general cleanup

Signed-off-by: ngn <ngn@ngn.tf>
This commit is contained in:
ngn 2025-01-20 04:31:15 +03:00
parent 95616fef4a
commit f8a1e6f79c
Signed by: ngn
GPG Key ID: A3654DF5AD9F641D
15 changed files with 108 additions and 389 deletions

View File

@ -1,18 +1,25 @@
FROM rust:slim as rust
WORKDIR /src
RUN apt-get update && apt-get install -y git pkg-config libssl-dev
RUN mkdir src && echo "fn main() {}" > src/main.rs
COPY Cargo.toml .
RUN sed -i '/.*build.rs.*/d' Cargo.toml
COPY Cargo.lock .
RUN sed -i '/.*build.rs.*/d' Cargo.toml
RUN cargo build --release
COPY . /src
RUN cargo build --release
FROM debian:bookworm-slim
RUN useradd -ms /bin/bash -u 1001 libmedium
RUN apt-get update && apt-get install -y ca-certificates
RUN mkdir /var/lib/libmedium && chown libmedium /var/lib/libmedium
COPY --from=rust /src/target/release/libmedium /usr/local/bin/
USER libmedium
ENTRYPOINT ["/usr/local/bin/libmedium"]

View File

@ -1,42 +1,20 @@
default: ## Debug build
all:
cargo build
clean: ## Clean all build artifacts and dependencies
@cargo clean
clean:
cargo clean
check: ## Check for syntax errors on all workspaces
check:
cargo check --workspace --tests --all-features
#cd utils/cache-bust && cargo check --tests --all-features
coverage: ## Generate HTML code coverage
./scripts/coverage.sh --coverage
dev-env: ## Download development dependencies
cargo fetch
doc: ## Prepare documentation
cargo doc --no-deps --workspace --all-features
docker: ## Build docker images
docker build -t realaravinth/libmedium:master -t realaravinth/libmedium:latest .
docker-publish: docker ## Build and publish docker images
docker push realaravinth/libmedium:master
docker push realaravinth/libmedium:latest
lint: ## Lint codebase
lint:
cargo fmt -v --all -- --emit files
cargo clippy --workspace --tests --all-features
release: ## Release build
cargo build --release
run: default ## Run debug build
run:
cargo run
test: ## Run tests
test:
cargo test --all-features --no-fail-fast
help: ## Prints help for targets with comments
@cat $(MAKEFILE_LIST) | grep -E '^[a-zA-Z_-]+:.*?## .*$$' | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
.PHONY: clean check lint run test

View File

@ -1,87 +1,5 @@
<div align="center">
<h1> LibMedium </h1>
<p>
# [ngn.tf] | libmedium
**Privacy-focused proxy for medium.com**
![](https://git.ngn.tf/ngn/libmedium/actions/workflows/build.yml/badge.svg)
</p>
[![Awesome Humane Tech](https://raw.githubusercontent.com/humanetech-community/awesome-humane-tech/main/humane-tech-badge.svg?sanitize=true)](https://github.com/humanetech-community/awesome-humane-tech)
[![status-badge](https://ci.batsense.net/api/badges/realaravinth/libmedium/status.svg)](https://ci.batsense.net/realaravinth/libmedium)
[![codecov](https://codecov.io/gh/realaravinth/libmedium/branch/master/graph/badge.svg)](https://codecov.io/gh/realaravinth/libmedium)
</div>
## Status
Usable. Should you run into a `HTTP 500 Internal Server Error`, please
file a bug report with the URL of the post you were trying to read and
the git commit hash of the build. Git commit hash can be obtained from
[/api/v1/meta/build](https://libmedium.batsense.net/api/v1/meta/build).
This proxy works by interacting with Medium's undocumented(probably
private but unauthenticated) API. So I've had to make assumptions and
tweak API schematics as I run into errors.
## Features
- [x] proxy images
- [x] proxy GitHub gists
- [x] render posts
- [x] syntax highlighting for gists
- [ ] user pages(WIP)
- [ ] RSS feeds
## Why?
Knowledge is the true wealth of humanity. If it weren't for our
ancestors, who chose to pass down their knowledge and experiences, we
would still be a primitive species. Whatever advancement that we as
a species have achieved is because we chose to share information.
To put a paywall on knowledge like that is both obscene and goes against
the very nature of humanity.
It is possible to run a sustainable publication business while still
respecting freedom. [LWN.net](https://lwn.net) is one of my favourite
publications that has been around forever. So it is possible. I hope
medium.com comes up with other, non-harmful ways to run a sustainable
business.
## Instances
| Instance | Country | Provider | Host |
| ------------------------------------------------------------------------- | ------- | ---------- | -------------------------------------- |
| https://libmedium.batsense.net | India | Airtel | @realaravinth |
| https://md.vern.cc | US | Hetzner | [~vern](https://vern.cc) |
| http://md.vernccvbvyi5qhfzyqengccj7lkove6bjot2xhh5kajhwvidqafczrad.onion/ | N/A | Hetzner | [~vern](https://vern.cc) |
| http://vernaqj2qr2pijpgvf3od6ssc3ulz3nv52gwr3hba5l6humuzmgq.b32.i2p/ | N/A | Hetzner | [~vern](https://vern.cc) |
| https://medium.hostux.net | France | Gandi | [hostux](https://hostux.net) |
| https://r.sudovanilla.org | US | Selfhosted | [SudoVanilla](https://sudovanilla.org) |
| https://libmedium.ducks.party | DE | Datalix | [ducks.party](https://ducks.party) |
## Deploy
1. Grab [`./config/default.toml`](./config/default.toml) and make
necessary changes
2. AMD64 pre-compiled images are available on DockerHub.
```
docker run -d \
-v ./config/default.toml:/etc/libmedium/config.toml \
-p 8082:7000 \
--restart always \
--name libmedium \
realaravinth/libmedium
```
If you are on a different architecture, run make docker and then run the
above command.
```
make docker
```
---
Inspired by [Scribe - An Alternative Medium Frontend](https://sr.ht/~edwardloveall/scribe)
A fork of the [libmedium](https://github.com/realaravinth/libmedium) project, with my personal changes.

7
config.toml Normal file
View File

@ -0,0 +1,7 @@
debug = true
#cache = "/var/lib/libmedium"
port = 7000
ip= "0.0.0.0"
domain = "example.com"
proxy_has_tls = false
#workers = 2

View File

@ -1,15 +0,0 @@
debug = true
source_code = "https://git.batsense.net/realaravinth/libmedium"
#cache = "/var/lib/libmedium"
[server]
# The port at which you want authentication to listen to
# takes a number, choose from 1000-10000 if you dont know what you are doing
port = 7000
#IP address. Enter 0.0.0.0 to listen on all availale addresses
ip= "0.0.0.0"
# enter your hostname, eg: example.com
domain = "localhost"
allow_registration = true
proxy_has_tls = false
#workers = 2

View File

@ -1,70 +0,0 @@
// ==UserScript==
// @name LibMedium proxy
// @version 0.1.1
// @description Re-writes medium.com URLs in point to libmedium
// @author Aravinth Manivannan
// @match https://*/*
// @match http://*/*
// @grant AGPLv3 or above
// ==/UserScript==
// websites to be proxied
const blacklist = [
"medium.com",
"blog.discord.com",
"uxdesign.cc",
"towardsdatascience.com",
"hackernoon.com",
"medium.freecodecamp.org",
"psiloveyou.xyz",
"betterhumans.coach.me",
"codeburst.io",
"theascent.pub",
"medium.mybridge.co",
"uxdesign.cc",
"levelup.gitconnected.com",
"itnext.io",
"entrepreneurshandbook.co",
"proandroiddev.com",
"blog.prototypr.io",
"thebolditalic.com",
"blog.usejournal.com",
];
// Location of the proxy
const libmediumHost = new URL("https://libmedium.batsense.net");
const isMedium = (url) => {
url = new URL(url);
if (blacklist.find((bl) => url.host.includes(bl))) {
return true;
}
let componenets = url.host.split(".");
let len = componenets.length;
if (
len > 1 &&
componenets[len - 1] == "com" &&
componenets[len - 2] == "medium"
) {
return true;
}
};
(function () {
"use strict";
if (!window.location.href.includes(libmediumHost.host)) {
let urls = document.links;
for (let i = 0; i < urls.length; i++) {
if (isMedium(urls[i])) {
let url = urls[i];
let path = url.pathname.split("-");
urls[i].pathname = `/utils/post/${path[path.length - 1]}`;
urls[i].host = libmediumHost.host;
urls[i].search = "";
}
}
}
})();

View File

@ -1,12 +1,5 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["config:recommended", ":dependencyDashboard"],
"labels": ["renovate-bot"],
"prHourlyLimit": 0,
"timezone": "Asia/kolkata",
"prCreation": "immediate",
"vulnerabilityAlerts": {
"enabled": true,
"labels": ["renovate-bot", "renovate-security", "security"]
}
"extends": ["config:recommended"],
"timezone": "Europe/Istanbul",
"prHourlyLimit": 20
}

View File

@ -1,88 +0,0 @@
#!/bin/bash
readonly GRCOV_DOWNLOAD="https://github.com/mozilla/grcov/releases/download/v0.8.2/grcov-linux-x86_64.tar.bz2"
readonly TMP_DIR=$(pwd)/tmp
readonly PROJECT_ROOT=$(pwd)
readonly GRCOV_TARBAL="$TMP_DIR/grcov.tar.bz2"
readonly GRCOV="$TMP_DIR/grcov"
clean_up() {
cd $PROJECT_ROOT
/bin/rm default.profraw lcov.info *.profraw || true
cd target
/bin/rm default.profraw lcov.info *.profraw || true
cargo clean
}
check_arg(){
if [ -z $1 ]
then
help
exit 1
fi
}
match_arg() {
if [ $1 == $2 ] || [ $1 == $3 ]
then
return 0
else
return 1
fi
}
download() {
if [ ! -e $GRCOV ];
then
echo "[*] Downloading grcov"
wget --quiet --output-doc=$GRCOV_TARBAL $GRCOV_DOWNLOAD;
cd $TMP_DIR
tar -xf $GRCOV_TARBAL;
cd $PROJECT_ROOT
fi
}
build_and_test() {
export RUSTFLAGS="-Zinstrument-coverage"
cd $PROJECT_ROOT
echo "[*] Building project"
cargo build
export LLVM_PROFILE_FILE="target/libmedium-%p-%m.profraw"
echo "[*] Running tests"
cargo test
echo "[*] Generating coverage"
$GRCOV target/ --binary-path \
./target/debug/ \
-s . -t lcov --branch \
--ignore-not-existing \
--ignore "../*" -o target/lcov.info
$GRCOV target/ --binary-path \
./target/debug/ \
-s . -t html --branch \
--ignore-not-existing \
--ignore "../*" -o target/coverage.html
}
run_coverage() {
cd $PROJECT_ROOT
mkdir $TMP_DIR || true
clean_up
download
build_and_test
}
check_arg $1
if match_arg $1 'c' '--coverage'
then
run_coverage
else
echo "undefined option"
exit 1
fi

View File

@ -60,7 +60,7 @@ async fn main() -> std::io::Result<()> {
PKG_NAME, PKG_DESCRIPTION, PKG_HOMEPAGE, VERSION, GIT_COMMIT_HASH
);
println!("Starting server on: http://{}", SETTINGS.server.get_ip());
println!("Starting server on: http://{}", SETTINGS.get_ip());
let data = Data::new();
@ -79,8 +79,8 @@ async fn main() -> std::io::Result<()> {
.app_data(data.clone())
.configure(routes::services)
})
.workers(SETTINGS.server.workers.unwrap_or_else(num_cpus::get))
.bind(SETTINGS.server.get_ip())
.workers(SETTINGS.workers.unwrap_or_else(num_cpus::get))
.bind(SETTINGS.get_ip())
.unwrap()
.run()
.await

View File

@ -133,7 +133,7 @@ const INDEX: &str = include_str!("../templates/index.html");
async fn index() -> impl Responder {
HttpResponse::Ok()
.content_type("text/html; charset=utf-8")
.body(INDEX.replace("SOURCE_CODE_REPLACE", &crate::SETTINGS.source_code))
.body(INDEX)
}
#[actix_web_codegen_const_routes::get(path = "crate::V1_API_ROUTES.proxy.asset")]

View File

@ -24,7 +24,9 @@ use serde::Deserialize;
use url::Url;
#[derive(Debug, Clone, Deserialize)]
pub struct Server {
pub struct Settings {
pub debug: bool,
pub cache: Option<String>,
pub port: u32,
pub domain: String,
pub ip: String,
@ -32,28 +34,13 @@ pub struct Server {
pub workers: Option<usize>,
}
impl Server {
#[cfg(not(tarpaulin_include))]
pub fn get_ip(&self) -> String {
format!("{}:{}", self.ip, self.port)
}
}
#[derive(Debug, Clone, Deserialize)]
pub struct Settings {
pub debug: bool,
pub cache: Option<String>,
pub server: Server,
pub source_code: String,
}
#[cfg(not(tarpaulin_include))]
impl Settings {
pub fn new() -> Result<Self, ConfigError> {
let mut s = Config::builder();
// setting default values
const CURRENT_DIR: &str = "./config/default.toml";
const CURRENT_DIR: &str = "./config.toml";
const ETC: &str = "/etc/libmedium/config.toml";
if let Ok(path) = env::var("LIBMEDIUM") {
@ -105,4 +92,9 @@ impl Settings {
fn check_url(&self) {
Url::parse(&self.source_code).expect("Please enter a URL for source_code in settings");
}
#[cfg(not(tarpaulin_include))]
pub fn get_ip(&self) -> String {
format!("{}:{}", self.ip, self.port)
}
}

View File

@ -6,59 +6,31 @@
<title>LibMedium</title>
</head>
<body>
<main>
<main class="index">
<div class="center">
<h1>LibMedium</h1>
<p>A free and privacy-respecting medium proxy</p>
<p>
<a
href="/@tylerneely/fear-and-loathing-in-lock-free-programming-7158b1cdd50c"
>Demo Article</a
>
| <a href="SOURCE_CODE_REPLACE">Source Code</a>
</p>
<div>
<a href="/@tylerneely/fear-and-loathing-in-lock-free-programming-7158b1cdd50c">
Demo
</a>
<span> | </span>
<a href="https://github.com/realaravinth/libmedium">
Source
</a>
<span> | </span>
<a href="https://git.ngn.tf/ngn/libmedium">
Modified Source
</a>
<span> | </span>
<a href="https://liberapay.com/realaravinth">
Donate
</a>
</div>
</div>
</main>
</body>
<style>
* {
padding: 0;
margin: 0;
}
body {
width: 100%;
min-height: 100vh;
display: block;
}
main {
display: flex;
flex-direction: column;
align-items: center;
width: 80%;
height: 100vh;
margin: auto;
justify-content: space-between;
}
.center {
margin: auto;
display: flex;
flex-direction: column;
align-items: center;
}
h1 {
font-size: 3rem;
}
@media screen and (max-width: 1200px) {
main {
width: 90%;
}
img {
display: block;
}
}
<. include!("./main.css"); .>
</style>
</html>

View File

@ -3,21 +3,48 @@
padding: 0;
}
body {
width: 100%;
display: flex;
min-height: 100vh;
display: block;
background: #000;
}
main {
.index {
display: flex;
flex-direction: column;
align-items: center;
width: 80%;
height: 100vh;
margin: auto;
justify-content: space-between;
}
.index .center {
margin: auto;
display: flex;
flex-direction: column;
align-items: center;
}
.index .center h1 {
font-size: 3rem;
}
@media screen and (max-width: 1200px) {
.index {
width: 90%;
}
}
.post {
width: 35em;
margin: auto;
display: flex;
flex-direction: column;
margin-top: 20px;
}
h1,
h2,
h3,
@ -25,27 +52,25 @@ h4,
h5,
h6 {
font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif !important;
font-weight: 900;
}
a {
color: rgb(0, 86, 179);
color: #02eded;
text-decoration: none;
}
a:visited {
color: rgb(0, 86, 179);
}
a:hover {
text-decoration: underline;
}
html {
color: #333;
color: #fff;
font-family: charter, Georgia, Cambria, "Times New Roman", Times, serif;
font-size: 26px;
line-height: 1.55rem;
}
p {
margin: 20px 0;
}
@ -83,11 +108,10 @@ pre {
line-height: 1rem;
padding: 20px;
border-radius: 6px;
background-color: rgba(175, 184, 193, 0.2);
background: #101010;
}
.code-block {
post.code-block {
display: block;
margin: 5px 0;
padding: 20px;
@ -105,7 +129,8 @@ iframe {
font-size: 16px;
line-height: 1.5rem;
}
main {
.post {
width: 90%;
}

View File

@ -5,7 +5,7 @@
</head>
<body>
<. let mut open_list = false ; .>
<main class="container">
<main class="post">
<h1><.= data.title .></h1>
<p class="meta">
<a class="author" href="https://medium.com/u/<.= data.creator.id .>" rel="noreferrer">

View File

@ -22,7 +22,7 @@
<meta property="twitter:description" content="<.= data.get_subtitle() .>" />
<meta
property="og:url"
content="http://<.= &*crate::SETTINGS.server.domain .>/<.= data.creator.name .>/<.= data.unique_slug .>"
content="http://<.= &*crate::SETTINGS.domain .>/<.= data.creator.name .>/<.= data.unique_slug .>"
/>
<meta property="og:image" content="<.= preview_img .>" />
<meta name="twitter:image:src" content="<.= preview_img .>" />
@ -72,5 +72,5 @@
<link rel="author" href="https://medium.com/<.= data.creator.name .>" />
<link
rel="canonical"
href="http://<.= &*crate::SETTINGS.server.domain .>/<.= data.creator.name .>/<.= data.unique_slug .>"
href="http://<.= &*crate::SETTINGS.domain .>/<.= data.creator.name .>/<.= data.unique_slug .>"
/>