ngn/libremdb
1
0
Fork 0
Code Issues 1 Pull Requests 10 Actions Packages Activity

Merge pull request #19 from httpjamesm/main

Browse Source 
Anonymous media proxy, local docker, private IP ratelimit and caching with redis
This commit is contained in:
zyachel 2022-11-13 12:11:01 +05:30 committed by GitHub
commit 31218adac1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
14 changed files with 279 additions and 120 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.env.local.example
View File

@ -7,3 +7,9 @@ NEXT_PUBLIC_URL=
# AXIOS_USERAGENT=
# default accept header is 'application/json, text/plain, */*'
# AXIOS_ACCEPT=
# for docker, just set the domain to the container name, default is 'libremdb_redis'
REDIS_URL=localhost:6379
# if you want to use redis to speed up the media proxy, set this to true
USE_REDIS = true

3
.gitignore vendored
View File

@ -33,3 +33,6 @@ yarn-error.log*
#just dev stuff
dev/*
yarn.lock
# docker
docker-compose.yml

34
Dockerfile Normal file
View File

@ -0,0 +1,34 @@
# Thanks @yordis on Github! https://github.com/vercel/next.js/discussions/16995#discussioncomment-132339
# Install dependencies only when needed
FROM node:lts-alpine AS deps
WORKDIR /opt/app
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile
# Rebuild the source code only when needed
# This is where because may be the case that you would try
# to build the app based on some `X_TAG` in my case (Git commit hash)
# but the code hasn't changed.
FROM node:lts-alpine AS builder
ENV NODE_ENV=production
WORKDIR /opt/app
COPY . .
COPY --from=deps /opt/app/node_modules ./node_modules
RUN yarn build
# Production image, copy all the files and run next
FROM node:lts-alpine AS runner
ARG X_TAG
WORKDIR /opt/app
ENV NODE_ENV=production
COPY --from=builder /opt/app/next.config.mjs ./
COPY --from=builder /opt/app/public ./public
COPY --from=builder /opt/app/.next ./.next
COPY --from=builder /opt/app/node_modules ./node_modules
ENV HOST=0.0.0.0
ENV PORT=3000
CMD ["node_modules/.bin/next", "start"]

28
README.md
View File

@ -64,12 +64,11 @@ Inspired by projects like [teddit](https://codeberg.org/teddit/teddit), [nitter]
- It doesn't have all routes.
I'll implement more with time :)
- I see connection being made to some Amazon domains.
For now, images and videos are directly served from Amazon. If I have enough time in the future, I'll implement a way to serve the images from libremdb instead.
- Is content served from third-parties, like Amazon?
Nope, libremdb proxies all image and video requests through the instance to avoid exposing your IP address, browser information and other personally identifiable metadata ([Contributor](https://github.com/httpjamesm)).
- Will Amazon track me then?
They may log your IP address, useragent, and other such
identifiers. I'd recommend using a VPN, or accessing the website through TOR for mitigating this risk.
Also nope. All Amazon will see is the libremdb instance making the request, not you. IP address, browser information and other personally identifiable metadata is hidden from Amazon.
- Why not just use IMDb?
Refer to the [features section](#some-features) above.
@ -87,7 +86,9 @@ Inspired by projects like [teddit](https://codeberg.org/teddit/teddit), [nitter]
A key named 'theme' is stored in Local Storage provided by your browser, if you ever override the default theme. To remove it, go to site data settings, and clear the data for this website. To permamently disable libremdb from storing your theme prefrences, either turn off JavaScript or disable access to Local Storage for libremdb.
- Information collected by other services:
libremdb connects to 'media-amazon.com' and 'media-imdb.com' for fetching images and videos. So, Amazon might log your IP address, and other information(such as http headers) sent by your browser.
~~libremdb connects to 'media-amazon.com' and 'media-imdb.com' for fetching images and videos. So, Amazon might log your IP address, and other information(such as http headers) sent by your browser.~~
None. libremdb proxies images anonymously through the instance for maximum privacy ([Contributor](https://github.com/httpjamesm)).
---
@ -114,7 +115,7 @@ Inspired by projects like [teddit](https://codeberg.org/teddit/teddit), [nitter]
- [ ] use redis, or any other caching strategy
- [x] implement a better installation method
- [ ] serve images and videos from libremdb itself
- [x] serve images and videos from libremdb itself
---
@ -128,7 +129,10 @@ As libremdb is made with Next.js, you can deploy it anywhere where Next.js is su
for Node.js, visit [their website](https://nodejs.org/en/).
for Git, run `sudo apt install git` if you're on a Debian-based distro. Else visit [their website](https://git-scm.com/).
2. Clone and set up the repo.
2. Install redis
You can install redis from [here](https://redis.io).
3. Clone and set up the repo.
```bash
git clone https://github.com/zyachel/libremdb.git # replace github.com with codeberg.org if you wish so.
@ -144,12 +148,16 @@ As libremdb is made with Next.js, you can deploy it anywhere where Next.js is su
libremdb will start running at http://localhost:3000.
To change port, modify the last command like this: `pnpm start -- -p <port-number>`.
### Docker
### Docker (Local & Recommended)
You can build the docker image using the provided Dockerfile and set it up using the example docker-compose file.
Change the docker-compose file to your liking and run `docker-compose up -d` to start the container, that's all!
### Docker (Built)
There's a [docker image](https://github.com/PussTheCat-org/docker-libremdb-quay) made by [@TheFrenchGhosty](https://github.com/TheFrenchGhosty) for [PussTheCat.org's instance](https://libremdb.pussthecat.org). You can use that in case you wish to use docker.
---
## Miscellaneous
### Automatic redirection

23
docker-compose.example.yml Normal file
View File

@ -0,0 +1,23 @@
# docker-compose.yml
version: '3'
services:
frontend:
container_name: libremdb
build:
context: .
network: host
ports:
- "3000:3000"
env_file: .env.local
depends_on:
- redis
restart: always
redis:
container_name: libremdb_redis
image: redis
# FOR DEBUGGING ONLY
# ports:
# - "6379:6379"
restart: always

2
package.json
View File

@ -15,7 +15,9 @@
"dependencies": {
"axios": "^0.27.2",
"cheerio": "1.0.0-rc.12",
"ioredis": "^5.2.3",
"next": "12.2.5",
"node-fetch": "^3.2.10",
"react": "18.2.0",
"react-dom": "18.2.0",
"sharp": "^0.31.0"

50
src/components/title/Basic.tsx
View File

@ -1,22 +1,27 @@
import { Fragment } from 'react';
import Image from 'next/future/image';
import Link from 'next/link';
import { Fragment } from 'react'
import Image from 'next/future/image'
import Link from 'next/link'
import { formatNumber, formatTime, modifyIMDbImg } from '../../utils/helpers';
import { Basic } from '../../interfaces/shared/title';
import styles from '../../styles/modules/components/title/basic.module.scss';
import {
formatNumber,
formatTime,
getProxiedIMDbImgUrl,
modifyIMDbImg,
} from '../../utils/helpers'
import { Basic } from '../../interfaces/shared/title'
import styles from '../../styles/modules/components/title/basic.module.scss'
type Props = {
className: string;
data: Basic;
};
className: string
data: Basic
}
const Basic = ({ data, className }: Props) => {
const titleType = data.type.id;
const titleType = data.type.id
const releaseTime =
titleType === 'tvSeries'
? `${data.releaseYear?.start}-${data.releaseYear?.end || 'present'}`
: data.releaseYear?.start;
: data.releaseYear?.start
return (
<section
@ -29,7 +34,8 @@ const Basic = ({ data, className }: Props) => {
className={styles.imageContainer}
style={{
backgroundImage:
data.poster && `url(${modifyIMDbImg(data.poster.url, 300)})`,
data.poster &&
`url(${getProxiedIMDbImgUrl(modifyIMDbImg(data.poster.url, 300))})`,
}}
>
{data.poster ? (
@ -39,11 +45,11 @@ const Basic = ({ data, className }: Props) => {
alt={data.poster.caption}
priority
fill
sizes='300px'
sizes="300px"
/>
) : (
<svg className={styles.image__NA}>
<use href='/svg/sprite.svg#icon-image-slash' />
<use href="/svg/sprite.svg#icon-image-slash" />
</svg>
)}
</div>
@ -51,7 +57,7 @@ const Basic = ({ data, className }: Props) => {
<h1 className={`${styles.title} heading heading__primary`}>
{data.title}
</h1>
<ul className={styles.meta} aria-label='quick facts'>
<ul className={styles.meta} aria-label="quick facts">
{data.status.id !== 'released' && (
<li className={styles.meta__text}>{data.status.text}</li>
)}
@ -72,7 +78,7 @@ const Basic = ({ data, className }: Props) => {
<p className={styles.rating}>
<span className={styles.rating__num}>{data.ratings.avg}</span>
<svg className={styles.rating__icon}>
<use href='/svg/sprite.svg#icon-rating'></use>
<use href="/svg/sprite.svg#icon-rating"></use>
</svg>
<span className={styles.rating__text}> Avg. rating</span>
</p>
@ -81,7 +87,7 @@ const Basic = ({ data, className }: Props) => {
{formatNumber(data.ratings.numVotes)}
</span>
<svg className={styles.rating__icon}>
<use href='/svg/sprite.svg#icon-like-dislike'></use>
<use href="/svg/sprite.svg#icon-like-dislike"></use>
</svg>
<span className={styles.rating__text}> No. of votes</span>
</p>
@ -93,7 +99,7 @@ const Basic = ({ data, className }: Props) => {
{formatNumber(data.ranking.position)}
</span>
<svg className={styles.rating__icon}>
<use href='/svg/sprite.svg#icon-graph-rising'></use>
<use href="/svg/sprite.svg#icon-graph-rising"></use>
</svg>
<span className={styles.rating__text}>
{' '}
@ -130,7 +136,7 @@ const Basic = ({ data, className }: Props) => {
<span className={styles.overview__text}>{data.plot || '-'}</span>
</p>
}
{data.primaryCrew.map(crewType => (
{data.primaryCrew.map((crewType) => (
<p className={styles.crewType} key={crewType.type.id}>
<span className={styles.crewType__heading}>
{`${crewType.type.category}: `}
@ -147,7 +153,7 @@ const Basic = ({ data, className }: Props) => {
))}
</div>
</section>
);
};
)
}
export default Basic;
export default Basic

52
src/components/title/Media.tsx
View File

@ -1,41 +1,43 @@
import Image from 'next/future/image';
import Link from 'next/link';
import { NextRouter } from 'next/router';
import { Media } from '../../interfaces/shared/title';
import { modifyIMDbImg } from '../../utils/helpers';
import Image from 'next/future/image'
import Link from 'next/link'
import { NextRouter } from 'next/router'
import { Media } from '../../interfaces/shared/title'
import { getProxiedIMDbImgUrl, modifyIMDbImg } from '../../utils/helpers'
import styles from '../../styles/modules/components/title/media.module.scss';
import styles from '../../styles/modules/components/title/media.module.scss'
type Props = {
className: string;
media: Media;
router: NextRouter;
};
className: string
media: Media
router: NextRouter
}
const Media = ({ className, media, router }: Props) => {
return (
<div className={`${className} ${styles.media}`}>
{(media.trailer || !!media.videos.total) && (
<section className={styles.videos}>
<h2 className='heading heading__secondary'>Videos</h2>
<h2 className="heading heading__secondary">Videos</h2>
<div className={styles.videos__container}>
{media.trailer && (
<div key={router.asPath} className={styles.trailer}>
<video
aria-label='trailer video'
aria-label="trailer video"
// it's a relatively new tag. hence jsx-all1 complains
aria-description={media.trailer.caption}
controls
playsInline
poster={modifyIMDbImg(media.trailer.thumbnail)}
poster={getProxiedIMDbImgUrl(
modifyIMDbImg(media.trailer.thumbnail)
)}
className={styles.trailer__video}
>
{media.trailer.urls.map(source => (
{media.trailer.urls.map((source) => (
<source
key={source.url}
type={source.mimeType}
src={source.url}
src={getProxiedIMDbImgUrl(source.url)}
data-res={source.resolution}
/>
))}
@ -44,15 +46,15 @@ const Media = ({ className, media, router }: Props) => {
)}
{!!media.videos.total &&
media.videos.videos.map(video => (
media.videos.videos.map((video) => (
<Link href={`/video/${video.id}`} key={video.id}>
<a className={styles.video}>
<Image
className={styles.video__img}
src={modifyIMDbImg(video.thumbnail)}
alt=''
alt=""
fill
sizes='400px'
sizes="400px"
/>
<p className={styles.video__caption}>
{video.caption} ({video.runtime}s)
@ -65,16 +67,16 @@ const Media = ({ className, media, router }: Props) => {
)}
{!!media.images.total && (
<section className={styles.images}>
<h2 className='heading heading__secondary'>Images</h2>
<h2 className="heading heading__secondary">Images</h2>
<div className={styles.images__container}>
{media.images.images.map(image => (
{media.images.images.map((image) => (
<figure key={image.id} className={styles.image}>
<Image
className={styles.image__img}
src={modifyIMDbImg(image.url)}
alt=''
alt=""
fill
sizes='400px'
sizes="400px"
/>
<figcaption className={styles.image__caption}>
{image.caption.plainText}
@ -85,6 +87,6 @@ const Media = ({ className, media, router }: Props) => {
</section>
)}
</div>
);
};
export default Media;
)
}
export default Media

85
src/pages/about/index.tsx
View File

@ -1,19 +1,19 @@
/* eslint-disable react/no-unescaped-entities */
import Link from 'next/link';
import Meta from '../../components/Meta/Meta';
import Layout from '../../layouts/Layout';
import Link from 'next/link'
import Meta from '../../components/Meta/Meta'
import Layout from '../../layouts/Layout'
import styles from '../../styles/modules/pages/about/about.module.scss';
import styles from '../../styles/modules/pages/about/about.module.scss'
const About = () => {
return (
<>
<Meta
title='About'
description='libremdb is a free & open source IMDb front-end. It allows you to see information about movies, tv shows, video games without any ads or tracking.'
title="About"
description="libremdb is a free & open source IMDb front-end. It allows you to see information about movies, tv shows, video games without any ads or tracking."
/>
<Layout full className={styles.about}>
<section id='features' className={styles.features}>
<section id="features" className={styles.features}>
<h2
className={`heading heading__secondary ${styles.features__heading}`}
>
@ -22,12 +22,12 @@ const About = () => {
<ul className={styles.features__list}>
<li className={styles.feature}>
<svg
aria-hidden='true'
focusable='false'
role='img'
aria-hidden="true"
focusable="false"
role="img"
className={styles.feature__icon}
>
<use href='/svg/sprite.svg#icon-eye-slash'></use>
<use href="/svg/sprite.svg#icon-eye-slash"></use>
</svg>
<h3
className={`heading heading__tertiary ${styles.feature__heading}`}
@ -41,12 +41,12 @@ const About = () => {
</li>
<li className={styles.feature}>
<svg
aria-hidden='true'
focusable='false'
role='img'
aria-hidden="true"
focusable="false"
role="img"
className={styles.feature__icon}
>
<use href='/svg/sprite.svg#icon-palette'></use>
<use href="/svg/sprite.svg#icon-palette"></use>
</svg>
<h3
className={`heading heading__tertiary ${styles.feature__heading}`}
@ -60,12 +60,12 @@ const About = () => {
</li>
<li className={styles.feature}>
<svg
aria-hidden='true'
focusable='false'
role='img'
aria-hidden="true"
focusable="false"
role="img"
className={styles.feature__icon}
>
<use href='/svg/sprite.svg#icon-responsive'></use>
<use href="/svg/sprite.svg#icon-responsive"></use>
</svg>
<h3
className={`heading heading__tertiary ${styles.feature__heading}`}
@ -79,7 +79,7 @@ const About = () => {
</li>
</ul>
</section>
<section id='faq' className={styles.faqs}>
<section id="faq" className={styles.faqs}>
<h2 className={`heading heading__secondary ${styles.faqs__heading}`}>
Questions you may have
</h2>
@ -91,21 +91,21 @@ const About = () => {
<p className={styles.faq__description}>
Replace `imdb.com` in any IMDb URL with any of the instances.
For example: `
<a href='https://imdb.com/title/tt1049413' className='link'>
<a href="https://imdb.com/title/tt1049413" className="link">
imdb.com/title/tt1049413
</a>
` to `
<a
href='https://libremdb.iket.me/title/tt1049413'
className='link'
href="https://libremdb.iket.me/title/tt1049413"
className="link"
>
libremdb.iket.me/title/tt1049413
</a>
` . To avoid changing the URLs manually, you can use extensions
like{' '}
<a
href='https://github.com/libredirect/libredirect/'
className='link'
href="https://github.com/libredirect/libredirect/"
className="link"
>
LibRedirect
</a>
@ -133,12 +133,21 @@ const About = () => {
</details>
<details className={styles.faq}>
<summary className={styles.faq__summary}>
I see connection being made to some Amazon domains.
Is content served from third-parties, like Amazon?
</summary>
<p className={styles.faq__description}>
For now, images and videos are directly served from Amazon. If I
have enough time in the future, I'll implement a way to serve
the images from libremdb instead.
Nope, libremdb proxies all image and video requests through the
instance to avoid exposing your IP address, browser information
and other personally identifiable metadata (
<a
href="https://github.com/httpjamesm"
target="_blank"
rel="noopener noreferrer"
className="link"
>
Contributor
</a>
).
</p>
</details>
<details className={styles.faq}>
@ -146,9 +155,9 @@ const About = () => {
Will Amazon track me then?
</summary>
<p className={styles.faq__description}>
They may log your IP address, useragent, and other such
identifiers. I'd recommend using a VPN, or accessing the website
through TOR for mitigating this risk.
Also nope. All Amazon will see is the libremdb instance making
the request, not you. IP address, browser information and other
personally identifiable metadata is hidden from Amazon.
</p>
</details>
<details className={styles.faq}>
@ -157,7 +166,7 @@ const About = () => {
</summary>
<p className={styles.faq__description}>
Refer to the{' '}
<a className='link' href='#features'>
<a className="link" href="#features">
features section
</a>{' '}
above.
@ -187,8 +196,8 @@ const About = () => {
</summary>
<p className={styles.faq__description}>
That's great! I've a couple of{' '}
<Link href='/contact'>
<a className='link'>contact methods</a>
<Link href="/contact">
<a className="link">contact methods</a>
</Link>
. Send your beautiful suggestions(or complaints), or just drop a
hi.
@ -198,7 +207,7 @@ const About = () => {
</section>
</Layout>
</>
);
};
)
}
export default About;
export default About

59
src/pages/api/media_proxy.ts Normal file
View File

@ -0,0 +1,59 @@
import { NextApiRequest, NextApiResponse } from 'next'
import redis from '../../utils/redis'
import axiosInstance from '../../utils/axiosInstance'
import { AxiosResponse } from 'axios'
const regex =
/^https:\/\/((m\.)?media-amazon\.com|imdb-video\.media-imdb\.com).*\.(jpg|jpeg|png|mp4|gif|webp).*$/
export default async function handler(
req: NextApiRequest,
res: NextApiResponse
) {
const mediaUrl = req.query.url as string | undefined
if (!mediaUrl || !regex.test(mediaUrl))
return res.status(400).json({
success: false,
message: 'Invalid query',
})
if (process.env.USE_REDIS === 'true') {
const cachedMedia = await redis.getBuffer(mediaUrl)
if (cachedMedia) {
res.setHeader('x-cached', 'true')
res.status(302).send(cachedMedia)
return
}
}
let mediaRes: AxiosResponse
try {
mediaRes = await axiosInstance(mediaUrl, { responseType: 'arraybuffer' })
} catch {
res.status(404)
res.json({
success: false,
message: 'Error from IMDb',
})
return
}
const data = mediaRes.data
if (process.env.USE_REDIS === 'true') {
// save in redis for 30 minutes
await redis.setex(mediaUrl, 30 * 60, Buffer.from(data))
}
// send media
res.setHeader('x-cached', 'false')
res.send(data)
}
export const config = {
api: {
responseLimit: false,
},
}

31
src/pages/privacy/index.tsx
View File

@ -1,14 +1,14 @@
import Meta from '../../components/Meta/Meta';
import Layout from '../../layouts/Layout';
import Meta from '../../components/Meta/Meta'
import Layout from '../../layouts/Layout'
import styles from '../../styles/modules/pages/privacy/privacy.module.scss';
import styles from '../../styles/modules/pages/privacy/privacy.module.scss'
const Privacy = () => {
return (
<>
<Meta
title='Privacy'
description='Privacy policy of libremdb, a free & open source IMDb front-end.'
title="Privacy"
description="Privacy policy of libremdb, a free & open source IMDb front-end."
/>
<Layout className={styles.privacy}>
<section className={styles.policy}>
@ -41,24 +41,11 @@ const Privacy = () => {
Local Storage for libremdb.
</p>
</div>
<div className={styles.item}>
<h2
className={`heading heading__secondary ${styles.item__heading}`}
>
Information collected by other services
</h2>
<p className={styles.item__text}>
libremdb connects to 'media-amazon.com' and 'media-imdb.com' for
fetching images and videos. So, Amazon might log your IP
address, and other information(such as http headers) sent by
your browser.
</p>
</div>
</div>
<footer className={styles.metadata}>
<p>
Last updated on <time>10 september, 2022.</time>
Last updated on <time>31 october, 2022.</time>
</p>
<p>
You can see the full revision history of this privacy policy on
@ -68,7 +55,7 @@ const Privacy = () => {
</section>
</Layout>
</>
);
};
)
}
export default Privacy;
export default Privacy

7
src/pages/title/[titleId]/index.tsx
View File

@ -20,6 +20,7 @@ import { AppError } from '../../../interfaces/shared/error'
// styles
import styles from '../../../styles/modules/pages/title/title.module.scss'
import Head from 'next/head'
import { getProxiedIMDbImgUrl } from '../../../utils/helpers'
type Props = { data: Title; error: null } | { error: AppError; data: null }
@ -50,7 +51,11 @@ const TitleInfo = ({ data, error }: Props) => {
<Head>
<meta
title="og:image"
content={data.basic.poster?.url || '/icon-512.png'}
content={
data.basic.poster?.url
? getProxiedIMDbImgUrl(data.basic.poster?.url)
: '/icon-512.png'
}
/>
</Head>
<Layout className={styles.title}>

4
src/utils/helpers.ts
View File

@ -53,6 +53,10 @@ export const modifyIMDbImg = (url: string, widthInPx = 600) => {
return url.replaceAll('.jpg', `UX${widthInPx}.jpg`);
};
export const getProxiedIMDbImgUrl = (url: string) => {
return `/api/media_proxy?url=${encodeURIComponent(url)}`;
}
export const AppError = class extends Error {
constructor(message: string, public statusCode: number, cause?: any) {
super(message, cause);

11
src/utils/redis.ts Normal file
View File

@ -0,0 +1,11 @@
import Redis from 'ioredis'
const redisUrl = process.env.REDIS_URL
if (!redisUrl) {
throw 'Please set the REDIS_URL environment variable.'
}
const redis = new Redis(redisUrl)
export default redis