Initial commit

This commit is contained in:
2026-11-15 18:15:03 -05:00
commit b76aea2899
36 changed files with 9424 additions and 0 deletions

247
README.md Normal file
View File

@@ -0,0 +1,247 @@
# opds-from-rss
Serve a **FreshRSS** account as an **OPDS 1.2** catalog, where every article is
downloadable as an **EPUB generated on the fly**. Downloading an article marks it
read in FreshRSS.
It is a single small container with **no runtime dependencies** — the whole server
is the Python standard library — and it keeps **nothing on disk**. The catalog is
rebuilt from the FreshRSS API as you browse it, and each book is assembled in
memory at the moment it is requested.
```text
e-reader ──OPDS──▶ categories ──▶ feeds ──▶ articles ──▶ EPUB (built in memory)
│ │
└──────────────── browse the unread queue ◀──────────────┘
mark read upstream
```
## How it works
* **Your account's shape becomes the catalog.** FreshRSS category labels become
navigation feeds, subscriptions become feed entries, and unread articles become
the acquisition entries you download.
* **One article, one book.** Each article is converted to an EPUB 3 (with an EPUB 2
NCX fallback) whose body is the content FreshRSS already stores, normalised into
strict XHTML. Images are fetched and embedded so the book reads offline.
* **Downloading is what marks it read.** After the EPUB has been delivered, the
matching item is tagged read upstream. A `HEAD` probe or an interrupted transfer
does **not** consume the article, and the call is idempotent.
* **Nothing is persisted.** No database, no cache directory, no temp files. The
container can run with a read-only filesystem, which the shipped Compose file
does.
## Try it out first
There is a self-contained demo stack that needs no configuration and no FreshRSS
account: it runs the catalogue against a fake FreshRSS holding three sample feeds.
```sh
make demo-up # or: docker compose -f docker-compose.demo.yaml up --build
```
Then open **http://127.0.0.1:8088/opds** and sign in as **demo / demo**. The fake
account holds three feeds across two categories, a starred article and an
already-read one. Downloading a book marks that article read in the fake server, so
the unread queue visibly drains as you read.
```sh
make demo-down # stop it; all demo state is in memory, so nothing persists
```
It publishes an unencrypted port on loopback with a known password, so keep it for
trying things out — for real use, follow the quick start below.
## Quick start
```sh
cp config.example.json config.json # edit: your FreshRSS URL, user, categories
cp .env.example .env # edit: the two secrets
make hash-password # paste the digest into config.json (auth.users)
docker compose up -d --build
```
Then point your e-reader's OPDS browser at `http://<host>:8080/opds` and sign in
with the user name and password from `auth.users`.
### FreshRSS prerequisites
You need an **API password**, which is separate from your login password:
FreshRSS → **Settings → Account → API**. Put that in `FRESHRSS_API_PASSWORD`.
The URL in `config.json` may be written with or without the `/api/greader.php`
suffix; it is added for you, and any path prefix (`https://host/freshrss`) is kept.
## Configuration
`config.json` is the only input. `config.example.json` documents every setting
in place, and a key beginning with `_` is treated as a comment — ignored, never
interpolated, and never reported as unknown.
Any string value may reference a secret instead of containing one:
| Syntax | Meaning |
| --- | --- |
| `${VAR}` | The environment variable; the server refuses to start if it is unset |
| `${VAR:-default}` | The environment variable, or a default when unset |
| `${file:/run/secrets/x}` | The contents of a file (Docker secrets); relative paths resolve against `config.json`. The trailing newline is trimmed. |
| `$${` | A literal `${` |
The settings most worth knowing about:
| Setting | Default | Notes |
| --- | --- | --- |
| `server.host` | `127.0.0.1` | **A container needs `0.0.0.0`**, which `config.example.json` sets. Exposure is controlled by the published port. |
| `server.port` | `8080` | `0` asks the OS for a free port. |
| `server.public_base_url` | unset | Pins the catalog's links. Set it in production so a spoofed `Host` header cannot shape them. |
| `server.trust_proxy_headers` | `false` | Enables `X-Forwarded-Proto/Host/Prefix`. Only behind a proxy you control. |
| `server.allowed_hosts` | `[]` | Optional allowlist; other `Host` values are refused. |
| `server.tls.cert_file` / `key_file` | unset | In-process TLS, if you would rather not run a reverse proxy. |
| `freshrss.items_per_page` | `30` | Articles per catalog page. |
| `freshrss.mark_read` | `true` | Set `false` for a read-only mirror. |
| `freshrss.read_window_items` | `200` | How far back the **Read** section looks (see limitations). |
| `catalog.show_read_section` | `true` | Adds a Read tree beside the Unread queue. |
| `catalog.show_starred` | `true` | Adds a Starred shelf. |
| `catalog.exclude_categories` / `exclude_feeds` | `[]` | Leave things out of the catalog entirely. |
| `epub.include_images` | `true` | Fetch and embed article images so books work offline. |
| `epub.cover_image` | `true` | Use the article's lead image as the book cover. |
| `epub.allowed_image_hosts` | `[]` | When set, only these hosts' images are embedded. |
| `auth.users[].password_sha256` | — | Preferred over `password`. Generate with `make hash-password`. |
| `cache.*` | 300s / 60s / 900s | In-memory TTLs. Nothing is written to disk. |
Validate before deploying — every problem is reported at once, with its JSON path:
```sh
make check # or: python3 -m app --config config.json --check-config
```
## What the catalog looks like
| Path | Kind | Contents |
| --- | --- | --- |
| `/opds` | navigation | Unread, Read, Starred, All unread |
| `/opds/unread` · `/opds/read` | navigation | Categories, with unread counts |
| `/opds/unread/c/<category>` | navigation | The feeds in that category |
| `/opds/unread/c/<category>/f/<feed>` | acquisition | Unread articles, newest first |
| `/opds/unread/all` · `/opds/read/all` | acquisition | The whole reading list, paginated |
| `/opds/starred` | acquisition | Starred articles |
| `/opds/search?q=…` | acquisition | Search over the recent window |
| `/opds/d/<feed>/<item>/<slug>.epub` | — | The EPUB, and the mark-read |
| `/opds/opensearch.xml` | — | OpenSearch description, so readers show a search box |
| `/healthz` | — | Liveness. No credentials, no upstream call |
| `/opds/healthz` | — | Diagnostics: upstream reachability, feed count, cache, mark-read state |
Opaque identifiers in paths are base64url, because FreshRSS ids contain `/`, `?`
and `:`. The `<slug>.epub` segment is decoration — it names the saved file and the
server ignores it when looking the article up.
Links are absolute, and every feed carries `rel="self"`, `rel="start"` and
`rel="search"`, with `rel="first"`/`previous`/`next` for pagination. Acquisition
entries link the EPUB twice (`open-access` and the generic acquisition relation),
because readers disagree about which to look for, plus `alternate` to the original
article.
### Reader setup
* **KOReader** — Tools → OPDS catalog → add `http://<host>:8080/opds` with your
credentials. Use `https://` if you put a proxy in front.
* **Calibre** — *Get books* → *Add a feed* → paste the same URL.
* **Moon+ Reader / Librera / FBReader** — add a *custom OPDS* catalog with that
URL and the same credentials.
Only OPDS **1.2 (Atom)** is served, deliberately: that is what shipping readers
implement. OPDS 2.0 JSON would be a second, unused surface.
## Deploying
The Compose file runs the container read-only, with all capabilities dropped, no
new privileges, published on **loopback only**, and with secrets coming from
`.env`. Put a TLS-terminating reverse proxy (or a Cloudflare tunnel, or similar)
in front of it — see below.
```sh
make up # build and start
make logs # follow
make down # stop
```
To reach it from the LAN instead of through a proxy, change the published port to
`"8080:8080"` in `docker-compose.yaml` and set `server.public_base_url`. Only do
that on a network you trust.
### Security notes
* **HTTP Basic requires TLS.** Basic credentials are base64-encoded, not
encrypted. Terminate TLS at a proxy, or set `server.tls.*` so the container does
it itself. The server logs a warning at startup when it is serving plain HTTP.
* **Authentication is mandatory and deliberate.** There is no "auth off" switch:
the catalog is your entire reading queue, and an unauthenticated OPDS endpoint on
a routable address is an open book. Passwords are compared as sha256 digests in
constant time, an unknown user takes the same time as a wrong password, and
repeated failures from one address are throttled (`auth.max_failures_per_minute`,
`auth.throttle_seconds`).
* **Image fetching is SSRF-guarded.** Image URLs come from feed content, so they are
attacker-controlled if you follow a hostile feed. Only `http`/`https` is
followed; every address a hostname resolves to is checked against the loopback,
link-local, private, multicast and reserved ranges (including v4-mapped v6
forms); the connection is dialled to the *validated address* rather than
re-resolving the name, which closes the DNS-rebinding window; TLS still verifies
the real hostname; redirects are validated hop by hop; and bodies are capped per
image and per article. A refused image is simply dropped from the book.
* **Set `server.public_base_url`** in production, and `server.allowed_hosts` if the
service is directly reachable, so a spoofed `Host` header cannot shape the links
in the catalog.
## Development
```sh
make test # 275 tests, no dependencies beyond Python 3.11+
make run # run locally against config.json
make demo # serve a built-in fake FreshRSS and try the whole thing offline
```
The suite needs no network: an in-process FreshRSS stand-in
(`tests/stub_freshrss.py`) speaks the real wire format, and the end-to-end tests
drive the actual HTTP server against it — browsing the catalog, downloading an
EPUB, unzipping it, and asserting on the `edit-tag` request that FreshRSS received.
`tools/fake-freshrss.py` exposes the same stub for a manual smoke test.
Layout: `config` validation, `urls` identifiers, `cache` TTL cache, `xhtml`
HTML→XHTML, `images` sniffing/sizing, `fetch` guarded fetching, `epub` the EPUB
writer, `freshrss` the API client, `auth`, `opds` the Atom builders, `server`
routing and catalog logic.
## Limitations, honestly
* **The Read section is bounded.** Unread filtering happens upstream (exact), but
read items are filtered locally because only the *unread exclusion* is a
documented upstream parameter — so Read shows at most `read_window_items` recent
articles. The same window bounds `/opds/search`, which is a local filter rather
than an upstream search call.
* **Article bodies come from FreshRSS**, not from the original pages. Fast,
offline-friendly, and it works for paywalled or JavaScript-heavy sites — but a
feed that only publishes teasers yields teaser-length books. Pair it with
FreshRSS's own full-text (Readability) configuration if you need more.
* **No cover thumbnails in the catalog.** Cover images are used inside the EPUB
(`cover-image`), but navigation feeds do not advertise thumbnails, to avoid
making every reader fetch one image per entry.
* **`javascript:`/`data:` URLs, scripts, styles, iframes and SVG do not survive**
into a book, by design. Neither do inline styles or `class` attributes: the book
gets one plain stylesheet instead (override with `epub.stylesheet`).
* **Basic auth only.** No OAuth, no per-user catalogs: every user sees the whole
account.
## Troubleshooting
| Symptom | Cause |
| --- | --- |
| Container exits immediately, `configuration file not found` | `config.json` is not at `/config/config.json`; check the volume mount, or set `CONFIG_PATH`. |
| `environment variable FRESHRSS_API_PASSWORD is not set` | `.env` is missing or unrenamed; `cp .env.example .env`. |
| Reader shows an empty catalog | `freshrss.url` or the API password is wrong. `curl -u user:pass http://host:8080/opds/healthz` says which. |
| Reader prompts for a password repeatedly | Wrong `auth.users` entry; note that passwords are compared as digests, so a `password_sha256` must match exactly. |
| Catalog loads, downloads fail with 502 | The container cannot reach FreshRSS, or the image host refused; check `docker compose logs`. |
| Books are missing images | The images were refused by the SSRF guard or exceeded `epub.max_image_bytes`; the log says which, per URL. |
| A downloaded article is still listed as unread | The reader refetched the catalog before the mark-read landed, or `freshrss.mark_read` is `false`. |
## License
MIT.