Technical overviewApplied AI engineering and output verification

arXiv Daily Digest

A daily research digest that has to prove every claim it publishes.

Source code

The repository is private. The architecture, engineering decisions, and verified evidence are documented below.

Role: Solo engineer across retrieval, the model pipeline, the verification checks, the published site, and the scheduled deployment
Evidence status: Live at papers.pesanth.com, running unattended each morning on a free hosted model tier, with a local model supported as an alternative
Verified: 2026-08-15

Purpose and scope

Every morning the digest reads the day's new AI submissions to arXiv, selects three, pulls each paper's body where arXiv renders it to HTML, and publishes technical notes as a static site. Because the notes name architectures, datasets and benchmark scores, two automated checks run before anything is published: the model must quote the fragment its result came from, and every figure it writes must appear in the source text. It is live at papers.pesanth.com.

What it is used for

Keep up with a field that posts more than a hundred papers a working day, in a few minutes rather than an hour.
Decide whether a paper is worth opening, from its mechanism and its measured results rather than its abstract.
See at a glance which claims were checked against the paper and which were not.
Keep a dated archive of what was read, so a paper can be found again months later.

Architecture

Sources

Feed

arXiv Atom API

One request per run, sorted newest first, with the window widening from 48 hours to a week when the day is thin

Full text

HTML rendering

The paper body for submissions arXiv renders, with references, appendices and math markup stripped and tables kept

Read · one paper at a time

Pipeline

Model step

Selector

Reads a 40 paper shortlist and returns three with a stated reason each

Model step

Summarizer

Produces mechanism, method specifics, headline figures and limitations from a 14k character extract

Inference

Model backend

A hosted free tier by default, or a local model, behind one interface

Check · against the same source

Verification

Gate

Citation check

The quoted fragment must appear in the source, compared on words so punctuation cannot fail an honest quote

Gate

Figure check

Every number in the result, the headline figures and the method details must appear in the source

Policy

Retry and disclose

One retry with the specific failure quoted back, then the summary is published with that check marked failed

Write · then render

Archive and publication

Record

Day archive

One JSON file per day, the record every other output is rendered from

Web interface

Static site

A filterable index and one page per day, each page self-contained with nothing fetched at load

Automation

Scheduled run

A systemd timer each morning, with the timezone pinned because the host clock is UTC

Public ingress

Cloudflare Tunnel

Outbound-only ingress in front of a read-only nginx container

Text equivalent: One request to the arXiv API returns the newest submissions, and the window widens in memory until there are enough candidates to choose from. A model shortlists three papers, then summarizes each from the body of the paper where arXiv publishes an HTML rendering. Two checks run against that same source text before the result is archived as JSON, which is the record everything else is rendered from. The static site is rebuilt from the whole archive on every run and served read-only through an outbound-only Cloudflare Tunnel.

Engineering decisions

Make the model quote the paper

A model asked to summarize a paper it half-recognizes will describe the paper it remembers, and that failure is invisible in fluent prose. Requiring a verbatim fragment turns it into a string comparison. A quote that cannot be found is discarded rather than published, and the page says the summary is unchecked.

Check the numbers, not just the quote

Depth introduced the failure it needed to guard against. Once notes name datasets and benchmark scores, a plausible figure is the easiest thing to invent and the hardest to notice. Prose can be vague and still be honest; a number cannot, so every figure is matched against the source before publication.

Read the paper, not the abstract

Abstract-only notes restate the paper's own marketing sentence. The body is taken from arXiv's HTML rendering, with references and appendices removed and tables kept because that is where the measured values live. No single section may take more than a third of the extract, so a long method section cannot crowd out the results.

Treat the archive as the record

Each day is stored as JSON and every other output is rendered from it, so a change to the page design re-renders every past day instead of stranding old ones in an old layout. The rebuild makes no model calls, which also means it needs no credential.

Degrade instead of failing

Nobody watches a job that runs at seven in the morning. An unusable selection falls back to the newest three and says so, a rate limit is waited out on the provider's own retry interval, a paper with no HTML rendering falls back to its abstract, and a failed run leaves the published site exactly as it was.

Verification evidence

99 automated tests pass without a network call or a model call, covering the invented quotes and the invented benchmark scores the two checks exist to catch.
A live run published three papers in about thirty seconds, two read from the paper body and one from its abstract, with the page stating which was which.
The checks were observed rejecting real output rather than only test fixtures: one summary in that run failed the citation check and was published marked unverified instead of being presented as checked.
The widening window was added after measurement, not assumption: a Saturday run found zero papers within 48 hours and 120 within 72, and would have published nothing.
The published site was exercised in a browser at desktop and 375px widths: the filter, the theme control, every disclosure and the copy actions work, and no element overflows the viewport.

verified-2026-08-15

$ python -m arxiv_digest
120 candidates from the last 96h
2 read in full · 1 abstract only
✓ 3 summaries · 1 quote unverified
✓ published papers.pesanth.com

Limitations

The summaries are machine written. The checks prove that the quote and every figure came from the paper, not that the summary is a correct reading of it.
Papers without an arXiv HTML rendering fall back to the abstract, which produces noticeably thinner notes. One of the three in the first full run was in that state.
Selection quality is unmeasured. There is no evidence that the three chosen papers are the three most worth reading, only that the choice was made against a stated set of interests.
The service depends on a free hosted tier, so a rate limit or a retired model name is a dependency outside the project's control. A local model is supported as the fallback.
The archive lives on a single machine with no off-site backup, and the site asks search engines not to index it, so machine-written notes do not compete with this portfolio's own pages.