Technical overviewApplied AI engineering and output verification

Daily Digest

One page a day from three sources, where every claim in a paper summary is checked against the paper first.

Source code

The repository is private. The architecture, engineering decisions, and verified evidence are documented below.

Open live site(opens in a new tab)Private repository
Role
Solo engineer: source integration, model pipeline, verification checks, published site, scheduled deployment
Evidence status
Live at papers.pesanth.com, running unattended each morning on a free hosted model tier, with a local model as fallback
Verified
2026-09-15

01

Purpose and scope

Each morning the digest builds one page from three sources. It reads the day's new AI papers on arXiv and summarizes each from the HTML body; before publishing, the model must quote the fragment a result came from and every figure must appear in the source. Hacker News and Contrary Research are added as curated one-line picks. It is live at papers.pesanth.com.

What it is used for

  • Keep up with a field posting a hundred-plus papers a day, in minutes.
  • Decide whether a paper is worth opening, from its mechanism and results.
  • See at a glance which claims were checked against the paper.
  • Python
  • LangGraph
  • arXiv API
  • Prismic API
  • Groq
  • Ollama
  • Docker
  • systemd

02

Architecture

Sources

Feed

arXiv Atom API

One request per run, window widens when thin

Full text

HTML rendering

Paper body, references and math stripped, tables kept

Picked feed

Hacker News

Day's stories, a few picked with a reason

Picked feed

Contrary Research

Editorial deep dives via Prismic, picked the same way

Read · one paper at a time

Pipeline

Model step

Selector

Shortlists papers and picks the other sources

Model step

Summarizer

Mechanism, method, figures, limitations

Inference

Model backend

Hosted free tier or local model

Check · paper summaries only

Verification

Gate

Citation check

The quoted fragment must appear in the source

Gate

Figure check

Every number must appear in the source

Policy

Retry and disclose

One retry, then publish marked failed

Write · then render

Archive and publication

Record

Day archive

One JSON file per day, the record

Web interface

Static site

Filterable index plus Hacker News and Contrary tabs

Automation

Scheduled run

A systemd timer each morning

Public ingress

Cloudflare Tunnel

Outbound-only ingress to read-only nginx

Text equivalent: One morning run gathers three sources: a model shortlists new arXiv papers and summarizes each from arXiv's HTML rendering, two checks run against that source text, and the result is archived as JSON. Hacker News and Contrary Research are picked with a one-line reason and not summarized. The static site is rebuilt from the archive and served behind an outbound-only Cloudflare Tunnel. The run is orchestrated as a LangGraph state graph, and each paper is summarized by a generate, verify and retry loop whose conditional edge returns to the model when a check fails; the model client and its rate pacing sit outside the graph.

03

Engineering decisions

Make the model quote the paper

Requiring a verbatim fragment turns a half-remembered summary into a string comparison; a quote that cannot be found is published marked unchecked.

Check the numbers, not just the quote

Once notes name datasets and scores, a plausible figure is easy to invent, so every number is matched against the source before publishing.

Pick the other sources, do not summarize them

Hacker News and Contrary are chosen with a one-line reason and never summarized, because a one-line reason makes no claim to verify.

Run the pipeline as a LangGraph state graph

The three sources and the publish step are nodes joined by a conditional edge that publishes only when a source produced something, and each paper's summarize step is a generate, verify and retry loop as its own graph. The model calls and their rate pacing stay outside the framework, so the run still fits the free tier's daily budget.

04

Verification evidence

  • 267 automated tests pass without a network call or a model call, covering the invented quotes and figures the checks exist to catch and both state graphs directly.
  • The checks were seen rejecting real output: one live summary failed the citation check and was published marked unverified.
  • An unattended morning run archived five papers and five Hacker News picks on its own.
verified-2026-09-15
$ python -m arxiv_digest
papers: 120 candidates in the last 96h
5 summaries · 4 read in full · 1 abstract
✓ checks: 1 quote unverified, flagged
picks: 5 Hacker News · 2 Contrary
✓ published papers.pesanth.com

05

Limitations