Architecture overview
Evidence base: live checkout reads of ARCHITECTURE.md (2026-09-01 review),
README.md, wrangler.toml, Dockerfile.api, public/llms.txt, and the
serving/ingestion entrypoints. Counts and schedules drift; treat numbers as
snapshots unless you re-run the cited commands.
Topology
Cloudflare Worker (worker.js) owns syntology.ai: static assets from
public/ (wrangler.toml [assets], run_worker_first = true so security
headers apply), billing/session, trial OAuth, MCP reverse proxy, graph-agent
proxy to App Runner, and several fixed Cypher reads against Aura’s HTTP
Query API (pulse, slice, neighborhood, atlas-api, code-library).
AWS App Runner runs uvicorn main:app --workers 1 from Dockerfile.api:
FastAPI surface for /papers/* (local corpus; empty in container by design),
thin /graph/<template> endpoints, /graph/ask (NL pipeline), draft/commit
agent, reviewer-bot, and mounted /mcp (mcp_server.py). Neo4j via
syntology_api_reader. Bedrock via task role (not static keys in the image).
Neo4j Aura Professional is the single graph. Writers on the Mac use
syntology_loader through WriteLedger (+ advisory lock file under
groundwork/19_structured_provenance/). Web tier uses syntology_web over
HTTP Query API only.
Local canebrake writer: acquisition → GROBID → parse → citation resolve → methods extraction → load; enrichment backfills; code harvest lane; QC suite; ~10 launchd jobs. Ops dashboard on localhost (read-only collectors).
CF Worker ──X-Syntology-Internal-Key──► App Runner ──reader──► Neo4j Aura
▲
Local Mac ──WriteLedger / syntology_loader────┘Data flow (stage sketch)
Directory-to-directory transforms. Stage numbers through 06 are sequential;
from 07 onward numbers are often write-order, not flow position.
| Phase | Scripts (examples) | Reads → writes |
|---|---|---|
| Acquire | acquire_*.py, comprehensive_paper_acquisition.py, 05/07/10/11 | venues/arXiv/OR → metadata + PDFs |
| Extract | 01_grobid_extract / 09_binned_parse, 02_parse_tei, 02b_backfill_from_s2 | PDFs → TEI → parsed JSON (+ review queue) |
| Cite / review | 04 / 04c, 03_openreview_sync | parsed → *_cited*, reviews_synced.json |
| Orchestrate | 06_process_new_papers.py | discovery JSON → full chain (subprocess per stage) |
| Methods | run_methods_full_corpus.py, apply_name_shape_veto.py | cited → extraction results |
| Load | load_papers_into_neo4j.py (+ other load_*.py) | cited + methods → Neo4j (MERGE, review gate, ledger) |
| Enrich | 15–22_*, embeddings | graph / corpus backfills |
| Code lane | discover → classify → harvest → probe → roles → verification ladder | GitHub + graph → CodeSample, IMPLEMENTS / HAS_*_IMPL |
| Serve | main.py + query_engine/ | graph → API/MCP responses |
Exit-code contract (everywhere): 0 clean · 1 findings · 2 drift ·
3 unaccepted exclusions · 4 unresolved partials. 06 treats 3/4 as
“refusing to overclaim; continue.”
Provenance discipline
- Derived files: stamp via
provenance.py(R1). - Filters account for exclusions (R3); review queue gates load (R4).
- Bulk graph writes:
WriteLedger+--intent(no placeholders). - Edges carry
provenance;CodeSample.verification_levelmust be backed by evidence on the node or readers treat as 0. - Reviews: raw per-reviewer rows; never averaged; no reviewer identity.
Identity spaces
Three disjoint paper identity spaces coexist: arXiv-keyed nodes, S2-keyed
citation stubs (s2_only), and venue-native ids stored in arxiv_id (CVF,
ACL, …). arxiv_ids.is_arxiv_id() is the shared definition of “usable as an
arXiv id.”