Architecture
A. Receipt reader
- 1
Read
The photo goes straight into Qwen3.5-4B, a vision-language model. Its vision encoder turns the image into up to ~800 image tokens and the language model writes the receipt as JSON. There is no separate OCR step.
receipts/vlm.py - 2
Sample
Besides the normal (greedy) reading, the model samples four more, giving five candidates.
vlm.generate - 3
Check
Each candidate is tested against the receipt's arithmetic. The checker keeps the one that passes the most rules; if even that one breaks a rule, the receipt goes to human review.
receipts/consistency.py - 4
Fine-tune (offline)
QLoRA trains a small LoRA adapter (0.67% of the weights) on 800 real CORD receipts while the 4-bit base model stays frozen; the adapter plugs into the language model at inference.
receipts/train_qlora.py
B. Financial filing search
- 1
Index (offline)
The 84 filings are split into 12,013 pages and each page is embedded with Qwen3-Embedding into a 1024-number vector.
rag/download_financebench.py - 2
Route
The question's company (name, ticker or nickname) and fiscal year select the matching filings.
financebench.build_router - 3
Search
The question is embedded the same way, and exact vector search finds the closest pages among the routed filings, in FAISS or in PostgreSQL with pgvector, where routing is a SQL WHERE clause.
rag/index.py · rag/pgstore.py - 4
Rerank
Qwen3-Reranker reads the question and each of the top 30 pages together and reorders them; the output is the evidence pages with their page numbers.
financebench.py