Automate the task. Audit every answer.
The productivity tax you can't skip is checking the model's work. Glasswork runs your task across a spread of open-source models, scores each on your task, picks the cheapest one that passes, and attaches a tamper-evident custody receipt to every single call — so you can hand off the work and actually trust it.
Narrated walkthrough: one task → 12 open models on Butterbase → the cheapest model that passes → the error surface vs a gold standard → a custody receipt you recompute in-browser, and a tampered one rejected. Captions: click CC in the player. Direct link: glasswork.butterbase.dev/demo.mp4.
A custody receipt only means something if the computation around it is reproducible. Glasswork's scoring and custody are fully deterministic — given a recorded output, the same rubric score and the same Merkle root recompute identically, every ingest. The model step runs at temperature 0 (a best-effort knob, not a guarantee) — which is exactly why we hash and bind the bytes that actually ran instead of trusting the model to repeat itself.
A Fractal Custody Object binds bytes, not just text — so the same spine scales past a single inference. An entire paper becomes a Fractal Custody Graph over all its parts: text, figures, tables, datasets, images, audio, any media. Bind every part and you can check the whole artifact against a gold standard — not "looks right," but provably the same, byte for byte. This demo is one leaf of that graph; the method is the same all the way up.
This isn't a mockup. Below is a real task run through 12 open models on Butterbase's gateway, graded against a hand-labeled gold set. Press Run, read the chart, then jump to Verify to recompute a custody receipt in your own browser.
Click any row to load its custody receipt in the Verify section. Pass bar (fixed before running): precision ≥ 0.80, recall ≥ 0.75.
| Model | Params | Precision | Recall | F1 | Cost / corpus | Verdict |
|---|
Most paper-to-result agents hand you an answer and stop. Glasswork hands you the answer plus its error surface: measured against a gold standard, exactly which facts each model recovered, which it missed, and a flag on any claim whose evidence isn't in the source (a would-be hallucination). Pick a model and see precisely where it fails.
Each inference emits two hash leaves — an operational leaf (model, tokens, cost) and a content leaf (hashes of the exact prompt and output bytes) — chained into a per-model Merkle root with RFC 6962 domain separation (leaf 0x00, node 0x01; duplicate-last pairing). Press Verify and your browser re-derives the root from the named bytes and compares. Reject iff mismatch.
SHA-256 via the browser's Web Crypto (SubtleCrypto). Nothing is sent to a server — the recompute happens on your machine. Download the full dataset (live JSON) to re-derive every root offline.
Five families through Butterbase's OpenAI-compatible gateway, same prompt, temperature 0 for reproducibility.
Precision penalizes any claim whose supporting span isn't in the document (hallucination); recall measures coverage against the gold set. No LLM judge in the loop.
The agent picks the lowest-cost model meeting the pre-set precision/recall bar on the corpus — a decision recorded as data.
Two hash leaves per call → a per-model Merkle root, domain-separated (leaf 0x00 / node 0x01). Published method, cited; verifiable in-browser.
AI gateway (inference), Postgres + Data API (catalog, gold set, 24 runs, 60 custody records), a serverless function (results), and the frontend deployment serving this page.
Glasswork is an open-method demo — the price that matters is the model spend it saves you. Same task, same pass bar, three strategies (measured, not estimated):
This honesty is the point. An agent you can trust is one that tells you the edges of what it measured.
The numbers are extraction quality on these documents (N shown), not a universal capability claim about any model.
A receipt tells you exactly which model produced which bytes at what cost. It does not say the answer is right, and it does not reproduce non-deterministic inference.
Savings are auditable computation from recorded token counts and the gateway's usage figures — trustworthy, but not an independently proven benchmark.
A model saying "this looks wrong" is a detector-arm signal. The custody spine tells you which bytes produced this. They are different things.
Real. Every number is 24 live inferences run through Butterbase's AI gateway and served from the app's own database — you can read it yourself at the public results function this page fetches.
No — and we say so on the page. Custody proves provenance: which model produced which exact bytes, at what token count and cost, tamper-evident. Whether the extraction is right is a separate question, measured by the precision/recall score.
Because bigger didn't win here. A 27B Gemma beat both the 70B and the 235B on recall, and the 70B Llama emitted invalid JSON on one document — unusable by an automated pipeline. You only find out by measuring on your task.
For every model that returned valid, parseable output, yes — none kept an ungrounded (hallucinated) span on this corpus. The 70B's score covers only the one document it could be parsed on; its recall (44%) honestly reflects the failure on the other.
All of it: inference (AI gateway), data (Postgres + auto-generated Data API), backend logic (a serverless function), and this website (frontend deployment). No other backend.
Yes — press Verify in the section above; your browser recomputes the Merkle root with SHA-256 and rejects a tampered one (try selecting openai/gpt-oss-20b). Or download the full dataset and re-derive every root offline.
Honest about what's genuinely wired. Butterbase runs the whole stack; on top, we ran the same task on Nebius, re-verified custody in a live Daytona sandbox, ran a pipeline on RocketRide Cloud, and loaded the graph into a live Neo4j AuraDB — every claim below is backed by a real artifact in the repo.
Every layer runs here: the AI gateway ran all 12 open models, Postgres + the Data API hold the catalog, gold set, 24 runs, and 60 custody records, a serverless function serves the dataset, and this site is a Butterbase frontend deploy. Built MCP-first, following the conventions of Butterbase's reference apps butter-support and butterbase-crm (schema → Data API → functions → deploy).
The custody layer applies this published method — domain-separated Merkle (leaf 0x00 / node 0x01) — and generates its own fresh demo data. Zenodo deposit, CC-BY / Apache-2.0, cited in the footer.
We ran the same extraction task on Nebius directly. Route-comparable: gemma-3-27b-it scored 0.94 recall on both providers, 4 of 5 shared models agree, and the flaky 70B failed on both. Real data in the repo (partners/nebius_compare.json).
Custody independently re-derived in a live Daytona sandbox — 11/12 roots reproduced byte-for-byte and the tampered receipt rejected (verdict PASS). Attestation + sandbox id in the repo (partners/daytona_attestation.json).
Ran the claim-extraction task as a real RocketRide Cloud pipeline — webhook→questions→llm_nebius→answers (node llm_nebius on Nebius Llama-3.3-70B). The Cloud returned the extracted claims (task tk_83947c…) — a genuine third route. Receipt in the repo (partners/rocketride_pipeline_receipt.json).
The decision + custody graph is loaded into a live AuraDB instance (Instance01): 58 nodes, 86 edges — Model, Run, CustodyRoot, Decision, Doc, Citation. The loadable Cypher/JSON is also in the repo (graph/decision_graph.cypher).
Next: bring custody roots into a CI/CD gate — fail the build if a receipt doesn't re-derive, so provenance is enforced before anything ships.
Thoughtful agents for productivity. Glasswork is our entry — a productivity agent you can hand off to and audit, end to end.