DEV Community: Advik Divekar

ESA's satellite is dead. Its ghost: 2,123 signals. TabPFN beat the published SOTA; 83¢ of Tinker stopped an LLM speaking for it

Advik Divekar — Sat, 31 Oct 2026 22:18:15 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

2022-06-02 03:17:03 UTC   I_PD3_THETA   0.0540137
2022-06-02 03:17:08 UTC   I_PD3_THETA   0.0462915
2022-06-02 03:17:13 UTC   I_PD3_THETA   0.0000000
2022-06-02 03:17:18 UTC   I_PD3_THETA   0.0000000
2022-06-02 03:17:23 UTC   I_PD3_THETA   0.0025708

Ten seconds of nothing. Recorded by a light sensor on a spacecraft that no longer exists.

ESA's OPS-SAT burned up in the atmosphere on 22 May 2024. Its engineers have moved on. Nobody is left to ask what happened at 03:17:13.

Ask a chatbot and it will tell you anyway. A sensor fault. A reboot. An eclipse. It will sound certain, and it will be making it up, because nothing in the archive says why.

So I built a séance with one house rule: nobody speaks for the dead.

The Last Satellite hands the archive to two open models:

TabPFN listens to every signal OPS-SAT left behind and decides which ones look wrong. On the official benchmark, it beats the best model in the paper that published the dataset, without ever running a training loop.
Qwen3-8B, fine-tuned on Tinker for 83 cents, reads the mission's documents and is only allowed to say what they say.

Ask it why the light went out at 03:17:13, and you get the most honest sentence a medium can give:

"Root cause of this signal behaviour is not documented in the archive."

One confession before we start. My model looked at those ten seconds and called them 0.098: probably fine. It was wrong. That miss is in the demo, on purpose.

What You're About to Read

The TabPFN result: on the official OPSSAT-AD test split (529 segments, run exactly once, logged), TabPFN-v2 scored AUCPR 0.996, F1 0.982, MCC 0.977. The best published model (an FCNN in Scientific Data, 2025) scored 0.979 / 0.946 / 0.932.
The same features as the paper, no training loop: using only the paper's 18 features, TabPFN still beats it: 0.989 / 0.950 / 0.937. It also beats my tuned Random Forest, XGBoost and LightGBM by about 0.02 AUCPR.
The Tinker result: one LoRA on Qwen3-8B took structured extraction from mission documents from 65.3% to 93.0% field-level exact match (validation), for 83 cents all-in. On fully held-out test documents it went from 72.2% to 86.3% (paired p = 6×10⁻¹⁶).
- Evidence copied verbatim: 81% → 100%.
- Investigation notes passing every automated check: 56% → 100%.
The cost: $0.22 for the final model. $0.83 for everything on Tinker: every run, the learning curve, a run I killed on purpose, and every evaluation.
The honesty record: 1 false alarm and 3 missed anomalies in 529 segments (the false alarm and a miss are in the demo, on purpose), and 0 invented root causes from either model. I also show where I missed my own 95% target, and the one task fine-tuning made worse.
You keep it: the adapter runs offline on a laptop through llama.cpp, with no network and no API key. It agrees with the hosted model on 100% of fields.

The problem: a dead satellite can't explain itself

OPS-SAT was ESA's flying laboratory: a CubeSat that teams outside ESA could upload experiments to. Over four and a half years it sent home thousands of telemetry fragments. Then, in the words of its mission-end coverage, it was "impacted by the recent unexpectedly strong increase in solar activity that began in early 2024", and it came down early.

Some of what it sent is now OPSSAT-AD:

2,123 telemetry segments from three magnetometers and six photodiodes;
each segment labelled nominal or anomalous by three ESA operations engineers and two ML experts.

The labels say that something was wrong. Nothing in the archive says why.

That's the trap. Point a modern AI at this archive and ask "what happened to segment 204?" and it will give you a confident, plausible, completely made-up story. A sensor fault. A reboot. Cosmic rays. The data can't contradict it, because the people who could are gone.

The failure you have to fear isn't "the model is wrong." It's "the model invents a memory the archive doesn't hold, and you believe it."

What exists today, and the gap

Option	What it gives you	What it can't tell you
Read the paper	A benchmark: FCNN at AUCPR 0.979	Which of these segments to look at first, or what the documents say about them
Train your own deep model	Maybe the paper's number, after a training loop	Whether its probabilities mean anything (calibration)
Ask a general chatbot	A fluent explanation	Whether a single word of it is in the archive
The Last Satellite	A calibrated anomaly queue, cited documents, a card per segment	It tells you what it doesn't know: "Root cause not documented in the archive."

What I built

The Last Satellite is a Mission Memory Engine: an archival investigation console with two open models, one evidence layer and one rule.

TabPFN-v2 reads every archived segment and gives a calibrated probability that it's anomalous. It ranks all 529 test segments into prioritise / queue / archive and shows which measured characteristics drove each score.
Qwen3-8B + a Tinker LoRA reads the mission's documents. It extracts 8-field, source-linked records, rebuilds the timeline (event dates kept apart from publication dates), explains telemetry terms with citations, and writes investigation notes split into KNOWN and NOT KNOWN.
Every statement on screen is either [measured] from the raw signal or [cited] from a document, verbatim.
A root cause is never asserted unless a source states one, and the archive has no per-segment incident reports. So it says so.
The analyst has the last word: Confirm, Reject or Investigate, stored as feedback.
Export: one click bundles documents, records, cards, feedback and every evaluation file into a portable mission-memory archive.
The console is one cinematic scroll from orbit to re-entry. Behind it, a shader-lit Earth whose city lights burn like candles, and a tiny CubeSat whose orbit decays as you read. At the end the planet warms to marigold and the satellite streaks in.

Why it fits Ghost in the Machine: this week runs from Halloween night into Día de los Muertos, when the line between past and present is supposed to be thinnest. OPS-SAT is a ghost in the most literal sense: a machine that no longer exists, still talking through 303,493 archived readings.

Día de los Muertos isn't about being afraid of the dead. It's about remembering them accurately: the ofrenda holds their real photos, their real favourite foods, their real stories. So the console is built like one. Marigold and candlelight, not horror. The satellite's own words are the offerings, every one sourced. And there's one rule a séance usually breaks: nobody gets to put words in the ghost's mouth. Not me, not the language model. Measured, cited, or unknown.

Demo

The video above is the whole tour. The console talks to the live FastAPI backend when it's running. Without it, it falls back to a labelled archived snapshot of real API responses, so it works anywhere.

The signal that should not be there. Pick one of six candles: six real segments from the official test split, labels hidden. Step through the signal, the 18 measured features, TabPFN's probability, the documents, the most similar past segments, and the investigation card. Then reveal the truth.

And the one it missed. The demo set deliberately includes the false alarm and a missed anomaly. Notice the banner on the summary: the fine-tuned interpreter's draft was rejected by the validator, so the card fell back to plain, deterministic text. More on that below.

Ask the archive. It tells you what it doesn't know.

The queue. 529 segments ranked by TabPFN, each with a card and three buttons: Confirm, Reject, Investigate.

Code

advikdivekar / wake

Mission Memory Engine for OPS-SAT (archival): TabPFN telemetry analyser + Tinker-tuned Qwen3-8B interpreter.

The Last Satellite: Mission Memory Engine

Its mission ended. Its memories did not.

OPS-SAT was ESA's flying laboratory, a CubeSat that re-entered the atmosphere on 22 May 2024. This project is an archival analysis system built on its real, public telemetry and documents. It has two halves:

Telemetry Analyser (TabPFN). For each archived telemetry segment it estimates P(anomalous), explains which measured characteristics drive the estimate, and ranks segments in an investigation queue.
Mission Memory Interpreter (Qwen3-8B + LoRA fine-tuned on Tinker). It turns public mission documents into structured, source-linked records. It keeps what is known separate from what is not, and it never invents a root cause.

An evidence layer joins the two into investigation cards. Every statement on a card is either measured or cited, and an analyst can confirm, reject or investigate it.

Nothing here is live. All numbers below are produced by scripts…

View on GitHub

How I built it

The rule behind all of it: the models read, code measures, and nothing is claimed that the archive cannot show.

Language models are brilliant readers and terrible witnesses. TabPFN is a superb estimator, but a probability is not an explanation. So each job goes to whatever can be held accountable for it, and five independent checks stand between the models and the screen:

The output parses into the 8-field schema (Pydantic).
Every quoted sentence exists verbatim in its source document (substring check).
Channel IDs and dates are cross-checked by an independent regex reader that never saw the model.
A "root cause" must be stated, unhedged, by a cited source. "May have been caused by…" maps to unknown.
Signal facts come from plain code, never from a model: zero runs, gaps, peaks, noise.

The data: real, public, and licence-aware.

Telemetry: OPSSAT-AD from Zenodo, md5-verified, CC-BY-4.0.
Documents: 20 public sources (the benchmark paper via Europe PMC's open-access API, the dataset code, Wikipedia, ESA pages, news reprints) chunked into 304 passages. Full text is stored only where the licence allows (CC-BY, CC-BY-SA, MIT); everything else gets ≤300-character excerpts and a link.
Timeline: every one of its 11 entries is a verbatim quote, checked against the stored text before it may appear.

Best Use of TabPFN

0.979 → 0.989, with no training loop

The published state of the art on this dataset is a fully connected neural network, trained, tuned and reported in Scientific Data in 2025. TabPFN-v2 never ran a training loop on this data. I handed it the training rows and the paper's own 18 features, and it beat the paper on the official test split. Then I engineered better features, and it beat the paper by more.

That's the point. The anomaly detector at the heart of a mission-memory console shouldn't need a GPU, a hyperparameter sweep and a week. It should need a table.

Official test split (529)	AUCPR	F1	MCC
FCNN, best published (Sci Data 2025)	0.979	0.946	0.932
TabPFN-v2, paper's 18 features	0.989	0.950	0.937
TabPFN-v2, selected features	0.996 [0.989–1.000]	0.982 [0.963–0.996]	0.977

Brackets are 95% bootstrap CIs over test segments.

My goal wasn't a big number. It was a big number I could defend line by line. Here is exactly how.

1. Reproduce the paper before you try to beat it

I downloaded OPSSAT-AD, verified both md5 checksums, and counted everything myself:

2,123 segments, 303,493 points, 9 channels, 20.4% anomalous;
1,594 train and 529 test segments.

The paper's text says 1,494 training segments; its own Table 1 and the released data both say 1,594.

Then I ported the published feature code. It reproduces 16 of the 18 features exactly (99.97% of all values). On the way it caught a nasty bug of mine: the raw file has 22,115 duplicate timestamps, and a non-stable sort silently changed every derivative feature.

2. Re-verify the bar programmatically

I didn't copy the benchmark from a PDF by eye. The script pulls the paper's full text (JATS XML) from Europe PMC and checks every number in Table 3. The bar to beat is FCNN: AUCPR 0.979, F1 0.946, MCC 0.932 (best unsupervised: MO-GAAL at 0.779). The 2025 correction notice only re-ordered figures; no table changed.

3. Choose everything on training data only

I ran 5-fold CV on the training split, grouped so neighbouring segments of a channel never straddle folds. Each candidate feature group had to earn its place:

Kept: zero runs & gaps, autocorrelation and robust spread, which together raised CV AUPRC from 0.9704 to 0.9768.
Dropped: spectral features and channel identity, which didn't help.
Ensemble size: 8 estimators won (CV 0.9781).

4. Let the operator choose the trade-off

Every threshold comes from training out-of-fold predictions, before the test split was opened:

max-F1 at 0.617;
prioritise at ≥ 0.80, where precision is ≥ 0.95;
queue at ≥ 0.05, where recall is ≥ 0.98.

5. Run the test split once

A guard logs every test-set run and refuses to run the same configuration twice. One shot:

415 nominal segments correctly left in the archive, 110 anomalies correctly flagged, 1 false alarm, 3 misses.

6. Checked before trusted

Same split, same features, same folds. Five baselines and a heuristic:

Official test split	AUCPR	F1	MCC
TabPFN-v2	0.996	0.982	0.977
Random Forest	0.994	0.973	0.966
XGBoost	0.993	0.968	0.960
LightGBM	0.990	0.969	0.960
Logistic regression	0.987	0.956	0.944
Isolation Forest (unsupervised)	0.401	0.499	0.362
"Lots of zeros or a gap" rule	0.293	0.527	0.414

A win, but on my features not a thrashing. Good features lift every model. Against Random Forest, TabPFN is a statistical tie (McNemar p = 0.63; AUPRC difference +0.002, 95% CI −0.0005 to +0.006). Against LightGBM and XGBoost, the paired-bootstrap AUPRC differences sit just above zero, and against logistic regression TabPFN wins with p = 0.03.

That's why the like-for-like table at the top matters most. On the paper's own features, TabPFN beats the tuned trees by ~0.02 AUCPR, and the paper's neural network, out of the box.

7. A probability that means what it says

The probability is the investigation queue, so it has to be honest. Brier score on test: 0.0073.

8. Things I got wrong first

TabPFN refused to run on CPU with more than 1,000 training rows. Apple's MPS backend was faster anyway, with identical AUPRC.
TabPFN and XGBoost segfault when loaded in the same process on macOS, because two OpenMP runtimes collide (exit 139). Now they run as separate pipeline stages.
The newest TabPFN weights want an account and a non-commercial licence. TabPFN v2 downloads without either (Prior Labs License: Apache-2.0 + attribution), so every number here is v2.
My first telemetry plot drew 553 "gaps" that were really just artefacts of downsampling. Gaps are now computed on raw samples only.

Best Use of Tinker

65.3% → 93%, for 83 cents, and it stopped inventing

Qwen3-8B out of the box makes up channel IDs and paraphrases quotes, and with a confident tone that is exactly how false memories get made. It scored 65.3%. Same model, same prompt, same validator: after 83 cents of Tinker, all-in, it scored 93.0%.

I didn't need a model that knows everything about spacecraft. I needed one that copies instead of inventing, and fine-tuning is how a small model learns that.

	base Qwen3-8B	Tinker LoRA
Field-level exact match, validation	65.3%	93.0% [90.7–94.8]
Field-level exact match, held-out test documents	72.2%	86.3% [83.2–88.9]
Channel IDs correct	70.8%	100%
Evidence copied verbatim from the source	81.1%	100%
"Unknown" used correctly	82.1%	99.8%
Whole record exactly right	0%	44.4%
Investigation notes passing every check (task D)	56%	100%
Errors caught by the independent validator	40	0
Unsupported root causes	0	0

My goal wasn't the biggest number. It was a real number at the lowest price I could defend. Here is exactly how.

1. Freeze the metric before anything trains

Before any model saw any data, I froze the headline: field-level exact match on held-out documents, scored on raw output before any repair. Companion metrics:

valid-schema rate;
verbatim-evidence rate;
unsupported-root-cause count;
correct use of "unknown";
Wilson CIs;
a paired McNemar test against the base model.

2. Shrink the bill before you shrink the model

Tinker bills per token, so the output is 8 compact lines that code turns into Pydantic JSON:

channel_id: CADC0873
sensor_type: magnetometer
units: ?
event_date: 2024-05-22
observation: On 22 May 2024 the magnetometer I_B_FB_MM_1 reported a run of zero values.
operational_response: ?
root_cause: ?
evidence: On 22 May 2024 the magnetometer I_B_FB_MM_1 reported a run of zero values.

A full training example, prompt and passage included, is about 380 tokens.

3. Price the options, then pick

Training per million tokens:

Qwen3-8B: $0.44.
Qwen3.5-4B: $0.737. A smaller model is not automatically a cheaper one.
GPT-OSS-20B: $0.396, but it's a reasoning model whose thinking tokens would cost more at sampling time and break the format.

4. Train: rank-16 LoRA, loss on the answer only, 5 minutes

renderer = renderers.get_renderer("qwen3_disable_thinking", tok)
mi, w = renderer.build_supervised_example(messages, train_on_what=TrainOnWhat.LAST_ASSISTANT_MESSAGE)

fb = tc.forward_backward(batch, loss_fn="cross_entropy")
op = tc.optim_step(tinker.AdamParams(learning_rate=lr_t, beta1=0.9, beta2=0.95, eps=1e-8))

Data: examples across four tasks (extraction, timelines, terminology with citations, investigation notes).
Schedule: batch 16, linear LR decay, 3 epochs, a checkpoint every epoch. The final run took 5.1 minutes.
Split by document family, never by chunk: the test documents (the benchmark paper plus Wikipedia) never appear in training, and the paper's arXiv preprint is excluded everywhere as a near-duplicate.
Budget guard: it prices every job from local token counts before submitting and refuses anything over the cap. I started with $0.93.

5. 65.3% → 93.0%: find the knee of the curve

I didn't train once and hope. I trained in stages, scored every run on the same validation set, and priced each one:

Run	Est. cost	Validation field exact match
base Qwen3-8B, untrained	n/a	65.3%
run 1	$0.025	90.8%
run 2	$0.049	91.8%
run 3	$0.097	92.2%
final model	$0.224	93.0%

The first 2.5 cents of training took the model from 65.3% to 90.8%. Every run after that added another point or so, and the final model landed at 93.0%, all inside an 83-cent budget.

6. Validation error analysis found my bugs

Midway through, most remaining "errors" were the model correctly refusing to quote code blocks, footnotes and markdown bullet lists that my labelling rules had picked as "observations".

I tightened the spec to prose-only sentences.
I killed a half-finished run rather than pay to finish it on bad labels.
I retrained once.
The test set stayed sealed the whole time.

7. Is the gain real or noise?

Paired, cell by cell, on the same 576 field cells: 95 cells only the fine-tune got right, 14 only the base got right. Exact McNemar p = 5.8×10⁻¹⁶.

The base model's 40 validator catches are what the validator is for: channel IDs and quotes that don't exist in the source. The fine-tune made zero. Even after the validator repairs the base model's output, it reaches 78.5%, still below the fine-tune's unrepaired 86.3%.

8. Rent the training. Keep the weights.

I exported the adapter as PEFT (one tensor rename: unembed_tokens → lm_head), converted it to GGUF with llama.cpp, and ran it over a Q4_K_M Qwen3-8B base:

92.5% field exact match on held-out passages, 100% field-for-field agreement with the same checkpoint on Tinker, 4.7 s per passage. M3 Pro, 18 GB, Hugging Face offline, no Tinker key in the environment.

The whole bill

Item	Est. cost
Learning-curve runs (runs 1–3)	$0.17
A run I stopped at 65/84 steps after finding label bugs	$0.17
Final model (3 epochs)	$0.22
Downloadable sibling adapter (see Known edges)	$0.10
All evaluation on Tinker	$0.17
Everything on Tinker	$0.83

Token counts at Tinker's list prices, logged before every job. Base-model validation ran locally for free.

Known edges (measured, not hidden)

I'd rather you hear these from me, with numbers.

I missed my own target. I set 95% before training. The model got 93.0% on validation and 86.3% on test. Most of the gap is one judgement call: which sentence is "the observation" in an unfamiliar document.
Fine-tuning made one task worse. Citing the right source for a telemetry term fell from 96.2% to 82.7% (p = 0.016). That task had the least training data, and the tuned model learned to answer "not in the archive" too eagerly.
The offline adapter is a sibling. Tinker doesn't let you download sampler checkpoints, which is all my final run had saved. So the offline adapter is a smaller sibling configuration, retrained with a downloadable checkpoint (validation 92.3% vs 93.0%). The training script now saves both kinds.
The labels are a spec, not human annotation. "Exact match" means agreement with a deterministic spec on unseen documents. Some fields (units, operational responses) appear in only a handful of held-out passages.
The validator rejects my own model. In the demo, 4 of 6 investigation cards fell back to deterministic text. In one, the interpreter wrote that the root cause "is documented", which contradicts its own NOT KNOWN list. The guard caught it. I'd rather show a plain card than a confident wrong one.
TabPFN's edge over Random Forest on my engineered features is a statistical tie. Its clear win is like-for-like, on the paper's features.

What went wrong on the way (and made it better): my base-model comparison was unfair at first. The prompt never named the eight fields and the parser rejected markdown, so I fixed both before training anything. A root-cause rule briefly treated "due to the underlying nature of the mission" as a cause, and a Q&A ranking tweak made one answer prettier but quietly removed the "cause unknown" warning from others. Both were caught and reverted.

Why does open innovation matter?

A closed API gives you an answer. Open weights and open data give you a way to check it. For a project whose whole job is refusing to invent memories, that difference is everything.

Where open beat closed, concretely:

What I needed	Closed API	What open let me do
Stop the model inventing channel IDs	Write a longer prompt and hope	Fine-tune the weights: invented IDs went to zero, for $0.22
Trust the anomaly probabilities	Take the vendor's word	Measure calibration myself on the public test split (Brier 0.0073)
Beat a published benchmark	Not possible to verify	Reproduce the paper's features to the decimal, then beat it like-for-like
Keep it running after the hype	Depends on the vendor staying up	Adapter + TabPFN run offline on a laptop, no key, no meter

The reasons behind that table:

I could audit the archive. The dataset authors published the data and the feature code, so I reproduced their features to the decimal and found a typo in their split count. A ghost story you can fact-check is a different kind of story.
I could change the model, not just the prompt. Base Qwen3-8B invented channel IDs. Twenty-two cents of LoRA training stopped it.
I could test the probability. With TabPFN's weights and the public test split, I measured calibration myself (Brier 0.0073) instead of trusting a number someone printed.
No meter running, nobody to deprecate it. The adapter and TabPFN run on the laptop that holds the archive. After a spacecraft dies, its memory shouldn't depend on someone else's API staying up.
Memory should be a commons. OPS-SAT existed because ESA opened a satellite to outsiders. Its memory should stay just as open.

The honest caveat: training ran on Tinker's servers, on public documents only. At run time, nothing leaves the laptop.

My agent session

I built The Last Satellite with Claude Code as a pair, from a written 12-hour plan with a cost estimate up front, a budget guard on every Tinker job and a guard on every test-set run.

It read the sources before writing code. It read Tinker's and TabPFN's docs, re-verified the paper's benchmark table from the full text, and read the original feature code before porting it.
It caught its own mistakes:
- a duplicate-timestamp sort bug;
- an unfair base-model prompt;
- label bugs, found through validation error analysis;
- a root-cause rule that was too generous.
It stopped spending when the data said to. It killed a half-finished run on bad labels, and when the final checkpoint turned out to be undownloadable, it asked me before spending the last of my credit.
It reverted its own "improvement" when it made the honesty guarantees worse.
It kept a running cost ledger from the first job.

Prize categories

Best Use of TabPFN. TabPFN is the engine of the console, not a bolt-on: every probability, every queue position and every "top measured characteristic" comes from it.

Official OPSSAT-AD test split, one logged run: AUCPR 0.996, F1 0.982, MCC 0.977, against the best published 0.979 / 0.946 / 0.932.
On the paper's own 18 features, with no training loop: 0.989 / 0.950 / 0.937, beating the paper and the tuned tree ensembles.
Rigour: calibrated (Brier 0.0073), with thresholds chosen on training data only, and compared against five baselines with bootstrap CIs and McNemar tests, including the tie I report against myself.

Best Use of Tinker. The LoRA fine-tune (loss on answers only, per-epoch checkpoints), the learning curve, a budget guard, held-out evaluation with paired statistics, and the adapter download behind the offline story all ran on Tinker. $0.22 for the final model, $0.83 for everything.

Same base model, same prompt, same validator: extraction 65.3% → 93.0% on validation (72.2% → 86.3% on held-out test documents), verbatim evidence 81% → 100%, investigation notes 56% → 100%, and 0 invented root causes.

Built solo, with Claude Code, over one long Halloween night, finished as Día de los Muertos began.

Two open models. Eighty-three cents. One rule: measure it, cite it, or call it unknown. If that sounds like the right way to remember a satellite, a ❤️ or 🦄 helps more people find it.

This Día de los Muertos, light a candle for a machine you loved that's gone: an old phone, a first laptop, a server that ran for a decade. What would you want its archive to remember? Leave it in the comments. 🕯️

A small, not-so-smart model went from 10% to 99.6% at reading property deeds for $1.15, and it says 'check this line' instead of bluffing

Advik Divekar — Sun, 25 Oct 2026 16:39:52 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

N 86°29' E, 103.2 feet.

That is one line of a property deed. Walk the rest of the lot and you should end up where you started.

You end up 27 feet away.

One digit is wrong. Maybe a clerk typed 103.2 for 130.2. Maybe the scan smudged. Maybe the AI that read it got creative. The drawing you'd make still looks fine, and nobody finds out until the survey crew does.

DeedReader reads that deed, closes the shape in plain code, plots it on a real satellite map, and when something doesn't add up it refuses to draw it. It stamps the result NEEDS HUMAN REVIEW, points at the line, and tries every one-digit fix to find the one that closes the parcel. In the example above: 103.2 → 130.2.

What You're About to Read

The result: one Tinker LoRA on Qwen3-8B moved exact deed reads from 10% (6/60) to 99.6% (2,988/3,000, 95% CI 99.3–99.8%) on fresh deeds. Same base model, same prompt, same checks.
The cost: $1.15 and 16 minutes of training for the final model. $0.17 of training already reaches 99.4%. The entire project, every baseline and every evaluation included, came to about $3.80.
The safety record: on the same 60 deeds, wrong-and-unflagged answers went from 5 to 0. Across 3,000 fresh deeds, 10 of the 12 imperfect outputs were caught by parsing and closure alone, and the other 2 are a label quirk in my own generator (same physical length, written two ways).
The design rule: the model reads. Plain, tested code does every piece of math. Five independent checks must agree before an answer is allowed to look confident.
You keep it: I downloaded the 369 MB adapter and ran it offline on a laptop. 150 of 150 deeds read exactly right, no network, no per-deed fee.

The problem: land is described in words

Every parcel of land has a legal description, and it reads like this:

Commencing at a cedar post at the corner of Tract 7, thence S 46°30′11″ W 1,011.25 feet to a corner post, thence along a curve to the right, R = 757.19 feet, to a point in the center of the road, thence N 23°16′52″ W 1,206.14 feet to a 4 inch concrete monument…

Someone has to turn that into a drawing. Convert every bearing. Plot every side, curved ones included. Check the shape closes. Work out the acreage. That someone is a surveyor, a title abstractor or a paralegal, and the job takes fifteen to thirty minutes a deed.

The scary part isn't the time. It's that one wrong digit quietly poisons everything downstream, and the drawing still looks plausible. The failure you have to fear isn't "the tool is wrong." It's "the tool is wrong and didn't tell me."

What exists today, and the gap

Option	What it gives you	What it can't tell you
By hand	A drawing you trust	It takes 15–30 minutes a deed
A regex parser	Instant and free	Only reads clean `N 41°00' E` calls, not 1800s prose or numbers spelled out in words
A base open model, no training	Reads messy prose	10% exactly right in my test, and 5 of 60 wrong with no warning at all
DeedReader	A closed parcel on a real map in seconds	Says NEEDS HUMAN REVIEW when anything doesn't add up, and points at the line

What I built

DeedReader is the lazy version of that job: paste, glance, approve.

The deed becomes a parcel on a real satellite map with true bearings and distances
Acres, square feet, perimeter and closure ratio, instantly
Click a sentence and its side lights up on the map. Click a side and the sentence lights up
Export to DXF for CAD, GeoJSON for GIS, or SVG
A deed that doesn't add up is never drawn as if it did. You get the line to check and the one-digit fix that would close it
Works fully offline from a downloaded adapter

Why it fits Slow and Steady: the tortoise didn't win by being fast. It won by never doing something stupid. A small model, a dollar of training, plain code for every number, and a system that would rather say "I don't know" than be quick and wrong. It automates the slowest job in land work, and earns trust the slow way: by showing its arithmetic.

Demo

The video above is the whole tour. The studio replays real recorded outputs of my fine-tuned model on deeds it never saw in training, while every safety check runs live in your browser.

Paste, glance, approve. A clean deed: every check passes, the parcel closes to 1:2.9 million, and the studio reports 43.44 acres. Click any sentence and its side lights up in the drawing. Export to SVG, DXF or GeoJSON in one click.

Break it on purpose. "Simulate a misread" corrupts a single number. Three independent checks fail at once, the banner turns red, and it names the line:

And the lazy payoff, side by side:

Code

advikdivekar / deed

DeedReader: a ~$1 fine-tuned model reads property deeds; plain code checks closure and plots the parcel

DeedReader

Paste a metes-and-bounds legal description. Get structured calls, a closure check done in plain code, a plotted parcel, and an honest NEEDS HUMAN REVIEW whenever anything is off. A fine-tuned open-weight model (Qwen3-8B + LoRA, trained on Tinker for about $2.5) does the reading; deterministic code does everything that must be right.

Built for the DEV Hacktoberfest Open-Source AI Challenge, Week 3. Read EVAL_REPORT.md before trusting any number here: everything is measured on synthetic deeds, and real-deed accuracy is unmeasured.

Results (synthetic test deeds; details and caveats in EVAL_REPORT.md)

base Qwen3-8B, 3-shot	tuned (v2)
deeds read exactly right (in-distribution)	10% (n=60)	99.6% (n=3,000)
flagged for a human	82%	0.4%
silent errors (wrong AND unflagged)	5 of 60	2 of 3,000 (0.07%, 95% upper bound 0.24%); 0 if `87 link` = `0.87 chain`
training cost	n/a	about $1.15 (est.)

Unseen style/unit, zero-shot: spelled-out numbers 85.6% exact (14.4% flagged, 0 silent)…

View on GitHub

How I built it

The rule behind all of it: the model reads, code decides.

Language models read messy text brilliantly and are not accountable for arithmetic. So Qwen3-8B plus a LoRA does exactly one job, turning prose into structured calls. Everything after that is plain code with tests: closure, area, unit conversion (feet, chains, poles, varas, links) and plotting.

Then five independent checks have to agree before an answer may look confident:

The output parses.
The parcel closes (1:5,000).
Every number appears in its own sentence of the deed.
Every landmark ("iron pin", "white oak") is backed by words in that sentence.
A second, independent reader, a regex parser that never saw the model, agrees on bearing, distance and curve direction.

The data: truth first, prose second. There is no free pile of labelled real deeds, so I turned the problem around. A generator draws a closed polygon with known bearings and distances, then writes it up in many styles: symbols, dashes, words, archaic chains and links, and damaged OCR, with easements, deed-book numbers and stray acreage as distractors. The truth exists before the text does, so every label is exact. I deliberately held back one writing style (every number spelled out in words) and one unit (Texas varas) to see what the model does with things it has never met.

Best Use of Tinker

10% → 99.6%, for $1.15

Qwen3-8B is a small model. Out of the box it doesn't know what a deed is, and it has nowhere near the knowledge of a giant LLM. It scored 10%. Same model, same prompt, same checks: the only thing that changed was one dollar and fifteen cents of Tinker training.

That's the point. I didn't need a model that knows everything. I needed one that does one narrow job perfectly, and fine-tuning is how a small model gets there.

	base Qwen3-8B, untrained	Tinker LoRA
Exactly right	6 / 60 (10%)	2,988 / 3,000 (99.6%)
Sent to a human	82%	0.4%
Wrong and unflagged	5 / 60	2 / 3,000 (0 by physical length)
Cost	n/a	$1.15

My goal wasn't a good number. It was a good number at the lowest price I could defend. Here is exactly how.

1. Shrink the bill before you shrink the model

Tinker bills per token, so I started with the output. My first format was JSON at 547 tokens per deed. I replaced it with one line per call:

POB iron pin
L N 41 0 0 E 121.9 ft | iron pin
C N 3 45 22 E 366.15 ft R 944.33 ft L | stone

Code converts it back to JSON. 145 tokens, 3.8× smaller, which cuts training, evaluation and local inference all at once, and costs nothing to run.

2. Price the options, then pick

Training per million tokens: Qwen3-8B $0.44 vs Qwen3.5-4B $0.737. A smaller model is not automatically a cheaper one. Qwen3-8B was cheaper to train and strong enough for the job.

3. Train: rank-32 LoRA, 3 epochs, 16 minutes

tc = await svc.create_lora_training_client_async(base_model="Qwen/Qwen3-8B", rank=32)

# loss only on the answer tokens, never on the deed prose
Datum(model_input=full[:-1],
      loss_fn_inputs=dict(weights=[0.0]*(len(prompt)-1) + [1.0]*len(completion),
                          target_tokens=full[1:]))

fb = await tc.forward_backward_async(batch, loss_fn="cross_entropy")
op = await tc.optim_step_async(AdamParams(learning_rate=lr_at(s), beta1=0.9, beta2=0.95, eps=1e-8))

2,300 training deeds, batch 16, peak learning rate 2e-4 with warmup and linear decay, 429 steps, about 2.6M tokens. Loss is masked to the answer, so the model learns to write calls instead of memorizing deed prose. Steps are pipelined (step k+1 is submitted before step k returns), a checkpoint is saved every epoch, and a budget guard prices every job from token counts and refuses to launch one that would burn half the cap. Loss falls from 1,806 on step 1 to under 0.1 by step 41.

4. Find the knee of the curve

I didn't train on everything and hope. Four runs, one epoch each, scored on the same 500 held-out deeds:

training deeds	est. cost	read exactly right	wrong and unflagged
250	$0.04	38.6%	19
500	$0.08	92.6%	0
1,000	$0.17	99.4%	0
2,000	$0.34	99.6%	0

Seventeen cents buys 99.4%. And look at the zeros: between 250 and 500 deeds the model stops making silent mistakes. That jump, not the accuracy, is where it becomes safe to trust.

5. Is the gain real or noise?

I paired the base model against the fine-tune on the same 60 deeds, deed by deed:

	base, untrained	fine-tuned
exactly right	6 / 60 (10%, CI 4.7–20.1%)	60 / 60
wrong and unflagged	5	0
deeds only this model got right	0	54

Every deed the base model got right, the fine-tune also got right, and it won the other 54. Paired sign test: p < 0.0001.

Then the real exam: 3,000 fresh deeds, generated before I froze the validator and scored exactly once. 2,988 exactly right. 99.6%.

Of the 12 imperfect outputs, 10 were caught by parse and closure alone. The other 2 are one ambiguity in my own generator: it writes 87 links as "87 link" in one style and "0.87 chain" in another, which is the same physical length. By physical length, the silent count is 0. I report both.

6. New styles and units cost about 150 examples

Held out on purpose, zero-shot, from the first model:

unseen test	exactly right	wrong and unflagged
numbers spelled out in words	85.6%	0 (14.4% flagged for review)
Texas varas	every deed flagged	0

Varas is the fun one. The model read every bearing and distance correctly but wrote the unit as var, a token it had never seen, and the strict decoder rejected it. So everything was flagged and nothing wrong got through. That is the safety net doing its job on a surprise.

I didn't patch the decoder to accept each spelling the model improvised, because that would be fitting the test set. The real fix was data: 150 examples each, retrain, score on fresh deeds:

fresh test set	n	exactly right	wrong and unflagged
in distribution	3,000	99.6%	2 (0 by physical length)
spelled-out numbers	250	99.6%	0
varas	250	100%	0

Onboarding a new style or unit is a 150-example, sub-dollar job. That's the workflow Tinker makes cheap enough to just do.

7. Make evaluation free the second time

Evaluation was the bigger bill ($1.35 vs $2.46 for all training), so every sampled output is cached and the checks score from the cache. Improving the validator and re-scoring 3,000 deeds cost $0. One deed through Tinker costs about $0.00015.

8. Rent the training. Keep the weights.

I downloaded the 369 MB adapter and ran it with MLX on the Apple GPU over a 4-bit Qwen3-8B base:

150 of 150 deeds read exactly right, 0 flagged, median 7.5 s per deed. M3 Pro, 18 GB, no network, no API key.

The whole bill

item	est. cost
baseline, no training	$0.11
learning-curve runs	$0.29
first full run	$1.01
final model (2,300 deeds, 3 epochs)	$1.15
all evaluation (4,000+ deeds)	$1.35
everything on Tinker	about $3.80

Token counts at Tinker's list prices.

Known edges (measured, not hidden)

I'd rather you hear these from me, with numbers.

The deeds are generated. That's what made exact labels and a 3,000-deed fresh test possible, and it's why the held-out style and unit matter. The next step is running real recorded deeds through the same pipeline, which is why every real-world answer still ends in a human approve button.
The baseline is the plain base model, not a heavily engineered one, and it was scored on 60 deeds to keep the bill low. That's why I paired it against the fine-tune on the same 60 and gave its confidence interval.
Clause checks look for ; or thence between calls. Where that structure is missing, they fall back to whole-text checks.
Curves are chord-form (chord bearing, length, radius, direction). Deflection angles and station/offset descriptions aren't supported yet.
It's a drafting aid. Not a substitute for a licensed surveyor.

What went wrong on the way (and made it better): my validator flagged perfect answers until I ran it on the generator's own ground truth and fixed three reading bugs. Two silent errors slipped past closure at 500 deeds (a dropped arc-minute and a flipped curve direction), which is exactly why the independent second reader exists.

Why does open innovation matter?

A closed API gives you a prompt box. Open weights gave me a steering wheel.

I could change the model, not just the prompt. The untrained small model got 10%. A one-dollar LoRA got 99.6%.
Deeds are private. They name people, parcels and mortgages. The weights run on the laptop that holds the file. Turn Wi-Fi off and it still works.
No meter running. After the one-time fine-tune, deed number one thousand costs electricity.
Nobody can deprecate it. The base model, adapter and code sit on disk. A better open model next year is a training run, not a migration.
I could look inside. The varas failure and the label tie all exist because I could inspect every output.

The honest caveat: training ran on Tinker's servers on generated deeds only. No real person's land record was ever sent anywhere, and at run time nothing leaves the laptop.

My agent session

I built DeedReader with Claude Code as a pair, from a written plan with a cost estimate up front and a budget guard on every training job.

It read Tinker's docs before writing a line against the API, and wrote closure-math tests before any model touched the project.
The baseline exposed a blind spot: many of the base model's landmark names were wrong, and closure can't see that. So it built the per-sentence grounding check.
It debugged its own validator against ground truth, and when varas failed it ran the real fix as an experiment on fresh data instead of patching around it.
It kept a running cost log from the first job.

Prize categories

Best Use of Tinker. The LoRA fine-tune (pipelined forward/backward passes, loss-masked completions, per-epoch checkpoints), the baseline, the data-size learning curve, 4,000+ evaluated deeds and the adapter download behind the offline story all ran on Tinker. $1.15 for the final model, about $3.80 for everything. Same base model, same prompt, same checks: 10% (6/60) → 99.6% (2,988/3,000), and every deed the base model got right, the fine-tune got right too.

Built solo, with Claude Code.

Three epochs. $1.15. 10% became 99.6%. If that sounds like a good trade, a ❤️ or 🦄 helps more people find it.

What's the strangest phrase you've ever seen in a property deed? Mine so far is "to a white oak, thence along the run." Leave yours in the comments.

A solution for all Ecom brand owners and marketing agencies helping ecom owners create video and image ads. Let me know your thoughts on it.

Advik Divekar — Sat, 17 Oct 2026 12:07:19 +0000

Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle Submission 💸

Advik Divekar

Oct 16

What if your whole marketing team fit on a laptop, and finished in 55 seconds?

#devchallenge #hf26challenge #ai #opensource

17 min read

What if your whole marketing team fit on a laptop, and finished in 55 seconds?

Advik Divekar — Fri, 16 Oct 2026 14:09:27 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What if your whole marketing team fit on a laptop, and finished in 55 seconds?

I used to run an ecommerce brand.

In small ecommerce, marketing is everything, and it comes in two kinds.

Performance marketing is the ads you pay for. Organic marketing is the posts, reels and listings you put out without paying for reach. Both run on a constant stream of fresh content. If it isn't live, you aren't selling.

Every product meant listing copy stitched together from two or three AI chats, then ad copy, photo edits, images for the listing, and video ads. Every step in a different tool. Every handoff a copy and paste.

It cost me in three ways:

Time. The gap between "I have a product" and "it's live" was where sales were lost.
Cashflow. A subscription for images, one for video, one for copy, one for editing. In a small business, tool bills quietly eat tight cashflow.
Privacy. My product details, my pricing and my workflow all sat inside other people's tools. None of it was ever private.

Then I watched my mom run into the same wall.

My mom is a cosmetician. She sells skincare and beauty products, and she is very good at it. Her marketing is organic: Instagram and Facebook posts and reels.

But one single image post took her 3 to 4 hours.

Canva for the picture. CapCut or another editor for the video. ChatGPT or Claude for the caption, then copy, paste, "make it shorter", copy, paste again. Then the post itself, on every platform she sells on. A reel could take days.

On top of that, she kept switching between ChatGPT, Claude, Higgsfield and whatever other AI tool she could find, often across several accounts to keep going. That irritated her a lot.

Those are hours she should have spent with customers. Instead she was working as a part-time designer and video editor.

Different business, same trap: too many tools, too much attention, and your product details sent to places you don't control.

So I built Mochi: one product photo and a few true facts become studio photos, an image deck, a 10-second video and copy for every platform you sell on. It runs on open models on your own computer, so nothing about your product goes to anyone else's server.

I built it for my mom and for sellers like me, and I tried it on a real brand to see if it works. More on that session is further down.

Here it is in action:

Here's the proof. A skincare brand, Velora, dropped three photos of a face cleanser into Mochi. This is the screen when it finished:

Fifty-five seconds. And this is what it had made.

9 studio photos on a designed backdrop:

A 5-slide image deck (this is slide 3 of 5):

A 10-second vertical video ad, with text cards built from the copy:

Listing copy, tags and an SEO title, each one editable:

No agency. No stack of subscriptions. No uploading your product to someone else's server. Just open models running on one computer.

What I Built

Mochi turns that whole chain of tools, tabs and copy-pasting into three taps.

You get:

Studio photos. Your phone photo, cut out and placed on a designed backdrop, and the product is never regenerated, so it stays your real product
An image deck. Slides ready for a feed or a catalog
A 10-second vertical video. Camera moves and text cards built from your own copy
Platform copy. Listing, social, Pinterest and a video brief, written for each place you sell

One thing I care about more than anything: Mochi checks its own copy for things you never told it. Models do invent claims, so every kit is checked by plain code. Missing facts are listed back to you ("Not provided, so left out of the copy"), and anything the copy says that isn't in your facts is flagged before you post.

For a skincare seller, that matters. "Reduces wrinkles" is a legal claim, not a style choice.

How to use it

Five steps, about two minutes of your time. The screenshots are from a real run on a phone.

The shortcut: add a photo, and if Mochi's guesses from it look right, tap Looks right: generate now. Everything below is the long way, for when you want control.

Step 1. Add your photos.
Tap Take photo or Add more. You can add up to 5. The first photo is your Main one, and it also opens and closes your video. Different angles make a better video and deck. A clear photo on a plain background works best. Mochi starts reading them right away.

Step 2. Check what Mochi found.
Mochi reads the label on your product and offers the text as notes you can tap to add. Fields it filled in are tagged "From photo, check it". Fill in what's missing, like price and features. Only add what is true. Blank fields are flagged as missing, never guessed.

Step 3. Pick your style.
Choose where you sell (Instagram, Facebook, Pinterest, Etsy, Shopify), a tone (from Straightforward to Luxury), a goal (Drive sales, Brand awareness, Gift appeal or Explain features), and a writing speed:

Fast takes about 20 seconds
Best quality uses a larger model and takes about a minute

Then pick a backdrop for your photos and video: Premium studio, UGC-style, Minimal, Warm and cozy, or Bold and colorful. Tap Generate kit.

Step 4. Let it build.
You see real progress, step by step: writing copy, building studio photos and video, adding text to the video. You can leave the page. It keeps running.

Step 5. Review, edit and share.
Your kit has four tabs:

Overview: the video, the image deck, and a "double-check before posting" list
Copy: listing, social, Pinterest and a video brief. Change the tone or refresh any section.
Photos: your studio photos, ready to download
Share: download or share the kit

The finished result of a real run is at the top of this post.

Nothing posts on its own. The only thing Mochi can send anywhere is a Shopify draft, and only when you tap it.

Demo

Every page of the web app

Landing: hero

Landing: the interactive studio

Landing: written the way each channel works

Landing: your next ad starts with a photo

Landing: upload once, get the lot

Landing: same facts, your voice

Landing: three steps, about a minute

Landing: call to action

Create: start

Create step 1: photos, and the text Mochi read from them

Create step 2: product facts

Create step 3: tone, goal, speed and visual style

Result: overview with video ad and image deck

Result: copy for every channel

Result: studio photos, white background and cut-outs

Result: share and export

Stores

Code

advikdivekar / sellcraft

Mochi

Your product. Your story. Ready to sell.

Small-seller marketing studio: upload a product photo, enter a few verified facts, pick a platform / tone / style / goal, and get platform-specific listing copy plus a video-ad brief. Copy is written by an open-weight model (Gemma) running locally through Ollama, so product data never leaves your machine. See plan.md for the full design.

Status: Working end to end: facts and photos → Gemma copy → studio photos, image deck and a 10-second video → export or Shopify draft. A LoRA fine-tune on Tinker, an independent Gemma judge and Sentry tracing are in; see docs/RESULTS.md for the measured numbers and docs/DECISIONS.md for why things are the way they are.

What it makes

From up to 5 photos and a few facts (the app measures each photo, flags blur/low-res/dark shots, cleans the cut-out, and picks a backdrop that makes your product…

View on GitHub

Backend: FastAPI, SQLite, Pydantic. Frontend: Next.js 16 and React 19. 135 backend tests pass. Every number in this post can be reproduced from the files in backend/finetune/results/, with the commands in docs/RESULTS.md.

How I Built It

Two lanes run at the same time: one for visuals and one for language. They meet at the deck and the video, which need the finished copy. The page polls the job every 1.5 seconds and shows real progress and partial copy as it arrives.

Every model has one job and one strict contract.

Two choices worth explaining:

The video is not AI-generated. It's image processing plus FFmpeg, so the product in the video is your product, not a regenerated lookalike. (I do denoise, upscale and retouch it.) Generative video tends to repaint the product.
The checker is plain code, not a second model. A model can talk itself into anything. A word list and a number check can't.

The copy is split into parallel parts (four core parts plus one per platform), each forced into a JSON schema. A bad call is retried once, and plain code then checks for forbidden words, invented numbers and unsupported claims.

Photos are read, never trusted. Up to 5 photos are shrunk to 768 px and read under strict rules. What comes back is only a suggestion, marked "From photo, check it".

Studio shots: measure, cut out, refine the product's pixels, fix the tone, pick a backdrop from what the photo shows, compose. If the cutout looks wrong, it falls back to a framed photo instead of a bad cutout.

The video: at least 4 shots rendered with FFmpeg (up to 3 at once), 0.4 s crossfades, text cards from your edited copy, checked to be 10 seconds at 1080x1920. Edit the copy and only the text layer re-renders.

Why it's fast. My first version took 90 to 145 seconds for one big generation on a laptop.

Change	Before	After
Fast mode (Gemma 4B) as the default	copy 53 s	copy 14 s
Split the kit into parallel parts	one 12B call, 90 to 145 s	45 to 60 s
Run both lanes together	the sum of both	the slower lane, plus a few seconds

A four-photo kit went from about 100 s to about 18 s in Fast mode. The trade-off: the 4B model gets more claims flagged than the 12B, the post-check still catches them, and Best quality (about 57 s) is one switch away.

The fine-tune, the evaluation and the Sentry setup each have their own section under Prize Categories.

Why Does Open Innovation Matter?

Go back to my two stories. In both, the real cost was not money alone. It was control.

1. Your product details stay yours. Pricing, ingredients, launch plans. With open models running locally, the data never leaves your machine. There's no "we don't train on your data" to trust.

2. No per-kit fee. Every kit costs electricity. A seller can run it five times, try three tones, and never wonder if it's "worth" the credits. When cashflow is tight, that matters.

3. I could change the model, not just the prompt. I fine-tuned a small open model to follow one rule: don't invent claims. Prompting alone left the 4B model at 23% clean kits. Training took it to 88%. You can't do that to a closed model, or download what you trained.

4. I could look inside and measure. I tested the fine-tune against an independent judge model, not just my own checker. With a closed API I'd have one number and no way to question it.

5. One setting swaps the model. Fast or Best quality is a single switch between two open Gemma models. The next Gemma release is a one-line change.

The honest limit: open models need a computer that can run them. I wrote the local setup into the README.

My Agent Session

I built Mochi with Claude Code, and every session is recorded with Entire. docs/DECISIONS.md lists 19 decisions, each with its reason, its commit and its Entire checkpoint, so you can read the conversation behind any choice. More in the Entire section below.

My First User

Velora, a skincare brand, ran its Sensicontrol Face Cleanser through Mochi on a phone, on 16 October 2026. The phone reached my computer over a Cloudflare tunnel, so for this demo the photos passed through Cloudflare. In normal use everything stays on the machine. I have the screen recording. Here is what happened, taken only from what is on screen.

I don't have a written quote from them, so I'm not putting words in their mouth. What I do have is the recording, and it shows what they actually did.

They followed the five steps above (the screenshots for steps 1 to 4 are from their session): three photos of the bottle, the label read off it, a persuasive tone, Best quality writing, Pinterest as the platform, and the warm and cozy backdrop.

What came out is shown at the top of this post: the finished kit in 55 seconds, the studio photos, the deck, the video and the copy. It was built on Best quality, with three photos, by a Gemma 3 12B model running on my own computer.

The part I care about most. Mochi told them what it had left out, and what it had written that they never said:

"Not provided, so left out of the copy": dimensions, verified benefits, target audience, care instructions
"Double-check before posting": Not in your facts: "available now". Remove it unless it's true.
"About your photos": it skipped the close-up slide because the photo didn't have enough pixels on the product

The kit was five slides, nine studio photos, a 10-second video, and copy for one platform.

What I learned from watching. All of this is from the recording:

They changed their mind before generating. They started with the Friendly tone, Instagram and Fast, then switched to Persuasive, Pinterest and Best quality. The choices clearly matter to people, so I'm glad nothing is locked in.
They left several facts blank, and the copy respected that. Target audience, dimensions, verified benefits and care instructions were never filled in, so Mochi left them out and said so, instead of guessing.
Best quality costs time. The form says Fast takes about 20 seconds and Best quality about a minute. They chose the slower one, and it took 55.

Prize Categories

I'm entering Tinker, Gemma, Sentry Agent Tracing and Entire. Each section covers what I used, why, what it was meant to fix, and what I found.

Best Use of Tinker

What I used it for. I fine-tuned Qwen3.5-4B with a LoRA adapter on Tinker to write marketing copy that follows one rule: never invent a claim the seller didn't give.

Why Tinker. Prompting a small model to follow that rule didn't work. Untuned Qwen3.5-4B produced a clean kit only 18% of the time. I wanted to change the model, not keep rewording the prompt, and Tinker let me train without running my own GPU servers.

How it worked.

A big teacher model, Qwen3.5-397B-A17B, writes and revises kits on Tinker
My checker and an independent judge (Gemma 12B, a different model family) reject any kit with an invented claim, number or forbidden word
The teacher revises with the checker's findings, at most 2 rounds
Only kits that pass both the checker and the judge become training data (zero hard violations)
A small student is trained on those: rank 32, 3 epochs, 319 examples, learning rate 2e-4
A held-out set of 60 products, which the training code refuses to read, tests it

The whole thing cost $5.41 of the $10 credit.

The full detailed version (tap to open it full size and zoom in):

Results (60 held-out synthetic products, one checker):

The full detailed version (tap to open it full size and zoom in):

That's a 4B model, a third of Gemma 12B's size, producing nearly twice as many clean kits, and about 10 times faster (4.9 s against 48.7 s per kit, though v2 runs on Tinker's cloud and Gemma on my laptop).

The honest result on the judge. v2 halves the untuned model's flags (0.43 to 0.20), but the confidence intervals overlap, and Gemma 12B is lower still at 0.13. With 30 kits per model I can't claim more than that.

Limits I'm stating on purpose:

The test set is synthetic, and the checker also filtered the training data
The fine-tune runs on Tinker, so its inputs leave the machine. The app's default (Gemma via Ollama) stays local.
The fine-tune is not in the app's path yet. I haven't exported it to GGUF for Ollama.
Hosted cost is about $3 per 1,000 kits, estimated from token counts

What I learned. A small model trained on the right examples can follow a narrow rule better than a larger model that's only prompted. The checker and the judge decide which examples count, so they matter as much as the training run.

Best Use of Gemma

What I used it for. Gemma does three jobs inside Mochi, all through Ollama on the seller's own computer:

The copy writer. Gemma 3 4B for Fast, Gemma 3 12B for Best quality
The photo reader. It reads up to 5 photos and returns suggested facts, each tagged "From photo, check it"
The independent judge. Gemma 3 12B, a different family from the Qwen teacher, reviews kits in my evaluation

The full detailed version (tap to open it full size and zoom in):

Why Gemma. It's open-weight, it runs on one machine, and it can be held to a JSON schema. That matters here: the seller's facts shouldn't leave their computer, and every part of a kit must come back as strict, checkable JSON.

What it was meant to fix. The cost and privacy problems from my ecommerce days: no per-kit fee, and nothing sent to anyone else's server.

Results.

Both sizes run the same prompt and the same checker, so I could compare them fairly: 23% clean kits on 4B and 45% on 12B
Speed is the trade-off. Fast (4B) builds a four-photo kit in about 18 s, Best quality (12B) in about 57 s
As the judge, Gemma 12B had the fewest flags of any model I tested (0.13 per kit)

What I learned. Two things. First, the model size is a user choice, not a technical one, so I made it a switch on the form. Second, the judge is Gemma, so it may be lenient toward Gemma's own writing. That bias favours the Gemma rows and works against my fine-tune's claim, so it can't be inflating my result.

Best Use of Sentry Agent Tracing

What I used it for. Tracing every model call and every job, without Sentry ever seeing a seller's product facts.

Why this was the hard part. A marketing tool handles a seller's most private material: product details, prices, plans. Sending prompts to a tracing service would break the promise the whole product makes. So the question was not "can I trace this", it was "can I trace this and prove nothing private leaks".

The full detailed version (tap to open it full size and zoom in):

Each model call is a gen_ai.chat span, with the model, token counts and latency
Each generation job is a transaction
Each flagged claim is an issue (which rule fired, and how many)
Each seller report is an event. The report button sends a section name and a fixed reason code, never free text

The flagged claims come from the copy post-check, so each issue is a rule the code caught, not a model's opinion.

The full detailed version (tap to open it full size and zoom in):

What it never sees. An allow-list scrubber removes everything that is not on the list of allowed metadata. Product facts, copy, photos and prompts can't pass it.

How I know it works. A canary test pushes a unique fake product through every input path of the real app, then fails if that marker shows up in any outgoing Sentry payload. It's in tests/test_tracing.py.

Results and honesty. The scrubber and the canary test are in place and pass. I am not claiming Sentry has caught a real production incident, because it hasn't. Mochi has had one real user session, not production traffic.

What I learned. Making tracing safe was a bigger design question than turning it on. The allow-list approach (what's permitted, not what's blocked) is the one that survives new fields being added later.

Best Use of Entire

What I used it for. To record how Mochi was built, so a reader can see why the code is the way it is, not just what it does.

Why. Most repos show a result. For a project where I changed my mind several times, the changes are the story. A privacy rule, a new default and a safety check each have a reason that a commit message can't hold.

What came of it.

About 113 checkpoints of Claude Code sessions, pushed to the repo
docs/DECISIONS.md: 19 numbered decisions, each with the reason, the commit and the Entire checkpoint ID
Anyone can run entire checkpoint explain <id> in a clone and read the prompts, files and reasoning behind a decision

The most useful example. Decision #11 records why Sentry only ever gets metadata, and how the scrubber enforces it. That kind of reasoning normally disappears. Here it is written down, with the conversation that led to it.

Honest limit. Checkpoints from before Entire was enabled were imported from local history and matched by time. They show the conversation a commit came out of, not a one-to-one record.

If you sell something and make your own content, I'd love to hear what's missing. Tell me in the comments.

Built solo, with Claude Code.