<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Jose Antonio Granero</title>
    <description>The latest articles on DEV Community by Jose Antonio Granero (@finallyjay).</description>
    <link>https://dev.to/finallyjay</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4161297%2Fdc941ae1-83c4-4f2f-8bf3-2a4bc47450a4.jpg</url>
      <title>DEV Community: Jose Antonio Granero</title>
      <link>https://dev.to/finallyjay</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/finallyjay"/>
    <language>en</language>
    <item>
      <title>Every rhyme rule I gave Gemma, it found a loophole. Catrina News turns today's tech headlines into Día de los Muertos calaveras</title>
      <dc:creator>Jose Antonio Granero</dc:creator>
      <pubDate>Sat, 31 Oct 2026 16:09:51 +0000</pubDate>
      <link>https://dev.to/finallyjay/every-rhyme-rule-i-gave-gemma-it-found-a-loophole-catrina-news-turns-todays-tech-headlines-into-1efb</link>
      <guid>https://dev.to/finallyjay/every-rhyme-rule-i-gave-gemma-it-found-a-loophole-catrina-news-turns-todays-tech-headlines-into-1efb</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-week4-2026-10-26"&gt;Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;&lt;a href="https://catrinanews.tech" rel="noopener noreferrer"&gt;Catrina News&lt;/a&gt;&lt;/strong&gt; is a daily broadside of &lt;em&gt;calaveras literarias&lt;/em&gt; about the tech companies in today's news, written by an open-weight model on my laptop and read aloud by the Catrina.&lt;/p&gt;

&lt;p&gt;A &lt;em&gt;calavera literaria&lt;/em&gt; is a short satirical poem Mexicans write for Día de los Muertos, imagining Death coming to take someone away. Most people know them as playful verses for friends and family, but they were born as something sharper: in 19th-century Mexican newspapers they mocked politicians and the rich, and José Guadalupe Posada's skeletons, the Catrina among them, were drawn for those pages.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fhhhfv17mmah60751mj0i.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fhhhfv17mmah60751mj0i.jpg" alt="Remate de Calaveras Alegres y Sandungueras, broadside by José Guadalupe Posada, Mexico, 1913: the first appearance of the Catrina" width="800" height="1073"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The broadside where the Catrina first appeared: José Guadalupe Posada, Mexico, 1913. Public domain.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;So I went back to the origin. The powerful of 2026 are tech companies, and the news gives them plenty of material:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;El iPhone Duo, un gran diseño,&lt;/em&gt;&lt;br&gt;
&lt;em&gt;prometía un lujo sin igual.&lt;/em&gt;&lt;br&gt;
&lt;em&gt;Mas la bisagra, con su empeño,&lt;/em&gt;&lt;br&gt;
&lt;em&gt;se quebró, ¡y al cliente un penal!&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The iPhone Duo, a great design, / promised a luxury without equal. / But the hinge, with its insistence, / broke, and a penalty for the customer!&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Every day it:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;pulls the last 24 hours of tech news with SerpApi's Google News engine,&lt;/li&gt;
&lt;li&gt;lets Gemma pick the ones worth satirizing (and drops anything about real people or tragedies),&lt;/li&gt;
&lt;li&gt;writes a calavera for each, &lt;strong&gt;with every rhyme and syllable checked by plain Python, not by the model&lt;/strong&gt;,&lt;/li&gt;
&lt;li&gt;adds a line-by-line English gloss,&lt;/li&gt;
&lt;li&gt;and has the Catrina read them aloud with ElevenLabs.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The result is printed as a Posada-style &lt;em&gt;hoja volante&lt;/em&gt; (broadside), in English or Spanish, under a motto that borrows the shout of his headlines: &lt;em&gt;¡Gran Calavera Tecnológica!&lt;/em&gt;, after broadsides like his &lt;em&gt;Gran Calavera Eléctrica&lt;/em&gt; (early 1900s).&lt;/p&gt;
&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;🌐 &lt;strong&gt;&lt;a href="https://catrinanews.tech" rel="noopener noreferrer"&gt;catrinanews.tech&lt;/a&gt;&lt;/strong&gt;: press &lt;strong&gt;Meet La Catrina&lt;/strong&gt; to hear her introduce herself, or &lt;strong&gt;Listen to the broadside&lt;/strong&gt; and she'll read every calavera.&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/0WV_of3JDlw" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;The video opens with La Catrina herself introducing the project (an ElevenLabs avatar with the same voice that reads the calaveras), then the broadside reads itself: press "Listen to the broadside" and she recites every calavera while her skull's jaw follows her voice.&lt;/p&gt;

&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/finallyjay" rel="noopener noreferrer"&gt;
        finallyjay
      &lt;/a&gt; / &lt;a href="https://github.com/finallyjay/catrinanews" rel="noopener noreferrer"&gt;
        catrinanews
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      Today's tech news as Día de los Muertos calaveras: a local open model writes them, code checks every rhyme, the Catrina reads them aloud.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;Catrina News 💀🌼&lt;/h1&gt;
&lt;/div&gt;
&lt;p&gt;&lt;strong&gt;Today's tech news, as Día de los Muertos calaveras.&lt;/strong&gt; A local open-weight model writes satirical verses about the companies in today's headlines, plain Python checks every rhyme, and the Catrina reads them aloud.&lt;/p&gt;
&lt;p&gt;👉 &lt;strong&gt;&lt;a href="https://catrinanews.tech" rel="nofollow noopener noreferrer"&gt;catrinanews.tech&lt;/a&gt;&lt;/strong&gt; · &lt;a href="https://github.com/finallyjay/catrinanews/README.es.md" rel="noopener noreferrer"&gt;Leer en español&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Built for the &lt;a href="https://dev.to/challenges/hacktoberfest-week4-2026-10-26" rel="nofollow"&gt;Hacktoberfest Open-Source AI Challenge, Week 4: Ghost in the Machine&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;a rel="noopener noreferrer" href="https://github.com/finallyjay/catrinanews/docs/screenshot.png"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fraw.githubusercontent.com%2Ffinallyjay%2Fcatrinanews%2FHEAD%2Fdocs%2Fscreenshot.png" alt="Catrina News, October 31, 2026"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;What's a calavera?&lt;/h2&gt;
&lt;/div&gt;
&lt;p&gt;A &lt;em&gt;calavera literaria&lt;/em&gt; is a short, satirical poem Mexicans write for Día de los Muertos, imagining Death (&lt;em&gt;la Flaca&lt;/em&gt;, &lt;em&gt;la Catrina&lt;/em&gt;) coming to take someone away. They began as jokes at the expense of the powerful in 19th-century newspapers, where José Guadalupe Posada's skeletons, the Catrina among them, were born.&lt;/p&gt;
&lt;p&gt;Most tools treat calaveras as sweet poems for loved ones. Catrina News goes back to the origin: the satire of the powerful. Today, that's tech companies.&lt;/p&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;How it works&lt;/h2&gt;
&lt;/div&gt;
&lt;div class="snippet-clipboard-content notranslate position-relative overflow-auto"&gt;
&lt;pre class="notranslate"&gt;&lt;code&gt;SerpApi + RSS ──► editor (Gemma) ──► poet (Gemma)&lt;/code&gt;&lt;/pre&gt;…&lt;/div&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/finallyjay/catrinanews" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;p&gt;Headlines come from &lt;a href="https://serpapi.com" rel="noopener noreferrer"&gt;SerpApi&lt;/a&gt;'s Google News engine: one request returns the last 24 hours of Spanish tech news as clean JSON, with the outlet's name and the article's real URL (the free Google News RSS only gives redirect links). Those links are what each calavera's "From …" points to.&lt;/p&gt;

&lt;p&gt;Everything is Python's standard library talking to &lt;strong&gt;Gemma 3 12B&lt;/strong&gt; through &lt;a href="https://ollama.com" rel="noopener noreferrer"&gt;Ollama&lt;/a&gt; on a 2020 13-inch MacBook Pro (M1, 16 GB). No frameworks.&lt;/p&gt;

&lt;h3&gt;
  
  
  The model writes; code listens
&lt;/h3&gt;

&lt;p&gt;A calavera has rules: four lines of about eight syllables, and lines 2 and 4 must rhyme &lt;em&gt;in consonance&lt;/em&gt; (everything from the stressed vowel on sounds the same: &lt;em&gt;capital / hospital&lt;/em&gt;). A 4B or 12B model can't reliably hear that. So the model never grades its own work.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/finallyjay/catrinanews/blob/main/rima.py" rel="noopener noreferrer"&gt;&lt;code&gt;rima.py&lt;/code&gt;&lt;/a&gt; does, in about 200 lines: Spanish stress rules, diphthongs and hiatus, the silent &lt;em&gt;u&lt;/em&gt; in &lt;em&gt;que&lt;/em&gt;, &lt;em&gt;b&lt;/em&gt; and &lt;em&gt;v&lt;/em&gt; sounding alike, synalepha between words, and the +1/−1 syllable rule for the last word of a line. When a stanza fails, the model gets feedback it can act on:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;No rima: el verso 2 termina en "director" (suena "-or") y el verso 4 en "mansión" (suena "-on"); deben rimar en consonante. El verso 3 tiene 11 sílabas; déjalo en unas 8.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Each headline gets three candidates with up to three rounds of feedback, and code picks the best.&lt;/p&gt;

&lt;h3&gt;
  
  
  What the numbers said
&lt;/h3&gt;

&lt;p&gt;I measured before building anything: 5 headlines × 2, up to 3 tries each.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Rhyme 2-4&lt;/th&gt;
&lt;th&gt;Rhyme 1-3 too&lt;/th&gt;
&lt;th&gt;Lines of 7–9 syllables&lt;/th&gt;
&lt;th&gt;Time per calavera&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Gemma 3 4B, plain retries&lt;/td&gt;
&lt;td&gt;9/10&lt;/td&gt;
&lt;td&gt;7/10&lt;/td&gt;
&lt;td&gt;18/40&lt;/td&gt;
&lt;td&gt;~9 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemma 3 4B + meter feedback + best-of-3&lt;/td&gt;
&lt;td&gt;10/10&lt;/td&gt;
&lt;td&gt;8/10&lt;/td&gt;
&lt;td&gt;30/40&lt;/td&gt;
&lt;td&gt;29 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Gemma 3 12B + meter feedback + best-of-3&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;10/10&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;9/10&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;34/40&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;59 s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Feedback works; planning doesn't.&lt;/strong&gt; Telling the 4B which sounds failed took it from 4/10 rhymes on the first try to 9/10. Asking it to choose its rhyming pair &lt;em&gt;first&lt;/em&gt; sounded smarter and made it far worse: &lt;strong&gt;2/10&lt;/strong&gt;, with invented words like &lt;em&gt;"sopio"&lt;/em&gt; and &lt;em&gt;"lumpo"&lt;/em&gt; to force a fit.&lt;/p&gt;

&lt;h3&gt;
  
  
  Every rule I added, the model found the loophole
&lt;/h3&gt;

&lt;p&gt;This was the ghost in my machine.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;em&gt;Panteón&lt;/em&gt; (graveyard) is the easiest rhyme in a calavera, and the 4B ended a line with it in 4 of its first 10. &lt;strong&gt;I banned it.&lt;/strong&gt; It switched to &lt;em&gt;millón&lt;/em&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;I banned any word already used in the sheet.&lt;/strong&gt; It kept the same &lt;em&gt;sound&lt;/em&gt; with new words: &lt;em&gt;razón, decepción, perfección, extinción, afición, lección&lt;/em&gt;… In one sheet, 12 of the 20 line endings were &lt;em&gt;-ón&lt;/em&gt; words.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;I banned rhyme &lt;em&gt;sounds&lt;/em&gt; already used in the sheet.&lt;/strong&gt; That finally worked, because a sound is something only the code can hear.&lt;/li&gt;
&lt;li&gt;The editor picked the iPhone Duo twice. &lt;strong&gt;I asked for one headline per topic.&lt;/strong&gt; It picked the iPhone Duo and "foldable phones". &lt;strong&gt;I gave it a fixed list of categories&lt;/strong&gt; to choose from (a JSON-schema &lt;code&gt;enum&lt;/code&gt;)… and it labelled all 11 headlines "apps and subscriptions". Describing each category with examples fixed it. Code still checks that no category repeats and that no two headlines share a product name.&lt;/li&gt;
&lt;li&gt;It once picked a headline about deepfake abuse, despite the prompt forbidding violence. Now a regex in code drops those, whatever the model says.&lt;/li&gt;
&lt;li&gt;The worst one: given a headline about alleged espionage, it picked it anyway (a named person &lt;em&gt;and&lt;/em&gt; a spy story, both forbidden) and wrote &lt;em&gt;"y al chino lo llevó al montón"&lt;/em&gt; ("and Death took the Chinese man to the pile"). Satire of a company had turned into a joke about a people. That sheet never went online. Now code drops headlines about spies, crimes or a named person's title ("CEO of…", "head of…"), and &lt;strong&gt;rejects any stanza that names a nationality or ethnicity&lt;/strong&gt;, sending the model back with &lt;em&gt;"la burla es para la empresa, nunca para un pueblo"&lt;/em&gt; (mock the company, never a people).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The pattern held all week: &lt;strong&gt;the model interprets, code verifies.&lt;/strong&gt; Each rule improved a number, and the model found the next way around it. Form is code's job. Whether the joke lands is the model's, and that's where size shows: the 12B writes &lt;em&gt;"te vende aire, ¡qué cruel plan!"&lt;/em&gt; (it sells you air, what a cruel plan!) where the 4B wrote &lt;em&gt;"lo envió al millón, ¡qué arroz!"&lt;/em&gt; (it sent him to the million, what rice!).&lt;/p&gt;

&lt;h3&gt;
  
  
  The gloss doubles as a nonsense detector
&lt;/h3&gt;

&lt;p&gt;Rhymes don't survive translation, so the Spanish is the original and Gemma writes a plain line-by-line English gloss. Unexpected bonus: filler that hides in Spanish verse is obvious in English. &lt;em&gt;"Lo llevó al reino del brazo"&lt;/em&gt; reads fine in Spanish; in English it's &lt;em&gt;"took him to the realm of the arm."&lt;/em&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The Catrina's voice
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://github.com/finallyjay/catrinanews/blob/main/voz.py" rel="noopener noreferrer"&gt;&lt;code&gt;voz.py&lt;/code&gt;&lt;/a&gt; calls ElevenLabs over plain REST, stitching each clip with the previous and next text so the sheet sounds like one reading. I picked the voice by measuring, not just by ear: of three voices reading the same calavera, &lt;em&gt;Lucia&lt;/em&gt; paused at the end of every line (6 pauses, against 2 for the first voice I tried), so you can actually hear the rhyme land.&lt;/p&gt;

&lt;h3&gt;
  
  
  She also introduces herself
&lt;/h3&gt;

&lt;p&gt;I first tried to animate Posada's original engraving with ElevenLabs' avatars: &lt;em&gt;"No face detected in the image."&lt;/em&gt; Fair enough, it's a skull. So I generated a portrait of a woman dressed as La Catrina with Flux 3, in ElevenLabs' own image tools: white skull face paint, a wide feathered hat, sepia like an old print. The avatar, speaking with the same voice, opens the demo video and greets you on the site (click to play, with English and Spanish subtitles timed to the pauses in her voice). On the site itself, the skull in the masthead talks: the Web Audio API measures her voice as it plays and opens the jaw to match, no service involved.&lt;/p&gt;

&lt;h3&gt;
  
  
  Printed once, like a broadside
&lt;/h3&gt;

&lt;p&gt;The site is static: HTML and MP3s, generated on my laptop and served by Render on a &lt;code&gt;.tech&lt;/code&gt; domain. No model, no API key and no GPU online. It's how a &lt;em&gt;hoja volante&lt;/em&gt; worked: printed at the shop, handed out on the street.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Satire needs a model nobody else controls.&lt;/strong&gt; These verses mock real companies. With an open-weight model on my own machine, no provider's content policy decides which companies can be laughed at, and no API bill grows with every headline.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Open weights let me measure.&lt;/strong&gt; I could run the same prompts on the 4B and the 12B, change one rule at a time and compare, for free, as many times as I wanted. That's how I found every loophole above.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It won't disappear.&lt;/strong&gt; The broadside is plain files. If any service I used shut down tomorrow, catrinanews.tech would still be there.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Best Use of Gemma&lt;/strong&gt;: Gemma 3 12B (and 4B for comparison) is the editor, the poet and the translator, running locally through Ollama.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Best Use of ElevenLabs&lt;/strong&gt;: the Catrina reads every calavera (stitched clips, a voice chosen by measuring pauses at line ends), her skull's jaw moves with that voice on the site, and an ElevenLabs avatar of La Catrina narrates the demo in the same voice.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Best Use of a .tech Domain&lt;/strong&gt;: live at &lt;a href="https://catrinanews.tech" rel="noopener noreferrer"&gt;catrinanews.tech&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Best Use of SerpApi&lt;/strong&gt;: SerpApi's Google News engine finds the day's tech news the Catrina writes about, with the source and link shown on every calavera.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Best Use of Render&lt;/strong&gt;: the broadside is served as a Render static site from a Blueprint (&lt;code&gt;render.yaml&lt;/code&gt;).&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>opensource</category>
      <category>ai</category>
    </item>
    <item>
      <title>Fridge Sloth 🦥: snap your fridge, get a grocery list and healthy meals, fully offline on Gemma</title>
      <dc:creator>Jose Antonio Granero</dc:creator>
      <pubDate>Sat, 24 Oct 2026 13:38:31 +0000</pubDate>
      <link>https://dev.to/finallyjay/fridge-sloth-snap-your-fridge-get-a-grocery-list-and-healthy-meals-fully-offline-on-gemma-2fkp</link>
      <guid>https://dev.to/finallyjay/fridge-sloth-snap-your-fridge-get-a-grocery-list-and-healthy-meals-fully-offline-on-gemma-2fkp</guid>
      <description>&lt;p&gt;&lt;em&gt;This is a submission for the &lt;a href="https://dev.to/challenges/hacktoberfest-week3-2026-10-19"&gt;Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Fridge Sloth&lt;/strong&gt; is for the moment you stand in front of the open fridge, door wide open, trying to remember what you need to buy and what on earth you could cook tonight.&lt;/p&gt;

&lt;p&gt;Instead, you take one photo. Fridge Sloth:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Looks at the photo&lt;/strong&gt; and lists the food it can actually see, with a rough "plenty / some / low" amount.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Checks it against your staples&lt;/strong&gt;, the things you always want at home (milk, eggs, tomatoes…), and builds a &lt;strong&gt;grocery list&lt;/strong&gt; of what's missing or running low.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Suggests three healthy meals&lt;/strong&gt; you can make mostly from what's already there: real dishes like omelettes, salads, stir-fries and wraps, 30 minutes or less, 5 steps max. Anything a recipe still needs goes onto your grocery list with one tap.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;strong&gt;Who gets to be lazier?&lt;/strong&gt; Anyone who does the weekly shop and then stares at a half-full fridge wondering what's for dinner. You skip the mental inventory, the list-writing and the "what can I even make with this?" moment.&lt;/p&gt;

&lt;p&gt;Everything runs locally on &lt;strong&gt;Gemma 3 (4B)&lt;/strong&gt; through &lt;strong&gt;Ollama&lt;/strong&gt;, on my 6-year-old M1 MacBook. No account, no API key, and your fridge photo never leaves your machine.&lt;/p&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/K0R_kFz1w3Y" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;The video is a real run of the app on my laptop with a public-domain USDA fridge photo (&lt;a href="https://commons.wikimedia.org/wiki/File:Food_Safety_-_Cut_Waste_in_Refrigerators_(20200605-FSIS-LSC-0017).jpg" rel="noopener noreferrer"&gt;source&lt;/a&gt;). The two waits are sped up and labelled with their real time: &lt;strong&gt;~37 s&lt;/strong&gt; for the scan and &lt;strong&gt;~26 s&lt;/strong&gt; for the recipes. I recorded it before the speed-up described below; the same scan now takes &lt;strong&gt;~20 s&lt;/strong&gt;. The voice-over was generated with &lt;strong&gt;ElevenLabs&lt;/strong&gt;.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Gemma saw:&lt;/strong&gt; milk, yogurt, juice, eggs, lettuce, cheddar cheese, bread, pesto…&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Grocery list:&lt;/strong&gt; butter, tomatoes, carrots, apples, orange juice, plus spinach and feta, which I'd just added to my staples. None of them are in the photo ✅&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Healthy meals:&lt;/strong&gt; a pineapple &amp;amp; lettuce omelette (20 min), a cheesy lettuce wrap (15 min) and a quick yogurt bowl (10 min). One tap put the omelette's olive oil on the list.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/finallyjay" rel="noopener noreferrer"&gt;
        finallyjay
      &lt;/a&gt; / &lt;a href="https://github.com/finallyjay/fridge-sloth" rel="noopener noreferrer"&gt;
        fridge-sloth
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      🦥 Snap your fridge, get a grocery list and healthy meals — runs fully offline on Gemma 3 + Ollama.
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;Fridge Sloth 🦥&lt;/h1&gt;
&lt;/div&gt;
&lt;p&gt;&lt;a href="https://github.com/finallyjay/fridge-sloth/actions/workflows/ci.yml" rel="noopener noreferrer"&gt;&lt;img src="https://github.com/finallyjay/fridge-sloth/actions/workflows/ci.yml/badge.svg" alt="CI"&gt;&lt;/a&gt;
&lt;a href="https://github.com/finallyjay/fridge-sloth/LICENSE" rel="noopener noreferrer"&gt;&lt;img src="https://camo.githubusercontent.com/fdf2982b9f5d7489dcf44570e714e3a15fce6253e0cc6b5aa61a075aac2ff71b/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4c6963656e73652d4d49542d79656c6c6f772e737667" alt="License: MIT"&gt;&lt;/a&gt;
&lt;a href="https://www.python.org/downloads/" rel="nofollow noopener noreferrer"&gt;&lt;img src="https://camo.githubusercontent.com/d9437b028482ebc68f658618ef016eedfd434dc28b3f6dfbf865fe0317a3243a/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f707974686f6e2d332e31342b2d626c75652e737667" alt="Python 3.14+"&gt;&lt;/a&gt;
&lt;a href="https://ai.google.dev/gemma" rel="nofollow noopener noreferrer"&gt;&lt;img src="https://camo.githubusercontent.com/382705fd3c848af479b0a1006a9bf0ba35879044c15a4b36622c420fa651777f/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6d6f64656c2d47656d6d61253230332d3432383546342e737667" alt="Gemma 3"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Snap a photo of your fridge, get a &lt;strong&gt;grocery list&lt;/strong&gt; and &lt;strong&gt;three healthy meals&lt;/strong&gt; you can cook with what's already there
Runs fully offline on &lt;a href="https://ai.google.dev/gemma" rel="nofollow noopener noreferrer"&gt;Gemma 3&lt;/a&gt; through &lt;a href="https://ollama.com" rel="nofollow noopener noreferrer"&gt;Ollama&lt;/a&gt;. Your fridge photo never leaves your machine.&lt;/p&gt;
&lt;p&gt;Built for the &lt;a href="https://dev.to/challenges/hacktoberfest-week3-2026-10-19" rel="nofollow"&gt;Hacktoberfest Open-Source AI Challenge, Week 3: Slow and Steady&lt;/a&gt;.&lt;/p&gt;
&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;How it works&lt;/h2&gt;
&lt;/div&gt;
&lt;div class="snippet-clipboard-content notranslate position-relative overflow-auto"&gt;&lt;pre class="notranslate"&gt;&lt;code&gt;┌──────────────┐   1. vision (no hints)   ┌──────────────────────┐
│ fridge photo │ ───────────────────────&amp;gt; │ items actually seen  │
└──────────────┘        Gemma 3           │ + amount left        │
                                          └──────────┬───────────┘
┌──────────────┐   2. match by name                  │
│ your staples │ ────────────────────────────────────┤
└──────────────┘     plain code, ~1 ms               │
                                          ┌──────────▼───────────┐
                                          │ grocery list         │
                                          │ (missing / low)      │
                                          └──────────┬───────────┘
                     3. healthy recipes              │
                          Gemma 3         ┌──────────▼───────────┐
                                          │ 3 meals, ≤ 30 min    │
                                          └──────────────────────┘
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Vision:&lt;/strong&gt; Gemma lists the food it can actually see in the photo, with a rough amount (&lt;code&gt;plenty&lt;/code&gt;, &lt;code&gt;some&lt;/code&gt;, &lt;code&gt;low&lt;/code&gt;)
It is &lt;em&gt;not&lt;/em&gt;…&lt;/li&gt;
&lt;/ol&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/finallyjay/fridge-sloth" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;/div&gt;


&lt;p&gt;It's deliberately tiny: a Python server that uses only the standard library (&lt;code&gt;http.server&lt;/code&gt; + &lt;code&gt;urllib&lt;/code&gt;), an optional tracing module, and one HTML page with vanilla JS. The only thing you install is Ollama:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;brew &lt;span class="nb"&gt;install &lt;/span&gt;ollama &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; ollama pull gemma3:4b
python3 server.py   &lt;span class="c"&gt;# open http://localhost:8000&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;There are 35 tests, and Gemma is mocked in them, so they run in CI without a model.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I Built It
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Stack:&lt;/strong&gt; Gemma 3 4B (vision) on Ollama, a zero-dependency Python server, and a single HTML page. The page shrinks phone photos to 1024 px in the browser before sending them, which is plenty for Gemma and much faster.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Structured output everywhere.&lt;/strong&gt; Every call to Gemma passes a JSON schema in Ollama's &lt;code&gt;format&lt;/code&gt; field, so the server always gets back data it can use (&lt;code&gt;{"seen": [{"item": "eggs", "amount": "plenty"}]}&lt;/code&gt;), never prose to parse.&lt;/p&gt;

&lt;h3&gt;
  
  
  The bug that taught me the most: Gemma "saw" whatever I told it to look for
&lt;/h3&gt;

&lt;p&gt;My first version sent the photo &lt;em&gt;together with&lt;/em&gt; the staples list and asked "which of these are in the fridge?". On the USDA photo, Gemma confidently reported butter, tomatoes and apples as "low". None of them are in the picture. Handing a small vision model a checklist primes it to find the items on it.&lt;/p&gt;

&lt;p&gt;The fix was to split the work:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Vision step, no hints:&lt;/strong&gt; "List every food you can actually see. Never guess hidden items." The staples list is not in this prompt at all.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Matching step:&lt;/strong&gt; your staples are compared with the items Gemma actually listed.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;That one change removed the invented items. At first, step 2 was a second, text-only Gemma call. It's now plain code, and the Sentry traces below are why.&lt;/p&gt;

&lt;p&gt;The same rule, &lt;strong&gt;never trust the model's claim, check it&lt;/strong&gt;, came back for the recipes. In one run a recipe listed a &lt;em&gt;banana&lt;/em&gt; "from your fridge" when there was none. Now any recipe ingredient that wasn't actually seen is moved to the grocery list.&lt;/p&gt;

&lt;h3&gt;
  
  
  Asking a small model for less
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;I first asked Gemma to also classify each item (ingredient, drink, condiment…). The 4B model was bad at it ("eggs: condiment"), and those wrong labels starved the recipe step of ingredients. Removing the field worked much better.&lt;/li&gt;
&lt;li&gt;"Healthy" needs to be spelled out. The first recipes included a "smoothie &amp;amp; cheese toast". The prompt now says what a healthy meal is (a real dish built on vegetables, whole grains, legumes, eggs, fish or lean protein; easy on butter, cream and salty sauces; never a drink presented as a meal) and that each meal must use at least two foods actually in the fridge.&lt;/li&gt;
&lt;li&gt;Matching staples doesn't need a model at all (see below).&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Watching the agent with Sentry Agent Tracing
&lt;/h3&gt;

&lt;p&gt;A scan took about a minute, and I wanted to know &lt;em&gt;where&lt;/em&gt; that minute went. Fridge Sloth has optional &lt;a href="https://docs.sentry.io/product/agents/getting-started/" rel="noopener noreferrer"&gt;Sentry Agent Tracing&lt;/a&gt;, off unless you set &lt;code&gt;SENTRY_DSN&lt;/code&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Each scan or recipe request is an &lt;code&gt;invoke_agent&lt;/code&gt; trace with a &lt;code&gt;gen_ai.chat&lt;/code&gt; span per Gemma call, carrying the model, token counts and Ollama's own timings (model load, prompt processing, generation).&lt;/li&gt;
&lt;li&gt;Every check the server runs on Gemma's output is a &lt;code&gt;gen_ai.execute_tool&lt;/code&gt; span: &lt;code&gt;dedupe_seen_items&lt;/code&gt;, &lt;code&gt;match_staples_by_name&lt;/code&gt; and &lt;code&gt;check_recipe_ingredients&lt;/code&gt;. In Sentry's &lt;strong&gt;Agent Activity&lt;/strong&gt; view, the "never trust the model" checks show up right next to the model calls, with what they caught.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;1. The traces exposed a real bug.&lt;/strong&gt; On a messy example photo, two scans failed after about 4½ minutes. The trace showed why: a single vision call ran for &lt;strong&gt;4.46 minutes&lt;/strong&gt;, and the next step never started. The 4B model was stuck in a loop, listing the same items until its JSON answer was cut off at about 10,000 characters.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fy5r228mvamglzsmt7sfo.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fy5r228mvamglzsmt7sfo.png" alt="Sentry trace of the failing scan: one gen_ai.chat vision call running 4.46 minutes, no matching step" width="799" height="274"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The fix: the schema now allows at most 30 items, which Ollama enforces while generating; every answer is capped in length; and a cut-off answer fails fast with a clear message. The same photo then scanned in 1.15 minutes, and that trace showed the next problem.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F28tu7vc4vwrdr38uu1f9.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F28tu7vc4vwrdr38uu1f9.png" alt="Sentry trace of the same photo after the fix: vision 44 s, then matching 25 s" width="799" height="274"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. The traces showed where to optimize.&lt;/strong&gt; The matching step, Gemma comparing two short lists of words, took &lt;strong&gt;~25 s&lt;/strong&gt;, a third of every scan. I first tried asking Gemma only about the staples plain code couldn't place ("is any of these here under another name?"). It was faster but &lt;strong&gt;wrong&lt;/strong&gt;: the 4B model over-claimed, deciding the spinach was the cheddar cheese. So matching is now plain code, with a strict rule: an item counts as a staple only if it &lt;em&gt;ends with&lt;/em&gt; it. "Cheddar cheese" is cheese and "free-range eggs" are eggs, but "tomato sauce" is not tomatoes and "milk chocolate" is not milk. Missing a synonym ("OJ") only means buying something you already have; a false match means &lt;em&gt;not&lt;/em&gt; buying something you need.&lt;/p&gt;

&lt;p&gt;The result, on the same photos and staples:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Photo&lt;/th&gt;
&lt;th&gt;Before&lt;/th&gt;
&lt;th&gt;After&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;USDA (clear)&lt;/td&gt;
&lt;td&gt;51.6 s&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;19.9 s&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Door shelves&lt;/td&gt;
&lt;td&gt;52.9 s&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;20.7 s&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Messy&lt;/td&gt;
&lt;td&gt;68.8 s&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;43.6 s&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A scan is now one Gemma call plus two checks that take about a millisecond each:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F3q02qz69hfaf3yx1apeu.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F3q02qz69hfaf3yx1apeu.png" alt="Sentry Agent Activity for a scan: one gemma3:4b call of 19.9 s, then dedupe_seen_items and match_staples_by_name tool calls" width="799" height="274"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Generation is what takes the time.&lt;/strong&gt; A recipe request is about 800 tokens; nearly all of its ~30 s is Gemma generating the answer, with model loading and prompt reading taking a few seconds. Shorter answers, not a faster prompt, are what would speed it up next.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fra5fmm6puvhyruh8mies.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fra5fmm6puvhyruh8mies.png" alt="Sentry Agent Activity for recipes: one gemma3:4b call of 31.6 s, then the check_recipe_ingredients tool call" width="799" height="274"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. Checking my own setup.&lt;/strong&gt; The looping scans' errors never showed up as Sentry issues, even though their traces did. Sentry's usage stats explained why: the error events had been rejected as &lt;strong&gt;too large&lt;/strong&gt;. By default, Sentry's Python SDK attaches each stack frame's local variables to error events, and here those held the prompt, Gemma's answer and the base64 photo, several times over. So errors from real scans were silently dropped, and smaller ones would have sent exactly the data I promised never to send. One setting (&lt;code&gt;include_local_variables=False&lt;/code&gt;) fixed both, and I checked it end to end: the same error with a real photo now arrives in Sentry at 11 KB, with no prompt, answer or photo in it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Privacy is kept,&lt;/strong&gt; in traces and error reports alike. Only metadata is sent: timings, token counts and how many items each check looked at. The photo, the prompts, Gemma's answers and even the food names never leave the machine, and &lt;a href="https://github.com/finallyjay/fridge-sloth/blob/main/tests/test_tracing.py" rel="noopener noreferrer"&gt;the tests&lt;/a&gt; check exactly that for both.&lt;/p&gt;

&lt;h3&gt;
  
  
  Gemma 3 4B vs 12B
&lt;/h3&gt;

&lt;p&gt;I also tried the 3× bigger model on the same photos, on the same laptop (before the matching speed-up):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;4B&lt;/th&gt;
&lt;th&gt;12B&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Scan, clear photo&lt;/td&gt;
&lt;td&gt;~45 s&lt;/td&gt;
&lt;td&gt;~44 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Scan, messy photo&lt;/td&gt;
&lt;td&gt;~49 s&lt;/td&gt;
&lt;td&gt;~109 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Recipes&lt;/td&gt;
&lt;td&gt;~30 s&lt;/td&gt;
&lt;td&gt;~95 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Messy photo&lt;/td&gt;
&lt;td&gt;missed the milk, called beer "juice"&lt;/td&gt;
&lt;td&gt;saw beer, salsa and bread, but still missed the milk and eggs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Recipes&lt;/td&gt;
&lt;td&gt;decent&lt;/td&gt;
&lt;td&gt;not better (a &lt;em&gt;cream bread pudding&lt;/em&gt;)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;12B reads messy photos a little better, but it's about 3× slower on recipes and the recipes weren't better. &lt;strong&gt;I kept 4B.&lt;/strong&gt; Switching is one environment variable (&lt;code&gt;FRIDGE_MODEL=gemma3:12b&lt;/code&gt;) if your machine is faster than mine.&lt;/p&gt;

&lt;h3&gt;
  
  
  Honest limitations
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;It works well on clear, well-lit photos. On messy, dim ones the 4B model still misreads things: it called a pack of beer "juice bottles", and in the demo photo it reads pineapple slices as banana (hence the yogurt &amp;amp; banana bowl).&lt;/li&gt;
&lt;li&gt;It only knows what's in the photo. Something hidden behind the milk shows up as missing.&lt;/li&gt;
&lt;li&gt;Staples are matched by name, so a synonym like "OJ" won't count as orange juice.&lt;/li&gt;
&lt;li&gt;Recipes are suggestions, not nutritional advice.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Why Does Open Innovation Matter?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A photo of the inside of your fridge is personal.&lt;/strong&gt; It shows what you eat, drink and can afford. With an open-weight model running locally, it never leaves your laptop. Even the tracing sends only timings and counts.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It runs on hardware people already own.&lt;/strong&gt; Gemma 3 4B handles both the vision and the recipes on a 6-year-old M1 laptop. No GPU server, no subscription, no per-request cost, so it can stay free.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;I can see and change everything.&lt;/strong&gt; I could swap models with one environment variable, constrain the output with a JSON schema that the runtime enforces token by token, cap the answer length, and design the pipeline around the model's real weaknesses. With a closed API I'd be guessing; here I could measure the problem and fix it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It works offline,&lt;/strong&gt; including in a kitchen with bad Wi-Fi.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Prize Categories
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Best Use of Gemma:&lt;/strong&gt; Gemma 3 4B does the AI work, reading the photo and writing the healthy recipes, entirely locally through Ollama.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Best Use of Sentry Agent Tracing:&lt;/strong&gt; every model call and every check on its output is traced (Agent Activity screenshots above). The traces exposed a real bug and showed where the time went, which led to scans 2.6× faster on clear photos. Following up on errors missing from Sentry's stats also uncovered a privacy problem in my own error reporting, now fixed.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Best Use of ElevenLabs:&lt;/strong&gt; the demo's voice-over was generated with ElevenLabs text-to-speech (&lt;code&gt;eleven_v4&lt;/code&gt;), one line per moment in the video, stitched so it sounds like one take. The script is &lt;a href="https://github.com/finallyjay/fridge-sloth/blob/main/scripts/narrate_demo.py" rel="noopener noreferrer"&gt;&lt;code&gt;scripts/narrate_demo.py&lt;/code&gt;&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;em&gt;Example photos are from Wikimedia Commons: USDA (public domain) and others under CC BY-SA. Full credits are in the repo.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>devchallenge</category>
      <category>hf26challenge</category>
      <category>ai</category>
      <category>opensource</category>
    </item>
  </channel>
</rss>
