Who's booking the hotel? What small local LLMs get wrong in meetings, and the code that catches it

Sethumadhavan N — Mon, 26 Oct 2026 06:39:04 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

I used this week's DEV challenge to make a product I had been thinking of making for some time now: a voice-based meeting assistant. I had made some earlier versions and prototypes, but most of them were wrappers around a bunch of LLM API calls that orchestrated the flow. I was burdened by two dependencies: the model's availability on the free tier, and the cost of using it for actual meetings.

Both had been hurdles. But Hacktoberfest has brought me not just a solution to my problem, but an opportunity to improve myself towards the role I aspire to.

When I read the challenge parameters, even the general ones that require open models excited me to be part of such a community and competition. But when the special theme dropped, I thought it was a sign from heaven. Slow and Steady, wow. It was perfect for me and my laptop. I had to wait 5 seconds for the challenge page to load, and then I saw that they were actually referring to my laptop. It made me super excited and determined to complete my project.

But this time, I did not just wrap something; I made research my primary focus. Research into which open model was right for my slow but steady laptop: which model made the best of the limited resources my laptop provides, to give a steady performance. And I was very glad to find the actual parameters that matter here.

So in this post (and a research paper I am planning), I will explain in detail the meeting assistant I built, which is powered by local models and runs steadily even on an 8 GB RAM laptop. Yes, it is comparatively slow, but it is a steady tool that provides cost independence, data privacy and, most importantly, complete customization. One could even fine-tune their own personal assistant, right down to the binaries (future work).

In short:

Just click your name and speak: the app transcribes each turn and writes the action items (owner, task, deadline) as sticky notes, one color per person. A white card means nobody has taken that task yet, and a red check marks anything the code is unsure about. You decide. Audio and notes never leave the laptop.

Demo

This video shows a short demo of the app and how to use it. We tested it in an actual meeting with my friends, planning a trip, but for a proper demo the video was recreated from a script based on that meeting.

A scripted re-enactment: screen-recording a live meeting wasn't feasible on this laptop. The voices are synthetic (Kokoro); the transcription, notes and timings are real, from one recorded take.

Code

I have shared the eval harness and the app code in the public repo.

Sethumadhavan004 / meeting-assistant

Local meeting assistant on an 8 GB CPU-only laptop, plus an eval of small open models.

Meeting Assistant: local, on an 8 GB laptop

A privacy-first meeting note-taker that runs entirely on a CPU-only, 8 GB laptop. Click your name talk; it transcribes with Moonshine, turns the meeting into owner · task · deadline with a small open model (Ternary Bonsai 4B by default), and then checks the model's work in plain code Audio and notes never leave the machine.

Behind it is an eval of three small open models on this exact laptop, because the first question was not "how do I build it" but "which model is good enough here, and what does each choice cost?"

Demo video: youtu.be/XEpmtArXQPA (70 s, a scripted re-enactment; see below) · Write-up: link coming (DEV)

What it does

Speaker buttons: click your name, speak, click the next person. Exact names, no guessing.
Speech-to-text on CPU with Moonshine, about 10× faster than real time here.
Action items…

View on GitHub

Setup is in the README. The eval method is in eval/README.md, and how the video was made is in demo/.

How I Built It

Open-source AI used (all local, CPU only):

Moonshine for speech-to-text.
Ternary-Bonsai-2-4B, run with PrismML's llama.cpp fork (the app's default model).
Granite 4.2 3B and Phi-4-mini, run with Ollama.
FastAPI for the local web app.
Kokoro for the demo voices only.

The setup:

A Ryzen 3 4300U with 8 GB of single-channel RAM and no GPU.
8 test meetings with 53 action items that I wrote by hand before any model ran, plus 2 held-out meetings for the second round.
A harness that logs quality, speed, tokens and RAM for every run.

The limited resources I had forced me to do something crucial that everyone should do: research what a model actually does, what is needed and what is not. Below is what I found.

What I found:

The speed limit is arithmetic. The numbers on paper didn't hold up in testing. The smallest model, Bonsai, should have been about twice as fast as the others; in practice its whole runs took longer than Phi's (36 s vs 27 s). Memory bandwidth (21.3 GB/s) divided by model size gives a ceiling on speed. Granite and Phi reached 72–73% of theirs. The 2-bit Bonsai reached only 45%, probably because it is compute-bound. The impact of compute is something that I was able to infer thanks to this study.

"Thinking" made it worse. I tested every model with thinking mode on and off, changing nothing else. With thinking mode on, Granite's recall fell from .94 to .28 and a run took 454 s instead of 44 s. 7 of 16 thinking runs hit the 3,000-token limit with no answer. On a small machine, "think harder" isn't free. We need special optimizations and fine-tuning at different levels, which is planned for future work.

Finding tasks is easy; owners are hard. All three models found 91–94% of the tasks. But "we need a venue" went to the speaker, Akhash, in 5 out of 5 baseline setups. A better system prompt cut invented deadlines from 6 to 0 (Phi and Bonsai), but it didn't fix owners. What these models miss is the subtle part that needs the whole picture: who actually took the task. To solve this, we use helper programs.
About ten lines of plain code fixed owners. Rule: if someone says "we need…" and never takes the task, the task is unowned. This fixed 11 wrong owners with 0 false alarms. (I wrote the rule after seeing these meetings, so it still needs testing on new ones.) The helper programs act as guardrails that enforce what is hard to get through the prompt alone with small models (SLMs).
Real use found what the eval couldn't. The first demo take lost the first half-second of every turn ("We need a hotel" → "Tell near"). The fix was one microphone stream per meeting, with a test that fails on the old code. Lesson: test the whole pipeline with audio, not just text. At the end of the day, actual implementation in an application is where new variables are introduced and one has to tackle them properly.

So in the end, to reach amaze, amaze, amaze, one has to enhance, enhance and enhance.

Why Does Open Innovation Matter?

Open innovation gives one an opportunity to improve not just oneself but also a community. And I am happy to share that I was able to build something because of such open innovation: a meeting assistant that is powered by local models. I did not just use them out of the box, but made an eval harness that compares the available options. This eval harness is my small contribution back to the open innovation without which I would not have been able to build this.

The use of open models also keeps the data on my laptop, and removes the cost dependency.

My actual laptop is very basic, and that is why I was able to research how I can make the best out of the resources I have. This sent me on a journey of understanding what the exact parameters and aspects are that affect the models, and how I can use them. The impact of RAM size is the most fundamental, but I was able to see that one specific model (Ternary-Bonsai-2-4B) reached a much lower share of its bandwidth ceiling, probably because of compute. That is something I was only able to know because I did the actual study. This post and the research paper that is in the works will contain even more insights that I was able to observe.

My Agent Session

The app's AI is all open-weight. For building it, I used Claude Code as a mentor and coding helper: we worked in stages, measured first, changed one thing at a time, and ran a review panel before recording the demo. That plan is in demo/plan-and-panel.md.

The Lazy Payoff

This enables one to properly document meetings that happen between people, with no manual effort other than clicking your name before you speak. In the demo, 18 seconds of speech were transcribed in 4.8 seconds, and the notes were ready 21 seconds after the meeting. The review of the generated notes is a necessary step, and as we improve the app and the harness around the model, we improve the performance as well.

This can be improved and could give better results with bigger models like Gemma 4, but this project was built on a laptop that can't handle them. I expect the performance would be better with them. I would be glad if someone reproduced it with my eval harness and shared how the performance varies.

I would appreciate any feedback or criticism on how one can do a better job, as that enables me to grow, learn and improve myself. So thank you all for that in advance.

I made this application for two purposes. One was to learn and understand how SLMs work, and how one can integrate them into an app at the deepest level, by actually doing it myself. The other was to make a tool that helps my productivity, acting as my primary note-taking app. Open innovation and local models have opened a path to a future in which one can create their own personally styled assistant. Voice-based assistants used to be sci-fi film concepts, but in the storage space needed to store that sci-fi movie, we now have a model and a harness that can actually do it.

And this is just the beginning, and I am super excited to be part of this New Age of Building.

Plus Ultra

The Double-Edged Sword

Sethumadhavan N — Thu, 15 Oct 2026 16:04:41 +0000

It increasingly feels as though nobody wants to hire freshers anymore, as the job that they once did can now be automated via an agent call, and the weekly refresh of tokens that Anthropic does to promote the new feature they drop twice a month does not help.

Feeling stranded is not a new feeling, but the entire landscape being shifted with each new announcement from the "Saviors of Humanity" companies is just too much stimulation for someone who is just starting out in the tech world.

But as someone who is trying to build his career in AI, I am facing the reality of how success is a culmination of hard work and opportunity. Working towards the goal is possible only in the environment that allows you to grow, making you close the distance between yourself and the goal.

But the current recruitment phase in the AI domain has become a double-edged sword.

The so-called big companies are too rigid in their hiring processes, and even though they have resources, they do not take the risk of research and development through actually creating talent in-house. Thus, in this space, the startups are the greatest risk-takers but also those who are most benefitted if their bets pay off. They are the ones who are already taking the risk, believing in an idea, betting against a market that is always clinging to its past, trying to build a product that can do real impact in the world.

Recruitment in startups can be said to be merit-based, where the talented can open that particular door by impressing the founders and tech leads, but what if there is potential but no concrete skills that are immediately deployable? Will they be willing to basically take potential candidates and then grow them into the engineers that solve real-world complex problems, someone who becomes a pillar of support in their collective journey towards achieving their goal?

It is true that we can't just bet on concepts and theories; in the end, only the results matter. One cannot hire a person because he writes a really nice blog or has a very cool project that has yet to become big. The type of person and the energy he brings to the table is a crucial factor that cannot be measured from an ATS score.

The diversity that gives the energy also contains the tension, as ideas collide between strong-minded people. The vibe and rhythm of a team are not just abstract concepts but real implications of how well-coordinated and motivated the team is. And maybe startups are not inclined towards this risk because they already have enough on their plate, and thus make exceptions for the exceptional ones.

If "Ifs" and "buts" were candy and nuts, we'd all have a Merry Christmas. So in the end, I guess we can agree there has to be a test of merit, a threshold that a candidate has to meet, but whether he is given the opportunity to prove himself is the more important question I would ask.

I mean, do the most daring group of individuals dare to wield this double-edged sword?

DEV Community: Sethumadhavan N