DEV Community: Utkarsh Mishra

Lets goo! Had to pull an all nighter to get this submission done before deadline. Finished with exams , got sick. Had only 24h to build something. Somehow figured out what to build and how to build. Check out POT - PeopleOfdPast

Utkarsh Mishra — Sun, 01 Nov 2026 04:23:45 +0000

Hacktoberfest Open-Source AI Challenge: Ghost in the Machine Submission 👻🌼

Utkarsh Mishra

Nov 1

It's time to come back to the POT - PeopleOfdPast

#devchallenge #hf26challenge #ai #opensource

18 min read

It's time to come back to the POT - PeopleOfdPast

Utkarsh Mishra — Sun, 01 Nov 2026 04:15:09 +0000

Welcome to the POT 👻

⚠️ A small note before trying the live demo

I'm hosting the application on Render's free tier, so there's one little thing to keep in mind.

If the demo hasn't received any traffic for a while, the backend service may have gone to sleep. The first request can take a little longer while Render wakes it up.

So, if the application appears to be loading or the first API request takes time, please give it a moment before refreshing the page.

It's a free-tier cold start, not necessarily a broken demo. :)

Building People of the Past for Hacktoberfest Open Source AI Challenge — Week 4: Ghost in the Machine

Every time I thought I had figured out the AI part of this project, the problem moved one layer deeper.

First, I needed a model that could hold a conversation. Then I realised a conversation without memory was just another chatbot. Once I started building memory, I had to think about retrieval, context, identity, privacy, and what happens when the AI confidently says something that was never actually told to it.

And somewhere in between, a modal decided it wanted to live in the top-left corner of the screen. :)

This is the story of building People of the Past (POT): the decisions that shaped it, the things that broke, the experiments that didn't go as planned, and the architecture I ended up with.

01. What did I build?

People of the Past started with a simple idea:

What if we could have meaningful conversations with people from the past, while preserving the memories and stories that make those people special?

I eventually split that idea into two experiences.

Links

Live Demo: https://youtu.be/zwuG8uBSCRQ
Source Code: https://github.com/utkarsh-g19/PeopleOfdPast
Hosted on Render: https://people-of-the-past-web.onrender.com/
Hacktoberfest Challenge: Open Source AI Challenge — Week 4: Ghost in the Machine

👻 Ghost Gallery — conversations with people from history

Ghost Gallery lets users explore conversations with notable historical figures.

The challenge here isn't simply making an AI imitate a historical personality. A convincing response that invents historical facts is not a good response.

I wanted the experience to be informed by historical profiles and source details, rather than relying entirely on whatever the model happens to know.

The historical profiles are stored separately from private TYLO companions. That distinction matters: historical information and personal family memories have very different privacy and grounding requirements.

❤️ TYLO — Talk to Your Loved Ones

TYLO is the more personal side of the project.

It allows someone to create an AI memory companion using information they provide about a loved one who has passed away.

Instead of asking users to write one enormous prompt, I built a structured profile that captures different dimensions of a person's life:

Section	Information collected
Their life	Life stages, relationship, family and friends, important dates
Their personality	Nature, nicknames, speaking style, favourite phrases
Your shared world	Special moments, stories, routines, interests and hobbies
What they believed	Values, likes and dislikes, lessons and advice

A profile also includes a photograph and an introduction.

The goal is to give the system more than a name and a generic personality description. It needs the little details: the nickname, the familiar phrase, the Christmas tradition, the way someone approached a difficult day.

Those details are what make a memory personal.

One distinction I want to make very clear: TYLO does not bring someone back, and it does not recreate their actual consciousness. It generates conversations using information and memories supplied by the user. It is a space for remembrance, not a replacement for the person.

02. The first big realisation: a prompt is not a memory system

My earliest instinct was to think about the model first.

Which model should I use? How should I write the system prompt? How do I make the responses sound natural?

But I eventually ran into a much more important problem.

Imagine someone creates a companion and tells it:

Their grandmother was always positive.
She loved being around people.
She had a particular nickname for them.
Every Christmas, the family had a special tradition.
She would often say, "Oh, come on, we will get it done."

Now imagine the user returns after several conversations and asks about that Christmas tradition.

A model that only sees the latest message has no reliable way of recalling it.

Sending every piece of profile information and the entire conversation on every request is not a sensible long-term solution either. Context needs to be selected.

That is where the TYLO memory architecture became the centre of the project.

I stopped thinking of TYLO as simply a chatbot with a personality prompt. I started treating it as a system with several sources of context:

The companion's profile.
Recent conversation history.
Saved memories and the companion summary.
Relevant older conversation exchanges retrieved through vector search.

The model generates the response. The memory system decides what information the model gets to work with.

That separation changed how I approached the backend.

03. How TYLO remembers: the actual memory architecture

This is the current architecture behind a TYLO message.

The database structure

I'm using MongoDB Atlas for both the application data and the vector retrieval layer.

The main collections are:

Collection	Responsibility
`personas`	Published historical profiles and their source details
`companions`	TYLO profiles, saved memories and summaries
`tylomessages`	Persisted TYLO conversation messages
`tylomemories`	Conversation exchanges and their 768-dimensional embeddings

The distinction between tylomessages and tylomemories is intentional.

The message collection stores the conversation history. The memory collection provides a retrieval-oriented representation of conversation exchanges, including their embeddings.

This allows the application to maintain recent conversational context while also looking for older, semantically relevant exchanges.

Embeddings and Atlas Vector Search

When a user sends a message, the API creates an embedding for the question.

The expected embedding size in this implementation is 768 dimensions.

That vector is used to query the MongoDB Atlas Vector Search index:

tylo_memory_vector_index

The current retrieval configuration is:

Setting	Value
Vector index	`tylo_memory_vector_index`
Expected vector dimensions	768
Candidate documents	50
Relevant exchanges returned	Up to 4
Ownership filters	`ownerSub` and `companionId`
Model compatibility filter	`embeddingModel`

Why retrieve 50 candidates if we only need up to four?

The idea is to search a wider candidate set and then provide a smaller, relevant selection to the generation step. The model doesn't need fifty old exchanges in its prompt. It needs the few that are useful for the current message.

There is also an important reason for filtering on embeddingModel: vectors generated by different embedding models should not be casually treated as interchangeable. Retrieval needs to respect the embedding model used to create the stored vectors.

What actually enters the model's context?

This is the part I wanted to make explicit because it is easy to say "we use vector search" without explaining what happens after the search.

For a typical TYLO message, the context is assembled from:

Context source	Maximum included
Companion profile	Relevant profile information
Recent conversation history	Latest 8 message records
Saved memories	Up to 3
Companion summary	Current summary
Retrieved conversation exchanges	Up to 4

The result is a bounded context assembled from several sources, rather than blindly sending the entire conversation.

The memory retrieval step also has a fallback.

If vector retrieval fails, the API logs the failure and continues with the companion profile and recent conversation history.

That doesn't magically make missing memories available. It does, however, prevent a retrieval failure from automatically turning into a completely failed conversation.

The important part is that memory creation and memory retrieval are separate stages.

A conversation exchange can be persisted first and processed for embedding in the background. On a later message, that exchange can become a candidate for semantic retrieval.

This is the foundation of TYLO's longer-term memory behaviour.

04. Why Auth0? Because TYLO is not an ordinary chatbot.

This was one partner decision I didn't want to treat as just another item in the technology stack.

Ghost Gallery deals with published historical information.

TYLO is different.

A user could upload a photograph of a loved one, describe their family relationships, share private experiences, explain personal traditions, and write down memories they may never have shared publicly.

That information deserves a proper identity and access-control layer.

I chose Auth0 because I needed to establish who was making a request before the API accessed their private companion data.

What the integration does

On the frontend, I'm using:

@auth0/auth0-react

On the Express API, token validation is handled using:

express-oauth2-jwt-bearer

The basic flow is:

The user authenticates through Auth0.
The frontend sends an Auth0 access token with a protected TYLO API request.
The Express API validates the token's issuer and audience.
The API reads the authenticated user's subject.
Companion and memory access is scoped to that identity.

This is especially important for vector search.

Consider two users who have both created a companion profile for their grandmother. Their memories might contain similar words, names, family relationships or traditions.

A semantic search that ignores ownership could return a memory belonging to the wrong user.

That is unacceptable for TYLO.

The vector retrieval query is therefore scoped using:

ownerSub
companionId
embeddingModel

The ownership identity must come from the validated token, not from an arbitrary user identifier supplied by the browser.

What Auth0 does not automatically do

I also want to be precise about the security claims.

Auth0 establishes and verifies identity and gives the API a trusted identity context. It does not automatically encrypt every MongoDB record, enforce every database filter, or make the application end-to-end encrypted.

Those responsibilities still belong to the application's backend, database configuration and data-handling design.

I have also not claimed to implement Auth0's more advanced AI-agent capabilities such as Token Vault, asynchronous authorization or fine-grained authorization unless those features are actually integrated.

For this project, the implemented value is the identity and protected API layer around private AI conversations.

That is why Auth0 mattered to the architecture rather than simply being another login button.

05. Choosing Gemma: my laptop had an opinion too.

Now for the model side.

I wanted to experiment with open-weight models and run local inference during development. Ollama made it straightforward to pull a model and connect it to the application.

Then I met the less exciting part of local AI development:

Hardware limitations.

I experimented with different local model options, but my laptop's available RAM made larger models uncomfortable to work with. I needed something practical enough to test repeatedly, not a model that turned every experiment into a hardware-management exercise.

I settled on experimenting with Gemma through Ollama, including the gemma4:e2b model.

It wasn't about declaring one model universally better than everything else. It was about finding a practical model that allowed me to keep building and testing.

The model was only half the work

I did not fine-tune Gemma or retrain its weights.

The improvements I worked on were at the application level:

Refining the instructions given to the model.
Supplying relevant companion profile information.
Selecting recent conversation context.
Retrieving older exchanges through vector search.
Including saved memories and the companion summary.
Adjusting how the application assembles the final context.

In other words, I was improving the information available to the model and the way the task was framed.

This distinction is important.

A model can produce a poor answer because its underlying capabilities are limited. But it can also produce a poor answer because the application supplied incomplete, irrelevant or misleading context.

Those are different problems, and they need different solutions.

The model provider is configurable

The backend is not tightly coupled to a single local Ollama process.

The inference architecture supports Cloudflare Workers AI when the required API credentials are configured. Otherwise, the local inference path uses Ollama through MODEL_BASE_URL, which defaults to:

http://localhost:11434

This separation helped keep the API architecture flexible.

It also created an important deployment lesson: Render does not automatically have access to the Ollama instance running on my laptop.

The API service needs a reachable inference endpoint. Cloudflare credentials or another correctly configured model endpoint must be supplied to the deployed API.

I don't want to claim that Render hosts Gemma inference. It hosts the application services; inference is handled separately.

06. The architecture: Render, Express and MongoDB Atlas

The application is split into separate services.

Layer	Technology	Responsibility
Frontend	React + TypeScript + Vite	User interface and application experience
Styling	Tailwind CSS	Layout and visual design
Routing	React Router	Navigation between experiences
Identity	Auth0	Authentication and access-token identity
Backend	Node.js + Express + TypeScript	API, validation and application logic
Database	MongoDB Atlas	Profiles, messages and memories
Retrieval	Atlas Vector Search	Semantic memory retrieval
Inference	Cloudflare Workers AI / Ollama	Model inference through configured provider
Deployment	Render	Frontend and API hosting

Why separate the frontend and API?

The React application is served as a Render static site.

The Express backend runs as a separate Render web service.

This keeps the responsibilities clear:

The frontend handles the user experience.
The API handles identity validation, database access and context assembly.
MongoDB Atlas stores application data and supports memory retrieval.
The configured inference provider generates the response.

Cloudflare credentials are not declared in render.yaml in the architecture snapshot I'm documenting. They need to be supplied through the API service's environment configuration to use the Cloudflare path in production.

Otherwise, the API falls back to the configured Ollama endpoint. On Render, localhost refers to the API service environment, not my personal laptop.

It sounds obvious once you draw the diagram. It is considerably less obvious when you're staring at a deployment that works locally and doesn't behave the same way remotely.

07. The testing: where the architecture had to justify itself

I don't think "I sent a message and the model replied" is a meaningful test for a project like this.

A memory companion can generate a warm, natural response and still get the important details wrong.

For TYLO, I wanted to look at whether the conversation reflected the profile, whether the memory architecture could supply relevant context, and whether the system respected the boundaries around private information.

Here are some of the checks and observations that shaped the implementation.

Test 1 — Does the profile influence the response?

I created a sample companion profile with information across the four sections: personality, speaking style, favourite phrases, shared experiences, beliefs and routines.

Then I tried a simple greeting.

The resulting conversation reflected the positive, encouraging tone described in the profile.

That was encouraging. It showed that the companion profile could influence the generated response rather than the model replying with a completely generic greeting.

But the response also included a present-day detail that had not been supplied in the profile.

That was an important observation.

The response sounded natural, but sounding natural is not the same as being grounded in memory.

It exposed a distinction I need to preserve throughout the project:

The system can use a person's personality and memories as conversational context, but it should not invent personal facts and present them as remembered experiences.

That is an ongoing quality and prompt-design challenge, not something I will pretend has been completely solved.

Test 2 — Can the application use older context?

A conversation has two different memory requirements.

The first is immediate continuity: understanding what was said a few messages ago.

The second is long-term recall: finding something meaningful from an older exchange that is no longer in the recent message window.

The current context assembly addresses both:

The latest 8 message records provide recent conversational continuity.
The companion summary provides compressed context.
Up to 3 saved memories provide additional profile-level information.
Atlas Vector Search retrieves up to 4 relevant older exchanges from a candidate set of 50.

The consequence is architectural: an older relevant exchange does not have to be present in the latest eight messages to be considered for the next response.

I haven't produced a formal retrieval-accuracy benchmark, so I won't attach a made-up percentage to this improvement. The retrieval configuration is implemented; its quality still needs systematic evaluation across a larger set of real test cases.

Test 3 — Can one user's memories leak into another user's conversation?

This is one of the most important tests for TYLO.

A normal search might find semantically similar memories across multiple users. But TYLO cannot treat all stored memories as one shared pool.

The retrieval query includes the verified owner identity and companion identifier.

That gives the memory search a defined ownership boundary, rather than relying on the model to decide which memories are appropriate.

The implementation uses ownerSub and companionId as filters.

The next level of verification here is a dedicated multi-user integration test: create separate users with similar companion profiles, seed similar memories, and verify that each user can retrieve only their own data. The filters are part of the architecture, but I won't describe that as a completed security audit.

Test 4 — What happens when retrieval fails?

Vector search is an additional source of context, not the only source.

The failure path logs a retrieval error and continues with the companion profile and recent history.

That gives the application a graceful-degradation path.

It is better to have a conversation with limited context than to silently pretend that missing memories were successfully retrieved.

Test 5 — Does the profile creation flow enforce the required information?

The TYLO creation flow was also tested beyond just its appearance.

The companion creation process requires a photograph and the structured profile details. Validation exists on the server as well as in the interface.

That matters because frontend validation alone can be bypassed.

The image flow also needed attention: image selection, JPEG compression, previewing the selected image and saving the resulting image data with the companion document.

The current implementation stores the photo as image data in MongoDB. That is straightforward for an initial version, but it increases document payload size. Moving images to dedicated object storage is a sensible future improvement.

What the tests actually establish

I want to be honest about the level of verification.

Area	What has been established	What is still needed
Profile creation	Required photo and profile validation, including server-side validation	Broader browser and edge-case testing
Image handling	Selection, compression, preview and persistence flow implemented	Evaluate object storage as usage grows
Conversation	A sample profile-driven response was generated	Larger grounded-response evaluation
Memory retrieval	Vector index, candidate retrieval, limits and filters are part of the architecture	Formal recall and relevance evaluation
Isolation	Owner and companion filters are used for retrieval	Dedicated multi-user security integration tests
API	Health endpoint returned successfully outside the restricted development environment	Full deployed request-path verification
Build	`npm run build` passed at the documented development snapshot	Re-run after subsequent changes
Full AI conversation	The application flow and a sample conversation are present	A complete end-to-end test across deployed Auth0, API, MongoDB and the configured inference provider

This is not a claim that the system is production-audited or that retrieval accuracy has been scientifically measured.

It is a record of what works, what the design does, and what still deserves testing.

08. The bugs that had nothing to do with AI

Some of the most frustrating problems had absolutely nothing to do with the model.

While building the frontend, I ran into several regressions that reminded me how much work exists between a feature working in isolation and the application feeling coherent.

A few examples:

The Explore Story page. It initially displayed a large chat composer where I wanted a focused story experience. The selected historical figure also needed to load directly from its slug. The navigation and route handling had to be corrected so a figure could be opened reliably.

The Talk Now modal. The modal had an issue with its lifecycle and could close unexpectedly. React Strict Mode exposed behaviour that wasn't as stable as I wanted. I had to fix the modal's interaction handling, including Escape and backdrop behaviour.

The modal position. After a CSS reset, the modal appeared pinned to the top-left instead of sitting in the centre. It was one of those bugs where the code looked reasonable until the browser rendered it. Explicit positioning fixed it.

The TYLO layout. The companion creation form was appearing before the existing memory spaces and conversations. That made the page feel like a setup screen even for someone who had already created companions. Reordering the experience made the saved spaces and conversations easier to reach.

Historical portraits. Finding images for historical figures was another problem. A URL returning an image does not prove that the image depicts the right person. Some candidate images needed stricter filtering and manual review. An unrelated film still or an incorrectly attributed portrait can undermine the whole experience.

I also used Codex to help inspect and fix targeted issues. It was useful for identifying regressions and making focused changes, but I still had to review the results and build the application again.

At one point, the changes had passed the build check, but the full live AI conversation had not been verified end to end. That distinction matters.

A successful build tells me the code compiles. It doesn't tell me that Auth0, the database, vector retrieval and the deployed inference provider all cooperate correctly in a real conversation.

09. The DigitalOcean plan that never happened

I initially considered provisioning a DigitalOcean Droplet to run model inference.

The idea was straightforward: host the model separately and have the API communicate with it.

But I got stuck at the billing setup and, honestly, I was too lazy to finish the provisioning process at that stage.

So I moved forward with the architecture I could actually work with: Render for the web application and API, MongoDB Atlas for persistence and retrieval, and a configurable inference provider.

It wasn't some grand technical decision after an extensive infrastructure benchmark.

Sometimes the engineering story is simply that the infrastructure you planned is not the infrastructure you end up using.

I am not claiming a DigitalOcean integration, because I did not provision one.

10. Why each partner technology belongs in this project

I didn't want to select partner categories just because a technology appeared somewhere in the dependency list.

Each one needs to have a meaningful responsibility in the application.

Best Use of MongoDB Atlas

MongoDB Atlas is the persistence and memory-retrieval layer.

It stores the historical profiles, TYLO companions, conversation messages and memory exchanges. Atlas Vector Search supports semantic retrieval over the 768-dimensional memory embeddings.

The tylo_memory_vector_index index, candidate limit, returned exchanges and owner/companion filters are part of the actual memory architecture.

Without this layer, TYLO would have a much weaker approach to long-term conversational recall.

Best Use of Render

Render hosts two distinct application services:

The React frontend as a static site.
The Express API as a separate web service.

The API handles protected requests, database operations, context assembly and communication with the configured model provider.

Render is responsible for hosting the application services. It is not the machine running my laptop's local Gemma model.

Best Use of Auth0 for AI Agents

TYLO works with private, user-provided information.

Auth0 establishes the authenticated identity used by protected API requests. The Express backend validates the access token, reads the user's subject and uses that identity when accessing companion and memory data.

The relationship between identity and memory retrieval is the important part here.

A user should not be able to access another person's companion simply because a vector search finds similar text.

I'm describing the Auth0 functionality actually integrated: authentication, access-token validation and protected API access. I'm not claiming Token Vault, asynchronous authorization or fine-grained authorization features that are not part of the current implementation.

Best Use of Gemma

Gemma was part of my local model experimentation through Ollama.

The hardware limitations on my laptop influenced the model I could practically use during development. I then iterated on the application-side instructions and context assembly to improve the quality of the conversations.

I did not fine-tune Gemma.

One important submission detail: the deployed inference provider is configurable. I will make sure the submitted demo's actual model configuration supports the Gemma category claim. A local development experiment alone should not be presented as proof that the deployed demo is currently running Gemma.

Categories I am not claiming

DigitalOcean: I considered it but never provisioned the Droplet.
ElevenLabs: It belongs to a different project direction, not the current POT implementation.
GitHub Copilot: I used Codex for coding assistance; I am not claiming GitHub Copilot integration.

I would rather enter fewer categories with a defensible technical explanation than list every partner name without meaningful usage.

11. Open innovation: why open models mattered to me

One of the most interesting parts of this project was being able to experiment with open-weight models rather than treating the AI layer as an invisible external service.

Using Ollama gave me a practical local development path. I could experiment with a model, modify the application prompt, change the context being supplied and inspect the response.

That made the model layer feel like something I could investigate and improve around.

But there is a distinction I want to preserve:

Open-weight model experimentation does not automatically make an application fully local or private.

POT also uses hosted infrastructure. MongoDB Atlas stores application data, Render hosts the application services, and the configured inference provider may be cloud-based.

The system's privacy and data-handling properties depend on the actual deployment and configuration, not simply on whether the model's weights are open.

That is especially important for TYLO.

12. What I would improve next

There are several things I would like to take further.

A proper memory evaluation suite. I want a repeatable collection of questions where the correct answer depends on an older profile detail or a previous conversation. I would measure whether the right exchange is retrieved, whether irrelevant memories are excluded and whether the generated answer stays within the available evidence.

Stronger grounding behaviour. The sample conversation showed that a warm response can still introduce unsupported details. I want the companion to distinguish between something explicitly remembered, a reasonable conversational response and something it simply does not know.

A dedicated privacy test suite. Multi-user tests should verify that similar companion profiles and overlapping memories never cross ownership boundaries. I also want broader tests around protected routes and companion access.

Object storage for photographs. Storing compressed image data directly with companion documents is convenient at this stage, but dedicated object storage will be a better direction as the number of users and images grows.

Historical source quality. Ghost Gallery needs a stronger process for reviewing profile information and portrait attribution. A historical conversation should not sound confident about information that is uncertain or unsupported.

A more complete deployed integration test. I want to verify the full path—from Auth0 authentication to the Express API, MongoDB retrieval, configured model inference and the streamed response—in the actual deployed environment.

13. The part I enjoyed the most

The most satisfying moment wasn't getting an API endpoint to return a response.

It was seeing a structured companion profile turn into an actual conversation.

A photograph, a relationship, a collection of everyday details, favourite phrases and stories—all of that information started to become useful conversational context.

It made me think differently about what a memory system needs to do.

It also made me more cautious about what it should say.

A convincing imitation is not necessarily a respectful one. The system needs to be thoughtful about the difference between what a person actually shared and what a language model is capable of inventing.

That is probably the biggest lesson I am taking away from this build.

The AI part is exciting. The engineering around it is what makes the experience meaningful.

And there is still a lot I want to get right.

Prize Categories

Overall Winner 👀
Best Use of MongoDB Atlas
Best Use of Render
Best Use of Auth0 for AI Agents
Best Use of Gemma

Imagine using your own tool to check if you are one of the winners of previous week's challenge 👀

Utkarsh Mishra — Fri, 30 Oct 2026 15:59:19 +0000

Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady Submission 🦥

Utkarsh Mishra

Oct 25

I struggled to understand Hacktoberfest winners projects, so I built ReadYou to summarize them—and let me talk to the projects 🎧

#devchallenge #hf26challenge #ai #opensource

11 min read

I built ReadYou because I kept losing time trying to figure out what interesting open-source projects actually do. It turns Hacktoberfest winners into short, source-grounded briefings—and lets you ask questions by text or voice.

Utkarsh Mishra — Sun, 25 Oct 2026 21:56:27 +0000

Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady Submission 🦥

Utkarsh Mishra

Oct 25

I struggled to understand Hacktoberfest winners projects, so I built ReadYou to summarize them—and let me talk to the projects 🎧

#devchallenge #hf26challenge #ai #opensource

11 min read

I struggled to understand Hacktoberfest winners projects, so I built ReadYou to summarize them—and let me talk to the projects 🎧

Utkarsh Mishra — Sun, 25 Oct 2026 21:48:51 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady.

(Please be patient while going through the video demo 🫶)

You know what I find frustrating?

Finding an open-source project that looks genuinely cool, starting to read about it, and then getting completely lost in the technical explanation. 😭

The architecture looks interesting. The developer clearly put a lot of thought into it. But then come the unfamiliar terms, workflows and design decisions—and the feeling that I need to understand five other technologies before I can understand this one project.

I was going through Hacktoberfest winners and spending way too much time trying to figure out what each project actually did.

I kept thinking: There has to be a less exhausting way to do this.

So I started building ReadYou.

What I Built

ReadYou collects Hacktoberfest winner announcements and turns the projects into readable technical briefings, short audio introductions, and conversations about individual projects.

Feature	What it does
🏆 Winner digest	Brings winning projects together in one place
🧠 Gemma 4	Generates project briefings and answers questions using project sources
🔎 Repository checks	Treats linked READMEs as candidates and excludes ones that don’t confidently match the submission
🎧 ElevenLabs audio	Turns project introductions into playable audio
🗣️ Voice Mentor	Lets you ask about a selected project by typing or speaking, then hear the answer
🔔 Scheduled updates	Checks for new winner announcements automatically and can email you when it finds them

The idea isn’t to replace the original developer’s documentation. I want to understand the big picture first, so when I open the README, I’m not assembling the entire project from scratch in my head.

Automated Email ( as soon as the scheduler discovers new relevant announcement , digest build is created to create summaries and email is sent to notify the user )

Job Scheduling ( scans immediately and at every 3h for any new updates i.e. the winner announcements )

ReadYou is my attempt to make that first step less painful.

Code: utkarsh-g19/ReadYou on GitHub

How I Built It

I’m a student developer, and this project became a fun excuse to experiment with local AI, backend architecture and voice technology. I used ChatGPT to reason through architecture and debugging, and Codex to inspect the repository and implement changes.

The most interesting moments weren’t when something worked immediately. They were when something looked like it worked, and then I found out it was doing the wrong thing.

😭 The bug that made me rethink my research pipeline

A DEV submission can contain several GitHub links. My first approach was to retrieve the READMEs and give them to Gemma along with the submission.

Simple, right?

Except some links pointed to dependencies, examples, frameworks or unrelated repositories. Gemma could read those READMEs and confidently describe features or technologies that the actual submission didn’t have.

That’s a nasty kind of bug: the summary looks convincing, but the information underneath is wrong.

I didn’t want to keep patching individual bad summaries. I wanted to stop feeding the model the wrong information in the first place.

Earlier approach 😬	Updated approach 🛠️
Fetch up to five linked READMEs	Treat them as candidates, not trusted sources
Assume a linked repository is relevant	Ask Gemma to compare it with the actual submission
Include candidate docs in the summary prompt	Include only documentation that passes a confidence threshold
Risk repeating a cached mistake	Regenerate selected summaries while retaining previous versions

The classifier returns structured JSON with a relevance decision and confidence score. The backend requires a score of at least 0.85. If classification is uncertain or fails, the README is excluded and ReadYou uses the DEV submission on its own.

That threshold doesn’t guarantee Gemma can never be wrong. It makes the pipeline cautious when the evidence isn’t convincing.

There was another trap: the cache could preserve a bad summary even after I fixed the research code. ReadYou stores processed winners by submission URL, so changing the prompt alone wouldn’t repair summaries already saved.

I added targeted summary regeneration and kept previous summary versions in the cache history. I used it to regenerate affected summaries from the Build for a Friend and Touch Grass winner announcements without wiping the rest of the application state.

I learned that a reliable AI application isn’t just about writing a better prompt. Sometimes the real fix is changing what the model is allowed to see.

🧠 Playing with Gemma 4 E2B

I’m running gemma4:e2b locally through Ollama.

I chose the E2B variant as a practical starting point for local inference on student-level hardware. I wanted to experiment with an open-weight model I could run and configure myself, instead of making every AI operation depend on a hosted model API.

I’m not claiming it’s the best model for every task. Smaller models have limitations, and I noticed them quickly once I started building the chatbot.

My first attempts at project chat could be repetitive, irrelevant or overly cautious. A question like “What is this project all about?” could fall through to Hacktoberfest boilerplate instead of explaining the project.

I remember thinking: Come on, you’re looking right at the project. Just tell me what it does! 😂

The issue wasn’t just the model. My source-selection logic sometimes prioritised the wrong material for the question. I improved it so broad overview questions look for project-specific sections such as “What I Built”, rather than challenge boilerplate.

I also found a fallback that could respond with “I couldn’t put that together clearly” and then paste a technical source sentence. I changed that path to give Gemma one short retry, asking it to explain the answer plainly using the same selected sources.

Then I ran into a broader problem: people can ask the same question in very different ways. Punctuation, typos or casual phrasing shouldn’t decide whether a question is about the project. I changed the shared chat flow so Gemma assesses whether the question is about the selected project and drafts its answer from the project sources in the same call. The backend checks the supporting quotes; an unrelated question gets a redirect, and an unsupported project question gets an uncertainty response.

That’s a workflow change for all projects, not a collection of project-specific answers. But the model can still misunderstand a question, so I wanted to be clear about what I’ve actually tested.

What my tests tell me—and what they don’t

The latest backend test run passed 45 tests. These cover behaviors such as accepting a casually formatted project question when Gemma marks it as relevant, distinguishing an unrelated question from a related-but-undocumented one, rejecting an invalid scope assessment, and checking that evidence quotes appear in their cited source sections.

Those tests verify the application’s handling of model output. They are not a benchmark of Gemma’s real-world accuracy. I haven’t yet run a systematic live-model evaluation across a fixed set of projects and paraphrased questions, so I can’t claim a success rate.

That’s still work I need to do: test ordinary questions, typos and punctuation, follow-ups, unrelated questions, and questions the project sources can’t answer—and record where the model gets the scope or answer wrong.

The chat service asks Gemma for claims with supporting quotes. The backend checks that each quote appears in the cited source section. That verifies the quote and citation match; it doesn’t prove the quote logically supports every claim.

I also worked on follow-up questions and language preferences, including Hindi. I want Voice Mentor to feel like an informal conversation with a developer friend, not a report generator that has learned to add question marks.

🎙️ ElevenLabs: I didn’t want another button that just says “Listen”

The basic audio feature takes a short project introduction and uses ElevenLabs text-to-speech to generate an MP3.

I also didn’t want replaying the same introduction to trigger another generation request every time. Generated introductions are cached locally, and duplicate requests can reuse a generation that’s already in progress.

But I wanted to experiment with something more interactive. That’s where Voice Mentor came in.

Instead of making people read a complicated explanation, I wanted them to ask questions like:

“Okay, but what does this project actually do?”
“Why did they choose this tech stack?”
“Can you explain that workflow more simply?”

Here’s how the voice interaction works:

Step	Technology	Responsibility
1. Ask aloud	ElevenLabs Scribe	Transcribes the spoken question
2. Reason	Gemma 4 via Ollama	Answers using the selected project sources
3. Speak back	ElevenLabs TTS	Turns the answer into audio

It’s a sequential pipeline, not real-time speech-to-speech. ElevenLabs handles speech recognition and synthesis; Gemma does the reasoning locally.

The interface shows listening, thinking and speaking states based on what the app is doing. Users can interrupt spoken answers with another question, and typing is available if they don’t want to use a microphone. Spoken answers are cleaned up so the app doesn’t read Markdown headings and URLs aloud, while the fuller answer and its sources stay on screen.

There’s still room to improve, especially around latency and answer quality, but this is the kind of interaction I wanted to explore.

🎨 A little funk in the interface

I wanted ReadYou to feel inviting to browse, not like a plain dashboard full of project links. So I used warm cream backgrounds, chunky dark borders, and bright orange, lime and lavender accents. The cards and labels are playful, but I’ve also been adjusting the typography so the project descriptions stay comfortable to read.

🏗️ Why I kept the backend separated

I didn’t want every feature crammed into one enormous server.js file.

The backend separates winner discovery, DEV article retrieval, GitHub research, repository relevance checks, summaries, chat and ElevenLabs integration into services and controllers. The digest service coordinates the pipeline and manages cached state.

Digest data and generated MP3s are stored locally. Chat sessions stay in backend memory and expire rather than being saved in the digest cache. File-based persistence and in-memory sessions have limits, but they kept the project understandable and easy for me to debug without adding infrastructure I didn’t yet need.

The announcement scheduler checks for new DEV Team posts when the backend starts and every three hours by default while it’s running. If email is configured, ReadYou can notify me when a scan finds new winner announcements. That means I don’t have to remember to check manually, though the scheduler only runs while the backend is online.

The frontend production build passed as well.

Why Open Innovation Matters to Me

I could have built ReadYou around a hosted model API and asked it to summarise every project. That might have been the quicker route to a first demo.

Instead, Gemma runs locally through Ollama. I can inspect the prompts, change the retrieval logic and experiment with the model’s behaviour.

That doesn’t make the whole application offline. ReadYou still fetches source material from DEV.to and GitHub, and ElevenLabs handles speech transcription and synthesis. Those voice features need an internet connection and an API key.

For me, open innovation means having the freedom to understand what I’m building, change it when it doesn’t work, and share what I learned with other developers.

Learning from Previous Winners

ReadYou’s summaries helped me quickly explore past ElevenLabs category winners: Padh Ke Batao, Outside and Auditly. I could get a quick sense of what each project did, then look closer at the ideas that caught my attention.

Padh Ke Batao uses voice to make complex documents easier to understand. It got me thinking about keeping ReadYou’s audio introductions clear and concise, and caching them so replaying one wouldn’t generate it again.

Outside makes listening interactive. It inspired me to let Voice Mentor users interrupt a spoken answer and ask a follow-up question.

Auditly showed me that reliable voice interaction depends on more than speech. In ReadYou, I focused on answering from each project’s selected sources and checking that evidence quotes appear in the cited text.

I didn’t recreate these projects or all their features. I adapted ideas I found useful to a different problem: helping developers understand open-source projects.

I started by struggling to understand other people’s projects. Along the way, those same projects taught me how to build mine better. ❤️

The Lazy Payoff 🦥

I haven’t measured exactly how many minutes ReadYou saves, so I’m not going to invent a number.

The intended payoff is straightforward: get a quick understanding of a winning project, listen to its introduction, ask questions when something doesn’t make sense, and decide where to spend your deeper reading time.

The bigger payoff for me has been learning how to put local inference, evidence retrieval, backend services, caching and external speech APIs together—and how to debug the system when the output looks right but the information underneath is wrong.

The small chore I wanted to avoid was repeatedly checking for new Hacktoberfest winners, then opening every submission just to work out which projects were worth a closer look.

ReadYou checks for new DEV Team announcements when the backend starts and every three hours while it’s running. If I configure email, it can notify me when a scan finds new winners. The digest gives me a starting point: a project briefing, a short introduction I can listen to, and Voice Mentor when I want to ask a question.

I haven’t measured how much time that saves, so I won’t make up a number. The useful part is that I can catch up on projects in one place and spend my reading time on the ones that interest me.

What’s Next?

There’s still work to do.

Better source coverage: Some DEV submissions are vague or don’t have useful READMEs. When the evidence isn’t there, ReadYou needs to say so rather than invent an explanation.
Faster local inference: Gemma’s response time depends on the hardware, and the sequential voice pipeline adds latency.
More voice testing: Transcription quality depends on the recording, language and service capabilities. Hindi handling has improved, but I haven’t established universal multilingual reliability.
More chat evaluation: I need to test live Gemma responses across more projects and question styles. Source checks and focused backend tests catch some failures, but they don’t guarantee a useful conversation.
More than Hacktoberfest winners: I’d like to adapt the same collect → check sources → summarize workflow to produce weekly briefings from technical reports and other useful publications across different websites.

I’d rather be honest about these limitations than call the project finished just because the demo works.

Prize Categories 🏆

Best Use of Gemma

Gemma 4 E2B is central to ReadYou’s research and explanation workflow. It generates project briefings, assesses whether linked repositories match a submission, and answers questions using selected project sources.

Best Use of ElevenLabs

ElevenLabs generates project introductions, transcribes spoken questions and produces spoken answers for Voice Mentor. It’s part of how users interact with the product, not just a voiceover added to a demo.

I think the interesting part is how the two fit together: Gemma helps make complicated engineering easier to understand, while ElevenLabs gives people another way to explore those explanations.

Using two partner technologies doesn’t automatically make this a winning entry. I hope the work that went into making them useful together stands out: selecting sources carefully, checking evidence quotes, handling uncertainty and making the experience feel conversational.

I started ReadYou because I was tired of struggling through technical explanations just to understand projects I found interesting.

Now I’m having a lot of fun building a tool that helps me do something about it.

Great projects deserve to be understood. ❤️

devchallenge #hf26challenge

First Post! Newbie

Utkarsh Mishra — Thu, 08 Oct 2026 20:30:14 +0000

Hey there! Another newbie into the picture of open source , hope every established ones here are willing to help me out understanding open source better!