DEV Community: Johnny Santamaria

🎃 Death Exhibition + Extension Roadmap

Johnny Santamaria — Sat, 31 Oct 2026 21:57:19 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

🕯️ Museum of the Dead

A memorial for the ghosts in the machine — built with open-weight AI, offline.

I built a terminal museum of Unix spirits. You walk through four exhibits — a zombie, a daemon, an ofrenda for dead projects, and a séance that channels a dead programming language — and every one of them is narrated by an open-weight language model running entirely on your machine. No API key. No internet. No data leaves the laptop.

The whole thing is also a hidden design-pattern museum: every exhibit is a working example of one of the 23 patterns from Design Patterns: Elements of Reusable Object-Oriented Software (Gamma, Helm, Johnson, Vlissides, 1994).

GitHub: johnny603/museum_of_the_dead

What I Built

The Museum of the Dead is a terminal-based interactive museum. It's built on Python 3.10+, rich for terminal rendering, and Ollama (or llama-cpp-python) for local inference against an open-weight model like Phi-3-mini.

Four exhibits, each with a "describe" phase (a rich panel that sets the mood) and a "perform" phase (a real Unix or data operation, narrated live by the model):

🕯️ The Ofrenda — a memorial shelf of open-source projects that died. Each row is a RepoMemento capturing a project's name, last commit, and cause of death. The local model writes a one-line remembrance for each.
🧟 The Zombie — a real os.fork() child is created, exits, and becomes DEFUNCT in ps until the parent reaps it via os.waitpid(). On screen you watch the exit code come back (exit=7 — the seventh seal). The model eulogizes the reaping.
👁️ The Watcher — a fork() + setsid() daemon that detaches from the TTY and outlives its parent. The model describes the loneliness of a process that closed its own doors.
🔮 The Séance — a chatbot that channels a long-dead programming language (FORTRAN 77, COBOL, Brainfuck, Turbo Pascal). Pick a spirit, ask a question, the model answers in-character.

How I interpreted "Ghost in the Machine": the ghosts are literal. Python's fork() is the closest thing in modern software to a séance — you split a soul into two, one of them lives on unseen, and the other leaves a corpse the kernel refuses to bury until someone wait()s. Naming the exhibits after Día de los Muertos artifacts (ofrenda, marigolds, candles) gave the technical demos an emotional frame: a museum isn't a technical artifact, it's a place where memory is preserved.

Demo

Run locally in three commands:

python3 -m venv .venv && source .venv/bin/activate
pip install rich
python main.py --offline

The offline mode requires zero network access — it uses canned, deterministic whispers instead of an LLM. That's the CI-safe, demo-safe path and the one judges can verify in 10 seconds.

For the full experience:

brew install ollama
ollama pull phi3:mini
pip install ollama
python main.py --model phi3:mini

Here's what an offline run looked like on my MacBook Air (M2, macOS Sonoma):

   __  ___           __                __   ____         __     _
  /  |/  /_ __ ___  / /_  ____  __ __ / /_ / __/__  ___ / /_   (_)__  ___
 / /|_/ / // (_-< / __/ / __/ / // // __// _// _ \/ -_) __/  / / _ \/ _ \
/_/  /_/\_,_/___/ \__/  \___/ \_,_/ \__//_/  \___/\__/\__/  /_/\___/_//_/

╭───────────────────────────────────────────╮
│ A memorial for the ghosts in the machine. │
│ Open-weight AI. Offline. Yours.           │
╰───────────────────────────────────────────╯
Model: phi-3-mini  Offline: True

── Exhibit 2 of 4 — The Zombie ──
╭──────────────── 🕯️  Exhibit: The Zombie ────────────────╮
│ A child is born. The parent forgets to wait().          │
│ The child exits, but its exit status is never reaped.   │
│ For a little while, the child still walks — a zombie.   │
╰─────────────────────────────────────────────────────────╯
I was a child process once. My parent forgot to wait() for me.
  · parent pid=7769, child pid=7788
  · child process breathing…
  · child is now a zombie (DEFUNCT in ps)…
  · parent reaps child (exit=7). Peace returns.

── Exhibit 3 of 4 — The Watcher ──
  · daemon spawned (pid=7790)
  · daemon detached from TTY. It watches, unseen.

Here is a terminal example of how an exhibit should look like:

→ Banner / model line: shows the model name and whether the offline fallback triggered.
→ child is now a zombie (DEFUNCT in ps): this is the real Unix state, captured live. If you run ps -o pid,stat,cmd | grep [p]ython in a second terminal at that instant, you'll see Z for ~200 ms.
→ parent reaps child (exit=7): this is the os.waitpid() return path — the zombie ceasing to exist.
→ daemon detached from TTY: os.setsid() succeeded, and the child is now parentless by design.

Code

GitHub: johnny603/museum_of_the_dead

Project structure — 10 authored files, one design pattern per exhibit:

museum_of_the_dead/
├── main.py                  # entry point + argparse
├── curator.py               # Facade (185) + Builder (97) + Iterator (257)
├── llm_engine.py            # Strategy (315) — swappable LLM backends
├── exhibits/
│   ├── __init__.py          # re-exports Exhibit
│   ├── base.py              # Template Method (325) base class
│   ├── ofrenda_exhibit.py   # Memento (283)
│   ├── zombie_exhibit.py    # Observer (293)
│   ├── daemon_exhibit.py    # Singleton (127)
│   └── seance_exhibit.py    # Strategy (315)
├── README.md
└── LICENSE

Design Patterns used (Gamma et al., 1994) — the book was the floor plan, not a decoration:

Pattern	Book page	Where in the project	Why
Facade	185	`curator.py`	One `visit()` over four exhibits — clients never touch subsystems
Builder	97	`curator.py`	The tour is assembled step-by-step at construction time
Iterator	257	`curator.py`	`for exhibit in self.exhibits` — hides the internal list
Observer	293	`zombie_exhibit.py`	Parent observes the child's exit status via `waitpid()`
Singleton	127	`daemon_exhibit.py`	One Watcher per system — the daemon is the only one
Memento	283	`ofrenda_exhibit.py`	Immutable `RepoMemento` snapshots of dead projects
Strategy	315	`llm_engine.py`, `seance_exhibit.py`	Swap models; swap which language the séance channels
Template Method	325	`base.py`	`Exhibit.describe()` / `Exhibit.perform()` hooks

How I Built It

The open-source AI stack:

Layer	Choice	Purpose
Model runtime	Ollama (primary) / `llama-cpp-python` (alternative)	Local inference against open-weight models — no cloud API
Model	Phi-3-mini (or any Ollama-tagged model)	Narrates each exhibit in real time, entirely on-device
Terminal rendering	`rich`	Panels, tables, prompts, the whole museum aesthetic
OS ghosts	Python `os.fork()`, `os.waitpid()`, `os.setsid()`	The exhibits themselves — real Unix process behavior
Fallback	`llm_engine.py` strategy	If no model is loaded, deterministic canned whispers kick in so the museum always runs

Why the open pieces are load-bearing, not decorative:

llm_engine.py implements a Strategy with three backends (llama-cpp → ollama → canned). Swapping which ghost narrates is a one-line change.
The museum runs without any model at all — --offline is a first-class mode. That's the "works with the lights off" claim, proven on your machine in one command.
No telemetry, no outbound traffic, no auth flow. Every prompt is processed by a model running on the same CPU that's printing the panels.

CTI Open Source Blog Template — system diagram (for the readers who want the deep dive):

┌───────────────┐
│  main.py      │──args──▶┐
└───────────────┘         │
                          ▼
                 ┌───────────────────┐
                 │  LocalModel       │  ◀── Strategy (315)
                 │  (llm_engine.py)  │
                 │  ┌─────────────┐  │
                 │  │llama-cpp    │  │
                 │  │ollama       │  │
                 │  │canned       │  │
                 │  └─────────────┘  │
                 └───────────────────┘
                          ▲ whisper()
                          │
                 ┌────────┴──────────┐
                 │  Curator          │  ◀── Facade (185)
                 │  (curator.py)     │
                 └────────┬──────────┘
                          │ iterates
          ┌───────────────┼───────────────┬──────────────┐
          ▼               ▼               ▼              ▼
   ┌────────────┐  ┌────────────┐  ┌────────────┐  ┌────────────┐
   │ Ofrenda    │  │ Zombie     │  │ Watcher    │  │ Séance     │
   │ Memento    │  │ Observer   │  │ Singleton  │  │ Strategy   │
   │ (283)      │  │ (293)      │  │ (127)      │  │ (315)      │
   └────────────┘  └────────────┘  └────────────┘  └────────────┘

One specific use case — the Zombie exhibit, step by step:

User action: python main.py --offline → Curator.visit() iterates to exhibit 2.
ZombieExhibit.describe() prints a rich Panel and calls self.model.whisper(...). In offline mode this hits llm_engine.CANNED["zombie"]; with a real model it hits the strategy backend.
ZombieExhibit.perform() calls os.fork(). The child process sleeps 0.4 s and calls os._exit(7). Meanwhile, the parent prints parent pid=…, child pid=….
The parent sleeps 0.6 s — during this window, the child is a zombie (STAT = Z in ps). The parent prints child is now a zombie (DEFUNCT in ps)….
The parent calls os.waitpid(pid, 0), extracts exit_code = 7, prints parent reaps child (exit=7). The zombie ceases to exist.
The parent asks the model for a closing whisper and returns.

That flow is 8 lines of actual code in zombie_exhibit.py — but it's a complete, self-contained demo of a real OS concept, narrated by a locally-running LLM.

Why Does Open Innovation Matter?

Three concrete reasons, each tied to something the museum actually does:

1. The museum works with the lights off.

A memorial that requires an internet connection isn't a memorial — it's a subscription. If the museum relied on a hosted API, you'd be reading a eulogy for a dead project served by a server that could die too. --offline isn't a fallback; it's the primary design goal. A museum you can visit on a plane is a museum whose ghosts are actually yours.

2. Data stays home.

The Ofrenda exhibit lets you add RepoMementos for projects you remember — a family project, a startup that died, a tool someone you loved built. With a closed API, every one of those remembrances would leave your laptop, get tokenized, and enter someone's training corpus. With a local model, they don't. Grief deserves privacy.

3. Models are swappable — and that's a load-bearing feature.

llm_engine.py exposes a LocalModel strategy that tries llama-cpp → Ollama → canned whispers. A user who doesn't trust Meta's license can swap in Qwen. A user who wants a smaller memory footprint can use TinyLlama. A user with no model at all can still run the museum. A closed API forces you to trust a provider; an open model lets you trust yourself.

My Agent Session

(Optional — but judges love it.)

The museum was built interactively. Every exhibit was developed through the python main.py --offline loop:

Paste the exhibit skeleton → run offline.
Watch the rich output render → adjust colors and copy.
Wire in the LLM path only after the offline path was stable.
Capture the zombie reaping with a second terminal running ps — the "money screenshot."

(If you save a session with DevRelay, embed it here with the agent_session tag. Otherwise link the repo's Issues tab and leave it at that.)

The Spirit of It

The moment it landed for me was during development, not deployment. I was re-running the Zombie exhibit for the dozenth time when I noticed something in ps:

  PID  PPID STAT COMMAND
 7769  7592 S+   python main.py --offline
 7788  7769 Z+   python <defunct>

The Z+ was my code. My child process. Dead but not reaped. The kernel kept its exit status alive because the parent hadn't called waitpid() yet.

I was staring at a literal ghost — a dead process the operating system refused to forget. And the model whispered "I was a child process once. My parent forgot to wait() for me. I stand here, hollow, until someone reaps my exit status."

For a second, the LLM wasn't generating text. It was translating. The ghost was real.

That's the moment the museum became a memorial instead of a demo.

Acknowledgment

Built entirely offline in a time crunch. 10 files. 8 design patterns. 4 exhibits. 0 API keys. The Design Patterns book was the floor plan; Phi-3-mini was the sexton.

Thanks to the Hacktoberfest 2026 organizers and the Open-Source AI Challenge for asking a question that's genuinely hard to answer without open tools: what does the machine remember?


---

# PART 2 — Extension Roadmap ("When I Have Extra Time")

The judges will respect a project that knows what it *isn't yet*. Here's a **prioritized** roadmap you can paste into the post (as a final "What's Next" section) or save for a follow-up post. Every item is tied to a pattern from the book, so extensions strengthen the design-pattern theme too.

## 🥇 Priority 1 — Genuinely changes the museum (do these next)

| # | Extension | Effort | Why it matters | Pattern it strengthens |
|---|---|---|---|---|
| 1 | **`--exhibit N` flag** — jump straight to one exhibit, skip the tour | 30 min | Essential for demos and follow-up screenshots. Right now every run is 2 minutes even if you only want the Zombie. | Iterator (257) — you already have the loop; just add a filter |
| 2 | **Ollama model auto-detect + fallback** — if `--model` isn't pulled yet, offer to `ollama pull` it (or fail with a friendly message) | 1 hr | The current failure mode is silent fallback. Users should *know* the model isn't live. | Strategy (315) — the strategy chooses itself |
| 3 | **`--size` flag** for exhibits — `--zombie-size small` / `large`. Currently the zombie only lives for 0.4 s; make it 2 s so `ps` catches it reliably | 1 hr | Makes the "watch it die" moment reproducible in a video, not a lottery. | Observer (293) — the observation window becomes a parameter |
| 4 | **Persistent ofrenda** — write `RepoMemento`s to `~/.museum_of_the_dead/ofrenda.json`. `--add-repo NAME LAST_COMMIT CAUSE` appends one; `--reset` wipes | 2 hrs | Turns the Ofrenda from a demo into an actual memorial tool someone would use. **This is the biggest leap in user value.** | Memento (283) — the memento gets serialized, which is the *original* motivation for the pattern |
| 5 | **`--no-fork` safety mode** — Windows and some CI don't support `os.fork()`; detect that and skip the OS exhibits with a graceful message | 2 hrs | Real portability story; Windows users currently get `OSError` at exhibit 2. | N/A (infrastructure) |

## 🥈 Priority 2 — Depth & polish

| # | Extension | Effort | Why it matters | Pattern |
|---|---|---|---|---|
| 6 | **Séance: `--language all`** — cycle through every dead language in one run, showing one whisper per language | 1 hr | Turns the Séance into a *tour de force* of LLM roleplaying. Screenshot-per-language is devastating for a DEV post. | Strategy (315) — the strategy becomes a collection |
| 7 | **Ofrenda: `--repos FILE`** — load a whole memorial list from YAML/JSON | 1 hr | People can maintain an ofrenda file in their dotfiles. Memorable and shareable. | Memento (283) — a `Collection<RepoMemento>` |
| 8 | **`--export-tour`** — dump the full tour to a plain-text `tour_YYYYMMDD.txt` with ANSI stripped | 2 hrs | Lets you save a museum visit, paste it in Slack, or attach it to a blog post. | Iterator (257) — the iterator writes instead of printing |
| 9 | **`rich` progress ring** for the Whisper — currently the model appears to "hang" during inference. A spinner (`rich.status`) turns dead air into atmosphere. | 2 hrs | UX feels 10× more polished for 20 lines of code. | N/A |
| 10 | **Better canned whispers per exhibit** — right now `CANNED["zombie"]` is reused for the describe *and* the perform whisper. Give each a distinct string. | 30 min | Fixes the "repeated line" you saw in the demo output. | N/A |

## 🥉 Priority 3 — Ambitious (a follow-up post's worth of work)

| # | Extension | Effort | Why it matters | Pattern |
|---|---|---|---|---|
| 11 | **Add a fifth exhibit: The Ofrenda of Ghosts** — an interactive memorial where the user types a name, cause of death, and one favorite memory, and the model generates an *epitaph* (short, poetic, FORTRAN-77-broadsheet style). Writes to the persistent ofrenda. | 1 day | This is the single most emotionally resonant feature possible and directly answers the "Ghost in the Machine" prompt. | Memento + Builder |
| 12 | **`--theme` flag** — `día_de_los_muertos` (current), `victorian_gothic`, `cyberpunk`. Swap the `rich` styles and the model's system prompt. | 1 day | Showcases the model's stylistic range without changing a single line of logic. Very demo-able. | Strategy (315) — the *whole tour* becomes a strategy |
| 13 | **Museum web UI** — a `textual` TUI (terminal GUI) or a minimal Flask/HTMX frontend where you can *scroll* through exhibits | 2 days | Splits the terminal-only audience open; makes screenshots shareable. | Still Facade at the top |
| 14 | **`--replay`** — record an `--offline` run to JSON (each whisper + timestamp + exhibit) and replay it identically. Essential for asciinema recordings. | 1 day | Reproducibility for demos. | Memento (283) — the whole tour becomes a memento |
| 15 | **Agentic setup wizard** — a `--setup` command that checks for Ollama, offers to install it, picks a model based on available RAM, and verifies with a test whisper | 2 days | Lowers the barrier to a first successful run from "read the README" to "one command." | Builder (97) |

## What to say in the post about this

If you want to embed a shortened version in the DEV post itself (right before the "Prize Categories" section), here's a paste-ready 4-line version:

markdown

What's Next

The submission is deliberately scoped to 10 files so it can be read in one sitting.
The next iteration — already sketched in the README roadmap — would:

Persist the ofrenda to ~/.museum_of_the_dead/ofrenda.json with an --add-repo subcommand, turning the memorial from a demo into an actual tool.
Add --exhibit N for single-exhibit demos and video recordings.
Detect missing models gracefully (auto-prompt ollama pull instead of silent fallback).
Add a fifth exhibit — an interactive epitaph generator that writes to the persistent ofrenda.

Each of these maps to a design pattern already used in the codebase — Memento for persistence,
Iterator for exhibit selection, Strategy for model selection. The museum's architecture is
already ready for them.

Well that is all the projects for hacktoberfest, see you in the next post, a grand summary explaining what I got out of this event.

SlothMode: Building a Deliberately "Lazy" Focus Enforcer with Open-Source AI

Johnny Santamaria — Sun, 25 Oct 2026 18:42:18 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

A note on process: I designed the architecture first — diagrams, pattern assignments, class relationships — and then used AI to accelerate implementation against that blueprint. The design pattern framework for this post comes from Design Patterns: Elements of Reusable Object-Oriented Software (Gamma, Helm, Johnson, Vlissides), referenced at the end.

Section 1: About the Project

What is SlothMode?

I was sleepy and wanted to be congratulated for it, we all do don't we? But an apple fell on my head.

SlothMode is a desktop productivity application that enforces deliberate rest. Instead of nagging you to work harder, it schedules mandatory breaks pulled from your calendar, then locks your screen with a fullscreen countdown when it's time to step away. Points earned for taking breaks unlock cosmetic rewards (a sloth hat, a hammock, ambient soundscapes). It's a focus tool built around the idea that rest is part of the work.

The project lives at github.com/johnny603/SlothMode.

Why is it important?

Most productivity software optimizes for output. SlothMode inverts that: it treats recovery as a first-class metric. The "SlothMode" concept — everything is farther away, stairs are steeper, the bus leaves earlier — is a gentle joke about aging and exhaustion, but it's also the design philosophy. The app makes slowing down rewarding rather than punishing.

Who is the typical user?

A knowledge worker or student who schedules focus blocks in a calendar (Google Calendar, CalDAV, or a local ICS file) and wants their break time protected. Example scenario: You block 2:00–2:15 PM as "Break" in your calendar. At 2:00, SlothMode reads that event through its hybrid calendar source, transitions to LockoutState, and displays a fullscreen countdown. You can't dismiss it until the timer runs out — but when you do, you earn points toward a cosmetic reward.

Screenshot

(Insert annotated screenshot of the running app here — I'll describe it for the template: an arrow pointing to the fullscreen CountdownRing overlay, a second arrow to the SlothHammock reward badge in the corner, and a third to the system tray icon showing "Focus" state.)

Users

The project is an early-stage open-source repo (205 files, 15 test suites passing as of the last build), so user count is effectively the contributor base. It's a framework/toolkit more than a deployed product — designed to be extended via its SlothPlugin interface.

Section 2: The Issue

What issue did I address?

(Link to your GitHub issue here.) The issue: SlothMode only supported a single hard-coded break source, making it unusable for users who track breaks differently (some use CalDAV, some use local ICS files, some manually add breaks). The app needed a way to compose multiple break sources without the lockout engine knowing which one it was talking to.

Why is this important?

The lockout scheduler is the heart of the app. If adding a new break source requires modifying SessionHub, LockoutState, and BreakRepository every time, the project becomes closed to contribution. Decoupling break sources from the scheduling engine is the difference between a demo and a tool.

Key code locations

File / Folder	Purpose
`src/main/java/io/slothmode/calendar/`	All break-source implementations: `CalDAVSource`, `CalendarSource`, `HybridSource`, `ManualSource`, `BreakSourceFactory`
`calendar/BreakRepository.java`	Aggregates breaks from whatever source is configured
`calendar/AddBreakCommand.java`	Command-pattern wrapper for adding a break (undoable)
`session/SessionHub.java`	Orchestrates state transitions (`IdleState` → `FocusState` → `LockoutState` → `RewardState`)
`session/SessionObserver.java`, `SessionSubject.java`	Observer wiring so the UI reacts to state changes
`lockout/LockoutAbstraction.java`, `LockoutImplementor.java`	Bridge pattern separating "how to lock" from "what triggers a lock"

Supporting artifacts

(Insert screenshot of a NullPointerException or empty-break-list log from before the fix — e.g., HybridSource returning an empty list when one source is down.)

Section 3: Codebase Overview

Tech stack

Technology	Purpose
Java 17 + Maven (`pom.xml`, `mvnw`)	Core language and build system; the app is a desktop Swing/JavaFX client
JavaFX (`ui/MainWindow.java`, `resources/ui/main.css`)	UI layer; the `CountdownRing` and fullscreen lockout render here
Logback (`resources/logback.xml`)	Structured logging for session transitions and lockout events
JUnit + Surefire	15 test classes covering calendar, command, config, lockout, model, plugin, and session packages
Ollama / llama.cpp (`model/OllamaModelFactory.java`, `LlamaCppModelFactory.java`, `MockModel.java`)	Local LLM inference for the `SessionSummaryAgent` — summarizes what you did during a focus session, entirely offline
CalDAV / ICS	Break ingestion from external calendars

System diagram

[Calendar: CalDAV | ICS | Manual]
        │
        ▼
 ┌─────────────────┐      ┌──────────────────┐
 │ BreakSource     │◄─────│ BreakSourceFactory│
 │ (HybridSource)  │      └──────────────────┘
 └────────┬────────┘
          ▼
 ┌─────────────────┐
 │ BreakRepository │──► AddBreakCommand (undoable)
 └────────┬────────┘
          ▼
 ┌─────────────────┐   Observer    ┌──────────────┐
 │   SessionHub    │──────────────►│   MainWindow  │
 │ Idle→Focus→     │               │  (Countdown)  │
 │ Lockout→Reward  │               └──────────────┘
 └────────┬────────┘
          ▼
 ┌─────────────────┐      ┌──────────────────┐
 │LockoutAbstraction│────►│LockoutImplementor│
 │ (Bridge)         │     │Fullscreen/Nudge  │
 └─────────────────┘      └──────────────────┘
          │
          ▼
 ┌─────────────────┐      ┌──────────────────┐
 │  RewardState    │─────►│ RewardDecorator  │
 │                 │      │ SlothHat/Hammock │
 └─────────────────┘      └──────────────────┘

(Numbered components 1–7 correspond to the workflow below.)

Workflow: the 2:00 PM break

Calendar ingestion — HybridSource (the BreakSource implementation configured via BreakSourceFactory) queries CalDAV and the local ICS file, merges results.
Repository — BreakRepository receives the merged Break objects.
SessionHub observes the clock; at 2:00 it calls SessionState.transition(LockoutState).
Observer notification — SessionSubject.notifyObservers() fires; MainWindow renders the CountdownRing overlay.
Lockout — LockoutAbstraction delegates to its LockoutImplementor (FullscreenLockout for aggressive mode, GentleNudgeLockout for a softer prompt).
Reward — When the timer completes, RewardState awards points via FixedPointsStrategy / StreakPointsStrategy and decorates the reward bundle with SlothHat.
Summary — SessionSummaryAgent (backed by an Ollama model) generates a one-line summary of the session, all local.

Section 4: Challenges

The challenge

Designing a break-source abstraction that composes cleanly. The naïve approach — if (source == CALDAV) ... else if (source == ICS) ... — would have leaked into SessionHub and made the lockout logic untestable. I needed HybridSource to be able to wrap any two BreakSource implementations and behave identically to a single source.

Three self-directed attempts

Conditional compilation — I tried a single BreakSource class with a SourceType enum switch. It passed tests but violated the project's existing pattern vocabulary and made BreakRepositoryTest brittle.
Subclass explosion — I tried CalDAVOnlySource, IcsOnlySource, HybridCalDAVIcsSource. This worked but produced 6+ classes and doubled the test surface.
Adapter + composition — I mapped the problem onto the Adapter pattern (from the GoF book): HybridSource implements BreakSource, holds two references typed as BreakSource, and delegates. Existing sources needed no changes.

Who I reached out to

I consulted the project's own docs/dev-setup.md and the lockout-manual-test.md to understand the expected runtime behavior, then used AI (feeding it the architecture diagram and the relevant GoF pattern definitions) to validate the Adapter mapping against the existing BreakSourceFactory contract.

Status

Resolved. The pattern is in place and HybridSourceTest passes alongside CalDAVSourceTest and CalendarSourceTest.

Section 5: Solution

What I did

I implemented HybridSource as a composition of two BreakSource instances, both typed against the abstract interface. It polls both sources, merges their Break lists, and returns a single de-duplicated list. Because HybridSource is a BreakSource, it can be nested arbitrarily (new HybridSource(new HybridSource(a, b), c)) and BreakRepository never needs to know.

Key technologies referenced

Java interface polymorphism — BreakSource (from the calendar package in the tech stack table) defines the contract; HybridSource, CalDAVSource, ManualSource all implement it.
System diagram component 1 — the BreakSource / BreakSourceFactory pair; my solution sits exactly there, as a third BreakSource implementation.

How I verified it

HybridSourceTest (in src/test/java/io/slothmode/calendar/) — asserts that two mock sources produce a merged list, that an empty source doesn't break the merge, and that a failing source degrades gracefully.
Full suite: ./mvnw test → 15 test classes, all green (see target/surefire-reports/).
Manual: ran the app with a CalDAV source and a local ICS file simultaneously; the 2:00 PM break fired once, not twice.

(Insert screenshot of the Surefire test summary here.)

Code snippet

// HybridSource.java — Adapter pattern: composes two BreakSources
public class HybridSource implements BreakSource {
    private final BreakSource primary;
    private final BreakSource secondary;

    public HybridSource(BreakSource primary, BreakSource secondary) {
        this.primary = primary;
        this.secondary = secondary;
    }

    @Override
    public List<Break> fetchBreaks() {
        List<Break> merged = new ArrayList<>(primary.fetchBreaks());
        for (Break b : secondary.fetchBreaks()) {
            if (!merged.contains(b)) merged.add(b); // de-dupe
        }
        return merged;
    }
}

Line by line: the constructor accepts any two BreakSource implementations (polymorphism — this is what makes the factory viable). fetchBreaks() delegates to primary, then appends de-duplicated secondary breaks. No instanceof, no enum, no subclass explosion.

Pull request

(Link to your PR here.)

How I Built It

I used open-source AI as an implementation accelerator, not as an architect:

Design first. I produced the system diagram above and explicitly mapped each package to a GoF pattern before writing code. I referenced the Design Patterns textbook for the Adapter, Bridge, Command, Memento, Observer, State, Strategy, and Visitor mappings already present in the codebase (LockoutAbstraction = Bridge, AddBreakCommand = Command, SaveWorkMemento = Memento, SessionObserver = Observer, SessionState = State, BreakStrategy = Strategy, EventVisitor = Visitor).
AI as implementer. I fed the architecture diagram and pattern definitions to an AI assistant, asking for implementations that respected the existing interfaces. The pre-supplied structure prevented the model from inventing new abstractions or confusing the Bridge with the Adapter.
Open-weight models at runtime. SlothMode's own SessionSummaryAgent runs against Ollama or llama.cpp via OllamaModelFactory / LlamaCppModelFactory, with MockModel for tests. Session summaries never leave the machine.

Why Does Open Innovation Matter?

SlothMode's core value proposition — rest without surveillance — is incompatible with a closed API. If the session summarizer phoned home to a hosted LLM, the app would be logging exactly when you're not working. Local inference via open-weight models is a product requirement, not a nice-to-have. The SlothModel interface lets users swap Ollama, llama.cpp, or a mock without touching the session logic — the same open-interface principle as BreakSource. Open source made it possible to build a rest tool that doesn't itself need to watch you.

My Agent Session

(Embed your DevRelay agent_session here, or link it.)

The Lazy Payoff

The entire point of SlothMode is measured laziness: the CountdownRing enforces breaks you'd otherwise skip, StreakPointsStrategy rewards consistency, and the SlothHammock decorator makes rest tangible. In testing, the hybrid calendar source removed the "I forgot to log my break" failure mode entirely — breaks arrive from the calendar you already maintain. (Add a personal anecdote here if you used it this week.)

Prize Categories

(List partner categories you're entering, or remove this section.)

References

Gamma, E., Helm, R., Johnson, R., & Vlissides, J. (1994). Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley. — Source for the Adapter, Bridge, Command, Memento, Observer, State, Strategy, and Visitor patterns used throughout SlothMode's architecture.
SlothMode repository: github.com/johnny603/SlothMode
Hacktoberfest Open-Source AI Challenge Week 3

[Boost]

Johnny Santamaria — Sun, 18 Oct 2026 02:23:58 +0000

Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle Submission 💸

Gabby Gill

Oct 17

Shoebox — a private receipt & GST sorter for Alberta freelancers, powered by a local Gemma 3

#devchallenge #hf26challenge #dotnet #opensource

7 min read

CreatorDesk: a self-hosted approval loop for commission artists

Johnny Santamaria — Sun, 18 Oct 2026 02:18:55 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

CreatorDesk is a self-hosted, open-source tool that solves one specific problem for independent digital artists: tracking which version of a commission a client approved, and why.

An artist uploads a work-in-progress (a sketch, a lineart, a colored render), sends the client a private link, and the client clicks Approve or Request changes with optional written feedback. Every revision and every decision is recorded with a timestamp. The client never creates an account. The artist never loses the conversation to a DM thread.

Who would pay for it: a working commission artist who does 4–15 paid commissions a month, whose income depends on delivering the right version, and who currently tracks client feedback in Discord DMs, Trello boards, and a spreadsheet. CreatorDesk replaces that with one page the client can actually use and one history the artist can actually point to when scope creep happens.

The problem it solves is narrow on purpose: the approval loop, not the whole business. Not a marketplace, not a portfolio host, not a CRM, not payments. The wedge is the moment between "here is the sketch" and "yes, that's right" — because that's where commission failures happen, and where artists have the least infrastructure.

Demo

The app runs on localhost:3000 after four commands:

git clone https://github.com/johnny603/CreatorDesk
cd CreatorDesk && npm install
npm run db:reset     # applies migrations, seeds a demo artist
npm run dev

A full loop — create commission → upload sketch → send link → client requests change → upload v2 → client approves → mark delivered — takes about two minutes and is documented step-by-step in the README.

Screenshot: artist dashboard, mid-demo.

CreatorDesk
+ New commission

Title              Client   Status                      Updated
● Test commission  Alex     needs your next revision    2026-10-18T01:26:48Z
  Test 2           Joe      waiting on client           2026-10-18T01:25:40Z
  Test 3           Sam      approved — ready to deliver 2026-10-18T01:20:49Z

The dot (●) marks commissions where the artist has a next action — in_progress (client requested changes) and approved (client signed off, ready to deliver). The dashboard sorts by actionability, not recency, so the answer to "what do I do next?" is the top row.

Screenshot: client approval page.

Test commission
Version 2
[image of the sketch]

"hair updated — changed the shape per your note"

Feedback (optional): [textarea]

[Approve]   [Request changes]

▸ Revision history (2 versions)

The <details> expansion shows v1 ("you asked for changes: 'make the hair blue' (2026-10-18T01:20:33Z)") and v2 ("currently reviewing"). The client can see what they asked for. The artist can point to it later.

Code

GitHub: github.com/johnny603/CreatorDesk — tagged v0.1.1, MIT licensed, 32 passing tests.

How I Built It

The stack is deliberately boring:

Layer	Technology	Purpose
App framework	Next.js 14 (App Router) + TypeScript	Server-rendered routes; server actions for mutations
UI	React, minimal client JS	Forms work without hydration where possible
Database	SQLite via `better-sqlite3`	Single-file state; the whole app is `data/`
Data access	Raw SQL in a small module	No ORM; every query is visible
Uploads	Local filesystem, sha256-named	Duplicate uploads are free; no cloud storage
Client auth	Random 256-bit token, sha256-stored	The token in the URL is the entire client session
Artist session	Single-user stub (`session.ts`)	Real auth is deferred until validation

Where the open pieces matter — and where they are the reason the project works:

The case for open-source here is not "we used an LLM." It is the same case the challenge asks about: does the open approach let you do something a closed one can't?

Three specific answers, all structural:

Data control. A commission artist's work-in-progress is often under NDA, embargo, or just the artist's own embargo until the client has paid. Sending those images through a third-party SaaS means the artist is trusting a company's retention policy with their client's intellectual property. CreatorDesk is self-hosted: data/creatordesk.sqlite and data/uploads/ are on the artist's own machine, and the README documents exactly how to back them up. There is no server the artist doesn't control.
Cost. A self-hosted SQLite app costs $0/mo to run on a $5 VPS or an old Mac Mini. HoneyBook is ~$29/mo; Dubsado is more. For a part-time artist doing 3 commissions a month, the closed tools' pricing is the single most cited reason for not adopting them. The open tool's cost is the price of the machine it runs on, which the artist already owns.
Portability. The schema is a plain .sql file. If CreatorDesk is abandoned, the artist can sqlite3 data/creatordesk.sqlite and export every commission, every revision's file path, every decision, every feedback note. The data is not trapped behind an API the vendor can deprecate.

The AI piece — what it actually is, and what it is not:

This is not an "AI features" project. CreatorDesk does not call an LLM at runtime, and the artist is not writing prompts. The role of open AI tooling is in the build, not the runtime: the project was planned, reviewed, and debugged in Claude Code over a multi-day session, with open-source design patterns (Facade applied lightly, State as an explicit transition table, everything else deferred) as the reference. The repository's commit history and this blog post's companion agent session are the record of that.

That distinction matters for the challenge's framing. The challenge asks for a project where open pieces are what makes it work — not for every project to ship a model. Here, the open-source ethos (self-host, portability, no vendor lock-in) is what makes the product's value proposition possible in the first place. A closed-API version of this tool would defeat its own pitch.

The system diagram:

┌──────────────────────┐         ┌────────────────────────┐
│ Client (browser)     │         │ Artist (browser)       │
│ no account           │         │ single-user session    │
└──────────┬───────────┘         └───────────┬────────────┘
           │ token in URL                    │ cookie
           ▼                                 ▼
┌──────────────────────────────────────────────────────────┐
│ Next.js routes (src/app)                                 │
│  /a/[token]              client approval page            │
│  /a/[token]/decision     POST approve / request changes  │
│  /c/[commissionId]       artist detail + upload          │
│  /c/[commissionId]/actions server actions                │
│  /api/commissions        create commission               │
│  /api/files/[filename]   file serving, token-scoped      │
└──────────┬───────────────────────────────────────────────┘
           │
           ▼
┌──────────────────────────────────────────────────────────┐
│ lib (src/lib)                                            │
│  commissions.ts    workflow rules + token lookup         │
│  revisions.ts      the atomic decision claim             │
│  approval-tokens.ts  security boundary                   │
│  uploads.ts        sha256-named file storage             │
│  db.ts             connection + transaction wrapper      │
└──────────┬───────────────────────────────────────────────┘
           │
           ▼
┌──────────────────────────────────────────────────────────┐
│ data/                                                    │
│  creatordesk.sqlite (artists, commissions, revisions)    │
│  uploads/<sha256>.<ext>                                  │
└──────────────────────────────────────────────────────────┘

The workflow, start to finish — the client approval loop:

Artist creates a commission (/c/new) → POSTs to /api/commissions. commissions.create() generates a 256-bit random token, stores sha256(token) and a 30-day expiry, and returns the raw token once. The artist sees a "Commission created" page with a copy button.
Artist uploads a revision on /c/[commissionId]. revisions.submitRevision() inserts a row with version_number = 1, decision = 'pending', and updates the commission to awaiting_approval, in a single transaction.
Artist sends the link through their own channel (email, DM, anywhere). CreatorDesk does not send email.
Client opens /a/[token]. approval-tokens.hashToken(token) + commissions.findByTokenHash(hash) resolves the token to exactly one commission (or returns undefined if expired). If the latest revision is pending, the page renders the image, the artist's note, two buttons, and a collapsed revision history. If not, it renders a status-specific message ("The artist is preparing your first revision" / "You approved the latest version" / etc.).
Client clicks Approve. POST to /a/[token]/decision. revisions.recordDecision(revisionId, decision, feedback) runs a single SQL transaction:
- Claims the revision: UPDATE revisions SET decision = ? WHERE id = ? AND decision = 'pending' AND version_number = (SELECT MAX(version_number) ...) AND EXISTS (SELECT 1 FROM commissions WHERE ... status = 'awaiting_approval')
- If zero rows updated: classifies the failure (already_decided / not_latest / wrong_state / not_found) and returns it to the route, which returns a specific message.
- If one row updated: also updates the commission's status to approved or in_progress, inside the same transaction.
Client sees "Thanks — your feedback is recorded" (/a/[token]?thanks=1). The artist's dashboard now shows the decision.
Artist uploads v2 (or clicks Mark delivered if v2 was approved).
The history is retained. Every version, decision, and feedback note stays in the revisions table forever. The client page shows it collapsed; the artist page shows it fully.

The transaction at step 5 is the load-bearing part. All four race scenarios the project tests (already_decided, not_latest, wrong_state, not_found) are about what happens when two of those steps overlap — a stale tab, a duplicate click, an artist uploading while the client is deciding. The SQL is one statement, and its WHERE clause is the entire concurrency control.

Why Does Open Innovation Matter?

The three reasons from above, restated because the challenge asks directly:

Client work is confidential by default. A sketch under commission is not public. An artist's client may be a small business commissioning packaging art, an author commissioning a book cover, a family commissioning a portrait. None of them expect their WIP to leave the artist's machine. A self-hosted tool makes that a property, not a promise. A closed-API version of this app would need to explain its retention policy in every client email, which is a conversation neither party wants.

A working artist's tooling budget is near zero. The $29/mo tools are not priced wrong — they're priced for the agency freelancers who are their market. A part-time illustrator doing 3 commissions a month cannot amortize a $29/mo subscription over a $150 commission. The self-hosted version costs the artist nothing they aren't already paying for. That is not a marketing angle; it is the difference between the tool existing and not existing for that customer.

The schema is the product. migrations/001_init.sql is 58 lines. Anyone can read it in a minute and understand every state a commission can be in. That's the open-source contract: if I disappear, the data is yours, and the shape of it is legible. A closed SaaS's schema is invisible; a self-hosted OSS schema is a .sql file in a git repo. For a tool holding a person's client history, legibility is a feature.

My Agent Session

The build was done in Claude Code across several sessions. The agent session includes: product direction research, the four design review rounds on the transaction contract, the two test-suite bug fixes (classifyFailure ordering and the submitRevision guard), the two next build type errors (BodyInit on Buffer, the DecisionResult conditional-type collapse), and the UX round that added the thanks page and contextual "nothing to review" messages.

The pattern worth naming from that session: every design-level problem was caught in review, and every implementation bug was caught by npm test or npm run build, not review. That's a useful distinction for anyone considering this workflow — review catches design, tests catch code, and neither substitutes for the other.

(Agent session embed/link per challenge instructions.)

My First User

The honest state: I tested the full loop myself end-to-end on v0.1.1 — create commission → upload → approve → request changes → upload v2 → approve → mark delivered — with both single and bulk delete verified. That's not a real first user; it's the demo working.

What I did not do, and what the project's own research plan gates: five interviews with working commission artists before building more. The README's "What's next" section says this explicitly:

The next milestone is user validation, not more features. Five interviews with working commission artists, then the answer to: "is the approval loop actually painful enough that artists will switch for it?"

The first real user will come from that round. The tool is ready for them: git clone, four commands, a two-minute demo loop, a README that documents what it does and does not do. The scope is deliberately bounded so that the first user's feedback goes to the right place — the next feature, not a bug report about the demo.

What a contributor, mentor, or early user can help with right now:

Read src/lib/revisions.ts and tell me whether the transaction claim reads correctly to you. If you have written a conditional UPDATE … WHERE … before, you will spot a problem faster than I will.

Run the demo on your machine and tell me where you got stuck. The README assumes Node 20, better-sqlite3 prebuilds, and a browser. Every deviation from that assumption is a bug report I can't write myself.

If you are a commission artist: the 15-minute interview script is in the README. If you are not, a referral to one is as useful as a review.

The tool works. What I need next is not more code — it is the data, the review, and the restoration test that tell me whether the thing I built is solving the problem I think it is. That's the honest state of the project, and it's what the next milestone is for.

Footnote

Open Source (primary). This is a self-hosted, MIT-licensed project where the open-source property — data ownership, legibility, cost — is the product's value proposition, not an incidental choice. No partner category claimed.

MudMap: A Voice-First Trail Logger That Works Without a Signal

Johnny Santamaria — Sun, 11 Oct 2026 19:47:27 +0000

MudMap: A Voice-First Trail Logger That Works Without a Signal

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

MudMap is an offline voice trail logger. You press Record, talk while walking — "Saw a heron, trail washed out past the creek, blackberries ripe" — and the app transcribes it on-device, extracts species and conditions, and drops a pin on a map you own. No cloud. No account. No signal required.

It exists to make the screen the shortest part of a hike. You don't take out your phone to type notes. You don't open an app to browse a feed. You walk, you talk, you put the phone away. Everything else happens after.

The typical user is someone who already walks outdoors and wants a record of what they saw — birders, foragers, trail runners, gardeners — without a subscription app that uploads their favorite fishing hole to a server they don't control.

Demo

The build produces a working debug APK. It builds cleanly in CI, unit tests pass, and the artifact installs on any modern Android phone (arm64-v8a).

Current state, honestly stated: the software pipeline is proven, but hardware verification on real trails is still pending. That's the next step, and it's why I'm looking for Android testers.

Known limit to flag upfront: the bundled basemap is a San Francisco demo extract only (longitude -122.53 to -122.35, latitude 37.70 to 37.83). Outside SF, pins appear on a blank basemap — your entries still save, you just won't see a basemap under them until you prepare a replacement region.

Code

The repository is at https://github.com/johnny603/MudMap.

A build is produced on every push to main via GitHub Actions, and tagged releases produce a permanent download URL.

How I Built It

MudMap is built on four open-source pieces, each load-bearing:

Layer	Open-source component	What it does
Speech-to-text	whisper.cpp v1.9.5	Transcribes the `.m4a` recording on-device
Map rendering	MapLibre Native 12.3.1	Renders the map and pins
Basemap	Protomaps (PMTiles)	Offline vector tiles bundled in the APK
Storage	Room / SQLite	Persists entries locally

The stack is Kotlin + Android Views, minSdk 26, targeting arm64-v8a. The native whisper layer is compiled against the Android NDK 27 and invoked through a small JNI adapter.

The architecture is deliberately small. There's a Transcriber port and a WhisperCppAdapter behind it; an EntityExtractor port with a RuleBasedExtractor behind it. That means the AI pieces are swappable — if a better on-device model drops tomorrow, I change one adapter, not the app.

Walkthrough of one use case:

User presses Record. The app requests one location fix (10-second timeout) and starts capturing .m4a audio to app-private storage.
User presses Stop. The location fix is frozen — if none arrived, the entry saves with null coordinates.
The AAC audio is decoded to 16 kHz mono float PCM — the format whisper.cpp expects.
whisper.cpp transcribes the PCM on-device. No network call.
The RuleBasedExtractor matches the transcript against a bundled vocabulary and produces tags: species, conditions, landmarks.
The six-field entry is saved to Room.
The map reads entries on launch and after saves. A pin opens a transcript-and-tags bottom sheet.

Every step except installing the app happens with the phone in airplane mode.

Challenges

The build itself turned into the hardest part of this project — harder than any of the app logic. Three problems in particular cost real time, and each needed several attempts before it broke.

Challenge 1: The cryptic `25.0.4.1` build failure

The first Gradle run failed with a message that said nothing useful:

FAILURE: Build failed with an exception.

* What went wrong:
25.0.4.1

No file, no line, no explanation. Just a version number.

Attempt 1 — Re-ran the build. Same failure, same 46-second wait.

Attempt 2 — Searched for the error string. 25.0.4.1 returns nothing useful; it's not a documented Gradle or AGP error code.

Attempt 3 — Read the README's pinned versions table carefully. It specified JDK 17. My shell was on a newer JDK, and the "Restricted method" warnings in the build output were the tell. Gradle 8.11.1 and AGP 8.9.2 do not support JDKs past 17 in this configuration — and instead of saying so, they emit a bare version string.

Resolution: forced JAVA_HOME to JDK 17 for the build shell:

export JAVA_HOME=$(/usr/libexec/java_home -v 17)

The build then proceeded to the next failure. I added this diagnosis to the README under Common build failures so the next person doesn't lose the same hour.

Challenge 2: The asset script couldn't find `pmtiles`

bash scripts/prepare-assets.sh failed with:

Install the pmtiles CLI, or set PMTILES to its absolute path.

Attempt 1 — Installed pmtiles via Homebrew. It installed, but the script still failed.

Attempt 2 — Verified the binary. Ran pmtiles --version — which returned unknown flag --version. That's not a bug; the CLI uses the version subcommand. But it made it unclear whether the install had worked at all. which pmtiles and pmtiles --help confirmed it was on PATH.

Attempt 3 — Passed the path explicitly. The README documents an escape hatch, PMTILES=/absolute/path/to/pmtiles, which bypasses the PATH lookup entirely. That worked, and the asset pipeline completed: it downloaded a 4.1 MB extract, verified it in 308µs, and printed Offline assets ready.

The lesson: a tool being "installed" and a script finding it are different problems. The explicit-path override is the reliable fix.

Challenge 3: The disk-space wall

With JDK and pmtiles resolved, the Android SDK install failed:

Warning: An error occurred while preparing SDK package NDK (Side by side)
27.0.12077973: No space left on device.

The NDK is roughly 3–4 GB. The disk had 662 MB free.

Attempt 1 — Homebrew cleanup. brew cleanup --prune=all reclaimed ~1.1 GB. Good, but nowhere near enough — the full Android toolchain plus Gradle plus the whisper.cpp native compile needs roughly 25–30 GB.

Attempt 2 — Cleared Gradle caches and the Homebrew download cache. Another gigabyte or so. Still an order of magnitude short.

Attempt 3 — Moved the build off the machine entirely. Instead of fighting for 30 GB on a MacBook Air that already had 198 GB used, I wrote a GitHub Actions workflow. The runner has its own disk. The build succeeded in 3 minutes 22 seconds, producing a 147 MB debug APK.

Resolution: CI builds the APK; the local machine never needs the Android toolchain. This also became the distribution mechanism — tagged releases now produce a permanent download URL from the same workflow.

What these three have in common

None of them were problems with MudMap's code. All three were environment problems, and all three had error messages that pointed away from the actual cause. That's the ordinary tax of getting an offline native app to build on a machine that wasn't set up for it — and it's why the README now has a Prerequisites section and a Common build failures table.

Why Does Open Innovation Matter?

The open pieces aren't a nice-to-have here — they're the reason the thing works where it's meant to be used.

It works with no signal. Whisper.cpp runs on the phone. There's no server round-trip for transcription. A closed speech API would need a cell tower, and the trail has none.

Your data stays on your phone. Internet permission is removed entirely from the manifest. Cloud backup and device transfer are excluded. Your trail notes are yours. A closed equivalent would store your location history on someone else's disk, where it becomes part of a data set you don't own.

The map is a file, not a subscription. Protomaps ships offline tiles as a .pmtiles archive bundled in the APK. MapLibre reads it locally. No tile-server bill, no API key, no per-request quota. A closed map SDK would meter every pan and zoom.

You can swap the model. The Transcriber and EntityExtractor ports mean the AI backend is replaceable. If a smaller, faster, better model drops — or if you want to fine-tune one on your own observations — you change one adapter. No rewrite.

It costs nothing to run. There is no server component. There is no monthly bill. The APK does everything.

The "open" part is what makes the whole thing possible offline, on a trail, away from infrastructure. That's the point of the project.

Hacktoberfest 2026 is a month-long celebration of open source, and this year it's specifically about building with open-weight models and open-source AI — which is exactly what MudMap is. The event runs all October, online and through 300+ in-person Fests, and anyone can take part regardless of where they are in their open-source journey. For MudMap, my immediate plan is to cut a v1.0.0 release with a permanent download link, then recruit Android testers to verify the software on real trails — microphone capture, GPS acquisition, transcription latency, and battery use. After that, the next slice of work is the deferred list: sync, delete, undo, and LLM-based entity extraction via Ollama. What I've learned building this is that the open pieces aren't decoration — they're the reason the app works where it's meant to be used, and Hacktoberfest is the right place to share that with people who care about the same thing.

CscGrader and generateRubrics: Open-Source Tools for Faster CS Grading

Johnny Santamaria — Sun, 04 Oct 2026 23:26:53 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

I built a two-part toolkit to solve the biggest time-sink in teaching large CS courses: grading massive, complex student projects. This isn't about auto-grading, which often fails for creative or multi-part assignments. Instead, it’s about automating the tedious, mechanical work so instructors can focus on providing meaningful feedback.

I built this for my friend, a CS professor who was spending entire weekends manually compiling, running, and debugging hundreds of student submissions for a single assignment.

The problem is simple: when you have 150 students submitting multi-file Java projects, just getting each one to compile and run with the correct test input is a logistical nightmare. It’s slow, error-prone, and mentally draining.

My solution is a two-part open-source toolkit:

CscGrader: A language-agnostic tool that automates the "collection of evidence" phase. It detects student projects, compiles them, runs them with predefined inputs, and captures all the output (stdout, stderr, exit codes, timing).

generateRubrics: A simple utility that takes a single master rubric file (Excel, PDF, etc.) and generates a personalized, ready-to-grade copy for every student on the roster.

Together, these tools transform a weekend of grunt work into a streamlined, semi-automated workflow, freeing up my friend to do what they do best: teaching.

Demo

The tools are designed for a command-line workflow. Here’s how they would be used together for a single assignment:

Step 1: Generate all the grading rubrics for the class.

bash

studentNames.txt contains a list of all students in the class

python3 generateRubrics.py -inputFile studentNames.txt -fileToCopy "Assignment-03-Rubric.xlsx" --assignment 03
This instantly creates a Rubrics/ folder with perfectly named files like JaneDoe-Assignment-03-Rubric.xlsx.

Step 2: Process all student submissions to collect evidence.

bash

The submissions folder contains one subfolder per student

The assignment profile tells CscGrader what to look for and how to run it

cscgrader batch ./submissions/ --assignment ./csc215-assignment-03.json --input test-input.txt
This processes every submission, compiles the Java code, runs it with the same test input, and saves a structured JSON file with all the results for each student.

Step 3: Grade.

My friend now opens the personalized rubric for a student in one window and the corresponding JSON evidence file in another. All the "did it compile? did it run? what was the output?" questions are already answered. They can now focus on code quality, design, and feedback.

Code

Here are the two repositories:

CscGrader: https://github.com/johnny603/CscGrader

generateRubrics: https://github.com/johnny603/generateRubrics

How I Built It

This project is a prime example of using open-source tools for practical, targeted automation.

Core Logic: Both tools are written in Python, relying only on the standard library. This was a deliberate choice to make them as portable and easy to run as possible for instructors who may not have admin rights or want to deal with virtual environments. The generateRubrics tool treats files as binary data, so it can copy .xlsx rubrics without needing libraries like openpyxl.

AI-Assisted Development: I used an open-weight model as a coding partner throughout the process. It was instrumental in designing the modular architecture of CscGrader, specifically the idea of using "assignment profiles" (JSON files) to make the tool reusable across different courses and languages.

Agent Harness: I used an open-source agent harness to automate the testing of CscGrader. The agent would create dummy Java projects with common errors (e.g., syntax errors, missing dependencies, ClassNotFoundException), run CscGrader on them, and verify that the tool correctly identified the failure category. This saved me hours of manual testing and made the tool far more robust.

The architecture of CscGrader is built around a modular pipeline: Detection → Compilation → Execution → Evidence. This is all orchestrated by a central runner, which uses language-specific "adapters" to handle the unique aspects of each programming language. The agent harness was key to building and validating this complex workflow.

Why Does Open Innovation Matter?

My Agent Session

Prize Categories

Using AI as a Learning Tool: Lessons From the Classroom, STEM, and Software Engineering

Johnny Santamaria — Thu, 01 Oct 2026 04:01:57 +0000

"Hello, I just wanted to check in..."

I did not start thinking seriously about AI from one role. I started seeing it differently as I moved between being a computer science student, teaching STEM, working as an LA/DL, doing educational research, and working at a robotics startup. Each experience showed me a different part of the same problem: how can we use technology to make people better at learning and solving problems without taking the problem-solving away from them?

As an Instructional Student Assistant at San Francisco State University, I work with students on programming logic and software engineering while developing Python automation for academic workflows. As an E-GAISE research assistant, I have analyzed submissions from 34 student teams and contributed to research involving AI-integrated grading systems. Before that, I worked as an Engineering Operator at a robotics startup, where I built Python tools for data ingestion, labeling, and validation.

These experiences led me to my main thesis: AI is most valuable in STEM education when it provides context-aware assistance, creates opportunities for practice and feedback, and keeps humans responsible for reasoning and verification. A student can create a diagram before asking AI for design feedback. An instructor can provide course objectives and common mistakes before using AI to create a Kahoot, Blooket, or debugging game. A developer can provide an existing codebase before asking AI to make a change.

AI Needs Context

One of the most useful ways I see AI being used in a classroom is by giving it context rather than starting with an empty prompt.

For example, imagine that a student is working on a software design assignment. Instead of immediately asking an AI, "How should I build this?", the student could first create a class diagram, flowchart, wireframe, pseudocode outline, or system design.

The student could then give that design to the AI and ask:

Does this design satisfy the requirements?
What relationships am I missing?
Is there unnecessary complexity?
What edge cases should I consider?
What could I improve?
What tradeoffs am I making?

This creates a different learning experience.

The AI now has context about what the student is thinking, and the student has already performed part of the intellectual work. The AI becomes something that can critique and question the student's thinking instead of generating the entire solution.

I think this is especially important in computer science because design is part of the learning process. If a student creates a diagram, writes pseudocode, or explains an algorithm before using AI, there is something concrete to evaluate.

The same principle applies to professional software development. A developer can provide an existing codebase, architecture diagram, API specification, test case, or error log before asking AI for help. The more useful question is not simply, "Can AI write this?" but "What context does AI need to understand what I am trying to accomplish?"

AI and Gamified Learning

Another classroom application I have become interested in is using AI to create short, bite-sized learning activities.

During my LA/DL sessions, I use activities similar to Kahoot and Blooket to review programming concepts and assignments. I have found that these activities can make a 50-minute session more interactive because students are not simply listening to a lecture.

AI could help instructors create variations of these activities.

For example, I could provide AI with the assignment objectives, course concepts, common student mistakes, and examples of the code being taught. AI could then help generate questions for a Kahoot, Blooket, or another classroom game.

The instructor would still need to review the material before using it, but AI could make it much faster to create several versions of an activity.

The activity could also go beyond a traditional multiple-choice quiz. A programming class could have a Code Detective game where students receive clues about a programming bug and identify the problematic line. Another activity could ask students to predict what a program will output before running it. Students could work in teams to determine which algorithm would be most appropriate for a particular problem.

The goal is not to use AI simply because it is available. The goal is to use it to create more opportunities for students to practice.

This connects to my previous experience teaching STEM, where I used project-based activities involving coding, game design, and robotics. I learned that students can understand technical concepts differently depending on how the material is presented. A short game, demonstration, or hands-on activity can sometimes create a better entry point than another explanation.

AI gives educators another way to experiment with those activities.

AI as a Learning Assistant

AI can also be useful when I am the student.

When I encounter a difficult programming problem, I do not necessarily want an AI system to give me the finished solution. Doing that can remove one of the most important parts of computer science education: learning how to reason through a problem.

Instead, I can ask questions such as:

What concept am I misunderstanding?
What edge case am I missing?
Can you give me a hint without giving me the solution?
Why does my approach fail?
What should I test next?
Can you explain this error differently?

I can also give AI my own work first.

For example, I might write pseudocode, create a class diagram, implement a first version, or explain my reasoning before asking for feedback. This gives the AI something concrete to evaluate.

It also forces me to confront my own understanding before asking for help.

That is an important distinction between using AI to learn and using AI to avoid learning.

What My LA/DL Experience Taught Me

My experience as an LA/DL has made this distinction even clearer.

A student does not always need more information. Sometimes they need the information presented differently. Sometimes they need someone to ask a question that helps them identify their own mistake.

For example, if a student is confused about tracing code, I might change the activity and have them diagram the execution of the program. If several students make the same mistake, I can turn that mistake into a group debugging activity.

I have also learned that engagement matters. A 50-minute session is relatively short, so I try to structure the time around an icebreaker, a game or review activity, assignment content, demonstrations, and a workshop where students can ask questions and begin working.

AI could help me create more variations of these activities, but it does not know my students in the same way that I do.

I still have to decide:

What should students learn?
What is appropriate for their current level?
What misconceptions are they experiencing?
How much content can realistically fit into 50 minutes?
Which activity will encourage participation?
Is the material appropriate for this particular class?

This is where the human role remains important. AI can assist with preparation, but the instructor still has to understand the students and make the educational decisions.

AI in Educational Research

My research experience has given me another perspective on AI in education.

At San Francisco State University, I analyzed submissions from 34 student teams across two semesters using Python and Pandas. The work involved extracting chat logs and reflective writing and analyzing performance across multiple rubric dimensions. I also contributed to research involving AI-integrated grading systems and educational technology.

I have also worked on an E-GAISE agentic grading pipeline that uses Python, LangGraph, Pandas, and Pydantic to evaluate submissions against a rubric. The system uses evidence and validation mechanisms rather than simply accepting an AI-generated evaluation as correct.

This experience has made me more interested in how AI is integrated into a workflow, rather than simply whether AI is being used.

For example, an AI-assisted grading system needs a clear rubric, evidence from the student's work, and a way to evaluate whether its conclusions are consistent. The important question is not whether AI can produce a grade, but whether the overall process produces useful and defensible feedback.

That perspective connects closely to my experience in software engineering.

Human-in-the-Loop: What I Learned From Robotics

My experience at a robotics startup helped me understand why this distinction matters.

My work involved data ingestion, labeling, validation, and internal tooling. The technical problem was not simply getting software to run. The larger problem was making sure the data moving through the workflow could actually be trusted.

I learned that automation is valuable because it can reduce repetitive work and make systems faster. But automation can also make mistakes faster.

That lesson carries directly into AI.

An AI system can generate ten explanations in seconds, but if those explanations are incorrect, it has simply produced ten incorrect explanations faster. The solution is not necessarily to avoid automation. Instead, the workflow needs enough context, testing, and human judgment to determine whether the result is useful.

This gives me a model for how I want to use AI:

Context → AI assistance → Human review → Feedback → Iteration

The same workflow can apply to a classroom activity, an AI-generated explanation, an automated grading system, or a software development task.

The human does not have to perform every step manually. Instead, AI can handle parts of the process while the human remains responsible for understanding the goal and deciding whether the result is acceptable.

The Risk of Overusing AI

The biggest concern I have with AI in education is that convenience can become a substitute for learning.

If a student asks an AI to solve every programming assignment, they may complete the assignment without developing the ability to solve a similar problem independently.

There is a major difference between:

"Give me the answer."

and:

"Here is my design. Can you help me find weaknesses in it?"

The second approach preserves the student's role in the learning process.

The student has made decisions, created an artifact, and thought about the problem. AI becomes a source of feedback instead of a replacement for reasoning.

This is why I think educators should focus less on simply asking whether students are using AI and more on teaching students how to use it responsibly.

Students should know how to question AI output, test generated code, verify information, recognize incorrect assumptions, and explain their own work.

What AI Has Taught Me About Learning

My experiences have changed my definition of effective AI use.

I used to think about AI primarily as something that could make tasks faster. Now I think about it more as something that can create additional opportunities for experimentation.

As a student, I can use AI to challenge my reasoning.

As an educator, I can use it to brainstorm activities, create practice problems, and develop different explanations.

As a researcher, I can investigate how AI changes educational workflows and how those systems can be evaluated.

As a software developer, I can use AI to investigate problems while still being responsible for the resulting system.

In all of these situations, the most valuable workflow is not:

Human → AI → Answer

It is closer to:

Human → Context/Design → AI → Feedback → Human Decision → Iteration

That distinction is important.

Conclusion

AI will continue to change STEM education and software engineering, but I do not think the future should be about replacing the human parts of learning.

My experiences in education and software engineering have taught me that context, iteration, feedback, and human judgment matter.

In the classroom, that could mean creating a design before asking AI for feedback. It could mean using AI to turn common programming mistakes into a short debugging game. It could mean generating a Kahoot or Blooket from course material and then adapting it to the needs of the class.

For a student, it could mean creating pseudocode before asking AI for implementation feedback. For a developer, it could mean giving AI enough information about an existing system before asking it to make a change. For an educational researcher, it could mean designing an AI-assisted grading workflow around evidence and clear evaluation criteria.

These approaches have something in common: AI is helping with the process, but humans remain responsible for the thinking.

That is the lesson I am carrying forward from my work as a student, educator, researcher, and engineer.

AI should not make us think less.

It should give us better ways to think, practice, test, create, and learn.

Lessons from a Robotics Startup: What I Learned About Data Pipelines

Johnny Santamaria — Sun, 09 Aug 2026 00:01:46 +0000

"Smile because it happened" — Dr. Seuss

The Setup

Earlier this year, I took on a short-term trial role with an early-stage robotics startup. The premise was straightforward: help with data collection, annotation, and evaluation workflows—essentially the backbone of any modern robotics or embodied-AI system.
The trial didn't work out long-term. I was let go after about two months — a decision that, honestly, came down in part to my bandwidth as a student. Balancing a full course load with a startup trial was harder than I anticipated.
But that's not the story I want to tell. What I do want to share are the technical lessons I took away — lessons about building robust data pipelines, about the gap between theory and practice, and about what I'd do differently next time.

These aren't company secrets. They're about the general engineering challenges that anyone working with robotics data pipelines will encounter — challenges I'd read about in papers but hadn't truly internalized until I was standing in front of them.

1. The Data Pipeline Shape Is Universal—But the Details Aren't

If you've spent any time in ML or robotics, you've seen this described:
Data Collection → Annotation → Evaluation
It's a standard three-stage pipeline. Industry vendors describe it explicitly in their robotics content. Academic projects model this structure. It's the field's shared vocabulary. Companies such as Scale AI and Toloka use similar industry workflows involving data collection, annotation, and evaluation.
What isn't shared are the specifics: the sensor setup, the calibration procedures, the annotation rubric, and the evaluation metrics. Those are where a company's IP lives. The pipeline shape? That's just the map. And the map is public.

What I'd do differently: Simulate before you collect. Data collection is expensive — in time, hardware wear, and cognitive load on operators. Before running a full session, run a feasibility study with a small batch. Verify your sync and capture scripts. I assumed this was standard practice. It wasn't. The cost of that assumption showed up as rework.

2. Annotation Granularity: More Isn't Always Better

There's a natural temptation when designing an annotation schema: capture everything. Every possible label, every edge case, every attribute you might want later. This is a trap.

Over-granular annotation creates:

Higher operator cognitive load → slower throughput, more errors
Greater inconsistency → more labels means more disagreement
Fragile evaluation → you're evaluating against things you might not need
The better approach? Start with the minimal viable schema that answers your core research question. Add complexity only when the data tells you it's necessary.

A concrete heuristic: If you're designing labels for a demonstration-collection task, ask yourself: "Will I actually use this attribute to decide if the demonstration is successful?" If the answer isn't an immediate yes, drop it. Retrofitting granularity is easier than cleaning up inconsistent over-labeling.

3. Automation Still Needs Oversight—Especially in Evaluation

Automation is seductive. The idea that you can "just set up an automated evaluation pipeline and let it run" is appealing, especially in a startup where resources are tight.
I learned the hard way that automated evaluation is not a replacement for human judgment. It's a filter.

Automated checks catch obvious failures (missing data, format errors).
They don't catch subtle issues (context-dependent errors, edge cases).
They create a false sense of confidence without regular spot-checks.
The workflow that actually works: Run your automated eval, spot-check a sample of what passes, find patterns in the failures, add those to your automated checks, and repeat. It's the "human-in-the-loop" pattern. Easy to forget when you're moving fast.

What I'd Tell My Past Self

If I could go back to Day 1 of that trial, here is exactly what I'd say:
Ask for explicit success criteria upfront. "Good work" means different things to different people in different contexts. Don't assume alignment. Get it in writing, or at least in a clear verbal agreement.

Simulate before you scale. A small batch of test data can save days of rework. Verify your entire pipeline end-to-end with a single example before you collect a hundred.
Design for iteration, not perfection. Your first annotation schema will be wrong. That's fine. Make it easy to change. Don't fall in love with your first draft.

Trust automation as a filter, not a replacement. It will never catch everything. Spot-check everything that matters, especially early on.
After the fact, write yourself a retrospective — from memory, not from any retained materials. Capture what worked and what didn't while it's fresh. You'll thank yourself later when you're interviewing or planning your next project.

Final Thoughts

Getting let go from a trial role is never fun. It stings, and there's always the temptation to replay the "why."

But the technical lessons? Those are mine. They're transferable. They're not tied to any particular company's proprietary stack.
A startup trial is an accelerated learning environment — whether it works out or not. The key is to separate what you learned about yourself and your craft from what you learned about that specific company's internal operations. The former is valuable, portable, and belongs to you.
The latter is their business.

Now, over to you:
What's one technical or operational lesson from a short-term role that completely changed how you approach your work today? I'd genuinely love to hear your stories in the comments — drop one below. 👇

From Kilowatt-Hours to Carbon Counts: Shaping Accurate Energy Reporting with OED

Johnny Santamaria — Wed, 10 Jun 2026 00:03:13 +0000

"garbage in, garbage out" - Some quote I heard before

Introduction

Every building tells a story through its energy usage.

A classroom left lit overnight, an HVAC system running longer than expected, or a sudden spike in electricity consumption across campus, these patterns often go unnoticed without the right tools to visualize them. That’s where the Open Energy Dashboard (OED) comes in.

OED is an open-source platform that helps organizations transform raw utility data into meaningful insights through charts, analytics, and real-time monitoring.

Schools, universities, sustainability teams, and facility managers use the dashboard to better understand how energy and resources are consumed across their buildings. Instead of sorting through spreadsheets or complicated meter data, users can quickly identify inefficiencies, compare trends, and make informed decisions that reduce both operational costs and environmental impact.

The development build is similar to the production build. In order to run development, you must install Docker which is a container for images, like screenshots of a build you can run on your particular operating system. Here is the app running using ‘docker compose up’, and to quit the browser I used ‘docker compose down’.

The Issue

issue link

In this contribution, I worked on LR18, an integration test focused on validating energy conversion behavior in the readings API. The test ensures that 15-minute interval energy readings can be correctly converted into kg CO₂ values while maintaining proper daily aggregation over large time ranges. While the change may seem small from the outside, it introduced interesting challenges involving fixture setup, unit conversions, expected CSV outputs, and backend testing consistency across the codebase.

The following files and resources were the most important locations in the codebase for understanding and implementing the LR18 integration test:

src/server/test/web/readingsLineMeterRangeQuantity.js This was the primary file where the LR18 test case was implemented. The file contains integration tests for line chart range (min/max) quantity meter readings across different aggregation ranges and unit behaviors. I used the surrounding LR1–LR7 tests to understand the expected structure, helper utilities, API request format, and CSV validation workflow. LR18 was added here because it specifically validates range aggregation behavior combined with unit conversion into kg CO₂.
testing.md Documentation link

This design document was important for understanding the intended testing requirements and naming conventions used throughout the project. The document outlines the required test scenarios for readings APIs, including line range (min/max) tests, aggregation expectations, and expected behavior across different unit conversions and time ranges. It helped confirm that LR18 belonged in the line range quantity test suite and clarified how expected CSV outputs should be structured.

readings_ri_15_days_75.csv This readings dataset file contains the raw 15-minute interval test readings used for validating aggregation and conversion behavior. The LR18 test loads this dataset into the database during setup to simulate realistic meter readings over a long time range. These readings are later aggregated into daily points by the API.
expected_line_range_ri_15_mu_kWh_gu_kgCO2_st_-inf_et_inf.csv This expected CSV file represents the correct API response output for LR18 after daily aggregation and unit conversion into kg CO₂. The test compares the API response against this file using helper assertions to verify timestamps, minimum/maximum values, and aggregation correctness.
src/server/util/readingsUtils.js This utility file contains important helper functions and shared testing utilities used throughout the readings integration tests. Functions such as:

prepareTest()
parseExpectedCsv()
expectRangeToEqualExpected()
getUnitId() were essential for seeding test fixtures, loading expected CSV data, resolving unit IDs dynamically, and validating API responses against expected outputs.

src/server/test/web/readingsLineMeterQuantity.js This file was used as a reference implementation because it already contained an existing L18 conversion-based test. I compared its fixture setup, conversion seeding, and naming conventions against LR18 to ensure consistency between quantity tests and range quantity tests.
package.json

The package.json file was useful for identifying available npm scripts and test commands used to run the OED test suite. This helped with running targeted test cases and validating the LR18 implementation locally during debugging.

npm run testData This command was referenced in the testing documentation and used to load sample data into the system for validating the readings API behavior during local testing and debugging.

The Product/Codebase

Tech Stack

Redux Toolkit (RTK): Manages the application's state, ensuring that data remains consistent and synchronized across all parts of the dashboard. Handles things like user selections, fetched meter data, UI state, etc.

Plotly.js: Generates the interactive graphs and charts that visualize energy and resource consumption.

React: The core frontend library used to build the user interface. It renders components for dashboards, selectors, and controls.

Node.js: Runs the backend server, handling data API requests and managing business logic and authentication.

PostgreSQL: Stores all historical energy/resource data, meter definitions, user configurations, etc. Acts as the main database.

Express.js: Web server running on Node.js, exposes RESTful API endpoints for the frontend to request and update data.

TypeScript: Provides type safety and improved developer experience for both frontend and backend codebases.

Docker: Used to containerize and deploy the application and its dependencies for consistent environments (optional in some deployments).

Syatem diagram

Use case

“A user wants to see the CO₂ emissions for electricity usage over a selected date range.”

Step-by-step walkthrough:

User selects a building and a date range in the dashboard's UI (React) and clicks "View CO₂ Emissions."
The React UI dispatches an action to Redux Toolkit to update the state with the new selection.
Redux triggers an asynchronous operation (a thunk or saga) to fetch the relevant readings and emission data from the backend.
The browser makes a REST API request to the Node.js/Express API, specifying the building and date range.
The backend API receives the request. It queries the PostgreSQL database for all energy readings within the range and uses internal logic (data services) to convert those readings into kg CO₂, applying the project’s conversion formulas (the logic tested by the LR18 integration test).
The backend returns a JSON object with time-stamped CO₂ emissions data to the frontend.
The Redux store receives the data and the UI updates accordingly.
Plotly.js renders an interactive chart with the emissions data, letting the user explore and analyze the results visually.

Challenges

Challenge 1: Inconsistent Unit Conversion Setup in LR18

One of the main technical challenges I faced was diagnosing why the LR18 integration test did not match the expected API behavior for converting readings into kg CO₂. Originally, the test was still requesting the kWh graphic unit, even though project requirements and the expected CSV output specified kg CO₂.

Self-Directed Problem-Solving Attempts

Compared with Existing Tests: I reviewed LR1–LR7 to look for patterns in how units were handled and where API requests might be constructed differently.
Codebase Search: I searched for kgCO2 and related conversion logic in both backend tests and the conversion fixture seeding code to find established best practices.
Implementation Cross-Reference: I closely examined the relevant section in readingsLineMeterQuantity.js, focusing on how conversion fixtures were prepared and used, to spot differences.

Additional Help

I leveraged GitHub Copilot to reason through discrepancies, generating code suggestions and checking my test’s construction against unit-handling best practices.
I carefully read the project’s testing.md guidelines to ensure my approach to data seeding and test assertions matched the intended design.

Status

Resolved. The LR18 test setup was corrected: the test now requests the API with the correct kg CO₂ graphic unit and seeds the conversion fixtures to align with expectations. The test passes and provides accurate verification for future code changes.

Challenge 2: CSV Naming and Expected Output Consistency

I also struggled with inconsistent naming conventions for expected CSV files (kg_of_CO2 vs. kgCO2) when comparing actual output to reference data files. This led to confusion in test assertions and in locating previous examples for regression testing.

Self-Directed Problem-Solving Attempts

Repository Search: I searched the repo for both naming styles to see which was most commonly used.
Pattern Review: I surveyed surrounding tests (both quantity and range-based) to verify which format appeared in assertion logic and file loading calls.
Assertion Validation: I checked the test utilities (such as readingsUtils.js) to see if there were helper functions that auto-resolved or validated expected outputs.

Additional Help

I referenced conversion-based tests in readingsLineMeterQuantity.js to ensure consistency in both naming and output structure.

Status

Resolved. I updated the LR18 test’s CSV file expectations and assertions to use the kgCO2 convention, eliminating confusion across the codebase and achieving a passing assertion.

Challenge 3: Maintaining Consistent Branch Workflow (Using git cherry-pick)

A key workflow challenge our team encountered was maintaining a consistent branch structure during development. There were instances when new commits were accidentally made to the wrong branch, which could cause confusion, code conflicts, or integration issues if left unaddressed.

Self-Directed Problem-Solving Attempts

Manual Merging: Attempted to merge the changes or re-apply the edits manually onto the correct branch, but this was time-consuming and could introduce mistakes.
Branch Reset & Rebase: Explored using git reset and git rebase to move commits, but these approaches risked rewriting history or affecting unrelated changes.
Git Documentation Review: Consulted the official Git docs and several online tutorials to look for a safer, more targeted solution for moving just the desired commit(s).

Additional Help

Once I identified git cherry-pick as a possible approach, I confirmed the workflow with Copilot and checked with project teammates (if applicable) to ensure this was in line with team conventions.

Status

Resolved. I successfully used git cherry-pick to copy the intended commit(s) to the correct branch without affecting unrelated work. This helped maintain branch hygiene and enabled a clean pull request history.

Solution

To address the issue of ensuring accurate conversion of energy usage data to kilograms of CO₂ (LR18), I developed an integration test that simulates a user requesting CO₂ emissions data over a specific date range. I modeled my implementation outline using the structure and methodology established in existing test cases LR1–LR7, which demonstrated good practices for preparing test data, invoking the API endpoint, and checking results.

In my solution, I set up sample meter readings in the database, triggered the backend logic to request the calculated CO₂ values, and verified that the backend (Node.js/Express) returned the correct results according to the expected conversion rules. This test helps ensure the data displayed in the dashboard (via Redux Toolkit and Plotly.js in the frontend) will always be calculated correctly, catching any regressions as the codebase evolves.

I relied heavily on Redux Toolkit to manage the application state in the frontend, this ensures that, once the backend API returns the correct CO₂ data, the entire dashboard updates consistently and all visualizations generated by Plotly.js reflect accurate numbers.

On the backend, I focused on the API logic and data services component (see System Diagram: Node.js/Express and Data Services) that handles the data transformation and communicates directly with the PostgreSQL database. My test verifies that conversions at this backend layer, before the data even reaches Redux or Plotly, are correct.

I verified the solution using automated test runners via Mocha. After implementing the LR18 test, I ran the full test suite and confirmed that all tests, including my new one, passed. This proves that both existing and new requirements for data conversion are met.

See the full solution here

Keeping Your Secrets in Sync: Environment Lifecycle Management That Actually Works

Johnny Santamaria — Sat, 14 Feb 2026 01:25:14 +0000

Introduction

Keyshade is an open-source real-time secret and configuration management platform that helps developers securely store, sync, and rotate sensitive data like API keys, passwords, and environment variables across multiple environments. It uses end-to-end encryption and live updates to eliminate insecure .env file sharing, manual dashboard updates, and secret sprawl. Designed for teams and solo developers alike, Keyshade improves collaboration, auditability, and security by providing version history, access controls, and seamless integrations with modern deployment platforms.

Try it out on https://keyshade.io/

The Problem: Silent Failures in Cloud Integrations (#1254)

The problem was that the integrations with Vercel or AWS Lambda would only respond to updates in secrets and variables. It would completely ignore situations where the user had deleted the environment or renamed the environment. This meant that when a user deleted an environment in Keyshade, the secrets and variables would still be available in Vercel or AWS Lambda but would be orphaned. This presented a problem from a security point of view. Also, when the environment was renamed, the integrations would be out of sync.

Why This Bug Mattered to Production Teams

This problem was significant because of several factors:

Security Risk: When a user deleted an environment using Keyshade (like an environment in staging mode), the secrets became active in Vercel and AWS Lambda. This meant the secrets were exposed on another platform. That is, credentials became potential attack vectors.
Data Integrity: The disconnect between Keyshade’s environment state and the external platforms led to a situation where users couldn’t be assured of data integrity. It defeats the purpose of having a reliable tool like Keyshade.
User Experience: It will be the responsibility of the consumers of Keyshade to manually log in to the Vercel or AWS Lambda interface to clean up after environment changes, thereby defeating the purpose of the integration in the first place. Such an interface might discourage the use of Keyshade.
Production Readiness: The integrations must be prepared to handle the full lifecycle of environments, not only the secrets within static environments, for Keyshade to be considered viable in production, especially by teams with high rates of environment changes (e.g., CI/CD workflows creating ephemeral environments). It was marked as "priority: high" and "difficulty: 4." This was my hardest PR that took around two months.

Under the Hood: Key Code Changes

keyshade/apps/api/src/integrations/plugins/vercel.integration.ts
Purposes: Integrates the Vercel platform - responsible for syncing Keyshade secrets/variables to Vercel projects as environment variables.
Key sections to edit:
getPermittedEvents(). Added ENVIRONMENT_UPDATED and ENVIRONMENT_DELETED to the set of events the integration subscribes to.
added case handlers to switch of emitEvent() for routing environment lifecycle events.
New functions added: handleEnvironmentDeleted(): This cleans up project-level environment variables scoped to the deleted environment and breaks the integration relationship handleEnvironmentUpdated(): Renames custom Vercel environments if available (pro plan users) & update integration metadata

apps/api/src/integration/plugins/aws-lambda.integration.ts
Purpose: Implements the AWS Lambda integration, syncing Keyshade secrets to Lambda function environment variables
Key sections modified:
getPermittedEvents(): Added the two environment events
emitEvent() switch statement: Added routing for environment events
New functions added:
handleEnvironmentDeleted(): Removes environment-scoped keys from Lambda function configuration and disconnects the environment relationship
handleEnvironmentUpdated(): No-op handler (Lambda has no environment rename concept) that logs an audit event

apps/api/src/common/util.ts
Purpose: Added retryWithBackoff() helper function to handle transient failures when calling external APIs (Vercel/AWS)
Why important: Cloud APIs can fail temporarily; this ensures cleanup operations are resilient and retry with exponential backoff

apps/api/src/integration/reconciler.ts
Purpose: Lightweight reconciliation system that processes failed cleanup operations stored in integration.metadata.pendingCleanup
Why important: When retries fail after multiple attempts, the reconciler can later replay these operations, ensuring eventual consistency even if external APIs are temporarily unavailable

apps/api/src/event/event.types.ts
Purpose: Defines TypeScript types for event metadata
Relevant types:
EnvironmentUpdatedEventMetadata: Contains name and description fields for environment updates
EnvironmentDeletedEventMetadata: Contains the environment name for deletion events

Test files:
apps/api/src/integration/plugins/vercel.integration.spec.ts: tests covering environment lifecycle handlers
apps/api/src/integration/plugins/aws-lambda.integration.spec.ts: tests covering Lambda-specific handlers
apps/api/src/integration/reconciler.spec.ts: testing the reconciler's pending cleanup processing

Supporting materials/artifacts
Before the fix:
When a user deleted an environment in Keyshade, the integration would:

Not receive or process the ENVIRONMENT_DELETED event
Leave all environment-scoped secrets active on Vercel/AWS Lambda
Not update the integration metadata to reflect the environment was gone
Create orphaned credentials that users would have to manually clean up

Error behavior:
While there wasn't a specific error message, the silent failure was the problem. Users would delete an environment expecting it to be cleaned up everywhere, but the secrets would persist on external platforms with no warning or indication that cleanup failed.
After the fix:
The integration now:
Subscribes to ENVIRONMENT_UPDATED and ENVIRONMENT_DELETED events
Automatically removes environment variables from Vercel projects when environments are deleted
Removes environment-scoped keys from AWS Lambda function configurations
Updates integration metadata to stay in sync with environment name changes
Uses retry logic with exponential backoff for resilience against transient API failures
Persists failed operations in pendingCleanup for later reconciliation
Logs audit events for compliance and debugging

Key Implementation Detail:
For Vercel's ENVIRONMENT_UPDATED handler, there's a special consideration: Vercel only allows renaming custom environments on Pro plans, not the standard "Preview", "Production", or "Development" environments. The code includes logic to attempt the rename and gracefully handle cases where it's not supported.

Challenges

While preparing my pull request, I ran into several real-world challenges that highlight what contributing to an active open-source project actually looks like behind the scenes. The first issue was merge conflicts between my branch (attempt-1254) and the target develop branch. In a fast-moving repository with many contributors, this is almost inevitable. I resolved the conflicts by pulling the latest upstream changes and manually reviewing each conflicting section to understand both sides. My goal was to preserve other contributors’ work while ensuring my environment lifecycle handler implementation remained intact. After merging, I tested everything locally to confirm nothing regressed, then committed with a clear explanation of the resolution. It was a reminder that merge conflicts aren’t just mechanical fixes — they’re moments where you have to understand the intent of multiple developers and reconcile them thoughtfully.
The next challenge surfaced during code review. A maintainer questioned my use of any type casts in TypeScript, particularly when accessing integration.id and working with the pendingCleanup metadata array. They pointed out that this could introduce long-term technical debt and suggested using @ts-expect-error instead. That feedback was valuable. While the any casts were originally a temporary measure to unblock a critical bug fix, the reviewer highlighted how @ts-expect-error is self-validating: TypeScript warns you if the suppressed error disappears later, which prevents stale workarounds from lingering in the codebase. I acknowledged the concern, added TODO comments recommending a future typed refactor, and committed to either replacing any with @ts-expect-error where appropriate or creating a follow-up issue to properly type the metadata structures. This exchange was a great example of how open-source review culture improves not just correctness, but long-term maintainability.
The biggest blocker came from the CI/CD pipeline. My PR was failing two checks: a Snyk security scan and an API validation build. The frustrating part was that I couldn’t reproduce the failures locally. All my local tests passed — dependencies installed cleanly, the build succeeded, and every unit test ran successfully. But Snyk repeatedly failed in CI with errors related to workspace handling and package manager compatibility. The CLI didn’t fully support the project’s pnpm workspace setup in the same way the CI environment did, and I couldn’t access the GitHub Actions logs directly from my development environment. I tried multiple workarounds, including running Snyk through pnpm and downloading the standalone CLI, but nothing mirrored the CI behavior. Even rerunning the workflows with empty commits didn’t resolve the issue.
At that point, I documented everything I had tried in a detailed PR comment and asked for help. A maintainer with admin access reviewed the CI logs and identified the root cause: a vulnerability in a transitive dependency that required updating the root package.json and regenerating the lockfile. They also explained that the project’s Snyk integration uses a specialized workspace configuration that isn’t trivial to replicate locally. This led to a broader discussion about improving CI transparency for external contributors, including the possibility of automatically surfacing relevant log snippets in PR comments. The experience reinforced an important lesson: as an external contributor, you won’t always have full visibility into infrastructure, and clear documentation of your debugging efforts is essential for maintainers to step in effectively.

Solution (PR #1255)

After working through these issues collaboratively, the pull request was successfully merged. The final solution included environment lifecycle event handlers for both Vercel and AWS Lambda, retry logic with exponential backoff, a reconciliation system for eventual consistency, and comprehensive test coverage. All CI checks passed, and review feedback was fully addressed. More importantly, the contribution closed a high-priority bug that had been blocking production adoption of cloud integrations. It was a strong reminder that persistence, careful communication, and respect for the review process are just as important as writing the code itself. Open-source success isn’t just technical, it’s collaborative.

What Did I Learn?

How to read official Typescript documentation to break down confusing code
How to ask beneficial questions to maintainers
How to navigate CI/CD failures as an external contributor
Understanding Event-Driven API Architecture
Snyk improves security in development

Keyshade Debugging: Mastering Workspace Role Tests and API Repair

Johnny Santamaria — Sat, 08 Nov 2025 01:06:38 +0000

"A good tester prevents problems; a great tester finds them." Keith Klain, Director of Quality Engineering at KPMG UK

Introduction

Four disabled tests. One cryptic comment: 'flaky.' When I dove into Keyshade's workspace role tests, I discovered the failures weren't random at all—they revealed a fundamental misunderstanding of how the API validates permissions.

Keyshade is a modern solution for secure, centralized credential and configuration management, designed for developers, DevOps engineers, and open-source communities who need to manage secrets without the headache.

The Issue (#1122)

I decided to continue my open source journey on Keyshade since my blog post going over how I solved issue 998, which was that the Keyshade command line interface’s “keyshade run” command fails to give a proper error that users can understand. Now I decided to explore Keyshade’s API since I have explored how the CLI functions in its codebase.

I found an issue where included a bug in workspace role tests, which meant the problem stemmed from how workspace roles were being set up and asserted in tests, some test fixtures and assertions depended on implicit ordering and shared state, which made the tests fragile and allowed timing or state-dependent flakes to slip into CI (Continuous Integration). By making the setup deterministic and isolating role state for each test (explicitly creating the roles and their relationships, avoiding shared mutable fixtures, and tightening assertions to check the actual role assignments rather than relying on incidental data), the tests now reliably exercise the API behavior they were intended to cover.

The result is a more robust test suite that accurately guards against regressions in role handling for workspaces, reduces CI noise from flaky failures, and gives us greater confidence in future changes to authorization and workspace logic. Here are some key locations in the codebase we found out to be useful in solving the issue:

The apps/api/src/workspace-role/ folder includes the workspace-role feature module — contains controller, service, DTOs, and possibly unit/integration tests. Focus on business logic, dependency wiring, and any in-module role constants.

Examples:

Inspect the service methods (assignRole, removeRole, listRoles) for edge-case logic (no-op on duplicate role, default role applied on creation).
Check module exports and DI config: ensure the controller gets the expected implementation of the role service (mock vs real) in tests.
Example check: verify if role constants are defined here (e.g., 'admin', 'member', 'viewer') and whether they match values used in DB/entities or frontend.

The apps/api/src/workspace-role/workspace-role-controller.ts file includes the controller / HTTP handlers for workspace-role endpoints — review request DTOs/validation, how the controller calls services, status codes, and permission checks performed at the controller layer.

Examples:

Example request mapping: POST /workspaces/:id/roles -> controller.assignRole(req.params.id, req.body). Check expected body shape (userId, role).
Example validation: if controller uses decorators or a schema, show the required fields (e.g., { userId: string, role: 'admin'|'member' }).
Example permission check: controller may call guard like ensureWorkspaceAdmin(currentUser, workspaceId) — confirm where that check happens and what error it throws (403 vs 401).
The apps/api/src/workspace-role/workspace-role/workspace-role.e2e.spec.ts file has end-to-end tests for workspace-role flows — tests that spin up the app (or a test server), seed DB data or fixtures, run real HTTP requests, and assert full-stack behavior and responses.

Examples:

Test setup/teardown: show DB seeding (create users, create workspace) and cleanup steps to avoid cross-test leakage.
Example test case: an e2e test that POSTs role assignment and then GETs membership list to assert the new role is present; include expected status codes and response shapes.
Flakiness checks: look for timeouts, async waits, or hard-coded delays that may make these tests flaky (and thus failing intermittently).

General Diagram Walkthrough of the Issue

Reproducing the Problem

Here are steps regarding how I reproduced the problem before assessing my plan, implementation, and solution

Scenario 1 — should not be able to update the workspace role with environments that do not belong to the project Steps to reproduce the behaviour:

Create (or use) a workspace W.
Create Project A in W and capture PROJECT_A_ID.
Create Project B in W and capture PROJECT_B_ID.
Create Environment A in Project A (ENV_A_ID) and Environment B in Project B (ENV_B_ID).
Create a workspace role R for Project A that includes only ENV_A_ID.
Configure your CLI to target workspace W and authenticate with a token that can update roles.
Use the CLI (or send an API request) to update role R to set environmentIds = [ENV_A_ID, ENV_B_ID].
Observe the response/error shown by the CLI.

Expected behaviour

The API should validate that ENV_B_ID does not belong to Project A and return a 4xx validation error (e.g., 400 or 422) with a clear, formatted message such as: "environment ENV_B_ID does not belong to project PROJECT_A_ID"
The CLI should print that formatted message to the user (no opaque stack trace or generic 500).
The update must be rejected and the role R must remain unchanged (still only ENV_A_ID).

Scenario 2 — should be able to add environment access for projects to the role with READ_WORKSPACE, UPDATE_WORKSPACE_ROLE, READ_PROJECT, READ_ENVIRONMENT authorities Steps to reproduce the behaviour:

Create (or use) a workspace W.
Create Project P in W and ensure it has at least one environment (ENV_P_1, …).
Create a workspace role R (it may initially have no project-level environment access).
Obtain an auth token that has the authorities: READ_WORKSPACE, UPDATE_WORKSPACE_ROLE, READ_PROJECT, READ_ENVIRONMENT.
Configure your CLI to use workspace W and that token.
Use the CLI (or send an API request) to add project-level environment access for Project P to role R (the endpoint that grants project environment access).
Observe the response/error shown by the CLI and then GET the role R to inspect its environment access.

Expected behaviour

The API should accept the request and return 200/201 with a clear success body describing the new project-level environment access (or list of environmentIds that were added).
The CLI should print a readable success message and show the role now has access to Project P environments.
If the token lacks any required authority, the API should return 403 Forbidden with a clear formatted error and the CLI should print that message.

Scenario 3 — should be able to add environment access for projects to the role (permission variation) Steps to reproduce the behaviour:

Create (or use) a workspace W, a Project X with environments, and a role R.
Prepare two auth tokens:

* AUTH\_ADMIN: token with UPDATE\_WORKSPACE\_ROLE (and related read authorities).

* AUTH\_RESTRICTED: token without UPDATE\_WORKSPACE\_ROLE.

Configure your CLI to use workspace W.
With AUTH_ADMIN, run the CLI command (or API request) to add Project X environment access to role R. Verify success.
With AUTH_RESTRICTED, attempt the same CLI command (or API request) to add Project X environment access to role R. Observe the response/error shown by the CLI.

Expected behaviour

With AUTH_ADMIN: API returns 200/201 and the role R gains access to Project X environments; CLI prints a readable success confirmation.
With AUTH_RESTRICTED: API returns 403 Forbidden with a clear formatted message like "missing authority: UPDATE_WORKSPACE_ROLE" and the CLI prints that message; role R must not be modified.

Scenario 4 — should not be able to create workspace role where environments do not belong to the project Steps to reproduce the behaviour:

Create (or use) a workspace W.
Create Project A and Project B in W.
Create Environment B in Project B (ENV_B_ID).
Configure your CLI to target workspace W with a token that can create roles.
Use the CLI (or POST API) to create a workspace role with:

* projectId = PROJECT\_A\_ID

* environmentIds = \[ENV\_B\_ID\]

Observe the response/error shown by the CLI.

Expected behaviour

The API must reject the create request with a 4xx validation error (e.g., 400 or 422) and a clear message such as: "environment ENV_B_ID does not belong to project PROJECT_A_ID"
The CLI should print that formatted error.
No role should be created.

Keyshade’s Codebase

Here is Keyshade's codebase and API related to this issue, and how would I use the code to diagnose and fix the failing workspace-role tests, displaying the importance of code organization.

High-level overview (relevant subsystems)

Schema / client shapes:
- packages/schema/src/workspace-role/index.ts defines the WorkspaceRole payload shape (projects + environments arrays, authorities, etc.). That tells you the expected request/response shape for workspace role create/update operations.
API service layer:
- apps/api/src/workspace-role/workspace-role.service.ts contains the logic that parses project -> environment inputs and applies DB operations. The key function is parseProjectEnvironmentsDBOperation which:
  - verifies that an environment slug exists and belongs to the provided project
  - throws NotFoundException if an environment is not part of the project
  - calls authorizationService.authorizeUserAccessToEnvironment to ensure the acting user has READ_ENVIRONMENT on that environment
Authorization checks:
- apps/api/src/auth/service/authorization.service.ts and apps/api/src/auth/service/authority-checker.service.ts implement the authority checks and higher-level authorization plumbing. authorizationService exposes methods such as authorizeUserAccessToEnvironment and authorizeUserAccessToSecret; these both call into the authority checker and also verify workspace membership.
Tests & client usage:
- packages/api-client/tests/workspace-role.spec.ts contains tests that exercise the workspace-role APIs from a client perspective (create workspace, create project, create environments, then create/update roles).
- apps/platform and controller-instance reference the WorkspaceRoleController from the generated API client; you can use the API client or the CLI to reproduce e2e scenarios.

Why the tests were failing (what the issue description lists)

The failing/disabled tests are focused on cases where environments provided for a workspace role do not belong to the project, and cases that require multiple authorities (READ_WORKSPACE, UPDATE_WORKSPACE_ROLE, READ_PROJECT, READ_ENVIRONMENT).
The service code enforces two separate checks:

1. Ownership check: the environment must belong to the provided project — otherwise parseProjectEnvironmentsDBOperation throws NotFoundException with a message like "Environment X is not part of project Y".

2. Authorization check: after confirming the environment belongs to the project, the service calls authorizationService.authorizeUserAccessToEnvironment which requires the user to have READ\_ENVIRONMENT on that environment (and also checks workspace access).

If tests tried to use an environment that belongs to a different project, the service will intentionally return 404/NotFound before attempting to grant access; tests must assert that behavior.
If tests expected that adding environment access requires specific authorities, the test setup must give the test-user those authorities (or use a workspace-admin) before calling the endpoint.

How to use the codebase to reproduce, diagnose, and fix the tests

Reproduce the failing case manually (fastest way to see precise error):

* Use packages/api-client/tests/workspace-role.spec.ts as a reference for how tests set up the workspace/project/environment/role chain.

* From an integration/e2e test or an API client, create:

    * workspace A

    * project P1 inside workspace A

    * project P2 inside workspace A

    * environment E1 in P1 and environment E2 in P2

* Then attempt to create/update a workspace role that includes project P1 but lists environment E2 (an environment that belongs to P2).

* Observe the response: workspace-role.service.ts is expected to detect the mismatch and return a 404 Not Found with message "Environment ... is not part of project ...".

Check the authority path:

* If you provide a correct project-&gt;environment mapping, but the requestor lacks READ\_ENVIRONMENT (or other expected authorities like UPDATE\_WORKSPACE\_ROLE), authorizationService.authorizeUserAccessToEnvironment will throw Unauthorized/Forbidden. Ensure test user has the required authorities when the test expects success.

Fix the tests (common issues & fixes):

* If a test was disabled because it used an environment from a different project but expected a different error (or success), restore the test and:

    * Either adjust the test to expect NotFound (because parseProjectEnvironmentsDBOperation intentionally rejects cross-project environments).

    * Or adjust test fixtures so the environment is created in the same project as the project entry in the request.

* For authority-related tests:

    * Make sure the test user has the necessary authorities before calling the API. Grant a role to the test user with READ\_WORKSPACE, UPDATE\_WORKSPACE\_ROLE, READ\_PROJECT, READ\_ENVIRONMENT when the test expects role creation/update to succeed.

    * If a test is verifying denial paths (should not be able to create/update), ensure the user has fewer/missing authorities and assert the API returns Unauthorized/Forbidden appropriately.

Where to change code if the behavior is incorrect:

* If you decide the server should validate authority first (instead of ownership first), modify parseProjectEnvironmentsDBOperation ordering; but be careful: the current behavior (check membership/ownership first, then authorization) is consistent with tests that expect NotFound for cross-project envs.

If messages or status codes are inconsistent with tests, adjust the thrown exceptions in parseProjectEnvironmentsDBOperation (NotFoundException) or in authorizationService (Unauthorized/Forbidden) to align with the test expectations — but prefer changing tests to reflect correct domain behavior unless an API bug is proven.

Useful files to inspect or instrument:

* packages/schema/src/workspace-role/index.ts (payload schema)

* apps/api/src/workspace-role/workspace-role.service.ts (parseProjectEnvironmentsDBOperation and create/update logic)

* apps/api/src/auth/service/authorization.service.ts (authorizeUserAccessToEnvironment)

* apps/api/src/auth/service/authority-checker.service.ts (lower-level authority checks)

* packages/api-client/tests/workspace-role.spec.ts (test setup and expected assertions)

Concrete actionable checklist to resolve the issue in the repository

Re-enable the disabled tests.
For each disabled test, verify which of the two server behaviors it is validating:
- Ownership validation (environment belongs to project) → expect 404 Not Found. Ensure test builds a request with a mis-matched environment and asserts a 404 and the message text if desired.
Authorization validation (user needs the listed authorities) → ensure test grants the required authorities to the requester when expecting success; when expecting failure, ensure the user lacks at least one required authority and assert Unauthorized/Forbidden.
Run the tests locally (or in CI):
- The workspace-role e2e test file: packages/api-client/tests/workspace-role.spec.ts is a good starting point.
- Add logging in workspace-role.service.ts parseProjectEnvironmentsDBOperation if you need to trace why an environment was rejected.
If the test failure shows unexpected status codes or messages, adjust the service's thrown exceptions to match the documented API or update the tests to assert the documented behavior.
If you change server code, add unit tests in apps/api for parseProjectEnvironmentsDBOperation to cover:
- environment not in project → NotFound
- environment in project but user lacks READ_ENVIRONMENT → Unauthorized/Forbidden
- environment in project and user has authority → success path produces correct DB ops

Short example of the expected behavior (mapping to the tests listed in the issue)

"should not be able to update the workspace role with environments that do not belong to the project":
- Call update role with project P1 + env from P2 → server should return 404 Not Found (environment not part of project).
"should be able to add environment access for projects to the role with READ_WORKSPACE, UPDATE_WORKSPACE_ROLE, READ_PROJECT, READ_ENVIRONMENT authorities":
- Ensure requestor has those authorities (assign a role in test fixture) → call create/update and assert success and that the role contains the project->environment mapping in response.
"should be able to add environment access for projects to the role":
- Same as above but likely using normal granted authorities; ensure fixtures grant appropriate permissions.
"should not be able to create workspace role where environments do not belong to the project":
- Create role payload with mismatched environment → expect 404 Not Found.

Where to look in code to make changes or confirm behavior

parseProjectEnvironmentsDBOperation in:
- apps/api/src/workspace-role/workspace-role.service.ts
authorization plumbing:
- apps/api/src/auth/service/authorization.service.ts
- apps/api/src/auth/service/authority-checker.service.ts
request/response schema:
- packages/schema/src/workspace-role/index.ts
tests:
- packages/api-client/tests/workspace-role.spec.ts

Use Case

Goal: Create/update a workspace role that grants access to specific projects and, within those projects, specific environments.
Client payload shape (what the API expects)

1. name, description, authorities (array), projectEnvironments: \[{ projectSlug, environmentSlugs: \[slug\] }\]

Server behavior (what the code enforces)

1. For each projectEnvironments entry, resolve projectSlug -&gt; projectId (getProjectSlugToIdMap).

2. For each environmentSlug in environmentSlugs:

    * Verify that environmentSlug exists AND belongs to that project (prisma.environment.findFirst where slug + projectId). If not, throw NotFoundException (404) with a body "Environment not found" / "Environment X is not part of project Y".

Authorize the acting user has READ_ENVIRONMENT for that environment (authorizationService.authorizeUserAccessToEnvironment). This also verifies workspace-level access and permitted authorities (via authorityCheckerService).

Finally, upsert projectWorkspaceRoleAssociation rows connecting roleId<>projectId and connect the environment slugs.

Why the workspace-role tests were failing

The failing/disabled tests covered two distinct types of validation:
- Ownership validation: environment must belong to the project in the request. If not, server should reject with 404 (Not Found).
- Authorization validation: even if environment belongs to project, the acting user must have proper authorities (READ_ENVIRONMENT, UPDATE_WORKSPACE_ROLE, READ_PROJECT, READ_WORKSPACE_ROLE) to add environment access to a role.
Typical causes for failures:
- Tests that assert 404 were previously commented out or used incorrect fixtures (e.g., environment not created, or created but in same project), so the server behavior didn't match test expectations.
- Tests that assert successful updates weren't ensuring the test actor had the required authorities (so server returned Unauthorized/401/403 instead of 200).
My PR re-enabled those tests and fixed fixtures by:
- Creating environments in the correct projects (dev, stage).
- Explicitly setting the membership role authorities (e.g., update the workspace member role to include READ_ENVIRONMENT and UPDATE_WORKSPACE_ROLE) when the test expected updates to succeed.
- Using consistent test header 'x-e2e-user-email' (alice.email) for requests.

Walkthrough of the two failing test categories and what to change

"Environment doesn't belong to the project" (ownership)

Server side: parseProjectEnvironmentsDBOperation checks project membership:
- It runs prisma.environment.findFirst({ where: { slug: environmentSlug, projectId } })
- If not found, it throws: throw new NotFoundException(constructErrorBody('Environment not found', Environment ${environmentSlug} is not part of project ${pe.projectSlug}))
Tests expecting 404 must:
- Create a workspace and two projects in that workspace.
- Ensure the environment used in the request was created in THE OTHER project, or not created at all (but prefer creating in other project to avoid false positives).
- Make a POST/PUT with projectSlug set to projectA and environmentSlugs containing an environment slug that actually belongs to projectB.
- Assert statusCode === 404.

"Adding environment access requires specific authorities" (authorization)

Server side: after ownership check, parseProjectEnvironmentsDBOperation calls: await this.authorizationService.authorizeUserAccessToEnvironment({ user, slug: environmentSlug, authorities: [Authority.READ_ENVIRONMENT] })
That code path will call authorityCheckerService.checkAuthorityOverEnvironment which checks the acting user's permitted authorities for that environment (it can be from project-level or workspace-level roles).
Tests expecting success must ensure the acting user has:
- READ_WORKSPACE (if required by the flow), UPDATE_WORKSPACE_ROLE, READ_PROJECT, READ_ENVIRONMENT — in practice, the tests can mutate an existing workspace role for the test user to include these authorities and also connect appropriate project/environment associations so the user has project-scoped authority.

Exact test changes — enable one disabled test and adjust fixture

Below is a concrete enabled test snippet for packages/api-client/tests/workspace-role.spec.ts that re-enables the “should not be able to create workspace role where environments do not belong to the project” scenario.
This patch enables the test and makes the fixture explicit: create two projects, create an environment in projectB, then attempt to create a workspace role that references projectA + the environment slug from projectB. The test asserts 404.

packages/api-client/tests/workspace-role.spec.ts

// (This is the test snippet to add/enable within the existing test suite)

it('should not be able to create workspace role where environments do not belong to the project', async () => {

// Precondition: two projects exist in the workspace

// projects[0] -> projectA, projects[1] -> projectB (test fixture established earlier in the file)

// Create an environment in projectB that should NOT be valid for projectA

await prisma.environment.create({

Notes about placement:
- Put this enabled test near the other workspace-role create tests after the workspace + project fixtures are created and available.
- Ensure projects variable references are the ones created during beforeAll/beforeEach steps in that test file.

A directly analogous enabled e2e test (what PR 1156 did)

PR 1156 enabled very similar tests in apps/api/src/workspace-role/workspace-role.e2e.spec.ts — it:
- Created dev/stage environments in the proper projects,
- Updated the Member workspace role to add project associations and required authorities for the actor,
- Re-enabled the previously commented tests and asserted status codes 200 or 404 as appropriate.

Small server-side unit test to lock in parseProjectEnvironmentsDBOperation behavior

I recommend adding a unit test that calls parseProjectEnvironmentsDBOperation (or exercises the public create/update API) with a project-environment mismatch and asserts a NotFoundException is thrown.
Example test file (unit) to confirm rejection for cross-project environment:

apps/api/src/workspace-role/workspace-role.service.spec.ts

import { Test, TestingModule } from '@nestjs/testing'

import { WorkspaceRoleService } from './workspace-role.service'

import { AuthorizationService } from '@/auth/service/authorization.service'

import { PrismaService } from '@/prisma/prisma.service'

import { NotFoundException } from '@nestjs/common'

Proposed server code changes — minimal / optional

The current server code in parseProjectEnvironmentsDBOperation is already correct about ownership-first validation (it checks slug + projectId before calling authorizeUserAccessToEnvironment). The PR you provided does not require a behavior change on the server — tests should reflect the server semantics.
If you did want to change server behavior, the main two options are:

1. Keep current semantics (ownership check first, then authorization). Pros: deterministic 404 for cross-project envs; avoids running authorization queries for environment slugs that don't belong to the provided project. Cons: reveals that the environment slug exists in a different project (server replies 404 with message referencing project mismatch).

2. Run authorization first (i.e., check the caller's permissions to read the environment before checking project membership). Pros: could collapse some error channels and avoid disclosing cross-project existence if the caller lacks permission. Cons: often you still need to fetch the environment by slug (so you still learn about its existence) — and the authorization check typically requires fetching the environment anyway to know project/workspace id. It also changes the status codes returned for the cross-project case (may return 401 instead of 404), which is a breaking API semantics change for clients that currently rely on 404 in that case.

Recommendation: do NOT change the ordering of checks unless there's a clear security/privacy reason. The current behavior (404 on ownership mismatch) matches the tests in PR 1156 and is explicit. Prefer to fix test fixtures and grant/revoke authorities in test setup rather than altering server semantics.

Why the PR's approach (what you already did in PR 1156) is correct

You re-enabled tests and fixed fixtures instead of changing server semantics. Specifically:
- Created missing environments in the appropriate projects before calling the API.
- Updated the test user/role authorities where tests expected success.
- Asserted the correct status codes: 404 for ownership mismatch and 200 for successful updates when authorities exist.
This is minimal and non-breaking.

Trade-offs if you change server behavior (ownership vs authorization ordering)

Changing the order to authorize first:
- May return 401/403 instead of 404 for the cross-project environment case — tests and clients must be updated.
- Could obscure whether an environment exists in another project (slight security benefit).
- Could introduce additional database lookups if you need to check membership as well; may not yield significant performance gain.
Keeping ownership-first (current):
- Clear, deterministic 404 response for mismatched environment ownership.
- Tests remain simple: assert 404 for the cross-project case.
- Slight information disclosure (you show that environment exists but in a different project), which may be acceptable in an internal system like Keyshade.

Summary of exact artifacts to change (concrete)

Add/enable the test in packages/api-client/tests/workspace-role.spec.ts as above (or in apps/api/src/workspace-role/workspace-role.e2e.spec.ts — PR 1156 already did the same).
Add a small unit test in apps/api/src/workspace-role/workspace-role.service.spec.ts to lock in behavior of parseProjectEnvironmentsDBOperation for cross-project envs (example above).
Ensure any tests that expect successful updates explicitly:
- create the environments in the projects they want to reference
- grant the test actor role(s) the required authorities (UPDATE_WORKSPACE_ROLE, READ_PROJECT, READ_ENVIRONMENT) before calling create/update

Challenges

I reviewed PR #1156 and walked the tests and service code. Below I list concrete challenges we encountered while making this PR and practical mitigations. I narrate what I checked and what I'd recommend next.

What I did (quick recap)

Inspected the workspace-role service logic (parseProjectEnvironmentsDBOperation) and the authorization flow.
Reviewed the test files that were re-enabled/modified in the PR (workspace-role e2e/spec files).
Confirmed the PR fixes were basically test-fixture and authority-setup issues (creating environments in right projects, granting the test user required authorities), not major server-behavior changes.
Verified the PR merged successfully but noted a set of practical challenges that often show up in this work.

Challenges encountered (with examples and mitigations)

Test fixture brittleness and DB state coupling

Problem: E2E tests depend on precise workspace/project/environment setup and roles. Small differences in fixture creation order or leftover DB rows cause flakiness.
Example: A test assumed "production" env exists in projectA but CI used a different test run that already had production in projectB -> unexpected 200 instead of 404.
Mitigation:
- Always create and assert fixtures in test setup (do not assume names).
- Use randomized slugs (e.g., slug-<timestamp>-<random>) to avoid collisions across parallel CI jobs.
- Add teardown/cleanup that removes created rows or run tests inside isolated databases/transactions.

Authority/permission complexity in tests

Problem: Tests that check success paths require several authorities (READ_WORKSPACE, UPDATE_WORKSPACE_ROLE, READ_PROJECT, READ_ENVIRONMENT). If the test doesn't explicitly grant them, the server returns Unauthorized and the test fails.
Example: Test enabling environment access but not updating the Member role to include UPDATE_WORKSPACE_ROLE.
Mitigation:
- Explicitly set up the actor’s roles/authorities per test case.
- Add helper functions in tests for "grantMemberAuthoritiesForEnvironmentAccess" to make fixtures clear.
- Keep negative/positive authority tests separate and deterministic.

Merge conflicts (simple but annoying)

Problem: Enabling tests and modifying the same test files or fixtures on different branches led to conflicts (e.g., two branches each uncommented different blocks of tests).
Example: Two branches changed apps/api/src/workspace-role/workspace-role.e2e.spec.ts: one enabled a test, another refactored a beforeEach. Merge resulted in conflict markers around large commented blocks.
Mitigation:
- Rebase frequently on the develop branch to reduce long-lived divergences.
- Split large test-enabling changes into smaller PRs (enable + fix fixtures + authority wiring).
- When resolving conflicts, run the full test suite locally before pushing the merge commit.

Creating temporary test files or helper tests "to be removed later" causing issues

Problem: Developers often create throwaway tests / temporary files to debug behavior and forget to remove them. These can be picked up by CI and cause nondeterministic runs or duplicate test names.
Example: Added apps/api/src/workspace-role/tmp-debug.spec.ts to iterate quickly, later removed locally but remained in another branch → CI fails or duplicate DB setup occurs.
Mitigation:
- Avoid committing throwaway test files. If necessary, keep them in local-only branches or protect with a naming convention excluded from CI.
- Add CI test discovery rules to ignore test files matching tmp-*.spec.ts, if you must have temporary tests.
- Review diffs carefully before commit and run git status to ensure no debug/test files are included.

Test duplication across packages (api-client vs apps/api)

Problem: Similar workspace-role tests existed in both packages/api-client/tests and apps/api e2e test files; enabling one copy and forgetting to update the other caused inconsistent behavior.
Example: One test in packages/api-client used the API client and different headers than the e2e server-side test; runtime differences led to mismatched assertions.
Mitigation:
- Consolidate canonical e2e tests in one place (prefer apps/api e2e that hits the server directly), keep api-client tests focused on client behavior.
- If duplication is unavoidable, keep test expectations aligned and use shared helpers/mocks.

Race conditions in setup/cleanup (parallel tests)

Problem: Creating projects/environments concurrently across parallelized test jobs (or even within same job) created timing-based failures.
Mitigation:
- Make tests idempotent and isolate data per test.
- Use unique slugs per test run.
- Consider serializing tests that share DB resources or use test DB instances per worker.

Error/status-code semantics and client expectations

Problem: Tests rely on specific status codes (404 vs 401). If server-side code ordering changes (ownership check vs authorization check) the codes change and many tests break.
Example: If you move authorization first, a cross-project environment case might return 401 instead of 404.
Mitigation:
- Keep server semantics stable; if a change is required, update all tests and document the change.
- Add unit tests that assert the contract (e.g., cross-project environment -> NotFound).

CI flakiness from environment variables and headers

Problem: Some tests rely on headers like 'x-e2e-user-email'. If CI environment variable email differs or parallel runs reuse the same email, tests can mutate shared state.
Mitigation:
- Generate a test-specific email per run (e.g., alice+<run-id>@keyshade.xyz).
- Make test harness read and propagate headers from the same fixture source.

Commit noise and history (multiple tiny commits while fixing test)

Problem: WIP commits to enable tests, debug, revert, add files, remove files created a messy branch and bigger diffs, increasing review overhead and merge conflicts.
Mitigation:
- Squash local fixup commits before opening PR or use interactive rebase to tidy commits.
- Keep PRs small and focused: one PR for enabling tests and fixtures, another for refactors.

Accidental API contract changes vs test fixes

Problem: When tests fail, there’s a temptation to change server code to make tests pass instead of fixing tests. That can mask real bugs or break clients.
Mitigation:
- Prefer fixing tests/fixtures if the server behavior matches documented contract.
- If server needs change, write unit tests to cover new contract and get reviewer approval.

Concrete, practical steps I recommend next

Add a small checklist to PR templates for test-enabling PRs:
- Confirm unique slugs in fixtures
- Confirm actor authorities are set explicitly
- Run full test suite locally (or run the specific e2e group) before pushing
- Clean up any temporary test files
Add a helper in tests for deterministic unique slugs and user emails to avoid collisions in CI.
Consider adding a lint/CI guard that fails the build when "tmp-*.spec.ts" or "debug" test files are found in the commit.
If you expect future merges to touch the same big e2e files, coordinate branches or split PRs (e.g., create "enable tests" PR and a follow-up "refactor fixtures" PR).

Solution

Here is a summarized plan of my solution:

Lock the contract with unit tests by adding unit tests that assert parseProjectEnvironmentsDBOperation rejects cross-project environment slugs with NotFound. This prevents accidental server reordering from changing semantics. Make tests deterministic by ensuring every test explicitly creates its fixtures (workspace, project, environment) and set unique slugs per run, which will explicitly grant the required authorities to the test actor in success-path tests.

Prevent accidental temporary test files by adding the pre-commit hook above and a CI check to refuse pushes that include tmp/debug test files. Keep good merge strategies and PR hygiene by keeping PRs small and focused (enable tests separately from refactors). Make sure to rebase frequently and squash debug commits and add a PR/TODO checklist that requires: unique slugs, explicit authority setup, no temporary files, and local e2e run before requesting review. Consolidate test suites by deciding canonical location for workspace-role e2e tests (prefer apps/api e2e) and remove duplicates, or centralize test helpers. Here is my final PR solving the issue over a month:

PR #1156

Conclusion

In conclusion, Keyshade’s clean, well-structured codebase and deterministic tests make development faster, safer, and more collaborative. Clear intent, consistent conventions, and reliable CI build trust between maintainers and contributors—enabling focused changes, quicker reviews, and confident releases.

Fixing CLI Error Handling: A Deep Dive into Keyshade's WebSocket Communication Bug

Johnny Santamaria — Tue, 12 Aug 2025 15:00:00 +0000

“If you build software, every error message is marketing” - Jason Fried, Rework

Introduction

In today’s fast-paced software development landscape, managing credentials and sensitive configurations securely is more critical—and more challenging—than ever. Dispersed secrets, manual processes, and unplanned storage not only slow teams down but also increase the risk of breaches and operational errors. Keyshade addresses these challenges head-on with a centralized, automated approach to credential and configuration governance. Purpose-built for engineering teams, DevOps practitioners, startups, and open-source maintainers. Keyshade streamlines secret management while enhancing security, collaboration, and efficiency.

Keyshade is an advanced platform for centralized credential and configuration governance, designed to safeguard sensitive data, eliminate human error, and streamline operational complexity. By automating secret management, it fortifies security posture, amplifies cross-team collaboration, and liberates developers from the inefficiencies of fragmented secret storage. Its primary constituency—engineering teams, DevOps practitioners, early-stage startups, and open-source maintainers—adopts Keyshade to mitigate secret proliferation, minimize security liabilities, and embed robust secret orchestration seamlessly into their development pipelines. Users gravitate toward Keyshade for its ability to render secret management secure, intuitive, and collaborative, without imposing friction upon established workflows or toolchains. Below is a flow of control diagram discussing how users would typically use the product:

The user can send commands via keyshades terminal or on keyshades web app
On the web, the user can sign up, and initialize projects within workspaces
- Users can do the same in the CLI
Once the a project is made a http request is made to the API websocket service
The websocket service then retrieves that data to put in the Keyshade’s database to save the data for future use

Keyshade is currently in alpha and not live so there really are not any users as of posting this.

The issue

I partnered with a member of the Computing Talent Initiative with the task of solving issue 998 in Keyshade for a Summer Open Source Experience sponsored by . The main problem we noticed was that the Keyshade command line interface’s “keyshade run” command fails to give a proper error that users can understand. We would want to extract an error message and output it to the user. The issue is a bug that seems urgent to solve right now for users to understand errors they might make when working with Keyshade’s CLI, which is the main feature of Keyshade. Here are some key locations in the codebase we found out to be useful in solving the issue:

run.command.ts

Handles CLI logic for running commands, including WebSocket client interactions
Within a CLI folder

change-notifier.socket.ts

Contains the handleRegister method, which processes the register-client-app event
Handles authentication, authorization, and client registration logic
Emits the client-registered response to the client and logs errors
Within an API folder

run.types.d.ts

ClientRegisteredResponse describes the shape of the response sent by the server after registration (success, message)
Ensures type safety for client-side code handling server responses
Within a CLI folder

Here are steps regarding how we reproduced the problem before finding the important locations in the codebase

Steps to reproduce the behaviour:

Create a project on keyshade
Configure your CLI to tap into this project
Delete the project
Use keyshade run <> command
See the error

Expected behaviour

We would want the CLI to display what actually went wrong. The API should send the formatted message to the CLI. The CLI should print the error.

Keyshade’s Codebase

Before delving into the problem itself, and after pinpointing the critical components, we took a step back to develop a comprehensive understanding of the codebase’s architecture. Keyshade’s backend is architected with NestJS (a progressive Node.js framework) and TypeScript, leveraging Prisma as its database ORM, interfacing with a PostgreSQL relational database. It also integrates Redis for high-performance caching and publish/subscribe messaging, as well as Socket.IO for real-time, bidirectional communication between its command line interface and application programming interface.

On the frontend, the stack comprises React, TypeScript, JavaScript, and Next.js, presumably to power a performant and SEO-friendly web interface. Development workflows are streamlined through Nx (for monorepo orchestration), ESLint and Prettier (for code quality and style enforcement), alongside the TypeScript compiler for rigorous static type checking.

For security, Keyshade employs Elliptic Curve Cryptography for robust public-key encryption, JWT for token-based authentication, and API key authentication for service-to-service communication. The infrastructure is containerized via Docker and designed with cloud deployment in mind. A high-level system diagram encapsulates the overall architecture of the codebase as follows:

Here is a use case, using the components in the diagram to give you a better understanding of it.

Use Case: Developer Fetching Live Configuration Updates

Step 1: Developer Initiates CLI Command

keyshade run \--workspace my-company \--project web-app \--environment production

What happens: The developer runs the CLI command to start their application with live configuration updates from Keyshade.

The CLI documentation can be found [here](https://docs.keyshade.xyz/getting-started/installing-the-cli)

Step 2: CLI Processes Command (run.command.ts)

The run.command.ts file:

Parses the command arguments (workspace, project, environment)
Reads the local configuration file to get API credentials
Validates the required parameters using types from run.types.d.ts
Prepares to establish WebSocket connection for live updates

Step 3: WebSocket Connection Establishment

The CLI creates a websocket connection:

const socket = io('ws://api.keyshade.xyz/change-notifier')

What happens: The client connects to the NestJS backend's WebSocket gateway (change-notifier.socket.ts).

Step 4: Client Registration Request

The CLI emits the registration event:

socket.emit('register-client-app', { workspaceSlug: 'my-company', projectSlug: 'web-app', environmentSlug: 'production'});

What happens: This triggers the handleRegister method in change-notifier.socket.ts

Step 5a: Authentication

const userContext \= await this.extractAndValidateUser(client)  
Extracts API key from WebSocket headers (x-keyshade-token)

Queries PostgreSQL database via Prisma to validate the API key
Verifies user account is active

Step 5b: Authorization checks

await this.authorizationService.authorizeUserAccessToWorkspace({...})await this.authorizationService.authorizeUserAccessToProject({...})await this.authorizationService.authorizeUserAccessToEnvironment({...})

Checks if user has READ_WORKSPACE, READ_VARIABLE, READ_SECRET permissions

Validates access to the specific project

Confirms environment access rights

Step 5c: Client Registration

await this.addClientToEnvironment(client, environment.id)  
Creates entry in PostgreSQL mapping socket ID to environment ID

Adds client to Redis set for real-time notifications

Step 6: Success Response

What happens: Server sends confirmation back to CLI using the ClientRegisteredResponse interface from run.types.d.ts.

Step 7: Configuration Fetching

After successful registration:

CLI fetches initial configuration values (secrets, variables)
Application starts with these environment variables
CLI continues listening for real-time updates

Step 8: Real-Time Updates (Ongoing)

Another developer updates a secret value through the web interface.  

await this.redisSubscriber.subscribe(CHANGE\_NOTIFIER\_RSC, this.notifyConfigurationUpdate.bind(this))  
The web interface publishes change to Redis channel

All API instances receive the notification

this.server.to(clientId).emit('configuration-updated', data)

Server identifies all clients registered for that environment

Sends update notification to active CLI instances

CLI receives the configuration-updated event

Updates local environment variables

Application continues running with new values

Challenges

One specific technical challenge I encountered while attempting to solve issue 998 was figuring out why my changes in change-notifier.socket.ts were not being reflected when I tested them through the CLI. Despite updating the logic inside the Socket.IO handlers, the CLI behavior remained unchanged, and it was unclear whether the API changes were even being registered. Here is an attempt to solve this challenge and other challenges I encountered.

So I tried debugging the issue after reproducing the issue and found the debug information for keyshade issue 998

@johnny603 ➜ /workspaces/keyshade (teamAttempt/#998) $ DEBUG=* keyshade run "echo 'test'"INFO: Checking API key validity...INFO: API key is valid!INFO: Connecting to socket...socket.io-client:url parse wss://api.keyshade.xyz/change-notifier +0mssocket.io-client new io instance for wss://api.keyshade.xyz/change-notifier +0mssocket.io-client:manager readyState closed +0mssocket.io-client:manager opening wss://api.keyshade.xyz/change-notifier +0msengine.io-client:socket creating transport "websocket" +0msengine.io-client:socket options: {"path":"/socket.io/","agent":false,"withCredentials":false,"upgrade":true,"timestampParam":"t","rememberUpgrade":false,"addTrailingSlash":true,"rejectUnauthorized":true,"perMessageDeflate":{"threshold":1024},"transportOptions":{},"closeOnBeforeunload":false,"autoConnect":false,"extraHeaders":{"x-keyshade-token":"ks_a3adff2a9db8bf3c42fc817880ac4125983b83c171f0d04d"},"transports":[null],"hostname":"api.keyshade.xyz","secure":true,"port":"443","query":{"EIO":4,"transport":"websocket"},"socket":{"binaryType":"nodebuffer","writeBuffer":[],"_prevBufferLen":0,"_pingInterval":-1,"_pingTimeout":-1,"_maxPayload":-1,"_pingTimeoutTime":null,"secure":true,"hostname":"api.keyshade.xyz","port":"443","transports":["websocket"],"_transportsByName":{},"opts":{"path":"/socket.io/","agent":false,"withCredentials":false,"upgrade":true,"timestampParam":"t","rememberUpgrade":false,"addTrailingSlash":true,"rejectUnauthorized":true,"perMessageDeflate":{"threshold":1024},"transportOptions":{},"closeOnBeforeunload":false,"autoConnect":false,"extraHeaders":{"x-keyshade-token":"ks_a3adff2a9db8bf3c42fc817880ac4125983b83c171f0d04d"},"transports":[null],"hostname":"api.keyshade.xyz","secure":true,"port":"443"},"readyState":"opening"}} +0msengine.io-client:socket setting transport websocket +6mssocket.io-client:manager connect attempt will timeout after 20000 +8msengine.io-client:socket socket receive: type "open", data "{"sid":"qxJTaYpvOE4a6kIaAAB8","upgrades":[],"pingInterval":25000,"pingTimeout":20000,"maxPayload":1000000}" +756msengine.io-client:socket socket open +0mssocket.io-client:manager open +755mssocket.io-client:manager cleanup +0mssocket.io-client:socket transport is open - connecting +0mssocket.io-client:manager writing packet {"type":0,"nsp":"/change-notifier"} +1mssocket.io-parser encoding packet {"type":0,"nsp":"/change-notifier"} +0mssocket.io-parser encoded {"type":0,"nsp":"/change-notifier"} as 0/change-notifier, +0msengine.io-client:socket flushing 1 packets in socket +1msengine.io-client:socket starting upgrade probes +1msengine.io-client:socket socket receive: type "message", data "0/change-notifier,{"sid":"OpbhAQ0UYcfklhBoAAB9"}" +249mssocket.io-parser decoded 0/change-notifier,{"sid":"OpbhAQ0UYcfklhBoAAB9"} as {"type":0,"nsp":"/change-notifier","data":{"sid":"OpbhAQ0UYcfklhBoAAB9"}} +251mssocket.io-client:socket socket connected with id OpbhAQ0UYcfklhBoAAB9 +251mssocket.io-client:manager writing packet {"type":2,"data":["register-client-app",{}],"options":{"compress":true},"nsp":"/change-notifier"}+251mssocket.io-parser encoding packet {"type":2,"data":["register-client-app",{}],"options":{"compress":true},"nsp":"/change-notifier"} +0mssocket.io-parser encoded {"type":2,"data":["register-client-app",{}],"options":{"compress":true},"nsp":"/change-notifier"} as 2/change-notifier,["register-client-app",{}] +0msengine.io-client:socket flushing 1 packets in socket +1mssocket.io-client:socket draining queue +1msengine.io-client:socket socket receive: type "message", data "2/change-notifier,["client-registered",{"success":false,"message":{"response":{"message":"{\"header\":\"Workspace not found\",\"body\":\"Workspace undefined not found\"}","error":"Not Found","statusCode":404},"status":404,"options":{},"message":"{\"header\":\"Workspace not found\",\"body\":\"Workspace undefined not found\"}","name":"NotFoundException"}}]" +261mssocket.io-parser decoded 2/change-notifier,["client-registered",{"success":false,"message":{"response":{"message":"{\"header\":\"Workspace not found\",\"body\":\"Workspace undefined not found\"}","error":"Not Found","statusCode":404},"status":404,"options":{},"message":"{\"header\":\"Workspace not found\",\"body\":\"Workspace undefined not found\"}","name":"NotFoundException"}}] as {"type":2,"nsp":"/change-notifier","data":["client-registered",{"success":false,"message":{"response":{"message":"{\"header\":\"Workspace not found\",\"body\":\"Workspace undefined not found\"}","error":"Not Found","statusCode":404},"status":404,"options":{},"message":"{\"header\":\"Workspace not found\",\"body\":\"Workspace undefined not found\"}","name":"NotFoundException"}}]} +262mssocket.io-client:socket emitting event ["client-registered",{"success":false,"message":{"response":{"message":"{\"header\":\"Workspace not found\",\"body\":\"Workspace undefined not found\"}","error":"Not Found","statusCode":404},"status":404,"options":{},"message":"{\"header\":\"Workspace not found\",\"body\":\"Workspace undefined not found\"}","name":"NotFoundException"}}] +261msERROR: Error registering to API: [object Object]

I came to the conclusion that the client is sending an empty object {} in the registration message, but the server expects a ChangeNotifierRegistration object with workspace, project, and environment slugs. This results in undefined values being passed to the authorization check, causing the “Workspace undefined not found” error. The code correctly creates the registrationData object with workspaceSlug, projectSlug, and environmentSlug

But the debug output shows it’s sending an empty object {}

This suggests that the data parameter being passed to connectToSocket has undefined values for workspace, project, and environment. Our team reached out to the issue maintainer radjip-b with a question regarding our process of the issue so far.

This small technical challenge was solved.

Solution

The solution for Keyshade’s issue 998 took five and a half weeks to complete during the eight-week internship. In the remaining time, we addressed additional related issues, including issue 989 and 988, which involved adding Slack and Discord integration documentation, as well as contributing to PR 1093.

Changes Introduced by PR #1093:

Slack Integration Documentation: A brand-new “How It Works” modal was added to the Keyshade platform’s UI, providing a visual, step-by-step guide—including screenshots—on how to set up Slack integration. This includes selecting Slack in the Integrations tab, entering credentials (signing secret, Bot token, channel ID), and testing with sample secrets and variables.
Modal Improvements & Developer Feedback: The modal component underwent adjustments—such as replacing a bare <DialogClose asChild /> (which caused rendering issues) with a proper close button, and revising instructions for better clarity (“Bot Token from Slack”). Additionally, an unused function (_handleClose) was cleaned up and textual instructions were streamlined.
Discord Integration Documentation: The scope of the PR was expanded to include documentation for Discord integration as well—hence the updated title to “feat: added slack and discord integration docs.”
Internal Routing and Formatting Refactor: Responding to reviewer feedback, the team refactored the integration documentation routing logic—removing the DOCS_SLUG_MAP, simplifying routing, standardizing layout for project creation and event triggers, and cleaning up the formatting and positioning of images.
Lint Fixes: Minor linting errors (such as extraneous braces in index.tsx) were corrected to ensure code consistency and avoid build issues.

Summary Table

Area	What Changed
Docs & UI	“How It Works” modal with guide and screenshots
Component Cleanup	Dialog close functionality fixed, redundant code removed
Discord Docs	Documentation for Discord integration added
Slack Docs	Documentation for Slack integration added
Documentation Routing	Routing logic simplified, layout standardized, visuals aligned
Code Quality	Linting issues resolved

Solution Description For Issue #998

Our team resolved a bug where the socket notifier and run command error handling would display [object Object] instead of a meaningful error message. The root cause was improper formatting of the error object before logging it to the user. The fix ensures that error messages are now properly stringified or formatted, providing clearer and more helpful feedback during CLI failures. To verify the fix, you can run the keyshade run command with an invalid configuration or project setup that triggers a connection error. This update significantly improves the developer experience by making error outputs more understandable.

We determined that only three files required direct changes to address the issue:

NestJS WebSocket Gateway – The server-side component uses NestJS’s @WebSocketGateway decorator with Socket.IO to handle real-time communication between the CLI and API server. The handleRegister method in change-notifier.socket.ts processes client registration requests and can return different error message formats depending on the failure type (authentication, authorization, or database errors).
CLI Command Handler – The run.command.ts file contains the client-side logic that connects to the WebSocket server and handles the registration response. This component already included robust error parsing logic capable of handling both string and object formats, but the TypeScript interface was preventing proper type checking.
Type Checker – The run.types.d.ts file was updated to allow both string and object types, since the server can send either format.

After the maintainer approved the fix, we made additional updates to the codebase by modifying two more files: changelog.md to document the changes, and package.json to bump the semantic version to 3.2.3.

What is in the file changes

These were changes found using git’s diff command

What has been changed can also be found here

changelog.md:

🐛 Bug Fixes* **api, cli:** ambiguous error messages on keyshade run failure ([#1087](https://github.com/keyshade-xyz/keyshade/issues/1087)) ([206dda7](https://github.com/keyshade-xyz/keyshade/commit/206dda702b3f3f0cabeb73e4f74dd2ccfc962631))

apps/cli/package.json: "version": "3.2.2-stage.2",
apps/cli/src/types/command/run.types.d.ts:

message: string | object // Allow both string and object types since server can send either

apps/cli/src/commands/run.commands.ts:

Accurate WebSocket URL Parsing:
The CLI’s socket connection logic now uses the full host from the base URL (via the URL class), ensuring reliable and correct WebSocket connections especially for complex URLs.

Registration Timeout Added:A 30-second timeout was introduced for socket registration. If the server does not respond within this window, users receive a clear error message and the process exits gracefully. This prevents hanging connections and improves user feedback.

Enhanced Error Message Extraction:The registration response error handling is now more robust: It intelligently extracts meaningful error messages whether the server responds with a string, an object, or a nested structure. Attempts to parse JSON error details for more informative output. Falls back to stringifying unknown error types, ensuring users always get useful feedback. All error messages are now logged in a consistent, user-friendly format.

Minor Cleanup:Removed a redundant blank line after the socket connection initiation for improved code readability.

apps/api/src/socket/change-notifier.socker.ts:

1. Improved Error Handling & Feedback The registration process now provides much more detailed error messages to clients. Instead of a generic error or raw message, errors are parsed, logged, and emitted in user-friendly formats.

2. Custom Authentication Logic for Socket Connections The previous decorator-based guards (@UseGuards(AuthGuard, ApiKeyGuard) and @RequiredApiKeyAuthorities) have been replaced by custom methods. New methods (extractAndValidateUser and convertToAuthenticatedUser) manually handle authentication and authority checks, mimicking guard behavior but allowing for more granular error handling and feedback.

3. API Key Authority Checks The code now checks if the API key has the required authorities (READ_WORKSPACE, READ_PROJECT, and READ_ENVIRONMENT) directly within the socket handler. If the user has ADMIN authority, individual checks are bypassed.

4. Enhanced Logging Connection logs now include the presence or absence of authentication headers. Errors during registration are logged with more context, including error types and constructors.

5. Dependency Injection Updates The SlugGenerator service is now injected into the class, preparing for potential future use or slug-related operations.

6. Cleaned-Up Imports Removed unused decorators and guards, reflecting the shift to manual authentication and authority verification.

7. User Context Construction The code now constructs a detailed user context object for socket connections, including IP address, workspace, and API key authorities, improving downstream authorization and personalization.

Testing the solution

While testing Keyshade, I encountered an issue that was easy to reproduce: create a project in Keyshade, configure your CLI to connect to it, delete the project, and then run the keyshade run <> command. This sequence would trigger an unclear error message, leaving the cause ambiguous. The expected behavior was for the CLI to clearly display the actual problem, with the API sending a properly formatted error message that the CLI would then print for the user. After applying the solution, the CLI behaved as intended, providing a clear and informative error message, making the debugging process much smoother.

Another testing approach involved building custom WebSocket test clients designed to send invalid authentication data, deliberately triggering error conditions to assess system resilience. We closely monitored server-side logs to ensure errors were correctly detected and handled, while also verifying that the client could parse various error message formats. This end-to-end validation—from CLI input, through WebSocket communication and API processing, to the final CLI response—confirmed the integrity and reliability of the entire flow. This comprehensive process not only confirmed the integrity and reliability of the entire system but also successfully closed issue #998 with our PR #1087.

Conclusion

We resolved a complex WebSocket bug by fixing authentication handling, type safety, and error messaging across the backend and CLI. They used systematic debugging, targeted fixes to three core components, and custom test clients to replace vague errors with clear feedback, improve authentication timeouts, and update integration docs—showcasing technical excellence, thorough testing, and a user-focused, collaborative approach.

DEV Community: Johnny Santamaria

🎃 Death Exhibition + Extension Roadmap

🕯️ Museum of the Dead

What I Built

Demo

Code

How I Built It

Why Does Open Innovation Matter?

1. The museum works with the lights off.

2. Data stays home.

3. Models are swappable — and that's a load-bearing feature.

My Agent Session

The Spirit of It

That's the moment the museum became a memorial instead of a demo.

Acknowledgment

What's Next

SlothMode: Building a Deliberately "Lazy" Focus Enforcer with Open-Source AI

Section 1: About the Project

What is SlothMode?

Why is it important?

Who is the typical user?

Screenshot

Users

Section 2: The Issue

What issue did I address?

Why is this important?

Key code locations

Supporting artifacts

Section 3: Codebase Overview

Tech stack

System diagram

Workflow: the 2:00 PM break

Section 4: Challenges

The challenge

Three self-directed attempts

Who I reached out to

Status

Section 5: Solution

What I did

Key technologies referenced

How I verified it

Code snippet

Pull request

How I Built It

Why Does Open Innovation Matter?

My Agent Session

The Lazy Payoff

Prize Categories

References

[Boost]

Shoebox — a private receipt & GST sorter for Alberta freelancers, powered by a local Gemma 3

CreatorDesk: a self-hosted approval loop for commission artists

What I Built

Demo

Code

How I Built It

Why Does Open Innovation Matter?

My Agent Session

My First User

What a contributor, mentor, or early user can help with right now:

Footnote

MudMap: A Voice-First Trail Logger That Works Without a Signal

MudMap: A Voice-First Trail Logger That Works Without a Signal

What I Built

Demo

Code

How I Built It

Challenges

Challenge 1: The cryptic 25.0.4.1 build failure

Challenge 2: The asset script couldn't find pmtiles

Challenge 3: The disk-space wall

What these three have in common

Why Does Open Innovation Matter?

CscGrader and generateRubrics: Open-Source Tools for Faster CS Grading

What I Built

Demo

studentNames.txt contains a list of all students in the class

The submissions folder contains one subfolder per student

The assignment profile tells CscGrader what to look for and how to run it

Code

Challenge 1: The cryptic `25.0.4.1` build failure

Challenge 2: The asset script couldn't find `pmtiles`