DEV Community: Orestis Ioannou

Why Saying ‘Hi’ to Our OpenCode Agent Used 177,000 Tokens

Orestis Ioannou — Thu, 29 Oct 2026 08:02:29 +0000

At Bold.org, people use our internal company agent to investigate production issues, look up information, and put together reports. It runs on OpenCode, connected to the systems where that information lives. I have gradually added more integrations as people found useful things to do with them, and for the most part that has worked well.

Over the last few days, though, we started reaching our AI usage limits much sooner than I expected. Astra is more expensive, and usage was growing steadily, so some increase made sense. But token consumption was climbing much faster than the number of sessions or automations could explain, even though I understood Astra was supposed to use fewer tokens to complete tasks. That was what bothered me. Was it spending more on reasoning? Were people keeping conversations open for too long?

So naturally, I asked an agent to help me investigate the agent.

Start with the database

Our hosted deployment was running OpenCode 1.18.31, with session history and per-call usage in SQLite. I asked for queries covering September 14 and 15, ran them against the hosted database, and brought the results back to the conversation. That let me look past the dashboard totals and see which sessions and model calls were consuming the tokens.

One Astra conversation had recorded 50.4 million tokens across 121 model calls, with 49.9 million in cache reads. Seeing that, I leaned toward long conversations as the explanation. The agent was repeatedly reading a large context, not generating millions of tokens of answers. Reasoning effort did not explain it either: the inspected sessions used the model default, and their reasoning tokens were a small part of the total.

But following sessions back to their first calls complicated that explanation. An Astra investigation had started with about 177K input tokens, before it had gathered any results. Other sessions started at a similar size. There was already a lot in the request before the conversation had a chance to become long.

I wanted to see how little I could ask and still get that overhead, so I opened a fresh Astra conversation and sent “hi.” It showed 177,444 tokens. No previous messages, no tool calls, nothing to investigate. At that point, blaming long conversations was getting difficult.

What was actually in the context?

My next suspicion was our instructions and skills. I had added plenty of those too, and perhaps I had been less economical with context than I thought. I used OpenCode TokenScope on the fresh session and shared its report with the agent to see what accounted for the input.

TokenScope separates recorded usage from estimates of the content behind it. The report confirmed 177,431 input tokens for a 13-token response, but the visible conversation was tiny. The skill catalog was estimated at about 3,400 tokens, and the tool definitions it could inspect at about 5,800. Those numbers did not explain the request, and cutting a few paragraphs from my instructions was clearly not going to get us there.

The important detail was what the report could not see: its tool inventory in our setup omitted the full MCP definitions. I asked the agent on the hosted machine to inspect those catalogs separately. That led to Meta Ads: 97 tools, with approximately 595K characters of descriptions and parameter schemas, accounting for about 72% of the measured MCP definition text.

This was not advertising data the agent had retrieved. It was the documentation needed to call the tools, supplied even when nobody was asking about advertising. To confirm it, I disabled Meta Ads and tried the greeting again. It dropped from 177,444 to 54,181 tokens without changing the model, reasoning effort, or skills. One integration explained roughly 69% of the original request.

I had been thinking about the cost of using those tools. I had not paid enough attention to the cost of making all of them available, all the time.

I did not want fewer tools

Disabling Meta Ads answered the question, but it was not a fix I wanted to keep. People used that integration. The problem was that a backend investigation had to carry all its definitions too, through request after request. Caching made that cheaper, but left the context occupied. I had also been noticing more frequent compactions, slower responses, and worse answers; this gave me another reason to care about the overhead beyond the usage limits.

What I wanted seemed straightforward: keep the tools available, but let the agent look up how to use them when it actually needed them. I asked whether OpenCode supported that. It turned out our installed version already did, through experimental Code Mode.

I enabled OPENCODE_EXPERIMENTAL_CODE_MODE=true on the OpenCode service, restored Meta Ads, and tried “hi” again. This time it used 13,274 tokens. The full set of integrations was back, but the starting context was roughly 92% smaller than before. I had not needed to remove tools or rewrite our skills after all.

The idea behind Code Mode is to let the model write code that calls tools, combines operations, and filters results, rather than bringing every intermediate response back into the conversation. OpenCode pairs that with on-demand discovery: the model starts with a compact catalog, looks up the signatures it needs, and runs JavaScript through execute in a confined runtime. MCP still handles the underlying calls; the full catalog no longer needs to accompany every request.

The 92% is a reduction in the greeting's recorded usage, not a claim about our final bill. An actual investigation will still load definitions and results as it works. The difference is that those tokens can now go toward the task somebody asked it to do.

A better default

OpenCode 2 makes Code Mode the default for MCP servers, which explains a slightly embarrassing part of this: I had been using its beta locally for at least a month. My own integrations were not accumulating the same upfront overhead, so my daily engineering setup had given me little reason to suspect it. The hosted agent is still on V1, where I needed to opt in.

This is not unique to OpenCode either. Claude Code has MCP Tool Search, and Codex defaults to deferred MCP discovery on supported configurations. The mechanisms differ, but on-demand discovery is already a normal default across several harnesses. I was discovering why it mattered in a deployment that had not enabled it yet.

I am glad the fix turned out to be straightforward, and that I could keep the MCP integrations people were already using instead of disabling them to bring usage down.

A note on how this article was written: AI helped me reconstruct the investigation from my OpenCode sessions and edit this article. The experience, decisions and opinions are mine.

I'm Orestis, VP of Technology at Bold.org. On the side I build clssrm, simple booking software for small studios. More of my writing is on oioannou.com.

AI Promised Less Work. I Have Never Worked More.

Orestis Ioannou — Wed, 28 Oct 2026 08:03:37 +0000

The pitch was that AI would give us our time back. Write less boilerplate, delegate the boring parts, go home earlier.

I am not complaining about the tools. I use agents for all of my engineering work now, and I wrote about how that setup evolved and why I still read the code. I can investigate and ship things that would not have been worth starting two years ago.

But somewhere this year I noticed that I am working more than I ever have, and that it feels heavier than it used to. Not because any single task is harder. Because there are so many more of them, all at once.

This post is me trying to understand that, with some data from my own GitHub history.

The old load and the new load

As a full-stack engineer, a normal week for me meant two or three serious tasks. A feature, a tricky bug, maybe a migration. Each one needed real focus, but I could hold one of them in my head at a time, finish it, and move to the next.

Today my week looks completely different. I have several agents running in parallel: one implementing a change, one investigating a production question, one working through a review, another doing research I will need later. None of them needs me all the time. Every one of them needs me sometimes.

That "sometimes" is the problem.

Each agent eventually comes back with a question, a plan to approve, a diff to read or a result to make sense of. Responding means switching contexts: reloading a different problem, a different repository and a different set of decisions.

Two or three serious tasks a week was tiring in a familiar way. Working with many agents is tiring in a new way. My attention is spread across more problems, and I find moving between them much more mentally draining.

What my GitHub history shows

I pulled my GitHub contribution calendar for every month from January 2019 to today and marked my roles at Bold.org, which I joined in January 2020: Full Stack Engineer until August 2022, Principal Engineer until August 2025, and Vice President of Technology since then. These are GitHub's contribution counts, including private work, so they mix commits, pull requests, reviews and issues. It is a rough signal of activity, not a measure of value. 2019 is an incomplete baseline: my job before Bold.org used a self-hosted GitLab, so GitHub only shows what I did outside it. But the shape is hard to ignore.

The first part of the curve is what I expected:

My busiest coding years at Bold.org were the early ones. 2020 was my highest year, with 2,711 contributions. 2021 dropped to 1,839 and 2022 to 1,498.
The Principal Engineer years were flat and lower. From August 2022 to July 2025 I averaged 114 contributions a month, against 173 as a Full Stack Engineer. My lowest year was 2023, with 1,189.
The first months as VP of Technology looked the same. From August to December 2025 I averaged 111 a month, and December 2025 was my quietest month since joining Bold.org, with 50.

As a Principal Engineer, more of my work went into architecture, deep dives into concepts, setting up projects and planning, rather than fixing bugs or iterating on features. Becoming VP of Technology continued that shift, with even less feature work expected. At least until AI came along.

Then 2026 happened:

I am back at my Full Stack Engineer pace. By October 27 I am at 2,102 contributions, already more than any full year from 2021 to 2025. Over the same period in 2020 I had 2,371, so 2026 is close to my busiest year but not above it.
August 2026 is the busiest month in the whole timeline. 427 contributions, against a previous high of 319 in April 2020.
My normal day moved back up. The median weekday was 3 or 4 contributions every year from 2022 to 2025. In 2026 it is 7, close to the 8 I had in 2020.
Fewer weekdays without GitHub activity. Through October 27, I had 18 weekdays with zero contributions, compared with 40–46 across each full year from 2022 to 2025. These are days without recorded GitHub activity, not days off work.
Heavy days became routine again. From 2022 to 2025 I had exactly one day per year with 24 or more contributions. In 2026 I already have 16, almost the 17 I had in all of 2020.

Almost all of this is private work. The dashed line in the chart is public contributions only: it was near zero from 2021 to 2025 and came back this year, with 191 so far, but that is still less than a tenth of 2026.

The twist is the role. In 2020 I was a full-stack engineer doing feature work full time, and this is roughly what that looked like on GitHub. Today I lead engineering at Bold.org and do far less feature work than I used to. On paper my graph should look like my Principal Engineer years, or lower. Instead it is back where it was when writing code was my whole job.

What changed is how much work feels worth taking on. When coding is cheap, I can fix bugs I would previously have left alone, improve patterns across the codebase, take on new projects and turn more ideas into something real. Much of this is work I would never have considered before. Agents did not just make my existing workload faster to finish. They made it easy to give myself a much bigger one.

Bursts, then burnout

Looking at the monthly totals, 2026 is not one steady climb. It is a series of bursts.

January and February were the first clear burst, with 277 and 233 contributions. This is when I started working seriously with agents, and when the observability experiment that became our internal company agent took off. Every week revealed something new I could suddenly delegate, and I wanted to try all of it.

March to June generally winds down, from 203 contributions in March to 77 in June. But April was a smaller burst along the way, another month when I had ideas and acted on them. I think improvements in the models were part of that renewed energy. Its monthly total of 187 is below January to March, but it includes a 29-contribution day and 68 public contributions, my highest public total of the year.

August is the biggest burst of all. Coming back from my summer break, the calendar shows a full week of zero contributions in early August. Then every weekday from August 10 to August 27 was between 14 and 43. August ended at 427 contributions, the busiest month in the whole timeline. September stayed high at 301.

Every cycle starts the same way. A new model or a new workflow makes something possible that was not possible last month, and my head fills with ideas. Because implementation is now cheap, there is no natural friction stopping me from pursuing all of them at once. So I do. I start more agents, open more threads and say yes to more experiments.

That works for a few weeks. Then the reviewing and context switching catch up with me, and I pull back to a more stable flow. I stay there until something exciting comes along, or until a holiday recharges me enough that I go straight back into another burst.

I count three of these cycles since January.

At the end of a burst, I feel tired of AI. I do not want to start another agent or think of more ideas. I do my normal work, but I stop chasing the extra projects and possibilities because I do not have the mental bandwidth for them.

The low season is not low anymore

The part that surprised me most is not the peaks. It is how much I am doing between them.

March did not feel like the bursts in January, February or August, but I was still doing substantial work. It had 203 contributions, against my Principal Engineer average of 114 a month. I was still acting on ideas, but that level of activity no longer felt like an exceptional push. Even when I think I am taking it easy, there are still agent results to work through, reviews to finish and small things that are now "just a quick prompt".

June was genuinely low, with 77 contributions, comparable to my quieter Principal Engineer months. Maybe that was when all the tiredness caught up with me. Not every quiet month is higher than before. But outside the bursts, I still find myself taking on more work than I used to. That is what I mean by the low season not feeling low anymore.

Where that leaves me

AI made room for work I would never have considered before. I like being able to fix more things and turn more ideas into something real. But the capacity to implement those ideas has grown much faster than my capacity to think about them. Eventually I stop wanting to start another agent and go back to just doing my normal work. I have not figured out a balance. For now, I am noticing that being able to do more keeps turning into doing more. And I am tired.

A note on how this article was written: AI helped me pull and analyze my GitHub contribution data, build the chart and draft this article. The experience, decisions and opinions are mine.

I'm Orestis, VP of Technology at Bold.org. On the side I build clssrm, simple booking software for small studios. More of my writing is on oioannou.com.

I Don't Write Code Anymore. I Still Read It.

Orestis Ioannou — Tue, 27 Oct 2026 08:06:49 +0000

Apparently, coding is solved and AGI is just around the corner. Meanwhile, I am asking the latest model why it is checking, yet again, whether an integer is a number.

I went back through two months of my OpenCode conversations and Plannotator feedback to understand that pattern. Across different projects and tasks, I kept pushing in the same direction: use the established approach, remove the unnecessary transformation, leave unrelated code alone, reconsider whether we need this behavior at all.

These conversations were with OpenAI's latest models: Sol 5.6, then Astra.

Code is solved. Bugs are not.

For my day-to-day work, producing an implementation is no longer the bottleneck. I can describe a change, discuss it with an agent, and get working code across the relevant parts of a project. What remains difficult is establishing that the change solves the actual problem without introducing behavior we did not need or bugs we did not have.

That is why I keep asking for simpler code.

Every additional condition introduces another path to reason about. Every additional representation creates another opportunity for the same information to disagree. Every unrelated refactor changes more behavior that must be understood and verified alongside the requested work.

The cost of generating those additions is now tiny. Their cost continues long after the current task is finished.

This matters especially in an established codebase. There are already patterns for how the system handles things, with behavior that callers rely on. An agent can follow those patterns, or it can introduce a second way of doing something because that looks convenient within the current task. The second approach may work in isolation while creating another area where bugs can develop.

I can see the appeal of frameworks such as Effect for agent-written code: explicit errors and dependencies, plus standard tools for concurrency and resource management, leave fewer mechanisms to invent from scratch. But not every codebase uses Effect or will migrate to it, and a well-typed, well-structured change can still be unnecessary.

A poorly designed function introduced today also becomes context for the next agent. Other agents start using it. Later, another agent changes it without knowing why it worked that way in the first place. An arbitrary choice has become a constraint on future work, and we now have to figure out which parts reflect an actual requirement and which parts were just the previous model's interpretation.

The next agent will make it work, with tests. But work how? The way we want, or the way it infers we want from the code it inherited?

The model can make its own complexity necessary

The most frustrating pattern starts with a design choice that was never a requirement.

The agent makes that choice, encounters the consequences, and writes more code to accommodate them. By the time I read the result, each addition has a reasonable explanation. Given the earlier decision, we need the extra handling. Given the extra handling, we need more tests. Given the new behavior, we need to account for another case.

Take an integer read from the database. The column is an integer, the API contract describes it as a number, and the Kysely query type agrees. Yet I still see agents repeatedly checking whether that value is a number as it passes through internal code. We have already established where it comes from and what it can be. Why does every use treat that guarantee as unknown again?

The check itself may be completely correct. It may never throw or have any meaningful performance impact. But what does it buy when the value already comes through a path that guarantees its type? It is treating the database as though we routinely bypass its constraints and manually create invalid values.

I see the same thing with locking and concurrency mechanisms. Before establishing whether an operation can actually run concurrently, the agent starts coordinating it. Then we need rules for acquiring and releasing the lock, recovering from failure, and testing those rules. If our execution model does not require that coordination, we have introduced new ways to fail while protecting against a problem we do not have.

I can ask the agent to improve each check or lock. The more useful question is whether the operation needs them at all.

We already had a principle for this: avoid premature optimization. When we wrote the code ourselves, the effort of implementing each idea also limited how much speculative work we could add. Agents can deliver the optimization, defensive branches, and supporting tests before we have established a need for any of them.

Even when it all works, that is extra cognitive load during review, extra context for the next agent, and extra tests in the PR and CI. We are generating code much faster than before, so more of the effort goes into deciding which of those additions are justified.

This is why asking for simplification sometimes takes several rounds. The agent removes the specific thing I pointed at, but the reasoning that produced it remains. I find another instance and have to explain the same objection again.

An impressive implementation of an unnecessary requirement still leaves us owning the unnecessary requirement.

Passing checks does not answer that question

Typechecking, linting, tests, and screenshots are essential to this workflow. They let agents catch and fix a huge amount without involving me. But a clean result does not establish that all the new code belongs in the change.

A typechecker checks the types we chose. A test checks the behavior we specified. If the agent introduced an unnecessary constraint and then wrote code and tests around it, those checks can all agree.

Even adversarial tests can reinforce this. Supply a string to the internal function that receives our database integer, and the test confirms that the defensive check handles it. We have proved that an invented error path works, without establishing that the real data path needs it.

Screenshots can help an agent verify that the UI is visually consistent. It can still consistently display the wrong information or take the user through the wrong flow. Looking right is another useful check, but it does not settle whether we built the right thing.

That is also why I keep asking what a test actually proves. Does it protect behavior that matters to a user? Would it catch a meaningful regression? Or would it mostly object if we reorganized the implementation?

One agent interpreted those questions as a preference against mock-heavy tests. I had to correct it: the issue was whether the test had meaning. Turning that judgment into a rule about mocks missed the point.

The same applies to complexity. “Fewer helpers” cannot substitute for understanding which responsibilities belong together. “More tests” cannot substitute for deciding which behavior is correct. These shortcuts are easy to put in an instruction file and easy to follow superficially.

I already put it in the prompt

I use skills, project instructions, and AGENTS.md. I keep them up to date, revisit them when I move to a new model, and check whether the agent's behavior matches the guidance. Maintaining that setup is already part of the work.

The questions about test value belong in a testing skill. But agents do not always follow those instructions, and I still have to check whether the tests reflect them.

While working on this article, I again asked an agent why it had added a helper instead of reusing or improving existing functions. Its answer:

The extra helper was my judgment call; repository guidance actually favors reuse.

That is why I struggle with “skill issue” or “fix your AGENTS.md” as a sufficient explanation. The guidance was already there. The agent could identify it and acknowledge that its implementation had gone another way.

There is a difference between making a principle available and applying it well. An agent can correctly explain why unnecessary complexity is undesirable and still introduce it in the next change. It can agree that a requirement is out of scope and later build something that depends on it.

More precise instructions help when I already know the specific mistake to prevent. But I often discover that mistake by reading the code. At that point, identifying the problem and writing the corrective prompt is the engineering work. Calling the resulting improvement better prompting does not explain how I could have skipped that judgment.

I want the agent to make routine implementation decisions independently. Specifying every branch and every boundary in advance would mean doing much of the implementation design myself. The useful delegation is letting it do that work, then being able to challenge the decisions that do not fit.

Long sessions can also lose agreed decisions. I asked for a command-line option to be removed, and an early compaction summary preserved that decision. Six compactions later, the option was back—with tests requiring it. I had also approved broader plans along the way, so compaction alone does not explain the regression. But the summaries had lost the explicit exclusion and eventually described the rejected behavior as intended. An earlier agreement was no guarantee of the final result.

That is the gap I keep encountering, even with the latest models and a workflow built around giving them useful context.

Why I still read production code

Plannotator makes this part of the workflow unusually smooth. I can annotate the exact part of a plan, diff, or explanation that bothers me, ask why it exists, and discuss a simpler approach. That feedback goes back to the agent with the relevant context. It makes it easy to reason about a change together and ask for another pass, instead of trying to describe code locations through a long prompt.

That matters because this is a recurring conversation, not a final approval button. I am very happy with how much faster I can investigate and implement things. Having a good way to challenge the result is part of what makes that speed useful.

I have been wrestling with this since Opus 4.5. I hoped newer models would make it substantially less necessary. There have absolutely been improvements, but not enough for me to confidently stop reading code intended for critical production use.

And trust me, I do want to stop reading code. I want this solved end to end. I just cannot treat it as solved yet.

Pet projects and fun experiments are a different tradeoff. If the consequences are small, I understand choosing to spend less time checking. But when someone says critical production code no longer needs to be read by a human, I think they are fooling themselves about what these tools can currently establish.

A note on how this article was written: AI helped me analyze my OpenCode conversations and Plannotator feedback, then draft and revise this article. The experience, decisions and opinions are mine.

I'm Orestis, VP of Technology at Bold.org. On the side I build clssrm, simple booking software for small studios. More of my writing is on oioannou.com.

Building an Internal Company Agent: 5 Things We Got Right and 5 I Would Do Differently

Orestis Ioannou — Sat, 24 Oct 2026 18:55:23 +0000

In my recent AI engineering setup article, I briefly explained how an observability project at Bold.org turned into an internal company agent.

The short version is that back in January 2026, Bold.org had structured backend events in ClickHouse, engineers could investigate them with SQL, and I wanted to see whether an agent could do the same. It worked, so I kept connecting more company systems. Within weeks, people across Bold.org were using it. It eventually became useful across engineering, product, support, operations, marketing, and growth.

That summary makes the path sound much cleaner than it was.

Several decisions helped the agent spread quickly. I also spent time solving problems I did not really have, left some important problems too late, and assumed things were obvious when they definitely were not.

This is not a universal guide to building agents. It is a practical retrospective from building one inside a real company: five things we got right and five I would approach differently today.

What this looked like in practice

The first workflow was straightforward: I gave the agent read-only access to structured backend events in ClickHouse so it could investigate production questions. I then connected more of the systems where people already worked, including Slack, Linear, PostHog, and Grafana.

The practical effect is bigger than a faster way to write SQL. Support can investigate many customer reports autonomously, without waiting for engineering or another team to provide context. For the workflows we target, investigations that previously took multiple hours and required finding the right access through three or four people can usually be completed in zero to one hour. The agent also automates parts of dashboard setup and analysis in PostHog and Grafana without requiring engineering to step in.

The system consumes billions of tokens per month. That makes usage analytics and cost awareness important, but it is also a useful signal that the tool is part of how people work rather than a demo that is occasionally opened.

I made one security boundary deliberately simple. The agent is gated behind Google sign-in for our company domain, and all of its remote access is read-only. Rather than relying on the model to be smart about when a change is safe, I did not give it permissions to write to remote systems in the first place.

What worked

1. Make usage visible early

By "visible," I mean visible inside the company.

It is tempting to build an internal tool privately, make it feel complete, write an announcement, and then ask everyone to use it. We did almost the opposite. The agent was used in shared Slack channels and connected to Linear, where people were already working and could see what it did.

This mattered more than any launch document.

Most people do not immediately know what an agent can do. A blank chat box does not explain its capabilities. Even a good list of examples still requires someone to translate those examples into their own work.

Seeing real usage solves that problem. Someone asks a question in Slack, the agent investigates it, and ten other people see both the result and the way the question was asked. A product manager notices that it can inspect a user flow. Somebody in support realizes it can help investigate a report. An engineer sees it gather context they were about to collect manually.

Each visible interaction becomes a small product demo based on an actual company problem.

Slack and Linear were important for another reason: they did not require people to adopt a new destination. Another internal web application would have asked everyone to remember that the tool existed, open it, and decide when to use it. In Slack and Linear, the agent appears where the question or task already exists.

The web interface is still useful, especially for private or longer work. But shared usage is what taught the company what was possible.

2. Connect the tools the company already uses

When I started this in January, MCP was still emerging and many of the systems we needed did not have useful servers. I closed the gap with small CLI tools, mostly thin wrappers around OpenAPI endpoints or existing SDKs, exposing only the operations needed for our workflows. They were intentionally boring: structured results, narrow permissions, proper error handling, and credentials kept away from the model.

The value was not just another action the agent could take. More access meant more company context: production data, tickets, analytics, dashboards, and operational information could be considered together. That gave the agent better evidence for its reasoning and made its answers more useful than anything it could infer from a single system.

3. Track how people actually use it

Usage analytics went in early. Not to create a leaderboard or measure who was "doing AI" correctly, but to understand what had been built.

Tool calls, token spend, recurring workflows, and broad usage patterns showed which tools were valuable, where requests were becoming expensive, and what capabilities people kept trying to use. That made it possible to improve the product based on behavior rather than guesses: unexpected usage suggested a workflow needed simplifying, while unused tools pointed to poor discoverability, descriptions, or limited value. Aggregate operational metrics were more useful than ranking individuals and kept the analytics focused on improving the tool rather than monitoring employees.

4. Close the feedback loop

People will try things you did not anticipate. They will also hit confusing behavior that feels obvious to the person who built the system.

Feedback was easy to share, and the agent was adjusted as people used it. This sounds basic, but it created a useful loop: somebody tried a workflow, reported where it failed, and could often try the improved version shortly afterwards.

That responsiveness encouraged more experimentation. People are much more willing to explore a new tool when they know a failure is not the end of the conversation.

Some basic documentation also helped. It did not need to explain every tool or become a complete AI training course. A short explanation of what the agent could access, examples of good requests, known limitations, and how to share feedback gave people enough confidence to start.

The combination matters. Documentation without feedback becomes stale. Feedback without documentation means answering the same introductory questions forever.

5. Design for the whole company

The agent started from an engineering and observability problem, so keeping it focused on engineers would have been the obvious path. That would have been a mistake.

Some of the most interesting usage came from product, support, operations, marketing, and growth. These teams often need information spread across several systems but do not have a convenient way to query or combine it. An agent with the right company context can remove a lot of that navigation. Support can be especially powerful here: people can investigate many reports themselves instead of waiting for engineering to gather the context.

Engineers can actually be harder to convince.

They already have coding agents, terminal workflows, preferred models, custom instructions, and strong opinions about harnesses. Asking them to move into a shared company interface can feel like asking them to give up a setup they spent months refining. I understand the resistance because I have the same opinions about my own tools.

Other departments may have fewer existing agent workflows to replace. Once they see a relevant example and trust the access model, the value can be more immediate.

Treating the agent as company infrastructure rather than an engineering experiment changed both its adoption and the features that mattered most.

Five things I would do differently

1. Design local access earlier

The biggest missing piece is making company tools available safely from local agent harnesses.

The shared web, Slack, and Linear interfaces are useful, but local agents have context that a company agent does not. They can work with a person's files, notes, project memory, terminal, and current repository. Engineers in particular do not want to abandon that context every time they need company information.

The ideal setup is not forcing everyone into one harness. It is allowing approved local harnesses to use company tools without exposing credentials or bypassing permissions.

That is a much harder problem than adding another interface. Authentication needs to represent the person making the request. Secrets cannot be handed to the local model. Tool access should remain scoped and auditable. A compromised or badly configured local environment should not gain broad company access.

2. Spend less time chasing the perfect architecture

Agent tooling changes constantly. Frameworks appear, models improve, MCP conventions evolve, and every week there is a new opinion about what AI should be, how it should be built, and which patterns or use cases are right or wrong.

It is very easy to keep redesigning the system around what everyone is discussing online.

I spent too much time considering whether I should rebuild parts of the agent around the latest approach. Most of that thinking had a short expiry date: a tool or pattern that looked essential one month could be irrelevant a few months later. Today I would pick one simple architecture that supports the immediate use case and move forward. Keep connectors small, keep boundaries clear, avoid coupling everything to one model or framework, and replace pieces only when there is a real reason rather than because a new diagram is popular on social media.

Internal tools have an advantage here: they do not need to become a generic platform for every possible customer. They need to solve the company's actual problems.

3. Make contribution paths obvious

The agent improved quickly, but too much of that improvement depended on a small number of people.

It was not obvious who could write a skill, add a connector, improve instructions, or contribute code. Even when the repository was available, people were understandably hesitant. Agent systems combine code, prompts, permissions, and unfamiliar conventions. Nobody wants to break a tool used across the company or accidentally expose something sensitive.

I have not solved this yet. People are welcome to use the agent, but it is not obvious how to contribute safely. The repository is available, but writing a skill, adding a connector, or changing instructions requires understanding code, prompts, permissions, and unfamiliar conventions.

A short contribution guide with examples, ownership, review requirements, and a safe test path would make this easier. Until then, too much of the improvement depends on a small number of people.

4. Build fewer evals and less tracing infrastructure

I spent too much time on evals, detailed traces, and comparing model behavior. Some visibility is essential: enough tracing to debug failures, usage analytics to control spend, and tests for critical tools and permissions. But this is not a frontier lab or a product with a dedicated evaluation team. It is an internal tool, and I cannot devote all my time to building infrastructure around hypothetical failures.

The goal is to make it simple, make it work, and keep it maintainable. A few representative workflows and permission tests are enough until recurring failures justify more. The evaluation system should grow from real problems rather than from the fear of not measuring enough.

5. Assume less prior AI knowledge

The largest mistake was probably assuming that capabilities obvious to me would be obvious to everyone else.

AI adoption varies enormously across departments and individuals. Some people work with agents all day. Others have only used the basic ChatGPT interface. Terms such as tools, memory, connectors, context, or skills may mean nothing to them, and there is no reason they should.

From the builder's perspective, the benefit seems clear: the agent can search several company systems, combine what it finds, and act on the result. From somebody else's perspective, it may look like another chat box. Why use this instead of ChatGPT? What does it know? What can it do? What is safe to ask? Will it make a change without permission?

I have not solved this completely either.

What helps most is turning abstract capabilities into visible functionality. Do not announce that the agent has a new connector. Show that it can prepare a specific report, investigate a known type of issue, or automate a repetitive workflow. Give people something relevant they can try immediately.

Adoption does not happen because the architecture is impressive. It happens when somebody sees the agent solve a problem they recognize.

The main lessons

These are the main lessons and mistakes from the first six months: make the agent visible early, connect it to the systems where context lives, keep access safe, and optimize for something simple enough to maintain.

The best decision was letting people see the agent being used before it felt finished. The recurring lesson is that what feels obvious to the builder is rarely obvious to everyone else.

I am interested in hearing how other teams are adopting AI in practice, what is working, and where workflows are still awkward or blocked by access, context, or trust. If you are working through similar questions or facing challenges introducing AI-native workflows in your team or company, I would love to hear more and see whether I can help.

I'm Orestis, VP of Technology at Bold.org. On the side I build clssrm, simple booking software for small studios. More of my writing is on oioannou.com.