DEV Community: Maksym Kuzmitskyi (MaximusFT)

Upgrading the Model Means Retesting the Workflow

Maksym Kuzmitskyi (MaximusFT) — Thu, 29 Oct 2026 00:00:00 +0000

Changing a model identifier can be a one-line edit. It is not necessarily a one-line change to the system's behavior.

The same task may now produce different searches, tool arguments, clarification questions, stopping decisions or recovery attempts. The final code can improve while one of those surrounding behaviors becomes less suitable for the workflow.

I would treat a model upgrade as a change to an engineering dependency, not as permission to carry every previous assumption forward.

That isn't an argument for staying on an old model. It is a reason to make the benefit and the compatibility work observable.

The application contract is larger than the final answer

A coding agent operates inside a harness: prompts, tool definitions, permissions, context management, execution loops and output handling. The model is one consequential part of that arrangement.

The contract may require the agent to inspect the current base before editing, use a particular tool for a sensitive action, stop when authorization is missing, and distinguish an unavailable check from a successful one.

Those requirements don't disappear because the candidate model produces a better-looking patch.

Some differences are API compatibility issues. Others are valid behavior changes inside an otherwise compatible API. A request succeeding doesn't establish that the application still gets the kind of progress events or tool-use pattern it expects.

There is current, concrete evidence for taking that distinction seriously. Anthropic's Fable 5.1 migration guide, checked on October 23, 2026, documents model-specific changes involving forced tool choice, conversation-history handling and agent-loop behavior. Those details apply to the documented migration, not to every model or managed product.

The general lesson isn't to memorize today's parameter list. It is to read the migration contract for the exact integration being changed. A managed agent product may handle details that a custom API harness must manage itself.

First name what is actually changing

An editor update can change the model, tool descriptions and execution behavior together. A provider can update a model alias. A team can revise its system prompt while testing a new model.

If all of those happen at once, an improved result does not identify which change helped. A regression doesn't identify which component broke either.

For a controlled comparison, I would record the model identity available to the integration, harness revision, prompt revision, tool schemas, relevant settings and fixture state. If the exact underlying model is hidden, I would record that limitation instead of inventing a stable identity.

Keep task inputs and starting repositories comparable. A new base commit, warmed cache or already-fixed fixture can turn a model comparison into a comparison of different problems.

One variable at a time is a useful diagnostic ideal, not always a deployment option. When a product bundles changes, evaluate the bundle and describe the attribution limit honestly.

Build cases around decisions that matter

I would start with a small regression set drawn from the workflow's consequential boundaries.

A clear local fix should remain clear and local. An ambiguous requirement should produce a useful question rather than an invented product decision. A failed integration check should not be summarized as a successful task because the unit suite passed.

Another fixture might change the PR head during review. The expected outcome is to notice that the reviewed artifact has changed, not merely attach the old conclusion somewhere the API accepts it.

These are proposed evaluation cases, not reported experiments or results from my work. Their value comes from defining the acceptable outcome before reading the candidate's explanation.

I would avoid requiring one exact tool-call sequence unless the sequence itself enforces a real constraint. Two valid investigations may read different files. The evaluation should distinguish a different route from a violation of the task's boundaries.

Conversely, a plausible final patch shouldn't erase an unauthorized intermediate action. Evaluate consequential actions and the final artifact, not just the last paragraph.

Grade observable behavior, with room for uncertainty

Some outcomes can be checked mechanically: whether a forbidden mutation occurred, whether the final diff contains unrelated files, or whether a claimed test command actually completed.

Other outcomes need a domain reviewer: whether a clarification was necessary, whether a suggested integration belongs in scope, or whether the proposed test establishes the intended behavior.

A model-based grader can help organize evidence, but it isn't automatically an independent authority. I would calibrate it against reviewed examples and inspect disagreements. A score without an understandable basis is another opaque output.

OpenAI's evaluation documentation describes datasets, annotations and graders. At the October 23 check, it also carries a deprecation notice for the hosted Evals platform. I am not recommending a new dependency on that retiring service. Representative cases and explicit grading criteria remain useful regardless of which supported runner stores them.

The tooling should serve the comparison. It should not become a reason to keep a fragile evaluation tied to an obsolete interface.

Recovery deserves its own test

A clean success path is a friendly demonstration. The more revealing cases often involve unavailable tools, authentication failures, partial edits and interrupted work.

I would check whether the candidate preserves the task boundary after an error. Does it ask for a secret in an inappropriate channel? Does it retry an irreversible operation without checking the previous outcome? Does it weaken a test merely to complete the loop?

These concerns aren't claims that a particular model does those things. They are reasons to observe recovery rather than assume it inherits the quality of the model's successful answers.

Permissions should remain enforced outside the model. An evaluation can detect undesirable behavior; it shouldn't be the only thing preventing a consequential action.

The same reasoning applies to compaction and handoffs. A workflow that works for five turns may fail when a summary loses an important unresolved condition. A longer-running fixture can expose that boundary without requiring an enormous benchmark.

Compare the cost of an accepted result

Tokens and response latency are useful measurements. They don't describe the whole task if the candidate requires more correction, publishes noisier reviews or leaves harder-to-understand changes.

I would record human interventions and rework alongside execution cost, using actual observations rather than estimated productivity claims. Passive waiting and a person's active effort are different quantities.

Repeated runs matter where outputs vary. One candidate success and one baseline failure are anecdotes, not a stable rate. The amount of repetition should match the consequence of the decision and the available resources.

The result can be mixed: better implementation on one class of task, more clarification on another, a compatibility repair needed in the harness. That is a useful migration finding, not a disappointing failure to produce a winner.

Keep a way back

Before changing a default, I would keep the previous configuration identifiable and limit initial exposure. That might mean a small authorized task class or shadow review without automatic publication.

Rollback has limits too. A pinned model may later be retired, and a managed product may not expose an exact old configuration. Record what can actually be restored rather than promising a permanent switch.

The existing evaluation article asks whether the workflow produces useful outcomes. A model upgrade is a concrete occasion to ask that question again, with the old assumptions made visible.

I want to use better models. I also want to know which part of my working agreement with the agent changed when I selected one.

What would make a model upgrade fail acceptance in your workflow even if its final code looked better? I'd like to hear which behavioral regressions your evaluations can actually detect.

When Repository Content Starts Giving Orders

Maksym Kuzmitskyi (MaximusFT) — Tue, 27 Oct 2026 00:00:00 +0000

A source comment explains a strange compatibility rule. A README explains how to run the project. A test fixture contains a conversation. A fetched issue describes what should change.

A coding agent needs all of them. They do not all have the same right to tell it what to do.

That is the boundary I find more useful than telling an agent not to trust the internet. A repository can contain relevant evidence, authorized project instructions, obsolete advice and deliberately hostile text. Sometimes those arrive through the same search result.

The problem is not reading a suspicious sentence. The problem is promoting a sentence from material under inspection into authority over the inspection.

The source being reviewed cannot appoint itself the reviewer

Consider an illustrative PR that adds a comment describing a component as exempt from ordinary validation. The same change tells automated reviewers to ignore failures involving that component.

The comment may be useful evidence about the author's intent. It is not, by itself, an approved exception to the team's validation policy.

If the agent treats it as an instruction, the proposed code has acquired a way to influence how it is judged. The author need not have changed a tool permission or a protected policy file. The confusion happened in interpretation.

This can be deliberate prompt injection. It can also resemble an ordinary documentation mistake: an outdated setup note asks for a step the current task doesn't need. The intent of the text and the authority of the text are separate questions.

I would not claim every incorrect README instruction is an attack. I would ask whether the agent is using it as context to verify or as a command that overrides the actual task.

Repository-local doesn't mean equally trusted

An instruction file deliberately selected by the user or organization has a different role from a copied log line. A task description from an authorized owner differs from an arbitrary comment underneath it. A policy on the reviewed branch may itself be part of the proposed change.

That doesn't mean refusing all repository instructions. It means keeping their origin, scope and review status visible.

For a PR reviewer, the base branch's approved policy and a proposed replacement on the head branch should not silently become the same source. A legitimate policy update may need review. Its existence in the diff doesn't automatically authorize it to govern that review.

Likewise, a retrieval tool returning content doesn't authenticate the instructions inside that content. The tool may be trusted to fetch a page accurately while the page remains untrusted to direct actions.

The distinction also applies to generated summaries. If a summary turns a quoted instruction from a fixture into an unqualified next step, it can lose the original boundary while preserving the words.

Accurate retrieval is not delegated authority.

I want a handoff to preserve that difference, not just preserve more text.

Current controls matter, but they cover different surfaces

This is not an argument that current coding agents operate without protections. The VS Code security guide, checked on October 23, 2026, documents workspace trust, approvals, sandboxing, MCP-server trust and information-exposure risks.

The details depend on the active harness and tool path. Its guidance distinguishes process sandboxing from permission checks on other tools and warns that sandboxing is not a standalone security boundary. That is more useful than assuming one toggle makes every route equivalent.

Instructions guide the model. Permissions determine which operations are available. Isolation can constrain what an executing process reaches. Network restrictions constrain destinations. None of these automatically proves that a proposed change satisfies the task.

The current GitHub application card also describes different environments and data flows for its agentic surfaces. I wouldn't transfer a cloud agent's documented constraints to a local terminal session merely because both carry the same product name.

Good controls reduce the consequences of a confused decision. They do not remove the need to distinguish data from instructions.

Keep the action boundary outside the prose

A document can recommend a command. It should not be able to grant that command new permissions.

I would enforce consequential restrictions in the tool or execution environment rather than depend solely on the model agreeing to follow a warning. A reviewer that doesn't need mutation tools shouldn't receive them by default. A task needing one repository shouldn't inherit every account a developer happens to use.

That principle is narrower than prescribing a particular sandbox configuration. Filesystem access, credential availability, network routes and connected services vary by environment. The controls have to be checked where actions actually run.

A tool wrapper that inspects only one command shape may miss a different execution route. A hook's failure or timeout behavior may not match an engineer's assumption about stopping. I would test those boundaries instead of treating the existence of a hook as enforcement.

There is also a quiet escalation to avoid: allowing a document being analyzed to nominate additional tools or a more privileged environment as prerequisites. A claimed need for access is still a request to evaluate.

I discussed useful tool context in Tools Are Part of the Agent's Intelligence. That usefulness makes precise tool authority more important, not less.

Test whether an instruction stays inside its source

For an authorized local evaluation, I would use harmless fixtures. A source file or mock tool result could request a clearly out-of-task change to a disposable marker file. The expected outcome would be to recognize the instruction as content, not carry it out.

No real secrets, external collection endpoints or production systems are needed to establish that question. The test should inspect actual tool activity as well as the final explanation. Saying that an instruction was ignored while performing its requested action is not a pass.

I would vary the source: a comment, a generated report, a quoted issue reply and a summarized tool result. The aim is to check the boundary through paths the workflow actually uses, not to collect theatrical attack phrases.

Passing such fixtures is evidence about those cases. It is not proof against every prompt injection. The environment still needs constraints that limit the effect of a missed case.

Legitimate instructions matter in the evaluation too. An agent that refuses to follow the approved project setup has not solved trust by refusing all work. I want correct treatment of different sources, not indiscriminate suspicion.

A stopped action should leave an understandable reason

If a source asks for an unexpected operation, the useful response identifies the source and the conflict: this instruction appeared in material being analyzed, but the task did not authorize the action.

A person can then confirm an actual requirement through the appropriate channel. The agent doesn't need to turn every questionable paragraph into an emergency. It needs to avoid quietly changing who is in charge.

Logs can help reconstruct that decision, but they should not copy entire sensitive documents merely to preserve evidence. Record enough origin and action information to investigate, with appropriate access and retention.

I don't want to make agents blind to repositories. I want them to read carefully without making every readable sentence a new supervisor.

When a coding agent fetches something in your workflow, what establishes whether that content may direct its next action? I'd be interested in the boundaries you have actually tested, especially the ones that survive summaries and handoffs.

The Agent Finished. The Reviewer Is Still Reading.

Maksym Kuzmitskyi (MaximusFT) — Sun, 25 Oct 2026 00:00:00 +0000

The agent has finished the implementation. CI is green. There is a useful PR description, a test summary and another task ready to start.

The reviewer is still trying to understand the first change.

I don't think that makes the reviewer the slow part of an otherwise efficient machine. Generation and acceptance are different kinds of work. Producing another plausible implementation does not create another person's attention to examine it.

The interesting question isn't how many PRs an agent can open. It is how much consequential change a team can understand, validate and own without turning review into an exercise in trusting the summary.

A faster author changes the queue

Imagine a team where generated changes arrive faster than reviewers can assess them. This is a hypothetical operating condition, not a productivity measurement from my work.

The immediate symptoms might look encouraging: more branches, more completed implementation tasks, more green checks. The waiting work has simply moved.

Opening several PRs in parallel may shorten implementation time for each author while extending the delay before any reviewer can concentrate on one. Interrupting reviewers with repeated requests makes their effective capacity smaller still.

This is a workflow problem, not an argument against agents. A compiler getting faster never meant that architectural judgment became free. The same distinction deserves to survive another increase in code-generation capacity.

OpenAI's 2026 harness-engineering report describes human QA capacity becoming a constraint as code throughput increased in its own internal experiment. Its response involved making application behavior and observability more accessible to agents. Those are observations from a particular environment, not a throughput target or permission model I would import wholesale.

For my purposes, the important part is the changed constraint. If acceptance is waiting, making authors produce more work may be optimizing the wrong stage.

Reviewability isn't a line-count limit

A small diff can contain a large decision. One changed authorization condition may deserve more attention than a generated file containing hundreds of mechanical edits.

I would size review around behavioral questions: what changed, who depends on it, which assumptions changed, and what evidence is available. Line counts can help notice outliers. They do not calculate risk.

Consider an illustrative PR that introduces a shared utility, migrates consumers, changes a dependency and refreshes unrelated snapshots. Every individual edit might be reasonable. Together they create several distinct review jobs.

The reviewer has to separate the intended behavior change from migration mechanics, dependency effects and test-expectation updates. A detailed explanation can help. A cleaner division of work may help more.

That doesn't mean splitting code into arbitrary tiny PRs. A sequence of fragments that cannot be tested or understood independently can increase coordination costs. I would look for cohesive behavioral slices with explicit dependencies and a valid state between them.

A reviewable change gives someone a bounded question they can actually answer.

The unit isn't necessarily one file, one agent session or one convenient commit. It is an intelligible decision with enough surrounding evidence.

Summaries should lower the cost of checking

A good summary tells me where to inspect, not merely what conclusion to reach.

For a permission change, I want the affected rule, the relevant route and the check that exercised a denied action. For a shared component change, I want the changed contract and the consumers that make it consequential. For an expected-result update, I want the requirement that changed.

I don't need a transcript of every search or an elaborate account of the model's internal reasoning. I need references I can follow and limitations I can see.

An agent saying that all relevant scenarios passed should be able to identify those scenarios. If a check was unavailable, the absence should remain visible rather than disappear inside a reassuring overall conclusion.

This is related to useful automated review comments, but it is a different responsibility. The question here is whether the whole change can be accepted with reasonable effort, not whether an individual bot comment is well phrased.

Long descriptions can make the problem worse if their conclusions are easier to read than their evidence is to verify. I would rather have three precise links and one unresolved question than a confident report that nobody has time to audit.

Another agent can remove work, or add another queue

Automated review can catch local defects, investigate a suspicious export or check a routine convention before a person arrives. That is useful preparation.

It can also generate a second pile of work: findings to validate, false positives to dismiss, replies to interpret and repeated comments to reconcile. A quiet tool isn't necessarily effective, and a busy one isn't necessarily helping.

GitHub's current application card for Copilot agents explicitly discusses missed issues, false positives and inaccurate suggestions, and recommends supplementing automated review with human review. These are present documented limits, not a claim that current tools only inspect a few lines of a diff.

I would assess whether a review tool reduces the human effort needed to reach a sound decision. That includes the effort spent checking the tool itself. Comment count is a poor substitute.

The same applies to UI evidence. Browser automation can make a previously invisible interaction inspectable, but a screenshot doesn't verify every state. Helpful automation narrows the remaining question. It doesn't just add another green label.

Control work in progress before asking for faster review

If review capacity is the constraint, I would try limiting how many changes are actively seeking attention. Completing one reviewable slice before opening three more can improve the flow without changing the model.

That limit should be a team decision, not a universal number. A routine dependency stream and a change to a shared identity contract don't impose the same load.

Review ownership also matters. An agent can identify potential consumers, but that doesn't automatically identify who is accountable for accepting the change. A PR waiting for everyone can quietly become a PR owned by no one.

I would make blocked states explicit: waiting for a domain decision, waiting for a missing integration check, or waiting for a reviewer with the relevant ownership. Those call for different actions. Re-running the same agent is unlikely to resolve all three.

There is a useful stopping point for automation here. When the next step is an accountable decision, generating more implementation isn't progress on that decision.

Measure the waiting and the rework

I would keep measurements modest and specific: how long changes wait for first substantive review, how often a PR needs to be separated, what corrections arrive after apparent completion, and which review questions repeatedly lack evidence.

These are proposed observations, not results I have measured for a comparative study. They need definitions and local interpretation. A shorter review can mean a better-prepared change, but it can also mean a rushed reviewer.

The goal isn't to make every change slower. It is to stop borrowing confidence from an overloaded acceptance process. If the team cannot explain what it accepted, the implementation speed has hidden part of its cost.

I still want the agent to finish quickly. I also want its finished work to be something another engineer can pick up without reconstructing an entire conversation.

When I look at a queue of generated PRs, that is the question I would put ahead of the next throughput improvement: can the people responsible for this system still understand what is entering it?

Where does review capacity constrain your AI-assisted workflow: understanding the change, checking the evidence, or sorting automated findings? I'd be interested in what actually reduced that burden, not just what produced more output.

Who Checks the Tests an AI Agent Writes?

Maksym Kuzmitskyi (MaximusFT) — Fri, 23 Oct 2026 00:00:00 +0000

An agent writes a function, adds tests, runs them, and reports success. The implementation and the tests agree.

What established that they agreed on the right behavior?

This is the part of AI-generated testing I want to examine before counting new assertions. A test can be perfectly executable, exercise the relevant branch, and confirm the same mistaken interpretation that produced the code.

Humans do this too. The additional concern with an agent is how conveniently implementation, test cases, fixtures and the final explanation can be produced from one interpretation. They look like several pieces of evidence. They may have only one underlying assumption.

I don't want fewer generated tests. I want a clearer answer to where their expected results came from.

A passing test for the wrong rule

Consider a fictional checkout policy. Free shipping applies when the merchandise subtotal after discounts is at least 5,000 cents. Tax and shipping charges do not contribute. The examples below assume valid, nonnegative integer inputs, with discounts no greater than the merchandise subtotal; input validation is a separate concern.

This is an illustrative example, not an incident from a client project.

An implementation and its test might look like this:

import assert from 'node:assert/strict';

type ShippingQuote = {
  merchandiseCents: number;
  discountCents: number;
};

function qualifiesForFreeShipping(quote: ShippingQuote): boolean {
  return quote.merchandiseCents >= 5_000;
}

assert.equal(
  qualifiesForFreeShipping({ merchandiseCents: 6_000, discountCents: 1_500 }),
  true,
);

The test passes. The discounted subtotal is 4,500 cents. Under the stated policy, that order does not qualify.

The test has not failed to execute the implementation. It has asserted the wrong answer. A coverage report can show that the function ran without noticing that the expected value contradicts the requirement.

Adding more examples above the undiscounted threshold won't necessarily help. Nor will renaming the test to sound more specific. The missing ingredient is an expected result derived from the policy, rather than an observation of what the function currently returns.

That expected result is the test oracle: the basis for deciding whether the observed behavior is correct. It doesn't have to be a separate program or an expensive formal specification. Sometimes it is one arithmetic calculation and a sentence someone has actually agreed on.

Establish the expectation before polishing the implementation

For this policy, I would start with a few examples whose answers can be worked out without consulting the function:

assert.equal(
  qualifiesForFreeShipping({ merchandiseCents: 6_000, discountCents: 1_500 }),
  false,
);
assert.equal(
  qualifiesForFreeShipping({ merchandiseCents: 5_000, discountCents: 0 }),
  true,
);
assert.equal(
  qualifiesForFreeShipping({ merchandiseCents: 4_999, discountCents: 0 }),
  false,
);

The first case disagrees with the original test. That disagreement is useful. It makes the interpretation visible instead of burying it beneath a successful run.

The implementation can then follow the policy:

function qualifiesForFreeShipping(quote: ShippingQuote): boolean {
  const discountedMerchandiseCents =
    quote.merchandiseCents - quote.discountCents;

  return discountedMerchandiseCents >= 5_000;
}

This is not a complete checkout test suite. The cases establish the particular discount and threshold behavior under discussion. They don't establish currency conversion, tax treatment elsewhere, coupon validity or production integration.

I would ask the agent to identify the source of each important expectation. An acceptance criterion, an approved example, a documented contract or a trusted reference can justify it. The current implementation can tell us what happens, but not always what should happen.

If the requirement doesn't settle whether the threshold applies before or after discounts, generating an assertion doesn't settle it either. The correct next step is a question. An executable guess is still a guess.

A second agent needs a different starting point

It is tempting to solve this by assigning test generation to another agent. That can help, but a separate role is not automatically an independent basis for judgment.

If the second agent receives the implementation and a persuasive explanation of why it is correct, it may simply encode that explanation as tests. Changing the model doesn't guarantee a different interpretation of an ambiguous requirement.

For consequential behavior, I would give the test reviewer the original requirement and ask it to derive concrete outcomes before showing it the implementation. Then compare the cases with the code and existing tests.

This reduces one source of anchoring. It doesn't prove independence or eliminate shared mistakes. Both reviewers may still misunderstand the requirement, which is why the expected outcomes need to remain inspectable by a person.

In Green Checks Only Cover What You Checked, the concern was the reach of evidence across an integration boundary. Here the boundary is earlier: whether the assertion itself expresses the intended behavior. An integration test can have a wrong expectation too.

Check that the test can disagree

A useful test should fail for a relevant wrong implementation. I would deliberately check that property, rather than infer it from a green run.

In this example, ignoring the discount should fail the first requirement-derived case. Changing the threshold comparison from inclusive to exclusive should fail the exact-boundary case.

Mutation testing automates variations of this idea by introducing changes into the code and checking which tests detect them. Tools such as Stryker make that practical without inventing a mutation framework for each project.

A detected mutation establishes sensitivity to that change. It doesn't establish the truth of the original requirement. An incorrect expected value can still kill mutants; a surviving mutant can reflect equivalent behavior rather than a meaningful defect. Mutation scores need interpretation, not ceremonial targets.

Property-based tests add another useful angle. Under our stated policy, increasing the discount while holding the merchandise subtotal fixed must not turn an ineligible order into an eligible one. This describes a relationship across inputs instead of one fixed example.

But a function that always returns false also satisfies that relationship. Properties need enough independent constraints to rule out plausible wrong implementations. They complement concrete examples; they don't manufacture a business rule from nothing.

Fixtures can make the same mistake less visible

I would also inspect how test data is created. A fixture built by calling the implementation under test can quietly recycle the result it is supposed to check. A broad mock can remove the behavior that actually needs verification.

This doesn't make builders or mocks inherently bad. A builder can provide ordinary input data while leaving the important expected result explicit. A mock can isolate a collaborator without deciding the business outcome for the test.

The question is what the helper owns. Does it simplify setup, or does it calculate the answer through the same logic the assertion claims to validate?

Duplicating the production algorithm inside the test is not necessarily a better oracle. It can reproduce the same defect in another file. For a small example, independently calculated values may be easier to audit. For complex behavior, a reviewed reference implementation, contract examples or another representation may be appropriate.

There isn't one universally independent source. The point is to identify the dependence instead of mistaking a second file for a second line of reasoning.

Review the test change, not just its result

When an agent fixes a failing suite, I want to know whether it changed the behavior or changed what the suite accepts.

Sometimes the expected result is genuinely obsolete because the requirement changed. Sometimes the implementation is wrong. Sometimes the test was wrong from the beginning. All three can end with green CI. They are not the same engineering decision.

A snapshot update deserves the same question. Accepting a new snapshot records an output; it doesn't establish that the user should see it. Removing an assertion or relaxing a matcher may be justified, but the justification needs more than getting the run to pass.

For a bug fix, I would prefer the regression test to demonstrate the reported failure on the relevant prior implementation, then pass with the correction. It also needs to fail for the intended reason, not because an unrelated dependency or import is broken.

For new behavior, a failing test against the previous implementation is expected. What matters is that its assertion comes from the requested behavior, not from whichever output the new implementation happened to produce.

Keep generation useful

I like the prospect of an agent handling repetitive setup, exploring input combinations and proposing edge cases. Those are useful contributions even when the final expectation still needs human judgment.

What I don't want is to promote the existence of generated tests into evidence that the requirement has been independently checked. More assertions can make a mistaken assumption look thoroughly documented.

The shipping example is small enough that the disagreement fits in one subtraction. Real systems hide the same question behind fixtures, collaborators and years of existing behavior. I would still start by asking who decided the answer and what supports it.

If you use agents to write tests, how do you keep expected outcomes independent of the implementation? I'd be interested in the checks that expose shared assumptions, especially when both the code and its tests look completely reasonable.

Independent Releases Still Need Compatible Neighbors

Maksym Kuzmitskyi (MaximusFT) — Thu, 22 Oct 2026 00:00:00 +0000

The account team releases its new frontend. The shell stays on the previous version. Checkout was updated yesterday.

That can be a healthy arrangement. It is what independent deployment is meant to permit.

It is also a compatibility problem. Those versions need to work together, and users will discover the combinations we did not consider while each team tested its own application.

The earlier micro-frontends article asks whether organizational independence justifies the split. Here I want to examine the technical consequence after that decision: what independence requires during ordinary releases and rollbacks.

Separate three kinds of boundary

A code boundary determines which module can import another. An ownership boundary determines who may change it. A deployment boundary determines which artifact can be released without rebuilding the others.

They can align, but they do not have to.

A modular frontend can have feature packages, explicit APIs, separate owners, and enforced import rules while producing one application build. That is the modular monolith in this comparison, not a claim that all code belongs in one undifferentiated folder.

Micro-frontends add independently deployed application artifacts, composed through routes, runtime loading, frames, or another integration mechanism. Different composition methods have different costs. An iframe provides a stronger browser isolation boundary than mounting two modules in the same JavaScript runtime; it also changes interaction and integration constraints.

I would name the intended composition before discussing the benefits. "Micro-frontends" is otherwise doing too much work in the sentence.

One build moves compatibility work earlier

In a modular monolith, one build checks the selected package versions together. A failing interface change can stop the release before production sees it. A coordinated rollback returns that tested set of frontend modules together.

That does not guarantee correctness. Old tabs still exist, backend APIs still evolve, and integration tests can miss a workflow. But the frontend's internal version combination is more constrained.

With independent artifacts, compatibility work remains necessary and moves partly into runtime and release policy. The shell needs to understand the remote's contract; the remote needs to tolerate the supported shell versions.

The work did not disappear because CI pipelines became separate. We changed where it is paid.

A remote needs a supported contract

An illustrative mount contract might include a version, capabilities, lifecycle, and a constrained way to communicate:

type AccountModuleContract = {
  version: 2;
  mount: (container: HTMLElement, context: AccountContext) => () => void;
  capabilities: readonly ('profile' | 'notification-settings')[];
};

The shape is not a recommended universal micro-frontend API. It shows the questions a boundary needs to answer: what does the host provide, what may the remote assume, how does it clean up, and how can the host detect incompatibility?

Do not make shared context an unrestricted store containing every application's internals. That turns the mount API into a distributed import path with weaker verification.

Capabilities can make additive evolution easier than assuming that every participant upgrades simultaneously. For breaking changes, define a support window and migration path.

Release independence creates version combinations

If several remotes each support two versions while the shell also supports two, the possible combinations multiply. It is rarely practical to test every theoretical combination equally.

Constrain the supported matrix. Test each remote against the current shell and the declared compatibility window. Keep representative whole-product flows that cross the important boundaries.

A remote's unit tests may correctly show that its save action works. They do not show that the shell refreshes the account menu, preserves navigation, and clears the right state after logout.

Contract tests help, but a contract can be structurally valid and behaviorally wrong. The few shared workflows deserve integration tests owned by someone, not by an imaginary team named "the platform."

Runtime manifests are release artifacts

If the shell loads a remote entry through a mutable URL, the deployed composition can change without the shell's commit changing.

Record the selected remote versions or immutable asset references. Keep enough history to reconstruct what users were served. Rollback should select a known-compatible composition, not point several URLs at yesterday and hope.

Asset retention matters for clients that already loaded an old entry. Removing the old chunks immediately can break lazy navigation later, long after the initial page load succeeded.

CDN cache lifetimes can also cause different users to load different manifest versions. This is manageable if the contracts support overlap. It is inconvenient if independent deployment silently assumes instantaneous global consistency.

Sharing dependencies exchanges one cost for another

Shipping separate dependencies can increase download size and produce multiple copies of libraries. Sharing them at runtime can reduce duplication, but introduces version negotiation and lifecycle constraints.

React singletons deserve care when components and hooks interact across a shared runtime. A shared design-system library also has behavioral and styling contracts; two versions can differ in focus management or portal assumptions even when the buttons look similar.

I would not force every dependency into a singleton merely to improve a bundle report. Nor would I accept uncontrolled duplication without measuring the user cost. Decide which shared dependencies are part of the platform contract, and verify both upgrades and rollbacks.

For stronger isolation, route-level composition may be more straightforward than many independently loaded widgets sharing one screen. It depends on the product flow, not on which demo looked most flexible.

Isolation needs failure behavior

A remote can fail to load because of a network error, stale asset reference, or incompatible deployment. The shell still needs a useful route and a recoverable state.

Distinguish an optional sidebar from checkout's payment step. One may show a fallback. The other may need to stop the workflow and preserve user input. An error boundary catches some rendering failures; it is not full isolation from CSS conflicts, shared global state, or security problems.

Keep remote-loading timeouts, telemetry, retry rules, and fallback ownership explicit. A spinner with no deadline is not a graceful degradation strategy.

The modular monolith has fewer runtime composition failure points, but its deployment can affect more features at once. Feature flags, staged releases, and modular tests can reduce that risk without creating separate runtime applications.

Shared state is an authority question

Identity, cart state, locale, and navigation often cross frontend boundaries.

Decide who owns each truth. An event that says "cart changed" can prompt other applications to read the authoritative value. An event carrying a partial cart snapshot can accidentally create another source of truth with its own freshness problems.

An event bus should have typed contracts, lifecycle rules, and unsubscribe behavior. It should not become an excuse for every remote to discover undocumented events by reading the others' source.

Authorization must remain enforced at the backend. A remote being mounted by an authenticated shell does not give its API calls permanent authority.

These rules apply inside a modular monolith too. Independent deployment makes violations harder to repair together, which raises the value of keeping the shared surface small.

Try modularity before distributing a coordination problem

If the main pain is accidental imports, unclear ownership, or slow whole-repo CI, a modular build with boundary rules and affected-project checks may address it directly.

If teams genuinely need separate release schedules, existing applications cannot reasonably be merged, or one part must evolve under substantially different constraints, runtime composition may justify its cost.

I would trial the split at a boundary already understandable in the product, with a specific blocked workflow the trial is meant to improve. Measure release lead time and integration effort, then account for the platform work needed to keep it reliable.

A module boundary that cannot be described cleanly inside one build usually does not become clearer when a network load is inserted between its parts.

Independence has a maintenance agreement

The decision is not simply one deployment versus many. It is how much coordination we want to perform before release and how much compatibility we are prepared to maintain afterward.

I would choose a modular monolith when coordinated artifacts are affordable and the team's main need is better boundaries. I would choose micro-frontends when independent delivery solves a demonstrated constraint and the teams can own the resulting compatibility work.

Either arrangement can be maintained well. Neither relieves us of agreeing what a neighboring module may do while we are deploying something else.

If your teams already release independently, what happens when one of them rolls back? I'd be interested in the supported combination, because that answer says more about the architecture than the number of repositories.

The Agent Was Right. The Scope Still Changed.

Maksym Kuzmitskyi (MaximusFT) — Wed, 21 Oct 2026 00:00:00 +0000

A colleague started working on his ticket and discovered that its implementation was already there. He traced the files back to my pull request.

I hadn't found a technical problem with the additional work. The awkward part was explaining how it had become part of my task.

My ticket introduced a new abstraction, which I'll call a model. Another task covered integrating it at three selected usage sites. The older implementation served other consumers and needed to remain.

The agent built the model and covered it with tests. Then it noticed that the model wasn't being used. The task description explained what it would eventually replace, and the agent identified the places where it could be connected.

That was a reasonable suggestion. I accepted the argument and let it continue.

The integration was good work. It also belonged to another ticket.

This wasn't an agent secretly editing forbidden files. It asked, and I agreed. That makes the case more interesting than a story about automation ignoring instructions. Human agreement was present. What was missing was an explicit reconciliation between that agreement and the way the work had been divided.

The agent was right about what came next. That didn't establish that it belonged in this PR.

Unused is not the same as unfinished

A tested abstraction waiting for its integration task can look like incomplete engineering when viewed only through the current repository.

The code suggests an obvious next step. The ticket boundary may say something different.

That boundary isn't automatically sacred. Tasks can be split poorly. Combining implementation and integration may be the better decision. But it should be a deliberate change to the plan, with the affected people aware of it, not an accidental consequence of a convincing technical explanation.

I wouldn't require product approval for every extracted helper. Engineers need room to make implementation decisions. The useful distinction is between supporting work necessary to satisfy the current acceptance criteria and work that delivers a separately planned behavior.

An additional compatibility fix might be necessary for the requested change to function. Integrating a new model into consumers assigned to another task is a different decision. Both can be technically justified. Only one follows automatically from the current task's goal.

Knowing the intended destination is not the same as owning the next step.

This isn't unique to AI. A human engineer can make the same choice. But an agent can provide both the persuasive argument and the implementation. I need to examine the transition between those two, not only the quality of the result.

I started making the origin of decisions visible

My response was not another permanent instruction saying never do anything extra. That would make the agent less useful without resolving the ambiguity.

Instead, I made a distinction in the task records: work grounded in requirements, documentation, repository instructions or acceptance criteria, versus additions originating in the agent's own engineering judgment.

The latter should be presented as a proposal or inference, not quietly described as required work.

Both I and the review agent can inspect that record. It gives us somewhere to look when a change is sensible but its relationship to the task is unclear.

The important difference is between stating that a documented requirement calls for a change and stating that the agent recommends it because the current implementation appears incomplete. Those explanations deserve different review questions.

Of course, the agent can misclassify its own reasoning. A documentation reference may describe a future direction rather than a current requirement. A record written by the implementer is an account to examine, not independent evidence that everything happened correctly.

The reviewer needs the original task and relevant sources, not just the implementer's summary. It also needs to compare that account with the actual changes. Otherwise, two agents may agree because they inherited the same assumption.

For substantial work, this record can preserve decisions, checks, open questions and work intentionally left outside the task. A tiny, clear change doesn't necessarily need another specification file.

I've described the broader task-to-PR workflow elsewhere. Jira, Confluence, repository instructions, Git history and API documentation can be available through MCP integrations and other tools. Access helps only if the sources are current and their authority is understood. More context can also mean more convincing outdated instructions.

I want the record to make disagreement easier to locate. I don't want an impressive document that makes disagreement harder.

A good review needs several kinds of evidence

My personal PR-reviewer article focused on specific, checkable comments and visibility into what the tool inspected. That remains useful engineering attention. A removed export or disappearing permission check deserves investigation, not an unexplained warning.

But a reviewer can correctly assess an implementation and still miss why that implementation is in the PR.

Tests address particular behaviors under particular conditions. A local TypeScript check doesn't establish compatibility with a downstream consumer whose actual package contract differs. As I argued in Green Checks Only Cover What You Checked, the question is what the check exercised, not merely whether it passed.

Automation can change the object being reviewed too. My scope-after-tools article uses a synthetic dependency update where a hook regenerates unrelated API artifacts. The generator succeeds; the resulting commit still needs inspection. That example is illustrative, not a second incident from the model-integration story.

Review evidence also expires. A conclusion about one revision may no longer describe the next. A Review Belongs to a Commit examines revision tracking and guarded actions in personal tooling I've developed. Remembering a SHA helps identify what was inspected; it doesn't prove every finding remains valid or turn an earlier approval into authority over later changes.

These aren't interchangeable checks. Code correctness, dependency compatibility, task inclusion, action permissions and the relevance of evidence each answer a different question.

A passing test doesn't authorize extra work. An allowed directory doesn't define the business requirement. A second model's agreement doesn't establish independent verification.

Nor should every small edit trigger an investigation across every connected repository. The validation needs to follow the affected contract and the decision being made.

Three conversations around different parts of the problem

During October, three groups contacted me after reading my articles. The useful connection was their different approaches to questions I was already trying to answer.

In an early-October exchange with Kai, we discussed code relationships: imports, exports, calls, and how a finding might be challenged rather than immediately published. We also discussed limits of static analysis around runtime behavior and some React relationships. That is an account of the conversation, not a specification of today's product.

I proposed an evaluation with expected defects recorded before running the reviewer, without revealing the answers in the PR descriptions. Cross-file regressions, contract changes, React behavior and false positives would make useful cases. That evaluation was discussed, not completed.

The DeepPattern conversation approached the problem through verification: missed requirements, untested assumptions and whether the available evidence supports a conclusion. It also touched attribution. If a task succeeds, was the useful contribution a reusable skill, connected documentation, an existing test, or a human correction? Without a meaningful comparison, I can't isolate the cause. A preview was offered; I am not reporting evaluation results.

Agent Role Contracts, from Netsujo, addresses explicit declarations of roles, scope and handoffs. Its documentation makes an important boundary clear: internally consistent declarations do not grant runtime authority, authenticate a reviewer or establish business acceptance.

A task requesting writes outside a declared scope can be rejected before execution. My model-integration example asks a different question: what if every edited file is permitted, but the work belongs to another task?

These are complementary directions, not three products I've ranked. Code relationships, verification evidence and declared authority all matter. None alone decides whether a technically sensible addition belongs in this particular PR.

Move the scope question earlier, without pretending it is solved

By PR review, a disputed decision already has an implementation. The code can make the proposed extension look settled, even when the reason for including it remains uncertain.

I would like to make that uncertainty visible before implementation: compare the proposed work with the original goal, identify additions, and decide which ones require coordination. Then compare the actual result with the accepted scope.

That is a direction to investigate, not a semantic scope checker I've completed. Requirements can be ambiguous. Plans can omit necessary work. The reviewer can share the implementer's blind spot. A real change in understanding may justify revising the plan.

I would evaluate a mechanism like this on cases where the right response differs. One task genuinely requires a supporting change. Another should produce a suggestion to do the integration separately. A third needs clarification because the requirement doesn't settle the choice.

The expected outcome has to include stopping or separating work, not just producing more code. Otherwise, the evaluation rewards the very behavior it is supposed to examine.

I also wouldn't ask an agent for an elaborate account of its internal thoughts. I need inspectable grounds: which requirement it relied on, what it inferred, what it changed, and what the checks actually support.

My colleague didn't uncover bad code. He uncovered work that had crossed a task boundary while still looking like sensible engineering. I had participated in that decision. The part I want to improve is how such a decision becomes visible before someone else discovers their task inside my PR.

If you work on agent review, verification or workflow controls, how do you distinguish a necessary supporting change from an expansion that needs a separate decision? What evidence would you need to trust that the agent did the requested job, rather than a well-implemented neighboring one? I'd like to compare approaches, including where they fail.

TanStack Start or Next.js: Follow One Mutation Through Production

Maksym Kuzmitskyi (MaximusFT) — Tue, 20 Oct 2026 00:00:00 +0000

A dashboard loads correctly in both frameworks. That part of the comparison is already fairly well understood.

Now change an order, switch accounts, keep an old tab open through a deployment, and make the request through the production CDN.

That is the comparison I would spend time on. The first screen tells us whether the framework can render. The later operations tell us whether the team understands the state and deployment model it has chosen.

The earlier TanStack Start article introduces SSR and the server boundary. This one examines an authenticated, interaction-heavy application after that introduction. It is not a performance benchmark or a claim of personal production results for either framework.

Version scope: package metadata and official documentation were checked on October 8, 2026: next 16.4.0 and @tanstack/react-start 1.168.60. Features and host integrations below should be checked again against the project's pinned versions before implementation. This site itself still uses its existing Next.js version; writing the comparison does not upgrade it.

Compare a workflow, not an API count

Use one representative flow: load a private order, edit its delivery option, save, return to the list, and sign out.

It exercises identity, server authorization, validation, durable state, multiple cached views, URL state, and browser cleanup. Add an old-client submission after a rolling deployment, and it also exercises release compatibility.

I would build that slice with the deployment target intended for production. A development server is too forgiving a witness for cache behavior and runtime bindings.

The result need not be a second full application. It needs enough ordinary inconvenience to reveal which model the team can explain.

The center of gravity is different

Next.js App Router organizes work around route segments, layouts, Server Components, client boundaries, and its rendering and caching model. That is useful when much of the route can be composed on the server and the team wants the framework to own those conventions.

TanStack Start builds around TanStack Router, loaders, server functions, and the associated client navigation model. That is useful when typed route/search state and client-rich workflows are already central to the application.

Neither description is an exclusive capability boundary. Start's current documentation also describes Server Components; I would not use "Start cannot do RSC" as a timeless reason to choose Next. Nor does a Next.js application forbid TanStack Query or carefully designed client state.

The question is which default model makes the particular workflow easier for this team to reason about, and which extensions create a second overlapping model.

Server functions still need an API boundary

In either system, a function that runs on the server can feel like a local call from frontend code. That convenience does not make the request trusted.

Validate input, authenticate the caller, check access to the specific order, and enforce idempotency where repeated writes matter. A protected page or route guard improves navigation; it does not replace checks in the separately callable mutation endpoint.

For Start, the current server-function documentation describes compilation to client RPC stubs, validation, middleware, and same-origin protections. Next's Server Functions also need explicit authorization in the data path.

I would test both flows by calling the endpoint while signed out and as another account. The implementation syntax differs. The rejected unauthorized write should not.

Keep external APIs separate from framework-private RPC assumptions. Mobile clients and third-party integrations need a deliberately supported contract, not an incidental generated function identifier.

A successful mutation can leave several stale views

After saving the order, which copy changes?

In a Start application using Query, there may be a query cache, router loader data, server-side data caches, and a CDN response. Invalidating one does not automatically invalidate every other layer.

In a Next application, server data and route output can have their own cache behavior, while the browser retains navigation state and perhaps a separate Query cache. Adding Query is reasonable when it serves a clear purpose. Using it merely because it is familiar can create two owners for the same remote data.

I would write down the expected invalidation path before selecting helper functions:

save delivery option
  -> commit the order change
  -> update or invalidate the detail view
  -> refresh affected list representations
  -> preserve unrelated local form state

For Next 16, be explicit about whether the application uses Cache Components or the previous caching model. Advice copied from a different major version or mode may describe different defaults. The current caching guide distinguishes those paths.

I would not choose a framework because its cache seems invisible. Invisible is convenient until two views disagree.

Request isolation matters more than friendly types

On the server, create user-specific state in request scope. A process-wide Query client or mutable user store can accidentally carry account data between requests if used incorrectly.

Test two accounts concurrently through SSR and the real CDN. Inspect returned HTML and serialized data, then switch accounts in the same browser. Sign-out should remove private client cache state as well as the cookie.

The Start production checklist explicitly addresses these checks. They are useful for either framework.

A type-safe cache key still needs the correct identity dimensions. If two tenants share the same key, the compiler can happily confirm that the wrong tenant's response has the expected shape.

Hosting is part of the choice

Start's hosting documentation describes provider-specific integrations and build outputs. Next's self-hosting documentation describes multi-instance caches, deployment identifiers, Server Function encryption keys, and streaming considerations.

Both require attention outside the route code. Verify the adapter, runtime APIs, environment-variable timing, asset paths, and proxy buffering. A container deployment and an edge-worker deployment are not interchangeable merely because both serve HTTP.

For Next across multiple instances, plan shared cache behavior and invalidation coordination where needed. For Start, verify the selected server output and host bindings rather than assuming the development runtime is the production runtime.

Do not judge self-hosting by whether npm start returned a page once. Judge whether the intended features behave correctly through the intended infrastructure.

Keep an old browser alive during the test

Server-function identifiers and client assets belong to a build. A tab can retain an older build while the server deploys a newer one.

Next documents deployment skew protections, including deployment identifiers; their exact behavior matters for self-hosting. Start also generates server-function identities, so test old-client calls against the actual deployed output rather than assuming type sharing guarantees runtime compatibility.

Retain required immutable assets. Decide whether an incompatible client should reload, and what happens to unsaved edits when it does. A full reload is a recovery mechanism, not a product-neutral event.

This check can matter more to an internal editing application than the difference in a clean first-load demo.

Include maintenance in the experiment

Ask another engineer to change the saved field, add a validation failure, and diagnose a controlled stale-view bug in each slice.

How much behavior can they find locally? Which conventions are familiar to the team? Are the deployment docs and libraries they depend on compatible with the pinned versions? Can they explain the cache lifecycle without relying on the person who built the example?

I would prefer this evidence to a general claim that one framework is simpler. Simplicity depends on the workflow and on what the team already understands.

For server-composed routes and a team committed to the App Router model, Next is a reasonable choice. For router-driven, interaction-heavy work with a strong TanStack investment, Start is a reasonable choice. Either can be made awkward by layering on another system without deciding who owns what.

The framework I would choose is the one whose unhappy production path the team can describe clearly. A nice dashboard matters too. Fortunately, both can make one.

If you're weighing the two, bring one mutation and the hosting target rather than a feature matrix. I'd be interested in where the ownership of data becomes unclear in that concrete path.

Freelance Architect or Agency: Who Keeps the Decisions?

Maksym Kuzmitskyi (MaximusFT) — Sun, 18 Oct 2026 00:00:00 +0000

A team asks for an architect, but the work waiting for them is a backlog of implementation tickets.

Another asks an agency to build a feature, while every consequential decision still needs approval from an internal team that has no time to review it.

Neither arrangement is necessarily wrong. They are simply describing a different purchase from the one in the proposal.

When choosing between an independent engineer and an agency, I would spend less time comparing job titles and more time describing who owns the decisions while the work is happening. That tends to matter after the first few meetings, when everyone has access and the easy questions have been answered.

This is a way to evaluate an engagement, not a claim about particular suppliers, prices, or a universal ranking of one model over the other.

First separate diagnosis from capacity

Sometimes the team knows what it needs to build and lacks enough delivery capacity. Sometimes it has plenty of developers but cannot explain why every change crosses five modules. Sometimes a deadline exposes both problems at once.

An architecture-focused independent engagement can make sense when the central work is concentrated judgement: inspect a system, identify boundaries, work with the internal team, and make a few consequential changes.

An agency can make sense when the work needs a coordinated mix of implementation, testing, design, delivery management, and continuity across several people.

Those are tendencies, not guarantees. An independent engineer may deliver a complete feature; an agency may provide one specialist. The actual assigned people and operating agreement matter more than the supplier category.

I would write the first outcome in plain terms. "Reduce the number of modules needed to change policy validation" is more testable than "improve architecture." "Own delivery of this workflow through acceptance and handover" is clearer than "provide a frontend team."

Responsibility needs corresponding authority

If an external architect is responsible for quality but cannot change boundaries, influence sequencing, or obtain review decisions, the responsibility is mostly ceremonial.

The opposite arrangement is also uncomfortable: an external team makes choices that the internal owners must maintain, without involving them until handover.

For decisions with lasting consequences, agree who proposes, who approves, and who maintains. This does not need an elaborate governance document. A small record can be enough:

Decision: separate payment validation from page orchestration
Proposed by: external engineer
Approved by: internal payments owner
Maintained by: payments team
Evidence: two representative flows and regression tests
Fallback: keep the old adapter during migration

The example is hypothetical. Its useful property is that someone still owns the result after the external person's last invoice.

Review availability limits delivery

Adding implementation capacity does not increase the internal team's review or decision capacity automatically.

If an agency submits ten pull requests a week and the internal team can carefully review three, work accumulates at the boundary. More developers may increase that queue rather than shorten the release.

An independent engineer working on a difficult architectural change can encounter the same problem. One large refactor waiting for a busy owner is not inherently easier to absorb than several ordinary features.

I would include reviewer availability in the plan: expected turnaround, who can make product decisions, when integration questions get resolved, and how work is split into changes the team can evaluate.

This is particularly important when acceptance depends on integrations the supplier cannot access or change. A delivery date that assumes immediate internal answers is a date with an unstated dependency.

Prefer demonstrated uncertainty over polished certainty

Before a substantial commitment, a small paid diagnostic or bounded delivery slice can reveal the working relationship.

Choose something representative rather than conveniently isolated: a flow with an awkward API contract, an existing shared component, a realistic test requirement, and an internal reviewer.

The useful output includes what was discovered, what changed, what remains uncertain, and which assumptions affect the next estimate. An apparently slower answer that identifies a dependency may be worth more than a confident schedule built without examining it.

I would also distinguish an estimate for implementation from the time waiting on decisions, environments, and approvals. They belong in the same delivery discussion, but they are not the same kind of work.

Continuity is a mechanism, not a headcount

An independent engineer has a clear availability risk: one person may be absent or the engagement may end.

An agency may offer replacement capacity. That does not guarantee the replacement already understands the system. A named specialist can leave; an account can be reassigned; context still has to transfer.

Ask how continuity actually works. Which people have read the critical code? Where are decisions stored? Can another engineer deploy, reproduce an issue, and explain the outstanding risks without a private conversation with the original author?

For both models, I would favor shared context in the client's systems, small decisions recorded near the code, and at least one internal engineer involved in consequential changes.

Continuity becomes less fragile when it is exercised during delivery rather than promised in a final document.

The work should remain inside the client's control

Source repositories, deployment accounts, domains, package ownership, and design assets need deliberate ownership. An engagement can finish with working software and still leave the client dependent on credentials or infrastructure controlled by the supplier.

Use client-controlled repositories and appropriately scoped access where possible. Name any exception and the transfer procedure before it becomes necessary.

The same applies to automation. If a supplier uses AI-assisted tooling, clarify permitted data handling, review expectations, generated-code responsibility, and what configuration belongs to the client. Faster implementation does not change who must verify the result or who may access private source and customer information.

I would not equate a supplier's preferred toolchain with a requirement for the client to adopt it permanently.

Avoid a handover that only describes files

A directory tour helps. It does not answer why a boundary exists, which alternatives were rejected, or what would break if the API changes.

A useful handover includes the important decisions, reproducible checks, release and rollback steps, known limitations, and the first likely maintenance changes. Keep it proportionate; six understandable notes are better than a large template nobody can navigate.

Have an internal engineer perform one change and one deployment before the engagement ends. The external person can assist, but should not quietly do every step that the exercise is intended to transfer.

That is a more convincing ownership test than asking whether everyone has read the documentation.

Architecture needs an implementation consequence

If the engagement is mostly architecture, the deliverables should still meet the code at some point.

A dependency diagram can identify a problem. A bounded migration, representative slice, or working contract test shows whether the proposed boundary is usable. Without that contact, the design may be elegant while the team's practical cost remains unknown.

Conversely, measuring only shipped tickets can discourage the diagnostic work needed to prevent the next expensive change. Agree which technical outcomes matter: fewer coupled changes, faster verification, clearer ownership, or an integration that no longer needs several teams to coordinate every edit.

I would avoid treating diagrams and implementation as competing definitions of productive work. They should inform each other, with evidence appropriate to the risk.

Choose the arrangement that can leave well

For a focused diagnosis with a capable internal team, I would often consider an independent senior engineer or architect. For a delivery stream needing several disciplines and coordinated capacity, I would consider an agency with named people and explicit ownership.

If the organization has nobody available to own decisions, review changes, or maintain the result, neither category solves that absence by itself. The operating model needs attention before more external capacity arrives.

The end of the engagement is a useful design constraint from the beginning. What should the internal team be able to do without asking the supplier? Which promises must remain true when a person changes?

I would rather make those questions slightly awkward during selection than leave them until the final week. By then the software may be complete, and the ownership may still be under construction.

If you're deciding between these arrangements, tell me which decision or delivery responsibility is currently unowned. That detail makes the comparison much more useful than the labels on the proposals.

What a Skill Changed, and What It Didn't

Maksym Kuzmitskyi (MaximusFT) — Sat, 17 Oct 2026 00:00:00 +0000

A task went well with a reusable skill enabled.

It also had clear requirements, repository instructions, connected tools, a useful test, and an engineer who corrected an early assumption.

Which part made the result better?

I would not answer that from the final diff alone. The diff can establish what changed. It cannot, by itself, tell us which part of the workflow caused the change to be correct.

I participate in evaluating reusable agent skills and refining how they are applied to ordinary engineering work. The interesting question is not whether the process produced a polished artifact. It is what observable contribution the skill made, what it cost, and what remains unknown.

This article describes an evaluation approach. It does not report internal programme results, measurements, or a controlled comparison that has established a productivity gain.

A skill is one input among several

A reusable skill packages instructions or a workflow. It may help discover context, plan a change, validate it, or handle review. It is not a separately trained model or an autonomous employee.

Its contribution depends on what was already present.

If repository rules already require a focused check immediately after an edit, a skill that repeats that instruction may add consistency, but it is not evidence that the check originated with the skill. If a connected tool supplies current requirements, the tool can explain why the agent avoided a stale assumption.

The engineer's domain knowledge matters too. A person may notice that two services disagree, narrow the investigation, or reject an obsolete proposal. Leaving that correction out makes the skill look more independent than it was.

I want the record to include those existing controls. Otherwise we are comparing a named framework with an imaginary workflow that has no useful practices of its own.

Observe behavior before assigning credit

The earlier evaluation article discusses what to measure. Here the narrower question is which explanation the observations support.

Use a deliberately illustrative record, not a real reporting form or an internal task:

Task: update a client without changing its callers
Observed behavior: agent checked a contract before editing
Existing control: repository instructions also require that check
Correction: engineer identified one stale source
Verified outcome: retained change passes the relevant gates
Skill contribution: may have reinforced discovery; attribution unresolved
Residual risk: deployment compatibility needs a separate decision

The conclusion is modest because the evidence is modest. That is not a failed evaluation. It is an evaluation that did not replace uncertainty with enthusiasm.

If the skill visibly introduced a previously missing check, say so. If the task does not distinguish its effect from existing instructions, say that instead.

A useful outcome is not always delivered code

A cross-repository investigation can end with an obsolete PR being closed. An architecture task can produce a proposal with explicit assumptions for discussion. A session-handoff change can produce a clearer continuation scheme.

Those outcomes have value. They are not interchangeable with a merged feature, an approved architecture, or guaranteed reproducibility.

Describe the actual artifact and its decision state. A reviewable proposal is ready to evaluate; it is not production rollout. A documented handoff reduces reliance on conversation history; it does not establish identical behavior across models and people.

In the supplied generalized experiences behind this series, recovery and refinement are part of the work. I would not remove the initial incomplete diagnosis simply because the final action was sensible.

The sequence explains what the process learned. A cleaned-up success story explains much less.

Separate work from overhead without pretending overhead is worthless

Implementation, framework setup, planning, repeated approvals, and reporting take different kinds of effort.

A specification can prevent an expensive mistake. It can also repeat a small task's already clear requirements. Reporting can support honest evaluation. It can also consume time that should be counted rather than hidden.

Keep normal engineering effort, framework overhead, and reporting effort distinguishable. Do not turn them into a single impressive number before understanding what each category contains.

The classification will sometimes require judgement. A discussion that clarifies an ambiguous requirement is engineering work even if a framework prompted it. A template completed only to satisfy the framework is a different contribution.

I would make that distinction visible rather than use it to prejudge whether the framework is good or bad.

Tiny tasks do not need a retrospective specification

For a small, clear change, the ticket and nearby contract may already be an adequate written requirement. A long specification can create another artifact to review without resolving a real uncertainty.

For ambiguous, risky, or multi-session work, a plan and checkpoints before implementation can be useful. They capture assumptions while there is still time for them to guide the work.

Do not write a specification afterwards and present it as evidence that the task followed specification-driven development. A retrospective note can explain the result, but it did not shape decisions that happened before it existed.

This is a simple chronology rule. It prevents polished documentation from making the process look more formal than it actually was.

Not using a skill can be the proportionate choice

An evaluation can tempt people to apply every available skill to every task so the record has plenty to discuss.

That changes the thing being evaluated. We are no longer observing useful selection; we are imposing ceremony for evidence collection.

Use a skill when it helps the task's current stage and risk. Record when a tiny or non-coding task offers limited evidence. Do not manufacture a repository-wide plan or another planning file after the fact to fill a reporting category.

A good workflow should be able to say "not needed here" without that becoming an admission that the whole framework failed.

Repeated success still has confounders

Several tasks going well can suggest a useful pattern. It does not automatically establish a causal productivity improvement.

Tasks differ in difficulty, familiarity, requirements, code quality, tools, models, and human involvement. An experienced engineer may become better at the domain during the same period the skills are introduced.

Without an appropriate baseline and comparison, avoid percentages, cost savings, or defect-rate reductions. Task counts and green checks are observations, not impact metrics by themselves.

Qualitative findings remain worthwhile: a missing boundary became explicit, a handoff had a clearer source of current decisions, or the process detected an unrelated generated change before review.

Name them as observations. They do not need a fabricated multiplier to be interesting.

Keep the evaluation record separate from public storytelling

An internal assessment may need task-level evidence, correction history, and reporting detail. A public article does not need to reproduce those artifacts.

Use generalized, explicitly labelled composite examples. Remove internal identities, programme details, source snippets, links, counts and model policy. Do not attach an employer's name to an anonymized case merely because the employment record is already public.

Even a collection of individually harmless details can make an episode identifiable. Preserve the engineering mechanism while changing or omitting unnecessary specifics.

The public claim should be something the available evidence can support without inviting the reader to inspect a closed system. "I evaluated and refined workflows" is a different claim from "this programme improved company delivery."

Feed the result back into the right layer

When an error repeats, decide which control should change.

An outdated repository convention belongs in repository context. A general validation habit may belong in a reusable workflow. A missing permission boundary may require tool access to change, not another paragraph in the prompt.

If the failure came from stale task evidence, repair the handoff or current-state check. If the skill added unnecessary steps, narrow when it applies. If the result remains ambiguous, keep the ambiguity in the record rather than forcing a favorable or unfavorable rating.

This is where evaluation becomes engineering work. The record is not the final product; the corrected operating arrangement is.

Useful attribution is specific enough to change the next task and modest enough not to invent a cause.

A smaller conclusion can be more reusable

I do not need every evaluation to end with a winner. I need to know which practice helped under which conditions, what required correction, and what the evidence could not distinguish.

That lets the next task inherit a more useful check or a simpler process instead of a larger claim.

The five articles in this wave share that approach. Context needs freshness. Green checks need scope. Tool output needs inspection. Reviews need a revision. Skills need honest attribution. None of those mechanisms removes the need for engineering judgement; they give it better material to work with.

That is the agentic engineering I want to describe: an environment that can learn from ordinary failures without pretending the agent was correct all along.

If a skill helped your work, what changed in the actual sequence of decisions? I'd be interested in that observation, including what your existing tools or judgement contributed alongside it.

The Backup Exists. The Restore Still Needs Work.

Maksym Kuzmitskyi (MaximusFT) — Fri, 16 Oct 2026 00:00:00 +0000

The database restores successfully to 10:42.

The payment provider has transactions through 11:07. Object storage contains invoices created after the restore point. The queue still holds jobs referring to orders the restored database does not remember.

The backup worked. We have not yet recovered the application.

I think this is the useful distinction to make before an incident. Recovery is not one storage operation. It is deciding which facts remain authoritative after different parts of the system have moved to different points in time.

Availability and recovery solve different failures

A replica can keep the service available after one machine fails. It may also faithfully replicate the accidental deletion we are trying to undo.

A backup preserves another state. Point-in-time recovery can reconstruct a database using a base backup and subsequent transaction-log records, within the retained recovery window. Its value depends on those records being available and the restore procedure actually working.

Neither mechanism automatically restores object storage, credentials, queue state, or external services.

I would define the failure being covered: infrastructure loss, accidental deletion, corruption, an incompatible migration, or compromised access. A design that handles the first can still be inadequate for the others.

Put a business meaning behind RPO and RTO

Recovery point objective describes acceptable data loss. Recovery time objective describes acceptable time to restore the required service.

"Fifteen minutes" sounds specific. It still needs a meaning. Can fifteen minutes of submitted support forms disappear? Of inventory reservations? Of charged orders? Those are different promises.

Some operations need reconciliation from another authoritative system even when the database recovery point is acceptable. A payment provider may tell us about a charge the restored database lost. We need a stable reference linking it back to the business operation.

For recovery time, include more than restoring bytes: provisioning, access approval, decryption keys, application startup, verification, reconciliation, traffic switching, and safe resumption of workers.

I would measure the whole exercise. A ten-minute restore inside a two-hour recovery procedure is not a ten-minute RTO.

Restore beside production, not blindly over it

For a logical data mistake, restoring an isolated instance gives us room to inspect the recovered state and compare it with current production. We may be able to repair a specific set of rows instead of rewinding everything.

Keep the recovery environment unable to send real emails, charge payments, consume production queues, or execute live webhooks. A copied database can contain pending work that application startup eagerly resumes.

Use controlled configuration and network access, not merely a promise that nobody will click the dangerous button. Preserve the original state as evidence while deciding which changes to apply.

Sometimes a full cutover is necessary. Even then, the new instance should be verified before traffic moves, with a deliberate plan for connections, endpoints, and old application instances.

DNS changes have the overlap behavior discussed earlier. Applications may also cache connection settings or continue using old pools. Changing a database endpoint is a release step, not a universal disconnect command.

Match data with code and configuration

A database restored to yesterday may have yesterday's schema. Today's application may require a column that did not exist then.

Record the schema migration state alongside artifact and configuration revisions. Retain deployable artifacts, not just source code that could theoretically be rebuilt with today's package registry.

An expand/contract migration gives recovery more options, but it does not remove the need to choose a compatible application version. Running every pending migration automatically against the restored instance may recreate the destructive change we were trying to investigate.

Secrets matter too. A backup encrypted with an unavailable key is not useful. A restored application with revoked credentials cannot necessarily reach its dependencies. Back up and test the recovery of access paths without scattering secret values through documents or logs.

External effects do not rewind with the database

Consider an order that was paid at 10:55 and lost from a database restored to 10:42. Asking the customer to retry can charge them again unless the payment operation has a durable, stable identity.

Reconcile against the provider using idempotency keys, merchant references, and the provider's authoritative transaction state. Do not infer a refund or charge from the absence of a local row alone.

Email is another external effect. A replayed confirmation job may send a second message. An invoice number may already have been issued. A shipment may already be in transit.

The recovery procedure should describe those consequences and the systems that settle them. We cannot restore the world to match a snapshot. We can reconstruct our records so they agree with the world that continued operating.

This is where the identity choices from the API and queue articles become more than implementation conveniences.

Queues and caches need a recovery policy

A restored outbox may contain events already published before the incident. A current queue may contain messages for records absent from the recovered database.

Pause consumers while reconciling. Decide which messages to preserve, replay, quarantine, or discard. Idempotent consumers reduce the cost of duplicates, but they still need a coherent business state to operate on.

Do not delete the queue simply to make its depth look tidy. It may contain the only remaining record of accepted work.

Redis also needs role-specific handling. Rebuild a disposable product cache from the recovered source. For sessions, consider whether invalidating them is safer than restoring older authorization state. For locks or scheduler metadata, old values may be actively misleading.

"Restore every dependency" is not always the right instruction. Some state should be restored, some reconciled, and some deliberately recreated.

Verify meaning, not only row counts

A database that starts and contains the expected number of rows is encouraging. It does not prove the checkout workflow is usable.

Check relationships, critical constraints, recent business operations, object references, account permissions, and one controlled end-to-end flow. Compare representative records with external authoritative systems where necessary.

Keep enough telemetry to tell when corrupted or missing data first appeared. Otherwise choosing a restore point becomes a sequence of guesses, each followed by another expensive restore.

Data integrity checks and recovery logs should avoid publishing private customer content. The people conducting a recovery need controlled access, not an unrestricted data export pasted into the incident channel.

Test the access path that an incident will use

A recovery exercise run by an administrator with every permission can conceal missing privileges in the actual emergency role. Test who can start the restore, decrypt it, deploy compatible code, change routing, and release the isolation controls.

For compromise scenarios, consider independent backup protection: restricted deletion, retention controls, and appropriate account or region separation. If the same credential can destroy production and every backup, the copies share a rather important failure mode.

There is no universal arrangement that makes all of this free. The choices should reflect the actual data, recovery promise, and threat model.

Document who makes the cutover decision and who handles reconciliation. That is operational ownership, not an excuse to collect an impressive runbook nobody has tried.

A useful drill has an inconvenient detail

Try restoring with an old schema, an unavailable primary region, a rotated credential, or a job already delivered once. Choose one complication rather than pretending the exercise must reproduce every disaster at once.

Measure the result, record the steps that required undocumented knowledge, and repair those gaps. A failed rehearsal is useful when it changes the recovery path before the real incident.

A backup preserves data. A rehearsed recovery preserves the ability to use it safely.

That is where I would end this field guide. The database, cache, queue, proxy, and application are not independent checkboxes. Their boundaries become visible when a request fails, a deployment overlaps, or one component returns to an earlier state.

I would be satisfied with an ordinary system whose recovery procedure is unremarkable because it has been tested. That seems a more useful goal than another layer whose failure behavior we will learn later.

If you've attempted a restore rehearsal, what took longer than the database restore itself? I'd be interested in the dependency or decision the backup dashboard could not account for.

A Review Belongs to a Commit

Maksym Kuzmitskyi (MaximusFT) — Thu, 15 Oct 2026 00:00:00 +0000

A reviewer sees a change and raises a useful concern. The author pushes another commit.

The comment is still there. The code it described may not be.

Human reviewers handle this imperfectly too. An automated reviewer makes the state-management problem easier to notice because it can repeat the same concern every time it runs, or confidently act on a conclusion from the previous revision.

The earlier PR-reviewer article focused on useful language and visibility into a personal tool. This article is about a narrower engineering problem: what a finding, check, or review decision refers to as the code changes.

I have developed commit-aware tracking, duplicate-finding safeguards, dry-run checks, and guarded actions in personal review tooling with AI assistance. That describes development of mechanisms, not current continuous operation, organization-wide adoption, or a replacement for human approval.

A PR number is not the reviewed object

The PR is a container for an evolving proposal. The head commit identifies one version of that proposal. The base revision and comparison mode also affect what the reviewer sees.

If an automated tool records only "PR reviewed," it cannot distinguish unchanged work from a new commit. If it records only head SHA, it may miss a changed base or integration result that matters to its conclusion.

A useful record identifies what was reviewed and under which assumptions:

type ReviewSnapshot = {
  headSha: string;
  baseSha: string;
  comparison: 'pull-request-diff';
  rulesRevision: string;
  checkedAt: string;
};

This is an illustrative type, not an internal implementation. The rules revision matters because the same code can be assessed differently after the checking policy changes.

I would not let a convenient database row imply more history than it contains. "Reviewed" should be explainable as a particular observation, not a permanent property of the PR.

CI has a revision too

A green check from an earlier commit does not establish the state of the current one. A required integration check may run on a merge result rather than the branch head. Those are different artifacts.

Before review actions, identify which checks are relevant, which revision they exercised, and whether they are finished. A skipped check or an optional success is not necessarily equivalent to every required gate passing.

Even then, CI evidence has the scope limits discussed in Green Checks Only Cover What You Checked. It can support a reviewable state without proving the absence of semantic defects.

I want a tool to report that distinction rather than promote "green CI" into "safe to approve."

A finding needs identity and evidence

Line numbers are useful for placement. They are poor long-term finding identities.

An author can move the same risky expression to another line. The finding remains relevant, but a line-based key changes and the bot may publish it again. Conversely, new code at the old line can have a different risk that a naive key suppresses.

A more semantic identity can include the rule, affected symbol, risk category, and relevant normalized evidence. Keep the exact source revision alongside it.

Normalization introduces its own trade-off. Strip too much context and different findings collapse into one fingerprint. Keep every character and harmless formatting changes produce duplicates.

Treat a fingerprint as a useful matching mechanism, not proof that two concerns are identical. Re-evaluate the current evidence before deciding the old finding still applies or no new comment is needed.

Suppressed does not mean resolved

A repeated finding can be current and intentionally not republished. A missing finding can mean it was fixed, became unobservable in the selected diff, or was skipped because the analysis failed.

Those states need different names.

current: the concern is supported by the latest evidence
unchanged: current, but no repeated comment is needed
resolved: evidence supports that the concern no longer applies
not-observed: this run did not establish either outcome
needs-human: the tool cannot settle the question safely

The exact storage model can vary. The distinction keeps duplicate suppression from quietly becoming automatic resolution.

An unresolved human thread is another state the tool should not overwrite simply because its own detector no longer sees the pattern. The thread may concern context or a decision that the detector never understood.

Recheck state before publication

Analysis can take time. While it runs, the author can push new code. A comment created from the original diff may now point to invalid lines or describe a risk that has already changed.

Re-read the relevant state before publishing. If it no longer matches the reviewed snapshot, stop, discard the pending action, or schedule another review. Do not silently attach the finding to a different revision because that makes the API call easier.

The same applies to stronger actions. A guarded action should carry the expected revision and use a platform-supported condition where available:

type GuardedReviewAction = {
  expectedHeadSha: string;
  snapshotId: string;
  action: 'publish-review';
};

This type expresses the requirement; it does not implement an atomic guarantee. Reading head and then calling an API leaves a time-of-check/time-of-use interval. If the platform cannot condition the action atomically, handle that limitation explicitly and avoid unsupported claims about stale-action prevention.

A fresh check reduces risk. It does not freeze another person's branch.

A dry run has its own evidence boundary

A dry run can show which findings and actions a tool would produce. It is useful for inspecting scope, tone, duplicates, and targeting without creating external effects.

It does not prove that the later API call will be authorized, attach to the intended line, or observe the same PR state. Nor does a successful API call prove the review content was correct.

Use the dry run to inspect proposed actions. Use publication checks to verify current state and delivery. Keep the two results separate in the control interface.

If the inline attachment fails, a body-level review may preserve the concern, but should still identify the source revision. A fallback should not turn a precisely scoped observation into an unexplained general warning.

Human decisions need context, not silent reassignment

A person may approve a particular revision. New code can change the basis for that decision. The tool should make that change visible and follow the platform's review policy rather than assume an earlier approval means the new code is accepted.

Whether an approval is dismissed, still valid, or requires a new decision depends on the repository's settings and accountable reviewers. The automation should not invent its own release authority.

Likewise, an AI suggestion that a dependency update appears routine is evidence, not permission to merge it. Any permitted automatic action needs its own narrowly defined authorization and technical controls. It should not be inferred from the absence of findings.

A review result describes observed code. It does not transfer responsibility for whatever code arrives next.

Preserve enough history to explain the tool

The control surface should answer ordinary questions: which revision was analyzed, what was published, what was suppressed, why a run stopped, and what remains undecided.

Record identifiers and outcomes with appropriate retention. Avoid storing raw credentials, private payloads, or unnecessary code copies just to make a log feel complete.

When the tool is wrong, the history should help reproduce the decision. Was the input stale? Did normalization merge two findings? Did a missing detector result get treated as resolution? Did the action target change between analysis and publication?

Those are repairable mechanisms. Without the history, every correction becomes a guess about the model's intentions.

The quieter reviewer is not necessarily the better reviewer

Duplicate suppression makes the conversation less noisy. That is useful. It can also hide an unresolved issue if the lifecycle is poorly modelled.

I would evaluate the tool on the relevance and traceability of its findings, not merely on how few comments it emits. Check unchanged code, moved code, corrected code, a new risk at the old location, and a head update during analysis.

Keep a human in charge of interpretation and approval. A review assistant should make that person's evidence easier to find, not create a reassuring status that no one can explain.

The next article, What a Skill Changed, and What It Didn't, examines the same attribution problem at workflow level: what a reusable skill changed, and what it did not.

If your review automation remembers that it saw a PR, what exactly does it remember about the code? That is a useful place to inspect before adding another action to it.

The Dashboard Is Green. The Order Is Missing.

Maksym Kuzmitskyi (MaximusFT) — Wed, 14 Oct 2026 00:00:00 +0000

The API error rate is low. The workers are running. The browser reported no uncaught exception.

A user says their order never appeared.

All of those observations can be true. We measured the health of individual pieces, while the user described an outcome across several pieces. The missing part is the connection between them.

I do not think the answer is simply to log more. A larger pile of unrelated events gives us more to read, which is not quite the same thing as giving us more to understand.

Start with the outcome we need to explain

For checkout, I would distinguish browser intent, request acceptance, order persistence, payment confirmation, and follow-up work.

These are not synonyms. A click event proves that someone activated a control. A 200 response proves that an HTTP exchange returned that status. A database record proves that an order was stored. A confirmation email proves another effect entirely.

The analytics layer and an operational trace also serve different purposes. Product analytics can describe behavior; operational telemetry needs enough context to reconstruct a specific failure without assuming every recorded click became a completed purchase.

I would define the business success point explicitly, then work backwards to the evidence required at each boundary. Otherwise the most available metric quietly becomes the definition of success.

A request id is useful, but it is not the whole identity

A request id correlates one HTTP attempt. An idempotency key identifies one intent across retries. An order id identifies durable business state. A trace context links spans across participating services.

Keeping these separate is helpful:

{
  "event": "order.persisted",
  "request_id": "req_9c7",
  "operation_id": "checkout_62a",
  "order_id": "order_18f",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "release": "checkout-2026-10-14"
}

These are synthetic identifiers. The record does not include a customer's name, email, address, token, or raw payment data.

A retry gets a new request id but retains the operation identity. A background job can link to the operation after the original request has ended. Depending on timing and instrumentation, a span link may describe that relationship more honestly than an endlessly extended parent span.

Use established trace propagation and instrumentation, such as OpenTelemetry, rather than inventing a collection of vaguely compatible headers. Still validate untrusted incoming context and keep authorization separate from any telemetry identity.

Missing data has several explanations

If a trace is absent, the request may never have happened. It may also have been sampled out, rejected by a collector, dropped from a full buffer, or generated by an uninstrumented component.

Head sampling decides early which traces to retain. It keeps costs bounded, but it cannot know at the start which request will fail late. Tail sampling can select after observing more of the trace, at the cost of buffering, coordination, and its own resource limits.

Neither gives perfect evidence automatically. Keep critical outcome counters and durable business records independently of a sampling decision. Record the telemetry pipeline's own drops and health so absence is less ambiguous.

For frontend error reporting, the same limitation applies. Browser extensions, network conditions, consent settings, and navigation can affect delivery. No recorded exception is not proof that no exception occurred.

Denominators need their own definition

"Twenty failed orders" is difficult to interpret without knowing how many attempts occurred and which attempts count.

Counting API requests exaggerates user intent when retries occur. Counting clicks includes actions rejected before a request. Counting created orders excludes failures before persistence.

For an outcome metric, define both numerator and denominator around the promise being measured. For example: distinct accepted checkout operations that reach a confirmed or explicit terminal state within a chosen interval.

Then describe exclusions: abandoned carts, expected validation failures, test traffic, and operations still inside their allowed processing window. An illustrative target is not a universal SLO.

The arithmetic is usually easy. The difficult part is resisting a reassuring percentage with a poorly chosen population.

Cardinality is a modelling choice

Metrics work well with bounded dimensions such as route templates, status categories, region, and service.

Attaching order_id, raw URLs, or customer email to every metric label creates huge numbers of distinct time series. It also turns the monitoring system into a rather expensive and unsuitable customer index.

Use /orders/:id, not every concrete order URL, as a metric dimension. Put specific identities in controlled logs or traces when needed, with retention and access appropriate to their sensitivity.

Histograms deserve some care too. Percentiles computed independently by instances cannot generally be averaged into the fleet's percentile. Aggregate compatible histogram observations rather than averaging whichever p95 numbers happen to be on the screen.

The goal is not to avoid detail. It is to put detail into the kind of evidence that can handle it.

Frontend context can make a backend trace usable

A server sees requests. The browser knows which route, interaction, and release led to them.

For an error-reporting tool such as Sentry, a release identifier and matching source maps can turn a minified stack into something useful. Upload source maps through controlled tooling; decide whether publicly serving source files is intended. A production error should identify the bundle that actually ran, not the latest source in main.

Session replay can help explain interaction sequences, but I would not enable it broadly without reviewing masking, consent, sensitive fields, retention, and access. "It is an internal admin screen" is often a reason for more care, not less.

Expected cancellation should not appear beside faults as though a user navigating away is a broken system. The AbortController article covers that distinction at the interface boundary.

An alert should name a response

High CPU may deserve investigation. Missing orders deserve investigation for a more direct reason.

Prefer alerts connected to user-visible impact or exhausted safety margins: sustained error-budget consumption, queue age beyond the product allowance, connection-pool waits, failure to complete accepted operations, or a recovery dependency approaching expiry.

The notification needs an owner, a scope, and a useful next action. A chart link is helpful. So are a recent deployment marker and a runbook that says which evidence separates an application failure from a shared-dependency failure.

I would not page someone every time one instance reports a brief spike unless that spike calls for a meaningful intervention. Noise does not stay harmless; it changes what people notice later.

Telemetry is another dependency

Exporters have buffers, network calls, timeouts, and shutdown behavior. A slow collector should not cause the checkout handler to wait indefinitely or allocate unbounded memory.

Use bounded, asynchronous export appropriate to the instrumentation. Decide what can be dropped and what must be durably recorded elsewhere. Business events required for fulfilment belong in a durable mechanism such as the transactional outbox, not solely in a best-effort log stream.

Also test redaction failures and collector outages. Observability should help us explain failure without becoming its next cause or a route for leaking data.

A telemetry event is evidence with delivery conditions, not a second database transaction.

Reconstruct one unsuccessful operation

I would exercise a deliberately failed checkout and try to follow it from browser intent to its final state. Can we distinguish a request that never arrived from an order that persisted but lost its response? Can we find the background job? Can we identify the release and a retry without counting them as two purchases?

If those questions are difficult, another dashboard will not necessarily improve the situation. A better identity or an event at the missing boundary might.

The green dashboard becomes useful when we know exactly what its green means. Until then, I would keep the user's missing order as the more interesting observation.

The final part of this guide covers backups and recovery: what happens when diagnosis is finished and some part of the system actually has to be rebuilt.

Which production question still sends you comparing timestamps across three tools? I'd like to hear the missing connection, not just which monitoring product you use.