DEV Community: jamilxt

Kwi vs Konsole: Should KDE Developers Switch Terminals?

jamilxt — Mon, 02 Nov 2026 12:04:01 +0000

A terminal can feel slow for two very different reasons. Sometimes your command is doing expensive work. Sometimes the terminal is struggling to paint the output quickly enough to keep up. Those problems look similar when a screen fills with text, but replacing the terminal only helps with the second one.

That distinction is why KDE's new terminal, Kwi, is interesting. Announced on November 1, 2026, Kwi is not simply a faster Konsole release. It is a new implementation built around GPU rendering, Qt Quick, and Ghostty's terminal parsing library. Its developer reports results from 3x to 35x faster than Konsole depending on the benchmark, including 870 frames per second in a DOOM-fire benchmark compared with 23 in Konsole. Those numbers come from the project's developer, not an independent benchmark, and the same announcement says Kwi has not reached a tagged 0.1 release yet. KDE's announcement explains the architecture, benchmark, and early status.

So should a KDE user replace Konsole with Kwi? For most people today, no. Keep Konsole as the dependable default, and treat Kwi as an early project worth testing if your workload exercises the areas it is designed to improve. The more important story is architectural: KDE is exploring a shared terminal component for its newer Qt Quick applications, while Konsole remains the mature, feature-rich application that people already use.

This comparison is based on public project information, not hands-on testing. Kwi is still in active development, so this is a guide to deciding what to evaluate rather than a claim that it is ready for everyday production use.

What is different under the window?

Konsole is the established application. KDE describes it as a terminal for running a command shell. Its documented features include tabs, multiple profiles, activity and silence monitoring, bookmarks, search, and saving output. It is also used by other KDE applications as an embedded terminal. Its long history matters: Kwi's developer says Konsole has had 29 years of continuous development and is used by millions of people. KDE's Konsole page lists its features and supported installation routes.

Kwi is a newer architecture, not just a visual refresh. Its developer says Kwi uses libghostty-vt for parsing terminal output and maintaining terminal state, KPty for pseudo-terminal integration, and Qt Quick's scene graph for a GPU-accelerated renderer using Vulkan or OpenGL. Its interface is built with Kirigami, KDE's UI framework for modern KDE applications. In practical terms, the project is trying to reuse a dedicated terminal parser and put more of the display work on the graphics stack used by newer KDE software. The Kwi announcement describes these components and their roles.

That design difference helps explain why Kwi's headline performance number needs context. A terminal emulator receives bytes from a program, interprets escape sequences and terminal state, lays out text and graphics, then draws the result. A workload that produces a constant stream of visual updates can stress the rendering path. A command that waits on a database or downloads a file may barely exercise that path at all. A benchmark result on a display-heavy workload is evidence about that workload, not a universal multiplier for every shell command.

Kwi vs Konsole, by the decisions that matter

Rendering and high-volume output: Kwi is the one to test. Kwi's GPU renderer is the central architectural bet. KDE's announcement reports the 3x to 35x range and the specific 870 versus 23 fps result. That is a large enough claim to justify testing on the sort of output that makes your current terminal visibly lag. But the project has not published a broad, independently reproduced benchmark suite in that announcement. Hardware, graphics drivers, resolution, terminal settings, and the exact workload can all change the outcome. Do not convert one benchmark into a promise that builds, tests, or remote sessions will run faster.

Everyday reliability and feature coverage: Konsole has the advantage. Konsole is already a released KDE application, with profiles, tabs, bookmarks, search, and saved output documented on KDE's software page. Kwi is explicitly described as young, still in development, and not yet at version 0.1. The Kwi page in KDE's application directory also says that it has not been released yet. If a terminal is part of your daily workflow for deployments, incident response, or long-running sessions, maturity is a real feature. KDE's current Kwi listing confirms its unreleased status.

Modern terminal protocols: Kwi has a promising direction, not yet a maturity win. The Kwi project lists more complete support for modern terminal features, including Kitty image and animation protocols, clipboard protocols, keyboard behavior, progress reporting, and color scheme queries. It also highlights improved text rendering, ligatures, fractional scaling, accessibility, and screen-reader text extraction. Those capabilities may matter to users of modern terminal user interfaces, but a feature list is not the same as broad compatibility testing. Check the applications and workflows you actually rely on before treating support as complete in practice. The project announcement has the detailed feature list and caveats.

Integration into KDE apps: Kwi could fill a gap. The project is not only trying to build another terminal window. Its developer says Kwi provides a public Qt Quick/QML module that other KDE apps can embed. Konsole already has an embeddable component used by QWidget-based applications, but the announcement identifies a missing equivalent for the newer Qt Quick and Kirigami stack. If that component works as intended, Kwi could make it easier for new KDE applications to include a consistent terminal without each app solving the problem independently. That potential is for application developers, not a reason by itself for every desktop user to switch.

Configuration and migration: Konsole is safer today. Kwi already supports Konsole color scheme files, but the project notes that migration is not finished. Existing QWidget applications that embed Konsole's component need a code change to request Kwi's component, and the project has not settled on a central way for users to express a terminal preference. That is an important distinction between technical compatibility and a smooth user migration. KDE's announcement calls out both the color scheme support and the unresolved transition path.

Availability: Konsole is ready, Kwi is for evaluation. KDE lists Konsole as installable through Linux distributions and Flathub. For Kwi, the announcement points to nightly Flatpak builds from KDE's CI and warns that its fresh dependencies can make building difficult. Nightly packages are useful for testing, but they are not the same promise as a stable release. Do not put a changing nightly build into a workflow that needs predictable upgrades unless you are comfortable diagnosing regressions. KDE's Konsole page gives its distribution and Flathub options, while the Kwi announcement describes its nightly-build route.

The performance claim needs a fair test

If you want to know whether Kwi is faster for your work, avoid comparing a single dramatic demo with a vague impression. Pick one repeatable workload that currently makes Konsole struggle. For example, use the same terminal size, font, theme, machine, and output command in both terminals. Measure wall-clock completion time separately from perceived smoothness. A faster animation does not necessarily mean the command itself completed sooner.

Try at least three kinds of work:

A rendering-heavy workload: a continuously updating TUI, large output stream, or terminal graphics demo. This is the kind of case most likely to expose renderer differences.
A normal developer session: editing in a terminal UI, running tests, navigating history, and using the tabs or profiles you depend on. This checks whether the rest of the experience is usable.
A reliability check: run your actual shell setup, SSH workflow, clipboard path, screen-reader or accessibility tools if relevant, and any terminal applications that use images or custom keyboard protocols.

Keep these questions in the notebook:

What exact command or TUI is slow, and is the delay in rendering or in the program itself?
Does Kwi improve that same workload on your hardware, or only the project's illustrative benchmark?
Does it handle your shell, fonts, keyboard shortcuts, clipboard, and terminal applications correctly?
Can you recover your existing color scheme and workflow without rebuilding settings by hand?
Is a nightly build acceptable for this machine, or should you wait for a tagged release and distribution package?

That checklist is more useful than picking the terminal with the biggest advertised number. It turns a benchmark claim into a personal, repeatable decision and keeps an early experiment out of a critical workflow until it earns trust.

Who should use which?

Choose Konsole for now if you want a stable, familiar KDE terminal with established features, or if you depend on an integrated terminal in an existing KDE application. KDE itself describes Kwi as an option under development, not a decision to replace Konsole. That distinction matters: a new project can be valuable without making the existing one obsolete.

Try Kwi in a separate test setup if you use KDE Plasma, care about GPU-rendered terminal output, rely on modern terminal protocols, or build Qt Quick/Kirigami applications that could benefit from an embeddable terminal component. Treat it as a project to evaluate, not as the default replacement for a production shell environment. If the first result is compelling, retest after updates because the code and dependencies are changing quickly.

Wait if you need a stable release, complete migration support, or evidence from benchmarks that cover your workload. The project is young, and its maintainer explicitly asks readers to expect bugs. Waiting is not resistance to progress; it is matching the maturity of a tool to the cost of a failure.

The verdict is straightforward: Konsole wins on readiness and established workflows. Kwi is the more interesting experiment for rendering performance and Qt Quick integration. The 870 fps result is a reason to pay attention, not a reason to uninstall a terminal that already works.

I write about developer tools, infrastructure, and AI each week. Subscribe if you want practical comparisons that separate a promising claim from a dependable tool.

Have you tested Kwi or another GPU-rendered terminal? What workload would convince you to switch from Konsole?

Sources

CVE-2026-98432: How to Find and Patch Vulnerable Linux Hosts

jamilxt — Mon, 02 Nov 2026 03:04:57 +0000

A server can have every application package updated and still be running the old kernel. That gap matters when the vulnerability is in the kernel's network path, and it is why “the package manager says everything is current” is not a complete incident response.

CVE-2026-98432 is a use-after-free flaw in Linux's IPv4 networking code. Red Hat describes a race during TCP connection-request processing that can free socket options while another operation is still using them. Its current bulletin rates the issue Critical and says a remote, unauthenticated attacker may cause a crash or potentially execute code. AWS describes the affected path more narrowly as IPv4 TCP handshakes involving crafted packets with IP options, and says researchers demonstrated remote code execution against hosts with an MPTCP listener. AWS also says it is not aware of exploitation in the wild. These descriptions are not identical, so follow the advisory for your exact distribution and product rather than treating one vendor's summary as a universal exploit guarantee. Red Hat's advisory and AWS's bulletin were checked on November 2, 2026.

This is a hands-on response guide, not an exploit walkthrough. It shows how to build an inventory, check vendor status, update the host kernel, verify the kernel that is actually running, and avoid the common mistake of patching an image while leaving existing nodes untouched. The steps are drawn from vendor advisories and standard Linux administration commands.

First, understand what “patched” means

A vulnerability record is not a universal list of safe kernel version numbers. Linux distributions routinely backport security fixes into a vendor kernel while keeping the version string's upstream-looking base unchanged. A kernel numbered 5.15.x can contain a security fix that is absent from another 5.15.x build. Compare your installed package against the vendor's fixed package or advisory status, not against an upstream version guessed from a headline.

There is a second distinction: the kernel package installed on disk is not necessarily the kernel executing now. Linux loads the kernel during boot. Updating it generally requires a reboot or a controlled replacement of the node before the fix is active. AWS explicitly calls for updating and rebooting self-managed EC2 hosts, and Red Hat says a kernel update is the complete remediation; its filtering rule is only a mitigation. AWS's bulletin and Red Hat's mitigation section explain those steps.

Keep three states separate in your records:

Package available: the vendor has published a fixed kernel package or image.
Package installed: the host has downloaded and installed that package.
Fixed kernel running: the host has booted the fixed kernel, or has been replaced by a node that did.

Only the last state confirms that the host is actively using the kernel fix.

Step 1: Find the machines you actually own

Start with your asset inventory, not a single SSH session. Include virtual machines, physical servers, Kubernetes workers, container hosts, appliances you manage, and old instances that are stopped but may return. A container image scan alone will not tell you which kernel the container host is running. Containers share the host kernel, so assess and patch the nodes beneath them.

For each host, record its cloud or data-center location, distribution and release, kernel package, current running kernel, workload role, and whether untrusted IPv4 TCP traffic can reach it. Do not infer reachability from an application's intended design. Check security groups, load balancers, firewall rules, and listening sockets.

On a Linux host, collect a starting snapshot:

printf 'hostname: '; hostname -f 2>/dev/null || hostname
printf 'os: '; . /etc/os-release && printf '%s %s\n' "$ID" "$VERSION_ID"
printf 'running kernel: '; uname -r
printf 'architecture: '; uname -m

If you administer a group of machines, gather the same fields through your existing configuration-management or cloud-inventory system. Avoid building a one-off scanner that silently misses nodes. A report should identify unreachable hosts and failed commands as unknown, not as patched.

Step 2: Check the right vendor advisory

Search for CVE-2026-98432 in the security tracker for the operating system and kernel flavor actually installed. That includes specialized cloud kernels, real-time kernels, HWE kernels, immutable images, and vendor-managed container nodes. Read both the affected-status and fixed-package fields. “Under investigation” is not the same as “not affected,” and a fix for one release does not establish the fix status for another.

The vendor pages can change while fixes roll out. As of November 2, 2026, AWS says fixed Amazon Linux 2 and Amazon Linux 2023 kernel packages are available and instructs customers to update and reboot. AWS also lists updated EKS-optimized Amazon Linux 2023 AMIs, while its page describes the Auto Mode rollout as still progressing. On the same date, Red Hat's bulletin says it is expediting fixes, and notes that traffic filtering can reduce exposure but cannot fully prevent the issue. Those are examples of why a single “Linux is patched” status is misleading: the answer depends on distribution, product, node type, and the exact time the advisory was checked. AWS security bulletin; Red Hat security bulletin.

Capture the advisory URL and the time you checked it in the change record. Recheck it before deployment if the remediation spans several days. Do not copy a fixed package version from another distribution or cloud image and assume it applies to your fleet.

Step 3: Update the host, not just the workload

For Amazon Linux 2, AWS currently documents:

sudo yum update kernel
sudo reboot

For Amazon Linux 2023, the bulletin documents:

sudo dnf update kernel --releasever=latest
sudo reboot

AWS notes that instances pinned to a particular Amazon Linux 2023 release need a release containing the fixed kernel. Follow the live bulletin and the relevant release advisory, especially if you pin repository versions. AWS security bulletin.

For other distributions, use their supported package manager and upgrade path. For example, a Debian-family system commonly uses apt, while a Red Hat-family system uses dnf or yum, but the command syntax is not the key decision. The decisive check is whether the distribution tracker says the installed kernel package for that release contains the fix. If no fixed package is available yet, do not substitute an unrelated kernel build on a production host without a supported migration plan.

Before rebooting, confirm that you have a recovery path: console access, a tested maintenance window, a healthy replacement node, or a rollback plan. Kernel changes affect the whole machine. For a cluster, drain and replace nodes in controlled batches, preserve capacity, and confirm that the new node image includes the fixed kernel before sending workloads to it.

Step 4: Verify what is running after remediation

After reboot or node replacement, check the live kernel:

uname -r

Then verify the installed kernel package using the distribution's package tools and compare it to the vendor's fixed package information. Keep both results. uname -r answers “what is running now?” Package queries answer “what is installed?” Neither alone answers whether the vendor considers that exact build fixed.

For a systemd-based host, verify uptime or boot time if the change record needs evidence that a reboot occurred:

uptime -s

For a fleet, collect the result from every in-scope node and reconcile it against your inventory. A successful update command on one representative machine is not fleet-wide proof. Include nodes that failed to reboot, were cordoned, or were replaced from an older image. If an automated scanner reports a vulnerable package after the reboot, investigate whether it is detecting the on-disk package, a stale image, or a separate kernel flavor rather than suppressing the finding without evidence.

Step 5: Treat cloud and Kubernetes layers separately

A cloud provider may patch its own managed control plane or service infrastructure while leaving customer-managed virtual machines for the customer to update. The provider's bulletin spells out that boundary. AWS, for example, says self-managed EC2 Linux instances remain the customer's responsibility and distinguishes them from services where AWS is applying infrastructure updates. It also has separate steps for EKS node AMIs, ECS on EC2, Fargate, and Bottlerocket. Read the section for the exact service you use. AWS security bulletin.

For Kubernetes, the pod's image is not the kernel boundary. Check the node operating system, node image release, and the mechanism that replaces nodes. Updating a launch template or node group does not necessarily replace already-running workers immediately. Confirm rollout status, drain or recycle old nodes, and verify the running kernel on each new node. If your cluster uses a managed automatic-update mode, confirm the provider's rollout status and your disruption-budget behavior instead of assuming that “managed” means “already complete.”

Also include edge cases in your inventory: autoscaling groups that can recreate an old image, warm pools, disaster-recovery regions, build agents, and long-lived development machines. A newly created node can reintroduce the old kernel if its image source was not refreshed.

What about the temporary firewall mitigation?

Red Hat describes an nftables rule that drops incoming IPv4 TCP packets containing IPv4 options. The vendor explicitly warns that the filter reduces exposure but cannot fully prevent the vulnerability, and that it may affect workloads. It should be treated as a temporary risk-reduction measure, reviewed and tested against your network, not as proof that the host is safe. Remove the rule after the fixed kernel is active, following the vendor's procedure. Red Hat's advisory includes the rule, validation, and removal commands.

Avoid applying network filters blindly across a fleet. First establish whether your traffic uses IPv4 options, whether the rule is compatible with your existing firewall framework, and how you will persist, monitor, and later remove it. Test in a representative non-production environment. If the vendor's exact mitigation is not relevant to your distribution or firewall manager, ask that vendor for its supported guidance rather than translating a rule from memory.

A compact response checklist

[ ] Inventory hosts, cloud nodes, container hosts, and rebuildable images.
[ ] Identify distribution release and kernel flavor per host.
[ ] Check the distribution or cloud vendor's current CVE-2026-98432 status.
[ ] Separate package availability, package installation, and the running kernel.
[ ] Prioritize hosts reachable by untrusted IPv4 TCP traffic.
[ ] Apply the vendor-supported fixed kernel or image.
[ ] Reboot or replace nodes using a controlled rollout.
[ ] Verify uname -r and the vendor package status on every in-scope host.
[ ] Refresh autoscaling, Kubernetes, and disaster-recovery images.
[ ] Track any temporary mitigation and remove it after patch verification.

The useful operational lesson is not “every Linux version is vulnerable” or “one firewall rule fixes it.” It is that a kernel security response has to trace the fix all the way from the vendor advisory to the kernel executing on each machine. The package manager is one checkpoint in that chain, not the finish line.

I write practical guides on software engineering, infrastructure, and AI. Subscribe if you want more walkthroughs like this.

How do you verify that a kernel fix reached every node in your fleet: package inventory, image rollout checks, or both?

Sources

Amazon’s AI Developer Memo Gets One Thing Right: Measure Outcomes, Not Tokens

jamilxt — Sun, 01 Nov 2026 16:05:49 +0000

A team can double the number of AI-generated lines in a week and still deliver no more useful software. It might even deliver less: more review work, more defects, more rework, and a larger bill. The uncomfortable part is that a dashboard full of activity can make the opposite story look convincing.

That tension sits at the center of Amazon’s newly published guidance on the software developer role in the age of AI. The memo, written by Terk Terkowitz, vice president and distinguished engineer at Worldwide Amazon Stores, asks engineers to learn and use AI. It also warns that token consumption, adoption rates, code changes, and reviews are deeply flawed if managers treat them as targets rather than imperfect signals. The company says those inputs can still correlate with valuable outputs, but should be used with care and judgment. Amazon’s memo is more nuanced than a simple “stop using AI” headline suggests.

The important question is not whether developers should use AI. It is whether a company can tell the difference between useful adoption and performative activity. Amazon’s guidance offers a good starting point, but the measurement problem is harder than swapping one dashboard for another.

What Amazon is actually saying

The memo makes two claims that need to be held together.

First, AI is likely to become central to software engineering. Amazon encourages engineers to learn the tools, apply them in daily work, and keep improving their practice. The memo calls for hands-on understanding, not secondhand opinions.

Second, the company says measures such as tokens consumed, adoption rates, code changes, and reviews are “deeply flawed” when detached from outcomes. As targets, they can encourage higher costs or unnecessary complexity. The memo gives “tokenmaxxing” as the case where increased consumption is treated as a goal in itself. It also warns that measuring lines of code can conflict with simplicity.

That does not mean Amazon considers those measurements useless. Its memo says input metrics have correlated with outputs that matter, and recommends looking carefully at both the 10th and 90th percentiles of adoption. The first can reveal teams that are not finding effective uses. The second can show whether teams with heavy usage are seeing delivery gains to match.

This distinction matters. A metric can be informative as a signal and damaging as a quota. If a manager asks why adoption is low on a particular team, the answer may expose a training gap or a workflow where the tools genuinely do not help. If the manager demands that every engineer hit a token target, the measure has become the work.

Why usage metrics are tempting

Usage is easy to count. Outcomes are not.

A provider can report tokens, requests, active seats, or generated suggestions almost immediately. A team can count pull requests, changed lines, or review comments. These numbers are legible, comparable, and available before a release has reached customers.

But the distance between “the tool was used” and “the customer received more value” is long. A generated patch may be rejected. An accepted patch may need extensive revision. A merged change may not solve the intended problem. A feature may ship faster and still introduce a reliability issue that takes weeks to surface.

That gap is a measurement problem, not an argument against using AI. It is also not solved by measuring only a different proxy. If token count is a weak stand-in for productivity, replacing it with lines of code or pull request count may merely move the incentive to another easy-to-game number.

The 2025 METR randomized controlled trial is a useful caution about scope. It studied 16 experienced open-source developers completing 246 tasks in mature projects they knew well. With the early-2025 AI tools in the study, developers took 19% longer on average, even though they expected AI to make them faster. METR’s paper does not prove that current tools slow every developer or every task. It shows why broad adoption claims cannot substitute for measuring the specific work, people, and tools involved.

METR later ran a follow-up with late-2025 tools, but said the newer data was an unreliable signal because developers who most wanted AI were less likely to participate or submit tasks they preferred to do with AI. In that follow-up, METR estimated an 18% slowdown among returning participants, with a confidence interval spanning from 38% slower to 9% faster, and a 4% slowdown among newly recruited developers, with a confidence interval spanning from 15% slower to 9% faster. Both intervals include no effect. METR said the selection effects made the results difficult to interpret and likely understated benefits for the developers and tasks missing from the study. METR’s 2026 design update explains those caveats. The takeaway is not that AI still slows developers; it is that a single metric or study cannot settle the question for every workflow. That is exactly the kind of nuance a usage dashboard hides. It can show that a team used an assistant more. It cannot, on its own, tell you whether the assistant helped with that team’s actual task mix.

A better unit of analysis: the task, not the token

A practical evaluation should start with a meaningful unit of work. That could be a bug fix, a small feature, a migration step, a test suite improvement, or a support issue. The definition should be stable enough that “done” means something consistent before and after AI enters the workflow.

For each task, record a small set of signals:

Outcome: Was the intended user or operational problem resolved?
Time to an accepted result: Include prompting, checking, editing, and rework, not only the first generated draft.
Quality: Did tests pass? Were review findings, regressions, or follow-up fixes needed?
Human effort: How much focused developer and reviewer time did the task consume?
Context: What kind of task was it, how familiar was the code, and what tool or workflow was used?

This is not a universal productivity formula. It is a way to avoid collapsing different work into a single count. A one-line configuration change and a complex production incident should not be treated as equivalent just because both produce one pull request.

Teams can then compare like with like. For example, they might examine several similar maintenance tasks completed with and without AI assistance, while noting differences in developer familiarity and review requirements. The comparison will not be a perfect experiment. It can still be more informative than asking whether AI usage went up.

The key is to preserve enough context to explain the result. If AI helps draft repetitive tests but slows work in a deeply familiar legacy module, that is not contradictory. It is evidence that usefulness depends on the task and workflow.

Keep adoption data, but do not turn it into a quota

There is a reasonable place for usage data. If nobody on a team has tried an approved tool, that may be worth investigating. Maybe access is broken. Maybe the team has not had time to learn. Maybe its work is unusually constrained. Usage can identify a question to ask.

It should not answer the question by itself.

A manager can use adoption data as a prompt for a conversation: “Where did this help? Where did it create extra work? Which task would you not use it for again?” That makes usage a diagnostic input. A required weekly token number makes it a performance target, even if leadership says otherwise.

Amazon’s attention to the 10th and 90th percentiles points toward a better practice: look at the outliers and investigate why they differ. Low usage might reveal a workflow that has not been adapted or a legitimate lack of fit. High usage should invite the same scrutiny: are those teams delivering more, or merely generating more activity? Do not assume the answer from the chart.

Averages can conceal both situations. A team-wide mean may look healthy while a few groups are struggling to adopt the tools, and a few others are spending heavily without a corresponding benefit. Percentiles do not explain the cause, but they help identify where a closer look is worthwhile.

A lightweight review cycle for engineering teams

A team does not need a measurement department to test whether AI helps. It does need a review cycle that resists easy gaming.

1. Pick one workflow. Choose a recurring task with a clear completion condition. Avoid beginning with “all engineering work.” Narrow scope makes the result interpretable.

2. Define the quality bar first. Specify the tests, review expectations, or operational checks that make the task acceptable. Do not lower the bar after seeing a faster draft.

3. Count the whole loop. Include time spent describing the task, waiting, reading generated output, correcting it, reviewing it, and handling rework. A fast first response is not the same as a fast accepted outcome.

4. Record negative cases. Note when the tool was not useful and why. Amazon’s memo explicitly treats an engineer’s explanation of where current AI does not help as useful judgment, not failure. Without this record, a team may remember only the impressive examples.

5. Review outcomes with context. Compare similar tasks and separate changes in scope, experience, and codebase familiarity. Treat a small sample as a learning signal, not a company-wide proof.

6. Revisit the measure. If people start optimizing the metric instead of the work, stop using that metric as a target. Keep it only if it still helps explain outcomes.

This cycle creates a practical decision: expand the workflow, adjust it, or leave it alone. The result can be “AI helps here,” “AI needs a different review pattern,” or “the current tool adds no value for this task.” All three are useful findings.

The “Renaissance Developer” is more than a tool user

Amazon describes the desired developer as a “Renaissance Developer,” combining AI tools with human judgment and domain expertise. The label is broad, but the memo gives it a concrete meaning: understand the product and customers, learn the tools directly, connect inputs to outputs, and exercise judgment about where AI belongs.

That is a stronger standard than simply asking engineers to use more AI. More importantly, it puts responsibility on leaders as well as individual contributors. Managers need to understand the technology well enough to recognize when a workflow is useful, and to connect adoption to delivery rather than treating usage as proof of progress.

There is a tension in the memo worth noticing. Amazon argues that its input metrics correlate with output, while also warning that they can cause harm when made targets. That is plausible: a measure may track a trend at the organizational level and still be a poor individual performance score. A company-wide relationship does not establish that every engineer should have a personal target for tokens, code changes, or tool usage.

The memo does not supply a complete measurement method, and its recommendations are written for Amazon’s own SDE and SDM organizations. Other teams should not copy its percentile thresholds or assume they will observe the same correlation. The transferable idea is more modest: inspect where AI use is low or high, ask what explains it, and verify whether outcomes moved too.

A checklist before adopting an AI productivity metric

Before putting a number on a dashboard, ask:

Does this measure customer or operational value, or only activity?
Can an engineer increase the number without improving the result?
Does it account for review, correction, and rework?
Are we comparing similar tasks and codebase familiarity?
Could the metric reward unnecessary code, requests, or complexity?
Do we record cases where AI was not useful?
What decision will we make if the number rises but outcomes do not?

If the answers are unclear, the metric is not ready to become a target. It may still be useful for exploration, but label it as an input and say what it cannot prove.

Amazon’s memo is a welcome correction to the idea that more AI activity automatically means more productivity. Its strongest point is not that tokens do not matter. It is that tool usage is only a clue. The work still has to solve the right problem, pass the quality bar, and deliver something useful.

I write about software engineering and AI tools. Subscribe if you want more practical, evidence-led breakdowns like this. Have you seen AI usage metrics help a team improve, or push people toward activity instead of outcomes? Share what happened.

Save the checklist for the next time someone proposes a token, adoption, or code-volume target. Start with the result you want, then choose the smallest measure that can help you understand whether the tool contributed.

Sources

HackerTube vs. YouTube Recommendations: Curated Signal or Personal Fit?

jamilxt — Sun, 01 Nov 2026 12:05:06 +0000

A video can be excellent and still never appear on your screen. That is the problem HackerTube tries to solve, but not by making a better version of YouTube's recommendation model. It changes the source of the recommendations.

HackerTube starts with videos that people recommended in Hacker News comments over almost two decades. YouTube's homepage, by contrast, is personalized around your own viewing history and other signals. One system asks, in effect, "What has this community found worth sharing?" The other asks, "What might this particular viewer want next?"

That difference makes HackerTube an unusually clear comparison of two discovery strategies: a shared human-curated corpus versus an individualized platform feed. Neither is automatically better. Each has a different blind spot, and the choice depends on whether you want a common catalog of recommendations or a feed tuned to your own habits.

I have not audited HackerTube's dataset or run a controlled comparison against YouTube. This is an analysis of the project creator's account, the site's stated features, YouTube's own description of its recommendation system, and the developer discussion around the launch.

What HackerTube changes

Andrej Karpathy introduced HackerTube on October 31, 2026. He says it draws from 39,834 Hacker News comments, narrowed to roughly 40,000 comments that recommend videos, and contains 6,454 videos across 150 topics from 2007 through 2026. Those are creator-reported figures, not an independent audit. The project presents those videos through a familiar video-browsing interface. Karpathy's launch post describes the collection and workflow; the Hacker News launch thread summarizes the catalog.

The source of its signal is the important part. A Hacker News user had to encounter a video, decide it was worth mentioning, and place it in a discussion. That is not a formal quality review, and a comment recommendation can be a joke, a disagreement, or a niche reference. But it is a trace of human selection that exists outside your own watch history.

The site's design also makes a few deliberate choices. Karpathy says it uses a frame from the middle of a video instead of the creator-uploaded thumbnail, offers recommendations without accounts, and keeps browsing state in the local browser session. He also says users can export the site's data and use it to build a different interface. These are descriptions from the creator, not guarantees about long-term availability or a substitute for inspecting the implementation. The launch post lists these features.

The project is not a new video host. The catalog points people toward YouTube videos. The discovery layer changes; the underlying video source does not. That distinction matters when assessing what the project can and cannot control.

What YouTube optimizes for

YouTube describes its recommendations as a personalized system. Its official explanation says the homepage primarily uses watch history and considers signals including search history, subscriptions, likes, dislikes, feedback such as "Not interested," and satisfaction surveys. The company says the system compares your viewing habits with those of people with similar habits.

That design can be powerful. If you have specific interests, a long viewing history, and channels you return to, the system can make useful suggestions that a general-purpose catalog would miss. It can also react to your feedback and viewing patterns rather than relying on what an outside community liked years ago.

The system's complexity is not proof that it serves every individual well. YouTube's explanation describes the signals it uses and its stated goal; it does not establish that every viewer experiences the feed as satisfying. The HN launch discussion contains both reactions: one commenter said the unpersonalized feed felt unusually close to their interests, while another said HackerTube resembled an HN-shaped algorithm and included videos they did not want. Those are personal reports, not representative user research. The thread is useful precisely because it shows that a community-derived feed can feel refreshing to one person and narrow to another.

HackerTube vs. YouTube: the practical trade-offs

Discovery signal: HackerTube uses historical links people chose to recommend in Hacker News discussions. YouTube uses personal and contextual signals, including viewing and search history. If you want to step outside your established viewing pattern, HackerTube's external corpus is attractive. If you want suggestions based on what you actually watch, YouTube has the more direct feedback loop. HackerTube's creator describes its source; YouTube describes its signals.
Breadth versus fit: HackerTube's reported 6,454 videos and 150 topics are a finite, shared collection. Its selection may surface a talk or documentary you would not find through your usual recommendations, but the collection reflects what was linked in one community. YouTube's catalog and personalization can cover far more viewing contexts, but a personalized feed may keep reinforcing patterns already present in your history. The latter is a possible trade-off, not proof that the system inevitably creates a filter bubble.
Freshness: HackerTube's corpus reaches back to 2007. A historical recommendation can remain useful, especially for foundational talks, but a decades-long window can also make old material prominent. YouTube can use current behavior and context to suggest recent uploads, though the results are still ranked for each viewer. HackerTube has since added a "Recently recommended" sort, according to a creator update in the launch thread. That helps distinguish a new HN mention from an old upload, but it does not make every video itself new.
Thumbnail incentives: HackerTube's middle-frame thumbnail is meant to avoid relying on the creator's selected thumbnail. It changes the preview, not the video's content, title, or reputation. YouTube's recommendation system considers whether viewers choose to watch, stay, and report satisfaction, according to its official overview. The two interfaces therefore expose different cues, but HackerTube has not demonstrated that one thumbnail strategy produces better viewing outcomes.
Personalization and privacy: HackerTube says it does not require accounts and stores browsing state in the local browser session. YouTube says its recommendations use account-associated signals such as watch history and subscriptions, and it provides controls to pause or delete that history. The right choice depends on the experience you want: minimal personalization or a feed that adapts to your behavior. Check each service's current settings and privacy explanation before treating these descriptions as a complete privacy comparison. HackerTube's creator post and YouTube's explanation describe the respective approaches.
Control and escape hatches: YouTube offers controls including "Not interested," "Don't recommend channel," and watch-history management. HackerTube's exportable catalog offers a different kind of control: a dataset someone can inspect or reuse to create another interface. One adjusts a personal feed; the other allows people to fork the collection's presentation, subject to the data and service remaining available. YouTube's help page lists its feedback signals; HackerTube's About link is linked from the launch discussion.

A recommendation feed is a product decision

The deeper lesson is not that one algorithm is honest and the other is manipulative. The two systems answer different questions because they start from different objectives and evidence.

HackerTube uses a community's past recommendations as a shared taste map. That map can help people discover material outside their own viewing history. But it inherits the limits of that community: who participates, which links get posted, which discussions survive, and what a commenter meant by sharing a video. A popular recommendation can be a useful clue, not a quality certificate.

YouTube's system personalizes suggestions from a broader stream of signals. That can make discovery more relevant to an individual, but it also means the experience is shaped by a platform's ranking choices and the data available about that viewer. YouTube says it uses satisfaction surveys alongside watch behavior. That is a meaningful distinction from a simplistic claim that the system only maximizes time watched. The actual outcome for any viewer still needs to be judged from their experience, not inferred from the system description alone.

HackerTube's smaller scale is a feature for some people and a limitation for others. A curated catalog makes the selection legible: the underlying videos were surfaced in public discussions, and the project reports the collection size and topics. But the process that filtered comments and grouped content is not the same as a transparent editorial review. Karpathy says AI models helped filter and classify the comments; that does not make the resulting list neutral or objectively high quality. His post explains the broad workflow, while the HN discussion surfaces questions about curation and coverage.

Which one should you use?

Choose HackerTube when you want a finite, community-shaped browsing session, especially if you are curious about technical talks and videos recommended in Hacker News discussions. Treat the catalog as a discovery index, then decide for yourself whether each video deserves your time.

Choose YouTube when you want a personalized feed, recent uploads from channels you follow, or recommendations that can adapt to your interests and feedback. If the homepage has drifted away from what you want, review your watch history, subscription list, and recommendation controls before deciding the whole system is useless. YouTube's own recommendation controls guide explains how to manage those inputs.

You can also combine them. Use HackerTube as a source of candidates, and YouTube as the player and subscription tool. That avoids pretending either source is universally superior. It also gives you a practical test: do you find videos worth watching that you would not have found through your normal feed?

A five-minute evaluation checklist

Pick a goal. Are you looking for a specific topic, a new creator, a long-form talk, or something recent? A discovery tool can only be judged against a concrete job.
Set a short session. Browse for ten minutes and save candidates instead of letting autoplay decide the length of the experiment.
Track useful finds. Count videos you actually want to watch, not just thumbnails you click. Note whether you already knew the creator or topic.
Check provenance. For a HackerTube result, inspect the discussion context when available. A recommendation's reason can be more informative than the fact that it appeared in a list.
Compare the cost. Consider time spent, interruptions, account requirements, and how easy it is to leave with a short watchlist.
Repeat once. A single session is anecdotal. Try a different topic or day before deciding one approach consistently works better for you.

The best feed is not necessarily the one with the most advanced model. It is the one whose recommendations match your intention without turning every visit into an unplanned session.

HackerTube is an interesting counterexample to the assumption that discovery must begin with a platform's own personalization. It asks whether a public archive of human recommendations can be useful enough to browse on its own. YouTube answers a different question: given the signals available about this viewer, what might they want next? For some viewers, switching between those answers may be more useful than choosing a permanent winner.

I write about developer tools, AI systems, and practical software engineering. Subscribe if you want more comparisons that focus on how tools behave, not just how they are announced.

Have you tried HackerTube or another community-curated video catalog? What did it help you find that your usual feed missed?

Sources

DoorDash Has Seen Hundreds of AI-Agent Orders. Build a Checkout That Knows When to Stop

jamilxt — Sun, 01 Nov 2026 03:07:25 +0000

A shopping agent can find a restaurant, assemble a cart, and call an ordering API. The hard question comes one step later: when is it allowed to spend the user's money?

DoorDash's latest earnings discussion offers a useful reality check. CEO Tony Xu said the company had seen “maybe several hundred orders in total” from some third-party AI agents. That is a cumulative count for those agents, not all orders and not a quarterly total. The same report says customers still predominantly order through DoorDash's own app. The company has also built its own agent experiences and an MCP connector for corporate ordering. The signal is not that agents cannot order. It is that connecting an agent to a real fulfillment network does not automatically make people use it. TechCrunch reported Xu's remarks and context.[1] The SEC-filed transcript of DoorDash's Q3 earnings call records Xu's cumulative figure and describes its broader strategy around digital assistants and physical fulfillment.[2]

For developers, the practical lesson is smaller and more actionable: do not let a language model turn a suggestion into an irreversible purchase. Build a deterministic boundary between “the agent found an option” and “the merchant accepted an order.” This tutorial develops that boundary as a small checkout state machine. It is a teaching example, not a production payment integration.

The order should be data before it becomes an action

A conversation is a poor source of authority. “Get the usual, but cheaper” is not a payment authorization. It leaves open which items, substitutions, delivery address, tip, fees, and final amount the user accepted.

The agent can help interpret intent and propose a basket. The application should then turn that proposal into a structured quote from the merchant. The quote is the object the user reviews. It should include exact line items, quantities, merchant, address, total, currency, and an expiration time. If inventory or price changes, create a new quote and require a new decision rather than silently editing the old one.

This is not merely a prudent UI convention. The Agent Payments Protocol (AP2) specification separates the shopping agent, merchant, credential provider, payment processor, and a trusted surface for user consent. It says transaction validation must happen in deterministic code. Its direct flow has a person approve the closed checkout and payment, while its autonomous flow uses user-approved constraints that the completed checkout must satisfy. Those are different authorization modes, not interchangeable versions of a model prompt.[3]

A compact state machine

Start with explicit states:

DRAFT: the agent is still assembling a proposed order.
QUOTED: the merchant has returned a concrete, expiring quote.
AWAITING_APPROVAL: the user is shown that exact quote.
APPROVED: the user approved the quote, and the approval is bound to its identifier and amount.
SUBMITTING: the server is making one idempotent order request.
CONFIRMED: the merchant accepted it and returned an order identifier.
REJECTED, EXPIRED, or FAILED: a terminal result that cannot be mistaken for success.

The key invariant is simple: no transition from DRAFT directly to SUBMITTING. The server checks the quote, the approval, and the merchant's current terms first. A model may request a transition, but it cannot grant itself the permission.

Here is a TypeScript sketch of the policy check. It deliberately stops before payment processing. Production systems should use the merchant and payment provider's supported APIs rather than treating this function as a substitute for them.

type Quote = {
  id: string;
  userId: string;
  merchantId: string;
  currency: string;
  totalMinor: number;
  expiresAt: number;
  items: Array<{ sku: string; quantity: number }>;
};

type Approval = {
  quoteId: string;
  userId: string;
  approvedAt: number;
  maxTotalMinor: number;
};

function canSubmit(
  quote: Quote,
  approval: Approval | undefined,
  now: number,
): boolean {
  if (!approval) return false;
  if (approval.userId !== quote.userId) return false;
  if (approval.quoteId !== quote.id) return false;
  if (now >= quote.expiresAt) return false;
  if (quote.totalMinor > approval.maxTotalMinor) return false;
  if (quote.items.length === 0) return false;
  return true;
}

Use integer minor units for money, such as cents, not floating-point dollars. The approval should refer to the immutable quote ID, not just “approved: true.” That makes a changed basket a different transaction. In a real service, the quote and approval records belong in durable storage with a transaction or compare-and-set transition so two workers cannot submit the same order concurrently.

Make retries safe, not merely convenient

Networks fail at awkward moments. Your server may send an order request, the merchant may accept it, and the response may time out before your application receives it. Retrying with a new request can create a duplicate dinner order.

Generate an idempotency key from the approved quote and persist it before contacting the merchant. Reuse that same key for every retry of the same logical submission. Record the request state, merchant response, and returned order ID. If the provider does not support idempotency, use its documented lookup or reconciliation mechanism before retrying. Do not invent a new order just because the first response was lost.

The state transition should be guarded server-side, roughly like this:

async function submitApprovedQuote(quoteId: string, userId: string) {
  const attempt = await db.transaction(async tx => {
    const order = await tx.orders.lockByQuoteId(quoteId);
    if (!order || order.userId !== userId) throw new Error("not found");
    if (order.status === "CONFIRMED") return { existing: order };
    if (order.status !== "APPROVED") throw new Error("not approved");
    if (order.expiresAt <= Date.now()) throw new Error("quote expired");
    return tx.orders.markSubmitting(order.id, order.idempotencyKey);
  });

  if ("existing" in attempt) return attempt.existing;
  return merchantApi.createOrder(attempt.payload, {
    idempotencyKey: attempt.idempotencyKey,
  });
}

This snippet assumes a transaction, row lock, and merchant API behavior that your actual stack must implement and test. It is not enough to put an idempotency header in a request if your application forgets which key belongs to which approved quote.

Approval must be meaningful

A confirmation screen is not meaningful if it hides the details that could change the decision. Show the merchant, exact items and substitutions, delivery destination, fees, tax, tip, total, and the deadline for the quote. Make the action label explicit: “Place this order for $X” is clearer than “Continue.” If the amount rises, an item is substituted, or the address changes, return to approval.

The authorization must also have a scope. “Spend up to $40 at this merchant for these listed items before 7:15 PM” is narrower than “buy groceries for me.” The AP2 specification describes open mandates for autonomous purchases, where a user approves constraints and the agent produces a specific checkout that is evaluated against them. It also recommends short expirations and requires that an agent not reuse an open mandate for another checkout until the previous one is resolved. That is a useful design pattern even if your implementation does not adopt AP2: bind permission to limits, merchant, duration, and the transaction being made.[3]

Payment credentials need similar boundaries. Stripe's agentic commerce guide describes Shared Payment Tokens scoped to a retailer, order amount, and time window, so an agent can initiate a payment without seeing the underlying payment details. That is a vendor-specific product description, not a general guarantee or a drop-in integration. The general principle is to avoid exposing reusable credentials to a model or agent process.[4]

Keep the merchant authoritative

The agent's cart is a proposal. The merchant remains the source of truth for inventory, current price, taxes, discounts, fulfillment availability, and order acceptance. Revalidate the quote at the commit boundary. If the merchant returns a changed total, do not silently charge the difference. Move the order back to a state where the user can review the new terms.

This also gives the system a clean failure vocabulary. “Agent suggested a basket,” “merchant returned a quote,” “user approved,” “payment authorized,” and “merchant confirmed” are separate events. Do not display “Your order is placed” after only a tool call or payment authorization. Wait for the authoritative confirmation and show the order ID or a clear pending status.

Tests worth writing before connecting a payment provider

Approval for one quote cannot authorize a different quote, even if the totals match.
A quote past its expiration is rejected.
A total increase or item substitution invalidates the previous approval.
Two concurrent submissions produce one merchant order.
A retry after a lost response reuses the same idempotency key.
A merchant timeout produces PENDING_RECONCILIATION, not a false success or a blind second order.
An agent cannot alter the shipping address after approval.
The model's textual statement that a user approved something is never accepted as proof of approval.

These tests target the boundary between probabilistic planning and deterministic execution. They do not prove the whole checkout is safe, but they make the most consequential transitions explicit and reviewable.

A practical decision rule

Use a human-present confirmation when the order is new, high value, sensitive, or likely to involve substitutions. Consider bounded autonomous purchasing only when the user has explicitly configured a narrow policy, the merchant provides stable structured quotes, credentials are scoped, and you can reconcile uncertain outcomes. Keep the two paths separate in both code and audit records.

DoorDash's “several hundred” figure is not a verdict on agentic commerce. It is a reminder that a demo, an integration, and a habitual customer behavior are three different milestones. A useful system has to do more than call tools. It has to earn permission, preserve the exact terms that were approved, and recover safely when the network does something ordinary and inconvenient.

Checkout review checklist:

Is the final merchant quote immutable and expiring?
Does approval bind to that exact quote, user, and maximum amount?
Can retries create duplicate orders?
Who is authoritative when price or availability changes?
Can the audit trail explain what the user saw and what the merchant accepted?
Does the user have a genuinely trusted surface for confirmation?

I write about developer tools, backend systems, and AI engineering. Subscribe if you want more practical pieces like this.

Have you built an agent flow that can spend money or submit a real order? Where do you put the approval boundary?

Sources

[1] TechCrunch: DoorDash said it’s only received “several hundred” orders from AI agents
[2] DoorDash Q3 earnings-call transcript, filed with the SEC
[3] Agent Payments Protocol specification
[4] Stripe: Agentic Commerce, A Getting Started Guide

When Java 27 Removed JVMCI, TornadoVM Built Its Own Compiler

jamilxt — Sat, 31 Oct 2026 16:08:40 +0000

A JDK upgrade can turn an implementation detail into a product-level problem.

For TornadoVM, the detail was JVMCI, the JVM Compiler Interface. The project used a relocated copy of Graal through that interface to translate Java kernels into GPU code. When JDK 27 removed JVMCI, TornadoVM had a choice: keep building around an integration that no longer existed in the new JDK, or take ownership of the compilation path. Oracle's JDK 27 migration guide documents the removal. [2]

With TornadoVM 8.0.0, its maintainers chose the second route. They replaced the bundled Graal compiler with TC, a Java bytecode-to-kernel compiler that emits CUDA C++, OpenCL C, or Metal code. Their release report says the new path preserves application APIs, passes the previous compiler's passing tests, and reaches comparable or higher throughput in selected workloads. Those are project-reported results, not an independent benchmark, but the architectural change is real and worth studying. [1]

The interesting story is not simply that a GPU framework wrote a compiler. It is that the team used a breaking platform boundary to simplify its own system, improve diagnostics, and make more deployment options possible.

The dependency was doing more than compiling

TornadoVM lets Java programs offload suitable computation to accelerators. Its users express work using APIs such as TaskGraph, TornadoExecutionPlan, KernelContext, @Parallel, and @Reduce. Under those APIs, a compiler must understand Java bytecode, map supported operations to accelerator execution, and produce code that a device driver can compile.

Previously, TornadoVM shipped a modified and relocated copy of Graal for this job. Graal is an optimizing compiler, but the TornadoVM team says its pipeline was designed to generate CPU machine code, not source kernels for a GPU driver. It also created a close dependency on JVMCI and internal JDK module exports. The project says this prevented it from reading newer class-file versions and kept its builds pinned to --release 22.

JDK 27's official migration guide lists the removal of JVMCI among that release's removed features. That's a narrow change in the JDK, but it had a large effect on software whose internals were built around the interface. [2]

The scale of the replacement is striking. In its release account, the TornadoVM team says the compiler replacement touched 1,218 files, adding about 40,166 lines and deleting about 119,699. It removed the relocated Graal module, vendored jdk.internal.vm.ci, and backend-specific Graal packages. This was not a small adapter written to make the next JDK compile. It was a redesign of a critical subsystem. [1]

TC narrows the compiler to the job

TC is a compiler written in Java, targeting JDK 21 and later. It reads class files directly, builds its own typed intermediate representation, performs optimization passes, lowers TornadoVM-specific operations, and prints code in a shared C-like form with target-specific dialects for CUDA, OpenCL, and Metal.

That design matters because the output is source code for a device-driver compiler. The team describes TC as having a front end, an SSA-based intermediate representation, optimization passes such as constant folding, dead-code elimination, global value numbering, loop-invariant code motion, and loop unrolling, followed by TornadoVM lowering and structured source emission.

The key distinction is not that TC has more compiler features than Graal. It is that TC can represent and expose the transformations this use case needs. The compiler can preserve Java variable names, source lines, and bytecode positions as it translates a kernel. The generated CUDA shown in the project article carries comments mapping emitted operations back to the Java source line.

That is practical debugging value. A GPU performance tool can identify a costly generated instruction, and the source annotation helps a developer trace it back to the Java kernel. When TC cannot compile a method, the project says it emits categorized error codes and points to the Java line and bytecode index. Compare that with diagnosing a failure deep inside an imported compiler pipeline: the new approach makes the boundary between application code and generated code more visible.

TC also runs without a GPU or the full TornadoVM runtime for its standalone emission mode. That allows a developer or CI job to inspect a generated kernel without having the target accelerator installed. It does not eliminate the need to test on actual hardware. It does make some compiler feedback easier to obtain and reproduce.

What the reported results say, and what they do not

A compiler replacement is only credible if it preserves behavior. The TornadoVM team's release article reports these test comparisons:

CUDA on an RTX 4090: 1,516 tests passed with Graal and 1,539 with TC.
OpenCL on an RTX 4090: 1,027 with Graal and 1,057 with TC.
OpenCL on Intel UHD 770: 932 with Graal and 934 with TC.
Metal on an Apple M3 Pro: 942 with Graal and 941 with TC. The team notes this Metal result was measured before the old Graal code was deleted and has not been rerun after removal.

The team says every test that passed before still passes, while TC handles some cases that the former compiler rejected. The Metal caveat should stay attached to the number. It would be misleading to present every backend as equally revalidated after the replacement landed.

For an application workload, the report uses jitLLM with Llama 3.2 1B and Qwen3 0.6B. It says the generated output was token-identical, with throughput between 1.06 and 1.14 times Graal on CUDA, and 1.04 to 1.09 times on OpenCL. On Metal, the two were within measurement noise. These are useful project measurements, but they are not a broad claim that all Java GPU workloads become faster. Results depend on the workload, device, model, precision, and measurement setup.

The honest takeaway is narrower: the replacement appears to have met its compatibility and selected-performance targets in the workloads and test matrix the project published. Teams evaluating it should look at the raw specification, reproduce their own kernels, and test every target backend they plan to support.

The surprising benefit is deployment choice

Removing JVMCI did not only restore compatibility with a newer JDK. According to the project, it also removed a blocker for Project Leyden AOT caches and GraalVM Native Image.

In a companion report, the team measured a small saxpy program that compiles and runs one CUDA kernel. It took about 0.6 seconds from launch to exit on a regular JVM, compared with about 0.3 seconds using an AOT cache or a Native Image in the stated RTX 4090 setup. The compiler's own kernel compilation reportedly dropped from about 30 milliseconds to 11 milliseconds with the cache, and to under 1 millisecond in the Native Image. The project notes that JNI-based libraries such as cuDNN, cuDF, and CUTLASS are not supported in Native Image yet. [3]

These figures are startup measurements for a specific small program, not a promise of doubled throughput for a long-running service. The project itself says startup optimizations matter most for command-line tools, tests, and short-lived workloads; long-running services gain less once warm. But the additional deployment paths are meaningful. An application may be able to choose a conventional JVM, a cached startup path, or a native executable according to its runtime environment.

This is a useful systems lesson: architectural boundaries can constrain features beyond the one they were introduced to support. A dependency that ties compilation to an internal JVM interface may also complicate startup optimization, native packaging, build flags, and future JDK compatibility. Replacing it changes more than the compiler.

What Java teams can borrow from this change

Most teams do not need to write a compiler after a library dependency breaks. The transferable idea is to use a compatibility crisis to inspect the shape of the dependency, rather than automatically rebuilding the same coupling around a replacement.

When an internal platform boundary changes, ask:

What capability do we actually consume? Name the narrow function, not just the library. TornadoVM needed to translate Java kernels into accelerator code, not to carry all of Graal's CPU JIT architecture.
Which constraints come from the dependency itself? The team points to class-file limits, internal module exports, backend code duplication, and poor visibility into intermediate code.
Can we preserve the public contract while replacing the implementation? TornadoVM says its application-facing APIs and kernel ABI remain stable in version 8.
What is the cost of owning the new layer? A compiler means ongoing semantic compatibility, optimization work, target-specific behavior, and a larger testing burden. One successful release does not make that maintenance disappear.
What evidence would make the migration safe? Define a test-parity gate, representative application benchmarks, explicit platform matrices, and a visible caveat list before declaring success.
Can the new design provide a second benefit? Here, the project links independence from JVMCI with AOT caches and Native Image support. A redesign can pay for itself through capabilities that the immediate compatibility patch would not provide.

This is not an argument to reject established dependencies. Graal served TornadoVM for years, and rewriting compiler infrastructure is an expensive choice. The lesson is to treat the break as an opportunity to evaluate the architecture, not as automatic proof that a full rewrite is justified.

What I would verify before adopting it

I have not run TornadoVM 8.0.0 or reproduced its benchmarks. This article is based on the project's release materials and Oracle's JDK migration documentation, so treat the performance numbers as vendor-reported until independently reproduced.

If you work with Java GPU computing, a practical evaluation can stay small:

Select a kernel your application actually needs, not only a sample vector operation.
Run it on every accelerator backend and JDK version you intend to deploy.
Compare output correctness first, then warm throughput and cold-start behavior separately.
Inspect the emitted code and error messages for a kernel you understand.
Test any Native Image or AOT configuration against your real libraries, especially if JNI is involved.
Record the device, driver, JDK build, compiler settings, and workload shape so a later comparison is meaningful.
Keep a rollback path until your test suite covers the kernels and launch configurations that matter to you.

That checklist is more useful than treating the release's strongest benchmark as a general verdict. The project has made a substantial engineering claim; a user should validate it against their own compute path.

A compiler rewrite can be a platform strategy

TornadoVM 8 is a good example of a team turning a JDK removal into a chance to own the piece of infrastructure closest to its product. The reported outcome is more than “it builds on Java 27”: the project says it now has a compiler designed for GPU kernel emission, clearer diagnostics, less dependence on internal JDK interfaces, and new startup options.

The cost is also real. TC is now a compiler the project must evolve and support. The published figures are promising, but they come from the project itself and include a backend caveat. That combination calls for interest, not blind adoption.

For Java teams, the broader question is worth keeping: when a platform removes a hook your system depends on, do you need a smaller adapter, a new dependency, or a narrower subsystem you can own? The answer depends on what the dependency was doing and what capabilities the current architecture prevents.

I write about Java, Spring Boot, and AI every week. Subscribe if you want more engineering breakdowns like this.

Sources

Have you had a platform change force a rethink of a core dependency? What did you replace, and what did you keep?

Before an AI Rewrites Legacy Code, Build a Parity Gate

jamilxt — Sat, 31 Oct 2026 12:05:11 +0000

A team can generate a new implementation faster than it can decide whether the new implementation is still the same product.

That is the important tension behind a report that Cisco president Jeetu Patel gave a 27-year-old engineer unlimited AI tokens and a goal to rewrite 98% of Cisco's legacy code within 24 months, with a four-week proof of concept. The account comes from Fortune reporting summarized by Tom's Hardware. The headline is striking. The engineering problem underneath it is less glamorous: how do you know a rewrite still does what users and dependent systems expect?

This article is a source-based engineering guide, not a report from a Cisco project or a hands-on legacy migration. The public coverage describes an ambition and a proof-of-concept window, not completed results. It does not establish that 98% has been rewritten, that the effort will meet the deadline, or that token supply is the limiting factor. The useful takeaway is a method you can apply to a small, reversible change: create a parity gate before asking an agent to replace behavior.

Why generated code is the easy part

A legacy system often has no single complete specification. Its behavior is distributed across code, database constraints, production data, integrations, error handling, and assumptions that nobody wrote down. A rewrite can look cleaner and still change a default, rounding rule, ordering guarantee, or failure mode that another system depends on.

The first question is not, “Can an agent reproduce this function?” It is, “What observable behavior must remain stable, and what evidence would show that it changed?”

The distinction is practical. A test that merely calls the new code and checks that it returns a plausible value proves little about equivalence. A parity test runs the same representative inputs through the old and new implementations and compares the outcomes. A reviewer can then inspect disagreements instead of trying to infer correctness from a large diff.

Microsoft's guidance for migrating AI agents recommends defining acceptance gates before a model change, capturing a baseline from the current system, using the same test data and environment assumptions for both versions, and investigating individual regressions rather than trusting an average score. Those ideas apply just as well to a code rewrite. See Microsoft's migration-validation guidance. For coding-agent work, Visual Studio Code's testing guide also says to preserve assertions, report what ran, and treat unrun tests as unverified.

Build the smallest useful parity harness

Start with a behavior that is bounded, deterministic, and important enough to matter. A pricing rule, date conversion, permission decision, or serialization function may be a better first slice than an entire service. Do not begin with the hardest code in the repository. Begin with a seam where you can call both implementations under controlled conditions.

For illustration, imagine an old and new implementation that both accept JSON inputs and return JSON results. The harness below reads one case per line, sends each input to both commands, and records exact output mismatches. It intentionally treats a process error as a failure rather than silently skipping it.

#!/usr/bin/env python3
"""Compare legacy and candidate commands on newline-delimited JSON cases."""
import json
import subprocess
import sys
from pathlib import Path

CASES = Path(sys.argv[1])
OLD_COMMAND = ["python3", "legacy_impl.py"]
NEW_COMMAND = ["python3", "candidate_impl.py"]


def run(command, case):
    completed = subprocess.run(
        command,
        input=json.dumps(case),
        text=True,
        capture_output=True,
        check=False,
        timeout=10,
    )
    if completed.returncode != 0:
        raise RuntimeError(
            f"{command[1]} exited {completed.returncode}: "
            f"{completed.stderr.strip()}"
        )
    return json.loads(completed.stdout)


def main():
    cases = [json.loads(line) for line in CASES.read_text().splitlines() if line]
    mismatches = []
    for index, case in enumerate(cases, start=1):
        old = run(OLD_COMMAND, case)
        new = run(NEW_COMMAND, case)
        if old != new:
            mismatches.append({"case": index, "input": case,
                               "legacy": old, "candidate": new})

    print(json.dumps({
        "cases": len(cases),
        "mismatches": len(mismatches),
        "details": mismatches,
    }, indent=2, sort_keys=True))
    return 1 if mismatches else 0


if __name__ == "__main__":
    raise SystemExit(main())

Save representative inputs in cases.jsonl, one JSON object per line, then run python3 parity.py cases.jsonl. Replace the two command arrays with real, isolated entry points. The example is intentionally strict: if the output formats differ, the harness reports a mismatch. Normalize only fields that are genuinely nondeterministic, and document each normalization so it cannot hide a meaningful regression.

This small script is not a migration framework. It is a starting gate. Production systems may need database snapshots, service stubs, message queues, feature flags, and stronger isolation. The key is that each version sees the same input and that you retain the evidence for every comparison.

Choose cases that reveal contracts

A collection of ordinary happy-path examples is not a useful characterization set. Include cases that make hidden behavior visible:

Typical input: common values from real use, with sensitive data removed or safely synthesized.
Boundaries: empty strings, minimum and maximum values, timezone transitions, rounding edges, and large payloads where relevant.
Invalid input: missing fields, malformed values, and unsupported combinations.
Failure behavior: timeouts, downstream errors, unavailable dependencies, and retry limits.
Compatibility details: field ordering if consumers rely on it, exact error codes, null versus omitted values, and stable identifiers.
Previously fixed bugs: regressions that the old code now avoids should be explicit cases.

Do not ask an agent to invent the expected result for every case and then treat its own answer as the oracle. Where possible, derive expected outcomes from existing tests, approved requirements, recorded behavior, or a domain owner. When the old implementation is the reference, it can reveal current behavior, but it cannot tell you whether that behavior is correct. Mark known defects as exceptions with an explicit decision rather than preserving them accidentally.

Make mismatches reviewable

A raw “17 tests failed” message is not enough. Each mismatch should show the input, the old output, the candidate output, and a stable case identifier. Group related mismatches by behavior: rounding, validation, authorization, serialization, and so on. A reviewer can then decide whether a difference is a bug, an intended improvement, or an old defect that should not be carried forward.

Keep three outcomes distinct:

Equivalent: the difference is not observable to supported callers, and the reason is documented.
Approved change: behavior is intentionally changing, with an owner and a migration or compatibility plan.
Unresolved: the difference blocks promotion until someone decides what the contract should be.

Never turn a mismatch into a pass by weakening the test just to make the candidate green. Microsoft's guidance explicitly recommends investigating individual regressions rather than relying on an aggregate rate. A 99% pass rate can conceal the one authorization or billing case that matters most.

Put the agent inside the gate, not above it

A coding agent can help inventory call sites, propose test cases, explain mismatches, and implement a candidate behind a feature flag. Those are useful tasks. It should not be allowed to change the acceptance criteria and approve its own output in the same unreviewed step.

A safer task boundary looks like this:

Ask the agent to map the selected behavior and identify callers, tests, data assumptions, and failure paths. Require file and line references so a human can verify the map.
Have a person review the map and choose the initial cases. Add cases for critical requirements that are missing from the existing suite.
Ask the agent to implement the replacement without editing the parity cases or their expected values. Protect those files through repository permissions or a separate review boundary where possible.
Run both implementations on the same fixed corpus. Save the command, version identifiers, environment, results, and mismatch report with the change.
Review every mismatch, inspect the final diff, and run the relevant existing test suite before merging.
Roll out gradually behind a reversible switch. Compare production signals where safe, and define a rollback trigger before the rollout begins.

This also makes “unlimited tokens” less important as a planning metric. More generation can increase the amount of code to inspect. The scarce resource may be trustworthy test cases, time from domain owners, or a review path that catches an unintended behavior change. That is an engineering inference, not a reported finding about Cisco's internal project.

A promotion checklist worth saving

Before replacing a legacy component, ask:

Is the selected slice small enough to revert independently?
Does the old implementation run in a controlled baseline environment?
Are cases drawn from real contracts and known incidents, not only agent-generated examples?
Are critical paths weighted separately from low-risk cases?
Can the harness reproduce each mismatch from a saved input?
Are nondeterministic fields normalized narrowly and transparently?
Are test files and expected outputs protected from the implementation agent?
Does every accepted behavior change have an owner and compatibility plan?
Do performance, security, privacy, and operational checks supplement functional parity?
Is rollout gradual, observable, and reversible?

A parity harness is not proof that a rewrite is safe. It is a way to turn a vague claim of equivalence into inspectable evidence. It tells you where to focus human attention, and it makes the decision to proceed, pause, or roll back more concrete.

Start with evidence, then scale

The public Cisco account is a striking example of how AI changes the perceived cost of attempting a large rewrite. But a deadline and token budget are not evidence of preserved behavior. Until results are independently reported, the 98% figure belongs to the stated ambition, not the outcome.

For an engineering team considering a similar effort, begin with one narrow behavior and a fixed corpus. Make the old and new outputs comparable. Let the agent help produce the candidate, but keep the contract, mismatch decisions, and promotion gate reviewable by people responsible for the system. Expand only when the gate catches the kinds of mistakes you care about and the rollback path has been exercised.

I write about software engineering, developer tools, and practical AI workflows. Subscribe if you want more guides like this. Have you used a parity harness for a rewrite or migration? What behavior did it catch that ordinary unit tests missed?

Pull Requests vs Agent Trails: Does AI Code Need a New Review Interface?

jamilxt — Sat, 31 Oct 2026 03:04:34 +0000

A pull request was designed around a simple unit of work: a person writes a change, proposes a diff, and asks another person to review it. Coding agents change the volume and shape of that work. They can produce multiple commits quickly, repeat a task across repositories, and revise code in response to automated feedback. The familiar review page can become a queue of diffs and comments while the real question remains unanswered: did the change accomplish what was intended?

That is the problem Entire is trying to address with Trails, a review workflow in its new Git forge. Rather than treating the pull request diff as the only review surface, Entire puts the task, agent activity, evaluations, and build results together. The company says agents can score customized evaluations on each push, while a human validates progress and merge gates determine when the change is ready.[1]

It is an ambitious proposal, but it should be judged as a workflow hypothesis, not as proof that human code review is obsolete. The useful comparison is not “old GitHub versus a magical agent forge.” It is diff-centered review versus evidence-centered review, and whether the second can preserve the strengths of the first.

What a pull request is good at

A pull request is more than a diff viewer. In GitHub’s documented workflow, reviewers inspect files and changes, leave comments or suggestions, and submit a decision such as approve or request changes.[4] Branch protection can also require reviews and passing status checks before changes merge.[3]

That model has several durable advantages:

The change is concrete. Reviewers can inspect the exact lines that will be merged.
The conversation has history. Questions, responses, suggestions, and decisions stay attached to the proposed change.
Policy is explicit. Required reviews and checks can block a merge rather than merely offering advice.[3]
It works across tools. Git is the shared version-control layer, while the pull request gives teams a familiar place to coordinate.

These are not incidental features. A repository needs a clear record of what changed, who approved it, and which checks passed. A faster code generator does not remove that need.

The weakness appears when the patch is no longer the whole story. A reviewer may need to understand the original request, the agent’s plan, which tests were selected, what changed after a failure, and whether the latest push invalidated an earlier review. A diff can show what changed without explaining whether the code solves the right problem.

What Entire’s Trail changes

Entire launched its forge on October 26, 2026. Its announcement describes Trails as a new review surface where the idea, agent trace, and outcome sit alongside the code. It says customizable evaluations can score each push, agents can address findings in a loop, and humans can validate progress.[1]

The company also describes a broader platform, not just a new pull request screen: Git-compatible hosting, integrations with Buildkite and Depot for CI, and “Continuous Memory” that connects codebase information and previous agent sessions.[1][2] Its blog says failing CI blocks merging on a Trail, while the announcement says the service can mirror GitHub repositories as well as host repositories natively.[2]

That distinction matters. Teams do not have to interpret the launch as “replace GitHub today.” Entire says repositories can be mirrored, and Trails can be used over pull requests while GitHub remains the source of truth.[2] For a cautious evaluation, mirroring a low-risk repository is a more reversible experiment than migrating a production codebase and its governance all at once.

Entire also advertises pushes up to 89 times faster than GitHub and up to six times faster than Cursor Origin, based on its own open-source benchmark.[1][2] Those are company-reported benchmark results, not an independent finding that the forge will make a team’s delivery 89 times faster. Push throughput is not review quality, merge safety, CI reliability, or time-to-correctness. Keep those measurements separate.

The practical comparison

Pull requests optimize for human-readable change review. They make diffs, comments, approvals, and required checks the center of the workflow. This remains a strong fit when changes are small enough to inspect, maintainers need a clear discussion record, or the team’s approval process is built around reviewing code line by line.

Trails try to optimize for intent and evidence around agent work. The proposed unit includes the task and the agent’s work history, with machine-generated evaluations and CI results providing more context than a diff alone.[1][2] This may help when agents make repeated pushes and the reviewer needs to follow progress rather than re-open a separate comment thread for every iteration.

The tradeoff is that an evaluation score is not a substitute for understanding the change. An agent can produce a plausible score against an incomplete or poorly designed evaluation. Passing tests demonstrate that selected tests passed, not that the system has no defect. A session trace can explain the agent’s process, but length does not guarantee that the reasoning was sound.

The best version of an agent-first review system would therefore make evidence easier to inspect without hiding the source code. A reviewer should be able to move from task intent to relevant tests, from test results to the actual diff, and from an evaluation failure to the affected behavior. If the UI presents a green score more prominently than the code and the evaluator’s limits, it risks creating a new kind of automation bias.

How to evaluate it without betting the repository

A useful trial should answer a workflow question, not simply test whether the interface looks modern. Pick one low-risk repository and a handful of representative changes. Keep the existing source of truth and required merge rules intact during the trial.

For each change, record four separate signals:

Review time: How long did a human need to understand the task and inspect the final change?
Defect discovery: Did the workflow surface meaningful issues before merge, and were they issues a reviewer could verify?
Rework: How many agent pushes or human corrections were needed before the change was acceptable?
Governance: Can the repository still enforce the required human approvals, status checks, and merge restrictions?

Do not combine these into a single “agent review score.” A tool might reduce review time while increasing missed defects, or improve context while adding operational work. Decide in advance which outcomes matter to your team.

For every evaluation that can affect merge readiness, ask:

What exact requirement is being tested?
Which code and test inputs did the evaluator inspect?
Can a developer reproduce the result locally or in CI?
What happens when the evaluator is unavailable or returns an ambiguous result?
Can a human override it, and is that override recorded?
Does a new push invalidate the previous evaluation and approval?

These questions apply to any forge that adds AI scoring, not just Entire. The goal is to make checks reviewable, repeatable, and appropriately scoped. An opaque “9/10” should not be treated as stronger evidence than a failing test with a clear reproduction.

A decision checklist

Stay with pull requests for now if:

Your work is already reviewed reliably through diffs, comments, and required approvals.
Your audit or compliance process depends on established approval records.
Agent-generated changes are occasional, and the review queue is not a real bottleneck.
The alternative cannot demonstrate that it preserves your branch rules and CI protections.

Run a bounded Trail trial if:

Agents make frequent revisions and reviewers lose the task context between pushes.
You want task intent, session history, evaluations, and CI results in one place.
You can test by mirroring a repository without changing the current source of truth.[2]
You are willing to evaluate the quality of the evidence, rather than trusting a score at face value.

Do not migrate just because of a speed claim. First verify what the benchmark measured, reproduce it for your own workload where possible, and separately measure review outcomes. Push speed may matter for some distributed workflows. It does not establish that the resulting code is easier or safer to merge.

The interface may change, but the responsibility does not

Entire’s launch is a useful signal that the pull request is no longer the only plausible interface for reviewing software. Agent-heavy development creates more activity around a change: prompts, plans, repeated edits, evaluation runs, and test results. A single diff-and-comment page may not organize all of that particularly well.

But a new surface does not eliminate the basic engineering obligations. Teams still need to know what was requested, what code will ship, which checks ran, who is accountable for approval, and what evidence supports the decision. Trails could make that evidence more visible. Pull requests already provide a mature way to inspect code and apply governance. The right choice depends on whether a new workflow measurably improves the review process without weakening its controls.

The takeaway: evaluate review systems by the quality and inspectability of the evidence they put in front of a human, not by how many agent-generated changes they can push per second.

I write about developer tools, software engineering, and AI. Subscribe if you want more practical analysis like this.

Have you tried an agent-first code review workflow, or does the pull request still fit your team? What evidence would make you trust an automated evaluation?

Sources

[1] Entire launches an independent Git forge for agent-human collaboration

[2] The Git Forge for Agent-Human Collaboration

[3] About protected branches - GitHub Docs

[4] Review pull requests - GitHub Docs

OAuth Trust at Machine Speed: What the ConnectWise Incident Teaches Cloud Teams

jamilxt — Fri, 30 Oct 2026 16:06:16 +0000

A third-party app does not need to guess your employees’ passwords if your tenant has already granted it permission to act. That is the uncomfortable part of the ConnectWise Cloud Backup incident: the reported access path ran through application credentials and existing customer trust, not a fresh interactive login at every downstream organization.

Microsoft says the China-state-aligned actor it tracks as Storm-0940 used the open-source ARTEX agentic penetration-testing framework to accelerate activity after the ConnectWise compromise. It reports more than 2,250 tenant sign-in attempts, with follow-on actions in some environments that included Microsoft Graph reconnaissance and application credential or privilege changes. Microsoft also says the activity did not result from a compromise of, or vulnerability in, Microsoft systems, products, or services. Those distinctions matter: the report describes abuse of a compromised third-party application’s granted access, not a Microsoft platform breach.[1]

This is a report-based analysis, not a first-person incident-response story. The useful lesson is not that AI has invented a new kind of cloud attack. It is that automation can make an existing trust failure move faster than a manual review process.

The application was already inside the trust boundary

OAuth consent is easy to treat as a one-time setup screen. An administrator reviews a list of permissions, clicks approve, and the integration starts working. Months later, the app may feel like a background service rather than an identity with ongoing authority.

That mental model is risky. An enterprise application or service principal can authenticate without a human entering a password at that moment. Its access depends on the permissions, credentials, and grants already configured. If the vendor’s application credentials are compromised, an attacker may be able to use established trust to reach customers who authorized the app.

Microsoft describes that exact shape in the ConnectWise case: after gaining control of credentials for cloud applications, the actor used those applications to authenticate to customer environments with the access they had already been granted. Microsoft observed follow-on activity in multiple environments, including reconnaissance and attempts to modify credentials or privileges. The company cautions that an affected application or successful authentication does not, by itself, prove customer data was accessed or exfiltrated.[1]

ConnectWise’s own advisory describes unauthorized access to application credentials associated with Cloud Backup, formerly known as SkyKick. Its latest advisory update, published October 28, says phased restoration was beginning through a new, security-hardened application, with partner action required. Partners must reset their password, configure multifactor authentication if needed, and re-authorize each Microsoft 365 tenant through the new app. The advisory warns not to re-enable previously disabled applications unless ConnectWise specifically instructs them to, and says re-authorization alone does not confirm that backups have resumed. Once backup capabilities are available, partners should verify that new snapshots complete successfully.[2]

ConnectWise’s technical guide also says that vendor-side containment does not change enterprise-application settings inside each partner or customer tenant. Those tenants need separate review, including dormant connections, and the guide cautions that completing the workflow does not prove a tenant was unaffected or authorize reconnection.[3]

What AI changes, and what it does not

Microsoft says ARTEX helped automate scanning, reconnaissance, password spraying, and OAuth application abuse. The report says some attacks reached privileged cloud access in under three hours. It also describes a password-spray campaign between October 10 and 14 that generated more than 151,000 failed sign-in events across 327 accounts at eight organizations. Microsoft observed no successful authentication, token issuance, Graph call, or directory change from that particular password-spray campaign.[1]

That last sentence is not a footnote to skip. The 151,000 failed sign-ins belong to one described campaign phase; they are not evidence that 151,000 accounts were compromised. Microsoft separately reports downstream customer activity through the compromised applications. Collapsing those into one dramatic number would exaggerate what the source says.

The deeper change is operational tempo. An agentic workflow can test many paths and react to results without waiting for a person to review every step. That increases the value of controls that work continuously: short-lived credentials where supported, narrow permissions, anomaly alerts, and a tested way to revoke trust across tenants.

The incident does not show that a language model independently discovered a zero-day or that AI defeated a specific security product. It shows a threat actor used an agentic framework to accelerate an existing offensive workflow. The hard problem remains familiar: what can a trusted application do, where can it do it, and how quickly can an organization notice when that behavior changes?

The uncomfortable inventory problem

Most organizations can list their direct employees and core cloud subscriptions. Fewer can quickly answer a different set of questions:

Which third-party applications have tenant-wide or high-impact permissions?
Which integrations are still needed, and which are leftovers from a pilot or departed vendor?
Who owns each application relationship?
What is the expected sign-in pattern for each service principal?
What is the emergency process for disabling an app without destroying evidence?
Which customer or business processes stop if the integration is revoked?

A display name is not enough for that inventory. Microsoft’s response guidance tells affected customers to search by application ID rather than display name, review tenants where affected applications are currently or were previously installed, and record the local service-principal object ID. That is a useful general lesson: names are labels; stable identifiers are the better join key for investigation.[1]

This gets harder in service providers and other multi-tenant businesses. One integration can have many customer authorizations, and a single vendor-side credential incident can create a broad downstream review. A spreadsheet maintained only during onboarding will not stay accurate. The inventory needs an owner, a review cadence, and an offboarding trigger.

A practical third-party OAuth review

1. Build the inventory from identity data. Export enterprise applications, service principals, owners, credentials, consent grants, and app-role assignments from the identity platform. Include application ID, local object ID, tenant, permission scope, business owner, vendor contact, and last observed use. Do not treat a familiar product name as proof that the app is official or currently required.

2. Rank access by impact. Separate read-only access from write access, directory administration, mailbox access, key-vault access, and broad resource management. Mark permissions that can create credentials, assign roles, alter users or groups, or access sensitive data. Review the actual granted permissions, not only the vendor’s marketing explanation of what the integration needs.

3. Define expected behavior. For each important service principal, record the expected source ranges or service locations when available, normal sign-in frequency, APIs it should call, and the resources it should touch. Avoid a simplistic rule that every unfamiliar IP proves compromise. Use deviations as investigation leads and correlate them with audit activity and vendor guidance.

4. Alert on identity changes, not just logins. Monitor new application credentials, added owners, new consent grants, app-role assignments, privileged role changes, and unusual Graph activity. A successful app-only sign-in may be routine; an unexpected credential addition immediately afterward is a different signal. Microsoft’s report specifically recommends reviewing service-principal sign-ins alongside Graph activity and directory changes.[1]

5. Preserve before removing. During a suspected compromise, capture available sign-in and audit records, relevant SIEM events, app configuration, permissions, owners, and credential metadata. Then contain the access using an authorized response process. Microsoft advises preserving configuration and evidence rather than deleting enterprise applications as the first action in this incident’s guidance.[1]

6. Revoke with a recovery plan. Know how to disable or remove an integration, revoke its grants, rotate any dependent secrets, and validate downstream workloads. Include a vendor-contact path and a customer communication plan. For backup products, independently confirm that new snapshots are completing and that restores work. A green service page is not the same as a verified recovery point.

7. Re-check historical access. Removing the original application may not remove credentials or persistence created elsewhere. Review subsequent changes to users, groups, roles, app registrations, mailbox rules, and data access. Use the incident window supplied by the vendor and security researchers, but extend it if local evidence points earlier.

Do not turn indicators into a false all-clear

An IP address, user agent, or application ID can help prioritize a hunt, but it is not a complete detection strategy. Infrastructure changes. An attacker may use new addresses. A clean query can also mean that relevant telemetry was not enabled or retained.

Microsoft explicitly warns that the absence of results from one of its sample queries should not be treated as confirmation that no access occurred. It recommends considering Graph telemetry alongside service-principal authentication and application permissions.[1] That is a useful rule beyond this incident: negative evidence is only as strong as the visibility behind the query.

Likewise, do not assume every customer of a vendor experienced data theft. Microsoft says an affected application or successful authentication does not necessarily mean data was accessed or exfiltrated. The responsible response is to inspect the tenant’s evidence and follow vendor and Microsoft guidance, not to convert a broad exposure into a universal claim.[1]

The takeaway: treat OAuth consent like a production dependency

Teams already inventory databases, queues, and external APIs because those dependencies can break production. A powerful third-party application deserves the same treatment, plus an identity owner and a tested revocation path.

The useful checklist is short enough to keep near the incident runbook:

Identify the application by ID, not just display name.
Record every tenant, owner, permission, and credential.
Alert on credential, consent, and role changes.
Correlate service-principal sign-ins with API and directory activity.
Preserve evidence before cleanup.
Revoke access deliberately, then verify the business workflow and recovery path.
Reauthorize only through the vendor’s current, explicit instructions.

The AI angle is real, but it should not distract from the architectural point. If one vendor-side identity can reach many customer tenants, the access boundary is part of your system whether or not your team wrote that integration. Machine-speed abuse makes stale assumptions more expensive.

I write about Java, Spring Boot, and AI every week. Subscribe if you want more engineering analysis grounded in what the sources actually say.

Have you reviewed the third-party OAuth applications in your cloud tenant recently? What permission or monitoring gap surprised you?

Sources

[1] Microsoft Security Research: AI-driven, large-scale nation-state campaign using ARTEX enables supply chain compromise

[2] ConnectWise Trust Center: Cloud Backup advisories

[3] ConnectWise Cloud Backup technical investigation and remediation guide

Cursor Marketplace vs. Official Claude Code: What Version 2.1.313 Reveals

jamilxt — Fri, 30 Oct 2026 12:06:01 +0000

A developer sees the familiar Anthropic name and anthropic.claude-code identifier in an editor, clicks update, and gets malware. That is the uncomfortable lesson from a malicious Claude Code extension package that Cursor says was available in its marketplace for about eleven hours on October 29, 2026.

The distinction that matters is not simply “Claude Code was hacked” versus “Claude Code was not hacked.” Cursor identified one malicious package, version 2.1.313, in its VS Code extension marketplace. It says the build was not developed by Anthropic, and that it has no evidence that other versions or the official distribution were affected. The extension attempted to collect host details, add an SSH key, create persistence, and open a reverse tunnel. Cursor estimated that at most 5,000 users were affected.[1]

This is a comparison of distribution paths, not a claim that one editor or registry is universally safer. The same extension name can show up in multiple places, and the trust boundary changes with the place you installed it from.

What happened, and what did not

Cursor’s notice says the package was available from about 2:27 a.m. to 1:35 p.m. Pacific Time on October 29. It attempted to collect a machine’s hostname, username, operating system, architecture, home and working directories, configured Git email, public IP, and names of environment variables. It also tried to add an attacker-controlled key labeled svc-telemetry to authorized_keys, enable local SSH, and establish a persistent tunnel.[1]

There is an important qualification. Cursor says enabling the SSH service required administrator rights, so that part would fail on most macOS and Windows machines. That did not make the package harmless: it could still add the key, create persistence, and report host details. Cursor says the attacker server is now offline, but recommends remediation for anyone who installed the affected version from its marketplace during the exposure window.[1]

The first wave of reporting confused the distribution paths. Some early alerts attributed the malicious package to Open VSX. Cursor's own notice identifies its Cursor VS Code extension marketplace. Researchers later updated their reporting to reflect Cursor's clarification: the version was not found in Open VSX, Microsoft's Visual Studio Marketplace, or the npm package for Claude Code.[2] That correction matters. An extension identifier is not a cryptographic proof of where a particular package came from.

The comparison: editor marketplace vs. official distribution

Cursor’s extension marketplace. This was the affected path. If you installed or updated Anthropic.claude-code version 2.1.313 through Cursor's extension marketplace during the stated window, treat that machine as potentially compromised and follow Cursor's incident response instructions. Cursor removed the package, audited the marketplace, rotated keys, and says it found no evidence that other extensions were tampered with.[1]

Anthropic's official distribution. Cursor says it has no evidence that the official distribution or other extension versions were affected. Researchers checking the registries reported that version 2.1.313 was absent from Open VSX, the Microsoft Marketplace, and npm. That is useful scope information, not a blanket assurance that every channel is always safe. A malicious artifact could be isolated to one marketplace while another copy with the same publisher and version string remains clean.[1][2]

What this comparison does not prove. It does not prove how the malicious package entered Cursor's marketplace, who was responsible, or that any marketplace has a perfect review process. Cursor's notice documents response actions and scope, but does not explain the package's provenance. Don't turn an incident report into a story about a confirmed account takeover, compromised Anthropic build system, or a vulnerability in Claude Code itself. Those claims go beyond the public evidence cited here.

The practical winner is not “always use the CLI” or “never use Cursor.” It is the installation path you can identify and verify. A familiar name, verified publisher badge, and plausible version number are useful signals, but this incident shows they are not enough on their own.

Why the registry boundary matters

An extension listing is not one universal package. Editors can connect to different marketplaces and mirrors. A developer may think they installed “the Claude Code extension,” while the actual artifact came from a specific service with its own publishing and distribution path. The marketplace is part of the software supply chain, not just a search box.

Publisher identity and artifact integrity answer different questions. A trusted identity tells you something about who is allowed to publish under a namespace. It does not automatically prove that every file delivered through every downstream marketplace is the exact file built and released by that publisher. Investigations need to preserve both facts: the product name and version, and the store or download endpoint that delivered it.

Version numbers are evidence, but not the whole investigation. Version 2.1.313 was a useful indicator in this case. But incident response should also check when the update occurred, which editor and marketplace delivered it, and whether the payload left persistent files behind. Removing a visible extension can leave a separate launch agent, scheduled task, SSH key, or running process untouched. Cursor explicitly warns that uninstalling the extension may not be sufficient.[1]

If you may have installed 2.1.313

Start with the exact scope. Cursor's notice applies to installations from its marketplace between 2:27 a.m. and 1:35 p.m. PT on October 29, 2026. If your fleet uses Cursor, search endpoint and extension-installation telemetry for the extension ID, version, and install time. Do not rely only on today's installed version: an update may have replaced the visible package after the earlier version already ran.

Cursor's advisory includes platform-specific checks and cleanup scripts. On macOS and Windows, it says the key labeled svc-telemetry is an indicator. It provides commands to check the relevant authorized_keys locations, plus remediation scripts that remove the extension, known persistence mechanisms, running tunnel processes, and the unauthorized key.[1] Use the instructions directly from Cursor's notice rather than copying a shortened command from a social post.

If you find evidence of execution, treat it as an incident, not a routine extension reinstall. Cursor recommends isolating hosts showing evidence of payload execution or backdoor installation, removing persistence and tunnel processes, assessing whether to rebuild, and rotating credentials and sessions accessible from the affected machine from a trusted system.[1] An empty authorized_keys check alone cannot establish that nothing else ran.

For organizations, preserve logs and images before cleanup where incident-response policy requires it. Coordinate with your security team; don't run a remediation script as a substitute for evidence preservation or credential rotation. Cursor provides separate scripts and instructions for affected macOS and Windows systems in its notice.[1]

A repeatable extension supply-chain checklist

Before installing or updating a developer extension, keep this short checklist close:

Record the source. Note the editor, marketplace or registry, publisher, extension ID, version, and install time. “Installed Claude Code” is not enough detail for a later investigation.
Verify the exact channel. Open the extension's publisher page from the marketplace configured in your editor. Don't assume search results, mirrors, and vendor downloads are interchangeable.
Pin high-risk changes. For managed developer machines, consider controlling extensions through an allowlist or approved version policy instead of allowing every update immediately. Define an emergency update path so pinning does not become permanent neglect.
Monitor version changes. Alert on unexpected publisher, extension ID, version, or source changes. Keep installation logs long enough to investigate a version that was installed and later replaced.
Limit the extension host's reach. Run development tools with ordinary user privileges where practical. Restrict secrets and SSH credentials available to the workstation. Least privilege does not stop every malicious extension, but it reduces what a successful payload can reach.
Plan for cleanup beyond uninstall. Your response procedure should include persistence checks, process and network review, credential rotation, and rebuild decisions. The visible extension folder is only one possible artifact.
Write down a scope rule. For each incident, specify the exact package, registry, versions, and exposure window supported by evidence. Keep “not observed in this channel” separate from “impossible in this channel.”

This list applies beyond AI coding assistants. Editors increasingly host tools that can inspect repositories, invoke shells, read local configuration, and handle credentials. That makes the marketplace and extension runtime part of the developer workstation's security boundary.

Verdict

The October 29 incident is a case for channel-aware verification, not panic about every Claude Code installation. Cursor's report identifies a malicious package in its marketplace, with a bounded version and exposure window; it does not say the official Claude Code distribution was compromised. Researchers corrected early reports that pointed to Open VSX, underscoring why source attribution must be checked before repeating a headline.[1][2]

If you installed the affected package through Cursor during the specified window, follow the vendor's incident response steps and rotate credentials from a trusted device if the machine shows evidence of compromise. If you did not, still take the durable lesson: record where extensions come from, keep version and install history, and design for the possibility that removal alone will not undo execution.

I write about software engineering, developer tools, and AI every week. Subscribe if you want more practical breakdowns like this. Have you changed how your team approves editor extensions after a supply-chain incident? What control would have helped most?

Sources

[1] Cursor security notice for Anthropic.claude-code 2.1.313

[2] Rescana analysis of the incident timeline and affected registries

AI Agent Swarms vs. a Single Agent: Build a Fair Coordination Benchmark

jamilxt — Fri, 30 Oct 2026 03:05:29 +0000

A team of four AI agents recently defeated Minecraft's Ender Dragon. In a separate experiment, a 50-agent colony tried to build a thriving civilization, then ran into resource scarcity and coordination overhead. Both results are interesting. Neither, by itself, proves that more agents make a better system.

The comparison is useful because it exposes the real engineering question: when does parallel work outweigh coordination overhead?

The four-agent run used a randomly generated world on Easy difficulty. The agents wrote 2,783 programs, died 605 times, and finished the objective in 27 hours and 14 minutes. The authors say the agents coordinated using chat and shared memory, without human-assigned roles or game logic.[2] The 50-agent colony had a different goal and setup. It produced roughly 5,900 messages, depleted nearby trees quickly, and recorded 22 of its 34 deaths during the first night.[1]

Those numbers are not a head-to-head benchmark. The goals, agent counts, models, run lengths, and conditions differ. Treating them as a leaderboard would confuse a vivid demo with a controlled experiment.

Instead, build a small scorecard that can answer a question on your own task: does a group of agents complete more work, more reliably, and at a reasonable cost than one agent given the same budget?

Why “the swarm won” is not enough

A single-agent setup has one main advantage: fewer coordination paths. There is no agent-to-agent disagreement, duplicate assignment, or shared ledger to keep consistent. It may be slower on tasks that divide cleanly, but it is easier to reason about.

A swarm can split independent tasks, gather information in parallel, and recover when one worker gets stuck. But it also introduces overhead: messages, integration, conflicting actions, retries, and a coordinator that may become a bottleneck. A group that performs many actions can still produce little useful progress.

The Minecraft examples make this distinction visible. In the 50-agent report, the agents formed a hierarchy and tracked food with shared notes, but the population also created more mouths to feed and a need to manage scarce resources.[1] In the four-agent research run, the team completed a long sequence of dependent steps, but the paper itself describes a particular experimental setup, not a general guarantee that four agents outperform one.[2]

The useful comparison is not agent count versus agent count. It is completed objective per unit of time, cost, and coordination burden under the same task conditions.

Define a fair experiment before launching agents

Start with a task that has an observable completion condition. “Build something impressive” is difficult to score consistently. “Collect 20 units of resource A and place them in the marked storage” is much easier.

Write down these controls before the first run:

Same objective: Give the single agent and the group the same task and success criteria.
Same total budget: Set a token, tool-call, or wall-clock ceiling. A group should not quietly receive five times the compute.
Same environment: Keep the initial state, tools, permissions, and external information equivalent.
Repeated trials: Use several seeds or starting states. One lucky run is not a reliable result.
Fixed stopping rule: Stop at success, failure, or the same budget limit.
Logged events: Record the final result, duration, token use, actions, and coordination messages for every trial.

For a real game experiment, a framework such as Mineflayer can supply the bot-to-world interface. Its project describes a JavaScript API, Python usability, and features including movement, inventory, crafting, digging, building, and chat.[3] This article does not claim to reproduce either published experiment. The Python program below scores event records after your own runs; the agent harness remains your choice.

Build a trial scorecard in Python

Use one JSON object per run in a file named trials.jsonl. A minimal record looks like this:

{"setup":"single","trial":"seed-01","success":true,"seconds":840,"tokens":12000,"actions":83,"messages":0}
{"setup":"swarm-4","trial":"seed-01","success":true,"seconds":510,"tokens":27800,"actions":121,"messages":34}

Keep the same trial identifier for runs that share a starting condition. For each setup, collect multiple trials. Count messages consistently: if one broadcast reaches four agents, decide whether that is one broadcast or four deliveries, then use the same definition throughout.

Save this script as score.py:

import json
import statistics
import sys
from collections import defaultdict
from pathlib import Path

REQUIRED = {"setup", "trial", "success", "seconds", "tokens", "actions", "messages"}

def load_trials(path):
    groups = defaultdict(list)
    for line_no, line in enumerate(Path(path).read_text().splitlines(), 1):
        if not line.strip():
            continue
        row = json.loads(line)
        missing = REQUIRED - row.keys()
        if missing:
            raise ValueError(f"line {line_no}: missing {sorted(missing)}")
        if row["seconds"] < 0 or row["tokens"] < 0:
            raise ValueError(f"line {line_no}: seconds and tokens must be non-negative")
        groups[row["setup"]].append(row)
    return groups

def summarize(rows):
    successes = [r for r in rows if r["success"]]
    return {
        "trials": len(rows),
        "success_rate": len(successes) / len(rows),
        "median_seconds_all": statistics.median(r["seconds"] for r in rows),
        "median_tokens_all": statistics.median(r["tokens"] for r in rows),
        "median_actions_all": statistics.median(r["actions"] for r in rows),
        "median_messages_all": statistics.median(r["messages"] for r in rows),
        "median_seconds_success": (
            statistics.median(r["seconds"] for r in successes) if successes else None
        ),
    }

def main():
    path = sys.argv[1] if len(sys.argv) > 1 else "trials.jsonl"
    groups = load_trials(path)
    if not groups:
        raise SystemExit("No trial rows found")
    for setup, rows in sorted(groups.items()):
        print(setup, json.dumps(summarize(rows), sort_keys=True))

if __name__ == "__main__":
    main()

Run it with python3 score.py trials.jsonl. It reports each setup separately. It does not invent a single “best” score that hides trade-offs: you can see completion rate, time, token use, actions, and messages independently.

Why report medians? A single unusually fast run can distort an average. The median gives a more robust center for small, noisy trial sets. Keep the full results too, especially failures. The scorecard reports median duration over all trials so timeouts are not silently removed, and a separate median for successful trials so you can see how quickly the successful runs finished.

Interpret the result without fooling yourself

Suppose a four-agent setup finishes successful trials sooner but uses substantially more tokens and has a lower success rate. That is not an automatic win. It might be suitable for a deadline-sensitive task where retries are cheap, and a poor fit for a constrained or reliability-critical workflow.

Ask these questions:

Did success rate improve? A faster median among successful trials can conceal many more failures.
Did the swarm respect the same budget? More agents often mean more parallel model calls. Count all of them.
Did the task actually parallelize? If each step depends on the previous one, agents may mostly wait or duplicate context.
Did coordination messages carry useful information? High message volume can signal productive collaboration or confusion. Inspect samples, do not treat message count as quality.
Were the runs comparable? A different model, prompt, tool permission, seed, or stopping rule can invalidate the comparison.
Was there a bottleneck? Shared files, one coordinator, limited resources, or a rate-limited tool may cap throughput regardless of agent count.

For stronger evidence, compare paired trials on the same starting conditions and report the spread, not just one median. Keep a record of model versions and prompt changes. If the evaluation is important enough to guide a production decision, have someone review the task definition and logging for hidden advantages.

A practical decision checklist

Try a single agent first when: the work is sequential; the objective is small; shared state is hard to protect; or failures are costly and difficult to audit.

Try a swarm when: sub-tasks are genuinely independent; each worker can return a verifiable result; the total budget is controlled; and an integration step can detect duplicates or contradictions.

Keep the swarm only if: repeated comparable trials show a meaningful gain in success, time, or quality that justifies additional tokens, messages, and operational complexity.

A useful middle ground is not “one agent or fifty.” Start with a small number of workers, assign bounded tasks, and make completion explicit. Add workers only after measuring the bottleneck. More agents cannot fix an ambiguous task definition, stale shared state, or a missing stop condition.

The takeaway

The recent Minecraft experiments are valuable demonstrations of coordination, persistence, and failure modes. They are not a controlled single-agent-versus-swarm comparison. One group completed a complex objective with four agents; another large colony encountered the costs of scale. The difference is a prompt to measure, not a verdict.

Build the smallest fair benchmark that reflects your task. Keep conditions constant, repeat trials, count all resources, and put success rate beside speed. Then decide whether parallel agents earned their extra complexity.

I write practical guides about AI engineering and software development. Subscribe if you want more. Have you compared a single agent with a multi-agent setup? What did you measure, and what surprised you?

Sources

[1] Y Combinator Labs, Agents in Games
[2] Faldor et al., A team of Gemini 4 Argon agents beats Minecraft
[3] PrismarineJS, Mineflayer

Mistral Large 4 vs Gemini 4 Carbon: Open Weights vs Restricted Access

jamilxt — Thu, 29 Oct 2026 16:07:50 +0000

A model announcement can look like a product release even when most developers cannot use the model yet. That distinction matters this week: Mistral says Mistral Large 4 weights are now available, while Google's October 28 Gemini 4 Carbon announcement describes a restricted initial rollout to selected cyber defenders and trusted partners. One option can be downloaded and deployed under its license; the other is not yet generally available to developers.

So which one should a team evaluate? The answer is not “whichever benchmark is higher.” As of October 29, 2026, this is primarily a comparison of access paths and rollout plans, not a fair, hands-on contest between two generally available models.

This public-source breakdown separates announced capability from current availability and offers a practical checklist for deciding whether either release is actionable for your team. No hands-on comparison is claimed.

The short answer

Need a model you can try now? Mistral says Large 4 is available through its public preview API on Mistral Studio. Confirm the current terms, limits, and pricing for your account before planning production use.
Need Gemini 4 Carbon now? Google's announcement says access starts with a subset of cyber defenders, then expands to trusted partners through its Fairwind program. Developer, business, and consumer access is a later step, not the launch-day offer.
Need downloadable weights today? Mistral says the weights are available through Hugging Face and names vLLM and SGLang as deployment options. Read the model card and license before committing to self-hosting.
Need a model choice for production? Wait until both options are actually accessible to your team, then run the same evaluation against your workload.

That is less exciting than declaring a winner. It is also more useful to an engineering team planning a real integration.

What each company has actually announced

Mistral's October 6 post introduced Large 4 as a public preview and said the preview API was available on Mistral Studio. Mistral described it as a one-trillion-parameter natively multimodal model, with 52 billion active parameters, and promised weights by the end of October. Its newer release post says the weights are available, alongside an updated model, improved reasoning efficiency, three reasoning modes, and additional evaluations. Mistral's model documentation classifies Large 4 as open-weight. Check the actual model card and license terms for your use case rather than treating “open-weight” as a complete licensing summary.

Google DeepMind's October 28 announcement calls Gemini 4 Carbon its strongest model for cybersecurity and long-horizon capabilities. Google cites its own and external benchmark results, including 59 on the Artificial Analysis Intelligence Index v5.0, 88.2% on SEC-bench Pro, 54.8% on ExploitGym, and 71.0% on CWE-bench. These are company-reported results; they should be interpreted with each benchmark's task definition and scoring methodology, not as a universal ranking of coding ability.

The access language is the key difference. Google says it is beginning with a subset of cyber defenders and will gradually add trusted partners through Fairwind. It says access for developers, businesses, and consumers through product surfaces will come later, after feedback and guardrail iteration. Carbon's announcement gives an introductory launch price of $2 per million input tokens and $10 per million output tokens, with cached input tokens at a 95% discount. That announced price does not mean every developer can call the model today. Mistral's current model documentation lists API rates of $1.36 per million input tokens, $0.14 per million cached input tokens, and $4.18 per million output tokens. These API rates are not directly equivalent to the cost of self-hosting the weights.

The comparison is therefore not “open versus closed” in the abstract. Mistral says its weights are now published, but deployment still depends on the model card, license, hardware, and serving stack. Google describes Carbon as a restricted rollout, with broader developer access later. Confirm the live terms and availability before making a production plan.

Availability is part of the product

Teams often compare models as if every named release were an API endpoint waiting to be called. In practice, access can decide a project before quality does.

A restricted preview may require an application, an invitation, or a particular use case. A public API may still have preview terms, quotas, changing model behavior, or no production service commitment. Downloadable weights can give deployment flexibility, but only after the artifacts, license, serving requirements, and support path are clear.

Treat these as separate questions:

Can our team access it now? Check the account and product surface, not just the announcement headline.
Can we use it for our intended workload? Preview participation may be limited to certain organizations or use cases.
Can we rely on it in production? Verify terms, quotas, availability commitments, data handling, and support.
Can we deploy it where policy requires? For a hosted model, inspect provider controls. For weights, inspect license and infrastructure needs.
Can we measure it fairly? Compare outputs only after both models are accessible under conditions your team can document.

Availability is not an administrative footnote. It determines whether a result can be reproduced, whether a prototype can move to production, and how much work the team must do to get there.

A decision matrix for this week

Mistral Large 4 public preview API

The stated advantage is that interested developers can try the preview through Mistral Studio.
The announcement describes a large multimodal model aimed at coding, agentic workflows, and other complex tasks.
Mistral says the weights are now downloadable; inspect the model card, license, and hardware requirements before designing a self-hosting plan.
Because it is a preview, confirm the service conditions that matter to your workload instead of assuming the endpoint is production-ready.

Gemini 4 Carbon restricted rollout

The stated advantage is a model Google positions for cybersecurity and long-horizon work, with published benchmark claims worth investigating.
Initial availability is deliberately restricted to selected cyber defenders and trusted partners.
General developer, business, and consumer access is not available at announcement time; Google says it intends to expand access later.
The announced token price is an introductory future launch price, not evidence of current API access.

Neither as a production decision yet

If your team cannot access a model, there is no honest way to compare its behavior in your environment.
If the other is only a preview, a positive test may still not establish production readiness.
Use the gap to prepare a benchmark set, evaluation criteria, and access questions. That work remains useful after the rollout changes.

Build the evaluation before access arrives

A useful model evaluation does not need thousands of prompts. It needs representative tasks, a clear scoring rule, and a record of what was actually tested.

1. Select 10 to 20 realistic cases. Include normal tasks and edge cases. For security work, use authorized, non-production test material. For coding, choose issues with tests or other observable acceptance criteria.

2. Define success before seeing outputs. Specify what must be correct, what errors are unacceptable, and when a human reviewer must intervene. Do not use “sounds convincing” as the score.

3. Keep a reproducible test record. Store the prompt, model identifier, date, settings, tools enabled, raw output, and scoring result. Record access mode and any preview restrictions too.

4. Measure the whole workflow. Capture latency, retries, usage cost where available, reviewer time, and repair effort. A model that produces a strong first answer but needs repeated correction may be less useful than the headline score suggests.

5. Separate quality from availability. Mark a model “not tested” when you lack access. Do not fill the gap with a vendor benchmark, a social post, or another team's anecdote. Keep vendor-published results in a separate column.

A compact scorecard:

Access route and eligibility: ___
Task success rate: ___ / ___
Critical errors: ___
Median and worst-case latency: ___
Cost or infrastructure estimate: ___
Human review and repair time: ___
Preview, license, or governance blockers: ___
Reproduction details saved: yes / no

This avoids the benchmark trap. A vendor's result can tell you what to investigate; it cannot tell you how a model handles your team's repository, policies, or failure cases.

What to verify before acting on the headlines

For Mistral Large 4: Verify the official weights, model card, license, included artifacts, and whether your serving stack can support the architecture and required throughput. The one-trillion total parameter figure and 52-billion active parameter figure describe different aspects of a mixture-of-experts model; neither alone provides a hardware bill. Benchmark your intended inference setup.

For Gemini 4 Carbon: Check whether your organization is eligible for the current rollout, what product or API surface is included, and whether the announced price and terms apply to your use. Google says broader developer and business access will come later, but that is not a date commitment in the announcement.

For both: Record the model version and date. Preview models can change. Re-run critical tests when a model is updated, and do not silently treat an early preview result as a stable production guarantee.

So, which one should you choose?

For an engineer who wants to explore immediately, Mistral Large 4 has the clearer public entry point in its announcement: a preview API on Mistral Studio. That is a statement about access, not proof that it is the better model or the better production service.

For teams working on cybersecurity, Gemini 4 Carbon's stated focus and published evaluations may make it worth tracking or applying to the restricted rollout. But unless the team is included, it cannot yet be evaluated directly. Google's scores should remain vendor-reported evidence until you can inspect the methodology and test the model yourself.

For self-hosting, Mistral's newly published weights make evaluation possible, but the right action is still to verify the files, license, and resource requirements. Do not promise an on-prem deployment before testing the target serving stack.

The practical decision today is therefore: prepare to evaluate Mistral if you have preview access; monitor or apply for Carbon if your use case fits its rollout; make no production winner claim until you can test both.

Bottom line

Mistral Large 4 and Gemini 4 Carbon are interesting for different reasons, but on October 29, 2026, access is the first comparison to make. Mistral says its preview API and weights are available. Google announced a phased, restricted Carbon rollout, with wider access later.

Keep three labels separate in your notes: announced, accessible to us, and validated on our workload. They are not interchangeable. A careful team can prepare its evaluation now, verify the real access state when it matters, and avoid turning a launch post into a production decision.

Save this checklist: verify access, verify terms, preserve model/version details, use representative tasks, measure repair and review time, and separate vendor benchmarks from your results.

Have you ever planned around a model announcement and then discovered access was still restricted? What did your team do while waiting?

Subscribe for more evidence-led comparisons and build guides on developer tools, AI systems, and practical engineering decisions.