DEV Community: Taha Echakiri

A Minimal CI/CD Pipeline for a Small Team: What to Automate First

Taha Echakiri — Mon, 02 Nov 2026 09:00:02 +0000

TL;DR

A CI/CD pipeline is an automated sequence that builds, tests and deploys software every time somebody changes the code, and it is described in a file that lives next to the code.
The first thing to automate is the test suite that runs on every push, because it gives developers feedback on each change.
The build artifact and the container image come second, since a deployment consumes those outputs and nothing else.
The deployment job follows the build artifact and sits behind a manual gate, so a person keeps the last word before production changes.
Secrets load from the runner environment at run time, and caching plus concurrency settings decide what the minutes cost.
Netics analysis organizes the rollout into six phases in a fixed order, and its first audit is a careful read of the pipeline file that already exists.

Consider a hypothetical small team in this situation. The deploy works, nobody remembers the exact sequence of commands, and the one person who does is on holiday. The pipeline can stay small when the team limits its scope to the delivery steps described below.

The GitHub Actions quickstart supplies background facts about workflow files, push triggers and run results, and GitLab's CI/CD pipeline and YAML references supply additional platform facts; the article's subject and six-phase ordering are Netics planning analysis. It starts with the cheapest automation that pays back the same phase.

A Pipeline Runs Stages of Jobs on a Machine

Both platforms describe the same object with slightly different words. GitHub Actions documents a workflow as a file in .github/workflows with a .yml or .yaml extension, triggered by an event such as a push, holding jobs that run on a runner such as ubuntu-latest, and each job holding steps that execute shell commands or call a published action such as actions/checkout@v6.

GitLab documents a pipeline as a .gitlab-ci.yml file built from YAML keywords, composed of global keywords, jobs executed by runners, and stages that group those jobs. Stages run in sequence and the jobs inside one stage run in parallel. When stages is left undefined, the default set is .pre, build, test, deploy and .post, and a job that names no stage lands in test.

The rules that matter for planning sit in one sentence of the GitLab documentation: if every job in a stage succeeds, the next stage starts, and if any job fails, later stages do not start. That single behaviour is why the first automation belongs at the front of the sequence.

Page capture of the GitHub Docs workflow syntax reference (retrieved 28 October 2026, docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax), the page that documents the keys a workflow file is built from.

The Test Suite on Every Push Comes First

Tests are the first automation because they run on the pipeline's configured runner and usually need neither deployment credentials nor permission from anybody outside the team. GitLab runs a pipeline on every commit pushed to a branch with no configuration at all, and a GitHub workflow can use this push trigger: on: [push].

Start with what a developer already runs locally: the linter and the unit tests. A fast suite keeps the feedback loop usable. When a suite becomes slow, teams can start skipping, muting, or rerunning it until it passes, which erodes trust in a green result.

The first pipeline should fail loudly and fail early. Order the steps so the cheapest check runs first, keep a broken build blocking, and leave integration tests for the stage after, once the short feedback loop is trusted.

A job from the GitHub Actions quickstart (retrieved 28 October 2026, docs.github.com/en/actions/get-started/quickstart) showing each step of a workflow run with the seconds it took, which is the feedback a developer reads after a push.

The Build Artifact and the Container Image Come Second

Once tests run on every push, the next gap is that the thing tested is rebuilt by hand at deploy time. Building once and passing the result forward removes that gap.

GitLab handles it by default: jobs in later stages download the artifacts created by jobs in earlier stages, and that behaviour can be tuned per job with the dependencies and needs keywords. The practical shape is one build job in this Netics analysis that produces the archive or the image, and later jobs that consume it without repeating the compile.

In this Netics sequence, container images deserve their own step when they are the artifact the team deploys. Build the image once per commit in this Netics analysis, tag it with the commit identifier, push it to a registry, and let every later stage reference that tag.

In a first pipeline, define caching and artifacts according to their separate roles. A cache holds dependencies so the next run starts faster. An artifact is the output you keep and deploy. According to GitLab's caching and artifact documentation, caches let subsequent jobs execute more quickly and artifacts pass build results between stages.

A pipeline detail page from the GitLab documentation (retrieved 28 October 2026, docs.gitlab.com/ci/pipelines) grouped by job dependencies, the view that shows which jobs must finish before the deploy can start.

A Deployment Job Sits Behind a Manual Gate

Both platforms provide a control point that lets the team keep a person in the loop during deployment.

GitLab documents manual jobs as the mechanism for exactly this: a pipeline can start automatically and still require a manual action to deploy to production, with a Run control on the pipeline graph, and a Run all manual control above a stage that contains only manual jobs. GitHub puts the same decision point in a deployment job that targets a protected environment, where a reviewer approves the run before it proceeds.

A deployment gate creates traceability around production releases. A deployment that waits for manual approval creates a record of who released what and when, and that record supports an incident review.

Put the gate on the deployment job only. A gate on the test stage trains people to click through approvals without reading them.

A merge request pipeline from the GitLab documentation (retrieved 28 October 2026, docs.gitlab.com/ci/pipelines) after the first stage passed, with the remaining stages queued behind it.

Secrets Load From the Runner Environment

The pipeline file is committed, reviewed and often visible to everyone who can read the repository, so credentials have to arrive from somewhere else.

GitLab documents CI/CD variables configured in the project settings and exposed to jobs at run time, and it adds a second control that costs nothing: variables marked as protected are available only to jobs running on protected branches, and runners marked as protected accept jobs only from those branches. The pair makes a deploy credential unreachable from a feature branch.

GitHub publishes secrets as a repository and organisation setting, with OpenID Connect available for cloud credentials that are exchanged at run time, which removes a long-lived key from the system altogether.

The recommendation for a first pipeline is direct. If a deploy credential appears in a YAML file, even one that is private today, rotate it and move it into the secret store before the next release. Repositories get forked, archives get exported, and logs get shared in chat.

Page capture of the GitLab CI/CD YAML syntax reference (retrieved 28 October 2026, docs.gitlab.com/ci/yaml), which documents stages, variables and the keywords the pipeline file is assembled from.

Caching and Concurrency Decide What the Minutes Cost

The bill arrives through the same counters that measure the pipeline, and both platforms expose the numbers.

GitLab calculates total pipeline duration while excluding pending queue time and the first run of any retried job, so that report omits time the team may still spend waiting for a runner. Alongside that, dependency caching with an integrity check keeps a cache honest when the underlying files change.

GitHub documents dependency caching and workflow concurrency as first-class settings. A concurrency group with cancel-in-progress set means a new push to the same branch cancels the run it supersedes, which is the cheapest optimisation available on a busy repository.

For a small team, start with a cache keyed on the dependency lockfile and a concurrency group per branch. Add a nightly scheduled run only after those two are in place.

Netics editorial visual: the automation order for a first pipeline, from the test suite on every push to the deployment gate and the cost controls that follow.

Netics Analysis: Six Phases of Rollout in a Fixed Order

A fixed sequence keeps the project from becoming a platform build. Take a concrete case. Consider an explicitly hypothetical software team whose release process remains manual. The sequence below is Netics analysis: the six-phase count, the split between phases one and two, the single build job and the single named releaser are Netics planning recommendations.

Phases one and two: the linter and the unit tests on every push, blocking merges into the main branch.
Phase three: one build job produces the artifact and the image tag once per commit.
Phase four: the deployment job sits behind a manual gate and one named person releases it.
Phase five: deploy credentials moved into the platform secret store, with protected branches and protected runners switched on.
Phase six: the cache keyed on the lockfile, a concurrency group per branch, and notification of a failed main-branch run into the team channel.

Each step is usable on its own, which is the property that makes the sequence survivable. A team that stops after phase two still has a working pipeline that catches broken code before it reaches a colleague.

Netics editorial visual: the six-phase sequence, one automation per phase, each usable before the next begins.

Where the First Audit Starts

An audit of an existing pipeline starts by reading the file and tracing its behavior. Open the pipeline file and answer these four questions in order: what runs on every push, what the build produces, who can release it, and where the credentials come from. Most teams find a pipeline that builds twice, deploys from a shell step, and holds a token in a variable that every branch can read.

Fixing those four answers is the work. For more information about Netics, book a consultation. For the corresponding platform reference, consult the GitHub Actions workflow syntax documentation. For the GitLab pipeline model, see the official CI/CD pipeline documentation. The official documentation describes the platform features used in this sequence. What the team buys with the effort is a release that a second person can perform, which is the difference between a holiday-proof system and a single point of failure.

For more on pipeline configuration, the official GitLab CI/CD pipeline documentation complements the discussion of human approval inside a longer automation chain in designing an automation pipeline that a human approves.

The GitHub Actions source is available in the official quickstart.

Originally published on the Netics blog.

Qdrant 1.20 Ranks Keyword Matches Inside the Full-Text Index

Taha Echakiri — Sun, 01 Nov 2026 19:00:11 +0000

TL;DR

Qdrant is an open-source vector database that stores embeddings beside payload fields and answers similarity searches over them.
Version 1.20, published on 22 October 2026, ranks keyword matches with BM25 directly on the full-text index, using payload text as its input.
Qdrant computes the BM25 statistics itself, average document length included, which keeps scoring accurate as documents are added and changed.
A text query ranks any indexed field across the Query API, and the k and b parameters are set per query.
A new code tokenizer splits identifiers on underscores and case changes, so getUserName, get_user_name and user name match each other.
Per-element indexes answer nested filters from the indexes alone, measured at 7 to 38 times faster on 100,000 points.
Turbo16 and Turbo8 shrink vector storage at two precision levels, and turbo16 kept recall@10 within 0.001 of float32 across the 12 search-benchmark datasets.
Internal authentication now guards the peer gRPC port by default, and inline storage takes the memory tier of the HNSW index, which makes the upgrade plan the first thing to read.
The upgrade needs a decision for clusters that configure only a read-only key.

Capture of the Qdrant 1.20 release post, qdrant.tech/blog/qdrant-1.20.x, taken 29 October 2026.

Keyword search inside a vector database used to be a sidecar. Teams that wanted BM25 fed every document through an embedding model to produce sparse vectors, stored those next to their dense vectors, and paid for the pipeline again whenever the corpus changed. Qdrant 1.20 moves that work into the index the database was already maintaining.

This article deliberately covers five groups relevant to retrieval and upgrade operations from a broader release that also adds snapshot storage on GCS and Azure Blob Storage, audit-log output to stdout or stderr, collection creation times, safer replica removal, local-time log timestamps and substring matching: BM25 ranking on the full-text index, a code tokenizer, per-element indexes for nested filters, two new TurboQuant datatypes, and a pair of security and memory-tier changes that alter how a cluster behaves after the upgrade. The release post describes BM25 ranking, code tokenization, per-element indexes and TurboQuant datatypes as product features. The internal-authentication and memory-tier changes require checks before an upgrade.

BM25 scoring runs on the full-text index and computes its own statistics

The old path forced an inference step on every document and every query. It also made the scoring statistics somebody else's problem: the average document length had to be tracked outside Qdrant, because the value was baked into each stored sparse vector and could not follow the data as it changed. Updating it meant re-encoding the whole corpus.

Netics editorial visual: where the BM25 inputs come from in each implementation. Sparse vectors carry the scoring inputs inside every vector; the full-text index derives them from the collection.

Version 1.20 derives those statistics from the collection itself, average document length included, so scoring is correct on the first query and stays correct as documents arrive. That is a smaller change than it sounds. The reason teams choose retrieval augmented generation for a production workload is that the corpus moves; a ranking function whose statistics are frozen in vectors moves with it only when someone rebuilds.

Sparse-vector BM25 remains supported, and the migration path the post describes keeps the data where it is: create the full-text index with the bm25 scoring type, configure the same text processing the sparse model used, then replace the sparse queries with text queries.

The text query ranks any indexed field across the Query API

The API shape is small. A full-text index on a field is created with the text type and the bm25 scoring type, and a query names the field through using while passing the search terms in a text object. Because it is a normal Query API query, it composes: a text query can run inside hybrid search alongside a dense vector, and it can rescore points another query retrieved.

Capture of the Qdrant indexing documentation, qdrant.tech/documentation/concepts/indexing, taken 29 October 2026.

The parameter placement is the part worth noticing. k and b are query-time settings, so tuning the length normalisation does not rebuild the index, and a collection can hold one index that serves several query profiles. That is a real operational difference from the sparse-vector path, where a change in the scoring configuration meant re-encoding.

The code tokenizer splits identifiers on underscores and case changes

The default word tokenizer breaks text at spaces and punctuation, which suits prose and fails on source code. getUserName becomes one token, so a search for user name finds nothing. Anyone who has searched a large repository knows the failure mode: the text is present, the index cannot see it.

The new code tokenizer splits identifiers at underscores and at case boundaries, acronyms included, and it also indexes the whole identifier. getUserName, get_user_name and user name all reach the same place, and hashmap still finds HashMap. Combined with BM25 on a full-text index, code search gains ranked results directly from payload text, which is a useful option for a self-hosted documentation or repository search box.

Qdrant 1.20 diagram from the release post, qdrant.tech/blog/qdrant-1.20.x: pointer 7 has no single offer that satisfies both conditions, point 9 does, which is the case a per-element index answers directly.

Per-element indexes answer nested filters directly from the indexes

Nested filters are the awkward case in any document store. A payload holds an array of objects such as offers, each with a shop and a price, and the filter asks for a point where one single offer satisfies both conditions. Indexes on offers[].shop and offers[].price could only shorten the candidate list; Qdrant then read each surviving payload, usually from disk, to check the pairing.

Version 1.20 adds a nested parameter to every payload index type, which builds a per-element index storing each array element separately. When every condition in a nested filter uses a field with a per-element index on the same array, the per-element indexes answer the filter directly. The query API stays as it was, so existing filters speed up as the indexes are created. On three nested filters over 100,000 points, the vendor measured 7 to 38 times faster than regular indexes with payload reads, with every payload already in memory; the gap is likely larger when payloads have to be fetched from disk.

Qdrant 1.20 diagram from the release post, qdrant.tech/blog/qdrant-1.20.x: float32 vectors go through the TurboQuant rotation and land on a turbo16 or turbo8 grid.

Turbo16 and Turbo8 shrink vectors at two precision levels

Storage is where the release pays for itself. float16 keeps equal relative precision for every coordinate, so a large coordinate carries a large rounding error, and embeddings with a few very large coordinates suffer for it. turbo16 applies the TurboQuant rotation first, spreading values across coordinates, then stores each vector on a single 16-bit integer grid. Same 2 bytes per dimension, and the vendor measured 7.5 times lower reconstruction error than float16 across 13 datasets.

turbo8 stores each dimension in one byte. In the vendor's 11-dataset benchmark, search ran 1.1 to 2.4 times faster than float16, recall came close to float16 at half the size, and the gain over turbo4 was largest at low dimensions. The 8-bit encoding is also available as TurboQuant quantization through bits8, which compresses about four times, against eight times for bits4; bits4 remains the default.

Netics editorial visual: the four numbers from the release post that a capacity decision turns on, with the benchmark each one came from.

Internal authentication now guards the peer gRPC port

Distributed Qdrant nodes talk over an internal gRPC API. In 1.20 that API requires authentication by default whenever an API key is configured, and it accepts only read-write keys: the read-only key and JWTs are rejected. The internal port can also be bound to its own address through cluster.p2p.host, so it can sit on a private interface while the public API keeps its current address.

The failure mode is specific and worth writing into the change ticket. A cluster that sets only read_only_api_key stops letting its peers talk to each other after the upgrade, and the fix is either setting api_key or setting enforce_internal_auth: false. Clusters that set only alt_api_key on 1.19.0 or 1.19.1 have to reach 1.19.2 first. Clusters without an API key see no change.

Inline storage takes the memory tier of the HNSW index

Inline storage places the original vectors in the HNSW index file next to the graph links, and until 1.20 Qdrant kept a duplicate copy in regular vector storage. The duplicate is gone, which saves disk. The consequence is that those vectors now inherit the memory tier of the index: a pinned index pins its vectors, and a cold index cools them. A collection that ran with vectors.memory: cold can therefore need more RAM after the upgrade, because the HNSW index defaults to cached.

Qdrant 1.20 diagram from the release post, qdrant.tech/blog/qdrant-1.20.x: the ID tracker moves between RAM, page cache and disk without changing the collection's vectors.

The same release extends memory tiers to the ID tracker, which maps each point ID to its location and version and historically sat in RAM at about 52 bytes per point, roughly 5 GB for 100 million points. pinned stays the default, cold reads from disk on demand, and cached loads the tracker into the page cache when a segment opens.

The upgrade order a 1.20 cluster needs

Read the release as an operations document. The documentation identifies two changes that can break a working cluster: the internal authentication default and the tier inheritance for inline-storage vectors. BM25, code tokenization, per-element indexes and TurboQuant datatypes are available for separate evaluation after those checks.

BM25 on the full-text index is the feature worth testing first. BM25 on the full-text index removes a pipeline and a data-consistency problem, and it costs one index creation to try on a copy of a collection. The nested per-element index can improve an application through index configuration alone, making it the cheapest experiment on the list.

Netics editorial checklist: the four items to settle before a Qdrant cluster moves to 1.20, in the order that avoids a failed rolling upgrade.

Upgrade one minor version at a time, so 1.18 goes to 1.19 and then to 1.20.0, and set hnsw_config.memory and the quantization memory tier before the upgrade to preserve the intended memory placement throughout the change. A stagger test on one node answers most of the questions here, which is cheaper than the same answer from a production incident.

For related operational analysis, see the Netics blog. For this upgrade, a first pass on one collection usually settles the questions above; compare the result with the Qdrant release notes. The release notes carry the numbers this article quotes, and a staging node can confirm the tier behaviour above; book a Netics review if you want help planning that test.

Originally published on the Netics blog.

Europe's Sovereignty Package Draws a Line Most Clouds Will Not Cross

Taha Echakiri — Sun, 01 Nov 2026 14:00:30 +0000

TL;DR

On June 3, 2026, the European Commission presented the Technological Sovereignty Package: the Chips Act 2.0 and the Cloud and AI Development Act (CADA), plus an EU Open Source Strategy and an energy digitalisation roadmap.
CADA would triple EU data-center capacity in five to seven years and introduces a common EU framework to grade cloud and AI sovereignty on four levels — from EU-resident data to full supply-chain control and freedom from third-country interference.
On June 25 the Commission disclosed a preliminary view that AWS and Azure should be designated gatekeepers under the Digital Markets Act (DMA) — a separate process from CADA, but the sovereignty debate reads them together.
The September status is where ambition meets reality: proposals are in Council and Parliament negotiation, and European defence officials are pushing back on the strictest sovereignty tiers.
For a French or EU SME the practical question is procurement: which of the four levels will public buyers actually require, and what evidence will they accept?

The European Commission presented its Technological Sovereignty Package on June 3, 2026. It bundles two legislative proposals — the Chips Act 2.0 and the Cloud and AI Development Act (CADA) — with an EU Open Source Strategy and a Strategic Roadmap for Digitalisation and AI in the Energy Sector. Commission President Ursula von der Leyen framed it in security terms: "We cannot afford to depend on others for the technologies that keep our hospitals running, our energy grids stable and our services secure."

Netics editorial visual: CADA's four-level sovereignty ladder from EU-resident data (level 1) to full supply-chain control and third-country freedom (level 4).

The scale of the dependency the package answers is blunt. The EU relies on non-EU countries for more than 80% of its digital products, services, infrastructure, and intellectual property, according to EU-institution data cited in the European Parliament's January 2026 resolution on technological sovereignty (El País, September 19, 2026). The package is the first attempt to make that dependency a procurement and investment problem rather than a talking point.

CADA's four-level sovereignty scale is the operational heart

The most innovative — and most contested — part of CADA is its definition of sovereignty. The Commission wants to move beyond the idea that a cloud is sovereign simply because data physically sits in the EU. As El País's report describes the draft proposal, the framework establishes four levels:

Level 1: data processed and stored on infrastructure located in the EU.
Level 2: level 1 plus independence from third countries and transparency on the software supply chain.
Level 3: level 2 plus a provider that is owned and controlled from within the EU.
Level 4: the highest tier — full transparency and control over the software supply chain, and freedom from interference by third countries.

That scale is the anti-"sovereign washing" mechanism: a vendor can no longer claim sovereignty by geography alone. It also becomes the grading tool for public procurement. The Commission's stated rationale is that the absence of a verifiable framework lets services present themselves as sovereign without criteria — CADA is designed to close exactly that gap.

The September reporting adds teeth and friction. Cloud Computing News, on September 7, 2026, described CADA's two-article risk-assessment structure: under Article 29, member states and EU entities would map public-sector activities that use cloud services; activities in areas including national security, defence, internal security, border management, justice, and law enforcement would have to use services recognised at assurance Levels 2, 3, or 4 under Article 30. Coverage on shattered.io adds that Article 30 ties the required assurance level to the sensitivity of the workload — a rural transit authority's records system faces lighter scrutiny than a signals-intelligence platform. The same coverage describes procurement rules for "highly critical state tenders" that would let contracting authorities exclude bidders based on ownership, financing structure, or exposure to third-country data-access law. Cloud Computing News is explicit that the proposal does not impose a blanket ban on US cloud providers: it allows exceptions where compliant services are unavailable or procurement processes fail to produce suitable offers.

Screenshot of the European Commission press-corner page (press release IP/26/1187, "Commission proposes tech sovereignty package to strengthen Europe's digital autonomy and resilience"), retrieved September 26, 2026.

Chips Act 2.0: resilience, not self-sufficiency

The second pillar, Chips Act 2.0, starts from a structural fact: Europe accounts for just under 10% of global semiconductor production, according to a Polytechnique Insights analysis cited by El País. AI-related components are expected to represent more than 70% of the global semiconductor market by 2030 per Commission figures. The proposal aims to strengthen European capabilities, deepen cooperation with like-minded partners, speed permitting, stimulate European demand, and introduce an excellence label for Europe's semiconductor regions.

The honest framing is in the Commission's own language: the goal is not for Europe to manufacture every chip it needs — that would be unrealistic — but to ensure Europe does not lose industrial capacities that could become critical if the global supply chain breaks down. The pandemic, US-China trade tensions, and restrictions on specific components are the evidence base. For buyers, the practical effect will be a label system (the excellence label) and faster permitting for strategic projects, not a near-term domestic alternative to TSMC or Samsung foundries.

The gatekeeper signal, and the procurement lever

Two facts make clear this is not only an industrial-policy exercise. On June 25, 2026, the Commission informed Amazon and Microsoft of its preliminary view that AWS and Azure — the EU's largest and second-largest cloud services — should be designated gatekeepers under the Digital Markets Act, saying they are an important gateway between businesses and their EU customers despite not meeting the DMA's quantitative thresholds; El País notes Brussels argues the two have entrenched market positions, high switching costs, and strong lock-in effects. And in April 2026 the Commission awarded contracts worth up to €180 million over six years for sovereign cloud services for EU institutions — an anchor-customer move for European providers.

Screenshot of the European Commission news page "Commission advances cloud sovereignty through strategic procurement" (April 17, 2026), retrieved September 26, 2026.

The procurement lever is the mechanism that makes the four-level scale binding in practice. A sovereignty framework that only policy documents use is a position paper; a sovereignty framework that sits inside tender evaluation is a market structure. CADA does the latter, which is why the defence pushback matters: Cloud Computing News reported, citing the Financial Times, that European defence officials — including from eastern and Nordic member states — are concerned the provisions could restrict the use of US cloud providers for some high-security workloads and complicate interoperability with NATO systems. That tension is not a bug in CADA; it is the design stress-test every serious sovereignty framework eventually hits.

What a French or EU SME should do now

The package will not be law this year; both proposals are in Council and Parliament negotiation. But the procurement direction is already legible, and SMEs building or buying cloud services should treat it as a decision input today, not a compliance item for 2028:

Map which of the four levels your own cloud and AI services would satisfy today, with evidence. "Data in the EU" is level 1; supply-chain transparency is level 2; EU ownership is level 3. The gap between your marketing and your evidence is exactly the gap CADA targets.
Watch the public-sector anchor: the €180 million EU-institutions contracts and the gatekeeper process for AWS and Azure define the reference point for what sovereign procurement will demand. Private-sector buyers in regulated verticals will likely inherit those expectations.
Treat the defence pushback as useful signal: the strictest tiers will be negotiated down or phased, so level 4 is unlikely to become a universal requirement. Level 2 — data residency plus supply-chain transparency — is the realistic floor for sensitive workloads.
Keep the skills argument this blog has made in view: a framework grades providers, but running sovereign infrastructure still requires the operational capability to operate it. Legislation is the demand side; skills are the supply side.

The full text of the proposals will change in negotiation. What will not change is the direction: European public procurement is moving toward verifiable sovereignty criteria, and the four-level scale is the reference model every vendor and buyer will be scored against.

Netics editorial visual: the four-level scale and what a French or EU SME should verify per level before a procurement decision.

Originally published on the Netics blog.

Three MoE Inference Optimizations NVIDIA Measured on DGX Spark

Taha Echakiri — Sun, 01 Nov 2026 14:00:02 +0000

TL;DR

The DGX Spark is a compact AI computer from NVIDIA for developers and researchers, built on the GB10 Grace Blackwell superchip with up to 128 GB of unified memory, and the model benchmarked on it here is Ling-3.0-flash from Inclusion AI, a mixture-of-experts model with 124 billion parameters.
NVIDIA's post published on 27 October 2026 measures three changes to that model's decode loop at batch size 1, and reports time per output token falling from 27.71 ms to 9.72 ms.
Profiling showed decoding was memory-bound: reading model weights drove execution time, and the MoE computation plus the output projection dominated it.
Humming MXFP4 kernels compile at run time for the model's real shapes, which measured +9.3% decode throughput.
Online FP8 quantization halves the LM head weight reads and brings the stack to +21.7% over baseline.
Multi-token prediction with NEXTN turns one verification into four tokens, and the synthetic workload accepted 3.90 tokens per round.
GSM8K measured a 42.5% throughput gain across 1,319 questions with accuracy unchanged within 0.08 of a point.
The synthetic workload overstates the gain a real workload sees, because repetitive output is easier to draft.
One missing interface makes the stack slower than baseline, which is why the vendor recommends validating each optimization at startup.

Official NVIDIA product photo of the DGX Spark family, from the product page at nvidia.com/en-us/products/workstations/dgx-spark, retrieved 29 October 2026.

A batch of one is the least glamorous benchmark in AI infrastructure and the one most people actually run. It is the shape of a coding assistant answering one developer, a document summariser serving one request, an agent waiting on its own next token. NVIDIA chose that shape for this measurement, and the result is a useful case study in how much of a decode step is spent reading weights off memory.

Decoding was memory-bound on a 124-billion-parameter model at batch size 1

Ling-3.0-flash keeps 5.1 billion of its 124 billion parameters active per token. It has 42 layers, 35 of them Kimi Delta Attention linear-attention layers and seven multi-head latent attention layers, and each MoE layer routes a token to eight of 512 routed experts alongside one shared expert. The LM head projects a hidden dimension of 2,560 onto a vocabulary of 157,184 tokens.

Capture of the NVIDIA technical blog post carrying the measurement, developer.nvidia.com/blog/3-ways-to-boost-moe-inference-on-nvidia-dgx-spark, taken 29 October 2026.

At batch size 1, arithmetic units sit idle and the decode step waits on memory bandwidth. Two operations carried the cost: the routed-expert computation, and the vocabulary projection whose entire weight block has to be read for every generated token. NVIDIA's three optimizations attack those two costs in two different ways, and the article's value is that it reports each one separately before stacking them.

Humming MXFP4 kernels compile at run time for the model's real shapes

The baseline ran the official W4A8 recipe: FlashInfer MXFP4 MoE kernels, CUTLASS dense GEMM, and the --enable-fp32-lm-head option. Swapping the MoE backend for Humming moved time per output token from 27.71 ms to 25.35 ms and decode throughput from 36.09 to 39.45 tokens per second, a 9.3% gain.

The mechanism explains the modest size of the gain. Humming uses NVRTC to generate and compile CUBINs at run time for the actual M, N and K dimensions, the data types and the GPU architecture, then caches them in ~/.humming/cache. That specialisation matters on a model with a narrow moe_intermediate_size of 768, and it changes the build experience: an individual kernel compiles in seconds, while the FlashInfer CUTLASS path uses ninja to compile 97 targets and takes tens of minutes. A shorter build is a practical gain that never shows up in a throughput table.

Netics editorial visual built from the post's end-to-end table: baseline against all three optimizations across all 1,319 GSM8K test questions.

Online FP8 quantization halves the LM head weight reads

The checkpoint stores the LM head in BF16 and lists it in modules_to_not_convert, so loading never quantizes it. Its vocabulary GEMM is 157,184 by 2,560, and every generated token reads that whole block. Under a bandwidth bottleneck it is the obvious target.

NVIDIA applies a dynamic FP8 quantization configuration when the LM head is constructed, leaving the rest of the model on the checkpoint's FP8 and MXFP4 settings. That halves the read volume for the layer, and the setting is one environment variable, SGLANG_ENABLE_FP8_LM_HEAD=1. The cumulative table shows 22.76 ms and 43.94 tokens per second, 21.7% over baseline and 11.4% over Humming alone. The flag is mutually exclusive with the baseline's FP32 logits option, which is worth knowing before someone enables both and wonders why nothing changed.

Netics editorial visual: the order the three techniques apply to a decode loop, and the startup check that confirms each one is active.

Multi-token prediction turns each verification into four tokens

The first two optimizations reduce the cost of a decode step. Speculative decoding reduces the number of steps, and Ling-3.0-flash ships a NEXTN prediction layer (num_nextn_predict_layers: 1) that works as a draft model with no extra training or download.

The configuration uses three draft steps and a four-token verification window: the draft predicts three tokens in sequence, and the target verifies four positions at once, accepting the correct prefix. The benefit depends on accept length and on the verification-cost ratio, which measured about 2.26. The interesting part of the analysis is where that cost came from. At batch size 1, small KV reads and larger weight reads made expert activation the principal verification cost: a normal decode activates eight experts for one token while verification over four tokens can activate up to 32.

On the synthetic 32K-input, 2K-output workload the stack reached 9.72 ms and 102.88 tokens per second, 185.1% over baseline, with an accept length of 3.90. That number is where a careful reader slows down.

Netics editorial visual: the four figures that decide whether this stack is worth deploying, each traceable to the post's tables.

GSM8K measured a 42.5% throughput gain across 1,319 questions

The second evaluation is the one worth trusting more. NVIDIA ran all 1,319 GSM8K test questions with five-shot prompting, temperature 0.6, top_p 0.95, top_k 29, a 32,768-token ceiling and chain-of-thought enabled, holding sampling, parser, seed and memory allocation fixed.

Across that set, time per output token went from 26.45 ms to 18.56 ms, decode throughput from 37.81 to 53.89 tokens per second, a 42.5% gain, and mean latency from 15.21 s to 11.49 s. P50 fell from 8.36 s to 6.11 s and P95 from 50.20 s to 38.60 s. GSM8K accuracy moved from 96.89% to 96.82%, and the run recorded zero errors and zero truncations. The measured gain is real and roughly a quarter of the synthetic claim.

The synthetic workload overstates the gain a real workload sees

NVIDIA states the caveat itself, which is rarer than it should be. Accept length was 3.90 on the synthetic run against 2.68 on GSM8K. Three differences explain most of the gap: synthetic output repeats, which makes drafting easy; the synthetic test decoded greedily while the real evaluation sampled at temperature 0.6 with top_p 0.95 and top_k 29; and the synthetic workload fixed output at 2K tokens while the median GSM8K output is about 310 tokens, which spreads the fixed overhead reductions over a much shorter generation.

Capture of the DGX Spark user guide landing page, docs.nvidia.com/dgx/dgx-spark/index.html, taken 29 October 2026.

That last point is the one to carry into a deployment decision. Speculative decoding pays per token generated, and the two other optimizations pay per step, so a workload with short answers collects most of the first two gains and a fraction of the third. The same discipline applies to any vendor benchmark: the useful question is which of the measured mechanisms your own traffic can reach. A previous analysis of how CUDA 13.4 turns a GPU fraction into an enforceable memory limit followed the same logic, because a capability only counts once the platform enforces it under your workload.

One missing interface makes the stack slower than baseline

Stacking the three optimizations is where the article earns its keep. Each worked alone. Together, the FP8 LM-head path and MTP need the draft model to share the entire LM-head module, because the draft has to interpret the same quantized weights as the target. The original Ling NEXTN implementation exposed only the weight tensor.

Netics editorial visual: the integration requirement behind the FP8 LM head and MTP pairing, and the failure mode when the interface is absent.

Without the interface the framework falls back silently: the draft receives FP8 weights and reads them as BF16, accept length collapses to 1.00, and throughput ends up below the configuration with no optimizations at all. Nothing fails, nothing logs an error, and the only symptom is that the machine is slower than it was before someone made it faster. NVIDIA's guidance is to validate at startup that every optimization is active, which is more reliable than inferring configuration state from a later performance number.

This paragraph belongs in an internal wiki. A silent fallback in an inference stack can create a capacity incident on launch day, so startup validation belongs in the deployment checklist. The model, the kernels and the quantization are NVIDIA's engineering; the discipline of proving each layer is active belongs to whoever operates the deployment. Related resources include Netics, and the deployment planning page.

Originally published on the Netics blog.

NVIDIA Mission Control 2.5 Brings Autonomous Recovery to Kubernetes Jobs

Taha Echakiri — Sun, 01 Nov 2026 09:00:02 +0000

TL;DR

NVIDIA Mission Control is a management software layer for an AI factory's compute estate: it schedules workloads, watches the hardware and coordinates recovery when a job stops.
Version 2.5, published on 26 October 2026, extends its autonomous recovery engine from Slurm-based HPC jobs to Kubernetes workloads scheduled by NVIDIA Run:ai.
The engine splits into autonomous job recovery, which classifies a failure from job context and failure history, and autonomous hardware recovery, which isolates nodes, runs diagnostics and keeps bad hardware out of scheduling.
During a hardware fault the job restarts on healthy capacity while the faulty node is diagnosed in parallel, so recovery begins before any root-cause analysis finishes.
A transient software error receives a retry under scheduler policy, and the hardware service stays out of the recovery path.
The post publishes figures for the control loop and each failure path, and it carries no measured recovery time or utilisation gain, so the numbers a buyer wants are still to come.

Large training runs fail in a way that is easy to underestimate. A job spanning hundreds of nodes for days keeps every worker in step, so one degraded GPU can hold up the whole collective. The usual response is a pager, a person, a cordon command and a restart, and every minute of that sequence burns accelerator time nobody is using.

NVIDIA's answer has been an autonomous recovery engine inside Mission Control, and until this release it only understood Slurm. Version 2.5 carries the same engine to Kubernetes workloads that Run:ai schedules.

Page capture of the NVIDIA post (retrieved 28 October 2026, developer.nvidia.com/blog/how-nvidia-mission-control-2-5-brings-autonomous-recovery-to-kubernetes): the announcement of 26 October 2026 by Ryan Prout and Casey Dugas, with the summary NVIDIA generated for it.

The Loop Separates Placement, Job Context and Hardware Health

The design that matters here is a division of labour. Three components hold one concern each: the scheduler decides where work goes, the job recovery service understands what the work was doing, and the hardware recovery service understands the state of the machine.

Run:ai acts as the Kubernetes default scheduler and ships as part of the software, owning workloads, queues and policies. The job recovery service watches running jobs, tracks where each one executes, and uses failure history and job topology to classify a recovery path before the resource manager turns that judgement into a restart, a retry, a re-queue or a different placement. The hardware recovery service monitors nodes and GPUs, pulls anomalies out of scheduling and runs diagnostics or remediation.

NVIDIA states the reason for the split plainly: "Keeping these responsibilities separate is what makes the model portable." A job recovery service that never touches hardware and a hardware service that never parses a training framework can both sit above Slurm and above Run:ai.

Figure 2 of the NVIDIA post (retrieved 28 October 2026, developer.nvidia.com/blog/how-nvidia-mission-control-2-5-brings-autonomous-recovery-to-kubernetes): the Kubernetes scheduler, the autonomous job recovery service and the autonomous hardware recovery service.

Run:ai Places the Job and Mission Control Reads Its History

Placement carries most of the credit in GPU scheduling conversations, and this release gives it a narrower role. Run:ai decides which queue and which capacity a job lands on. The recovery engine decides what to do when that job stops.

Job topology is the interesting input. A failure in a tightly coupled training job with a shared collective is expensive to ignore, and a failure in a job with independent shards leaves most of the run intact. The recovery service classifies from that context, which is why the same symptom can produce a restart on one job and a re-queue on another.

Keeping the logic in a service, separate from the scheduler, has a practical consequence for a platform team: the recovery behaviour stays the same when the scheduler changes, and the scheduler can keep placing work without waiting for a diagnosis.

Figure 1 of the NVIDIA post (retrieved 28 October 2026, developer.nvidia.com/blog/how-nvidia-mission-control-2-5-brings-autonomous-recovery-to-kubernetes), a matrix of node health against scheduler availability, with the states recovery automation moves nodes out of.

A Hardware Fault Starts Two Tracks at Once

The first failure path is the one operators know best. A GPU begins to fail, and the symptom arrives as ECC errors, a hung worker or a communication timeout inside the job.

Recovery splits into two tracks that run at the same time. The job recovery service captures the job context and identifies the suspect node or GPU, then the scheduler integration restarts or reschedules the workload onto healthy capacity. Meanwhile the hardware recovery service isolates the node, records why, and starts diagnostics that continue after training has resumed.

The gain here is time. The workload returns to productive compute without waiting for a root-cause answer, and the suspect node leaves the scheduling pool so the next job cannot land on it.

Figure 3 of the NVIDIA post (retrieved 28 October 2026, developer.nvidia.com/blog/how-nvidia-mission-control-2-5-brings-autonomous-recovery-to-kubernetes): the job restarting on healthy nodes while remediation continues on the faulty one.

A Transient Software Error Gets a Retry Under Scheduler Policy

The second path is the more interesting design decision. A training worker can exit, hit a recoverable timeout, or lose coordination for a moment while the node underneath it stays perfectly healthy.

An aggressive recovery system cordons the node anyway, and a cluster that loses a node to every software hiccup quietly shrinks its own capacity. Mission Control reads the failure history first. When the pattern looks transient, the workload retries under scheduler policy, and the software-error figure in the post shows the path: failure context captured, retry decision taken, workload restarted from a checkpoint, training resumed.

Repeated or non-retryable failures take the other branch and reach the operators. Restraint is the property to look for in a recovery system, and NVIDIA names it as the value of this path.

Figure 5 of the NVIDIA post (retrieved 28 October 2026, developer.nvidia.com/blog/how-nvidia-mission-control-2-5-brings-autonomous-recovery-to-kubernetes): a software error interrupting a job, the retry decision, the workload restarting from a checkpoint, and repeated failures routed to escalation.

Early Degradation Removes a Node Before a Job Lands

The third path is preventive. A node starts showing repeated GPU health warnings or connectivity problems, and it qualifies as risky capacity before any job has failed on it.

The hardware recovery service can remove that node from scheduler eligibility and start diagnostics before a training job is placed there. The job recovery service may never be involved, because the capacity left the pool before it could break anything.

For a scheduler, a cleaner pool is worth as much as a faster restart. Placement decisions made against known-bad capacity generate the outages that a recovery service then has to clean up.

The Hardware Path Records Why a Node Left Service

A machine that disappears from scheduling without a record is an operational debt. The next engineer on call sees a missing node, no reason and no maintenance history.

The hardware track keeps that record. Figure 4 of the post lays the sequence out in seven steps: the faulty node is removed from service, labelled as a break fix candidate, annotated with a reason for audit tracking, found by a poller that looks for nodes removed from scheduler service, checked for agent connectivity, held under a maintenance lock so two systems do not do the same work, and finally handed to hardware diagnostics. The node returns to service only when it is ready.

Figure 4 of the NVIDIA post (retrieved 28 October 2026, developer.nvidia.com/blog/how-nvidia-mission-control-2-5-brings-autonomous-recovery-to-kubernetes): the seven breakfix steps, from removing a faulty node from service to the maintenance lock and hardware diagnostics.

What the Post Measures and What It Leaves Open

This is a design announcement backed by figures of the control loop. It contains no recovery time, no mean-time-to-repair comparison and no utilisation gain, which means the case for adopting it still rests on architecture, and the measurements are still to come.

That absence is worth stating plainly, because the purchasing decision usually turns on the opposite question: how many GPU hours does autonomous recovery return per month. A team can measure that itself on one cluster before a wider rollout, and the measurement needs a baseline taken before the change.

The release also lists four additional highlights. They are central Keycloak authentication across services, security and Kubernetes hardening updates, an AI-enabled network anomaly detection service in beta, and early access for NVIDIA Vera Rubin NVL72. Plan on the recovery engine, and treat the rest as roadmap.

Netics editorial visual: what autonomous recovery covers on a Kubernetes AI cluster, from the failure signal to the recovered job.

Where Recovery Design Meets Capacity Planning

The recovery engine answers a question that every operator of shared GPU infrastructure faces: what happens after something breaks. The answer in 2.5 is a restart on healthy capacity, a node removed from the pool, and a record that explains both.

Two things decide the value in practice. The first is checkpointing, since a retry that restarts from zero wastes most of the recovery. The second is the baseline, since a team that cannot say how much time it lost last quarter cannot show what the engine returns.

Mission Control 2.5 is available now for existing users of the software. Capacity planning also shapes sovereign infrastructure and cloud exit work, and the scheduling layer beneath it was covered when NVIDIA open-sourced its GPU orchestration components.

Netics editorial visual: placement, job recovery and hardware recovery as three separate responsibilities, with the record each one leaves behind.

Originally published on the Netics blog.

Google Cloud Turns the Wiz Runtime Sensor Into a GKE Add-On

Taha Echakiri — Sat, 31 Oct 2026 19:00:03 +0000

TL;DR

The Wiz Runtime Sensor is a monitoring component from Wiz that watches what running containers actually do inside a Kubernetes cluster, at kernel level.
Google Cloud announced it on 17 October 2026 as a managed add-on for Google Kubernetes Engine, in preview, for both Standard and Autopilot clusters.
GKE installs the sensor, injects its credentials from Secret Manager and updates it outside the cluster version cycle.
A custom organization policy can reject any new cluster created without the add-on, which moves enforcement to the platform team.
The sensor reads system calls, processes, network connections and file access, with a fixed memory limit on each node, and sends that telemetry to the Wiz Security Graph.
Preview status, the egress proxy path and the gap between a GKE-managed and a Helm-managed install are the first things an operator should verify.

Runtime security has been the awkward purchase in Kubernetes for years. Every cluster needs an agent, every agent needs a credential, and the team that owns the clusters is rarely the team that owns the agent's licence. Google Cloud and Wiz have now put the agent inside the platform itself.

The announcement is short and specific: the Wiz Runtime Sensor becomes a first-party add-on for GKE, in preview, covering Standard and Autopilot clusters, with GKE handling deployment, credential injection and updates. The detection engine is the visible part of the release. The release also moves credential handling and fleet enforcement into the platform, changing the operating model.

A Runtime Sensor Watches What Containers Do at Kernel Level

A container image can be scanned before it runs, and a manifest can be checked before it is admitted. Runtime observation covers processes that begin after an application launches.

The Wiz Runtime Sensor covers that window. It observes system calls, running processes, network connections and file access. The GKE documentation demonstrates a reverse-shell alert, while the Wiz engineering post lists reverse shells, binary execution from /tmp, image drift and privilege-escalation attempts. Processing happens on the node through an eBPF program, so the library does not need to be linked into the application.

Resource use is bounded on purpose. The sensor keeps a fixed memory limit on each node, which matters on a cluster where workload density changes through the day, and the announcement points at AI workloads and web services as the cases where that predictability helps.

The architecture figure published with the Google Cloud announcement (retrieved 28 October 2026, cloud.google.com/blog/products/identity-security/wiz-sensor-for-gke-turnkey-threat-detection-for-kubernetes), showing credential sync, sensor deployment and the threat telemetry path back to Wiz.

GKE Installs the Sensor and Pipes Credentials From Secret Manager

The part of this release that changes an operating model is the credential path. Distributing a vendor API key into a namespace as a Kubernetes Secret is the standard shortcut, and it is the step that turns a security product into a secret to rotate at scale.

Here the key lives outside the cluster. GKE uses its Secret Manager secret sync integration to hold the Wiz credentials in a central vault that only a dedicated service account can read, then retrieves them and injects them into the sensor pods. Updating a key in Secret Manager automatically re-synchronizes the Kubernetes Secret. Using updated registry or API credentials then requires a manual restart of the Runtime Sensor DaemonSet.

GKE documentation adds the mechanical detail: enabling the add-on creates the gke-managed-wiz namespace, installs the Runtime Sensor components there, and pulls the images from the Wiz private container registry.

Wiz's own view of the chain (retrieved 28 October 2026, wiz.io/blog/wiz-sensor-on-gke-1-click-runtime-protection-now-native): the GKE control plane, Google Cloud Secret Manager, the node-level eBPF engine and the Wiz Security Graph.

One gcloud Flag Enables It on a New or Existing Cluster

Enabling the sensor follows the path a platform administrator already knows. On a new cluster the announcement documents a single flag on the cluster create command, --addons=WizSensor=ENABLED, alongside the two Secret Manager references that name the registry secret and the API key.

On a running cluster the equivalent is --update-addons=WizSensor=ENABLED, and the prerequisites are explicit: Workload Identity Federation for GKE and Secret Manager secret sync have to be in place before any of it works. That pairing is the real gate, since a cluster without workload identity cannot present the service account the vault checks.

For teams that restrict outbound traffic, the announcement documents standard HTTP and mutual TLS proxy configurations so telemetry reaches Wiz while egress firewall rules stay intact.

Page capture of the GKE installation documentation (retrieved 28 October 2026, docs.cloud.google.com/kubernetes-engine/security/install-wiz-sensor-addon), which lists the prerequisites and the namespace the add-on creates.

An Organization Policy Makes Runtime Coverage a Create-Time Condition

Fleet-wide enforcement is where most runtime programmes fail. A team enables the agent on the clusters it knows about, and the cluster somebody spins up on a Friday afternoon arrives without it.

The add-on is designed to close that gap through Organization Policy Service custom constraints. The announcement publishes the constraint itself: a resource type of container.googleapis.com/Cluster, the method types CREATE and UPDATE, and a condition that the cluster's addon configuration has the sensor enabled. Apply it at the organization, folder or project level and a cluster created without runtime protection is refused.

Read that constraint as a governance instrument. It makes agent installation an API-enforced rule that platform teams can apply when clusters are created or updated.

A Wiz console view from the vendor's own post (retrieved 28 October 2026, wiz.io/blog/wiz-sensor-on-gke-1-click-runtime-protection-now-native) showing a sensor deployment reported by GKE with its active sensor pods.

Two practical notes about the constraint. It evaluates the cluster's add-on configuration at create and update time, so a later change to that configuration is checked as well as a brand new cluster. Applying it at the folder level covers the environments nobody remembers to check, which is where most coverage gaps start.

Telemetry Reaches the Wiz Security Graph and Answers a Narrower Question

Detection rules answer specific questions about runtime activity. Image scanning identifies vulnerable libraries in an image and supplies posture context. Runtime evidence shows which of those libraries are loaded and executing in memory.

The announcement frames the Wiz Security Graph as the place where those two views meet: cluster runtime activity correlated with image scans, cloud posture and identity. The practical effect is prioritisation around vulnerable code that is loaded and running.

Diagnostic data is collected when a threat fires, so an investigation starts with evidence attached.

Page capture of the Google Cloud announcement (retrieved 28 October 2026, cloud.google.com/blog/products/identity-security/wiz-sensor-for-gke-turnkey-threat-detection-for-kubernetes), the source of the enablement flags and the organization policy above.

Independent Updates Keep Detection Rules Ahead of the Cluster Version

Agent lifecycle is the operational tax of runtime security. Every agent image is a new thing to test, and every change usually waits for a maintenance window.

GKE documentation states the arrangement directly: "Any updates to the Runtime Sensor components are independent of GKE patch version upgrades." The announcement adds that sensor versions and detection rules published by Wiz roll out without GKE version bumps, node pool recreation or workload disruption.

That independence cuts both ways, and a platform team should decide which way it wants before enabling anything. A detection engine that updates on the vendor's schedule keeps pace with new attack behaviour. It also changes what runs on your nodes outside your release calendar, which is a governance question for a regulated environment.

Netics editorial visual: the split between what the managed add-on now owns and what stays with the platform team.

The Preview Label, the Proxy and the Helm Alternative Are the First Checks

Three details decide whether this is a production option this quarter.

The first is the preview status. GKE documentation places the feature under the Pre-GA terms of service, which means the usual "available as is" language and limited support commitments. A regulated workload can wait, and the wait is cheap because nothing forces an early decision.

The second is egress design. A sensor that cannot reach Wiz reports nothing, so the proxy configuration is part of the architecture and belongs in the firewall request early. The third is the install path. The Helm chart remains available, offers more configuration freedom, and leaves the components unmanaged by GKE, with issues reported to Wiz. Choose per estate: the add-on for fleet coverage, the chart for clusters that need tuning.

Consider a hypothetical case. Imagine a Paris platform team running forty clusters across three environments and two cloud projects. The add-on plus one organization constraint covers the fleet in a week, and the two clusters running bespoke networking stay on the chart.

Netics editorial visual: the preview status, the egress path and the install path, the three checks that come before any rollout.

Where Runtime Coverage Fits Beside the Configuration Review

Runtime detection complements the controls that decide what a workload may do before it starts. Admission policies, least-privilege service accounts, network rules and the sensor work together to record configured boundaries and activity inside them.

The add-on makes that layer cheaper to run, and the enforcement constraint makes it harder to forget. Both are good news for a small platform team that has been maintaining three copies of an agent across a growing fleet.

The open question is what the platform team gives up in exchange for the managed lifecycle: a vendor-owned update channel on every node, and a detection engine that lives outside the cluster's own release process. That trade is reasonable for most estates, and it deserves an explicit answer taken on purpose. This level of hardening belongs in security audit and hardening work, and the credential-handling argument connects to the earlier analysis of hardening a Fortinet estate with AI assistance.

Originally published on the Netics blog.

Oracle PeopleSoft Exploitation: How One Encoded Character Bypassed a WAF Rule

Taha Echakiri — Sat, 31 Oct 2026 14:00:07 +0000

TL;DR

On September 25, 2026, Mandiant and Google Threat Intelligence Group (GTIG) reported renewed mass exploitation of CVE-2026-35273 by UNC6240 (ShinyHunters), following the June 2026 campaign that used the flaw as a zero-day against education.
The bypass is one URL-encoded character: the actor requests /%50SEMHUB/ instead of /PSEMHUB/, because many WAF and proxy rules match the literal path before URL decoding.
Web shells landed on dozens of systems globally — higher education, technology, IT services, healthcare, agriculture, transportation, and government.
A typical probe is five to 15 POST requests carrying a serialized Java object; unpatched servers answer with the host operating system and the check stays silent.
This is a renewal, not a first sighting: the June wave hit education as a zero-day, and Oracle patched on June 10, 2026.
The post-exploitation chain includes JSP shells, fileless command execution, SIDEEYE via Ple64.exe, Neo-reGeorg tunnels, and MeshAgent persistence; hunt the whole path, not just files.
Patch first: WAF rules and path-based blocking are not a substitute for patching. Hunt encoded variations and unexpected .jsp files; rotate credentials as if the host were already compromised.

A renewed campaign aimed at every sector

On September 25, 2026, Mandiant and Google Threat Intelligence Group published the follow-up report, ShinyHunters Renewed Mass Exploitation Campaign Targeting Oracle PeopleSoft, on the Google Cloud blog. CVE-2026-35273, the Java deserialization flaw in Oracle's Environment Management Hub (PSEMHUB), is being mass-exploited again by UNC6240 (ShinyHunters). What changed since June is not the vulnerability: the actor modified its exploit to bypass the web application firewall (WAF) rules deployed against the endpoint.

The mechanism is quietly brutal. The actor requests /%50SEMHUB/ in place of /PSEMHUB/, and the report explains why that defeats a rule: "Many WAF and reverse proxy rules match the literal path before URL decoding, while the PeopleSoft application server decodes the request and routes it to the vulnerable servlet." A rule that matches a string can be bypassed by a request that is not the string and yet is the same path. Scope grew with technique: GTIG shows the actor "deploying web shells on dozens of systems globally, spanning higher education, technology, IT services, healthcare, agriculture, transportation, and government". The education-only focus of June is gone.

Official Google Cloud blog figure (Mandiant/GTIG, 2026-09-25): the /%50SEMHUB/ encoded-path request reaches the PSEMHUB servlet because the WAF rule matches the literal /PSEMHUB/ path before URL decoding.

A one-character encoding that defeats a literal-path rule

%50 is the URL-encoded form of the letter P. WebLogic decodes /%50SEMHUB/ back into /PSEMHUB/ and serves the application normally; a rule that matches the literal string /PSEMHUB/ before decoding never sees the match. GTIG's warning covers the whole class of variants: "Defenders should assume that threat actors may use any percent-encoded, mixed-case, or otherwise non-normalized variant of /PSEMHUB/, and should enforce blocking on the normalized path."

The uncomfortable part is how foreseeable this was. The June guidance already recommended blocking external access to /PSEMHUB/* at the perimeter while "noting that WAF body-inspection rules alone were insufficient." The September post states the consequence: "The current campaign demonstrates that UNC6240 adapted to published defensive guidance, targeting organizations that implemented WAF rules but did not patch the vulnerability." At each inspection boundary, normalize the path before matching it; otherwise the rule and the application may evaluate different representations of the same request.

GTIG's own quick guide is unambiguous: "WAF rules and path-based blocking are not a substitute for patching." That sentence is the story of this wave. A WAF rule is a mitigation with a lifespan: it expresses a string match at one point in the request pipeline, and the application server downstream may normalize the same request differently. Patching removes the component the request addresses — no encoding variant helps once the servlet is gone. This is the same controls-versus-conditions lesson we drew from the September GitLab mass exploitation of CVE-2026-85706: a rule that an attacker can normalize around is a condition, not a control.

Netics editorial diagram: the WAF rule blocks the literal /PSEMHUB/ path while the encoded /%50SEMHUB/ request passes; WebLogic decodes and routes to the same servlet. A mitigation, not a control.

The probe that confirms quietly before anyone owns the host

Before exploitation, the pattern is consistent. GTIG observed that "targeted servers typically received five to 15 POST requests to /%50SEMHUB/hub containing a serialized Java object." Unpatched servers respond "without writing files or disrupting the service, allowing the threat actor to quietly confirm exploitability." On hosts validated but not yet exploited, "organizations may see this request in logs, with no follow-on activity." The asymmetry is the operational point: the probe is short, silent, and leaves no file behind — it is visible only to a team that actually reads access logs.

Two exploitation methods follow, both abusing Java deserialization in the PSEMHUB hub servlet. The first is web shell deployment: a burst of POSTs to /%50SEMHUB/hub followed by JSP files such as x.jsp in PSEMHUB.war, repeated so every node behind a load balancer receives a copy. The second is fileless command execution: POSTs that return command output directly in the HTTP response, with nothing written to disk. "Detections that rely on JSP file creation will not identify this method", the report warns — the hunt has to look for shell processes (cmd.exe or /bin/sh) spawned by the WebLogic Java process, not just new files.

Official Google Cloud blog figure from the June 2026 companion post "ShinyHunters Targets Education Sector with Oracle PeopleSoft Exploit": the ShinyHunters extortion notice that set the tone for the campaign's ransom stage.

What the shell opens once it lands

Post-exploitation stops being PeopleSoft-specific. The primary shell x.jsp takes hex-encoded commands via POST and returns output prefixed with R:. A second servlet, u.jsp, decodes Base64 chunks in 150 KB increments to stage larger binaries while bypassing HTTP request-size limits. On Windows hosts the actor uploaded Ple64.exe, a 5.2 MB binary masquerading as a signed Light Alloy media player installer but carrying a three-stage chain that loads the SIDEEYE backdoor in memory; the sample was signed with a valid EV certificate issued to Tobias Weihmann Software Development OU via Sectigo. Alongside it came the Neo-reGeorg tunneling servlets tunnel.jsp and tunnel.jspx, routing SOCKS5 proxy traffic through ordinary HTTP for internal discovery and lateral movement.

On Linux, persistence runs through MeshAgent, a legitimate remote management tool: binaries and configuration land in /tmp (meshagent, meshagent.msh, meshagent.db) under the PeopleSoft service account, with outbound connections to Microsoft-masquerading domains including azurenetfiles.net. MITRE's mapping is familiar — T1190 (exploit public-facing application), T1505.003 (web shell), T1048 (exfiltration over alternative protocol). The consequence is blunt: a quarter of observed commands executed as root or NT Authority\SYSTEM. Where a shell is found, GTIG is unambiguous: organizations should "treat the host as compromised, preserve evidence, and rotate all credentials accessible from the PeopleSoft tier".

Netics editorial diagram of the UNC6240 cycle: probe POSTs to /%50SEMHUB/hub, silent exploitability confirmation, dual web shells (x.jsp, u.jsp), tunneling, and MeshAgent persistence.

Patch, disable EMHub, rotate credentials, watch the egress

The remediation guide is short. Apply the Oracle Security Alert patch for CVE-2026-35273. Disable EMHub in multi-server configurations, or remove the PSEMHUB application entirely in single-server ones; EMHub and the Integration Broker listening connector are administrative, system-to-system components, and restricting them from the public internet does not break standard PeopleSoft Internet Architecture sessions. Search PIA WebLogic access logs for /PSEMHUB/ and any percent-encoded variant, POSTs to /hub from external sources, and unexpected .jsp or .jspx files. Check every node, not only the first identified, because the burst exists precisely to replicate shells across load-balanced nodes. Inspect PSEMHUB.war for non-shipped files, rotate credentials readable by the application service account — psappsrv.cfg connection strings, Integration Broker credentials, cloud credentials — and monitor outbound traffic. "UNC6240 has a well-established pattern of data theft extortion," so prepare for extortion communications.

A renewal, not a first sighting

Honesty requires the counterpoints, because they are present in the source. This is a renewal, not a first sighting: the June wave exploited the CVE as a zero-day between May 27 and June 9, 2026, predominantly against higher education, and Oracle shipped an out-of-band Security Alert on June 10, 2026. Organizations that followed the June guidance and disabled EMHub have no servlet left to probe this September — the campaign primarily bites teams that chose a WAF rule over the patch.

Mass-exploitation economics are why the response has to be mechanical: a low-noise probe, a confirmation that writes nothing, a persistence toolkit that survives the web tier. The patch is the only control that closes the CVE; logs, endpoint checks, and egress review are the detection layer, valuable only while the window is still open. Check your access logs for /%50SEMHUB/ tonight, whatever your WAF vendor promised. New threat-intelligence breakdowns land on neticslabs.com.

Originally published on the Netics blog.

Kubernetes Starts Writing a Conformance Standard for Workloads

Taha Echakiri — Sat, 31 Oct 2026 14:00:01 +0000

TL;DR

Workload conformance is a proposed shared standard for how an application behaves while it runs on Kubernetes, covering things like tolerating a node drain or surviving an upgrade.
The Kubernetes community announced a Workload Conformance Working Group on 27 October 2026, sponsored by two Special Interest Groups.
Certified Kubernetes tests a cluster and Kubernetes AI Conformance tests whether a cluster can run AI workloads, so operators still assess each application by hand before maintenance.
The group plans three deliverables: a specification per Kubernetes minor version, a method for testing each requirement against a running workload, and upstream conformance tooling.
Every requirement needs an objective test that a test binary can run, and the specification focuses on workload runtime behaviour.
The target is a stable first specification and working tests within two to three Kubernetes release cycles.
The group is asking operators for exactly the evidence most teams already have: what broke during the last drain or upgrade.

Page capture (retrieved 27 October 2026, kubernetes.dev/blog/announcing-workload-conformance-wg): the section that sets out the three deliverables the working group's charter commits to.

Certified Kubernetes tests the cluster. Operators still check each application by hand before routine maintenance or a Kubernetes upgrade because workloads currently lack a shared conformance standard. The new working group exists to close that gap by writing requirements that a program can check.

Two Existing Programs Test the Cluster and Its AI Readiness

Kubernetes already runs two conformance tracks. Certified Kubernetes tests the cluster. Kubernetes AI Conformance tests whether a cluster can run AI and ML workloads.

Workload behaviour sits between those programs and application-level assessment, and that is the layer the new group targets.

Netics editorial visual: the path the charter lays out, from a scope list to a test binary that runs upstream.

The Working Group Writes a Specification Per Minor Version

The charter sets three deliverables. A specification of workload behavior for each Kubernetes minor version. A method for testing each requirement against a running workload. Upstream conformance tooling that implements those tests.

The scope list is broad and specific at the same time: security, resilience during disruptions, resource use, networking, storage, API usage and observability. The group is still deciding the specific requirements and the methods for testing them.

One rule is already fixed, and it is the one that decides whether this works. Every requirement needs an objective test that a test binary can run, so nothing subjective goes into the specification.

Every Requirement Needs a Test a Binary Can Run

That rule has a consequence worth naming for anyone who has written a platform standard. A requirement concerning a workload's ability to tolerate disruption during a node drain enters the document only once someone can express the check as an executable test.

Netics editorial visual: the objective-test rule the charter fixes, and the boundary that keeps application logic outside the specification.

Subjective quality statements drop out at the door. The planned binary focuses on workload-conformance requirements; packaging formats, policy engines and security scanners remain implementation choices.

Three Deliverables, Two Sponsor SIGs

Two special interest groups sponsor the work. The announcement describes the working group as a forum where Kubernetes SIGs can shape the specification. The group seeks input from both workload maintainers and cluster operators as it writes the specification.

Where a requirement touches another SIG's area, node behaviour or networking for example, the working group opens the conversation with that SIG. The group describes itself as a forum for SIGs to shape the direction while the specification is still being written.

Drain, Upgrade and Privilege Failures Are What Operators Report

The announcement names the failures the group expects to formalise and presents them as familiar operational problems.

Page capture (retrieved 27 October 2026, kubernetes.dev/blog/announcing-workload-conformance-wg): how the work is organised between SIG Architecture, SIG Apps and the working group.

A routine node drain shows which application cannot tolerate disruption. An upgrade breaks an application that still depends on a removed API version. Workloads ask for privileges they do not need, and reserve resources they never use.

Kubernetes documents much of this, including security guidance and handling disruptions. The waste is repetition: every cluster operator runs the same assessment, and application maintainers answer the same questions from every environment they support.

How to Contribute an Example, a Review or a Test

Here is the practical part. The specification will be built from real examples, and the group asks operators two direct questions: how do you assess workloads today, and what broke during your last upgrade or maintenance window?

Answer both. Those two answers are the evidence a testable requirement needs, and they cost nothing to write down if the team already keeps a maintenance log.

Maintainers are asked what the application needs from the cluster, and where the existing guidance left them guessing.
Reviewers could strengthen proposed requirements, while contributors could help write tests and tooling as the work develops.
Early testers should run the binary against their own workloads and report results that differ between runs or environments.
Long-term maintainers should be recruited early to give the specification and tooling durable ownership.

Netics editorial visual: the assessment every operator repeats today, set against the same requirements written down and testable.

The group meets every other Wednesday at 15:00 UTC and works in #wg-workload-conformance on Kubernetes Slack, with an open mailing list.

A Target of Two to Three Release Cycles

Read the timeline carefully before planning anything around it. The group aims for a stable first specification and working tests within two to three Kubernetes release cycles, and that is a planning target. The announcement presents the specification and test binary as planned deliverables.

For current maintenance windows, teams still assess each application by hand, and the group asks them to record what breaks during an upgrade or maintenance window.

For an infrastructure team that already writes runbooks, this is a chance to make one of them reusable. Describe the drain behaviour of your service in the vocabulary the group is asking for: what happens to in-flight requests, how long termination takes, what the pod disruption budget allows. The same description belongs in your own sovereign infrastructure and cloud exit programme, where a migration window is a drain with higher stakes.

Netics editorial visual: the four kinds of contribution the announcement asks for, from a maintenance log to a test binary run.

Upgrade planning is covered in a related Netics article, Kubernetes 1.34 Stops Receiving Patches on 27 October 2026.

One thing this standard will test is its own authors. A per-minor-version specification can fragment results across version pairs, while timing-sensitive drain tests can fluctuate across environments and a binary pass can create false confidence about application logic outside the scope. That narrowness is the reason a binary can check it, and the reason each result carries the date and the environment it came from.

Originally published on the Netics blog.

Vertiv Sells a Rack-Scale AI Package Validated Before It Ships

Taha Echakiri — Sat, 31 Oct 2026 09:00:03 +0000

TL;DR

Vertiv SmartIT is a pre-engineered rack package for high-density AI deployments, and it bundles rack infrastructure, DC power conversion and distribution, direct-to-chip liquid cooling, in-rack cooling distribution and monitoring into one factory-validated design.
Vertiv announced it on 26 October 2026 and says it is available now, aimed at enterprises, research institutions and colocation providers that run AI hardware in existing buildings as well as new ones.
Factory validation moves the assembly work out of the data hall: Vertiv states that commissioning narrows to the facility connections, and that brownfield enterprise space and Class A commercial buildings can carry AI-class density without full reconstruction.
The package lists four NVIDIA systems, GB300 NVL72, Vera Rubin NVL72, DGX Rubin NVL8 and HGX Rubin NVL8, on the MGX and OCP ORV3 reference architectures, which sets the density a buyer can promise before any site survey.
The facility end stays with the building: the power feed, the cooling loop and heat rejection, floor loading, and the network uplinks are the buyer's scope, and a single rack is the smallest unit Vertiv describes while a full pod is the largest.
Nothing in the announcement gives a measured power envelope, a cooling capacity or an independent cost comparison, so the useful reading is a scoping document for what to ask before ordering.

Official Vertiv product photograph (retrieved 26 October 2026, vertiv.com/en-us/products-catalog/facilities-enclosures-and-racks/integrated-solutions/vertiv-smartit): the SmartIT rack from the front, with the compute and network bays populated.

Vertiv builds the parts that keep a data hall alive: power distribution, cooling, racks. Its new product combines engineering with a packaging decision, and that packaging is what a buyer of AI hardware actually faces. Buying accelerators has become routine, and making the room around them work remains difficult. SmartIT is Vertiv's answer to that second problem, sold as one item.

The rack ships as one validated package

The press release describes a single order covering rack infrastructure, DC power conversion and distribution, direct-to-chip liquid cooling, in-rack cooling distribution and monitoring, integrated and validated before shipment. The product page repeats the framing and adds the operational detail that matters to whoever will run it: monitoring support for Redfish, SNMP and Modbus, proactive leak detection, automated shunt-trip, and a one-year limited warranty under Vertiv's standard terms.

Netics editorial visual: the commercial shape of the offer, one order and one support contact across rack, power, cooling and monitoring, available now according to Vertiv.

Scott Armul, Vertiv's chief product and technology officer, frames the product around coordination: AI is changing what data center infrastructure has to do, and the answer is bringing power, cooling and IT together in one validated architecture. That sentence names the actual problem. In a conventional build, the rack vendor, the power vendor and the cooling vendor each validate their own box, and the seams between them get validated by whoever is on site with a clipboard.

The list of supported systems does more work than a marketing paragraph would. GB300 NVL72, Vera Rubin NVL72, DGX Rubin NVL8 and HGX Rubin NVL8 are named on NVIDIA's MGX and OCP's ORV3 reference architectures, and Vertiv says it already deployed the platform with NVIDIA to support the DGX GB300, and showed a unit alongside an HGX Rubin NVL8 demonstration system at NVIDIA GTC Berlin in October. A buyer reading that list learns which generations the vendor expects to sit inside this enclosure, which confirms compatibility and leaves capacity unspecified.

Netics editorial visual: the integration order before and after factory validation, with on-site assembly replaced by factory build and site work limited to the facility connections.

What factory validation leaves with the facility team

Factory validation transfers a precisely defined part of the work away from the site. The release is explicit that the commissioning scope narrows to the facility connections. Everything upstream of those connections is now a building problem, and buildings vary more than racks do.

Direct-to-chip liquid cooling makes that split sharper than air cooling did. A rack that expects a coolant loop needs a supply temperature, a flow rate, a pressure budget, a fluid specification and a path for the heat to leave the building. Vertiv's package carries the in-rack distribution and the leak detection that shuts the flow; the heat rejection plant, the pipework to the row and the floor's structural rating sit outside the order.

Official Vertiv product photograph (retrieved 26 October 2026, vertiv.com/en-us/products-catalog/facilities-enclosures-and-racks/integrated-solutions/vertiv-smartit): the rear of the rack, showing cable entry and the rear panel layout that the facility connections land on.

Vertiv claims a specific benefit here that is worth separating from the rest: brownfield enterprise facilities and Class A commercial buildings can support AI-class density without full reconstruction. That claim covers the enclosure and the cooling it brings with it. The building's available electrical capacity remains unspecified, a figure that is frequently the binding constraint in an office block designed for desks.

The supported systems set the density you can promise

Four systems across two reference architectures is a wide compatibility statement, and compatibility is where a procurement conversation usually starts. Facility planning requires per-rack power and heat figures, both unspecified in the release.

A vendor listing four accelerator systems in one enclosure is telling you the enclosure, the power shelves and the coolant distribution are sized for a family of loads. How wide that band runs, and which configuration sits at the top of it, is the number a facility engineer needs before signing a lease or ordering a feed upgrade. Vertiv's own product page states the deployment range as a single rack to a full pod and leaves per-rack power and heat unspecified.

Where a pre-engineered rack fits in a small estate

Smaller organisations rarely buy one of anything. A first AI deployment often lands in a room that already does something else, which is the case this product is pointed at: the same building block from a single rack to multi-rack, without a redesign between phases.

Page capture (26 October 2026, prnewswire.com/news-releases/vertiv-simplifies-high-density-ai-infrastructure-deployment-across-data-center-environments-302919919.html): the announcement as distributed, with the product image Vertiv supplied.

Planning a phased build around a validated block changes the internal conversation. The capacity decision splits into two tracks that can run in parallel: the room, which the facilities team owns, and the compute, which the platform team owns. Teams that keep the AI capacity inside their own walls usually treat that as a sovereign infrastructure and cloud exit question first, and the earlier piece on DGX Spark at 64GB: What $4,999 of Local Compute Buys a Small Team covered the other end of the same decision.

Netics editorial visual: the four items the building still provides after the package is validated, from the power feed to the network uplinks.

Vertiv sells adjacent pieces for that row: SmartRun for integration across the row, rack PDUs for distribution and metering, and Unify for monitoring and controls across power and thermal infrastructure. A buyer who takes the rack package and stops there inherits a monitoring boundary: the enclosure reports on itself, leaving the row without monitoring coverage. That boundary is worth naming in the purchase order.

What the announcement leaves unproven

Three things are missing, and their absence is normal for a launch release.

There is no measured power envelope per configuration, no coolant flow or heat rejection figure, and no independent test of the factory-validation claim. The one-year limited warranty is a short window for capital equipment built to stay in place, and its terms are Vertiv's standard terms, which a buyer should read before comparing prices.

Official Vertiv product photograph (retrieved 26 October 2026, vertiv.com/en-us/products-catalog/facilities-enclosures-and-racks/integrated-solutions/vertiv-smartit): the rear and side of the rack, where power distribution and the cooling connections meet the row.

There is also no published price, which turns the comparison into a scoping exercise with each vendor's sales team. The honest summary of SmartIT is a packaging claim with a concrete bill of materials attached: rack, DC power, coolant distribution, monitoring and leak response, validated as one item. Facility readiness remains the buyer's responsibility. That is a useful thing to be able to order. The teams that get value from it are the ones who already know what their building can supply.

Originally published on the Netics blog.

CUDA 13.4 Turns a GPU Fraction Into an Enforceable Memory Limit

Taha Echakiri — Fri, 30 Oct 2026 19:00:03 +0000

TL;DR

GPU fractioning is the practice of running several workloads on one physical GPU, each with its own share of memory and compute, so a small job stops consuming a whole device.
NVIDIA released CUDA 13.4 on 27 October 2026 with a memory boundary the application cannot see, plus an open-source GPU Fractioning Operator that applies per-container shares on every node.
The scheduler writes the contract into four Pod annotations, and any Kubernetes scheduler that can write those annotations can enforce GPU fractions with CUDA.
KAI Scheduler v0.18.0 is the first integration, through a GPU-sharing mode called NvFractions that is enabled with one Helm flag and installs the operator as a dependency.
Time-slicing gives each workload its own error domain and about 1/n of the time; SM-sharing runs everything at once on split streaming multiprocessors inside one shared error domain.
The guaranteed memory request and optional burst limit are the two numbers to verify before a team enables this on a shared cluster.
Memory enforcement settles the accounting question and leaves bandwidth, cache and failure domains exactly where they were.

Figure 1 from the NVIDIA technical blog (retrieved 27 October 2026, developer.nvidia.com/blog/memory-enforced-gpu-sharing-in-kubernetes-with-nvidia-cuda-13-4): time-slicing alternates workloads W1 and W2 along a time axis, while SM-sharing places workloads A, B and C on different streaming multiprocessors at the same moment.

Every platform team with a GPU node knows the arithmetic problem. As an illustration, a batch job that needs 8 GB of memory can hold a device with 80 GB while the other 72 GB sit idle because the pod asked for one GPU and the scheduler granted one GPU. NVIDIA's answer in CUDA 13.4 is to make the fraction real at the level where it can be enforced: the node.

CUDA 13.4 Makes the Memory Boundary Transparent to the Application

NVIDIA's GPUs already offered three ways to share a device. Multi-instance GPU splits a GPU into hardware-isolated instances that are transparent to the application. Time-slicing lets several processes take turns and enforces no memory limit. Multi-Process Service lets them run concurrently in separate address spaces, which NVIDIA calls sm-sharing.

Official NVIDIA artwork published with the 27 October 2026 post: the CUDA memory enforcement feature that the GPU Fractioning Operator configures on each node.

CUDA 13.4 adds enforcement on top of that. The release enforces a memory boundary that is transparent to the application, in a new time-sharing mode, alongside the compute sharing mode that already existed in MPS. Four behaviours matter:

An exact memory amount. A workload asks for a memory size and CUDA holds it there. An allocation above that size fails, exactly as it would on a smaller GPU.
A range. The workload gets a guaranteed minimum, plus a maximum it can grow into when memory is free.
A compute mode. Each workload selects its compute mode; time-slicing is the default, and sm-sharing is also available.
A transparent limit. Each workload sees only a fraction of the GPU memory.

The failure mode is the interesting part. An allocation above the enforced amount fails, just as it would on a smaller GPU.

The Operator Runs MPS on Every Node and Applies What the Scheduler Wrote

CUDA provides the enforcement capability, and a Kubernetes cluster cannot reach it on its own. On every GPU node, something has to run and supervise the daemons these features depend on, apply each Pod container's memory limits and compute mode before the process starts, and report what each fraction actually uses. NVIDIA puts that work on the node, and says so directly: "This work belongs on the node, independent of whichever scheduler placed the workload."

Page capture of the NVIDIA post (retrieved 27 October 2026, developer.nvidia.com/blog/memory-enforced-gpu-sharing-in-kubernetes-with-nvidia-cuda-13-4) showing the three jobs the operator performs on every GPU node.

The result is gpu-fractioning, an open-source Kubernetes operator hosted under the KAI Scheduler organisation, running next to the NVIDIA GPU Operator that supplies the driver and the container runtime setup. It has three jobs: set up and run MPS on every GPU node, apply each workload's memory limits and compute mode, and measure memory and compute use per fraction.

That split is the architectural decision worth studying. The scheduler chooses placement. The node enforces the limit. Two different programs, with a written contract between them.

Four Annotations Carry the Memory Contract

The contract is small enough to read in one screen. A scheduler records a workload's allocation and compute mode in per-container Pod annotations, and the operator reads them and configures the assigned GPUs.

Annotation	Meaning
`nvidia.com/container..gpu-memory.request`	Guaranteed GPU memory
`nvidia.com/container..gpu-memory.limit`	Maximum the container may grow into
`nvidia.com/container..gpu-compute.mode`	`time-slicing` (default) or `sm-sharing`
`nvidia.com/container..gpus.devices`	Which GPUs the container was assigned

Netics editorial visual: the two compute modes the operator applies, from the time each workload receives to the error domain it shares.

NVIDIA states the general rule that follows: any Kubernetes scheduler that can write these annotations can enforce GPU fractions with CUDA. The scheduler keeps its own obligations, which are choosing a GPU with enough free memory and never mixing compute modes on one GPU.

Time-Slicing and SM-Sharing Divide Compute Two Ways

Compute can be divided as well, and call n the number of workloads sharing a GPU. Either way each workload gets roughly 1/n of the compute. The two modes schedule that share differently: time-slicing takes turns, SM-sharing runs workloads together.

Time-slicing is serial. Workloads take turns on the whole GPU, each gets about 1/n of the time when all of them use their full slice, and unused time is shared equally between the others. This is the default, and each workload runs in its own error domain, which is full error isolation.

SM-sharing is parallel. The streaming multiprocessors are split between workloads, which run concurrently while the hardware allocates and releases SMs as needed. When every workload is busy the split settles at roughly 1/n, and a single active workload can use them all. In this mode, workloads share a single error domain.

Netics editorial visual: the four annotations that carry a fraction from the scheduler to the node, with what each one asks for.

Read that last sentence twice before enabling anything. SM-sharing gives better utilisation and narrower blast protection. NVIDIA's own guidance is to use it for workloads you trust to run together. Each container declares a compute mode; KAI Scheduler tracks the single mode active on each shared GPU, and different GPUs on one node can use different modes.

A 40Gi Request on Four GPUs, and Half a GPU for a Notebook

NVIDIA ships two examples and they cover the two shapes most teams need.

The first is an inference server with a guaranteed 40Gi on each of four GPUs, a 60Gi limit it can grow into when neighbours are idle, sm-sharing, and gpu-fraction-num-devices: "4". The request applies to each device, so 40Gi means 40Gi on every one of the four.

The second is a development or batch job that asks for gpu-fraction: "0.5", a limit of gpu-fraction.limit: "0.8", time-slicing, and two devices. Half of each GPU guaranteed, growth to 80% when memory is free, and each job in its own error domain so a crash stays local.

Netics editorial visual: what the CUDA 13.4 memory boundary does at the allocation, with the guaranteed minimum and the burstable maximum.

One detail in the second example deserves attention. The limit is optional, and when it is omitted the limit equals the request, so the workload is capped at exactly what it asked for. A portion limit must be greater than the gpu-fraction and less than 1. Those constraints are what stops a team from writing a limit that silently re-creates the whole-device behaviour it was trying to remove.

What to Verify Before You Enable NvFractions

Run through this list on one node before it reaches a shared cluster.

Confirm the NVIDIA GPU Operator is present, since the fractioning operator relies on it for the driver and the container runtime setup.
Decide whether each workload needs an absolute size or a portion, because the limit annotation differs between the two request types.
Write the burst limits on purpose. A generous limit is a performance decision with a neighbour cost attached.
Monitor memory use for every fraction as its own accounting unit.
Test whether nvidia-smi inside the container reports the fraction you expect, which is the sanity check NVIDIA uses in its own example.

Page capture of the NVIDIA post (retrieved 27 October 2026, developer.nvidia.com/blog/memory-enforced-gpu-sharing-in-kubernetes-with-nvidia-cuda-13-4) showing the annotation table and the Helm flag that enables NvFractions.

Memory Enforcement Leaves Bandwidth and Failure Domains Alone

A fraction of memory is a fraction of one number. The bandwidth of the device, the L2 cache and the computational throughput remain shared, and time-slicing still serialises execution. A 40Gi memory allocation defines capacity; benchmark the workload separately to measure the performance of that share on the target GPU.

Hardware isolation remains the MIG story. MIG splits a GPU into instances that are isolated in hardware, and CUDA 13.4 adds enforcement to the sharing modes that sit below that line. Choosing between them is a question about tenants: a shared cluster of internal teams can live with a shared error domain, while a customer-facing one may want the hardware boundary and accept the coarser split.

The operational value sits in metering. Once a fraction is a declared, enforced quantity, a platform team can bill an internal team for 40Gi of a GPU the way it already bills for memory in a virtual machine. That is the change worth planning for, and it lands next to the broader work of building a sovereign infrastructure and cloud exit plan, where every shared resource needs an owner and a number.

Netics editorial visual: the checks that belong on one node before a fractional policy reaches a shared cluster.

NVIDIA states that any Kubernetes scheduler that can write the annotations can enforce GPU fractions with CUDA.

Two things are still open. NVIDIA documents the operator against KAI Scheduler and states that any scheduler able to write the annotations can use it, and it does not publish the per-fraction overhead of running MPS on every node. The second one is measurable on your own hardware before a rollout, which is the honest way to decide whether the density gain pays for the supervision.

Originally published on the Netics blog.

Valkey 9.1 Turns the Redis Licensing Shock Into a Real Upgrade Path

Taha Echakiri — Fri, 30 Oct 2026 14:00:34 +0000

TL;DR

Google Cloud's page marks Valkey 9.1 GA; its structured datePublished is 2026-09-25 while the live page displays September 26, 2026.
The mechanism: lock-free SPMC, MPSC, and SPSC queues replace round-robin socket assignment and list polling, plus dynamic scaling (30% CPU ignition, queue-depth auto-scaling).
The governance story: Redis Inc.'s 2024 license shift pushed Google Cloud and other leaders to back Valkey under the Linux Foundation; two years later the fork is a managed upgrade path.
New capabilities: database-level ACLs for environment isolation, CLUSTERSCAN, and new atomicity commands (HGETDEL, MSETEX, HSETEX).
Read "up to 3x" as a vendor-chosen, benchmark-based comparison.
Before migrating: audit command coverage, ACL configuration, and latency targets against your own workloads.

Google Cloud's live page displays September 26, 2026; its structured datePublished field says 2026-09-25. The post announces general availability of Memorystore for Valkey 9.1, with a headline claim of "up to 3x queries per second (QPS) at microsecond latency compared to Memorystore for Redis Cluster." Two stories sit under that sentence — how a cache gets three times faster, and the 2024 licensing shift that created the project. Teams choosing a cache layer today need both.

The 2024 license shock is the reason Valkey 9.1 exists

Google's own timeline starts in 2024, when Redis Inc. shifted its licensing away from the permissive open-source BSD license to a dual-license model. Google Cloud and other technology leaders responded by backing Valkey, an open-source alternative governed by the Linux Foundation. Procurement teams should sit with that sequence: the project exists because a license changed, not because a benchmark demanded it. Two years on, the fork is a managed service with a faster engine than the original — a viable exit after a license shift, though a young one: Valkey is about two years old, a maturing path rather than a fresh one. Google's positioning is nonetheless confident: "Valkey is particularly compelling for organizations scaling AI and microservices to handle millions of concurrent users."

The claim is backed by shipped controls, most usefully database-level access control. ACL rules previously applied globally across an instance; Valkey 9.1 restricts user access at the specific numeric database level. Google's example is a three-environment cluster — a production user scoped to db=0, a staging user to db=1, a dev user to db=2 — isolating workloads without key prefixes and at no additional cost, environment separation without running three clusters.

Official Google Cloud blog figure: database-level ACL isolation — access restricted to numeric databases (db=0, db=1, db=2), separating production, staging, and dev keyspaces in one cluster.

Operationally, that separation depends on both sides of the connection being right: the client must select the intended numeric database and the ACL must restrict its user to that database. Verify db=0/1/2 mappings in each connection pool, then test a denied cross-environment access before treating the arrangement as isolation. This is a configuration boundary, not a substitute for application authorization or separate failure domains.

The mechanism behind the 3x: queues instead of polling

Google describes the old design plainly: previously, Valkey assigned client sockets to I/O threads statically in a round-robin fashion, requiring the main thread to continuously poll lists of pending clients. Busy-wait list polling is the cross-thread CPU waste that shows up at high QPS. Valkey 9.1 replaces list polling with a lock-free, multi-queue messaging architecture: the main thread dispatches read and write jobs into a single-producer multi-consumer (SPMC) queue, where free workers pull tasks on demand (work-stealing, no starvation); completed work returns through a multi-producer single-consumer (MPSC) queue popped instantly, eliminating busy-wait list iteration; single-producer single-consumer (SPSC) queues handle thread-affine memory cleanup and high-volume epoll offloading. Scaling is dynamic too, through what Google calls an ignition phase: when main-thread CPU usage crosses 30%, the engine activates the first background I/O thread before queue bottlenecks form, and queue-depth auto-scaling moves the worker count on real-time SPMC backlog, "ensuring extra cores are used only when needed and parked when idle." That mechanism is the argument for 3x.

Official Google Cloud blog figure: the lock-free multi-queue design — SPMC work queues to I/O threads, an MPSC completion queue back, SPSC queues per thread for memory cleanup and epoll offloading.

That is the mechanism Google connects to the performance claim, not a universal result for every service workload. Validate with your own request mix, connection count, and latency percentiles under production-like concurrency.

Scaling, the honest reading, and a scan that survives failovers

Scanning keys across a large cluster used to mean querying nodes individually — an approach that was neither cluster- nor failover-aware, so scans could miss keys, return duplicates, or fail if slot migrations or node failovers occurred mid-scan. CLUSTERSCAN fixes this with a topology-aware cursor that encodes the current slot, a fingerprint of the local hashtable, and the local cursor. Two strategies: a sequential full-cluster scan for simple scripts, and a parallelized mode where the SLOT argument partitions the cluster's 16,384 slots across multiple workers.

That creates a useful acceptance test beyond the vendor benchmark: run sequential and SLOT-parallel scans against the same keyspace, compare returned key sets, and repeat during a slot migration or node failover. Record duplicate, missing, and cursor-resume outcomes separately. A demo proves the command runs; reconciliation under topology change is what tells a team whether its own scan consumer is safe.

Official Google Cloud blog figure: parallel CLUSTERSCAN workers partitioning the 16,384 slots — sequential scans for simple jobs, SLOT-scoped parallel scans for high throughput.

Now the inconvenient part. The 3x figure is "up to 3x," measured against Memorystore for Redis Cluster — the comparison the vendor chooses against its own previous service. The source carries the caveat itself: the sizing figures are based on open-source benchmarks, with real gains varying by workload. Customer traction is vendor-stated: Google says over 95% of the top 100 Google Cloud customers already rely on Memorystore for demanding, high-throughput workloads.

What else ships in 9.1: commands, sizes, and the 9.0 base

HGETDEL reads a hash field and deletes it atomically in a single network round-trip — the pattern behind single-use authentication tokens and short-lived session state. MSETEX sets multiple keys with a single, shared expiration time, removing multi-command pipeline overhead. HSETEX gains NX and XX conditional flags for writes that must not clobber an existing value. Valkey 9.1 also builds on 9.0 from Google Cloud Next '26: native JSON support and Bloom filters for AI and vector workloads, plus six new node sizes — Custom-Pico (1.25 GB), Custom-Micro (2.5 GB), and Custom-Mini (3.5 GB), available only for cluster mode disabled environments, up to Highmem-XXLarge with 110 GB RAM and 16 vCPUs per node.

What to check before moving a Redis Cluster workload to Valkey 9.1

Migration is now a supported, generally available workflow: provision the target instance with your shard count and node sizing, establish continuous dual-sync online replication, validate data synchronization against replication metrics, then execute the cutover by switching application connection endpoints. "This workflow is generally available." The checks are the usual managed-Redis list, with a licensing twist. Audit command coverage: every Redis command your application uses must exist in the Valkey command set, kept in full in the docs. Decide the ACL configuration before cutover so environment separation is in place on day one. Set latency targets from your own workload, not the 3x headline — the same latency-economics argument we made when GPT-6 prompt caching launched.

Take a concrete case — hypothetical, but the shape is common. A European e-commerce SME runs checkout session state on self-managed Redis OSS and pays an engineer to watch failovers; the dual-license model made its procurement team uneasy, and a managed Valkey path removes both the license question and the operational load, with DB-level ACLs separating staging from production inside one cluster. That journey is as much a procurement decision as a performance one.

None of this argues against migrating — it argues for checking first. The next time a vendor quotes 3x, look for the mechanism before the chart. For Valkey 9.1 it is checkable: lock-free queues, a 30% ignition threshold, ACL scoping per database — a release a team can evaluate against its own workloads, which is exactly the position a license shift should put you in. Teams making that call can read how Netics works on cache-layer performance and open-source governance decisions.

Netics editorial diagram: the before/after — round-robin socket assignment with list polling versus the lock-free SPMC/MPSC/SPSC queue design and dynamic I/O worker scaling.

Originally published on the Netics blog.

Cloudflare Reclaims 100 TB of RAM by Rebuilding Its Hash Ring

Taha Echakiri — Fri, 30 Oct 2026 14:00:03 +0000

TL;DR

Consistent hashing is the technique that spreads requests across a pool of servers so that adding or removing one server moves a small slice of the work; Cloudflare's open-source implementation is called pingora-ketama, and its Pingora Backend Router uses it to decide which server holds a cached file.
Each server carried 100,000 hash points, because a 160-point baseline is multiplied by a disk-space weight of 625, and supporting several feature combinations multiplied the rings again, which left some processes holding 6 GB of hash structures.
The predicted error margin starts near 99% with one hash per server and falls to about 8% at 160 points; each further step down costs almost ten times the hashes, and the last 90,000 points bought a 0.7% improvement.
Thirty-two-bit hashes also collide, and Cloudflare's simulations show error rising between 10,000 and 100,000 hashes per server for a 2,048-server data centre, which is what made a 90% cut safe.
A second change packed each hash point into six bytes, down from eight, which cut the memory used by consistent hashing by 25%.
The migration kept both rings in memory, chose a ring per request by hash and rolled out location by location, so the cache did not invalidate; the old path came out at 100% and the network reclaimed 100 TB of RAM.
A smaller operator can copy three habits from the work: measure the skew of the distribution, ship a routing change as a migration with both paths live, and inspect the size of the structure that holds the index.

Load balancers are the part of a platform that nobody measures until a bill or an out-of-memory kill arrives. Cloudflare measured, found that a single routing structure was holding gigabytes per process, and then did the arithmetic before touching the code. The post that describes the work is a useful template for anyone running a consistent-hashing layer of their own. Its central lesson is the value of measuring how much accuracy the system needs.

What the Pingora Backend Router does with a hash ring

Pingora Backend Router is Cloudflare's internal load-balancing service for cacheable requests. Its job is to send every request for the same URL to the same server, which is what allows one copy of a cached file per data centre and a stable way to find it again. Consistent hashing is the mechanism: server identities and request identities are hashed into a 32-bit space, and each request goes to the nearest server hash at or before its position.

The design pays off when the pool changes. Adding or removing a server moves a share of the keys and leaves the rest where they were, which is the property that keeps a cache useful during maintenance. The cost is statistical: with one hash point per server, the sizes of the ranges differ, and the share of traffic each server receives follows the range it owns. Cloudflare's engineers computed the expected value and the standard deviation for a pool of N servers, then divided one by the other to get a coefficient of variation of about 99% at 100 servers.

The head of Cloudflare's engineering post at blog.cloudflare.com/saving-100-tb-of-ram-with-math, retrieved 2026-10-25, which opens with the ticket that started the work: excessive memory usage from pingora-ketama in Pingora Backend Router.

Where the memory went: 100,000 hashes for every server

One hash per server produces a bad spread, so every implementation gives each server several. NGINX hardcodes a baseline of 160 hash points per server, and Pingora uses the same default. At 100 servers, that baseline yields a coefficient of variation of about 8%. Cloudflare needed one more thing: servers with more disk should receive proportionally more traffic.

The ketama scaling does the weighting. The number of hashes for a server is multiplied by a weight, and Cloudflare weights by disk space. A weighting factor of 625 on top of the 160 baseline produces 100,000 hash points per server. Then the feature combinations arrive: compliance requirements and caching features mean only some servers may answer a given request, so each combination needs its own ring, and a handful of features multiplies into dozens of rings. The result, as the post puts it, was memory use of 6 GB in some cases.

Netics editorial diagram of the four numbers that frame the change, each taken from Cloudflare's post: the network-wide saving, the cut in hash points per server, the saving from the storage format and the error reduction the final 90,000 points delivered.

The 25% saving that came from a six-byte struct

The storage change is the part a smaller team can copy without any probability work. The structure that held one hash point carried a 32-bit hash and a 32-bit server index, eight bytes in memory. The index only needs 16 bits, because the router will not coordinate more than about 65,000 servers, so an obvious edit is to declare the index as a 16-bit integer.

Rust alignment rules turn that edit into a no-op: a struct is padded to a multiple of its largest field, so four bytes of hash plus two bytes of index still occupies eight bytes. Cloudflare's answer stores both values in a six-byte array and exposes them through getters, which produces the same machine code as a packed struct without the usual objections to packing. The post measures the effect as a 25% reduction in the memory used for consistent hashing.

That saving attacks the constant factor in front of the count. Reducing the point count was the larger lever, and the curve below shows the price of each further step.

Cloudflare's own chart at blog.cloudflare.com/saving-100-tb-of-ram-with-math, retrieved 2026-10-25, plotting the error from prediction against the number of hashes per server for 100 servers; each step down in error costs roughly ten times the points.

Why 90% fewer hashes kept the same spread

Accuracy has a shape, and the shape is the argument for cutting hashes. Each step down in the error margin costs almost ten times the hash points, so the second hundred thousand points do far less work than the first. With a weighting factor of 625 the calculation lands at 100,000 points per server, and Cloudflare's own summary of that number is blunt: the last 90,000 hashes added were buying a minuscule 0.7% reduction in error.

Thirty-two-bit hashes bring a second limit. As the number of points grows, collisions grow with it, following the birthday paradox, and a collision randomly drops a contribution that was meant to balance the pool. Simulating 32-bit hashes against the prediction shows the error rate starting to rise between 10,000 and 100,000 hashes per server in a data centre with 2,048 servers. That combination, flat returns and rising collisions, is what let the team remove 90% of the points without a measurable loss in distribution.

Cloudflare's simulation chart at blog.cloudflare.com/saving-100-tb-of-ram-with-math, retrieved 2026-10-25, comparing 2,048-server pools of different sizes; with 32-bit hashes the collision-driven error begins to climb between 10,000 and 100,000 hashes per server.

Cutting over without invalidating the cache

A better ring is still a different ring. Switching the whole network at once would send requests to servers that do not hold the file, which turns a memory optimisation into a stampede towards the origins. Cloudflare treated the change as a migration and kept both rings alive in the router, with the existing migration framework deciding per request which ring selects the backend.

That design gave two things at once: the choice stayed stable for a given request hash, and the rollback path stayed open without a redeploy. The rollout then moved in layers, from small validation locations through progressively larger groups of data centres, with the share of traffic on the new ring and the set of locations allowed to use it controlled separately. Scoping the rollout by location confined cache churn to selected data centres and kept the blast radius small. During the change the team watched backend-selection traces, ring-version counters, connection errors, process memory, startup time, cache behaviour and origin traffic.

Cloudflare's memory chart at blog.cloudflare.com/saving-100-tb-of-ram-with-math, retrieved 2026-10-25, comparing PBR memory in the week of the change with a few weeks before; the sharp drop is the day the build with the large, unused hash rings was decommissioned, and the difference comes to 100 TB.

What a smaller operator can copy from this

Cloudflare's evidence covers PBR's 32-bit hashes and the server counts it tested. Netics therefore treats the 90% cut as a configuration-specific result. Operators should rerun the distribution and collision tests with their own server counts, weights, hash widths and eligibility rules before choosing a point count. Three habits travel well. Measure the distribution before tuning it: a coefficient of variation, or simply the busiest server against the average, tells you whether your ring is skewed and whether a change helped. Treat every routing change as a migration with two code paths, because the rollback is what makes the change safe to attempt. Check the size of the structures that hold the index, since padding inside a struct is invisible in the source and visible in the resident memory.

The release half matters too. Everything described ships in the pingora-ketama crate behind an unadvertised cargo feature: the v2 ring carries the compacted storage, a faster sorting method and a configurable base number of hashes per node, while the v1 ring is untouched and both can run at once. That is the same option the network rollout used, exposed to anyone who depends on the crate. For teams that run their own load balancers, the practical follow-on is an hour with a profiler and the question of what the ring costs in memory today. It is the kind of measurement that sits behind monitoring that explains an incident, and a dashboard of averages hides it.

Netics editorial diagram of the migration sequence Cloudflare describes: two rings in memory at once, a per-request choice that is stable for a given hash, a rollout scoped by data centre, and the removal of the old path once the migration reached 100%.

Originally published on the Netics blog.