DEV Community: Marco Orta

Ollama on an AMD GPU: Five Failures That Drop You to CPU Without Telling You

Marco Orta — Wed, 28 Oct 2026 10:22:27 +0000

The worst Ollama failure on an AMD GPU isn't the one that takes your server down. It's the one that takes nothing down: Ollama starts, responds, answers correctly — and does it on CPU, at 15.89 tokens per second instead of 132.48. An 8.3x penalty that shows up in no log unless you go looking, because as far as Ollama is concerned that isn't an error. It's a fallback.

This article is the summary of three weeks measuring a 24 GB Radeon RX 7900 XTX on a Ryzen 9 7950X3D, across Windows, native Linux and inside WSL2. It isn't an install guide — there are plenty of those — but an inventory of the five ways this setup breaks without saying so, with the numbers for each and with what I proved doesn't fix them, which is usually the more useful half.

The tok/s here belong to one specific card in one specific environment. What does transfer verbatim are the failure modes and the method for catching them.

First: the thirty-second check

If you take one thing from this article, take this. Ollama probes the GPU exactly once, at startup, and never looks again. So there are two questions to answer every time, and neither is answered by noticing that the chat "feels fast":

1. Which backend did the server declare at startup? It's in the startup log (%LOCALAPPDATA%\Ollama\server.log on Windows, journalctl -u ollama on Linux):

library=Vulkan compute=0.0 name=Vulkan0 description="AMD Radeon RX 7900 XTX"
libdirs=ollama,vulkan  total="24.0 GiB" available="23.2 GiB"

What you're looking for is library=. If it says cpu, that's it: there is no GPU, and nothing else is going to tell you.

2. Did the model load in full? ollama ps gives you the split:

ollama ps
# NAME              SIZE     PROCESSOR
# gpt-oss:20b       16 GB    100% GPU

Anything that isn't 100% GPU is a model split between GPU and CPU, and that doesn't raise an error either.

One detail that cost me a whole verifier script: don't require compute=gfx1100 or a pci_id. The Vulkan backend reports compute=0.0 and pci_id="" where ROCm gave real values, so a script checking those fields will fail a perfectly healthy system. Read library= and libdirs=, nothing else.

Failure 1: the CPU's integrated GPU steals the job

This one is treacherous because the guilty hardware isn't the hardware you're looking at.

If you have a desktop Ryzen with integrated graphics — nearly all of the 7000 series and later do — your machine has two AMD GPUs, not one. And ROCm's enumeration orders them however it likes:

ROCm0: AMD Radeon(TM) Graphics (36694 MiB)   <- the integrated one
ROCm1: AMD Radeon RX 7900 XTX (24560 MiB)

The integrated one comes first and reports 36 GB of VRAM (which it doesn't have: that's system RAM). llama.cpp binaries pick it, load kernels compiled for gfx1100, and blow up:

ggml-cuda.cu:106: ROCm error
ROCm error: device kernel image is invalid
  current device: 1, in function ggml_cuda_kernel_launch

What doesn't fix it — and I tried everything before giving up:

Attempt	Result
`--device ROCm1` (pick it by name)	⛔ same error
`HIP_VISIBLE_DEVICES=0` and `=1`	⛔ same error, or "no usable GPU found"
`ROCR_VISIBLE_DEVICES=1` + `--device ROCm0`	⛔ same
`ROCBLAS_TENSILE_LIBPATH` to the kernel directory	⛔ same (the `gfx1100` kernels were there)

What does fix it: disabling the integrated GPU in the BIOS. With a single device there's nothing to pick wrong and it starts first try, with no flags:

	before	after
`--list-devices`	integrated + dedicated	only the RX 7900 XTX
standalone `llama-server`	⛔ `kernel image is invalid`	✅ 136 tok/s
Ollama	118 tok/s	132.5 tok/s

The reasonable fear was losing VRAM by moving the monitor onto the dedicated card. Measured: it costs 0.2 GiB. With the display running 3440×1440 at 165 Hz off the big card, Ollama still reports total 24.0 GiB / available 23.8 GiB. The trade is free.

And a note so you don't lose your mind: under Ollama this doesn't happen, because Ollama masks the integrated GPU on its own and to it the dedicated card is ROCm0. That's why you can have Ollama working and llama-server failing on the same machine with the same weights. You're not imagining it: they're two different enumerations.

Failure 2: updating anything resets the backend

I said it above, but it earns its own section because it's the most expensive one: Ollama probes the GPU at startup and that's it.

After an AMD driver reinstall, the service just carried on like this:

inference compute  id=cpu  library=cpu

No error. No warning. 15.89 tok/s against 132.48. Restarting Ollama put it back on library=ROCm compute=gfx1100 and back to 132.

There's a second version of the same failure, and it's worse because you don't trigger it: every Ollama update reinstalls the ROCm bundle. If you'd deliberately switched to Vulkan, the update quietly puts you back on ROCm. Nobody tells you; the number you'd been measuring for weeks simply stops being the same number.

The operating rule that came out of this: run the check after touching drivers, BIOS or Ollama version. Not after something goes wrong — after touching anything, always, even when nothing seems wrong.

Failure 3: inside WSL2 there is no GPU for Ollama, and there won't be

This is the one that ate the most time, so here's the detail so it doesn't eat yours.

The question was reasonable: if native Linux beats Windows by 6.7 to 19.5% in tok/s, can I take it inside WSL and skip the reboot? The answer is that Ollama in WSL doesn't use the GPU at all. And it isn't a driver problem.

What I measured, in order:

Check	Result
`/dev/kfd` and `/sys/class/kfd`	⛔ don't exist
`/dev/dxg`	✅ exists (the WSL path)
Does HIP compute in WSL?	✅ YES — probe built with `hipcc`: `hipGetDeviceCount rc=0 n=1`, `gfx1100`
Ollama detects GPU	⛔ `library=cpu`
`HSA_ENABLE_DXG_DETECTION=1`	⛔ changes nothing
`llama-server --list-devices` with ROCm	⛔ `Available devices: (none)`
`llama-server --list-devices` with Vulkan	⛔ `Available devices: (none)`

The paradox that explains it: HIP does work in WSL, through the HSA runtime that talks to /dev/dxg. But Ollama enumerates GPUs by reading the KFD topology from sysfs, which WSL doesn't expose. The driver isn't missing — Ollama is looking through a door WSL doesn't have.

With OLLAMA_DEBUG=1 the real cause surfaces, otherwise buried:

failure during llama-server GPU discovery
error="llama-server --list-devices failed: signal: segmentation fault (core dumped)"

There's a recipe going around — Ollama issue 16551, which is the identical case: same card, same WSL2, same ROCm 7.2 — whose author reports library=ROCm compute=gfx1100 at 100% GPU by dropping in the ROCm bundle and setting HSA_ENABLE_DXG_DETECTION=1. Tried here, does not reproduce.

What about the Vulkan backend? In WSL it would need the dzn ICD (Vulkan over D3D12), which Ubuntu doesn't package. You could build Mesa with -Dvulkan-drivers=microsoft-experimental and see what happens, but that's a lot of work for an uncertain payoff.

Practical conclusion: for Ollama, WSL is out. The two exits are Windows Ollama (which does use the GPU, and is what exists today) or booting native Linux. And if you develop inside WSL like I do, the good news is you don't need to move Ollama: with WSL2 in networkingMode=mirrored, localhost:11434 from WSL reaches Windows Ollama with zero configuration.

Failure 4: the model doesn't fit, gets split, and keeps answering

When you ask for more context than fits in VRAM, Ollama doesn't say no. It splits the model between GPU and CPU and carries on. The symptom is size_vram < size in /api/ps, and nothing else.

Measured with qwen3.8 (27.3B) on the 7900 XTX, with OLLAMA_FLASH_ATTENTION=1 and the KV cache at q8_0:

`num_ctx` requested	On GPU
262,144	⛔ 63.4% — a third on CPU
131,072	⛔ 92.4%
122,880	✅ 100% (with 0.6 GiB of headroom)

With 0.6 GiB free, anything you open on the desktop knocks it over. That ceiling belongs to that afternoon and that display, not to the model.

And here's the warning that nearly cost me a bad decision: ollama ps underestimates real VRAM by 4.3 to 6.8 GiB. Cross-checked against the system counter, across four models:

Model	`ollama ps`	Real VRAM
`gpt-oss:20b`	12.33 GiB	16.59 GiB
`muse-glimmer` (27.9B)	15.49 GiB	20.77 GiB
`gemma4` (25.8B)	17.24 GiB	22.37 GiB
`qwen3.8` (27.3B)	16.61 GiB	23.36 GiB

For memory decisions, the ps column is useless. It's only good for telling you whether Ollama split to CPU. If you're working out whether a model fits, measure system VRAM, not what Ollama declares.

Free tool: LLM Token Counter & AI Cost Calculator — Count the tokens in your text or prompt and estimate the cost across GPT-5, Claude, Gemini, Kimi K3, Grok and DeepSeek. Exact OpenAI counting, per-call and monthly costs, with prompt caching.

Failure 5: it trims your context and lets you believe you have it

A subtler variant of the previous one. Ask muse-glimmer or gpt-oss for num_ctx: 262144 and they load perfectly happily — and the context_length returned by /api/ps says 131072. What you asked for isn't what you got.

The underlying cause is that these models have an architecture ceiling below the card's. And it explains something counterintuitive that shows up when you measure: on-disk size does not predict maximum context. What decides is whether the model uses a sliding window (SWA) or dense KV:

Model	SWA	KV growth from 16K to 128K
`muse-glimmer`	2048	~0
`gpt-oss`	128	+0.36 GiB
`gemma4`	1024	+0.40 GiB
`qwen3.8`	— (dense KV)	+1.35 GiB

Three of the four hit their architecture ceiling and never notice the VRAM. The only one that actually pays for context is the only one that overflows.

Always read the real context_length from /api/ps, not the one you sent.

ROCm or Vulkan in 2026: what each one wins

This is the year's underlying shift, and the reason this article lands now. Ollama added experimental Vulkan support in 0.12.6 and it has been reaching the binaries through 2026, with the argument of opening the door to the AMD and Intel GPUs ROCm never supported — which is most consumer cards.

On my machine, forcing each backend with the same Ollama and the same weights, Vulkan wins:

Model	Vulkan's gain over ROCm
`gpt-oss:20b`	+17.8%
`gemma4`	+30.1%
`muse-glimmer`	+30.3%

Disjoint ranges across two independent rounds, so the signal is real. And it lines up with what others report: on an RX 9070 XT, Llama 2 7B Q4_0 decodes at 137 tok/s on Vulkan against 101 on ROCm.

With two caveats almost nobody adds.

First: the delta compares two things at once. On Windows it's ROCm 7.1 against AMDVLK; on Linux it's ROCm 7.2.4 against RADV. A worse ROCm inflates exactly the same number as a better Vulkan, and they aren't separated.

Second is a cost I didn't see coming. Vulkan leaves 0.6 GiB less VRAM available: 23.2 GiB declared against 23.8 with ROCm. For three of my four models it makes no difference because they weren't near the limit. For qwen3.8, which had exactly 0.6 GiB of headroom, it eats the lot:

`num_ctx`	On Vulkan
122,880 (the ROCm ceiling)	⛔ 93.8% on GPU
98,304	✅ 100%

The real trade with this lever isn't speed against stability: it's speed against context. If your model is near the card's limit, Vulkan costs you window. If it isn't, it's free.

What remains unmeasured — by me and by almost everyone — is Vulkan's stability over multi-hour sessions. For a ten-minute chat it doesn't matter; for an agent looping all afternoon, it does.

The full checklist

What I do now every time I touch this machine:

Disable the integrated GPU in the BIOS if you have a Ryzen with graphics. It costs 0.2 GiB and removes an entire axis of ambiguity.
Restart Ollama after touching drivers, BIOS or version. Not optional: it probes once.
Check library= in the startup log. Not compute=, not pci_id.
Check that ollama ps says 100% GPU before trusting any measurement.
Read the real context_length from /api/ps, not the one you sent.
For memory decisions, measure system VRAM, not the ollama ps column.
Don't try it in WSL. Point at localhost:11434 against Windows Ollama.
Close Chrome, Spotify and Slack before measuring. Less for the tok/s than for the range: a real 2% difference disappears if noise widens it.

And the one that covers all of them: an empty result is not a zero. When an instrument returns nothing, the reading isn't "there is none"; it's "I couldn't look." On this machine, Get-Process | Modules returned 0 modules and I nearly read it as "no layers loaded." It was 0 because the process was running elevated and I couldn't see inside. What saved it was having a control case next to it — explorer, same user, 389 modules. Without that contrast, the zero reads as data.

If you're weighing up the hardware

The honest conclusion after all of this: a consumer Radeon runs local models perfectly well in 2026, and the numbers sit alongside what gets published for NVIDIA in the same bracket. ROCm 7.2 is the first release where Ollama, LM Studio, llama.cpp and vLLM all work on Radeon without hand-patching.

What still costs more than on NVIDIA isn't the tok/s: it's the diagnosis. Everything in this article is time you wouldn't have spent on a 4090 — not because the hardware is worse, but because the path is less trodden and the failures are silent instead of loud.

If that trade works for you — and it works for me, because 24 GB of VRAM at this price doesn't exist on the other side — this article is the shortcut past the five holes.

Frequently asked questions

How do I know whether Ollama is using my AMD GPU?

With two checks, not one. First, the server startup log (%LOCALAPPDATA%\Ollama\server.log on Windows, journalctl -u ollama on Linux) must declare library=ROCm or library=Vulkan; if it says library=cpu, there is no GPU. Second, ollama ps must say 100% GPU: any other value means the model is split between GPU and CPU. Ollama raises no error in either case, it just runs slower.

Why does Ollama run on CPU after a driver update?

Because Ollama probes the GPU only once, at startup, and never looks again. If you update drivers, change the BIOS or install a new version while the service is running, it keeps the stale detection and falls back to CPU with no error at all. Measured on an RX 7900 XTX: 15.89 tok/s on CPU against 132.48 on GPU, an 8.3x penalty. The fix is to restart Ollama and re-check the library= line in the log.

Can I use an AMD GPU with Ollama inside WSL2?

No. Verified in August 2026 with an RX 7900 XTX and ROCm 7.2: Ollama enumerates GPUs by reading the KFD topology from sysfs, and WSL exposes neither /dev/kfd nor /sys/class/kfd. HIP does compute in WSL through /dev/dxg, but Ollama does not look there, and HSA_ENABLE_DXG_DETECTION=1 does not change it. Both ROCm and Vulkan return "Available devices: (none)". The practical exit is to point at localhost:11434 against Windows Ollama: with WSL2 in networkingMode=mirrored it works with no configuration.

Is ROCm or Vulkan better for Ollama on an AMD GPU?

It depends on whether your model is near the VRAM limit. On an RX 7900 XTX, Vulkan wins between 17.8% and 30.3% in tokens per second depending on the model, with disjoint ranges across two rounds. But Vulkan leaves 0.6 GiB less VRAM available (23.2 against 23.8 GiB), so a model that was tight on context stops fitting. The trade is not speed against stability: it is speed against context window.

Why does llama.cpp fail with "device kernel image is invalid" on a Ryzen?

Because the machine has two AMD GPUs: the dedicated card and the CPU's integrated graphics. ROCm enumerates the integrated one first, the binary picks it, and kernels compiled for gfx1100 do not work on that chip. Neither --device by name nor HIP_VISIBLE_DEVICES nor ROCR_VISIBLE_DEVICES fixes it. The only thing that works is disabling the integrated GPU in the BIOS, and it costs just 0.2 GiB of VRAM. Ollama does not suffer from this because it masks the integrated GPU on its own.

How much VRAM does a model in Ollama actually need?

More than ollama ps says, which underestimates by 4.3 to 6.8 GiB. Measured across four models on a 24 GB card: gpt-oss:20b declares 12.33 GiB and occupies 16.59; qwen3.8 declares 16.61 and occupies 23.36. To decide whether a model fits you have to read the system VRAM counter; the ollama ps column is only good for telling you whether the model was split to CPU.

Originally published on ortamarco.me.

GitHub Actions Node 20 Deprecation: What Broke on 23 September (and Why node20 Isn't the Problem)

Marco Orta — Wed, 28 Oct 2026 10:22:23 +0000

GitHub removed Node 20 from the Actions runners on 23 September 2026, and an action that declares using: node20 still doesn't fail for that: since 16 June the runner rewrites node20 to node24 before it executes anything, and it does the same with node12 and node16. What went away on the 23rd was the escape hatch: the ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION variable no longer brings Node 20 back. The next date with real consequences is 2 November, when macos-14 is retired.

That inverts who is at risk. If your CI has been green since June without that variable, the 23rd changed nothing for you on GitHub-hosted runners. The ones that break are precisely the teams that already hit an action that can't handle Node 24, set the variable to keep moving, and filed the real fix under later. And around that date there are four more with real consequences: the self-hosted runner minimum version (25 September), retention for checks and runs (1 October), the macos-14 brownouts (October) and its retirement (2 November).

I checked this against the runner's source code and against this site's own CI, which uses two node20 actions. Everything below comes with its source.

The calendar, by date

Date	What changes	What breaks	How to detect it
14, 16 and 18 Sep (11:00-15:00 ET)	Brownouts ahead of the self-hosted runner minimum version	Outdated runners cannot register or run jobs during the window	Runner deprecations API (below)
17 Sep	`ubuntu-22.04` and `ubuntu-22.04-arm` enter deprecation	Nothing fails yet; queues may be longer	`grep` for the label
21-30 Sep	`windows-11-arm` moves to the Visual Studio 2026 image	The toolchain underneath changes	`grep windows-11-arm`
23 Sep	Node 20 leaves the runners	`ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION` is ignored; Linux ARM32 runners lose support	`grep` for the variable
25 Sep	Self-hosted runner minimum version is enforced	Registering requires ≥ 2.329.0; a runner that doesn't install each release within 30 days stops receiving jobs	Deprecations API
1 Oct	Actions retention now covers checks, workflow runs and statuses	Anything past your retention period (90 days by default) is deleted	Repo/org retention setting
5-31 Oct	Eight `macos-14` brownouts	Jobs scheduled inside the window fail	`grep macos-14`
2 Nov	`macos-14` retired	Jobs with that label terminate with an error	`grep macos-14`
17 Apr 2027	`ubuntu-22.04` retired (brownouts before, in March and April)	Jobs terminate with an error	`grep` for the label

1. Node 20: what actually happens on 23 September

What the notice says

GitHub's changelog post from 19 September 2025, with an editor's note dated 25 August 2026, sets two dates: from 16 June 2026 runners use Node 24 by default, and the ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true opt-out back to Node 20 "will only work until we upgrade the runner and remove Node20 on September 23rd, 2026." It adds two platform limits: Node 24 is incompatible with macOS 13.4 and lower, and it has no official ARM32 support.

What the notice does not explain is what happens to an action whose action.yml still says using: node20. That lives in the code.

What the runner code says

In HandlerFactory.cs (identical at tag v2.337.0, the current release), the runner does two things before launching a JavaScript action:

If the action declares node12 or node16, it bumps it to node20.
If it ends up on node20, it asks NodeUtil.DetermineActionsNodeVersion which version to use.

That function has three phases, driven by feature flags on GitHub's side:

// Phase 3: Always use Node 24 regardless of environment variables
if (requireNode24)
{
    return (Constants.Runner.NodeMigration.Node24, null);
}
// ...
// Phase 2: Node 24 is the default
if (useNode24ByDefault)
{
    if (allowUnsecureNode)
    {
        return (Constants.Runner.NodeMigration.Node20, null);
    }
    return (Constants.Runner.NodeMigration.Node24, null);
}

PR #3948, which introduced this, is blunt about it: in phase 3, "All actions use Node 24" and "No opt-out options available." No branch returns an error for declaring node20. The action runs on a different Node, which is very different from not running.

Note the order too: node12 → node20 → node24. An old action with using: node16 (for example actions/checkout@v3) ends up on Node 24 as well.

The proof: this site's CI

This blog's CI workflow uses actions/checkout@v4 and actions/setup-node@v4. The official action.yml files at those tags declare using: node20; from v5 onward they declare node24 (I checked tags v4 through v7 of both). The run on 11 September, on runner 2.337.0 with the ubuntu-24.04 image, finished green with this annotation:

Node.js 20 is deprecated. The following actions target Node.js 20 but are being
forced to run on Node.js 24: actions/checkout@v4, actions/setup-node@v4.

And in the log of those two actions' steps, this line:

Node 20 is being deprecated. This workflow is running with Node 24 by default.
If you need to temporarily use Node 20, you can set the
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true environment variable.

The same annotation appears on the very first run of that workflow, on 23 August. In other words, my two node20 actions have never run on Node 20 in this CI, and I didn't touch anything to make that happen. 23 September doesn't change their runtime, because that change already happened in June. Bumping them to v5 or later clears the annotation, and it's worth doing, but there's no clock on it.

What it looks like when it does break

There are three real ways to fail, and none of them is a GitHub message saying "Node 20 no longer exists."

You had the variable set. Some action of yours tripped on Node 24 after 16 June, you set ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true, and it stayed on Node 20. In phase 3 the runner ignores the variable without telling you: the requireNode24 branch returns null as its message. The action goes back to Node 24, and the error you see is its own code's error, the same one you saw the first time. One hint that phase 3 is live: with the current runner code, the "If you need to temporarily use Node 20…" line disappears from the log, because it is only printed while Node 24 is the default but not yet mandatory.

The action uses something Node 24 removed. Per the Node 24 deprecations table and the 24.0.0 release notes, these reached end-of-life in that release:

API	Status in Node 24	How it fails
`tls.createSecurePair()`	Removed (DEP0064)	`TypeError: ... is not a function` when called
`fs.truncate()` with a file descriptor	Removed (DEP0081)	Throws `ERR_INVALID_ARG_TYPE`: it expects a path. Use `fs.ftruncate()`
`dirent.path`	Removed (DEP0178)	Doesn't throw on its own: the property is gone and you read `undefined`. Use `dirent.parentPath`
`OutgoingMessage.prototype._headers` / `_headerNames`	Removed (DEP0066)	You read `undefined`. Use `getHeaders()`

dirent.path is the sneakiest one on the list, because reading it throws nothing: the failure shows up later, wherever that path gets used, or never, if it's only concatenated into a string and comes out as undefined/file. If you maintain an action that walks directories, search for that property before you trust the green check.

Your self-hosted runner can't run Node 24. On Linux ARM32, the same NodeUtil.cs has a kill switch that fails the step with:

Linux ARM32 runners are no longer supported. Please migrate to a supported platform.

Before it's flipped, the runner keeps those actions on Node 20 with a warning whose default date is the Node 20 removal date: "Linux ARM32 runners are deprecated and will no longer be supported after September 23rd, 2026." GitHub hasn't published the exact day the switch flips; the changelog only says ARM32 loses support after the Node 20 deprecation. On macOS, Node 24's BUILDING.md requires macOS 13.5 or later.

2. The audit: what to look for and where

In one repository

Run this from the root:

# 1. The switch that stops working on 23 September
grep -rn 'ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION' .github/

# 2. Image labels with a date. No "runs-on:" on purpose,
#    so matrices show up too (os: [ubuntu-22.04, ...])
grep -rnE 'ubuntu-22\.04|macos-14|windows-11-arm' .github/

# 3. Your own or local actions that declare an old Node
grep -rnE --include=action.yml --include=action.yaml \
  "using:[[:space:]]*['\"]?node(12|16|20)" .

Search 2 has a blind spot: runs-on: ${{ vars.RUNNER }} or a label passed as an input to a reusable workflow won't show up in any grep. For those, the log is the source of truth. The "Set up job" step prints the real image:

gh run view <run-id> --log | grep -m1 'Image:'
# Image: ubuntu-24.04

If the matrix is long or nested, converting it to JSON and filtering it with jq is usually faster than reading the YAML by eye:

Free tool: JSON and YAML Converter — Convert JSON to YAML and YAML to JSON instantly, with configurable indentation, download and copy. All in your browser, nothing uploaded.

To find out which runtime each third-party action you use declares, including SHA-pinned ones, you have to read its action.yml at that exact ref. This script does it with gh:

grep -rhoE "uses:[[:space:]]*['\"]?[^[:space:]'\"#]+@[^[:space:]'\"#]+" .github/ \
  | sed -E "s/uses:[[:space:]]*['\"]?//" | sort -u \
  | while read -r ref; do
      spec=${ref%@*}; ver=${ref#*@}
      case "$spec" in ./*|docker://*) continue ;; esac
      repo=$(cut -d/ -f1-2 <<<"$spec"); sub=$(cut -d/ -f3- <<<"$spec")
      for f in action.yml action.yaml; do
        rt=$(gh api "repos/$repo/contents/${sub:+$sub/}$f?ref=$ver" --jq .content 2>/dev/null \
             | base64 -d | grep -E '^[[:space:]]*using:' | tr -d " '\"" | cut -d: -f2)
        [ -n "$rt" ] && { echo "$rt  $ref"; break; }
      done
    done

On this repo it prints:

node20  actions/checkout@v4
node20  actions/setup-node@v4

It handles subpaths (github/codeql-action/init@v3) and SHAs. A SHA pointing at a node20 release doesn't fail on the 23rd, for the same reason @v4 doesn't: the runner forces it onto Node 24. What's different with a SHA is how it gets updated: nothing moves on its own, so Dependabot or a human has to bump it.

One warning before bumping a major: read the release notes. actions/setup-node v5 turned on automatic caching when package.json has a packageManager field, and v6 limited it to npm. If you already pass cache: npm explicitly, as my CI does, it doesn't affect you.

Across an organization

GitHub code search reaches every repo at once:

gh search code --owner YOUR_ORG 'ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION'
gh search code --owner YOUR_ORG 'ubuntu-22.04 path:.github/workflows'
gh search code --owner YOUR_ORG 'macos-14 path:.github/workflows'
gh search code --owner YOUR_ORG node20 --filename action.yml

Two limits of the code search API that will bite you: it only searches the default branch (a workflow on a release branch won't show up) and it allows 10 requests per minute. While preparing this post I hit HTTP 403: API rate limit exceeded as soon as I chained a handful of searches; add a sleep between them if you automate it.

3. Self-hosted runners: two dates in three days

If you only use GitHub-hosted runners, skip this section. If you run your own, 25 September matters more than the 23rd.

The 12 June changelog post resumes enforcement of two rules on github.com:

To register (or re-register) a runner you need version 2.329.0 or later.
To keep running jobs, the runner must install each new release within 30 days of its publication. "A runner pinned to 2.329.0 that never updates again will not pick up jobs."

Full enforcement begins on 25 September 2026 (the notice's table labels that date as GitHub Enterprise Cloud, and the same notice states the change applies to github.com), with brownouts first: on 14, 16 and 18 September, from 11:00 AM to 3:00 PM Eastern, out-of-date runners can neither register nor run jobs. What you see, per the notice, is that "workflows targeting unsupported runners may remain queued or fail."

The way to find out when your version expires is an endpoint GitHub added on 3 September. I queried it today:

gh api repos/OWNER/REPO/actions/runners/deprecations/2.335.1
# {"runner_version":"2.335.1","runtime_deprecates_at":"2026-09-24T15:30:55Z"}

gh api repos/OWNER/REPO/actions/runners/deprecations/2.337.0
# {"runner_version":"2.337.0","runtime_deprecates_at":null}

A runner on 2.335.1 falls out of support on 24 September, so by the time full enforcement starts on the 25th it no longer gets jobs. You get the installed version on the machine itself with ./config.sh --version.

The self-hosted checklist, in order:

Update the runner to the current release and leave auto-update on. If it's off, you need a manual cadence shorter than 30 days.
Rebuild the images, VM templates and containers you create runners from. The changelog asks you to recreate runners built from older cached images or templates: if the image ships a runner older than 2.329.0, it won't register.
Linux ARM32: move to arm64. Node 24 has no official support there, the runner's linux-arm package only bundles the Node 20 binary (externals.sh), and the code that fails the step is already in the runner.
macOS 13.4 or earlier: upgrade the OS. GitHub lists Node 24 as incompatible with those versions.
Actions that already declare node24 (such as actions/checkout@v5) require runner v2.327.1 at minimum. If you've done step 1, you already have it.

4. Retention, 1 October: history gets deleted

This change breaks no build, but it deletes data. According to the 27 August changelog post, starting 1 October 2026 checks, workflow runs and statuses follow the same retention setting that already governed artifacts and logs, 90 days by default. Until now they were kept for "400+ days" regardless of your configuration.

For public repositories the maximum is 90 days.
The change is not retroactive: raising retention later won't restore anything.
Check and workflow run metadata isn't billed as storage; the associated artifacts and logs are.

What breaks is anything that reads that history beyond your period: deployment metrics computed from gh run list, audits that link to a specific run, quarterly reports. Export what you need before 1 October.

What the changelog does not say: what happens to a pull request open for more than 90 days whose required check expires. I couldn't find it documented. If you have long-lived PRs under branch protection, look at them after 1 October before assuming anything.

5. Images: macos-14 has a date, ubuntu-22.04 doesn't yet

macos-14 (actions/runner-images#13518): deprecation started on 6 July and support ends on 2 November 2026. Before that there are eight brownouts during which jobs fail: 5, 12, 16, 19, 23, 26, 29 and 30 October, each from 14:00 UTC to 00:00 UTC the next day. It covers macos-14, macos-14-large and macos-14-xlarge. The replacement is macos-15, macos-26 or macos-latest, which started pointing to macOS 26 with the migration that began on 15 June.

ubuntu-22.04 (actions/runner-images#14254): on 17 September deprecation only begins. The issue warns about longer queue times at peak hours, not failures. Retirement is on 17 April 2027, with brownouts that the issue schedules for March and April (no year given, but they fall before the retirement). It also covers ubuntu-22.04-arm. The replacement is ubuntu-24.04, ubuntu-26.04 or ubuntu-latest.

If someone tells you ubuntu-22.04 breaks this week, it doesn't. Migrate calmly, but don't leave it for March.

What's in the changelog and breaks nothing

cache-mode (10 September): a new key to limit cache access per workflow or job. It's opt-in: workflows that don't set it "continue to use the existing secure defaults."
The separate Code Quality path (20 August): only affects you if you have reports filtering on dynamic/github-code-scanning/codeql or on the github-advanced-security actor.
actions/checkout and pull_request_target (18 June): this one does break, but it already happened. Since 20 July the floating tags, @v4 included, refuse to check out fork code in pull_request_target. If your workflow depended on that, it's already red. It's the same family of risk I covered in the worm that got in through npm trusted publishing.

What I'm doing with my CI

Nothing urgent. actions/checkout@v4 and actions/setup-node@v4 have been running on Node 24 since June, the workflow doesn't use the opt-out variable, it runs on ubuntu-latest (currently ubuntu-24.04) and on GitHub-hosted runners. 23 September changes nothing for it.

I am going to bump both actions to their current major, for two reasons unrelated to the date: a permanent yellow annotation trains you to ignore annotations, and an action its authors no longer test on the runtime it actually runs on is debt. If you're also thinking about your own project's Node rather than the actions', I covered that in what breaks when upgrading to Node 26.

Frequently asked questions

What happens to GitHub Actions on 23 September 2026?

GitHub removes Node 20 from the runners. Since 16 June 2026 runners had already been running actions on Node 24 by default, and ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true let you go back to Node 20. From 23 September that variable stops working and every JavaScript action runs on Node 24. Self-hosted runners on Linux ARM32 lose support, because Node 24 has no official support for that platform.

Does an action that declares using: node20 fail after 23 September?

Not for declaring it. The runner code (HandlerFactory.cs and NodeUtil.cs in actions/runner) rewrites node12 and node16 to node20, and node20 to node24; in the final phase it ignores the environment variables and always picks Node 24, without returning an error. The action fails if its code uses something Node 24 removed, such as tls.createSecurePair or fs.truncate with a file descriptor, or if it runs on a self-hosted Linux ARM32 runner or on macOS 13.4 or earlier. The logs show the annotation: The following actions target Node.js 20 but are being forced to run on Node.js 24.

Do I have to update actions/checkout@v4 and actions/setup-node@v4?

It's a good idea, but no date forces it. The v4 tags of both declare node20 and v5 onward declare node24. On GitHub-hosted runners, the v4 tags have already been running on Node 24 since June and keep working. When bumping setup-node, read the release notes: v5 turned on automatic caching when package.json has packageManager, and v6 limited it to npm.

How do I find affected workflows across my organization?

With GitHub code search: gh search code --owner YOUR_ORG with ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION, with ubuntu-22.04 path:.github/workflows, with macos-14 path:.github/workflows, and with node20 --filename action.yml for your own actions. The API only searches the default branch and allows 10 requests per minute. Labels that come from variables or expressions don't show up in any search: for those, check the Image: line of the Set up job step in the log.

Does ubuntu-22.04 break on 17 September 2026?

No. On 17 September deprecation only begins, and the notice talks about longer queue times at peak hours. Jobs start failing during the brownouts that issue 14254 in actions/runner-images schedules for March and April, before the image is retired on 17 April 2027. The one with a date this fall is macos-14: brownouts in October and retirement on 2 November 2026.

What do self-hosted runners need to do before 25 September?

Update the runner. From 25 September 2026 GitHub requires version 2.329.0 to register a runner, and each new release must be installed within 30 days to keep receiving jobs. There are brownouts first on 14, 16 and 18 September, from 11:00 AM to 3:00 PM Eastern. The endpoint GET /repos/OWNER/REPO/actions/runners/deprecations/VERSION returns when your version stops running jobs. Also rebuild the images you create runners from.

What changes with GitHub Actions retention on 1 October 2026?

Checks, workflow runs and statuses start following the artifact and log retention setting, which defaults to 90 days; before, they were kept for more than 400 days. For public repositories the maximum is 90 days, and the change is not retroactive. If you have metrics or audits that read old runs, export the data before that date.

Originally published on ortamarco.me.

npm Supply Chain: The Worm That Walked In Through Trusted Publishing

Marco Orta — Wed, 28 Oct 2026 10:22:19 +0000

For a year, npm security advice was a single sentence and everybody repeated it: stop publishing with long-lived tokens and move to trusted publishing with OIDC. It was good advice. It still is. And on May 11, 2026, the Shai-Hulud worm published dozens of official @tanstack/* packages by walking in exactly that way — not by forging provenance, but by genuinely generating it from the project's own legitimate workflow.

That distinction is the whole article. A malicious package with an authentic Sigstore signature, issued by Fulcio, logged in Rekor, backed by the real repository context. Every indicator you were taught to check said "fine."

This doesn't mean trusted publishing is useless, or that anyone should go back to tokens. It means something more uncomfortable: it's a layer, not a perimeter, and we spent a year selling it as the second thing. Here's what it actually protects, what it leaves out, and the four measures that would have stopped the last twelve months of attacks.

The timeline, with dates

Worth having in front of you, because the cadence matters more than any single incident:

Date	What happened
Jul 2025	Trusted publishing with OIDC goes generally available on npm
Aug 26, 2025	S1ngularity attack
Sep 8, 2025	Phishing against `debug`, `chalk` and 16 more utilities — live for two hours
Sep 15, 2025	Shai-Hulud wave 1: 700+ packages compromised
Nov 23, 2025	Shai-Hulud 2.0
Dec 9, 2025	npm permanently revokes every classic token
Feb 2026	npm CLI 11.10.0 ships `min-release-age` and bulk OIDC configuration
Mar 30, 2026	`axios@1.14.1` compromised — 100M weekly downloads
May 11, 2026	Shai-Hulud against TanStack, entering through trusted publishing itself
May 14, 2026	`node-ipc`: three malicious versions with an 80 KB credential stealer
Jun 1, 2026	32 `@redhat-cloud-services` packages, payload named "Miasma"
Aug 2026	New wave: `keyv` and 400+ packages

Ten months, eight campaigns. This is no longer a series of incidents: it's the registry's normal operating state.

How it came in through the good door

The TanStack attack is worth following step by step, because each link dismantles a different assumption. Upwind's reconstruction lays it out like this:

A "dropper" dependency lands in package.json and runs during installation via npm's lifecycle hooks. Nothing exotic: that's documented behaviour.
The dropper downloads and runs the Bun runtime. That way it doesn't depend on the installed Node and it sidesteps monitoring that watches for Node executions.
Under Bun runs a heavily obfuscated 2.3 MB payload (router_init.js) that daemonises itself and stays alive after the install finishes.
Credential sweep: filesystem, cloud APIs, and — this is the good part — GitHub Actions runner memory, scraped from /proc/*/mem. Targets: AWS keys, npm tokens, Kubernetes service accounts, AI assistant configs.
OIDC abuse: rather than needing a stored npm token, it exchanges the GitHub Actions OIDC token for dynamic publish access. The workflow's id-token: write permission is the key.
Propagation: with that access it downloads target packages, injects itself, bumps the version by exactly +3 and republishes.
And along the way it mints a Sigstore bundle through Fulcio and Rekor — so the malicious tarball ships with provenance generated from the trusted workflow context.

Point 7 is the one to internalise. There was no forgery. The chain of trust worked exactly as designed; what was compromised was the link before it, the CI runner. Provenance answers "was this built where it says?", and the answer was yes. It does not answer "is what was built what the maintainer wrote?", which was the question that mattered.

The published indicators, in case you need to sweep: workflows talking to api.masscan[.]cloud or 83.142.209.194, and persistence artifacts named router_init.js, pgmonitor.py or pgsql-monitor.service.

So is trusted publishing worth it?

Yes, very much so. But the split is worth being explicit about, because the generic advice never is:

Threat	Does OIDC stop it?
npm token stolen off a laptop	✅ Yes — there's no token to steal
Token leaked in a CI log or a `.env`	✅ Yes
Maintainer phishing (the `debug`/`chalk` case)	✅ Largely
Token still valid months after a maintainer leaves	✅ Yes
CI runner compromised during the build	⛔ No
Malicious dependency running in `postinstall`	⛔ No
CI cache poisoning	⛔ No
Maintainer's GitHub account compromised	⛔ No

The correct reading: OIDC eliminates the long-lived secret, which was the dominant vector through 2025. It does not defend the process that issues the publish. And as the first vector closes, attacks move to the second. That is precisely what the timeline above shows.

One number to calibrate how many people are still in the first box: according to Aikido's analysis, of the 51,370 most-downloaded packages only 11,001 use trusted publishing — 21.4%, covering barely 25% of download volume. Three quarters of what you install every day is still published with tokens. And adoption moves incident by incident: from a baseline of ~35 new packages a week it spiked to 372 during Shai-Hulud 2.0, then the urgency deflated again.

So the message isn't "trusted publishing failed." It's "migrate anyway, and also do these four things."

The four measures that would actually have stopped this

1. Dependency cooldown: don't install what shipped this morning

This is by far the best impact-to-effort ratio available, and still the least used.

The idea is offensively simple: tell your package manager to ignore versions published less than N days ago. Almost all of these campaigns are detected and pulled within hours — the malicious debug and chalk versions were live for two hours — so a modest delay takes you entirely out of the exposure window. Of ten attacks analysed, eight had windows shorter than a week.

All four managers support it natively now, with the trap that the units don't match:

Manager	Key	Unit	Since
npm	`min-release-age`	days	CLI 11.10.0 (Feb 2026)
pnpm	`minimumReleaseAge`	minutes	10.16 (Sep 2025)
Yarn	`npmMinimalAgeGate`	minutes	Berry 4.10.0
Bun	`minimumReleaseAge`	seconds	—

In npm, one line in .npmrc:

min-release-age=7

In pnpm, inside pnpm-workspace.yaml, with an exclusion list for whatever you publish yourself:

minimumReleaseAge: 10080          # 7 days in minutes
minimumReleaseAgeExclude:
  - '@my-company/*'

Two practical warnings. First: the cooldown is enforced at install time, not at update-suggestion time, so your dependency bot can open the PR and the npm ci fail afterwards; set the same threshold in both places. Second: Bun counts seconds, not minutes: seven days is 604800, and anyone who copies pnpm's 10080 ends up with a cooldown of under three hours without noticing. Yarn does accept durations like 7d, and its exemption list is not a *Exclude key but npmPreapprovedPackages.

If you want to see what it would have held back in your project before switching it on, dep-cooldown, an open-source tool of mine, reads your npm, pnpm, Yarn or Bun lockfile and gives you the age of every resolved version, including on the day you installed (--as-of). And npx dep-cooldown --config all prints the key for all four managers with the unit already converted.

2. Separate the runner that installs from the runner that publishes

This is the one that would have cut the TanStack attack dead.

The structural problem is that in most workflows the same job runs npm ci and npm publish. That means third-party code executing during install lives in the same process that holds — or can request — the OIDC token.

The right shape is to split it, with id-token: write existing only in the publishing job, which installs nothing from third parties:

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      contents: read          # no id-token here
    steps:
      - uses: actions/checkout@v5
      - run: npm ci --ignore-scripts
      - run: npm run build
      - run: npm pack
      - uses: actions/upload-artifact@v4
        with: { name: tarball, path: '*.tgz' }

  publish:
    needs: build
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write         # the permission lives only here
    steps:
      - uses: actions/download-artifact@v4
        with: { name: tarball }
      - run: npm publish *.tgz --provenance

Note the --ignore-scripts on the install step. Lifecycle hooks are steps 1 and 2 of the attack; if your build doesn't need them — and many don't — turning them off in CI is free. If some dependency genuinely does need them, the exception list is shorter than you think and worth writing by hand.

3. Treat the CI cache as untrusted input

The initial vector of the May wave was cache poisoning. It's an unintuitive vector because the cache looks like your own infrastructure, and it isn't: in GitHub Actions, a lower-privileged branch can write into a cache that the release branch later reads.

The minimum:

Never use pull_request_target without an explicit trust boundary. That trigger runs with the base repository's secrets and the PR's code in front of it; it's the foot in the door.
Segregate cache keys by branch or by purpose. The release workflow should not share a key with the PR workflow.
In the publishing job, don't restore a cache at all. That job should be as boring as possible: download an artifact, upload it. Nothing else.

4. Stop reading the provenance badge as a verdict

Provenance is still useful: it tells you where a package was built. What you can't do after May is read it as "this is safe."

In practice that means dependency review doesn't get delegated to a badge. What does work:

Pin with package-lock.json and always use npm ci in CI, never npm install. That's the difference between reproducing a tree and resolving a new one every time.
Watch for odd version jumps. The worm bumped versions by exactly +3. A patch that skips three numbers with no changelog is a cheap and surprisingly good signal.
Rotate credentials when a wave hits you, and rotate everything: npm, GitHub, Actions secrets, AWS, Vault, Kubernetes, SSH. These payloads sweep the entire runner, not just the publish token.

Free tool: JWT Decoder and Verifier — Decode the header and payload of a JSON Web Token and verify its HS256/384/512 or RS256/384/512 signature.

And what npm changed on its own

Part of the work has already been done for you, and it's worth knowing because it affects scripts you may still have lying around:

Classic tokens have been revoked since December 9, 2025. Not deprecated: revoked. If an old pipeline of yours stopped publishing on that date, this is why.
npm login no longer hands you a long-lived token, but a two-hour session that expires on its own and enforces 2FA for publishing.
Granular write tokens expire after 90 days at most. The eternal token no longer exists, even if you ask for it.
And since npm CLI 11.10.0 you can configure OIDC in bulk across many packages at once, which was the real friction in migrating an org with fifty packages.

If you publish packages, migrating to trusted publishing is literally four lines of YAML — the id-token: write and the --provenance from the example above — plus setting the trusted publisher on the package page. Do it. But do it knowing you've just closed the 2025 vector, not the 2026 one.

What I'd do this month, in order

If you have half an hour, in decreasing order of return:

Set min-release-age=7 (or your manager's equivalent) in every repository. Five minutes, cuts off most known campaigns.
Swap npm install for npm ci --ignore-scripts in every CI workflow that doesn't need hooks.
If you publish packages, split the workflow into build and publish, with id-token: write only in the second.
Migrate to trusted publishing anything still publishing with a granular token.
Audit your pull_request_target triggers and any cache keys shared across branches.

None of the five is expensive. And here's the uncomfortable part: the first one, the most effective of all, requires trusting nobody and understanding no attack. Just waiting a week.

Frequently asked questions

Is trusted publishing with OIDC still worth it after the May 2026 attack?

Yes, and you should migrate anyway. OIDC completely eliminates the long-lived token, which was the dominant vector through 2025: there is nothing to steal from a laptop, from a CI log, or from a maintainer via phishing. What it does not cover is the process that issues the publish: if the CI runner is compromised during the build, the attacker exchanges the OIDC token exactly as the legitimate workflow would. It is a layer, not a perimeter.

How did the Shai-Hulud worm generate valid Sigstore provenance?

Because it did not forge it: it genuinely generated it. The malicious payload ran inside the project's own GitHub Actions runner, requested an OIDC token using the workflow's id-token: write permission, exchanged it for npm publish access, and minted a Sigstore bundle through Fulcio and Rekor from that trusted context. Provenance answers "was this built where it says?", and the answer was yes. It does not answer "is what was built what the maintainer wrote?".

What is a dependency cooldown and how do I configure it?

It tells your package manager to ignore versions published less than N days ago, so you never install a malicious version inside its lifetime, which is usually measured in hours. In npm it is min-release-age in .npmrc, measured in days (since CLI 11.10.0, February 2026); in pnpm it is minimumReleaseAge, measured in minutes (since 10.16); in Yarn it is npmMinimalAgeGate in minutes, and in Bun minimumReleaseAge in seconds. A seven-day threshold would have blocked eight of the ten most recent attacks analysed.

Why should the install job be separate from the publish job in CI?

Because if the same job runs npm ci and npm publish, the third-party code that executes during installation lives in the same process that has access to the OIDC token. That is how the TanStack attack got in: a malicious dependency ran in a lifecycle hook, scraped the runner memory and exchanged the token. The fix is for id-token: write to exist only in a publish job that installs nothing from third parties and just uploads an already-built artifact.

What share of npm packages already use trusted publishing?

Not many: of the 51,370 most-downloaded packages, 11,001 use it — 21.4%, covering around 25% of total download volume. In other words, three quarters of what gets installed daily is still published with tokens. Adoption spikes with each incident (372 new packages during the week of Shai-Hulud 2.0, against a baseline of about 35 per week) and falls back once the urgency passes.

Do npm classic tokens still work?

No. npm permanently revoked every classic token on December 9, 2025. Since then npm login issues a two-hour session instead of a long-lived token, and granular write tokens expire after 90 days at most. If an old pipeline stopped publishing around those dates, that is the cause.

Originally published on ortamarco.me.

Why Online Stores Crash During Mexico's Buen Fin (and How to Prevent It): The 2026 Technical Guide

Marco Orta — Wed, 28 Oct 2026 10:22:16 +0000

El Buen Fin, Mexico's answer to Black Friday, runs from Friday November 13 to Tuesday November 17, 2026. Almost everything published about getting a store ready for it is about banners, coupons and email. This guide is the other half: what breaks technically when the visitors you paid to attract all show up at once, and what to do beforehand. Nearly all of it is written down in your host's, your payment gateway's and your platform's documentation, and almost nobody reads it until the store is already down.

Last year's numbers show why it matters. According to Mexico's Ministry of Economy, the 2025 edition sold 219.2 billion pesos, and digital commerce was 21% of the total, growing 31% year over year. The public AMVO report (Mexico's online sales association) counted 1.686 billion page views on e-commerce sites and traffic 56% higher than the previous edition (which went from four days to five). The retailers' association ANTAD reports that the last day alone took 26.1% of online sales. The peak doesn't spread out: it arrives, and it arrives all at once.

If you sell into Mexico from abroad, or build stores for Mexican clients, the local details are what catch people out: the cash-at-OXXO payment method, the Mexican shared-hosting plans small businesses actually run on, and a gateway landscape where Mercado Pago matters as much as Stripe. I've built WooCommerce stores with catalogues in the thousands, Mercado Pago payments and WhatsApp checkout. This is what I check on each one before a date like this, with a source for every limit.

What those five days look like

Buen Fin 2025	Figure	Source
Total sales	219.2 billion pesos	Ministry of Economy
Online sales	21% of the total, +31% YoY (45.9 billion per AMVO)	Economy / AMVO
Traffic to online stores	+56% over the 5 days, +24% on comparable days	AMVO (Similarweb data)
Average ticket	$1,063 MXN online vs $780 overall	Mexican Banking Association, cited by Economy
How people paid online	Credit 48%, debit 43%, store cards 17%, card-free instalments 12%, cash at retail chains 12%	AMVO
Consumer agency complaints	220 filed, 205 settled; mostly unhonoured prices and refused delivery	Profeco

Two rows matter for the technical side. One in eight online buyers paid cash at a retail chain (OXXO and similar), which has a consequence for your inventory that I explain below. And 37% of consumers, per the AIMX survey the ministry cites, fear a data leak: an expired-certificate warning or a checkout page that looks off on your busiest day costs sales even if the site stays up.

Not everything that fails is yours, either. During Buen Fin 2024, Santander and BBVA customers reported card payment failures; during Hot Sale in May 2026 BBVA confirmed on X that its systems were down, with over five hours of Downdetector reports. And on November 18, 2025, the day after Buen Fin ended, Cloudflare had a global outage of more than three hours. You can't stop your customer's bank from going down. You can have a second payment method ready when it does.

1. Shared hosting has a ceiling, and it's in writing

The shared hosting plan that costs a few hundred pesos a year isn't "slow": it has exact limits, and when you hit them it doesn't slow down, it returns errors. Hostinger publishes its limits on its plan parameters page and says reaching them produces "503 – Service temporarily unavailable":

Hostinger plan	CPU	RAM	Concurrent PHP workers	MySQL connections
Web Single	1	1 GB	20	25
Web Premium	1	2 GB	40	50
Unlimited (formerly Business)	2	3 GB	60	75

HostGator Mexico puts it differently in its help centre: a cap of 25% CPU sustained for 90 seconds or more, and 25 concurrent processes per cPanel account.

Do the maths with your own store. An uncached WooCommerce product page holds a PHP worker while it renders. If it takes one second and you have 20 workers, your ceiling is about 20 pages per second; visitor number 21 doesn't wait in line, they get an error. That's why caching, next, isn't an optimization: it decides whether those limits ever reach you.

2. Caching: what can be cached and what can't

The idea is simple: most visitors, the ones just browsing, should get a prebuilt copy of the page and cost your server nothing. Only the cart, checkout and account pages need to reach the server every time.

What almost nobody knows is that Cloudflare doesn't cache HTML by default, on free or paid plans. Its documentation is explicit: "The Cloudflare CDN does not cache HTML or JSON by default." If you put your store behind Cloudflare and configured nothing else, images, CSS and JavaScript are cached, but every page still hits your host. The free plan gives you up to 10 Cache Rules and one is enough. This expression is for a Spanish-language WooCommerce store (adjust the paths to yours):

(http.host eq "tutienda.mx"
 and not starts_with(http.request.uri.path, "/wp-admin")
 and not http.request.uri.path contains "/carrito"
 and not http.request.uri.path contains "/finalizar-compra"
 and not http.request.uri.path contains "/mi-cuenta"
 and not http.cookie contains "woocommerce_items_in_cart"
 and not http.cookie contains "wordpress_logged_in")

With the "Eligible for cache" action and a short edge TTL (5 to 10 minutes), browsers get the cached copy, and anyone with something in the cart or a logged-in session goes straight to your server. Two warnings:

Purge the cache when prices change. If your discounts start at midnight on the 13th, a copy cached at 11:58 pm still shows the old price, and "the price wasn't the advertised one" is among the complaints Profeco receives most. Purge when you switch the offers on.
A cookie on every page breaks everything. Cloudflare won't cache a response carrying Set-Cookie, and some plugins (currency switchers, geolocation, popups) send one on every visit. Check it like this:

curl -sI https://tutienda.mx/producto/tu-mas-vendido/ | grep -iE "set-cookie|cf-cache-status"

What you want to see, from the second request on, is cf-cache-status: HIT and no set-cookie. If you see DYNAMIC or BYPASS, that page is still hitting your server.

Two things look like safety nets and aren't. Always Online is on every plan, but it only kicks in on 52x errors, serves Internet Archive snapshots and can't serve dynamic content: nobody checks out from an archived copy. And Cloudflare's Waiting Room, the virtual queue big retailers use, only exists on Business and Enterprise plans.

On WooCommerce, also check cart fragments: the script that updates the cart counter by calling the server on every page. WooCommerce acknowledges that on heavily trafficked stores "this could severely impact the load on the server", and since version 7.8 it no longer loads on every page, but plenty of themes and plugins turn it back on. Open a product page with your browser's dev tools and look for a request to ?wc-ajax=get_refreshed_fragments. If it fires on pages without a mini cart, that's one server request per visit that caching can't remove.

And slim down your product photos: they're most of a store's page weight, and on mobile, where most Mexican shoppers buy, every megabyte shows.

Free tool: Image Compressor — Compress and optimize JPG, PNG or WebP images without visible quality loss, with quality control, format change and resizing. All in your browser.

3. Your payment gateway has a limit too

The server isn't the only thing that can say no. Stripe publishes its rate limits: 100 requests per second in live mode, with most individual endpoints capped at 25. And it says plainly that "a sudden increase in charge volume, such as a flash sale, might result in rate limiting"; if you expect a spike, it asks you to contact support beforehand. Mercado Pago doesn't publish a number, but its Orders API answers 429 Too Many Requests with a Retry-After header when you exceed it.

For a small business, 100 charges a second is plenty. The problem shows up when the integration makes several calls per purchase (create customer, create payment intent, poll status) or retries without backing off: a badly handled 429 becomes "we couldn't process your payment" for a customer who had the funds.

Bank verification (3-D Secure, the "confirm in your app" screen) changes the flow too. Mexico has no regulatory mandate for it, but Mercado Pago recommends enabling it in optional mode, and when the bank asks for a challenge the payment sits in pending with detail pending_challenge. If your integration treats "pending" as "declined", you're cancelling good orders. AMVO lists "payment method rejected" among the barriers to buying online.

4. The problem nobody mentions: cash at OXXO locks your inventory

This is what worries me most for a store with limited stock, and I didn't see it in any Buen Fin guide I read. When a customer chooses to pay cash, the gateway issues a voucher with an expiry date, and until they pay, the item is in limbo:

Gateway	Voucher validity	If they pay late
Stripe (OXXO)	5 days by default, configurable from 1 to 7; confirmation arrives the next business day	The voucher expires (docs)
Mercado Pago (OXXO, Paycash)	Configurable from 1 to 30 days; they recommend 3. Crediting takes up to 2 business hours	"The amount will be refunded" (docs)
Conekta	Set by your integration with `expires_at`; signalled by the `order.expired` webhook	Up to your integration (docs)

Now the WooCommerce side. Its "Hold stock" setting reserves items for orders pending payment for as long as you configure (60 minutes is the recommended value) and then cancels them, according to its guide. But the same guide says delayed payment methods leave the order on hold, where that automatic cancellation doesn't apply. If your gateway plugin doesn't cancel the order when the voucher expires, the item stays reserved for days by someone who may never pay, while you tell the customer holding a card that it's sold out.

Before the 13th:

Shorten voucher validity for the campaign. One or two days instead of five or thirty.
Check that your plugin cancels the order on expiry. Place a test cash order with the minimum validity, don't pay it, and check the next day that the order was cancelled and the stock came back.
If you sell over WhatsApp with payment links, give them a short expiry. Clip, for example, lets a link expire the same day.

5. Overselling: two people buy the last unit

Since version 4.3, WooCommerce reserves stock during checkout to stop two simultaneous purchases taking the same unit (PR #26395). Even so, issue #44273, where the team confirms that "a race condition like this is possible", is still open today. On a normal day it almost never happens. During Buen Fin, with hundreds of people on your best seller, is when it does.

For low-stock products: keep a buffer (list 18 if you have 20), cap quantity per customer, and decide in advance what you'll do if it happens, with an apology and a fast refund. Refused delivery was among the complaints that dominated Profeco's 2025 figures. Shopify's flash sale guide recommends switching to manual payment capture for the same reason: charge only for what you can actually ship.

6. What runs in the background

Order confirmation emails, payment webhooks and stock updates in WooCommerce go through Action Scheduler, which by default processes batches of 25 actions for 30 seconds, triggered by WP-Cron. And WP-Cron only runs when someone visits the site. With caching done right, hardly anyone reaches the server, and tasks can pile up. The result: customers who paid and don't get their confirmation, and message you asking whether the order went through.

The fix is a real server cron instead of the visit-driven one: set DISABLE_WP_CRON in wp-config.php and schedule a call to wp-cron.php every minute from your hosting panel. Almost every shared host allows it.

7. Test it first with fake traffic

All of the above can be checked without waiting for the 13th. A load test simulates many visitors at once and shows you where errors start. Two free options:

k6, by Grafana: free on your own machine, and its free cloud tier includes 500 virtual user hours a month.
Loader.io: the free plan allows 10,000 clients per test for one minute, run from US data centres.

A minimal k6 script that visits the home page and your best seller:

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '2m', target: 50 }, // ramp to 50 concurrent visitors
    { duration: '5m', target: 50 }, // hold
    { duration: '1m', target: 0 },
  ],
  thresholds: {
    http_req_failed: ['rate<0.01'],     // under 1% errors
    http_req_duration: ['p(95)<2000'],  // 95% under 2 seconds
  },
};

export default function () {
  check(http.get('https://tutienda.mx/'), { home: (r) => r.status === 200 });
  sleep(3);
  check(http.get('https://tutienda.mx/producto/tu-mas-vendido/'), { product: (r) => r.status === 200 });
  sleep(5);
}

Run it with k6 run test.js. Three rules: test pages, never your payment gateway; on shared hosting, tell your provider first, because a load test looks a lot like an attack; and aim at three to five times your normal peak hour. There's no public figure for how much a small store's traffic rises during Buen Fin compared with an ordinary day; if your analytics kept last year's, use that. If the test fails at 50 visitors, you know what to fix, and you have seven weeks to do it.

If your store runs on Shopify or Tiendanube

Infrastructure isn't your problem: Shopify handled 489 million requests per minute at its edge over Black Friday and Cyber Monday 2025, and its guide says special preparation is only needed if you expect "tens of thousands of customers" starting checkout within minutes. Yours are the other sections: theme weight and the apps you load on every page, the gateway and its pending states, cash voucher validity, and your stock buffer.

The two-weeks-before checklist

What	How you verify it
HTML caching on for browsers	`cf-cache-status: HIT` on the second request to a product page
No page sends `Set-Cookie` to visitors	The same `curl -sI` as above
Cart, checkout and account excluded from cache	Add something to the cart in another window and check the counter updates
New prices visible	Purge the cache when offers go live and check in a private window
Short cash-voucher validity	Test OXXO order you don't pay: on expiry, order cancelled and stock back
"Pending" payments treated as pending	Test purchase with 3-D Secure that you don't complete
Real server cron	`DISABLE_WP_CRON` in `wp-config.php` and a task every minute
Buffer on best sellers	Listed stock below real stock
SSL certificate valid past November 17	Domain health report (HTTPS, redirect and HSTS)
Load test passed	k6 or Loader.io with no errors at three times your peak
Plan B for payments	Payment link and WhatsApp ready in case the gateway or the bank fails

If you need a store or site in Mexico built to survive a date like this, that's what I do.

Frequently asked questions

When is Buen Fin 2026?

From Friday November 13 to Tuesday November 17, 2026. It is the 16th edition and runs five days because it includes Monday November 16, the observed Revolution Day holiday. Business registration is free at elbuenfin.org, from September 8 to November 12.

Why do online stores crash during Buen Fin?

Usually because every visit reaches the server uncached and the host runs out of the processes it allows. Shared plans have fixed limits (20 PHP workers on Hostinger's entry plan, 25 processes per cPanel account on HostGator Mexico) and return a 503 error past them. The payment gateway can also fail when it rate-limits, or the customer's bank, which is out of your hands.

Does Cloudflare's free plan protect a store from Buen Fin traffic?

It helps, but not just by switching it on. Cloudflare does not cache HTML by default, so without a Cache Rule every page still hits your host. The free plan allows up to 10 rules, and one is enough to cache pages for browsers while excluding cart, checkout and account. The virtual queue (Waiting Room) only exists on Business and Enterprise plans.

Does a pending OXXO payment lock my inventory?

It can, for days. Stripe OXXO vouchers expire after 5 days by default (configurable from 1 to 7) and Mercado Pago's after 1 to 30. In WooCommerce, delayed-payment orders go on hold and the 60-minute hold-stock setting only cancels orders pending payment, so the item stays reserved until the gateway plugin cancels the order when the voucher expires. Shorten validity for the campaign and test it with an order you do not pay.

How do I load test an online store?

With k6, free and open source, or Loader.io, whose free plan runs up to 10,000 clients in a one-minute test. Simulate visitors on the home page and best sellers, never on the payment gateway, aim at three to five times your peak hour, and warn your shared hosting provider first, because a load test looks like an attack.

Originally published on ortamarco.me.

Laravel 13: What Actually Breaks When You Upgrade (and What Doesn't)

Marco Orta — Mon, 21 Sep 2026 08:29:07 +0000

Laravel 12 stopped receiving bug fixes on August 13, 2026. Since then only security patches remain, and those run out on February 24, 2027. If you have a production app on 12, you are already in the last window.

The good news is that Laravel 13 really is a small upgrade. The official docs estimate 10 minutes and say most applications can upgrade "without changing much application code." That is true.

The bad news is the nuance almost nobody is covering: minimal breaking changes is not zero breaking changes. The official upgrade guide flags two high-impact changes, two medium-impact ones, and roughly fifteen low-impact ones. And if your application caches PHP objects, uses upsert on MySQL, or excludes the CSRF middleware in tests, one of them is going to bite.

This article is the complete list, with the concrete code for each change. If what you need is the general upgrade procedure — backup, branch, composer update, verification — that lives in the guide to upgrading Laravel to the latest version; here we go straight to what breaks.

First: where you are and how urgent this is

This is the official support table, not the one making the rounds:

Version	Supported PHP	Released	Bug fixes until	Security until
Laravel 13	8.3 – 8.5	Mar 17, 2026	Q3 2027	Mar 17, 2028
Laravel 12	8.2 – 8.5	Feb 24, 2025	Aug 13, 2026	Feb 24, 2027
Laravel 11	8.2 – 8.4	Mar 12, 2024	Sep 3, 2025	Mar 12, 2026 ❌
Laravel 10	8.1 – 8.3	Feb 14, 2023	Aug 6, 2024	Feb 4, 2025 ❌

Two things worth reading carefully in that table:

Laravel 11 has received no security patches since March 12, 2026. If you're there, you aren't "a little behind" — you're uncovered.
Laravel 12 is not LTS. Laravel stopped shipping LTS releases years ago; the current policy is uniform across every major — 18 months of bug fixes and 2 years of security.

flowchart TB
    A["Which version are you on?"] --> B["Laravel 12"]
    A --> C["Laravel 11"]
    A --> D["Laravel 10 or older"]
    B --> B1["✅ Jump straight to 13<br/>Read this list and do it"]
    C --> C1["⚠️ No security patches<br/>since Mar 12, 2026"]
    C1 --> C2["Go 11 → 12 → 13,<br/>one major at a time"]
    D --> D1["🚨 Fully end of life"]
    D1 --> C2

Entry requirement: PHP 8.3

Laravel 13 requires PHP 8.3 minimum and supports up to 8.5. No more, no less: if you read somewhere that "in practice it needs 8.4," that's wrong — 8.3 works.

php -v          # you need 8.3.0 or higher
composer -V     # and an up-to-date Composer 2.x

If you're coming from Laravel 12 on PHP 8.2, this is the only real infrastructure work in the upgrade. Do it before touching composer.json, not at the same time, so that if something breaks you know what broke it.

High impact

There are two. Both affect practically every application.

1. Dependencies in `composer.json`

No surprises, but there are more packages than people remember:

{
  "require": {
    "php": "^8.3",
    "laravel/framework": "^13.0",
    "laravel/tinker": "^3.0"
  },
  "require-dev": {
    "laravel/boost": "^2.0",
    "phpunit/phpunit": "^12.0",
    "pestphp/pest": "^4.0"
  }
}

The two most commonly forgotten are laravel/tinker to ^3.0 and the jump to PHPUnit ^12.0. If you use Pest, you need ^4.0.

And if you installed the global Laravel installer:

composer global update laravel/installer

With Herd, updating Herd is enough.

2. The CSRF middleware changes name and behavior

This is the significant change in the release, and the one most likely to quietly break your tests.

VerifyCsrfToken is renamed to PreventRequestForgery, and it isn't just a rename: it now performs origin verification by reading the browser's Sec-Fetch-Site header, on top of the usual token validation.

use Illuminate\Foundation\Http\Middleware\PreventRequestForgery;
use Illuminate\Foundation\Http\Middleware\VerifyCsrfToken;

// Laravel <= 12.x
->withoutMiddleware([VerifyCsrfToken::class]);

// Laravel >= 13.x
->withoutMiddleware([PreventRequestForgery::class]);

VerifyCsrfToken and ValidateCsrfToken still exist as deprecated aliases, so your app won't explode on day one. But:

Referencing them directly stops being a good idea, especially when excluding middleware in tests or route definitions.
The middleware configuration API now exposes preventRequestForgery(...).

Where this really bites is test suites that call withoutMiddleware([VerifyCsrfToken::class]) expecting to disable the protection: because the alias points elsewhere, the exclusion may not apply the way you assumed. Grep for it before upgrading:

grep -rn "VerifyCsrfToken\|ValidateCsrfToken" app/ tests/ routes/ bootstrap/

Since you're hardening request forgery protection anyway, this is a good moment to check which security headers your domain is actually returning in production:

Free tool: HTTP Security Headers — Analyse a site's HTTP security headers (HSTS, CSP, X-Frame-Options and more) and get a grade plus the ones you are missing.

Medium impact

3. `serializable_classes` in the cache config

The default cache configuration now includes serializable_classes => false. It's a hardening measure against deserialization gadget chain attacks: if your APP_KEY leaks, without this protection an attacker could inject serialized PHP objects into the cache and chain magic methods until they achieve code execution.

The price is that if your application deliberately stores PHP objects in cache, it stops working unless you explicitly declare which classes may be unserialized:

// config/cache.php
'serializable_classes' => [
    App\Data\CachedDashboardStats::class,
    App\Support\CachedPricingSnapshot::class,
],

If you were caching arbitrary objects, you have two ways out: an explicit allow-list like the one above, or migrating to object-free payloads (plain arrays). The second is more work and the better idea.

Watch out for one detail: these errors don't show up at deploy time, they show up the first time someone reads a cache entry that contained an object. Which means in production, a while later. Flush the cache on deploy and cover it with a test.

4. `upsert` with MySQL or MariaDB

Laravel now validates that uniqueBy isn't empty, and throws InvalidArgumentException instead of generating invalid SQL:

// Laravel >= 13.x: this now blows up with InvalidArgumentException
DB::table('stats')->upsert($rows, [], ['views']);

The curious part is that the MySQL and MariaDB drivers ignore the value of uniqueBy — they always use the table's primary and unique indexes to detect existing records. The validation applies regardless. So if your code passed an empty array because "MySQL ignores it anyway," you now have to pass the real columns.

Low impact (the ones that will actually bite)

This is where the real value is, because nobody lists these and they're the ones that surface three weeks after the deploy.

5. Cache prefixes and session cookie name

The default prefixes move from underscore to hyphen:

// Laravel <= 12.x
Str::slug((string) env('APP_NAME', 'laravel'), '_').'_cache_';
Str::slug((string) env('APP_NAME', 'laravel'), '_').'_database_';
Str::slug((string) env('APP_NAME', 'laravel'), '_').'_session';

// Laravel >= 13.x
Str::slug((string) env('APP_NAME', 'laravel')).'-cache-';
Str::slug((string) env('APP_NAME', 'laravel')).'-database-';
Str::slug((string) env('APP_NAME', 'laravel')).'-session';

It only affects you if you don't have those values defined in your configuration and rely on the framework fallback. The symptom is ugly and confusing: the entire cache is invalidated at once and every active session is logged out the moment you deploy, because the keys and the cookie name changed.

To avoid it, pin the values explicitly before upgrading:

CACHE_PREFIX=myapp_cache_
REDIS_PREFIX=myapp_database_
SESSION_COOKIE=myapp_session

6. `JobAttempted` now exposes the exception

The event swaps a boolean property for the exception object:

// Laravel <= 12.x
$event->exceptionOccurred;   // bool

// Laravel >= 13.x
$event->exception;           // Throwable|null

It's a change for the better — now you know what failed, not just that it failed — but any listener reading exceptionOccurred stops working. And since it's a boolean read inside an if, the failure is silent: null is falsy, so your listener simply stops reacting.

7. `QueueBusy`: `$connection` becomes `$connectionName`

A pure rename, for consistency with the rest of the queue events. If you monitor queues with your own listener, update it.

grep -rn "QueueBusy" app/

8. `Container::call` respects nullable defaults

$container->call(function (?Carbon $date = null) {
    return $date;
});

// Laravel <= 12.x: a Carbon instance
// Laravel >= 13.x: null

It now matches the constructor injection behavior introduced in Laravel 12. If you had code relying on the container resolving the class despite the = null, the result changes without warning.

9. Domain route precedence

Routes with an explicit domain are now evaluated before routes without one. This fixes the catch-all subdomain case, which previously depended on registration order.

If you do multi-tenancy by subdomain, or you declare domain routes after general ones, review your matching. It's one of the few changes on this list that can alter which controller handles a request.

10. `DELETE` with `JOIN`, `ORDER BY`, and `LIMIT` on MySQL

Previously, in a delete with a join, the ORDER BY and LIMIT clauses were silently ignored. Now they're compiled and sent to the database.

In practice: code you believed was bounded (->limit(100)) but was actually deleting without a limit can now throw QueryException, because MySQL and MariaDB don't support that syntax in joined deletes.

It's an awkward change but a healthy one — it was lying to you. If you run into it, inspect the generated SQL before touching anything:

Free tool: SQL Formatter / Beautifier — Format and beautify your SQL code with ease.

11. Polymorphic pivot table names

When inferring the table name for a polymorphic pivot model with a custom class, Laravel now pluralizes. If you relied on the singular inferred name, declare the table explicitly on the pivot:

class Taggable extends MorphPivot
{
    protected $table = 'taggable';   // this used to be inferred
}

12. Collection serialization restores eager-loaded relations

When a model collection is serialized and restored — the usual case in queued jobs — relations loaded with with() are now present again after deserialization.

That's usually what you want. But if your job counted on the relations not being there (to force a fresh reload from the database, say), it now works with potentially stale data. That bug is especially treacherous because nothing fails: it just processes old data.

13. `Str` factories reset between tests

Laravel now clears custom Str factories during test teardown. If you defined a deterministic UUID or ULID generator once — in a setUpBeforeClass or a global helper — and expected it to persist across methods, you now have to define it in each test or in setUp.

Symptom: tests that pass in isolation and fail in the suite.

14. `extend` callbacks are bound to the manager

Custom driver closures registered via extend now have the manager as $this, not whatever was there before. If you used $this inside the closure expecting the service provider, move it into a capture:

// Before: $this was the service provider
Cache::extend('mydriver', function ($app) {
    return $this->createStore($app);      // no longer works
});

// Now
$provider = $this;
Cache::extend('mydriver', function ($app) use ($provider) {
    return $provider->createStore($app);
});

15. Bootstrap 3 pagination view names

// Laravel <= 12.x
pagination::default
pagination::simple-default

// Laravel >= 13.x
pagination::bootstrap-3
pagination::simple-bootstrap-3

Only relevant if you reference those view names directly. The old names were confusing — default wasn't the actual default — and now they're explicit.

16. `Js::from` stops escaping Unicode

It now uses JSON_UNESCAPED_UNICODE by default. If you have tests or output comparisons expecting è instead of è, update the expectations. Any accented character triggers this.

17. The PHP 8.5 polyfill and global helpers

This is the easiest one to miss. Laravel 13 adds a dependency on symfony/polyfill-php85, which on PHP below 8.5 defines global functions such as array_first() and array_last() if they don't already exist.

The conflict: the historical array_first() from laravel/helpers accepted a callback to return the first element matching a condition. The polyfilled one only returns the array's first element. Same apparent signature, different behavior.

If you still carry laravel/helpers or your own global helpers with those names, always use Arr:

use Illuminate\Support\Arr;

Arr::first($array, fn ($value) => $value->active);

Very low impact: only if you maintain packages

If you implement framework contracts yourself, there are new methods to add. If you only consume Laravel, skip this section entirely.

Contract	Method to add
`Illuminate\Contracts\Cache\Store`	`touch($key, $seconds)`
`Illuminate\Contracts\Bus\Dispatcher`	`dispatchAfterResponse($command, $handler = null)`
`Illuminate\Contracts\Routing\ResponseFactory`	`eventStream(...)`
`Illuminate\Contracts\Auth\MustVerifyEmail`	`markEmailAsUnverified()`
`Illuminate\Contracts\Queue\Queue`	`pendingSize`, `delayedSize`, `reservedSize`, `creationTimeOfOldestPendingJob`

On top of that, instantiating a model while that model is still booting now throws LogicException:

protected static function boot()
{
    parent::boot();

    (new static())->getTable();   // LogicException on Laravel 13
}

And the HTTP client's throw and throwIf signatures now declare their callbacks explicitly, which only matters if you override those methods in a custom response class.

What does NOT break, whatever you may have read

A couple of false claims are circulating about this upgrade:

app/Http/Kernel.php does not disappear in Laravel 13. That file was removed in Laravel 11, with the new application structure. If you're coming from 12 you no longer have it.
Laravel 13 does not require PHP 8.4. The minimum is 8.3 and the supported range goes up to 8.5.
VerifyCsrfToken is not removed. It remains a deprecated alias. Migrating it is wise, but it won't take your deploy down.
There are no "zero breaking changes." The documentation itself says minimal, and flags two high-impact changes. The gap between "minimal" and "zero" is exactly this article.

The upgrade, in practice

With the full list in front of you, the procedure is short:

# 1. New branch, always
git checkout -b upgrade/laravel-13

# 2. PHP 8.3+ before anything else
php -v

# 3. Pin cache and session prefixes so you don't invalidate everything
#    (CACHE_PREFIX, REDIS_PREFIX, SESSION_COOKIE in your .env)

# 4. Grep for what you know changes
grep -rn "VerifyCsrfToken\|ValidateCsrfToken" app/ tests/ routes/ bootstrap/
grep -rn "exceptionOccurred\|QueueBusy" app/
grep -rn "array_first\|array_last" app/

# 5. Update dependencies
composer update --with-all-dependencies

# 6. Clear everything
php artisan optimize:clear

# 7. Tests
php artisan test

Laravel also offers an assisted route: if you have Laravel Boost ^2.0 installed, the /upgrade-laravel-v13 command walks the upgrade from Claude Code, Cursor, OpenCode, Gemini, or VS Code. And for large projects, Shift is still the paid option that automates the bulk of it.

One last step almost nobody takes, and it saves surprises: diff your configuration files against laravel/laravel on the 13.x branch. Many changes on this list — serializable_classes being the obvious one — live in config/, not in the framework, and don't arrive on their own with composer update.

One catch when you copy, though: the 13 skeleton's config/session.php ships 'serialization' => 'json'. If your app was on php and you sync that value as-is, every active session is invalidated on deploy and every user has to log in again. For a seamless upgrade, keep it at php. Moving to json is the recommended hardening against deserialization attacks, but only if you don't store PHP objects in the session, and only when you're fine forcing that re-login.

Free tool: Text Diff Checker — Compare two texts and highlight the differences line by line, with unified and side-by-side views and ignore case or whitespace. All in your browser.

Checklist before you merge

[ ] PHP 8.3 or higher on local, CI, and production
[ ] laravel/framework ^13.0, laravel/tinker ^3.0, phpunit ^12.0 or pest ^4.0
[ ] Zero references to VerifyCsrfToken in tests and routes
[ ] CACHE_PREFIX, REDIS_PREFIX, and SESSION_COOKIE pinned explicitly
[ ] config/cache.php reviewed if you cache PHP objects
[ ] Every upsert() call passes a non-empty uniqueBy
[ ] JobAttempted and QueueBusy listeners updated
[ ] Domain routes reviewed if you use subdomains
[ ] Test suite green as a whole, not just file by file
[ ] Cache flushed on deploy

Conclusion

Laravel 13 is an honest upgrade: 10 minutes for most apps, and the new capabilities — the first-party AI SDK, JSON:API resources, Queue::route(), PHP attributes for middleware and jobs, Cache::touch(), vector search with whereVectorSimilarTo() — arrive without demanding a rewrite in exchange.

But "minimal" is not "zero," and the changes that actually hurt aren't the two high-impact ones — those are documented and surface immediately — but the low-impact ones that fail silently: the cache that invalidates wholesale, the listener that stops reacting, the job that processes stale data, the test that only fails in the suite.

That date has passed: August 13, 2026. Since then, every bug you find in Laravel 12 is yours.

Frequently asked questions

When does Laravel 12 support end?

Laravel 12 stopped receiving bug fixes on August 13, 2026 and stops receiving security patches on February 24, 2027. Laravel applies the same policy to every major version: 18 months of bug fixes and 2 years of security. Laravel 12 is not an LTS release.

Which PHP version does Laravel 13 require?

Laravel 13 requires PHP 8.3 as a minimum and supports up to PHP 8.5. It is not true that it requires PHP 8.4: it works correctly on 8.3. If you are coming from Laravel 12 on PHP 8.2, raising the PHP version is the only real infrastructure work in the upgrade.

What are the high-impact breaking changes in Laravel 13?

There are two. The first is updating dependencies in composer.json: laravel/framework to ^13.0, laravel/tinker to ^3.0, laravel/boost to ^2.0, phpunit/phpunit to ^12.0, and pestphp/pest to ^4.0. The second is that the CSRF middleware VerifyCsrfToken is renamed to PreventRequestForgery and adds origin verification through the Sec-Fetch-Site header. VerifyCsrfToken still exists as a deprecated alias.

Is app/Http/Kernel.php removed in Laravel 13?

No. That file was removed in Laravel 11 with the new application structure, not in Laravel 13. If you are coming from Laravel 12 your project no longer has it and there is nothing to do about it.

Why did my entire cache disappear after upgrading to Laravel 13?

Because the default cache, Redis, and session prefixes changed from underscore to hyphen: from myapp_cache_ to myapp-cache-. It only affects applications that rely on the framework fallback instead of defining those values in their configuration. The effect is that every cache key changes and active sessions are logged out. To avoid it, pin CACHE_PREFIX, REDIS_PREFIX, and SESSION_COOKIE in your .env before upgrading.

How long does upgrading from Laravel 12 to Laravel 13 take?

The official documentation estimates 10 minutes for a typical application. In practice, the time goes into raising PHP to 8.3 if you were on 8.2, reviewing third-party packages that do not yet declare Laravel 13 compatibility, and checking the low-impact changes that fail silently. With Laravel Boost 2.0 installed you can use the /upgrade-laravel-v13 command to have an AI assistant guide the process.

Originally published on ortamarco.me.

The AI model your business runs on is being retired: the 2026 shutdown calendar

Marco Orta — Mon, 21 Sep 2026 08:20:03 +0000

OpenAI already shut down the Assistants API on 26 August 2026. The next cutoffs are 24 September, when it retires the Videos API and sora-2; 28 September, for the last legacy completion snapshots; and 23 October, for gpt-3.5-turbo and gpt-4. If your business runs a chatbot, an email classifier or any AI automation that was built for you in 2023 or 2024, there is a very high chance it is wired to one of those last two models. And on shutdown day it will not degrade gracefully: it will return an error and stop working.

This is not speculation or industry chatter. These are dates published by the providers themselves on their own deprecation pages, with six months' notice. The problem is that the notice arrives by email to the developer account that created the API key — which half the time belongs to an agency that no longer works with you.

This article is the full calendar, how to find out in three minutes whether it affects you, and what to do if it does. If what you need is what each model costs, that lives in the guide to OpenAI, Claude and Gemini API pricing; here we go straight to what shuts down and when.

What nobody told you when you bought it

AI models are not software you buy once. They are a service with an expiry date, and the provider decides the date.

OpenAI puts it plainly in its documentation: "software relying on OpenAI models may need occasional updates to keep working." Anthropic writes almost the same sentence. Translated into the language of the person paying the invoice: the chatbot you were handed as "finished" carries mandatory maintenance, and nobody put it in the quote.

This changes a decision plenty of businesses got wrong. When you asked for a quote to automate something with AI, you compared the implementation price and the monthly token cost. What you did not compare — because nobody showed you — was how often you would be paying for a migration. And on that, as we will see, providers are nothing alike.

First: does this affect you? The three-minute check

Before reading the calendar, find out which model you actually use. Not what you think you use: what is written in the code.

If you have access to the OpenAI dashboard

Sign in to platform.openai.com with the account that pays for the API.
Go to Usage and switch the breakdown to group by model.
Look at the last 30 days. Those names are what you are really running.

If you have access to the Anthropic console

Open the Usage page in the Claude Console.
Click Export.
The CSV gives you usage broken down by API key and by model. Anthropic documents this exact procedure for auditing deprecated models.

If you have access to nothing

This is the most common case, and it is the finding in itself. If nobody at your company can log into the provider's dashboard:

The API key belongs to a third party. The retirement notice goes to them, not to you.
You cannot audit the spend, so you also do not know whether you are overpaying.
You cannot migrate without them, which means the continuity of a business process depends on that person picking up the phone.

All three are governance problems, not technical ones, and they are fixed the same way: the AI provider account is opened in the company's name, paid by the company, and the external developer is added as an invited user. If this is your situation, that is the first job — before any migration.

Free tool: LLM Token Counter & AI Cost Calculator — Count the tokens in your text or prompt and estimate the cost across GPT-5, Claude, Gemini, Kimi K3, Grok and DeepSeek. Exact OpenAI counting, per-call and monthly costs, with prompt caching.

OpenAI's shutdown calendar

These are the dates published on OpenAI's official deprecations page, checked on 16 August 2026 and reconfirmed on 13 September 2026. I have bolded the ones that hit small businesses hardest.

Shutdown	What dies	Recommended replacement
26 Aug 2026 (already happened)	Assistants API	Responses API + Conversations API
24 Sep 2026	`sora-2`, `sora-2-pro`, Videos API	no replacement
28 Sep 2026	`gpt-3.5-turbo-instruct`, `babbage-002`, `davinci-002`, `gpt-3.5-turbo-1106`	`gpt-5.6-terra`
1 Oct 2026	`gpt-5.4-cyber`	`gpt-5.6-cyber`
23 Oct 2026	`gpt-3.5-turbo-0125`, `gpt-4-0613`, `gpt-4-turbo`, `gpt-4-1106-preview`, `gpt-4o-2024-05-13`, `o1`, `o1-pro`, `o3-mini`, `o4-mini`, `gpt-4.1-nano`, `gpt-image-1`	`gpt-5.6-sol` / `gpt-5.6-terra` / `gpt-5.6-luna` (`gpt-image-1` → `gpt-image-2`)
31 Oct 2026	Evals: existing evals become read-only	Promptfoo (OpenAI's migration path)
30 Nov 2026	`v1/prompts` API and reusable prompts, Agent Builder, Evals dashboard and API	Prompts in your own code · Agents SDK or ChatGPT Workspace Agents · Promptfoo
1 Dec 2026	`gpt-image-1-mini`, `gpt-image-1.5`, `chatgpt-image-latest`	`gpt-image-2`
11 Dec 2026	`gpt-5-2025-08-07`, `gpt-5-mini-2025-08-07`, `gpt-5-nano-2025-08-07`, `gpt-5-pro-2025-10-06`, `o3-2025-04-16`, `o3-pro-2025-06-10`	`gpt-5.6-sol` (`reasoning.mode: pro` for the pro ones) / `gpt-5.6-terra` (mini) / `gpt-5.6-luna` (nano)
20 Jan 2027	`gpt-realtime`, `gpt-audio`, `gpt-4o-audio`, `gpt-4o-realtime` and mini variants	`gpt-realtime-2.1` (`gpt-realtime-2.1-mini` for the realtime minis) / `gpt-audio-1.5`
26 Feb 2027	`whisper-1`, `gpt-4o-transcribe`, `gpt-4o-mini-transcribe`, `gpt-4o-transcribe-diarize`	`gpt-transcribe` / `gpt-live-transcribe`

Two of those dates are closer than the headline one, and easy to miss because they read like small print next to it. 24 September retires the Videos API along with sora-2 and sora-2-pro — with no replacement listed, so anything built on video generation needs a different plan, not just a different model string. 28 September retires the last of the old completion-style snapshots (gpt-3.5-turbo-instruct, babbage-002, davinci-002, gpt-3.5-turbo-1106) into gpt-5.6-terra. Neither is wired into as many small-business chatbots as gpt-3.5-turbo or gpt-4 are — but if your stack touches either one, your date is 24 or 28 September, not 23 October.

And a third one shows that six months' notice does not apply to everything: gpt-5.4-cyber was announced on 11 September and shuts down on 1 October, twenty days later. The six-month minimum OpenAI promises covers generally available models, not its whole catalogue, so do not assume it for variants like this one.

Three readings of that table matter more than the table itself:

1. 23 October is the big one. gpt-3.5-turbo and gpt-4-0613 are the models that appear literally in every tutorial from 2023 and 2024. Any integration copied from a blog, a course or the documentation of that era has them hardcoded. And because they worked without complaining for three years, nobody has touched them since. (For what actually changed in the replacements, see the GPT-5.6 launch rundown.) ⚠️ And if the GPT-6 Astra launch on 3 September tempts you to migrate straight there: for a chatbot or a classifier that is an expensive mistake. Astra costs 20x the input and 33x the output of gpt-3.5-turbo, and its million-token window hides a 2x surcharge past 272K. The migration ladder by workload type is in GPT-6 Astra for developers. And if you're the one migrating the code: changing the name isn't enough, because LangChain and LlamaIndex still default to gpt-3.5-turbo and the substitutes reject tools on Chat Completions unless you turn reasoning off. The details, with audit greps, are in what breaks in your code on October 23.

2. Whole APIs are shutting down, not just models. The Assistants API, Agent Builder, the Evals platform and reusable prompts all disappear. That is not fixed by changing a string: it is rewriting the integration. Anyone who built on Assistants is looking at days of work, not minutes.

Reusable prompts are the sneakiest case, because the prompt text is not in your repository: it is stored in the OpenAI dashboard, and your code only references it with prompt: { id: "pmpt_…" } in the Responses API call. On 30 November the v1/prompts API and those prompt objects shut down, so the official migration guide asks you to take the content out of the dashboard, move it into your code and send it as input. Evals also has an intermediate step: from 31 October your existing evals become read-only, a month before the dashboard and API disappear.

3. Pinned snapshots die even when the model "lives on". Look at gpt-5-2025-08-07. GPT-5 is not disappearing as a family, but that specific snapshot is. If your code pins the date — which is the good practice for reproducible results — then you are on the shutdown calendar even while running a model that feels current.

Anthropic's calendar works differently

Anthropic publishes the same thing on its model deprecations page, but with one difference that changes how you plan.

Model	Status	Retirement
`claude-opus-4-1-20250805`	Retired	5 Aug 2026
`claude-opus-4-20250514`, `claude-sonnet-4-20250514`	Retired	15 Jun 2026
`claude-3-7-sonnet-20250219`, `claude-3-5-haiku-20241022`	Retired	19 Feb 2026
`claude-3-haiku-20240307`	Retired	20 Apr 2026
`claude-sonnet-4-5-20250929`	Active	not sooner than 29 Sep 2026
`claude-haiku-4-5-20251001`	Active	not sooner than 15 Oct 2026
`claude-opus-4-5-20251101`	Active	not sooner than 24 Nov 2026
`claude-opus-5`, `claude-sonnet-5`, `claude-fable-5`	Active	2027

Careful not to read those "not sooner than" entries as shutdown dates: they are floors, not announcements. As of 13 September 2026 none of the three Claude 4.5 models has an announced deprecation, and with the 60 days' notice Anthropic commits to, none of them can be retired before 12 November.

The difference is in the notice period, and it is a big one:

Provider	Committed advance notice
OpenAI	6 months for generally available models · 3 months for specialised variants · 2 weeks for preview models
Anthropic	60 days for publicly released models
Google (Gemini)	Its deprecations page sets no notice period: it tracks "the announced earliest shutdown dates"

Sixty days is not much. claude-opus-4-1 was announced on 5 June and switched off on 5 August: two months exactly. If your business slows down in August, or your developer is on holiday, or nobody simply reads that email, the entire notice window passes you by.

This does not make Anthropic a bad choice — its models are excellent, and the retirement pace is the flip side of a very high release pace. It means the notice period is a product feature you should compare before choosing, the same as price or latency. And it appears in none of the comparison posts doing the rounds.

Gemini: two shutdowns between September and October

Google publishes its own on the Gemini API deprecations page, checked on 13 September 2026. Between September and December 2026 there are two dates, plus one that has already passed and is worth checking if your integration dates from 2025:

Shutdown	What dies	Recommended replacement
1 Jun 2026 (already happened)	`gemini-2.0-flash`, `gemini-2.0-flash-001`, `gemini-2.0-flash-lite`, `gemini-2.0-flash-lite-001`	`gemini-3.6-flash` / `gemini-3.1-flash-lite`
30 Sep 2026	`gemini-omni-flash-preview`	`gemini-omni-1.1-flash`
2 Oct 2026	`gemini-2.5-flash-image`	`gemini-3.1-flash-image`

Two details in that table. gemini-omni-flash-preview, the video generation model, launched on 30 June and shuts down three months later: the textbook example of why you do not build a business process on a preview. And for gemini-2.5-flash-image — the original Nano Banana — the official page lists gemini-3.1-flash-image-preview as the replacement, but that preview was already shut down on 25 June 2026 and the same page gives the stable version, gemini-3.1-flash-image (Nano Banana 2), as its replacement, which is where it makes sense to migrate directly.

Gemini 2.5's text models (gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-lite) still have no announced shutdown date.

The trap that catches the most people: where you run it changes the date

This is the detail nobody has written up properly, and the one that produces the most expensive surprises.

The dates above are the original provider's. If you consume the same model through somebody else's cloud, the dates are different.

Anthropic says so explicitly: its dates apply to the Claude API, the Claude Platform on AWS and Microsoft Foundry. Amazon Bedrock and Google Cloud Vertex AI set their own retirement schedules, so the same model's status and dates can differ.
The same is true of OpenAI: Azure OpenAI publishes its own retirements table, which is not always in sync with OpenAI direct.

flowchart TB
    A["Same model,<br/>three different dates"] --> B["Provider's direct API"]
    A --> C["Azure OpenAI / Microsoft Foundry"]
    A --> D["Amazon Bedrock / Google Vertex"]
    B --> B1["Date on the official<br/>deprecations page"]
    C --> C1["Microsoft's own schedule"]
    D --> D1["Cloud operator's<br/>own schedule"]
    B1 --> E["⚠️ Check the table for<br/>WHERE YOU RUN IT,<br/>not the model maker's"]
    C1 --> E
    D1 --> E

The practical consequence: if your integration goes through Bedrock or Azure, the table you need is not the one I just published. It is your cloud's. Setting the alarm for the wrong date is as bad as not setting one.

What else breaks, and it isn't a model

Beyond the shutdowns, there are silent retirements that break code without any model dying.

The live example right now: Anthropic deprecated the temperature, top_p and top_k parameters as of Claude Opus 4.7. It does not ignore them — it returns a 400 error if you send them with anything other than the default value. Any integration that tuned the model's "creativity" with temperature: 0.7 — which is practically every integration from 2023–2024 — stops working the moment you move to a current model.

This matters because it dismantles the naive plan. Plenty of people assume migrating means swapping "gpt-4" for "gpt-5.6-sol" and done. Sometimes it is. But you may also be facing:

Parameters that are no longer accepted (the case above).
Different response formats, if you used structured output.
Entire APIs rewritten (Assistants → Responses).
Prompts that stop performing the same. A new model is not the old one but better: it is a different one. Prompts tuned by trial and error against gpt-4 can do worse on gpt-5.6-sol until they are readjusted.

That last point eats the most budget and is the one that never gets quoted.

The migration procedure

If you have a technical team, this is the order that works. If you do not, skip to the next section.

1. Inventory before touching anything

Search the whole codebase for model strings, not just the obvious file:

# OpenAI models hardcoded in the code
grep -rn "gpt-3.5-turbo\|gpt-4-0613\|gpt-4-turbo\|gpt-4o-2024" \
  --include="*.php" --include="*.js" --include="*.ts" --include="*.py" .

# Prompts stored in the OpenAI dashboard (v1/prompts shuts down 30 Nov 2026)
grep -rn "pmpt_\|prompt_id" .

# Anthropic models
grep -rn "claude-3\|claude-sonnet-4-2025\|claude-opus-4-2025" .

# And what nearly everyone forgets: environment variables and config
grep -rn "MODEL" .env .env.example config/ 2>/dev/null

Also check the places where the model name is not in the repository at all: n8n or Make panels, serverless functions, cron jobs, and any third-party automation consuming your key.

This kind of break stays invisible until the exact day for a specific reason: a hardcoded model ID needs no validation to keep working. It succeeds every single time you call it, for as long as the model is alive, then fails on the very next call after retirement. There is no warning header on the 200 responses the week before, no gradual dip in quality to notice — the API goes straight from working to a 400 or 404 the moment the shutdown date arrives. If nothing in your code specifically catches that error, the person using your product just sees something break, with nothing in the interface pointing at the real cause.

That is also why it is worth pulling your API logs now, not only grepping your code. A spike in 400 or 404 responses from api.openai.com or api.anthropic.com around a retirement date is the model dying in production — and it is usually a customer who reports it first, not a dashboard.

2. Pin the version, don't use the alias

If your code says plain "gpt-4", you are using an alias the provider repoints whenever it likes. That looks convenient and is exactly the opposite: it means your system's behaviour can change on a Tuesday without you touching anything.

The correct practice is to pin the dated snapshot and control yourself when you move up. Yes, that puts you on the shutdown calendar — but it puts you there predictably, which is precisely what you want.

3. Compare side by side before switching

Do not swap the model in production and wait to see what happens. Take 30 or 50 real cases from your history, run them through the old model and the new one, and compare outputs side by side. It is an afternoon's work and it is the difference between a migration and an incident.

Free tool: Text Diff Checker — Compare two texts and highlight the differences line by line, with unified and side-by-side views and ignore case or whitespace. All in your browser.

4. Put the provider behind your own layer

This is the work that makes the next shutdown boring. Instead of calling the provider's SDK from twenty places, call it from one:

// A single place where the decision of which model to use lives.
// The rest of the application asks for "reasoning" or "classification",
// not for "gpt-5.6-sol".

const MODELS = {
  reasoning:      { provider: 'openai',    model: 'gpt-5.6-sol' },
  classification: { provider: 'openai',    model: 'gpt-5.6-terra' },
  summarisation:  { provider: 'anthropic', model: 'claude-sonnet-5' },
} as const;

export async function complete(task: keyof typeof MODELS, prompt: string) {
  const { provider, model } = MODELS[task];
  return provider === 'openai'
    ? callOpenAI(model, prompt)
    : callAnthropic(model, prompt);
}

With that in place, the next shutdown is editing one line of an object and re-running step 3's comparison. Without it, it is find-and-replace across twenty files and hoping.

You do not need a framework for this. LangChain or LlamaIndex solve the same problem and bring far more baggage than a small business needs; thirty lines of your own usually do the job without adding a dependency that can also be deprecated.

If you have no technical team: what to demand, and from whom

For the owner or operations lead who will not be touching code, the job is different — and more important.

What to ask, in writing, from whoever maintains your integration:

Which exact model each process uses, with the full name and snapshot date. "We use ChatGPT" is not an answer.
Which platform it runs on — direct API, Azure, Bedrock, Vertex — because the real date depends on it.
The shutdown date for each one, taken from that platform's table.
Who receives the deprecation emails. If it is an account you do not control, change that this week.
What the migration costs and who pays for it. This is the uncomfortable question, and it needs asking now, not in October.

And what belongs in the contract from here on: deprecation maintenance is not an enhancement or a nice-to-have — it is what keeps the system switched on. Either it is covered by a retainer, or it is priced up front. Discovering it on shutdown day, with the process already down, is the worst negotiating position available.

I say this from the uncomfortable side of the table: I make a living building these integrations. And even so, a client who does not understand that this carries maintenance is a client who will feel ripped off a year from now — rightly.

What the switch costs (and why you sometimes come out ahead)

Migrating is not always a net expense. Newer models tend to be considerably cheaper per token than the ones they replace, and considerably more capable, so three effects offset each other:

In your favour: the replacement's price per million tokens is usually lower. A process that cost X on gpt-4 can cost a fraction on its successor.
In your favour: more capable models need shorter prompts for the same quality, and a shorter prompt is fewer tokens.
Against you: reasoning models burn internal tokens that you also pay for. Switching to one of those by accident can multiply the bill even as the unit price drops.

The only honest way to know is to measure it with your real data before switching. The numbers and the method are in the OpenAI, Claude and Gemini API pricing guide, and if you want to estimate a specific prompt's consumption before sending it, the token counter alongside does it with no signup.

Free tool: Tokenizer Visualizer — Paste text and see exactly which tokens a language model splits it into, with their IDs. Check why Spanish costs more tokens than English.

The radical way out: make it un-switchoffable

There is one way to make this article never apply to you: run the model yourself.

An open model running on your own hardware cannot be retired by anyone. It keeps working exactly the same three years from now, with the same outputs, no deprecation emails and no date on the calendar. For stable internal processes — classifying email, extracting data from documents, summarising — it is a real option, and today it runs on affordable hardware.

It is not free: you pay in hardware, in electricity, and in quality that trails the frontier models. I wrote the full analysis, with the machine I use and measured numbers, in AI without sending data outside: a local LLM for small firms. The continuity section of that article is exactly this problem seen from the other side.

For most people the sensible call is not picking a side but splitting: provider models for what needs frontier quality, your own model for the stable processes you would rather not tie to anybody's calendar.

Frequently asked questions

What exactly happens on shutdown day?

The request fails. There is no degradation and no automatic substitution: the model stops being reachable and the API returns an error. Anthropic says it literally — "requests to retired models will fail" — and OpenAI that the model or endpoint "will no longer be accessible." If your application does not handle that error, what the end user sees is a broken screen.

I use ChatGPT in the browser, not the API. Does this affect me?

Far less. The dates in this article are for the API, which is what integrations consume. In the ChatGPT app, retired models are swapped for the current one automatically and at most you notice it answers differently. The serious risk is to anything automated.

How do I know if my chatbot uses the Assistants API?

Ask whoever built it, and if you cannot, check the OpenAI dashboard: if assistants appear under Assistants, it is in use. The code gives it away too — it will call endpoints under /v1/assistants or /v1/threads. If you have not migrated yet, that one is no longer a countdown: the Assistants API already shut down on 26 August 2026, so a chatbot still calling it is already broken, not about to be.

Is migrating just changing the model name?

Sometimes yes, sometimes no. Swapping the string is enough if you only used plain text generation. But you may need to rewrite if you used the Assistants API, if you send already-deprecated parameters — temperature returns a 400 error on current Claude models — or if your prompts were tuned against the old model. Budget for the output comparison, which is the part that actually takes time.

How often will this happen again?

Permanently. OpenAI gives at least 6 months' notice for generally available models; Anthropic, 60 days. At that pace, the realistic plan is a model review every six months and a small migration once a year. It is recurring maintenance, like upgrading the framework or renewing an SSL certificate.

Does using LangChain or a chatbot SaaS protect me?

Partly, with caveats. An abstraction layer makes the switch cheaper, but it does not stop the underlying model dying or remove the need to revalidate quality. And if the middleman is a SaaS, you inherit its calendar and the risk that it deprecates features of its own — which is exactly what OpenAI did with Agent Builder, its own product, retiring on 30 November 2026.

Conclusion: put it in the calendar today

Boiled down to what to do this week:

Find out which model you actually use. OpenAI dashboard grouped by model, or the CSV export from the Claude console. Three minutes.
Check which platform it runs on, because that is what determines the real date.
Set an alarm for every date that applies to you: 24 September 2026 if you use Sora or the Videos API; 28 September if you still call old completion-style models; 30 September or 2 October if you generate video or images with Gemini (gemini-omni-flash-preview, gemini-2.5-flash-image); 23 October if any gpt-3.5-turbo or gpt-4 shows up in your inventory; 31 October and 30 November if you use Evals, Agent Builder or prompts stored in the dashboard; 1 December if you generate images with gpt-image-1.5 or gpt-image-1-mini; and 11 December if you pin gpt-5 or o3 snapshots. (If you were still calling the Assistants API, that alarm already went off on 26 August — check today whether it is broken.)
If your code calls the Responses API with prompt: { id: "pmpt_…" }, copy that prompt into your repository. The text lives in the OpenAI dashboard, not in your code, and on 30 November 2026 the v1/prompts API and reusable prompts shut down. The official guide says to move the content into your code and send it as input. If the account belongs to your agency, ask them for it now, not on 30 November.
Verify who receives the deprecation emails. If it is not somebody at your company, fix that before anything else.
And ask what the migration costs, while there is still room to negotiate it calmly.

Model shutdowns are not an exotic technical problem: they are ordinary maintenance for a technology moving fast. What is a problem is finding out about them on the day the chatbot stops answering.

If you have an AI integration in production and cannot answer question 1 on that list, get in touch and we will review it. Auditing which models you run and when they die is a short job, and it is better done weeks before the shutdown than the day after.

Originally published on ortamarco.me.

Node 26: What Breaks When You Upgrade, and When the Jump Is Worth It (LTS Guide)

Marco Orta — Mon, 21 Sep 2026 08:19:46 +0000

Node 26 shipped on May 5, 2026 and enters LTS on October 28. It's a calm upgrade — the list of things that actually break is short, and almost all of it has been warned about for years — but it has one property no previous version had: it's the last release of the two-per-year model. Starting with Node 27, the project moves to one release per year, and every release is LTS. The odd/even distinction that spent a decade deciding which version you could run in production disappears.

That makes this upgrade more than a version bump: it's the last train on the old schedule, and it's worth boarding knowing what's inside. Here's what really breaks, sorted by impact, and at the end the full calendar so you can decide when to move.

First: where are you coming from?

The effort depends on your starting point:

From Node 24 (the current LTS): the comfortable scenario. The incompatible changes are few and very localized; for most projects the migration is checklist work, not refactoring work.
From Node 22: still reasonable, but with more fronts — you absorb everything that changed in 24 (V8, permissions, minimum glibc) plus what's new in 26. Node 22 leaves maintenance in April 2027, so you have a real deadline.
From Node 25: odd releases were never for production, and this was the last one that will ever exist. Jump to 26 without thinking twice.

The schedule change: Node 27 debuts a new model

The official project announcement sums it up:

Aspect	Through Node 26	From Node 27 on
Major releases per year	2	1
LTS versions	Even-numbered only	All of them
Short-lived odd versions	Yes	Gone
Alpha channel	Didn't exist	6 months, allows semver-major changes
Total support per version	36 months	36 months (unchanged)

The concrete dates that affect you:

Node 26: Active LTS from October 28, 2026, maintenance from October 20, 2027, end of life on April 30, 2029.
Node 27: alpha channel opens October 28, 2026 (the same day 26 goes LTS), release on April 22, 2027, and it will also be LTS.
Node 24: Active LTS until October 20, 2026, when it moves to maintenance (critical and security fixes only); end of life on April 30, 2028.

The exact dates come from the schedule.json in Node's release repository, the file the project itself works from; the new schedule was announced on the Node.js blog.

The practical consequence: you'll no longer have to memorize which versions "count". Every release from here on is production-grade, and one release per year means fewer migrations per decade for your team. If you maintain libraries, the part that concerns you is the alpha channel: six months to test breaking changes before each release ships.

High impact

1. The internal `_stream_*` modules are gone

The removal with the most shrapnel, because it doesn't break your code: it breaks old dependencies. The internal modules _stream_readable, _stream_writable, _stream_duplex, _stream_transform, _stream_passthrough and _stream_wrap — deprecated since Node 12 — have been removed entirely. Any require('_stream_readable') now throws MODULE_NOT_FOUND.

No package maintained in recent years does that, but ecosystems drag fossils along. To find out whether you have one:

grep -rn "_stream_" node_modules --include="*.js" -l | head

If something shows up, you have two exits: update that package to a version using the public API (node:stream), or — if it's abandoned — replace it. The readable-stream polyfill covers most cases where the package is yours and you just need a mechanical migration.

2. Undici 8: the global `fetch` gets strict

Node 26 ships Undici 8.0, the implementation behind the global fetch. It brings stricter header validation and changes to redirect handling. If your code builds requests with dynamically generated headers — from a database, from user input, from another API — what used to pass silently can now throw a TypeError.

The pattern that shows up most in practice:

// If any value carries \n or \r (e.g. a token copied with a trailing newline),
// Undici 8 rejects it instead of silently sanitizing.
const res = await fetch(url, {
  headers: { Authorization: `Bearer ${token}` },
});

The right fix is not catching the error: it's sanitizing the value at the edge, where it enters your system (token.trim() when you read it). Having the request blow up before it leaves is the good behavior — with Undici 7 that token with a \n would travel and fail at the server on the other side, which is far harder to debug.

3. Native addons: mandatory recompilation

NODE_MODULE_VERSION moves to 147, so every compiled addon (bcrypt, sharp, better-sqlite3, canvas…) needs to be rebuilt or ship prebuilds for the new ABI. The error is unmistakable:

Error: The module was compiled against a different Node.js version using NODE_MODULE_VERSION 137.

npm rebuild solves it locally; in CI and Docker, rebuilding the image on the new base is enough. The detail that can actually stall you: the build requirements went up — GCC 13.2 minimum, and Python 3.9 support was dropped from the build toolchain. If you compile addons inside an old Debian/Alpine image, update the image before fighting the wrong error.

Free tool: Docker Compose Validator — Validate your compose.yaml against the official Compose schema and catch key typos, wrong types and YAML errors, with the exact line. All in your browser.

Medium impact

4. `module.register()` is runtime-deprecated

The async module-hooks API — the one used by instrumentation loaders, on-the-fly transpilation and APM agents — now emits a runtime deprecation. The designated replacement is the synchronous module.registerHooks() API.

This reaches you mostly through third parties: observability agents (Datadog, New Relic, OpenTelemetry) and tools like tsx live in that layer. The action item isn't rewriting anything of yours: it's updating those dependencies to the versions that already migrated before moving production to 26.

5. `writeHeader()` removed from the HTTP server

http.Server.prototype.writeHeader() was an undocumented alias of writeHead(). It's gone. If it appears in your code, the fix is literal, letter for letter:

// Before
res.writeHeader(200, { 'Content-Type': 'application/json' });
// Now
res.writeHead(200, { 'Content-Type': 'application/json' });

A grep -rn "writeHeader" src/ tells you in seconds whether you're affected. Frameworks (Express, Fastify, Nest) have always used the documented API; this bites hand-written HTTP code from many years ago.

6. `type: "module"` packages: the extensionless-CJS exception is gone

In packages declared as ESM, loading a CommonJS file without an extension relied on a historical resolver exception that Node 26 removes. If a "type": "module" package of yours had a require of a path without an explicit .cjs/.js, it now fails. The fix is adding the extension — which is what ESM resolution always asked for.

Low impact

localStorage with no backing file returns undefined instead of throwing. If you probed availability with try/catch around localStorage, invert the check: it's a plain if now.
QuotaExceededError becomes a DOMException-derived interface. Only affects code comparing the error by constructor or numeric code.
Several crypto APIs advance in the deprecation cycle — one reaches end of life and others start warning at runtime. If you see no warnings with --pending-deprecation today, you're not affected.
assert accepts printf-style messages in assertion errors. It's an addition, but it changes text output: if your CI parses assert messages (bad idea, while you're here), review it.

What does NOT break, even if you've read otherwise

The Temporal API doesn't break Date. Temporal arrives enabled by default in Node 26, but it's an addition: all your Date code keeps working exactly as (badly as) before. Migrating is optional and gradual — I have a full guide to Temporal in Node 26 with the conversion table from date-fns and dayjs.
Running TypeScript directly is now stable. node app.ts works with no flags: type stripping left experimental status. The only thing that "breaks" is the --experimental-transform-types flag, which was removed precisely because the feature graduated — if you had it in an npm script, delete it and move on.
V8 14.6 only adds. Map.prototype.getOrInsert(), Iterator.concat() and the rest of the Chromium 146 features are additions with zero migration cost.

The upgrade, in practice

The order that produces the fewest surprises:

Install 26 next to your current version, not over it. With fnm or Volta it's one command, and rolling back takes seconds.
npm rebuild (or delete node_modules and reinstall) to regenerate native addons against ABI 147.
Run the suite with deprecations visible:

   node --pending-deprecation --trace-deprecation ./node_modules/.bin/vitest run

The module.register() and crypto ones show up here, with a stack trace to the culprit.

grep for the fossils: _stream_, writeHeader(. Two minutes that save you a production surprise.
Update your agents (APM, telemetry, loaders) to versions that declare Node 26 support — it's the layer that takes longest.
Review the engines field in your package.json and your critical dependencies: npm ls --depth=0 plus a glance at EBADENGINE warnings during install.

When should you upgrade?

New project: start on 26 today. There's no reason to begin something that will live for years on 24.

Production on 24: the sensible window opens on October 28, 2026, when 26 enters LTS and eight days after 24 moves to maintenance — with 24 supported until April 2028 there's no rush, but no reason to wait until the end either: the longer you wait, the further this list drifts from fresh memory.

Production on 22: move now. April 2027 looks far away until it doesn't, and the 22 → 26 jump is bigger than 24 → 26.

Frequently asked questions

When does Node 26 enter LTS?

On October 28, 2026, according to schedule.json in the nodejs/Release repository. Node 26 shipped as the Current release on May 5, 2026, moves to maintenance on October 20, 2027, and its end of life is scheduled for April 30, 2029. It has no announced codename yet.

What changes in the Node.js release schedule starting with Node 27?

Starting with Node 27 (April 2027) the project moves from two major releases per year to one, and every release will be LTS: the odd/even distinction disappears. A six-month alpha channel is added (the Node 27 alpha opens on October 28, 2026) where breaking changes are tested before release. Total support per version remains 36 months.

What breaks when upgrading from Node 24 to Node 26?

Little, and very localized: the removed internal stream* modules (affects very old dependencies), the removed writeHeader() HTTP server alias, Undici 8 with stricter header validation in fetch, the runtime deprecation of module.register(), and native addon recompilation due to the ABI change to NODE_MODULE_VERSION 147. For most projects it is checklist work, not refactoring work.

Can I run TypeScript directly in Node 26?

Yes. Type stripping is stable in Node 26: node app.ts works with no flags. The --experimental-transform-types flag was removed precisely because the feature is no longer experimental; if you had it in a script, just delete it.

How long is Node 24 supported?

Node 24 moves from Active LTS to maintenance on October 20, 2026 (critical and security fixes only from then on) and its end of life is scheduled for April 30, 2028, so there is no immediate urgency to leave it. The recommended migration window to 26 opens on October 28, 2026, when 26 enters LTS.

Does the Temporal API in Node 26 break code that uses Date?

No. Temporal ships enabled by default but it is a new API that coexists with Date: it does not change or remove any existing behavior. Migrating to Temporal is optional and can be done gradually, file by file.

Originally published on ortamarco.me.