DEV Community: Waseem Anjum

How to Test an MCP Server: Vitest, In-Memory Transport, Streamable HTTP and the Inspector CLI

Waseem Anjum — Sat, 31 Oct 2026 06:05:29 +0000

Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com.

Testing an MCP server means checking three layers: the pure functions behind your tools, the MCP contract (tool list, schemas, results and errors), and the transport your clients actually use. You can cover all three with Vitest and the official TypeScript SDK, without starting an AI client, and the whole suite below runs in about 300 ms.

I wrote this while building Benchline, my set of browser tools that are also a stateless MCP server. Every tool there is a pure function first and an MCP tool second, and that split is what makes testing easy. Below is a small, complete example server you can copy, the tests I run against it, and the MCP Inspector CLI commands for manual checks. Everything was run with @modelcontextprotocol/sdk 1.32.2, Vitest 5.0.5 and Node 22.20, and I re-ran the full suite on October 31, 2026 (1.32.2 is still the latest SDK release) with the same result.

TL;DR

Unit-test the logic as plain functions. Most bugs live there, and these tests need no MCP at all.
Contract-test the server in memory with InMemoryTransport.createLinkedPair() and a real SDK Client: list tools, check schemas, call each tool, and assert on errors.
Test the transport once over Streamable HTTP: the SDK client connects, initialize returns 200, wrong Accept headers get 406, GET gets 405 on a stateless server.
Check by hand with npx @modelcontextprotocol/inspector --cli <url> --transport http --method tools/list.
In SDK 1.32.2, a schema failure and an unknown tool both came back as a tool result with isError: true, not as a thrown error. Write your assertions so they hold either way.

The server under test

Two read-only tools: slugify, with a typed output schema, and json_format, which must report bad input as an error result instead of crashing.

```ts title="src/server.ts"
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { z } from "zod";

export function slugify(input: string) {
return input.normalize("NFKD").replace(/[\u0300-\u036f]/g, "").toLowerCase()
.replace(/[^a-z0-9]+/g, "-").replace(/^-+|-+$/g, "");
}

export function createServer() {
const server = new McpServer({ name: "demo-tools", version: "1.0.0" });

server.registerTool("slugify", {
title: "Slugify",
description: "Turn a title into a URL slug (lowercase ASCII, words joined by hyphens).",
inputSchema: { text: z.string().min(1).max(500) },
outputSchema: { slug: z.string() },
annotations: { readOnlyHint: true, idempotentHint: true, openWorldHint: false },
}, async ({ text }) => {
const slug = slugify(text);
return { content: [{ type: "text", text: slug }], structuredContent: { slug } };
});

server.registerTool("json_format", {
title: "Format JSON",
description: "Pretty-print a JSON string with the given indent. Returns an error result for invalid JSON.",
inputSchema: { json: z.string(), indent: z.number().int().min(0).max(8).default(2) },
annotations: { readOnlyHint: true, idempotentHint: true, openWorldHint: false },
}, async ({ json, indent }) => {
try {
return { content: [{ type: "text", text: JSON.stringify(JSON.parse(json), null, indent) }] };
} catch (e) {
return { isError: true, content: [{ type: "text", text: Invalid JSON: ${(e as Error).message} }] };
}
});

return server;
}




The key design choice is that `slugify` is exported on its own. The MCP handler is a thin wrapper, so the logic is tested like any other function.

## Layer 1: unit tests for the tool logic



```ts title="test/unit.test.ts"
import { describe, it, expect } from "vitest";
import { slugify } from "../src/server.ts";

describe("slugify (pure function)", () => {
  it("lowercases and joins words", () => expect(slugify("Hello, MCP World!")).toBe("hello-mcp-world"));
  it("strips accents", () => expect(slugify("Crème Brûlée")).toBe("creme-brulee"));
  it("trims separators", () => expect(slugify("  --a--  ")).toBe("a"));
});

Nothing MCP-specific here, and that is the point. In Benchline, each tool folder has a pure lib/ covered by Vitest. The MCP layer only maps inputs and outputs.

Layer 2: contract tests with the in-memory transport

The SDK ships InMemoryTransport, which links a client and a server inside one process. You test the real protocol (JSON-RPC messages, schema validation, result shapes) with no ports and no network.

```ts title="test/mcp.test.ts"
import { describe, it, expect, beforeAll, afterAll } from "vitest";
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
import { createServer } from "../src/server.ts";

let client: Client;
beforeAll(async () => {
const [clientT, serverT] = InMemoryTransport.createLinkedPair();
await createServer().connect(serverT);
client = new Client({ name: "test", version: "1.0.0" });
await client.connect(clientT);
});
afterAll(() => client.close());

describe("MCP contract (in-memory)", () => {
it("lists tools with object schemas and descriptions", async () => {
const { tools } = await client.listTools();
expect(tools.length).toBeGreaterThan(0);
for (const t of tools) {
expect(t.name).toMatch(/^[a-z][a-z0-9_]{0,63}$/);
expect(t.description?.length).toBeGreaterThan(20);
expect(t.inputSchema.type).toBe("object");
expect(t.annotations?.readOnlyHint).toBe(true);
}
expect(new Set(tools.map((t) => t.name)).size).toBe(tools.length);
});

it("returns structured content that matches the output schema", async () => {
const r = await client.callTool({ name: "slugify", arguments: { text: "Hello MCP" } });
expect(r.isError).toBeFalsy();
expect(r.structuredContent).toEqual({ slug: "hello-mcp" });
});

it("applies schema defaults", async () => {
const r = await client.callTool({ name: "json_format", arguments: { json: '{"a":1}' } });
expect((r.content as any)[0].text).toBe('{\n "a": 1\n}');
});

it("reports bad input as a tool error, not a crash", async () => {
const r = await client.callTool({ name: "json_format", arguments: { json: "{oops" } });
expect(r.isError).toBe(true);
expect((r.content as any)[0].text).toMatch(/^Invalid JSON/);
});

it("rejects arguments that fail the input schema", async () => {
const r = await client.callTool({ name: "slugify", arguments: { text: "" } }).catch((e) => e);
if (r instanceof Error) expect((r as any).code).toBe(-32602);
else expect((r as any).isError).toBe(true);
});

it("errors on unknown tools", async () => {
const r = await client.callTool({ name: "nope", arguments: {} }).catch((e) => e);
expect(r instanceof Error ? true : (r as any).isError).toBe(true);
});
});




What these tests catch:

- **Schema regressions.** Every tool must have an object `inputSchema` and a useful description. A model picks tools by name and description, so an empty description is a real bug.
- **Annotation drift.** If a tool is meant to be read-only, assert `readOnlyHint`. Clients can use annotations to decide when to ask the user for confirmation.
- **Duplicate names** after a refactor.
- **Error shape.** A handler that throws on bad input is worse than one that returns `isError: true` with a clear message, because the model can read the message and retry.

Two results surprised me, so I logged them before writing the assertions. With SDK 1.32.2, calling `slugify` with an empty string (which fails `min(1)`) and calling a tool that doesn't exist both resolved to a result with `isError: true`. The unknown-tool text was `MCP error -32602: Tool nope not found`. Older SDK versions and other server frameworks may throw a JSON-RPC error instead, so the tests accept both shapes.

## Layer 3: the Streamable HTTP transport

Clients like Cursor, Claude Code and VS Code connect over HTTP, so test that path once. This server is stateless: a new `McpServer` and transport per request and no session ID. Benchline's public endpoint is stateless in the same way: no API key and no session.



```ts title="src/http.ts"
import { createServer as createHttpServer } from "node:http";
import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
import { createServer } from "./server.ts";

// Stateless Streamable HTTP: a new server + transport per request, no session id.
export function startHttp(port = 0) {
  const http = createHttpServer(async (req, res) => {
    if (req.url !== "/mcp") { res.writeHead(404).end(); return; }
    if (req.method !== "POST") { res.writeHead(405, { Allow: "POST" }).end(); return; }
    const chunks: Buffer[] = []; for await (const c of req) chunks.push(c as Buffer);
    const body = JSON.parse(Buffer.concat(chunks).toString() || "null");
    const server = createServer();
    const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: undefined, enableJsonResponse: true });
    res.on("close", () => { transport.close(); server.close(); });
    await server.connect(transport);
    await transport.handleRequest(req, res, body);
  });
  return new Promise<{ url: URL; close: () => void }>((resolve) =>
    http.listen(port, "127.0.0.1", () => {
      const { port } = http.address() as { port: number };
      resolve({ url: new URL(`http://127.0.0.1:${port}/mcp`), close: () => http.close() });
    }));
}

if (import.meta.url === `file://${process.argv[1]}`) {
  const { url } = await startHttp(Number(process.env.PORT ?? 3321));
  console.log(`MCP server on ${url}`);
}

```ts title="test/http.test.ts"
import { describe, it, expect, beforeAll, afterAll } from "vitest";
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";
import { startHttp } from "../src/http.ts";

let srv: Awaited>;
const url = () => process.env.MCP_URL ? new URL(process.env.MCP_URL) : srv.url;
beforeAll(async () => { if (!process.env.MCP_URL) srv = await startHttp(); });
afterAll(() => srv?.close());

const init = { jsonrpc: "2.0", id: 1, method: "initialize",
params: { protocolVersion: "2025-06-18", capabilities: {}, clientInfo: { name: "curl", version: "1" } } };

describe("Streamable HTTP transport", () => {
it("works with the official SDK client", async () => {
const c = new Client({ name: "http-test", version: "1.0.0" });
await c.connect(new StreamableHTTPClientTransport(url()));
const { tools } = await c.listTools();
expect(tools.length).toBeGreaterThan(0);
await c.close();
});

it("answers initialize without a session id (stateless)", async () => {
const r = await fetch(url(), { method: "POST", body: JSON.stringify(init),
headers: { "content-type": "application/json", accept: "application/json, text/event-stream" } });
expect(r.status).toBe(200);
expect(r.headers.get("mcp-session-id")).toBeNull();
});

it("rejects a client that does not accept both content types", async () => {
const r = await fetch(url(), { method: "POST", body: JSON.stringify(init),
headers: { "content-type": "application/json", accept: "application/json" } });
expect(r.status).toBe(406);
});

it("returns 405 for GET when there is no SSE stream", async () => {
const r = await fetch(url(), { headers: { accept: "text/event-stream" } });
expect(r.status).toBe(405);
});
});




All four passed. Facts worth pinning in tests, because a framework upgrade can silently change them:

- `initialize` returned **200** with no `mcp-session-id` header, so clients don't need to track a session.
- A request with `Accept: application/json` only returned **406**. The Streamable HTTP spec requires clients to accept both `application/json` and `text/event-stream`, and some hand-written clients and proxies don't.
- `GET /mcp` returned **405**, which tells clients there is no server-to-client SSE stream on a stateless server.
- In stateless mode, `tools/list` worked without an `initialize` call first. A raw curl got back a 1,120-byte JSON response for the two tools.

Set `MCP_URL` to run the same file against a deployed server: `MCP_URL=https://your-host/mcp npx vitest run test/http.test.ts`. I ran it against the local server started with `node src/http.ts`, and all 4 tests passed.

Response size is worth a test too. Everything in `tools/list` is sent to the model, so it costs context on every conversation. I measured why that matters in [MCP vs API](https://fabwaseem.com/blog/mcp-vs-api).

## Manual checks with the MCP Inspector CLI

The [MCP Inspector](https://github.com/modelcontextprotocol/inspector) has a web UI and a CLI mode. The CLI is handy in a terminal or a CI smoke step (I first ran it with version 2.12.0, and the same commands gave the same output with 2.13.0, the latest release on October 31, 2026):



```bash title="Inspector CLI"
# list tools
npx @modelcontextprotocol/inspector --cli http://127.0.0.1:3321/mcp \
  --transport http --method tools/list

# call one tool
npx @modelcontextprotocol/inspector --cli http://127.0.0.1:3321/mcp \
  --transport http --method tools/call --tool-name slugify --tool-arg 'text=Hello MCP World'

The call printed:

{
  "content": [{ "type": "text", "text": "hello-mcp-world" }],
  "structuredContent": { "slug": "hello-mcp-world" }
}

And raw curl, for when you suspect the client:

```bash title="curl"
curl -s http://127.0.0.1:3321/mcp \
-H 'content-type: application/json' \
-H 'accept: application/json, text/event-stream' \
-d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}'




## Running it in CI



```json title="package.json (scripts)"
{
  "scripts": {
    "test": "vitest run",
    "test:remote": "vitest run test/http.test.ts"
  }
}

npm test runs all three layers. In 3 test files and 13 tests, Vitest reported a duration of 295 ms. Run test:remote with MCP_URL set after each deploy as a smoke test. It catches a broken proxy, a missing route or a CDN that strips headers, which is exactly what unit tests can't see.

What to test, as a checklist

Every tool: unit tests for its logic, including edge cases and bad input.
tools/list: unique names, object input schemas, descriptions long enough to be useful, expected annotations.
Each tool through callTool: one happy path, one error path that returns isError: true with a readable message.
Output schemas: structuredContent matches what you declared.
Transport: SDK client connects; initialize status and session header; Accept handling (406); GET behavior (405 when stateless).
Auth, if you have it: no token gets 401 with a WWW-Authenticate header, and a wrong-audience token is rejected. The MCP security guide covers the cases worth testing.
Size: keep an eye on the tools/list byte count as you add tools.

If you are building your first server, start with how to build an MCP server in Next.js and stateless MCP servers. Then connect it to your editor with how to add a remote MCP server.

FAQ

How do I test an MCP server without an AI client?

Use the official SDK's Client class in your test runner. Connect it to your server with InMemoryTransport.createLinkedPair() for fast in-process tests, or with StreamableHTTPClientTransport for the HTTP path, then call listTools and callTool and assert on the results. No model or editor is needed.

What is InMemoryTransport in the MCP SDK?

InMemoryTransport, exported from @modelcontextprotocol/sdk/inMemory.js, creates a linked pair of transports inside one process. Connect your server to one end and a Client to the other, and they exchange real JSON-RPC MCP messages without any network, which makes contract tests fast and deterministic.

How do I use the MCP Inspector CLI?

Run npx @modelcontextprotocol/inspector --cli followed by the server URL, --transport http and a method. For example --method tools/list lists tools, and --method tools/call --tool-name slugify --tool-arg 'text=Hello' calls one. It prints the JSON result, which makes it useful for scripts and CI smoke checks.

Should an MCP tool throw or return isError?

Return a result with isError: true and a clear text message for expected failures such as invalid input. The model can read the message and try again. In my tests with SDK 1.32.2, even schema validation failures and unknown tool names came back as isError results.

Why does my MCP server return 406 Not Acceptable?

The Streamable HTTP transport requires the client to send an Accept header listing both application/json and text/event-stream. A request with only Accept: application/json got 406 from the SDK's server transport in my test. Fix the client or the proxy that rewrites the header.

Originally published on fabwaseem.com.

Free OG Image Generator: Open Graph Image Sizes and Dynamic OG Images in Next.js

Waseem Anjum — Fri, 30 Oct 2026 07:03:55 +0000

Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com.

A free OG image generator turns a title, a subtitle and a few colors into a 1200×630 PNG that link previews can display. I built one you can use right now, with no account and no watermark: open the free OG image generator. It exports a ready PNG, gives you a live image URL, and writes the matching Next.js opengraph-image.tsx file for the template you picked.

This guide covers what an OG image is, the correct Open Graph image size for every major platform, how to use the tool, how to generate dynamic OG images in Next.js with next/og, the exact meta tags you need, and how to debug a preview that refuses to update.

What is an OG image?

An OG (Open Graph) image is the picture a social network or messaging app shows when someone shares a link. The page declares it with an og:image meta tag, defined by the Open Graph protocol, and the platform downloads that URL. Without one, the platform shows no image or guesses one from the page, which rarely looks good.

The Open Graph protocol was created by Facebook and is now read by LinkedIn, Slack, Discord, WhatsApp, iMessage and most chat apps. X (Twitter) reads its own twitter:* tags first and falls back to Open Graph tags when they are missing.

Definition: an OG image is the image referenced by a page's og:image meta tag. A dynamic OG image is one rendered on request (or at build time) from code, usually with the page title baked in, instead of a static file designed by hand.

What is the correct Open Graph image size?

Use 1200×630 pixels, a 1.91:1 aspect ratio, as a PNG or JPG under 5 MB. Facebook recommends at least 1200×630 for high-resolution displays, LinkedIn asks for at least 1200×627 at 1.91:1, and X's large card crops toward 2:1 but displays a 1200×630 image fine. One 1200×630 file covers almost every platform.

Keep the important text inside a safe zone: leave roughly 60 to 80 px of padding on every side, because some apps crop the edges or round the corners. Large, high-contrast titles survive small mobile previews far better than paragraphs of small text.

What OG and Twitter card sizes does each platform use?

Each platform publishes slightly different rules, but they converge on 1.91:1. The table below lists the recommended size and the hard limits from each platform's own documentation, so you can design one image and know where it might be cropped.

Platform	Recommended size	Ratio	Limits from official docs	Tag read
Facebook	1200 × 630	1.91:1	Minimum 200 × 200; 600 × 315 for large previews; max 8 MB	`og:image`
LinkedIn	1200 × 627	1.91:1	Minimum 1200 × 627; max 5 MB	`og:image`
X (summary_large_image)	1200 × 600 to 1200 × 675	2:1	Minimum 300 × 157; max 4096 × 4096; under 5 MB; JPG, PNG, WEBP or GIF	`twitter:image`, falls back to `og:image`
X (summary)	144 × 144 or larger	1:1	Max 4096 × 4096; under 5 MB	`twitter:image`
Slack	1200 × 630	1.91:1	Unfurls use Open Graph tags	`og:image`
Discord	1200 × 630	1.91:1	Embeds use Open Graph tags	`og:image`
WhatsApp	1200 × 630	1.91:1	Small files load more reliably	`og:image`
Next.js file convention	1200 × 630	1.91:1	`opengraph-image` max 8 MB, `twitter-image` max 5 MB (build fails above)	generates both

Sources: Facebook image guidelines, LinkedIn sharing image requirements, X summary card with large image and the Next.js opengraph-image docs.

How do I use the free OG image generator?

Open the tool, pick a template, type your title, subtitle and author, choose a font and three colors, then click Download PNG. The preview updates as you type and always renders at 1200×630. You can also copy the image URL to use directly in og:image, or copy the Next.js code.

Step by step:

Go to tools.fabwaseem.com/t/og-image-generator.
Choose one of six templates: Blog post (title, dek and byline), Product / SaaS (launch card with a mark), Minimal (type and empty space), Gradient (full-bleed color), Code snippet (editor window) or Personal / profile (name, role and portrait).
Fill in the title, subtitle and author. For the product and profile templates you can add a logo or avatar URL (HTTPS only; PNG, JPEG, WebP or GIF up to 1.5 MB).
Pick a font: Inter, Syne, Space Grotesk, Playfair Display, JetBrains Mono or Urbanist.
Set the background, text and accent colors.
Click Download PNG, Copy URL, or Copy code for an opengraph-image.tsx file.

The PNG is rendered when its URL is requested and is not stored as a file. Browsers and the CDN may cache the exact URL for a day, so the same card loads fast the next time.

Can I use the generated image URL directly in my meta tags?

Yes. The Copy URL button gives you an absolute HTTPS link to /api/og-tool with your settings in the query string, and that URL always returns a 1200×630 PNG. Paste it into og:image and twitter:image. For production sites, downloading the PNG or using the generated Next.js file keeps you independent of a third-party URL.

Here is the shape of a generated URL (shortened):

https://www.fabwaseem.com/api/og-tool?template=blog&title=Ship+the+post+people+actually+share&subtitle=A+readable+1200%C3%97630+card&author=Waseem+Anjum&font=syne

How do I generate dynamic OG images in Next.js?

Add an opengraph-image.tsx file to any App Router segment and default-export a function that returns an ImageResponse from next/og. Next.js turns that route into an image and injects the og:image, og:image:width, og:image:height, og:image:type and og:image:alt tags into the page head automatically. No meta tags are written by hand.

This is the pattern from the official Next.js docs, adapted for a blog post route with a dynamic slug:

// app/blog/[slug]/opengraph-image.tsx
import { ImageResponse } from 'next/og'
import { readFile } from 'node:fs/promises'
import { join } from 'node:path'
import { getPost } from '@/lib/posts'

export const alt = 'Blog post cover'
export const size = { width: 1200, height: 630 }
export const contentType = 'image/png'

export default async function Image({
  params,
}: {
  params: Promise<{ slug: string }>
}) {
  const { slug } = await params
  const post = await getPost(slug)
  const interSemiBold = await readFile(
    join(process.cwd(), 'assets/Inter-SemiBold.ttf')
  )

  return new ImageResponse(
    (
      <div
        style={{
          width: '100%',
          height: '100%',
          display: 'flex',
          flexDirection: 'column',
          justifyContent: 'flex-end',
          padding: 72,
          background: '#0C0C0C',
          color: '#F5F5F5',
        }}
      >
        <div style={{ display: 'flex', fontSize: 28, color: '#8FB4C9' }}>
          yoursite.com/blog
        </div>
        <div style={{ display: 'flex', fontSize: 72, lineHeight: 1.05, marginTop: 16 }}>
          {post.title}
        </div>
      </div>
    ),
    {
      ...size,
      fonts: [
        { name: 'Inter', data: interSemiBold, style: 'normal', weight: 600 },
      ],
    }
  )
}

A few facts from the docs worth knowing before you ship it:

The default export receives params as a Promise, so await it (Next.js 15 and later).
Generated images are statically optimized at build time and cached, unless they use request-time APIs or uncached data.
ImageResponse uses Satori and Resvg to turn JSX and CSS into a PNG. Only flexbox and a subset of CSS work; display: grid does not. Every div with more than one child needs display: 'flex'.
The ImageResponse reference lists a 500 KB maximum bundle size, which includes your JSX, CSS, fonts and images. Fonts must be TTF, OTF or WOFF.
Add a twitter-image.tsx with the same code if you want a separate X image; otherwise X falls back to og:image.

The free generator writes this file for you. Its output loads the chosen Google Font at render time with a small loadFont helper, so you can paste it into a segment and run it without adding font files.

Which meta tags does an OG image need?

At minimum you need og:title, og:description, og:url, og:type and og:image with an absolute HTTPS URL, plus twitter:card set to summary_large_image for a large X card. Adding og:image:width, og:image:height and og:image:alt helps platforms render the first share without waiting to download the image.

If you are not on Next.js, paste this into your <head>:

<meta property="og:type" content="article" />
<meta property="og:title" content="Your page title" />
<meta property="og:description" content="One sentence that earns the click." />
<meta property="og:url" content="https://example.com/blog/your-post" />
<meta property="og:image" content="https://example.com/og/your-post.png" />
<meta property="og:image:width" content="1200" />
<meta property="og:image:height" content="630" />
<meta property="og:image:alt" content="Short description of the image" />
<meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:title" content="Your page title" />
<meta name="twitter:description" content="One sentence that earns the click." />
<meta name="twitter:image" content="https://example.com/og/your-post.png" />

In the Next.js App Router you can set the same values with the Metadata API. Set metadataBase in your root layout so relative image paths resolve to absolute URLs:

// app/layout.tsx
import type { Metadata } from 'next'

export const metadata: Metadata = {
  metadataBase: new URL('https://example.com'),
  openGraph: {
    images: [{ url: '/og.png', width: 1200, height: 630, alt: 'Example' }],
  },
  twitter: { card: 'summary_large_image' },
}

How do I test and debug an OG image?

Paste the page URL into the Facebook Sharing Debugger and the LinkedIn Post Inspector, then click scrape again to clear their cache. Open the og:image URL in a browser to confirm it returns a 200 image. Finally, drop the link into Slack or Discord and check the unfurl. Most broken previews are a relative URL, a blocked crawler or a stale cache.

Testing tools:

Facebook Sharing Debugger: shows exactly which tags Facebook read and lets you force a re-scrape.
LinkedIn Post Inspector: refreshes LinkedIn's cached preview.
curl -I https://example.com/og.png: should return 200 and content-type: image/png.
curl -s https://example.com/page | grep og:image: confirms the tag is in the server HTML, not added later by client JavaScript.

Common causes and fixes:

Symptom	Likely cause	Fix
No image at all	`og:image` is a relative path	Use an absolute https URL; in Next.js set `metadataBase`
Old image keeps showing	Platform cache	Re-scrape in the debugger; add a version query like `?v=2`
Works in production, not on localhost	Crawlers cannot reach localhost	Test on a preview deployment or a tunnel
Image cropped badly on X	Ratio far from 2:1 or text near the edges	Use 1200×630 with 60 to 80 px padding
Next.js build fails on the image	File over 8 MB (OG) or 5 MB (Twitter)	Compress, or generate with code
`ImageResponse` layout is broken	Unsupported CSS such as grid	Use flexbox and `display: 'flex'` on parents
Tag missing in page source	Set in a client component	Set metadata on the server (Metadata API or file convention)

Static OG image or dynamic OG image: which should I use?

Use a static 1200×630 file for pages that rarely change, such as the home page, and dynamic OG images for content that grows, such as blog posts, docs and product pages. Dynamic images keep every share on-brand automatically, while a static file is the simplest and fastest option for a handful of pages.

	Static file	Dynamic (`next/og`)	Free generator URL
Setup	Design once, drop in `app/`	Write `opengraph-image.tsx`	Copy one URL
Per-page titles	Manual	Automatic	One URL per page
Best for	Home, landing pages	Blogs, docs, catalogs	Quick launches, side projects
Build impact	None	Rendered at build or on request	None

The generator sits in the middle: design visually, then either download a static PNG or copy the code to make it dynamic.

Frequently asked questions

Is this OG image generator really free?

Yes. It runs in the browser with no account, no sign-up and no watermark on the PNG. You can download images, copy the image URL or copy the Next.js code as often as you like.

What size is an OG image in Next.js?

The Next.js docs use size = { width: 1200, height: 630 } in opengraph-image.tsx, and ImageResponse defaults to 1200×630. Image files must stay under 8 MB for opengraph-image and 5 MB for twitter-image, or the build fails.

Do I need separate Twitter card images?

Usually not. X reads twitter:image first and falls back to og:image. Add twitter:card with the value summary_large_image so X shows the large card. Create a separate twitter-image only if you want a different design for X.

Why is my OG image not showing?

The most common reasons are a relative image URL, a page that blocks crawlers, a tag added by client-side JavaScript, or a cached old preview. Use an absolute https URL, check the server HTML and re-scrape in the Facebook Sharing Debugger.

Can I use WebP or SVG for OG images?

Use PNG or JPG for the widest support. X accepts WEBP, but SVG is not supported by the major platforms, and the Next.js file convention accepts JPG, PNG and GIF for static image files. The free generator always outputs PNG.

Does an OG image help SEO?

It does not directly change Google rankings, but it improves click-through when links are shared on social networks and in chat apps. For AI search visibility, pair it with good metadata and structured data, as covered in my Next.js SEO for AI search guide.

Key takeaways

Design every OG image at 1200×630 (1.91:1), under 5 MB, with 60 to 80 px of padding.
og:image must be an absolute HTTPS URL; in Next.js, set metadataBase.
In Next.js, an opengraph-image.tsx with ImageResponse from next/og generates the image and the meta tags for you. Use flexbox only.
Add twitter:card = summary_large_image; X falls back to og:image for the picture.
Always re-scrape in the Facebook Sharing Debugger and LinkedIn Post Inspector after changing an image.

Ready to make one? Create your free OG image now: pick a template, download the PNG or copy the Next.js code. For more of what I build, see my projects or head back to the home page.

Originally published on fabwaseem.com.

Free AI SEO Checker: How to Rank in ChatGPT, Perplexity and AI Overviews

Waseem Anjum — Fri, 30 Oct 2026 04:10:31 +0000

Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com.

An AI SEO checker tests whether AI answer engines such as ChatGPT search, Perplexity and Google AI Overviews can reach, read and quote a page. I built a free one that runs a GEO audit on any public URL in a few seconds and gives you a score out of 100 plus a fix for every warning: open the free AI SEO checker. No account needed.

This guide explains what an AI SEO check covers, what each check means, how the score works, how to run the tool, and what the research and the search engines themselves say about how to rank in ChatGPT and other AI answers.

What is an AI SEO checker?

An AI SEO checker is a tool that audits a web page for the signals AI answer engines rely on: whether AI crawlers are allowed in robots.txt, whether the page has a sitemap, llms.txt, structured data, clean metadata and headings, and whether its opening text answers the topic directly. It reports what passes, what fails and how to fix it.

Definition: AI SEO, also called GEO (generative engine optimization) or AEO (answer engine optimization), is the practice of making content easy for AI systems to find, understand and cite in generated answers. A GEO audit is a checklist review of a page against those requirements.

The term GEO comes from the research paper GEO: Generative Engine Optimization by Aggarwal and colleagues from Princeton and IIT Delhi, published at KDD 2024.

TL;DR

AI answer engines can only cite pages they are allowed to crawl and can parse. Most failures are technical: blocked crawlers, missing metadata, no structured data, or a page that buries the answer.
Google says AI Overviews and AI Mode need no special optimization beyond normal SEO: the page must be indexed and eligible for a snippet.
The GEO paper found that adding citations, quotations and statistics raised visibility in generated answers by up to about 40%, while keyword stuffing did not help.
Run a free GEO audit at tools.fabwaseem.com/t/ai-seo-checker, fix the failures first, then the warnings.

How do I rank in ChatGPT and other AI answers?

To rank in ChatGPT search, Perplexity and Google AI Overviews, make sure their crawlers can reach the page, that the page is indexed by the underlying search engine, and that it answers the question clearly in the first lines with sourced facts. Add structured data that matches the visible text, keep titles and headings precise, and publish a sitemap.

There is no paid placement or secret switch. What the engines and the research agree on:

Source	What it says
Google: AI features and your website	No extra requirements for AI Overviews or AI Mode. The page must be indexed and eligible for a snippet. Allow crawling, use internal links, keep important content in text, and make structured data match visible text.
OpenAI crawlers	OAI-SearchBot is used for ChatGPT search results; GPTBot is for training; ChatGPT-User fetches pages for user actions. Each respects robots.txt separately.
Perplexity crawlers	PerplexityBot indexes pages for Perplexity answers; Perplexity-User fetches on a user's request.
Anthropic crawlers	ClaudeBot and related agents follow robots.txt.
GEO paper (KDD 2024)	Citing sources, adding quotations and adding statistics gave the largest visibility gains; keyword stuffing did not help.

The practical order is: be crawlable, be indexed, be clear, be citable. The checker covers the first three directly and gives hints for the fourth.

What does the free AI SEO checker test?

The checker fetches the URL you enter plus its robots.txt, llms.txt and sitemap, then runs a set of checks in three groups: on-page (HTTPS, response time, title, meta description, canonical, Open Graph, headings, answer-first lead), discovery (JSON-LD, llms.txt, sitemap) and AI crawler rules for nine named bots.

Every check, with the weight it carries in the score:

Check	Group	Weight	Passes when
HTTPS	Page	8	The final URL after redirects is https
Response time	Page	6	HTML arrives in 800 ms or less (warn up to 1,800 ms)
Title	Page	8	A title of 15 to 65 characters
Meta description	Page	8	A description of 50 to 160 characters
Canonical	Page	6	An absolute rel=canonical URL
Open Graph	Page	6	og:title, og:description and og:image present
Headings	Page	8	One H1, then H2 and H3 in order without skipped levels
Answer-first lead	Page	8	The opening paragraph is a short, direct answer (roughly 40 to 320 characters, 70 words or fewer)
JSON-LD	Discovery	10	At least one valid JSON-LD block with a schema.org type
llms.txt	Discovery	10	`/llms.txt` exists with an H1 and a summary or links
Sitemap	Discovery	8	A sitemap is found from robots.txt or `/sitemap.xml`
AI crawlers	Crawlers	2 each	Each bot's robots.txt rule matches the expected access

The crawler rows check these user agents: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended, CCBot and Applebot-Extended (expected: allowed), plus Bytespider (expected: blocked, since it is a high-volume scraper rather than a citation crawler). If you deliberately block training bots such as GPTBot or CCBot, that row will fail; the choice is yours, and the tip explains the trade-off.

How is the AI search score calculated?

The score is a weighted average out of 100. Each check has a weight, and a pass earns its full weight, a warning earns half, and a fail earns nothing. The total earned is divided by the total possible weight and rounded. Discovery checks such as JSON-LD and llms.txt weigh most because they decide whether an engine understands the page at all.

So a page that passes everything except a warning on the answer-first lead loses 4 of 104 possible points and scores around 96. That is what this site scored when I tested the homepage: every check passed except the answer-first lead, which the tool flagged with "The opening text is there, but it is not a short direct answer."

The tool labels the score with a band:

Score	Band	What to do
85 to 100	Strong	Polish warnings, focus on content quality
70 to 84	Good	Fix the remaining fails and the heaviest warnings
50 to 69	Mixed	Fix every fail, starting with crawler access and discovery
Below 50	Weak	Start with crawler access, sitemap and metadata

How do I run a GEO audit with the tool?

Open the AI SEO checker, paste a full public URL, and press Check this URL. In a few seconds you get a score, a pass, warn or fail status for each check, a one-line detail of what was found, and a fix with a link to a longer guide. Fix the fails first, redeploy, and run it again.

Step by step:

Pick the page that matters most. Usually your homepage, a pricing page or your best article. The audit is per URL, so check key templates separately (home, blog post, product page).
Paste the full URL, including https://, into the Page URL field. Only public http and https addresses are accepted.
Run the check. The tool fetches the page, robots.txt, llms.txt and the sitemap, with an 8-second timeout per request.
Read the score and the groups. On the page covers metadata, headings and the opening answer. Discovery covers JSON-LD, llms.txt and the sitemap. The crawler section lists each AI bot.
Fix fails first. A blocked OAI-SearchBot or PerplexityBot means those engines cannot read the page at all, which matters more than a long title.
Fix warnings next. Each row's "How to fix this" link explains the change.
Re-run after deploying to confirm the score moved. The tool allows a few runs per visitor every 10 minutes.

How do I fix the most common AI SEO failures?

The most common failures are a robots.txt that blocks AI search crawlers, a missing llms.txt, no JSON-LD, a missing or duplicate meta description, and a page that opens with filler instead of an answer. Each has a small, specific fix, and most take minutes in a modern framework such as Next.js.

Failure	Why it matters	Fix
OAI-SearchBot or PerplexityBot blocked	That engine cannot fetch the page to cite it	Allow them in robots.txt; named groups do not inherit `*` rules under RFC 9309, so repeat your disallows in each group
No llms.txt	Agents lack a quick map of your site	Create one with the free llms.txt generator
No JSON-LD	Engines must guess what the page is	Add Article, Organization, Person, Product or FAQPage markup that matches the page (Google's guide)
Weak or missing meta description	Snippets and summaries pull poor text	Write 120 to 160 characters that answer the page's main question
Skipped heading levels	Structure is harder to parse into sections	One H1, then H2s, then H3s
No answer in the first paragraph	Engines quote clear, direct statements	Open with a 40 to 60 word answer, then the detail
Slow HTML	Crawlers may time out or crawl less	Cache HTML, cut server work, use a CDN
No sitemap	New pages are found more slowly	Publish `/sitemap.xml` and list it in robots.txt (sitemaps.org)

A robots.txt that allows AI search crawlers while keeping private paths closed looks like this:

User-agent: *
Allow: /
Disallow: /admin/

User-agent: OAI-SearchBot
Allow: /
Disallow: /admin/

User-agent: PerplexityBot
Allow: /
Disallow: /admin/

User-agent: Bytespider
Disallow: /

Sitemap: https://example.com/sitemap.xml

For a full Next.js implementation of robots, sitemap, metadata and JSON-LD, see my guide to Next.js SEO for AI search. For social previews, the free OG image generator makes the og:image the checker looks for.

How do I write content that AI engines cite?

Write each section so it can be lifted out and quoted on its own: a question as the heading, a direct 40 to 60 word answer first, then detail, tables and steps. Cite primary sources, include concrete numbers you can back up, and define terms plainly. The GEO paper found these citation, quotation and statistics changes gave the biggest gains.

From the GEO paper, in its own benchmark:

Method	Effect on visibility in generated answers
Cite sources, add quotations, add statistics	Largest gains, about 30 to 40% on the position-adjusted word count metric
Fluency and easy-to-understand rewrites	About 15 to 30%
Authoritative, persuasive tone	No significant improvement
Keyword stuffing	Little or no improvement, and worse on Perplexity in their test

The paper also reports that lower-ranked pages gained most: in one test, citing sources raised visibility for pages ranked fifth in search results by 115%. Treat these as research results on a benchmark, not guarantees, but the direction matches what the engines say: clear, sourced, well-structured text wins.

A practical writing checklist:

A question-style H2 for every main question people ask.
A first sentence that answers it without preamble.
One definition per new term.
At least one comparison table.
Numbered steps for any process.
Links to official documentation for every factual claim.
An FAQ section with short, self-contained answers, marked up as FAQPage if it suits the page.

Is AI SEO different from regular SEO?

Mostly no. AI search reuses the foundations of regular SEO: crawlable, indexed, fast pages with accurate metadata and helpful content. The differences are emphasis. AI engines quote passages, so answer-first writing, clear sections, cited facts and structured data matter more, and there are extra crawler user agents to allow in robots.txt.

Area	Regular SEO	AI SEO / GEO
Goal	Rank a blue link	Be quoted or cited in an answer
Crawlers	Googlebot, Bingbot	Plus OAI-SearchBot, PerplexityBot, ClaudeBot and others
Content unit	The page	The passage or section
Signals	Links, relevance, experience	The same, plus clarity, citations and direct answers
Extra files	robots.txt, sitemap	Optionally llms.txt
Measurement	Rankings, clicks	Citations, referral traffic from AI tools, Search Console "Web" data

Google notes that traffic from AI Overviews and AI Mode is counted in Search Console's Performance report under the Web search type, so you measure it there alongside normal search.

Frequently asked questions

Is the AI SEO checker free?

Yes. The AI SEO checker is free with no account and no email gate. A small rate limit (a few runs per visitor every 10 minutes) keeps it fast and stops abuse.

What is a GEO audit?

A GEO audit is a review of a page against what generative engines need: crawler access, indexability, metadata, structured data, llms.txt, a sitemap, clear headings and answer-first content. The checker automates the technical part and returns a score with fixes.

Can I guarantee my site appears in ChatGPT or AI Overviews?

No. No tool or agency can guarantee it. Google states that meeting every requirement does not guarantee crawling, indexing or serving. You can remove the technical blockers and make the content easy to cite, which improves your chances.

Should I block GPTBot?

It depends on your goals. GPTBot collects training data, while OAI-SearchBot powers ChatGPT search results. OpenAI documents them separately, so you can block GPTBot and still allow OAI-SearchBot if you want to appear in ChatGPT search without contributing training data.

Does llms.txt improve my AI SEO score?

In this tool, yes: llms.txt is one of the heaviest checks, at 10 points. In the real world, Google says you do not need it for AI Overviews. It helps agents and assistants that read it, and it costs little to add.

How often should I run an AI SEO check?

Run it after every deploy that touches templates, robots.txt or metadata, and on each new key page. A robots.txt change can silently block an AI crawler, so a quick re-check after infrastructure changes is worth it.

Key takeaways

AI SEO starts with access: allow OAI-SearchBot, PerplexityBot and the other bots you want, and remember named robots.txt groups do not inherit * rules.
Google says no special optimization is needed for AI Overviews; normal SEO fundamentals apply.
Write answer-first sections with cited sources and real numbers; the GEO paper found these gave the biggest visibility gains, and keyword stuffing did not.
Add JSON-LD that matches visible text, a sitemap and an llms.txt.
Audit key pages for free with the AI SEO checker, fix fails first, then re-run.

Run your free AI SEO check

Paste your most important page into the free AI SEO checker and fix the first failure it shows. If llms.txt is missing, make one with the llms.txt generator. All my free tools are on the tools page, and my work is on the projects page.

Originally published on fabwaseem.com.

MongoDB vs Postgres JSONB in 2026: Inserts, Lookups, Updates and Storage, Benchmarked

Waseem Anjum — Fri, 30 Oct 2026 04:10:09 +0000

Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com.

Most "MongoDB vs Postgres JSONB" debates are fought with benchmarks from 2014 and opinions from Twitter. I run both in my day job and side projects, so I put the current versions, MongoDB 9.0.4 (released October 13, 2026) and PostgreSQL 18.6, on the same VM, loaded the same 200,000 order documents into each, and measured the things a real app does all day: inserts, lookups by an indexed field, containment filters, partial updates of nested fields, aggregations and disk usage.

The short version: neither one wins everywhere. Postgres answered B-tree indexed reads more than twice as fast through the Node.js drivers, but MongoDB loaded data almost 4x faster, updated nested fields about 50% faster and used 5.6x less disk. And one index choice that most JSONB tutorials recommend, a GIN index on the whole document, turned out to be the most expensive line in my Postgres schema.

TL;DR: MongoDB vs Postgres JSONB

If your app is relational with some flexible fields, keep it in Postgres and index the JSONB paths you query with B-tree expression indexes, not one big GIN index. If your data really is documents that get written and updated a lot, and disk or cache size matters, MongoDB did that work faster and far more compactly in my tests.

200,000 docs, 16 clients (median of 3)	PostgreSQL 18.6 JSONB	MongoDB 9.0.4
Bulk insert, 1,000 per batch	12,337 docs/s	46,448 docs/s
Single inserts	24,594 docs/s	18,506 docs/s
Point lookup by `orderNo`	44,701 ops/s (B-tree), 29,417 (GIN)	19,139 ops/s
Latest 10 orders of a customer	19,322 ops/s	8,855 ops/s
Containment filter, `LIMIT 20`	2,531 ops/s	4,605 ops/s
Count of 1,411 matching docs	2,094 ops/s	1,067 ops/s
Nested field update	16,786 ops/s	24,603 ops/s
Append to an array	16,542 ops/s	26,755 ops/s
Revenue by country (full scan)	182.8 ms	154.6 ms
Top 10 SKUs (unwind items)	161.3 ms	283.1 ms
Disk: data + indexes after load	308.4 MB	55.2 MB
Disk after 1.5 to 2.3 million updates	1,231.5 MB	138.8 MB

How I tested

Everything ran on one Linux VM: 8 vCPUs (Intel Xeon, KVM), 16 GB of RAM, Debian 13. The databases and the Node.js client shared the machine and talked over loopback TCP, so there's no network latency in these numbers. The VM is shared with other workloads, so I logged the load average and free memory with every result and re-ran everything after one early run stalled (more in the limitations).

PostgreSQL 18.6 (PGDG build) in its own cluster with shared_buffers = 512MB, max_wal_size = 4GB and otherwise default settings, including fsync = on and synchronous_commit = on.
MongoDB 9.0.4 Community, the official Debian 12 tarball, as a standalone server with --wiredTigerCacheSizeGB 0.5 so both databases got the same 512 MB of cache. Everything else default: snappy compression, journaling on, default write concern.
Node.js 22.20.0 with pg 8.23.1 and mongodb 7.7.0, the latest npm releases on October 29, 2026.
Data: 200,000 generated e-commerce orders from a seeded random generator, the same documents for both databases. Each order has a customer, status, total, tags, a nested shipping object, 1 to 5 line items and an events array. That's about 770 bytes as JSON, 979 bytes per row as JSONB and 806 bytes as BSON.
Method: 16 concurrent clients (a 16-connection pool on both sides) in a closed loop for 10 seconds per test, 3 runs per test, and I report the median. Aggregations ran 5 times per run.

Every query I timed returned the same answer on both databases: for example, both found exactly 1,411 orders that are shipped, going to the UAE and tagged gift, and both aggregations returned identical totals.

The schema and the indexes

Postgres gets one jsonb column plus an identity key. To compare the two ways people index JSONB, I created both: a GIN index over the whole document and B-tree expression indexes on the fields the app looks up.

```sql title="postgres.sql"
CREATE TABLE orders (
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
doc jsonb NOT NULL
);
-- one index for any containment query (@>)
CREATE INDEX orders_gin ON orders USING gin (doc jsonb_path_ops);
-- B-tree expression indexes for the hot lookups
CREATE UNIQUE INDEX orders_orderno ON orders ((doc->>'orderNo'));
CREATE INDEX orders_customer ON orders
(((doc->>'customerId')::int), (doc->>'createdAt') DESC);






```js title="mongo-indexes.js"
db.orders.createIndex({ orderNo: 1 }, { unique: true });
db.orders.createIndex({ customerId: 1, createdAt: -1 });
db.orders.createIndex({ status: 1, "shipping.country": 1 });
db.orders.createIndex({ tags: 1 });

Two things to know about the Postgres side. The query has to match the index: doc->>'orderNo' = $1 uses the B-tree, doc @> '{"orderNo": "..."}' uses the GIN index, and neither one can use the other's index. And JSONB has no date type, so createdAt is an ISO 8601 string. It sorts correctly as text, but in MongoDB it's a real Date.

Insert throughput

Inserts (median of 3)	PostgreSQL 18.6	MongoDB 9.0.4
Bulk, 1,000 docs per call, all indexes	12,337 docs/s	46,448 docs/s
Bulk, without the GIN index	26,918 docs/s	not applicable
Single inserts, 16 clients, defaults	24,594 docs/s (p50 0.485 ms)	18,506 docs/s (p50 0.571 ms)
Single inserts, `synchronous_commit = off` / `{ j: true }`	31,459 docs/s	16,070 docs/s

For bulk loads MongoDB was 3.8x faster: 200,000 orders in 4.3 seconds against 16.2 seconds for Postgres with a multi-row INSERT ... SELECT unnest($1::jsonb[]). Half of the Postgres cost was the GIN index. Without it the same load ran at 26,918 docs/s, 2.2x faster.

Single inserts flipped the result. With 16 clients each inserting one document at a time, Postgres did 24,594 per second with every commit flushed to the WAL. Postgres groups concurrent commits into one flush, which is why durability cost it so little here. MongoDB's default write concern gave 18,506 inserts per second, and asking it to wait for the journal on every insert ({ j: true }) gave 16,070.

Point lookups: GIN vs B-tree expression index vs MongoDB index

Fetching one order by its unique orderNo is the most common query in most apps.

Lookup by `orderNo` (median of 3)	16 clients	1 client, p50	1 client, p99
Postgres, B-tree expression index (`->>`)	44,701 ops/s	0.057 ms	0.316 ms
Postgres, GIN `jsonb_path_ops` (`@>`)	29,417 ops/s	0.123 ms	0.506 ms
MongoDB, `{ orderNo: 1 }` index	19,139 ops/s	0.076 ms	0.377 ms

The B-tree expression index was the fastest way to find one document in either database, and 52% faster than the GIN index on the same table. EXPLAIN shows why: the B-tree query is a plain index scan, while the GIN query is a bitmap index scan plus a bitmap heap scan with a recheck, and the planner guessed 20 matching rows for a unique value because it has no real statistics for containment. With GIN you also pay for an index that covers every key and value in the document.

MongoDB's own query planner used its fast EXPRESS_IXSCAN path, and with one client its p50 (0.076 ms) sat between the two Postgres indexes. With 16 clients it delivered less than half the throughput of the B-tree. I didn't profile the client side, but part of the gap is likely decoding: pg hands JSONB to V8's native JSON.parse, while the MongoDB driver decodes BSON in JavaScript.

Customer feed: the latest 10 orders

A compound index on customer and date, then ORDER BY ... LIMIT 10. Both databases read the index backwards from the newest entry and stopped after 10 rows.

Latest 10 orders for a random customer	Ops/s	p50	p99
Postgres, expression index on `(customerId::int, createdAt DESC)`	19,322	0.67 ms	2.40 ms
MongoDB, `{ customerId: 1, createdAt: -1 }`	8,855	1.55 ms	4.50 ms

Postgres returned the 10 documents 2.2x faster. Same pattern as the point lookups: when the access path is a known B-tree, Postgres plus pg moved more rows per second.

Containment filters and counts

Now an ad-hoc filter: orders with a given status, shipping country and tag, such as shipped to the UAE with the gift tag. In Postgres that's one @> against the GIN index. In MongoDB it's a normal filter that used the { status, shipping.country } index and checked tags on the fetched documents.

SELECT doc FROM orders
WHERE doc @> '{"status": "shipped", "shipping": {"country": "AE"}, "tags": ["gift"]}'
LIMIT 20;

db.orders.find({ status: "shipped", "shipping.country": "AE", tags: "gift" }).limit(20);

Filter on 3 fields (4 rotating filters)	Postgres GIN `@>`	MongoDB
First 20 matches	2,531 ops/s (p50 5.76 ms)	4,605 ops/s (p50 2.86 ms)
Count all matches (1,411 for the UAE filter)	2,094 ops/s (p50 6.83 ms)	1,067 ops/s (p50 14.21 ms)

MongoDB was 1.8x faster at returning the first 20, because it can walk the compound index and stop as soon as it has 20 documents. Postgres builds a bitmap from the GIN index for all matches before it reads the first row, so LIMIT 20 saves less. For a full count Postgres was 2x faster: the GIN index matched all three conditions at once, while MongoDB had to fetch every shipped UAE order to check its tags. A tags field in MongoDB's compound index would change that, and so would a B-tree on the hot fields in Postgres. The point is that GIN gives you decent speed on any combination of fields without planning indexes, not the best speed on any one of them.

Partial nested updates

Updating one nested field is where the two storage engines differ the most. Here are three updates by orderNo: set shipping.status and updatedAt, push an event onto the events array, and increment a counter.

-- Postgres
UPDATE orders
SET doc = jsonb_set(doc, '{shipping,status}', to_jsonb($2::text))
WHERE doc->>'orderNo' = $1;

UPDATE orders
SET doc = jsonb_set(doc, '{events}',
  (doc->'events') || jsonb_build_array(jsonb_build_object('type', $2::text, 'at', now())))
WHERE doc->>'orderNo' = $1;

// MongoDB
db.orders.updateOne({ orderNo }, { $set: { "shipping.status": status, updatedAt: new Date() } });
db.orders.updateOne({ orderNo }, { $push: { events: { type: "note", at: new Date() } } });
db.orders.updateOne({ orderNo }, { $inc: { views: 1 } });

Updates by `orderNo`, 16 clients (median of 3)	Postgres, all indexes	Postgres, no GIN	MongoDB
Set a nested field	16,786 ops/s	22,979 ops/s	24,603 ops/s
Append to an array	16,542 ops/s	22,154 ops/s	26,755 ops/s
Increment a counter	17,604 ops/s	23,298 ops/s	26,638 ops/s

MongoDB was about 50% faster than Postgres with the full set of indexes. jsonb_set reads like a partial update, but Postgres stores JSONB as one value, so every update writes a complete new copy of the row. Worse, it can't be a HOT (heap-only tuple) update once any index is built on an expression over doc, because Postgres only knows the indexed column changed. I checked this on a small table: 5,000 jsonb_set updates gave 0 HOT updates with a doc->>'orderNo' index and 2,215 without it. So every update also inserts new entries into every index, including the GIN index, which holds an entry for every value in the document. Dropping the GIN index made Postgres updates 32 to 37% faster and closed most of the gap.

Aggregations

Two reports over all 200,000 orders: revenue and order count per shipping country for shipped and delivered orders, and the 10 best-selling SKUs, which needs unwinding the items array.

SELECT doc->'shipping'->>'country' AS country, count(*) AS n,
       sum((doc->>'total')::numeric) AS revenue
FROM orders
WHERE doc->>'status' IN ('shipped', 'delivered')
GROUP BY 1 ORDER BY 3 DESC;

SELECT i->>'sku' AS sku, sum((i->>'qty')::int) AS qty
FROM orders, jsonb_array_elements(doc->'items') AS i
GROUP BY 1 ORDER BY 2 DESC, 1 LIMIT 10;

db.orders.aggregate([
  { $match: { status: { $in: ["shipped", "delivered"] } } },
  { $group: { _id: "$shipping.country", n: { $sum: 1 }, revenue: { $sum: "$total" } } },
  { $sort: { revenue: -1 } },
]);
db.orders.aggregate([
  { $unwind: "$items" },
  { $group: { _id: "$items.sku", qty: { $sum: "$items.qty" } } },
  { $sort: { qty: -1, _id: 1 } },
  { $limit: 10 },
]);

Full-collection aggregation (median of 3 x 5 runs)	Postgres 18.6	MongoDB 9.0.4
Revenue by country (70% of orders match)	182.8 ms	154.6 ms
Top 10 SKUs (about 600,000 line items)	161.3 ms	283.1 ms

A split result. The grouped sum was close (MongoDB took 15% less time, which is inside the run-to-run noise for that Postgres query), and Postgres took 43% less time on the unwind. Postgres used parallel workers for these scans (2 per query by default) and still had to parse ->> paths and cast text to numeric for every row. EXPLAIN ANALYZE showed the revenue query sorting on disk with the default work_mem of 4 MB, which I left untuned. If reports like this matter, move the summed fields into real columns, or raise work_mem for the reporting role.

Storage size

After loading 200,000 orders	PostgreSQL 18.6	MongoDB 9.0.4
Table / collection data	212.6 MB	45.6 MB (161.2 MB before compression)
Indexes	95.7 MB (GIN 71.9 MB)	9.7 MB
Total	308.4 MB	55.2 MB
Total without the GIN index	236.5 MB	not applicable

MongoDB used 5.6x less disk. WiredTiger compresses collections with snappy and indexes with prefix compression by default, so 161 MB of BSON became 46 MB. Postgres only compresses a JSONB value when it's bigger than about 2 KB (the TOAST threshold), and these documents averaged 979 bytes, so none were compressed. The GIN index alone was larger than MongoDB's collection and all five of its indexes together.

Then I ran the update tests from the previous section and measured again. Those runs also grew the documents (new events, a views counter, an updatedAt field), so average document size went up by about a fifth in both databases.

After the update tests	PostgreSQL, all indexes	PostgreSQL, no GIN	MongoDB
Updates applied	1,461,709	2,128,223	2,308,992
Total size	1,231.5 MB	627.0 MB	138.8 MB
Of which indexes	512.2 MB (GIN 466.0 MB)	50.6 MB	9.7 MB

Postgres grew 4x with autovacuum running at default settings, and the GIN index grew 6.5x to 466 MB even though no indexed value changed. MongoDB's indexes stayed the same size, because updating fields that aren't indexed doesn't touch its indexes. Its collection grew to 129.1 MB, of which 67.4 MB was free space WiredTiger will reuse. Postgres will reuse its dead space too after vacuum, but a heavily updated JSONB table with a whole-document GIN index needs much more disk and cache than the same data in MongoDB.

The same operations in Node.js

This is the module I used to check that each query does the same thing in both databases. A test inserted 3 orders, ran every function against both and asserted identical results. It passed.

```ts title="orders.mjs"
import pg from "pg";
import { MongoClient } from "mongodb";

const pool = new pg.Pool({ connectionString: process.env.PG_URL });
const mongo = new MongoClient(process.env.MONGO_URL);
const orders = mongo.db("shop").collection("orders");

// Insert
export const pgInsert = (doc) => pool.query("INSERT INTO orders (doc) VALUES ($1)", [doc]);
export const mgInsert = (doc) => orders.insertOne(doc);

// Point lookup: ->> matches the B-tree expression index; @> would go to GIN
export const pgByOrderNo = async (no) =>
(await pool.query("SELECT doc FROM orders WHERE doc->>'orderNo' = $1", [no])).rows[0]?.doc;
export const mgByOrderNo = (no) => orders.findOne({ orderNo: no });

// Customer feed: latest 10 orders
export const pgFeed = async (customerId) => (await pool.query(
SELECT doc FROM orders WHERE (doc->>'customerId')::int = $1 ORDER BY doc->>'createdAt' DESC LIMIT 10, [customerId])).rows.map((r) => r.doc);
export const mgFeed = (customerId) =>
orders.find({ customerId }).sort({ createdAt: -1 }).limit(10).toArray();

// Containment filter
export const pgFilter = async (status, country, tag) => (await pool.query(
"SELECT doc FROM orders WHERE doc @> $1::jsonb LIMIT 20",
[JSON.stringify({ status, shipping: { country }, tags: [tag] })])).rows.map((r) => r.doc);
export const mgFilter = (status, country, tag) =>
orders.find({ status, "shipping.country": country, tags: tag }).limit(20).toArray();

// Partial nested update
export const pgShip = (no, status) => pool.query(
UPDATE orders SET doc = jsonb_set(doc, '{shipping,status}', to_jsonb($2::text)) WHERE doc->>'orderNo' = $1, [no, status]);
export const mgShip = (no, status) =>
orders.updateOne({ orderNo: no }, { $set: { "shipping.status": status } });




Two driver details that cost me time. `pg` serializes a plain object to JSON for a `jsonb` parameter, but a JavaScript `Date` inside it becomes an ISO string, and it comes back as a string. And `pg` returns `numeric` and `count(*)` as strings (`"361.5"`, `"3"`), while MongoDB returns numbers, so convert before you compare or add.

## Which should you choose?

- **Your data is relational and some of it is flexible: Postgres JSONB.** Joins, foreign keys, transactions across tables and SQL reporting still matter more than raw document speed for most apps. Use B-tree expression indexes (or generated columns) for the paths you query often. They were the fastest lookups in this whole test.
- **Add a GIN index only when you need ad-hoc containment queries.** It made any combination of fields queryable, but it halved bulk insert speed, slowed updates by about a quarter and was the biggest object on disk, and it grew fastest under updates. If you only filter on a few known keys, index just those paths, for example a GIN index on `(doc->'tags')` queried with `doc->'tags' @> '["gift"]'`. It should cost far less, though I didn't measure that variant.
- **Your data is documents that get written and updated all day: MongoDB.** It loaded 3.8x faster, updated nested fields about 50% faster and stored the same orders in 18% of the space, which also means more of your working set fits in RAM.
- **Updates are frequent and documents are large: avoid whole-document rewrites in Postgres.** Put fast-changing fields like status, counters and timestamps in normal columns next to the JSONB, so updates can be HOT and don't rewrite every index.

If you're also deciding what else Postgres can absorb, I measured [Postgres full-text search against Elasticsearch and Meilisearch](https://www.fabwaseem.com/blog/postgres-full-text-search-vs-elasticsearch) and [Postgres job queues against BullMQ on Redis](https://www.fabwaseem.com/blog/postgres-job-queue-pg-boss-vs-bullmq). For primary keys on big tables, see [UUIDv7 in Postgres](https://www.fabwaseem.com/blog/uuidv7-postgres), and for the ORM layer, [Drizzle vs Prisma](https://www.fabwaseem.com/blog/drizzle-vs-prisma).

## Limitations of this benchmark

- One VM, a single node of each database, everything on loopback. No replica set, so MongoDB's write concern never waited for a second node, and Postgres had no replica either. Real networks add a round trip to every call.
- 200,000 documents of under 1 KB each, which fits in RAM. With a working set larger than memory, MongoDB's 5.6x smaller footprint would likely matter even more, but I didn't test that.
- The VM is shared with other workloads. My first full run stalled for about a minute while the VM was almost out of free memory, so I discarded it and re-ran everything. Then MongoDB crashed mid-run because a snapshot agent on the VM dropped a temporary file into its journal folder, so I restarted it with log file removal turned off (`log=(remove=false)`) and re-ran all MongoDB tests from scratch. Most tests stayed within 10% of their median across the 3 runs, but a few varied by up to 47% (one Postgres aggregation run, the GIN lookups and some update runs). That's why I report medians, and why I'd treat any gap under about 20% as noise.
- Node.js drivers on both sides, so client-side decoding (JSON vs BSON) is part of every read number.
- I didn't tune `work_mem`, autovacuum or fillfactor in Postgres, or anything in MongoDB beyond the cache size. I also didn't test transactions, sharding, schema validation or change streams.


---

*Originally published on [fabwaseem.com](https://www.fabwaseem.com/blog/mongodb-vs-postgres-jsonb).*

Free llms.txt Generator: What llms.txt Is and How to Create One

Waseem Anjum — Thu, 29 Oct 2026 01:05:16 +0000

Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com.

An llms.txt generator builds the /llms.txt file that tells AI agents and chat assistants what your site is and which pages matter. I built a free one that reads your sitemap and writes a spec-style file in a few seconds, with no account: open the free llms.txt generator. It also writes an optional llms-full.txt with a short excerpt of every page.

This guide explains what llms.txt is, what it does and does not do for SEO, the exact format from the official proposal, how to use the generator step by step, how to serve the file from Next.js, and how to check that it works.

What is llms.txt?

llms.txt is a Markdown file at the root of a website (/llms.txt) that gives large language models a short, curated map of the site. It starts with an H1 name and a one-line summary, then lists the important pages as Markdown links with notes. Agents read it to find the right page without parsing the whole site.

The format was proposed by Jeremy Howard of Answer.AI on September 3, 2024, and the llms.txt proposal was updated to v2 in August 2026. The proposal lives in an open GitHub repository.

Definition: llms.txt is a Markdown index for AI agents, placed at /llms.txt (or at a subpath such as /docs/llms.txt), that summarizes a site and links to its most useful pages. llms-full.txt is a common companion file that includes the content itself, or excerpts of it, in one document.

TL;DR

llms.txt is a Markdown file at /llms.txt with an H1, a blockquote summary and H2 sections of links.
Only the H1 is required. An ## Optional section marks links an agent can skip.
It is a guide for agents at answer time, not an access control file like robots.txt.
Google says you do not need it to appear in AI Overviews or AI Mode, so treat it as a low-cost extra for agents and AI assistants, not a ranking trick.
Generate one free at tools.fabwaseem.com/t/llms-txt-generator, edit it, and serve it as plain text.

What is llms.txt used for?

llms.txt is used to give AI agents a fast, clean starting point. A coding agent can read a library's llms.txt to find the right API page. A chat assistant browsing your site can read it to learn who you are and where your pricing, docs or blog live. It saves tokens because the agent skips navigation, ads and scripts.

The proposal says the files are used most heavily for software documentation, and that the AI labs publish their own: OpenAI (developers.openai.com/llms.txt), Anthropic (docs.anthropic.com/llms.txt) and Google's Gemini API docs (ai.google.dev/gemini-api/docs/llms.txt). It also lists platforms that generate the file automatically, including Mintlify, GitBook, Yoast SEO, AIOSEO and Wix.

Typical uses:

Site type	What llms.txt points to
Developer docs	Quick start, API reference, SDK guides, changelog
SaaS product	Product overview, pricing, features, docs, support policy
Personal or portfolio site	About, projects, CV, contact, best articles
Blog or publication	Pillar articles, topic hubs, author page
Store	Category pages, shipping and returns policy, size guides

Does llms.txt help SEO or AI search rankings?

llms.txt does not directly improve Google rankings. Google's guidance on AI features and your website says there are no extra requirements for AI Overviews or AI Mode and that you do not need new machine-readable files or AI text files to appear in them. Its value is for agents that choose to read it.

So be honest about the trade-off. llms.txt costs a few minutes to publish and can help any assistant or coding agent that fetches it. It will not replace the basics that every AI answer engine still depends on: crawlable pages, a sitemap, clear titles, structured data that matches the page, and content that answers the question in its first lines. If you want to check those basics, run your site through the free AI SEO checker; it also checks for llms.txt.

What is the llms.txt format?

The format, from the official proposal, is Markdown sections in a fixed order: an optional byte-order mark, an H1 with the site or project name (the only required part), a blockquote summary, optional paragraphs or lists with no headings, then zero or more H2 sections that each contain a list of [name](url) links with optional : notes.

Here is the shape:

# Site name

> One-sentence summary with the key facts an agent needs.

Optional notes about how to read the links below.

## Docs

- [Quick start](https://example.com/docs/start.md): Install and first request
- [API reference](https://example.com/docs/api.md): Every endpoint and parameter

## Optional

- [Changelog](https://example.com/changelog)

Rules worth remembering:

Element	Required?	Rule
H1 (`# Name`)	Yes	The only required section
Blockquote (`> summary`)	No	Short summary with key facts
Free text	No	Paragraphs or lists, no headings
H2 file lists	No	Markdown list of `[name](url): notes`
`## Optional`	No	By convention, links an agent can skip
Location	Yes	`/llms.txt` at the root or any subpath

The proposal also suggests offering a clean Markdown version of important pages at the same URL with .md added, and linking them with rel="alternate" type="text/markdown". That is optional, and many sites start with the index alone.

What is the difference between llms.txt, llms-full.txt, robots.txt and sitemap.xml?

robots.txt says which crawlers may fetch which paths. sitemap.xml lists every indexable URL for search engines. llms.txt is a short, curated Markdown guide for AI agents with notes about each link. llms-full.txt is a larger companion that holds page content or excerpts in one file, so an agent can read everything in a single request.

File	Purpose	Audience	Format	Size
robots.txt	Allow or block crawling (RFC 9309)	All crawlers	Plain text directives	Tiny
sitemap.xml	List every indexable URL (sitemaps.org)	Search engines	XML	Can be large
llms.txt	Curated map with notes	AI agents and assistants	Markdown	Small, fits in context
llms-full.txt	Content or excerpts in one file	AI agents	Markdown	Larger

The proposal makes the same point: a sitemap often lacks LLM-readable versions, leaves out helpful external links, and is usually too big to fit in a model's context window. llms.txt is the short version.

How do I create an llms.txt file with the free generator?

Enter your homepage URL in the free llms.txt generator. It reads Sitemap: lines in your robots.txt, then your sitemap (falling back to homepage links), visits up to 30 pages, groups them into sections and writes llms.txt plus llms-full.txt. Review the preview, edit it, then copy or download it.

Step by step:

Open the tool at tools.fabwaseem.com/t/llms-txt-generator.
Pick a mode. Use Site URL to crawl from your homepage, or Paste URLs to list up to 30 exact pages you want included.
Enter your homepage, for example https://example.com, and run it. The tool reads robots.txt for Sitemap: entries, then the sitemap. If there is no sitemap, it uses links from your homepage.
Check the summary. The H1 and blockquote come from your homepage title and meta description. Rewrite the summary so it states who you are and what you offer in one sentence.
Review the sections. Pages are grouped by path: the homepage under Main, /blog/ or /articles/ under Blog, /docs/-style API paths under API, /pricing and /features under Product, /about and /careers under About, and privacy, terms and login pages under Optional. Other paths get a section named after their first folder.
Edit the notes. Each link carries a note from the page's description. Shorten them to what an agent needs to know.
Copy or download llms.txt, and llms-full.txt if you want it. The full file adds a short excerpt (up to about 800 characters) of each page.
Publish the file at https://yourdomain.com/llms.txt and confirm it loads as plain text.

What the tool does and does not do, so you know the limits:

Behavior	Detail
Pages read	Up to 30 pages per run, up to 300 sitemap URLs considered
Sources	robots.txt `Sitemap:` lines, sitemap index (up to 4 child sitemaps), homepage links
Safety	Public http/https only; private, loopback and link-local addresses refused after DNS
Limits	8-second timeout per request, size caps per file, a few runs per visitor every 10 minutes
Output	Editable llms.txt and llms-full.txt, copy and download
Account	None, free

How do I add llms.txt to a Next.js site?

In the Next.js App Router, either put a static llms.txt file in the public/ folder, or create a route handler at app/llms.txt/route.ts that returns the text with a text/plain content type. The route handler is better when your pages come from a CMS, because the file stays in sync with your content.

Static option: save the generated file as public/llms.txt. Next.js serves it at /llms.txt.

Dynamic option:

// app/llms.txt/route.ts
import { getPublishedPosts } from "@/lib/posts"

export const revalidate = 3600

export async function GET() {
  const posts = await getPublishedPosts()
  const lines = [
    "# Example Studio",
    "",
    "> Example Studio builds Next.js apps and publishes guides on self-hosting and AI search.",
    "",
    "## Main",
    "",
    "- [Home](https://example.com/): Services and contact",
    "- [Projects](https://example.com/projects): Case studies",
    "",
    "## Blog",
    "",
    ...posts.map((p) => `- [${p.title}](https://example.com/blog/${p.slug}): ${p.excerpt}`),
  ]
  return new Response(lines.join("\n"), {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  })
}

This site does exactly that: fabwaseem.com/llms.txt is built from published posts and projects. For the rest of the setup (metadata, sitemap and JSON-LD), see my guide to Next.js SEO for AI search.

How do I check that my llms.txt works?

Open https://yourdomain.com/llms.txt in a browser and confirm it returns HTTP 200 with a text/plain or text/markdown content type, starts with an H1, and has absolute https links that all load. Then give only that URL to an AI assistant and ask it questions about your site to see whether it finds the right pages.

A quick checklist:

curl -I https://yourdomain.com/llms.txt returns 200.
The first line is # Your name.
The blockquote is one or two sentences with real facts, not slogans.
Every link is absolute and returns 200, with no redirects to login pages.
The file is small. If it is long, move secondary links to ## Optional.
robots.txt does not block the path for the crawlers you care about. Each AI company documents its own user agents: OpenAI, Anthropic and Perplexity.

The AI SEO checker runs the H1 and link checks for you as part of a full report.

What should I put in llms.txt?

Put the pages that answer the questions people actually ask about you: what you do, pricing or services, docs or guides, key articles, and how to contact you. Use a one-sentence factual summary, short notes per link, and an Optional section for legal pages and archives. Leave out thin pages, tag archives, search pages and anything behind a login.

Good and bad examples:

Do	Avoid
`> Acme is a time-tracking app for agencies, from $9 per user per month.`	`> The best, most innovative solution for everyone.`
`- [Pricing](https://acme.com/pricing): Plans, limits and the free tier`	`- [Click here](https://acme.com/p?id=7)`
10 to 40 curated links	Every URL from the sitemap
Absolute https URLs	Relative links or staging domains
Legal pages under `## Optional`	Legal pages at the top

Common llms.txt mistakes

The most common mistakes are serving the file as HTML instead of plain text, missing the H1, using relative links, dumping the entire sitemap, and writing a marketing tagline instead of a factual summary. Each one makes the file harder for an agent to parse or less useful once parsed.

Served as HTML. A framework's 404 page or a rewrite returns HTML. Check the content type.
No H1. The H1 is the one required element.
Headings in the intro. Only H2 headings are allowed, and only for file lists.
Relative links. Agents may read the file out of context. Use full URLs.
Stale links. Regenerate after big content changes, or build it from your CMS.
Expecting a ranking boost. It helps agents that read it. It is not a Google ranking factor.

Frequently asked questions

Is the llms.txt generator free?

Yes. The generator at tools.fabwaseem.com/t/llms-txt-generator is free, needs no account and adds no watermark or branding to the file. There is a small per-visitor rate limit so the tool stays fast for everyone.

Where do I put the llms.txt file?

Put it at the root of your domain so it loads at https://yourdomain.com/llms.txt. The proposal also allows subpaths, such as /docs/llms.txt for documentation only, and agents should use the most specific file that covers a page.

Does Google use llms.txt?

Google's documentation for AI Overviews and AI Mode says you do not need new machine-readable files or AI text files to appear in those features. Treat llms.txt as a help for agents and assistants that read it, not as a Google ranking signal.

What is llms-full.txt?

llms-full.txt is a companion file that puts page content, or excerpts of it, into one Markdown document so an agent can read everything in one request. The generator writes one with a short excerpt per page. Keep llms.txt as the short index.

How often should I update llms.txt?

Update it whenever you add or remove important pages. If your site uses a CMS, generate the file from your content with a route handler so it never goes stale. Otherwise, rerun the generator after major releases.

Is llms.txt the same as robots.txt?

No. robots.txt controls which crawlers may fetch which paths. llms.txt does not grant or block anything. It is a Markdown guide that tells agents what your site contains and which pages matter.

Key takeaways

llms.txt is a Markdown map for AI agents at /llms.txt: an H1, a summary and H2 lists of links.
It complements robots.txt and sitemap.xml. It does not replace them and is not a Google ranking factor.
Keep it short and factual, with absolute links and an ## Optional section for secondary pages.
In Next.js, serve it from public/ or a route handler that builds it from your content.
Generate a draft in seconds with the free llms.txt generator, then edit it.

Try the llms.txt generator

Paste your homepage into the free llms.txt generator, edit the summary, and publish the file today. Then run the page through the AI SEO checker to see what else AI answer engines need from your site. More free tools are on the tools page, and you can see what I build on my projects page.

Originally published on fabwaseem.com.

How to Self-Host Next.js with Coolify: Complete Production Guide

Waseem Anjum — Thu, 29 Oct 2026 01:05:08 +0000

Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com.

To self-host Next.js with Coolify, install Coolify on a Linux VPS, connect your Git repository, pick the Nixpacks or Dockerfile build pack, set port 3000 and your environment variables, then point a domain at the server. Coolify builds the image, runs the container, and puts it behind a reverse proxy with automatic HTTPS, so every git push becomes a production deploy without a managed platform in the middle.

This guide walks through the whole path: server sizing, installing Coolify, choosing between Nixpacks and a hand-written Dockerfile with output: 'standalone', wiring up PostgreSQL, handling build-time vs runtime environment variables, and the production details that usually bite people later (memory limits, image optimization, caching, health checks and rollbacks). It is the same kind of setup that serves this site, fabwaseem.com, and projects like Foliora.

Why self-host Next.js with Coolify instead of a managed platform?

Managed platforms are excellent at zero-config deploys. The trade-off is that pricing, limits and runtime behavior are decided for you. Coolify is an open-source, self-hostable alternative that gives you a similar push-to-deploy workflow on hardware you control.

What you get:

Predictable cost. You pay for the server, not per request, per build minute or per seat.
One box, many apps. Next.js apps, APIs, workers, PostgreSQL, Redis and static sites can share a server.
Full Node.js runtime. next start runs as a long-lived Node process, so server actions, route handlers, ISR and middleware behave like they do locally.
No lock-in. Under the hood it is Docker containers and a reverse proxy. You can leave with your Dockerfile.

What you take on: OS updates, backups, monitoring and capacity planning. If you are not willing to own those, stay on a managed host.

When is a managed platform still the better choice?

If you rely heavily on a global edge network, per-region functions or a vendor's integrated analytics and image CDN, a managed platform saves real work. Self-hosting shines for full-stack apps with a database, background jobs and steady traffic, where one well-sized VPS does the job.

What do you need before you start?

Requirement	Minimum	Notes
Server	Linux VPS with SSH access, amd64 or arm64	Ubuntu LTS or Debian is the easy path
CPU / RAM	2 cores, 2 GB RAM (Coolify's documented minimum)	Next.js builds are memory-hungry; give yourself headroom
Disk	10 GB free	Docker images and build cache grow fast; plan for more
Domain	An A record you control	Needed for automatic HTTPS
Repository	GitHub, GitLab, Bitbucket or any Git URL	GitHub App integration enables push-to-deploy
Ports	22, 80, 443 open (8000 for the first login)	Close 8000 once the dashboard has a domain

The official Coolify installation guide lists the full requirements and supported distributions.

How do you install Coolify on a VPS?

SSH into a fresh server as root (or a sudo user) and run the official install script:

curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash

The script installs Docker if needed, pulls the Coolify containers and prints the dashboard URL. Then:

Open http://YOUR_SERVER_IP:8000 and create the admin account immediately. Whoever registers first owns the instance.
Back up /data/coolify/source/.env. It contains APP_KEY, which encrypts values in Coolify's database; you need it to restore.
In Settings, give the dashboard its own domain (for example coolify.example.com) so it is served over HTTPS, then close port 8000 in your firewall.

A basic firewall with UFW looks like this:

ufw default deny incoming
ufw default allow outgoing
ufw allow OpenSSH
ufw allow 80/tcp
ufw allow 443/tcp
ufw enable

Docker publishes container ports through its own iptables rules, which can bypass UFW. Don't publish database ports to the host unless you mean to; keep them on Coolify's internal network.

How do you deploy a Next.js app on Coolify?

Create a project in the dashboard (Projects → Add). Projects group an app with its databases and environments.
Add a resource → Application → choose your Git source. With the GitHub App, Coolify gets a webhook and redeploys on every push to the branch you select.
Pick a build pack. Start with Nixpacks for a zero-config build, or Dockerfile for full control (covered below).
Set the exposed port to 3000, the default for next start.
Add a domain in the format https://app.example.com. Using https:// tells Coolify to request a Let's Encrypt certificate through its proxy.
Add environment variables (next section), then hit Deploy and watch the build log.

Point the domain's A record at your server's IP before deploying so the certificate challenge can succeed. If you sit behind Cloudflare's proxy, use Full (strict) SSL mode so you don't create a redirect loop.

Nixpacks or Dockerfile: which build pack should you use?

	Nixpacks	Dockerfile
Setup	None; detects Next.js from `package.json`	You write and maintain the Dockerfile
Image size	Larger; ships the full `node_modules`	Small with `output: 'standalone'`
Control	Limited; tweak with env vars or `nixpacks.toml`	Total: base image, build args, user, layers
Node version	Detected from `engines` or `.nvmrc`	Pinned in `FROM`
Best for	Prototypes, previews, quick side projects	Production apps you'll run for a long time

The Coolify Next.js guide documents both paths. Nixpacks is the fastest way to see your app live. For production, I prefer a Dockerfile: builds are reproducible, the image is small and the Node version doesn't change under you.

If you stay on Nixpacks, pin Node explicitly so a detector update doesn't surprise you:

{
  "engines": {
    "node": ">=22 <23"
  }
}

How do you build a production Dockerfile with standalone output?

Next.js can trace exactly which files the server needs and emit a self-contained folder. Enable it in next.config.ts:

import type { NextConfig } from 'next';

const nextConfig: NextConfig = {
  output: 'standalone',
};

export default nextConfig;

next build now writes .next/standalone with a minimal server.js and only the traced dependencies. Static assets are not copied in automatically, so the Dockerfile has to copy .next/static and public itself. See the output option in the Next.js docs.

A multi-stage Dockerfile based on the pattern in the official Next.js examples:

# syntax=docker/dockerfile:1
FROM node:22-alpine AS base

FROM base AS deps
RUN apk add --no-cache libc6-compat
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci

FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
ENV NEXT_TELEMETRY_DISABLED=1
RUN npm run build

FROM base AS runner
WORKDIR /app
ENV NODE_ENV=production
ENV NEXT_TELEMETRY_DISABLED=1
RUN addgroup --system --gid 1001 nodejs \
 && adduser --system --uid 1001 nextjs

COPY --from=builder /app/public ./public
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs
EXPOSE 3000
ENV PORT=3000
ENV HOSTNAME=0.0.0.0
CMD ["node", "server.js"]

Two lines matter more than they look:

HOSTNAME=0.0.0.0 makes the server listen on all interfaces. Without it, the container can listen only on its internal hostname and the proxy returns a 502 or "no available server".
The non-root nextjs user limits the blast radius if the app is ever compromised.

If you use Prisma, add RUN npx prisma generate in the builder stage before npm run build, and make sure the generated client ends up in the traced output (it does by default when imported from server code).

Commit the Dockerfile, switch the build pack to Dockerfile in Coolify, keep port 3000, and redeploy.

How do you handle environment variables and secrets?

This is the most common source of "works locally, broken in production". Next.js has two kinds of variables:

NEXT_PUBLIC_* variables are inlined into the client bundle at build time. Changing them in the dashboard does nothing until you rebuild.
Server-only variables (DATABASE_URL, API keys, auth secrets) are read at runtime by the Node process.

In Coolify each variable has a Build Variable toggle. Turn it on for anything next build needs: every NEXT_PUBLIC_* value, plus anything read during static generation. With a Dockerfile build pack, build variables are passed as build args, so declare them in the builder stage:

FROM base AS builder
ARG NEXT_PUBLIC_SITE_URL
ENV NEXT_PUBLIC_SITE_URL=$NEXT_PUBLIC_SITE_URL

Keep real secrets runtime-only. Don't bake them into image layers, because anyone with the image can read them. Validate variables at startup with a schema (for example Zod) so a missing value fails the deploy loudly instead of throwing on the first request.

How do you add a PostgreSQL database in Coolify?

Inside the same project, add a resource → Database → PostgreSQL. Coolify starts it on its internal Docker network and shows two connection strings:

Internal URL, using the container name as host. Use this as DATABASE_URL for apps on the same server. Traffic never leaves the box.
Public URL, only if you enable public access. Leave that off unless you really need external access, and prefer an SSH tunnel for ad-hoc admin work.

Run migrations as part of deploy, not by hand. With Prisma, a clean pattern is to run them before the server starts:

{
  "scripts": {
    "build": "prisma generate && next build",
    "start:prod": "prisma migrate deploy && node server.js"
  }
}

If you go this route, the runner image needs the Prisma CLI and the prisma folder. Alternatively, use Coolify's pre-deployment command setting to run npx prisma migrate deploy in the new container before traffic switches. Then turn on scheduled backups for the database and send them to S3-compatible storage. A backup that only lives on the same disk is not a backup.

What should you configure for production?

Memory and CPU limits

Coolify lets you set resource limits per application. Set them. A runaway Node process or a memory leak should restart one container, not take down the server and every other app on it. Builds are the hungriest step, because next build on a mid-size app can spike well above the app's runtime usage, so leave headroom or use a separate build server once you run several apps.

Health checks and zero-downtime deploys

Add a lightweight route handler and point Coolify's health check at it:

// app/api/health/route.ts
export const dynamic = 'force-dynamic';

export function GET() {
  return Response.json({ ok: true });
}

With a passing health check, Coolify only switches traffic to the new container once it is healthy, and keeps the old one if the new one fails. Your users never see a half-started app.

Image optimization

next/image optimizes on the server. In a self-hosted container that means the Node process does the work and caches results on local disk. Recent Next.js versions use sharp automatically, but it pays to keep an eye on CPU during traffic spikes. If images are a big part of the site, offload them to a media CDN with a custom loader and let the app server focus on rendering.

Caching and ISR

ISR and the data cache write to the container's filesystem by default. That works perfectly on a single container, but the cache resets on every deploy and isn't shared between replicas. If you scale out horizontally, configure a shared cache handler (Redis is the common choice). The Next.js self-hosting guide covers cache handlers, image optimization and streaming behind a proxy.

Rollbacks

Every deployment is kept as an image, so rolling back is a click in the Deployments tab. Pair that with database migrations that are backward-compatible (add columns first, remove them in a later release) so rolling back the app never breaks the schema.

How do you troubleshoot common Coolify + Next.js problems?

Symptom	Likely cause	Fix
502 / "no available server"	App listens on the wrong host or port	Set `HOSTNAME=0.0.0.0`, match "Ports Exposes" to `PORT`
Build killed, exit code 137	Out of memory during `next build`	Add RAM or swap, or move builds to a build server
`NEXT_PUBLIC_*` value is empty	Not available at build time	Mark it as a Build Variable and redeploy
Unstyled pages or 404 on `/_next/static`	Static folder not copied into standalone output	Copy `.next/static` and `public` in the runner stage
Certificate never issues	DNS not pointing at the server, or port 80 blocked	Check the A record, open 80/443, then redeploy
Redirect loop behind Cloudflare	Flexible SSL mode	Switch Cloudflare to Full (strict)

When something fails, read the deployment log first, then the container logs in the app's Logs tab. Nearly every problem above shows up clearly in one of the two.

Is self-hosting Next.js with Coolify worth it?

If you run more than one app, need a database next to your app and want costs you can predict, yes. Coolify gives you most of the managed-platform experience (Git deploys, previews, HTTPS, rollbacks, backups) on a server you own, and the escape hatch is a plain Dockerfile. Start with Nixpacks to get live, move to a standalone Dockerfile for production, and put limits, health checks and off-site backups in place before you put real traffic on it.

Want to see what runs on this kind of stack? Browse my projects, including Snagbox, or read the Docker multi-stage build docs to tune your images further.

Related: not sure Coolify is the right panel? Coolify vs Dokploy compares idle RAM, disk, architecture and security advisories, measured on identical fresh Ubuntu VMs.

FAQ

Can you self-host Next.js with Coolify for free?

Coolify itself is open source and free to self-host. You still pay for the server it runs on, and optionally for off-site backup storage and a domain.

Does Next.js work fully when self-hosted?

Yes. Running next start (or the standalone server.js) on Node.js supports server components, server actions, route handlers, middleware, ISR and image optimization. Features tied to a specific vendor's platform need their own replacement.

Should I use Nixpacks or a Dockerfile for Next.js on Coolify?

Nixpacks is zero-config and great for getting live or for previews. A multi-stage Dockerfile with output: 'standalone' gives a smaller image, a pinned Node version and reproducible builds, which is the better fit for production.

Why are my NEXT_PUBLIC environment variables empty after deploying?

NEXT_PUBLIC_* values are inlined into the client bundle during next build. In Coolify, mark them as Build Variables (and declare them as ARG in a Dockerfile), then redeploy so the build picks them up.

How much RAM do I need to run Next.js on Coolify?

Coolify's documented minimum is 2 CPU cores and 2 GB RAM for the platform itself. Next.js builds can spike memory well above runtime usage, so leave headroom, add swap, or use a separate build server as you add apps.

Is Coolify a good Vercel alternative for Next.js?

For full-stack apps with a database, workers and steady traffic, Coolify is a strong alternative with predictable costs and no lock-in. If you depend on a global edge network or vendor-integrated features, a managed platform may still save you work.

Originally published on fabwaseem.com.

Postgres Job Queues in 2026: SKIP LOCKED vs pg-boss vs BullMQ (Redis and Postgres), Benchmarked

Waseem Anjum — Wed, 28 Oct 2026 19:11:09 +0000

Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com.

A Postgres job queue used to mean one of two things in Node.js: a hand-written FOR UPDATE SKIP LOCKED query or pg-boss. Since BullMQ 6 shipped in July 2026, there is a third option: BullMQ now runs on PostgreSQL as well as Redis, with the same Queue and Worker API. So I put all four on one box and measured them: enqueue speed, processing throughput, how fast an idle worker picks up a new job, what happens when a worker is killed mid-job, and how much space a waiting job takes.

Three results surprised me. pg-boss with its default worker settings processed 5 jobs per second with 10 workers, because each worker fetches one job and then waits out a 2-second polling interval. One option fixed that, and pg-boss then matched BullMQ on Redis at about 29,000 jobs/s. The hand-written SKIP LOCKED worker got a killed worker's job back in 0.01 seconds, while every library took 60 seconds with its defaults. And once each job did 10 ms of simulated work, all four landed within about 12% of each other.

TL;DR: SKIP LOCKED vs pg-boss vs BullMQ

If your data already lives in PostgreSQL and you process up to a few thousand jobs per second, you don't need Redis for a job queue. Use pg-boss, but set batchSize and turn on LISTEN/NOTIFY, or its defaults will cap you at a few jobs per second. Write your own SKIP LOCKED queue only for simple, short jobs. Pick BullMQ on Redis when you need the highest throughput or BullMQ's features like flows, rate limiting and its dashboards, and try BullMQ's Postgres backend if you want that API without running Redis.

50,000 no-op jobs, 10 workers (median of 3)	Jobs/s	Idle pickup p50	Killed job back after	Bytes per waiting job
SKIP LOCKED, my 60 lines, 1 job per fetch	12,588	1.71 ms (LISTEN/NOTIFY)	0.01 s	213
SKIP LOCKED, 50 jobs per fetch	148,341	not tested	not tested	213
pg-boss 12.38.0, defaults	5	9,665 ms (fell behind)	15+ min (default expiry, not measured)	297
pg-boss 12.38.0, `batchSize: 50` + notify	29,323	3.19 ms	60 s (expireInSeconds 30)	297
BullMQ 6.5.0 on Redis 8.0.2	27,461	1.25 ms	60 s	618 (RAM)
BullMQ 6.5.0 on PostgreSQL 18.6	7,045	2.69 ms	60 s	576

"Idle pickup" is the time from add() to the handler starting while workers wait for work. "Killed job back after" is how long a job stayed stuck after I SIGKILLed the worker holding it. The pg-boss throughput row with batchSize: 50 ran without notify, because notify only changes how fast workers wake up for new jobs.

How I tested

Everything ran on one Linux VM: 8 vCPUs (Intel Xeon, KVM), 15 GB of RAM, Debian 13. Database, Redis and Node.js shared the machine and talked over loopback TCP, so there's no network latency in these numbers. On a real network every round trip costs more, and the per-job gaps grow.

PostgreSQL 18.6 from the PGDG apt repo, with shared_buffers = 1GB and max_connections = 300. Everything else stayed at defaults, including fsync = on and synchronous_commit = on, so every commit waited for the WAL flush.
Redis 8.0.2, the Debian 13 package, with Debian's default config: RDB snapshots, no AOF. That means Redis acknowledged writes before they reached disk while Postgres flushed every commit. That's how most people run them, but it's a big part of the throughput gap, so keep it in mind.
Node.js 22.20.0 with the latest npm releases on October 28, 2026: pg 8.23.1, pg-boss 12.38.0, bullmq 6.5.0 and ioredis 6.0.0.
Each test ran in a fresh Node process against an empty queue: I dropped the table or schema (or ran FLUSHALL on Redis) first. Payload: { userId, email, template }, about 70 bytes of JSON. I report the median of 3 runs unless I say otherwise.
"Workers" means jobs in flight at once: BullMQ's concurrency, pg-boss's localConcurrency and the number of polling loops (each with its own connection) in my SKIP LOCKED worker.

The scripts and raw JSON results are in my working folder. Every number below comes from them.

The contenders

SKIP LOCKED, hand-written. One table, one index, and a worker that claims rows with FOR UPDATE SKIP LOCKED inside a transaction. When the handler finishes, the transaction commits and the row is gone. If the handler throws, the transaction rolls back and the job is available again. A statement-level trigger sends NOTIFY on insert so idle workers wake up at once.

pg-boss 12.38.0. The established Postgres queue for Node.js: retries with backoff, cron schedules, dead-letter queues, priorities, singleton and debounce policies, group concurrency and flows. It also uses SKIP LOCKED under the hood.

BullMQ 6.5.0 on Redis. The most popular Node.js queue, built on Redis Lua scripts. This is the setup from my AI automation pipeline with BullMQ post.

BullMQ 6.5.0 on PostgreSQL. New in v6. You install pg and pass createPostgresBackend as the last constructor argument. BullMQ creates its own bullmq schema with tables and SQL functions, claims jobs with FOR UPDATE SKIP LOCKED and waits with LISTEN/NOTIFY, according to its source. The BullMQ docs still call Redis "the default and the most battle-tested option".

Enqueue speed

How fast each one accepts new jobs: 5,000 add() calls one after another, 20,000 with 50 in flight, and 20,000 in bulk batches of 500.

Enqueue (jobs/s, median of 3)	One at a time	p50 per add	50 in flight	Bulk, 500 per call
SKIP LOCKED (`INSERT`)	5,651	0.117 ms	14,611	178,287
pg-boss `send()` / `insert()`	1,301	0.716 ms	10,463	49,164
BullMQ on Redis `add()` / `addBulk()`	15,359	0.056 ms	67,909	40,202
BullMQ on Postgres `add()` / `addBulk()`	4,263	0.172 ms	12,358	33,151

Redis wins single adds by a wide margin, which is what you'd expect from an in-memory store that doesn't wait for disk. In bulk, Postgres catches up: a multi-row INSERT of 500 jobs commits once, so the flush cost is shared. My plain INSERT ... SELECT FROM jsonb_array_elements(...) was the fastest bulk path of all. pg-boss's single send() was the slowest at 0.72 ms per call (I didn't profile why), but its bulk insert() beat both BullMQ backends.

Processing throughput

I loaded 50,000 jobs, started the workers and timed how long it took to drain the queue. The handler did nothing, so this measures pure queue overhead. Runs were capped at 30 seconds, and a capped run reports jobs processed divided by 30.

Drain 50,000 no-op jobs (jobs/s, median of 3)	1 worker	10 workers	50 workers
SKIP LOCKED, 1 job per fetch	2,791	12,588	12,433
SKIP LOCKED, 50 jobs per fetch	63,972	148,341	88,266
pg-boss, defaults (`batchSize: 1`)	0.5 (capped)	5 (capped)	25 (capped)
pg-boss, `batchSize: 50`, `burstWhenBatchFull`	5,763	29,323	28,389
BullMQ on Redis	10,556	27,461	29,969
BullMQ on Postgres	982 (capped)	7,045	9,185

A few things stand out:

Batching is the biggest lever on Postgres. Claiming 50 rows per query cut the per-job cost so much that my 60-line worker drained 50,000 jobs in a third of a second. The catch with my version: all 50 jobs share one transaction, so one failing job rolls back the whole batch.
More workers stopped helping at 10. With Postgres, Node.js and 50 connections fighting over 8 vCPUs, 50 workers were no faster than 10, and for batched SKIP LOCKED they were slower.
BullMQ on Postgres was about 4x slower than on Redis at 10 workers (7,045 vs 27,461 jobs/s). The BullMQ docs say to expect "~1.5–2× fewer jobs/s" for processing. On my box the gap was bigger, and with one worker it was about 11x. Each job state change is a durable Postgres transaction here, while Redis with default settings never waits for disk.

The pg-boss default that ran 5 jobs per second

This is the result I'd most like every pg-boss user to see. With work() and default options, 10 workers processed 150 jobs in 30 seconds.

The reason is in pg-boss's worker loop. Each worker fetches batchSize jobs (default 1), runs the handler, and then waits for pollingIntervalSeconds (default 2) before fetching again. It only skips the wait when a NOTIFY arrives or when the fetch returned a full batch with burstWhenBatchFull on, and that option is documented as ignored when batchSize is 1. So localConcurrency: 10 with defaults means 10 jobs every 2 seconds, however long the backlog is.

The fix is one object:

```ts title="pg-boss worker that keeps up"
import { PgBoss } from "pg-boss";

const boss = new PgBoss({
connectionString: process.env.DATABASE_URL,
useListenNotify: true, // one dedicated LISTEN connection
});
await boss.start();
await boss.createQueue("emails", { notify: true }); // NOTIFY on every new job

await boss.work(
"emails",
{
localConcurrency: 10,
batchSize: 50, // fetch up to 50 jobs per query
burstWhenBatchFull: true, // keep fetching while batches come back full
},
async (jobs) => {
for (const job of jobs) await sendEmail(job.data);
},
);




With `batchSize: 50` and `burstWhenBatchFull: true`, the same 10 workers drained 50,000 jobs at 29,323 jobs/s, a bit faster than BullMQ on Redis in my runs. The handler always receives an array of jobs, now up to 50 at a time. If it throws, the whole batch fails and retries, so either keep handlers idempotent or set `perJobResults: true` and return a result for each job.

## Pickup latency: polling vs LISTEN/NOTIFY

Throughput is about backlogs. Most apps spend their time with an empty queue, where the question is how long a new job waits before a worker starts it. I started 10 idle workers, added 300 jobs at 20 per second and measured from just before `add()` to the moment the handler started.

| Idle pickup, 300 jobs at 20/s (median of 3) | p50 | p95 | p99 |
| --- | --- | --- | --- |
| SKIP LOCKED + `NOTIFY` trigger | 1.71 ms | 2.54 ms | 19.01 ms |
| pg-boss, defaults (polling every 2 s, `batchSize: 1`) | 9,665 ms | 18,446 ms | 18,704 ms |
| pg-boss, `batchSize: 50`, polling every 2 s | 972 ms | 1,905 ms | 1,972 ms |
| pg-boss, `useListenNotify` + queue `notify: true` | 3.34 ms | 6.23 ms | 14.90 ms |
| pg-boss, notify + `batchSize: 50` | 3.19 ms | 4.38 ms | 20.57 ms |
| BullMQ on Redis | 1.25 ms | 2.11 ms | 2.80 ms |
| BullMQ on Postgres | 2.69 ms | 3.54 ms | 18.70 ms |

With defaults, pg-boss couldn't even keep up with 20 jobs per second: only 130 of the 300 jobs had started 10 seconds after the last one was added, and the backlog kept growing. With `batchSize: 50` it kept up, but each job still waited for the next poll, about 1 second on average and up to 2. Turning on LISTEN/NOTIFY brought pickup down to 3 ms, close to BullMQ.

LISTEN/NOTIFY holds one dedicated connection per process, and pg-boss's docs warn it won't work through PgBouncer in transaction pooling mode. Polling keeps running as a fallback, so a broken listener costs latency, not jobs.

## Crash recovery: what happens to a job when a worker dies

I added one job, let a worker in a child process start it, killed that process with `SIGKILL` 200 ms later, started a fresh worker and timed how long the job took to come back.

| Worker killed mid-job | Job available again after |
| --- | --- |
| SKIP LOCKED (job claimed inside a transaction) | 0.01 s |
| BullMQ on Redis, defaults | 60.0 s |
| BullMQ on Postgres, defaults | 60.0 s |
| pg-boss, `expireInSeconds: 30` | 60.0 s |
| pg-boss, `heartbeatSeconds: 10` | 60.0 s (1 run) |

The SKIP LOCKED worker recovers instantly because the claim is a row lock inside an open transaction. When the process dies, its TCP connection closes, Postgres rolls the transaction back and the next worker can lock the row. Nothing has to time out.

The libraries mark the job as active and rely on a timer to notice it was abandoned. BullMQ's defaults are a 30-second `lockDuration` and a 30-second `stalledInterval`, and the job came back after 60 seconds on both backends. pg-boss expires abandoned jobs in its maintenance pass, which runs every 60 seconds by default (`monitorIntervalSeconds` and `superviseIntervalSeconds`), so `expireInSeconds: 30` and `heartbeatSeconds: 10` both came back at 60 seconds. With pg-boss's default `expireInSeconds` of 900, a crashed job would stay stuck for at least 15 minutes. I didn't wait that one out.

The instant recovery has a limit. If a worker's machine disappears instead of its process (a network partition or a VM that freezes), Postgres doesn't see the socket close until TCP keepalives give up. Set `tcp_keepalives_idle` and friends on the server, and `idle_in_transaction_session_timeout` as a backstop.

## Enqueue a job in the same transaction as your data

This is the feature Redis can't give you. A classic bug: you insert an order, commit, then crash before you enqueue the confirmation email. Or you enqueue first, the order insert fails, and the customer gets an email for an order that doesn't exist. If the job lives in the same database, it can commit or roll back with the order.

pg-boss supports this with the `db` option, which takes any object with an `executeSql` method:



```ts title="Enqueue inside your own transaction (pg-boss)"
const client = await pool.connect();
try {
  await client.query("BEGIN");
  const { rows } = await client.query(
    "INSERT INTO orders (total) VALUES ($1) RETURNING id", [total]);
  await boss.send("emails", { orderId: rows[0].id }, {
    db: { executeSql: (text, values) => client.query(text, values) },
  });
  await client.query("COMMIT");
} catch (err) {
  await client.query("ROLLBACK"); // the job disappears with the order
  throw err;
} finally {
  client.release();
}

I tested it: after ROLLBACK, both the order and the pg-boss job were gone. After COMMIT, both were there. My SKIP LOCKED table behaved the same way, since it's just an INSERT.

BullMQ's Postgres backend can't do this today. Its backend interface "intentionally exposes no connection or transaction type", in the words of its source, so queue.add() always runs on BullMQ's own pool. With either BullMQ backend you need an outbox table or an enqueue after commit.

Storage per waiting job

I enqueued 100,000 jobs in bulk and measured the growth of the queue's tables and indexes (pg_total_relation_size) or Redis used_memory. One run each.

100,000 waiting jobs	Total	Per job
SKIP LOCKED table + index	21.3 MB	213 bytes
pg-boss	29.7 MB	297 bytes
BullMQ on Postgres	57.6 MB	576 bytes
BullMQ on Redis	61.8 MB (RAM)	618 bytes

None of these matter for a normal backlog. They matter when a backlog grows to millions of jobs: on Redis it's RAM, and on Postgres it's disk. Both libraries also keep finished jobs by default: pg-boss deletes completed jobs after 7 days (deleteAfterSeconds), and BullMQ keeps completed jobs until you set removeOnComplete, so set a retention you can live with.

With real work, the queue barely matters

No-op jobs exaggerate the differences. I reran the drain with a handler that waits 10 ms (like a fast API call) and 50 jobs in flight, which caps out at 5,000 jobs/s in theory.

20,000 jobs, 10 ms each, 50 in flight (median of 3)	Jobs/s	Share of the 5,000/s ceiling
SKIP LOCKED, 1 job per fetch	4,189	84%
pg-boss, `batchSize: 50`	4,587	92%
BullMQ on Redis	4,469	89%
BullMQ on Postgres	4,037	81%

Once a job does real work, every option except pg-boss's defaults spends most of its time in your handler. For the AI jobs I usually queue, where one model call takes seconds, the queue's overhead is noise. Pick on features, operations and failure behavior, not on no-op benchmarks.

A SKIP LOCKED queue in about 60 lines

If your jobs are short and you only need "run this once, retry on failure", this is all it takes. It's the worker I benchmarked, trimmed for reading:

```sql title="schema.sql"
CREATE TABLE jobs (
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
queue text NOT NULL,
payload jsonb NOT NULL,
run_at timestamptz NOT NULL DEFAULT now(),
attempts int NOT NULL DEFAULT 0
);
CREATE INDEX jobs_fetch_idx ON jobs (queue, run_at, id);

-- Wake idle workers on insert (one NOTIFY per statement, not per row)
CREATE FUNCTION jobs_notify() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN PERFORM pg_notify('jobs', ''); RETURN NULL; END $$;
CREATE TRIGGER jobs_notify AFTER INSERT ON jobs
FOR EACH STATEMENT EXECUTE FUNCTION jobs_notify();






```ts title="queue.ts"
import pg from "pg";

const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL, max: 14 });

export async function enqueue(queue: string, payload: unknown, db: pg.Pool | pg.PoolClient = pool) {
  await db.query("INSERT INTO jobs (queue, payload) VALUES ($1, $2)", [queue, payload]);
}

export async function startWorkers(
  queue: string,
  concurrency: number,
  handler: (payload: any) => Promise<void>,
) {
  const waiters = new Set<() => void>();
  const listener = new pg.Client({ connectionString: process.env.DATABASE_URL });
  await listener.connect();
  await listener.query("LISTEN jobs");
  listener.on("notification", () => { waiters.forEach((w) => w()); waiters.clear(); });

  const loop = async () => {
    const client = await pool.connect();
    for (;;) {
      await client.query("BEGIN");
      const { rows } = await client.query(
        `DELETE FROM jobs WHERE id IN (
           SELECT id FROM jobs
           WHERE queue = $1 AND run_at <= now()
           ORDER BY run_at, id
           FOR UPDATE SKIP LOCKED
           LIMIT 1)
         RETURNING id, payload`, [queue]);
      if (rows.length === 0) {
        await client.query("COMMIT");
        // sleep until NOTIFY, or 1 s as a fallback poll
        await new Promise<void>((r) => { waiters.add(r); setTimeout(r, 1000); });
        continue;
      }
      try {
        await handler(rows[0].payload);
        await client.query("COMMIT");   // job done, row deleted
      } catch {
        await client.query("ROLLBACK"); // row is back in the queue
      }
    }
  };
  for (let i = 0; i < concurrency; i++) loop();
}

Because enqueue accepts a client, you get transactional enqueue for free: pass the client of your open transaction. I ran these two files as written: 200 jobs with 4 workers, 20 of which threw on their first try, were all processed exactly once with 0 rows left over, and a job enqueued inside a rolled-back transaction never ran.

What it doesn't do is everything pg-boss and BullMQ spent years on: retry limits and backoff (a rollback doesn't save the attempts increment), delayed retries, cron, dead-letter queues, priorities, rate limits, deduplication, metrics and a UI. It also holds a connection and an open transaction for the whole job. That's fine for jobs that take milliseconds. For jobs that take minutes it ties up your pool, and long-open transactions hold back vacuum. For long jobs, claim with a lease and commit right away:

```sql title="Lease pattern for long jobs"
-- add once: ALTER TABLE jobs ADD COLUMN locked_until timestamptz;
UPDATE jobs SET locked_until = now() + interval '5 minutes', attempts = attempts + 1
WHERE id = (
SELECT id FROM jobs
WHERE queue = $1 AND run_at <= now()
AND (locked_until IS NULL OR locked_until < now())
ORDER BY run_at, id
FOR UPDATE SKIP LOCKED
LIMIT 1)
RETURNING id, payload, attempts;
-- on success: DELETE FROM jobs WHERE id = $1




The trade-off is the same one the libraries make: a crashed job now waits for its lease to expire instead of coming back in 10 ms.

## BullMQ on PostgreSQL: setup

Switching an existing BullMQ app to Postgres is mostly one argument:



```ts title="BullMQ 6 with the PostgreSQL backend"
import { Queue, Worker, createPostgresBackend } from "bullmq";

const opts = {
  connection: { connectionString: process.env.DATABASE_URL, max: 20, migrate: true },
};

const queue = new Queue("emails", opts, createPostgresBackend);
await queue.add("welcome", { userId: 42 });

new Worker("emails", async (job) => sendEmail(job.data), { ...opts, concurrency: 10 }, createPostgresBackend);

migrate: true creates or upgrades the bullmq schema on first connect. You can also call runMigrations() from a deploy step instead. BullMQ requires PostgreSQL 13 or newer and recommends 14 or newer, per its PostgreSQL backend guide. Size the pool for your worker concurrency, because each worker's queries come out of it and the LISTEN connection is extra.

It worked without surprises in my tests, and the identical API is a real advantage if you already know BullMQ. But it's three months old, and on my box it was the slowest at processing no-op jobs. I'd use it for new projects that want BullMQ's features on a Postgres-only stack, and keep busy existing queues on Redis for now.

Which Postgres job queue should you choose?

You already run Postgres and want a full-featured queue: pg-boss. Retries, cron, dead letters, priorities and transactional enqueue, with no extra infrastructure. Set batchSize and burstWhenBatchFull on busy queues, useListenNotify plus queue notify where latency matters, and a lower expireInSeconds than the 15-minute default for short jobs.
You need the most throughput, or BullMQ's ecosystem: BullMQ on Redis. It had the fastest single enqueues, the lowest pickup latency and strong throughput without tuning. You pay with a second datastore and no transactional enqueue.
You like BullMQ but don't want Redis: BullMQ on PostgreSQL. Same code, fewer moving parts, lower throughput, and no enqueue inside your own transaction yet.
Your jobs are short and simple: a SKIP LOCKED table. Fastest in batch mode, instant crash recovery and no dependency. You own retries, scheduling and monitoring from then on.

The rough throughput limits from my no-op runs (about 7,000 to 150,000 jobs/s depending on the option) are far above what most apps need. If your queue does fewer than a few hundred jobs per second, choose on features and operations.

If you're also deciding where search or IDs live, I ran similar tests for Postgres full-text search vs Elasticsearch and UUIDv7 primary keys in Postgres. For the ORM layer on top, see Drizzle vs Prisma, and for a full worker with retries, backoff and a dead-letter queue, my BullMQ AI automation pipeline.

Limitations of this benchmark

One VM, everything on loopback. Real networks add a round trip to every query and widen the gaps between per-job and batched approaches.
Redis ran without AOF, Postgres with full durability. With appendfsync always, Redis would pay a disk flush per write too. I didn't test that, or synchronous_commit = off on Postgres.
No-op handlers measure queue overhead only. The 10 ms test is closer to real life.
Crash recovery ran 3 times per option except heartbeatSeconds: 10 (once). Storage ran once per option.
I didn't test retries, delayed jobs, priorities, cron or flows, which is where the libraries earn their keep.

Coolify vs Vercel for Next.js: Cost, Features and Trade-offs

Waseem Anjum — Tue, 27 Oct 2026 22:03:08 +0000

Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com.

Coolify vs Vercel comes down to who runs the infrastructure. Vercel is a fully managed platform that deploys Next.js with zero configuration and bills per seat plus usage, while Coolify is a free, open-source control plane that runs your app as Docker containers on servers you own, for a flat server cost.

Pick Vercel when you want zero ops, global edge delivery and polished previews out of the box. Pick Coolify when you want predictable costs, databases and background workers next to your app, and almost no platform lock-in. I've run Next.js on both: this site, fabwaseem.com, used to live on Vercel with a Neon database and now runs on Coolify with its own PostgreSQL. This guide compares the two on the things that actually decide the choice (pricing model, Next.js feature support, previews, databases, long-running work, scaling and lock-in), using the official numbers as of October 2026, and ends with a migration checklist if you decide to switch.

If you've already picked Coolify, my step-by-step guide on how to self-host Next.js with Coolify covers the full setup.

What is the real difference between Coolify and Vercel?

Vercel sells a finished platform. You connect a repository and get a managed build fleet, a serverless functions runtime, a global CDN, automatic TLS, preview URLs, analytics and spend controls. You never see a server. Vercel also maintains Next.js, so the integration is first-party.

Coolify gives you the control plane and leaves the hardware to you. In its own words, Coolify "provides the deployment control plane and coordinates Docker on servers you connect." It handles Git integration, builds (Nixpacks, Railpack, a Dockerfile or Docker Compose), the reverse proxy and certificates, one-click databases and services, preview deployments, health checks and rollbacks. The server itself is yours: its capacity, OS updates, backups and recovery.

Almost everything else in this comparison follows from that one difference.

Coolify vs Vercel: how do they compare side by side?

Here is the short version, based on Vercel's and Coolify's own documentation and pricing pages as of October 2026.

	Vercel	Coolify
Model	Managed platform	Docker on your servers
Free option	Hobby, non-commercial	Self-hosted, all features
Paid tier	Pro: $20/seat/month	Cloud: from $5/month
Billing	Seats + usage	Flat server cost
Next.js	First-party	Full, via `next start`
Edge CDN	Built in	Add your own
Previews	Every branch push	Per PR + wildcard DNS
Databases	Marketplace add-ons	One-click, on your server
Max run time	300s (800s on Pro)	No platform limit
Persistent disk	No	Volumes
Scaling	Automatic	Manual
Ops and outages	Vercel	You
Lock-in	Low to medium	Very low

The details behind each row are below.

How much does Vercel cost compared to Coolify?

The two pricing models are different shapes, so compare them on your situation rather than on a single headline number.

What does Vercel charge?

Vercel's Hobby plan is free, but it's restricted to non-commercial personal use. According to Vercel's fair use guidelines, commercial use includes any deployment built for financial gain, "including a paid employee or consultant writing the code", so client sites, SaaS products and sites with ads need Pro. Hobby also comes with fixed caps (for example 100 GB of Fast Data Transfer, 1 million function invocations and 5,000 image transformations a month), and projects are paused when you exceed them. You can't buy extra usage on Hobby.

The Pro plan costs a $20/month platform fee that includes one deploying seat and $20 of monthly usage credit. Each additional Owner or Member seat is $20/month; Viewer seats are free. Pro also includes a Flat Rate CDN tier of 1 million CDN requests and 1 TB of data transfer a month. Once the credit is used, usage is billed on demand, for example $0.60 per million function invocations, Active CPU from $0.128 an hour, and $0.05 per 1,000 image transformations. New teams get a default on-demand budget of $200 with alerts, and you can configure a hard limit that pauses projects.

What does Coolify charge?

Self-hosted Coolify is free forever with no feature restrictions, according to the Coolify pricing page. You pay your server provider, and that's it. Coolify itself needs at least 2 CPU cores, 2 GB of RAM and 10 GB of disk, and your apps need room on top of that.

If you don't want to run the Coolify dashboard yourself, Coolify Cloud costs $5/month for two connected servers plus $3/month for each additional one. You still bring your own servers; Cloud only manages the Coolify instance.

A worked example

Take a team of three developers who all deploy. On Vercel Pro, that's the $20 platform fee (one seat included) plus two extra seats at $20 each: $60/month before any usage beyond the $20 credit. On Coolify, team members are unlimited, so the same three people add nothing. The bill is your server cost, whether you run one app or ten.

The flip side is that a server costs the same at zero traffic as at peak, and it scales in steps (a bigger box, another box) rather than smoothly. There is also the cost that never shows up on an invoice: your time. Updates, backups and monitoring are now your job. If an hour of your time is worth more than the monthly difference, Vercel is the cheaper option.

Does Next.js work the same on Coolify as on Vercel?

For features, yes. The Next.js guide to deploying to platforms is explicit: "To run Next.js, your platform needs a Node.js server. That's it." A single next start process handles Server Components, ISR, Partial Prerendering, Cache Components, Server Actions, Proxy (the new name for Middleware) and after(). The only extra dependency is sharp for image optimization. The same page notes that Vercel's own adapter uses the same public adapter API as every other platform.

The difference is performance, not correctness. On Vercel, static assets and cached pages are served from a global edge network. On a single Coolify server, everything comes from one origin in one region. For most sites, putting a CDN such as Cloudflare in front for static assets closes most of that gap.

Start from a config that is portable to both:

// next.config.ts
import type { NextConfig } from 'next'

const nextConfig: NextConfig = {
  // Lean, self-contained server bundle for Docker. Harmless on Vercel.
  output: 'standalone',
  // Same value on every instance of one release, so clients and servers agree.
  deploymentId: process.env.DEPLOYMENT_VERSION,
}

export default nextConfig

What changes when you run more than one instance?

This is where self-hosting gets real. By default, the ISR and data cache live on each server's local disk, so a revalidateTag() call on one container doesn't reach the others. Once you scale out, you need three things the platform otherwise handles for you:

A shared cache handler (Redis is the usual choice).
The same NEXT_SERVER_ACTIONS_ENCRYPTION_KEY on every instance, or Server Actions fail with "Failed to find Server Action" errors.
A consistent deploymentId per release, so rolling deploys don't cause version skew.

// next.config.js (multi-instance setup)
module.exports = {
  output: 'standalone',
  cacheHandler: require.resolve('./cache-handler.js'), // Redis-backed handler
  cacheMaxMemorySize: 0, // disable the per-process in-memory cache
  deploymentId: process.env.DEPLOYMENT_VERSION,
}

On a single Coolify server running one container, you need none of this. That's one reason a single, well-sized server is often the sweet spot.

What about image optimization?

On Vercel, next/image transformations are a metered resource. On Coolify, they're free in dollars but cost CPU and memory on your server. If your site is image-heavy, a custom loader that points to an image CDN keeps both bills predictable, and it works the same on either platform:

// lib/cloudinary-loader.ts
// next.config.ts: images: { loader: 'custom', loaderFile: './lib/cloudinary-loader.ts' }
export default function cloudinaryLoader({
  src,
  width,
  quality,
}: {
  src: string
  width: number
  quality?: number
}) {
  const params = ['f_auto', 'c_limit', `w_${width}`, `q_${quality || 'auto'}`]
  return `https://res.cloudinary.com/<your-cloud>/image/upload/${params.join(',')}/${src}`
}

When is Vercel the better choice?

Vercel wins clearly in these situations:

Nobody on the team wants to own a server. For a commercial project that has to be live today with no ops time available, Pro is worth the money.
Your audience is global and latency-sensitive. A built-in edge network is hard to match with one server in one region.
Previews are central to how you review work. Every branch push gets a preview URL with no DNS or webhook setup.
Traffic is spiky or unpredictable. Functions scale automatically, and you pay for usage instead of provisioning for the peak.
It's a personal, non-commercial project. Hobby is free and perfectly good for that.
You already depend on Vercel-only products. If you use Edge Config, Vercel Blob or Vercel's analytics heavily, staying avoids a rewrite.

When is Coolify the better choice?

Coolify wins when the app is more than a frontend:

Several apps and databases on one bill. A single server can run a Next.js frontend, a NestJS API, PostgreSQL, Redis and a worker, all on the same private Docker network. An app like Foliora, a Next.js portfolio and blog platform on Prisma and PostgreSQL, maps naturally onto one Coolify project with the database right next to it.
Long-running work. Vercel Functions stop at their maximum duration (300 seconds by default, up to 800 seconds on Pro). Queues, scheduled jobs and media processing don't fit that model well. A self-hostable media download API like Snagbox is exactly the kind of workload that belongs in a long-running container.
Persistent files. Uploads, generated files and SQLite databases can live on a volume instead of a separate storage service.
Predictable cost. No per-seat pricing and no usage meter. The bill only changes when you change the server.
Control over where data lives. You choose the provider and the region.
Minimal lock-in. Everything is a Docker image, so moving to another server or provider is a redeploy, not a migration project.

What do you give up when you self-host with Coolify?

Be honest with yourself about this list before you switch, because it's the real price of the lower bill:

You're the ops team. OS security patches, Docker updates, Coolify updates and disk space (old images pile up) are yours to watch.
Backups are your responsibility. Coolify can schedule dumps for PostgreSQL, MySQL, MariaDB and MongoDB, but its docs are clear that a rollback "does not restore databases or persistent files". Send backups off the server.
One server is one point of failure. If it goes down, everything on it goes down until you fix it or fail over.
Security starts at install. Coolify's install guide warns that "anyone who reaches the registration page first can become the instance admin and gain root access to your server", so create the admin account immediately.
Previews run untrusted code. A pull request can execute code on your server, so keep preview access to repository members and never give previews production credentials.

Add a health check so Coolify's rolling updates only switch traffic to a container that actually works:

// app/api/health/route.ts
import { connection } from 'next/server'

export async function GET() {
  await connection() // always run at request time, never prerendered
  return Response.json({ status: 'ok', time: new Date().toISOString() })
}

Point the health check at /api/health in the app's settings. If your app can't serve requests without the database, check the connection there too.

How do you migrate a Next.js app from Vercel to Coolify?

Coolify's documentation recommends deploying first and moving DNS last. This is the order I'd follow:

Audit what's Vercel-specific. Search for platform SDKs and config that won't move with you:

   grep -rn "@vercel/" package.json app src lib 2>/dev/null
   cat vercel.json 2>/dev/null   # rewrites, headers and crons live here

Rewrites and headers can move into next.config.ts. Vercel cron jobs become Coolify scheduled tasks or a small worker container.

Make the build portable. Add output: 'standalone' and either let Nixpacks detect the app or use a multi-stage Dockerfile (the self-hosting guide has one you can copy).
Deploy to Coolify without touching DNS. Set environment variables and mark every NEXT_PUBLIC_* value as a build-time variable, because those are inlined during next build.
Move the data with writes stopped. For PostgreSQL, a dump and restore is the simplest safe path:

   pg_dump "$OLD_DATABASE_URL" --format=custom --no-owner --no-acl --file=app.dump
   pg_restore --dbname="$NEW_DATABASE_URL" --no-owner --no-acl app.dump

Test on the generated Coolify domain. Check the health endpoint, logins, forms, image optimization and revalidation.
Switch DNS. Lower the TTL a day ahead, then point the A record at your server and let Coolify issue the certificate.
Keep Vercel running until you're sure. Watch the runtime logs for a few days, then remove the old project.

So, should you use Coolify or Vercel for Next.js?

Use Vercel if your time is the scarcest resource, your traffic is global or spiky, and the app is mostly a frontend. Use Coolify if you run several apps or services, need databases, workers or persistent files next to your app, and want a bill that doesn't move with traffic or team size. Plenty of teams do both: marketing sites on Vercel, APIs and data on a Coolify server.

For full-stack work, a single, well-sized Coolify server with off-site backups and a CDN in front is a strong default. Vercel stays the easy answer when a project is frontend-only or there's simply no one to look after a server. If you're weighing this for a product of your own, take a look at what I build and the projects I've built.

Related: comparing self-hosted panels instead? In Coolify vs Dokploy I installed both on identical VMs: Coolify idled at about 460 to 520 MiB of RAM and 3.0 GB of disk, Dokploy at about 831 MiB and 5.0 GB.

FAQ

Is Coolify a good alternative to Vercel for Next.js?

Yes, for most full-stack apps. Next.js only needs a Node.js server, so features like Server Components, ISR, Server Actions and Proxy work on Coolify through next start or a standalone Docker image. What you give up is Vercel's managed global edge network and zero-ops experience: you look after the server, updates and backups yourself.

Is Coolify really free?

Self-hosted Coolify is free forever with every feature included; you only pay for the servers you run it on. If you'd rather not host the Coolify dashboard yourself, Coolify Cloud costs $5 a month for two connected servers plus $3 a month for each additional server, and you still bring your own servers.

Can I use Vercel's free Hobby plan for a client or business website?

No. Vercel's fair use guidelines restrict Hobby to non-commercial personal use. Commercial use includes sites that take payments, advertise a product or service, show ads, or are built by a paid employee or consultant. Those need the Pro or Enterprise plan.

Does ISR work when you self-host Next.js on Coolify?

Yes. On a single next start instance, ISR and on-demand revalidation work out of the box, with the cache stored on the container's disk. Regenerated pages don't survive a redeploy unless you persist the cache, and if you run several instances you need a shared cache handler, usually Redis, so revalidateTag reaches every instance.

How long can a request run on Vercel compared to Coolify?

Vercel Functions run for 300 seconds by default. Hobby is capped at 300 seconds, Pro can go up to 800 seconds, and a 30-minute extended maximum is in beta for some runtimes. Coolify runs normal containers, so there is no platform time limit, although long jobs are still better handled by a background worker than by a web request.

Can I move from Vercel to Coolify without downtime?

Yes, if you deploy first and move DNS last. Deploy to Coolify, set environment variables, test on the generated domain, lower your DNS TTL ahead of time, then switch the A record and keep the Vercel project running until the new deployment is stable. The only planned pause is a short write freeze while you dump and restore the database.

Originally published on fabwaseem.com.

Next.js SEO for AI Search: How to Get Cited by ChatGPT, Perplexity and AI Overviews

Waseem Anjum — Tue, 27 Oct 2026 22:03:07 +0000

Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com.

Next.js SEO in 2026 means two audiences at once: classic search crawlers like Googlebot, and AI answer engines like ChatGPT search, Perplexity and Google AI Overviews. The good news is that both reward the same foundations. The App Router gives you a typed Metadata API, file-based sitemap.ts and robots.ts, and server-rendered HTML, so you can ship a site that is easy to crawl, easy to understand and easy to quote.

This guide walks through exactly what this site, fabwaseem.com, implements: per-page metadata with canonical URLs, a dynamic sitemap, a robots.txt that names each AI crawler, JSON-LD for posts and projects, an llms.txt file, and a content structure built around question headings and direct answers. Every code sample targets the current Next.js 16 App Router.

What is AI search optimization (GEO and AEO)?

Generative engine optimization (GEO), also called answer engine optimization (AEO), is the practice of making your pages easy for AI systems to find, understand and cite when they generate an answer. In practice it is technical SEO plus content written in self-contained, quotable chunks: clear definitions, direct answers, tables and sourced facts.

The terms are newer than the work. Google states plainly that "there are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary" (Google Search Central). So GEO is not a separate trick. It is solid SEO, plus a writing style that survives being chopped into a snippet.

Term	Optimizes for	Main output
SEO	Ranking in classic search results	Blue links, rich results
AEO	Being the direct answer to a question	Featured snippets, voice and AI answers
GEO	Being cited inside generated answers	Citations in ChatGPT, Perplexity, AI Overviews

How do AI answer engines find and cite pages?

AI answer engines cite pages they can crawl, index and retrieve for a query. ChatGPT search uses OpenAI's OAI-SearchBot, Perplexity uses PerplexityBot, and Google AI Overviews draw on Google's normal index. If a crawler is blocked, the page cannot render content server-side, or the answer is buried, you are unlikely to be cited.

A few facts from the official documentation are worth knowing:

To be shown as a supporting link in AI Overviews or AI Mode, "a page must be indexed and eligible to be shown in Google Search with a snippet" (Google Search Central).
Google AI Overviews and AI Mode "may use a 'query fan-out' technique", issuing multiple related searches across subtopics. That rewards pages that answer many related sub-questions clearly.
OpenAI documents separate crawlers for search (OAI-SearchBot), user-triggered fetches (ChatGPT-User) and model training (GPTBot) (OpenAI bots). Perplexity documents PerplexityBot and Perplexity-User (Perplexity bots).

The pipeline is simple: crawlers discover URLs (sitemap and internal links), check permission (robots.txt), fetch HTML (server rendering), understand it (metadata, headings, JSON-LD), and then retrieval systems pick the passage that best answers the question. Each step below maps to one piece of that chain.

How do you set metadata in the Next.js App Router?

Use the Metadata API: export a static metadata object or an async generateMetadata function from any layout.tsx or page.tsx. Next.js merges it down the route tree and renders the <title>, description, canonical, Open Graph and Twitter tags into server HTML. Set metadataBase once in the root layout so relative URLs resolve to absolute ones (Next.js docs).

Start in the root layout with site-wide defaults and a title template:

// app/layout.tsx
import type { Metadata } from 'next'

export const metadata: Metadata = {
  metadataBase: new URL('https://www.fabwaseem.com'),
  title: {
    default: 'Waseem Anjum: Senior Full Stack Developer',
    template: '%s | Waseem Anjum',
  },
  description:
    'Senior full stack developer building with Next.js, React, Node.js, NestJS and TypeScript.',
  alternates: {
    canonical: '/',
    types: { 'application/rss+xml': '/blog/rss.xml' },
  },
  openGraph: { type: 'website', siteName: 'Waseem Anjum', locale: 'en_US' },
  twitter: { card: 'summary_large_image' },
  robots: { index: true, follow: true },
}

export default function RootLayout({ children }: { children: React.ReactNode }) {
  return (
    <html lang="en">
      <body>{children}</body>
    </html>
  )
}

How do you generate dynamic metadata for blog posts?

Export generateMetadata from the dynamic route and read params, which is a Promise in current Next.js, so you must await it. Return a per-post title, description, canonical URL and Open Graph article data. Next.js deduplicates the fetch or cached data call, so the page and its metadata share one lookup.

// app/blog/[slug]/page.tsx
import type { Metadata } from 'next'
import { notFound } from 'next/navigation'
import { getPost } from '@/lib/posts'

type Props = { params: Promise<{ slug: string }> }

export async function generateMetadata({ params }: Props): Promise<Metadata> {
  const { slug } = await params
  const post = await getPost(slug)
  if (!post) return {}

  return {
    title: post.seoTitle ?? post.title,
    description: post.seoDescription ?? post.excerpt,
    alternates: { canonical: `/blog/${post.slug}` },
    openGraph: {
      type: 'article',
      title: post.title,
      description: post.excerpt,
      publishedTime: post.publishedAt.toISOString(),
      modifiedTime: post.updatedAt.toISOString(),
      authors: ['Waseem Anjum'],
      images: post.coverImage ? [{ url: post.coverImage, width: 1600, height: 900 }] : [],
    },
  }
}

export default async function PostPage({ params }: Props) {
  const { slug } = await params
  const post = await getPost(slug)
  if (!post) notFound()
  return <article>{/* render post */}</article>
}

On this site, every post page ships exactly one canonical link, index, follow robots meta, and RSS and Atom alternates. You can confirm it on the Coolify vs Vercel post by viewing the source.

How do you create a sitemap in Next.js?

Add app/sitemap.ts and default-export a function that returns MetadataRoute.Sitemap, an array of URLs with optional lastModified, changeFrequency, priority, images and alternates. Next.js serves it at /sitemap.xml. Because it is a normal async function, you can query your database for every published post and project (Next.js docs).

// app/sitemap.ts
import type { MetadataRoute } from 'next'
import { getPublishedPosts, getPublishedProjects } from '@/lib/content'

const SITE = 'https://www.fabwaseem.com'

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  const [posts, projects] = await Promise.all([getPublishedPosts(), getPublishedProjects()])

  return [
    { url: SITE, lastModified: new Date(), images: [`${SITE}/opengraph-image`] },
    { url: `${SITE}/projects`, lastModified: new Date() },
    { url: `${SITE}/blog`, lastModified: new Date() },
    ...projects.map((p) => ({
      url: `${SITE}/projects/${p.slug}`,
      lastModified: p.updatedAt,
    })),
    ...posts.map((post) => ({
      url: `${SITE}/blog/${post.slug}`,
      lastModified: post.updatedAt,
      images: post.coverImage ? [post.coverImage] : undefined,
    })),
  ]
}

Keep lastModified honest. Use the real update time from your database rather than new Date() on every URL, so crawlers can trust it. The live fabwaseem.com sitemap lists the home page, author page, every project page, every post, and the category and tag archives, with image entries where they exist. For more than 50,000 URLs, split it with generateSitemaps.

How do you allow AI crawlers in robots.txt with Next.js?

Add app/robots.ts returning MetadataRoute.Robots. Give each crawler its own rule when you want different behavior, list your sitemap, and keep private routes like /api/ and /admin disallowed. To be cited by AI search, allow the search crawlers: OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User and Claude-SearchBot (Next.js docs).

// app/robots.ts
import type { MetadataRoute } from 'next'

const PRIVATE = ['/api/', '/admin', '/admin/']

const AI_SEARCH = [
  'OAI-SearchBot', 'ChatGPT-User',
  'PerplexityBot', 'Perplexity-User',
  'Claude-SearchBot', 'Claude-User',
]
const AI_TRAINING = ['GPTBot', 'ClaudeBot', 'Google-Extended', 'Applebot-Extended', 'CCBot']
const BLOCKED = ['Bytespider']

export default function robots(): MetadataRoute.Robots {
  return {
    rules: [
      { userAgent: '*', allow: '/', disallow: PRIVATE },
      ...[...AI_SEARCH, ...AI_TRAINING].map((userAgent) => ({
        userAgent,
        allow: '/',
        disallow: PRIVATE,
      })),
      ...BLOCKED.map((userAgent) => ({ userAgent, disallow: '/' })),
    ],
    sitemap: 'https://www.fabwaseem.com/sitemap.xml',
  }
}

Which crawlers should you allow? It depends on whether you want visibility, training use, or both:

User agent	Operator	Purpose	fabwaseem.com
Googlebot	Google	Search and AI Overviews	Allowed
Google-Extended	Google	Gemini training and grounding (not Search)	Allowed
OAI-SearchBot	OpenAI	ChatGPT search results	Allowed
ChatGPT-User	OpenAI	Fetches a page when a user asks	Allowed
GPTBot	OpenAI	Model training	Allowed
PerplexityBot	Perplexity	Perplexity search index	Allowed
Claude-SearchBot	Anthropic	Claude search results	Allowed
Bytespider	ByteDance	Crawling	Blocked

Two details trip people up. First, blocking Google-Extended does not remove you from AI Overviews; Google says robots directives for Googlebot are the control for Search, and you limit snippets with nosnippet, max-snippet or noindex (Google Search Central). Second, robots.txt is only half the story: a CDN or firewall bot rule that challenges these crawlers blocks them just as effectively, and Google explicitly lists checking "any CDN or hosting infrastructure" as a best practice.

How do you add JSON-LD structured data in Next.js?

Render a native <script type="application/ld+json"> tag inside your page.tsx or layout.tsx, with the object passed through JSON.stringify and < replaced by \u003c to prevent XSS. The Next.js docs recommend this exact pattern and note that next/script is not needed because JSON-LD is data, not executable code (Next.js docs).

A small reusable component keeps every page consistent:

// components/json-ld.tsx
import type { Thing, WithContext } from 'schema-dts'

export function JsonLd<T extends Thing>({ data }: { data: WithContext<T> }) {
  return (
    <script
      type="application/ld+json"
      dangerouslySetInnerHTML={{
        __html: JSON.stringify(data).replace(/</g, '\\u003c'),
      }}
    />
  )
}

Then build a BlogPosting from the same record that renders the page, so the markup always matches the visible text:

// app/blog/[slug]/page.tsx (inside PostPage)
import type { BlogPosting, WithContext } from 'schema-dts'
import { JsonLd } from '@/components/json-ld'

const url = `https://www.fabwaseem.com/blog/${post.slug}`
const blogPosting: WithContext<BlogPosting> = {
  '@context': 'https://schema.org',
  '@type': 'BlogPosting',
  headline: post.title,
  description: post.excerpt,
  image: post.coverImage ?? undefined,
  datePublished: post.publishedAt.toISOString(),
  dateModified: post.updatedAt.toISOString(),
  mainEntityOfPage: url,
  url,
  author: {
    '@type': 'Person',
    name: 'Waseem Anjum',
    url: 'https://www.fabwaseem.com/author/waseem-anjum',
  },
  keywords: post.tags.map((t) => t.name).join(', '),
}

return (
  <article>
    <JsonLd data={blogPosting} />
    {/* ... */}
  </article>
)

Which schema types matter for a developer blog?

Each post on this site emits BlogPosting (also typed TechArticle), BreadcrumbList and FAQPage, and tutorial posts add HowTo. Project pages describe each project as a SoftwareApplication with breadcrumbs, and the author page uses Person. Google says structured data helps it "understand the content of the page" (Google Search Central).

Be realistic about rich results. Since 2023 Google only shows FAQ rich results for well-known, authoritative government and health sites (FAQPage docs). FAQ markup is still useful because it labels question and answer pairs cleanly for any parser, but do not expect the expandable FAQ snippet on a personal blog. Google's AI features page also says there's "no special schema.org structured data that you need to add" for AI Overviews, so treat JSON-LD as clarity, not a ranking switch.

What is llms.txt and should a Next.js site have one?

llms.txt is a proposed standard: a Markdown file at /llms.txt that gives language models a concise summary of your site and links to the most useful pages. The spec asks for an H1 with the site name, a blockquote summary, then sections of links (llmstxt.org). It is cheap to add, but Google says it does not need it for AI features.

That second point matters for setting expectations. Google states: "You don't need to create new machine readable files, AI text files, or markup to appear in these features" (Google Search Central). So llms.txt will not help with AI Overviews. It is useful for coding assistants and agents that fetch it directly, and the Next.js docs themselves publish one. This site serves both a short llms.txt and an expanded llms-full.txt generated from the same database as the pages.

In the App Router, a route handler builds it at request time:

// app/llms.txt/route.ts
import { getPublishedPosts, getPublishedProjects } from '@/lib/content'

const SITE = 'https://www.fabwaseem.com'

export const revalidate = 3600

export async function GET() {
  const [posts, projects] = await Promise.all([getPublishedPosts(), getPublishedProjects()])

  const body = [
    '# Waseem Anjum',
    '',
    '> Senior full stack developer building with Next.js, React, Node.js, NestJS and TypeScript.',
    '',
    '## Projects',
    ...projects.map((p) => `- [${p.name}](${SITE}/projects/${p.slug}): ${p.summary}`),
    '',
    '## Blog',
    ...posts.map((p) => `- [${p.title}](${SITE}/blog/${p.slug}): ${p.excerpt}`),
    '',
    '## Optional',
    `- [Full context](${SITE}/llms-full.txt)`,
  ].join('\n')

  return new Response(body, {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  })
}

How do you write content that ChatGPT and Perplexity will cite?

Write in self-contained passages that answer one question each. Phrase headings as the questions people ask, put a 40 to 60 word direct answer right under each heading, define terms in one sentence, and back claims with links to primary sources. Retrieval systems pull passages, not whole pages, so every section should make sense on its own.

This is the structure every post on this blog follows:

Lead with a definition. The first two sentences say what the thing is and who it's for. It gives engines a sentence that stands on its own.
Use question headings. Match the wording people type or say, such as "How do you create a sitemap in Next.js?". Query fan-out means engines look for answers to many sub-questions.
Answer first, explain second. One short paragraph that fully answers the heading, then detail, code and caveats.
Prefer tables and numbered steps. Comparisons and procedures extract cleanly into generated answers.
Cite primary sources inline. Link the official docs for every factual claim. It shows the claim is checkable and helps readers.
Never invent numbers. If you can't source a statistic, leave it out. Made-up stats are the fastest way to lose trust.
Show who wrote it. A real author with a profile page and Person markup supports Google's guidance on helpful, people-first content.
Add a TL;DR, key takeaways and FAQs. They give engines short, quotable summaries that match how people ask.

Does page speed or rendering affect AI citations?

Yes, indirectly. Many AI crawlers don't run JavaScript the way a browser does, so content that only appears after client-side rendering may be invisible to them. Server Components render your article to HTML on the server by default, which is one of the strongest reasons to build content sites on the App Router.

Keep the article body in Server Components and reserve 'use client' for interactive pieces such as a share button or theme switcher. Check what crawlers see with curl https://your-site.com/blog/post | grep "<h2". If your headings aren't in the raw HTML, fix that before anything else.

How do you get new pages indexed faster?

Submit your sitemap in Google Search Console and Bing Webmaster Tools, link new posts from existing pages, and ping IndexNow when you publish. IndexNow is a protocol supported by Bing and other engines that lets you notify them of changed URLs with one request (IndexNow docs). It takes one request per publish, so it's worth the small effort.

// lib/indexnow.ts
export async function pingIndexNow(urls: string[]) {
  const key = process.env.INDEXNOW_KEY!
  const res = await fetch('https://api.indexnow.org/indexnow', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json; charset=utf-8' },
    body: JSON.stringify({
      host: 'www.fabwaseem.com',
      key,
      keyLocation: `https://www.fabwaseem.com/${key}.txt`,
      urlList: urls,
    }),
  })
  return res.ok
}

Also publish an RSS or Atom feed (this blog serves /blog/rss.xml and /blog/atom.xml) so aggregators and feed-reading agents pick up new posts quickly. Hosting matters less than people think: this site self-hosts on a VPS with Coolify, covered in how to self-host Next.js with Coolify, and every feature here works the same on a single next start server.

Next.js SEO checklist for AI search

Use this as a pre-launch checklist. Every item is something fabwaseem.com ships today.

Area	What to ship	Next.js feature
Titles and descriptions	Unique per page, title template	`metadata`, `generateMetadata`
Canonical URLs	One per page, absolute	`metadataBase`, `alternates.canonical`
Social previews	Open Graph and Twitter images	`opengraph-image.tsx`, `openGraph`
Discovery	Sitemap with real `lastModified`	`app/sitemap.ts`
Crawl access	AI search bots allowed, private routes blocked	`app/robots.ts`
Understanding	BlogPosting, Breadcrumb, Person, FAQPage	JSON-LD in `page.tsx`
LLM context	`llms.txt` and `llms-full.txt`	Route handlers
Rendering	Article HTML on first response	Server Components
Freshness	RSS/Atom feeds, IndexNow ping	Route handlers
Content	Question headings, direct answers, sources	Your writing

If you're still choosing where to host, the trade-offs are in Coolify vs Vercel for Next.js. SEO works equally well on either.

FAQ

Is Next.js good for SEO?

Yes. The App Router renders pages on the server by default, so crawlers receive full HTML without running JavaScript. It also has a built-in Metadata API for titles, descriptions, canonicals and Open Graph tags, plus file conventions for sitemap.xml, robots.txt and Open Graph images. Good SEO still depends on your content and internal linking.

Do I need llms.txt to appear in ChatGPT or Google AI Overviews?

No. Google says you don't need new machine-readable files, AI text files or special markup to appear in AI Overviews or AI Mode. llms.txt is a proposed standard that some agents and coding assistants read directly. It's cheap to add as a route handler, but treat it as a bonus, not a ranking factor.

Which robots.txt user agents control ChatGPT and Perplexity?

OpenAI uses OAI-SearchBot for ChatGPT search, ChatGPT-User for pages fetched during a conversation and GPTBot for training. Perplexity uses PerplexityBot for its index and Perplexity-User for user-triggered fetches. To be cited in their answers, allow the search and user agents; blocking GPTBot only affects training.

Does blocking Google-Extended remove my site from AI Overviews?

No. Google-Extended controls use in Gemini training and grounding in some other Google systems, not Search. AI Overviews and AI Mode are part of Search, so they follow Googlebot's robots rules. To limit what appears, use nosnippet, data-nosnippet, max-snippet or noindex on the page.

How do I add JSON-LD in the Next.js App Router?

Render a script tag with type application/ld+json inside page.tsx or layout.tsx, and pass the object through JSON.stringify with every < replaced by \u003c to prevent XSS. The Next.js docs recommend a native script tag rather than next/script. You can type the object with the schema-dts package.

Does FAQ schema still help SEO in 2026?

FAQ rich results are now limited to well-known, authoritative government and health websites, so most blogs won't get the expandable FAQ snippet. FAQPage markup is still valid structured data that labels question and answer pairs clearly, and visible FAQs written as real questions give AI engines short, quotable answers.

Originally published on fabwaseem.com.

Sign-In with Ethereum in Next.js 16: SIWE with viem, iron-session and a Replay-Proof Nonce (Tested)

Waseem Anjum — Tue, 27 Oct 2026 19:09:38 +0000

Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com.

Sign-In with Ethereum (SIWE) lets a user log in by signing a plain-text message with their wallet, as defined in EIP-4361. In Next.js you need three pieces: a route that issues a nonce, a route that verifies the signed message, and a session cookie. viem ships every SIWE helper you need, so no extra siwe package is required.

I built the whole flow in a Next.js 16.4.3 app with viem 2.57.11 and iron-session 9.0.1, then attacked it. Two things surprised me. The common "keep the nonce in the session cookie" pattern let me replay a captured login with HTTP 200. And with viem's default verification mode, every login waited about 1.06 seconds when the RPC endpoint was down, against 5.9 ms with one option changed. Both fixes are below.

TL;DR

Create the message with createSiweMessage, sign it in the wallet, and verify it on the server with viem's SIWE helpers. Always pass the expected domain and nonce. Without them, verification only checks the signature.
Store nonces server-side and delete them on first use (Redis SET NX EX then GETDEL). A nonce that lives only in the cookie can be replayed with the old cookie.
Verify plain wallets locally with mode: "eoa". Keep an RPC for smart-contract wallets (ERC-1271 and ERC-6492), but don't make every login wait on it.
Hard-code the domain from config, never from the request's Host header.

Login verify, 50 runs on the box	p50	p95
`verifySiweMessage`, default mode, RPC unreachable	1,064 ms	1,077 ms
`verifySiweMessage`, default mode, public RPC	24.2 ms	33.6 ms
`verifyHash` with `mode: "eoa"`, RPC unreachable	5.9 ms	8.9 ms
`verifyHash` with `mode: "eoa"`, public RPC	5.1 ms	6.5 ms

Times are for the whole POST /api/siwe/verify request against next start, measured from a Node client on the same machine.

How Sign-In with Ethereum works

The browser asks your server for a nonce, a random single-use value.
The browser builds an EIP-4361 message containing your domain, the wallet address, the nonce, a chain ID and timestamps, and asks the wallet to sign it (personal_sign).
The server checks that the message names your domain, contains the nonce it issued, hasn't expired, and was signed by the address in the message. Then it starts a session.

This is what a signed message looks like, from my test run:

app.example.com wants you to sign in with your Ethereum account:
0x13F7713E3A0499BF531879888D8E6dd5f93FDea7

Sign in to Example.

URI: https://app.example.com/login
Version: 1
Chain ID: 1
Nonce: 6a5d8a5d6a69ad7863e7c51413d00d684959afce06879effaa9e1c6e9cc86e84ff0f0ed03cf62b106dab75ac0d37c66f
Issued At: 2026-10-26T05:10:48.993Z
Expiration Time: 2026-10-26T05:15:48.993Z

The domain line is what protects users from phishing. Many wallets warn the user when it doesn't match the site asking for the signature, and your server must reject a message signed for any other domain.

Setup

npm i viem iron-session

// lib/session.ts
import { cookies } from "next/headers";
import { getIronSession, type SessionOptions } from "iron-session";

export type SiweSession = { nonce?: string; address?: `0x${string}`; chainId?: number };

export const sessionOptions: SessionOptions = {
  password: process.env.SESSION_SECRET!, // at least 32 characters
  cookieName: "siwe_session",
  cookieOptions: { httpOnly: true, sameSite: "lax", secure: process.env.NODE_ENV === "production" },
};

export async function getSession() {
  return getIronSession<SiweSession>(await cookies(), sessionOptions);
}

// lib/siwe.ts
import { createPublicClient, http } from "viem";
import { mainnet } from "viem/chains";

// Only needed for smart-contract wallets (ERC-1271 / ERC-6492).
export const publicClient = createPublicClient({ chain: mainnet, transport: http(process.env.RPC_URL) });

// From config, never from the request's Host header.
export const SIWE_DOMAIN = process.env.SIWE_DOMAIN!; // e.g. "app.example.com"

iron-session 9 is ESM-only and needs Node 22.13 or later, according to its README. I ran everything on Node 22.20. The sealed session cookie in my test was 342 bytes.

Step 1: issue a nonce (and remember it on the server)

// lib/nonce-store.ts
// In production use Redis: SET siwe:<nonce> 1 EX 300 NX on issue, GETDEL on use.
// A Map works for one process and for tests.
const issued = new Map<string, number>();

export async function issueNonce(nonce: string, ttlMs = 5 * 60_000) {
  issued.set(nonce, Date.now() + ttlMs);
}

export async function consumeNonce(nonce: string) {
  const exp = issued.get(nonce);
  issued.delete(nonce); // single use, even if verification fails
  return exp !== undefined && exp > Date.now();
}

// app/api/siwe/nonce/route.ts
import { generateSiweNonce } from "viem/siwe";
import { getSession } from "@/lib/session";
import { issueNonce } from "@/lib/nonce-store";

export async function GET() {
  const session = await getSession();
  session.nonce = generateSiweNonce();
  await session.save();
  await issueNonce(session.nonce);
  return Response.json({ nonce: session.nonce }, { headers: { "Cache-Control": "no-store" } });
}

generateSiweNonce() returned a 96-character hex string in my test. EIP-4361 only requires at least 8 alphanumeric characters, so that is plenty.

Step 2: sign in the browser

With wagmi 3 (3.7.9 when I wrote this) you already have a connected address and chain. useConnection replaces useAccount, and mutation hooks now return mutateAsync. Build the message with viem and ask the wallet to sign it:

// components/sign-in-button.tsx
"use client";
import { createSiweMessage } from "viem/siwe";
import { useConnection, useSignMessage } from "wagmi";

export function SignInButton() {
  const { address, chainId } = useConnection(); // useAccount() in wagmi 2
  const { mutateAsync: signMessage } = useSignMessage();

  async function signIn() {
    if (!address || !chainId) return;
    const { nonce } = await fetch("/api/siwe/nonce").then((r) => r.json());
    const message = createSiweMessage({
      domain: window.location.host,
      address,
      statement: "Sign in to Example.",
      uri: window.location.origin,
      version: "1",
      chainId,
      nonce,
      issuedAt: new Date(),
      expirationTime: new Date(Date.now() + 5 * 60_000),
    });
    const signature = await signMessage({ message });
    const res = await fetch("/api/siwe/verify", {
      method: "POST",
      headers: { "content-type": "application/json" },
      body: JSON.stringify({ message, signature }),
    });
    if (!res.ok) throw new Error("Sign-in failed");
  }

  return <button onClick={signIn} disabled={!address}>Sign in with Ethereum</button>;
}

The client may put any domain it likes in the message. That is fine, because the server decides which domain it accepts.

Step 3: verify on the server

The simplest version uses verifySiweMessage:

// app/api/siwe/verify/route.ts (simple version)
import { parseSiweMessage, verifySiweMessage } from "viem/siwe";
import { getSession } from "@/lib/session";
import { consumeNonce } from "@/lib/nonce-store";
import { publicClient, SIWE_DOMAIN } from "@/lib/siwe";

export async function POST(req: Request) {
  const { message, signature } = await req.json();
  const session = await getSession();
  const nonce = session.nonce;
  if (!nonce || !(await consumeNonce(nonce))) {
    return Response.json({ ok: false, error: "nonce used or expired" }, { status: 401 });
  }
  session.nonce = undefined;

  const ok = await verifySiweMessage(publicClient, { message, signature, domain: SIWE_DOMAIN, nonce });
  if (!ok) { await session.save(); return Response.json({ ok: false }, { status: 401 }); }

  const { address, chainId } = parseSiweMessage(message);
  session.address = address;
  session.chainId = chainId;
  await session.save();
  return Response.json({ ok: true, address });
}

I tested what verifySiweMessage accepts and rejects with a fresh private key:

Case	Result
Valid message and signature	`true`
Different nonce passed to verify	`false`
Different domain passed to verify (phishing site)	`false`
Checked 6 minutes later, message expired after 5	`false`
Message signed by another key	`false`
One character of the statement changed	`false`
Valid signature, no `domain` or `nonce` passed	`true`

The last row matters. The domain and nonce parameters are optional, and when you leave them out the function only proves that the address signed some SIWE message. That message could have been signed for another site. Always pass both.

The replay bug: a nonce in the cookie isn't single-use

Many tutorials store the nonce only in the session cookie and clear it after login. I tested that version (the same route without consumeNonce):

GET /api/siwe/nonce returns a cookie containing the nonce.
POST /api/siwe/verify with that cookie, message and signature returns 200, and the response sets a new cookie without the nonce.
Sending the same request again with the original cookie returned 200 again: a second valid session from one signature.

Clearing the nonce only changes the new cookie. The old cookie is still a valid, sealed copy of the session, and the server has no memory that it was used. Anyone who captured the request (a proxy log, a compromised browser extension, a shared machine) could mint sessions until the message expired. With the server-side store, the same replay returned 401 nonce used or expired.

In production, back issueNonce and consumeNonce with Redis so every instance sees the same state:

// lib/nonce-store.ts (Redis)
import { Redis } from "ioredis";
const redis = new Redis(process.env.REDIS_URL!);

export const issueNonce = (nonce: string) => redis.set(`siwe:${nonce}`, "1", "EX", 300, "NX");
export const consumeNonce = async (nonce: string) => (await redis.getdel(`siwe:${nonce}`)) === "1";

GETDEL (Redis 6.2+) reads and deletes in one atomic step, so two parallel requests can't both use the same nonce. A Postgres table with DELETE ... RETURNING works the same way if you don't run Redis.

The 1-second login: verify plain wallets locally

verifySiweMessage supports smart-contract wallets through ERC-6492, so by default it first verifies on-chain with an eth_call. Only when that call fails does it fall back to recovering the address locally. In my test:

With a public RPC, a login took 24.2 ms at p50, mostly the RPC round trip.
With the RPC unreachable, each login took 1,064 ms. My offline test client counted 4 RPC attempts per verification, matching viem's transport default of 3 retries, before the local fallback succeeded.

viem's lower-level verifyHash has a mode: "eoa" option that tries local ECDSA recovery first and only goes on-chain if that fails. verifySiweMessage's TypeScript type doesn't accept mode (true in 2.57.11, which I tested, and still true in 2.57.12, the latest release on October 27, 2026), so I rebuilt the same checks from viem's exported helpers:

// lib/verify.ts
import { hashMessage, type Hex } from "viem";
import { verifyHash } from "viem/actions";
import { parseSiweMessage, validateSiweMessage } from "viem/siwe";
import { publicClient, SIWE_DOMAIN } from "@/lib/siwe";

export async function verifySiwe(message: string, signature: Hex, nonce: string) {
  const parsed = parseSiweMessage(message);
  if (!parsed.address) return null;
  // checks domain, nonce and the time window (expirationTime / notBefore)
  if (!validateSiweMessage({ message: parsed, domain: SIWE_DOMAIN, nonce })) return null;
  const ok = await verifyHash(publicClient, {
    address: parsed.address,
    hash: hashMessage(message),
    signature,
    mode: "eoa", // local ecrecover first, RPC only for contract wallets
  });
  return ok ? parsed : null;
}

Swap verifySiweMessage for verifySiwe in the route and logins took 5.1 ms (public RPC) and 5.9 ms (RPC down) at p50. Smart-contract wallets still verify on-chain, so keep a reliable RPC for them.

Reading the session

// app/api/me/route.ts
import { getSession } from "@/lib/session";

export async function GET() {
  const { address } = await getSession();
  return address ? Response.json({ address }) : Response.json({ error: "signed out" }, { status: 401 });
}

For pages, call getSession() in a Server Component or check the cookie in proxy.ts (the Next.js 16 name for middleware). If you also need email or OAuth logins, Better Auth with Next.js 16 covers that side, and SIWE can sit beside it as one more sign-in method.

Security checklist

Domain from config. Compare against a fixed SIWE_DOMAIN. Reading it from the Host header lets a proxy or a forged header decide what you accept.
Single-use nonces on the server, deleted on first use, with a short TTL (5 minutes is plenty).
Short expiry. Set expirationTime on the message. viem's validation rejects expired messages, as my 6-minute test showed.
Rate-limit the nonce and verify routes per IP, like any login endpoint.
Treat the address as an identity, not a permission. Look up roles in your database after login.
Sign out by destroying the session (session.destroy()). Remember that a stateless cookie stays valid until it expires, so keep the session TTL modest or track session IDs server-side if you need instant revocation.

For the API side of the same idea (input validation, SSRF and auth checks), see SSRF protection in Node.js and the secure NestJS + Prisma API starter.

Test setup

Next.js 16.4.3 (next build + next start), React 19.3.1, viem 2.57.11, iron-session 9.0.1, Node 22.20.0, on an 8 vCPU Linux box.
Wallets: fresh keys from generatePrivateKey(), signing with account.signMessage (the same EIP-191 personal_sign a browser wallet uses).
"RPC unreachable" pointed RPC_URL at a closed local port. "Public RPC" used a free public Ethereum mainnet endpoint.
Each latency row is 50 timed logins after 2 warm-ups. I did not test real smart-contract wallets.

Originally published on fabwaseem.com.

Production Next.js Dockerfile: Multi-Stage, Standalone and Non-Root

Waseem Anjum — Mon, 26 Oct 2026 22:03:36 +0000

Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com.

A production Next.js Dockerfile is a multi-stage build that installs dependencies, runs next build with output: 'standalone', and copies only the traced server, static assets and public/ folder into a small runtime image that runs node server.js as a non-root user. The result is an image that starts fast, carries no dev dependencies or source code, and runs the same way on any Docker host.

This guide gives you a complete, copy-paste Next.js Dockerfile for Next.js 16 and Node.js 24 LTS, explains every line, and shows how to handle the parts that usually go wrong: sharp for image optimization, build-time vs runtime environment variables, health checks, and file permissions. I built and ran this exact Dockerfile against a fresh create-next-app project before writing this, and the image sizes below are measured, not estimated.

If you deploy with Coolify, this Dockerfile drops straight into the Dockerfile build pack. For the platform side (server setup, domains, databases), see How to Self-Host Next.js with Coolify; this post focuses on the image itself.

What is the best Dockerfile for Next.js in production?

The best production Dockerfile for Next.js uses three stages (deps, builder, runner), enables output: 'standalone' in next.config, starts from an official Node.js LTS image such as node:24-slim, copies .next/standalone, .next/static and public/ into the final stage, switches to the built-in node user, and starts with node server.js instead of next start.

That is also the shape of the official vercel/next.js with-docker example, which currently pins Node 24 slim. The version below keeps that structure and adds a health check, build cache mounts and a clear split between public and secret environment variables.

Quick definitions

Term	What it means
Multi-stage build	One Dockerfile with several `FROM` stages; only the last stage ships. Build tools stay behind.
`output: 'standalone'`	Next.js traces which files each route needs and copies them, plus a minimal `server.js`, into `.next/standalone`.
Output file tracing	The `@vercel/nft` analysis Next.js runs during `next build` to find every file a page can load.
Runner stage	The final, small image that actually runs in production.
`HEALTHCHECK`	A Dockerfile instruction that tells Docker how to test whether the container is still working.

TL;DR: the production Next.js Dockerfile

Enable standalone output first:

// next.config.ts
import type { NextConfig } from "next";

const nextConfig: NextConfig = {
  output: "standalone",
};

export default nextConfig;

Then add this Dockerfile at the project root:

# syntax=docker/dockerfile:1
ARG NODE_VERSION=24-slim

# 1) Install dependencies (cached until the lockfile changes)
FROM node:${NODE_VERSION} AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm npm ci --no-audit --no-fund

# 2) Build the app with output: "standalone"
FROM node:${NODE_VERSION} AS builder
WORKDIR /app
ENV NODE_ENV=production
ENV NEXT_TELEMETRY_DISABLED=1
COPY --from=deps /app/node_modules ./node_modules
COPY . .
# Public, build-time values only. They are inlined into the JS bundle.
ARG NEXT_PUBLIC_SITE_URL
ENV NEXT_PUBLIC_SITE_URL=${NEXT_PUBLIC_SITE_URL}
RUN npm run build

# 3) Minimal runtime image
FROM node:${NODE_VERSION} AS runner
WORKDIR /app
ENV NODE_ENV=production \
    NEXT_TELEMETRY_DISABLED=1 \
    PORT=3000 \
    HOSTNAME=0.0.0.0

COPY --from=builder --chown=node:node /app/public ./public
RUN mkdir .next && chown node:node .next
COPY --from=builder --chown=node:node /app/.next/standalone ./
COPY --from=builder --chown=node:node /app/.next/static ./.next/static

USER node
EXPOSE 3000

HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
  CMD ["node", "-e", "fetch('http://127.0.0.1:'+(process.env.PORT||3000)+'/api/health').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))"]

CMD ["node", "server.js"]

Add a .dockerignore so local junk and secrets never reach the build context:

.git
.next
node_modules
npm-debug.log*
Dockerfile*
.dockerignore
.env*
!.env.example
coverage
README.md

Build and run:

docker build -t my-next-app --build-arg NEXT_PUBLIC_SITE_URL=https://example.com .
docker run -p 3000:3000 -e DATABASE_URL="postgres://..." my-next-app

How do multi-stage Docker builds make Next.js images smaller?

A multi-stage build lets the build stages carry the full node_modules, TypeScript, ESLint and the compiler, while the final stage copies only the build output. Docker ships just the last stage, so none of the build tooling, source code or package cache ends up in the production image.

The Docker multi-stage build docs describe this pattern: each FROM starts a fresh stage, and COPY --from=<stage> pulls selected files forward. For Next.js the three stages do one job each:

deps copies only package.json and the lockfile, then runs npm ci. Because source files are not copied yet, this layer stays cached until your dependencies change.
builder brings in node_modules and your source and runs next build.
runner starts from a clean Node image and copies the standalone server, static files and public/.

How much smaller is the image? (measured)

I built a fresh Next.js 16.4.2 app from create-next-app (App Router, TypeScript, Tailwind) three ways on the same machine and compared docker images output:

Build	Base image	Image size
Single stage: `COPY . .`, `npm ci && npm run build`, `npm start`	`node:24`	1.97 GB
Multi-stage + standalone (this guide)	`node:24-slim`	287 MB
Multi-stage + standalone	`node:24-alpine`	229 MB

The application files inside the standalone runner (/app) took 59 MB; the rest is the Node.js base image. Your numbers will differ with your dependencies, but the order of magnitude is typical, because the standalone trace keeps only the packages the server actually imports.

What does output: 'standalone' do in Next.js?

output: 'standalone' tells next build to create a .next/standalone folder that contains a minimal server.js and only the files from node_modules that your routes need. You can run that folder with node server.js and no npm install, which is why it is the foundation of every small Next.js Docker image.

From the Next.js output docs, three details matter for Docker:

It does not copy public/ or .next/static. Those are expected to go to a CDN, so the Dockerfile copies them next to server.js manually. Forget this and your CSS, JS chunks and images return 404.
PORT and HOSTNAME control where the server listens. Inside a container you want HOSTNAME=0.0.0.0, otherwise the server may bind only to the container's own interface name and the published port will not answer.
Monorepos need outputFileTracingRoot. If your app imports packages from outside its folder, set it to the repository root so tracing can include them. If a file is still missing, outputFileTracingIncludes adds it explicitly.

Should you use next start or node server.js in Docker?

Use node server.js with standalone output. It runs the same Next.js server without needing the full node_modules tree or the next CLI, and it runs directly as PID 1 rather than through npm. That means docker stop sends SIGTERM straight to Node instead of to an npm wrapper process.

Which Node.js base image should you use for Next.js?

Use the current Node.js LTS line; as of October 2026 that is Node.js 24 ("Krypton"), per the Node.js release schedule. Pick node:24-slim (Debian, glibc) as the safe default, or node:24-alpine (musl) for a smaller image if your native dependencies support musl.

Option	Pros	Watch out for
`node:24-slim`	glibc, widest native-module compatibility; the official example's default	Larger than Alpine
`node:24-alpine`	Smallest official Node image	musl libc; some native modules need musl builds or extra packages
`node:24` (full)	Has build tools for compiling native modules	Very large; use only in a build stage if needed

Pin the version with an ARG (as in the Dockerfile above) so upgrading Node is a one-line change, and rebuild regularly to pick up base image security patches.

How do you run a Next.js container as a non-root user?

Switch to the non-root node user that ships in every official Node.js image with USER node, and give it ownership of the files it needs with COPY --chown=node:node. Create .next and chown it first, because Next.js writes its prerender and image cache there at runtime.

The Node.js Docker best practices recommend exactly this: the official images already include a node user (UID 1000), so there is no need to create your own. I confirmed it in the built container: id returns uid=1000(node) gid=1000(node).

Two permission mistakes cause most "EACCES" crashes:

Copying files without --chown, so they belong to root and the cache directory is read-only for node.
Mounting a volume over /app/.next/cache that is owned by root on the host. Fix the host directory's owner or skip the mount.

How do you make sharp work in a Next.js Docker image?

With Next.js 15 and later you usually do nothing: next pulls in sharp as a dependency, and standalone tracing copies it into the image along with its prebuilt libvips binaries for Linux. Run npm ci inside the container (not on your Mac or Windows host) so the correct Linux binaries are installed.

In my test build, the runner's node_modules contained sharp plus @img/sharp-linux-x64 and @img/sharp-libvips-linux-x64 without any extra configuration, and calling require('sharp') inside the running container worked (sharp 0.35.5).

If you do see the sharp missing in production warning, check these in order:

Do not copy a host node_modules into the image. Keep node_modules in .dockerignore.
Match the platform. Building on Apple Silicon for an x64 server? Use docker build --platform linux/amd64 so the right binaries are chosen.
Force the files into the trace. Add outputFileTracingIncludes: { '/*': ['node_modules/sharp/**/*'] } to next.config, as the Next.js docs show for native assets.

Remember the default image optimizer runs on your server's CPU. If you serve lots of images, the self-hosting guide explains how to swap in a custom loader that points to an image CDN.

How do you handle environment variables in a Next.js Docker build?

Treat them as two different things. Variables prefixed with NEXT_PUBLIC_ are inlined into the browser JavaScript at next build, so they must be passed as build arguments. Everything else (database URLs, API keys, auth secrets) is read by the server at runtime, so pass it with docker run -e or your platform's environment settings and never bake it into the image.

The Next.js environment variables guide spells this out: once built, a NEXT_PUBLIC_ value is frozen into the bundle. Changing it later requires a rebuild.

Variable type	Example	Where to set it	Rebuild needed to change?
Public (browser)	`NEXT_PUBLIC_SITE_URL`	`ARG` + `ENV` in builder stage, `--build-arg`	Yes
Server secret	`DATABASE_URL`, `AUTH_SECRET`	`docker run -e`, Compose `environment`, Coolify env vars	No
Server config read at request time	`REVALIDATE_TOKEN`	Runtime env	No

What if the build itself needs a secret?

Some builds need a secret, for example to fetch data during prerendering or to install a private package. Do not use ARG for it, because build arguments can show up in image history. Use a BuildKit secret mount, which exposes the value only for one RUN step:

RUN --mount=type=secret,id=npm_token,env=NPM_TOKEN npm ci

docker build --secret id=npm_token,env=NPM_TOKEN .

A simpler rule that avoids most of this: keep pages that need secrets dynamic, so they read runtime env on the request instead of at build time.

How do you add a health check to a Next.js container?

Add a tiny route handler that returns 200, then point a Dockerfile HEALTHCHECK at it using Node's built-in fetch, because slim and Alpine images do not include curl by default. Docker marks the container healthy or unhealthy, and orchestrators like Coolify, Compose and Swarm use that status to decide when traffic can switch to a new deploy.

The health route (App Router):

// src/app/api/health/route.ts
import { connection } from "next/server";

export async function GET() {
  await connection(); // opt out of prerendering: answer on every request
  return Response.json({ status: "ok", uptime: process.uptime() });
}

await connection() makes the handler run on every request, so a stuck server cannot hide behind a prerendered response. The Dockerfile HEALTHCHECK reference explains the options used above:

--start-period=20s gives the server time to boot before failures count.
--interval=30s and --timeout=5s set how often and how long each probe runs.
--retries=3 means three failures in a row flip the status to unhealthy.

Running the test image, docker inspect reported healthy and /api/health returned {"status":"ok",...}. Keep this endpoint cheap. If you also want a readiness check that pings the database, make it a separate route so a slow database does not get the whole container restarted.

How do you speed up Next.js Docker builds?

Order layers from least to most frequently changed, and use BuildKit cache mounts. Copy the lockfile and run npm ci before copying source, so dependency installs are reused on every code-only change. The --mount=type=cache,target=/root/.npm flag keeps npm's download cache between builds without putting it into the image.

More options, from Docker's build cache optimization guide and the official example:

Keep .dockerignore tight so a changed README or .git folder does not invalidate COPY . ..
Cache .next/cache with RUN --mount=type=cache,target=/app/.next/cache npm run build to reuse compiler output. The official example notes the trade-off: that cache is then not in the final image, so fetch responses cached during the build are not available at startup.
Use the lockfile your project already has. The official example detects npm, Yarn or pnpm; if you use pnpm, swap the install line for corepack enable pnpm && pnpm install --frozen-lockfile and copy pnpm-lock.yaml.

How do you deploy this Next.js Dockerfile to Coolify or any host?

Because the image is self-contained, any Docker host can run it: a plain VPS with docker run, Docker Compose, Kubernetes, Fly.io, Railway or Coolify. Expose port 3000, set runtime environment variables on the platform, pass NEXT_PUBLIC_ values as build arguments, and put a reverse proxy with HTTPS in front.

In Coolify specifically:

Choose the Dockerfile build pack and keep the Dockerfile at the repository root.
Set Ports Exposes to 3000.
Add runtime variables (like DATABASE_URL) as normal environment variables, and make sure NEXT_PUBLIC_ ones are available at build time so they reach the ARG in the builder stage.
Coolify can use the container's health check during deploys, so the HEALTHCHECK above helps it avoid routing traffic to a container that never became healthy.

I walk through the full Coolify setup, including domains and PostgreSQL, in the self-hosting guide. If you are still deciding between platforms, Coolify vs Vercel for Next.js compares cost and trade-offs. This same pattern runs this site and projects like Foliora.

Running more than one replica?

Each container has its own .next/cache, so ISR and image caches are not shared between replicas by default. The Next.js self-hosting guide covers configuring a shared cache handler (for example, Redis-backed) when you scale horizontally. A single container per app does not need this.

Common Next.js Docker errors and fixes

Symptom	Likely cause	Fix
CSS/JS 404s, unstyled page	`.next/static` not copied	`COPY --from=builder /app/.next/static ./.next/static`
Images in `public/` return 404	`public/` not copied	`COPY --from=builder /app/public ./public`
Container runs but port does not answer	Server bound to the wrong interface	`ENV HOSTNAME=0.0.0.0`
`EACCES` writing `.next/cache`	Files owned by root	`mkdir .next && chown node:node .next`, `COPY --chown`
`NEXT_PUBLIC_` value is empty or stale	Set only at runtime	Pass as `--build-arg` and rebuild
`Cannot find module` at runtime	File not traced (monorepo or dynamic `require`)	`outputFileTracingRoot` / `outputFileTracingIncludes`
Sharp warning in logs	Host-built `node_modules` or wrong platform	Ignore host `node_modules`, build with `--platform`

Related: the builder stage runs your type check and any Prisma commands, so two toolchain changes affect it: TypeScript 7 makes next build type checking faster, and Prisma 8 changed what an unpinned npx prisma installs.

Related: thinking about Bun in your Docker image? In Bun vs Node.js in 2026 a Next.js 16.4 app installed in 1.15 s with bun vs 15.4 s with npm, and next build worked on Bun 1.4.3.

Key takeaways

Enable output: 'standalone' and use a three-stage Dockerfile; in my measured test that cut a fresh Next.js 16 image from 1.97 GB to 287 MB on Debian slim.
Standalone output does not include public/ or .next/static; copy both into the runner yourself.
Run as the built-in node user and give it ownership of .next, so the runtime cache can be written.
NEXT_PUBLIC_ variables are build-time and baked into the bundle; secrets belong in runtime env or BuildKit secret mounts, never ARG.
Add a cheap /api/health route and a Node fetch-based HEALTHCHECK so your platform can roll out new containers safely.

FAQ

Do I need output: 'standalone' to run Next.js in Docker?

No, but it is strongly recommended. Without it you must ship the full node_modules and run next start. With it, next build produces a minimal server.js plus only the traced dependencies, so the runtime image is far smaller and needs no npm install.

Why are my CSS and JS files returning 404 in the Next.js Docker container?

Standalone output does not include the public folder or .next/static. Copy both into the runner stage: .next/static to ./.next/static and public to ./public, next to server.js. The standalone server then serves them automatically.

Should I use node:24-alpine or node:24-slim for Next.js?

Use node:24-slim as the safe default because it uses glibc and works with almost every native module, including sharp's prebuilt binaries. Alpine is smaller but uses musl, which some native dependencies do not support. In my test the Alpine image was 229 MB vs 287 MB for slim.

How do I pass environment variables to a Next.js Docker image?

Pass NEXT_PUBLIC_ variables as build arguments, because Next.js inlines them into the browser bundle during next build. Pass server-only values like DATABASE_URL at runtime with docker run -e or your platform's settings. Never put secrets in ARG; use a BuildKit secret mount if the build needs one.

Do I need to install sharp manually in a Next.js Docker image?

Usually not on Next.js 15 and later. Next.js depends on sharp and standalone tracing copies it, with its prebuilt Linux libvips binaries, into the image. Install dependencies inside the container and keep host node_modules out of the build context so the correct platform binaries are used.

How do I add a Docker health check without curl?

Slim and Alpine Node images do not include curl, so use Node's built-in fetch in the HEALTHCHECK command to call a lightweight /api/health route and exit 0 on a 2xx response or 1 otherwise. Add a start period so the server has time to boot.

Originally published on fabwaseem.com.

AGENTS.md vs CLAUDE.md vs Cursor Rules: A Next.js Guide with a Free Rules Pack

Waseem Anjum — Mon, 26 Oct 2026 22:03:30 +0000

Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com.

AGENTS.md vs CLAUDE.md comes down to scope: AGENTS.md is an open, tool-neutral Markdown file that many coding agents read, while CLAUDE.md is Claude Code's own project memory file. Cursor rules (.cursor/rules/*.mdc) and GitHub Copilot instructions (.github/copilot-instructions.md) are tool-specific formats that add scoping by file pattern. Most Next.js teams need more than one of them.

Free and open source: I packaged production rules for Next.js, TypeScript, React, Node.js, NestJS, Prisma, Drizzle, Tailwind CSS and shadcn/ui into one repo that renders every format from a single source. Get it here: github.com/fabwaseem/agent-rules (MIT).

This guide explains what each file does, which agent reads which file, how they interact (Claude Code has specific rules for when it reads AGENTS.md), and gives you real rule snippets you can copy. Everything below was checked against the official docs linked in each section.

What is AGENTS.md?

AGENTS.md is a plain Markdown file at the root of a repository that tells AI coding agents how to work in that codebase: build and test commands, conventions, and things to avoid. It has no required fields or frontmatter. Agents read the nearest AGENTS.md in the directory tree, so a nested file in a subproject takes precedence over the root one.

The format is documented at agents.md, which describes it as "a simple, open format for guiding coding agents" used by more than 60,000 open-source projects. Think of it as a README for agents: the README is for humans, AGENTS.md is for the tool that edits your code.

A minimal example:

# AGENTS.md

## Workspace
- Package manager: `pnpm`. Do not add a second lockfile.
- Run `pnpm test` and `pnpm typecheck` before you finish.
- Do not hand-edit generated output (`dist/`, `.next/`, generated database clients).

What is CLAUDE.md?

CLAUDE.md is the project instructions file for Claude Code. Claude reads it at the start of every session from ./CLAUDE.md or ./.claude/CLAUDE.md, plus your personal ~/.claude/CLAUDE.md. It is plain Markdown, can import other files with @path/to/file, and Anthropic recommends keeping each file under 200 lines because longer files cost context and reduce adherence.

Those details come straight from the Claude Code memory docs. Two things matter most in practice:

Imports. A line like @AGENTS.md in CLAUDE.md pulls that file into context at launch, up to four hops deep. This is how you share one source of truth between Claude and every other agent.
Instructions are context, not enforcement. The docs say so directly: to block an action no matter what, use a hook, not a sentence in CLAUDE.md.

Does Claude Code read AGENTS.md?

Yes, but only in one situation by default. Claude Code reads AGENTS.md as project instructions when there is no CLAUDE.md, .claude/CLAUDE.md or CLAUDE.local.md in your working directory or above it. If a CLAUDE.md exists, Claude reads only the CLAUDE.md files, unless that file imports AGENTS.md or you change the Project instructions setting.

Here is the behavior from the official docs, summarized:

Your repository has	Claude Code reads
`AGENTS.md` only	`AGENTS.md`
`AGENTS.md` and `CLAUDE.md`	`CLAUDE.md` only
`CLAUDE.md` that contains `@AGENTS.md`	`CLAUDE.md` with `AGENTS.md` included

Reading AGENTS.md directly also needs a recent Claude Code version (v2.1.277 or later per the docs). The safest setup for a mixed team is a short CLAUDE.md that imports AGENTS.md and adds only Claude-specific notes. That is exactly what the agent-rules installer writes.

AGENTS.md vs CLAUDE.md vs Cursor rules vs Copilot instructions: what is the difference?

They differ in who reads them and how they are scoped. AGENTS.md is shared by many agents and scoped by directory. CLAUDE.md is Claude Code only. Cursor .mdc rules can attach to specific file globs or load when the agent judges them relevant. Copilot path instructions use an applyTo glob. Use AGENTS.md for shared rules and the others for targeted ones.

	AGENTS.md	CLAUDE.md	Cursor `.mdc`	Copilot instructions	Claude Code skill	Windsurf / Devin
Path	`AGENTS.md` (nested allowed)	`CLAUDE.md`, `.claude/CLAUDE.md`	`.cursor/rules/*.mdc`	`.github/copilot-instructions.md`, `.github/instructions/*.instructions.md`	`.claude/skills/<name>/SKILL.md`	`.devin/rules/.md`, `.windsurf/rules/.md`
Frontmatter	None	None	`description`, `globs`, `alwaysApply`	Path files: `applyTo`	`name`, `description`	`trigger` plus `globs` or `description`
Scope	Nearest file in the tree	Claude Code sessions	Always, by glob, agent-selected or `@` mention	Repo-wide or matching `applyTo`	Loaded on demand	Always, glob, model decision or manual
Best for	Rules every agent shares	Claude-specific additions	Rules tied to certain files	Copilot Chat, review and coding agent	Multi-step procedures	Cascade / Devin workspace rules

Sources: agents.md, Cursor Rules docs, Claude Code memory, Claude Code skills, GitHub Copilot repository instructions and Windsurf / Devin rules.

How do Cursor rules work in a Next.js project?

Cursor reads .mdc files from .cursor/rules/, and the frontmatter decides when each rule loads. alwaysApply: true includes it every session. alwaysApply: false with globs attaches it when a matching file is in context. A description with no globs lets the agent pull it in when relevant. With neither, it loads only when you @ mention it.

One gotcha: Cursor ignores a plain .md file inside .cursor/rules/. The extension must be .mdc. Here is the real App Router rule from the repo, trimmed:

---
description: "Next.js App Router structure, Server Components, routing files, and proxy.ts."
globs: "app/**/*.ts,app/**/*.tsx,src/app/**/*.ts,src/app/**/*.tsx,next.config.*,proxy.ts,middleware.ts"
alwaysApply: false
---

# Next.js App Router

- New UI goes in the App Router. Do not add a `pages/` route.
- Components are Server Components by default. Add `"use client"` only on the leaf
  that needs state, effects, or browser APIs.
- Read `params` and `searchParams` as promises and `await` them.
- Set document metadata with the Metadata API. Do not add `next/head`.
- Import `server-only` in modules that read secrets or talk to the database.
- On Next.js 16, request interception lives in `proxy.ts` and exports `proxy`.
- Do not flip `cacheComponents` on as a drive-by.

Because it has globs, this rule costs nothing when you are editing a Prisma schema or a README. That is the main advantage of Cursor rules over one giant AGENTS.md.

What should go in AGENTS.md for a Next.js app?

Put the things every agent must know in every session: the package manager and scripts, how to run tests and typecheck, which folders are generated, and short, specific conventions. Keep stack details that only matter for some files in scoped rules. Short and concrete beats long and vague, because agents follow specific instructions more reliably.

A good checklist for a Next.js AGENTS.md:

Commands: install, dev, test, typecheck, lint, build.
Package manager: one, named explicitly, with "do not add a second lockfile".
Generated paths: .next/, dist/, coverage, Prisma client.
Conventions: Server Components by default, Zod validation at boundaries, server actions that authorize inside the action.
Git: commit format and what not to do (no force-push on shared branches).
Definition of done: tests pass, types check, no new warnings.

Avoid generic advice like "write clean code". It costs tokens and changes nothing.

How do GitHub Copilot instructions differ?

Copilot uses a repo-wide .github/copilot-instructions.md plus optional path-specific files in .github/instructions/ that end in .instructions.md and declare an applyTo glob. Copilot Chat, code review and the Copilot coding agent read them. Copilot also reads AGENTS.md files, with the nearest one taking precedence, but applyTo files are how you scope rules to certain paths.

A real path-specific file from the repo's NestJS example:

---
applyTo: "**/*.controller.ts,**/*.service.ts,**/*.module.ts,**/*.guard.ts,**/*.dto.ts"
---

# NestJS

- A feature is a module: `*.module.ts`, a controller, a service, and DTOs.
- Controllers translate HTTP. Services hold business rules. A controller does
  not import Prisma or Drizzle directly.
- Do not read `process.env` inside a service. Inject `ConfigService`.
- Authentication is a guard. Resource authorization happens in the service.

When should you use a Claude Code skill instead of a rule?

Use a rule for a constraint and a skill for a procedure. A rule says "never do X in these files". A skill is a step-by-step workflow, like making a commit or adding a server action, that Claude loads only when its description matches the task or when you type /name. Skills live in .claude/skills/<name>/SKILL.md with name and description frontmatter.

Per the skills docs, name is lowercase letters, numbers and hyphens. Here is the commit skill from the pack:

---
name: git-commit
description: Use when creating a git commit or pull request and the change should follow this repository's commit conventions.
---

# Commit a change

1. Inspect `git status` and `git diff`. Stage only the files that belong to this change.
2. Run typecheck and the tests that cover the change. Do not commit a known failure.
3. Write a Conventional Commit subject: `type(scope): imperative summary`.
4. Do not add `Co-authored-by` or other trailers unless the user asked.
5. Do not amend or force-push a commit that is already on a remote.

What is agent-rules and what does it include?

agent-rules is an MIT-licensed pack of production rules for AI coding agents, written once as .mdc source files and rendered into AGENTS.md, CLAUDE.md, Cursor rules, Copilot instructions, Claude Code skills and Windsurf/Devin rules. You pick the stacks you use, and the installer writes only those, so the formats never drift apart.

Stacks covered:

Stack	What the rules argue for
Core	Strict TypeScript, no `any` or `enum`, Zod at runtime, Conventional Commits, WCAG 2.2 AA, Vitest
Next.js	App Router only, Server Components first, `params` as promises, `proxy.ts` on Next.js 16, server actions that validate and authorize
React	Composition, derived state, `ref` as a prop on React 19
Node.js	Validate env once at startup, handle `SIGTERM`, timeouts on outbound calls
NestJS	Feature modules, thin controllers, whitelist validation, guards for auth
Prisma	Follow the installed major (Prisma 7 driver adapters), new migration per change, one client per process
Drizzle	Schema in TypeScript, `drizzle-kit generate` then `migrate`, bound parameters only
Tailwind CSS	v4 `@import "tailwindcss"` and `@theme` tokens, `cn()` for conditional classes
shadcn/ui	`components/ui` is source you own, semantic colors, compose what the CLI added

It also ships 12 Claude Code skills (commit, add tests, security review, accessibility pass, Next.js server action, Next.js cache, React component, Node service, Prisma change, Drizzle change, NestJS feature and shadcn compose) and two committed examples: a Next.js SaaS setup and a NestJS API.

How do you install the rules into your project?

Clone the repo, build the installer and point it at your project. It detects your stacks from package.json and components.json, asks you to confirm, then writes each format. Re-running updates only its own marked blocks in AGENTS.md, CLAUDE.md and copilot-instructions.md, so text you wrote outside them stays.

git clone https://github.com/fabwaseem/agent-rules.git
cd agent-rules && npm install && npm run build

# preview what would be written
node dist/cli.js --target ../my-next-app --dry-run

# install chosen stacks and formats
node dist/cli.js --target ../my-next-app \
  --stacks core,nextjs,react,prisma,tailwind,shadcn \
  --agents agents,claude,cursor,copilot

Prefer no tooling? Copy files by hand from examples/nextjs-saas: take AGENTS.md, CLAUDE.md, .cursor/rules/, .github/ and .claude/skills/ and drop them into your repo. An npx one-liner is coming soon once the package is published to npm.

The installer needs Node.js 20.19 or later. It also includes a validate command that checks Cursor frontmatter keys, Copilot applyTo, skill names and Windsurf triggers, including the 12,000-character Windsurf workspace rule limit.

Should you keep one file or several?

Keep one shared source and generate the rest. Hand-maintaining five formats means they drift: someone updates Cursor rules and forgets AGENTS.md. A single AGENTS.md plus a CLAUDE.md that imports it works for small repos. Larger repos benefit from glob-scoped Cursor and Copilot files so each conversation only carries relevant rules.

Repo size	Recommended setup
Solo or small app	`AGENTS.md` + `CLAUDE.md` with `@AGENTS.md`
Team Next.js app	Above + glob-scoped `.cursor/rules/.mdc` and `.github/instructions/`
Monorepo	Root `AGENTS.md` + nested `AGENTS.md` per package + scoped rules

If you run your own agent tooling, the same idea applies to servers too. I wrote about exposing tools to agents in How to Build an MCP Server in Next.js, and about getting content cited by AI engines in Next.js SEO for AI Search. You can see more of what I build on the projects page.

Related: when a section of AGENTS.md grows into a multi-step procedure, move it into an Agent Skill that loads only when needed. See Agent Skills: how to write a SKILL.md for Claude Code, Cursor, Codex and Copilot.

Key takeaways

AGENTS.md is the open, shared format. Use it for rules every agent should follow.
Claude Code reads AGENTS.md only when no CLAUDE.md exists, so add @AGENTS.md to your CLAUDE.md if you keep both.
Cursor needs .mdc files in .cursor/rules/, and globs keep rules out of unrelated conversations.
Copilot path rules use .instructions.md files with an applyTo glob.
Generate every format from one source so they never drift. agent-rules does this for the Next.js and TypeScript stack.

FAQ

Is AGENTS.md the same as CLAUDE.md?

No. AGENTS.md is an open format many coding agents read. CLAUDE.md is Claude Code's own project instructions file. Claude Code reads AGENTS.md only when no CLAUDE.md exists, so teams usually keep a short CLAUDE.md that imports AGENTS.md with an @AGENTS.md line.

Can I use AGENTS.md with Cursor?

Yes. Cursor supports AGENTS.md as plain project instructions. For rules that should apply only to certain files, use .mdc files in .cursor/rules/ with a globs pattern, which Cursor attaches only when matching files are in context.

Why is my Cursor rule not loading?

The most common cause is the file extension. Cursor ignores .md files in .cursor/rules/; they must be .mdc. Also check the frontmatter: a rule with alwaysApply false and no globs or description only loads when you @ mention it.

Does GitHub Copilot read AGENTS.md?

Yes. GitHub's docs say you can store AGENTS.md files anywhere in a repository and the nearest one takes precedence. Copilot also reads .github/copilot-instructions.md and path-specific .instructions.md files with an applyTo glob.

How long should AGENTS.md or CLAUDE.md be?

Short. Anthropic recommends under 200 lines per CLAUDE.md because longer files cost context and reduce adherence. Keep commands, conventions and must-not-do rules at the root, and move file-specific rules into scoped Cursor or Copilot files.

Is agent-rules free?

Yes. agent-rules is MIT-licensed on GitHub at github.com/fabwaseem/agent-rules. Clone it and run the installer, or copy the rendered files from the examples folder. An npx installer is coming once the package is on npm.

Originally published on fabwaseem.com.