DEV Community: alex

bounded queues and back‑pressure for high‑throughput websockets in next.js

alex — Fri, 23 Oct 2026 18:19:59 +0000

when a realtime dashboard needs to push thousands of events per second to dozens of browsers, a naive ws.send() loop can saturate the V8 event loop, drop frames, and eventually crash the server. i ran into this while extending a Next.js API route to broadcast sensor data from a Rust edge worker. the problem isn’t just bandwidth – it’s how quickly the Node process can drain its internal socket buffers.

why back‑pressure matters

Node’s WebSocket implementation respects the underlying TCP back‑pressure: socket.bufferedAmount grows when the OS can’t keep up. if you keep queuing messages without checking that metric, the process memory spikes and GC pauses become noticeable. the usual pattern of for (msg of msgs) ws.send(msg) ignores this signal, leading to “write after end” errors when the client disconnects mid‑burst.

a bounded queue solution

instead of hammering the socket, we can introduce a fixed‑size async queue. producers push payloads into the queue; a consumer task drains it at a rate the socket can sustain. if the queue is full, producers back‑off or drop low‑priority events. this gives a predictable memory ceiling and lets us surface back‑pressure upstream.

// simple bounded async queue (capacity = 5000)
class BoundedQueue<T> {
  private buffer: T[] = []
  private resolvePush!: () => void
  private resolvePop!: (value: T) => void
  constructor(public readonly capacity: number) {}

  async push(item: T) {
    while (this.buffer.length >= this.capacity) {
      await new Promise(r => (this.resolvePush = r)) // wait for space
    }
    this.buffer.push(item)
    this.resolvePop?.()
  }

  async pop(): Promise<T> {
    while (this.buffer.length === 0) {
      await new Promise(r => (this.resolvePop = r)) // wait for item
    }
    const val = this.buffer.shift()!
    this.resolvePush?.()
    return val
  }
}

// usage inside a Next.js API route
export default async function handler(req, res) {
  const ws = new WebSocket(req)
  const queue = new BoundedQueue<string>(5000)

  // producer: pipe Rust edge worker events into queue
  rustWorker.on('data', (msg) => queue.push(JSON.stringify(msg)))

  // consumer: respect socket back‑pressure
  while (ws.readyState === WebSocket.OPEN) {
    const payload = await queue.pop()
    if (ws.bufferedAmount > 1_000_000) {
      // pause draining until OS catches up
      await new Promise(r => setTimeout(r, 10))
    }
    ws.send(payload)
  }
}

tradeoffs to watch

latency vs drop rate – a small capacity reduces memory but may discard events during spikes. decide whether stale data is acceptable or if you need a replay buffer.
complexity – adding an async queue introduces another failure surface (e.g., dead‑locks if push/pop promises never resolve). unit‑test the queue under saturation.
horizontal scaling – if you scale the API route across instances, each will have its own queue. coordinating a global rate limit may require a shared broker like Redis Streams.
CPU overhead – the while loops with setTimeout are cheap, but in a hot path they can add ~0.2 ms per iteration. profile with node --prof if you see CPU spikes.

production lesson

the biggest surprise was how quickly ws.bufferedAmount grew under burst traffic from the Rust edge worker. once the queue capped at 5 k messages, memory usage stayed flat and the server recovered gracefully after the spike. without that guard, the pod would OOM in minutes.

anyone built a similar back‑pressure layer for websockets in a serverless or edge environment? what queue size or drop‑policy worked best for your latency‑sensitive use case?

taming cold starts in supabase edge functions with ts & pg

alex — Tue, 20 Oct 2026 02:25:04 +0000

when a Next.js app offloads auth and webhook handling to Supabase Edge Functions, the first request after a period of inactivity can spike latency past 2 seconds – a noticeable jank for realtime dashboards.

why the pause happens

edge functions run on isolated V8 isolates that are torn down after a configurable idle timeout (default ~5 minutes). on warm invocations the isolate reuses any in‑process state, but a cold start forces a fresh container, npm install, and most importantly a new pg client connection. the connection handshake alone can add 500‑800 ms, especially when TLS is involved.

keep‑alive vs pool inside the isolate

unlike traditional server processes you cannot keep a long‑living connection pool across isolates. the usual pg.Pool will still open a new socket on each cold start, then close it when the isolate dies. the trick is to lazily create a single client and reuse it only while the isolate lives.

import { createClient } from '@supabase/supabase-js'
import { Client } from 'pg'

let pgClient: Client | null = null

export const handler = async (event: any) => {
  if (!pgClient) {
    // first call in this isolate – establish the connection once
    pgClient = new Client({
      connectionString: process.env.SUPABASE_DB_URL,
      ssl: { rejectUnauthorized: false }
    })
    await pgClient.connect()
  }

  const { data } = await pgClient.query('SELECT now()')
  return new Response(JSON.stringify({ now: data[0].now }))
}

the pgClient lives in module scope, so subsequent invocations reuse the same socket until the isolate is reclaimed.

trade‑offs to watch

idle timeout – if your traffic pattern has >5 min gaps, every request still incurs the full cold start. you can tune EDGE_FUNCTIONS_TIMEOUT in Supabase dashboard, but longer lifetimes increase resource consumption and cost.
connection limits – each warm isolate holds an open DB connection. on a high‑scale deployment you could exhaust the Postgres max‑connections count. a pragmatic guard is to cap concurrent edge functions with Supabase quotas or implement a tiny “connection broker” that proxies through a lightweight pooler like PgBouncer.
error handling – when an isolate is reclaimed the client is abruptly closed, causing ECONNRESET on in‑flight queries. wrap queries in a retry block that re‑creates the client if pgClient is null or the query throws a connection error.
TLS handshake cost – you can shave a few hundred ms by using Supabase’s native pg connection string that enables sslmode=require without full certificate validation, but be aware of security posture in production.

monitoring cold starts

Supabase logs expose X-Edge-Function-Invocation headers with cold-start:true. ship a tiny middleware that records the latency delta and feeds it to your observability stack – you’ll quickly see whether your traffic pattern justifies a higher idle timeout or a fallback to a traditional serverless function.

anyone else juggling edge isolates with Postgres? how do you balance connection churn vs latency in your own stack

taming cold‑start latency in supabase edge functions (typescript)

alex — Sun, 18 Oct 2026 01:13:30 +0000

while building a low‑latency analytics dashboard on supabase edge functions, the first request after a few minutes of inactivity consistently hit ~800 ms latency. the spike broke the UI's 200 ms frame budget and forced me to rethink the function deployment.

why edge cold‑starts matter

edge functions run in lightweight containers that spin down after a period of inactivity. when a new request arrives the platform pulls the latest bundle, resolves dependencies and starts the runtime. this process is fast compared to traditional VMs, but it’s still enough to cause noticeable jitter in real‑time UIs.

three pragmatic knobs to pull

bundle size – every kilobyte adds network and unzip time. using esbuild with --bundle --minify strips dead code and inlines small assets.
runtime pre‑warm – a tiny cron job that pings the function every 30 s keeps the container alive without costing much quota.
cold‑start guard – detect a cold start inside the function and return a short‑lived placeholder while the heavy work finishes in the background.

example: a lean typescript edge handler

import { serve } from "@supabase/functions-js"

// small helper that can be tree‑shaken
export const getStats = async (userId: string) => {
  const res = await fetch(`https://api.example.com/stats/${userId}`)
  return res.json()
}

serve(async (req) => {
  // simple cold‑start flag – runs once per container lifecycle
  const isCold = !globalThis.__warm
  globalThis.__warm = true

  if (isCold) {
    // fire‑and‑forget heavy init (e.g. DB pool)
    setTimeout(() => getStats("placeholder"), 0)
    return new Response(JSON.stringify({ status: "warmup" }), { status: 202 })
  }

  const { userId } = await req.json()
  const data = await getStats(userId)
  return new Response(JSON.stringify(data), { headers: { "content-type": "application/json" } })
})

trade‑offs to watch

pre‑warm traffic: pinging every 30 s adds a few hundred requests per day; check your quota limits and consider a serverless‑friendly cron (supabase scheduled functions or GitHub actions).
stateful globals: using globalThis is cheap but only works for data that can survive a container restart. never store user secrets here.
error handling: the placeholder response should be idempotent; the client must be ready to retry or show a loading spinner.
bundle vs native deps: pulling in heavy native modules (e.g. pg-native) forces the platform to compile a larger binary, dramatically increasing cold‑start time. prefer pure‑js alternatives when latency is critical.

wrapping up

by shrinking the bundle, keeping the container warm, and explicitly handling the first request, I cut the 800 ms cold‑start tail to under 120 ms on repeat calls. the pattern works for most supabase edge functions that serve UI‑critical data.

what strategies have you used to mitigate edge cold‑starts in your stack, and how do you balance quota cost against latency guarantees?

taming bursty websocket traffic with supabase edge functions

alex — Fri, 16 Oct 2026 14:38:07 +0000

when you spin up a simple supabase edge function to proxy a websocket, it works fine for a few concurrent users. the moment a product launch or a live‑coding session pushes hundreds of connections within seconds, the function starts timing out, and you see 502s in the client console. this article walks through the root cause and a pragmatic pattern that keeps latency low without over‑engineering the stack.

why bursts break the naive setup

edge functions run in isolated containers that spin up on demand. each new container has a cold‑start penalty of ~50‑150 ms and a hard limit of 10 MB of inbound payload per request. a websocket handshake is just an HTTP upgrade, but every subsequent frame is still processed through the same request context. when dozens of connections arrive simultaneously, the platform throttles the number of active containers, queueing the upgrades. the result is dropped handshakes and jittery latency that quickly cascades into client‑side reconnect storms.

a lean edge‑function solution

instead of proxying every frame through the edge, keep the edge function solely as a connection broker. it validates the auth token, writes a short‑lived entry to supabase realtime, and returns a pre‑signed URL that points directly to a dedicated websocket server (e.g., a cheap node process on fly.io or a rust‑based tokio service). the client then connects straight to that server, bypassing the edge for the data path.

// supabase/edge/functions/connect.ts
import { serve } from "std/server";

serve(async (req) => {
  const auth = req.headers.get("authorization") ?? "";
  const user = await verifyJwt(auth);
  if (!user) return new Response("unauth", { status: 401 });

  // generate a signed URL valid for 30 seconds
  const wsUrl = `wss://ws.myservice.dev/${user.id}?exp=${Date.now()+30000}`;
  const signed = signUrl(wsUrl, process.env.SIGNING_KEY);
  return new Response(JSON.stringify({ url: signed }), {
    headers: { "content-type": "application/json" },
  });
});

the edge function now does less than 5 ms of work per request, stays well within the 10 MB limit, and avoids any per‑frame processing.

tradeoffs & failure modes

cold‑start still matters – the first batch of connections will experience the spin‑up latency; mitigate with a cron “ping” that keeps a container warm during expected traffic windows.
signing key leakage – expose the signing key only to the edge runtime; rotate regularly and embed a short TTL in the URL to limit replay attacks.
backend scaling – the downstream websocket server must handle the burst; autoscaling on fly.io works but introduces its own cold‑start; consider a rust tokio binary with a low memory footprint to spin up quickly.
observability gap – you now have two hops (edge → broker → ws); instrument both sides with correlated request IDs to trace dropped frames.

the pattern shifts the heavy lifting off the edge platform, preserving its cheap request‑based billing model while still leveraging supabase auth.

what’s your approach to high‑throughput websocket bursts?

have you built a similar broker layer, or do you rely on a different platform altogether? share the edge cases you hit and how you mitigated them.