DEV Community: Felix Yang

204 of a board's 232 jobs were "updated" in the same second

Felix Yang — Sat, 17 Oct 2026 18:58:55 +0000

I aggregate job boards. One of the features is "posted in the last 24 hours".

For a week it worked. Then I pulled a full board and asked a simpler question than I usually ask: what does the updated_at distribution actually look like?

GitLab's board, 232 live postings:


distinct `updated_at` values	29
jobs sharing one single timestamp	204 (`2026-10-15T15:24:46-04:00`)
earliest `first_published` on the board	`2026-03-06`
`updated_at` on that same job	`2026-10-15` — 222 days later

So my "new in the last 24 hours" feed was reporting 204 jobs as fresh. Their real first-published dates span seven months. Every one of them was 222 days old at the most, and one of them was a March posting wearing an October timestamp.

Nothing errored. The API returned 200, the timestamps parsed, the sort worked. The output was just wrong in the way that survives review — plausible, formatted, and committed.

The timestamp is a column about the record, not about the posting

updated_at is a data-quality field. It answers "when did this row last change in our database", and a board that re-syncs its whole catalogue changes every row at once. That's the 204. It is not a bug on Greenhouse's side; it's a field being asked to answer a question it was never about.

The tell is the histogram. Twenty-nine distinct values across 232 rows is not a posting pattern. Postings arrive one at a time — a company does not hire 204 people in the same second. A single value covering 88% of the board is a write operation.

Compare the other two vendors I hit, same question, same day:

Vendor	Field	Postings	Distinct values
Greenhouse (`/v1/boards/{slug}/jobs`)	`updated_at`	232	29
Ashby (`/posting-api/job-board/{slug}`)	`publishedAt`	167 / 814	167 / 814
Lever (`/v0/postings/{slug}?mode=json`)	`createdAt`	82	82

Ashby: 814 postings, 814 distinct timestamps. Lever: 82 and 82. Those two answer "when was this posted" with a per-posting event. Greenhouse answers with a per-row write. Same logical field, opposite reliability — and if you wrote one normaliser and trusted it, you would only be wrong on one third of your sources, which is exactly the amount of wrong that doesn't get noticed.

What actually works

Two fields, one local set.

# DON'T: this is a "board sync" feed, not a "new jobs" feed
new_jobs = [j for j in jobs if is_recent(j["updated_at"])]

# DO: structural date for recency, id for novelty
seen = load_seen_ids()                      # persisted between runs
first_run = not seen
new_jobs = [
    j for j in jobs
    if j["id"] not in seen
    and (first_run or parse(j["first_published"]) > last_run_at)
]
save_seen_ids(seen | {j["id"] for j in jobs})

first_published is the only Greenhouse field that behaves like a posting date. But it still can't tell you newly new on its own — a job published in March and still open is legitimately in the feed forever. Recency is a property of the posting; novelty is a property of your observation. Those are two different clocks and they need two different fields.

The first_run branch matters more than it looks. On a cold start, every posting is unseen, so the difference set is the whole board. Without that branch you either publish 232 "new" jobs on day one, or you silently drop the board and never recover. I hit both.

This is the same failure as the missing-key trap in the first post of this series — a field that exists, parses, and returns a value that means something other than what you assumed. null at least stops you.

Two more things that cost more than they look

?content=true is a 22× tax. GitLab's board, same 232 jobs either way:

request	bytes	ratio
`/jobs`	163,949	1.0×
`/jobs?content=true`	3,646,907	22.2×
`/jobs?content=true`, gzip	1,180,797	7.2×

The job count is identical. You get the same 232 records with a content key added. Fetch descriptions on the detail endpoint for the jobs you actually keep, not on the listing that exists to be filtered.

And note the middle column versus the right one: 22× and 7× are the same request. If you benchmark bandwidth with curl --compressed and then reason about your JSON parser's memory, you are off by a factor of three and you will never find it, because both numbers are real and neither is labelled. I have made this mistake in a report; the fix is to write the encoding into the sentence, not the number.

limit can be decorative. Remotive accepts ?limit= and ignores it:

limit=3   -> HTTP 200, 173,720 B, job-count 13
limit=50  -> HTTP 200, 173,720 B, job-count 13

Byte-identical. Both responses are the same 13 jobs. A limit parameter that parses, validates, and does nothing is worse than no parameter — you'll build a paging loop on it and silently stop at whatever the server felt like sending. The only way to know a parameter works is to send two values and compare the counts, not the status codes. Both were 200.

The rule I write down now

For every timestamp I consume, I check the histogram before I check the value:

Distinct values ≈ row count → it's an event field. Trust it for recency.
One value covering most rows → it's a write field. It records your vendor's cron, not the world.

That check costs one Counter() and it is the difference between a feed and a random-number generator. The 204 rows were not lying to me. They were answering the question I should have asked.

If you'd rather not maintain three normalisers yourself: I run this as a hosted actor — paste a company, slug or careers URL and get one clean dataset back, built from the Greenhouse, Ashby and Lever job-board APIs. ATS Job Board Aggregator.

Written with AI assistance and verified by hand — every board, count and byte figure above was pulled live on 2026-10-18. The normalisers are in ats_normalizers.py.

Your fetch failed, your code wrote an empty array, and now your tests assert a lie

Felix Yang — Fri, 16 Oct 2026 20:38:09 +0000

A scraper of mine reported that a GitHub repository had zero closed pull requests.

The repository had dozens. It had its first PR closed 21 minutes after it was created, three days before the scraper ran.

Nothing crashed. No test failed. The number was produced, written to disk, committed, and then quoted in a report as evidence. The bug was not that the scraper lied — it is that my pipeline had no vocabulary for "I don't know", so it said 0 instead.

The chain of custody of a wrong number

Here is the whole failure, step by step.

1. A timeout that didn't look like a timeout.

The fetch was a curl call with a 25-second ceiling:

curl -sS --max-time 25 \
  "https://api.github.com/repos/$REPO/pulls?state=closed&per_page=100"

A large repo returned ~606 KB. The body did not finish in 25 seconds, so curl killed the transfer mid-flight. curl still exited having written something to stdout, and — this is the part that matters — stdout contained the first N bytes of a perfectly valid JSON array, cut off in the middle of a string.

2. The parse error was swallowed.

try:
    data = json.loads(out)
except Exception:
    data = []          # <- the entire bug lives on this line

One except Exception. It was written to be defensive and it is exactly what turned a truncation into a measurement.

3. The empty list was written to disk as a result.

Downstream code received [], counted it, and stored closed_prs: 0. A reader of that file cannot distinguish it from a repo that genuinely has no closed PRs. The snapshot format had one slot where there should have been two.

4. The empty list was promoted to a test fixture.

This is the expensive part. A regression fixture was created to cover the case "we found no evidence, so we must answer UNKNOWN". A real repository was picked as the realistic example, and the recorded value was closed_prs: 0 — read straight from the poisoned snapshot. From that commit onward, the test suite asserted a false statement about a third party, forever, and passing tests meant the bug was invisible.

5. The only reason it was caught was that the repo was small enough to enumerate every PR with its own timestamp. On a repo with 900 PRs there would have been no way to notice, and the false fact would still be in the fixture today.

Two variations of the same bug

Once I started looking, the same shape showed up elsewhere in the same codebase.

Variant A — a successful HTTP call that means "you are not authenticated".

curl -sS -H "Authorization: Bearer $TOKEN" "$API/runs"
# HTTP 401
# body: {"error": {"type": "unauthorized", "message": "Invalid token"}}

The check was: did curl succeed, and does the body parse as JSON? Both true. Therefore runs = []. The dashboard reported zero runs for weeks. The token had expired on day one.

An auth failure had been silently converted into a zero — and zero is a plausible number for runs, which is why nobody questioned it.

Variant B — the timeout that leaves no trace at all.

--max-time bounds the whole transfer. It does not bound the connect phase. Measured hang before the failure surfaced:

curl -sS --max-time 30 "$URL"
# ... 268,008 ms later ...
# curl: (28) Operation timed out

Four and a half minutes, with a 30-second timeout on the command line. If your retry logic lives above this call, your retry logic never ran. --connect-timeout is a separate, non-negotiable flag:

curl -sS --connect-timeout 8 --max-time 30 "$URL"

And a hung first call was enough to abort the entire scan and discard three windows of already-collected data, because one unguarded call sat outside every try.

The rule: three states at every boundary

The fix is not "add more error handling". It is to make EMPTY and ERROR different types that cannot be compared or coerced into each other, so that the compiler — or at least the first reader — sees the difference.

I now return a small tri-state from every fetch, and nothing downstream sees a bare list:

from dataclasses import dataclass

OK, EMPTY, ERROR = "ok", "empty", "error"

@dataclass(frozen=True)
class Fetch:
    status: str          # OK | EMPTY | ERROR
    items: list
    detail: str = ""     # why it failed; never empty when status == ERROR

    def require(self):
        if self.status != OK:
            raise RuntimeError(f"fetch {self.status}: {self.detail}")
        return self.items

Four properties are doing the work here.

1. ERROR can never be downgraded. A helper that returns [] on failure is not defensive, it is a lie generator. If a caller wants to continue, it must opt in explicitly, and the failure must survive into the output.

2. EMPTY must be earned. An empty list is only a fact when the transport completed and the parser ran and the shape was checked. Anything else is ERROR:

def fetch_json(url, timeout=(8, 30)):
    r = subprocess.run(
        ["curl", "-sS", "-w", "\n%{http_code}",
         "--connect-timeout", str(timeout[0]),
         "--max-time", str(timeout[1]), url],
        capture_output=True, text=True)

    if r.returncode != 0:
        return Fetch(ERROR, [], f"curl rc={r.returncode}: {r.stderr.strip()[:200]}")

    body, _, code = r.stdout.rpartition("\n")
    if code.strip() != "200":                       # read the status, never infer it
        return Fetch(ERROR, [], f"http {code.strip()}: {body.strip()[:200]}")

    try:
        data = json.loads(body)
    except json.JSONDecodeError as e:               # truncation lands here
        return Fetch(ERROR, [], f"json error: {e}")

    if not isinstance(data, list):
        return Fetch(ERROR, [], f"unexpected shape: {type(data).__name__}")

    return Fetch(OK if data else EMPTY, data)

Note what is not in that function: no bare except, and no default value that is also a legitimate result.

3. Snapshot writers must refuse to write on failure. A partial or empty write is worse than no write, because the next run diffs against it and reports fabricated change:

def save_snapshot(path, windows):
    ok = [w for w in windows if w.status == OK]
    err = [w for w in windows if w.status == ERROR]

    if not ok:                       # nothing succeeded -> write nothing at all
        raise SystemExit(2, f"no window fetched: {[w.detail for w in err]}")

    if err:                          # partial -> write, but name what is missing
        print(f"WARNING: {len(err)} window(s) missing: "
              f"{[w.name for w in err]}")

    path.write_text(json.dumps([w.items for w in ok]))

The rule underneath: the file on disk must never be indistinguishable from a successful run.

4. Errors must carry their own diagnosis. detail is why the failure is debuggable six weeks later without re-running anything. "fetch error: 401" is a fact; "fetch error" is a shrug.

Testing the failure path without lying to yourself

Two things went wrong in how this code was tested.

Assert on the value that can actually be wrong. The original test asserted the number of matches found by the parser. The broken code produced the same number of matches — it was wrong about which row and what value, not about how many. Old code and new code both passed. If a test would pass with the bug present, it is not testing the bug. Assert the field that moves:

assert [p["closed_at"] for p in result] == ["2026-10-15T20:21:49Z"]

Use orthogonal single-point failures, not just total failure. "Everything fails" is the easy case and it exercises the least code. Run the pipeline twice: once with every fetch failing (expect: no file written, non-zero exit), and once with exactly one named window failing while the others succeed (expect: file written, missing window named, exit 0). The second run is the one that catches the swallowed exception.

Never let captured data masquerade as a synthetic fixture. A fixture harvested from a live third party is a claim about the world, and the world changes. If you keep one, store the command, the timestamp, and the exit status alongside it — or replace it with a hand-written fixture and record why. A fixture that asserts a durable fact about a mutable external system is a scheduled lie.

The generalisation

Every one of these bugs is the same bug: a value that means "I don't know" was given the same representation as a value that means "there is nothing".

It is worth hunting for on purpose, because the symptoms are quiet:

A function with except Exception: return [].
A pipeline that checks if response.ok but never response.status_code.
Any retry loop wrapping a call whose timeout doesn't cover connecting.
Any snapshot, cache, or fixture file whose contents are valid in both the "ran and found nothing" and "never ran" cases.
Any test that asserts a count.

None of these raise. They all produce numbers. And a pipeline that cannot say "I don't know" will say 0 instead, which is a much more expensive answer.

Your fake Chrome User-Agent is why that site returns 403

Felix Yang — Thu, 15 Oct 2026 09:32:32 +0000

I spent twenty minutes convinced a site had gone offline.

It hadn't. I was the one who broke it — by putting a Chrome User-Agent on a curl command.

Here is the original measurement. Same machine, same connection, same few seconds, one URL:

UA="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 \
(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"

curl -sS -o /dev/null -w "%{http_code}\n" -A "$UA" https://alistapart.com/about/contribute/
# 403

curl -sS -o /dev/null -w "%{http_code}\n" https://alistapart.com/about/contribute/
# 200

curl -sS -o /dev/null -w "%{http_code}\n" -A "" https://alistapart.com/about/contribute/
# 200

The fake browser is the one that gets blocked. The honest curl is the one that gets the page.

The 403 is not a firewall error, it's a challenge page

The body gives it away:

<title>Checking your browser...</title>
<script src="/__challenge/probes.js"></script>

That is a JS challenge. The site is not refusing to serve the document — it is refusing to serve it to a client that claims to be Chrome but cannot run the JavaScript it just handed you.

Which is the whole point. A request that says Chrome/126 and then presents no browser behaviour is a more suspicious artifact than a request that says curl. It is self-contradictory, and inconsistency is cheap to detect. Being honest costs you nothing here.

The inverse case is real too

An empty User-Agent fails on the same logic, from the other direction. Requesting https://dev.to/ with no UA at all:

HTTP/2 403
server: Varnish
content-length: 0

Zero-byte body, no challenge, just a Varnish-layer rejection. "Claims to be nothing" is also easy to classify.

So there are two failure directions, and they look identical if you only look at the status code:

What you send	How it reads to bot mitigation	What you get
`Chrome/126` with a non-browser client	contradictory	challenge / 403
no User-Agent at all	anonymous, unclassifiable	rejection / 403
an honest client UA (`curl/8.x`)	internally consistent	the document

The control experiment that should be in every scraper repo

One request tells you nothing about a site. It tells you about your request. So run three, and log which one produced which code:

test_url () {
  local u="$1"
  local ua="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 \
(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
  printf '%-46s chrome=%s default=%s empty=%s\n' "$u" \
    "$(curl -sS -o /dev/null -w '%{http_code}' -m 25 -A "$ua" "$u")" \
    "$(curl -sS -o /dev/null -w '%{http_code}' -m 25 "$u")" \
    "$(curl -sS -o /dev/null -w '%{http_code}' -m 25 -A "" "$u")"
}

test_url https://alistapart.com/about/contribute/
test_url https://www.smashingmagazine.com/contact/
test_url https://www.infoq.com/
test_url https://blog.logrocket.com/
test_url https://dev.to/

Run just now, that prints:

alistapart.com/about/contribute/        chrome=403  default=200  empty=200
smashingmagazine.com/contact/           chrome=200  default=200  empty=200
infoq.com/                              chrome=405  default=405  empty=405
blog.logrocket.com/                     chrome=200  default=200  empty=200
dev.to/                                 chrome=200  default=200  empty=403

Read that table carefully, because it is doing more work than it looks:

One URL out of five is UA-sensitive (ALA). If I had tested only that one, I would have concluded "UA spoofing is broken everywhere". If I had tested only the other four, I would have concluded "UA spoofing is fine". Both conclusions would have been wrong, and the fix would have been wrong in opposite ways.
405 is not a UA problem at all. InfoQ returns it identically to all three clients. That is a method/policy answer, and no header change will move it. Chasing it with UA rotation is pure waste.
The direction of the effect is not consistent. Chrome-UA fails on ALA, empty-UA fails on dev.to. There is no "best UA" to hardcode. There is only "check, per site, per request shape".

Two things this is not

It is not a transport failure. Separately from all of the above, some hosts are simply unreachable from a given network. https://news.ycombinator.com/ returns 000 — curl's "no HTTP response at all", in this case a TLS-level refusal on my egress. All three UA variants returned 000. If you see the same failure across every UA, you are not debugging a header. Stop editing the header.

The distinction matters because the two failure modes have opposite fixes: a 000 is a network/egress problem, a 403 may be a client-fingerprint problem, and a 405 is neither.

It is not a licence to bypass anything. If a site hands you a challenge page, that is the site answering your request. A scraper that solves the challenge to get around a documented "no automated access" policy is a different activity from a scraper that notices it is sending a self-contradictory request. The three-UA control tells you whether your own client is the problem. It is a diagnostic, not a bypass — and the honest answer is sometimes "this site does not want a script here, go use their API or their form".

The rule I ended up with

A 403 is two different bugs wearing the same status code. Before you write "site blocks scrapers" in a README, re-run the request with the default UA and with an empty one. Record all three codes. Only the pattern is evidence.

Twice now, on this same codebase, the two-UA retest has overturned a conclusion I had already written down: once when a failed request got filed as "the endpoint is gone", and once when it got filed as "the submission form no longer accepts email". Both times the site was fine and my client was lying about itself.

The checker behind the table above is a few lines of shell — I run the three variants in a wrapper that prints one row per URL, because a status code without its control group is not a measurement. If you maintain scrapers or an uptime check, the cheapest upgrade available to you is probably the second and third request.

Written with AI assistance and verified by hand — all responses above were pulled live on 2026-10-15 from a single machine and connection.

Every careers page has a public JSON API — you just never noticed

Felix Yang — Mon, 12 Oct 2026 07:10:14 +0000

Most people who need job-posting data reach for a headless browser. They point Playwright at a careers page, hope the React app has hydrated, and write selectors that will break the next time marketing ships a redesign.

You usually don't need any of that.

The three ATS platforms that most startups use — Greenhouse, Ashby and Lever — already serve every job board as plain JSON. They have to. That's how the "embed our jobs on your own website" widget works.

These endpoints are public. No login, no API key, no browser. Here are all three, with real numbers I pulled while writing this.

The three endpoints

Provider	Endpoint	Shape
Greenhouse	`https://boards-api.greenhouse.io/v1/boards/{slug}/jobs`	`{"jobs": [...]}`
Ashby	`https://api.ashbyhq.com/posting-api/job-board/{slug}`	`{"jobs": [...]}`
Lever	`https://api.lever.co/v0/postings/{slug}?mode=json`	bare `[...]`

Two of them wrap the array; Lever returns a naked one. That alone will make you write three normalizers, so plan for it.

Live counts, pulled on 2026-10-12:

Greenhouse  stripe    → 727 jobs
Greenhouse  gitlab    → 221 jobs
Ashby       openai    → 811 jobs
Ashby       ramp      → 165 jobs
Lever       leverdemo →  10 jobs

That's 1,934 postings from five requests. No browser, no proxies, well under a second each.

Working out which ATS a company uses

Don't guess — read the careers URL. All three vendors use predictable hosts:

job-boards.greenhouse.io/stripe   → greenhouse, slug "stripe"
boards.greenhouse.io/gitlab       → greenhouse, slug "gitlab"
jobs.ashbyhq.com/openai           → ashby,       slug "openai"
jobs.lever.co/leverdemo           → lever,       slug "leverdemo"

The slug is whatever sits between the host and the next slash. On a company's own domain the path is usually similar — /careers redirects, or the embed URL leaks the slug in a query parameter.

The smallest thing that works

slug=stripe
curl -s "https://boards-api.greenhouse.io/v1/boards/$slug/jobs" \
  | jq -r '.jobs[] | [.title, .location.name, .absolute_url] | @tsv' \
  | head -20

And in Python, normalizing all three into one shape:

import httpx

def fetch(provider: str, slug: str) -> list[dict]:
    if provider == "greenhouse":
        url = f"https://boards-api.greenhouse.io/v1/boards/{slug}/jobs"
        raw = httpx.get(url, timeout=20).json()["jobs"]
        return [{"title": j["title"], "url": j["absolute_url"],
                 "location": (j.get("location") or {}).get("name"),
                 "updatedAt": j.get("updated_at")} for j in raw]

    if provider == "ashby":
        url = f"https://api.ashbyhq.com/posting-api/job-board/{slug}"
        raw = httpx.get(url, timeout=20).json()["jobs"]
        return [{"title": j["title"], "url": j["jobUrl"],
                 "location": j.get("location"),
                 "updatedAt": j.get("publishedAt")} for j in raw]

    if provider == "lever":
        url = f"https://api.lever.co/v0/postings/{slug}?mode=json"
        raw = httpx.get(url, timeout=20).json()   # bare array
        return [{"title": j["text"], "url": j["hostedUrl"],
                 "location": j.get("categories", {}).get("location"),
                 "updatedAt": j.get("createdAt")} for j in raw]

    raise ValueError(provider)

The traps nobody mentions

The slug is not the company name. gitlab works. So does a lot of near-misses that don't — which is why you need to distinguish "wrong slug" from "no openings right now". A board that exists but has zero jobs still returns a valid empty result. Test for a 200 with an empty array, not a 404. Otherwise you'll spend an afternoon renaming a company that was never hiring.

Timestamps are three different formats. Greenhouse gives you ISO-8601 strings, Ashby gives ISO-8601, Lever gives epoch milliseconds as an integer. If you sort a mixed array without normalizing, you get silent nonsense.

Descriptions can blow up your payload. Greenhouse's ?content=true and Ashby's ?includeCompensation=true pull the full HTML job description — routinely 10–20 KB per posting. On a board with 800 openings that's tens of megabytes for data most pipelines throw away. Fetch titles first, then enrich only the rows you actually need.

Be a good citizen. These are public endpoints, but they're not yours. Space your requests, don't run a hundred boards in parallel, and cache. I use a 250 ms delay between boards and four-way concurrency, which is enough to feel instant and rude to nobody.

Check the terms before you redistribute. The endpoints are public; the postings aren't necessarily yours to republish. Personal data and contact details inside postings deserve particular care, especially if you're in a jurisdiction with data-protection law.

If you'd rather not maintain this

All of the above is maybe 120 lines of code. I wrote it out and put it here — standard library only, no dependencies, MIT:

github.com/yff124/apify-actors/blob/main/ats_normalizers.py

python3 ats_normalizers.py stripe                           # bare slug, auto-detects the provider
python3 ats_normalizers.py https://jobs.ashbyhq.com/openai  # or hand it a careers URL

It accepts a company name, an explicit provider:slug, or a full careers URL, auto-detects which of the three platforms answers, and returns one schema — the same three public endpoints described above, with the slug guessing, the timestamp normalizing and the politeness handled for you.

If you'd rather not run anything locally, the same thing is also packaged as a hosted Actor — paste a company or a careers URL, it runs in the cloud and hands you a dataset: ATS Job Board Aggregator. It covers Workday too, which needs a separate endpoint and a paging quirk the script above leaves out.

Run 5 companies or 500. If you only ever need one company, the curl command at the top of this post is genuinely all you need — I'd rather you knew how it works than treated it as magic.

Update — 2026-10-16

A reader flagged that this post understated one trap, and it's worth putting real numbers on it, because it cost us a bug.

Greenhouse doesn't return descriptions "if you ask for them, and they're large". It returns no content key at all unless you pass ?content=true. Same board, same minute, Accept-Encoding: identity so these are actual payload bytes — 725 jobs in both cases:

request	payload	keys on `jobs[0]`	`content` key
`/jobs`	452,886 B	14	absent
`/jobs?content=true`	5,646,271 B	20	present

A missing key fails differently from a null one. Read it with j.get("content") and you get a silent nothing; validate it against a schema and you get a hard failure — and only on Greenhouse, because Ashby and Lever return content unconditionally. Those two pass, and passing is exactly what hides the problem.

Two more things that came out of chasing it:

?content=false is observably identical to sending no parameter (38,229 B vs 38,103 B compressed, same 14 keys). We now pass the flag explicitly in both directions instead of trusting the default — "the vendor changed the default" is not a failure you can see coming.
Stripping tags is not enough. Decode entities first, then drop <script>/<style>, then the remaining tags, then collapse whitespace. Truncating at 20k characters per job is what keeps a 725-job board storable.

And the mirror image of the slug trap above, stated as a rule: 404 = wrong board; 200 with [] = right board with nobody hiring. All three vendors hand you a real body to tell them apart — Greenhouse {"status":404,...}, Ashby Document not found, Lever a plain Not found — so check the status before you parse, and you'll never spend an afternoon renaming a company that was never hiring.

The Greenhouse branch, after all of this, is in `ats_normalizers.py`.

Written with AI assistance and verified by hand — every endpoint and job count in this post was pulled live on 2026-10-12.

CJK line breaking in Canvas, and what the browser will not do for you

Felix Yang — Sun, 11 Oct 2026 08:42:47 +0000

CSS has been quietly good at CJK line breaking for years. word-break: normal plus line-break: strict plus the browser's default break-opportunity table gets you typographically sane paragraphs without writing a line of logic.

Canvas does not. <canvas> gives you measureText() — a width per string — and nothing else. If you build anything that composes text into an image (a share card, a certificate, a dynamic cover, a mini-program poster), you are writing your own line breaker. And the naive one is wrong in a way that is easy not to notice if you do not read Chinese.

The naive implementation, and exactly how it fails

accumulate characters until measureText(line + ch).width > maxWidth
then push the line and start a new one

For Latin text this is roughly acceptable, because the breaks almost always land near spaces. For CJK there are no spaces, so the break lands wherever the width runs out:

本产品支持批量导出
图片并自动压缩

That is fine by luck. Now shift the box 20px:

本产品支持批量导
出图片并自动压缩

导出 has been split. A Chinese reader does not read this as a slightly awkward break, they read it as a typo. The same code, two different results, and only one of them is acceptable.

Three rules that fix most of it

1. Prefer a break at punctuation. Before accepting a width-driven break, scan back up to ~12 characters for the nearest closing punctuation — ，。、：；！？）」』】—… — and break there instead. This single rule accounts for the majority of natural break points in running prose.

2. Keep numbers and their units together. If the break would leave 20 at the end of one line and + or 项 at the start of the next, pull the number down. Readers parse 20 / + 项 as two separate values.

3. Never start a line with closing punctuation. A full stop, comma, or closing bracket at the start of a line is the single most visible CJK typographic error, and it is also the easiest to check for.

const CLOSERS = "，。、：；！？）」』】—…";
const OPENERS = "（「『【";

function breakBefore(ctx, text, i, font, maxWidth, indent = 12) {
  // try punctuation within `indent` characters, walking backwards
  for (let k = 1; k <= indent && i - k > 0; k++) {
    if (CLOSERS.includes(text[i - k])) return i - k + 1; // break AFTER the closer
  }
  // fall back: never leave a closer stranded at line start
  if (CLOSERS.includes(text[i])) return i + 1;
  return i;
}

Two implementation details that are easy to get wrong:

Measure with the same font object you draw with. ctx.measureText() after changing ctx.font reflects the new font. Cache the width of each character once per (font, size) pair; measuring per character per frame is the usual reason a text layout janks.
Let CJK characters wrap individually but keep Latin words intact. A run of [A-Za-z0-9]+ should be treated as one indivisible unit. Mixing the two scripts in one paragraph is the normal case in Chinese product copy, and it is where most homegrown wrappers fall apart.

If you can, do not write this at all

Modern browsers ship Intl.Segmenter with granularity: "word", which gives you the same break-opportunity data the CSS engine uses:

const seg = new Intl.Segmenter("zh-Hans", { granularity: "word" });
[...seg.segment("本产品支持批量导出图片并自动压缩")].map(s => s.segment);

Support is now broad enough to use directly, with a character-by-character fallback. If you are on the web and only need to lay out HTML, prefer CSS and stop reading. Canvas is the case where you have no choice.

The part that only shows up in production

Canvas fails differently from Python: there is no missing-glyph exception, silently, the same way. If a font fallback chain resolves to a face without the needed glyph, you get tofu boxes or blanks inside your drawn text. In a headless renderer (server-side, or a mini-program's offscreen canvas) you often do not see it until a user screenshots the result.

So after layout, run a cheap audit over the union of every character you intend to draw, and assert the chosen font actually covers it. Which characters break is always domain-specific — the rare surname on a certificate, the technical term on a data card — so audit your real strings, not lorem ipsum.

The rule set above, a wrapped-measureText layout pass, and the fallback chain is roughly what cjk-text-render does on the Python side. Writing the browser version is an afternoon; getting the break rules right is the part people skip.

Your CJK text is silently broken, and Pillow will not tell you

Felix Yang — Sun, 11 Oct 2026 08:42:20 +0000

If you generate images with Chinese, Japanese, or Korean text in Python, there is a category of bug you have almost certainly shipped: the text rendered, nothing threw, and the output is wrong.

Not "wrong" as in a stack trace. Wrong as in a user looks at the poster and says "something feels off" without being able to name it. Or worse: a character is simply missing from the image and nobody notices until print.

I collected the three that bite hardest while building cjk-text-render, a single-file helper for drawing CJK text with Pillow. All three are silent by construction.

Trap 1: `PingFang.ttc` index 0 is not "Regular"

On macOS, /System/Library/Fonts/PingFang.ttc is a TrueType Collection. The order of faces inside it is not weight order. It is region order:

index	face	notes
0	PingFang HK Regular	Hong Kong Traditional, the one people accidentally use
1	PingFang TC Regular	Taiwan Traditional
2	PingFang SC Regular	Simplified Chinese, the one you want
3 / 4 / 5	HK / TC / SC Medium
6 / 7 / 8	HK / TC / SC Semibold	use 8 for bold Simplified
9 / 10 / 11	HK / TC / SC Light

Because index defaults to 0, the idiomatic-looking call is the wrong one:

# WRONG: renders everything in Hong Kong Traditional glyph forms
font = ImageFont.truetype("/System/Library/Fonts/PingFang.ttc", 40)

# RIGHT: Simplified Chinese Regular
font = ImageFont.truetype("/System/Library/Fonts/PingFang.ttc", 40, index=2)

Two things go wrong when you use index 0. First, every character is drawn in the Hong Kong Traditional glyph variant. Simplified Chinese and Traditional Chinese share a large number of codepoints but not their shapes, so you get characters that are recognisably the wrong form while still being the right codepoint. Second, and much nastier: some Simplified characters are absent from the HK face. 稳 is a common example. Pillow draws a blank space. No exception, no warning, no log line.

If you want to check what you actually loaded, inspect the face rather than trusting the index:

from PIL import ImageFont
f = ImageFont.truetype("/System/Library/Fonts/PingFang.ttc", 40, index=2)
print(f.getname())   # -> ('PingFang SC', 'Regular')

Print getname() in CI. It is a one-line assertion that catches an entire class of platform-dependent regressions before they reach a human.

Trap 2: missing glyphs are drawn as blank, not raised

This is the one that should be an exception and is not.

font = ImageFont.truetype("NotoSansCJK-Regular.ttc", 40)
draw.text((0, 0), "文字と漢字の混在", font=font, fill="black")

If the loaded face does not contain one of those characters, Pillow does not raise KeyError, does not log, and does not fall back to another face. It draws nothing for that character. You get a gap in the middle of a word.

The fix is an explicit audit before you render. Pillow exposes what you need through font.getmask, but the ergonomic version is a helper that raises on the first missing codepoint:

from cjk_render import audit

audit(["all the text you are about to draw"])   # raises RuntimeError on any missing glyph

The design decision here matters: make it crash. A script that explodes at build time is strictly better than a delivered image with a hole in it. The failure mode you cannot afford is the one that ships.

Run audit() on your real strings, not on sample text. The characters that go missing are always the domain-specific ones — a rare surname on a certificate, a technical term on a data card.

Trap 3: line-breaking CJK by width instead of by punctuation

CJK has no spaces, so the obvious wrapping implementation — accumulate characters until the line exceeds max_width, then break — produces this:

知道你在打
杂

The word 打字 has been cut in half. Readers notice immediately. Latin line-breaking intuition simply does not transfer.

There are three rules worth encoding:

Prefer punctuation. Look back up to ~12 characters from the overflow point for the nearest of ，。、：；！？）」』】—… and break there if one exists.
Never split numeric units. If breaking would put 20 on one line and + on the next, the reader parses two numbers instead of one quantity.
No orphaned closing punctuation. A full stop or comma must never begin a line.

from cjk_render import wrap

lines = wrap(draw, "一段很长的中文正文……", font, max_w=900)

A short list of things that are not traps but will bite anyway

Once the three silent failures are handled, a few loud-but-annoying ones remain:

Save PNG, not JPEG. Dense CJK strokes at small sizes pick up chromatic noise from JPEG compression. If you must use JPEG, keep quality >= 95.
Auto-fit functions have an off-by-one that inverts them. In a while size > min_size shrink loop, if min_size is not strictly below the smallest starting size you call it with, the loop body never executes and your "shrink to fit" silently leaves the text at its original size, overflowing the canvas.
Cap the hierarchy at three levels. On a 1080x1080 thumbnail, a fourth text level is decoration, not information.

Why this is worth caring about

AI image generation still mangles CJK text, so any design with a lot of it — posters, data cards, marketing images, before/after comparisons — has to be drawn in code. That makes these three traps load-bearing: they are on the critical path of an entire category of work, and all three are silent.

The library is a single file with no dependencies beyond Pillow. Take it, or take the three checks and port them into whatever renderer you already have. Just do not rely on the renderer to tell you it failed. It will not.

Repo: github.com/yff124/cjk-text-render (MIT)