DEV Community: clearpath-data

New US private funding rounds, daily, from Form D notices in Python

clearpath-data — Tue, 27 Oct 2026 18:49:02 +0000

When a US company raises money privately, under Regulation D for example, it usually files a short notice called Form D with the U.S. Securities and Exchange Commission. The notice is due within 15 days of the first sale, and it's public. It names the issuer, its state and industry, the size of the offering, how much has been sold so far and which exemption it relies on.

That makes Form D a handy, free signal for questions like which companies in my sector started raising money this week? Deal sourcing, market maps, local business reporting and sales research all start there.

The filings sit in the SEC's public EDGAR archive, which has no API key and no sign-up. This post builds a small script that:

finds the latest published business day;
lists that day's Form D and Form D/A (amendment) filings;
reads each notice and keeps the offering facts, skipping pooled investment funds;
writes a CSV, and runs once a day.

Python 3.10+ and requests. The XML parsing uses the standard library.

Step 0: be a polite client

The archive is free to use, with two fair-access conditions from the SEC's Accessing EDGAR Data page:

Declare who you are in the User-Agent header: a name plus a contact email.
Keep it under 10 requests per second in total. Going over can get your IP address temporarily blocked.

The script reads the User-Agent from an environment variable so your email stays out of the code:

export SEC_USER_AGENT="Your Name your-email@example.com"

It also spaces requests 0.25 s apart (4 per second at most) and backs off when the server says no:

# new_form_d.py  -- usage: python new_form_d.py [--date YYYY-MM-DD] [--limit N] [--include-funds]
import argparse
import csv
import datetime as dt
import os
import pathlib
import re
import time
import xml.etree.ElementTree as ET

import requests

ARCHIVES = "https://www.sec.gov/Archives/edgar"
# Fair access: say who you are (name + contact email), and stay well under 10 requests per second.
USER_AGENT = os.environ.get("SEC_USER_AGENT", "Your Name your-email@example.com")
MIN_INTERVAL = 0.25  # seconds between requests, so at most 4 per second

session = requests.Session()
session.headers.update({"User-Agent": USER_AGENT})
_last_request = 0.0


def get(url: str) -> requests.Response:
    """GET with a simple rate limit and a short back-off on 403/429/5xx."""
    global _last_request
    for attempt in range(3):
        wait = _last_request + MIN_INTERVAL - time.monotonic()
        if wait > 0:
            time.sleep(wait)
        _last_request = time.monotonic()
        r = session.get(url, timeout=30)
        # 403 can mean "slow down", or a User-Agent the site doesn't accept. Either way, back off.
        if r.status_code in (403, 429) or r.status_code >= 500:
            time.sleep(10 * (attempt + 1))
            continue
        r.raise_for_status()
        return r
    raise RuntimeError(f"Gave up on {url} (HTTP {r.status_code}). Check your User-Agent and request rate.")

Step 1: which days are published, and what was filed

The archive has a daily index for every business day. Each quarter's folder has an index.json listing, and each day has a form.YYYYMMDD.idx file: a fixed-width text list of every filing that day, sorted by form type. Form D notices have the form type D; amendments are D/A.

Form Type   Company Name                                                  CIK
      Date Filed  File Name
---------------------------------------------------------------------------------
D                ASAL International                                            1770205     20261023    edgar/data/1770205/0001770205-26-000003.txt
D/A              ABS ALPHA LTD                                                 1359353     20261023    edgar/data/1359353/0001013594-26-001074.txt

The last part of the file name is the filing's accession number, its unique ID. We need it, with the CIK (the filer's ID), to find the notice itself.

def published_days(year: int, quarter: int) -> list[dt.date]:
    """Business days in a quarter whose daily index has been published."""
    listing = get(f"{ARCHIVES}/daily-index/{year}/QTR{quarter}/index.json").json()
    days = set()
    for item in listing["directory"]["item"]:
        m = re.fullmatch(r"form\.(\d{8})\.idx", item["name"])
        if m:
            days.add(dt.datetime.strptime(m.group(1), "%Y%m%d").date())
    return sorted(days)


LINE = re.compile(r"^(D|D/A)\s+(.+?)\s+(\d+)\s+(\d{8})\s+edgar/data/\d+/(\d{10}-\d{2}-\d{6})\.txt\s*$")


def form_d_filings(day: dt.date) -> list[dict]:
    """Every Form D and D/A filed on `day`, from that day's form index."""
    quarter = (day.month - 1) // 3 + 1
    text = get(f"{ARCHIVES}/daily-index/{day.year}/QTR{quarter}/form.{day:%Y%m%d}.idx").text
    filings = []
    for line in text.splitlines():
        m = LINE.match(line)
        if m:
            form, company, cik, _, accession = m.groups()
            filings.append({"form": form, "company": company.strip(), "cik": cik, "accession": accession})
    return filings

Step 2: read each Form D

Each Form D filed online has a machine-readable primary_doc.xml in the filing's folder: /Archives/edgar/data/<CIK>/<accession without dashes>/primary_doc.xml. The useful parts:

XML path	What it is
`primaryIssuer/entityName`, `issuerAddress/stateOrCountry`	Who is raising, and where its principal place of business is
`offeringData/industryGroup/industryGroupType`	Industry, e.g. `Biotechnology`, `Other Technology`, `Pooled Investment Fund`
`offeringData/federalExemptionsExclusions/item`	Exemptions: `06b` = Rule 506(b), `06c` = 506(c), `04` = Rule 504, `3C.x` = Investment Company Act exclusions
`offeringData/offeringSalesAmounts/totalOfferingAmount`	Offering size in USD, or the word `Indefinite`
`offeringData/offeringSalesAmounts/totalAmountSold`	Amount sold so far
`offeringData/typeOfFiling/dateOfFirstSale`	A date, or `yetToOccur`
`relatedPersonsList/relatedPersonInfo`	Executive officers, directors and promoters, with names and addresses

The script keeps the offering facts and only counts the related persons. More on that below.

def text(node: ET.Element | None, path: str) -> str:
    found = node.find(path) if node is not None else None
    return (found.text or "").strip() if found is not None else ""


def read_offering(filing: dict) -> dict:
    """Fetch one filing's Form D document and keep the offering facts (no personal names)."""
    folder = filing["accession"].replace("-", "")
    url = f"{ARCHIVES}/data/{filing['cik']}/{folder}/primary_doc.xml"
    doc = ET.fromstring(get(url).content)
    issuer, offering = doc.find("primaryIssuer"), doc.find("offeringData")
    amounts = offering.find("offeringSalesAmounts")
    first_sale = text(offering, "typeOfFiling/dateOfFirstSale/value")
    if text(offering, "typeOfFiling/dateOfFirstSale/yetToOccur") == "true":
        first_sale = "yet to occur"
    return {
        "accession": filing["accession"],
        "form": filing["form"],
        "issuer": text(issuer, "entityName") or filing["company"],
        "state": text(issuer, "issuerAddress/stateOrCountry"),
        "entity_type": text(issuer, "entityType"),
        "industry": text(offering, "industryGroup/industryGroupType"),
        "exemptions": " ".join(e.text for e in offering.findall("federalExemptionsExclusions/item")),
        "first_sale": first_sale,
        "offering_usd": text(amounts, "totalOfferingAmount"),  # a number or "Indefinite"
        "sold_usd": text(amounts, "totalAmountSold"),
        "investors": text(offering, "investors/totalNumberAlreadyInvested"),
        "related_persons": len(doc.findall("relatedPersonsList/relatedPersonInfo")),
        "url": url,
    }

Step 3: skip the funds, write the CSV

A large share of each day's Form D filings come from pooled investment funds (hedge, private equity and venture funds raising from their own investors). If you're after operating companies, drop them; --include-funds keeps them.

def main() -> None:
    p = argparse.ArgumentParser()
    p.add_argument("--date", type=dt.date.fromisoformat, help="filing date; default: latest published day")
    p.add_argument("--limit", type=int, default=0, help="read at most N filings (0 = all)")
    p.add_argument("--include-funds", action="store_true", help="keep pooled investment funds")
    p.add_argument("--out-dir", default="data")
    args = p.parse_args()

    day = args.date
    if day is None:
        today = dt.date.today()
        day = published_days(today.year, (today.month - 1) // 3 + 1)[-1]
    filings = form_d_filings(day)
    print(f"{day}: {len(filings)} Form D filings ({sum(f['form'] == 'D/A' for f in filings)} amendments)")

    rows, funds = [], 0
    for filing in filings[: args.limit or None]:
        row = read_offering(filing)
        if row["industry"] == "Pooled Investment Fund" and not args.include_funds:
            funds += 1
            continue
        rows.append(row)

    out = pathlib.Path(args.out_dir) / f"form-d-{day}.csv"
    out.parent.mkdir(parents=True, exist_ok=True)
    with out.open("w", newline="", encoding="utf-8") as f:
        w = csv.DictWriter(f, fieldnames=list(rows[0]) if rows else ["accession"])
        w.writeheader()
        w.writerows(rows)
    print(f"{len(rows)} offerings kept, {funds} pooled funds skipped -> {out}")


if __name__ == "__main__":
    main()

A run for Friday 23 October 2026. --limit 15 reads only the first 15 notices in the index, which keeps this demo to 16 requests. Drop it to read the whole day, one request per notice:

$ python new_form_d.py --date 2026-10-23 --limit 15
2026-10-23: 331 Form D filings (143 amendments)
6 offerings kept, 9 pooled funds skipped -> data/form-d-2026-10-23.csv

Even in this small slice, most notices were pooled funds. Some rows of the CSV (columns trimmed):

form,issuer,state,industry,exemptions,first_sale,offering_usd,sold_usd,investors
D,ASAL International,TX,Business Services,06b,yet to occur,Indefinite,0,0
D,Advanced Glass Holdings, Inc.,IL,Other,06b,2026-10-09,2000000,2000000,10
D,Ascoltare Holdings LLC,FL,Biotechnology,06b,2025-05-09,4300000,1700000,10
D,Atlas 2026, LLC,OH,Other Real Estate,06b,2026-09-21,4750000,4750000,22

(The index lists issuers alphabetically, which is why they all start with "A". Names with commas are quoted in the real CSV; the quotes are dropped here for readability.)

A few things to read from this:

Ascoltare Holdings has sold 1,700,000 USD of a 4,300,000 USD offering. Its first sale was in May 2025, so this round has been open for a while.
Advanced Glass Holdings and Atlas 2026 report the whole offering as sold, from 10 and 22 investors.
ASAL International hasn't sold anything yet ("yet to occur"), and it didn't state a total, so the offering size is Indefinite.
Amendments (D/A, 143 of the 331 that day) come after the new notices in the index. An amendment can mean a bigger round, more money sold, or just a yearly update. Its previousAccessionNumber (in typeOfFiling/newOrAmendment) points to the filing it amends.

Step 4: run it every day

The default --date is the latest day in the index. The daily file is published after the business day closes, so running the next morning picks up the previous business day. A crontab line for 07:00 on weekdays:

0 7 * * 1-5 cd /home/me/form-d && SEC_USER_AGENT="Your Name your-email@example.com" /usr/bin/python3 new_form_d.py >> run.log 2>&1

Each notice is one request, so a full day takes as many requests as there are filings, at 4 per second.

Things to watch

People's names. Form D lists the issuer's executive officers, directors and promoters, often with an address, and the signature block names the signer. The filing is public, but once you store or combine those names you're processing personal data, and privacy law (GDPR, CCPA and others) can still apply. The script keeps only a count. If you need names, keep only what you need, store them securely and delete them when you're done.
It's a notice, not an audit. The figures are what the issuer reported. The SEC doesn't review or approve a Form D, and a filing isn't proof that any money was raised.
"Indefinite" amounts. Some issuers don't state a total, so offering_usd is the text Indefinite, not a number. Convert with care.
403 means slow down, or say who you are. A missing or unaccepted User-Agent gets a 403 page titled "Your Request Originates from an Undeclared Automated Tool". The same status is used for rate limiting. If you see it, check your User-Agent and slow down; don't retry in a tight loop.
Bulk history. For past quarters, the SEC also publishes Form D data sets as downloadable files, so you don't need one request per filing.

This article was written with the help of AI. All code was run against the live SEC EDGAR archive on 27 October 2026, and the output shown is real (trimmed where noted).

A daily list of new UK companies by SIC code with the Companies House API

clearpath-data — Mon, 26 Oct 2026 20:17:05 +0000

Every UK company declares what it does with one or more SIC codes when it registers. That makes the Companies House register a neat source for a simple question: which companies in my industry were incorporated yesterday? Market research, local-economy reporting, a "new in town" newsletter, or just curiosity all start there.

Companies House has a free API with an advanced search endpoint that filters by incorporation date and SIC code. This post builds a small script that:

finds the right SIC codes for your industry;
pulls every company incorporated on a given day with those codes;
writes a CSV and a short summary;
runs once a day.

Python 3.10+, requests, and PyYAML for the SIC lookup.

Step 0: a free API key

Authentication is HTTP Basic with a twist. From the authorisation guide: "the Companies House API takes the username as the API key and ignores the password, so can be left blank."

Keep the key out of your code:

export COMPANIES_HOUSE_API_KEY=your-key-here

Step 1: find your SIC codes

SIC codes are five digits (the SIC 2007 scheme): 62012 is "Business and domestic software development", 56101 is "Licensed restaurants". Companies House publishes the descriptions it uses in its public api-enumerations repository, so you can search them from Python. (The human-readable version is Companies House's condensed SIC list.)

# sic_lookup.py  -- usage: python sic_lookup.py software
import sys

import requests
import yaml

URL = "https://raw.githubusercontent.com/companieshouse/api-enumerations/master/constants.yml"


def sic_descriptions() -> dict[str, str]:
    data = yaml.safe_load(requests.get(URL, timeout=30).text)
    # The file also holds older 4-digit SIC 2003 codes; current filings use 5-digit SIC 2007.
    return {str(k): v for k, v in data["sic_descriptions"].items() if len(str(k)) == 5}


if __name__ == "__main__":
    names = sic_descriptions()
    words = [w.lower() for w in sys.argv[1:]]
    for code, desc in sorted(names.items()):
        if all(w in desc.lower() for w in words):
            print(code, desc)

$ python sic_lookup.py software
46510 Wholesale of computers, computer peripheral equipment and software
47410 Retail sale of computers, peripheral units and software in specialised stores
58290 Other software publishing
62011 Ready-made interactive leisure and entertainment software development
62012 Business and domestic software development

$ python sic_lookup.py restaurant
56101 Licensed restaurants
56102 Unlicensed restaurants and cafes

Step 2: the advanced search endpoint

GET https://api.company-information.service.gov.uk/advanced-search/companies takes, among others:

Parameter	What it does
`incorporated_from`, `incorporated_to`	date range (YYYY-MM-DD); use the same date for one day
`sic_codes`	comma-separated list; a company matches if it has any of them
`size`	results per page, documented as "a range of 1 to 5000"
`start_index`	offset for paging

Two details from the docs that the code below handles:

A 404 on this endpoint is documented as "No companies found", so treat it as an empty result, not an error.
The rate limit is "up to 600 requests within a five-minute period", after which you get 429 until the window resets. One day of one industry is usually a single request, so you're very unlikely to hit it, but a backfill over many days could.

# new_companies.py
import argparse
import collections
import csv
import datetime as dt
import os
import pathlib
import time

import requests

API = "https://api.company-information.service.gov.uk"
PAGE_SIZE = 5000  # documented maximum for advanced search ("a range of 1 to 5000")

session = requests.Session()
# The API key is the Basic-auth username; the password is empty.
session.auth = (os.environ["COMPANIES_HOUSE_API_KEY"], "")


def get(path: str, params: dict) -> dict:
    for _ in range(3):
        r = session.get(API + path, params=params, timeout=30)
        if r.status_code == 429:  # 600 requests per 5-minute window
            reset = int(r.headers.get("X-RateLimit-Reset", time.time() + 60))
            time.sleep(max(reset - time.time(), 1))
            continue
        if r.status_code == 404:  # documented as "No companies found"
            return {"hits": 0, "items": []}
        r.raise_for_status()
        return r.json()
    raise RuntimeError("Rate-limited three times in a row")


def new_companies(day: dt.date, sic_codes: list[str]) -> list[dict]:
    """All companies incorporated on `day` with any of `sic_codes`."""
    rows, start = [], 0
    while True:
        page = get("/advanced-search/companies", {
            "incorporated_from": day.isoformat(),
            "incorporated_to": day.isoformat(),
            "sic_codes": ",".join(sic_codes),
            "size": PAGE_SIZE,
            "start_index": start,
        })
        items = page.get("items", [])
        for c in items:
            addr = c.get("registered_office_address", {})
            rows.append({
                "company_number": c["company_number"],
                "company_name": c["company_name"],
                "incorporated": c.get("date_of_creation", ""),
                "company_type": c.get("company_type", ""),
                "sic_codes": " ".join(c.get("sic_codes", [])),
                "locality": addr.get("locality", ""),
                "postcode": addr.get("postal_code", ""),
                "url": f"https://find-and-update.company-information.service.gov.uk/company/{c['company_number']}",
            })
        start += len(items)
        if not items or start >= page.get("hits", 0):
            return rows

Each item in the response already carries the company number, name, status, type, date_of_creation, SIC codes and registered office address, so there's no need for a second call per company.

Step 3: write the CSV and a summary

def main() -> None:
    p = argparse.ArgumentParser()
    p.add_argument("--sic", required=True, help="comma-separated SIC 2007 codes, e.g. 62012,62020")
    p.add_argument("--date", type=dt.date.fromisoformat, default=dt.date.today() - dt.timedelta(days=1))
    p.add_argument("--out-dir", default="data")
    args = p.parse_args()

    sic = [s.strip() for s in args.sic.split(",") if s.strip()]
    rows = new_companies(args.date, sic)

    out = pathlib.Path(args.out_dir) / f"new-companies-{args.date}-{'-'.join(sic)}.csv"
    out.parent.mkdir(parents=True, exist_ok=True)
    with out.open("w", newline="", encoding="utf-8") as f:
        w = csv.DictWriter(f, fieldnames=list(rows[0]) if rows else ["company_number"])
        w.writeheader()
        w.writerows(rows)
    print(f"{len(rows)} companies incorporated on {args.date} with SIC {', '.join(sic)} -> {out}")
    towns = collections.Counter(r["locality"].strip().title() for r in rows if r["locality"])
    for town, n in towns.most_common(5):
        print(f"  {town:<20} {n}")


if __name__ == "__main__":
    main()

Running it for software companies (custom software 62012 plus IT consultancy 62020) on Friday 23 October 2026:

$ python new_companies.py --sic 62012,62020 --date 2026-10-23
215 companies incorporated on 2026-10-23 with SIC 62012, 62020 -> data/new-companies-2026-10-23-62012-62020.csv
  London               97
  Manchester           5
  Brighton             3
  Bury St. Edmunds     3
  Leicester            3

The first rows of the CSV (some columns trimmed):

company_number,company_name,incorporated,sic_codes,locality
17536836,ORBON TECHNOLOGIES LTD,2026-10-23,62012 62090,Romford
17536871,BARLLAST TECHNOLOGIES LTD,2026-10-23,62012 62020 62090 63110,London
17536870,STALVIX LTD,2026-10-23,62020 62090,London

Notice the extra codes: a company can choose up to four SIC codes, and the filter matches on any of them, so 62090 and 63110 show up alongside the ones you asked for.

And restaurants and cafés for the day before:

$ python new_companies.py --sic 56101,56102 --date 2026-10-22
70 companies incorporated on 2026-10-22 with SIC 56101, 56102 -> data/new-companies-2026-10-22-56101-56102.csv

Step 4: run it every day

The default --date is yesterday. Running for today gives a partial list, because companies keep being registered through the day. A crontab line that runs at 07:00 and reads the key from a file only you can read:

0 7 * * * cd /home/me/new-companies && COMPANIES_HOUSE_API_KEY=$(cat ~/.ch_key) /usr/bin/python3 new_companies.py --sic 62012,62020 >> run.log 2>&1

Don't skip weekends. Companies are incorporated at weekends too: on Sunday 25 October 2026 there were 34 new companies with SIC 62012, against 156 on Friday 23 October.

Things to watch

SIC codes are self-declared. A company picks its own codes at registration, and some pick a broad one like 62090 ("Other information technology service activities") or 82990. Expect some noise and, for a tight list, filter by name or check the companies by hand.
Personal data. For many small companies the registered office is an accountant's or a director's home address. The script keeps only town and postcode and never fetches officers. If you go further (officer names, full addresses), you're processing personal data and UK GDPR applies, even though the register is public.
Paging. If you query a wide date range or many codes, hits can exceed one page. The loop follows start_index until it has everything.

This article was written with the help of AI. All code was run against the live Companies House API on 26 October 2026, and the output shown is real (trimmed where noted).

BIC and ISIN to LEI in Python with the GLEIF API

clearpath-data — Mon, 26 Oct 2026 20:14:51 +0000

If you work with payments, trades or counterparty lists, you end up holding one identifier and needing another. A bank shows up as a BIC (DEUTDEFFXXX), a bond or share as an ISIN (US0378331005), and the regulator, the KYC team or the reporting template wants the LEI, the 20-character Legal Entity Identifier.

The Global Legal Entity Identifier Foundation (GLEIF) runs a free, public API that holds those mappings. No key, no signup. This post walks through:

validating LEI, ISIN and BIC strings offline, so typos never reach the API;
BIC → LEI in batches;
ISIN → LEI, and why that one has to go one code at a time;
enriching a CSV with LEI, legal name, country and LEI status.

Everything uses requests and the standard library (Python 3.10+).

The API in one minute

The endpoint is https://api.gleif.org/api/v1/lei-records. It returns JSON:API documents, and you filter with query parameters:

https://api.gleif.org/api/v1/lei-records?filter[bic]=DEUTDEFFXXX
https://api.gleif.org/api/v1/lei-records?filter[isin]=US0378331005

Two things from GLEIF's API documentation matter here:

Rate limit: "Rate limiting is currently set at 60 requests, per minute, per user, for all users." Above that you get HTTP 429.
ISIN mappings come from the ISIN-to-LEI mapping process run with the Association of National Numbering Agencies (ANNA). Not every ISIN has one.

GLEIF's terms of use say the data "are provided under the CC0 licence", so you can store and reuse the results.

Step 1: validate before you call

All three identifiers can be checked offline. An LEI ends in two check digits (ISO 17442: read the code as a number with A=10 … Z=35, and it must leave remainder 1 when divided by 97). An ISIN ends in a Luhn check digit (ISO 6166). A BIC has no check digit, but it does have a fixed shape, and it has a catch: GLEIF stores 11-character BICs, so the common 8-character form DEUTDEFF returns nothing until you append XXX.

# identifiers.py
import re

LEI_RE = re.compile(r"^[A-Z0-9]{18}[0-9]{2}$")
ISIN_RE = re.compile(r"^[A-Z]{2}[A-Z0-9]{9}[0-9]$")
BIC_RE = re.compile(r"^[A-Z]{4}[A-Z]{2}[A-Z0-9]{2}([A-Z0-9]{3})?$")


def _letters_to_digits(s: str) -> str:
    """A=10, B=11, ... Z=35; digits stay as they are."""
    return "".join(str(int(c, 36)) for c in s)


def is_valid_lei(lei: str) -> bool:
    """ISO 17442: the 20-character code, read as a number, mod 97 must equal 1."""
    lei = lei.strip().upper()
    return bool(LEI_RE.match(lei)) and int(_letters_to_digits(lei)) % 97 == 1


def is_valid_isin(isin: str) -> bool:
    """ISO 6166: convert letters to numbers, then run the Luhn check."""
    isin = isin.strip().upper()
    if not ISIN_RE.match(isin):
        return False
    digits = _letters_to_digits(isin)
    total = 0
    for i, d in enumerate(reversed(digits)):
        n = int(d)
        if i % 2 == 1:
            n *= 2
            if n > 9:
                n -= 9
        total += n
    return total % 10 == 0


def normalize_bic(bic: str) -> str | None:
    """Uppercase, drop spaces, and expand 8-character BICs to 11 with 'XXX'."""
    bic = bic.replace(" ", "").strip().upper()
    if not BIC_RE.match(bic):
        return None
    return bic + "XXX" if len(bic) == 8 else bic

A quick check, with one deliberate typo in the last digit of each code:

LEI  7LTWFZYICNSX8D621K86: True
LEI  7LTWFZYICNSX8D621K87: False
ISIN US0378331005: True
ISIN US0378331006: False
BIC  'deutdeff' -> DEUTDEFFXXX
BIC  'BNPA FRPP XXX' -> BNPAFRPPXXX
BIC  'not-a-bic' -> None

Step 2: BIC → LEI, in batches

The bic filter accepts a comma-separated list, and every LEI record lists the BICs mapped to it in attributes.bic. That means one request can resolve up to a page of codes, and you can match each result back to the code you asked for. (Deutsche Bank's record, for example, carries more than 70 BICs, so check membership rather than assuming one BIC per record.)

# gleif.py
import time

import requests

from identifiers import is_valid_isin, normalize_bic

API = "https://api.gleif.org/api/v1/lei-records"
session = requests.Session()
session.headers["Accept"] = "application/vnd.api+json"


def get(params: dict) -> dict:
    """GET with a polite retry on HTTP 429 (GLEIF allows 60 requests a minute)."""
    for attempt in range(4):
        r = session.get(API, params=params, timeout=30)
        if r.status_code == 429:
            time.sleep(int(r.headers.get("Retry-After", 10)) + attempt * 5)
            continue
        r.raise_for_status()
        return r.json()
    raise RuntimeError("Still rate-limited after 4 attempts")


def summarize(record: dict) -> dict:
    a = record["attributes"]
    return {
        "lei": a["lei"],
        "legal_name": a["entity"]["legalName"]["name"],
        "country": a["entity"]["legalAddress"]["country"],
        "lei_status": a["registration"]["status"],         # ISSUED, LAPSED, ...
        "next_renewal": a["registration"]["nextRenewalDate"][:10],
    }


def bics_to_lei(bics: list[str], chunk: int = 50) -> dict[str, dict | None]:
    wanted = {b: normalize_bic(b) for b in bics}
    found: dict[str, dict] = {}
    codes = sorted({c for c in wanted.values() if c})
    for i in range(0, len(codes), chunk):
        part = codes[i:i + chunk]
        data = get({"filter[bic]": ",".join(part), "page[size]": 200})["data"]
        for rec in data:
            for code in rec["attributes"].get("bic") or []:
                if code in part:
                    found[code] = summarize(rec)
    return {b: found.get(c) if c else None for b, c in wanted.items()}

Step 3: ISIN → LEI, one at a time

The isin filter also takes a list, but the LEI record that comes back doesn't contain its ISINs. With three ISINs in, you get three records out and no reliable way to say which ISIN produced which record. So for ISINs, make one request per code and pace yourself under the rate limit:

def isin_to_lei(isin: str) -> dict | None:
    isin = isin.strip().upper()
    if not is_valid_isin(isin):
        return None
    data = get({"filter[isin]": isin})["data"]
    return summarize(data[0]) if data else None

(If you need the reverse direction, GLEIF also serves all ISINs for one LEI at /lei-records/{lei}/isins.)

Running both:

for bic, hit in bics_to_lei(["DEUTDEFF", "BNPAFRPPXXX", "ZZZZGB2LXXX"]).items():
    print(f"{bic:<12} -> {hit['lei'] + '  ' + hit['legal_name'] if hit else 'no LEI found'}")
for isin in ["US0378331005", "DE0007164600", "GB0002634946"]:
    hit = isin_to_lei(isin)
    print(f"{isin} -> {hit['lei'] + '  ' + hit['legal_name'] if hit else 'no LEI found'}")

Output on 26 October 2026:

DEUTDEFF     -> 7LTWFZYICNSX8D621K86  DEUTSCHE BANK AKTIENGESELLSCHAFT
BNPAFRPPXXX  -> R0MUWSFPU8MPRO8K5P83  BNP PARIBAS
ZZZZGB2LXXX  -> no LEI found
US0378331005 -> HWUPKR0MPOU8FGXBT394  Apple Inc.
DE0007164600 -> 529900D6BF99LW9R2E68  SAP SE
GB0002634946 -> 8SVCSVKSGDWMW2QHOH83  BAE SYSTEMS PLC

Note that the 8-character DEUTDEFF matched only because normalize_bic padded it.

Step 4: enrich a CSV

The usual real-world input is a spreadsheet export with a bic column, an isin column, or both. This script adds the LEI fields plus a match column, so you can see why a row didn't match:

# map_csv.py  -- usage: python map_csv.py input.csv output.csv
import csv
import sys
import time

from gleif import bics_to_lei, isin_to_lei
from identifiers import is_valid_isin, normalize_bic

FIELDS = ["lei", "legal_name", "country", "lei_status", "next_renewal", "match"]


def main(src: str, dst: str) -> None:
    with open(src, newline="", encoding="utf-8") as f:
        rows = list(csv.DictReader(f))

    bic_hits = bics_to_lei([r["bic"] for r in rows if r.get("bic")])

    isin_hits = {}
    for isin in {r["isin"] for r in rows if r.get("isin")}:
        isin_hits[isin] = isin_to_lei(isin)
        time.sleep(1.1)  # stay under 60 requests a minute

    for r in rows:
        hit, how = None, ""
        if r.get("bic") and bic_hits.get(r["bic"]):
            hit, how = bic_hits[r["bic"]], "bic"
        elif r.get("isin") and isin_hits.get(r["isin"]):
            hit, how = isin_hits[r["isin"]], "isin"
        if not hit and r.get("isin") and not is_valid_isin(r["isin"]):
            how = "invalid_isin"
        elif not hit and r.get("bic") and not normalize_bic(r["bic"]):
            how = "invalid_bic"
        r.update({k: (hit or {}).get(k, "") for k in FIELDS[:-1]})
        r["match"] = how or "none"

    with open(dst, "w", newline="", encoding="utf-8") as f:
        w = csv.DictWriter(f, fieldnames=list(rows[0].keys()))
        w.writeheader()
        w.writerows(rows)
    print(f"{sum(r['match'] in ('bic', 'isin') for r in rows)}/{len(rows)} rows matched -> {dst}")


if __name__ == "__main__":
    main(sys.argv[1], sys.argv[2])

Input:

name,bic,isin
Deutsche Bank,DEUTDEFF,
BNP Paribas,BNPAFRPPXXX,
Apple,,US0378331005
SAP,,DE0007164600
Typo example,,US0378331006
Unknown BIC,ZZZZGB2LXXX,

Output (26 October 2026):

4/6 rows matched -> out.csv

name,bic,isin,lei,legal_name,country,lei_status,next_renewal,match
Deutsche Bank,DEUTDEFF,,7LTWFZYICNSX8D621K86,DEUTSCHE BANK AKTIENGESELLSCHAFT,DE,ISSUED,2027-06-02,bic
BNP Paribas,BNPAFRPPXXX,,R0MUWSFPU8MPRO8K5P83,BNP PARIBAS,FR,ISSUED,2027-02-06,bic
Apple,,US0378331005,HWUPKR0MPOU8FGXBT394,Apple Inc.,US,ISSUED,2027-03-08,isin
SAP,,DE0007164600,529900D6BF99LW9R2E68,SAP SE,DE,ISSUED,2027-01-26,isin
Typo example,,US0378331006,,,,,,invalid_isin
Unknown BIC,ZZZZGB2LXXX,,,,,,,none

Things to watch

lei_status matters. An LEI that isn't renewed on time shows as LAPSED. It still identifies the entity, but if the LEI is going into a regulatory report, check whether a lapsed one is acceptable there. Keeping the status and next_renewal columns, not only the code, makes that easy.
No match isn't an error. Plenty of BICs and ISINs have no LEI mapping. Keep the none rows so someone can look at them.
Branch BICs. A BIC with a branch code (last three characters other than XXX) only matches if that exact BIC is in the mapping. If it isn't, you can retry with the XXX form, but then record that the match came from the head-office BIC, not the branch.
Cache. For large lists, store results and re-check on a schedule instead of calling the API for every run.
Bulk alternative. For very large lists, GLEIF also publishes downloadable BIC-to-LEI and ISIN-to-LEI relationship files, so you can join locally instead of calling the API.

This article was written with the help of AI. All code was run against the live GLEIF API on 26 October 2026, and the output shown is real (trimmed where noted).

Watching a company's hiring week to week: snapshot and diff Greenhouse, Lever and Ashby job boards

clearpath-data — Mon, 19 Oct 2026 09:13:08 +0000

"Spotify has 82 open jobs" is a weak signal. "Spotify opened 17 roles and closed 14 in ten days" is a much stronger one, and you can't get it from a careers page. You get it by taking snapshots and diffing them. Here's a small, dependency-light way to do that for companies on Greenhouse, Lever or Ashby.

Step 1: one snapshot function for three ATSs

All three ATSs publish public JSON job boards. For a diff you only need a stable key per posting (its public URL) and something human-readable (the title):

ATS	Endpoint	URL field	Title field
Greenhouse	`boards-api.greenhouse.io/v1/boards/{slug}/jobs`	`absolute_url`	`title`
Lever	`api.lever.co/v0/postings/{slug}?mode=json`	`hostedUrl`	`text`
Ashby	`api.ashbyhq.com/posting-api/job-board/{slug}`	`jobUrl`	`title`

The slug is the board name in the careers URL (jobs.lever.co/spotify → spotify).

"""Save today's open jobs for one company board as {job_url: title}."""
import datetime, json, pathlib, sys
import requests

FETCH = {
    "greenhouse": lambda s: [(j["absolute_url"], j["title"]) for j in
        requests.get(f"https://boards-api.greenhouse.io/v1/boards/{s}/jobs", timeout=30).json()["jobs"]],
    "lever": lambda s: [(j["hostedUrl"], j["text"]) for j in
        requests.get(f"https://api.lever.co/v0/postings/{s}", params={"mode": "json"}, timeout=30).json()],
    "ashby": lambda s: [(j["jobUrl"], j["title"]) for j in
        requests.get(f"https://api.ashbyhq.com/posting-api/job-board/{s}", timeout=30).json()["jobs"]],
}

def snapshot(ats, slug, folder="snapshots"):
    jobs = dict(FETCH[ats](slug))
    path = pathlib.Path(folder) / f"{ats}-{slug}-{datetime.date.today()}.json"
    path.parent.mkdir(exist_ok=True)
    path.write_text(json.dumps(jobs, indent=1, ensure_ascii=False))
    return path, len(jobs)

if __name__ == "__main__":
    print(*snapshot(sys.argv[1], sys.argv[2]))

Running it on 19 Oct 2026:

$ python snapshot.py lever spotify
snapshots/lever-spotify-2026-10-19.json 82
$ python snapshot.py greenhouse stripe
snapshots/greenhouse-stripe-2026-10-19.json 729
$ python snapshot.py ashby openai
snapshots/ashby-openai-2026-10-19.json 814

Run it from cron (or GitHub Actions) once a day. Each snapshot is a small JSON file, so a year of daily snapshots per company is easy to keep in git.

Step 2: diff two snapshots

"""Compare two snapshots: which postings opened, which closed."""
import json, sys

old, new = (json.load(open(p)) for p in sys.argv[1:3])
opened = sorted(set(new) - set(old), key=new.get)
closed = sorted(set(old) - set(new), key=old.get)
print(f"{len(old)} -> {len(new)} open jobs | +{len(opened)} opened | -{len(closed)} closed")
for url in opened[:5]:
    print("  +", new[url])
for url in closed[:5]:
    print("  -", old[url])

I had a real Spotify baseline from 9 October (all 79 postings on its Lever board that day), so here is the 10-day diff:

$ python diff.py baseline-lever-spotify-2026-10-09.json snapshots/lever-spotify-2026-10-19.json
79 -> 82 open jobs | +17 opened | -14 closed
  + AV Engineer - Platform
  + Art Director - Mexico
  + Audiobook Licensing & Partnerships Manager - Japan
  + Client Partner ANZ
  + Client Partner, Emerging & Scaled
  - Android Engineer - Experience
  - C++ Engineer - Experience
  - Client Partner - Emerging & Scaled, Independent Agency (UK)
  - Director, Portfolio & Monetization Architecture
  - Growth Manager

What the net number hides

The headline count moved by +3. Underneath, 31 postings changed: 17 opened and 14 closed, so roughly 40% of a 79-job board turned over in ten days. If you only track counts, you'd conclude nothing happened.

I also counted the same diff by title instead of URL. It gives 18 titles opened and 15 closed (some titles appear more than once on the board), so the churn isn't just the same job re-posted under a new URL.

A few practical notes from building this:

Key on the URL, not the title. Companies often run several postings with the same title in different cities. Title-keyed dicts silently collapse them.
"Closed" means "no longer on the board". That could mean filled, paused or moved. Treat it as a signal, not a fact.
Lever dates are epoch milliseconds (createdAt); Greenhouse and Ashby use ISO 8601. The diff approach sidesteps dates entirely, which is part of why I like it.
Be polite. These are companies' public career feeds. Once a day is plenty, and cache everything.

New postings vs. disappeared postings

Some job APIs let you filter to recently posted jobs. That tells you what's new, while the snapshot diff also tells you what disappeared. For hiring-trend work you usually want both.

All the code is in github.com/clearpath-data/ats-jobs-api.

This article was created with the help of AI and reviewed for accuracy before publishing.