DEV Community: Alex B

A vendor bar for a day nothing traded - why our completeness check was 97% wrong

Alex B — Thu, 29 Oct 2026 14:05:44 +0000

The first check we built to measure missing price data was 97% wrong, and nothing in the data said so.

Every backtest starts with an assumption nobody writes down: the price history is complete. Every trading day a
stock was listed, there is a bar.

Ours wasn't. And the first check we built to measure how incomplete it was turned out to be 97% wrong - with nothing in
the data to tell us.

The gap

We compared our own daily price files against the exchange calendar, stock by stock, inside each stock's own
first-to-last trading span since June 2025. 30,581 missing stock-days across 738 listed common stocks.

Most of them were names that had changed ticker or listed recently. The list of symbols our fetcher asked for only knew
names it already held, so a renamed company quietly stopped being fetched. One large bank was missing 86 daily
sessions after its ticker change.

The first check: "is there a bar?"

To separate real gaps from days a stock genuinely did not trade, we asked a market-data vendor one question per
missing day: do you have a bar for this stock on this day?

It said yes for 29,793 of the 30,581. That looked like a clean result: nearly every gap is real, go fill them.

What the bars actually were

Before filling anything, we read a few of the worst cases by hand. The bars were there - with zero volume and zero
trades. The price was simply the previous day's close, carried forward.

The vendor serves a bar for a day nothing traded. 19,066 of the "real" gaps were these placeholders.

Asking a different question - does the vendor's bar have trades? - the real number was 10,727 sessions across 240
stocks. All of them were filled the same morning (11,408 daily rows and 869,391 one-minute rows), and the same count
now reads 0. It runs every night.

Why this matters for a backtest

A placeholder bar is not harmless:

Volume filters lie. A stock looks listed and priced on a day it had no market at all.
Returns get smoothed. A flat day carried forward hides the real move, which then lands all at once on the next day that traded.
"Completeness" checks pass. Any check that asks does a row exist? is satisfied by a row that says nothing happened.

We found the same thing in our own stored files: three sampled days held 189, 467 and 432 zero-volume rows.

The lesson

A check that asks does a record exist? measures the vendor's habits, not the market. Ask the question the record is
supposed to answer: did this stock trade?

What I don't know

Whether the placeholder bars are documented behaviour, or a quirk of the way we ask for historical data.
What the right storage policy is. A day with no trades arguably has no bar at all - but many tools expect one.

A question for you

Who has a reliable public rule for "did this stock trade today" that doesn't depend on the vendor's own bar - the
exchange's daily volume files, consolidated trade counts, something else? If you've solved it, I'd like to learn how.

Want this checked on your own data or backtest? Backtest & Data Audit, one-week pilot: https://altdataforge.com/audit

Research and education only - not investment advice.

How do you know your company matching is right? We had to grade the grader

Alex B — Mon, 26 Oct 2026 14:05:24 +0000

Every agent that reads financial text has to decide which company a name means. Here is how we graded ours - and why we had to grade the answer key first.

Every AI agent that reads financial text has to answer one question before anything else: which company is this?
"Apple" in a filing. "MS" in a headline. A name that changed three years ago, or a ticker that now belongs to someone
else. Get it wrong and everything downstream is confidently wrong.

We have been building a lookup for exactly that: give it a company name as it was written, in a filing or an article,
and a date, and it returns the company that name meant on that date - including renamed and delisted ones. This week
we measured it properly. Three things surprised me.

1. We had to check the answer key first

To grade a matcher you need an answer key: a set of names where careful labellers have written down the right company.
We split ours in two - one half to tune on, one half held out and never looked at while tuning.

Before trusting any score, we had a second model act as a referee on 891 items of the key, without ever seeing what our
lookup had answered. It cost about $3.50 of API usage. What it found:

485 confirmed as they stood.
226 widened: the original answer was right, but the same company also existed as a second record, and the two had been offered as rivals. One company, two entries - both are now accepted.
101 settled: items the labellers had marked "can't tell" now have an answer.
2 reversed: outright wrong labels.

So the key was rarely wrong; it was incomplete. Every change is stored as its own record with a reason, on top of the
original, never as a silent edit.

On the checked key, the lookup scores 0.927 on the held-out half (629 matches; lower bound 0.904). The part I find
most useful: answers the lookup gave where the key said "can't tell" fell from 27 to 10 - most of them turned out
to be the same company listed twice. Real wrong answers barely moved (37 to 36). Checking the key changed what we
thought was a mistake, not how many mistakes we make.

If you grade an entity matcher against an answer key nobody has audited, you are partly grading the key.

2. Same question, three different orderings

Some names legitimately point to several entities - Morgan Stanley is a good example. When the lookup returned those
tied alternatives, their order depended on a random seed that changes every time a program starts. Same question,
three runs, three orderings. The answer itself never changed - only the order of the alternatives. For a person that is
a shrug; for an agent that takes the first item in a list, it is a coin flip. Ties now break on a stable identifier, and
the same question gets the same list every time.

3. News is not filings

Filings are formal; company names sit in predictable places. News prose is not. On FiNER-ORD, a public, human-labelled
set of financial news text (Shah et al., 2023), rule-based name finding scored an F1 of 0.350. A free open model
(GLiNER), fine-tuned on our filings data, scored 0.820. Rules that work on filings fall apart on headlines.

What I don't know yet

How well matching (not just finding) works on news text - there is no human-labelled answer key for that yet.
Whether 0.927 holds outside SEC filings.

What's next

We are working on letting AI agents ask this lookup directly - "which company did this name mean on this date?" -
with the answer key's score attached to every answer. If you are building an agent that needs that, reply or write to
AlexB@AltDataForge.com and I'll let you know when there is something to try.

Next post: when a ticker changes hands - how one company's history quietly leaks into another's when a symbol is
reused, and what it takes to stop it.

A question for you

If you run entity resolution for agents or research: how do you grade it - and has your answer key ever turned out to
be incomplete rather than wrong? If someone has published a good open answer key for company matching, link it and I'll
credit it.

Want this checked on your own data or backtest? Backtest & Data Audit, one-week pilot: https://altdataforge.com/audit

FiNER-ORD: Shah et al., 2023, arXiv:2302.11157 (CC BY-NC 4.0; used here for evaluation only). The free SEC data
dictionaries this work sits on: https://alexbocio.github.io/us-gov-data-dictionary/

Why your AI agent gets fund holdings wrong: the SEC's "Ticker" field isn't a ticker

Alex B — Sat, 17 Oct 2026 14:58:25 +0000

If you are building an agent or a model on SEC fund-holdings data, it will read a field called IDENTIFIER_TICKER and trust it. Here is what that field actually contains.

If you build fund-ownership data from SEC Form N-PORT, you will read a field called IDENTIFIER_TICKER and treat it as a stock symbol. The SEC's data-set documentation describes it in one word: "Ticker".

The form itself says more. Form N-PORT, Item C.1.e, asks for "at least one" other identifier: an ISIN, or a "Ticker (if ISIN is not available)", or some other identifier. Nothing requires the ticker to be an exchange symbol. It is a fallback.

So we measured what funds actually put there - every quarter the SEC has published, 2019 Q4 to 2026 Q2:

17,361,050 rows carry a ticker.
72% are on common-stock lines, and they look like real tickers: MSFT, AMZN, AAPL.
On mortgage-backed lines (676,336 rows) the "tickers" are agency pool-type prefixes: FN 161,550 rows, GN 109,156, FNMA 55,140, FHLMC 45,223.
On bond lines (2,312,102 rows) the ticker is the ISSUER's stock symbol, not the bond's: AT&T bonds carry "T" on 86,298 rows.
On derivative lines you find futures contract codes (TYZ2, ESZ2) and currency codes (USD, EUR).

Why it matters: FG is a Freddie Mac pool prefix. It is also the NYSE ticker of F&G Annuities & Life. Join N-PORT's ticker field to a ticker list and Freddie Mac mortgage pools turn into holdings of an insurance company. Bond holdings turn into stock holdings. Ownership counts, "smart money" signals and flow features all inherit it - silently.

Two more traps in the same files:

Placeholders that look like data: ISSUER_CUSIP is "N/A" on 47.5 million rows, "000000000" on 15.2 million and "999999999" on 210 thousand. ISSUER_LEI is "N/A" on 67.6 million. Count those as filled and your coverage numbers are wrong.
The SEC's machine-readable field list ships unchanged with every historical quarter, so a field "defined" there may not exist in older files (the VaR table is listed back to 2019 but only appears from 2022 Q1).

What to do: before trusting any identifier on an N-PORT line, check what kind of holding the line is (the ASSET_CAT field), and treat placeholder values as blank.

What I don't know yet:

Whether anyone publishes an open mapping from N-PORT pool prefixes (FN, GN, FG...) to the agency programs they stand for. I could not find one.
How much of the bond-line "issuer ticker" habit comes from filers themselves and how much from the software vendors that prepare their filings.

We published the full N-PORT field dictionary - the SEC's definitions with citations, the official code lists, and what is actually in each field - free, under CC BY 4.0: https://alexbocio.github.io/us-gov-data-dictionary/sec_nport.html

Source data: SEC Form N-PORT data sets, 2019 Q4 - 2026 Q2.

If you build fund-ownership data from N-PORT: how do you handle the non-stock lines today - map them, or drop them? And if someone has already solved this better, link it in the comments and I'll credit it in the dictionary.

One developer, one year, 1.8 billion rows: a point-in-time alternative-data platform

Alex B — Mon, 12 Oct 2026 19:33:37 +0000

In the last year I built, alone, a quantitative research data platform: about 1.8 billion rows across 764 tables of alternative data, SEC filings, government records and market data, engineered to be trusted in a backtest. I am self-taught. The discipline came from the work itself: 206 architecture decision records and 680 research notes, one for every problem worth not solving twice.

The hard problems were not the ones anyone warns you about. They were data problems.

The stack, in the terms the field uses

Point-in-time (as-of) data. Every record carries when it became public, not just when it happened, and 103 tables are bitemporal (valid time and recorded time). That is the defence against look-ahead bias, the most common way a backtest flatters itself.
Survivorship-bias-free history. Delisted and renamed companies stay in the data with their history, and a window in which a stock dies is scored with its delisting return instead of being dropped.
Entity resolution and a security master. Record linkage across tickers, CIKs, LEIs and CUSIPs, with point-in-time symbology: tickers get renamed, reused and delisted, so "which company was this on that date" is answered as of the date.
Corporate actions. Split and dividend adjustment validated against an independent source rather than against itself; the split-adjustment tool is open source.
Information extraction (NLP) from regulatory filings. 8-K events, earnings guidance and reported results, press releases, IPO prospectuses, SEC comment letters, material contracts, Schedule 13D/13G beneficial ownership, insider Forms 3/4/5, foreign-issuer reports and XBRL financial statements, turned into structured, dated facts.
A medallion lakehouse. Bronze, silver and gold layers on Parquet, PostgreSQL for the serving tables, about 456 data contracts guarding them, and data lineage that answers "what breaks downstream if this table changes" in one command.
MLOps and model validation. Feature pipelines, walk-forward and purged cross-validation, backtesting and paper trading, because a model that cannot fail a test is a horoscope.
Domain-specific language models. Small models fine-tuned for narrow financial extraction tasks on a desktop GPU.
Agentic workflows. Model Context Protocol (MCP) servers that let AI agents query the data, and agent-to-agent orchestration around them.
All of it on one workstation, with a resource broker that schedules heavy jobs so the machine does not eat itself.

By the numbers: 4,776 commits since 21 November 2025, 5,281 Python files, 738 of them tests.

Three lessons in data quality

1. Labels lie; specifications do not. The SEC's fund-holdings data (Form N-PORT) has a field called "Ticker". Measured across every quarter the SEC has published, about 28% of its 17.4 million values are not stock tickers: mortgage-pool codes, bond issuers' symbols, futures contracts. One pool code, FG, is also an NYSE ticker, so a naive join turns Freddie Mac mortgage pools into shares of an insurance company. The fix was metadata management, not cleverer code: the SEC's own form instructions say the field is a fallback identifier. I now build a data dictionary (a codebook, in the ISO/IEC 11179 sense) from each agency's official documentation before touching its data. The first seven dictionaries turned up twelve defects in the SEC's own published files.

2. A test that compares data with itself proves nothing. One evening every data-quality check on prices passed. The next morning an independent reconciliation found 236 days of prices rewritten onto the wrong side of a stock split. Every check had been comparing the pipeline with its own output. Now a data-quality claim needs an external witness.

3. Coverage is not precision. The easy metric in entity resolution is match rate. The useful one is precision. A shortcut that raised one match rate by ten points was right only 38% of the time against an independent identifier. It never shipped.

What is public

Two open-source releases so far:

split-adjustment-tool - verify, repair and apply stock-split adjustments against your own as-traded price tape. Vendor-neutral Python (Polars), Apache 2.0, release 0.4.0: https://github.com/AlexBocio/split-adjustment-tool
us-gov-data-dictionary - machine-readable data dictionaries for seven SEC data sets: 1,469 fields, each with the SEC's own definition and citation, official code lists, and a profile of what the published files actually contain. W3C CSVW and SKOS formats, CC BY 4.0: https://github.com/AlexBocio/us-gov-data-dictionary

More agencies are next. If you work with public financial data and it has ever quietly lied to you, I would like to hear the story.