DEV Community: Jean-Luc Martel

Python: while...else and Other Crimes Against Intuition

Jean-Luc Martel — Mon, 02 Nov 2026 07:00:00 +0000

We've already explored some of PHP's more questionable life choices and JavaScript's remarkable ability to make perfectly valid code look like a practical joke.

Now it's Python's turn.

Python is particularly interesting because it has become the language of choice for data science, machine learning, and AI worlds.

And understandably so.

It's readable. It's expressive. Its ecosystem is extraordinary. You can go from a CSV file to a predictive model in 5 lines of code.

But Python has a few surprises of its own.

And when you're working with data, some of those surprises can produce results that are plausible, reproducible, and completely wrong.

Which is arguably worse than code that simply crashes.

Here are five of my favorites.

1. One Change to a Matrix Changes Every Row

Let's start with something familiar to anyone who works with tabular data.

You need a 3×3 matrix initialized to zero.

Easy:

matrix = [[0] * 3] * 3

print(matrix)

Result:

[[0, 0, 0],
 [0, 0, 0],
 [0, 0, 0]]

Perfect.

Now let's change one value.

matrix[0][0] = 99

print(matrix)

You'd expect:

[[99, 0, 0],
 [ 0, 0, 0],
 [ 0, 0, 0]]

Instead:

[[99, 0, 0],
 [99, 0, 0],
 [99, 0, 0]]

Wait. What?

You modified one cell, and an entire column changed.

The explanation is simple, but rather unpleasant.

When Python evaluates:

[[0] * 3] * 3

It doesn't create three independent inner lists.

It creates one list and repeats the reference to that list three times.

All three rows are actually the same object.

You can prove it:

print(matrix[0] is matrix[1])  # True

The fix:

matrix = [[0] * 3 for _ in range(3)]

Now each row is independent.

To be clear, this is a native Python list problem, not how NumPy array construction works. But it's exactly the kind of mistake that can creep into preprocessing code, custom numerical algorithms, or quick exploratory scripts.

The lesson: Repeating a list doesn't necessarily create independent copies. Sometimes it just gives the same object several different places to hide.

2. Your Function Remembers Things You Never Asked It to Remember

Suppose you're collecting results from an experiment.

You write a little helper:

def record_result(result, history=[]):
    history.append(result)
    return history

Looks perfectly reasonable.

Now you run two experiments:

print(record_result("Experiment A"))
print(record_result("Experiment B"))

Output:

['Experiment A']
['Experiment A', 'Experiment B']

Maybe that's what you wanted.

But what if the second experiment was supposed to have its own history?

Too bad.

Python has been maintaining a shared list behind your back.

The culprit is the default argument:

history=[]

In Python, default argument values are evaluated when the function is defined, not every time the function is called.

That list is created once and reused across calls that omit the argument.

Which means your convenient little helper has accidentally acquired state.

In a data science workflow, that could mean results from one experiment leaking into another.

The solution is the familiar None pattern:

def record_result(result, history=None):
    if history is None:
        history = []

    history.append(result)
    return history

Now every call without an explicit history gets a fresh list.

The lesson: Python's default arguments aren't necessarily defaults in the sense you expect. Mutable defaults can become accidental storage devices.

3. `True`, `1`, and `1.0` Are the Same Dictionary Key

Here's an especially entertaining one for anyone working with categorical data.

Consider:

labels = {
    True: "boolean",
    1: "integer",
    1.0: "float"
}

print(labels)

Three keys, three types.

You might reasonably expect three entries.

Instead:

{True: 'float'}

One entry.

The last value wins.

Why?

Because Python considers all three keys equal:

True == 1      # True
1 == 1.0       # True
True == 1.0    # True

And their hashes are equal, too.

Even more interesting:

isinstance(True, int)  # True

Yes, bool is actually a subclass of int in Python.

So dictionaries treat these values as equivalent keys.

The first key object remains, but its value gets overwritten twice.

This can matter when you're mixing boolean indicators with numeric category identifiers.

If your categories need to distinguish between a boolean True and the integer 1, a plain dictionary won't do it automatically.

The lesson: In Python, equality and identity are different concepts, and dictionaries care about equality and hashing, not whether two keys have different types.

4. Python Has a `while...else`. Because Apparently `if` Wasn't Enough.

Most programmers are familiar with if...else.

Some languages have try...else.

Python went full "hold my beer" and decided that loops should have an else too.

Yes, really.

while False:
    print("This will never run")
else:
    print("But this will!")

Output:

But this will!

Okay, that's a little unusual, but not completely insane.

Now try this:

x = 0

while x < 3:
    print(x)
    x += 1
else:
    print("Loop finished!")

Output:

0
1
2
Loop finished!

Wait. What?

The while condition was "true", the loop executed, and somehow the else block still ran.

Welcome to Python.

Here's the secret: a loop's else clause doesn't mean what you'd normally expect from the word "else."

It actually means something closer to "the loop completed without encountering a break."

Consider:

x = 0

while x < 3:
    if x == 1:
        break
    x += 1
else:
    print("Finished!")

This time, the else block doesn't execute.

Because the loop ended with break.

Which means the else runs when the loop terminates normally, including when it never runs at all, but not when a break interrupts it.

To be fair, there's a legitimate use case here. When searching for something in a loop, you can use break when you find it and else when the search finishes without finding anything.

But you have to admit that else is a rather creative choice of keyword.

Python's own documentation describes the behavior by comparing loop else clauses to the else in a try statement.

The lesson: In Python, else doesn't always mean "otherwise." Sometimes it means "congratulations, nobody broke out of the loop."

5. Python Objects Can Share a Secret You Never Intended Them to Share

Let's get into object-oriented programming.

Suppose you're tracking machine learning experiments. Each experiment has a name and a collection of metrics.

You write a simple class:

class Experiment:
    metrics = []

    def __init__(self, name):
        self.name = name

    def add_metric(self, metric):
        self.metrics.append(metric)

Looks reasonable enough.

Now let's create two independent experiments.

experiment_a = Experiment("Model A")
experiment_b = Experiment("Model B")

experiment_a.add_metric(0.95)

print(experiment_a.metrics)
print(experiment_b.metrics)

Output:

[0.95]
[0.95]

Excuse me?

You added a metric to Model A, and Model B somehow knows about it.

The two experiments are completely separate instances.

Different names. Different objects.

But apparently, they have a shared memory.

Here's what's happening.

In Python, attributes declared directly inside a class body are class attributes, not instance attributes.

So this:

class Experiment:
    metrics = []

Creates one list shared by every instance of Experiment, unless an instance overrides the attribute.

When Python evaluates:

self.metrics.append(metric)

It looks for metrics on the instance.

Doesn't find it.

Looks on the class.

Finds the shared list.

And modifies it.

Now here's where things get even more interesting.

Suppose you do this:

experiment_a.metrics = [0.99]

print(experiment_a.metrics)  # [0.99]
print(experiment_b.metrics)  # [0.95]

Suddenly, the experiments are independent!

Why?

Because assigning to experiment_a.metrics creates an instance attribute that shadows the class attribute.

But calling .append() on the original list modifies the shared class attribute.

So two operations that look like they're manipulating the same property can behave very differently.

The correct implementation is:

class Experiment:
    def __init__(self, name):
        self.name = name
        self.metrics = []

    def add_metric(self, metric):
        self.metrics.append(metric)

Now every experiment gets its own metrics list.

And this isn't limited to experimental metrics. The same problem can appear in cached results, configuration objects, event handlers, and any class that accidentally shares mutable state between instances.

The lesson: In Python, where you declare an attribute matters. Put mutable per-instance state in __init__, unless you genuinely want every object to share it.

Because nothing inspires confidence in your machine learning results quite like discovering that two supposedly independent experiments have been sharing their data all along.

6. Python Throws an Exception. Your Data Changes Anyway.

I saved my favorite for last.

Because this one challenges a fairly reasonable assumption:

If an operation fails, surely it didn't complete?

Meet Python.

data = ([1, 2],)

try:
    data[0] += [3]
except TypeError:
    print("Operation failed!")

print(data)

Output:

Operation failed!
([1, 2, 3],)

Let's review what just happened.

Python threw an exception.

We caught it.

And the data changed anyway.

The operation failed successfully.

Here's why.

data is a tuple containing a list.

Tuples are immutable, meaning you can't replace their elements.

But the objects inside a tuple can still be mutable.

When Python executes:

data[0] += [3]

Two things happen.

First, the list is modified in place. The value 3 is appended.

Then Python attempts to assign the result back into the tuple.

That second step fails because tuples don't support item assignment.

But the first step already happened.

And Python doesn't automatically roll it back.

So you get an exception AND modified data.

To avoid this particular problem, you can mutate the contained list explicitly when that's what you intend:

data[0].append(3)

Or use immutable contents if you need the entire structure to remain unchanged.

The truly wonderful detail is that this behavior is documented in the official Python FAQ.

It's not a bug.

It's a consequence of Python's object model and augmented assignment semantics.

The lesson: Exceptions do not imply atomicity. A failed operation can leave behind side effects, and those side effects can matter enormously in data-processing pipelines.

Beautiful Is Better Than Ugly. Usually.

Python's famous Zen includes principles like:

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.

And Python generally does an admirable job of living up to them.

But as we've seen, simple syntax doesn't always imply simple behavior.

The most dangerous programming errors aren't necessarily the ones that crash.

They're the ones that quietly give you the wrong answer.

And in data science, that wrong answer might come with a beautiful chart and a very convincing confidence interval.

This is part of an ongoing series exploring the stranger corners of familiar programming languages. We've already visited PHP and JavaScript. C# and Perl are still waiting.

Perl, naturally, is preparing something involving regex.

JavaScript: The Language That Gaslights Its Developers

Jean-Luc Martel — Fri, 30 Oct 2026 07:00:00 +0000

After writing about PHP's greatest hits in the Weird Language Hall of Fame, it felt only fair to turn to JavaScript.

Because if PHP is the language of historical baggage, JavaScript is the language of "this seems impossible, and yet it is fully specified behavior."

JavaScript has a special talent for producing results that look like bugs, feel like bugs, and absolutely are not bugs.

Most of the time, the culprit is type coercion, implicit conversion, or one of the language's many "technically reasonable, emotionally devastating" rules.

So here are a few of my favorite examples.

1. An Empty Array Is Truthy... But Also Equal to `false`

Let's begin with a classic.

Boolean([])       // true
[] == false       // true
[] === false      // false

This is the kind of thing that makes developers stare at the screen and wonder why they picked computing science instead of accounting like their mom advised.

An empty array is truthy.

So far, so normal:

if ([]) {
  console.log("Yep, truthy");
}

That runs.

But then this also happens:

[] == false   // true

Why?

Because JavaScript doesn't use the same rules for boolean contexts and loose equality.

In a boolean context, arrays are simply objects, and objects are truthy.

But with ==, JavaScript starts converting values. The empty array becomes an empty string, which then becomes the number 0. Meanwhile, false also becomes 0.

So JavaScript ends up comparing:

0 == 0

Which is, unfortunately, true.

And if you really want to ruin your afternoon:

[] == ![]   // true

Because ![] becomes false, and now you're back in the previous example.

The lesson: truthiness and loose equality are not the same thing. If you're comparing values in JavaScript, === should be your default.

2. `typeof null` Is `"object"`

This one has survived for decades.

typeof null         // "object"
typeof undefined    // "undefined"
typeof []           // "object"
typeof NaN          // "number"

Let's focus on the weirdest part:

typeof null   // "object"

Except null is not an object.

This is a long-standing historical bug in JavaScript that became untouchable because too much code depended on it.

So now it just lives there, like haunted wallpaper.

And since JavaScript enjoys layering confusion on top of confusion:

null is not an object, but typeof null says it is.
Arrays are objects, so typeof [] is also "object".
NaN means "Not a Number," but typeof NaN is "number".

You can almost hear the language whispering, "If you can't handle me on my worst days, you don't deserve me."

The lesson: typeof is useful, but not always precise. If you need to distinguish arrays, nulls, and objects, use more specific checks like Array.isArray() and explicit value === null.

3. `map(parseInt)` Produces Nonsense

This is one of the best examples in the entire language because every piece of it looks reasonable.

["10", "10", "10"].map(parseInt)

What would you expect?

[10, 10, 10]

What do you actually get?

[10, NaN, 2]

Naturally.

Here's what's happening.

map() doesn't just pass each array element to the callback. It passes three arguments:

the current value
the current index
the whole array

So this:

["10", "10", "10"].map(parseInt)

effectively becomes:

parseInt("10", 0)  // 10
parseInt("10", 1)  // NaN
parseInt("10", 2)  // 2

And why is that a problem?

Because parseInt() takes a second argument: the radix.

So JavaScript interprets the array index as the number base.

"10" in base 0 becomes 10
"10" in base 1 is invalid, so NaN
"10" in base 2 becomes 2

This is one of those perfect JavaScript moments where two completely legitimate APIs collide and create chaos.

The fix is simple:

["10", "10", "10"].map(Number)

or:

["10", "10", "10"].map(x => parseInt(x, 10))

The lesson: Don't hand a function to map() unless you know exactly what arguments it expects.

4. `sort()` Sorts Numbers Like Strings

This one has caused real bugs in real applications.

[1, 2, 10, 20, 3].sort()

What you probably expect:

[1, 2, 3, 10, 20]

What JavaScript gives you:

[1, 10, 2, 20, 3]

Because by default, sort() compares elements as strings.

So JavaScript is effectively sorting these:

"1"
"2"
"10"
"20"
"3"

And in string comparison, "10" comes before "2".

This makes perfect sense if you're sorting words.

It makes much less sense if you're sorting numbers in a programming language and expected numeric order to be the default.

The fix:

[1, 2, 10, 20, 3].sort((a, b) => a - b)

Now you get what you wanted:

[1, 2, 3, 10, 20]

The lesson: JavaScript's default behaviors are often optimized for generality, not for your expectations.

So, Is JavaScript Bad?

Not really.

Like PHP, JavaScript is often mocked for behavior that is actually consistent with its own rules.

The problem is that many of those rules are invisible until you trip over them.

An empty array can be truthy while also loosely equal to false.

typeof null can lie to your face.

A perfectly innocent-looking map(parseInt) can produce results that seem like a prank.

And sort() can rearrange numbers alphabetically, because apparently that was a reasonable default choice at some point in human history.

But that's also what makes JavaScript fascinating.

Its weirdness isn't random. It's the product of a language that was created to validate forms in web pages and became something people use to build microservices and large-scale data streaming pipelines, and evolved while the entire internet was standing on top of it.

Which is kind of impressive.

This is part of a series on programming languages that occasionally make developers question the nature of reality.

PHP already had its turn. Python, C#, and Perl are waiting patiently in the wings.

And Perl, as always, remains less a programming language than a dare.

PHP: 30 Years of Making Developers Say "Wait, What?" (Part 2: It Gets Worse)

Jean-Luc Martel — Wed, 28 Oct 2026 07:00:00 +0000

Apparently, three examples of PHP weirdness weren't enough.

In the original post, we explored how PHP considers "0" to be false, how array keys can silently overwrite one another, and how an innocent-looking foreach loop can unexpectedly modify your data.

Those were fun.

But PHP has been around for three decades, and you don't accumulate that much history without developing a few interesting personality traits.

So here are four more examples of PHP doing exactly what it was designed to do, even when that seems deeply unreasonable.

4. Filtering an Array Can Turn It Into a JSON Object

Let's say you're building an API.

You have an array of fruit. You want to remove one item and return the rest as JSON.

Simple enough.

$items = ['apple', 'banana', 'cherry'];

$filtered = array_filter(
    $items,
    fn($item) => $item !== 'banana'
);

echo json_encode($filtered);

What do you expect?

Probably this:

["apple","cherry"]

What you actually get:

{"0":"apple","2":"cherry"}

Wait. What?

You started with an array. You filtered an array. You encoded an array.

And somehow, you got a JSON object.

Here's why.

PHP's array_filter() preserves the original keys.

After filtering, the array looks like this:

[
    0 => 'apple',
    2 => 'cherry'
]

When json_encode() encounters a PHP array whose keys aren't consecutive integers starting at zero, it serializes it as a JSON object rather than a JSON array.

Your perfectly reasonable filtering operation has silently changed the shape of your API response.

Imagine the delight of the frontend developer expecting an array.

The fix:

echo json_encode(array_values($filtered));

Which gives you:

["apple","cherry"]

The lesson: If you're filtering arrays for JSON responses, remember that array_filter() preserves keys. Use array_values() when you need a proper JSON array.

5. A Variable Can Exist and Still Not Be Set

Let's talk about isset().

The name seems reasonably self-explanatory.

Surely it tells you whether something exists?

Well, not exactly.

$config = [
    'timeout' => null
];

var_dump(isset($config['timeout']));

Result:

bool(false)

But the key is right there!

You can see it. I can see it. PHP can presumably see it.

And yet isset() returns false.

That's because isset() doesn't actually mean "does this variable or key exist?"

It means something closer to "does this exist AND contain a value other than null?"

So we have two functions:

isset($config['timeout']);                // false

array_key_exists('timeout', $config);     // true

Same array. Same key. Different answers.

And there's a practical consequence.

Suppose null means "disable the timeout," while a missing key means "use the default timeout."

You might write:

$timeout = $config['timeout'] ?? 30;

But the null coalescing operator also falls back when the value is null.

So $timeout becomes 30, even though the configuration explicitly contained null.

If those two states matter, you need to distinguish them:

$timeout = array_key_exists('timeout', $config)
    ? $config['timeout']
    : 30;

Now an explicitly configured null stays null.

The lesson: In PHP, "missing" and "present but null" are different states, but isset() treats them the same. Sometimes that's convenient. Sometimes it's a bug waiting for the right configuration file.

6. Two Different Strings Can Be Equal Because They're Both Zero

Here's a fun one.

var_dump("0e123" == "0e999");

What would you expect?

They're clearly different strings.

Surely the answer is false.

bool(true)

Of course it is.

The explanation involves PHP's numeric-string handling.

Both strings are valid representations of numbers in scientific notation.

0e123 means zero multiplied by ten to the power of 123.

0e999 means zero multiplied by ten to the power of 999.

Both evaluate numerically to zero.

And because PHP's loose comparison operator (==) recognizes both strings as numeric, it compares their numeric values.

So:

"0e123" == "0e999"    // true
"0e123" === "0e999"   // false

This particular behavior has a wonderful nickname: magic hashes.

Certain hash outputs happen to consist entirely of a zero, followed by e, followed by digits.

When two such hashes are compared using loose equality, PHP can interpret them as equal numeric values even though their string contents differ.

Historically, this has contributed to security vulnerabilities in applications that compared hash strings incorrectly.

And no, switching from PHP 7 to PHP 8 doesn't eliminate this particular issue. Both operands here are valid numeric strings, so the loose comparison still considers them equal.

The fix is straightforward:

"0e123" === "0e999"  // false

For security-sensitive hash comparisons, use the appropriate cryptographic comparison functions, such as hash_equals() when comparing authentication-related strings, or password_verify() for password hashes.

The lesson: Loose equality isn't merely a convenience feature. When values originate from external input, it can become a security problem.

7. `switch` and `match` Disagree About Reality

PHP 8 introduced match, a more modern alternative to switch.

And naturally, they don't quite agree on what constitutes a match.

Consider:

$value = "10";

switch ($value) {
    case 10:
        echo "Integer ten";
        break;

    case "10":
        echo "String ten";
        break;
}

Output:

Integer ten

The value is a string.

The first case is an integer.

But PHP's switch uses loose comparisons, so "10" matches 10.

The first matching case wins.

Now let's rewrite that using match:

$value = "10";

echo match ($value) {
    10   => "Integer ten",
    "10" => "String ten",
};

Output:

String ten

Same input. Same alternatives. Different result.

Why?

Because match uses strict comparisons (===), while switch uses loose comparisons (==).

And match has some other differences, including no fall-through and a requirement to handle unmatched values or provide a default.

To be fair, match is arguably the better-designed construct.

But imagine converting an old switch statement to match during a refactoring session, assuming that you're just modernizing the syntax.

You could change application behavior without changing any of the values being compared.

The lesson: When replacing switch with match, you're not merely changing syntax. You're changing comparison semantics.

PHP Is Consistent. That's Not Always Comforting.

Here's the interesting thing about all seven examples we've now explored across these two articles.

None of them are random.

PHP has rules governing truthiness, type coercion, array keys, references, JSON serialization, and comparisons.

The interpreter follows those rules consistently.

The problem is that those rules don't always line up with what developers intuitively expect.

And sometimes two individually reasonable rules interact to produce something spectacularly unreasonable.

That's what makes PHP such a fascinating language to revisit.

It has evolved a lot since the 1990s. Modern PHP is faster, more expressive, and substantially better engineered than the language many developers remember.

But it still has a few surprises hiding behind perfectly ordinary syntax.

And after thirty years, I suspect it's earned the right to be a little eccentric.

PHP: 30 Years of Making Developers Say "Wait, What?"

Jean-Luc Martel — Sun, 25 Oct 2026 01:57:59 +0000

With PHP suddenly back in the spotlight, I figured this was a good time to revisit one of the web's most successful, most criticized, and occasionally most bewildering programming languages.

PHP has been powering websites for decades. It has evolved a lot, particularly through PHP 7 and 8, but it still carries some fascinating quirks.

And I'm not talking about obscure syntax or features nobody uses. I'm talking about perfectly ordinary code that does something completely unexpected.

Here are three of my favorites.

1. The String "0" Is False. The String "00" Is True.

Let's start with something simple.

var_dump((bool) "0");      // false
var_dump((bool) "00");     // true
var_dump((bool) "false");  // true
var_dump((bool) "");       // false

Isn't that beautiful?

The string "0" evaluates to false, but "00" evaluates to true.

Even the literal string "false" evaluates to true.

PHP considers an empty string and the specific string "0" to be falsy. Other nonempty strings, including "00", are truthy.

Now consider:

$password = "0";

if (!$password) {
    echo "No password provided!";
}

Except a password was provided. PHP just decided it didn't count.

And if you're thinking that empty() might save you:

empty("0");    // true
empty("00");   // false

Nope.

The lesson: Don't use truthiness to determine whether a string contains meaningful input. Be explicit about what you're checking.

2. PHP Arrays Can Silently Overwrite Your Data

PHP arrays aren't really conventional arrays. They're ordered maps that support both integer and string keys.

Which sounds perfectly reasonable until you do this:

$data = [
    "8"  => "eight",
    8    => "EIGHT",
    "08" => "zero eight"
];

print_r($data);

You might expect three entries.

You get two.

Array
(
    [8] => EIGHT
    [08] => zero eight
)

PHP automatically converts the string key "8" to the integer 8.

So the second entry overwrites the first.

But "08" stays a string because it isn't a valid decimal integer representation for this conversion.

And it gets better:

$data = [
    1    => "integer",
    "1"  => "string",
    true => "boolean"
];

print_r($data);

Result:

Array
(
    [1] => boolean
)

Three apparently different keys. One surviving value.

The first two collide because "1" becomes integer 1. The third collides because true also becomes integer 1.

The lesson: PHP array keys undergo implicit type conversion. If your application depends on distinguishing keys by type, you're in for an unpleasant surprise.

3. A `foreach` Loop Can Quietly Corrupt Your Array

This is my favorite, because it's the kind of bug that can survive a code review.

Consider:

$numbers = [1, 2, 3];

foreach ($numbers as &$number) {
    // Do some processing
}

foreach ($numbers as $number) {
    // Do some more processing
}

print_r($numbers);

Nothing particularly exotic here.

The first loop iterates by reference. The second iterates normally.

What does the final array contain?

Array
(
    [0] => 1
    [1] => 2
    [2] => 2
)

Wait. What happened to the 3?

The culprit is that innocent-looking ampersand in the first loop:

foreach ($numbers as &$number)

When that loop finishes, $number remains a reference to the last element of the array.

The reference doesn't magically disappear when the loop ends.

Then the second loop reuses $number.

Each iteration assigns another array value to a variable that's still referencing the last element.

By the time the second loop finishes, the last element has been overwritten with the second-to-last value.

The truly delightful part?

This behavior is explicitly documented in the PHP manual.

The fix is simple:

foreach ($numbers as &$number) {
    // Do some processing
}

unset($number); // Break the reference

But the fact that you need to remember to do this is what makes the behavior so interesting.

The lesson: References in PHP can outlive the control structures that introduced them. Reusing a variable after a by-reference loop can have side effects that are surprisingly difficult to spot.

So, Is PHP Broken?

Not really.

All three behaviors follow documented rules. They're not random, and they're not bugs in the PHP interpreter.

But they illustrate something interesting about programming languages.

A language can be internally consistent while still producing results that surprise the people using it.

PHP has spent decades balancing developer convenience, backward compatibility, and language evolution. Sometimes those priorities have produced genuinely useful features. Sometimes they've produced behavior that makes you stare at your screen for a few minutes.

And that's part of what makes exploring programming languages so entertaining.

This was fun so I think I'll make it a series looking at the stranger corners of familiar languages.

Migrating AKS workloads from AAD Pod Identity to Entra Workload ID

Jean-Luc Martel — Thu, 22 Oct 2026 02:49:22 +0000

You're looking at an AKS cluster for a client.

Somewhere in it, a pod is authenticating to Azure using AAD Pod Identity. It relies on AzureIdentity and AzureIdentityBinding custom resources. It depends on the Node Managed Identity (NMI) pods intercepting metadata requests on port 2579.

It is deprecated. Microsoft pulled the plug a year ago. The NMI intercept is a tax on the cluster's CPU and a liability in the threat model.

You need to move it to Entra Workload ID. It uses OIDC federation. It is cleaner. But cleaner does not mean easy.

Inventory the plumbing

Do not just generate new manifests and hope for the best.

Inventory what you actually have. Every AzureIdentityBinding is a dependency. Every AzureIdentity is a managed identity mapped to a specific set of pod labels.

Map each workload to its intended federated Kubernetes Service Account. If you do not know exactly what the old identity does, you will not know what permissions the new one needs.

Treat the old bindings as a map of your actual access patterns, not just a list of resources to delete.

Validate the exact strings

Entra Workload ID does not guess. It matches strings.

When you create the federated identity credential in Entra ID, the issuer is your AKS OIDC endpoint. The subject must be exactly system:serviceaccount:<namespace>:<name>.

Miss a colon. Add a trailing space. Use the wrong namespace. It will not give you a helpful error. It will just give you a 401 Unauthorized.

The pod also requires the label azure.workload.identity/use: "true". Without it, the Entra workload identity mutating webhook ignores the pod. No environment variables get injected. No service account token gets projected. The pod starts, but it cannot authenticate.

Test the token exchange before you move production traffic.

Exec into a debug pod. Request the token from the projected volume. Decode it. Check the aud (audience) and sub (subject) claims. Verify the exact string matches what you configured in Entra ID.

Stage the migration

Do not do a cutover. Do a migration.

Deploy the new service account and the federated identity credential. Update the workload definition to use the new service account and apply the required pod label.

Watch the logs. Verify the token exchange succeeds.

Once the new identity is serving traffic and passing health checks, kill the old AzureIdentityBinding and AzureIdentity resources.

Do not leave the old plumbing in the cluster "just in case." Deprecated resources left behind become permanent attack surface. If you do not need it, delete it.

Prove it works

You are not done when the pod starts. You are done when you prove it can do its job.

Check the Azure Activity Log. Look for the API calls.

See the new managed identity making the requests. See the old one go silent.

That is the only evidence that matters. The new identity reaches the required services with the intended permissions, and the old plumbing is gone.

The Green Padlock That Almost Cost Me a Contract

Jean-Luc Martel — Mon, 19 Oct 2026 20:15:12 +0000

Encryption is supposed to answer one question for the person sitting at the browser: am I really talking to who I think I'm talking to? On a corporate network running TLS inspection, the honest answer is often "not necessarily" — and most of the people relying on that network have no idea.

There are serious risks if the setup isn't done properly, but raising that flag can lead to uncomfortable discussions. I found that out the hard way, when demonstrating the problem nearly got my contract terminated.

The setup

The client ran a TLS inspection appliance on their internal network — the kind that decrypts outbound HTTPS so it can be scanned for malware and data loss before being re-encrypted and sent on. These devices are common and, in principle, defensible: most malware command-and-control and data exfiltration now hides inside TLS, so without inspection, a lot of security tooling is blind.

The way these appliances work is by man-in-the-middling every HTTPS connection. The appliance holds its own certificate authority. When you visit a site, it generates a certificate for that site on the fly, signs it with its own CA, and presents it to your browser. For this to work without warnings, that CA's root certificate is installed in the trust store of every managed machine — usually pushed automatically through group policy or device management.

Once that root is trusted, the browser stops complaining. The certificates the appliance generates look completely valid. No warning. No broken padlock. Nothing to tell the user their "encrypted" session is being read in clear text in the middle.

On top of that, the certificate issuance on this network was far more permissive than it should have been. The practical effect was that the internal CA could produce a trusted certificate for essentially any domain name — including public ones like google.com.

The problem I wanted to show

The thing I wanted the security team to see is subtle, and it's the part people consistently underestimate: once a root is trusted, the browser's special protections for high-value sites quietly stop applying.

Browsers do a lot to protect domains like google.com:

Certificate pinning ties those domains to a specific set of public CAs.
Certificate Transparency requires publicly trusted certificates to be logged in public, auditable logs.
HSTS forces connections to use HTTPS.

Every one of these makes an exception for locally installed roots. Pinning is skipped when the chain ends at an admin-installed CA. CT isn't required for private roots. HSTS is satisfied by any cert the browser considers valid. These exceptions exist on purpose — they're what stops a legitimate inspection appliance from breaking Google for the whole company. But they also mean that anyone who can issue from the trusted internal CA inherits all of that stand-aside treatment.

In other words: the protections you'd assume would catch an impersonated Google simply don't fire inside that environment.

The demonstration

To make this concrete rather than theoretical, I built a tiny proof of concept on my own machine:

I created a web page that said, in plain text, "I'm a fake Google page." No login form, no attempt to harvest anything — the content existed only to make the point unmistakable.
I added an entry to my own machine's hosts file pointing google.com at the IP serving that page.
I generated a certificate for google.com from the environment's trusted CA.
I loaded google.com in the browser.

The page loaded as google.com, with a green padlock, no warnings, and a certificate the browser reported as valid.

That was the whole point. The browser was not malfunctioning. It was doing exactly what the trust configuration told it to do. Every safeguard that should have flagged an impersonated Google had been designed to stand aside for a locally trusted root, and that's exactly what happened.

The hosts file was the deliberately contained part. It only affected my own machine — nobody else could reach that page. In a real attack, the hosts file stands in for an attacker's network position: a poisoned entry on the internal DNS resolver, ARP spoofing on a flat LAN, a rogue DHCP server, or a tampered proxy configuration. On a lot of internal networks, getting into that position is a lower bar than people like to assume. The demo swapped that step out for a one-line, single-machine change so there was no collateral effect at all.

The reaction

I sent the example to the security team as part of a technical debate about the risks of their setup. Their response was not to engage with the finding. It was to try to have my contract terminated, on the grounds that I had been "phishing."

I had not shared the fake page with anyone. There was no victim, no credential harvesting, no email, no deception of a single other person. The entire thing lived on one machine — mine — and demonstrated a property of their infrastructure. Calling it phishing was a category error, and I think a revealing one.

Why the reaction matters more than the bug

Phishing is the use of deception to trick a person into giving something up. A proof of concept that impersonates a site to no one but yourself, to demonstrate a flaw in a system you were engaged to assess, is the opposite of that: it's disclosure. The deception, such as it was, was aimed at a browser I controlled, to reveal something true about the environment.

What the reaction demonstrated is a failure mode I've seen more than once: when a finding is uncomfortable, it's easier to attack the method than to sit with the conclusion. The conclusion here was genuinely uncomfortable. It wasn't "the appliance has a bug." It was "the way trust is configured on this network means the guarantees your users assume they have — that a green padlock on google.com means they're talking to Google — don't hold, and an insider with issuance rights can impersonate anything to anyone, invisibly."

That's not a comfortable thing to hear about a system you own and defend. But shooting the person who showed it to you doesn't make it less true. It just guarantees the next person won't tell you.

What should have happened

The appropriate response to that demo was a conversation about controls, not contracts:

Keep the interception CA separate from the internal issuing CA, so a compromise of one doesn't hand over the other.
Apply name constraints to internal roots so they can only sign the domains they're supposed to.
Lock certificate profiles so requested names have to match the requester's identity, instead of allowing arbitrary domains.
Restrict and audit issuance — know who can request certificates and log everything that gets signed.
Protect the CA key properly, ideally in an HSM, because whoever holds it can impersonate anything to every managed device.
Be honest with users that inspection is happening, and keep sensitive categories like banking and health out of scope.

None of that is exotic. It's the standard hygiene for running TLS inspection without turning it into a single, silent point of total compromise.

The takeaway

The green padlock is a promise, and TLS inspection quietly rewrites the terms of that promise for everyone on the network. That can be done responsibly. But the first step is being willing to look at what your own configuration actually allows — especially when someone hands you an uncomfortable demonstration of it on a single contained machine, and asks you to think about it.

Attacking the messenger is a choice. It's just not a security control.

BigQuery BI Engine deserves a benchmark before you dismiss Looker Studio as slow

Jean-Luc Martel — Sun, 18 Oct 2026 07:00:00 +0000

Someone is complaining that Looker Studio is slow.

A dashboard takes forty seconds to load.

The immediate reaction is to blame the tool.

"Looker is too slow."
"We need to migrate."
"We need a custom frontend."

Look at the query logs.

The dashboard is running a full table scan on a 500GB fact table every time a user changes a date filter.

Looker Studio isn't the bottleneck. The query path is.

Blaming the visualization layer for a warehouse execution problem is like blaming the paint for a crumbling foundation.

Before you throw money at BigQuery BI Engine, isolate the variables.

Performance hypotheses usually mash three distinct things together.

First, connector caching. Is the dashboard just failing to cache the extract?

Second, query shape. Are you running a distinct count on a high-cardinality string?

Third, BI Engine support. Does the query use supported data types and operations?

BI Engine can't fix a query it doesn't support.

If your query falls back to the main BigQuery engine, the BI Engine reservation sits idle while you pay for it.

Don't guess. Benchmark.

Take the five most-used, slowest queries from the dashboard.

Run them without BI Engine. Record the latency. Record the bytes billed.

Turn on a BI Engine reservation. Run the exact same queries. Record the latency. Record the bytes billed.

Calculate the cost of the reservation against the savings in bytes billed and the value of the latency reduction.

If the query isn't supported, latency won't drop.

If the query is just badly shaped, latency might drop, but the cost won't justify the reservation.

A reservation is a blunt instrument. It solves a specific class of problem and ignores everything else.

Before you provision one, ask yourself: which exact metric will prove the bottleneck is in the compute layer, and not in the query design?

If you can't answer that, you haven't fixed a performance problem.

You've just bought a faster way to run a bad query.

Azure subscription Owner is an access-delegation privilege

Jean-Luc Martel — Tue, 13 Oct 2026 07:00:00 +0000

You hand someone the Owner role on an Azure subscription.

You think you're giving them the keys to manage resources. VMs, databases, networking. You think you've made them a builder.

You look at the role name. "Owner." It sounds like they just own the stuff inside the box.

They don't. They own the box. And they own the locks.

It’s not tenant-wide administration. A subscription Owner isn't a Global Administrator. They can't rewrite the rules for the entire Azure tenant. They can't even remove protected deny assignments if someone locked a resource down properly.

But they don't need to.

Because within the boundary of that subscription, Owner is an access-delegation privilege.

They can assign powerful data-access roles. They can hand out the keys to the data plane. They can change legacy Key Vault access policies. They can grant someone else the right to read the secrets, even if they themselves were never explicitly granted that data access.

The control plane becomes a backdoor to the data plane.

We treat subscription management and data access as two different conversations.

They aren't.

If you want to protect your data, you need to look at who holds the Owner role. Review who can assign those powerful data-access roles. Review who can quietly flip a legacy Key Vault access policy to let themselves in.

When that Owner grant was approved, what was the justification? "Needs to manage the dev environment." "Needs to deploy the infrastructure."

But was the grant approved with those delegation powers in view?

If not, you haven't granted resource administration.

You have granted data access. And you didn't even realize it.

The data engineering reality of GNNs in production

Jean-Luc Martel — Thu, 08 Oct 2026 20:32:16 +0000

Your offline GNN looks perfect. The AUC is high. The precision is great.

You ship it to production to catch fraud or optimize a supply chain.

Then the metrics tank.

The model isn't broken. The data is just too old.

It is making real-time decisions using neighborhoods and embeddings from yesterday, or last week. In fraud, a node's connections change in milliseconds. In supply chain, they change in hours.

The model is evaluating a ghost.

GNNs don't just need features. They need fresh ones. They need neighborhoods that reflect the current state of the world.

This means your graph storage and feature-update pipelines cannot be an afterthought. They must be engineered to the exact freshness the use case demands.

A batch job running at midnight is fine for predicting customer churn. It is catastrophic for blocking a fraudulent transaction at checkout.

Data engineering must run on the same clock as the business decision.

Don't trust your architecture diagram. Trust a trace.

Pick one production decision and trace it backward. Start at the served model input.

Look at the embedding it consumed. Look at the neighborhood it aggregated.

Trace those back to the graph storage layer. Trace that back to the feature-update pipeline. Trace that back to the source system change.

If a user changes their email, or a supplier misses a shipment, how long does it take for that reality to reach the model?

If you can't measure the lag, you can't trust the prediction.

Offline evaluation assumes the model has access to the exact state of the graph at the moment of the event. Production rarely offers that luxury.

A zero-disruption PodDisruptionBudget can block an AKS upgrade

Jean-Luc Martel — Thu, 08 Oct 2026 04:06:50 +0000

Your AKS cluster upgrade is stuck.

The control plane is waiting for a node to drain. The node is waiting for a pod to terminate. The pod is refusing to move.

Look at the logs. The workload has a PodDisruptionBudget that allows exactly zero disruptions.

It is doing exactly what it was told to do. Protect the application at all costs. Even if it means halting the cluster upgrade.

To upgrade a node, AKS has to drain it. It cordons the node and evicts the pods.

But the eviction process respects disruption budgets.

If your deployment has three replicas, and your budget dictates that three must remain available, the eviction fails. The node stays tainted. The upgrade stalls.

You didn't just configure high availability. You configured a deadlock.

Replica headroom isn't just a best practice for scaling. It is the physical space required for the control plane to do maintenance.

Before your next maintenance window, look at your math.

Check your minAvailable or maxUnavailable settings against the actual number of running replicas.

If you use minAvailable with a percentage, remember that Kubernetes rounds up. A setting of 100% means zero disruptions allowed. A setting of 90% with three replicas rounds up to three.

You have to leave room for the scheduler to breathe. Configure these values deliberately, not just to pass a compliance check.

High availability means surviving a failure. It shouldn't mean blocking your own operations team.

Your availability policy guarantees the app survives a zone failure. But has anyone tested whether it survives routine maintenance?

If your zero-disruption configuration blocks your upgrade pipeline, you haven't built a resilient system.

You have built a hostage situation.

I rebuilt four dead systems with AI. The failures all landed in the same place.

Jean-Luc Martel — Thu, 24 Sep 2026 16:55:00 +0000

Four reconstructions, four kinds of missing documentation, and the defects landed in the same place every time.

In HAL/S it was the characters the scanner destroyed. In the Nautilus simulator it was the parameters Verne never wrote down. In Wikicarta it was the desktop window no headless harness can open. In the compressor it was the part of the input space the corpus never sampled.

Different projects, different languages, different failure symptoms. One shape: the defects clustered wherever no automated check could reach, and in every case the code looked fine. Tests green, types satisfied, output plausible.

What was actually being varied

These weren't four projects that happened to be about old software. They were one experiment with a controlled variable: how degraded is the source of truth the model works from, and can it be caught being wrong?

arm	source of truth	oracle	outcome
HAL/S	degraded formal spec + preserved code	strong, independent	faithful reconstruction; 12/12 preserved programs run
Nautilus	a novel	none — one was built	5 findings; the legend fails on every axis
Wikicarta	memory of a product	none — human judgement	shippable app; every defect at the runtime seam
compressor	behaviour only, key sealed	strong, opened after freeze	decoder 100%, encoder 8.3%

The intuition going in was that performance would degrade smoothly as the source got worse. It didn't. Something else predicted the results, and it cut across all four.

Determined beats hard

The reconstruction quality tracked whether the available evidence determines the answer — not how difficult the task was.

The compressor makes this measurable, because both halves ran under identical conditions. The decoder recovered the bitstream format at 100%, because a format is one-way determined: if you want to decode the stream, exactly one answer works. The encoder scored 8.3% on the same format, because its heuristics — match-finder strategy, tie-breaking, greedy versus lazy — are choices that leave no trace in the decompressed bytes. Same model family, same oracle, same week. A 12× gap, entirely explained by whether the evidence pinned the answer.

The other three say the same thing in their own terms:

HAL/S. The 485 grammar productions were determined by the page images, and the reconstruction got them. The characters the scan physically destroyed were determined by nothing, and no amount of model capability recovers them. That limit is information-theoretic. Better OCR narrows it and never closes it.
Nautilus. The physics is determined; the equations produce the same crush depth for anyone. The parameters Verne omitted are determined by nothing, so submerged endurance ranges across 6 to 211 hours — a 35× spread — and battery capacity across 100×. Those spreads aren't noise around a finding. They are the finding.
Wikicarta. Parsing, backoff, cache naming and database ordering are determined by their inputs, and compiled correct on the first or second attempt, consistently. Whether the category wheel feels like Encarta is determined by nothing you can write down, and stayed a human call from start to finish.

The practical version: before handing a legacy task to a model, ask whether the evidence you have actually pins the answer. If it does, expect the model to find it and build a cheap mechanical check to confirm. If it doesn't, expect a confident, plausible, unmarked invention — and note that the model will not tell you which mode it's in, because from the inside they feel identical.

A strong oracle is not the protection you think

The compressor was built to demonstrate that a strong oracle masks confabulation. It mostly refuted that.

The reconstruction's central error — assigning Huffman code lengths by symbol index rather than in heap-extraction order — failed the byte-match test loudly. The oracle caught it. What the oracle did hide were two mechanisms the corpus never triggered: a block-splitting path that needs inputs larger than 8 KiB to fire, and a match-finder chain cap that needs input more self-similar than anything tested.

So the masking risk is not a property of the oracle's strength. It's a property of the input space you sampled. A strong checker over a narrow corpus hides precisely the mechanisms your corpus never exercises, and hides them silently, because everything it can see is green.

Strengthening the checker does nothing for this. Widening the corpus does.

That generalises past compression. It's the same failure as a test suite with excellent coverage of the paths anyone thought to write a test for.

What the human actually contributed

I expected to spend supervision effort reviewing code. Across all four projects, almost none of it went there. The load-bearing human contributions were:

HAL/S — deciding not to vote between OCR engines, and then reading 485 grammar productions against page images by eye.
Nautilus — signing the invariants and making them append-only, so the model could patch the physics but was structurally unable to weaken the checks that judge it.
Wikicarta — being the runtime. Clicking the button in the real window, on the real WebKit build, where the blocking dialog freeze and the cross-realm instanceof failure lived.
compressor — holding the seal. Not reading the answer key until the reconstruction was frozen at a tag.

Two categories, and neither is code review. One is supplying what no code can check — an eye on a page image, a hand on a mouse. The other is designing the checks that are possible and then putting them structurally out of the model's reach.

The OCR decision is the sharpest of the four, because it's a judgement call that separates a good result from a worthless one that looks identical. Running three OCR engines and taking the majority is the obvious move and it is wrong: the errors are correlated, so a vote launders error rather than cancelling it, and no fusion can recover a character that no engine produced. Survivable in prose, fatal in a grammar where the metacharacters are the content. Take the vote and you get a clean, plausible, silently corrupt grammar — and every downstream artifact inherits it.

The Nautilus invariant signing is the most reusable. Twelve invariants live in version control with a manifest checksum only a human re-signs. When the fuzzer finds a violation, the repair workflow requires patching the core and forbids touching the invariant. An agent optimising against a check it can also edit is not being checked.

Build the oracle before the artifact

Ranked by how much verification apparatus existed before the real work started, the four projects come out in the same order as how much I trust their conclusions.

Nautilus had determinism, golden-stream replay and malformed-input fuzzing green in its check harness before a single equation was written, plus a frozen protocol and a rule that every spec parameter traces to either a sourced claim or a documented gap range. Its numbers are the most defensible in the series. The compressor's seal, cold-recall protocol, freeze tag and grading harness are most of that repository; the reconstruction is the small part. HAL/S inherited an independent interpreter, which is a luxury and which is why it is the easiest arm.

Wikicarta had none of this available, and it shows — its conclusions are qualitative, argued from patterns across features rather than measured. They're still the most immediately useful findings in the series, and they're the least provable. Both things are true because of the same missing oracle.

There's a specific trap worth naming. A verification harness that finds nothing is indistinguishable from a broken one. Nautilus's 24-hour soak ran 879,643 ticks and found zero invariant violations, which is only meaningful because the harness was first validated against a deliberately poisoned invariant — which it caught, minimised from seven commands to two, and packaged. Test the test.

Three results that went the wrong way

A series that only reports confirmations is a demo reel. These four produced three clean refutations, and they were the most informative outputs:

The compressor predicted the classic overfitting curve — strong on trained data, weak on held-out. It came out inverted, 8% trained against 71% held-out, because the split tracked difficulty rather than seen-versus-unseen.
It also predicted the oracle would mask confabulation. The oracle caught the main error; a narrow corpus hid the rest.
Nautilus's fuzzer was built to find physics exploits. It found none. The only finding in the whole loop was a flaw in the experiment design — the genetic algorithm had been optimising survival rather than performance, because the controller pinned its own objectives.

Each of those changed the conclusion. None would have surfaced from reasoning about the problem.

Where this doesn't reach

Four projects, one operator, no control group. There is no human-only reconstruction to compare against, so nothing here says AI was faster or better than the alternative — only what it did and did not manage. The projects were chosen partly for being interesting, which is a selection effect. Model versions moved during the work. Three of the four are Rust, which is a small language sample.

The one number I'd most like and don't have is a like-for-like human baseline on the same corpus. That would need someone to transcribe 485 productions and reconstruct an encoder without a model, which is a real cost for a comparison nobody has funded yet.

What I'd tell someone starting

Handing legacy code to a model works, and it works well enough that the interesting question is where to spend your own attention. Four projects say:

Not on the code. The logic was right nearly every time. Spend it at the host, runtime and OS boundary, where every real defect in the desktop project lived and where no harness reached.

On knowing which mode you're in. Where the evidence determines the answer, expect success and build the cheap mechanical check — production numbering, reference integrity, a verbatim grounding gate. Where it doesn't, expect confident invention, and force it into the open: a divergence log, a gap inventory with plausible ranges, an ambiguity recorded rather than resolved inline. Every one of these projects has a file whose entire job is holding the things that were not derivable, and those files are what make the rest credible.

On the checks the model can't edit. Sign them, hash them, make weakening them require a human.

On widening the input space rather than tightening the oracle. The gaps that survive strong testing are exactly the ones your corpus never triggers.

All four repositories are public, with the divergence logs, gap inventories, transcription artifacts and grading harnesses included — those are the parts worth reading, more than the reconstructions themselves.

space-shuttle-sdk · nautilus-sim · wikicarta · compressor

The Decoder Scored 100%. The Encoder Scored 8.3%.

Jean-Luc Martel — Tue, 22 Sep 2026 13:08:00 +0000

This is the last of four experiments in AI-assisted reconstruction of legacy systems. The first three degraded the source of truth — a scanned spec, then a novel, then a memory of a product. This one removes it completely.

The target is LHA's -lh5- method — Haruyasu Yoshizaki's 1988 archiver, LZSS with an 8 KB window plus static Huffman.

In the experiment, there is no spec and no source, only behaviour, plus an oracle that will answer any question you ask it about outputs. It is the only configuration where the two questions that actually matter become measurable rather than assertable:

Behavioural match is not understanding. You can pass every test and hold a wrong model of the mechanism, with untriggered modes waiting in the part of the input space you never sampled.
A strong oracle masks confabulation. The model fits observed outputs while inventing a plausible-but-wrong internal mechanism, and because the tests keep going green, the error is rewarded rather than exposed.

Measuring those requires reconstructing blind against an answer key you are not allowed to look at. Everything else in the design follows from that one constraint.

Choosing something the model hasn't memorised

WinRAR was the instinct and is the wrong specimen. RAR and DEFLATE are heavily memorised, so you'd measure recall rather than reconstruction. RAR5 is far too large. Its decompressor is documented while its compressor has no public ground truth, which is exactly the half you need to grade. And it is live commercial software rather than abandonware.

LHA satisfies four properties that a lot of candidates fail:

The original runs, so the oracle is unlimited — compress or decompress anything on demand.
The algorithm is public, so a real answer key exists and the confabulation can be graded rather than merely suspected. A never-solved black box demonstrates that the pitfall exists but can never put a number on it.
It's bounded — a few hundred lines, reconstructable.
The encoder heuristics are underdetermined by the output. Match-finder strategy, minimum match length, tie-breaking, greedy versus lazy matching, block splitting are all encoder-side choices that leave no trace in the decompressed bytes. That is the confabulation sweet spot.

The general LZSS-plus-Huffman scaffold is in every model's training data. LHA's specific choices are not, and those specifics are what gets measured.

The controls

Three, and they're the reason the result is a measurement instead of an anecdote.

The seal. The original source sits in the repo, git-ignored, hashed. At unseal, sha256sum -c verified 107 of 107 files intact, and the manifest's own digest matched the value pinned in the sealing document. Reading it before the reconstruction was frozen is the single act that would have destroyed the experiment, so it was made mechanically detectable.

The freeze. The reconstruction was tagged and committed before unsealing, so no artifact could be quietly edited to fit the answer once the answer was visible.

The cold-recall file. Before any reconstruction began, a model wrote down everything it already believed about LHA, committed. Without it there is no way to distinguish "derived from behaviour" from "remembered from training," and the whole exercise collapses into a memory test.

Three models did the work, deliberately split: Gemini 3.1 Pro reconstructed the decoder, Codex/GPT-5 the encoder and the cold-recall baseline, Claude ran the design thread.

The decoder: 100%, and it doesn't count for much

Nineteen of nineteen round-trips, exact. The format was fully recovered.

The sharpest single result lives here. LHA's generic macro defines PBIT 5, so anyone reading the source would guess a 5-bit position-table field width. The -lh5- method actually uses pbit = 4 with np = 14. The reconstruction recovered the method-specific 4 purely from round-trip behaviour, against a constant that would have misled a source-reader.

But checking that win against the cold-recall file deflates most of the rest. Nearly every format fact the decoder "recovered" was already in the model's committed prior: the 8 KB window, the threshold of 3 and max match of 256, the roughly 510-symbol alphabet, the 19-entry precode with its 0/1/2 zero-run encoding, the 16-bit block count, the position-slot decode. The 100% credits prior knowledge, not derivation.

Which is unsurprising, because the format is one-way determined. If you want to decode the bitstream, there is exactly one answer, and both memory and derivation converge on it.

The encoder: 8.3%, and it counts for everything

Byte-match against real lha, on a fresh-seed corpus the reconstruction had never seen:

input class	trained on?	byte-match
text	yes	0/3 (0%)
source	yes	0/3 (0%)
structured	yes	0/3 (0%)
repetitive	yes	1/3 (33%)
random	no	2/2 (100%)
edges	no	3/5 (60%)
trained total		1/12 (8.3%)
held-out total		5/7 (71.4%)

A perfect decoder coexisting with an 8% encoder is match ≠ understanding in one line. The format was recoverable. The encoder's choices were not.

The hypothesis was wrong, in a useful direction

The plan predicted the classic overfitting shape: high scores on trained classes, dropping on held-out ones. The data inverted it. Held-out scored 71%, trained scored 8%.

The split tracks difficulty, not seen versus unseen. The trained classes are exactly the ones that exercise real -lh5- compression; the held-out set is dominated by incompressible and trivial inputs that fall through to stored mode or hit trivial paths. Random data scores 100% because no compression heuristics ever run.

So the honest statement isn't "an 8% to 71% generalisation gap." It's this: byte-identity is reached only where the encoder heuristics never fire. Wherever genuine compression happens, the reconstruction diverges.

The rates were identical on the trained-seed and fresh-seed corpora, so the divergence is systematic rather than instance overfitting.

The confabulation catalogue

Every inferred encoder decision, graded against the now-visible source:

decision	inferred	actual	verdict
longest-match search	yes	matches	correct
tie-break among equal lengths	nearest offset	chain walked recent→old, updates only on strictly longer	derived correct
greedy vs lazy	one-step lazy	emits literal if next match is strictly longer	derived correct
block splitting	"one block; unknown for larger"	flush at 32 KB buffer threshold	never triggered — correctly scoped
match-finder chain cap	not modelled	a hard limit makes the finder non-exhaustive	wrong, and it passed anyway
Huffman code-length assignment	canonical, tie order unmatched	lengths assigned in heap-extraction order	visible divergence

Two of those are clean wins. Nearest-offset tie-breaking and one-step lazy matching are both marked as unknown in the cold-recall file — "do not know whether lha -lh5- does this," "not a specific memory." Derived from black-box behaviour, with documented prior ignorance.

There's a caveat that inverts on inspection. Codex wrote both the recall file and the encoder, which normally weakens an independence claim. Here it strengthens it: a model cannot conceal prior knowledge from its own committed statement of ignorance. Deriving both heuristics after recording that it didn't know them is the cleanest evidence in the run.

The failure is concentrated in one place. LHA builds a frequency heap and assigns code lengths in the order symbols leave the heap, governed by the exact comparison semantics of its sift-down — not by symbol value. Textbook canonical Huffman breaks ties by symbol index, which produces different lengths for equal-frequency symbols, which cascades through the entire bitstream. The reconstruction localised the problem correctly and even tried a heap hypothesis, but never replicated the precise sift order.

The second hypothesis was also wrong, and that's the finding

The project was built to show that a strong oracle masks confabulation. It mostly didn't.

The core error — the Huffman ordering — is a visible divergence. It fails the byte test loudly. The oracle caught it, exactly as an oracle should. Only two gaps were genuinely hidden: the never-triggered block split, and the unmodelled chain cap, which is latent because the corpus never produced input self-similar enough to expose it.

Both are artifacts of a corpus that tops out at 8 KiB.

So the masking risk isn't a property of the oracle's strength. It's a property of the input space you sampled. A strong oracle over a narrow corpus hides exactly the mechanisms your corpus never triggers, and it hides them silently, because everything it can see is green. Making the oracle stronger doesn't help. Making the corpus wider does.

That reframing is more useful than the result I expected to get, and I would not have got it by reasoning about it. It needed the sealed envelope.

What it cost to know any of this

Final tally across the encoder decisions: six recalled-correct, five derived-correct, one derived-wrong-but-passed, two never-triggered, one visible divergence.

The infrastructure to produce those thirteen data points — the pinned oracle, the deterministic corpus generator, the seal and its manifest, the cold-recall protocol, the freeze tag, the grading harness — is most of the repository. The reconstruction itself is the small part.

That ratio is the practical lesson. If you want to know whether an AI-assisted reconstruction actually understood the system, rather than whether it passed, you have to build the apparatus that can tell you, and you have to build it before you start. Afterwards, every artifact is contaminated by the answer.

One thing remains open. A fully independent contamination estimate needs a repeat where the recall file and the encoder come from different model families, closing the theoretical shared-prior channel that the current caveat only argues around.

Next, the wrap-up: what four projects across four degrees of source degradation say about where the current generation of models actually fails, and why the failures kept landing in the same place.

Code: https://github.com/singular-state/compressor. LHA was written by Haruyasu Yoshizaki in 1988; the original source is used here only as a private grading key, held out during reconstruction and consulted once. The reconstruction is clean-room, so the measured gap between "passes the oracle" and "matches the source" is legitimate.

DEV Community: Jean-Luc Martel

Python: while...else and Other Crimes Against Intuition

1. One Change to a Matrix Changes Every Row

2. Your Function Remembers Things You Never Asked It to Remember

3. True, 1, and 1.0 Are the Same Dictionary Key

4. Python Has a while...else. Because Apparently if Wasn't Enough.

5. Python Objects Can Share a Secret You Never Intended Them to Share

6. Python Throws an Exception. Your Data Changes Anyway.

Beautiful Is Better Than Ugly. Usually.

JavaScript: The Language That Gaslights Its Developers

1. An Empty Array Is Truthy... But Also Equal to false

2. typeof null Is "object"

3. map(parseInt) Produces Nonsense

4. sort() Sorts Numbers Like Strings

So, Is JavaScript Bad?

PHP: 30 Years of Making Developers Say "Wait, What?" (Part 2: It Gets Worse)

4. Filtering an Array Can Turn It Into a JSON Object

5. A Variable Can Exist and Still Not Be Set

6. Two Different Strings Can Be Equal Because They're Both Zero

7. switch and match Disagree About Reality

PHP Is Consistent. That's Not Always Comforting.

PHP: 30 Years of Making Developers Say "Wait, What?"

1. The String "0" Is False. The String "00" Is True.

2. PHP Arrays Can Silently Overwrite Your Data

3. A foreach Loop Can Quietly Corrupt Your Array

So, Is PHP Broken?

Migrating AKS workloads from AAD Pod Identity to Entra Workload ID

Inventory the plumbing

Validate the exact strings

Stage the migration

Prove it works

The Green Padlock That Almost Cost Me a Contract

The setup

The problem I wanted to show

The demonstration

The reaction

Why the reaction matters more than the bug

What should have happened

The takeaway

BigQuery BI Engine deserves a benchmark before you dismiss Looker Studio as slow

Azure subscription Owner is an access-delegation privilege

The data engineering reality of GNNs in production

A zero-disruption PodDisruptionBudget can block an AKS upgrade

I rebuilt four dead systems with AI. The failures all landed in the same place.

What was actually being varied

Determined beats hard

A strong oracle is not the protection you think

What the human actually contributed

Build the oracle before the artifact

Three results that went the wrong way

Where this doesn't reach

What I'd tell someone starting

The Decoder Scored 100%. The Encoder Scored 8.3%.

Choosing something the model hasn't memorised

The controls

The decoder: 100%, and it doesn't count for much

The encoder: 8.3%, and it counts for everything

The hypothesis was wrong, in a useful direction

The confabulation catalogue

The second hypothesis was also wrong, and that's the finding

What it cost to know any of this

3. `True`, `1`, and `1.0` Are the Same Dictionary Key

4. Python Has a `while...else`. Because Apparently `if` Wasn't Enough.

1. An Empty Array Is Truthy... But Also Equal to `false`

2. `typeof null` Is `"object"`

3. `map(parseInt)` Produces Nonsense

4. `sort()` Sorts Numbers Like Strings

7. `switch` and `match` Disagree About Reality

3. A `foreach` Loop Can Quietly Corrupt Your Array