DEV Community: math4mad

The Geometry of Shadows: What a Bridge Railing Taught Me About the Mathematics of LLMs

math4mad — Sun, 18 Oct 2026 04:54:41 +0000

While running one morning, I found myself staring at the shadows cast by a stone bridge railing. The railing had a series of identical, carved镂空 (hollowed-out) structures. As I ran past, the angle of the sun shifted, and the shadows on the ground transformed.

At one moment, the shadow looked like a swimming fish; a few steps later, it looked like a rolling wave; later still, it was just an abstract, elongated blob.

It struck me that I was witnessing a perfect physical metaphor for how human language—and Large Language Models (LLMs)—generate meaning.

The stone bridge hole is the concept. The sunlight is the context. The shadow on the ground is the utterance.

This realization perfectly mirrors the "Central Problem" of language modeling, beautifully stripped of its usual jargon in Joseph L. Breeden’s paper, The Simple Mathematics of Large Language Models.

The Problem of the Single Shadow

In his paper, Breeden highlights a fundamental flaw in early AI approaches: treating words as static, unchanging symbols. He uses the classic example of the word "bank":

"I deposited money at the **bank." (a financial institution)
"We walked along the river **bank." (the land beside a river)
"The plane **banked* sharply left."* (tilted)

If we assign a single, fixed vector to the word "bank" in a high-dimensional space, we are essentially trying to capture all three meanings in one static shape. But as Breeden points out, a fixed vector for "bank" cannot simultaneously be close to "money," "river," and "tilt."

In my running metaphor, a single, fixed vector is like looking at the bridge hole only at exactly high noon. You get one specific shadow. But language is dynamic. The meaning of "bank" depends entirely on the "angle of the light" provided by the surrounding words.

The Mathematics of Light: Learned Weighted Averaging

How does an LLM solve this? Breeden explains that the core mechanism of a Transformer is not magic; it is Learned Weighted Averaging.

Let’s map Breeden’s mathematics to the bridge railing:

1. The Bridge Hole (The Token Embedding $e$
Before any context is applied, the word "bank" is just an initial embedding vector ($e$). It is the physical stone structure itself. It contains the potential for all its meanings, but it is currently unshaped by the environment.

2. The Angle of the Sunlight (Influence Scores,$a_i \cdot b_j$)
To determine what the shadow will look like, we need light. In an LLM, the "light" comes from the surrounding words (the context).
Breeden explains that the model computes an influence score between the target word and its context. It does this by projecting the vectors through learned matrices ($W^A$ and $W^B$).

When the context is "deposited money," the "light" hits the bridge hole from a financial angle.
When the context is "river," the light hits it from a geographical angle. The dot product ($a_i \cdot b_j$) measures how perfectly the angle of the light aligns with the shape of the hole.

3. The Shadow on the Ground (Context-Dependent Representation, $h_i$)
Finally, the light passes through the hole and casts a shadow. Mathematically, the model takes a weighted average of the context vectors (using the content projection $W^C$), resulting in a brand new vector ($h_i$).
This new vector is the shadow. It is a context-dependent representation. The "bank" in the first sentence is mathematically pulled toward the geometry of finance; the "bank" in the second sentence is pulled toward the geometry of nature. They are the same material, but entirely different products.

Why This Metaphor Matters

What makes Breeden’s explanation so profound is that it removes the anthropomorphic jargon of "Queries," "Keys," and "Values." There is no little homunculus inside the computer "looking up" information in a database.

There is only geometry and light.

When an LLM reads the sentence "The plane banked sharply left," it doesn't "know" what a plane is. Instead, the high-dimensional vectors for "plane" and "sharply left" act as a specific angle of light. This light passes through the "bank" bridge hole and casts a shadow that aligns perfectly with the geometric region of "physical motion" in the model's latent space.

Furthermore, this explains the difference between languages. In English, "bank" is a single, multi-purpose bridge hole that requires complex, dynamic lighting (Attention mechanisms) to cast the right shadow. In Chinese, the language has already carved three separate, distinct bridge holes: 银行 (bank), 岸 (riverbank), and 倾斜 (to tilt). The Chinese model requires less "light manipulation" because the physical structures themselves are already disentangled.

Conclusion: Meaning is a Projection

As I finished my run, I realized that the bridge railing wasn't just casting shadows; it was performing a physical projection.

Large Language Models do not store a dictionary of definitions. They store a high-dimensional geometry of concepts. Meaning is not a static object you can hold in your hand; it is a projection. It is the dynamic, mathematical shadow cast when the light of context passes through the structure of language.

And just like the shadows on the pavement, the beauty of language lies not in the stone itself, but in how it changes with the light.

References:
Breeden, J. L. (2026). The Simple Mathematics of Large Language Models. CRC Working Papers.

post according to talk with Qwen 3.7 plus model

What is Abstraction anyway: Part Two

math4mad — Wed, 07 Oct 2026 02:36:55 +0000

What is Abstraction anyway: Part Two

The Super Power of Abstraction

1. Where we left off

In Part One, we used leaves to show what abstraction is: we took the same method, applied it to different objects, and got strikingly similar results. That was abstraction — a way of finding what is common.

But abstraction is more than a neat trick. It has a hidden power. In this part, we dig into how that power was born — not in a math classroom, but out of everyday life: a hunt, and a trade.

2. The hunt

Imagine a small tribe living in the forest. Every day, the same problem returns: where is the food, and how many people should go get it?

At first, the whole tribe went out together, searching in one direction. But one direction means a small space. So the tribe changed the plan.

They split into small groups. Each group went a different way. Suddenly, their search space multiplied — five small teams could cover five different directions instead of everyone crowding one path.

When each group came back, they brought back more than meat. They brought back two things: a number and a direction.

"There is a herd of deer to the northeast, about a day away."
"There are wild ducks by the southern river."

Each report was really a vector — a quantity and a direction tied together.

Now the leader could do something new. Instead of guessing, they could compare the reports: this direction has more game and is farther, so send more people; that direction has little game and is near, so send fewer.

The vector pointed the way, and it decided how many hands to send. The hunt was no longer a guess. Spread the search, decide together, catch the prey — and the payoff multiplied.

3. The super power of abstraction

Look at what happened. The tribe did not invent a magic weapon. They did one simple thing: they pulled the "how many" and the "which way" out of the actual deer and ducks, and turned them into symbols that could be compared and counted.

That is the super power of abstraction. It lets you take a messy, particular situation and boil it down to something you can reuse, compare, and decide with. A number without the deer. A direction without the walking. Combined, they become a tool for thinking.

Once the tribe had this power, they could use it again and again — for the next hunt, and the one after.

4. The trade

Now imagine the tribe by the river, where two peoples meet. One raises cattle, the other keeps sheep. They cannot eat each other's food until they agree on a way to trade.

So they agree on a rule:

One cow is worth five sheep.

That little rule is a function. Write it as f(cow) = 5 sheep. It does not care about this particular cow or that particular sheep — it is a dependence between two kinds of things, a rule you can reuse all day, every day.

But here is the real magic. The function can compose.

Tomorrow they meet a third people who grow wheat, and they agree: five sheep are worth two sacks of wheat. Write it as g(sheep) = 2 wheat.

Now, without any negotiation, the first people already know:

f(g(sheep)) = 10 wheat, for the same five sheep.

They never touched the wheat, never haggled, and yet they can already convert their cows into wheat. Each new rule in the market layers onto the old ones, and the whole village's ability to trade multiplies.

Carry this far enough, and the layers of functions begin to point at one thing everyone can measure against — and that, in time, is the seed of money. Money is just the universal function that maps everything onto one common scale.

The trade did not invent a coin. It invented a rule for changing one thing into another — and that rule, reused and composed, is what moves an economy.

5. Where it came from

So the two great powers of mathematics — the quantity behind numbers and vectors, and the correspondence behind functions — were not born in a classroom.

They came from ordinary life. A hunt taught people to pull out "how many" and "which way." A trade taught people to pull out "a rule that changes one thing into another."

Any time, anywhere — abstraction was already working, long before anyone gave it a name.

Think about it: can you find another example like this in your own life?

What is Abstraction anyway: Part One

math4mad — Wed, 07 Oct 2026 02:15:55 +0000

What is Abstraction anyway: Part One

1. An example from real life

Once, while I was running, it occurred to me that I might take a picture of a leaf against the sunlight. So I did — and this is what I got:

2. Maybe another one?

If you look carefully, you will notice that although we used a different leaf each time, we ended up with strikingly similar pictures — and we used the same method to create them.

3. What is this called?

When we use the same method to handle different objects and obtain similar results, that is almost what we call abstraction. But why?

4. When using this method, be careful about boundaries

The method above works well for thin leaves, but it won't work for a cactus.

5. Results

We extract the same method from different operations — this is what we call abstraction. The payoff: we can use it again and again.

If you think abstraction belongs only to mathematics and science, you are wrong. It comes from our normal life — any time, anywhere.

Think about it: can you find another example like this?