DEV Community

Jax Sinclair profile picture

Jax Sinclair

Semiconductor analyst. I write about AI silicon bottlenecks: advanced packaging, HBM, test, and the yield math underneath. Numbers first.

Work

Semiconductor analyst

A 671B model reads 36GB per token and still needs 671GB of HBM

A 671B model reads 36GB per token and still needs 671GB of HBM

Comments
7 min read

Want to connect with Jax Sinclair?

Create an account to connect with Jax Sinclair. You can also sign in below to proceed if you already have an account.

Already have an account? Sign in
Halving the KV Cache Doubles Capacity and Buys 1% of Speed

Halving the KV Cache Doubles Capacity and Buys 1% of Speed

1
Comments 2
6 min read
Decode Is 300x Memory-Bound. Batching Fixes That Until the Context Gets Long.

Decode Is 300x Memory-Bound. Batching Fixes That Until the Context Gets Long.

1
Comments 4
6 min read
The KV Cache Is Your Inference Bill: Bytes Per Token, From Three Real Configs

The KV Cache Is Your Inference Bill: Bytes Per Token, From Three Real Configs

1
Comments 4
4 min read
AI Packages Are Bending Under Their Own Size. The Money Moved to Prediction.

AI Packages Are Bending Under Their Own Size. The Money Moved to Prediction.

Comments
3 min read
Advanced Packaging's Yield Killer Isn't Assembly. It's Test.

Advanced Packaging's Yield Killer Isn't Assembly. It's Test.

Comments
3 min read
loading...