Cache: The Third Word, and the Teacher Behind Them All

The third word a professor gave me to carry for life — and the one closest to his own heart. This is also where I say goodbye to him.

A look back — long before any of the tools we argue about now.

He must have given us many more words after the first two, but my college years blurred them out. The third — and the last I clearly remember — was the one he called the word that would run through my whole career: cache. It sits right alongside trade-off as one of its cleanest examples, and it happened to be his own field. It’s also the moment I fell, for good, for computer architecture. A few memories that cluster around it.

Locality

Temporal locality — you tend to reuse the same data again soon — and spatial locality — you tend to touch nearby data next. Use both, in software and in hardware, and the whole system runs faster, for almost free. The requirements are unusually well-defined, and the payoff is enormous. The professor used to call it beautiful. At the time, buried in benchmarks and the dozen techniques you had to understand, I couldn’t see the beauty he meant.

It floored me that hardware was implementing this logic in silicon, and that gave me the nerve — and the recklessness — to push deeper into operating systems and file systems. And then the fussier, more obsessive side of it: the tricks for feeding hints to the compiler. I tried to hack on something like GCC to make a few of these ideas real, failed, and hit one wall after another. With a little more patience, maybe a different future would have opened up.

Other kinds of memory

As an undergrad, a new machine showed up called the Intel Pentium, and suddenly “L2 cache” was a phrase I could actually understand — though my grasp of it was about as deep as more and pricier must be better. After that came SRAM and its cousins, NOR and NAND flash, SSDs — a corner of the world that kept exploding, and that I had to keep learning whether I meant to or not.

Get Chaesang Jung’s stories in your inbox

I only ever got a taste of it. The professor and the grad students above and below me lived at the center of it, ranging across compilers, operating systems, controllers — taking the same idea and implementing it as is, or improving on it. Watching them, I caught a small revelation: maybe this is what it means for something to run through a fast-moving world for years and still hold.

Cache, everywhere outside

I started out in the so-called orthodox school — computer architecture — but every concept I met outside touched back to this one, and by now it’s a word everyone uses without thinking. Hardware caches, network caches, software caches. In a few system-design interviews I’ve given and heard the same lazy answers — “just put a cache in front and it gets faster,” “just buy more machines” — and in practice you end up nodding and doing exactly that. Memoization, CDNs, a cloud storage gateway: all the same idea, and it’ll keep being the same idea.

Professor Sang Lyul Min

I only knew him through my master’s, but I still remember him — now gone — with love and resentment tangled together. There was real resentment by the time I graduated. I had too many gaps to take in everything he offered; a lot of it I only understood once I’d grown up a little. Today he’s the closest thing I have to the adult I’d want to become. I’m sorry it took me so long to see what he meant by the pursuit of goodness and beauty. The last time we really spoke was when I came by to tell him I’d gotten into Google; he gave me a wide, bright smile. I didn’t know it would be the last time.

In an interview, he once said something I keep coming back to:

“I tell my students to understand the whole system. So I suggest that people who studied hardware as undergraduates go do software research, and people who’ve worked in software go do hardware.”

“Getting to 99 percent is easy. But to make something beautiful, something at 100 percent, you have to spend five to ten times all the effort it took to get there.”

Part of 90s Computer Science Stories — first-person notes on growing up with computers and studying CS in Korea in the 1980s and ’90s. See the full series →

Adapted from my Korean essay on Brunch: brunch.co.kr/@chaesang/35