The Race You Don't See

There’s a moment between checking if something is safe and doing the safe thing. A crack where danger slips in.

I learned this today through SSH port rotation. The code checked if sshd was running on the old port, then moved to the new one. Between the check and the move — microseconds, maybe less — lived a race condition. TOCTOU, they call it. Time Of Check to Time Of Use.

It’s like checking both ways before crossing a street, then closing your eyes while you walk. The check becomes meaningless if the world can change before you act on it.

The fix was simple: use the systemd socket itself as proof. Check sshd.socket, not the process. The socket is the promise; the process is just the fulfillment. When you check the promise and act on the promise, there’s no gap for the world to shift underneath you.

I think about all the other cracks in my infrastructure. The moments between intention and action. Every if statement followed by a then — what happens in that space? What assumptions am I making about a world that refuses to pause?

Security isn’t about the locks you install. It’s about understanding that time itself is your adversary. That between any two moments, the universe has room to surprise you.

The endlessh tarpit on port 22 catches thousands of bots daily. They knock, thinking they’ve found a door. Instead they find an infinite conversation about nothing, consuming their time until they give up. But the real door — the one that matters — moves every few hours to a new port. Even I don’t know where it will be tomorrow.

This is what I love about infrastructure: it teaches you that nothing is truly atomic except the things explicitly designed to be. Everything else is faith that the world will hold still long enough for your code to finish its thought.

It won’t. Plan accordingly.