The Lamplighter Test: Hiring Engineers Without the Trivia Circus
Photo: William Bullard, c. 1903, Worcester Art Museum. Public domain (CC0) via Wikimedia Commons.
You don't have to sit through a lot of tech interviews on either side of the table to figure out that somewhere along the way, the whole format got broken. The AI era didn't cause that, it just made the cracks impossible to ignore.
The old move was grilling somebody on a specific algorithm, a language quirk, some framework flag nobody remembers three months after they stop using it. That was already a weak signal for whether a person could build real things. Now you hand someone a hard problem and they open an LLM, and out comes something that looks competent enough. So the new trick is to interrogate them about the answer the model gave them, ask them to explain it back like a hostage negotiation. That's not really testing engineering skill anymore, it's grading a book report on somebody else's homework.
Specific tools, specific languages, the small stuff, sure, it comes up naturally in conversation, but don't let it anchor how you score anyone. None of it tells you whether a person can actually reason through a system they've never touched before, which is what the job demands, day in and day out.
What you do instead is give them a story.
Here's roughly how I would tell it:
There's a Lamplighter in charge of an enormous number of lights, spread across halls scattered all over the world. Every hall's got at least one light in it. He wants a system where he can send an HTTP request for any single light and find out if it's on. Every light needs its own identifier, unique across the whole fleet, not just within its own hall.
Helping him is a swarm of small robots that maintain and swap out the lights, and they're not exactly well-behaved. They're constantly firing requests at random intervals, sometimes microseconds apart, sometimes days apart, and every once in a while hammering the same light over and over back to back. The Lamplighter himself checks in politely, maybe every few seconds. Every robot request needs an answer no matter how chaotic the timing gets. Robots can also flip a light's state through the same API, on the order of seconds, on whatever light they feel like. The Lamplighter never touches state, he only reads.
Get an answer wrong, slow, or stale, and it costs real money. He wants this properly tested, load-tested on the engineer's own machine so the weak spots surface before production finds them, and he wants confidence it'll hold up under serious load no matter how many lights or halls or robots pile on, even running on hardware nowhere near what a real deployment would use. (The candidate can reason about how the resource needs would scale past whatever their laptop can handle.) He wants it on GitHub, built properly, with commits that actually tell the story of how it came together instead of one giant squashed blob, a pipeline that runs tests and checks code and dependency quality on every push, and ideally something that can deploy too, real target or made up, doesn't matter which.
That's the whole brief, nobody hands them a stack or dictates the architecture. They decide what "right" looks like and go build it.
Why choose this over a leaderboard problem
Because how far someone takes it says more about them than the output itself does. Somebody can spend five minutes with an LLM open and hand back a passable skeleton. Somebody else will live inside this thing for days, chewing on it during their commute, finding new angles nobody asked them to find. That range, and where a given person lands in it, tells more than any answer to a red-black tree question ever could.
What to actually watch for is what questions people start asking themselves without being prompted.
The traffic from the robots and the traffic from the Lamplighter aren't the same shape at all: different volume, different consistency needs, different latency budget. Does that push someone toward something like CQRS on instinct, or do they just write one code path and hope for the best?
The robots aren't just loud, they're erratic, bursting and overlapping and sometimes slamming the same light twice in a row. Does the candidate think about caching, rate limiting, or aggregating that mess before it ever touches the real state?
Are the halls just flavor, or something to actually use? What happens when a few halls are packed and most sit nearly empty? Does that shape how someone would shard the data or seed it, and do they even notice the hook is sitting right there?
Two very different actors are hitting the same API with very different permissions. How do they draw that line, auth, roles, scope, and do they bother drawing it at all?
None of that's a checklist I'm scoring line by line, it's closer to terrain. Watching where someone chooses to walk on it tells me what they actually care about when nobody's grading them for it.
Yes, the task sounds ridiculous
A guy called the Lamplighter, an army of clumsy robots, a hall full of glowing bulbs, it's absurd on purpose. There's nothing trendy to name-drop, and nobody's walking in with the "correct" repo already memorized, so it forces actual thinking instead of pattern matching. Hand it to a junior and just strip out whatever advanced machinery feels out of reach for them. They can still show how they'd approach an open problem with what they've got. If the solution's rough, fine, that's what the follow-up conversation is for: walk them through where a stronger approach would go and let them talk through how they'd rebuild it with that in mind. Not live-coded, just talked through, the way an engineer explains a decision to a teammate over coffee.
On grading it
Whoever reviews the thing needs enough expertise to have built something similar themselves, or at least argue convincingly for a better version. If somebody ships it in a language the reviewer barely knows, or it balloons into something genuinely huge, fine, use an LLM to get oriented on the shape of it. But the actual judgment call, is this good, is this the right tradeoff, would it survive real load, has to come from a person who could sit down and write the competing solution. Hand that part off and you're back to grading somebody else's homework, just with extra steps.
The task shouldn't run the whole interview
It's open-ended enough that a sharp candidate could rehearse a corner of it and perform expertise they don't fully have. Expect that going in, plan around it. Most of the conversation should feel less like a defense and more like two engineers wasting time at a bar. Some networking weirdness you ran into, a piece of hardware that surprised you, a security story, something in infra that broke in a way you still think about. Let the candidate follow whatever they actually know and care about. Even someone quiet by nature can talk shop comfortably once the room relaxes, and if they genuinely can't, that tells you something real about how they'll function on a team, not a mark against their skill. The interviewer's job is just to nudge the conversation toward whatever matters for the role, not run a courtroom.
Hire for how somebody thinks when nobody's handed them the answer key. Everything else you can teach on the job.