Forty-five minutes in, I'd walked through systems I built. The architecture, the integrations, the auth decisions, what broke in production and what I did about it. There are working products online with demos anyone can open. Nobody opened them.
Then came the coding exercise. Share your screen. Open the editor. Turn off AI, disable autocomplete. Write a program that calls an API and returns a picture of a bird and a fact about it.
The endpoint was supplied. So were the library, the response shape, and the expected output. The product and architecture decisions were already out of the picture. What remained was implementing a tiny, prescribed task without the tools I'd normally use.
I've shipped a lot of things. I have never once shipped a bird.
When I talk to someone who knows a field I know, I can usually hear it inside five minutes. They ask about a condition I haven't mentioned. They tell me why an approach works here and turns into a problem two systems over. Change an assumption on them and they follow it downstream without stopping to reload.
I've found that in software and I found it in a woodworking shop. People who have actually done the work hand you details you can push on. The part that took three times longer than the estimate. The shortcut they took once and will never take again. You always have somewhere left to go.
A first impression still needs checking. Keep going with the follow-ups, because somebody can stumble over a sentence and still understand the problem cold, and somebody else can be very smooth and very empty. But by the end of a real technical conversation, an interviewer should be holding evidence.
So why spend forty-five minutes asking about my experience, decline to look at the results, and then let a small unaided API call cast the deciding vote?
1. The woodworking shop
I owned a woodworking shop with expensive equipment in it. A jointer, a planer, a table saw with a fence I could trust. That equipment took rough lumber and turned it into square, four-sided, dimensioned stock faster than I could ever do it by hand. That was the entire point of buying it.
A stack of perfectly milled boards still wasn't furniture. I had to know what I was building, how the joints would carry load, where the material would move across a winter, and whether the finished piece was any good. If I couldn't put it together, the machinery didn't excuse that. The gap was mine.
Some excellent woodworkers can walk onto a site with basic tools and build something beautiful. I respect that ability. It doesn't follow that using a planer proves somebody lacks it, and making a person repeat every prep operation by hand will not tell you whether they can deliver the finished piece. It'll tell you they can push a hand plane.
I bought equipment to take back time that didn't need to be spent, and to get consistency I couldn't hold by hand at volume. What I got in exchange was attention I could spend on the design and on the parts of the job the machines hadn't solved.
AI isn't a planer. It can hand you something confidently wrong, which makes verification part of the engineering problem instead of a formality. But verification is a thing you can design, automate, and hold to a standard.
The objective is a well-built result. I don't hand out extra credit for preserving unnecessary labor on the way there.
2. Nobody panics about the teleprompter
Here's a tool that's been around since the 1950s.
An executive can deliver a prepared speech from a piece of glass. A news anchor can read a newsroom's script. We understand that the person on camera may be working from words prepared with other people. Imagine interrupting a CEO halfway through a town hall: hang on, kill the prompter, let's find out if he actually believes any of this.
I'd have questions about what he said. Whether he'd memorized it wouldn't settle them.
Apply the same test to a quantitative analyst: take away the computer and ask for long division on a legal pad. You could learn something about their arithmetic. You'd still have questions about whether they understand the risk they're being asked to model.
Spellcheck. Route planning. CAD. Compilers, for that matter. We could spend the afternoon removing assistance and congratulate ourselves on how much harder we'd made the job.
The honest objection is that these tools aren't equivalent. A teleprompter displays a prepared script; a generative model can supply reasoning the person never had. Fine. That difference is worth examining. Tell me which ability this role requires unaided, why this exercise measures it, and I'll take that seriously.
What I won't accept is the reflex. Finding a tool in someone's hand is the beginning of a question, not the answer to it.
3. Walowitz is mine
I've spent the better part of a decade building my own assistant. Walowitz is part of it, in my editor and across the other places I use it. I haven't released it publicly. I built it for myself.
Cognitive relay is one reason I built it. I can read a raw log, see the relationship in it fast, and still lose the explanation somewhere between knowing the answer and getting it out of my mouth on demand. Anxiety makes that worse. I have understood a problem and blown the explanation in front of people. It doesn't feel like a communication problem. It feels like being found out.
For recall, the material I want back is my own writing: notes, comments, ledgers, and decisions I recorded while doing the work. Retrieved passages carry references back to their source. That lets me return to the record rather than pretend a new model-generated explanation is something I wrote years ago.
Exact identifiers and semantic search do different jobs, so I run both and put exact matches first when they're available. Asking for the function where I handled a specific case isn't the same request as asking for everything related to an idea. The combined results are deduplicated before entering the context.
Walowitz cancels retrieval when a newer request overtakes it and expires temporary context on purpose. Material from the previous subject can be completely accurate and completely wrong for the conversation happening now. Skip that and you've built a very sophisticated way to remember the wrong thing at exactly the right moment.
Recall isn't permission to act. As I extend the assistant into screens for my kids, a request to trace the letter B shouldn't inherit authority to touch the router. If I ever confuse those scopes, I have considerably larger problems than a coding interview.
The practice ledger has separate states for aided, unaided, and unrecorded turns. Zero means nobody measured this. It doesn't mean unaided. Within a tracked turn, an aided label doesn't revert merely because the display later clears. The ledger still has paths that don't record aid. I won't pretend it is finished.
I chose those states because a ledger based only on the session's opening mode would record a preference and call it behavior. A flattering default becomes indistinguishable from a real measurement once somebody writes it down.
Pick one of those decisions. Keep asking. Change a condition and see whether I can follow the consequences. We can discuss those choices without distributing private source or a former employer's work.
But first, apparently, we need to establish whether I can fetch the bird unassisted.
4. Knowing what not to build
The first decision on anything is whether it should exist. Who needs it, what are they trying to do, what would make it worth using.
While one team builds a capability from scratch, somebody else may find a maintained service or an open model that already does it and spend those weeks shipping the part nobody has solved. Checking that is part of the job. License, operating cost, what you'd depend on, what happens when it changes under you.
There are real reasons to build your own. The available option might require sending customer records somewhere they're not allowed to go, or become unaffordable at the volume you're planning for. I want those limits investigated before anybody commits the team to months of work, not discovered afterward.
That judgment is invisible in a test where the build-versus-buy decision was made for me before I opened the editor.
5. Companies already changed the test
Karat surveyed 400 engineering leaders across the United States, India and China. Seventy-one percent said AI was making technical skills harder to assess, while 62% of organizations still prohibited it in technical interviews.
Karat co-founder Mo Bhende put it to IEEE-USA plainly: "the fundamental job has changed, but technical interviews, as we know them, have not."
Canva announced in June 2025 that backend, frontend and machine-learning candidates were expected to use AI tools. Its pilot found that some candidates who could code struggled anyway, because they couldn't guide the tools or recognize a bad suggestion. Canva kept code fluency and technical depth as requirements. Allowing assistance exposed a weakness the old exercise hadn't been designed to assess.
Meta's July 2025 internal memo gave two reasons for developing AI-enabled interviews: they better represent the working environment, and they make AI-based cheating less effective. 404 Media reported the memo with WIRED. Read that second one twice. Allowing visible assistance was part of the stated rationale for protecting the assessment.
CoderPad's 2026 survey found its hiring respondents split 34% prohibiting, 46% allowing broadly or with constraints, and 20% deciding case by case. Asked what shows skill when AI is allowed, 66% picked catching and fixing its mistakes and 56% picked explaining trade-offs and correctness.
Excluding AI is a choice. It isn't a settled professional requirement, and it shouldn't be presented to a candidate as one.
Anthropic's candidate guidance, updated July 2025, prohibits AI in live interviews and take-homes unless permission is given. The same page describes using Claude to develop questions, draft communications, analyze hiring metrics and source candidates. Claude doesn't make the hiring decisions.
Preparing a question and answering one are different activities, so different rules on the two sides aren't automatically hypocrisy. But the restriction still owes the applicant an explanation. Which ability does this job require unaided, and why does this exercise deserve the weight you're giving it?
6. What about cheating
The concern is legitimate and I'm not going to wave at it.
Fabric's January 2026 report analyzed 19,368 interviews on its own AI interview platform. It flagged 38.5% of candidates for cheating behavior, and roughly 61% of those flagged scored above its passing threshold. That's one vendor's detection and scoring, not an independently established industry rate, and it doesn't establish that every flagged person cheated.
If the people you believe haven't demonstrated competence are clearing your threshold at that rate, the scoring deserves investigation alongside their behavior. Adding a ban doesn't establish that the score measures the ability you're hiring for.
There's risk pointed the other way too. A 2020 NC State and Microsoft study put 48 computer science students through a whiteboard problem. The ones solving it while watched and narrating performed about half as well as the ones working privately. Observation and narration both changed at once, and they were students, not principal engineers. The size of that gap is still worth noticing before anyone insists the stressful version is the honest one.
AI can also undermine learning, and I'm not leaving that out because it complicates my case. In Anthropic's January 2026 randomized study, 52 mostly junior engineers learned an unfamiliar Python library. The AI group averaged 50% on a quiz covering concepts they'd used minutes earlier, against 67% unaided. The speed gain wasn't statistically significant.
Inside the AI group, people who asked conceptual questions or followed generated code with questions to understand it scored better than people who mostly delegated. That comparison was exploratory and can't establish cause. It still gives an interviewer a behavior worth investigating rather than treating every use of assistance as equivalent.
I'm not claiming AI always makes experienced engineers faster, either. METR's February 2026 update revisited its earlier 19% slowdown finding and reported that the follow-up couldn't reliably estimate the current effect, largely because developers declined to participate or withheld tasks they didn't want to attempt without AI. The experiment was missing people and work that could have changed the answer.
A fast, plausible mistake is still a mistake.
7. Keep the birds, then change the response
I'm not asking anybody to stop testing me. I spent part of my twenties in military aviation, where competence was checked by people qualified to check it. I have no objection to proving I can do the work.
A failure scenario can justify taking a capability away. Then the question is how the person handles that failure in the job they're trained to do. That's an explanation I can understand. "We always turn the tools off" isn't one.
So keep the bird exercise. Then make it worth the hour.
Change the response halfway through. Introduce a timeout. Hand me an implementation that looks reasonable and has a defect sitting in it, and ask what needs testing before anyone should trust it. Let me use my tools, including automated review, and watch whether my checks find the problem and what I do once they have. Make me explain why the evidence I produced addresses the changed requirement.
Then take a problem with real decisions in it. Talk about what the user needs and let me investigate the options. Ask why I'd use an existing service instead of building one. Change a constraint and follow it with me. I can use research or a model to explore it, and you can ask what made me accept one approach and abandon another.
Work I can legitimately share gives you another place to start. A demo leads into the decisions behind it without exposing a former employer's code.
Nobody needs to complete an unpaid production project for this. A bounded exercise plus a serious technical conversation gives you several kinds of evidence instead of one convenient proxy. Give candidates comparable access and consistent criteria. That addresses an important part of fairness without pretending the working environment doesn't exist.
If your process can make time to watch my screen share but not to open what I built, the priorities are backward.
And if you're using a tool to assess me, own how you interpret its output. That standard runs both directions or it isn't a standard.
8. What I'd want to know before I said yes
I wouldn't want to join a company that expects end-to-end AI engineering and treats an unaided API call as the deciding test, particularly if the people running it can't connect it to the role. I'd question whether their understanding of the job had caught up to what they're asking for. I don't need to prove their whole stack is obsolete before deciding to pass.
There's a question worth asking out loud in either direction. Am I being hired to work inside an established engineering practice, or to modernize the software and change how it gets built? The second one comes with budget decisions, approval processes, and people outside the development team who have to agree. That's a different job, and whoever takes it needs the authority and the resources to do it.
A paycheck doesn't resolve a mandate nobody defined. I'm not interested in spending my first six months arguing for the methods I was supposedly hired to bring.
I'm asking you to put the bar where the work is.