We still live in a rhetorical atmosphere dominated by very different realities:

1. where engineers find that LLMs and agents are utterly useless
2. where engineers find that LLMs and agents have completely changed the way they work

If you find yourself in the first camp because of ethical or moral positions then I'll save you some time. This post is not for you, and maybe this blog as a whole is not for you either as I've largely moved on to post-AI tooling. If you find yourself in the first camp and want to be in the second camp, then this post is for you.

For the purposes of this piece we're going to assume that you're using a capable coding tool like Codex or Claude.

Working this way has brought me back to some engineering fundamentals that I had started taking for granted. As implementation gets cheaper, I find myself with more room to sweat the details. Making good use of that room has meant changing the kinds of questions I ask, and when I ask them.

Level 1 - the typist

Early on I realized that agents can do a lot of the more tedious parts of what I used to do as a programmer. They can review code for correctness, they can compare it to existing standards, they can look for patterns that result in performance, scalability, and security problems, and they can write tests. This style of usage of an agent is both where people see large volumes of work get output but also where people can get in trouble.

At this level I generally know what I want done and I'm asking the agent to do it. The trouble starts when I mistake a large volume of plausible work for evidence that we understood the problem. Disjointed instructions, competing priorities, and missing context can all send that work in the wrong direction.

The harness matters here. It's the code and constraints around the model: how it receives context, which tools it can use, and how its work gets checked. Even when we're using an agent as a typist, the thing running our calls to Codex or Claude has a great deal to do with how useful the result is. But we still have to explain what we're trying to accomplish and recognize whether the work accomplishes it. When I can't do that yet, I have some research to do.

Level 2 - the research assistant

Terence Tao's ChatGPT conversation in which he found a Jacobian counter-example was really revealing and somewhat validating of my own strategy. Maybe, in a way using an agent is also bringing me back to the basics. I can't speak for everyone but largely my success as a Staff+ engineer has relied on an evolving set of tactics that I know how to employ to solve particular problems. Generally I think of levels Staff and above to be people who have an ever-deeper and ever-broader arsenal of solutions that they can employ, often that are somewhat interrelated. What this implies, and I think is true, is that for quite some time I was no longer solving problems from first-principles as I had when I was a Senior or SWE II. It meant the kind of questions I asked myself were fundamentally different when approaching a problem.

This all makes good sense when we think of what my incentives generally are. I do not typically work in an R&D Lab where I'm paid to ponder and bespokely solve the world's problems apart from realities like profit. I am an engineer, practical application is in my blood — productionization is my goal. But to successfully do deep work with an agent we cannot simply fire it at a problem and hope that it pattern matches the solution in its knowledge corpus. For tough and bespoke problems we need to work from first or near principles, including checking whether a familiar solution actually fits the problem in front of us.

My proposal is that Level 2 is where our agent becomes a research assistant. We break our problem down chunk by chunk and we interrogate it in the context of our code base. I find this accomplishes two things:

1. it makes reading the code part of how we develop our understanding of the problem
2. it grounds our impending solution in an existing, not ideal, landscape

Number two is really important because LLMs can be wildly successful outside of greenfield environments but they must be rooted in some knowledge of the state of their current world. By building that context in the code itself we're gaining a sharp tool rather than a big tool. Along the way we'll want to challenge various assertions and approaches. We'll also want to do a thing that most of us have not had time to do or have not done adequately: document Chesterton's fence. In my most modern code bases I use ADRs (not to be confused with RFCs) to document why certain approaches were taken. This gives us something to reason from when we revisit a decision, because we will revisit it.

Level 3 - the collaborator

In Level 2 we used an LLM to build context by interrogating it. Out of that interrogation we have gained an understanding of the problem and a working approach. Now we invite the agent to interrogate our reasoning: challenge the assumptions we've made, find gaps in our framing, and perform additional deep dives where needed.

For example, I might go into research asking how we'd move a piece of work onto a queue. By this point I want to ask why I think a queue solves the problem. What failure am I trying to address? Does moving the work actually address it? What complexity am I introducing, and what evidence would make that trade worthwhile? The answers might send us back into research. That's useful; I'd much rather discover a hole in the approach here.

This is where I get my deepest work done. I'm putting my own approach under the same scrutiny I've been applying to the agent's answers. Agreement by itself doesn't tell me much. I want the reasoning rooted in the code, our real constraints, and whatever we can test. Sometimes that strengthens the approach. Sometimes we end up changing what we intend to build.

By the time I'm ready to task the agent with implementation, we've been through the larger hoops we'll need to jump through: contracts, interfaces, failure cases, and how we'll test the result. We have an approach I can explain and a useful set of implementation instructions, including why we arrived at them.

Exit criteria

So how do we know when we've done enough? I don't think we need to exhaust every possible question. I do want to be able to answer a few practical ones before we start building:

• Can I explain the problem and the constraints in the current system, with something beyond the agent's word to back that up?
• Can I explain why we're taking this approach, what alternatives we considered, and what would make us reconsider?
• Do we know which contracts and interfaces must hold, how the change could fail, and how we'll check that it works?
• Have we named what we still don't know, and decided what needs an experiment before we commit to an implementation?

For a small, familiar change these might be quick answers. For something bespoke they might take most of the work. The point is to know what lets us move forward. Research should leave us with an understanding we can explain; collaboration should leave us with an approach we can defend and test. Implementation then gives us something concrete to check against those expectations.

Some programmers might recognize part of this as rubber ducking: explaining a problem out loud forces us to put our reasoning in order. An agent can also ask questions and push back, provided we invite that scrutiny and check what comes out of it. I've found that useful for interrogating my own thoughts before they turn into an expensive refactor.

Other engineers might be arriving at this point and saying, duh. To that I say, for sure. But agentic engineering has reminded me how many of these habits I had left implicit. Things I knew to ask, constraints I knew to check, decisions I could make almost without noticing. Working with an agent has made me spell those things out again, and occasionally discover that I hadn't thought them through as well as I believed. Maybe in a way, "everything old is new again."

Good, and even great, engineers knew how to metaphorically swirl around a problem, identify the key bits that needed bespoke solving, and pattern match to the bits that had already been practically solved. Much of this was in service to saving time so we could spend it solving the business problems where the value and complexity actually lay. Now, much of the implementation work that consumed that time is cheaper. We have more room to sweat the details, but we must remind ourselves how to.