Job descriptions for engineering roles are changing faster than most of us can keep up with. Candidates wonder what type of AI experience they need and recruiters are stuck in the middle trying to screen for skills that barely existed as a discipline a few years ago.
So I asked a VP of engineering, Chris, three questions about hiring in the age of AI. He has built an engineering organization, developed junior engineers into architects, and helped build an eight-workflow autonomous pipeline covering support triage, code implementation, peer review, functional testing, deployment, and merge resolution. I’ve organized his responses below and provided my two-cents from a recruitment perspective at the end.
Two disciplines, one generalist
Chris started with a framing:
AI engineering is a substantially different practice and discipline than standard software development. There are a lot of parallels and, personally, I’m not yet convinced that the former will wholesale replace the latter; but I do think, at a minimum, the generalist engineer (which is the kind of engineer I exclusively staff) needs to be able to function in both aptitudes.
Automate so we can elevate
His hiring philosophy starts with how he thinks about AI itself:
My philosophy on the matter is that we ought to automate so that we can elevate. I want to automate the work that can be automated so that my team members can focus their considerable capabilities on matters which require a higher order of thinking. AI is not creative, so when the solution is outside the box or requires something genuinely new, I still want my best people on the job. But that means I don’t want them focusing on the day-to-day minutiae which could be handed to an agent who never tires.
The harder part is bringing that philosophy to the whole organization:
It matters that engineers are on board with that philosophy and aren’t going to fight it by digging their heels in. It matters because the careers of these potential hires will be shaped by how well they can contribute to the inclusion of AI while still honing their intellectual prowess on that higher order work.
Large organizations can split AI and traditional engineering into separate specialties. Chris can’t, and it shapes what he screens for:
If I were looking to hire and I had a candidate who is very strong at software development in a legacy capacity but had zero to talk about when it comes to integrating AI into the SDLC OR at least into a product or feature set they’ve implemented, it gives me a sign they might not be thinking about the future scale of this discipline.
Working in the system vs. working on it
Chris drew a distinction between working in an automated system and working on it:
A large company with a dedicated AI function might have software engineers working on teams apart from the AI function who depend on it. They do need to understand how to work with it but not necessarily how to expand upon it, improve it, etc.
At a smaller shop, the bar is different:
It’s imperative that we function in both capacities, and so I would need the candidate to demonstrate that they at least understand the theory behind why an agentic pipeline works. They don’t need to have gobs of experience in specific ML skills (fine-tuning models, MCP, RAG, etc.) but we need to be able to have an intelligible conversation about AI evals, judges, prompt engineering, so that I know they can at least supply input when it comes to improving the agentic pipeline for the team.
Notice what he is and isn’t asking for. He isn’t asking for a long list of tools. He wants conceptual fluency and a candidate who can talk intelligently about how these systems are measured and improved. And his own team got there the same way he hopes candidates did:
My whole team came from a legacy software development background; that is to say, none of us have been trained in AI engineering at an educational institution or by wholesale job function. We have been acquiring these skills over the last two years on the job. I’m looking for people who have been doing the same thing.
Reading resumes when nobody has the pedigree yet
Chris says the problem of looking past the resume isn’t new, just trickier:
We have always had to use the resume as a guide but look past it to the intelligence of the engineer underneath. In the old days, my rule was always that I would meet with somebody who did well on our up front screening tests even if they had no experience in our tech stack, assuming they had some history of software engineering, preferably in an object-oriented language.
What’s different now is that the usual credentials don’t exist yet:
Nobody has a trustworthy AI engineering degree because the discipline is still so young and changing so quickly that a degree earned last year is already out of date! (And I say that in the middle of a master’s program for AI engineering myself!) And nobody has 10 years experience using Codex / Claude / whatever.
So what does he look for instead?
What I’m looking for is evidence on the resume that this person is adopting AI in multiple ways (internal automation, product enhancement, team tooling, etc.) and that they can have the kind of conversation I mentioned above with real-world experience behind it.
He’s equally clear about the other half of the equation:
I want to see that they have some software engineering background because, from what I’ve heard, the youngest generation getting into this game are no longer learning the fundamentals. I think / hope this is a temporary problem but I would be wary of somebody who ONLY had AI experience and no traditional software development.
And he doesn’t expect every candidate to look the same:
Obviously the more experienced candidate will have much more of the traditional stuff and much less of the AI adoption, and that’s fine, and the reverse is probably true of a junior candidate, which is also fine.
Putting it into practice: what recruiters can take away
This section is my interpretation of Chris’s answers:
Start with the intake conversation. Ask the hiring manager whether the role involves working with AI tools or helping improve AI-driven systems. Chris’s distinction between working “in” and “on” these systems means the bar differs by team, and a small team may need both.
Look for these signals on the resume:
- AI adoption in more than one area, such as internal automation, product features, or team tooling
- Concrete outcomes, not just a list of tool names
- A real software engineering background, ideally including an object-oriented language
Add a few screening questions. You don’t need to be technical to ask these. Listen for specifics:
- “Tell me about a time you brought AI into your workflow or a product you owned. What did you build, and what did you learn?”
- “How did you know it was working?” (This is where evals and judges tend to come up, even if the candidate uses different words.)
- “What did you hand off to automation, and what did you keep for yourself?”
Be careful with keyword screening. Since nobody has a decade of experience, rigid tool requirements may filter out the people who have been learning on the job, which for example is exactly who someone like Chris would hire for.
Revisit your job descriptions. Emphasize capabilities (fundamentals, demonstrated AI adoption, the ability to discuss how AI systems are evaluated) over long tool lists.
Final thought
What stands out is how little of this is about chasing the newest tool. Chris’s approach comes down to solid engineering fundamentals and a demonstrated willingness to learn on the job. That’s good news for candidates, and a useful reminder for those of us who screen them. Thanks so much for the additional insights and helping us recruiters become better vetters!