AI is changing how quality engineers work. I think we all know that by now.
I spoke with a QE leader about how she’s using AI in testing, where integration projects create quality risks, and what skills she believes are becoming more important for QE engineers as AI takes over more of the repetitive work. Some of her answers were highly technical, so I wanted to break them down from a recruiter’s perspective.
It’s important for us recruiters to understand what these engineers are actually doing so we can ask better questions, recognize stronger answers, and avoid treating technical skills like a checkbox.
1. AI Isn’t Just Writing Test Cases
One of the biggest themes from the conversation was that AI is being used across several different parts of the testing process, and the important part wasn’t simply which AI tools she uses. It was how she uses them.
Take something as basic as writing queries. She uses AI to help generate and troubleshoot SQL and SOQL queries, and if you’re not technical, the simple version is this: a query is basically a question you ask a database, something like “show me all customers who’ve been active in the last 30 days.” Instead of manually writing the database language needed to ask that question, she can use AI to help generate it. SOQL is Salesforce’s query language, similar to SQL but designed specifically for working with Salesforce data.
But here’s the part a recruiter should actually care about: “this candidate has used ChatGPT to write SQL” tells you almost nothing. The better question is whether the engineer understands what the query is supposed to do and can verify whether AI generated the correct answer.
That’s the theme that runs through the entire convo: AI can help produce the work, but the QE needs to understand whether the work is correct.
So instead of asking:
“Have you used AI for SQL?”
Try:
“How have you used AI to help with SQL or database testing, and how do you verify the output is correct?”
You’re now testing for understanding rather than tool exposure.
Same pattern for data validation. Imagine a company migrating information from an old system into Salesforce. You don’t just want to know whether the migration finished. You want to know whether the right information moved, and moved correctly.
For example, a customer status stored as “1” in the old system might need to show up as “Active” in Salesforce. Someone has to verify that data didn’t get lost, change incorrectly, or land in the wrong place along the way.
She normally uses a dedicated tool called QuerySurge to compare data between systems, but when that tool wasn’t available, she used AI to help perform some of that validation instead.
That’s a useful thing to listen for in an interview. Not just:
“I use AI for data validation.”
But whether the candidate can explain what they were trying to validate, what could go wrong, what method they used, and how they knew the result was correct.
A strong example from the conversation though was test case generation.
She’s used Claude to generate test cases directly from acceptance criteria, the requirements that spell out what a feature is supposed to do.
Say the requirement is:
“A customer should be able to reset their password using their registered email address.”
AI can generate the positive scenario: valid email, reset email sent.
The negative scenario: email doesn’t exist, system correctly blocks it.
And edge cases: an extremely long email, a form submitted multiple times, or the email service being down.
That’s useful.
But her expertise is still what makes it work, because she has to look at what AI generated and ask whether it actually understood the requirement.
AI might miss an important scenario, create tests that aren’t needed, misread the business rule, or produce something that sounds technically correct but doesn’t actually validate anything.
She described a process where AI generates the test cases and she reviews and validates them before they ever get used. That’s what separates:
“I use Claude to generate my test cases.”
from:
“I use Claude to generate positive, negative, and edge cases from acceptance criteria, then review the output against the requirements, identify gaps, modify the cases, and only then add them to our test management system.”
If you hear the first version from a candidate, push further:
“How do you review AI-generated test cases before using them?”
Or better:
“Can you give me an example of something AI generated that you had to correct?”
That second question is particularly useful because it forces the candidate to demonstrate judgment, not just usage.
She also made a point of not defaulting to the same AI tool for everything. She’s used ChatGPT, Claude, Copilot and NotebookLM for different kinds of problems rather than treating whichever tool her company issued as the default for every task.
You don’t need to know the differences between all of these as a recruiter.
What you’re listening for is the reasoning behind the choice.
“I have a problem, and I determine which tool is appropriate for it” is a very different answer than “my company gave me Copilot, so I use Copilot for everything.”
A good follow-up question is simply:
“How do you decide which AI tool to use for a particular testing task?”
You’re looking for the why, not a list of product names.
All of this points to the same underlying theme: human-in-the-loop.
AI can generate queries, test cases, automation, data comparisons, documentation or internal knowledge agents. She built one herself using Copilot Studio to help onboard new interns, with the agent’s knowledge base scoped to SharePoint documents.
But in every case, she remains responsible for deciding whether the output is actually correct. That’s the thread connecting everything in this section: the tool changes, but her judgment is the constant.
2. QE Is Becoming More Development-Adjacent
This is where the conversation moved from how she uses AI today to something recruiters should be paying closer attention to going forward: what the QE role is becoming.
As AI takes over more of the repetitive work in the environments she’s working in, such as generating test cases, writing initial queries, and drafting documentation, she said automation has become a central focus for QE engineers, especially on fast-paced projects with limited headcount.
Teams are leaning on AI not just to test faster, but to actually write automation scripts. That work is bringing QE closer to development, particularly when engineers are expected to understand, evaluate, and modify automation code.
Her framing was direct: QE engineers sit inside tools like Claude, and the job isn’t just prompting the tool. It’s knowing what you’re looking at and what needs to change.
That’s a meaningful shift for recruiters to internalize because it means QE candidates increasingly need to be able to read AI-generated code, not just AI-generated test cases, and decide whether it’s actually right.
It’s the same human-in-the-loop judgment from section one, just applied one layer deeper: to code instead of test scenarios.
A QE engineer who can prompt an AI to write an automation script but can’t evaluate whether that script is doing the right thing still has an important piece of the work left to do.
So if a candidate tells you they “use AI for automation,” it’s worth asking:
“Can you give me an example of AI-generated automation code you had to review or change?”
Or:
“How do you determine whether AI-generated automation code is actually doing what you intended?”
Those questions do the same job as the test case follow-up in section one: they separate someone who’s used the tool from someone who understands the work well enough to catch the tool’s mistakes.
3. Integration Testing: When Two Systems Both Work but the Data Is Wrong
I moved this part of the conversation away from AI and into something I’ve seen recruiters often struggle evaluating: integration and data quality, where two or more systems have to exchange information correctly.
Picture data moving from a source system, through middleware, into Salesforce, then into a data warehouse, then into reporting.
Every individual system in that chain might work perfectly on its own. The problem happens between them, which is exactly what integration testing is meant to catch.
Middleware, in simple terms, is software that helps different systems communicate, move data, and sometimes transform that data along the way.
And that transformation step is where things can go wrong.
One of the main points I took away was:
Don’t assume the mapping document is correct.
A mapping document tells the technical team how information from one system is supposed to map into another.
For example:
| Source System | Target System |
|---|---|
| Customer_Name | Account_Name |
| Customer_Email | |
| Status | Customer_Status |
A recruiter might hear “there’s a mapping document” and assume the requirements are fully covered.
But her point was essentially the opposite: the mapping document itself can be incomplete.
A field can be missing.
A business rule can change without the document being updated.
A requirement might never have made it into the document in the first place.
So the QE needs to check the business requirements first, and then ask whether the mapping document actually reflects them, rather than treating the mapping document as the source of truth on its own.
That gives you a much sharper interview question than:
“Have you worked with data mapping?”
Ask instead:
“How do you make sure a mapping reflects the actual business requirements during a migration?”
A strong QE should be able to walk you through the process, not just confirm they’ve seen a document.
She also talked about data fallout, records that simply don’t make it through the process.
If you migrate 100,000 customer records and only 99,500 arrive on the other side, those missing 500 are an example of data fallout.
The QE needs to figure out what didn’t move, why, whether the problem is isolated or widespread, and whether the business can actually function with what’s missing.
If a candidate says they have migration experience, ask:
“How did you identify records that failed to migrate from one system to another?”
You’re not asking them to diagram the whole architecture. You’re giving them a chance to show they understand data completeness, not just whether the migration technically ran.
The most technical, and probably the best, example she gave was transformation errors, where the data arrives but means something different than it should.
Imagine the source system stores a true/false value as 1 and 0, while the new system is supposed to display those values as Yes and No.
The transformation should be:
1 → Yes
0 → No
But what happens if the transformation rule is wrong?
You could end up with:
1 → No
The data moved.
The field exists.
Nothing crashed.
But the meaning of the data is wrong.
That’s a quality problem.
It’s why integration testing isn’t just:
“Did the data get from A to B?”
It’s also:
“Did the data arrive with the correct meaning?”
And that’s why she described this specific category of error as requiring very close, detailed review rather than a quick pass.
It’s also a good reminder that someone who’s worked on “a data migration” hasn’t necessarily worked on the quality side of one.
What This Means for Recruiters
There’s a larger lesson running through all of this: modern QE isn’t simply about knowing a collection of tools. The quicker you understand this, the better recruiter you’ll become.
It’s increasingly about understanding systems, requirements, data, and risk well enough to use those tools effectively and, increasingly, well enough to evaluate the code and automation those tools produce.
AI can help a QE:
- Generate test cases
- Write queries
- Create automation
- Analyze information
- Validate data
- Build internal agents
But the engineer still has to understand what they’re testing, whether the result is correct, and, more and more, whether the automation code AI just handed them actually does what it’s supposed to.
An integration QE, meanwhile, shouldn’t just know that two systems exchange data. They should understand the specific ways that exchange can break down:
- Missing records
- Incorrect mappings
- Incorrect transformations
- Requirements that never made it into the mapping document
- Data that arrives successfully but carries the wrong meaning
So instead of screening for whether someone has used Claude, ask how they use AI during testing.
Instead of asking whether they have data migration experience, ask what kinds of problems they had to validate during the migration.
Instead of asking whether they’ve worked with integrations, ask what can go wrong between two systems even when both are working individually.
And increasingly, ask what happens when the automation script AI wrote for them turns out to be wrong.
Those questions get you closer to the engineer’s actual level of understanding.
And that’s ultimately the goal of better recruiting: moving from checking whether someone has touched a technology (check the box mentality) to understanding whether they know how to use it to solve quality problems.