AI is changing how quality engineers work, and as a recruiter, I wanted to understand what that looks like in practice, beyond the buzzwords.
I had a great conversation with Srilata, a Principal QA Engineer, about how she uses AI in testing, where integration and data projects create quality risks, and how the QE role is shifting toward engineering. She helped to turn this conversation into the article below, written from the engineering perspective.
Read it first, then scroll to the end, where I’ve pulled out what I took away as a recruiter. 👇
AI Is Changing Quality Engineering, But Engineering Judgment Still Matters
AI is changing how Quality Engineers work. I think we all know that by now.
But from my perspective as a Principal QA Engineer, the more important question isn’t whether we’re using AI. It’s how effectively we use it without compromising quality, accuracy, security, or engineering judgment.
AI can generate test cases, write SQL, create automation scripts, analyze data, generate documentation, and even help build internal knowledge agents. These capabilities can significantly improve productivity.
But AI doesn’t replace the responsibility of the Quality Engineer.
It changes where our expertise needs to be applied.
The QE still needs to understand the requirements, business rules, architecture, data, integrations, risks, and expected behavior well enough to determine whether the AI-generated output is actually correct.
That human-in-the-loop mindset is becoming increasingly important.
1. AI Isn’t Just About Generating Test Cases
One of the biggest misconceptions about AI in testing is that its value is simply generating test cases.
In reality, AI can support multiple areas of the quality engineering lifecycle.
I’ve used AI to help generate and troubleshoot SQL and SOQL queries, analyze requirements, identify potential test scenarios, assist with documentation, and accelerate problem-solving.
For example, a QE may ask AI to generate a SQL query to identify customers who have been active within the last 30 days.
The productivity gain is obvious.
But the real engineering question is:
Does the QE understand what the query is supposed to accomplish, and can they validate that the generated query produces the correct result?
That’s where engineering expertise matters.
Using AI to generate a query is one thing.
Understanding the data model, joins, filters, business rules, expected results, and validating the output is another.
AI can generate the work. The QE remains accountable for the quality of the work.
The same applies to data validation.
Consider a data migration from a legacy system into Salesforce.
It’s not enough to confirm that the migration completed successfully.
We need to validate:
- Was every required record migrated?
- Were fields mapped correctly?
- Were business rules preserved?
- Were transformations accurate?
- Was any data lost?
- Were records duplicated?
- Did data land in the correct destination?
- Does the migrated data still have the correct business meaning?
Tools such as QuerySurge can be extremely useful for comparing data across systems. But when a dedicated tool isn’t available, AI can also assist with developing queries, comparison logic, and validation approaches.
Again, the tool is not the quality strategy.
The engineer’s understanding of what needs to be validated is the quality strategy.
2. AI-Generated Test Cases Still Need Engineering Review
Another area where AI provides significant value is test case generation.
For example, given an acceptance criterion such as:
A customer should be able to reset their password using their registered email address.
AI can quickly generate:
Positive scenarios
- Valid registered email
- Reset email successfully generated
- Password successfully reset
Negative scenarios
- Unregistered email
- Invalid email format
- Expired reset link
Edge cases
- Multiple reset requests
- Extremely long input
- Expired token
- Email service unavailable
- Repeated submissions
That’s valuable.
But AI can also misunderstand requirements, overlook critical business rules, create redundant scenarios, or generate tests that sound technically correct but don’t actually validate the intended behavior.
That’s why I don’t look at AI-generated test cases as finished test cases.
I look at them as a starting point for engineering analysis.
My approach is:
Generate → Review → Challenge → Refine → Validate → Execute
The QE needs to compare the generated scenarios against the acceptance criteria, requirements, business rules, architecture, and risk areas.
The most important question isn’t:
“Did AI generate the test cases?”
It’s:
“Did the QE identify what AI missed?”
That is where experience makes a difference.
3. The Right AI Tool Depends on the Problem
I also don’t believe in using one AI tool for every problem.
Different tools can be useful for different activities, whether that’s ChatGPT, Claude, Copilot, NotebookLM, or specialized enterprise AI capabilities.
The important skill isn’t memorizing AI tool names.
It’s understanding:
What problem am I trying to solve, and which tool is appropriate for that problem?
For example, different AI tools can serve different purposes, ChatGPT/Claude for generating and refining test scenarios, Codex for code assistance and automation, Microsoft Copilot agents for analyzing and summarizing large sets of documentation, and ROVO for creating a more structured onboarding experience for new QA hires.
The technology is impressive.
But governance and validation are equally important.
We need to understand:
- What data is the AI accessing?
- What sources is it using?
- How current is the information?
- How do we validate its responses?
- What information should never be exposed to the model?
- How do we prevent incorrect information from becoming institutional knowledge?
That is where Quality Engineering extends beyond traditional functional testing.
4. QE Is Becoming More Development-Adjacent
I believe this is one of the most significant changes happening in Quality Engineering.
As AI takes over more repetitive activities, such as generating test cases, writing initial queries, creating documentation, and producing automation code, the QE role is becoming increasingly development-adjacent.
Automation is becoming even more important, particularly for teams working with fast release cycles and limited resources.
AI can generate an automation script in seconds.
But generating the script is not the end of the job.
The QE needs to be able to:
- Read the code
- Understand the framework
- Validate the logic
- Identify incorrect assumptions
- Modify the implementation
- Assess maintainability
- Validate test coverage
- Determine whether the automation actually tests the intended behavior
This creates a new expectation for modern Quality Engineers.
You don’t necessarily need to write every line of code manually, but you need enough engineering depth to understand, challenge, modify, and validate the code AI produces.
If an AI-generated automation script looks perfect but validates the wrong business behavior, it’s still a failed test.
5. Integration Testing: When Everything Works, But the Data Is Wrong
Another area that continues to require deep QE expertise is integration and data quality.
Consider a typical enterprise flow:
Source System → Middleware → Salesforce → Data Warehouse → Reporting
Every individual system can be functioning correctly.
And yet the overall business process can still fail.
That’s the nature of integration testing.
Middleware may transform, enrich, filter, or route information between systems. Every one of those steps introduces potential quality risks.
One principle I strongly believe in is:
Don’t assume the mapping document is correct simply because it exists.
A mapping document may say:
| Source | Target |
|---|---|
| Customer_Name | Account_Name |
| Customer_Email | |
| Status | Customer_Status |
But the mapping document itself needs to be validated against the actual business requirements.
A field could be missing.
A business rule could have changed.
A requirement could have been misunderstood.
Or a requirement may never have made it into the mapping document.
Therefore, the QE shouldn’t treat the mapping document as the ultimate source of truth.
The QE should validate:
Business Requirements → Mapping → Transformation Logic → Target Data
That end-to-end traceability is critical.
6. Data Fallout Is a Quality Risk
Data migration provides another great example.
Suppose 100,000 customer records are migrated and only 99,500 arrive in the target system.
The migration technically completed.
But 500 records are missing.
That’s data fallout.
The QE needs to determine:
- Which records failed?
- Why did they fail?
- Is the issue isolated or systemic?
- Were specific data conditions responsible?
- Can the records be reprocessed?
- What is the business impact?
- Are there downstream impacts?
This is why simply saying:
“I’ve worked on data migration.”
doesn’t tell me very much about someone’s actual QE experience.
The more meaningful question is:
“How did you validate data completeness and identify records that failed to migrate?”
7. Data Transformation Can Be Even More Dangerous
One of the most subtle integration defects occurs when data successfully moves from one system to another, but its meaning changes.
For example:
Source system:
1 → True
0 → False
Target system:
1 → Yes
0 → No
Now imagine the transformation rule is implemented incorrectly:
1 → No
Nothing crashed.
The record migrated.
The field exists.
The integration completed successfully.
But the business meaning is wrong.
That is a quality defect.
This is why integration testing isn’t simply:
“Did the data move from A to B?”
It is:
“Did the data move from A to B completely, accurately, and with the correct business meaning?”
That distinction is fundamental to effective Quality Engineering.
8. The Future of QE: Less Checkbox, More Engineering
The evolution of AI doesn’t make Quality Engineering less important.
In many ways, it raises the bar.
The role is moving from executing predefined test cases toward understanding and managing quality risks across the entire technology ecosystem.
A modern QE needs to understand:
Requirements
What are we actually supposed to build?
Systems
How do the components interact?
Data
Is the information complete, accurate, and meaningful?
Integrations
What can break between systems?
Automation
Are we testing the right things efficiently?
AI
Is the output accurate, explainable, secure, and fit for purpose?
Risk
What happens when something goes wrong?
AI can help us accelerate all of these activities.
But it doesn’t remove the need for engineering judgment.
If anything, it makes that judgment more important.
The Human-in-the-Loop Principle
For me, this is the central theme.
AI can:
- Generate test cases
- Write SQL/SOQL
- Create automation
- Analyze information
- Compare data
- Generate documentation
- Assist with troubleshooting
- Build knowledge agents
But the Quality Engineer remains responsible for determining whether the result is correct.
The future QE isn’t simply someone who knows how to use AI.
It’s someone who knows when to use it, how to use it, how to challenge its output, and how to validate the result against requirements, business rules, data, architecture, and risk.
That’s the difference between using AI as a productivity tool and using AI as an engineering capability.
And as a Principal QA Engineer, that’s the direction I believe Quality Engineering is heading:
From test execution to quality engineering.
From tool knowledge to engineering judgment.
From checkbox validation to risk-based thinking.
And from simply testing software to engineering quality across the entire ecosystem.
What I took away as a recruiter
Thank you, Srilata, for breaking this down. Here’s how I’ll apply it when screening QE talent.
Tech and concepts worth knowing:
- AI tools: ChatGPT, Claude, Copilot, NotebookLM, Codex, ROVO
- Data and queries: SQL, SOQL, Salesforce, QuerySurge
- Integration: middleware, mapping documents, data migration, data warehouse and reporting
- Automation: AI-generated scripts that the engineer can read, challenge, and fix
Questions that go beyond the checkbox:
- “How have you used AI for SQL or data validation, and how did you verify the output was correct?”
- “Can you give me an example of something AI generated that you had to correct?”
- “How do you decide which AI tool to use for a given task?”
- “How do you make sure a mapping document reflects the actual business requirements?”
- “On a migration, how did you identify records that failed to move, and what did you do about them?”
- “What can go wrong between two systems even when both are working individually?”
- “What data should never be exposed to an AI model, and how do you validate what it tells you?”
The pattern: don’t ask whether someone has used a tool. Ask how they know the output is right.