I spoke with Jay, a Senior Quality Engineer who has spent years working across QA automation, SDET, DevOps, and multiple generations of automation frameworks.
One thing that stood out to me was how differently he thinks about the SDET role compared to how these positions are often written and screened.
A lot of SDET hiring still comes down to a checklist: Selenium, Cypress, Playwright, Java, C#, API testing, CI/CD. Those things matter, but Jay’s perspective was that the strongest engineers are thinking beyond the individual tools. They understand the framework underneath them, how the pieces of a codebase fit together, how to evaluate a new technology, and how to leave the system in a state that another engineer can actually work with.
That distinction becomes even more important as AI gets incorporated into the workflow.
AI Isn’t Just For Writing Tests
Jay uses tools like Cursor and GitHub Copilot as part of his day-to-day engineering work. He described Cursor to me as essentially “Google on steroids,” but the interesting part wasn’t simply that he uses it to generate code.
He uses AI to interrogate the existing codebase.
If he’s working on a new class, interface, or test, he can ask the tool to help him understand what already exists, locate relevant methods, find how different pieces of the framework connect, and investigate whether there is already something he can reuse.
That can eliminate a lot of the manual searching engineers have historically had to do across repositories and files.
He also uses AI when looking at framework design and technical debt. If an organization has accumulated multiple repositories or overlapping automation frameworks, AI can help him investigate what’s actually there and identify duplication or opportunities to consolidate.
That’s a very different use case from simply asking AI to “write me a Playwright test.”
And I think that’s an important when evaluating AI experience in a QE candidate.
A candidate telling you they use AI to generate tests doesn’t tell you much by itself.
A better question might be:
“Tell me about a time you used AI to understand or navigate an existing automation framework.”
The answer can tell you whether they’re using AI as a shortcut for thinking or as a tool to accelerate the thinking they’re already capable of doing.
The Engineer Still Has to Know What Good Looks Like
Jay made an important point about AI-generated tests: just because the code runs doesn’t mean the test is good.
When AI creates or modifies something, he still needs to understand where the code connects to the rest of the framework. Is the appropriate class being called? Does the method already exist? Are the necessary interfaces there? Are there edge cases that should be covered?
He can use AI to investigate those questions, but he’s still responsible for deciding whether the answer makes sense.
That gets at one of the biggest problems with evaluating AI skills right now: There is a difference between using AI to produce code and using AI effectively as an engineer.
Jay brought up the growing problem of “vibe coding,” where someone can get AI to produce something that looks technically convincing without actually understanding what it does. I personally have seen this happen to my recruiting peers looking for quality talent.
And I can see that being a problem in QE because a test that executes successfully can still be testing the wrong thing.
For recruiters, I’d be much more interested in questions like:
“When AI generates an automated test for you, how do you validate that it’s actually testing the intended behavior?”
Or:
“How do you use AI when you’re working in an unfamiliar codebase?”
Those questions get closer to engineering judgment than simply asking whether someone has used Copilot or Cursor.
The SDET Isn’t Just the Person Who Knows the Automation Tool
Jay’s description of the SDET role was probably my biggest takeaway from the conversation.
He sees a strong SDET as someone who needs to be highly technical, basically operating at a level comparable to their development counterparts, while also being able to explain technical concepts in plain language and pivot when the situation changes.
Jay talked about being comfortable with different technologies rather than becoming overly attached to one particular tool. He gave the example of introducing Cypress and having his assumptions challenged along the way. The point wasn’t that Cypress was automatically better or worse. The point was being willing to learn something new, evaluate it, and change your approach.
That’s different from hiring someone because they are a “Playwright expert.”
If someone has spent five years with Selenium but understands automation architecture, design patterns, CI/CD, test strategy, and how to evaluate new technology, I wouldn’t automatically dismiss them because Playwright isn’t prominent on their resume.
I’d want to know how they approached the transition.
A much more revealing question would be:
“You’ve worked with multiple automation frameworks. When you moved from one to another, what did you have to rethink versus what stayed fundamentally the same?”
That answer can tell you a lot about whether you’re talking to someone who understands the underlying engineering or someone who primarily knows the syntax of a particular tool.
A Lot of Senior QE Work Is Actually Cleanup
Another part of our conversation that I found interesting was his discussion of legacy automation.
When people hear “automation framework,” it’s easy to picture an engineer starting from scratch and building something modern. That’s personally where my mind first goes.
But in reality, senior QE’s often inherit years of accumulated debt. There may be multiple repositories, multiple frameworks, duplicated functionality, stale test cases, and old code that nobody is quite sure why they still have.
Jay uses AI to help interrogate those systems and understand what is actually there.
That matters because one of the most valuable things a senior QE can do isn’t necessarily adding another 500 automated tests. It can be figuring out why the organization has several different systems doing essentially the same job and determining what can be consolidated, removed, or modernized.
That’s a very different skill set than simply knowing how to write an automated test.
For recruiters, I’d ask:
“Have you ever inherited an automation framework? What did you look at first?”
And then:
“How did you decide what to keep, replace, consolidate, or remove?”
Don’t Underestimate the Planning
We also talked about something that tends to get lost when teams are focused on delivery: the analysis and configuration work that happens before an engineer starts building.
From the outside, that work can look like nothing is happening.
But understanding the existing architecture, determining how the framework should be structured, figuring out what can be reused, and establishing patterns that other engineers can follow can save significant time later.
Jay referred to this as “human portability.” The idea is that the framework shouldn’t only make sense to the person who built it. If another engineer joins the team, they should be able to understand the patterns, find the relevant code, and make changes without having to track down the original developer for an explanation of how everything works.
That also connects back to AI.
The better structured and documented the codebase is, the easier it becomes for AI tools to interrogate and work with that codebase effectively.
So the goal isn’t simply “use AI.” It’s building engineering systems where both humans and AI can understand what is happening.
The Recruiting Takeaway
The more I talk with experienced QE engineers, the less useful I find the traditional tool checklist as a proxy for seniority.
Playwright experience matters. So does Selenium. So does Cypress. So do programming languages and CI/CD tools.
But those are pieces of the job. The bigger questions are how someone thinks about automation, how they approach unfamiliar systems, how they evaluate technology, how they deal with technical debt, and whether they can use tools like AI without outsourcing their engineering judgment to them.
Thanks to Jay for sharing his perspective and giving me a few things to think about as I continue learning more about modern QE.