I’ve been sitting in calls with engineering leaders alongside my engineering manager, Ryan, talking about business initiatives, quality engineering, automation strategy, and the problems companies are trying to solve.
Companies are trying to move faster. They’re looking for ways to reduce costs, eliminate bottlenecks, push testing earlier in the development process, adopt AI, and modernize their automation stacks.
All of those things make sense.
But I kept hearing about companies that had cut their QA organizations down significantly, pushed more testing onto developers, and eventually came back (to us) looking for help rebuilding their quality engineering capabilities.
At the same time, I’m hearing from companies struggling through migrations to Playwright. And then there’s AI, which can generate a test in seconds but doesn’t necessarily know whether it’s testing the right thing.
So I finally asked Ryan to help me understand what’s going on.
I asked him three questions:
What are companies struggling with when they move to Playwright?
What did companies underestimate when they reduced their QA organizations?
And what are we seeing in candidates who don’t make it through our technical assessments?
They all came back to the same thing: expertise still matters.
The Playwright problem isn’t always Playwright
One of the questions I wanted to understand was why so many companies are coming to us for help transitioning to Playwright.
If Playwright is supposed to make modern test automation easier, what’s making the transition so difficult?
Ryan’s answer wasn’t simply, “People don’t know Playwright.”
A lot of these companies have engineers who know their existing frameworks extremely well. They may have years of experience with Selenium, Appium, Cucumber, or other legacy tooling.
The problem is that companies don’t necessarily want to stop and retrain everyone from scratch.
They want to take what they’ve already built and move it into a newer framework.
And that’s where things get complicated.
Legacy automation frameworks can create a lot of maintenance overhead. Flaky tests fail. Developers have to investigate the failure. Someone discovers the locator was bad. The test gets fixed. Then it happens again. Now you’re spending engineering time figuring out whether a failed test represents an actual product problem or a problem with the test itself.
Part of what makes Playwright attractive is that more functionality comes out of the box, configuration is simpler, and cross-browser testing is easier to manage.
But moving to a new tool doesn’t automatically solve the underlying engineering problem.
You still need people who understand what they’re testing, why they’re testing it, how the suite should be structured, and how to recognize when a test isn’t actually giving you meaningful confidence.
What happens when you remove the check?
This was the question I was most curious about.
We’ve seen many companies reduce their QA organization and eventually come to us needing help building a Quality Engineering Center of Excellence.
So I asked Ryan: What did they underestimate?
His answer was essentially specialization.
Developers are not automatically testers just because they can write code. That doesn’t mean developers shouldn’t test. They absolutely should. In fact, Ryan’s recommendation wasn’t to put QA back into a silo and create a giant handoff between development and testing.
Quite the opposite.
He wants developers and quality engineers working together as one product team, with QE having access to the development codebase and developers having visibility into the automation.
The goal isn’t to create a wall between development and QA.
It’s to eliminate the wall while still keeping the expertise.
Ryan gave me an analogy I really liked: If you go to a doctor with a heart problem, you don’t assume your doctor can simply switch specialties and perform the same job as a lung specialist because they went to medical school.
There is foundational knowledge they share, but specialization exists for a reason.
It’s similar in engineering.
Developers absolutely should test. But writing software and testing software require overlapping, not identical, expertise.
A developer can write a test that passes while still allowing a bug into production. A quality engineer is trained to think differently about how the product can fail, what hasn’t been tested, and whether the test actually proves what you think it proves.
So when a company removes that specialized expertise, it hasn’t necessarily eliminated the need for it.
It has just moved the responsibility somewhere else.
And sometimes that “somewhere else” is the customer.
That’s when the cost calculation changes.
Finding a bug internally is one thing. Having your customer find it, open a support ticket, escalate it to the business, and eventually send it back to engineering is another.
You may have saved money on the QA organization. But you haven’t necessarily saved money on quality.
AI doesn’t eliminate the need for testing expertise either
This came up when I asked Ryan about candidates taking our technical assessments.
This is where my recruiter brain gets conflicted.
I talk to a lot of really good engineers. I want to help people get through the process. And our technical assessments can be tough.
I always though “Ryan is pretty picky about what he wants to see”.
But then I look at our engineering team and realize something: We don’t lose people very often because we hired the wrong technical fit.
So maybe the pickiness is doing something. I asked him what he sees when candidates fail.
Two things stood out.
The first is AI.
Candidates are increasingly feeding the requirements into AI, letting it generate the test, running it, seeing that it passes, and submitting it.
The problem?
They aren’t necessarily looking at what the AI actually produced. Ryan’s expectation is that you use AI, but you still need to watch it, sanity-check it, and understand what it’s doing. Because a test can pass and still be a bad test.
That leads to the second problem: basic testing judgment.
Here’s the kind of example that makes this obvious: Imagine you’re testing for a specific piece of text on a page. The text exists twice. Your automation finds the text. The test passes. But did you actually test the right section of the page?
Maybe not.
The test technically passed. But it didn’t prove what you thought it proved.
That’s not a Playwright problem. That’s not an AI problem. That’s a testing problem.
The answer isn’t “less QA”. It’s better integration.
I don’t think the lesson from these conversations is that every company needs to rebuild a traditional QA organization.
That’s not what Ryan is advocating either.
The more interesting answer is integration.
Developers should be testing. Quality engineers should understand development. QE should have access to the codebase. Automation shouldn’t live in a silo. AI should absolutely be part of the workflow. And companies should modernize their tooling.
But none of those things eliminate the need for people who understand quality engineering.
Maybe that’s the check and balance.
Not development versus QA. Not manual versus automation. Not humans versus AI.
It’s having enough different expertise on the team to challenge the assumptions everyone else is making.
I’ve spent a lot of my career looking at this from the recruiting side: Can we find the person with the right skills?
Sitting in these engineering leadership conversations has made me ask a different question:
What happens when an organization decides it doesn’t need that skill anymore?
Sometimes you don’t find out what the expertise was worth until you remove it.