I’m a tech recruiter, not an engineer.
And sometimes I come across QE projects that sounds impressive, but I have to stop and ask myself:
Okay… what the heck did they actually build? Why does it matter? How can I use this info to sell my client on why they should interview this person?
And that basically what happened when I came across a QE build from Kim.
At first glance, it was a lot of technical language: API schemas, contract guards, CI-gated evaluations, Git commits, architecture decision records.
Easy enough to read right past it. But once I broke it down, I realized there was a lot here that recruiters should understand because this isn’t simply someone who knows how to write automated tests.
So, what did they actually build?
At a high level, she built a plugin for Claude Code, a collection of skills that can be used individually or orchestrated together, each one pointing you to whichever other skill or tool actually fits the step you’re on. Or as I like to call it, “a QA toolkit.”
Worth noting: the toolkit’s biggest-impact skill is audit-test, it works by making a small deliberate change to the code (a mutation) and checking whether the test still catches it, so you’re not left with a hollow, false-positive “green.” But one of the major piece I want to talk about is called contract-guard.
The easiest way to understand that is to first understand an API contract.
An API is essentially a way for different pieces of software to communicate with each other.
The API contract defines what that communication is supposed to look like.
For example, an application might expect an API to return:
customer_name = "John"
customer_age = 42
If someone changes customer_age from a number to a string, or removes a field entirely, another application depending on that API could break.
Kim built this tool to solve a scenario she sees often: tests that were working fine yesterday, or a month ago, suddenly start “breaking”, with no code changes involved. That kind of failure makes people lose confidence in automation fast, because it looks like the tests themselves are unreliable. Instead of digging through it manually every time, contract-guard does the digging for you.
So contract-guard evaluates whether the existing tests are capable of detecting different kinds of API contract changes.
And she didn’t just build it and say, “Yep, it works.” She tested it against 1,152 real, public API schemas.
The results were interesting.
It detected changed field types 98.6% of the time when they were marked as required. (“Required” is a flag set directly in the API schema).
But when a field was deleted or renamed and that field wasn’t marked as required, it only detected the change about 25% of the time. Kim reminded me that the skill is only as good as the API schema it has to compare to.
That’s actually one of the things I find most interesting about the project.
Kim didn’t just measure whether the tool “worked.”
She measured what kinds of problems it could find and where it had limitations.
The tool now reports those capabilities on a field-by-field basis.
There’s another piece I found interesting
They also added a check that compares the Git commit referenced by a QA report against the commit that actually ran.
Why does that matter?
Imagine someone gives you a test report saying:
“All tests passed.”
Sounds good.
But what if that report was generated from an older version of the code?
You might think you’re looking at evidence that the current code passed testing when you’re actually looking at evidence from something that ran yesterday.
The Git verification helps prevent that.
In other words, they’re not just asking:
“Did the tests pass?”
They’re also asking:
“Can I trust the evidence telling me that the tests passed?”
That’s a very different level of thinking.
Why should recruiters care about any of this?
You don’t need to become an engineer.
You don’t need to understand every line of code.
But if you recruit Quality Engineering professionals, understanding projects like this helps you recognize the difference between someone who can execute testing and someone who thinks deeply about quality.
A resume might say:
- API testing
- Test automation
- CI/CD
- Playwright
- Claude
- Python
- QA architecture
Those keywords don’t tell you much by themselves.
But when someone can show you that they built a system that evaluates whether tests can actually detect API contract changes, measured its effectiveness against more than 1,000 schemas, identified gaps, and then improved the reporting?
Now you have something much more meaningful to talk about.
That’s where I think recruiters can add value.
We don’t have to be engineers but we should be able to understand what problem an engineer solved, how they approached it, and why the work matters.
4 skills I would take away from this project
1. API Testing & Contract Testing
Kim clearly understands that APIs have contracts and that changes to those contracts can break systems that depend on them.
Recruiter question:
“Can you explain what an API contract is and what your tool is checking when that contract changes?”
You don’t need to know the perfect technical answer.
You’re listening for whether the candidate can explain a complicated concept clearly and whether they actually understand the problem they’re solving.
2. Test Automation & QA Architecture
This wasn’t just a collection of individual automated tests.
She built a reusable system with multiple QA skills, a router, CI-gated evaluations, and supporting documentation around the architecture.
That’s a different level of automation thinking.
Recruiter question:
“What problem were you trying to solve by building this as a reusable QA system instead of just writing individual tests?”
That question gets you into architecture and engineering thinking.
3. Quality Engineering & Test Strategy
This might be the biggest takeaway for me.
The whole idea behind QA Skills came from something Kim said: the bottleneck in AI-assisted development has shifted to QA, to testing what AI actually outputs. Humans simply can’t review all AI-generated code. So these skills were built as a judgment layer, helping narrow down which tests need a human eye and which are safe enough to trust, especially given how complicated it is to trust the same AI that wrote the code to also grade its own work. That mindset shows up in how she approached testing the tool itself. She didn’t stop at: “The test passed.” She asked: “What can this test actually detect?”
Recruiter question:
“You found that your tool detected changed field types 98.6% of the time but deleted or renamed optional fields only about 25% of the time. What did you learn from that?”
Now you’re getting into how the candidate thinks about gaps, limitations, and improving a testing system.
4. CI/CD & Test Evidence
The Git commit validation is another signal.
She’s thinking about whether test results are trustworthy, not simply whether tests are green.
That’s an important distinction in modern engineering environments.
Recruiter question:
“Why does verifying the Git commit behind a test report matter, and what problem were you trying to prevent?”
Again, you don’t need to know the implementation.
You’re trying to understand whether the candidate understands why the control exists.
You don’t have to be technical to recruit technical people.
I won’t lie…it helps. But I do think we can be curious enough to understand what we’re actually looking at and connect the dots.
When I see a QA resume that says: “Built automated API testing framework.”
I don’t want to immediately check the API testing box. I want to know:
What did you build?
What problem did it solve?
How did you know it worked?
What did it fail to catch?
What did you learn?
Those questions will tell you a heck of a lot more about an engineer than a list of technologies ever will.
And the more we understand the work, the better questions we can ask.
A big nod to Kim for the education!