AI feels like a magic genie.

It writes test cases. It drafts strategies. It even proposes entire automation frameworks.

It feels like a superpower.

But the truth is that AI doesn't think like you. It doesn't know your product. It doesn't care about your users, products, or quality goals. It cannot weigh your risks or context.

If you outsource your judgment, you lose credibility. You create blind spots that damage quality.

An AI tool could draft a professional-looking test plan for a login feature, and everything might look perfect.  Until production, when users hit "forgot password" at scale, and the whole system collapses.

Why? The AI never thought about load, and nobody stopped to question it. That's how outsourcing your judgment quietly plants a blind spot that later explodes as a quality failure.

The answer isn't to avoid AI. The answer is to use it with control, transparency, and your brain fully engaged.

In this article, we'll cover six key tactics that will help you maintain control over quality while reaping AI's benefits.

Document AI's Role

Imagine you're reading a flawless test plan only to find out that it was mostly written with AI. Would you trust it the same way? Probably not.

When you hide AI involvement, you lose trust. Your stakeholders start doubting your strategies, test cases, and testing reports. They wouldn’t be able to trust your work for any decision-making.

If you use AI to draft, refine, or format your work, be open about it. Document it the way you'd cite a source.

To do this, start by identifying how you're using it. Different testers use AI for different categories of work, such as:

• Generation: Drafting test cases or plans
• Improvement: Refining your existing work artifacts
• Suggestion: Offering ideas you hadn't considered
• Correction: Cleaning grammar or fixing layout

You can document such usage modes and detailed citations in multiple locations, such as strategy documents, acknowledgment sections, or footers of testing artifacts (presentations, documents, etc.).

If team members can't trace which parts were AI-generated, they may skip critical reviews, leading to undetected gaps in coverage. A simple line such as "This test plan was refined with Microsoft Copilot" can make your work transparent.

You can also track how much time it saved. Later, that record tells you whether AI truly added value or just gave you cleanup work.

Remember, transparency builds trust; secrecy breaks it. Transparency doesn't weaken you. It strengthens you.

Use AI in Ask Mode Not Agent Mode

Most modern generative AI-based tools work in two modes: Ask mode and agent mode.

Ask mode means you are the driver of the overall AI experience. You give AI a specific, clear, and small task: "Rephrase this email", "Suggest edge cases for a login feature", "Format a set of test scenarios into a table", and it executes only that task, nothing more.

Agent mode, on the other hand, means that you hand over complete control. The AI is free to drive the entire task. For example: "Create a full automation framework for Playwright,” and the AI makes all the decisions.

Agent mode appears efficient at first. But it comes at a cost.

You inherit complexity without the context. You don't own the decisions. Later, when the code breaks, you're stuck in maintenance hell. Many test suites reach a state where fixing one area breaks another, requiring more time to debug this complexity instead of actually testing the product.

Ask mode is safer. It lets AI assist you in your work without stealing your judgment and control.

Delegating full control creates brittle frameworks that collapse under real-world changes, damaging reliability.

So the bottom line is: augment (by asking), don't delegate fully (to an agent).

Practice Critical Evaluation of AI Outputs

AI writes with polish. Sometimes, it adds too much.

Smooth words can trick you into believing that they're correct. But as testers, our job is not to agree with everything we receive; we question.

If testers accept polished outputs without questioning, bugs slip through disguised as “good” coverage.

Here are three quick questions you can ask yourself while reviewing AI-generated outputs:

• "Huh?" - Pause and check: Does this make sense for your context?
• "Really?" - Challenge it: It might make sense in your context, but how can you verify it as a tester?
• "So?" - Reflect on it: So what? Does this matter for your current testing mission?

Use these questions as a checklist. They help you pause, notice gaps, and judge whether the output is truly useful.

Once you are done with the above questions, look for:

• Assumptions: What assumptions did AI make about my system?
• References: What sources did it cite, and are they real and relevant to the context?
• LLM Syndromes: Are there signs of hallucinations?

As a final step, cross-check the AI output against project references such as project requirements, testing logs, user reports, and actual code behaviour.

Pro tip: Always read AI's output twice. The second read often reveals flaws the first pass might have missed. AI polish is like a soft illusion. Don't let it fool you.

By doing this, you end up with AI-assisted outputs that aren't just polished words, but artifacts you can trust for your project and safely use for decision-making.

Spot Low-quality AI Content in Testing Artifacts

The four classes of AI assistance in content are:

• Purely AI-generated (e.g, AI drafts a full test plan)
• AI-generated and AI-refined (e.g, AI drafts test cases and polishes them as per a set of instructions)
• Human-generated and AI-refined (e.g, you write test ideas or scenarios and AI formats them)
• Purely human-written

The real danger is in the first two classes; they lack human judgment and input.

Weak AI content in testing artifacts such as test documentation, requirement documents, test strategy, and presentations often shows symptoms like:

• Over-generalization: AI often gives vague claims and general statements. On the other hand, testers focus on precision.
• Patterned words: AI writing is usually very predictable. On the contrary, testers are pattern breakers and convey what's not obvious.
• Lack of real-world experience: This is the biggest issue with AI. It lacks real-world experience and experimentation. Testers, on the other hand, work from real-world experience, context, and history.
• Over-perfection: AI over-polishes its content. This is different from humans, as humans often make subtle mistakes, typos, and omissions.
• Tone or style mismatch: Every tester has a unique voice and style. AI produces uniform, monotonous content. This creates a big tone and style mismatch for potential readers who know you in the real world.

When you see these signs, don't ship the work as-is. Rewrite it, add your unique insights. AI can assist, but it should never erase your style.

Rule of thumb: If your peers can't tell it's you, AI has stolen too much space in your work and may steal your reputation soon.

Use Context Engineering Tactics

AI outputs are only as good as the input you give it. Garbage in, garbage out.

Generic prompts? Generic answers.

Instead, give it specifics about your context: Purpose, personas, constraints, business goals, testing mission, etc.

Here are two tactics that you can use to inject more context into AI-assisted work:

• Ask AI to question you before answering. This reveals gaps.

  Start by appending this to your regular prompts:

  *Before you begin, ask me a few questions that you think will help you create the best output possible.*

• Ask AI to critique its own output. This turns the AI into a reviewer.

  You can ask this once you receive your AI-generated output:

  *Critically evaluate the above answer as an expert reviewer and list down the potential limitations in this answer.*

Without context, AI suggestions may ignore critical risks, leaving the product exposed. With context, AI shifts from a generic assistant into a real co-pilot.

Educate Peers And Build Risk Awareness

Most testers often over-trust AI because it sounds smart and authoritative. That overconfidence is a risk, and it's important to talk about such risks, as our job is about safeguarding trust.

Show them how over-reliance on AI can:

• Harm your credibility and reputation
• Erase your unique voice and style as everyone adopts the same AI phrasing
• Create false confidence in your strategies
• Reduce the quality of your overall product and testing work

As testers, our job isn't only to find bugs, it's to protect user trust. So, educate your peers. Show them the risks; don't keep this wisdom to yourself.

If teams normalize uncritical AI use, poor-quality practices scale across projects, amplifying risks instead of reducing them.

Over-trust will damage credibility. It will dull your judgment.

AI in testing isn't just about tools; it's also about culture.

The testers who evaluate where AI fits, challenge its outputs, and use their own judgment will shape high-quality products.

The future of testing won't be decided by who uses AI the most. It belongs to the testers who set the tone for how it's used wisely, transparently, and in service of quality.

References:

• Is There Something in Your "I"? - Satisfice, Inc.
• A Super-Quick Guide to Evaluating "AI" Claims – DevelopSense
• The fine print of using AI tools in testing - AI Disclaimer
• Appendix: ChatGPT Sucks at Being a Testing Expert - Satisfice, Inc.
• The 70% problem: Hard truths about AI-assisted coding