Testing With GenAI

● Structure of prompts and why this matters.● Core prompting techniques ● System, user and meta prompts: best practices● Test Analysis with Gen AI ● Test Design and implementation with Gen AI 

● Automated regression: nuances and rules ● Techniques and metrics to evaluate results of GenAI test tasks. Quality gates. Problem of non-determinism. Hallucinations, Reasoning errors and Biases in LLM output● Mitigation of defects in LLM output ● Data Privacy and security risks ● Environmental Impact and AI regulations and standards

● LLM-powered test infrastructure: architecture and key components. ● Retrieval-Augmented Generation.● LLM-powered agents and Fine-tuning techniques● LLMOps

● GenAI strategy in software testing processes: risks, aspects and phases● How to select LLMs: costs, performance and fine-tuning aspects● How to manage change when adopting GenAI in testing for organization and team. AI first approach.● Exam structure and tests.

What exam formats are available, and how do they differ?● Which exam providers are available in different countries, and how do you choose the right one?● How and when should you register, and what do you need to prepare?● What happens on exam day: what to bring, what is allowed, and what is not?

● Overview of the modern AI tooling for testers: - chat assistants vs agentic CLI tools vs IDE assistants. - how LLMs actually work in practice: tokens, context, cost, and why outputs differ between runs
Tools: Claude, Claude Code, OpenAI Codex, Gemini; overview of GitHub Copilot and Cursor

● Prompt engineering hacks as a basis for context engineering● Context engineering:- the skill that has replaced simple prompting as the core AI competency. - setting up an AI-powered QA workspace: project instructions, context files, team conventions, and protection of reference data.
Tools: Claude Projects, Claude Code, Codex CLI, Gemini CLI, git

● Trust, but verify: - systematic evaluation of AI-generated testware- detecting hallucinations and reasoning errors in generated tests- security in practice: prompt injection attacks on testing workflows- data privacy rules for working with real company data.
Tools: Claude Code, promptfoo, git

Reviewing group progress, discussing participants’ questions, and working through practical challenges from the first three workshops together.

● From prompts to autonomous components: - skills- custom agents- MCP integrations.
● Packaging QA expertise into reusable AI building blocks that the whole team can use. Overview of the commercial agentic testing platform landscape.
Tools: Claude Code (Skills, agents, MCP), GitHub MCP, Playwright MCP. Claude Code is the primary demonstration track; Codex CLI is presented as an alternative.

● AI inside CI/CD: from interactive use to unattended automation. Building a real pipeline: automatic impact analysis of code changes, smart regression scope selection, and auto-generated test reports with guardrails, budget control, and human oversight built in.
Tools: GitHub Actions; headless agent modes in Claude Code, OpenAI Codex CLI, and Gemini CLI; Slack/Jira integrations (overview of how to)

● Team receive an unfamiliar feature and independently build the complete AI-first testing workflow from context setup and test design to automated execution, evaluation, and reporting. Data-driven defense of tool and model choices. Personal GenAI adoption roadmap for each participant's own team.
Tools: Full toolset of the course — participants choose and justify their own stack

 Reviewing the completed workflow, exploring challenging implementation points in more detail, and answering the remaining questions.