Workshops: Agentic Engineering Practices
Self-Healing CI with AI Agents: Builds, Flaky Tests, Updates
Red builds, flaky tests and a growing queue of dependency update pull requests drain hours from every team. In this workshop your team designs a CI loop in which AI agents diagnose failures, propose fixes and repair update PRs on their own branches, while guardrails on permissions, retries, cost and review keep the pipeline trustworthy.
Who is this workshop for?
- DevOps and platform engineers who own CI/CD pipelines
- Developers tired of babysitting red builds and Renovate or Dependabot backlogs
- QA engineers fighting unstable test suites
- Engineering leads who want to automate pipeline maintenance safely
- Security engineers who must approve agents running inside CI
What you will take away
- Classify CI failures (real regression, flaky test, infrastructure noise, dependency breakage) and decide which an agent should handle.
- Build a flow in which an agent analyses logs, reproduces the failure and opens a fix PR instead of pushing to main.
- Detect, quarantine and fix flaky tests with agent support.
- Let agents repair failing Renovate or Dependabot update PRs, including changelog research.
- Set guardrails: minimal permissions, retry limits, cost caps and mandatory human review.
Agenda
Module 1
Anatomy of a red pipeline
- Failure categories and what an agent can realistically fix
- Logs, artefacts and test reports as agent context
- When automation hurts: masking real bugs and a culture of retries
Module 2
The fix-PR loop
- Triggering an agent on failure in GitHub Actions or GitLab CI
- Headless Claude Code or OpenAI Codex in the pipeline, or handing failures to the GitHub Copilot coding agent
- Exercise: an agent that turns a red build into a reviewable fix PR
Module 3
Flaky tests
- Detecting instability through reruns and history
- Quarantine with an owner and an expiry date, not forever
- Typical root causes: timing, ordering, shared state, test data
Module 4
Dependency update PRs
- Grouping and scheduling updates in Renovate or Dependabot
- Agent fixes for breaking upgrades, with release notes as evidence
- Auto-merge only for low-risk updates with full checks
Module 5
Guardrails and operations
- Token scopes, secrets and why untrusted PR content must not steer the agent
- Loop limits, timeouts and cost budgets per run
- Metrics: time to green, acceptance rate of agent fixes, reverted fixes
What self-healing should and should not mean
Self-healing CI does not mean a pipeline that quietly turns itself green. An agent that disables a failing test, loosens an assertion or raises a timeout has hidden a problem, not solved it. The goal is the opposite: faster diagnosis, a proposed fix with evidence and a human who decides.
Done well, your developers start the day with a short list of fix PRs and explanations instead of a wall of red builds. Done badly, trust in the test suite erodes.
Security in the pipeline
Agents in CI run with credentials and process content written by others: pull request descriptions, issue text, dependency changelogs. That makes them a target for prompt injection. We show how agents can handle such input without being able to cause damage:
- Separate workflows for trusted and untrusted triggers
- Short-lived, minimally scoped tokens and no production secrets
- Isolated runners with restricted network egress
- No direct pushes to protected branches, full logs of every run
Built on your pipeline
We work with your CI system and a test repository that mirrors your real pipeline, so experiments never touch production branches. GitHub Actions, GitLab CI, Azure Pipelines and Jenkins are all possible. The day is led by neooutput AI Code Engineers, remotely or on-site, in English or German.
Frequently asked questions
Will agents in CI drive up our costs?
They can if runs are unbounded. With triggers limited to relevant failures, per-run budgets, timeouts and a cap on retries, costs stay predictable, and we show you how to monitor them.
Should agents auto-merge their own fixes?
For code fixes we advise against it. Auto-merge can make sense for low-risk dependency updates with good test coverage; everything else should pass human review.
Does this replace our flaky-test process?
It supports it. Agents speed up detection and root-cause analysis, but ownership, quarantine rules and the decision to delete or rewrite a test stay with your team.
Workshop
Get in touch
How many hours a week does your team lose to red builds?
Tell us roughly how many people will join and pick dates that work – we will get back to you within 24 hours with a proposal for content, format and schedule.
Prefer to talk first?
15 minutes, free and without obligation – with our AI Trainers and AI Code Engineers.
Book a call for free (opens in new tab) +49 761 519 751 50 (24/7)Request a training / workshop
Workshop: Self-Healing CI with AI Agents: Builds, Flaky Tests, Updates · 1 day
More workshops
- Workshops: Agentic Engineering PracticesTaking Over Vibe-Coded Prototypes: Workshop for Dev Teams
- Workshops: Agentic Engineering PracticesDesign Systems & Storybook with AI Coding Agents: Workshop
- Workshops: Agentic Engineering PracticesPair & Mob Programming with an AI Agent: Team Workshop
- Workshops: Agentic Engineering PracticesAgent-Ready User Stories: Scrum & Kanban with Coding Agents
- Workshops: Agentic Engineering PracticesObservability with Coding Agents: OpenTelemetry Workshop
- Agentic Coding ToolsLovable vs Bolt.new vs v0: AI App Builders Compared
