Workshops: Agentic Engineering Practices

Self-Healing CI with AI Agents: Builds, Flaky Tests, Updates

Red builds, flaky tests and a growing queue of dependency update pull requests drain hours from every team. In this workshop your team designs a CI loop in which AI agents diagnose failures, propose fixes and repair update PRs on their own branches, while guardrails on permissions, retries, cost and review keep the pipeline trustworthy.

Who is this workshop for?

  • DevOps and platform engineers who own CI/CD pipelines
  • Developers tired of babysitting red builds and Renovate or Dependabot backlogs
  • QA engineers fighting unstable test suites
  • Engineering leads who want to automate pipeline maintenance safely
  • Security engineers who must approve agents running inside CI

What you will take away

  • Classify CI failures (real regression, flaky test, infrastructure noise, dependency breakage) and decide which an agent should handle.
  • Build a flow in which an agent analyses logs, reproduces the failure and opens a fix PR instead of pushing to main.
  • Detect, quarantine and fix flaky tests with agent support.
  • Let agents repair failing Renovate or Dependabot update PRs, including changelog research.
  • Set guardrails: minimal permissions, retry limits, cost caps and mandatory human review.

Agenda

  1. Module 1

    Anatomy of a red pipeline

    • Failure categories and what an agent can realistically fix
    • Logs, artefacts and test reports as agent context
    • When automation hurts: masking real bugs and a culture of retries
  2. Module 2

    The fix-PR loop

    • Triggering an agent on failure in GitHub Actions or GitLab CI
    • Headless Claude Code or OpenAI Codex in the pipeline, or handing failures to the GitHub Copilot coding agent
    • Exercise: an agent that turns a red build into a reviewable fix PR
  3. Module 3

    Flaky tests

    • Detecting instability through reruns and history
    • Quarantine with an owner and an expiry date, not forever
    • Typical root causes: timing, ordering, shared state, test data
  4. Module 4

    Dependency update PRs

    • Grouping and scheduling updates in Renovate or Dependabot
    • Agent fixes for breaking upgrades, with release notes as evidence
    • Auto-merge only for low-risk updates with full checks
  5. Module 5

    Guardrails and operations

    • Token scopes, secrets and why untrusted PR content must not steer the agent
    • Loop limits, timeouts and cost budgets per run
    • Metrics: time to green, acceptance rate of agent fixes, reverted fixes

What self-healing should and should not mean

Self-healing CI does not mean a pipeline that quietly turns itself green. An agent that disables a failing test, loosens an assertion or raises a timeout has hidden a problem, not solved it. The goal is the opposite: faster diagnosis, a proposed fix with evidence and a human who decides.

Done well, your developers start the day with a short list of fix PRs and explanations instead of a wall of red builds. Done badly, trust in the test suite erodes.

Security in the pipeline

Agents in CI run with credentials and process content written by others: pull request descriptions, issue text, dependency changelogs. That makes them a target for prompt injection. We show how agents can handle such input without being able to cause damage:

  • Separate workflows for trusted and untrusted triggers
  • Short-lived, minimally scoped tokens and no production secrets
  • Isolated runners with restricted network egress
  • No direct pushes to protected branches, full logs of every run

Built on your pipeline

We work with your CI system and a test repository that mirrors your real pipeline, so experiments never touch production branches. GitHub Actions, GitLab CI, Azure Pipelines and Jenkins are all possible. The day is led by neooutput AI Code Engineers, remotely or on-site, in English or German.

Frequently asked questions

Will agents in CI drive up our costs?

They can if runs are unbounded. With triggers limited to relevant failures, per-run budgets, timeouts and a cap on retries, costs stay predictable, and we show you how to monitor them.

Should agents auto-merge their own fixes?

For code fixes we advise against it. Auto-merge can make sense for low-risk dependency updates with good test coverage; everything else should pass human review.

Does this replace our flaky-test process?

It supports it. Agents speed up detection and root-cause analysis, but ownership, quarantine rules and the decision to delete or rewrite a test stay with your team.

Workshop

Get in touch

How many hours a week does your team lose to red builds?

Tell us roughly how many people will join and pick dates that work – we will get back to you within 24 hours with a proposal for content, format and schedule.

Prefer to talk first?

15 minutes, free and without obligation – with our AI Trainers and AI Code Engineers.

Book a call for free (opens in new tab) +49 761 519 751 50 (24/7)

Request a training / workshop

Workshop: Self-Healing CI with AI Agents: Builds, Flaky Tests, Updates · 1 day

Dates that work for you (optional)

Suggestions about 2 weeks ahead – pick as many as you like

Reply usually within 24 hours · jocapps is ISO 27001 certified · Privacy