DeepDojo.AIPublished 2026-11-15中文

Debugging CI Failures With Claude Code

Claude CodeAI Coding

A test has passed on GitHub Actions ten times in a row, and this time it fails. You scroll to the bottom of the log, and the last error points at a file this commit never touched.

The easiest move is to paste that last error into Claude Code and ask it to "fix it." But GitHub Actions logs are long, and the last line is often just a downstream symptom. The root cause might sit in an earlier step or another job. It might be a flaky test that passes some runs and fails others, or an outage in an external dependency that happened at the same time.

Have it fix whatever the last line points to, and at best you waste an afternoon. At worst it breaks code that was working.

To investigate a CI failure, answer four questions first:

  • Where did it fail: which workflow, which job, which command, and what exit code?
  • When did the failure start, and does it reproduce reliably?
  • Which commit might have introduced it?
  • Is someone already fixing it?

Those four questions map to four layers of investigation. Claude Code can dig into each one, as long as every judgment comes with evidence you can re-check: reruns on the same commit, the pass rate over the last dozen or so runs, the same check run before and after a commit, and any open fix PRs.

01 Find the Failing Step

Start by confirming the workflow, job, commit, failing command, and exit code. The log mixes in plenty of output that looks like an error, so sort it out: warnings and non-fatal errors are one thing, and the failing:, fatal:, or condition check that actually made the script exit is another.

The "last line points at an untouched file" puzzle from the opening is often solved right here. The assertion that made the command exit with a non-zero code may sit dozens of lines higher.

02 Check Whether It's Flaky

A flaky check sometimes passes and sometimes fails on the same input. To judge that, look at the last 10 to 20 runs of the same job; the workflow's overall result can mislead you, because a failure in another job also marks the whole workflow as failed.

gh run list --workflow=<workflow-name>
gh run view <run-id> --json jobs

--json jobs on gh run view lists the status of every job in a run. You want three things: the recent pass rate, when it last passed, and whether failures cluster after a certain point.

Passes and failures across different commits don't prove the check is flaky. Compare reruns on the same commit, with the same input and environment, first; then look into differences in timing, network, or resources.

For the test that had passed ten times in a row, start with a same-commit comparison:

  1. Rerun the same commit and check that the input and environment are comparable. Two passes and one failure out of three supports looking for instability; three passes still can't rule out a rare bug.
  2. Check whether failures cluster in time. If they mostly happen in the early morning, look into scheduled jobs and resource contention, but you'll still need resource records or a controlled repro to confirm the cause.

"Flaky" describes an unstable result. It doesn't explain the root cause. After the report says "this is flaky," it still needs to say how far the investigation got.

03 Find the Commit That Introduced It

Compare the last passing run, the first consistent failure, and the commits in between. Commits near the first failure are only candidates: a dependency outage can line up with a commit purely by coincidence.

Where possible, run the same check on the commits before and after, in the same environment. With many commits, narrow the range with git bisect. As the Git documentation explains, it uses binary search to find the commit that introduced a bug: you mark one commit known to be "bad" and one known to be "good," and it keeps picking a commit in between for you to test until it finds where the problem first appeared.

04 Look for an Existing Fix

Search open PRs using the error message, file names, and workflow name:

gh pr list --state open --search "<keywords>"

If someone is already working on it, report their progress instead of duplicating the work.

05 Hand the Investigation to Claude Code

All four layers can go to Claude Code at once, as long as the request asks for evidence:

Analyze this GitHub Actions failure.
Confirm the workflow, job, commit, failing command, and exit code; distinguish fatal errors, warnings, and flakiness;
check related commits and existing fix PRs. Give verifiable evidence for every judgment.

The final report can follow this structure:

Summary: the direct cause of exit code 1.
Flakiness: how often it recurs and the recent pass rate.
Breaking Commit: the commit and evidence, if it can be identified.
Root Cause: the cause supported by both logs and history.
Existing Fix PR: progress on any existing fix.
Recommendation: next steps to fix and verify.

Every section should point to specific evidence. "The failure started around the time of a certain commit" is only a lead to check. Running the same test on the commits before and after is what tells you whether the change and the failure are related. For how to keep facts, inferences, and open items apart in a report, checking research done with Claude Code goes into more detail.

If you run this process often, you can package it as a command. The dx plugin from the third-party repo ykdojo/claude-code-tips provides /dx:gha <url>: pass it a GitHub Actions link and it analyzes the run along similar steps. Review the scripts and permissions it brings in before installing.

06 Check In Now and Then While CI Runs

CI reruns often take several minutes. You don't need to watch the log. Query it now and then with gh run view <run-id>, filtered to the job you care about:

gh run view <run-id> | grep "<job-name>"

gh run watch streams output continuously; use it only when you really need live updates. Have Claude Code check at 1, 2, 4, then 8 minutes and analyze only when there's a clear failure. Managing multiple Claude Code sessions at once has the full checking protocol.

After all four layers, that test that failed after ten passes might turn out to be an environment blip, or a rare real bug. With every judgment in the report backed by evidence you can re-check, and anything unclear marked as unconfirmed, it becomes clear whether the next step is fixing code, adding a test, or waiting for a dependency to recover.