Failure scenarios

A regression the log does not name

Shape: a test fails with an error about a value it did not construct. The traceback names the test file; the bug is somewhere else.

E       AttributeError: 'dict' object has no attribute 'expires_at'
tests/unit/test_auth.py:47: AttributeError

Healthy run: one read_file on the module that produces the value, then a diagnosis naming the function and the changed return type. Confidence 8–9.

Why this is the interesting case: a script cannot do it. Everything in the log is true and none of it is the answer. This is the case the tool budget and the “open the source” instinct exist for.

A dependency that will not resolve

Shape: the build fails during installation. The resolver prints the conflicting constraints in full.

ERROR: Cannot install app==1.4.0 and urllib3<2 because these have conflicting dependencies.
The conflict is caused by:
    The user requested urllib3<2
    botocore 1.34.2 depends on urllib3>=2.0.7

Healthy run: zero tool calls. The log contains the answer verbatim, so opening requirements.txt to confirm it wastes a turn and adds nothing. A run that reads three files here is a run that has been trained to look busy.

Watch for: the fix suggestion is often the weakest part. Pinning urllib3 resolves the build; whether it is what you should do depends on constraints TraceCI cannot see.

A broken workflow or tooling config

Shape: failure at a setup step — Set up Python, actions/cache, a matrix expansion. Nothing has run yet, and the application code is irrelevant.

Healthy run: a read_file on the workflow file, sometimes a list_directory to confirm a path exists. Category config, and the root cause names the YAML key.

Watch for: workflow files that use reusable workflows or composite actions defined in other repositories. TraceCI reads the repository under investigation, so a failure inside an external action is visible only through its output.

A lint or type error

Shape: a static check rejected the code. Rule code, file and line are printed together.

app/rate_limit.py:64:9: F821 undefined name 'window_start'
Found 1 error.

Healthy run: zero or one tool calls, high confidence, and a patch that is usually correct — this is the category where the suggested fix is most reliable, because the tool already stated exactly what it wanted.

Watch for: a lint failure after a configuration change. The error is in your code but the cause is a new rule being enabled, and that distinction is only visible in the diff.

Infrastructure

Shape: runner out of disk, a registry timeout, a service container that never became healthy, a cancelled job. Nothing to do with your change.

Healthy run: zero tool calls and a category of infra. This is the one category where re-running the job is a legitimate response.

Watch for: infrastructure failures that are really resource regressions. A job that started running out of memory after a change is categorised as infra but caused by the diff — check whether the diff is empty before accepting the label.

Flakiness

Shape: a test that passes on re-run. Timing, ordering or shared state.

Healthy run: confidence in the 4–6 range, and language that says the failure is consistent with non-determinism rather than asserting it. A single run cannot prove flakiness, and a diagnosis claiming it at confidence 9 is overreaching.

Best use: run TraceCI on the failed run rather than re-running the job. It often identifies the shared fixture or ordering assumption, which a green re-run destroys the evidence for.

Quick calibration table

These are expectations, not guarantees. A run well outside its row is worth a second look — not because it is necessarily wrong, but because it is unusual.

Each scenario has a matching branch in the lab repository, prefilled as an example in the workspace.