Every support engineer has lived this ticket. The customer writes, “The upgrade last night broke checkout.” A colleague replies within minutes, “Yes, that version has a known memory leak.” Someone starts preparing a rollback. Two hours later it turns out the upgrade had nothing to do with it; a certificate expired on a downstream payment endpoint. This FAQ covers evidence-based troubleshooting in managed DevOps support: how to treat every explanation as a claim, trace it to evidence, and record what is still unknown before you change production.

What is evidence-based troubleshooting, in one sentence?

It is the practice of acting on what logs, metrics, traces, changelogs and reproductions actually show, rather than on the first explanation that sounds right. The first explanation is a hypothesis. It earns the right to drive a change only after something supports it.

Why not just go with the most likely cause?

Because “most likely” is usually based on recency and memory, not data. The last change made is always the first suspect, and often a fair one. But in support work, acting on a wrong cause has two costs: the actual problem keeps hurting the customer, and the unnecessary fix (a rollback, a restart, a config change) introduces its own risk. Ten minutes spent checking is cheaper than an hour spent undoing.

What counts as evidence?

In rough order of strength:

• A reproduction. If you can trigger the fault on demand and make it disappear by reversing one variable, you have strong evidence.
• Correlated telemetry with timestamps. Errors starting at the exact minute of a change, on exactly the affected component.
• Primary documentation. The vendor’s changelog, release notes or issue tracker entry for the precise version in use.
• A published incident report. Useful when a provider has already confirmed a fault on their side.
• Someone’s memory. Valuable as a lead, weakest as proof.

A colleague says “that version has a known bug”. What do we do?

Ask for the source. A known bug should have an issue number, a changelog line or a vendor advisory. Read it and check the conditions: affected versions, required configuration, symptoms described. Frequently the bug is real but applies to a different setup. If no source turns up, treat it as a rumour and keep investigating.

What about the provider’s status page?

Status pages are useful primary sources for the provider’s view of their own service, but they are not complete. A green status page does not prove the provider is healthy for your region or your specific API, and a red one does not prove your issue is theirs. Correlate their reported window with your own telemetry before concluding anything.

The customer insists on a specific cause. How do we handle that?

Respectfully, and with data. Acknowledge their explanation, explain what you are checking, and share what you find. Customers generally appreciate “here is the log evidence pointing elsewhere” far more than a fix that does not work. When your evidence agrees with them, say so quickly and act.

How do we record what we don’t know?

Keep a short “open questions” block in the ticket or incident channel. For example: “Unknown: whether the error rate on the EU cluster is related; investigating.” This prevents two common failures: people assuming a question has been answered when it has not, and the incident being closed with loose ends that reappear next week.

Who owns the evidence trail on a shared ticket?

Whoever is currently driving the investigation. When a ticket changes hands between shifts, the handover note should list each hypothesis, what was checked against it, and the result. Without that, the next engineer either repeats work or, worse, inherits a guess dressed up as a finding.

Does this slow down incident response?

Slightly at the start, considerably less overall. Teams that check evidence before acting make fewer wrong changes, so they spend less time recovering from their own fixes. Where an immediate mitigation is genuinely needed, take it, but label it as a mitigation and keep investigating the cause. The broader reliability case for this way of working is laid out in our article on building resilient IT operations with cloud and reliability practices.

How should this show up in a postmortem?

Every causal statement in the write-up should point to its evidence: the graph, the log query, the changelog entry. Separate confirmed facts from contributing factors and from things the team still suspects but cannot prove. A postmortem that blurs those categories teaches the wrong lessons. If you want a grounding in the operational principles behind this, our post on core principles for dependable software delivery is a sound reference.

Can monitoring and AIOps do this for us?

They help surface correlations quickly, which is valuable. But a correlation flagged by a system is still a claim that needs checking against the underlying data and a plausible mechanism. Our overview of intelligent operations and observability covers where automated analysis fits and where human verification remains essential.

Is this habit useful outside support work?

Very. The same discipline applies wherever polished claims meet real consequences. A team member researching cosmetic treatment abroad, for example, faces confident marketing and dramatic photos. A resource like Best Cosmetic Hospitals, which describes its procedure guides as citation-backed and lists sources such as the FDA and ASPS, makes it easier to apply the same “where is the evidence?” test, while being clear that it publishes information, not medical advice.

What is the one rule to remember?

Treat every explanation as a hypothesis until evidence promotes it. Find the source, check the conditions, write down the unknowns, and only then change production.