Just started on a support or managed-services desk? Your first live outage can feel like being handed a script in a language you have not studied. Words fly past: sev two, IC, mitigate, scribe, postmortem. This primer slows everything down. It explains IncidentOps in everyday terms, walks through the life of one outage from first alert to follow-up, and ends with ten terms you can tape next to your monitor.<\/p>\n
IncidentOps is the practice of dealing with an unplanned service disruption as an organised team instead of a crowd. The Ops Atlas explainer on IncidentOps<\/a> splits it into four activities: declaring that something is an incident, commanding the response, communicating, and learning. It also admits plainly that the name is informal, a loose cousin of the established term incident management, so you will see the longer phrase on most job descriptions and in vendor documentation.<\/p>\n Beginners often assume every alert is an incident. It is not. The Atlas lists what falls inside the practice: declaration, severity levels, on-call and escalation, roles, update cadence, status pages, mitigation, reviews, follow-up actions and incident metrics. It sends other topics elsewhere: attacker-driven events belong to security response, setting reliability targets belongs to site reliability work, designing alerts belongs to observability, invoking disaster recovery has its own playbook, and removing underlying causes over weeks is problem management. Planned outages are simply maintenance.<\/p>\n For a support engineer that boundary is helpful. You are not expected to solve every kind of failure. You are expected to know which door to knock on.<\/p>\n A monitor trips, or a customer writes in. You check whether the problem is real and whether it matches your written criteria for an incident. Typical criteria cited in the Atlas include customers noticing, a second team being needed, or a problem that lasts around an hour without a fix. If any are true, declare it. Declaring something small costs a few minutes; waiting too long costs hours.<\/p>\n The commander is the person coordinating, not the person fixing. A commander asks: who is doing what, what do we know, what will we try next? The people changing production are a small group, and everyone else stays out of their way. If you are new, your best contribution on the first few calls is usually scribe, writing down the timeline.<\/p>\n Someone else, the communications lead, posts regular updates. The Atlas suggests an opening post around five minutes after kickoff then a scope note soon after, then fresh messages no further than twenty minutes apart across the opening hours. When it is over, the closing note says plainly whether data was lost.<\/p>\n The team looks for a quick mitigation, such as reversing a release or moving load to a healthy server. They do not need a complete explanation first. Locating the fault is usually enough to act on; understanding the reason can wait.<\/p>\n When service is back, the team watches for a while to verify that it stays back. If the incident runs across shifts, the outgoing commander hands over with a clear spoken confirmation. Then comes the review, ideally scheduled within days, with an owner named before the call ended.<\/p>\n Behind every call sits an on-call rota, a schedule saying who gets paged. Escalation is the ladder: if the first person does not acknowledge within an agreed time, the page moves to the next. Support staff sometimes meet this from the other side, as the customer-facing person who triggers a page. The Atlas notes a fairness limit worth knowing: Google’s guidance caps on-call load at about a quarter of a person’s time, so a team of six sharing round-the-clock cover with two slots a week would exceed it and would need closer to eight people. If your team looks stretched, that is a legitimate topic to raise.<\/p>\n Beginners often ask what to actually write. A useful update answers three questions in plain words: what customers experience, what the team knows or is doing, and when the next message will arrive. Here is a generic example: “Some customers cannot save changes. We have found the area responsible and are reversing a recent change. Next update in twenty minutes.” Notice there are no internal system names, no guesses about the cause and no promises about when everything will be fixed. The Atlas puts it memorably, quoting a vendor’s guidance, that clear is better than concise.<\/p>\n Dashboards often feature the acronym MTTR. The Atlas points out that it stands for at least four different things, so ask which one. Because incident lengths are uneven, with a few very long events among many short ones, an average can move a lot without any real improvement. Its worked example uses eleven incidents, where one nasty outage dragged the mean up by three-quarters of an hour. When someone quotes you a single average, politely ask for the median and the range.<\/p>\n It is tempting to skip the review once the pressure is off. Resist. The Atlas recommends that the facilitator should not have been directly involved, that the actions be few, and that every action carry one owner and one date. It also warns against ending a review at “human error”. The better question is what made the mistake easy to make. This habit protects newcomers too, because it keeps the focus on systems rather than on whoever clicked last.<\/p>\n The commonest one is jumping in to fix. It feels helpful, yet two people changing the same system without telling each other is how small problems become big ones. The second is going silent: if you are investigating something, say so in the channel. The third is leaving the writing of the review to somebody else. Even a short note from the person who first saw the alert is valuable, because the first few minutes are the hardest to reconstruct later.<\/p>\n To place your growing skills in the wider picture of support careers, a guide to the various DevOps role titles and career paths<\/a> maps out where incident work leads, and the thinking behind it all connects to core principles for dependable software delivery<\/a>.<\/p>\n Nobody expects a beginner to run the call. Learn the vocabulary, ask the scribe questions, and in a few months you will be the one explaining the glossary to the next new colleague.<\/p>\n","protected":false},"excerpt":{"rendered":" New to support or managed services? Follow one outage from first alert to review, then keep the ten-term glossary beside you for your first live call.<\/p>\n","protected":false},"author":8,"featured_media":5513,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3948],"tags":[],"class_list":["post-5512","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops-support"],"_links":{"self":[{"href":"https:\/\/www.devopssupport.in\/blog\/wp-json\/wp\/v2\/posts\/5512","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.devopssupport.in\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.devopssupport.in\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.devopssupport.in\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/www.devopssupport.in\/blog\/wp-json\/wp\/v2\/comments?post=5512"}],"version-history":[{"count":1,"href":"https:\/\/www.devopssupport.in\/blog\/wp-json\/wp\/v2\/posts\/5512\/revisions"}],"predecessor-version":[{"id":5608,"href":"https:\/\/www.devopssupport.in\/blog\/wp-json\/wp\/v2\/posts\/5512\/revisions\/5608"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.devopssupport.in\/blog\/wp-json\/wp\/v2\/media\/5513"}],"wp:attachment":[{"href":"https:\/\/www.devopssupport.in\/blog\/wp-json\/wp\/v2\/media?parent=5512"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.devopssupport.in\/blog\/wp-json\/wp\/v2\/categories?post=5512"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.devopssupport.in\/blog\/wp-json\/wp\/v2\/tags?post=5512"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Where the practice starts and stops<\/h2>\n
One outage, step by step<\/h2>\n
1. Something looks wrong<\/h3>\n
2. Someone takes command<\/h3>\n
3. Customers hear from you<\/h3>\n
4. Stop the damage<\/h3>\n
5. Confirm, close and hand over<\/h3>\n
Mini-glossary<\/h2>\n
\n
How escalation and rotas fit in<\/h2>\n
What a good update looks like<\/h2>\n
Numbers you may hear, and a caution<\/h2>\n
Why the review matters most<\/h2>\n
Newcomer slips and how to sidestep them<\/h2>\n
Your first week checklist<\/h2>\n
\n