Imagine two support desks covering the same kind of managed service on the same kind of quiet Tuesday night. Both employ capable engineers. Both have monitoring. At just past two in the morning, a storage volume on a customer’s application server starts filling faster than usual. This is the story of how each desk handles it, and what the contrast says about writing if-then plans for support work.
Desk one: deciding on the spot
On the first desk, the engineer on shift sees the alert and starts thinking. Is this a known pattern? Should the customer be called now or at a more civilised hour? Could old logs be removed, or would that risk deleting something needed? The engineer scans the ticket history, finds two similar cases handled differently, and spends fifteen minutes weighing them.
Nothing goes badly wrong. The engineer clears space, notes the action and moves on. But the decision relied on one person’s memory, one person’s confidence and one person’s tolerance for risk at an awkward hour. A different engineer on a different night might have decided differently, and the customer would never know why the experience varied.
Desk two: deciding in advance
The second desk has a short document pinned beside the alerts. Among its lines is this one: if a storage volume passes the agreed fill level and is growing, then rotate the application logs and notify the account owner by message. The engineer reads the line, performs the action, and sends a brief note. Total time: a few minutes. The deeper investigation of why growth accelerated waits for morning.
The engineer did not need to be braver or smarter than the first one. The difficult thinking had already been done by the team, in daylight, with the ticket history open.
Side-by-side comparison
Here is how the two approaches line up across the things support managers usually care about.
- **Speed to first action**: the improvising desk spends time deliberating; the planned desk moves quickly. The gap matters most for fast-growing problems.
- **Consistency**: improvised responses vary between engineers and nights; plans give customers the same treatment every time.
- **Stress on the engineer**: choosing under pressure is tiring; following a plan is lighter, although the engineer still carries responsibility for noticing when a plan does not fit.
- **Handover quality**: shift handovers on the improvising desk depend on narrative notes; on the planned desk they can reference named plans, which keeps messages short.
- **Flexibility for novel situations**: improvisation wins here, since no plan covers a truly unforeseen failure.
- **Upkeep**: improvisation costs nothing to maintain; plans must be reviewed, or they decay into misleading clutter.
The table, so to speak, is not one-sided. Planned response excels at repeat problems. Improvisation is irreplaceable for the odd ones. The sensible position is a blend: a compact set of plans for known patterns, with a clear route for everything else.
What the plans on desk two look like
A few examples in the style used by desks that do this well:
- If a monitored service fails its health check after a change, then roll back the change and notify the owner.
- If the same alert fires twice within the agreed window, then page the senior on duty rather than silencing it.
- If a customer reports an outage that monitoring has not seen, then check the monitoring path first and note the gap.
- If a backup job fails once, then re-run it; if it fails twice, then raise a ticket for the owner.
Each plan starts with something observable and ends with one action. None tries to cover every case, and none asks the engineer to evaluate a long list of conditions.
Where the comparison gets interesting
A common objection is that plans make engineers mechanical. In practice the reverse tends to happen. Because routine decisions are handled, engineers have more attention for the unusual ones. The sense of calm on desk two is not complacency; it is spare capacity.
Another objection is cost. Writing and reviewing plans takes time. That is true, and it is worth putting a figure on it using your own history: how many minutes of deliberation did your last ten routine incidents consume? If the answer is large, the investment is easy to justify. If the answer is small, you may need only two or three plans.
Principles that underpin this thinking, such as fast feedback and shared ownership, are described in an outline of core DevOps principles for effective delivery. And for support engineers who are expected to understand code-defined infrastructure as well as operations, an explanation of how infrastructure as code leads toward reliability engineering explains how rules and automation can reinforce each other.
Making the switch without a big project
Desks rarely adopt plans through a grand programme. The easiest route is incremental. Take whichever alert has been noisiest lately. Draft a single condition plus a single reaction for it, and keep both under one line. Place it where the alert appears. After two weeks, review whether it fired and whether it helped. Then pick the next noisiest.
Name an owner for each plan and record who can authorise an exception. That small addition prevents the most common failure of written rules: the engineer who follows the plan when it obviously does not fit, because nobody said they could do otherwise.
Engineers on the move
Support rotas sometimes include travel, and a dropped connection can leave an engineer unreachable. A natural plan: if I am travelling during a rota week, then I arrange connectivity before I leave. Some engineers browse a travel resource organised into countries, providers, plans and guides for SIM and eSIM selection to settle this early, so the question never arises at a gate.
Checking whether the plans help
Should you prefer facts to hunches, line up a few weeks of tickets from before and after. Look at the time from alert to first action, at how often the same alert reopened a ticket, and at whether handover notes got shorter. Those three signals are modest but honest, and they avoid the trap of counting how many plans exist. A desk with four plans that fire often and help is better off than one with forty that nobody opens.
Gather the observations in a plain shared note rather than building a dashboard. The aim is to decide which plans to keep, not to produce a report.
Which desk would you rather join?
Most people, asked this, choose the second. They are not choosing bureaucracy; they are choosing a team where the common decisions have been made kindly and in advance, and where a new colleague can act with confidence on their first night. The improvising desk may be full of talented people, but its talent is spent on questions that did not need asking.
Start with one plan this week. It is the cheapest improvement a support team can make, and unlike most improvements, it pays back the first time the alert goes off.