The call came in mid-morning: "Internet is down." I did what most IT people do. I assumed I knew the cause before I asked a single clarifying question. We spent the next six hours chasing the ISP, rebooting routers, and escalating to a vendor who had nothing to do with the problem. The fix, when we finally found it, took twenty minutes. It had been sitting there the whole time.
That day taught me more about IT triage than any training I had received up to that point. The failure was not technical. It was diagnostic. We jumped to a conclusion before we understood the shape of the problem, and every hour we spent after that was wasted because we were solving the wrong thing.
What Actually Happened
This was during my time at a large international NGO, managing IT infrastructure across multiple field offices. When users reported that "the internet is down," my first instinct was the main upstream link. That is what goes wrong in a multi-office setup when everything goes dark at once, right? So we escalated to the ISP. We restarted the edge router. We verified the fiber link. All of it came back clean.
Two hours in, someone on the team asked a question that should have been the first question: "Is this affecting everyone, or just certain offices?" It was affecting certain offices. Not all of them. That single answer eliminated the upstream link as the cause entirely, because the upstream link serves everyone equally. We had spent two hours in completely the wrong layer of the stack.
More questions followed. Which services specifically? Users could load some sites but not others. Internal tools were fine. External collaboration platforms were not. That pointed us toward DNS resolution, not connectivity. We checked the DNS forwarder configuration. A weekend maintenance window had updated some infrastructure records, and one of the DNS forwarders was still pointing at a server that had been decommissioned. Certain offices used that forwarder. Others did not. Which explained why the outage was uneven.
We updated the DNS record. Everything came back in about fifteen minutes. Six hours of pain for a twenty-minute fix, all because we started with an assumption instead of a question.
Five Questions Before You Touch Anything
I now have a short triage discipline that I run through before I open any configuration, restart any service, or pick up the phone to a vendor. Five questions. Ninety seconds. They have saved me from the wrong-assumption trap more times than I can count.
The answers to these five questions will eliminate large parts of your infrastructure from the suspect list before you run a single diagnostic tool. That is the point. Not to gather more data. To discard wrong theories early.
Why Scope Is Everything
Scope is the single most powerful filter in IT diagnosis. It works like this:
- If only one user is affected, the problem almost certainly lives in their device, their account, or their local network. Not in shared infrastructure.
- If one team or department is affected, look at shared resources: a group policy, a shared drive permission, a shared mailbox, a licence assigned to that team.
- If one office or location is affected, the problem is in that location's infrastructure: local switch, local DNS, local firewall rules, or a regional ISP issue.
- If everyone is affected simultaneously, then and only then should you be looking at upstream infrastructure: core switches, internet links, authentication servers, or a provider outage.
Most of the time, "everyone" turns out to mean "several people who happened to call around the same time." Taking sixty seconds to verify scope before escalating to your ISP or your network vendor will save you hours and a good deal of embarrassment.
The Habit That Adds Hours to Every Incident
The habit that kills IT diagnostic time is starting with a fix instead of a question. It comes from good intentions. You want to be useful immediately. You want the problem to go away. So you restart the service, reboot the server, or call the vendor before you know what the problem actually is.
The cost is real. When you restart a service before you understand the cause, you may clear the symptom temporarily while the underlying issue remains. You also lose any live state the system was holding that would have helped you diagnose it. I have watched incidents stretch to four or five hours because someone rebooted the affected server in the first ten minutes, clearing all the logs and memory state that would have told us exactly what failed and why.
The urge to act immediately is understandable. But an untargeted action in the first five minutes often adds an hour to your resolution time. Ninety seconds of triage questions is the faster path.
The Tools I Reach for After the Five Questions
Once scope is established, a few commands confirm the hypothesis fast. On Windows: ping separates DNS failures from routing failures; Resolve-DnsName in PowerShell shows which DNS server is answering and what it returns; Test-NetConnection checks whether a specific port is reachable on a specific host. Event Viewer on the affected machine, filtered to the relevant time window, will usually show you exactly what failed and when. In a Microsoft 365 environment, check the M365 Admin Center service health dashboard first. If Microsoft is having an incident, your local triage is irrelevant.
What I Changed After That Incident
After the DNS incident at the NGO, I did two things that I still do in every IT role. First, I wrote a one-page triage card: the five questions, the scope logic, and the three command-line tools most likely to confirm the answer. It lives in a shared folder every IT team member can access during an incident, because incidents are exactly when you most want to have a checklist and least want to rely on memory.
Second, I made change documentation a strict part of any maintenance window. Every configuration change, every server update, every script run outside of business hours gets a one-line note in a shared log with the timestamp, the system affected, and the person who made the change. When the next incident hits and someone asks "what changed recently?", the answer takes thirty seconds instead of thirty minutes of Slack archaeology.
The One Thing to Do This Week
If your team does not have a triage card, write one today. Paste the five questions at the top. Add the scope matrix. Add your two or three most-used diagnostic commands. Pin it somewhere every team member can reach during an incident, because incidents are exactly when you most want a checklist and least want to rely on memory. The next outage will test whether you actually use it.
If you want to compare notes on how you run IT triage on your team, or if you are building out a first-response playbook and want another set of eyes, reach out. Happy to look at what you have and tell you where the gaps usually are.
Comments