You get a shell on a deliberately broken box, a vague complaint like "it feels slow", and about forty minutes. Run a fixed order and say it out loud: orient, then load, memory, disk I/O, network, and logs last. Each step either accuses a subsystem or clears it, and the sentence you say before each command is scored as heavily as the fix, because the interviewer is testing whether you form hypotheses or just type commands you memorised. Below: that order, the command at each step, the fallback when the box has no tooling, and the line to say.
What the broken-box format actually is
This round is not a quiz. Someone has broken a throwaway VM or container, handed you a shared terminal over tmux, SSH or a screen share, and is watching your keystrokes. Four things follow from that shape:
- There are usually one to three planted faults, chosen to be findable in the time given. Nobody plants a kernel bug. They plant a full disk, a runaway process, a wrong file descriptor limit, a broken resolver.
- The interviewer is playing the box. Ask whether the host is taking live traffic, or what deployed in the last hour, and you will usually get a real answer. Candidates who never ask are marked down for it.
- The box is often stripped. No
sar, noiotop, sometimes no package manager and no network to install one. That is deliberate. - A restart clears the symptom and destroys the evidence. Say what it would cost before you reach for it, and say it before the silence gets long, because quiet stretches read as guessing even when you are right.
The reference most interviewers carry in their heads is Brendan Gregg's first sixty seconds checklist from the Netflix performance engineering team: uptime, dmesg | tail, vmstat 1, mpstat -P ALL 1, pidstat 1, iostat -xz 1, free -m, sar -n DEV 1, sar -n TCP,ETCP 1, top. Knowing the list is table stakes. Knowing what each line would have to say to move you to the next subsystem is the interview.
The triage order, the command, and the sentence
Work top to bottom. Do not skip ahead because a hunch pulled you there, and if you do skip, say why.
| Step | The question it answers | First command | If the tooling is missing |
|---|---|---|---|
| 0. Orient | What is this host, and what changed | uptime, uname -a, dmesg -T | tail -40, systemctl --failed | Always available |
| 1. Load | Is anything running, or is everything waiting | top, then pidstat -u 1 5 and mpstat -P ALL 1 | /proc/loadavg, /proc/pressure/cpu |
| 2. Memory | Reclaimable, or genuinely gone | free -m, vmstat 1 5, dmesg -T | grep -i "killed process" | /proc/meminfo, /proc/pressure/memory |
| 3. Disk I/O | Saturated, or simply full | iostat -xz 1, iotop -oPa, df -h, df -i | /proc/diskstats, /proc/PID/io, /proc/pressure/io |
| 4. Network | Reaching the dependency at all, and how fast | ss -s, ss -ltn, ss -ti, curl -w timing split | /proc/net/dev, /proc/net/netstat, /proc/PID/limits |
| 5. Logs | Does the timeline match the story | journalctl -u svc --since "15 min ago", journalctl -p err -b | /var/log/, the dmesg ring buffer |
What to say at each step
These lines are not filler. Each one names a hypothesis and the observation that would kill it, which is the thing being graded.
Orient
"Before I change anything: what does this host serve, is it taking live traffic, and when did the symptom start? I am going to read state before I touch state." Then uptime, for how long the box has been up and how load has trended across one, five and fifteen minutes. Falling across those three columns is a different problem from climbing.
Load
"Load is 38 on 8 cores. Before I blame CPU I want to know how much of that is runnable and how much is uninterruptible sleep, because Linux counts D state in the load average." In vmstat 1 5 the r column is runnable and b is blocked. High r with high user time is real CPU work, and mpstat -P ALL 1 says whether one core is saturated, which usually means a single-threaded process. High b with high wa is a storage problem, so say that and jump to step three.
Memory
"I am reading the available column, not free, because a small free number is usually page cache doing its job." Then si and so in vmstat: sustained swap traffic is the difference between tight memory and thrashing. If dmesg -T shows an OOM kill, read the whole line rather than the process name. It carries the RSS and the cgroup, and the cgroup tells you whether this was a container limit rather than the host running dry.
Disk I/O
"w_await is 400 milliseconds on one device while the others sit idle, so this is one disk and not the whole box." In iostat -xz 1, discard the first sample: it averages since boot. Read r_await and w_await for latency and aqu-sz for queue depth, and distrust %util on SSD and NVMe, where parallel queues make 100 percent mean busy rather than saturated. Then df -h and, in the same breath, df -i.
Network
"The listen queue is sitting at its backlog limit, so we are dropping connections at accept rather than failing to route." For a listening socket, ss -ltn shows the current backlog in Recv-Q and the limit in Send-Q. Confirm the drops with nstat -az TcpExtListenOverflows TcpExtListenDrops. To split latency, run curl -o /dev/null -s -w 'dns %{time_namelookup} conn %{time_connect} tls %{time_appconnect} ttfb %{time_starttransfer}\n' https://dependency/ and let the four numbers say whether the cost is DNS, TCP, TLS or the application.
Logs
"Everything so far points at storage from about 14:02, so I am opening the logs to confirm that timeline, not to go hunting." Logs read first are an unbounded search; logs read last are a one-minute confirmation. Use --since with the time your evidence turned, and read journalctl -k separately, because kernel messages about device resets, OOM and conntrack rarely reach the service unit's own journal.
When the box has no tooling: reading /proc instead
This is the part that separates candidates. If iostat is missing, do not stall and do not try to install anything. Say "sysstat is not here, so I will read the counters it would have read" and go to /proc. All of these exist on a stock kernel with nothing installed.
# load, plus pressure stall info on kernel 4.20+ (absent if PSI is disabled) cat /proc/loadavg cat /proc/pressure/cpu /proc/pressure/io /proc/pressure/memory # memory: MemAvailable is the number that matters, not MemFree grep -E 'MemTotal|MemAvailable|SwapTotal|SwapFree|Dirty|Writeback' /proc/meminfo # per-device I/O. cols 12 and 13 are I/Os in flight and ms spent doing I/O. # sample twice, one second apart: a delta near 1000 means busy the whole second grep -E ' (sd[a-z]|nvme[0-9]n[0-9]|vd[a-z]) ' /proc/diskstats # who is blocked, and in which kernel function ps -eo state,pid,ppid,wchan:32,comm | awk '$1 ~ /D/' # per-process I/O and file descriptor pressure, no lsof required cat /proc/<pid>/io grep 'Max open files' /proc/<pid>/limits ls /proc/<pid>/fd | wc -l # interface errors and TCP drops when sar and ip -s are unavailable cat /proc/net/dev grep -E 'ListenOverflows|ListenDrops|TCPSynRetrans' /proc/net/netstat
Two of these carry disproportionate weight. /proc/pressure/io answers "is anything actually waiting on storage" in one line, which no amount of squinting at %util does. And wchan names the kernel function each D-state process is parked in, so a screen full of processes stuck in an NFS or device-mapper function tells the story before you open a single log.
Five faults that get planted, and the tell for each
| Symptom | What was planted | The command that confirms it | The wrong turn |
|---|---|---|---|
df says 100 percent, du says half that | A deleted file still held open by a running process | lsof +L1, or ls -l /proc/*/fd 2>/dev/null | grep deleted | Deleting more logs, which frees nothing, because the blocks are already unlinked |
"No space left on device" with free space in df -h | Inode exhaustion, usually millions of small files under one directory | df -i, then find /var -xdev -printf '%h\n' | sort | uniq -c | sort -rn | head | Hunting for large files for ten minutes |
| Load of 40, CPU 95 percent idle | Processes blocked on slow storage or a hung mount | The b column in vmstat 1, then ps -eo state,pid,wchan:32,comm filtered to D | Blaming CPU and living inside top sorted by CPU |
| The port is listening, clients time out | Accept backlog full, or a low file descriptor limit on the unit | ss -ltn Recv-Q against Send-Q, nstat -az TcpExtListenOverflows, /proc/PID/limits | Restarting the service, which clears the queue and deletes the evidence |
| Intermittent latency to one dependency | Conntrack table full, or a resolver timing out to its first nameserver | dmesg for "nf_conntrack: table full", /proc/sys/net/netfilter/nf_conntrack_count against _max, and getent hosts against dig +short | Testing with an IP address, which silently skips the broken step |
Two follow-ups are near certain. "How would you mitigate this in five minutes without root-causing it" wants a reversible action with an explicit cost, for example rotating and truncating to buy disk headroom while you find the writer. "How would you stop it recurring" wants an alert on the leading indicator rather than the symptom: inode usage rather than bytes free, listen overflows rather than error rate.
Where a copilot is the wrong tool for this round
Worth saying plainly on a page published by a company that sells one: a live debug round is the weakest case for AI assistance of any kind. The output you need is a five-character command, not a paragraph, and if you did not already know the command you cannot interpret what it prints. The interviewer is watching a terminal you are driving, so every pause belongs to you, and a silent gap while you read something is more conspicuous here than in a behavioural round where thinking time is expected. Terminal output scrolling under a cursor is also poor material to capture, and much of what matters is a number that changed between two samples rather than anything sitting on screen.
The reasonable use is before the round. OfferPilot AI runs on Windows 10 and 11 only, so it is a preparation tool on your own machine, not something you carry onto a bastion host. Its overlay is excluded from screen capture at the operating system level, so it does not appear in a Zoom, Teams, Meet or OBS share or in a screenshot, but that covers screen sharing and screen recording only. It does nothing about a webcam feed, a lockdown browser, eye tracking, or a person watching you in the room. Where it earns a place in an SRE loop is the architecture and incident-review conversations either side of the debug round.
How to rehearse until the order is automatic
Break a machine on purpose and fix it against a timer. A local VM is enough. Fill a filesystem with fallocate, then unlink the file while a process still holds it open and watch df and du disagree. Exhaust inodes with a loop that creates empty files. Pin one core with a spin loop and confirm mpstat -P ALL 1 shows a single core hot rather than all of them. Set LimitNOFILE low on a systemd unit and drive it until it refuses connections. Each takes minutes to build and shows you what the output looks like when it is genuinely broken.
Then rehearse the talking, the half most people skip. Run the whole order on a healthy box and narrate every step aloud to a machine that is not broken. If you can narrate a clean run in three minutes, you can narrate a dirty one under pressure. For the rounds either side, the company Q&A generator builds a bank around the stack the role uses, and the question banks cover the incident-review and behavioural sides. The free tier is 5 minutes a day with no card, and paid plans meter listening time only rather than wall-clock session length.
Related guides
- Data Engineer SQL Interview Round: What to Expect in 45 Minutes
- API Design Interview vs System Design Interview: The Tells
- React Live Coding Interview Help: One Multi-Scroll Capture
- DevOps Interview vs SRE Interview: Two Different Question Sets
- Full Stack Take-Home vs Live Coding: What Each Format Grades