Latency Hunting: A Methodology

Users complain "The app is slow." Where do you look?

Step 1: Isolate the Segment

Ping the Load Balancer IP. Then ping the backend instance IP. If LB is fast (20ms) but instance is slow (200ms), the issue is in your VPC or Application, not the internet.

Step 2: MTR (My Traceroute)

Don't use standard traceroute. Use MTR. It sends continuous packets to every hop.

mtr -r -c 100 google.com

Look for the hop where packet loss begins. If loss starts at Hop 1, it's your local Wifi. If it starts at Hop 12, it's an internet exchange point.

Step 3: Packet Capture (Wireshark/TCPDump)

Capture traffic on the server. Look for TCP Retransmissions. This indicates a dirty network where packets are being dropped and resent.

Look at the Delta Time between the HTTP Request and the HTTP Response. If the network delta is 1ms but the response takes 5 seconds, your Database query is slow. It's not the network.