How We Measure Results

What we measure

Every engagement opens by agreeing which numbers count as evidence. Below is the default set for a routing or failover engagement. It is deliberately short: a metric nobody looks at after kickoff is instrumentation debt, not observability.

Metric	What it tells you	How it is captured
p50 / p95 / p99 latency by region	Whether slowness is universal or confined to users your median hides.	RUM bucketed by client geography and served region, plus synthetic probes from fixed vantage points for a controlled comparison.
DNS resolution time	Cost of the lookup preceding every cold connection, and whether your authoritative tier or a resolver path is responsible.	Resolver-side timing from probe nodes plus the client's domainLookupEnd − domainLookupStart delta.
Time to first byte	Path plus origin think-time; separates distance from slow backends when read alongside connect time.	RUM navigation timing and synthetic requests against the same URL from several regions.
Failover detection time	How long a fault exists before anything reacts — the part of an outage nobody sees in a dashboard.	Fault injection timestamped from health-check logs, not from the alert that eventually fired.
Failover completion time (RTO)	Wall-clock time from fault to a healthy path carrying production traffic.	Game-day exercises measured end to end, including TTL expiry and connection-pool drain, not just the control-plane call.
Data loss window (RPO)	How much writable state a regional failover would discard.	Replication lag sampled continuously at the database tier and reported as a distribution, not an average.
Egress cost per GB by path	Which routing decisions are expensive, independent of this month's volume.	Billing exports joined to flow logs so bytes attribute to a path — inter-region, inter-AZ, internet or interconnect.
Health-check false-positive rate	Whether your failover logic can be trusted, or will flap under load and cause the incident it was built to prevent.	Health-check history correlated against origin error rate over the same window.
Error rate by origin	Whether a "network problem" is one origin degrading while the load balancer keeps sending it work.	Edge and load balancer logs split by backend target, with 5xx separated from 4xx and connection resets.

Baseline before change

An after-number on its own means nothing. Traffic mixes shift, a campaign moves your user geography, a provider silently changes a peering path — any of those moves a latency percentile more than a competent configuration change would. Without a before-measurement taken with the same instrument over a comparable window, you cannot tell which happened. So the first phase is measurement, not change:

Agree the metric set and record in advance which numbers we expect to move and which should stay flat. The flat ones are the control — if they move too, something other than our change is at work.
Collect over a window containing your real cycles: a full week at minimum, longer if load is seasonal or batch-driven.
Record conditions as well as results — probe locations, sampling rate, resolver set, TTLs in force, exact queries. A baseline you cannot reproduce is an anecdote.
Freeze the instrument. If collection changes mid-engagement, old and new are kept as separate series rather than plotted as one line.

This is also the honest answer to "how much faster will it be?" — unknown until the baseline exists. The process is sequenced around that, and every shape of engagement opens with measurement.

What we will not promise

Three commitments are common in this market and all three are unearnable. The useful question is not whether the firm making one is honest, but what it would have to know that it cannot know.

A guaranteed latency figure before measuring your traffic

Latency is a property of your users, your origins and the paths between them, so a millisecond figure quoted before seeing your data is somebody else's topology. Your gain depends on where users actually are, how much of your TTFB is origin think-time rather than transit, whether connections are cold or reused, and how much headroom your routing has left — any of which can make the same change worth a great deal or nothing.

An uptime guarantee for infrastructure we do not operate

We design and tune; you and your providers run it. Availability is decided by the code you deploy, the change process you follow at 2am, and the regions you sit in. A firm guaranteeing your uptime is either taking an operational role it has not staffed, or writing exclusions enough to make the guarantee unenforceable. We help set an SLO, design failure modes that suit it, and measure whether it holds. We will not underwrite it.

A vendor-independent claim about a provider's SLA

Only the provider can commit to its own availability, with its own definition of "unavailable", measurement interval and credit mechanism. Those definitions matter more than the count of nines: a monthly percentage computed over five-minute intervals absorbs a lot of brownout your users would call an outage. We read those documents with you rather than restating them as our own commitment.

Why numbers disagree

Expect your dashboard, our measurements and your provider's console to disagree. Reconciling them exactly is rarely worth the effort; understanding why they differ usually is, because the reason is often the finding.

Synthetic probes versus real user monitoring

Probes give a repeatable signal from a well-connected fixed location: good for detecting change, bad for representing users. RUM gives the truth about the population you have, contaminated by device CPU, congested last-mile links and blockers. Use probes to attribute a change, RUM to decide whether it mattered.

Percentile choice and computation

A p99 over an hour and a p99 over a day are different quantities, and neither can be recovered from the other — percentiles do not average across regions or buckets. Many backends store approximate quantiles from bounded-error sketches, so two systems can honestly report different p99s for identical input.

Sampling and cardinality limits

Tracing backends sample, often dropping most successful requests while keeping errors, which biases any latency distribution built from traces. Metrics backends drop label combinations past a cardinality ceiling, so a per-region, per-endpoint, per-status breakdown can quietly stop being complete at the granularity a routing decision needs.

Clock skew

Any metric derived by subtracting timestamps taken on two machines carries their clock offset; browser clocks are not disciplined at all. Where a measurement spans hosts — replication lag, one-way delay — we say so and treat small differences as noise.

Resolver geography versus user geography

Geo and latency-based DNS decide from the resolver's address, not the user's. A user on a public resolver may be steered to that resolver's egress region; EDNS Client Subnet fixes this only when resolver and authoritative tier both support it and the subnet is forwarded. It is a frequent cause of a regional tail that looks inexplicable until you segment by resolver — read alongside Anycast DNS and geo routing.

Provider dashboards versus your telemetry

A provider measures at its edge; you measure in your application or the browser. The two observe different segments of the same request, with different inclusion rules for retries, aborted connections, TLS handshakes and health-check traffic, over different intervals. Both can be correct and still differ. We reconcile shape and direction of movement rather than absolute value, and always say which source a number came from.

Illustrative scenarios

These scenarios are composites showing typical scope and sequence — what we would measure and change, in what order. They are not client results, not case studies, and contain no performance figures.

Global SaaS with a single-region origin

The symptom is usually "the app feels slow in APAC". First establish whether that is transit distance, DNS, TLS setup or origin think-time, by segmenting TTFB and connect time by client region and comparing RUM against probes on the same paths. Then: baseline a full traffic cycle; separate cacheable responses from dynamic ones; move the dynamic path onto a terminating edge so TLS and TCP setup happen near users; only then consider read replicas or a second region, since that is the expensive change and should follow a measurement.

Retailer with seasonal spikes

The failure mode is behaviour at the knee of the load curve, so baselines must cover a quiet period and a representative peak, and error rate by origin matters more than p50. Then: map where capacity actually binds — connection limits, backend concurrency, edge cache hit ratio; check whether health-check thresholds tuned for quiet hours will false-positive under load and evict healthy origins exactly when you need them; confirm autoscaling and health checking do not fight during warm-up; set TTLs before the event window.

Consolidating two clouds after an acquisition

The pressure is to unify quickly; the risk is unifying before you know what traffic between the two estates looks like. Then: inventory flows and attribute egress per path from billing and flow logs, since inter-cloud chatter is where cost appears; resolve overlapping address space and split-horizon DNS before touching routing; choose interconnect versus internet transit per flow class; only then converge the two failover designs, because two sets of RTO and RPO assumptions cannot merge without deciding which now applies. The multi-cloud and failover work are sequenced, not parallel.

How reporting works

A written findings document

Prose and data, not slides. It states what was measured, under what conditions, what we found, what we changed, and what we deliberately did not change and why. Where a number is an estimate or a single observation, it says so. The document is yours to circulate without us in the room.

A walkthrough session

A working session with your engineers to challenge the conclusions and argue the trade-offs. Most of the value transfers here, because the reasoning matters more than the recommendation — your team makes the next decision without us.

Re-measurement against the baseline

At the end we re-run the original measurements with the frozen instrument and report before and after side by side, including the control metrics predicted not to move. A missed target is reported as plainly as a met one. If the data is too noisy to support a conclusion, we say so rather than pick the favourable window.

Related: the FAQ covers scope and access, cloud monitoring the tooling, the disaster recovery checklist RTO and RPO in depth, and the editorial policy how claims here are reviewed.

Measure first, then decide

To know what a routing or failover change is worth to you, the honest first step is a baseline. Email [email protected] with two lines about the symptom — no form, no phone queue, a reply within one business day.

Start a conversation Read the technical insights

How we measure results — and what we will not promise