Measuring OTP Delivery Latency — Time-to-OTP Metrics and SLOs

OTP delivery latency — often called time-to-OTP — is the elapsed time between the moment a code is requested and the moment it is available to the user or system that needs it. The most useful way to measure it is per activation, using timestamps you control, aggregated into percentiles and segmented by country and service. Averages hide the slow tail that actually frustrates users and breaks tests, so build your dashboards and internal SLOs around p50, p90 and p99 rather than the mean.

This guide explains what to timestamp, how to compute and store the metric, which factors move it, and how to turn your own baseline into objectives your team can act on.

Define the metric precisely

"How long did the SMS take?" sounds simple until two teams compute it differently. Before you instrument anything, agree on the start and end events.

Candidate start events

• T0 — activation created. The moment your system calls POST /v1/activations. Useful for capacity planning, but it includes the time your test or user spends entering the number.
• T1 — code requested on the target platform. The moment your app or test clicks "Send code". This is the most honest start point for delivery latency, because everything after it is outside your UI.

Candidate end events

• T2 — SMS received by the gateway. For OTP Route, this is when the activation transitions to received, signalled by the activation.sms_received webhook.
• T3 — code consumed. The moment your system reads the code and submits it.

The core metric is T2 − T1: delivery latency. Track T2 − T0 as end-to-end activation time and T3 − T2 as your own processing time. Splitting the interval this way tells you immediately whether a regression lives in the network or in your code.

Instrumenting with OTP Route

Capture timestamps on your side

Use a monotonic clock for durations within one process and UTC wall-clock time for anything that crosses process boundaries. Record T0 when the activation is created and T1 when the send-code action fires:

import time, datetime, requests, os

API = "https://otproute.com/api/v1"
H = {"Authorization": f"Bearer {os.environ['OTPROUTE_API_KEY']}"}

def utc_now():
    return datetime.datetime.now(datetime.timezone.utc)

t0 = utc_now()
act = requests.post(f"{API}/activations", headers=H,
                    json={"service": "example_app", "country": "gb"}, timeout=15).json()

# ... enter act's number into the form ...
t1 = utc_now()  # immediately after clicking "Send code"

store_event(activation_id=act["id"], t0=t0, t1=t1, country="gb", service="example_app")

Use the webhook for T2

Polling introduces measurement error equal to your polling interval: if you poll every five seconds, your observed T2 can be late by up to five seconds. Webhooks avoid that. When the activation.sms_received event arrives, verify the signature first, then record T2.

The X-OTPRoute-Signature header carries t=<unix>,v1=<hex>. The t value is the time the event was signed, which gives you a server-side timestamp independent of your receiver's queueing. Record both:

// Express handler (signature verification omitted here; see the HMAC guide)
app.post("/hooks/otproute", express.raw({ type: "application/json" }), (req, res) => {
  const header = req.get("X-OTPRoute-Signature") || "";
  const signedAt = Number(header.split(",").find((p) => p.startsWith("t="))?.slice(2));
  const event = JSON.parse(req.body.toString("utf8"));

  if (event.type === "activation.sms_received") {
    metrics.recordReceipt({
      activationId: event.data.id,          // payload fields illustrative; see /docs
      t2Signed: new Date(signedAt * 1000),  // when the event was signed
      t2Received: new Date(),                // when your receiver got it
    });
  }
  res.sendStatus(200);
});

The gap between t2Signed and t2Received measures webhook transport plus your ingress — useful for spotting a slow load balancer or a saturated receiver. Signature verification is covered in securing OTP webhooks with HMAC, and the end-to-end receiver pattern in webhook-based OTP capture.

Record terminal outcomes too

An activation that never receives an SMS has no T2, and silently dropping it from your dataset will make latency look better than reality. Record activation.expired and activation.cancelled events as censored observations: the latency was at least the time until expiry or cancellation. Report them as a separate non-delivery rate alongside the percentiles.

Store and aggregate correctly

Store raw events, not just averages

Keep one row per activation: IDs, country, service, T0–T3, terminal status. Raw rows let you recompute any percentile, segment or window later. Pre-aggregated averages cannot be un-averaged.

Use percentiles

Latency distributions are typically skewed with a long tail. The mean is pulled around by a handful of slow deliveries and describes almost nobody's experience. Report:

• p50 — the typical experience.
• p90 — what a noticeable minority sees.
• p99 — the tail that drives support tickets and flaky tests.

If you use a metrics system with histograms, choose bucket boundaries that resolve the range you actually care about, and compute percentiles from histograms rather than averaging per-host percentiles, which is mathematically invalid.

Segment before you conclude

A global p90 can hide a single country with a problem. Slice by:

What affects OTP latency

Most of the delivery path sits between the sending platform and the receiving number. Understanding the moving parts helps you decide which ones you can influence.

• The sender's own pipeline. The platform generating the code may queue, rate-limit or batch outbound SMS, and may route through different aggregators per destination.
• Carrier routing and interconnects. Messages can traverse several networks before reaching the destination carrier. Each hop adds variance.
• Destination-country rules. Some markets apply sender registration, content filtering or time-of-day restrictions that can delay or block messages.
• Sender-side anti-abuse controls. Platforms may slow or withhold codes for requests they consider risky, such as rapid repeated attempts.
• Your own retries. Clicking "Resend" can invalidate the first code or trigger throttling. Measure from the first request and record resends separately.
• Your receiver. A slow webhook endpoint or a long polling interval inflates observed latency even when delivery was fast.

The levers you control are country choice and fallbacks (country routing for OTP), retry discipline, and your own receiver performance. Broader delivery-failure patterns are covered in reducing failed OTP deliveries.

Building the dashboard

A practical OTP latency dashboard has four panels:

1. Time-to-OTP percentiles over time — p50, p90 and p99 as lines, filterable by country and service.
2. Non-delivery rate — the share of activations ending expired or cancelled without a code, on the same time axis.
3. Heatmap by country × hour — reveals time-of-day effects in specific markets.
4. Processing time (T3 − T2) — confirms your own stack is not the bottleneck.

Annotate the dashboard with deploys and configuration changes. When latency shifts, the first question is always "did we change something?"

The OTP Route developer console's usage logs are a useful cross-check when an individual activation looks wrong; your own dashboard remains the place for trends.

Setting internal SLOs

Avoid adopting a latency target from a blog post or a vendor slide. Targets only mean something relative to your traffic mix, countries and senders. A sound process:

1. Collect a baseline. Run at least a few weeks of measurement across the countries and services you care about.
2. Write the SLI precisely. For example: "the proportion of activations for service X in country Y where T2 − T1 is below a threshold, measured over a rolling 28-day window."
3. Choose thresholds from the data. Set the threshold near your observed p90 or p95 so the objective is achievable but still detects regressions.
4. Set separate objectives per segment. A single global SLO lets a healthy large market mask a struggling small one.
5. Define the response. Decide in advance what happens when the error budget is spent — switch the default country, raise test timeouts, or open an investigation with the sender's team.
6. Review quarterly. Distributions drift as senders and carriers change their infrastructure.

Link SLOs to test timeouts

If you run SMS OTP tests in CI, derive the test timeout from the measured tail rather than guessing. A timeout tuned slightly above your p99 keeps false failures rare while still failing fast enough to cancel the activation and retry. Because unreceived activations are refunded, the cost of a timeout is runner time, not wallet balance.

A note on responsible measurement

Measure latency against applications you own or are authorised to test. OTP Route is built for QA, app verification flows and operational use; respect the terms of the platforms involved and do not generate traffic designed to probe or abuse third-party systems.

See the API docs for event payload details and the OTP gateway overview for how activations are routed.

Frequently asked questions

Topics: OTP delivery latency · time to OTP · SMS latency monitoring · OTP SLO · OTP percentiles · SMS delivery metrics · OTP observability