When Your Load Generator Mimics Normal Traffic But Not Incident Peaks: Where to Correct

You run a load test. Looks fine. Graphs are flat. Then the real incident hits—and everything falls over. Sound familiar? That's because your load generator mimicked normal traffic but missed the spike patterns that actually break things. Here's where to correct that gap.

Where This Bites You in Real Work

Real-world protocol stress scenarios

A payment gateway fails at 11:38 AM on a Tuesday. Not Black Friday — just a routine morning. The load generator shows 2,000 TPS handled cleanly for weeks. Yet the seam blows out: upstream PostgreSQL hits 100% CPU, connection pool saturates, and every new request queues. What the generator never stressed is the incident peak — a 12:1 read-to-write ratio shift after a cache eviction event. The traffic pattern looked normal because it copied a Tuesday at 10 AM. Wrong order. That hurts.

I have seen this exact shape three times in production post-mortems. Teams run protocol-level stress tests against a single endpoint, measure p50 latency, declare success. Meanwhile, the real failure hides in the corner case: a database connection storm triggered by 400 clients all reconnecting after a 3-second network blip. The test suite simulates steady-state traffic. It never simulates the staggered retry storm — every client reconnecting within a 500-millisecond window. That pattern alone can take down an haproxy cluster in 17 seconds. Honest—I timed it once.

The catch is subtle. Load generators that replay HTTP logs miss the protocol-level handshake cost entirely. They send payloads, sure. But they skip TLS negotiation, they skip authentication token refresh, they skip the exact moment a connection pool drains and every goroutine blocks on conn.acquire(). That's where the protocol stress bites — in the seam between transport and application state.

Database connection storms

Most teams skip this: testing what happens when 200 app instances all lose their database connections at once. The load generator sees 2,000 queries per second and says "fine." But queries are already inside established connections. A storm is not about query volume — it's about connection establishment under lock contention. I once watched a PgBouncer farm collapse because 48 app nodes tried to SELECT 1 on fresh connections within the same second. The test harness had never forced a full disconnect-and-reconnect cycle. It only measured warm-pool throughput. Different failure mode entirely.

What usually breaks first is the listener backlog. TCP SYN queue fills, kernel starts dropping handshakes, clients retry, retries pile up. That positive feedback loop — each failure spawns two more attempts — is invisible when your generator holds connections open for 30 minutes. The real workload drops connections every few seconds. A pattern mismatch. Teams revert to simple throughput tests because connection-storm scenarios are harder to script. Harder, not optional.

‘We tested at 5x normal traffic. The database fell over at 1.2x — but only when every connection was fresh.’

— ops engineer, post-incident retrospective, 2023

That quote captures the disconnect. 5x traffic on warm connections tells you little about 1.2x traffic on cold connections. The load generator mimics normal traffic. Normal traffic doesn't include the 30-second window after a blue-green deploy where every instance reconnects simultaneously. Yet that window happens every deployment. And deployment is when you least want a surprise.

Cache stampedes

Cache stampedes follow the same broken logic. A generator fills Redis with keys, then measures gets at 10,000 ops. Fine. But the incident peak looks different: key expires, 400 threads miss simultaneously, all query the database, all try to SET the same key. The generator never expires keys mid-test. It never simulates the exact microsecond when TTL hits zero across a hot partition. The database sees a spike 40x above baseline — but only for 200 milliseconds. Long enough to blow the connection pool. Short enough that your 1-minute average latency chart looks clean. That's where this bites you in real work: the metric hides the failure.

I fixed one of these by adding a 50-millisecond jitter window to cache re-computation. The test harness had shown no problem because it never forced simultaneous misses. We had to build a synthetic stampede generator — spawn 500 coroutines, expire the same key, measure how many hit the database. First run: 497 hits. After jitter: 3 hits. That 99.4% reduction came from stress-testing the incident peak, not the steady state. The load generator that mimics normal traffic can't catch this. It's not built to. You have to build the secondary test yourself.

A rhetorical question then: how many of your existing stress tests would catch a cache stampede? If the answer requires digging through config files, the answer is likely zero. That's the cost of testing what is easy instead of testing what breaks.

Foundations Readers Confuse

Average Throughput vs Peak Capacity

The single most common mental tripwire I see is teams treating their load generator like a speedometer. They set average requests per second from production dashboards, run a 30-minute test, and call it done. That sounds fine until a real incident hits—because average throughput hides the jagged edges. A service can handle 1,000 req/s smoothly for hours, then collapse at 1,200 for three minutes during a cache-miss storm. The difference isn't the number; it's the buy-in. Most generators don't ramp fast enough to create the abrupt wallop that triggers connection pool exhaustion or GC thrash. We fixed this once by replaying a five-minute log slice where traffic jumped 4x in twelve seconds. The engineering lead stared at the graph and said, 'Our test never looked like that.' He was right—nobody had told the tool to hurt the system, only to walk it.

Wrong order. You build a capacity model around sustained load, then assume that same ceiling holds for bursts. It rarely does. The catch is that thread pools, database connection limits, and request queuing all behave non-linearly when demand spikes faster than the autoscaler can react. Your average-throughput test is essentially a marathon runner's heartbeat check—useful, but irrelevant if the actual injury is a sudden sprint.

Single Pattern vs Multiple Failure Modes

Another quiet failure: teams design one traffic pattern—usually a flat line or a gentle sawtooth—and call it representative. But production doesn't send one shape. You get the coordinated flash crowd after a marketing blast, the slow drip of a retry storm from a misconfigured client, the oscillating heartbeat of a cron job hitting every node at once. Each pattern stresses a different subsystem. Slow drip fills the backlog queue; flash crowd burns CPU on new connections; oscillation confuses the load balancer's smoothing algorithm. What usually breaks first is the part you didn't model.

Most teams skip this: they re-use the same scenario file for six months. The generator keeps humming, the graphs look green, and then a real incident exposes a failure mode that was never in the script. That hurts. I once watched a team chase a tail-latency regression for two weeks—only to realize their load test never sent the tiny percentage of requests that triggered a slow authentication path. The pattern they chose was safe. Too safe.

Field note: emergency plans crack at handoff.

‘You don't need a bigger load generator. You need a better liar about what production actually does.’

— senior SRE, after three on-call rotations from a pattern mismatch

Trade-off here: covering multiple failure modes means more test variants, more data to store, and longer validation cycles. Teams with tight deployment cadences often revert to a single pattern because it's faster. That's a cross-roads. Every new scenario adds maintenance cost—drift, stale fixtures, false positives from expired tokens or shifted baselines. But the alternative is knowing your system survives only the one story you told it. Not yet a universal truth, but close.

Patterns That Usually Work

Chaos engineering for spikes — but targeted

Most teams blast random failures at a system and call it a day. That misses the point. Incident peaks follow predictable shapes — they surge, they oscillate, they decay — and your load generator needs to replicate those shapes, not just flatline at 10x baseline. I have seen this fail hardest when engineers set a concurrency ceiling and walk away. The real trick is injecting transient overload that mimics a cascading failure: one service slows, queuing builds, clients retry, and suddenly your ingress path sees a waveform that looks nothing like a normal ramp. You can build this with a simple state machine that modulates request rate based on latency feedback from the target. That sounds fine until you hit the pitfall — if your chaos tool shares infrastructure with the production path, you amplify risk instead of learning from it. Run chaos for spikes in a shadow environment with production traffic replayed, or you will mistake self-inflicted noise for genuine system behavior.

Wrong order will kill you here.

Start with the failure mode, then build the spike. Most teams reverse this — they build a spike and hunt for what breaks. The catch is that your system may tolerate high load but collapse under specific patterns like sudden drops in request size (header parsing gets hammered) or rapid alternation between hot and cold endpoints. Our load generator once produced a perfect sine wave of traffic. Real incidents look more like a rock tossed into still water — sharp initial splash, ripples that grow quieter, then silence. So simulate that. Use a simple script that sends a burst, waits for error rates to climb, then sends another burst while the first surge is still draining. That reproduces the worst of most real-world peaks: overlapping retry storms.

'We thought 20K requests per second was our breaking point. It was 3K — if you sent them in two-second waves with a half-second gap.'

— engineer on a healthcare API team, after their third on-call rotation that week

Traffic modeling from incident logs — your second brain

Your incident logs already contain every spike pattern you need. Most teams never extract them. Grab the request timestamps from your last five significant incidents, bin them into 100-ms windows, and you have a load profile that's empirically real. The beauty is that this captures the behavioral fingerprints humans miss — like the fact that downstream cache clears often precede the surge by exactly the database query timeout duration. I have used this to rebuild a spike that looked flat in aggregate but was actually a sawtooth of 200-millisecond bursts. The hard part is that logs from normal operations are useless here; you need the incident-specific trace data, which often lives in separate storage or gets rotated out after 30 days. Set up a pipeline that exports incident traffic shapes to a dedicated bucket the minute an alert fires. That one change turns post-mortems into reusable load tests.

The trade-off is fidelity versus portability.

Exact replays of recorded spikes contain noise — client-specific delays, TLS negotiation overhead, garbage collection pauses — that may not reproduce in a test environment. We fixed this by extracting only the inter-arrival times between requests and the endpoint distribution, then rebuilding the spike from those parameters. The resulting load felt synthetic but behaved realistically under stress. A second pitfall: replaying a single incident shape teaches you about that incident, not about edge cases you have never seen. So collect at least three distinct spike types (slow-burn, flash crowd, cascading retry) and parameterize their mix ratios. That way your test suite doesn't overfit to last month's outage.

Control theory ramps — feedback-driven load generation

Set a target latency or error rate, not a raw request rate. Control theory gives you a loop that adjusts load based on how the system responds in real time. This is how real traffic behaves: users leave when pages hang, so the load collapses; then they try again a few seconds later, causing a smaller spike. A proportional-integral controller can mimic that oscillatory damping naturally. The catch is that most load-testing tools assume a static rate. You have to write a small adapter that feeds back the SLO metrics into the generator. I have done this with a Python script that polls the target's p99 latency every 200ms and decreases request rate by 20% if latency exceeds a threshold, then slowly ramps back up. That produces a spike that looks eerily similar to a misconfigured auto-scaler fighting itself.

But here is where most teams revert.

Control loops introduce latency in the test itself. If your feedback cycle is slower than the incident's time-to-failure, you're testing your test harness instead of your system. Keep the feedback period under 500 milliseconds, and accept that you will overshoot during the first few iterations. That overshoot is data — it tells you how much damping your system needs before it oscillates into failure. A final warning: control theory works beautifully for gradual spikes but fails for instantaneous ones (a cache server dying mid-request). For those, fall back to the chaos-engineering approach: a preset burst profile that triggers when you manually inject the fault. Blend both patterns, and your load generator stops being a toy and starts being a credible mimic of the worst days you have not yet seen.

Anti-Patterns and Why Teams Revert

Linear Ramp-Ups

The most seductive mistake in protocol stress testing is the steady stair-step ramp. You start at 100 virtual users, add 50 every minute, and watch the response-time chart climb in a clean slope. That feels scientific. It looks repeatable. The catch is that real incident peaks don't arrive like a metronome — they slam in all at once, often while the system is already handling degraded state. I have seen teams run a linear ramp, declare the system "good to 2,000 concurrent connections," and then collapse at 600 because the actual traffic arrived as a wall, not a staircase. The ramp hides GC thrash, hides connection-pool exhaustion, hides the moment the database queue doubles faster than it can drain. It's a nice chart. It's a useless prediction. The psychological reason teams cling to this is obvious: a linear ramp produces clean failure curves that make for tidy slides. Ugly data gets you questioned. So the ramp stays.

Fixed Think Times

Another anti-pattern: every virtual user waits exactly 3.5 seconds between requests. Why 3.5? Because someone once measured the median think time in production and froze it into a config file. That measurement was taken during a Tuesday afternoon — not during Black Friday, not during a cascading cache miss, not during the five minutes when every client retries simultaneously. Fixed think times squash the single most destructive real-world factor: synchronization of retry storms. When one connection stalls, others don't politely wait 3.5 seconds. They retry in two seconds, then one second, then all at once. Your load generator with fixed think times never produces that pile-up.

Teams revert to fixed think times because they're easy to defend. "We used the production average." That sounds rigorous. It's not. It's a static snapshot of a dynamic system — the load-test equivalent of steering by a photograph. The harder, more honest approach is to model think times as distributions — skewed, bursty, sometimes with a long tail that looks like a mistake but is actually a rogue mobile client. That takes work. Many teams choose the clean number instead.

Reality check: name the preparedness owner or stop.

'We kept the think time constant because changing it made the test 'unstable' — the failure rate jumped around.'

— QA lead, after a post-mortem where the 'unstable' test correctly predicted their production outage three months early

Canned User Journeys

The third trap: recording a single user flow — login, search, add to cart, checkout — and replaying it on a loop, forever. This is what most commercial load-testing tools sell you as a "scenario." It's also what breaks first when production traffic does something slightly different: a user who refreshes mid-checkout, a user who abandons the cart and starts a new search, a user whose session token expires and forces re-login. Canned journeys test one path. Real traffic tests every degenerate branch.

What usually breaks first is session reuse — the load generator quietly recycles tokens because the script says "if error, retry with same credentials," and the system happily accepts replay traffic that looks nothing like human behavior. The failure then gets blamed on "authentication performance" rather than on the fact that your load model is a puppet show. Teams revert to canned journeys because they're cheap to build and easy to document. A single flowchart fits on one slide. A probabilistic model of user behavior — with branching, cancellation, back-button spam, and malformed requests — doesn't. The cost of that simplicity is that your test predicts yesterday's traffic, not tomorrow's meltdown. And tomorrow's meltdown is the one that wakes you up at 3 a.m.

Maintenance, Drift, and Long-Term Costs

Test Script Decay — the Silent Rot

Scripts age like milk, not wine. I have seen teams invest two weeks building a gorgeous load-test suite, only to return six months later and find every assertion silently failing. The API contract shifted — a header renamed, a status code changed from 200 to 201, a pagination cursor required instead of an offset. The load generator kept running, green lights across the dashboard. But it wasn't sending anything close to real traffic. That false green is worse than a red alert. At least a red alert makes you look.

The decay creeps in when nobody owns the test suite full-time. A single dev rotates off the project; the next person treats the scripts like sacred artifacts, too afraid to touch them. Wrong order. You need to treat load tests like production code — code-reviewed, versioned, and wired into your CI pipeline so they break loudly, not silently. Without that, your peak-season simulation becomes a museum exhibit.

What breaks first? Environment-specific URLs hardcoded in the test config. Session tokens that expire faster than the test duration. Third-party mock endpoints that drift from the real service. The fix is boring but necessary: parameterize everything, run a daily smoke test, and flag any deviation over 5% in response structure. We fixed this once by scheduling a Friday-afternoon "load-test audit" — 45 minutes, no exceptions.

Monitoring Burden — When the Cure Costs More

Keeping tests faithful imposes a monitoring tax. You now watch two systems: the application under load and the load generator itself. Most teams skip this. They assume the generator is a black box that always behaves. It doesn't. I have watched a distributed load cluster drift time sync by three seconds, causing all traffic patterns to compress into shorter windows. The result? A false peak that looked like a two-second burst — but was really a batch of delayed requests arriving together. The team spent a week chasing a database contention bug that didn't exist.

The monitoring burden grows in proportion to your test fidelity. High-fidelity traffic replay requires tracking request order, latency distributions, and response correlation. That's a lot of telemetry for something that only runs once a quarter. The trade-off is real: you trade simplicity for accuracy, and accuracy demands observability. If your threshold for detection is a guess, you're operating on hope.

One rhetorical question per section, then: how many teams actually monitor the monitor? Few. And that's where false confidence plants its flag.

False Confidence — the Most Expensive Outcome

False confidence feels productive. The dashboards show green bars, the response-time percentiles sit below threshold, and your team declares the system ready for Black Friday. But those green bars only reflect the traffic you modeled — not the traffic you will actually see. The gap between modeled and real peaks widens with every drift, every skipped dependency, every assumption that went unvalidated. That gap is where money gets lost.

We ran the test every night. Every night passed. Then the real spike arrived and killed us in eleven minutes.

— Site reliability engineer, postmortem retrospective, private conversation

The hard truth: a passing load test is not proof of readiness; it's proof that your generator matched your assumptions. The difference matters when incident traffic includes patterns you never scripted — a cascading retry storm, a cache stampede, a database connection pool exhausted by slow queries from a different microservice. Those scenarios don't appear in your test unless you deliberately inject them. And most teams revert to "happy path" stress because it hurts less to write. That hurts more to debug later. Specific next action: before your next major load test, run a fault-injection session where every service dependency fails. If your generator doesn't survive that, neither will your customers.

When Not to Use This Approach

Steady-state validation

Some systems just don't live on the edge. Think about a batch-processing pipeline that runs every morning at 3 AM, processing yesterday's transactions for four hours straight. Peak traffic? Irrelevant. What kills this system is slow memory growth over 90 minutes, or a connection pool that never recycles under constant load. I have seen teams burn three weeks building a beautiful spike-generating harness for a service that never actually sees spikes—its production traffic looks like a flat line with 5% jitter. The spike obsession cost them steady-state analysis, and the actual outage hit at minute 47 of a sustained run, not in a burst.

Wrong tool for that problem.

If your workload is predictable—if the CPU graph is a straight line with no hockey-stick inflection—then spike replication adds noise, not signal. You need hours-long soak tests with real data volumes, not two-second bursts at 100x baseline. The catch is that most load generators treat "steady" as easy and "spiky" as hard, so teams gravitate toward the harder thing. That hurts. A monitoring dashboard that never wavers is exactly when you should skip spike testing entirely and focus on memory profiles, garbage collection logs, and disk I/O curves over time. Trade the adrenaline for patience.

Limited observability

Here is a hard truth: if you can't see what broke during a spike, replicating the spike tells you nothing. I have walked into postmortems where the team replayed a 10x traffic jump, watched the error rate climb, and then stared at logs that said nothing except "timeout." No traces. No histogram of latency by percentile. No connection state snapshots. The spike arrived—and then it left without a clue. What usually breaks first in those scenarios is the team's confidence, not the system; they start chasing shadows because every spike looks like the same opaque wall of red metrics.

Flag this for emergency: shortcuts cost a day.

Most teams skip this:

• Instrument your endpoints for p99 latency per route before you touch spike generator settings
• Ensure distributed traces sample at least 10% during burst windows—otherwise the rare slow path hides
• Expose connection-pool depth and thread-queue length as real-time metrics, not aggregates

Without that observability, spike testing becomes theater. You press go, the chart turns red, and you learn nothing actionable. Better to spend that time building a dashboard that shows you the whole picture first. Then spike. Not the other way around—that order matters. A rhetorical question for the reader: would you rather fix a known bottleneck from steady-state traces, or guess at an invisible one from a spike you can't inspect?

Regulatory compliance

Some environments forbid you from generating certain traffic patterns at all. Healthcare systems handling PHI, financial platforms under SOX or PCI audits—contractually, you must not run tests that mimic authenticated user surges unless every single payload is scrubbed. The cost of data de-identification for a 500-user spike often exceeds the cost of the hardware itself. I once watched a compliance officer shut down a perfectly good load test because the generated transactions contained synthetic but structurally valid account numbers that looked "too real" under regex scans. Honest work, dead on arrival.

That sounds fine until you realize regulators also expect evidence of resilience under stress. So where to test? Some teams run spike tests on isolated networks using synthetic identities generated by the compliance team itself—slow, painful, but audit-proof. Others skip spike testing entirely and instead prove linear scalability with steady-state profiling, arguing to auditors that capacity planning beats chaos testing. Not every regulator buys that argument, but most accept documentation of known throughput limits rather than seeing spike data they can't approve.

The compromise: use empty payload spikes—metric-only bursts that verify routing and autoscaling without real data flowing. It's not the same as a full spike test, but it avoids the compliance trap. If your legal team says "no spike" flat out, listen. Build soak tests instead. Spike obsession is a luxury not everyone can afford, and pretending otherwise risks your deployment pipeline getting quarantined.

'We stopped spike testing entirely after the third compliance audit rejection. Our steady-state validation caught more bugs in six months than spike tests ever did.'

— infrastructure lead, healthcare SaaS platform, 2023 retrospective

Open Questions / FAQ

What tool supports spike injection?

Most load tools—k6, Locust, Gatling—claim they can do spikes. Try it. You will discover they ramp smoothly, then hold, then ramp down. That's a plateau, not a spike. A real spike needs a sub-second jump from, say, 50 concurrent users to 1,200, then a jagged descent that matches what a flash crowd actually does—erratic, not linear. I have watched teams spend three days tuning Gatling scenarios only to find their CI pipeline could not sustain the connection burst. What actually works? A lightweight companion process that overwrites the target endpoint count at the OS level. We fixed this by pairing Locust with a small Go binary that issues raw HTTP/2 frames on a cron trigger. The tool is secondary; the ability to inject before the orchestrator sees the event is primary.

How to determine spike magnitude?

Look at your real incident logs—not the dashboard averages. Dashboards lie. Averages mask the 90th-percentile burst that arrives in two seconds and vanishes. Most teams skip this: they pick a number like "2× normal peak" and call it done. That's arbitrary and, honestly, dangerous. The correct magnitude is the smallest spike that made your pager go off last quarter. I have seen a 4.3× spike melt a service that handled 6× during a planned stress test—because the 4.3× arrived as a single concurrent wave, not a distributed ramp. The catch is you can't know ahead of time. You run five exploratory injections, measure the recovery slope, and then set the formal magnitude at the point where recovery time doubles. That number changes every six weeks. Build the calibration into your pipeline, not a spreadsheet.

Can machine learning help?

Not reliably—not yet. The appeal is obvious: let a model watch production traffic patterns and generate spike profiles automatically. I have seen three teams sink six months into this. They ended up with a model that produced statistically valid distributions that never triggered the real alarm. Why? ML sees median behavior well. Spikes live in the tail—the rare, ugly, correlated failures that training sets under-sample. A Bayesian model might guess the timing of a spike, but it can't guess the application-level payload that causes a cascading lock escalation. That said, ML can help with one narrow task: flagging when your current spike magnitude has drifted below the 5th percentile of observed anomalies. That's a drift detector, not a generator. Worth running as a sidecar. But don't let it drive the injection engine—the false-negative rate will bite you during a live incident.

We ran 14,000 spike simulations before we found the one that matched our real outage. The ML model had discarded that pattern as noise.

— SRE lead, after a Black Friday incident that the spike generator missed

The takeaway for your next experiment: pick one tool that you control at the socket level, measure spike magnitude from your actual pager data (not synthetic multiples), and treat ML as a drift watchdog, not a prophet. Run that setup alongside your next production deployment. Compare the recovery curves. Then adjust. That's where the correction lives—not in the theory, but in the seam between your generator and what your system actually felt.

Summary + Next Experiments

Key takeaways — what this actually means for your next load test

Normal traffic mimics a gentle river. Incident peaks are flash floods. Your load generator, if it only ever reproduces the river, will leave you blind when the dam breaks. The core fix is not more data or fancier dashboards — it’s a deliberate mismatch between your warm-weather profile and your stress profile. I have seen teams burn two weeks building a “realistic” generator that replayed Tuesday at 2 PM, then wondered why the site fell over on Black Friday. Realistic is a trap when your reality has no spikes.

The tricky bit is admitting your generator lies to you. Most teams skip this: they validate the generator against logged traffic, see a 94% match, and call it good. But 94% on volume means nothing if that missing 6% is the 10x burst that collapses your database connection pool. You need a second generator — or a second mode — that deliberately amplifies arrival rate, shrinks think time, and throws in correlated failures (a search storm followed by a checkout wave). That hurts. It should.

Immediate actions — do these before your next test cycle

Three steps, in order. First, pull your last three real incident logs and identify the peak request rate and the peak concurrency. Not the average — the absolute ceiling right before the SRE team hit the emergency brake. Second, build a separate test scenario that starts at your normal baseline, then steps to 1.5× that ceiling within thirty seconds. Let it hold for two minutes. Measure what breaks, not whether the front page loads. Third, schedule this spike scenario monthly, on the same hardware your production runs today. Don't mix it into your routine load test — teams who do revert to the mean, smoothing the spike back into a tolerable hum.

What usually breaks first is the connection pool, not the CPU. Or the Redis cache eviction policy, not the query planner. You will know you're done when the failure mode changes from “timeout” to “node crash” — that's the point where you start tuning for real.

‘If your generator never surprises your system, your system will always surprise you in production.’

— overheard in an outage post‑mortem, where the load generator had matched normal traffic to 97% accuracy. The remaining 3% took the site down for 41 minutes.

Further reading — where to go when your eyes glaze over

Stop chasing perfect generator fidelity. Instead, read the Tail‑at‑Scale paper (Dean and Barroso, 2013) — yes, it's old — and apply its latency‑tolerance logic to your burst handling. Then look at Netflix’s Chaos Monkey for inspiration: they inject failures on purpose. Your task is easier: you only need to inject bursty volume, not random instance kills. A single afternoon of wiring a Poisson arrival generator to your test harness will teach you more than a week of tuning a replay tool to mirror normal traffic. That's not a metaphor. Code it. Run it. Watch the seam blow out. Then fix it.