Performance Testing to Find System Requirements

Note: this happened a while ago, and I don’t remember the exact numbers anymore. Every RPS figure, user count, and server spec below is a placeholder, made up purely to illustrate the shape of the problem. The situation is real; the numbers are not.

What would you do if you were asked to run performance tests and hand the sales team a system requirement they could quote during a pitch, for a brand new application with no historical usage data and no idea what real usage patterns would look like?

The more I looked at the ask, the more ambiguous it became. It was clear I wasn’t going to get firm answers from stakeholders. I’d have to make a lot of assumptions myself and use them as a baseline for testing.

So I started by asking myself a few questions.

Who is the audience, and what’s the usage pattern?

Who would actually use this application, and what would a concurrent user count realistically look like?

The application was built for radiologists preparing reports. It was designed to run on-premises, in institutions like hospitals and radiology clinics, deployed as a Docker Compose stack on a single VM, with no option to scale dynamically. That ruled out “just add more servers” as an answer, the hardware had to be sized up front.

To answer my own question, I turned to one of the doctors my company had on staff to help with product development. I asked them to use the application the way a radiologist actually would, while I watched server logs and a browser session recording to capture the request pattern.

That gave me a rough picture of three things:

1. Auto-save called a server API repeatedly while a report was being edited. My hunch was that this would be the bottleneck, since it looked like it was firing at roughly 2 requests per second per active session.
2. Auto-correct was also a high-frequency endpoint, hit constantly as the radiologist typed.
3. Report submission was a heavier, higher-latency endpoint, but it only fired once per report, when the doctor finished and submitted.

That gave me a working model of what a normal session looked like on the server. Assume 10 concurrent users. Auto-save alone, at roughly 2 RPS per session, would mean around 20 RPS hitting that one endpoint. I coded that request pattern into a Gatling simulation, since it let me model concurrent users and ramp-up profiles closely enough to what I’d observed, and had something to actually run against.

What hardware do I even test against?

The next question was what server configuration to test on, but answering that required already having a server with a known capacity. Classic chicken-and-egg problem: I needed a baseline to test, but the baseline itself was the thing I was supposed to discover.

Since the whole stack would ship as Docker Compose on a single VM, the test had to target one machine’s worth of CPU and RAM, not a cluster. I picked a starting point, a VM with a fixed amount of CPU and RAM, ran the Compose stack on it, noted the spec down as an assumption, and moved on. If it turned out to be wrong, I could always test other configurations once I had a working result.

Running the test, hitting the first wall

With the load pattern and machine defined, I ran the test. The server got overwhelmed almost immediately: requests piled up in a queue, and client-side response times became painfully slow.

Digging into it, the root cause turned out to be a handful of GET endpoints running slow due to missing indexes. Once that was fixed, I re-ran the test, and this time the server comfortably absorbed the load. The result: a machine with 8 cores and 16 GB of RAM could handle 10 concurrent users.

The numbers were too high to sell

I published the result, and the obvious problem surfaced immediately: that hardware requirement was a hard sell. Nobody wants to hear “you need a server this expensive” as the opening line of a sales pitch.

So I reframed the question: this spec is what you need for 10 concurrent users. If you have fewer, I can find you a smaller requirement.

Back to testing, this time with lower load and less powerful hardware. That run succeeded too. I pushed further and tried an even smaller configuration, where response times came in just barely within acceptable limits, no real margin left.

By the end, I had three concrete data points instead of zero:

1. Bare minimum hardware to run the software at all.
2. Recommended spec for a single concurrent user.
3. Recommended spec for a small number of concurrent users.

I started with almost nothing to anchor the testing against, and ended up with a usable, tiered answer the sales team could actually quote.

The takeaway

This is the part of the job that doesn’t show up in a ticket description: you’re handed a question with no clear inputs, and you have to manufacture a reasonable starting point yourself, test it, and let the results tell you where to go next. Each pass through that kind of ambiguity leaves me with a slightly clearer answer than the one I started with, and that’s usually enough to move forward.

Have you been in a similar spot, where you started with almost nothing and the picture only became clear as you went? How did you handle it? I’d like to hear about it.