What Is Throughput?

If Latency is how fast you can drive Throughput is how wide the highway is.

It is the measure of capacity.

Simple Definition

Throughput is the amount of material or items passing through a system or process. In software it is usually measured in Requests Per Second (RPS) or Transactions Per Second (TPS).

It answers the question “How much work can we handle at once?”

A single server might handle 500 RPS. A cluster of ten servers handles 5000 RPS.

Rate of successful message delivery (RPS)

Throughput counts the successful actions. If your server is rejecting 50% of traffic your throughput is low even if the traffic is high.

Throughput vs Latency

These two concepts are often confused but they are different.

• Latency: Speed. (How long does one request take?)
• Throughput: Capacity. (How many requests can we handle per second?)

You can have a system with high latency (slow) but high throughput (handles millions of slow requests). Think of a cargo ship. It is slow (high latency) but it carries a massive amount of goods (high throughput).

Visualizing

How do you show capacity on a diagram?

Arrow thickness or load balancer distribution

In a System Architecture Diagram you can use arrow thickness to represent volume. A thick arrow from the Load Balancer to the App Server implies high throughput.

You can also label the connection lines. “Expected Load: 10k RPS.”

This helps architects size the infrastructure. “If we have 10k RPS we need a bigger database instance.”

Related Terms

To talk about scale you need these terms:

• Bandwidth: The maximum theoretical throughput of a network connection.
• Bottleneck: The part of the system that limits the overall throughput.
• Load Testing: The process of artificially generating high throughput to see when the system breaks.

For more on visualizing high-scale systems check out our System Design Guide and What Is Caching?.