← All tutorials

Skill Zuno · skillzunos.com · System Design

Latency and Throughput tutorial

Learn Latency and Throughput on Skill Zuno. This Latency and Throughput tutorial covers New, Easy, Medium, Hard, and Diff levels so you can study practical Latency and Throughput problems on skillzunos.com.

Open Latency and Throughput on Skill Zuno

Also see the Latency and Throughput interview questions.

Latency is how long one operation takes; throughput is how many operations a system completes per unit of time. Both shape user experience and capacity, and improving one can hurt the other. Interviews cover typical latency numbers, percentiles such as p99, Little's law, and techniques such as batching, caching and parallelism.

Latency and Throughput topics covered

  • Latency versus throughput — Why a system can have high throughput and still feel slow, and the reverse.
  • Latency numbers every engineer should know — Memory, SSD, network within a data centre and across continents, in orders of magnitude.
  • Percentiles and tail latency — Why p99 matters more than the average, and how fan-out amplifies the tail.
  • Little's law — Relating concurrency, throughput and latency to size thread pools and connection pools.
  • Batching, pipelining and parallelism — Raising throughput by doing more per round trip or more at once.
  • Queueing and saturation — Why latency shoots up as utilisation approaches 100 percent.

What you can learn

  • Latency and Throughput tutorial for beginners and experienced developers
  • Level-based practice: New, Easy, Medium, Hard, and Diff
  • System Design topics alongside Latency and Throughput on skillzunos.com

More System Design tutorials

All tutorials