← All tutorials
Skill Zuno · skillzunos.com · System Design
MapReduce tutorial
Learn MapReduce on Skill Zuno. This MapReduce tutorial covers New, Easy, Medium, Hard, and Diff levels so you can study practical MapReduce problems on skillzunos.com.
Open MapReduce on Skill ZunoAlso see the MapReduce interview questions.
MapReduce is a programming model for processing huge datasets in parallel across a cluster. A map step turns input records into key-value pairs, a shuffle groups them by key, and a reduce step combines each group. Introduced by Google and popularised by Hadoop, its ideas live on in Spark and modern data pipelines. Interviews cover the phases, fault tolerance, data locality and when to use it.
MapReduce topics covered
- Map, shuffle and reduce — The three phases, illustrated with the classic word-count example.
- Distributed file systems — How HDFS and GFS split files into replicated blocks for parallel processing.
- Data locality — Moving computation to where the data is instead of moving data across the network.
- Fault tolerance — Re-running failed tasks, and handling slow stragglers with speculative execution.
- Combiners and partitioners — Reducing shuffle traffic and controlling which reducer receives each key.
- MapReduce versus Spark — Why Spark's in-memory processing replaced disk-based MapReduce for most workloads.
What you can learn
- MapReduce tutorial for beginners and experienced developers
- Level-based practice: New, Easy, Medium, Hard, and Diff
- System Design topics alongside MapReduce on skillzunos.com