Big Data Computing - Course Materials

A 400-level undergraduate course in distributed data systems, built and taught for senior Data Science majors at the U.S. Naval Academy. Covers distributed storage, batch and stream computation, and the theory behind them - with hands-on labs on a real local cluster.

What's here

• Lecture materials and notebooks on distributed storage theory, Hadoop, Spark / PySpark, and Kafka
• Docker Compose lab environments that let each student spin up a locally-simulated multi-node Hadoop / Spark / Kafka cluster on their own machine - no shared-cluster contention, no setup friction

Topics

• Distributed storage and the theory behind HDFS
• Batch computation with Hadoop MapReduce and Spark / PySpark
• Stream processing with Kafka
• Cluster architecture and multi-node simulation via Docker Compose

Why it exists

Built to replace a shared cloud-based lab model. Moving every student onto a self-contained, reproducible local cluster removed setup friction and let the course focus on the concepts.

---

Built and taught by Brett Gentile, CS Instructor, U.S. Naval Academy.