Grokking Streaming Systems: Real-time event processing

"A must-read for engineers and solution architects who work with streams on a regular basis." - Damian Esteban

Every action by a user or a system process generates valuable data for your application or organization. Streaming systems capture and process these events, turning disconnected bits into coherent, useful information sources. Grokking Streaming Systems is a simple guide to the complex concepts you need to start building your own streaming systems. In this friendly, framework-agnostic tutorial, you'll learn how to handle real-time events and how to design and implement a system that's a perfect fit for your needs. Each new idea is carefully explained with diagrams, clear examples, and fun dialogue between perplexed personalities!

about the technology

Many modern organizations rely on real-time event data to ensure top performance. In its raw state, most event data is unfiltered and hard to analyze. Streaming systems address this problem by processing event data so it can be used to create alerts, analysis dashboards, and automated responses, and to trigger other actions within a system. From live financial information to monitoring for signs of a DDoS attack and blocking rogue agents, streaming systems provide a big boost to the health, security, and flexibility of your applications.

about the book

Grokking Streaming Systems helps you unravel what streaming systems are, how they work, and whether they're right for your business. Written to be tool-agnostic, you'll be able to apply what you learn no matter which framework you choose. You'll start with the key concepts and then work your way through increasingly complex examples, including tracking a real-time count of IoT sensor events and detecting fraudulent credit card transactions in real-time. You'll even be able to easily experiment with your own streaming system, by downloading the custom-built and super-simplified streaming framework designed for this book. By the time you're done, you'll be able to easily assess the capabilities of streaming frameworks, and solve common challenges that arise when building streaming systems.

what's inside

- Implement and troubleshoot streaming systems

- Design streaming systems for complex functionalities

- Assess parallelization requirements

- Spot networking bottlenecks and resolve back pressures

- Group data for high-performance systems

- Handle delayed events in real-time systems

about the reader

For readers interested in data processing. Examples in Java.

about the author

Josh Fischer and Ning Wang are both Apache Committers, and part of the project management committee for the Apache Heron distributed stream processing engine. Josh is a software engineer at Scotcro and has worked with moving large datasets in real time for organizations such as 1904labs and Bayer. Ning is a software engineer at Amplitude building real-time data pipelines. He was a key contributor of Apache Heron in Twitter's Real-time Compute team.