Last week in Stream Processing & Analytics 4/25/2016

This is the 11th installment of my blog series around Stream Processing and Analytics.

First two interesting tweets I found last week. The first one by Steve Wilkes brings it straight to the point:

The second one by Neha Narkhede reveals some impressive metrics about the usage of Kafka @ LinkedIn. 1.4 trillion messages a day on 1400 brokers. Kafka is really a game changer!

Last but not least I would like to quote from Mark Palmer’s latest article on 8 Predictions for the Internet of Analytics which I really enjoyed reading:

  • Streaming analytics will become a fundamental topic in computer science. Forrester’s Streaming Analytics Wave defines a set of computer science criteria to define streaming analytics: time windowing, aggregation, correlation, and integration with interactive analytics. These fundamentals are not well understand by the computer science community, are not yet taught in school, and are therefore not yet well known.
  • Data streams will be as important as data lakes. Data lakes contain data at rest; data streams contain data in motion. But most IT applications today are designed around data at rest. In the coming decade, data streams will become as important as data at rest.
  • Streaming analytics and traditional analytics will become increasingly intertwined. In order to apply analytics to streams, you need to know what to look for. Traditional analytics help you look through the rearview mirror at the past, and predict important conditions. Streaming analytics are about looking forward, through your windshield, looking at real-time conditions, and acting.


So that’s it for this week. As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts



Apache Beam

Apache Storm

Apache Spark Streaming

Apache Flink

Apache Apex

Apache Kafka

Apache NiFi / Hortonworks DataFlow

Apache Metron


New Presentations

New Videos

New Podcasts

New Releases / Components

Upcoming Events

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!