Last week in Stream Processing & Analytics 4/25/2016

This is the 11th installment of my blog series around Stream Processing and Analytics.

First two interesting tweets I found last week. The first one by Steve Wilkes brings it straight to the point:

The second one by Neha Narkhede reveals some impressive metrics about the usage of Kafka @ LinkedIn. 1.4 trillion messages a day on 1400 brokers. Kafka is really a game changer!

Last but not least I would like to quote from Mark Palmer’s latest article on 8 Predictions for the Internet of Analytics which I really enjoyed reading:

  • Streaming analytics will become a fundamental topic in computer science. Forrester’s Streaming Analytics Wave defines a set of computer science criteria to define streaming analytics: time windowing, aggregation, correlation, and integration with interactive analytics. These fundamentals are not well understand by the computer science community, are not yet taught in school, and are therefore not yet well known.
  • Data streams will be as important as data lakes. Data lakes contain data at rest; data streams contain data in motion. But most IT applications today are designed around data at rest. In the coming decade, data streams will become as important as data at rest.
  • Streaming analytics and traditional analytics will become increasingly intertwined. In order to apply analytics to streams, you need to know what to look for. Traditional analytics help you look through the rearview mirror at the past, and predict important conditions. Streaming analytics are about looking forward, through your windshield, looking at real-time conditions, and acting.

 

So that’s it for this week. As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Comparison

Apache Beam

Apache Storm

Apache Spark Streaming

Apache Flink

Apache Apex

Apache Kafka

Apache NiFi / Hortonworks DataFlow

Apache Metron

StreamSets

New Presentations

New Videos

New Podcasts

New Releases / Components

Upcoming Events

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

Advertisements

Last week in Stream Processing & Analytics 4/18/2016

This is the 10th installment of my blog series around Stream Processing and Analytics.

Two days later than planned, was traveling and had again trouble with my power supply 😦

So what happened in the world of Stream Processing? For me the most interesting news last week was the release of Storm 1.0.

I’m a  storm user for more than 3 years now and this is really a significant release that delivers several features that pertain to enterprise readiness, operational simplicity and ease of use. I really like that Storm now has native Windowing and State Management Support, Automatic Back Pressure Support and the new connectors for Cassandra, Elasticsearch and Kafka.

Nathan Marz, the founder and creator of Storm also tweeted about it:

And Ian Hellström already updated his stream processing overview chart with Storm 1.0.0.

apache-streaming6

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Apache Beam

Apache Storm

Apache Spark Streaming

Apache Flink

Apache Kafka

Apache NiFi / Hortonworks DataFlow

Apache Metron

Striim

StreamSets

IBM Quarks

New Presentations

New Videos

New Releases / Components

Upcoming Events

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

Last week in Stream Processing & Analytics 4/11/2016

This is the 9th installment of my blog series around Stream Processing and Analytics.

First I have to mention a blog article I have somehow missed last month. It nicely compares the various streaming frameworks available from the Apache software foundation.

 

Last week Forrester published it’s updated Forrester Wave for Big Data Streaming Analytics products. Forrester Research defines Big Data Streaming Analytics as

Software that can filter, aggregate, enrich, and analyze a high throughput of data from multiple, disparate live data sources and in any data format to identify simple and complex patterns to provide applications with context to detect opportune situations, automate immediate actions, and dynamically adapt.

Here the Leaders, the Strong Performers and Contenders as seen by Forrester:

cez8rzywwaadrg8

Source: The Forrester Wave: Big Data Streaming Analytics, Q1 16

 

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Apache Storm

Apache Spark Streaming

Apache Flink

Apache Kafka

Apache NiFi / Hortonworks DataFlow

StreamSets

New Presentations

New Videos

New Releases / Components

Upcoming Events

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

Last week in Stream Processing & Analytics 4/4/2016

This is the 8th installment of my blog series around Stream Processing and Analytics. It’s two days later, due to some technical difficulties. Forgot power adapter at home on Monday and I’m traveling 😉 87 EUR and one day later I’m back in business 🙂

As expected, there were quite a lot of topics around stream processing and streaming analytics at the Strata conference last week.

Jay Krebs and Neha Narkhede from Confluent both mentioned it on Twitter:

And Jack Vaughan summarized it in his blog article:  “Moving streams of data is a must in many modern applications. As a result, streaming analytics applications with Spark Streaming, Kafka and other components are coming to the big data forefront.

Definitely very interesting times ahead 🙂

As usual, just find what I have noticed last week:

News and Blog Posts

General

Comparison

Apache Storm

Apache Flink

Apache Kafka

Goggle Cloud Dataflow / Apache Beam

MapR Streams

Apache NiFi / Hortonworks DataFlow

Oracle Stream Explorer

StreamSets

New Presentations

New Podcasts

New Videos

New Books

New Releases / Components

Upcoming Events

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!