Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 21:21 on April 17, 2018
Tags: flink ( 92 ), kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Processing & Analytics – 17.4.2018

This is the 111th edition of my blog series blog series around Stream Processing and Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

What Is Stream Processing? A Gentle Introduction by Srinath Perera
Understanding Batch, Microbatch, and Stream Processing by Jon Block

Apache Kafka / Kafka Streams / Confluent Platform

We ❤ syslogs: Real-time syslog Processing with Apache Kafka and KSQL—Part 2: Event-Driven Alerting with Slack by Robin Moffatt
Event Sourcing With Kafka by Adam Warski
Confluent Platform 4.1 with Production-Ready KSQL Now Available by Rohit

Apache Flink

Apache Flink Basic Transformation Example by Upender Chinthala

Spark Streaming

Bringing Complex Event Processing to Spark Streaming by Alex Kotopoulis
Continuous Deployment with Spark Streaming (Part I) by Tommy Shiou
Continuous Deployment with Spark Streaming (Part II) by Tommy Shiou
Spark receiver as a key part of your streaming application by Oleksii Yermolenko

StreamSets

Efficient Splunk Ingest for Cybersecurity by Pat Patterson

Apache NiFi / HDF

Ingesting JMS Messages From TIBCO With Apache NiFi by Tim Spann
Moving into the big (data) league with rt-ai Edge and Apache NiFi by Richards

Presentations

Scaling stream data pipelines with Pravega and Apache Flink by Till Rohrmann & Flavio Junqueira
Operating Flink on Mesos at Scale by Jörg Schad and Biswajit Das
Stream Processing Revolutionizing Big Data by Srikanth Satya
Flink real-time analysis in CloudStream Service of Huawei Cloud by Jinkui Shi & Radu Tudoran
Deep learning on HDP by Tim Spann
Writing Continuous Applications with Structured Streaming in PySpark by Jules Damji
Akka, Spark or Kafka? Selecting The Right Streaming Engine For the Job by Dean Wampler
10 essentials steps for kafka streaming services by Ben Stopford
Streaming Machine learning Distributed Pipeline for Real-Time Uber Data Using Apache APIs: Kafka, Spark, HBase by Carol McDonald
Microservices Part 3 Service Mesh and Kafka by Araf Karsh Hami
Kafka streams by Giridhar Addepalli
What turns stream processing from a tool into a platform? by Sephan Ewen

Videos

Analyzing Data Streams in Real Time with Amazon Kinesis by Allan Macinnis & Milian Brahmbhatt
Akka, Spark or Kafka? Selecting The Right Streaming Engine For the Job by Dean Wampler
The Power of Kafka in the Oracle Cloud: A Deep Dive into Event Hub Cloud Service by Arturo Viveros
10 essentials steps for kafka streaming services by Ben Stopford

New Releases

Confluent Platform 4.1.0

Upcoming Events

17.4.2018 (Laurel, US) – Huge steps for Apache NiFi – Talks on NiFi in Finance and Powerful New Features (Meetup)
17.4.2018 (Berlin, GE) – Deep in the weeds with Apache NiFi, MiNiFi and NiFi Registry (Meetup)
17.4.2018 (Leeds, UK) – Look Ma, no Code! Apache Kafka and KSQL (Meetup)
18.4.2018 (online) – Top 10 FAQs for KSQL, Streaming SQL for Apache Kafka (Confluent Webinar)
18.4.2018 (Munich, GE) – Confluent and Payback talk about Kafka, KSQL and Spark (Meetup)
19.4.2018 (online) – Accelerate Decision Making with Real-Time Analytics on AWS (AWS Webinar)
24.4.2018 (San Francisco, US) – First StreamSets User Group Meetup (Meetup)
25.4.2018 (Berlin, GE) – KSQL and Stream all things with Gwen Shapira and Matthias J Sax (Meetup)
13.6.2018 (St. Louis, US) – Apache Heron (Meetup)
14.6.2018 (Hamilton, US) – Predictive Maintenance with IoT, Apache NIFi, MiniFi and Blockchain (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 22:34 on April 9, 2018
Tags: flink ( 92 ), kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Processing & Analytics – 9.4.2018

This is the 110th edition of my blog series blog series around Stream Processing and Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Real-Time Sentiment Analytics Becomes Fashionable for Retail by Michael Vizard

Apache Kafka / Kafka Streams / Confluent Platform

We ❤️ syslogs: Real-time syslog Processing with Apache Kafka and KSQL—Part 1: Filtering by Robin Moffatt
Of Streams and Tables in Kafka and Stream Processing, Part 1 by Michael G. Noll
Getting started with contributing to Apache Kafka (Part 1): Build and run Kafka from source code by Manasvi Gupta
“Kongo” Part 2: Exploring Apache Kafka application architecture: Event Types and Loose Coupling by Paul Brebner

Apache Flink

Stateful stream processing with Apache Flink by Fabian Hueske & Stephan Ewen

Spark Streaming

Continuous Deployment with Spark Streaming (Part I) by Thommy Shiou

Apache NiFi / HDF

Apache NiFi – Secure Keytab Access by Bryan Bende
Automate workflow deployment in Apache NiFi with the NiFi Registry by Pierre Villard
Using A TensorFlow “Person Blocker” With Apache NiFi by Tim Spann

Presentations

Kafka Connect & Kafka Streams/KSQL – the ecosystem around Kafka by Guido Schmutz
Apache Kafka® and Stream Processing at Pinterest by Liquan Pei & Boyang Chen & Shawn Nguyen
Event-driven Architectures with Spring Cloud by Kenny Bastani
Serverless Data Processing at Scale: An Amazon.com Case Study by Kashif Imran & Haoyu Chen
Building Enterprise-Grade Serverless Apps by Tim Wagner & Cecilia Deng & Satish Malireddi
Enterprise IIoT Edge Processing with Apache NiFi by Timothy Spann

Videos

HDInsight: Streaming Petabytes of IoT Data in Real-Time by Raghav Mohan

Book

Concepts and Patterns for Streaming Services with Apache Kafka by Ben Stopford

New Releases

Apache NiFi 1.6.0

Upcoming Events

10.4.2018 (Lugano, CH) – Apache Kafka for near real-time event processing (Meetup)
17.4.2018 (Laurel, US) – Huge steps for Apache NiFi – Talks on NiFi in Finance and Powerful New Features (Meetup)
17.4.2018 (Berlin, GE) – Deep in the weeds with Apache NiFi, MiNiFi and NiFi Registry (Meetup)
17.4.2018 (Leeds, UK) – Look Ma, no Code! Apache Kafka and KSQL (Meetup)
18.4.2018 (online) – Top 10 FAQs for KSQL, Streaming SQL for Apache Kafka (Confluent Webinar)
18.4.2018 (Munich, GE) – Confluent and Payback talk about Kafka, KSQL and Spark (Meetup)
19.4.2018 (online) – Accelerate Decision Making with Real-Time Analytics on AWS (AWS Webinar)
24.4.2018 (San Francisco, US) – First StreamSets User Group Meetup (Meetup)
25.4.2018 (Berlin, GE) – KSQL and Stream all things with Gwen Shapira and Matthias J Sax (Meetup)
13.6.2018 (St. Louis, US) – Apache Heron (Meetup)
14.6.2018 (Hamilton, US) – Predictive Maintenance with IoT, Apache NIFi, MiniFi and Blockchain (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 20:32 on April 4, 2018
Tags: flink ( 92 ), kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), Stream Processing ( 241 ), streamsets ( 74 )

Last Week in Stream Processing & Analytics – 4.4.2018

This is the 109th edition of my blog series blog series around Stream Processing and Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

AI-Focused Economist Summit Highlights Importance of Real-Time Data by DataStax Blog
How AI Transforms the Real-time Customer Experience by Joe McKendrick
Give meaning to 100 billion analytics events a day by Alban Perillat-Merceroz
Spark Streaming vs Flink vs Storm vs Kafka Streams vs Samza : Choose Your Stream Processing Framework by Chandan Prakash

Apache Kafka / Kafka Streams / Confluent Platform

Kafka and Spark Streams: Living Happily Ever After by Anmol Sarna
Continuous Conversion of Kafka topics from JSON to Avro with KSQL by Robin Moffatt
Monitoring Kafka Consumer Lag in Seconds by Jeremy Donahue
Running Out Of Memory-Mapped Files with Apache Kafka by Jeremy Donahue

Apache Flink

Stateful stream processing with Apache Flink by Fabian Hueske & Stephan Ewen

Heron

Introducing Heron on Nomad by Boyang Jerry Peng

Streamsets

Kafka + TLS/Kerberos in Cluster Streaming Mode is coming! by Adam Kunicki

Apache NiFi / HDF

Presentations

Building a company-wide data pipeline on Apache Kafka – engineering for 150 billion messages per day by Yuto Kawamura
Building Stream Processing as a Service by Steven Wu

Videos

Introduction to Apache Kafka by Jakub Scholz
Running Apache Kafka on Kubernetes by Jakub Scholz
Real time analytics with Apache Kafka for HDInsight by Raghav Mohan

Upcoming Events

10.4.2018 (Lugano, CH) – Apache Kafka for near real-time event processing (Meetup)
17.4.2018 (Laurel, US) – Huge steps for Apache NiFi – Talks on NiFi in Finance and Powerful New Features (Meetup)
17.4.2018 (Berlin, GE) – Deep in the weeds with Apache NiFi, MiNiFi and NiFi Registry (Meetup)
24.4.2018 (San Francisco, US) – First StreamSets User Group Meetup (Meetup)
25.4.2018 (Berlin, GE) – KSQL and Stream all things with Gwen Shapira and Matthias J Sax (Meetup)
13.6.2018 (St. Louis, US) – Apache Heron (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!