Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 07:28 on June 14, 2017
Tags: flink ( 58 ), heron ( 15 ), kafka ( 67 ), kafka-connect ( 5 ), kafka-streams ( 22 ), nifi ( 60 ), spark-streaming ( 60 ), Stream Processing ( 63 )

Last week in Stream Processing & Analytics – 13.6.2017

This is the 70th edition of my blog series blog series around Stream Processing and Analytics!

This weeks edition is really large, as both the videos from Spark Summit SF and Kafka Summit NY talks have been published last week.

Last weekend I have watched some of the presentations of Kafka Summit NY, obviously I was not able to watch them all, but what I have seen so far is definitely worth the time! Thanks to both Confluent and Databricks for providing the recordings of all the sessions!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Stream Processing for Enterprise IT by Robin Miller
Stream processing in e-commerce by Alexander Dean

Apache Kafka / Kafka Streams / Confluent Platform

Introduction to Apache Kafka™ for Python Programmers by Matt Howlett
Load Testing Kafka With Ranger by Matija Gobec
An Introduction to Kafka Streams by Sangeet Aguli
Announcing Data Collector ver 2.6.0.0 by Kirit Basu
Configuring Kafka Connect to log REST HTTP messages to a separate file by Robin Moffatt
A super quick comparison between Kafka and Message Queues by Hendrik Swanepoel

Spark Streaming

What’s the big deal with event-by-event streaming in Spark’s 2.2 release?

Heron

Heron on DC/OS by John Crawford

Apache Flink

Apache Flink on DC/OS (and Apache Mesos) by Till Rohrmann
Apache Flink® at MediaMath: Rescaling Stateful Applications in Production by Stefan Richter & Seth Wiesman and Michael Winters

Apache NiFi / Hortonworks HDF

Real-Time SQL On Event Streams by Mark Payne
Announcing the general availability of Hortonworks DataFlow 3.0 by Wei Wang
Dynamic IIoT Dataflows for Demand Based Surveillance via Apache NiFi by Chris Herrera

StreamSets

Introducing the Data Collector Support Bundle by Wagner Camarao
Announcing Data Collector ver 2.6.0.0 by Kirit Basu

New Presentations

Connect K of SMACK:pykafka, kafka-python or? by Shuhsi Lin
Easy, Scalable, Fault-Tolerant Stream Processing with Structured Streaming in Apache Spark by Michael Armbrust & Tathagata Das
Etl is Dead; Long Live Streams by Neha Narkhede
Building Data Product Based on Apache Spark at Airbnb by Jingwei Lu & Liyin Tang
SSR: Structured Streaming on R for Machine Learning by Felix Cheung
StreamSets and spark by Hari Shreedharan
How to implement a real-time streaming data analytics engine by Juantomás García
Governance using Apache Atlas: Why and How by Vimal Sharma
Structured Streaming for Columnar Data Warehouses by Jack Gudenkauf
Fast Data at ING – the why, what and how of the streaming analytics platform in a financial enterprise by Bas Geerdink
Real-time Machine Learning Analytics Using Structured Streaming and Kinesis Firehose by Caryl Yuhas & Myles Baker
The History and Future of the Blurring of Stream Processing & OLTP by John Hugg

New Videos

Building Continuous Application with Structured Streaming and Real Time Data Source by Nan Zhu & Arijit Tarafdar
Scalable Monitoring Using Apache Spark and Friends by Utkarsh Bhatnagar
Dynamic DDL: Adding Structure to Streaming Data on the Fly by David Winters & Hao Zou
Real-Time Machine Learning Analytics Using Structured Streaming and Kinesis Firehose by Caryl Yuhas & Myles Backer
A Practical Approach to Building a Streaming Processing Pipeline for an Online Advertising Platform by Yifan Wang & Amit Ramesh
SSR: Structured Streaming on R for Machine Learning by Felix Cheung
The Top Five Mistakes Made When Writing Streaming Applications by Mark Grover & Ted Malaska
Structured-Streaming-as-a-Service with Kafka, YARN, and Tooling by Jim Dowling
Structured Streaming Demo by Michael Armbrust
Easy, Scalable, Fault-Tolerant Stream Processing with Structured Streaming in Apache Spark (part1, part2) by Michael Armbrust & Tathagata Das
Expanding Apache Spark Use Cases in 2.2 and Beyond by Matei Zaharia &Tim Hunter & Michael Armbrust
How to Write Batch or Streaming Data Pipelines with Apache Beam in 15 mins by James Malone
Structured Streaming for Columnar Data Warehouses by Jack Gudenkauf
Introducing Exactly Once Semantics in Apache Kafka by Apurva Mehta
Data Processing at LinkedIn with Apache Kafka by Joel Koshy & Kartik Paramasivam
The Rise of the Streaming Platform by Jay Kreps
The Real-time Event Driven Bank: A Kafka Story by Ferd Scheepers
Stream It Together: 3 Realities of Modern Programming by Neha Narkhede
Kafka in the Enterprise: What if it Fails? by Anton Gorshkov
Apache Kafka Core Internals: A Deep Dive by Jun Rao
How to Lock Down Apache Kafka and Keep Your Streams Safe by Rajini Sivaram
Running Hundreds of Kafka Clusters with 5 People by Thomas Crayford
Hardening Kafka for New Use Cases with Venice by Matthew Wise
Portable Streaming Pipelines with Apache Beam by France Perry
The Best Thing Since Partitioned Bread: Rethinking Stream Processing with Apache Kafka’s new Streams API by Michael Noll
Microservices with Kafka: An Introduction to Kafka Streams with a Real-Life Example by Alexis Seigneurin
Hanging Out With Your Past Self In VR: Time-Shifted Avatar Replication Using Kafka Streams by Greg Fodor
Building Advanced Streaming Applications using the Latest from Apache Flink and Kafka by Jamie Grier
The Data Dichotomy: Rethinking Data & Services with Streams by Benjamin Stopford
Easy, Scalable, Fault-tolerant Stream Processing with Kafka and Spark’s Structured Streaming by Tathagata Das
Scalable Real-time Complex Event Processing at Uber by Shuyi Chen
Capture the Streams of Database Changes by Randall Hauch
Billions of Messages a Day – Yelp’s Real-time Data Pipeline by Justin Cunningham
California Schemin’! How the Schema Registry has Ancestry Basking in Data by Chris Sanders
Achieving Predictability And Compliance With The Data Distribution Hub at Bank of New York Mellon by Rajesh Kaveti
Every Message Counts: Kafka as a Foundation for Highly Reliable Logging at Airbnb by Youssef Francis
The Source of Truth: Why The New York Times Stores Every Piece of Content Ever Published in Kafka by Boerge Svingen
Single Message Transformations Are Not the Transformations You’re Looking For by Ewen Cheslack-Postava
Cloud Native Data Streaming Microservices with Spring Cloud and Kafka by Marius Bogoevici
Etl is Dead; Long Live Streams by Neha Narkhede

New Releases

Upcoming Events

14.6.2017 (Online) – Real-time Data Pipelines with Attunity and Apache Kafka (Attunity Webinar)
20.6.2017 (Düsseldorf, GE) – Big Data & Stream Processing with Apache Spark
22.6.2017 (Canberra, Australia) – Data-in-Motion: Recent advances in Apache projects for streaming data (Meetup)
22.6.2017 (Zurich, CH) – Data stream processing at PSI (Meetup)
27.6.2017 (London, UK) – Stateful Stream Processing (Meetup)
28.6.2017 (Munich, GE) – Rethinking Stream Processing with Apache Kafka (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 19:34 on June 6, 2017
Tags: beam ( 24 ), confluent ( 6 ), flink ( 58 ), heron ( 15 ), kafka ( 67 ), nifi ( 60 ), spark-streaming ( 60 ), storm ( 42 ), Stream Processing ( 63 ), streaming-analytics ( 62 ), streamsets ( 45 )

Last week in Stream Processing & Analytics – 5.6.2017

This is the 69th edition of my blog series blog series around Stream Processing and Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

MQTT: The Nerve System of IoT by Abhinaya Balaji
Data Sketching by Graham Cormode
Real-Time Stream Processing for Internet of Things by Asena Atilla Saunders

Comparison

Amazon Kinesis vs. Apache Kafka For Big Data Analysis by Christopher Low

Apache Kafka / Kafka Streams / Confluent Platform

Build Services on a Backbone of Events by Ben Stopford
The Future of ETL Isn’t What It Used To Be by Gwen Shapira
How Pandora Uses Kafka by Alex Woodie
Running Apache Kafka on Docker by Suci Lin

Spark Streaming

Performance Tuning of an Apache Kafka/Spark Streaming System – Telecom Case Study by Mathieu Dumoulin
Making Apache Spark the Fastest Open Source Streaming Engine by Michael Armbrust

Storm

Securing Apache Storm – part IV by Colm O hEigeartaigh
Securing Apache Storm – part III by Colm O hEigeartaigh
Securing Apache Storm – part II by Colm O hEigeartaigh
Securing Apache Storm – part I by Colm O hEigeartaigh

Heron

Heron on DC/OS by John Crawford

Apache Flink

Apache Flink 1.3.0 and the Evolution of Stream Processing with Flink by dataArtisans

Apache NiFi

Real-Time SQL On Event Streams by Mark Payne

Apache Ignite

Getting Started With Apache Ignite (Part 2) by Akmal Chaudhri

New Presentations

Introducing Confluent Cloud: Apache Kafka as a Service by Clarke Patterson
Flink meet DC/OS – Deploying Apache Flink at Scale by Elizabeth K. Joseph & Ravi Yadav
Spark Streaming – Large-scale near-real-time stream processing by Venkateswaran Kandasamy
Partitioning for Scalability: Real-Time Streaming Big Data Applications by Vikram Patil
Apache Flink and IoT: How Stateful Event-Time Processing Enables Accurate Analytics by Aljoscha Krettek
Event-driven applications with streams and snapshots (Apache Flink) by Stephan Ewen
Kappa architecture in the telecom industry by Nicolas Seyvet
Hazelcast Jet – Riding the Jet Streams by Christoph Engelbert
Portable stateful big data processing in Apache Beam by Aljoschka Krettek
Realizing the promise of portability with Apache Beam by Tyler Akidau
Building Effective Near-Real-Time Analytics with Spark Streaming and Kudu by Jeremy Beard
Structured Streaming Using Spark 2.1 by Yaswanth Yadlapalli
Dimensions Computation With Apache Apex by Devendra Tagare
Big & Fast Data for Identity & Telemetry services by Softserve

New Videos

Introducing Confluent Cloud: Apache Kafka as a Service by Clarke Patterson
Stream All Things: Patterns of Modern Data Integration by Gwen Shapira
Stateful stream processing with Apache Spark by Yuriy Taras
Unified processing with the Samza High-level API by Yi Pan
Streaming Data Pipelines with Brooklin by Samarth Shetty
Kafka at half the price by Dong Lin
How Superset and Druid Power Real-Time Analytics at Airbnb by Hakka Labs
Partitioning for Scalability: Real-Time Streaming Big Data Applications by Vikram Patil

New Podcast

New-World BI with BigQuery, Kafka, Streamsets and Looker by Mark Rittman & Steward Bryson

New Releases

Apache Flink 1.3.0

Upcoming Events

7.6.2017 (San Francisco, US) – Integrated Dataflow Processing with Spark and StreamSets (Meetup)
8.6.2017 (Online) – Load Testing Streaming Platforms (Neoty Webinar)
8.6.2017 (Utrecht, NL) – Kafka June Meetup (Meetup)
8.6.2017 (Online) – ETL is Dead; Long Live Streams (Confluent Webinar)
12.6.2017 (Columbia, US) – Latest developments in Apache NiFi and MiNiFi (Meetup)
13.6.2017 (Berlin, GE) – Rethinking Stream Processing with Apache Kafka (Meetup)
14.6.2017 (Online) – Real-time Data Pipelines with Attunity and Apache Kafka (Attunity Webinar)
22.6.2017 (Canberra, Australia) – Data-in-Motion: Recent advances in Apache projects for streaming data (Meetup)
22.6.2017 (Zurich, CH) – Data stream processing at PSI (Meetup)
27.6.2017 (London, UK) – Stateful Stream Processing (Meetup)
28.6.2017 (Munich, GE) – Rethinking Stream Processing with Apache Kafka (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 16:22 on May 23, 2017
Tags: aws ( 2 ), flink ( 58 ), IoT ( 7 ), kafka ( 67 ), kafka-streams ( 22 ), nifi ( 60 ), spark-streaming ( 60 ), Stream Processing ( 63 ), streaming-analytics ( 62 )

Last week in Stream Processing & Analytics – 22.5.2017

This is the 67th edition of my blog series blog series around Stream Processing and Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Anomaly Detection in Telecommunications Using Complex Streaming Data by Ted Dunning
The Future of Event Stream Processing by Michael Drogalis
How to avoid downtime and disruption when moving data by David Richards
The Anatomy of an IoT Solution by Hans de Visser

Apache Kafka / Kafka Streams / Confluent Platform

Reading Data Securely From Apache Kafka to Apache Spark by Mark Grover
Kafka Topic Design Checklist by Jesse Anderson
Building Real-Time data pipelines with Kafka Connect by Slim Baltagi
Trialing Single Message Transformations in Kafka Connect by Mike Donovan

Spark Streaming

Taking Apache Spark’s Structured Structured Streaming to Production by Bill Chambers & Michael Armbrust
Running Streaming Jobs Once a Day For 10x Cost Savings by Burak Yavuz & Tyson Condie

Apache Flink

A Year of Blink at Alibaba: Apache Flink in Large Scale Production by Xiaowei Jiang

AWS Kinesis

Build a Visualization and Monitoring Dashboard for IoT Data with Amazon Kinesis Analytics and Amazon QuickSight by Karan Desai

Apache NiFi

Monitoring NiFi – Scripted Reporting Task by Pierre Villard
Monitoring NiFi – Ambari & Grafana by Pierre Villard
Monitoring NiFi – Workflow SLA by Pierre Villard
Monitoring NiFi – Site2Site reporting tasks by Pierre Villard
Monitoring NiFi – Logback configuration by Pierre Villard
Record-oriented Data with NiFi by Mark Payne

New Presentations

Real Time Analytics – Stream Processing by Mahesh Madushanka
Streaming Analytics for IoT-Oriented Applications by Adrian Bowles
Geo-Distributed Big Data and Analytics by Ted Dunning
Top 5 mistakes when writing Streaming applications by Mark Grover & Ted Malaska
Fast Cars, Big Data How Streaming can help Formula 1 by Carol McDonald
Stateful streaming data pipelines by Chandni Singh & Timothy Farkas
Stream Me Up, Scotty: Transitioning to the Cloud Using a Streaming Data Platform by Chrix Finne & Bob Lehmann
Microservices interaction at scale using Apache Kafka by Ivan Ursul
Capture the Streams of Database Changes by Randall Hauch
Stream Processing Metamorphosis – A Kafka’s tale by João Vazão Vasques
Event processing with Apache Storm by Nuno Caneco

New Podcasts

Apache Kafka, Streaming Data Integration and Schema Registry by Mark Rittman with Gwen Shapira

Upcoming Events

22.5.2017 (Los Angeles, US) – Apache Kafka & the Riseof real-time (Meetup)
23.5.2017 (online) – Deploy and Scale a Real-Time Streaming Data Platform with Confluent on DC/OS (Mesosphere Webinar)
23.5.2017 (London, UK) – Stream Analytics with SQL on Apache Flink (Meetup)
24.5.2017 (online) – Simplifying Real-Time Architectures for IoT with Apache Kudu (Cloudera Webinar)
30.5.2017 (Frankfurt, GE) – First Kafka Meetup (Meetup)
7.6.2017 (San Francisco, US) – Integrated Dataflow Processing with Spark and StreamSets (Meetup)
12.6.2017 (Columbia, US) – Latest developments in Apache NiFi and MiNiFi (Meetup)
22.6.2017 (Canberra, Australia) – Data-in-Motion: Recent advances in Apache projects for streaming data (Meetup)
27.6.2017 (London, UK) – Stateful Stream Processing (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 22:33 on May 17, 2017
Tags: beam ( 24 ), confluent ( 6 ), flink ( 58 ), kafka ( 67 ), kafka-streams ( 22 ), nifi ( 60 ), spark-streaming ( 60 ), Stream Processing ( 63 ), streaming-analytics ( 62 ), streamsets ( 45 )

Last week in Stream Processing & Analytics – 17.5.2017

This is the 66th edition of my blog series blog series around Stream Processing and Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Fast Cars, Big Data – How Streaming Data Can Help Formula 1 by Carol McDonald
Thoughts on Building a Successful Streaming Analytics Platform by George Vetticaden
A Digital Reference Architecture for the Industrial Internet Of Things (IIoT) by Vamsi Chemitiganti
Introducing DevHigh5 Project Pravega: Storage reimagined for a streaming world by Salvatore DeSimone

Apache Kafka / Kafka Streams / Confluent Platform

Kafka Summit New York: The Rise of the Streaming Platform by Clarke Patterson
Reading data securely from Apache Kafka to Apache Spark by Mark Grover
Installing and Running Kafka on an AWS Instance by Sahil Sawhney
Managing Spark and Kafka Pipelines by Larry Murdock
Getting Started with the Kafka Streams API using Confluent Docker Images by Michael Noll
Kafka Consumer Architecture – Consumer Groups and subscriptions by Cloudurable
The new version of Kafka Topics UI is now available! by Christina Daskalaki
Startup Confluent makes Kafka cloud play to ease streaming deployment by Jack Vaughan
Securing Apache Kafka with Kerberos by Colm O hEigeartaigh
Apache Kafka – Introduction & Installation by Rudra Narayan Garnaik
Visualizing the flow of data as it moves through a Kafka cluster by James Cheng

Spark Streaming

How to Get Started with Spark Streaming and MapR Streams Using the Kafka API by Carol McDonald

Apache Beam

Apache Beam publishes the first stable release by Davor Bonaci

Apache Flink

Introducing Official Docker Images for Apache Flink by Patrick Lucas & Ismaël Mejía

StreamSets

Visualizing and Analyzing Salesforce Data with Neo4j by Pat Patterson
Calling External Libraries from the JavaScript Evaluator by Pat Patterson

Apache NiFi

Apache NiFi and TLS Toolkit Ansible Roles to create a multi-node secure NiFi cluster by Roger Carhuatocto
Apache Nifi Sensitive Value Encryption by Kya
Record based processors in Apache NiFi 1.2 by Haimo Liu

New Presentations

Intro to Apache Kafka by Jason Hubbard
Stream processing using Kafka by Himani Arora & Prabhat Kashyap
Rethinking Services With Stateful Streams by Ben Stopford
Connect S3 with Kafka leveraging Akka Streams by Seiya Mizuno
Stream Processing with MicroProfile and Apache Kafka by Matthias Wessendorf
Real-time Data Pipelines with SAP and Apache Kafka by Carole Gunst
Processing IoT Data with Apache Kafka by Confluent
Functional streams with Kafka – A comparison between Akka-streams and FS2 by Luis Reis & Rui Batista
Open Blueprint for Real-Time Analytics with In-Stream Processing (ISP) by Victoria Livschitz
Dataservices – Processing Big Data The Microservice Way by Josef Adersberger

New Video

Scalable Stream Processing: A Survey of Storm, Samza, Spark and Flink by Felix Gessert
Improve your App UX with one model processing insights from batch and streams by Robert Kubis
Apache Ignite: Real Time Processing of IoT Generated Streaming Data by Denis Magda
Rethinking Microservices with Stateful Streams by Ben Stopford
Fast Cars, Big Data – How Streaming Can Help Formula 1 by Tugdual Grall

New Releases

Apache Beam 2.0.0

Upcoming Events

17.5.2017 (Mountain View, US) – Stream Processing with Apache Kafka & Apache Samza (Meetup)
18.5.2017 (online) – Conquering Data in Motion with a Modern Ingestion Platform (Cloudera Webinar)
22.5.2017 (Los Angeles, US) – Apache Kafka & the Riseof real-time (Meetup)
23.5.2017 (online) – Deploy and Scale a Real-Time Streaming Data Platform with Confluent on DC/OS (Mesosphere Webinar)
23.5.2017 (London, UK) – Stream Analytics with SQL on Apache Flink (Meetup)
24.5.2017 (online) – Simplifying Real-Time Architectures for IoT with Apache Kudu (Cloudera Webinar)
7.6.2017 (San Francisco, US) – Integrated Dataflow Processing with Spark and StreamSets (Meetup)
12.6.2017 (Columbia, US) – Latest developments in Apache NiFi and MiNiFi (Meetup)
22.6.2017 (Canberra, Australia) – Data-in-Motion: Recent advances in Apache projects for streaming data (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 22:42 on May 10, 2017

Last week in Stream Processing & Analytics – 10.5.2017

This is the 65th edition of my blog series blog series around Stream Processing and Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Four Keys to Enterprise IoT Architecture by Joe McKendrick
Part 1: Hortonworks Thoughts on Building a Successful Streaming Analytics Platform by George Vetticaden
Real-Time Data Analysis: Where the Jobs Are by Joe McKendrick

Comparison

Apache Kafka vs RabbitMQ: Fit For Purpose / Decision Tree by Slim Baltagi

Apache Kafka / Kafka Streams / Confluent Platform

Pepper-Box Kafka Load Generator by Satish Bhor
Watermarks, Tables, Event Time, and the Dataflow Model by Eno Thereska
2017 Apache Kafka Report by Confluent
The 2017 Apache Kafka Survey: Streaming Data on the Rise by Luanne Dauber
Kafka ‘Massively Simplifies’ Data Infrastructure, Report Says by Alex Woodie
Robust Usage of Apache Kafka on Heroku by Heroku
The Many Meanings of Event-Driven Architecture: Kafka Edition by Jesse Anderson
Confluent Now Offers Apache Kafka as a Service with Confluent Cloud by Sue Walsh
Announcing Confluent Cloud: Apache Kafka as a Service by Neha Narkhede
Joins on stateful stream processing using Kafka Streams’ KTables and GlobalKTables by Daniel Lebrero
Optimizing your Kafka Deployment by Yeva Byzek
The Continued Rise of Apache Kafka by RedMonk
Apache Kafka Startup Confluent Launches A Cloud Service With Bold Words For AWS by Alex Konrad
Apache Kafka – Usage Patterns by Gagan

Spark Streaming

Event-time Aggregation and Watermarking in Apache Spark’s Structured Streaming by Tathagata Das

Apache Flink

Flink Forward San Francisco 2017 Recap: On the State of Stream Processing with Apache Flink by dataArtisans
Continuous Queries on Dynamic Table by Fabian Hueske, Shaoxuan Wang, and Xiaowei Jiang

StreamSets

StreamSets Named a 2017 Cool Vendor in Data Management by Gartner
StreamSets benefits from focus on continuous data integration by Matt Aslett

Heron

Heron Going Exactly-Once by Sanjeev Kulkarni

Apache NiFi

My first experiences with Apache NiFi by Marcel-Jan Krijgsman
Speeding up Apache NiFi compilation by Richards Technotes
A New Era of Open-Source Streaming by Tim Spann
Apache NiFi 1.2.0 – Component Versioning by Bryan Bende
IoT: Ingesting GPS Data from Raspberry PI Zero Wireless with USB GPS Device by Timothy Spann

New Presentations

Big Data Architectural Patterns and Best Practices on AWS by Amazon AWS
Red Hat JBoss AMQ and Apache Kafka : which to use ? by Christian Posta & Paolo Patierno
What is apache Kafka? by Kenny Gorman
Kafka Tutorial – introduction to the Kafka streaming platform by Cloudurable
Azure real-time analytics and kappa architecture with Kafka and Cassandra clusters by Vitalii Bondarenko
Challenges of a multi tenant kafka service by Thomas Alex
From Three Nines to Five Nines – A Kafka Journey by Allen Wang
Pinot: Near Realtime Analytics @ Uber by Xiang Fu
Stream Processing with SMACK by Tamás István Ujj
Oracle Stream Analytics – Industry Examples by Prabhu Thukkaram
Apache Kafka – Scalable Message-Processing and more ! by Guido Schmutz
Introducing Exactly Once Semantics To Apache Kafka by Jason Gustafson, Guozhang Wang, Sriram Subramaniam & Apurva Mehta
Introduction to Kafka Streams with a Real-Life Example by Alexis Seigneurin
Cloud Native Data Streaming Microservices With Spring Cloud and Kafka by Marius Bogoevici

New Video

Visual VM with Kafka by Patrick Bailey
Comparison of Apex and Spark Streaming by Bhupesh Chawada

New Podcast

Data Intensive Appications with Martin Kleppmann by SE Daily

New Releases

Upcoming Events

16.5.2017 (online) – Geo-Distributed Big Data and Analytics (MapR Webinar)
16.5.2017 (Copenhagen, DK) – Meet Gwen Shapira, the Master of Apache Kafka! (Meetup)
18.5.2017 (online) – Conquering Data in Motion with a Modern Ingestion Platform (Cloudera Webinar)
23.5.2017 (online) – Deploy and Scale a Real-Time Streaming Data Platform with Confluent on DC/OS (Mesosphere Webinar)
24.5.2017 (online) – Simplifying Real-Time Architectures for IoT with Apache Kudu (Cloudera Webinar)
7.6.2017 (San Francisco, US) – Integrated Dataflow Processing with Spark and StreamSets (Meetup)
12.6.2017 (Columbia, US) – Latest developments in Apache NiFi and MiNiFi (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 20:54 on May 1, 2017
Tags: kafka ( 67 ), kafka-streams ( 22 ), nifi ( 60 ), Stream Processing ( 63 ), streaming-analytics ( 62 ), streamsets ( 45 )

Last week in Stream Processing & Analytics – 1.5.2017

This is the 64th edition of my blog series blog series around Stream Processing and Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

A Big Data Reference Architecture for IoT by Tim Spann & Vamsi Chemitiganti

Apache Kafka / Kafka Streams / Confluent Platform

Hello Kafka Streams by Vishnu Viswanath
Test Strategy for Samza/Kafka Services by Devi Sridharan
Understanding When to use RabbitMQ or Apache Kafka by Pieter Humphrey
How Apache Kafka Works and Why by Stefan Groschupf

StreamSets

Create a Custom Expression Language Function for StreamSets Data Collector by Pat Patterson

Apache NiFi

Ingesting GPS Data From Onion Omega2 Devices with Apache NiFi by Tim Spann
Ingesting iBeacon Data via BLE to MQTT WiFi Gateways via Apache NiFi by Tim Spann
Advanced Apache NiFi Flow Techniques by Tim Spann

Azure Stream Analytics

Data driven troubleshooting of Azure Stream Analytics jobs by Samartha Chandrashekar

New Presentations

What will be new in Apache NiFi 1.2.0 by Koji Kawamura
Event Driven Services Part 2: Building Event-Driven Services with Apache Kafka 23 views by Ben Stopford
Putting the Micro into Microservices with Stateful Stream Processing by Ben Stopford
Schema Registry & Stream Analytics Manager by Harsha Chintalapani
Using Akka Streams For Real Time Decision Making by Dustin Lyons
Twitter Sentiment Analysis Example by Edureka
Spark Tutorial For Beginners by Edureka
Bridging Batch and Streaming Data Integration by Shirshanka Da
Apache Storm by Nguyen Quang

New Video

Introduction to Apache Kafka by James Ward
Using Akka Streams For Real Time Decision Making by Dustin Lyons

Upcoming Events

6.5.2017 (Fremont, US) – Hands-on workshop in Real Time Stream Processing and Analytics (Meetup)
9.5.2017 (London, UK) – Reactive in Practice (CakeSolutions Event)
16.5.2017 (online) – Geo-Distributed Big Data and Analytics (MapR Webinar)
23.4.2017 (online) – Deploy and Scale a Real-Time Streaming Data Platform with Confluent on DC/OS (Mesosphere Webinar)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!