Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 08:37 on March 20, 2019
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 19.3.2019

This is the 149th edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Getting Ready for Real-Time Decisioning by John Dillon
7 Commandments for Event-Driven Architecture by RJ Zaworski
SOA vs. EDA: Is Not Life Simply a Series of Events? by Jerry Mathew

Apache Kafka / Kafka Streams / Confluent Platform

Kafka Connect Deep Dive – Error Handling and Dead Letter Queues by Robin Moffatt
Debezium meets Quarkus by Jiri Pechanec
How to know if Apache Kafka is right for you by Amit Rathi
Apache Kafka in a nutshell by Vajo Lukic
KSQL: Streaming SQL Engine for Apache Kafka by Hojjat Jafarpour & Rohan Desai & Damian Guy
Top 6 insights you should know before using the Kafka Connect BigQuery sink connector by Jaroslaw Kijanowski
The Case for Database-First Pipelines by Gwen Shapira

Apache Pulsar / Streamlio

Streamlio, an open-core streaming data fabric for the cloud era by George Anadiotis

Apache Flink

Flink and Prometheus: Cloud-native monitoring of streaming applications by Maximilian Bode

NiFi

Announcing NiFi Processors for Apache Pulsar by David Kjerrumgaard

StreamSets

Join Us at the First Annual DataOps Summit by Sean Anderson

Presentations

Embrace the Anarchy: Apache Kafka’s Role in Modern Data Architectures by Robin Moffatt
Solutions for bi-directional Integration between Oracle RDMBS & Apache Kafka by Guido Schmutz
Event driven microservices by Andrew Schofield
Crossing the Streams: Rethinking Stream Processing with Kafka Streams and KSQL by Tim Berglund & Viktor Gamov
Architecting for Real-Time Insights with Streaming Data by Frank Munz

New Videos

Embrace the Anarchy: Apache Kafka’s Role in Modern Data Architectures by Robin Moffatt

Upcoming Events

18.3.2019 (Palo Alto, US) – A Series of Talks on Apache Heron (Meetup)
19.3.2019 (Paris, FR) – Kafka, the power of events and unbounded data (Meetup)
19.3.2019 (Seattle, US) – Confluent and Nordstrom discuss Events, Logs, and Microservices! (Meetup)
19.3.2019 (Boston, US) – Real Time Analytics with Complex Event Processing (Meetup)
20.3.2019 (Portland, US) – Deploying Kafka Streams Applications with Docker and Kubernetes (Meetup)
20.3.2019 (Utrecht, NL) – A spring with Stream Processing and Apache Flink (Meetup)
20.3.2019 (Brussels, BE) – Real-time SQL Stream Processing at Scale with Apache Kafka and KSQL (Meetup)
20.3.2019 (online) – Deploying Flink on Kubernetes (Ververica Webinar)
20.3.2019 (Sunnyvale, US) – Stream Processing with Apache Kafka & Apache Samza
21.3.2019 (Munich, GE) – 5th Apache Flink Meetup Munich (Meetup)
21.3.2019 (Amsterdam, NL) – Apache Kafka, KSQL, Demos & Booking.com (Meetup)
21.3.2019 (Warsaw, PL) – Developing Streaming Data Sources V2 for Spark Structured Streaming (Meetup)
4.4.2019 (New York, US) – Stream Processing Beyond Streaming with Apache Flink, by Co-Creator of Flink (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 18:37 on March 12, 2019
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 12.3.2019

This is the 148th edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Innovation Insight for Event-Thinking by Confluent
Reducing System Complexity with Event Sourcing by Jesse Anderson

Apache Kafka / Kafka Streams / Confluent Platform

Solutions to Communication Problems in Microservices using Apache Kafka and Kafka Lens by Paul Downey
Apache Kafka With Scala Tutorial by Shubham Dangare
Using Jakarta EE/MicroProfile to Connect to Apache Kafka: Part 1 — Hello World by Otavio Santana
What happened in the Kafka community in February 2019? by Mickael Maison
Spring for Apache Kafka Deep Dive – Part 2: Apache Kafka and Spring Cloud Stream by Soby Chacko
Anomalia Machina 8 – Production Application Deployment with Kubernetes: Massively Scalable Anomaly Detection with Apache Kafka and Cassandra by Paul Brebner
CloudBank’s Journey from Mainframe to Streaming with Confluent Cloud by Ricardo Ferreira
Putting ML in production I: using Apache Kafka in Python by Javier Rodriguez Zaurin
Building Reliable Reprocessing and Dead Letter Queues with Apache Kafka by Ning Xia

Apache Pulsar

Efficient Stream Processing with Pulsar Functions by jowanza

Apache Flink

Flink and Prometheus: Cloud-native monitoring of streaming applications by Maximilian Bode

Spark

Spark Streaming and Kafka, Part 3 – Analysing Data in Scala and Spark by Janis Rumnieks

WSO2 Stream Processor

Introduction to Apache Kafka Integration for Siddhi by Niruhan Viswarupan
HA Deployment of WSO2 Stream Processor with Kafka by Niruhan Viswarupan

StreamSets

Scaling Data Collectors on Azure Kubernetes Service by Dash Desai
From zero to prod ETL in 30 minutes with Streamsets by Jeffery Schmitz
Solving Data Quality in Streaming Data Flows by Vinu Kumar

Presentations

Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) by Kai Waehner

New Videos

Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) – Friends, Enemies or Frenemies? by Kai Waehner

New Podcast

Containerized Kafka On Kubernetes with Viktor Gamov by Confluent

New Releases

Apache Beam 2.11.0

Upcoming Events

12.3.2019 (London, UK) – Embrace the Anarchy: Apache Kafka’s Role in Modern Data Architectures (Meetup)
12.3.2019 (Boston, US) – What is Apache Kafka and Wayfair’s Journey with Apache Kafka (Meetup)
18.3.2019 (Palo Alto, US) – A Series of Talks on Apache Heron (Meetup)
19.3.2019 (Paris, FR) – Kafka, the power of events and unbounded data (Meetup)
20.3.2019 (Portland, US) – Deploying Kafka Streams Applications with Docker and Kubernetes (Meetup)
20.3.2019 (Utrecht, NL) – A spring with Stream Processing and Apache Flink (Meetup)
20.3.2019 (Brussels, BE) – Real-time SQL Stream Processing at Scale with Apache Kafka and KSQL (Meetup)
20.3.2019 (online) – Deploying Flink on Kubernetes (Ververica Webinar)
21.3.2019 (Munich, GE) – 5th Apache Flink Meetup Munich (Meetup)
21.3.2019 (Amsterdam, NL) – Apache Kafka, KSQL, Demos & Booking.com (Meetup)
4.4.2019 (New York, US) – Stream Processing Beyond Streaming with Apache Flink, by Co-Creator of Flink (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 23:01 on March 4, 2019
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 4.3.2019

This is the 147th edition of my blog series blog series around Stream Data Integration and Stream Analytics! I have slightly changed the title to better reflect the content of by weekly blog series.

According to Gartner’s “Market Guide for Event Stream Processing”, Stream Data Integration and Stream Analytics are the two subsegments of the overall Stream Processing domain. I really like this definition, as it easily allows to categorize products either into one of these two segments.

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

The Importance of Event-First Thinking by Jan Stenberg
Why the industrial IoT is more important than consumer devices — and 7 more surprising IoT trends by Fredric Paul

Apache Kafka / Kafka Streams / Confluent Platform

Filesystem events to Elasticsearch / Kibana through Kafka Connect / Kafka by Maarten Smeets
Spring for Apache Kafka Deep Dive – Part 1: Error Handling, Message Conversion and Transaction Support by Gary P. Russell
Journey to Event Driven – Part 3: The Affinity Between Events, Streams and Serverless by Neil Avery
Resizing persistent volumes with Strimzi by Jakub Scholz
UDFs in KSQL: DateAdd by Saeed Barghi
All About the Kafka Connect Neo4j Sink Plugin by Michael Hunger & Andrea Santurbano
5 Machine Learning Trends for 2018 Combined With Apache Kafka Ecosystem by Kai Waehner
Who’s tweeting about Dick’s Sporting Goods and KSQL? by Viktor Gamov
Rendezvous with Kafka : A simple guide to get started by Suman Pattnaik

Apache Flink

The Unexpected Quest for Business Intelligence by Steven Le Roux

Apache NiFi

Auto Loading Nars in Apache Nifi by NiFi.rocks
Apache NiFi Overview by Manoj GT
Edge to AI: Apache Spark, Apache NiFi, Apache NiFi MiNiFi, Cloudera Data Science Workbench Example

Spark

Spark Streaming and Kafka, Part 2 – Configuring a Kafka Connector by Janis Rumnieks

Oracle Cloud Infrastructure Streaming

Ingest and Process High-Volume, Continuous Data in Real Time with Oracle Cloud Infrastructure Streaming by Somnath Lahiri

Presentations

The Changing Face of ETL: Event-Driven Architectures for Data Engineers by Robin Moffatt
Who’s Tweeting about #DSGTECH and #KSQL by Viktor Gamov
Architectures with Apache Kafka in AWS / Managed Streaming for Apache Kafka (MSK) by Frank Munz
Apache Kafka and KSQL in Action: Let’s Build a Streaming Data Pipeline! by Tim Vincent
Integration for real-time Kafka SQL by Amit Nijhawan
Streaming Visualization by Guido Schmutz
Patterns of Streaming Applications by Monal Daxini
Microservices, Events, and Breaking the Data Monolith with Kafka by Comelia Davis & Ben Stopford

New Videos

Patterns of Streaming Applications by Monal Daxini
ATM Fraud Detection with Apache Kafka and KSQL by Robin Moffatt
Stream All Things – Patterns of Modern Data Integration by Neil Avery

Upcoming Events

5.3.2019 (Brentwood, US) – Tips and Tricks about Apache Kafka in the Cloud for Java Developers (Meetup)
5.3.2019 (London, UK) – Lies Enterprise Architects Tell (Meetup)
7.3.2019 (Cologne, GE) – Accelerate data – Accelerate Decission – Accelerate Business (Meetup)
12.3.2019 (London, UK) – Embrace the Anarchy: Apache Kafka’s Role in Modern Data Architectures (Meetup)
12.3.2019 (Boston, US) – What is Apache Kafka and Wayfair’s Journey with Apache Kafka (Meetup)
20.3.2019 (Portland, US) – Deploying Kafka Streams Applications with Docker and Kubernetes (Meetup)
20.3.2019 (Utrecht, NL) – A spring with Stream Processing and Apache Flink (Meetup)
20.3.2019 (Brussels, BE) – Real-time SQL Stream Processing at Scale with Apache Kafka and KSQL (Meetup)
20.3.2019 (online) – Deploying Flink on Kubernetes (Ververica Webinar)
21.3.2019 (Munich, GE) – 5th Apache Flink Meetup Munich (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!