Last Week in Stream Data Integration & Stream Analytics – 26.7.2019

This is the 165th edition of my blog series blog series around Stream Data Integration and Stream Analytics!

This time it is actually for two weeks, as I was too busy to release one earlier. From next week I hope to get back to a weekly cycle. As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Five Advantages of Log-Based Change Data Capture by Gunnar Morling
Communicating Data Changes Across Service Boundaries… Safely! by Hans-Peter Grahsl

Apache Kafka / Kafka Streams / Confluent Platform

Autoscaling Kafka Streams applications with Kubernetes by Jaroslaw Kijanowski
Streaming Cassandra at WePay – Part 1 by Joy Gao
Streaming Cassandra at WePay – Part 2 by Joy Gao
Bust the Burglars – Machine Learning with TensorFlow and Apache Kafka by Erik-Berndt Scheper
Open Sourcing Brooklin: Near Real-Time Data Streaming at Scale by Celia Kung
Getting Started with the MongoDB Connector for Apache Kafka and MongoDB by Robert Walters
Kafka Streams, Jigsaw and Docker walk into a bar by Jaroslaw Kijanowski
Bridging the HTTP protocol to Apache Kafka by Paolo Patierno
Why I Can’t Wait for Kafka Summit San Francisco by Gwen Shapira
Fault Tolerance in Distributed Systems: Tracing with Apache Kafka and Jaeger by Aaron Burk
An introduction to event-driven architecture and Apache Kafka by Neil McAllister

Apache Pulsar

Sentiment Analysis of Tweets using Apache Pulsar by Jerry Peng
Apache Pulsar as One Storage System for Both Real-time and Historical Data Analysis by Yijie Shen

Apache Flink

Apache Pulsar Flink Connector by Eren Avşaroğulları
Flink Network Stack Vol. 2: Monitoring, Metrics, and that Backpressure Thing by Nico Kruber & Piotr Nowojski

StreamSets

Enhanced Error Diagnostics in StreamSets Data Collector 3.9.0 by Pat Patterson
A New Definition of DataOps by John Schmidt

New Presentations

From Zero to Hero with Kafka Connect by Robin Moffatt
Top Ten Kafka® Configs by Bill Bejeck
Real-time processing of large amounts of data by Perry Krol

New Videos

From Zero to Hero with Kafka Connect by Robin Moffatt
End-to-End Integration: IoT Edge to Confluent Cloud by Kai Wähner
Ask Confluent #14: In Control of Kafka by Dan Norwood
Kubernetes as a Streaming Data Platform with Kafka, Spark, and Scala by Gerard Maas
View from Apache Flink on Evolution & Outlooks for the Modern Stateful Stream Processor by Tzu-Li Tai

New Releases

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 20:59 on July 10, 2019
Tags: kafka ( 232 ), kafka-streams ( 160 ), nifi ( 222 ), spark ( 137 ), spark-streaming ( 203 ), Stream Processing ( 225 )

Last Week in Stream Data Integration & Stream Analytics – 10.7.2019

This is the 164th edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Server-Sent Events (SSE): A Conceptual Deep Dive by
Kieran Kilbride-Singh
Event-Driven Architectures: A Simple Analogy by Kevin Larsen
AoA and AoD: Real-Time Location Services Drive IoT Value by Fabio Belloni

Apache Kafka / Kafka Streams / Confluent Platform

Lessons learned from running Kafka at Datadog by Emily Chang
KSQL and Registered topics by Jaroslaw Kijanowski
KSQL in Football: FIFA Women’s World Cup Data Analysis by Francesco Tisiot
Kafka Monthly Digest: June 2019 by Mickael Maison
Analysis of streaming data with Kafka by Rob Cowell
Persistent storage improvements by Jakub Scholz
Deploying Kafka Streams and KSQL with Gradle – Part 3: KSQL User-Defined Functions and Kafka Streams by Stewart Bryson

Apache NiFi

NiFi & NiFi Registry on the Google Cloud Platform with Cloud Source Repositories by Pierre Villard

Apache Flink

On Stream Processing and Apache Flink. Q&A with Fabian Hueske by Fabian Hueske

New Videos

Real Time Investment Alerts using Apache Kafka at ING Bank by Marcos Maia & Tim v Baarsen
Deep Learning For Real Time Streaming Data With Kafka And Tensorflow by Yong Tang
How to use Apache Kafka to transform a batch pipeline into a Real-Time-One? by Stephane Maarek
The internals of stateful stream processing in Spark Structured Streaming by Jacek Laskowski

New Podcasts

Change Data Capture with Debezium ft. Gunnar Morling by Confluent Streaming Audio

New Releases

Apache Pulsar 2.4.0

Upcoming Events

9.7.2019 (Moutain View, US) – Apache Druid Bay Area Meetup @ Intuit (Meetup)
11.7.2019 (Utrecht, NL) – From Zero to Hero with Kafka Connect (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 19:01 on July 2, 2019
Tags: kafka ( 232 ), kafka-streams ( 160 ), nifi ( 222 ), spark ( 137 ), spark-streaming ( 203 ), Stream Processing ( 225 )

Last Week in Stream Data Integration & Stream Analytics – 2.7.2019

This is the 163st edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Design Considerations for Real-Time Operational Data Architectures for Industry 4.0 by Mahendraprabu Sundarraj

Apache Kafka / Kafka Streams / Confluent Platform

A real showcase of Kafka at Wirecard Brazil by Cauê Ferreira
An overview of the Kafka Connect RabbitMQ Source Connector by Danny Kay
Building a Scalable Search Architecture by Pere Urbón-Bayes
What’s New in Apache Kafka 2.3 by Colin McCabe
High Availability Kafka Service in Microsoft Azure Cloud by Ada Yan
Microservices, Apache Kafka, and Domain-Driven Design by Kai Waehner
Building Kafka Streams applications with Quarkus and Eclipse MicroProfile by Gunnar Morling
Designing the .NET API for Apache Kafka by Matt Howlett
Building zero-latency data lake using Change Data Capture by Ofir Ventura
Optimizing Your Apache Kafka Deployment by Yeva Byzek
Kafka Listeners – Explained by Robin Moffatt
Change data capture from Oracle to Kafka with triggers by Janne Keskitalo
Kafka Connectors Without Kafka by Dawid Kublik

Apache Flink

A Practical Guide to Broadcast State in Apache Flink by Fabian Hueske

New Slides

Data Integration with Apache Kafka: What, Why, How by Pat Patterson
Streaming database events over Kafka using Debezium by Clement Demonchy & Job Teaser
The Changing Face of ETL: Event-Driven Architectures for Data Engineers by Robin Moffatt
Testing Kafka based applications by Nikita Zhevnitskiy
Integrating Databases and Apache Kafka by Robin Moffatt
Deep Dive on Apache Flink State by Seth Wiesman
Streaming ETL in Practice with Oracle, Apache Kafka, and KSQL by Robin Moffatt

New Videos

What’s new in Kafka 2.3 by Stephane Maarek
Apache Kafka 2.3 | What’s New in this Release + Updates and KIPs by Tim Berglund
From zero to hero with Apache Kafka’s connect API by Robin Moffatt

New Podcasts

Rolling Kafka Upgrades and Confluent Cloud by Gwen Shapira

New Releases

Upcoming Events

1.7.2019 (Paris, FR) – Apache Beam meetup 1 at Criteo (Meetup)
2.7.2019 (Vienna, AU) – Apache Kafka® & New Ways of Stream Processing with KSQL (Meetup)
3.7.2019 (Munich, GE) – From Zero to Hero with Kafka® Connect (Meetup)
4.7.2019 (Istanbul, TU) – Strimzi: Distributed Streaming with Apache Kafka in Kubernetes (Meetup)
9.7.2019 (Moutain View, US) – Apache Druid Bay Area Meetup @ Intuit (Meetup)
11.7.2019 (Utrecht, NL) – From Zero to Hero with Kafka Connect (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!