Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 20:02 on October 19, 2019
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 18.10.2019

This is the 172nd edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading!

News and Blog Posts

General

To Become a Real-time Business, You Need Event-Streaming Architecture by Jackie Vendetti

Apache Kafka / Kafka Streams / Confluent Platform

Exploit Apache Kafka’s Message Format to Save Storage and Bandwidth by Seyed Morteza Mousavi
Killing Kafka: The Pitfalls of Over-architecting by Stephanie Sherriff
Kafka Summit 2019 San Francisco: a review by Simon Aubury
Deploying Kafka Connect Connectors by Yaroslav Tkachenko
Kafka Serialization and Deserialization (SerDes) Examples by Yves Byzek
On Track with Apache Kafka – Building a Streaming ETL Solution with Rail Data by Robin Moffatt

Apache Pulsar

Getting Started with Apache Pulsar using Python by Ankit Sahay
5 More Reasons to Choose Apache Pulsar Over Kafka by Chris Bartholomew
An Introduction to Stream Processing with Pulsar Functions by Matteo Merli & Jerry Peng
Why Nutanix Beam went ahead with Apache Pulsar instead of Apache Kafka? by Yuvaraj Loganathan

Apache NiFi

Migrating Apache Flume Flows to Apache NiFi: Kafka Source to Multiple Sinks by Tim Spann

New Presentations

Making sense of sensors with SQL Streaming by Hans-Peter Grahsl
Do Flink on Web with FLOW by Dongwon Kim & Haemee Park
Apache Pulsar – First Overview by Ricardo Paiva
Apache Kafka Event-Streaming Platform for .NET Developers by Viktor Gamov
Machine Learning & Kafka KSQL stream processing by Simon Aubury
Benefits of Stream Processing and Apache Kafka Use Cases by Mark Frei
RabbitMQ & Kafka by Madhav Sathe & Zoe Vance
Event Streaming Architecture for Industry 4.0 by Abdelkrim Hadjidj & Jan Kunigk

New Videos

RabbitMQ & Kafka by Madhav Sathe & Zoe Vance
Apache Kafka Event-Streaming Platform for .NET Developers by Viktor Gamov
Streaming with Spring Cloud Stream and Apache Kafka by Soby Chacko & Oleg Zhurakousky
From Mainframe to Microservices with Pivotal Platform and Kafka: Bridging the Data Divide by Dmitry Milman & Ankur Kaneria
ReactSphere: Probabilistic data structures for Kafka Streams by Mateusz Owczarek & Miron Ficak
Flink Forward Europe 2019 Playlist

New Podcasts

Change Data Capture, Debezium, Streaming and Kafka by Airhacks FM Podcast
Apache Kafka on Kubernetes, Microsoft Azure, and ZooKeeper by Confluent Streaminig Audio

New Releases

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 21:36 on October 10, 2019
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 10.10.2019

This is the 171st edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading!

News and Blog Posts

Apache Kafka / Kafka Streams / Confluent Platform

Neo4j Gets Hooks Into Kafka by Alex Woodie
Messing Around with Apache Kafka Using the Confluent Cloud Free Trial by Bruno Borges
Confluent makes the case for serverless Apache Kafka on its Cloud, not AWS by Virginia Backaitis
Implementing a Queryable User Profile Store using Kafka Streams by Torben Meyer
Kafcache: Memcached + Kafka Streams by madewithtea
Why Scrapinghub’s AutoExtract Chose Confluent Cloud for Their Apache Kafka Needs by Ian Duffy
Messing around with Apache Kafka using the Confluent Cloud free trial by Richard Seroter
Internet of Things (IoT) and Event Streaming at Scale with Apache Kafka and MQTT by Kai Waehner
How to Deploy Confluent Platform on Pivotal Container Service (PKS) with Confluent Operator by Todd McGrath
How to Run Apache Kafka with Spring Boot on Pivotal Application Service (PAS) by Todd McGrath
Strategies for Handling Unchanged Postgres TOAST Values by Gunnar Morling
Using Snowflake, Debezium and Kafka with Lenses by Mihalis Tsoukalos
Strimzi, Apache Kafka and Tracing by Jakub Scholz
Kafka Monthly Digest: September 2019 by Mickael Maison
Kafka Summit San Francisco 2019 Session Videos by Tim Berglund
How LinkedIn customizes Apache Kafka for 7 trillion messages per day by Jon Lee & Wesley Wu
Logs & Offsets: (Near) Real Time ELT with Apache Kafka + Snowflake by Adrian Kreuziger

Apache Flink

Here’s What Makes Apache Flink scale by Kartik Khare
Apache Flink and the Backpressure Smackdown by Owen Rees-Hayward
Announcing Stateful Functions: Distributed State, Uncomplicated by Stephan Ewen & Marta Paes
Real-time experiment analytics at Pinterest using Apache Flink by Pinterest Engineering

StreamSets

Announcing StreamSets Data Collector 3.11.0 and StreamSets Data Collector Edge 3.11.0 by Dash Desai

New Presentations

Building stream processing applications for Apache Kafka using KSQL by Robin Moffatt
Now You See Me, Now You Compute: Building Event-Driven Architectures with Apache Kafka by Michael Noll
Kafka 102: Streams and Tables All the Way Down by Michael Noll
Event Sourcing, Stream Processing and Serverless by Ben Stopford
Kafka on Kubernetes: Keeping It Simple by Nikki Thean
Using Kafka to Discover Events Hidden in your Database by Anna McDonald
On Track with Apache Kafka: Building a Streaming ETL solution with Rail Data by Robin Moffatt
Towards Stateful FaaS on Streaming Dataflow by Marios Fragkoulis & Adi Akhter
Stateful Functions – Building general-purpose Applications and Services on Apache Flink by Stephan Ewen
Solutions for bi-directional integration between Oracle RDBMS and Apache Kafka by Guido Schmutz

New Videos

Blending Event Stream Processing with Machine Learning Using Kafka Ecosystem by Andrea Spina
Kafka Streams in Production: From Use Case to Monitoring by Alexander Kudryashow

New Podcasts

Kafka Data Pipelines with Robin Moffatt by SE Daily
Data Modeling for Apache Kafka – Streams, Topics & More with Dani Traphagen by Confluent Streaming Audio
Improving Fairness Through Connection Throttling in the Cloud with KIP-402 ft. Gwen Shapira by Confluent Streaming Audio

New Releases

StreamSets Data Collector 3.11.0

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 20:56 on October 2, 2019
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 2.10.2019

This is the 170th edition of my blog series blog series around Stream Data Integration and Stream Analytics!

It is the Kafka Summit San Francisco 2019 edition, which I had the privilege to speak at and attend. Two very interesting days indeed! I have included all the presentation I fond reading tweets.

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading!

News and Blog Posts

General

Every Company is Becoming a Software Company by Jay Kreps
Event Storming meets Domain Driven Design with Alberto Brandolini: A Workshop Review by Fränze Deuber-Winski
Building Audit Logs with Change Data Capture and Stream Processing by Gunnar Morling

Apache Kafka / Kafka Streams / Confluent Platform

How to use Kubernetes to deploy stateful Kafka Streams applications by Abhishek Gupta
Incremental Cooperative Rebalancing in Apache Kafka: Why Stop the World When You Can Change It? by Konstantine Karantasis
Real-Time Analytics and Monitoring Dashboards with Apache Kafka and Rockset by Shruti Bhat & Kai Waehner
Schema Validation with Confluent 5.4-preview by Guozhang Wang
How to count events in Kafka Streams? by Lukas Chrzaszcz
Copying data between Kafka clusters with Kafkacat by Robin Moffatt
Building A Relational Database Using Kafka by Robert Yokota
Building Scalable Real Time Analytics, Alerting and Anomaly Detection Architecture at Dream11 by Dream11 Engineering
Kafka Summit San Francisco 2019: Day 1 Recap by Tim Berglund
Free Apache Kafka as a Service with Confluent Cloud by Priya Shivakumar
Messing around with Apache Kafka using the Confluent Cloud free trial by Richard Seroter
Kafka Transforming Into ‘Event Streaming Database’ by Alex Woodie

Apache Flink

Here’s What Makes Apache Flink scale by Kartik Khare

Spark Streaming

Brand Safety with Structured Streaming, Delta Lake, and Databricks by Rahul Jain and Vini Jaiswal

StreamSets

The StreamSets Cloud Beta is Open for Participation! by Pat Patterson
StreamSets Transformer Extensibility — Part 2: Spark MLeap Bundles to S3 by Dash Dasei
StreamSets Cloud Unlocking Insights: Amazon S3 to Snowflake by Dash Dasei

New Presentations

Streaming Visualization by Guido Schmutz
Location Analytics – Real-Time Geofencing using Kafka by Guido Schmutz
Fundamentals Big Data and AI Architecture by Guido Schmutz
Faas composition using Kafka and cloud-events by Neil Avery
A Zero Code Tracking Pipeline with Apache Kafka at METRO Markets by René Kerner
Building and Evolving a Dependency-Graph Based Microservice Architecture by Lars Francke
Cross the Streams Thanks to Kafka and Flink by Christophe Philemotte
Event-Driven Stream Processing and Model Deployment with Apache Kafka, Kafka Streams, TensorFlow, gRPC by Kai Waehner
Being an Kafka Developer Hero in the World of Cloud by Ricardo Ferreira
The art of the event streaming application. streams, stream processors and scale by Neil Avery
Streaming Data on the Shoulders of Giants by Hans-Peter Grahsl
Stream Processing Live Traffic Data with Kafka Streams by Tim Ysewyn & Tom Van den Bulck
Kafka on Kubernetes: Does it Really Have to Be “The Hard Way”? by Viktor Gamov

New Videos

Streams Must Flow: Deployment of Fault-tolerant Stream Processing Applications with Kafka Streams and Kubernetes by Viktor Gamov

New Podcasts

Kafka Data Pipelines with Robin Moffatt by SE Daily
Understand What’s Flying Above You with Kafka Streams ft. Neil Buesing by Streaming Audio
Scaling Apache Kafka with Todd Palino by Streaming Audio
MySQL, Cassandra, BigQuery, and Streaming Analytics with Joy Gao by Streaming Audio

New Releases

Debezium 0.10 Final Released

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!