Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 14:31 on December 14, 2021
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 14.12.2021

This is the 236th edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading and stay safe!

News and Blog Posts

General

Tech Radar on Event Streaming Platforms by Dima Kalashnikov

Kafka

Migrating Data from On-Premises Data Platforms to Databricks Delta Lake in AWS by Paul Earsy
How To Integrate Apache Druid With Apache Kafka by Lahaul Seth
A Guide to Stream Processing and ksqlDB Fundamentals by Dave Klein
O Kafka, Where Art Thou? by Gunnar Morling
Scaling Kafka at Honeycomb by Liz Fong-Jones
Confluent Platform 7.0: Data Streaming Across Multiclouds by B. Cameron Gain
Best Tutorials for Getting Started with Apache Kafka by Evan Bates
Introducing 18 New, Fully Managed Connectors in Confluent Cloud by Sheryl Li & Diby Malakar
Kafka Monthly Digest: November 2021 by Mickael Maison
Apache Kafka for Conversational AI, NLP and Chatbot by Kai Waehner
Getting Started with Apache Kafka in Python by Robert Zych
Serverless Stream Processing with Apache Kafka, AWS Lambda, and ksqlDB by Bill Bejeck
The Cloud-Native Chasm: Lessons Learned from Reinventing Apache Kafka® as a Cloud-Native, Online Service by Gwen Shapira et al.
Debezium UI support for Single Message Transformations by Mark Drilling

Flink

Advise on Apache Log4j Zero Day (CVE-2021-44228) by Konstanin Knauf

StreamSets

The Dollars and Sense of DataOps by StreamSets

New Presentations

Event Streaming CTO Roundtable for Cloud-native Kafka Architectures by Kai Waehner
The Top Five Use Cases & Architectures for Data In Motion in 2022 by Kai Waehner

New Videos

Lessons Learned From Designing Serverless Apache Kafka ft. Prachetaa Raghavan by Streaming AudioksqlDB Fundamentals: How Apache Kafka, SQL, and ksqlDB Work Together ft. Simon Aubury by Confluent Streaming Audio
Lessons Learned From Designing Serverless Apache Kafka ft. Prachetaa Raghavan by Confluent Streaming Audio (#190)
Using Apache Kafka as Cloud-Native Data System ft. Gwen Shapira by Confluent Streaming Audio (#189)
Analytics Pipelines using Apache Kafka, ksqlDB, and Apache Druid by Neil Buesing

New Podcasts

ksqlDB Fundamentals: How Apache Kafka, SQL, and ksqlDB Work Together ft. Simon Aubury by Confluent Streaming Audio
Lessons Learned From Designing Serverless Apache Kafka ft. Prachetaa Raghavan by Confluent Streaming Audio (#190)
Using Apache Kafka as Cloud-Native Data System ft. Gwen Shapira by Confluent Streaming Audio (#189)

New Releases

Debezium 1.8.0.CR1 Released

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 07:25 on November 23, 2021
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 22.11.2021

This is the 235th edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading and stay safe!

News and Blog Posts

Kafka

Kafka Monthly Digest: October 2021 by Mickael Maison
Easily Copy or Migrate Schemas Anywhere with Schema Linking by Robert Yokota
Announcing ksqlDB 0.22.0 by Matthias J. Sax
How to Efficiently Subscribe to a SQL Query for Changes by Alan Sheinberg
Streaming Data Exchange with Kafka and a Data Mesh in Motion by Kai Waehner
Migrating Kafka clusters with MirrorMaker2 and Strimzi by Roman Martin & Manuel Schindler

Flink

Flink Backward – The Apache Flink Retrospective by Johannes Moser

Apache NiFi

NiFi as a Function in DataFlow Service by Pierre Villard

New Videos

The Architect: Bare Metalsson Rejects the Cloud by Bare Metalsson 🤣🤣🤣
From Batch to Streams: Building Value from Data In-Motion by Ricardo Ferreira
Handling Message Errors and Dead Letter Queues in Apache Kafka ft. Jason Bell by Confluent Streaming Audio

New Podcasts

Handling Message Errors and Dead Letter Queues in Apache Kafka ft. Jason Bell by Confluent Streaming Audio
A Bootiful Podcast: Fellow Java Champion Kate Stanley on Kubernetes and Apache Kafka by Josh Long

New Releases

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 22:07 on November 9, 2021
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 9.11.2021

This is the 234th edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading and stay safe!

News and Blog Posts

Kafka

A Guide to Kafka Streams and Its Uses by Bill Bejeck
Stream Governance – How it Works by Tushar Thole
Is Apache Kafka an iPaaS or is Event Streaming its own Software Category? by Kai Waehner
Condition Monitoring and Predictive Maintenance with Apache Kafka by Kai Waehner
Readings in Streaming Database Systems by Michael Drogalis
The Future of SQL: Databases Meet Stream Processing by Matthias J. Sax & Michael Noll
4 Key Design Principles and Guarantees of Streaming Databases by Guozhang Wang
How Do You Change a Never-Ending Query? by Almog Gavra
Scaling Apache Druid for Real-Time Cloud Analytics at Confluent by Zohreh Karimi & Harini Rajendran
Building Real-Time Hybrid Architectures with Cluster Linking and Confluent Platform 7.0 by Hasan Jilani
Why should you use a Canary in your Apache Kafka coal mine? by Paolo Patierno

Spark

What’s new in Apache Spark 3.2.0 – RocksDB state store by Bartosz Konieczny

New Presentations

Change Data Streaming Patterns in Distributed Systems @ Flink Forward 2021 by Gunnar Morling & Hans-Peter Grashl

New Videos

Event API Products for Kafka by Ari Hermawan & Aaron Lee

New Podcasts

Automating Infrastructure as Code with Apache Kafka and Confluent by Confluent Streaming Audio (#183)
Real-Time Stream Processing with Kafka Streams ft. Bill Bejeck by Confluent Streaming Audio (#184)
Confluent Platform 7.0: New Features + Updates by Confluent Streaming Audio (#185)

New Releases

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 07:20 on October 22, 2021
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 21.10.2021

This is the 233rd edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading and stay safe!

News and Blog Posts

General

When to Use Reverse ETL and when it is an Anti-Pattern by Kai Waehner
Using Debezium to Create a Data Lake with Apache Iceberg by Ismail Simsek

Kafka

Reliable Messaging in Microservices – Implementing Outbox Pattern using Kafka, Debezium, Micronaut, and Oracle Database on Kubernetes by Sina Nourian
How to Securely Connect Confluent Cloud with Services on AWS, Azure, and GCP by Shiva Mogili
Confluent Brings Multicloud Stream Governance to Kafka by Cameron Gain
Fast Path to Streaming Data Analysis by Neil Buesing
Apache Kafka in the Public Sector – Blog Series about Use Cases and Architectures by Kai Waehner
Apache Kafka in the Public Sector – Part 2: Smart City by Kai Waehner
Apache Kafka in the Public Sector – Part 3: Government and Citizen Services by Kai Waehner
Apache Kafka in the Public Sector – Part 4: Energy and Utilities by Kai Waehner
Apache Kafka in the Public Sector – Part 5: National Security and Defense by Kai Waehner
Incremental Snapshots in Debezium by Jiri Pechanec
Accelerate Your Cloud Data Warehouse Migration and Modernization with Confluent by Kevin Balaji & Alicia Moniz
Building a realtime ticket booking solution with Kafka, FastAPI, and Ably by Ben Gamble
Introducing Single Message Transforms and New Connector Features on Confluent Cloud by Piyush Bhansali & Sheryl Li
Apache Kafka and R: Real-Time Prediction and Model (Re)training by Patrick Neff
Kafka Monthly Digest: September 2021 by Mickael Maison
Kafka mesh filter in Envoy by Adam Kotwasinski
A True Atomic Microservices Implementation with Debezium to Ensure Data Consistency by Braindose
MongoDB Change Data Capture via Debezium Kafka Connector with a .NET 5 Client by Vasil Kosturski
Spring for Apache Kafka 101 by Viktor Gamov
Kafka Streams 101 by Daniel Ciocîrlan
Using ksqlDB for Real-Time Lead Management and Reporting at Leadnomics by Danny Scott
Extracting Value from IoT Using Azure Cosmos DB, Azure Synapse Analytics, and Confluent Cloud by Ramya Oruganti & Abhishek Gupta
Apache Kafka vs Microsoft Azure Service Bus by Ritam Das
Kafka Connect and Elasticsearch vs. PostgreSQL Pipelines: Initial Performance Results (Pipeline Series Part 8) by Paul Brebner
Kafka Connect and Elasticsearch vs. PostgreSQL Pipelines: Final Performance Results (Pipeline Series Part 9) by Paul Brebner
Confluent’s Oracle CDC Connector Now Supports Oracle Database 19c by Nathan Nam
Migrating Kafka transparently between Zookeeper clusters by Edmund Mok

Spark

Native Support of Session Window in Spark Structured Streaming by Jungtaek Lim & Yuanjian Li & Shixiong Zhu

StreamSets

Ingest Continuous Data into your Snowflake Data Cloud by Thomas Ganka & Kate Guttridge
The Dollars and Sense of DataOps by StreamSets

New Presentations

Telco 4.0 – Payment and FinServ Integration for Data in Motion with 5G and Apache Kafka by Kai Waehner

New Videos

Getting Started with Spring for Apache Kafka ft. Viktor Gamov by Confluent Streaming Audio (#182)
Automating DevOps for Apache Kafka and Confluent ft. Pere Urbón-Bayes by Confluent Streaming Audio (#180)

New Podcasts

Automating DevOps for Apache Kafka and Confluent ft. Pere Urbón-Bayes by Confluent Streaming Audio (#180)
Powering Event-Driven Architectures on Microsoft Azure with Confluent by Confluent Streaming Audio (#181)
Getting Started with Spring for Apache Kafka ft. Viktor Gamov by Confluent Streaming Audio (#182)

New Releases

Debezium 1.7.0.Final

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 17:40 on September 29, 2021
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 29.9.2021

This is the 232nd edition of my blog series blog series around Stream Data Integration and Stream Analytics! There has been some time since my last blog, so this one is once more rather large! Will try to get back to a weekly schedule ….

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading and stay safe!

News and Blog Posts

Kafka

What’s new in Kafka 3.0 by Apache Kafka
What’s New in Apache Kafka 3.0.0 by Konstantine Karantasis
Kafka for Neo4j Aura Is Available Now by David Allen
Kappa Architecture is Mainstream Replacing Lambda by Kai Waehner
Messaging Modernization with Confluent by Jeff Bean
Announcing ksqlDB 0.21.0 by Walker Carlson
Trigger AWS Lambda Functions Directly from an Apache Kafka Topicc by Jobin George & Joseph Morais
Completed Kafka Connectivity by Yannic Klem
Kafka Connect Fundamentals: What is Kafka Connect? by Tim Berglund

Flink

Real-Time Exactly-Once Ad Event Processing with Apache Flink, Kafka, and Pinot by Jacob Tsafatinos, Yuriy Bondaruk, Yupeng Fu, and James Kwon
Apache Flink 1.14.0 Release Announcement by Stephan Ewen & JOhannes Moser

StreamSets

Our StreamSets Data Integration Patterns by Instadeq Team

New Presentations

All at Sea with Streams – Using Kafka to Detect Patterns in the Behaviour of Ships by Robin Moffatt

New Videos

All at Sea with Streams – Using Kafka to Detect Patterns in the Behaviour of Ships by Robin Moffatt
Apache Kafka 3.0 – Improving KRaft and an Overview of New Features by Tim Berglund

New Podcasts

Apache Kafka 3.0 – Improving KRaft and an Overview of New Features by Confluent Streaming Audio (#177)
Designing a Cluster Rollout Management System for Apache Kafka ft. Twesha Modi by Confluent Streaming Audio (#178)
Intro to Kafka Connect: Core Components and Architecture ft. Robin Moffatt by Confluent Streaming Audio (#179)

New Releases

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 06:17 on September 20, 2021
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 19.9.2021

This is the 231st edition of my blog series blog series around Stream Data Integration and Stream Analytics! There has been some time since my last blog, so this one is once more rather large! Will try to get back to a weekly schedule ….

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading and stay safe!

News and Blog Posts

General

Introducing WebSockets — Building Real-time Apps by Jung Taejin
How to integrate legacy systems with event-driven architecture? by Mortaza Ghahremani
Event Driven Architecture (3) Operational Data Store using event streaming by Architech
Many Faces of Event-driven Architecture by Dunith Dhanushka
How to integrate legacy systems with event-driven architecture? by Mortaza Ghahremani
Understanding Protobuf Compatibility by Robert Yokota

Kafka

Monitor Apache Kafka messages using Elasticsearch/Logstash/Kibana stack by Shashank Srivastava
Netflix Builds a Reliable, Scalable Platform with Event Sourcing, MQTT and Alpakka-Kafka by Eran Stiller
Hydrating a Data Lake using Log-based Change Data Capture (CDC) with Debezium, Apicurio, and Kafka Connect on AWS by Gary A. Stafford
Make a mock “real-time” data stream with Python and Kafka by Maria Patterson
Apache Kafka in Cybersecurity for Situational Awareness in Real-Time by Kai Waehner
Driving New Integrations with Confluent and ksqlDB at ACERTUS by Jeffrey Jennings
From Pets to Cattle: Going Cloud Native with Apache Kafka by Jennifer Riggins
Optimizing Kafka Streams Apps on Kubernetes by Splitting Topologies by Victor Künstler
Apache Kafka in Cybersecurity for Digital Forensics by Kai Waehner
Going ZooKeeper-less With the Debezium Container Image for Apache Kafka by Gunnar Morling
Announcing Elastic Data Streams Support for Confluent’s Elasticsearch Sink Connector by Kundan Vyas
How ksqlDB Works: Internal Architecture and Advanced Features by Michael Drogalis
Run Self-Managed Kusto Kafka Connector Serverless in Azure Container Instances by Niels Berglund
Real-time Einstein Insights Using Kafka Streams by Rohit Deshpande
Event Sourcing Outgrows the Database by Ben Stopford
Kafka Monthly Digest – August 2021 by Mickael Maison
Confluent Unlocks the Full Power of Event Streams with Stream Governance by David Araujo
Getting Started with GraphQL and Apache Kafka by Gerard Klijs
Kafka Summit Americas 2021 Recap by Tim Berglund
Operating Apache Kafka with Cruise Control by Viktor Somogyi-Vass
Enabling Seamless Kafka Async Queuing with Consumer Proxy by Yang Yang, Zhifeng Chen, Qichao Chu, Haitao Zhang, & George Teo

Flink

Stateful Functions 3.1.0 Release Announcement by Seth Wiesman & Igal Shilman & Tzu-Li (Gordon) Tai
Implementing a Custom Source Connector for Table API and SQL – Part One by Ingo Buerk & Daisy Tsang
Implementing a custom source connector for Table API and SQL – Part Two by Ingo Buerk & Daisy Tsang

Pulsar

Building Scalable Streaming Applications with DataStax Astra Streaming by Jeff Davies

Redpanda

Schema Registry – The event is the API by Ben Pope

RabbitMQ

RabbitMQ In Event-Driven Communication Between Microservices by Darshana Wickramasinghe
Observability with RabbitMQ and microservices by Dmitrii Tarasov

AWS

Adopting an Event-Driven Architecture: A comparative look at AWS messaging solutions — SNS by Mario Bittencourt
Introducing Amazon MSK Connect – Stream Data to and from Your Apache Kafka Clusters Using Managed Connectors by Danilo Poccia

New Presentations

Cloud-Native 5G, MEC and OSS/BSS/OTT Telco with Apache Kafka and Kubernetes by Kai Waehner
Rethinking Financial Services with Data in Motion by Jeffrey Lam
Event sourcing – what could possibly go wrong? by Andrzej Ludwikowski

New Videos

Data Mess to Data Mesh | Kafka Summit Americas 2021 Keynote by Jay Kreps
Eberhard Wolff about Kafka Patterns and Anti-Patterns by Erik Wilde
Kafka – The New Database Monolith by Eberhard Wolf
Panel Discussion about Kafka, Edge, Networks and 5G in Oil & Gas and Mining Industry by Kai Wähner
Kafka in the Wild by Laura Schornack & Maureen Penzenik
Event-Driven applications: Apache Kafka and Python by Francesco Tisiot
Event-driven APIs & Schema governance for Apache Kafka by Hugo Guerrero
All at Sea with Streams – Using Kafka to Detect Patterns in the Behaviour of Ships by Robin Moffatt
Multi-Cluster Apache Kafka with Cluster Linking ft. Nikhil Bhatia by Confluent Streaming Audio (#174)
Using Apache Kafka and ksqlDB for Data Replication at Bolt by Confluent Streaming Audio (#173)

New Podcasts

Using Apache Kafka and ksqlDB for Data Replication at Bolt by Confluent Streaming Audio (#173)
Multi-Cluster Apache Kafka with Cluster Linking ft. Nikhil Bhatia by Confluent Streaming Audio (#174)
What Is Data Mesh, and How Does it Work? ft. Zhamak Dehghani by Confluent Streaming Audio (#175)
How to Build a Strong Developer Community with Global Engagement ft. Robin Moffatt and Ale Murray by Confluent Streaming Audio (#176)
Designing Data Intensive Applications with Martin Kleppmann by Coding over Cocktails

New Releases

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 21:39 on August 24, 2021
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 24.8.2021

This is the 230th edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading and stay safe!

News and Blog Posts

General

Introduction to event-driven architecture by Aiven
Stream Your Database Changes with Change Data Capture by Taron Foxworth
Outbox Pattern for reliable data exchange between Microservices by Eresh Gorantla

Kafka

Introducing the Debezium UI by Mark Drilling
Hydrating a Data Lake using Query-based CDC with Apache Kafka Connect and Kubernetes on AWS by Gary A. Stafford
Announcing ksqlDB 0.20.0 by Chittaranjan Prasad
Confluent builds out Kafka event streaming platform by Sean Michael Kerner
Using HTTP Bridge as a Kubernetes sidecar by Jakub Scholz
Confluent and Google Cement Their Ever-Stronger Partnership with Another Smart Analytics Partner of the Year Award by Vinod Devan
Panel Discussion about Kafka, Edge, Networking and 5G in Oil and Gas and Mining Industry by Kai Waehner
ClickHouse + Kafka = ❤ by Tim Lavnik
Announcing the Confluent Q3 ’21 Release by Dan Rosanova
Debezium Community Stories With… Sergei Morozov by Gunnar Morling
Implement a Cross-Platform Apache Kafka Producer and Consumer with C# and .NET by Jim Galasyn

New Presentations

Apache Kafka in the Transportation and Logistics by Kai Waehner

New Videos

Kafka Connect for Azure Cosmos DB by Azure Cosmos DB
Gaming and Betting Use Cases and Architectures for Data in Motion by Kai Waehner & Sami Ahmed & Jordan Crabb & Zachary Hamilton
Streaming Analytics Made Easy: Kinesis Data Analytics Studio Run on Apache Flink by Deepthi Mohan & Jeremy Ber

New Podcasts

Druid: Event-Driven Data with Eric Tschetter by SE Daily
Placing Apache Kafka at the Heart of a Data Revolution at Saxo Bank by Confluent Streaming Audio (#172)

New Releases

ksqlDB 0.20.0

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 18:19 on August 11, 2021
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 11.8.2021

This is the 229th edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As my blog is usually quite Kafka-centric, there is something I would like to especially point out this week: the new Confluent Developer site went live last week with tons of really good material on Kafka. This is the place to go to get started with Kafka as well as if you already using Kafka 🙂 with many great animated diagrams, such as this one:

Learn about the fundamentals of Kafka, event streaming, and the surrounding ecosystem. — Source: Confluent Developer Site – by Confluent

For an overview what is available, check out Jay Kreps latest blog article.

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading and stay safe!

News and Blog Posts

Kafka

Confluent Developer Site by Confluent
The New One-Stop Shop for Learning Apache Kafka by Jay Kreps
Designing and Architecting the Confluent CLI by David Hyde
Kafka Monthly Digest – July 2021 by Mickael Maison
Kafka for Cybersecurity (Part 6 of 6) – SIEM / SOAR Modernization by Kai Waehner
Change Data Capture (CDC) With Kafka Connect and the Debezium Cassandra Connector (Part 1) by Paul Brebner
Instaclustr Announces the General Availability of Graviton2 for Managed Apache Kafka by Instaclustr
Helpful Tools for Apache Kafka Developers by Dave Klein
Field Notes: Deliver Messages Using an IoT Rule Action to Amazon Managed Streaming for Apache Kafka by Siddhesh Keluskar
Announcing the Azure Cosmos DB Sink Connector in Confluent Cloud by Nathan Nam
Hydrating a Data Lake using Query-based CDC with Apache Kafka Connect and Kubernetes on AWS by Gary A. Stafford

New Presentations

Apache Kafka and API Management / API Gateway – Friends, Enemies or Frenemies? by Kai Waehner
Kafka and GraphQL: Misconceptions and Connections by Gerard Klijs
Real-time Analytics with Upsert Using Apache Kafka and Apache Pinot by Yupeng Fu
Use ksqlDB to migrate core-banking processing from batch to streaming by Mark Teehan

New Videos

Confluent CEO Jay Kreps on growing the cloud data business by CNBC
Level-up your gaming telemetry using Kafka Streams by Evan Shortiss

New Podcasts

Minimizing Software Speciation with ksqlDB and Kafka Streams ft. Mitch Seymour by Confluent Streaming Audio (#170)
Advanced Stream Processing with ksqlDB ft. Michael Drogalis by Confluent Streaming Audio (#171)

New Releases

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 16:40 on August 3, 2021
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 3.8.2021

This is the 228th edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading and stay safe!

News and Blog Posts

General

Turning Microservices Inside-Out by Bilgin Ibryam

Kafka

Kafka for Cybersecurity (Part 3 of 6) – Cyber Threat Intelligence by Kai Waehner
Kafka for Cybersecurity (Part 4 of 6) – Digital Forensics by Kai Waehner
Kafka for Cybersecurity (Part 5 of 6) – Zero Trust and Air-Gapped Environments by Kai Waehner
Real-World Deployments of Kafka in the Automotive Industry by Kai Waehner
Cleaning messy sensor data in Kafka with ksqlDB by Simon Aubury
Design Considerations for Cloud-Native Data Systems by Gwen Shapira
Making Apache Kafka Serverless: Lessons From Confluent Cloud by Prachetaa Raghavan
Speed, Scale, Storage: Our Journey from Apache Kafka to Performance in Confluent Cloud by Alok Nikhil & Adithya Chandra
Kafka Summit APAC 2021 Recap by Tim Berglund
Reduce Your Data Infrastructure TCO with Confluent’s New Splunk S2S Source Premium Connector by Piyush Bhansali
From On-Prem to Cloud-Native: Multi-Tenancy in Confluent Cloud by Anna Povzner & Anastasia Vela
Protecting Data Integrity in Confluent Cloud: Over 8 Trillion Messages Per Day by Rohit Shekhar & Alok Thatikunta & Marc Selwan
Event-driven OpenFaaS with Managed Kafka from Aiven by Alex Ellis
Kafka and Starburst: 3 Considerations for Accelerating Time to Value by Clark Bradley
Streaming JSON Data Into PostgreSQL Using Open Source Kafka Sink Connectors (Pipeline Series Part 6) by Paul Brebner

Pulsar

DataStax Astra Streaming: Why and How we Built Apache Pulsar-as-a-Service by Prabhat Jha & Chris Bartholomew

Flink

Flink on TiDB: Reliable, Convenient Real-Time Data Service by Jia Lin

New Presentations

The Log of All Logs: Raft-based Consensus Inside Kafka by Guozhang Wang
Cloud-Native Kafka, Integration and Kubernetes by Christina Lin

New Videos

Leveraging Data in Motion Kafka Summit APAC 2021 Keynote by Jun Rao
Closing Keynote: The Physics of Streaming by Tim Berglund
Extending Kafka for Internet Scale Distribution of Data by Mike Eisterer
Stream Processing with Apache Kafka and .NET by Niels Berglund

New Podcasts

Consistent, Complete Distributed Stream Processing ft. Guozhang Wang by Confluent Streaming Audi (#168)
Powering Real-Time Analytics with Apache Kafka and Rockset by Confluent Streaming Audio (#167)
Collecting Data with a Custom SIEM System Built on Apache Kafka and Kafka Connect ft. Vitalii Rudenskyi by Confluent Streaming Audio (#169)
Explaining how Kafka works with Robin Moffatt by Coding over Cocktails Podcast

New Releases

Debezium 1.7.0.Alpha1 Released

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!