gschmutz 07:24 on June 29, 2020
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 29.6.2020

Black Lives Matter (@Blklivesmatter) | Twitter

This is the 194th edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading and stay safe!

News and Blog Posts

Apache Kafka / Kafka Streams / Confluent Platform

Kafka on Kubernetes, the Strimzi way! (Part 2) by Abhishek Gupta
Should I backup my Kafka cluster? And how? by Jaroslaw Kijanowski
Learning Kafka: Kafka Connect + Wikipedia by Siarhei Krukau
Build Real-Time Observability Pipelines with Confluent Cloud and AppDynamics by Oli Watson
The Cost of Apache Kafka: An Engineer’s Guide to Pricing Out DIY Operations by Gwen Shapira & Joshua Buss
Connecting Kafka to MinIO by Alex
How Merging Companies Will Give Rise to Unified Data Streams by Naveen Nandan
Unifying Streams and State: The Seamless Path to Real-Time by Michael Drogalis
Announcing ksqlDB 0.10.0 by Steven Zhang
Announcing the Snowflake Sink Connector for Apache Kafka in Confluent Cloud by Nathan Nam
What Apache Kafka Costs — Pricing Out Open-Source DIY Operations by Gwen Shapira

Flink

Flink on Zeppelin Notebooks for Interactive Data Analysis – Part 2 by Jeff Zhang

Spark Streaming

Monitor Spark streaming applications on Amazon EMR by Amir Shenavandeh

New Presentations

How Apache Kafka helps to create Data Culture – How to Cross the Kafka Chasm by Andreas Grütter & Patrick Bözli
Real-Time Streaming Analytics with 100,000 Cars Using MQTT, Kafka and InfluxDB 2.0 on Kubernetes by Kai Waehner
4K – Kubernetes with Knative, Kafka and Kamel by Kamesh Sampath
Kafka on Pulsar:bringing native Kafka protocol support to Apache Pulsar by Sijie Guo & Pierre Zemb
Ten reasons to choose Apache Pulsar over Apache Kafka for Event Sourcing by Robert van Moelken
Tips & Tricks for Apache Kafka Ops by Jen Snipes & Kat Grigg
Stream Processing with ksqlDB by Tim Berglund
Interactive querying of streams using Apache Pulsar by Jerry Peng
Building a Streaming Microservice Architecture: with Apache Spark Structured Streaming and Friends by Scott Haines
Performant Streaming in Production: Preventing Common Pitfalls when Productionizing Streaming Jobs by Max Thoene & Stefan van Wouw

New Videos

How to install Kafka Connect connector plugins by Robin Moffatt
Loading a CSV file into Kafka (Kafka Connect in Action) by Robin Moffatt
How Apache Kafka helps to create Data Culture – How to Cross the Kafka Chasm by Andreas Grütter & Patrick Bözli

New Podcasts

Growing the Event Streaming Community During COVID-19 by Confluent Streaming Audio

New Releases

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 21:18 on June 17, 2020
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 17.6.2020

Black Lives Matter (@Blklivesmatter) | Twitter

This is the 193rd edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading and stay safe!

News and Blog Posts

General

Demystifying the Event Driven Architecture – An open solution by
Eric D. Schabell

Comparison

Pulsar vs Kafka – Comparison and Myths Explored by Kai Waehner

Apache Kafka / Kafka Streams / Confluent Platform

Test Your Kafka Cluster With Python by Geoffrey Mariette
Announcing the MongoDB Atlas Sink and Source Connectors in Confluent Cloud by Nathan Nam
Real-Time Fleet Management Using Confluent Cloud and MongoDB by Gnanaguru Sattanathan
My Python/Java/Spring/Go/Whatever Client Won’t Connect to My Apache Kafka Cluster in Docker/AWS/My Brother’s Laptop. Please Help! by Robin Moffatt
Stream Processing with IoT Data: Challenges, Best Practices, and Techniques by Jesse Yates
Consistent Metastore Recovery for ksqlDB Using Apache Kafka Transactions by Steven Zhang
Mirroring to Azure Event Hub evolution: using Mirror Maker 2 by Paolo Patierno
Kafka on Kubernetes, the Strimzi way! (Part 1) by Abhishek Gupta
Using Kafka as a Temporary Data Store and Data-loss Prevention Tool in The Data Lake by Lucio Daza
Scaling Apache Kafka to 10+ GB Per Second in Confluent Cloud by Dan Rosanova
Change Data Captures CDC from MySQL Database to Kafka with Kafka Connect and Debezium by Ramu Kakarla
Integration Of Apache Kafka With Mule 4 to Publish/Consume Data To/From Kafka Topics by Rashi Goel
Kafka Connect – Offset commit errors (I) by El Javi
Kafka Connect – Offset commit errors (II) by El Javi
Project Metamorphosis Month 2: Cost-Effective Apache Kafka for Use Cases Big and Small by Jay Kreps

Hazelcast Jet

How Hazelcast Jet Compares to Apache Spark by Vladimir Schreiner
Processing 10M queries / second on a single node using Jet and gRPC by Frantisek Hartman Marko Topolnik

Flink

Flink Community Update – June’20 by Marta Paes
Flink on Zeppelin Notebooks for Interactive Data Analysis – Part 1 by Jeff Zhang

Spark Streaming

Spark Streaming with HTTP REST endpoint serving JSON data by Chaitanya Chandurkar
Optimized Real-time Analytics using Spark Streaming and Apache Druid by Jatinder Assi

New Presentations

Kafka as a Platform: the Ecosystem from the Ground Up by Robin Moffatt
Integrating Oracle and Kafka by Robin Moffatt
AIA Tech Talk #1: Dive into Apache Kafka by Kenneth Cheung
Event Hub (i.e. Kafka) in Modern Data Architecture by Guido Schmutz
Introduction to data streaming by Nicolas Fraenkel
Data Engineer, Patterns & Architecture The future: Deep-dive into Microservices Patterns with Stream Process by Igor De Souza

New Videos

Demo: Scaling Kafka to 10GB/sec with Confluent Cloud by Confluent
Machine Learning through Streaming at Lyft by Sherin Thomas
Join the Data Movement: MongoDB and Apache Kafka by Robert Walters & Jeff Bean
Integrating Oracle and Kafka by Robin Moffatt
Solutions for bi-directional integration between Oracle RDBMS and Apache Kafka by Guido Schmutz
Miroservices, Events, and breaking the Data Monolith with Kafka by Viktor Gamov

New Podcasts

Exploring Event Streaming Use Cases with µKanren ft. Tim Baldridge by Confluent Streaming Audio

New Releases

Debezium 1.2.0.CR1

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!

gschmutz 19:11 on June 2, 2020
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Data Integration & Stream Analytics – 2.6.2020

Black Lives Matter (@Blklivesmatter) | Twitter

This is the 192nd edition of my blog series blog series around Stream Data Integration and Stream Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week. Happy reading and stay safe!

News and Blog Posts

General

Event-Driven Data Preprocessing by Victor Nitu

Apache Kafka / Kafka Streams / Confluent Platform

Quarkus – Using Apache Kafka with Reactive Messaging by Quarkus
Facts about Kafka every business should know by Ola Puchta-Górska
Fullstack Kafka by Chris Moune
Kafka Streams Interactive Queries Go Prime Time by Navinder Pal Singh Brar
Kafka vs. Pulsar vs. RabbitMQ: Performance, Architecture, and Features Compared by Confuent
Apache Kafka and Event Streaming in Pharma and Life Sciences by Kai Waehner
Kafka for Engineers by Dave Taubler
Getting Started with Protobuf in Confluent Cloud by Ricardo Ferreira
JDBC Nested Set Sink Connector
Building a Clickstream Dashboard Application with ksqlDB and Elasticsearch by Bill Bejeck
Learning All About Wi-Fi Data with Apache Kafka and Friends by Robin Moffatt
How to Prepare for the Confluent Certified Operator for Apache Kafka (CCOAK) exam by Stéphane Maarek
Best Practices to Secure Your Apache Kafka Deployment by Rick Spurgeon & Nikoleta Verbeck
Elastically Scaling Confluent Platform on Kubernetes by Nick Bryan
Confluent CLI 1.0 is Now Generally Available for Cloud and Platform by David Hyde
How to build a real-time analytics platform using Kafka, ksqlDB and ClickHouse ? by Florian Hussonnois

StreamSets Data Collector

Announcing StreamSets Data Collector 3.16.0 by Dash Desai
Oracle 19c Bulk Ingest And Change Data Capture Into Databricks Delta Lake by Dash Desai
Using Kubernetes Secrets in Data Collector Pipelines by Mark Brooks

New Presentations

Machine Learning with Apache Kafka in Pharma and Life Sciences by Kai Waehner
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka by Guido Schmutz

New Videos

Testing ksqDB 🚀 applications by Viktor Gamov
An introduction to ksqlDB by Robin Moffatt

New Podcasts

AMA with Tim Berglund | Streaming Audio Special by Confluent Streaming Audio
Scaling Apache Kafka in Retail with Microservices by Confluent Streaming Audio

New Releases

Please let me know if that is of interest. Please tweet your projects, blog posts, and presentations & videos to @gschmutz to get them listed in next week’s edition!