Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 16:20 on November 19, 2018
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Processing & Analytics – 18.11.2018

This is the 133rd edition of my blog series blog series around Stream Processing and Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Moments From My So-Called Real-Time Life by Joe McKendrick
Managing State in Serverless by Mark Boyd

Apache Kafka / Kafka Streams / Confluent Platform

Kafka change data capture breaks database encapsulation by Chris Riccomini
Kafka Connect Deep Dive – Converters and Serialization Explained by Robin Moffatt
Apache Kafka Supports 200K Partitions Per Cluster by Jun Rao
Kafka hands-on. Part I: development environment by Sergei Beilin
Connecting Apache Kafka to Azure CosmosDB — Part I by Hans-Peter Grahsl
Connecting Apache Kafka to Azure CosmosDB — Part II by Hans-Peter Grahsl
Does Apache Kafka do ACID transactions? by Andrew Schofield
Microsoft Announces the General Availability of Azure Event Hubs for Apache Kafka by Steef-Jan Wiggers

Spark Streaming

Hooking up Spark and Scylla: Part 4 by Itamar Ravid

Apache Flink

Broadcast State Pattern in Apache Flink: 4 important considerations by Markos Sfikas
Flink SQL for powerful querying of data streams and data at rest by
Aljoscha Krettek and Timo Walther
Stream processing: An Introduction to Event Time in Apache Flink by Markos Sfikas
Years in Big Data. Months with Apache Flink. 5 Early Observations With Stream Processing by Jeff Bean

Apache Pulsar

Real-Time Analytics with Pulsar Functions by David Kjerrumgaard

Apache NiFi

Real-Time Stock Processing With Apache NiFi and Apache Kafka Part 1 by Timothy Spann

Druid

How Druid enables analytics at Airbnb by Pala Muthiah and Jinyang Li

Presentations

Demystifying Kafka by Nakul Mishra
Introducing Apache Kafka and why it is important to Oracle, Java and IT professionals by Lucas Jellema
Scaling big with Apache Kafka by Nikolay Stoitsev
KSQL – The Open Source SQL Streaming Engine for Apache Kafka by Kai Waehner
Bridge to Cloud: Using Apache Kafka to Migrate to AWS by Priya Shivakumar & Konstantine Karantasis & Rohit Pujari
Streaming Data Microservices with Akka Streams, Kafka Streams, and Kafka by Dean Wampler
Stream Processing takes on everything by Konstantin Knauf
The Future of Streaming: Global Apps, Event Stores and Serverless by Ben Stopford
Building Ebay using Serverless and stream processing by Neil Avery
Bringing DevOps to Data Integration by Girish Pancha
Build Event-Driven Microservices With Apache Kafka by Gwen Shapira
(Big)Data Serialization with Avro and Protobuf by Guido Schmutz

Videos

Apache Kafka and KSQL in Action : Let’s Build a Streaming Data Pipeline! by Robin Moffatt
Data Streaming for Microservices using Debezium by Gunnar Morling
Embrace the Anarchy : Apache Kafka’s Role in Modern Data Architectures by Robin Moffatt
Processing Fast Data with Apache Spark: The Tale of Two Streaming APIs by Gerard Maas
Demystifying Kafka by Nakul Mishra
Beyond the DSL—Unlocking the power of Kafka Streams with the Processor API by Antony Stubbs
Running Kafka in Kubernetes a practical guide by Katherine Stanley

New Releases

Upcoming Events

27.11.2018 (online) – Stream into Kafka Series: Monitoring and Protecting Data in Kafka (StreamSets Webinar)
27.11.2018 (Prague, CZ) – Streaming applications – from basic to advance usage (Meetup)
28.11.2018 (Atlanta, US) – Stream Processing Best Practices – In the Cloud and with Apache Ignite (Meetup)
29.11.2018 (online) – Run Apache Kafka on Kubernetes with Red Hat AMQ streams (Redhat Webinar)
6.12.2018 (Cambridge, US) – StreamSets Boston User Group (Meetup)
7.12.2018 (Sao Paulo, BR) – Apache Kafka® Rethinked: From Messaging to a Streaming Data Platform (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 23:19 on November 7, 2018
Tags: kafka ( 248 ), kafka-streams ( 176 ), nifi ( 238 ), spark ( 153 ), spark-streaming ( 219 ), Stream Processing ( 241 )

Last Week in Stream Processing & Analytics – 7.11.2018

This is the 132nd edition of my blog series blog series around Stream Processing and Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Doing Business In Real Time, Responding To What Matters As It Occurs by Kin Lane

Apache Kafka / Kafka Streams / Confluent Platform

That’s a Wrap! Kafka Summit San Francisco 2018 Roundup by Tim Berglund
Apache Kafka on Kubernetes – Could you? Should you? by Gwen Shapira
The Kafka API Battle: Producer vs Consumer vs Kafka Connect vs Kafka Streams vs KSQL ! by Stéphane Maarek
Using Monix with Kafka, Avro and Schema Registry by Jendrik Poloczek
ATM Fraud Detection with Apache Kafka and KSQL by Robin Moffatt
How to run Kafka on Openshift, the enterprise Kubernetes, with AMQ Streams by Hugo Guerrero
Confluent Platform Now Validated on Oracle Cloud Infrastructure by Pinkesh Valdria
How to Send Data through Logstash or Kafka from Elastic APM by Gil Raphaelli
How to Work with Apache Kafka in Your Spring Boot Application by Igor Kosandyak
Announcing the general availability of Azure Event Hubs for Apache Kafka by Shubha Vijayasarathy
Testing Producer Deduplication in Apache Kafka and Apache Pulsar by Jack Vanlightly
Apache Kafka Supports 200K Partitions Per Cluster by Apache Kafka

Apache Pulsar

Event Processing Design Patterns with Pulsar Functions by David Kjerrumgaard
Comparing LogDevice and Apache Pulsar by Ivan Kelly
Querying Data Streams with Apache Pulsar SQL by Jerry Peng

StreamSets

Adaptive Data Integration and Operations on Oracle Cloud Using StreamSets by
Clarke Patterson

Apache NiFi

Load Balancing Data Across a NiFi Cluster by Mark Payne

Presentations

Apache Kafka and KSQL in Action : Let’s Build a Streaming Data Pipeline by Robin Moffatt
Data Streaming for Microservices using Debezium by Gunnar Morling
A Global Source of Truth for the Microservices Generation by Ben Stopford
A Tour of Apache Kafka by Matt Howlett
Fun with Kafka, Spark and Offset Management by Simona Meriam
Building Pinterest Real-Time Ads Platform Using Kafka Streams by Liquan Pei & Boyang Che
Serving the Real-Time Data Needs of an Airport with Kafka Streams and KSQL by Sönke Liebau
Kafka on OpenShift: make it easy with AMQ Streams by Paolo Patierno
Stream, stream, stream: Different streaming methods with Spark and Kafka by Itai Yaffe & Ron Tevel
Kafka Connect by Oleg Kuznetsov

Videos

Kafka Summit 2018

New Releases

Upcoming Events

13.11.2018 (online) – Stream into Kafka Series: Machine Learning with Tensorflow and Apache Kafka (StreamSets Webinar)
13.11.2018 (London, UK) – Exactly-once Semantics In Apache Kafka with Jay Kreps (Meetup)
14.11.2018 (Los Gatos, US) – Druid Bay Area Meetup @ Netflix (Meetup)
14.11.2018 (Paris, FR) – Flink SQL in Action, Low latency decision for Bot and Flink goes Cloud Native (Meetup)
27.11.2018 (online) – Stream into Kafka Series: Monitoring and Protecting Data in Kafka (StreamSets Webinar)
27.11.2018 (Prague, CZ) – Streaming applications – from basic to advance usage (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!