Last Week in Stream Processing & Analytics – 29.1.2018

This is the 100th edition of my blog series blog series around Stream Processing and Analytics! Happy anniversary 🙂

As every week I was also updating the following two lists with the presentations/videos of the current week:

• Presentations from SlideShare
• Videos from YouTube

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

• Is Batch ETL Dead, and is Apache Kafka the Future of Data Processing? by Daniel Bryant
• How Much More Complicated Is Real-Time Big Data? by Jesse Anderson
• Streaming Data in the Enterprise Hits a Tipping Point: Our 2018 Predictions by Kostas Tzoumas
• From big data to fast data by Raul Estrada
• What To Do With Your IoT Data in 2018 by Katherine Lazarevich

Apache Kafka / Kafka Streams / Confluent Platform

• Debezium 0.7.2 Is Released by Gunnar Morling
• Monitoring Kafka Data Pipeline by Randhir Singh
• Using Mule With Kafka Connector by Greg Lawson
• Partitioning with Apache Kafka and Vert.x by Piotr Mińkowski
• Apache Kafka is not for Event Sourcing by Jesper Hammarbäck
• Custom Partitioning and Analysis using Kafka SQL Windowing by Treselle Engineering

Heron

• Heron on Kubernetes with Helm by Heron

Apache Pulsar

• Apache Pulsar: Distributed Pub-Sub Messaging System by Furkan Kamaci

Apache Flink

• Managing Large State in Apache Flink®: An Intro to Incremental Checkpointing by Stefan Richter and Chris Ward

Apache NiFi

• Introduction to Apache NiFi by Mark Payne
• Introduction to Real Big Data Engineering by Tim Spann
• Monitoring Energy Usage Utilizing Apache NiFi, Python, Apache Hive and a SmartPlug Device by Tim Spann
• Provenance Site to Site Reporting – via Apache NiFi by Tim Spann
• Post data to NIFI using PowerShell, or more broadly, using powershell to post data to a URL using Rest. by Digital Plumber

StreamSets

• 9TH BIT brings StreamSets Data Connector Edge to South Africa to address data drift by Mark Bowen

Presentations

• Kafka Streams: the easiest way to start with stream processing by Yaroslav Tkachenko
• Streaming ETL – from RDBMS to Dashboard with KSQL by Björn Rost
• Understanding time in structured streaming by Madhukara Phatak
• Apache Flink by Robert Metzger
• Debugging Apache Spark by Joey Echeverria
• Local Apache NiFi Processor Debug by Deon Huang

Videos

• Deploying Hazelcast IMDG and Apache Kafka for IoT Stream Processing and Analytics byAndrew Stevenson & Viktor Gamov
• SQL over Streams, Ask the Experts by InfoQ
• Microservices and Fast Data: Industry and Architecture Trends by Dean Wampler & Matt Aslett
• Deep Dive: Amazon Kinesis Video Streams by Amazon Web Services
• Let your domain events flow by Bernd Rücker & Martin Schimak
• Apache Flink: Building a Stream Processor for fast analytics by Robert Metzger
• Sustainable Computing at Scale for Smart Cities by Yogesh Simmhan

New Releases

• Debezium 0.7.2
• Heron 0.17.4

Upcoming Events

• 29.1.2018 (Vancouver, CA) – Data Pipelining and Refining at Scale (Meetup)
• 30.1.2018 (Helsinki, FI) – Apache Kafka Meetup @ Rovio (Meetup)
• 31.1.2018 (London, UK) – Building Streaming Data Pipelines into Elasticsearch with Apache Kafka and KSQL (Meetup)
• 1.2.2018 (Rom, IT) – Meetup #AperiTech di Roma Apache Kafka (Meetup)
• 13.2.2018 (online) – Empowering Streams through KSQL: Part 1: Exploring KSQL Patterns (Confluent Webinar)
• 13.2.2018 (Princeton, US) – Apache NiFi Mardi Gras (Meetup)
• 15.2.2018 (Sunnyvale, US) – Learn about unified queuing and streaming at Yahoo scale with Apache Pulsar (Meetup)
• 15.2.2018 (Ottawa, CA) – Building Multi-Region & Multi-Cloud Services with Kafka (Meetup)
• 20.2.2018 (online) – Empowering Streams through KSQL: Part 2: Live Coding a KSQL Application (Confluent Webinar)
• 21.2.2018 (online) – Unlock Value from Big Data with Apache NiFi and Streaming CDC (Hortonworks Webinar)
• 22.2.2018 (Boulder, US) – Messaging,storage,or both: The real-time story of Pulsar & Apache DistributedLog (Meetup)
• 1.3.2018 (Mexico City, MX) – Nuestro primer Kafka Meetup en Ciudad de México (Meetup)
• 15.3.2018 (online) – Empowering Streams through KSQL: Part 3: Deploying and Operating KSQL (Confluent Webinar)
• 17.4.2018 (Laurel, US) – Huge steps for Apache NiFi – Talks on NiFi in Finance and Powerful New Features  (Meetup)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!