Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 20:24 on August 21, 2017
Tags: kafka ( 248 ), mqtt, nifi ( 238 ), Stream Processing ( 241 ), streamsets ( 74 )

Last Week in Stream Processing & Analytics – 21.8.2017

This is the 78th edition of my blog series blog series around Stream Processing and Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Five Questions to Embrace Diversity in Your Data Architecture by insideBIGDATA
Making sense of stream processing by Pat Patterson
Twitter, Yahoo Vets Pool Real-Time Processing Knowledge in Startup Streamlio by Susan Hall
MQTT v5 : what is on the way ? by Paolo Patierno

Apache Kafka / Kafka Streams / Confluent Platform

Leveraging the Power of a Database ‘Unbundled’ by Ben Stopford
How Kafka Helped Rabobank Modernize Alerting System by Alex Woodie
Data Ingestion with Spark and Kafka by Gary Dusbabek
How was Game Of Thrones S07 E05? Tweet Analysis with Kafka, BigQuery and Tableau by Francesco Tisiot
Kafka Consumer Architecture – Consumer Groups and Subscriptions by
Jean-Paul Azar
Kafka Topic Architecture – Replication, Failover, and Parallel Processing by
Jean-Paul Azar
Kafka Producer Architecture – Picking the Partition of Records by Jean-Paul Azar
Self-Learning Kafka Streams With Scala (Part 2) by Himanshu Gupta

StreamSets

Announcing Data Collector v2.7.0.0 by Kirit Basu

Apache NiFi / Hortonworks HDF

Processing Social Media Feeds in Stream with Apache NiFi 1.0.0 and NLTK by Timothy Spann

New Presentations

Big Data CEP using scale-out Drools in Apex Hadoop native platform by Tushar Gosavi & Priyanka Gugale
Apache Storm – A Real-time Processing System by masifqadri
Apache Spark – A Real-time Processing Tool by masifqadri
Analyzing Streaming Data in Real-time with Amazon Kinesis by Ray Zhu
Real-time Streaming Applications on AWS, Patterns and Use Cases by Ryan Nienhuis
Looking for Patterns – FlinkCEP Library by Dawid Wysakowicz
Building Microservices with Apache Kafka by Colin McCabe
ETL as a Platform: Pandora Plays Nicely Everywhere with Real-Time Data Pipelines by Lawrence Weikum
Survival of the Fittest – Streaming Architectures by Michael Hansen
Data Warehouse Modernization: Accelerating Time-To-Action by Pat Patterson & Ankur Desai
Kafka Connect implementation at GumGum by Karim Lamouri

New Videos

Continuent Webinar: Real-time Data Loading from Oracle and MySQL into Kafka by continuent
Big Data Complex Event Processing (CEP) using scale out Drools in Apex, a Hadoop native platform by Priyanka Gugale

New Releases

StreamSets Data Collector 2.7.0.0

Upcoming Events

24.8.2017 (San Francisco, US) – Visualizing and Analyzing Salesforce Data with StreamSets and Neo4j (Meetup)
27.8.2017 (online) – Embrace Streaming Analytics and Transform Your Business (AWS Webinar)
30.8.2017 (Dublin, IR) – The SMACK Stack (Meetup)
31.8.2017 (online) – Simplifying real-time architectures for IoT with Apache Kudu (Cloudera Webinar)
7.9.2017 (Berlin, GE) – Apache Flink Meetup Berlin @SAP (Meetup)
16.9.2017 (McLean, US) – Spark Saturday DC (Meetup)
18.9.2017 (Zurich, CH) – First Kafka Meetup Zurich (Meetup)
19.9.2017 (online) – Real-time marketing analytics with stream processing (O’Reilly Webcast)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 14:57 on August 15, 2017
Tags: aws ( 3 ), flink ( 92 ), kafka ( 248 ), kafka-connect ( 8 ), kafka-streams ( 176 ), nifi ( 238 ), pulsar ( 16 ), spark ( 153 )

Last Week in Stream Processing & Analytics – 14.8.2017

This is the 77th edition of my blog series blog series around Stream Processing and Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Complete event trend detection in high-rate data streams by Adrian Colyer
How to Check the Vital Signs of Real-Time Patient Monitoring by Kevin Jordan
Don’t Have Real-Time Analytics? Your Insights Are Already Too Late by Shelby Blitz
Apache Metron Streaming Architecture Stellar: “the Excel of Cybersecurity” by Casey Stella
How To Conquer Your Dataflow Chaos by Arvind Prabhakar
Lessons from LinkedIn: Faster Insights Through a Unified Data Ecosystem by Kat Campise

Apache Kafka / Kafka Streams / Confluent Platform

So, you want to build a Kafka Connector? Source edition by Chris Riccomini
The Simplest Useful Kafka Connect Data Pipeline In The World … or Thereabouts by Robin Moffat
Kafka Architecture by Jean-Paul Azar
What Is Kafka? by Jean-Paul Azar
Kafka Producer in Java by Jean-Paul Azar
Why Kafka Streams didn’t work for us? – Part 1 by Alexis Seigneurin
Why Kafka Streams didn’t work for us? – Part 2 by Alexis Seigneurin
Why Kafka Streams didn’t work for us? – Part 3 by Alexis Seigneurin
Cassandra to Kafka Data Pipeline (Part 1) by Vladimir Vajda
Dresses, Stream Processing and Apache Kafka by Cristina Negrean
How to Publish Data Collector Tables to Apache Kafka by Soniya

Amazon AWS

Building a Real World Evidence Platform on AWS by Aaron Friedman

Spark

Apache Spark’s Structured Streaming with Amazon Kinesis on Databricks by Jules Damji
Apache Spark Streaming by Shreya Gupta
Spark Streaming Kerberized Kafka by Talend Community

Apache Flink

Apache Flink: Stream Processing by lakshaysuri
An Introduction to Apache Flink by Dursun Koç

Apache Storm

Analyzing Twitter Data With Apache Storm, Power BI, and Azure SQL Server by Nischal S

Apache NiFi / Hortonworks HDF

Running SQL on FlowFiles using QueryRecord Processor (Apache NiFi 1.2+) by Andrew Lim
TensorFlow and NiFi: Big Data AI Sandwich by Tim Spann
Apache NiFi on Hortonworks HDF Verses … Microsoft Flow? by Ian Roseman
MiNiFi: Transmitting and Storing From Devices to Apache Hive by Tim Spann
Creating Word Clouds From DataFlows With Apache NiFi and Python by Tim Spann

New Presentations

NiFi Best Practices for the Enterprise by Greg Keys
Building Streaming Data Applications Using Apache Kafka by Slim Baltagi
Monitoring Big Data Hadoop Apex Apps by Ajay Gupta
Hadoop made fast – Why Virtual Reality Needed Stream Processing to Survive by Greg Fodor & Gehrig Kunz
Introducing Hazelcast Jet by Rahul Gupta
HDF 3.0 Deep Dive by Aldrin Piri

New Releases

Apache Pulsar 1.19.0-incubating

Upcoming Events

15.8.2017 (Milwaukee, US) – Sensors, Spark and Kafka: Applied Machine Learning (Meetup)
15.8.2017 (Los Angeles, US) – Kafka Connect Implementation at GumGum and more (Meetup)
16.8.2017 (online) – Pandora Plays Nicely Everywhere with Real-Time Data Pipelines (Confluent Webinar)
16.8.2017 (online) – Data Warehouse Modernization: Accelerating Time-to-Action (MapR Webinar)
16.8.2017 (online) – Tungsten Replicator for Kafka – Real-time Data Loading from Oracle and MySQL into Apache Kafka (Continuent Webinar)
24.8.2017 (San Francisco, US) – Visualizing and Analyzing Salesforce Data with StreamSets and Neo4j (Meetup)
27.8.2017 (online) – Embrace Streaming Analytics and Transform Your Business (AWS Webinar)
30.8.2017 (Dublin, IR) – The SMACK Stack (Meetup)
16.9.2017 (McLean, US) – Spark Saturday DC (Meetup)
18.9.2017 (Zurich, CH) – First Kafka Meetup Zurich (Meetup)
19.9.2017 (online) – Real-time marketing analytics with stream processing (O’Reilly Webcast)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!

gschmutz 22:08 on August 7, 2017
Tags: confluent ( 8 ), flink ( 92 ), IoT ( 10 ), kafka ( 248 ), nifi ( 238 ), spark ( 153 ), storm ( 46 ), Stream Processing ( 241 ), streaming-analytics ( 84 )

Last Week in Stream Processing & Analytics – 7.8.2017

This is the 76th edition of my blog series blog series around Stream Processing and Analytics!

As every week I was also updating the following two lists with the presentations/videos of the current week:

As usual, find below the new blog articles, presentations, videos and software releases from last week:

News and Blog Posts

General

Real-Time Anomaly Detection Streaming Microservices with H2O and MapR – Part 3: Production Deployment of ML Model With a Streaming Microservice by Mathieu Dumoulin
Event sourcing on blockchain with Ethereum, TypeScript and React by Krzysztof Ciesielski
Event Driven Microservices Patterns by Carol McDonald
Edge Computing Unlocks the Business Value of IoT Data by RTInsights Team
Flogo: Built From the Ground Up for Edge Computing by Matt Elis
NBCUniversal Changes the Game for TV Advertising with DC/OS and the SMACK Stack by Mesosphere

Apache Kafka / Kafka Streams / Confluent Platform

We’ll Say This Exactly Once: Confluent Platform 3.3 is Available Now by Geherig Kunz
How WePay uses stream analytics for real-time fraud detection using GCP and Apache Kafka by Wei Li
Messaging as the Single Source of Truth by Ben Stopford
Scaling Blockchains with Apache Kafka by Alex Miller
Kafka Streams: Unit Testing by Anuj Saxena
Install Apache Kafka On Windows by howtoprogram
Are You Ready for Kafka Summit San Francisco? by Tim Berglund
Kafka Streams and Unit Testing by Anuj Saxena

Apache Flink

Flink Forward Berlin Preview: The Apache Flink Ecosystem by Michael Winters

Apache NiFi / Hortonworks HDF

Ingest Salesforce Data Incrementally Into Hive Using Apache NiFi by
Saikrishna Teja Bobba
Building a Custom Processor in Apache NiFi 1.2 for TensorFlow Using the Java API by Timothy Spann
How to collect data like a spy — Part 1: How to collect data like a spy by Mark Craddock
How to collect data like a spy — Part 2: Getting NiFi up and running by Mark Craddock
How to collect data like a spy — Part 3: How to collect social media data like a pro by Mark Craddock
How to collect data like a spy — Part 4: Turning JSON into a database with AWS Athena by Mark Craddock
How to collect data like a spy — Part 5: Connecting rStudio to Amazon Athena by Mark Craddock
How to collect data like a spy — Part 6: Creating Maps of the Data in RStudio by Mark Craddock
Creating WordClouds From DataFlows with Apache NiFi and Python by Timothy Spann
Convert CSV to JSON, Avro, XML using ConvertRecord (Apache NiFi 1.2+) by Andrew Lim
Installing a local Hortonworks Registry to use with Apache NiFi by Andrew Lim

New Presentations

Streaming Real-Time Events Using Apache APIs by Carol McDonald
Exactly-once Data Processing with Kafka Streams by Guozhang Wang
How Yelp Leapt to Microservices with More than a Message Queue by Justin Cunningham
Deep learning and streaming in Apache Spark 2.2 by Matei Zaharia
Kafka RealTime Streaming by Viyaan Jhiingade

New Videos

Google Dataflow: The New Open Model for Batch and Stream Processing by Robert Kubis
Building an automation-driven Lambda architecture by Malwarebytes

New Releases

Upcoming Events

9.8.2017 (Bellevue, US) – Spark Structured Streaming : Introduction and Internals (Meetup)
9.8.2017 (Sydney, AU) – Data-in-Motion: Recent advances in Apache projects for streaming data (Meetup)
10.8.2017 (online) – Why VR Needed Stream Processing to Survive (Confluent Webinar)
15.8.2017 (Milwaukee, US) – Sensors, Spark and Kafka: Applied Machine Learning (Meetup)
16.8.2017 (online) – Pandora Plays Nicely Everywhere with Real-Time Data Pipelines (Confluent Webinar)
16.8.2017 (online) – Data Warehouse Modernization: Accelerating Time-to-Action (MapR Webinar)
24.8.2017 (San Francisco, US) – Visualizing and Analyzing Salesforce Data with StreamSets and Neo4j (Meetup)
27.8.2017 (online) – Embrace Streaming Analytics and Transform Your Business (AWS Webinar)
16.9.2017 (McLean, US) – Spark Saturday DC (Meetup)
18.9.2017 (Zurich, CH) – First Kafka Meetup Zurich (Meetup)
19.9.2017 (online) – Real-time marketing analytics with stream processing (O’Reilly Webcast)

Please let me know if that is of interest. Please tweet your projects, blog posts, and meetups to @gschmutz to get them listed in next week’s edition!