Feb 18, 2021

Oracle to Kafka — Playing with Confluent’s new Oracle CDC Source Connector in Docker

Oracle to Kafka — Playing with Confluent’s new Oracle CDC Source Connector in Docker It’s really exciting to have a new option for streaming Oracle data into Kafka. With Confluent releasing their “Oracle CDC Source Premium Connector” there’s a new way to capture data that has been added to, updated, or deleted from Oracle RDBMS with Change Data Capture (CDC). Think of this as a low touch way to stream both the Oracle data and schema changes into Kafka and Schema Registry.

Feb 4, 2021

Cat tracking with Amazon Web Services— AWS IoT, DynamoDB and SageMaker

Cat tracking with Amazon Web Services— AWS IoT, DynamoDB and SageMaker If you read 1 Cat, 3 Clouds — Where is Snowy sleeping (Part 1) you’ll know I’m trying to build a real-time location tracker for my cat. Following on from that initial blog (which described the hardware and networking), this “Part 2” article describes data processing with Amazon Web Services. I’ll be using native AWS services for data ingest, data processing and mapping visualisation to build a low cost pet tracking device to show Snowy’s location in real-time.

Jan 10, 2021

1 Cat, 3 Clouds — Part 1 - Where is Snowy sleeping?

1 Cat, 3 Clouds — Where is Snowy sleeping? Our cat Snowy is an outdoor cat — but we are beginning to wonder where she travels once she’s left the house. Let me share how I built a low cost pet tracking device to show Snowy’s location in real-time. Plus, what’s it like to build the same IoT data project in AWS, Azure & Google Cloud in 2021? This is article 1 in a 4 part series.

Sep 21, 2020

Pi Peripheral - Controlling computer actions with a $10 Raspberry Pi Zero

Pi Peripheral Landing gear down and strobe lights active. How to add physical buttons, switches and dials to control computer actions with a $10 Raspberry Pi Zero. Why would you want to do this? Well you might want to add some additional buttons or switches to your favorite game. Or maybe you can streamline your video production but switching cameras with a foot pedal. For me, I wanted to add tactile controls to Microsoft Flight Simulator 2020.

Jun 9, 2020

Home office ergonomics — with TensorFlow

Home office ergonomics — with TensorFlow Monitor good posture — with machine learning. Ensure proper office ergonomics with TensorFlow, a webcam and OpenCV Python real-time computer vision libraries. Code available at https://github.com/saubury/posture-watch Real-time posture monitoring — as demonstrated with a teddy bear If you sit behind a desk for hours at a time, you’re possibly going to slump. This may result in neck and back pain. This can be avoided by teaching a machine to recognise “good” posture from “bad” posture.

May 18, 2020

🍹 Cocktail Pi

🍹 Cocktail Pi Cocktails based on your mood created by a Raspberry Pi bartender. Drinks are selected based on your emotion and multilingual voice prompts let you know when your drink is available. How to build a fully automated home build cocktail maker. Code available at **https://github.com/saubury/cocktail-pi**🍸 Pumps I used 4 peristaltic pumps to provide a “food safe” way to pump the liquids from the drink bottles. These pumps provide a steady rate of flow of liquids when powered.

May 2, 2020

Kafka with AVRO vs., Kafka with Protobuf vs., Kafka with JSON Schema

Kafka with AVRO vs., Kafka with Protobuf vs., Kafka with JSON Schema Kafka serialisation schemes — playing with AVRO, Protobuf, JSON Schema in Confluent Streaming Platform. The code for these examples available at https://github.com/saubury/kafka-serialization Apache Avro was has been the default Kafka serialisation mechanism for a long time. Confluent just updated their Kafka streaming platform with additional support for serialising data with Protocol buffers (or protobuf) and JSON Schema serialisation.

Feb 23, 2020

Kafka stream processing: sorting socks with ksqlDB - Part 2

Kafka stream processing: sorting socks with ksqlDB Sorting socks with a streaming solution? Pair socks with ksqlDB, Kafka and Kafka Connect. Socks — sorted! ksqlDB is a pretty cool event streaming platform. A few commands allow you to build a detailed real-time stream processing application. Capture, transform and perform continuous transformations on Kafka with a simplified SQL dialect. In addition, ksqlDB allows you to capture events from an external system using Kafka connect.

Feb 10, 2020

Sorting my socks with deep learning — Part 1

Sorting my socks with deep learning — Part 1 Can I arrange my sock drawer using machine learning? An adventure in pairing my socks using transfer learning to build a custom sock image classification model. A small project to train and deploy an object recognition model on AWS DeepLens hardware for identifying my socks. Sock classification madness to neatly paired The Idea I have a crazy idea that I can pair my mismatched socks with machine learning.

Jan 14, 2020

Kafka Connect Secrets Management with ksqlDB

Kafka Connect Secrets Management with ksqlDB Get data into and out of Kafka with ease using ksqlDB … without sharing your passwords Kafka Connect is a scalable, flexible framework for streaming data between Apache Kafka and other data systems. Got data in a database table and want to stream it into Kafka? No problem, a simple bit of configuration and a JDBC driver will get you going. What’s even better is this Kafka Connect configuration can be setup and managed with ksqlDB with Confluent Platform 5.