github.com/jessewei/cp-demo ↗

About jessewei/cp-demo

This is a comprehensive demonstration of the Confluent Platform built around a real-world use case of streaming Wikipedia edits in real time. The project ingests live edit data from Wikimedia Foundation IRC channels using Kafka Connect, processes it through ksqlDB and Kafka Streams, and outputs the results to Elasticsearch for visualization in Kibana. The example showcases the full data pipeline including schema management via Confluent Schema Registry, message replication, and monitoring through Confluent Control Center.

All components in the demo have end-to-end security enabled, making it suitable for learning how to deploy secure event streaming applications in production environments. The architecture demonstrates multiple key technologies working together: Kafka as the event streaming backbone, various Kafka connectors for source and sink operations, stream processing frameworks, and supporting infrastructure for data governance and observability.

The project is designed as an educational resource paired with official Confluent documentation and tutorials. It targets developers and architects looking to understand how to build and manage event streaming applications using the Confluent Platform, with particular emphasis on security considerations and operational management across distributed systems.