github.com/alibaba/canal ↗

About alibaba/canal

Canal is Alibaba's open-source MySQL binlog incremental subscription and consumption component, designed to parse MySQL binary logs and provide real-time data change capture and streaming. The project originated from Alibaba's need to synchronize data across geographically distributed data centers, evolving from business-level triggers to a robust log-based change data capture solution that supports MySQL versions 5.1 through 8.0.

The core mechanism works by having Canal emulate a MySQL slave, sending dump protocol requests to the MySQL master and receiving binary log events in real time. Canal then parses these binary log objects and makes the incremental changes available for downstream consumption. Common use cases include database replication and backup, real-time index construction and maintenance, cache invalidation, and data pipeline enrichment with business logic.

Canal follows a client-server architecture with protobuf 3.0 for protocol communication, enabling multi-language client support including Java, C#, Go, PHP, Python, Rust, and Node.js. Recent versions introduced significant enhancements including 150% performance improvements, native Prometheus monitoring, Kafka and RocketMQ message delivery, Aliyun RDS binlog support, and a web-based administration interface called Canal Admin for dynamic configuration and operational management through an online UI dashboard.