github.com/taskcluster/taskcluster ↗
CI at Scale
Open this visualization on its own page →
Contributors
238
Lines of Code
119,308
From
2014-07-02
To
2022-02-01
About taskcluster/taskcluster
Taskcluster is the task execution framework that powers Mozilla's continuous integration and release infrastructure at scale. It provides a distributed system for defining, scheduling, and executing tasks across many workers, enabling large organizations to manage complex CI and CD pipelines efficiently.
The project is organized as a monorepo containing multiple interconnected services, libraries, and tools. Key components include the Queue Service for task management, Worker Manager for provisioning workers across cloud providers, authentication and secrets management services, GitHub integration, and various client libraries for JavaScript, Python, Go, Rust, and Shell. The infrastructure also includes a web-based UI, database layer, and specialized tools like Worker Runner and Websocktunnel for communication and task execution.
Taskcluster is primarily written in JavaScript with significant Go components and serves Mozilla's extensive CI needs. It's designed to scale across many machines and cloud environments, with a comprehensive architecture that handles worker provisioning, task queuing, authentication, notifications, and result indexing. The project welcomes contributions and maintains detailed documentation for both users and developers.