Vitess Database Clustering System

Vitess is a free and open-source database clustering system for horizontal scaling of MySQL, originally developed at YouTube (by Sugu Sougoumarane and Mike Biscoglio starting in 2010) to handle YouTube's massive database workload. Vitess was open-sourced in 2012 and became a CNCF graduated project in 2019 (the second after Kubernetes). It is used by Slack, Square, Flipkart, Noop, GitHub, and JD.com. Key features: sharding: Vitess transparently shards MySQL tables across multiple MySQL instances (shards). Sharding can be range-based or hash-based, with the ability to reshard (split or merge shards) without downtime. VSchemas (Vitess schemas) define how tables are sharded and route queries to the correct shards. Connection pooling: VTGate (the proxy layer) multiplexes application connections to a pool of MySQL connections, reducing connection overhead and preventing connection exhaustion. A single VTGate can handle tens of thousands of application connections while maintaining a small number of MySQL connections per shard. Query rewriting: VTGate parses and rewrites SQL queries, routing them to the appropriate shards. Cross-shard queries are scattered and gathered. ACID transactions: support for distributed transactions (two-phase commit) across shards. Read replication: reads can be distributed across primary and replica MySQL instances. VTTablet (the sidecar) manages individual MySQL instances, handling query deduplication, transaction management, and failover. Migration: VReplication enables online schema changes, resharding, and data migration with minimal downtime. Key range and vindex: vindexes define the sharding strategy, including lookup vindexes (secondary indexes) and functional vindexes (hash-based). Observability: integration with Prometheus and Grafana. Topology service: etcd or ZooKeeper-based cluster topology. Go. Apache-2.0.