CubeFS Distributed File System

CubeFS (formerly ChubaoFS) is a free and open-source cloud-native distributed storage system developed by JD.com (Jingdong, one of China's largest e-commerce companies, headquartered in Beijing). Originally created in 2016 to address the storage challenges of JD.com's massive e-commerce platform (processing over 100 PB of data and billions of files daily), CubeFS was open-sourced in 2019 and graduated as a CNCF (Cloud Native Computing Foundation) project in 2022. Key features: multi-protocol support: provides POSIX-compliant file access (mounted as a local filesystem via FUSE), S3-compatible object storage, and HDFS-compatible access for big data workloads. This allows applications to access the same data through different protocols simultaneously. Architecture: consists of three components: Master (metadata management, cluster management via Raft consensus), MetaNode (metadata partition servers storing inode and dentry information), and DataNode (data partition servers storing file data chunks). Volume: logical storage unit with configurable replication factor (1, 2, or 3 replicas) or erasure coding for cost-efficient storage. Erasure coding reduces storage overhead by up to 50% compared to 3-replica setups. Data replication: strong consistency via Raft consensus protocol for metadata and chain replication for data. Multi-tenant: resource isolation and quota management per volume. Snapshot: create point-in-time copies of volumes. POSIX compatibility: applications can use standard file operations (open, read, write, stat) without modification. S3 compatibility: supports S3 API (PutObject, GetObject, ListObjects) with bucket management. Go. Apache-2.0.