SeaweedFS

SeaweedFS is a fast distributed storage system designed for files and objects, providing S3-compatible API, distributed key-value store, and a distributed file system with POSIX-like semantics. Created by Chris Lu in 2014, SeaweedFS has over 23,300 stars as of 2026 and is optimized for storing and serving large numbers of files (billions) with fast access speed. The architecture uses a two-tier design: Volume Servers (storing actual file data as content-addressable volumes, each volume being a fixed-size append-only file with an index for O(1) lookup) and Master Servers (managing volume-to-server mapping, volume creation, and replication, using Raft consensus for HA). Key features include: S3 API compatibility (full AWS S3 REST API support including multipart upload, presigned URLs, bucket policies, lifecycle rules, versioning, and object locking), WebDAV support for mounting as a network drive, filer interface providing a hierarchical file system view on top of the flat object store (with metadata stored in LevelDB, RocksDB, Redis, MySQL, PostgreSQL, Cassandra, MongoDB, or etcd), FUSE mount for mounting as a local filesystem on Linux and macOS, async replication across data centers, erasure coding for storage efficiency (Reed-Solomon erasure coding configurable k+m parameters), tiered storage (hot data on SSD, cold data on HDD or cloud), cloud storage backend support (S3, GCS, Azure Blob, Backblaze B2, Alibaba OSS for tiering), Kubernetes CSI driver for persistent volumes, data encryption at rest (AES-256-GCM), data compression (zstd, LZ4), garbage collection, and volume rebalancing. Written in Go, deployed as a single binary.