SeaweedFS

SeaweedFS is an open-source distributed file system designed to store and serve a large number of files efficiently. Created by Chris Lu in 2014, SeaweedFS has over 23,000 stars as of 2026, making it one of the most popular self-hosted storage solutions. Unlike traditional distributed file systems that manage file metadata and data together, SeaweedFS uses a unique architecture inspired by Facebook's Haystack design: a master server manages file metadata (volume locations and file IDs) while volume servers store the actual file data. Key features include: key-value file storage (each file is assigned a unique file ID consisting of a volume ID and a file key, enabling O(1) lookups), master server cluster (multiple master servers using Raft consensus for high availability and automatic failover), volume server management (automatic volume rebalancing, replication, and tiered storage across SSD and HDD), replication (configurable replication factors with rack-aware and data-center-aware placement), S3-compatible API (Seaweed S3 Gateway providing full S3 compatibility for integration with existing applications and tools), FUSE mount (mounting SeaweedFS as a local file system via Weed FUSE), HDFS compatibility (providing an HDFS-compatible interface for big data frameworks), WebDAV support (mounting as a network drive on macOS and Windows), distributed queue (Message Queue mode for event streaming), TTL support (automatic file expiration), file deduplication, encryption at rest, distributed caching via CDN proxies, and cloud tiering (transparently offloading cold data to S3, GCS, or Azure Blob).