Serf Decentralized Cluster Membership

Serf is a free and open-source decentralized solution for cluster membership, failure detection, and orchestration that is lightweight, highly available, and fault tolerant. Developed by HashiCorp (headquartered in San Francisco, California, founded by Mitchell Hashimoto and Armon Dadgar in 2012) and first released in 2013, Serf is built on top of the SWIM (Scalable Weakly-consistent Infection-style Process Group Membership) protocol and the gossip protocol. Key features: membership management: Serf maintains a list of all nodes in a cluster, automatically detecting when nodes join or leave. Each Serf agent runs on every node and communicates via UDP-based gossip protocol. Failure detection: Serf uses a sophisticated gossip protocol (based on SWIM with Lifeguard enhancements) to detect node failures. When a node becomes unreachable, Serf detects the failure and propagates this information throughout the cluster within seconds. Custom events: broadcast custom events (deploy notifications, configuration changes) to all members. Events are delivered via gossip with best-effort delivery. Tags: each node can have key-value tags (metadata) such as role=web-server, datacenter=us-east-1, version=1.2.3. Tags are propagated through gossip and can be queried. Event handlers: Serf can execute user-defined scripts when events occur (member-join, member-leave, member-failed, user events), enabling automated cluster orchestration such as reconfiguring load balancers. LAN and WAN modes: supports both LAN (optimized for low latency) and WAN (optimized for high latency and packet loss) gossip configurations. Key-value store for sharing cluster-wide configuration data. Cross-platform (Go): Linux, macOS, Windows, BSD. Used by Consul (which embeds Serf for gossip protocol) and Nomad. MPL-2.0.