LanceDB Vector Database

LanceDB is an open-source vector database built for artificial intelligence and machine learning applications created by Chang She and Lei Xu in 2022, headquartered in San Francisco, providing efficient storage, indexing, and retrieval of multi-billion-vector embeddings with sub-millisecond query latency, built on the Lance columnar data format designed specifically for machine learning workloads, supporting both local embedded storage for zero-setup development and distributed cloud storage for production scale, with the embedded mode requiring no server process, enabling vector search directly from any Python, JavaScript, or Rust application with minimal overhead. The vector indexing uses an inverted file index with product quantization for approximate nearest neighbor search, providing tunable trade-offs between recall and latency through configurable number of probes, sub-vectors, and bits per sub-vector, with the index supporting both flat search for exact results on smaller datasets and IVF-PQ for approximate search on billion-scale datasets, while the disk-based index enables searching datasets larger than available RAM through memory-mapped files. The multi-modal data support stores embeddings alongside their source data including images, text, audio, and video through the Lance format, enabling retrieval of not just vectors but also associated metadata, raw data, and model outputs in a single query, with the columnar format enabling efficient filtering and projection on metadata columns without loading the entire dataset into memory. The hybrid search combining vector similarity with full-text search and metadata filtering. The versioning through Lance format. The Python, JavaScript, and Rust SDKs. The integration with PyTorch, TensorFlow, and Hugging Face. The managed cloud offering. Designed by Chang She. Designed for machine learning engineers.