Singer Data Extraction Framework

Singer is an open source framework for writing extractors and loaders that move data between sources and destinations in the Extract-Load-Transform (ELT) data pipeline pattern. Originally developed at Stitch (now part of Talend) in 2017, the project provides a standardized specification for how data extraction taps and loading targets communicate through JSON-based message streams. The Singer specification defines two types of components: Taps that extract data from source systems such as APIs, databases, SaaS applications, and files, and Targets that load extracted data into destination systems such as data warehouses, databases, and cloud storage. Communication between taps and targets follows a simple Unix pipe pattern: a tap outputs JSON records and schema information to stdout, which is piped into a target that consumes the stream and writes data to the destination. The Singer community has built hundreds of open source taps and targets covering popular data sources including Google Analytics, Salesforce, Shopify, Stripe, HubSpot, Slack, MySQL, PostgreSQL, MongoDB, and Elasticsearch, and destinations including Snowflake, BigQuery, Redshift, Amazon S3, and PostgreSQL. This growing catalog enables users to mix and match components to build custom data pipelines without writing custom code.