Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
-
Updated
Jun 13, 2024 - Python
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
Flink CDC is a streaming data integration tool
An orchestration platform for the development, production, and observation of data assets.
Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ
Fancy stream processing made operationally mundane
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
🧙 Build, run, and manage data pipelines for integrating and transforming data.
Implementing best practices for PySpark ETL jobs and applications.
SQL stream processing, analytics, and management. We decouple storage and compute to offer instant failover, dynamic scaling, speedy bootstrapping, and efficient joins.
The open source high performance ELT framework powered by Apache Arrow
Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
Database Reporting Tool and Tasks (.Net)
Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
Extensible data integration Java framework for building XML and non-XML fragment-based applications
Hop Orchestration Platform
Privacy and Security focused Segment-alternative, in Golang and React
Addax is a versatile open-source ETL tool that can seamlessly transfer data between various RDBMS and NoSQL databases, making it an ideal solution for data migration.
React components to build CSV files on the fly basing on Array/literal object of data
Add a description, image, and links to the etl topic page so that developers can more easily learn about it.
To associate your repository with the etl topic, visit your repo's landing page and select "manage topics."