Data Engineering Project - Full ETL Pipeline
-
Updated
Jun 11, 2024 - Python
Data Engineering Project - Full ETL Pipeline
With the help of this repository you can evaluate my professional level. Everything I know about Data Engineering is stored here.
A framework for writing Unstract Tools/Apps
Repo for all data enginnering projects
No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents
Stellar ETL will enable real-time analytics on the Stellar network
fluent iteration
An Apache Airflow ETL pipeline to process employee data by downloading, validating, transforming, and loading it into a PostgreSQL database.
Solução completa dedicada a realizar ETL de dados de cotações de moedas usando Python. Fonte dos dados: https://docs.awesomeapi.com.br/api-de-moedas
Jayvee is a domain-specific language and runtime for automated processing of data pipelines
Make stream processing easier! Easy-to-use streaming application development framework and operation platform.
Move your data with ease.
A CLI tool for transforming large RDF datasets using pure SPARQL.
Service for bulk-loading data to databases with automatic schema management (Redshift, Snowflake, BigQuery, ClickHouse, Postgres, MySQL)
DIRECT, the Data Integration Run-time Execution Control Tool, is a data logistics control framework that can be used to monitor, log, audit and control data integration / ETL processes.
Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage and metadata. Runs and scales everywhere python does.
Tutorial for Pydantic and SQLAlchemy
One ETL tool to rule them all
Crypto scraping project based on ETL process where data is getting scrap from an online website and performed transformation and load it into the SQL Server Database.
Add a description, image, and links to the etl-pipeline topic page so that developers can more easily learn about it.
To associate your repository with the etl-pipeline topic, visit your repo's landing page and select "manage topics."