Trouvez votre prochaine possibilité
J&M Group
Résumé du poste
Develop and manage data pipelines, jobs, and automated tasks using various cloud platforms. Focus on dataflow management, Delta Lake indexing, and the movement of data within a Medallion Architecture.
Détails du poste
Required Skills Experience developing ETL processes with Informatica, Azure Data Factory and Databricks (Python, Lakeflow, and SQL Optimization) Experience with solution development with Databricks in Medallion Architecture. Experience with Cloud data engineer must be familiar and experienced with different programming languages (Python, SQL, t-SQL, PL/SQL, Informatica, ADF, SSIS. Microsoft Fabric) and be able to integrate with many different platforms to create data pipelines, jobs, automate tasks, and write scripts. Experience with Data pipeline and workflow development, orchestration, deployment, and automation and specializing in programming and pipelines to create and managing the dataflow, Delta Lake indexing, parallelism and movement of data.
Ce que vous ferez
Develop and manage data pipelines, jobs, and automated tasks using various cloud platforms. Focus on dataflow management, Delta Lake indexing, and the movement of data within a Medallion Architecture.
Exigences
Requires extensive experience with ETL tools like Informatica and Azure Data Factory, alongside proficiency in Python and SQL. Candidates must be skilled in Databricks and the implementation of cloud data engineering workflows.
Compétences indiquées
- SQL · Souhaitée
- Python · Souhaitée
Autres compétences pertinentes
Relevées dans la description du poste. Confirmez les exigences importantes ci-dessus.
- Informatica
- Azure Data Factory
- Databricks
- Python
- SQL
- T-SQL
- PL/SQL
- SSIS
- Microsoft Fabric
- Medallion Architecture
- ETL
- Delta Lake
- Data Pipelines
- Workflow Orchestration
- Lakeflow
- SQL Optimization
Domaines d’emploi
- Data & Analytics
- Technology
- Software
- Engineering
- Consulting