Azure Data engineer

Job summary

Develop and maintain data pipelines using Azure Data Factory and Snowflake to support enterprise data warehousing. Implement change data capture and ETL/ELT processes using Databricks and PySpark.

Job details

Role: Azure Data Engineer Skillset: Snowflake, ADF, Python & SQL are must and Databricks is a good to have. Skills required - Python Modules: • Standard Libraries (required) • Azure SDK (preferred) • Keyring (preferred) • Pandas (preferred) • Snowflake Connector (preferred) ADF Components: • Flow Control Logic • Lookups • Copy Activities • Child pipeline execution • Script calls • Batch service calls Dataflows • EMS , Event subscription, Event hub, Function Apps etc Snowflake Objects: • COPY INTO Scripts/Commands • Tasks • Dynamic Tables Databricks: • Notebooks to perform change data capture • Familiarity with Spark/PySpark ETL & Development Basics - • CI/CD in Azure DevOps • Knowledge of EDW , ETL, ELT processes, Data warehousing • Advanced SQL knowledge • Code deployment process • Troubleshooting pipelines • ADO • Agile methodologies

What you’ll do

Develop and maintain data pipelines using Azure Data Factory and Snowflake to support enterprise data warehousing. Implement change data capture and ETL/ELT processes using Databricks and PySpark.

Requirements

Must be proficient in Python, SQL, Snowflake, and Azure Data Factory. Experience with Azure DevOps, CI/CD, and Agile methodologies is required.

Listed skills

• SQL · Preferred
• CI/CD · Preferred
• Python · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Snowflake
• Azure Data Factory
• Python
• SQL
• Databricks
• PySpark
• Azure DevOps
• ETL
• ELT
• Data Warehousing
• CI/CD
• Agile Methodologies
• Azure SDK
• Pandas
• Event Hub
• Function Apps

Job areas

• Data & Analytics
• Technology
• Software
• Engineering
• Consulting