Opens LinkedIn
- Employment type
- Contract
- Experience level
- Senior · 5+ years
- Posting language
- English
- Working hours
- 40 hours per week
- Seniority
- Mid-Senior level
- Application method
- Apply on LinkedIn (LinkedIn sign-in required)
Job summary
Develop scalable data ingestion, transformation, and orchestration pipelines using Azure Databricks, Apache Airflow, and related Azure services. Build reusable engineering components and notebooks, implement automated data quality checks, and optimize and troubleshoot data workloads.
Job details
Strong hands-on experience with Azure Data Factory (ADF), Azure Databricks, Python, PySpark, SQL, and Azure cloud services. Expertise in developing scalable data ingestion, transformation, and orchestration pipelines using Databricks and Apache Airflow. Experience implementing Medallion Architecture (Bronze, Silver, Gold), Delta Lake, and reusable data engineering frameworks. Develop reusable Databricks notebooks, data transformation components, orchestration workflows, and automated data quality checks. Strong experience with Azure DevOps, Git, CI/CD pipelines, and automated deployment processes. Experience with Spark performance tuning, pipeline troubleshooting, data reconciliation, and workload optimization. Knowledge of data governance, metadata management, Unity Catalog, and enterprise data engineering best practices.
What you’ll do
Develop scalable data ingestion, transformation, and orchestration pipelines using Azure Databricks, Apache Airflow, and related Azure services. Build reusable engineering components and notebooks, implement automated data quality checks, and optimize and troubleshoot data workloads.
Requirements
Requires strong hands-on experience with Azure Data Factory, Azure Databricks, Python, PySpark, SQL, and Azure cloud services. Candidates should also have experience with Medallion Architecture, Delta Lake, Azure DevOps, Git, CI/CD, Spark optimization, data governance, metadata management, and Unity Catalog.
Listed skills
- SQL · Preferred
- CI/CD · Preferred
- Git · Preferred
- Python · Preferred
Other relevant skills
Identified from the job description. Confirm important requirements above.
- Azure Data Factory
- Azure Databricks
- Python
- PySpark
- SQL
- Azure Cloud Services
- Apache Airflow
- Medallion Architecture
- Delta Lake
- Data Engineering Frameworks
- Data Quality
- Azure DevOps
- Git
- CI/CD
- Spark Performance Tuning
- Unity Catalog
Job areas
- Data & Analytics
- Technology
- Software
More jobs from Viva Tech Solutions
Java Full Stack Engineer - Artificial Intelligence
- On-site
- Toronto, ON
- Posted Nov 3, 2026
Guidewire Tester
- On-site
- Markham, ON
- Posted Oct 16, 2026