Databricks Data Engineer

Job summary

Design, build, and optimize enterprise-scale data pipelines and AI-ready data platforms using Databricks and AWS, including Delta Lake and Medallion Architecture. Lead end-to-end delivery from requirements gathering through production deployment, with responsibility for governance, performance, CI/CD, data quality, and monitoring.

Job details

Role - Databricks Data Engineer Location: Ottawa, Canada Employment Type: Contract Experience: 8+ Years Company: Celebal Technologies Role Overview We are seeking a hands-on Lead Databricks Data Engineer to design, develop, and optimize enterprise-scale data pipelines and AI-ready data platforms using Databricks and AWS. Key Responsibilities Design and develop scalable ETL/ELT pipelines using Databricks, PySpark, Python, and Advanced SQL. Implement Delta Lake and Medallion Architecture (Bronze/Silver/Gold). Manage data governance, lineage, and access controls using Unity Catalog. Build reusable data models, dimensional models, and AI-ready data products. Optimize Spark workloads, clusters, and pipeline performance. Implement Git, CI/CD, DataOps, data quality checks, and production monitoring. Own end-to-end data engineering delivery, from requirements gathering to production deployment. Required Skills 8+ years of hands-on enterprise data engineering experience. Strong production Databricks experience, including PySpark, Delta Lake, and Unity Catalog. Expertise in Advanced SQL, Python, Spark, and dimensional data modeling. Experience with AWS or another major cloud platform. Strong understanding of data governance, performance tuning, and CI/CD. Ability to independently design and deliver scalable data engineering solutions. Preferred Skills Databricks Genie / AI/BI, semantic layers, and AI-ready data products. Manufacturing, Supply Chain, ERP/MES, or Tableau experience. Cloud migration, ML/MLOps, and data platform modernization.

What you’ll do

Design, build, and optimize enterprise-scale data pipelines and AI-ready data platforms using Databricks and AWS, including Delta Lake and Medallion Architecture. Lead end-to-end delivery from requirements gathering through production deployment, with responsibility for governance, performance, CI/CD, data quality, and monitoring.

Requirements

Requires 8+ years of enterprise data engineering experience and strong production expertise with Databricks, PySpark, Delta Lake, Unity Catalog, Python, Spark, and advanced SQL. Candidates should also have cloud-platform experience and be able to independently design scalable solutions; experience with AI/BI, manufacturing or supply chain, Tableau, cloud migration, or MLOps is preferred.

Listed skills

• CI/CD · Preferred
• Amazon Web Services · Preferred
• Python · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Databricks
• PySpark
• Python
• Advanced SQL
• Apache Spark
• Delta Lake
• Unity Catalog
• AWS
• ETL/ELT
• Medallion Architecture
• Dimensional Data Modeling
• Data Governance
• CI/CD
• DataOps
• Performance Tuning
• Data Quality

Job areas

• Data & Analytics
• Technology
• Software
• Engineering