itjobs.ca Logo
iLink Digital logo

Sr Data Engineer -

iLink Digital17 days ago
Toronto, Ontario, Canada
Senior Level
Full-Time

About the role

Job Summary We are seeking a highly skilled Databricks Engineer with AI/ML experience to design, build, and optimize scalable data and machine learning platforms on Databricks. The role involves end-to-end ownership of data pipelines, ML workflows, and production AI systems. Key Responsibilities Design and implement scalable ETL pipelines using Databricks & Spark Build Lakehouse architecture using Delta Lake Develop and deploy ML models using MLflow Implement MLOps pipelines for training, testing, and serving models Optimize cluster performance and reduce compute cost Build RAG and LLM-based solutions using Mosaic AI Integrate analytics with BI tools (Power BI, Tableau) Implement data governance using Unity Catalog Collaborate with Data Scientists and Business teams Ensure data quality, security, and compliance Required Skills Mandatory 5+ years of Databricks & Apache Spark Strong Python & PySpark Experience with Delta Lake & Lakehouse MLflow & MLOps experience Cloud platform (AWS/Azure/GCP) Git & CI/CD Preferred Experience with LLMs & Generative AI RAG pipelines & Vector Databases Deep Learning frameworks Databricks certifications Power BI integration

Requirements Job Summary We are seeking a highly skilled Databricks Engineer with AI/ML experience to design, build, and optimize scalable data and machine learning platforms on Databricks. The role involves end-to-end ownership of data pipelines, ML workflows, and production AI systems. Key Responsibilities Design and implement scalable ETL pipelines using Databricks & Spark Build Lakehouse architecture using Delta Lake Develop and deploy ML models using MLflow Implement MLOps pipelines for training, testing, and serving models Optimize cluster performance and reduce compute cost Build RAG and LLM-based solutions using Mosaic AI Integrate analytics with BI tools (Power BI, Tableau) Implement data governance using Unity Catalog Collaborate with Data Scientists and Business teams Ensure data quality, security, and compliance Required Skills Mandatory 5+ years of Databricks & Apache Spark Strong Python & PySpark Experience with Delta Lake & Lakehouse MLflow & MLOps experience Cloud platform (AWS/Azure/GCP) Git & CI/CD Preferred Experience with LLMs & Generative AI RAG pipelines & Vector Databases Deep Learning frameworks Databricks certifications Power BI integration

About iLink Digital

IT Services and IT Consulting