Data Engineer & Data Designer

Job summary

Design, develop, and maintain scalable data pipelines and workflows using Databricks, Python, and SQL, integrating data from Azure and Oracle databases. Ensure data quality and pipeline performance, troubleshoot processing issues, and collaborate with stakeholders while following security, governance, and documentation practices.

Job details

Job Title: Data Engineer Location: Toronto, ON Job type: onsite Job description : Key Responsibilities: • Design, develop, and maintain scalable data pipelines for data ingestion, transformation, and integration. • Work extensively with Databricks for data processing, transformation, and analytics. • Develop and maintain data workflows using Python and SQL. • Integrate data from multiple sources, including Azure databases and Oracle databases. • Write optimized queries and implement efficient data transformation logic. • Perform data cleansing, validation, and quality checks to ensure accuracy and consistency. • Troubleshoot data pipeline failures and resolve performance and data processing issues. • Collaborate with business analysts, data architects, and other engineering teams to understand data requirements. • Follow best practices for data security, governance, performance optimization, and documentation. • Support data solutions involving AI or machine learning where applicable. Required Skills and Qualifications: • 5–7 years of experience in data engineering or data development. • Strong hands-on experience with Databricks. • Experience working with Azure database technologies. • Strong experience with Oracle Database and SQL. • Proficiency in Python programming for data processing and automation. • Experience with ETL/ELT processes, data integration, and data transformation. • Strong analytical, troubleshooting, and problem-solving skills. • Good communication and collaboration skills. Preferred Skills: • Exposure to AI, machine learning, or AI-enabled data solutions. • Experience with Spark, PySpark, Delta Lake, or Azure data services. • Experience optimizing data pipelines and processing large datasets

What you’ll do

Design, develop, and maintain scalable data pipelines and workflows using Databricks, Python, and SQL, integrating data from Azure and Oracle databases. Ensure data quality and pipeline performance, troubleshoot processing issues, and collaborate with stakeholders while following security, governance, and documentation practices.

Requirements

Requires 5–7 years of data engineering or data development experience, strong hands-on Databricks experience, and proficiency in Python, SQL, Azure database technologies, and Oracle Database. Experience with ETL/ELT, data integration and transformation is required; exposure to AI or machine learning, Spark, PySpark, Delta Lake, and Azure data services is preferred.

Listed skills

• SQL · Preferred
• Machine learning · Preferred
• Python · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Databricks
• Python
• SQL
• Azure Database Technologies
• Oracle Database
• Data Engineering
• Data Pipelines
• ETL/ELT
• Data Integration
• Data Transformation
• Data Quality
• Spark
• PySpark
• Delta Lake
• Data Governance
• Machine Learning

Job areas

• Data & Analytics
• Technology
• Software
• Engineering

More jobs from Lorven Technologies Inc.