Data Engineer (Databricks / Apache Spark)

Original job posting

Design, develop, maintain, and optimize data processing solutions and Apache Spark jobs using Databricks. Support data pipelines and processing frameworks, troubleshoot workload issues, and contribute to delivery, deployment, and issue resolution.

Job details

Data Engineer (Databricks / Apache Spark) Contract Term: 7 months, renewal Work Location: Toronto, ON. hybrid Service Description: Provide data engineering services to design, develop, and optimize data processing solutions using Databricks and Apache Spark. Support the development of scalable data transformation and processing capabilities to meet project requirements. Role Description Design, develop, and maintain data processing solutions using Databricks. Develop and optimize Apache Spark jobs for data transformation and processing. Collaborate with project team members to support data engineering requirements and delivery activities. Troubleshoot and resolve issues related to Databricks and Spark workloads. Support the implementation and enhancement of data pipelines and data processing frameworks. Deliverables Databricks notebooks and data processing solutions. Apache Spark data transformation jobs. Data ingestion and transformation workflows. Optimized and reusable data engineering code. Unit-tested data processing components. Technical documentation of implemented data engineering solutions. Support for deployment and issue resolution activities. Required Qualifications / Experience Strong experience with Databricks Strong experience with Apache Spark

What you’ll do

Design, develop, maintain, and optimize data processing solutions and Apache Spark jobs using Databricks. Support data pipelines and processing frameworks, troubleshoot workload issues, and contribute to delivery, deployment, and issue resolution.

Requirements

The role requires strong experience with Databricks and Apache Spark. It also involves developing scalable data transformation and processing capabilities and collaborating with project team members to deliver data engineering solutions.

Listed skills

• Technical Documentation · Preferred
• Troubleshooting · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Databricks
• Apache Spark
• Data Engineering
• Data Processing
• Data Transformation
• Data Pipelines
• Data Ingestion
• Troubleshooting
• Unit Testing
• Technical Documentation
• Deployment Support

Job areas

• Data & Analytics
• Technology
• Software