Applications are currently unavailable for this posting.
Browse current jobs- Employment type
- Full-time
- Experience level
- Senior
- Posting language
- English
- Application method
- Apply on LinkedIn (LinkedIn sign-in required)
Job summary
Build and optimize scalable data pipelines and real-time streaming solutions using Python, PySpark, Spark SQL, and Azure Databricks. Work with Azure data services and complex SQL, and collaborate across teams to translate business requirements into technical solutions.
Job details
Position Name – Sr. Data Engineer Type of hiring – Fulltime Location – Remote Canada Looking for a Sr Data Engineer who has experience in AI & Claude, Python, PySpark, Azure Databricks, Spark SQL, Dataframes, Datasets, Real-time Data Streaming, Azure Data Services & Advanced SQL. Job Description: Role Overview 5-8 Years of hands-on experience in software development, with a strong emphasis on Python programming. Experience in AI (Artificial Intelligence) & Claude code is mandatory. Deep expertise in PySpark for building and optimizing scalable data pipelines. Proficient in designing and implementing real-time data streaming solutions using Spark SQL, Dataframes, and Datasets. Solid experience with Azure Databricks, including working with Delta tables, workflows, and scripting in Python. Familiar with Azure cloud services, especially Azure Data Lake, Azure Data Factory, and EventHub or Kafka for streaming data ingestion. Skilled in writing complex SQL queries and building views involving joins across multiple tables. Experience across the Software Development Lifecycle (SDLC), with a strong grasp of Agile Scrum methodologies. Proficient in tools like Jira (project management), Confluence (documentation), and GitHub (version control and CI/CD pipelines). Soft Skills & Collaboration: Strong collaboration mindset with enthusiasm for mobbing sessions and cross-team coordination. Excellent problem-solving and code review skills. Strong communication skills with the ability to understand business requirements and translate them into technical solutions. Proactive and responsive during business hours for any ad-hoc calls or meetings. Demonstrated ability to prioritize effectively, think creatively, and adapt quickly to new technologies and feedback.
What you’ll do
Build and optimize scalable data pipelines and real-time streaming solutions using Python, PySpark, Spark SQL, and Azure Databricks. Work with Azure data services and complex SQL, and collaborate across teams to translate business requirements into technical solutions.
Requirements
Requires 5–8 years of software development experience, strong Python skills, mandatory experience with AI and Claude Code, and deep expertise in PySpark and Azure Databricks. Candidates should also have experience with Azure data services, streaming ingestion, advanced SQL, the software development lifecycle, Agile Scrum, and collaboration and code review.
Listed skills
- Python · Preferred
- Jira · Preferred
- GitHub · Preferred
Other relevant skills
Identified from the job description. Confirm important requirements above.
- Python
- PySpark
- Azure Databricks
- Spark SQL
- Real-Time Data Streaming
- Azure Data Lake
- Azure Data Factory
- Azure Event Hubs
- Kafka
- Advanced SQL
- Delta Tables
- Claude Code
- Agile Scrum
- Jira
- Confluence
- GitHub
Job areas
- Data & Analytics
- Technology
- Software
- Engineering
More jobs from Yochana
Sr. Data Engineer
- Remote
- Canada
- Posted Nov 18, 2026
Senior Full Stack Developer (Angular & Java)
- Hybrid
- Montréal, QC
- Posted Nov 18, 2026
Principal Mobile Engineer
- Remote
- Canada, United States
- Posted Nov 18, 2026
Sr. Data Engineer with AI & Claude
- Remote
- Canada
- Posted Nov 17, 2026