Back to job search
Yochana logo

Sr. Data Engineer with AI & Claude

  • Canada
  • Remote
  • Posted Nov 17, 2026
  • 1 position

Applications are currently unavailable for this posting.

Browse current jobs
Employment type
Full-time
Experience level
Senior
Posting language
English
Application method
Apply on LinkedIn (LinkedIn sign-in required)

Job summary

Build and optimize scalable data pipelines using Python, PySpark, and Azure Databricks, and design real-time streaming solutions with Spark SQL, DataFrames, and Datasets. Work with Azure data services and streaming ingestion tools, develop complex SQL queries and views, and collaborate across teams throughout the software development lifecycle.

Job details

Position Name – Sr. Data Engineer with AI & Claude Type of hiring – Fulltime Location – Remote Canada Looking for a Sr Data Engineer who has experience in AI & Claude, Python, PySpark, Azure Databricks, Spark SQL, Dataframes, Datasets, Real-time Data Streaming, Azure Data Services & Advanced SQL. Exp - > 8 Years Job Description: Role Overview 5-8 Years of hands-on experience in software development, with a strong emphasis on Python programming. Experience in AI (Artificial Intelligence) & Claude code is mandatory. Deep expertise in PySpark for building and optimizing scalable data pipelines. Proficient in designing and implementing real-time data streaming solutions using Spark SQL, Dataframes, and Datasets. Solid experience with Azure Databricks, including working with Delta tables, workflows, and scripting in Python. Familiar with Azure cloud services, especially Azure Data Lake, Azure Data Factory, and EventHub or Kafka for streaming data ingestion. Skilled in writing complex SQL queries and building views involving joins across multiple tables. Experience across the Software Development Lifecycle (SDLC), with a strong grasp of Agile Scrum methodologies. Proficient in tools like Jira (project management), Confluence (documentation), and GitHub (version control and CI/CD pipelines). Soft Skills & Collaboration: Strong collaboration mindset with enthusiasm for mobbing sessions and cross-team coordination. Excellent problem-solving and code review skills. Strong communication skills with the ability to understand business requirements and translate them into technical solutions. Proactive and responsive during business hours for any ad-hoc calls or meetings. Demonstrated ability to prioritize effectively, think creatively, and adapt quickly to new technologies and feedback.

What you’ll do

Build and optimize scalable data pipelines using Python, PySpark, and Azure Databricks, and design real-time streaming solutions with Spark SQL, DataFrames, and Datasets. Work with Azure data services and streaming ingestion tools, develop complex SQL queries and views, and collaborate across teams throughout the software development lifecycle.

Requirements

The role requires more than eight years of experience according to the posting, with strong Python and PySpark expertise and mandatory experience with artificial intelligence and Claude Code. Candidates should also have experience with Azure Databricks and Azure data services, advanced SQL, real-time streaming, Agile Scrum, and collaboration and communication tools such as Jira, Confluence, and GitHub.

Listed skills

  • Python · Preferred
  • GitHub · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

  • Python
  • Artificial Intelligence
  • Claude Code
  • PySpark
  • Azure Databricks
  • Spark SQL
  • Real-Time Data Streaming
  • Azure Data Lake
  • Azure Data Factory
  • Event Hubs
  • Kafka
  • Advanced SQL
  • Delta Tables
  • Agile Scrum
  • GitHub
  • Data Pipeline Optimization

Job areas

  • Data & Analytics
  • Technology
  • Software
  • Engineering

More jobs from Yochana

See all jobs from Yochana