GCP Senior Data Engineer

Job summary

Lead a team of 10 engineers while building and optimizing data pipelines and managing GCP data infrastructure, including BigQuery, Cloud Storage, and Data Fusion. Establish engineering standards and monitoring, improve performance and cost efficiency, and collaborate on self-service data platforms, analytics, and machine learning use cases.

Job details

Role Name: GCP Senior Data Engineer Grade Work site: Canada (Remote Description and Location Required Skills Leadership: Experience leading data engineering teams onshore and offshore. This is a lead position, not a senior data engineer position. Candidates without leadership experience will not be considered for this role. Technical: Proficiency in SQL, Python, Java/Scala, and distributed systems (Spark, Beam). GCP Expertise: Hands-on with BigQuery, Dataflow, Dataproc, Pub/Sub, Composer (Airflow), and Terraform. DevOps: Familiarity with CI/CD, Git, and infrastructure-as-code. Nice to Have GCP Certifications (e.g., Professional Data Engineer). Experience with real-time processing, Vertex AI, or multi-cloud environments. Key Responsibilities Pipeline Development: Build and optimize ETL/ELT pipelines using Dataflow (Apache Beam), Dataproc (Spark), Pub/Sub, and Cloud Functions. Data Infrastructure: Architect and manage BigQuery, Cloud Storage, and Data Fusion for efficient data ingestion, transformation, and storage. Team Leadership: Lead a team of 10 engineers, review code, establish data engineering standards, CI/CD, and monitoring (e.g., Cloud Monitoring, Logging). Performance & Cost: Optimize pipeline performance, query efficiency, and GCP cost management (e.g., slot reservations, partitioning). Collaboration: Work with data architects, scientists, and analysts to deliver self-service data platforms and support advanced analytics/ML use cases. Data Quality & Governance: Implement data validation, lineage, and metadata management (e.g., Data Catalog, dbt).

What you’ll do

Lead a team of 10 engineers while building and optimizing data pipelines and managing GCP data infrastructure, including BigQuery, Cloud Storage, and Data Fusion. Establish engineering standards and monitoring, improve performance and cost efficiency, and collaborate on self-service data platforms, analytics, and machine learning use cases.

Requirements

This lead role requires experience leading onshore and offshore data engineering teams, along with proficiency in SQL, Python, Java or Scala, and distributed systems such as Spark and Beam. Candidates should have hands-on experience with GCP services including BigQuery, Dataflow, Dataproc, Pub/Sub, Composer, and Terraform, plus familiarity with CI/CD, Git, and infrastructure as code.

Listed skills

• SQL · Preferred
• CI/CD · Preferred
• Team Leadership · Preferred
• Java · Preferred
• Terraform · Preferred
• Python · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Team Leadership
• SQL
• Python
• Java
• Scala
• Distributed Systems
• Apache Spark
• Apache Beam
• BigQuery
• Dataflow
• Dataproc
• Pub/Sub
• Cloud Composer
• Terraform
• CI/CD
• Data Pipeline Development

Job areas

• Data & Analytics
• Technology
• Software
• Management & Leadership
• Consulting

More jobs from Epsilon Solutions Ltd.