Data Engineer

Job summary

Design, build, test, and maintain end-to-end data pipelines and platforms, including ingestion, transformation, warehousing, and validation. Build AWS infrastructure with Terraform, develop transformations using dbt and SQL, automate and validate pipelines with Python, and monitor and improve platforms after launch.

Job details

Data Engineer (Contract, LLM Exposure a Plus) Overview We're looking for a Data Engineer to join us on a 6+ month contract. You'll design, build, and own data pipelines and platforms for our clients, from ingestion and transformation through to warehousing and validation. Some projects also bring LLMs into the mix, so you'll get hands-on exposure to applying them alongside solid data engineering. The work spans a range of client needs, so expect to build new systems from scratch as well as improve existing ones. Responsibilities - Own end-to-end development, testing, and validation frameworks for the platforms you build. - Build scalable products from scratch, anticipating how users will break them and designing for future issues, not just today's cases. - Build infrastructure as code with Terraform on AWS services such as Redshift and S3. - Develop data transformations using dbt and SQL. - Write Python scripts for pipeline automation and validation. - Validate outputs with test cases and AI-agent-assisted testing. - Monitor platforms and deliver new development after launch. Technical Requirements - Solid foundation in data engineering, including designing and building end-to-end data pipelines. - Strong understanding of data processing (batch and incremental) and data storage (warehouses, data lakes, data modeling). - Strong experience building data pipelines with Python and SQL. - Hands-on AWS experience, especially Redshift and S3. - Terraform for infrastructure as code. - dbt for data transformation. - Experience writing automated test cases and validation frameworks. - Familiarity with LLMs, including when to use deterministic versus non-deterministic (LLM) approaches. Why Join Us We move fast and give people real ownership. You'll work in a high-ownership environment where your decisions shape the product from day one, with a small team that values initiative, clear thinking, and getting things shipped. If you enjoy building from scratch, solving hard problems end to end, and seeing your work used by real clients, we'd love to hear from you.

What you’ll do

Design, build, test, and maintain end-to-end data pipelines and platforms, including ingestion, transformation, warehousing, and validation. Build AWS infrastructure with Terraform, develop transformations using dbt and SQL, automate and validate pipelines with Python, and monitor and improve platforms after launch.

Requirements

Requires a solid data engineering foundation, experience designing end-to-end pipelines, and strong Python and SQL skills. Candidates should have hands-on experience with AWS Redshift and S3, Terraform, dbt, automated testing and validation, and familiarity with LLMs and when to use deterministic versus non-deterministic approaches.

Listed skills

• SQL · Preferred
• Automated testing · Preferred
• Data Validation · Preferred
• Amazon Web Services · Preferred
• Terraform · Preferred
• Python · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Data Engineering
• Data Pipeline Development
• Python
• SQL
• AWS
• Amazon Redshift
• Amazon S3
• Terraform
• dbt
• Data Transformation
• Data Warehousing
• Data Modeling
• Automated Testing
• Data Validation
• LLMs
• Infrastructure as Code

Job areas

• Data & Analytics
• Technology
• Software
• Engineering