Senior Software Developer, LLM Infrastructure

Job summary

Build and maintain scalable infrastructure to host and serve open-source LLMs while implementing intelligent model-routing frameworks. Develop evaluation and benchmarking tooling to validate model performance and drive cost-optimization initiatives.

Job details

Build and maintain scalable infrastructure to host and serve open-source LLMs while implementing intelligent model-routing frameworks. Develop evaluation and benchmarking tooling to validate model performance and drive cost-optimization initiatives. Requires 6+ years of software engineering experience in building production-ready distributed systems or data platform infrastructure. Strong proficiency in Python, Kubernetes, and Terraform is essential, along with practical experience in the modern AI ecosystem.

What you’ll do

Build and maintain scalable infrastructure to host and serve open-source LLMs while implementing intelligent model-routing frameworks. Develop evaluation and benchmarking tooling to validate model performance and drive cost-optimization initiatives.

Requirements

Requires 6+ years of software engineering experience in building production-ready distributed systems or data platform infrastructure. Strong proficiency in Python, Kubernetes, and Terraform is essential, along with practical experience in the modern AI ecosystem.

Benefits

• Health benefits
• Life insurance
• Long-term group savings with employer match
• 20 vacation days
• 4 wellness days
• Unlimited sick and mental health days
• 90 days away work program
• Equity compensation

Listed skills

• Python · Preferred
• Kubernetes · Preferred
• Terraform · Preferred
• Amazon Web Services · Preferred
• PostgreSQL · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Python
• Kubernetes
• Terraform
• Distributed systems
• Data platform infrastructure
• LLM infrastructure
• Model routing
• Evaluation frameworks
• GPU infrastructure
• vLLM
• Ray
• FastAPI
• AWS
• Kafka
• Postgres
• Snowflake

Job areas

• Software
• Engineering
• Data & Analytics
• Technology
• Finance & Accounting

More jobs from Wealthsimple