Senior Software Engineer (Remote | $60–$120/hr)

Job summary

Create challenging software engineering tasks, reproducible environments, reference solutions, tests, and objective evaluation criteria for advanced AI models. Evaluate AI-generated code and technical submissions for correctness, robustness, completeness, and quality, then provide structured feedback to improve AI coding capabilities.

Job details

Software Engineer Position: Software Engineer (SWE) Type: Contractor Compensation: $60–$120 per hour equivalent (paid per accepted task) Location: Fully Remote (Worldwide) Time Commitment: 15+ hours per week About the Opportunity We are seeking experienced Software Engineers with prior professional AI-training or model-evaluation experience to support projects focused on improving advanced AI systems. In this role, you will apply your software engineering expertise to create challenging technical tasks, evaluate AI-generated code, identify implementation failures, and establish objective standards for correct solutions. This opportunity is intended for candidates who have already completed substantive AI-training, model-evaluation, or benchmarking work. Key Responsibilities Create realistic and technically challenging software engineering tasks to assess advanced AI models. Develop clear task instructions, reproducible environments, reference solutions, tests, and evaluation criteria. Evaluate AI-generated code and technical deliverables for correctness, robustness, and completeness. Identify issues involving implementation, debugging, dependencies, performance, testing, and reproducibility. Assess software design decisions, code quality, and the reliability of proposed solutions. Establish objective standards for evaluating technical submissions. Provide structured feedback to support the improvement of AI coding capabilities. Required Qualifications Prior paid professional experience training, evaluating, benchmarking, or improving AI systems. Strong practical proficiency in Python. Strong knowledge of software design, debugging, testing, version control, and dependency management. Ability to assess code quality, identify technical issues, and validate software solutions. Preferred Qualifications Experience creating AI-training tasks, reference solutions, tests, graders, or benchmark problems. Experience reviewing technical submissions from other AI trainers. Codeforces rating of approximately 1900 or higher, or strong results in another programming competition. Familiarity with Linux, Docker, CI/CD, command-line tools, and sandboxed environments. Experience with long-horizon or agentic coding evaluations. Availability to begin immediately. Compensation & Engagement Compensation: $60–$120 per hour equivalent, paid per accepted task. Engagement Type: Independent Contractor. Work Arrangement: Fully remote, worldwide. Time Commitment: 15+ hours per week. Schedule: Flexible. Application Process Submit Your Application: Provide your application details, previous AI-training experience, and public technical evidence. AI Interview: Complete an interview that includes technical and integrity checks. Hiring Manager Review: Your application, experience, and technical qualifications will be evaluated. Onboarding: If selected, complete project onboarding and begin your first assigned task.

What you’ll do

Create challenging software engineering tasks, reproducible environments, reference solutions, tests, and objective evaluation criteria for advanced AI models. Evaluate AI-generated code and technical submissions for correctness, robustness, completeness, and quality, then provide structured feedback to improve AI coding capabilities.

Requirements

Candidates must have prior paid professional experience training, evaluating, benchmarking, or improving AI systems, along with strong practical Python proficiency. They should be skilled in software design, debugging, testing, version control, dependency management, and validating code; experience creating AI-training tasks or benchmarks is preferred.

Listed skills

• Python · Preferred
• Linux · Preferred
• Docker · Preferred
• CI/CD · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Python
• Software Design
• Debugging
• Testing
• Version Control
• Dependency Management
• Code Evaluation
• AI Model Evaluation
• Benchmarking
• Task Design
• Reference Solutions
• Test Development
• Code Quality Assessment
• Linux
• Docker
• CI/CD

Job areas

• Software
• Technology
• Engineering
• Data & Analytics

More jobs from Synthires