Software Engineer (Remote | $60–$120/hr)

Job summary

Create challenging software engineering tasks, reproducible environments, reference solutions, tests, and objective evaluation criteria for advanced AI models. Review AI-generated code and technical submissions for correctness, robustness, completeness, and quality, and provide structured feedback to improve AI coding capabilities.

Job details

Software Engineer Position: Software Engineer (SWE) Type: Contractor Compensation: $60–$120 per hour equivalent (paid per accepted task) Location: Fully Remote (Worldwide) Time Commitment: 15+ hours per week About the Opportunity We are seeking experienced Software Engineers with prior professional AI-training or model-evaluation experience to support projects focused on improving advanced AI systems. In this role, you will apply your software engineering expertise to create challenging technical tasks, evaluate AI-generated code, identify implementation failures, and establish objective standards for correct solutions. This opportunity is intended for candidates who have already completed substantive AI-training, model-evaluation, or benchmarking work. Key Responsibilities Create realistic and technically challenging software engineering tasks to assess advanced AI models. Develop clear task instructions, reproducible environments, reference solutions, tests, and evaluation criteria. Evaluate AI-generated code and technical deliverables for correctness, robustness, and completeness. Identify issues involving implementation, debugging, dependencies, performance, testing, and reproducibility. Assess software design decisions, code quality, and the reliability of proposed solutions. Establish objective standards for evaluating technical submissions. Provide structured feedback to support the improvement of AI coding capabilities. Required Qualifications Prior paid professional experience training, evaluating, benchmarking, or improving AI systems. Strong practical proficiency in Python. Strong knowledge of software design, debugging, testing, version control, and dependency management. Ability to assess code quality, identify technical issues, and validate software solutions. Preferred Qualifications Experience creating AI-training tasks, reference solutions, tests, graders, or benchmark problems. Experience reviewing technical submissions from other AI trainers. Codeforces rating of approximately 1900 or higher, or strong results in another programming competition. Familiarity with Linux, Docker, CI/CD, command-line tools, and sandboxed environments. Experience with long-horizon or agentic coding evaluations. Availability to begin immediately. Compensation & Engagement Compensation: $60–$120 per hour equivalent, paid per accepted task. Engagement Type: Independent Contractor. Work Arrangement: Fully remote, worldwide. Time Commitment: 15+ hours per week. Schedule: Flexible. Application Process Submit Your Application: Provide your application details, previous AI-training experience, and public technical evidence. AI Interview: Complete an interview that includes technical and integrity checks. Hiring Manager Review: Your application, experience, and technical qualifications will be evaluated. Onboarding: If selected, complete project onboarding and begin your first assigned task.

What you’ll do

Create challenging software engineering tasks, reproducible environments, reference solutions, tests, and objective evaluation criteria for advanced AI models. Review AI-generated code and technical submissions for correctness, robustness, completeness, and quality, and provide structured feedback to improve AI coding capabilities.

Requirements

Candidates must have prior paid professional experience training, evaluating, benchmarking, or improving AI systems, along with strong Python proficiency and practical software engineering skills. Experience creating AI-training tasks, graders, or benchmarks, competitive programming results, and familiarity with Linux, Docker, CI/CD, and agentic coding evaluations are preferred.

Listed skills

• Python · Preferred
• Linux · Preferred
• Docker · Preferred
• CI/CD · Preferred

Other relevant skills

Identified from the job description. Confirm important requirements above.

• Python
• Software Design
• Debugging
• Testing
• Version Control
• Dependency Management
• Code Evaluation
• Benchmarking
• AI Model Evaluation
• Task Design
• Reference Solutions
• Performance Analysis
• Linux
• Docker
• CI/CD
• Command-Line Tools

Job areas

• Software
• Technology
• Engineering
• Data & Analytics

More jobs from Synthires