hiringremote.Get free job alerts

Software Engineer – AI Code Evaluation & Benchmarking (US candidates only)

See listing · Turing · remote

Open to people living in the United States.

Check history

CheckedStatusSlots leftChangeHow
15 Nov 2026Open––Script, platform’s board
14 Nov 2026Open––Script, platform’s board
13 Nov 2026Open––Script, platform’s board
12 Nov 2026Open––Script, platform’s board
11 Nov 2026Open––Script, platform’s board
10 Nov 2026Open––Script, platform’s board
9 Nov 2026Open––Script, platform’s board
8 Nov 2026Open––Script, platform’s board
7 Nov 2026Open––Script, platform’s board
6 Nov 2026Open––Script, platform’s board

The script reads the platform’s own job board every night. A job not confirmed for 10 days comes off the list.

An older role: first posted 156d ago, and Turing still listed it when it was last checked on 15 Nov 2026. Newer roles tend to fill faster. See the jobs hiring now.

Most people who don't get in fail the screening, not the CV. Five minutes here first: How to pass the Turing assessment and interviews.

What the work is

Role Overview

We are looking for experienced Software Engineers to help evaluate, benchmark, and improve the coding capabilities of frontier AI models. In this role, you will assess AI-generated code, validate solutions against real-world software engineering tasks, identify correctness and quality issues, and contribute to the development of high-quality evaluation datasets and benchmarks.

This position is ideal for engineers who enjoy code review, debugging, problem-solving, and applying strong software engineering judgment to complex technical scenarios. Your work will directly contribute to measuring and improving the performance of advanced AI coding systems.

What Does Day-to-Day Look Like?

  • Review and evaluate AI-generated code for correctness, efficiency, maintainability, and adherence to requirements.
  • Analyze software engineering tasks and validate whether proposed solutions meet expected outcomes.
  • Debug code, reproduce issues, and verify fixes across different programming environments.
  • Assess model-generated explanations, reasoning, and implementation approaches for technical accuracy.
  • Create, refine, and maintain evaluation datasets, benchmarks, and grading rubrics for coding tasks.
  • Identify edge cases, failure modes, and areas where AI systems struggle with software engineering problems.
  • Document findings clearly and provide structured feedback to improve evaluation quality and consistency.
  • Collaborate with project teams to establish quality standards and evaluation methodologies.

Who gets hired

  • Bachelor's or Master's degree in Computer Science, Software Engineering, or a related technical field.
  • 3+ years of professional software engineering experience.
  • Strong proficiency in one or more of the following languages: Python, Java, C/C++, Go, Swift, Objective-C, PHP, or SQL.
  • Strong understanding of data structures, algorithms, software design principles, and debugging methodologies.
  • Experience performing code reviews and evaluating code quality in production or large-scale codebases.
  • Ability to analyze complex technical problems and assess solution correctness with minimal supervision.
  • Familiarity with version control systems (e.g., Git) and modern software development workflows.
  • Strong written communication skills and attention to detail.
  • Experience with AI/ML data annotation, NLP, prompt engineering, model evaluation, or LLM-related projects is a plus.
  • Experience evaluating AI-generated code, benchmark creation, or software quality assessment is highly preferred.

Perks of Freelancing With Turing

  • Work in a fully remote environment.
  • Opportunity to work on cutting-edge AI projects with leading LLM companies.

Offer Details

  • Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST.
  • Engagement type : Contractor assignment (no medical/paid leave)
  • Duration of contract : 1 month; [expected start date is next week]
  • Location: US only

Evaluation Process

  • Online automated coding challenge for Python and Docker test (RHLF)

The platform posted this on 12 Jun 2026.

Similar jobs hiring now

Open the full list
  1. Posted 14 NovCMC, Manufacturing & Quality Expert$70–100/hrAI expert network10 slots leftUSUSChecked auto, 15 Nov
  2. Posted 13 NovAndromeda Delta - Audio Transcription Adjudicator V2 - Portuguese (Brazil)$8.93/hrWelo DataBRBRChecked auto, 15 Nov
  3. Posted 13 NovDataGen & EvalsSee listingTuringCountries not statedNot statedChecked auto, 15 Nov
  4. Posted 13 NovDomain Expert: BiologySee listingTuringCountries not statedNot statedChecked auto, 15 Nov
  5. Posted 13 NovBioinformatics / Computational Genomics Expert$200/taskTuringCountries not statedNot statedChecked auto, 15 Nov
  6. Posted 13 NovMusic AnnotatorSee listingTuringCountries not statedNot statedChecked auto, 15 Nov
  7. Posted 13 NovUS Medical Expert – DermatologySee listingTuringCountries not statedNot statedChecked auto, 15 Nov
  8. Posted 13 NovAndromeda Delta - Audio Transcription Adjudicator V2 - Spanish (Spain)$21.9/hrWelo DataESESChecked auto, 15 Nov