What the work is
About the Role
We are looking for experienced software engineers with strong technical leadership skills and hands-on experience working with high-quality public GitHub repositories. In this role, you will analyze real-world codebases, configure development environments, triage issues, evaluate test quality, and assess how effectively LLMs solve software engineering tasks.
This is an opportunity to combine practical software engineering with AI research and contribute to the development of next-generation coding agents.
What You’ll Do
- Analyze and triage issues across popular open-source repositories.
- Set up and configure development environments for real-world codebases.
- Dockerize repositories and establish reliable local execution workflows.
- Run, modify, debug, and test projects locally.
- Evaluate unit test coverage, reliability, and overall test quality.
- Assess LLM performance on bug-fixing and software engineering tasks.
- Identify challenging repositories and issues for AI evaluation benchmarks.
- Collaborate with researchers and engineers to design high-quality evaluation scenarios.
- Review code and provide technical guidance to junior engineers.
- Lead or coordinate small teams when required.
Who gets hired
- 5+ years of professional software engineering experience.
- Strong proficiency in Python and JavaScirpt.
- Practical experience with Git and Docker.
- Basic understanding of software development pipelines and CI workflows.
- Ability to understand, navigate, and modify complex codebases.
- Experience running and testing real-world software projects locally.
- Strong debugging, code review, and problem-solving skills.
- Ability to work independently and communicate technical findings clearly.
Offer Details
- Commitments required: 40 hours per week with 4 hours/ day overlap with PST
- Engagement type: Contractor
- Duration of contract: 3 months
Pay
See listing, fully remote. How payouts and tax work.
Check history
How we score →today–Open