Most people who don't get in fail the screening, not the CV. Five minutes here first: How to pass the Turing assessment and interviews.
What the work is
Role Overview
We are seeking a Subject Matter Expert (SME) in Market Intelligence, Media Production, or Technical Communications to author, evaluate, and calibrate high-fidelity domain tasks, realistic workplace scenarios, and benchmark evaluation rubrics.
This unified opening covers contributors across three specialized disciplines: Market Research & Analytics, Media Production & Direction, and Editorial & Technical Communications. Candidates will be placed in either an Expert Contributor (Mid-Level) or Senior Specialist / Reviewer (Senior Level) track based on overall domain depth, years of practical experience, and peer review background.
Who gets hired
- 3 to 15+ years of demonstrable industry experience within one of the target functional tracks.
- Strong written communication and documentation skills, with an ability to justify editorial, methodological, or artistic critiques objectively.
- High attention to detail, capable of identifying subtle logical inconsistencies, formatting errors, or rubric deviations.
- Comfort operating within structured review cycles and fast-paced calibration environments.
Responsibilities
- Benchmark Authoring & Task Design: Develop high-difficulty, specialized domain tasks, realistic workplace scenarios, and gold-standard solution sets tailored to your specific functional discipline.
- Rubric Application & Scoring: Apply evaluation criteria defined by the Functional SME Lead to assess outputs with strict adherence to accuracy, terminology, and quality benchmarks.
- Peer Review & Calibration (Senior Track): Review, calibrate, and critique pod members' outputs; resolve edge-case ambiguities and identify rubric discrepancies to ensure day-to-day quality control across the team.
- Iterative Refinement: Incorporate calibration feedback and iterative critique rounds to align deliverables with pod standards.
Education & Experience
- Bachelor's degree or equivalent practical experience in any field.
- Experience in AI evaluation, data annotation, content review, quality assurance, or a related analytical role is preferred but not required.
Offer Details
- Commitments Required: at least 4 hours per day and upto 40 hours per week with 4 hours of overlap with PST.
- Engagement type: Contractor
- Engagement Length: 12 weeks
Evaluation Process -
- Applicants needs to submit Job Interest Form & complete the assessment.
- Final selected candidates will be contacted with the next steps and onboarding requirements.
The platform posted this on 26 Oct 2026.