Page 1 of 3

Response Evaluator (LLM Project)

Quantigo AI Independent Contractor

Quantigo AI is seeking Response Evaluator agents to support Large Language Model (LLM) projects. The role focuses on evaluating, reviewing, and improving AI-generated responses to ensure accuracy, relevance, and quality across diverse use cases.


Key Responsibilities
● Evaluate AI/LLM-generated responses for correctness, relevance, and groundedness that aligns with guidelines
● Compare multiple model outputs and select or rank the best responses
● Identify errors, biases, or gaps in model outputs and provide structured feedback
● Follow detailed evaluation rubrics and project-specific instructions
● Maintain high quality and consistency while meeting productivity targets
Required Qualifications
● Minimum 1–2 years of experience working on LLM, AI evaluation, or NLP-related projects
● Strong analytical and critical thinking skills
● Excellent written comprehension and attention to detail
● Ability to follow complex guidelines and apply consistent judgment
● Comfortable working independently in a remote, contract-based environment ● Exceptional communication skill in English (Spoken and Written)
Preferred Qualifications
● Prior experience as an AI evaluator, annotator, or prompt/response reviewer
● Familiarity with generative AI tools and language models is a big plus.
● Experience working on data annotation or model training projects
Contract Details
Individual contractor role (not an employment position)
Remote work
Flexible hours based on project requirements