ai evaluation engineer

Micromine in Australia is seeking a Machine Learning Engineer to evaluate, benchmark and improve LLM and agent-based AI systems across the product suite. You’ll design evaluation frameworks, measure model performance and provide insights that drive reliable, high-impact AI solutions for the mining industry.You'll work with PyTorch, develop benchmarks, and...

Profectus is seeking an experienced AI Test and Evaluation and Assurance Specialist to support the safe adoption of AI-enabled capability within Australian Government contexts from Canberra. The role provides independent, evidence-based assessment of AI-enabled systems, governance artefacts and advisory support for risk and suitability at senior levels.The...

Mercor is seeking expert Evaluators in Nonprofit/philanthropy to review AI-generated documents, spreadsheets, and slide decks for accuracy and quality. You will apply deep subject-matter knowledge to grade the outputs with domain-specific rubrics.This remote hourly engagement requires native or professional English, and strong skills in Microsoft Office and...

StarRez is seeking a Senior Quality Engineer - AI to shape how we evaluate, validate, and improve AI-powered product experiences across our SaaS platform for student housing. You will balance hands-on work with coaching peers to raise quality standards.You will define practical evaluation standards, rubrics, and repeatable pipelines that reduce risk and...

StarRez, the global leader in student housing software, is seeking a Senior Quality Engineer - AI to shape how we evaluate, validate, and improve AI-powered product experiences. You will balance hands-on work with mentoring the team and influence product readiness across LLM, RAG, prompts, and tool-use.You’ll partner with engineers, product managers, and...

Remote | Freelance | Upcoming Project We are seeking fluent English Reviewers/QA Specialists for an upcoming writing annotation project. Job Description: Evaluate conversations between users and agents, assessing responses for helpfulness, quality, naturalness, and overall user satisfaction. The ideal candidate must have strong English writing skills,...

Australian English AI Product Evaluator🎯 The ProjectProductive Playhouse is building a talent pool of Australian English speakers for an upcoming project testing and evaluating leading AI chatbots. The objective is to enhance response quality through direct user interaction with the various AI models. Open to freelancers based outside the U.S.   This...

Obsidian is seeking expert Evaluators in Government/public administration to review AI-generated documents, spreadsheets, and slide decks for accuracy and quality. You will apply deep subject-matter expertise to grade outputs and provide structured feedback. This is a remote, hourly engagement. Candidates should demonstrate 5+ years in government work,...

Mercor is seeking expert Evaluators in Education / school to review and assess AI-generated work products (documents, spreadsheets, and slide decks) for accuracy, rigor, and domain quality. You will apply deep subject-matter expertise to grade outputs.This is a remote, hourly engagement. You will evaluate artifacts, identify errors, and provide clear,...

Obsidian is seeking expert Evaluators in Humanities / arts / culture to review AI-generated documents, slides, and spreadsheets for accuracy and quality. This is a remote, hourly engagement, requiring deep subject-matter expertise to grade outputs.You will evaluate artifacts against rubrics, identify errors, and provide clear written feedback to improve...

Mercor is hiring experienced musicians to evaluate generative music AI models, in partnership with a leading AI lab. You will assess AI-generated music across a wide range of genres and rate it against detailed quality standards, working in Telugu and English.Start date is immediate and the project runs up to six months with a flexible commitment around 20...

Donor Republic is seeking a Software Engineer – AI Products to own the backend and AI-systems engineering for Lantern and the Assessment Hub. You’ll design prompt architectures, evaluation frameworks, and guarded outputs while maintaining a clean frontend within the established design system.The role requires 3–5 years of full-stack experience with a...

Mercor is seeking expert Evaluators in Training / onboarding / L&D to review AI-generated artifacts for accuracy and domain quality. This is a remote, hourly engagement where you will apply deep subject-matter expertise to grade outputs.Responsibilities include evaluating artifacts, identifying errors, and delivering structured feedback to improve content...

Mercor seeks expert Evaluators in Healthcare/clinical to review AI-generated documents, spreadsheets, and slide decks for accuracy and domain quality. You will apply deep subject-matter expertise to grade outputs. This is a remote, hourly engagement.Required are 5+ years in Healthcare/clinical, fluent English, and strong proficiency with MS Office and Google...

Mercor is seeking detail-oriented writing experts to contribute to a high-impact video AI research project with a leading lab. Annotators will watch two short video clips side by side, compare them to a rubric across dimensions such as prompt adherence, motion quality, artifacts, and physical plausibility, and assign scores.Ideal candidates have strong...

Prolific seeks an AI Trainer fluent in Marathi to evaluate AI-generated text and voice snippets for authenticity and naturalness. This role involves assessing the quality of AI's understanding of the Marathi language, ensuring it captures the depth of its nuance.Successful candidates should possess strong communication skills in both Marathi and English, as...

Mercor is partnering with a leading AI research organization to hire experienced marketing professionals. The role focuses on defining what excellent work looks like by designing task-specific grading criteria and scoring completed samples with written justifications.You will evaluate both AI-generated and human work, applying structured, evidence-based...

Mercor is seeking expert Evaluators in Education / school to review AI-generated documents, spreadsheets and slide decks for accuracy and quality. This remote, hourly engagement relies on your deep subject-matter expertise to grade outputs against rubrics and provide structured feedback.Required are 5+ years in Education, native or professional English, and...

Are you ready for new challenges and new opportunities?Join our team!Current job opportunities are posted here as they become available.Subscribe to our RSS feeds to receive instant updates as new positions become available.Conversation Evaluator for AI Training - EnglishWe are seeking fluent English Reviewers/QA Specialists for an upcoming writing...

Overview In this role, you will evaluate AI-generated responses and provide structured written feedback. This is a great opportunity for sharp, analytical thinkers to contribute to high-impact AI research projects. Basic Qualifications Bachelor's degree from a top-500 globally ranked university preferred Strong analytical and written communication...