Appen Logo

Appen

Ukrainian LLM Evaluator

Sorry, this job was removed Sorry, this job was removed at 08:07 a.m. (PST) on Sunday, May 11, 2025
Be an Early Applicant
Remote
Hiring Remotely in United States
Remote
Hiring Remotely in United States

Similar Jobs

Yesterday
Remote
United States
140K-165K Annually
Senior level
140K-165K Annually
Senior level
Cloud • eCommerce • Enterprise Web • Information Technology • Software
The Senior iOS QA Engineer will develop tests for the iOS SDK, ensure code quality through automation, and contribute to product development strategies.
Top Skills: AppiumFlutterGitlabObjective-CReact NativeSwiftXctest
Yesterday
Remote
Hybrid
United States
120K-165K
Senior level
120K-165K
Senior level
Cloud • Enterprise Web • Other • Productivity • Software • Analytics • Design
The Professional Services Solutions Architect will implement Altium solutions for enterprise customers, providing technical guidance and managing customer relationships. Key duties include architecture solutions, creating statements of work, and collaborating with engineering and R&D teams.
Top Skills: Altium Enterprise SolutionsCloud InfrastructuresEcad SoftwarePlm Integrations
Yesterday
Remote
Hybrid
Mab, LA, USA
73K-198K Annually
Senior level
73K-198K Annually
Senior level
eCommerce • Food • Sales • Software
Lead a team of sales executives to drive revenue growth through coaching, strategy development, and performance monitoring, while fostering collaboration and accountability.
Top Skills: ExcelSalesforce
Join Project Spearmint, a multilingual AI response evaluation project reviewing large language model (LLM) outputs in different languages, focused on either Tone or Fluency. Native-level fluency in a target language, along with strong English comprehension, is required.

As an evaluator, you will review short, pre-segmented datasets and assess model-generated replies based on specific quality dimensions. Your input will help validate evaluation frameworks and establish baseline quality metrics for future model development.

Key Responsibilities:
- Evaluate model replies in your native language based on either Tone or Fluency.
- Assess the overall quality, correctness, and naturalness of responses.
- Read the user prompt and two model replies, then rate each using a five-point scale.
- Provide brief rationales for any extreme ratings.

Project Breakdown:

Batch 1 – Tone: Determine whether replies are helpful, insightful, engaging, and fair. Flag formality mismatches, condescension, bias, or other tonal issues.

Batch 2 – Fluency: Assess grammatical accuracy, clarity, coherence, and natural flow.


This is a project-based opportunity with CrowdGen, where you will join the CrowdGen Community as an Independent Contractor. If selected, you will receive an email from CrowdGen regarding the creation of an account using your application email address. You will need to log in to this account, reset your password, complete the setup requirements, and proceed with your application for this role.

Make an impact on the future of AI – apply today and contribute from the comfort of your home.

Appen Kirkland, Washington, USA Office

12131 113th Ave NE, Kirkland, WA, United States, 98304

What you need to know about the Seattle Tech Scene

Home to tech titans like Microsoft and Amazon, Seattle punches far above its weight in innovation. But its surrounding mountains, sprinkled with world-famous hiking trails and climbing routes, make the city a destination for outdoorsy types as well. Established as a logging town before shifting to shipbuilding and logistics, the Emerald City is now known for its contributions to aerospace, software, biotech and cloud computing. And its status as a thriving tech ecosystem is attracting out-of-town companies looking to establish new tech and engineering hubs.

Key Facts About Seattle Tech

  • Number of Tech Workers: 287,000; 13% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Amazon, Microsoft, Meta, Google
  • Key Industries: Artificial intelligence, cloud computing, software, biotechnology, game development
  • Funding Landscape: $3.1 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Madrona, Fuse, Tola, Maveron
  • Research Centers and Universities: University of Washington, Seattle University, Seattle Pacific University, Allen Institute for Brain Science, Bill & Melinda Gates Foundation, Seattle Children’s Research Institute
By clicking Apply you agree to share your profile information with the hiring company.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account