Terac Logo

Terac

AI Evaluators: Assessing A Shopping Assistant

Reposted 6 Days Ago
Remote
Hiring Remotely in United States
50-50 Hourly
Entry level
Remote
Hiring Remotely in United States
50-50 Hourly
Entry level
Evaluate conversations between users and an AI shopping assistant, identify logical errors, inaccuracies, and poor recommendations, and create structured rubrics and verifiers for judging response quality. This remote engagement requires at least 20 hours per week and focuses on improving model performance and e-commerce response quality.
The summary above was generated by AI
What We're Researching

We're hiring AI evaluators to assess the accuracy and helpfulness of a new digital shopping assistant. This project focuses on understanding how well the system handles real-world e-commerce queries and where it falls short in its logic. Your analysis will directly feed into improving the underlying model and its response quality.

How It Works

You will review real interaction traces between users and the shopping assistant within our custom platform. As you analyze these conversations, you will pinpoint specific failures, logical errors, or unhelpful product recommendations. From there, you will create structured rubrics and verifiers to consistently judge future response quality. This is an ongoing remote engagement requiring 20+ hours per week.

Who This Is For

This opportunity is ideal for quality assurance specialists, AI data evaluators, and e-commerce professionals with a strong eye for detail. We welcome applicants with prior experience in prompt engineering, complex data annotation, or software testing. You should be comfortable analyzing text interactions deeply and building structured evaluation frameworks from scratch.

What You'll Do
  • Review real user interaction traces with an AI shopping assistant

  • Identify logical failures, inaccuracies, or poor recommendations in the text

  • Create structured rubrics and verifiers to judge response quality

  • Commit to 20+ hours per week of evaluation work on our internal platform

Who Should Apply
  • Experience in data evaluation, quality assurance, or AI training

  • Strong analytical skills with the ability to spot subtle errors in text

  • Familiarity with e-commerce search and digital shopping experiences

  • Ability to commit to a sustained workload of 20+ hours per week

Compensation

$50 per hour

 
Ready to participate?

Start your paid interview now

 
About Terac

Terac is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.

 

Learn more at terac.com or on YouTube at @jointerac.

Similar Jobs

15 Minutes Ago
Remote or Hybrid
45K-85K Annually
Junior
45K-85K Annually
Junior
Artificial Intelligence • Fintech • Insurance • Marketing Tech • Software • Analytics
Handles inbound calls and warm leads, consults customers on insurance needs, recommends appropriate Property and Casualty products, and converts prospects into policyholders. The role includes paid licensing and sales training, commission opportunities, customer communication, lead conversion, and flexible remote work. Representatives must work a weekday and weekend schedule, maintain a dedicated home workspace, and meet specified wired internet requirements.
Top Skills: PcWired High-Speed Internet
15 Minutes Ago
Remote or Hybrid
79K-130K Annually
Mid level
79K-130K Annually
Mid level
Automotive • Hardware • Internet of Things • Mobile • Software • App development • PropTech
Analyzes chatbot conversations and interaction data to identify performance issues, trends, anomalies, and root causes. Leads retraining, response optimization, AI workflow testing, QA, and performance measurement. Coordinates cross-functional improvement initiatives, defines AI-ready knowledge standards, reports business impact, and advises customer experience and technical support teams on chatbot quality and best practices.
Top Skills: Artificial IntelligenceChatbotsMS OfficeNatural Language Processing
15 Minutes Ago
Remote
USA
225K-275K Annually
Senior level
225K-275K Annually
Senior level
Artificial Intelligence • Cloud • Information Technology • Energy • Infrastructure as a Service (IaaS)
Leads Riot’s corporate M&A, strategic investments, joint ventures, and development capital structuring for data center infrastructure. Responsibilities include sourcing and executing transactions, building valuation and returns models, managing diligence, negotiating with capital partners and advisors, preparing board and investment committee materials, and supporting post-close integration. The role partners with executive leadership, finance, legal, tax, development, and site acquisition teams.

What you need to know about the Seattle Tech Scene

Home to tech titans like Microsoft and Amazon, Seattle punches far above its weight in innovation. But its surrounding mountains, sprinkled with world-famous hiking trails and climbing routes, make the city a destination for outdoorsy types as well. Established as a logging town before shifting to shipbuilding and logistics, the Emerald City is now known for its contributions to aerospace, software, biotech and cloud computing. And its status as a thriving tech ecosystem is attracting out-of-town companies looking to establish new tech and engineering hubs.

Key Facts About Seattle Tech

  • Number of Tech Workers: 287,000; 13% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Amazon, Microsoft, Meta, Google
  • Key Industries: Artificial intelligence, cloud computing, software, biotechnology, game development
  • Funding Landscape: $3.1 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Madrona, Fuse, Tola, Maveron
  • Research Centers and Universities: University of Washington, Seattle University, Seattle Pacific University, Allen Institute for Brain Science, Bill & Melinda Gates Foundation, Seattle Children’s Research Institute

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account