Evaluate AI systems on personalized, real-world life tasks. Use extensive LLM experience to assess whether outputs are useful, realistic, safe, complete, and tailored to context, preferences, constraints, and tradeoffs. Provide detailed written judgments across areas such as food, health, productivity, careers, and learning, contributing to improvements in AI assistants and personal workflows. Expected commitment is 20–40 hours per week.
This role is for one of our clients
Compensation: $50 - $200 per hour
A leading AI research organization is seeking advanced LLM power users to evaluate how well AI systems handle personalized, real-world life tasks.
This role is for people who use AI tools heavily in their personal lives and can clearly judge whether an AI response is useful, personalized, realistic, and successful.
Requirements
Who We’re Looking For
Strong candidates have:
- Heavy personal usage of LLM products
- Experience using AI for multi-step tasks, planning, research, decision-making, or personal workflows
- Familiarity with tools such as ChatGPT, Claude, Gemini, Perplexity, Cursor, Windsurf, Codex, or other AI agents
- Ability to explain what makes an AI output good, bad, incomplete, unsafe, or unrealistic
- Strong written judgment and attention to detail
Why This Work Matters
LLMs are quickly becoming personal assistants for everyday decisions, but truly useful AI needs to do more than produce generic advice. It needs to understand context, preferences, constraints, tradeoffs, and what success looks like in real life. Your evaluations will help improve how AI systems support people with practical, high-context tasks across food, health, productivity, careers, and learning. This work directly contributes to making AI assistants more personalized, trustworthy, and useful for real-world personal workflows.
Engagement Details
- Expected commitment: 20-40 hours/week
We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
Similar Jobs
Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
Provides sustaining process engineering support for semiconductor solar cell wafer and assembly production. Maintains process control using SPC, troubleshoots equipment and manufacturing anomalies, and drives continuous improvement to reduce cycle time and cost while improving performance. Supports solar technology R&D, process and product development, safety initiatives, and innovative solutions. Requires semiconductor processing experience and collaboration with manufacturing operations teams.
Top Skills:
Analytical TechniquesCC++Dry EtchingHigh-Temperature AnnealingLabviewLean PrinciplesMaterial CharacterizationPhotolithographySemiconductor ProcessingSolar Cell AssembliesSolar Cell WafersSQLStatistical Process Control (Spc)Thin-Film DepositionVBAWet Etching
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Principal Software Engineer leading frontend and full-stack architecture for Atlassian’s Growth Platform. The role defines technical strategy for messaging, personalization, notifications, and experimentation systems; designs scalable microservices, APIs, data models, and React/TypeScript components; drives performance, reliability, security, accessibility, observability, and CI/CD standards; and influences multiple teams through architecture reviews, cross-functional partnership, and mentorship.
Top Skills:
APIsAutomated TestingCi/CdClient SdksClient-Side State ManagementCore Web VitalsCSSData ModelingDistributed SystemsDistributed TracingHTML5Javascript Es6+MicroservicesObservabilityReactTelemetryTypescript
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Provides organization-wide technical leadership for Atlassian’s Dev AI platforms and autonomous developer initiatives. Defines multi-year architectures for agent runtimes, orchestration engines, execution sandboxes, memory systems, and context retrieval. Drives distributed-system architecture, reliability, security, responsible AI governance, evaluation, and operational standards. Leads cross-organizational technical initiatives, resolves architectural ambiguity, mentors senior engineers, partners with product and engineering leaders, and remains hands-on with strategic prototypes and architectural validation.
Top Skills:
Agent OrchestrationAi Evaluation FrameworksArtificial IntelligenceAtlassian Teamwork GraphAutonomous Ai AgentsBitbucketCode IntelligenceCode SynthesisDistributed SystemsJIRAResponsible AiRetrieval SystemsRuntime SandboxesSemantic SearchTelemetry
What you need to know about the Seattle Tech Scene
Home to tech titans like Microsoft and Amazon, Seattle punches far above its weight in innovation. But its surrounding mountains, sprinkled with world-famous hiking trails and climbing routes, make the city a destination for outdoorsy types as well. Established as a logging town before shifting to shipbuilding and logistics, the Emerald City is now known for its contributions to aerospace, software, biotech and cloud computing. And its status as a thriving tech ecosystem is attracting out-of-town companies looking to establish new tech and engineering hubs.
Key Facts About Seattle Tech
- Number of Tech Workers: 287,000; 13% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Amazon, Microsoft, Meta, Google
- Key Industries: Artificial intelligence, cloud computing, software, biotechnology, game development
- Funding Landscape: $3.1 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Madrona, Fuse, Tola, Maveron
- Research Centers and Universities: University of Washington, Seattle University, Seattle Pacific University, Allen Institute for Brain Science, Bill & Melinda Gates Foundation, Seattle Children’s Research Institute
