E-commerce Evaluation Specialist
- company
- Mercor
- location
- India
- work mode
- remote
- posted
- 5 Aug
- **About The Job**
- **Mercor**
- connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include
- **Benchmark**
- **General Catalyst**
- **Peter Thiel**
- **Adam D'Angelo**
- **Larry Summers**
- and
- **Jack Dorsey**
- **Position:**
- Shopify Storefront Agent Evaluator
- **Type:**
- **Contract**
- **Compensation:**
- **$40–$50/hour**
- **Location:**
- **Remote**
- **Commitment:**
- **up to 5 hours/week**
- **Role Responsibilities**
- Evaluate the Storefront Agent by simulating real shopper interactions, including product discovery, comparisons, and inquiries about sizing, stock, shipping, and returns.
- Grade responses based on Relevance, Accuracy, and Safety to ensure they meet shopper expectations and adhere to store policies.
- Provide concise feedback on any failures to ensure clarity for team members not present during evaluation.
- Work independently to deliver precise and repeatable judgments on agent performance.
- **Qualifications**
- **Must-Have**
- Consumer e-commerce fluency — you shop online regularly and understand effective product recommendations.
- Careful, consistent judgment — apply the rubric uniformly across responses.
- Clear, concise written English — articulate reasons for response failures in a sentence or two.
- **Preferred**
- Hands-on Shopify merchant experience — familiarity with shopper behavior on Shopify stores.
- Experience using AI shopping assistants or chat agents — understand common failure points.
- Prior work evaluating, red-teaming, or annotating AI model outputs.
- Familiarity with retail operations: catalog and variant structure, inventory, shipping, and returns policy.
- **Application Process (Takes 20–30 mins to complete)**
- Upload resume
- AI interview based on your resume
- Submit form
- **Resources & Support**
- For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
- For any help or support, reach out to: support@mercor.com
- *PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.*
More ecommerce jobs from Mercor
- E-commerce Evaluation Specialist, Greater Montreal Metropolitan Area, CA
- E-commerce Evaluation Specialist, GB
- E-commerce Evaluation Specialist, US
- Shopify Specialist - E-commerce Expert, San Francisco, CA, US
- Shopify Specialist - E-commerce Expert, Toronto, Ontario, CA
- Shopify Specialist - E-commerce Expert, Mumbai Metropolitan Region, IN
- Shopify Specialist - E-commerce Expert, GB
- Shopify Specialist - E-commerce Expert, Greater Montreal Metropolitan Area, CA
- Ecommerce Specialist | Upto $55/hr, San Francisco, CA, US
- Ecommerce Specialist | Upto $55/hr, US