Convos — Turning AI Evaluation Into a Game
A data marketplace where people pit AI models against each other, vote for the better answer and earn rewards, turning everyday AI battles into licensable preference data.
Convos turns human interactions with AI models into useful preference data. People compare answers from different AI models, in blind battles or modes such as Direct Battle and Speed Dating, vote for the better result and earn rewards. Those interactions feed private, licensable multimodal datasets and AI benchmarking, on a data marketplace built on Story Protocol. AI evaluation and data labeling are valuable, but usually boring and technical. The product has to bring people back again and again without the experience feeling like unpaid annotation work.
Client
M0
Services
Product Design UX Design Gamification & Incentive Design Landing Page Design
Industries
AI Evaluation, Data Marketplace, Web3, Human Feedback
Date
May 2026
What if evaluating AI models felt less like labeling data and more like playing a game? As Design Lead at M0, I worked on the landing page and product communication, the AI battles and model-comparison experience, contributor incentives and reward flows, and the broader marketplace experience. Design Challenges & Solutions 1. Evaluation Feels Like Work Challenge: Traditional AI evaluation feels like a job. Solution: Designed battle-based interactions, quick comparison modes, rewards and progression, turning annotation into a lightweight consumer activity. 2. Brand Bias in Every Vote Challenge: Knowing which model wrote an answer pulls votes toward familiar brands. Solution: Kept models anonymous in blind battles, so people judge the quality of the answer rather than the company behind it. 3. Why My Vote Matters Challenge: People need to understand why their votes matter. Solution: Connected credits and rewards, leaderboard feedback and visible model rankings, creating an immediate reward loop that ties each vote to collective benchmarking.
Outcome Convos is live at convos.to, where two anonymous models answer each prompt, every vote earns credits and model rankings update from human comparisons, with rewards for voters on Solana, EVM and Story Protocol. Key Learnings Make the Valuable Thing Fun Learning: People contribute more when the task feels like play rather than work. Implementation: Comparison became a battle, with quick modes and progression. Design Out the Bias Learning: The quality of the data depends on what people can see while they vote. Implementation: Blind battles keep attention on the answers, not the brands. Close the Loop Learning: Contributors stay when they can see the effect of their effort. Implementation: Rewards and leaderboard feedback connect each vote to visible results.
