Task Design for Crowdsourcing Complex Cognitive Skills

ABSTRACT Task design for crowdsourcing is a key factor limiting the quality of crowd-sourced results. This case study presents our design process for a complex cognitive task: generating Dimension/Values for categorizing ideas. Conveying the task to workers was a formidable design challenge. We present five strategies and lessons learned from testing with workers. Decomposing the cognitive process and testing mastery of each cognitive subprocess enabled us to finally convey the task requirements and obtain useful results.