Skill ontology-based model for Quality Assurance in Crowdsourcing

TitleSkill ontology-based model for Quality Assurance in Crowdsourcing
Publication TypeConference Paper
Year of Publication2014
AuthorsMaarry, K. E., W. - T. Balke, H. Cho, S. -won Hwang, and Y. Baba
Conference NameWorkshop on Uncertain and Crowdsourced Data (UnCrowd), DASFAA
Date Published04/2014
Conference LocationBali, Indonesia

Crowdsourcing continues to gain more momentum as its potential becomes more recognized. Nevertheless, the associated quality aspect remains a valid concern, which introduces uncertainty in the results obtained from the crowd. We identify the different aspects that dynamically affect the overall quality of a crowdsourcing task. Accordingly, we propose a skill ontology-based model that caters for these aspects, as a management technique to be adopted by crowdsourcing platforms. The model maintains a dynamically evolving ontology of skills, with libraries of standardized and personalized assessments for awarding workers skills. Aligning a worker’s set of skills to that required by a task, boosts the ultimate resulting quality. We visualize the model’s components and workflow, and consider how to guard it against malicious or unqualified work-ers, whose responses introduce this uncertainty and degrade the overall quality.

DASFAA14_UnCrowd_2.pdf100.82 KB