Crowdsourcing, a term coined by Jeff Howe, is when you take a large task, break it down into smaller - micro - pieces and send it out to a large group of people to perform each microtask in parallel for a monetary reward.
Microtask crowdsourcing traditionally covers a different set of scenarios. Tasks primarily rely on basic human abilities, including visual and audio cognition, as well as natural language understanding and communication (sometimes in different languages) and less on acquired skills (such as subject-matter knowledge). As such, a great share of the tasks addressed via microtask platforms like MTurk or CrowdFlower could be referred to as ‘routine’ tasks - recognizing objects in images, transcripting audio and video material and text editing.
To be more efficient than traditional outsourcing (or even in-house resources), the tasks need to be highly parallelized. This means that the actual work is executed by a high number of contributors in a decentralized fashion; this not only leads to significant improvements in terms of time of delivery, but also offers a means to cross-check the accuracy of the answers (as each task is typically assigned to more than one person) and reward the workers according to their performance and productivity.
In this workshop, you will not only learn the fundamentals of crowdsourcing but also get your hands dirty by designing and executing a task on CrowdFlower and getting and analyzing the results from the workers. Also, you will learn about the successful applications of crowdsourcing in different domains.
Duration: 8,5 hours
Target Group: Beginner
|09:00 - 09:15||Welcome with coffee and tea and introduction organizers|
|09:15 - 10:00||Introduction to Human computation|
|10:00 - 11:00||Microtask crowdsourcing fundamentals 1|
|11:00 - 11:15||Coffee and tea break|
|11:15 - 12:30||Hands-on 1: Designing a microtask on CrowdFlower|
|12:30 - 13:30||Lunch break|
|13:30 - 14:30||Microtask crowdsourcing fundamentals 2|
|14:30 - 15:30||Hands-on 2: Executing a microtask on CrowdFlower|
|15:30 - 16:00||Coffee and tea break|
|16:00 - 16:30||Discussion of the results|
|16:30 - 17:30||Applications of crowdsourcing, summary and conclusion|
Prerequisite knowledge or skills required
Learning objectives and goals
You will learn about one of the most promising trends today: machine learning!
Learn the fundamentals of human computation
Learn how to design and execute a hands-on crowdsourcing task on CrowdFlower
Learn the applications of crowdsourcing
Required materials and resources
Laptop (browser: Chrome, Firefox, Safari)
Course materials and resources
All slides will be made available online