A crowdsourcing framework for the production and use of film and television data
New Review of Hypermedia and Multimedia,
Vol. 17, No. 1, April 2011, 73Á97
A crowdsourcing framework for the production and use of film and television data
GARY GEISLER*, GEOFF WILLARD and CARLOS OVALLE
School of Information, University of Texas at Austin, Austin, TX, USA
(Received 19 July 2010; final version received 30 November 2010)
This paper outlines a framework that would enable the detailed indexing of film and television media through crowdsourcing. By making it easier to generate detailed data about these media on a large scale, fans and scholars can more efficiently produce a wide range of artifacts that reflect their interests in this content. Our development of a test collection included detailed indexing of 12 feature films and 8 television programs. We describe the conditions that make crowdsourcing an ideal approach for accomplishing this work on a larger scale; present a three-level development framework; and discuss how automated indexing, crowdsourcing quality, and copyright concerns might influence continued development of the project. Our framework highlights the potential of both multimedia indexing and crowdsourcing and can serve as a model for others embarking on projects that involve indexing, annotating, or labeling large multimedia collections. Keywords: Crowdsourcing; Film and television; Indexing; Moving images; Video;
Visualization
1. Introduction
The rapid growth of many of today’s most popular web sites*Facebook,
YouTube, and Wikipedia, for example*can be largely attributed to the work not of the managers or developers of those sites, but to the efforts of what would traditionally be referred to as those sites’ consumers. That is, much of the actual content the sites provide (articles, videos, photos, messages, comments, and metadata) is contributed by a significant portion of the sites’ own customers, acting not only as consumers of the product but producers of it as well. Benkler has said that this
Vol. 17, No. 1, April 2011, 73Á97
A crowdsourcing framework for the production and use of film and television data
GARY GEISLER*, GEOFF WILLARD and CARLOS OVALLE
School of Information, University of Texas at Austin, Austin, TX, USA
(Received 19 July 2010; final version received 30 November 2010)
This paper outlines a framework that would enable the detailed indexing of film and television media through crowdsourcing. By making it easier to generate detailed data about these media on a large scale, fans and scholars can more efficiently produce a wide range of artifacts that reflect their interests in this content. Our development of a test collection included detailed indexing of 12 feature films and 8 television programs. We describe the conditions that make crowdsourcing an ideal approach for accomplishing this work on a larger scale; present a three-level development framework; and discuss how automated indexing, crowdsourcing quality, and copyright concerns might influence continued development of the project. Our framework highlights the potential of both multimedia indexing and crowdsourcing and can serve as a model for others embarking on projects that involve indexing, annotating, or labeling large multimedia collections. Keywords: Crowdsourcing; Film and television; Indexing; Moving images; Video;
Visualization
1. Introduction
The rapid growth of many of today’s most popular web sites*Facebook,
YouTube, and Wikipedia, for example*can be largely attributed to the work not of the managers or developers of those sites, but to the efforts of what would traditionally be referred to as those sites’ consumers. That is, much of the actual content the sites provide (articles, videos, photos, messages, comments, and metadata) is contributed by a significant portion of the sites’ own customers, acting not only as consumers of the product but producers of it as well. Benkler has said that this
References: Lumpur, Malaysia, New York: Springer, pp. 811Á821, 2009. September 2008, University College Cork, Cork, Ireland, pp. 1Á6, 2008. 9Á15, 2008. N. Anderson, 2010. OpenStreetMap: Crowd-sourcing the world, a street at a time. Ars Technica. Available online at: http://www.arstechnica.com/open-source/news/2010/06/crowd-sourced-world-map.ars (accessed 12 July 2010). J. Bacon, The Art of Community: Building the New Age of Participation, Sebastopol, CA: O’Reilly Media, Inc., 2009. J. Band, 2007. Educational fair use today. Association of Research Libraries. Available online at: http:// www.arl.org/bmÂdoc/educationalfairusetoday.pdf (accessed 10 July 2010). Y. Benkler, The Wealth of Networks: How Social Production Transforms Markets and Freedom, New Haven, CT: Yale University Press, 2006. F. Berman and H. Brady, 2005. Final report: NSF SBE-CISE Workshop on Cyberinfrastructure and the Social Sciences pubs/CyberInfrastructure_FINAL.pdf (accessed 28 October 2010). A. Bruns, ‘‘Produsage’’, in Proceedings of the 6th ACM SIGCHI Conference on Creativity & Cognition, 13Á15 June 2007, New York, New York: ACM, 2007. August 2009, Singapore, Singapore: World Scientific, pp. 286Á295, 2009. 7Á11 June 2005, Denver, CO, New York: ACM, pp. 69Á78, 2005. O. Dekel and O. Shamir, ‘‘Vox populi: Collecting high-quality labels from a crowd’’, in Proceedings of the 22nd Annual Conference on Learning Theory, 18Á21 June 2009, Montreal, Quebec, 2009. June 2008, Chapel Hill, NC, 2008. R. Glott, P. Schmidt and R. Ghosh, 2009. Wikipedia survey*first results. Wikipedia. Available online at: http://www.upload.wikimedia.org/wikipedia/foundation/a/a7/Wikipedia_General_Survey-Overview_0.3.9. pdf (accessed 15 July 2010). L. Hoffmann, ‘‘Crowd control’’, Communications of the ACM, 52(3), pp. 16Á17, 2009. J. Howe, 2006. The rise of crowdsourcing. Wired, 14(6). Available online at: http://www.wired.com/wired/ archive/14.06/crowds.html (accessed 9 July 2010). Language Processing, 31 May, Stroudsburg, PA: Association for Computational Linguistics, 2009. State University Press, pp. 51Á69, 1995. H. Jenkins, Fans, Bloggers, and Gamers: Exploring Participatory Culture, New York: New York University Press, 2006. Jinni, 2010. The movie genome. Available online at: http://www.jinni.com/movie-genome.html (accessed 25 May 2010). E. Katz, The Film encyclopedia: The Most Comprehensive Encyclopedia of World Cinema in a Single Volume, (5th ed)., New York: Collins, 2005. Lost, 2004. Television Program, ABC Studios. ACM, pp. 77Á85, 2009. MIT Center for Educational Computing Initiatives, 2006. The virtual screening room. Available online at: http://www.ceci.mit.edu/projects/virtual_screening_room/index.htm (accessed 20 May 2010). Pandora, 2010. Pandora FAQ. Available online at: http://www.blog.pandora.com/faq/ (accessed 25 May 2010). 26th Annual International Conference on Machine Learning, 14Á18 June, New York: ACM, pp. 889Á896, 2009. E.S. Raymond, The Cathedral & the Bazaar, Sebastopol, CA: O’Reilly, 1999. New York: ACM, pp. 740Á741, 2004. C. Shirky, Here Comes Everybody: The Power of Organizing Without Organizations, New York: Penguin Press, 2008.