{"id":379,"date":"2009-10-27T19:45:20","date_gmt":"2009-10-27T18:45:20","guid":{"rendered":"http:\/\/staff.science.uva.nl\/~cgmsnoek\/?p=379"},"modified":"2009-10-27T19:45:20","modified_gmt":"2009-10-27T18:45:20","slug":"the-mediamill-trecvid-2009-semantic-video-search-engine","status":"publish","type":"post","link":"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/","title":{"rendered":"The MediaMill TRECVID 2009 Semantic Video Search Engine"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" class=\"alignright size-medium wp-image-380\" title=\"tv09\" src=\"http:\/\/staff.science.uva.nl\/~cgmsnoek\/wp-content\/uploads\/2009\/10\/tv09-300x175.png\" alt=\"tv09\" width=\"300\" height=\"175\" align=\"right\" srcset=\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2009\/10\/tv09-300x175.png 300w, https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2009\/10\/tv09-1024x599.png 1024w, https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2009\/10\/tv09.png 1435w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/p>\n<p>The draft notebook paper for TRECVID 2009 by the MediaMill team, containing members of the University of Amsterdam, INESC-ID and the University of Surrey, <a href=\"http:\/\/staff.science.uva.nl\/~cgmsnoek\/pub\/MediaMill-TRECVID2009-notebook.pdf\" target=\"_blank\">is now available<\/a>. In the paper we describe our TRECVID 2009 video retrieval experiments. The MediaMill team participated in three tasks: concept detection, automatic search, and interactive search. Starting point for the MediaMill concept detection approach is our top-performing bag-of-words system of last year, which uses multiple color descriptors, codebooks with soft-assignment, and kernel-based supervised learning. We improve upon this baseline system by exploring two novel research directions. Firstly, we study a multi-modal extension by the inclusion of 20 audio concepts and fusing using two novel multi-kernel supervised learning methods. Secondly, with the help of recently proposed algorithmic refinements of bag-of-words, a bag-of-words GPU implementation, and compute clusters, we scale-up the amount of visual information analyzed by an order of magnitude, to a total of 1,000,000 i-frames. Our experiments evaluate the merit of these new components, ultimately leading to 64 robust concept detectors for video retrieval. For retrieval, a robust but limited set of concept detectors necessitates the need to rely on as many auxiliary information channels as possible. For automatic search we therefore explore how we can learn to rank various information channels simultaneously to maximize video search results for a given topic. To improve the video retrieval results further, our interactive search experiments investigate the roles of visualizing preview results for a certain browse-dimension and relevance feedback mechanisms that learn to solve complex search topics by analysis from user browsing behavior. The 2009 edition of the TRECVID benchmark has again been a fruitful participation for the MediaMill team, resulting in the top ranking for both concept detection and interactive search.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The draft notebook paper for TRECVID 2009 by the MediaMill team, containing members of the University of Amsterdam, INESC-ID and the University of Surrey, is now available. In the paper we describe our TRECVID 2009 video retrieval experiments. The MediaMill team participated in three tasks: concept detection, automatic search, and interactive search. Starting point for [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-379","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>The MediaMill TRECVID 2009 Semantic Video Search Engine - Cees Snoek<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The MediaMill TRECVID 2009 Semantic Video Search Engine - Cees Snoek\" \/>\n<meta property=\"og:description\" content=\"The draft notebook paper for TRECVID 2009 by the MediaMill team, containing members of the University of Amsterdam, INESC-ID and the University of Surrey, is now available. In the paper we describe our TRECVID 2009 video retrieval experiments. The MediaMill team participated in three tasks: concept detection, automatic search, and interactive search. Starting point for [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/\" \/>\n<meta property=\"og:site_name\" content=\"Cees Snoek\" \/>\n<meta property=\"article:published_time\" content=\"2009-10-27T18:45:20+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/staff.science.uva.nl\/~cgmsnoek\/wp-content\/uploads\/2009\/10\/tv09-300x175.png\" \/>\n<meta name=\"author\" content=\"Cees\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Cees\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/\",\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/\",\"name\":\"The MediaMill TRECVID 2009 Semantic Video Search Engine - Cees Snoek\",\"isPartOf\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/staff.science.uva.nl\/~cgmsnoek\/wp-content\/uploads\/2009\/10\/tv09-300x175.png\",\"datePublished\":\"2009-10-27T18:45:20+00:00\",\"author\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/#primaryimage\",\"url\":\"http:\/\/staff.science.uva.nl\/~cgmsnoek\/wp-content\/uploads\/2009\/10\/tv09-300x175.png\",\"contentUrl\":\"http:\/\/staff.science.uva.nl\/~cgmsnoek\/wp-content\/uploads\/2009\/10\/tv09-300x175.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.ceessnoek.info\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The MediaMill TRECVID 2009 Semantic Video Search Engine\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.ceessnoek.info\/#website\",\"url\":\"https:\/\/www.ceessnoek.info\/\",\"name\":\"Cees Snoek\",\"description\":\"research on video and image ai\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.ceessnoek.info\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\",\"name\":\"Cees\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"caption\":\"Cees\"},\"sameAs\":[\"http:\/\/www.CeesSnoek.info\"],\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The MediaMill TRECVID 2009 Semantic Video Search Engine - Cees Snoek","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/","og_locale":"en_US","og_type":"article","og_title":"The MediaMill TRECVID 2009 Semantic Video Search Engine - Cees Snoek","og_description":"The draft notebook paper for TRECVID 2009 by the MediaMill team, containing members of the University of Amsterdam, INESC-ID and the University of Surrey, is now available. In the paper we describe our TRECVID 2009 video retrieval experiments. The MediaMill team participated in three tasks: concept detection, automatic search, and interactive search. Starting point for [&hellip;]","og_url":"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/","og_site_name":"Cees Snoek","article_published_time":"2009-10-27T18:45:20+00:00","og_image":[{"url":"http:\/\/staff.science.uva.nl\/~cgmsnoek\/wp-content\/uploads\/2009\/10\/tv09-300x175.png","type":"","width":"","height":""}],"author":"Cees","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Cees","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/","url":"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/","name":"The MediaMill TRECVID 2009 Semantic Video Search Engine - Cees Snoek","isPartOf":{"@id":"https:\/\/www.ceessnoek.info\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/#primaryimage"},"image":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/#primaryimage"},"thumbnailUrl":"http:\/\/staff.science.uva.nl\/~cgmsnoek\/wp-content\/uploads\/2009\/10\/tv09-300x175.png","datePublished":"2009-10-27T18:45:20+00:00","author":{"@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1"},"breadcrumb":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/#primaryimage","url":"http:\/\/staff.science.uva.nl\/~cgmsnoek\/wp-content\/uploads\/2009\/10\/tv09-300x175.png","contentUrl":"http:\/\/staff.science.uva.nl\/~cgmsnoek\/wp-content\/uploads\/2009\/10\/tv09-300x175.png"},{"@type":"BreadcrumbList","@id":"https:\/\/www.ceessnoek.info\/index.php\/the-mediamill-trecvid-2009-semantic-video-search-engine\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.ceessnoek.info\/"},{"@type":"ListItem","position":2,"name":"The MediaMill TRECVID 2009 Semantic Video Search Engine"}]},{"@type":"WebSite","@id":"https:\/\/www.ceessnoek.info\/#website","url":"https:\/\/www.ceessnoek.info\/","name":"Cees Snoek","description":"research on video and image ai","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.ceessnoek.info\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1","name":"Cees","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","caption":"Cees"},"sameAs":["http:\/\/www.CeesSnoek.info"],"url":"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/379","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/comments?post=379"}],"version-history":[{"count":2,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/379\/revisions"}],"predecessor-version":[{"id":382,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/379\/revisions\/382"}],"wp:attachment":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/media?parent=379"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/categories?post=379"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/tags?post=379"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}