{"id":1563,"date":"2016-10-26T16:07:37","date_gmt":"2016-10-26T15:07:37","guid":{"rendered":"http:\/\/www.ceessnoek.info\/?p=1563"},"modified":"2016-10-26T20:19:18","modified_gmt":"2016-10-26T19:19:18","slug":"video-stream-retrieval-of-unseen-queries-using-semantic-memory","status":"publish","type":"post","link":"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/","title":{"rendered":"Video Stream Retrieval of Unseen Queries using Semantic Memory"},"content":{"rendered":"

\"streamretrieval\"The BMVC2016 paper Video Stream Retrieval of Unseen Queries using Semantic Memory<\/em>\u00a0by Spencer Cappallo<\/a>, Thomas Mensink and Cees Snoek is now available<\/a>. Retrieval of live, user-broadcast video streams is an under-addressed and increasingly relevant challenge. The on-line nature of the problem requires temporal evaluation and the unforeseeable scope of potential queries motivates an approach which can accommodate arbitrary search queries. To account for the breadth of possible queries, we adopt a no-example approach to query retrieval, which uses a query’s semantic relatedness to pre-trained concept classifiers. To adapt to shifting video content, we propose memory pooling and memory welling methods that favor recent information over long past content. We identify two stream retrieval tasks, instantaneous retrieval at any particular time and continuous retrieval over a prolonged duration, and propose means for evaluating them. Three large scale video datasets are adapted to the challenge of stream retrieval. We report results for our search methods on the new stream retrieval tasks, as well as demonstrate their efficacy in a traditional, non-streaming video task.<\/p>\n","protected":false},"excerpt":{"rendered":"

The BMVC2016 paper Video Stream Retrieval of Unseen Queries using Semantic Memory\u00a0by Spencer Cappallo, Thomas Mensink and Cees Snoek is now available. Retrieval of live, user-broadcast video streams is an under-addressed and increasingly relevant challenge. The on-line nature of the problem requires temporal evaluation and the unforeseeable scope of potential queries motivates an approach which […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-1563","post","type-post","status-publish","format-standard","hentry","category-science"],"yoast_head":"\nVideo Stream Retrieval of Unseen Queries using Semantic Memory - Cees Snoek<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Video Stream Retrieval of Unseen Queries using Semantic Memory - Cees Snoek\" \/>\n<meta property=\"og:description\" content=\"The BMVC2016 paper Video Stream Retrieval of Unseen Queries using Semantic Memory\u00a0by Spencer Cappallo, Thomas Mensink and Cees Snoek is now available. Retrieval of live, user-broadcast video streams is an under-addressed and increasingly relevant challenge. The on-line nature of the problem requires temporal evaluation and the unforeseeable scope of potential queries motivates an approach which […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/\" \/>\n<meta property=\"og:site_name\" content=\"Cees Snoek\" \/>\n<meta property=\"article:published_time\" content=\"2016-10-26T15:07:37+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2016-10-26T19:19:18+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2016\/10\/streamretrieval-300x167.png\" \/>\n<meta name=\"author\" content=\"Cees\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Cees\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/\",\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/\",\"name\":\"Video Stream Retrieval of Unseen Queries using Semantic Memory - Cees Snoek\",\"isPartOf\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2016\/10\/streamretrieval-300x167.png\",\"datePublished\":\"2016-10-26T15:07:37+00:00\",\"dateModified\":\"2016-10-26T19:19:18+00:00\",\"author\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/#primaryimage\",\"url\":\"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2016\/10\/streamretrieval-300x167.png\",\"contentUrl\":\"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2016\/10\/streamretrieval-300x167.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.ceessnoek.info\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Video Stream Retrieval of Unseen Queries using Semantic Memory\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.ceessnoek.info\/#website\",\"url\":\"https:\/\/www.ceessnoek.info\/\",\"name\":\"Cees Snoek\",\"description\":\"research on video and image ai\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.ceessnoek.info\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\",\"name\":\"Cees\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"caption\":\"Cees\"},\"sameAs\":[\"http:\/\/www.CeesSnoek.info\"],\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Video Stream Retrieval of Unseen Queries using Semantic Memory - Cees Snoek","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/","og_locale":"en_US","og_type":"article","og_title":"Video Stream Retrieval of Unseen Queries using Semantic Memory - Cees Snoek","og_description":"The BMVC2016 paper Video Stream Retrieval of Unseen Queries using Semantic Memory\u00a0by Spencer Cappallo, Thomas Mensink and Cees Snoek is now available. Retrieval of live, user-broadcast video streams is an under-addressed and increasingly relevant challenge. The on-line nature of the problem requires temporal evaluation and the unforeseeable scope of potential queries motivates an approach which […]","og_url":"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/","og_site_name":"Cees Snoek","article_published_time":"2016-10-26T15:07:37+00:00","article_modified_time":"2016-10-26T19:19:18+00:00","og_image":[{"url":"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2016\/10\/streamretrieval-300x167.png","type":"","width":"","height":""}],"author":"Cees","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Cees","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/","url":"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/","name":"Video Stream Retrieval of Unseen Queries using Semantic Memory - Cees Snoek","isPartOf":{"@id":"https:\/\/www.ceessnoek.info\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/#primaryimage"},"image":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/#primaryimage"},"thumbnailUrl":"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2016\/10\/streamretrieval-300x167.png","datePublished":"2016-10-26T15:07:37+00:00","dateModified":"2016-10-26T19:19:18+00:00","author":{"@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1"},"breadcrumb":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/#primaryimage","url":"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2016\/10\/streamretrieval-300x167.png","contentUrl":"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2016\/10\/streamretrieval-300x167.png"},{"@type":"BreadcrumbList","@id":"https:\/\/www.ceessnoek.info\/index.php\/video-stream-retrieval-of-unseen-queries-using-semantic-memory\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.ceessnoek.info\/"},{"@type":"ListItem","position":2,"name":"Video Stream Retrieval of Unseen Queries using Semantic Memory"}]},{"@type":"WebSite","@id":"https:\/\/www.ceessnoek.info\/#website","url":"https:\/\/www.ceessnoek.info\/","name":"Cees Snoek","description":"research on video and image ai","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.ceessnoek.info\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1","name":"Cees","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","caption":"Cees"},"sameAs":["http:\/\/www.CeesSnoek.info"],"url":"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/1563","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/comments?post=1563"}],"version-history":[{"count":4,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/1563\/revisions"}],"predecessor-version":[{"id":1577,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/1563\/revisions\/1577"}],"wp:attachment":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/media?parent=1563"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/categories?post=1563"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/tags?post=1563"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}