{"id":4119,"date":"2022-10-16T12:00:28","date_gmt":"2022-10-16T11:00:28","guid":{"rendered":"https:\/\/www.ceessnoek.info\/?p=4119"},"modified":"2022-10-16T12:00:42","modified_gmt":"2022-10-16T11:00:42","slug":"eccv-2022-less-than-few-self-shot-video-instance-segmentation","status":"publish","type":"post","link":"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/","title":{"rendered":"ECCV 2022: Less than Few: Self-Shot Video Instance Segmentation"},"content":{"rendered":"\n

The ECCV 2022 paper by entitled: “Less than Few: Self-Shot Video Instance Segmentation<\/em>” by Pengwan Yang, Yuki M Asano, Pascal Mettes and Cees Snoek is now available<\/a>. The goal of this paper is to bypass the need for labelled examples in few-shot video understanding at run time. While proven effective, in many practical video settings even labelling a few examples appears unrealistic. This is especially true as the level of details in spatio-temporal video understanding and with it, the complexity of annotations continues to increase. Rather than performing few-shot learning with a human oracle to provide a few densely labelled support videos, we propose to automatically learn to find appropriate support videos given a query. We call this self-shot learning and we outline a simple self-supervised learning method to generate an embedding space well-suited for unsupervised retrieval of relevant samples. To showcase this novel setting, we tackle, for the first time, video instance segmentation in a self-shot (and few-shot) setting, where the goal is to segment instances at the pixel-level across the spatial and temporal domains. We provide strong baseline performances that utilize a novel transformer-based model and show that self-shot learning can even surpass few-shot and can be positively combined for further performance gains. Experiments on new benchmarks show that our approach achieves strong performance, is competitive to oracle support in some settings, scales to large unlabelled video collections, and can be combined in a semi-supervised setting.<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n","protected":false},"excerpt":{"rendered":"

The ECCV 2022 paper by entitled: “Less than Few: Self-Shot Video Instance Segmentation” by Pengwan Yang, Yuki M Asano, Pascal Mettes and Cees Snoek is now available. The goal of this paper is to bypass the need for labelled examples in few-shot video understanding at run time. While proven effective, in many practical video settings […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-4119","post","type-post","status-publish","format-standard","hentry","category-science"],"yoast_head":"\nECCV 2022: Less than Few: Self-Shot Video Instance Segmentation - Cees Snoek<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"ECCV 2022: Less than Few: Self-Shot Video Instance Segmentation - Cees Snoek\" \/>\n<meta property=\"og:description\" content=\"The ECCV 2022 paper by entitled: “Less than Few: Self-Shot Video Instance Segmentation” by Pengwan Yang, Yuki M Asano, Pascal Mettes and Cees Snoek is now available. The goal of this paper is to bypass the need for labelled examples in few-shot video understanding at run time. While proven effective, in many practical video settings […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/\" \/>\n<meta property=\"og:site_name\" content=\"Cees Snoek\" \/>\n<meta property=\"article:published_time\" content=\"2022-10-16T11:00:28+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-10-16T11:00:42+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2022\/04\/pengwan-self-shot.png\" \/>\n<meta name=\"author\" content=\"Cees\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Cees\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/\",\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/\",\"name\":\"ECCV 2022: Less than Few: Self-Shot Video Instance Segmentation - Cees Snoek\",\"isPartOf\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2022\/04\/pengwan-self-shot.png\",\"datePublished\":\"2022-10-16T11:00:28+00:00\",\"dateModified\":\"2022-10-16T11:00:42+00:00\",\"author\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/#primaryimage\",\"url\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2022\/04\/pengwan-self-shot.png\",\"contentUrl\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2022\/04\/pengwan-self-shot.png\",\"width\":1314,\"height\":480},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.ceessnoek.info\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"ECCV 2022: Less than Few: Self-Shot Video Instance Segmentation\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.ceessnoek.info\/#website\",\"url\":\"https:\/\/www.ceessnoek.info\/\",\"name\":\"Cees Snoek\",\"description\":\"research on video and image ai\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.ceessnoek.info\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\",\"name\":\"Cees\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"caption\":\"Cees\"},\"sameAs\":[\"http:\/\/www.CeesSnoek.info\"],\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"ECCV 2022: Less than Few: Self-Shot Video Instance Segmentation - Cees Snoek","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/","og_locale":"en_US","og_type":"article","og_title":"ECCV 2022: Less than Few: Self-Shot Video Instance Segmentation - Cees Snoek","og_description":"The ECCV 2022 paper by entitled: “Less than Few: Self-Shot Video Instance Segmentation” by Pengwan Yang, Yuki M Asano, Pascal Mettes and Cees Snoek is now available. The goal of this paper is to bypass the need for labelled examples in few-shot video understanding at run time. While proven effective, in many practical video settings […]","og_url":"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/","og_site_name":"Cees Snoek","article_published_time":"2022-10-16T11:00:28+00:00","article_modified_time":"2022-10-16T11:00:42+00:00","og_image":[{"url":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2022\/04\/pengwan-self-shot.png","type":"","width":"","height":""}],"author":"Cees","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Cees","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/","url":"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/","name":"ECCV 2022: Less than Few: Self-Shot Video Instance Segmentation - Cees Snoek","isPartOf":{"@id":"https:\/\/www.ceessnoek.info\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/#primaryimage"},"image":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/#primaryimage"},"thumbnailUrl":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2022\/04\/pengwan-self-shot.png","datePublished":"2022-10-16T11:00:28+00:00","dateModified":"2022-10-16T11:00:42+00:00","author":{"@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1"},"breadcrumb":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/#primaryimage","url":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2022\/04\/pengwan-self-shot.png","contentUrl":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2022\/04\/pengwan-self-shot.png","width":1314,"height":480},{"@type":"BreadcrumbList","@id":"https:\/\/www.ceessnoek.info\/index.php\/eccv-2022-less-than-few-self-shot-video-instance-segmentation\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.ceessnoek.info\/"},{"@type":"ListItem","position":2,"name":"ECCV 2022: Less than Few: Self-Shot Video Instance Segmentation"}]},{"@type":"WebSite","@id":"https:\/\/www.ceessnoek.info\/#website","url":"https:\/\/www.ceessnoek.info\/","name":"Cees Snoek","description":"research on video and image ai","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.ceessnoek.info\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1","name":"Cees","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","caption":"Cees"},"sameAs":["http:\/\/www.CeesSnoek.info"],"url":"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/4119","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/comments?post=4119"}],"version-history":[{"count":1,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/4119\/revisions"}],"predecessor-version":[{"id":4120,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/4119\/revisions\/4120"}],"wp:attachment":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/media?parent=4119"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/categories?post=4119"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/tags?post=4119"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}