{"id":3888,"date":"2021-08-17T15:26:22","date_gmt":"2021-08-17T14:26:22","guid":{"rendered":"https:\/\/www.ceessnoek.info\/?p=3888"},"modified":"2021-08-17T15:26:36","modified_gmt":"2021-08-17T14:26:36","slug":"iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection","status":"publish","type":"post","link":"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/","title":{"rendered":"ICCV 2021: Social Fabric: Tubelet Compositions for Video Relation Detection"},"content":{"rendered":"\n

The ICCV 2021 paper “Social Fabric: Tubelet Compositions for Video Relation Detection” by Shuo Chen, Zenglin Shi, Pascal Mettes and Cees Snoek is now available<\/a>. This paper strives to classify and detect the relationship between object tubelets appearing within a video as a \u27e8subject-predicate-object\u27e9 triplet. Where existing works treat object proposals or tubelets as single entities and model their relations a posteriori, we propose to classify and detect predicates for pairs of object tubelets a priori. We also propose Social Fabric: an encoding that represents a pair of object tubelets as a composition of interaction primitives. These primitives are learned over all relations, resulting in a compact representation able to localize and classify relations from the pool of co-occurring object tubelets across all timespans in a video. The encoding enables our two-stage network. In the first stage, we train Social Fabric to suggest proposals that are likely interacting. We use the Social Fabric in the second stage to simultaneously fine-tune and predict predicate labels for the tubelets. Experiments demonstrate the benefit of early video relation modeling, our encoding and the two-stage architecture, leading to a new state-of-the-art on two benchmarks. We also show how the encoding enables query-by-primitive-example to search for spatio-temporal video relations. Code: https:\/\/github.com\/shanshuo\/Social-Fabric<\/a>.<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n","protected":false},"excerpt":{"rendered":"

The ICCV 2021 paper “Social Fabric: Tubelet Compositions for Video Relation Detection” by Shuo Chen, Zenglin Shi, Pascal Mettes and Cees Snoek is now available. This paper strives to classify and detect the relationship between object tubelets appearing within a video as a \u27e8subject-predicate-object\u27e9 triplet. Where existing works treat object proposals or tubelets as single […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-3888","post","type-post","status-publish","format-standard","hentry","category-science"],"yoast_head":"\nICCV 2021: Social Fabric: Tubelet Compositions for Video Relation Detection - Cees Snoek<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"ICCV 2021: Social Fabric: Tubelet Compositions for Video Relation Detection - Cees Snoek\" \/>\n<meta property=\"og:description\" content=\"The ICCV 2021 paper “Social Fabric: Tubelet Compositions for Video Relation Detection” by Shuo Chen, Zenglin Shi, Pascal Mettes and Cees Snoek is now available. This paper strives to classify and detect the relationship between object tubelets appearing within a video as a \u27e8subject-predicate-object\u27e9 triplet. Where existing works treat object proposals or tubelets as single […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/\" \/>\n<meta property=\"og:site_name\" content=\"Cees Snoek\" \/>\n<meta property=\"article:published_time\" content=\"2021-08-17T14:26:22+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-08-17T14:26:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/chen-social-fabric-1024x336.png\" \/>\n<meta name=\"author\" content=\"Cees\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Cees\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/\",\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/\",\"name\":\"ICCV 2021: Social Fabric: Tubelet Compositions for Video Relation Detection - Cees Snoek\",\"isPartOf\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/chen-social-fabric-1024x336.png\",\"datePublished\":\"2021-08-17T14:26:22+00:00\",\"dateModified\":\"2021-08-17T14:26:36+00:00\",\"author\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/#primaryimage\",\"url\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/chen-social-fabric.png\",\"contentUrl\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/chen-social-fabric.png\",\"width\":1876,\"height\":616},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.ceessnoek.info\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"ICCV 2021: Social Fabric: Tubelet Compositions for Video Relation Detection\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.ceessnoek.info\/#website\",\"url\":\"https:\/\/www.ceessnoek.info\/\",\"name\":\"Cees Snoek\",\"description\":\"research on video and image ai\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.ceessnoek.info\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\",\"name\":\"Cees\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"caption\":\"Cees\"},\"sameAs\":[\"http:\/\/www.CeesSnoek.info\"],\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"ICCV 2021: Social Fabric: Tubelet Compositions for Video Relation Detection - Cees Snoek","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/","og_locale":"en_US","og_type":"article","og_title":"ICCV 2021: Social Fabric: Tubelet Compositions for Video Relation Detection - Cees Snoek","og_description":"The ICCV 2021 paper “Social Fabric: Tubelet Compositions for Video Relation Detection” by Shuo Chen, Zenglin Shi, Pascal Mettes and Cees Snoek is now available. This paper strives to classify and detect the relationship between object tubelets appearing within a video as a \u27e8subject-predicate-object\u27e9 triplet. Where existing works treat object proposals or tubelets as single […]","og_url":"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/","og_site_name":"Cees Snoek","article_published_time":"2021-08-17T14:26:22+00:00","article_modified_time":"2021-08-17T14:26:36+00:00","og_image":[{"url":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/chen-social-fabric-1024x336.png","type":"","width":"","height":""}],"author":"Cees","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Cees","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/","url":"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/","name":"ICCV 2021: Social Fabric: Tubelet Compositions for Video Relation Detection - Cees Snoek","isPartOf":{"@id":"https:\/\/www.ceessnoek.info\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/#primaryimage"},"image":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/#primaryimage"},"thumbnailUrl":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/chen-social-fabric-1024x336.png","datePublished":"2021-08-17T14:26:22+00:00","dateModified":"2021-08-17T14:26:36+00:00","author":{"@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1"},"breadcrumb":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/#primaryimage","url":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/chen-social-fabric.png","contentUrl":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/chen-social-fabric.png","width":1876,"height":616},{"@type":"BreadcrumbList","@id":"https:\/\/www.ceessnoek.info\/index.php\/iccv-2021-social-fabric-tubelet-compositions-for-video-relation-detection\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.ceessnoek.info\/"},{"@type":"ListItem","position":2,"name":"ICCV 2021: Social Fabric: Tubelet Compositions for Video Relation Detection"}]},{"@type":"WebSite","@id":"https:\/\/www.ceessnoek.info\/#website","url":"https:\/\/www.ceessnoek.info\/","name":"Cees Snoek","description":"research on video and image ai","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.ceessnoek.info\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1","name":"Cees","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","caption":"Cees"},"sameAs":["http:\/\/www.CeesSnoek.info"],"url":"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3888","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/comments?post=3888"}],"version-history":[{"count":1,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3888\/revisions"}],"predecessor-version":[{"id":3889,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3888\/revisions\/3889"}],"wp:attachment":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/media?parent=3888"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/categories?post=3888"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/tags?post=3888"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}