{"id":2158,"date":"2020-03-23T11:39:44","date_gmt":"2020-03-23T10:39:44","guid":{"rendered":"http:\/\/www.ceessnoek.info\/?p=2158"},"modified":"2020-03-30T10:40:52","modified_gmt":"2020-03-30T09:40:52","slug":"cvpr-2-4-actor-transformers-for-group-activity-recognition","status":"publish","type":"post","link":"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/","title":{"rendered":"CVPR 2\/4: Actor-Transformers for Group Activity Recognition"},"content":{"rendered":"\n<p>The CVPR 2020 paper:\u00a0<em>Actor-Transformers for Group Activity Recognition<\/em>\u00a0by <a href=\"https:\/\/kgavrilyuk.github.io\">Kirill Gavrilyuk<\/a>, Ryan Sanford, Mehrsan Javan and Cees Snoek is\u00a0<a href=\"http:\/\/isis-data.science.uva.nl\/cgmsnoek\/pub\/gavrilyuk-transformers-cvpr2020.pdf\">now available<\/a>. This paper strives to recognize individual actions and group activities from videos. While existing solutions for this challenging problem explicitly model spatial and temporal relationships based on location of individual actors, we propose an actor-transformer model able to learn and selectively extract information relevant for group activity recognition. We feed the transformer with rich actor-specific static and dynamic representations expressed by features from a 2D pose network and 3D CNN, respectively. We empirically study different ways to combine these representations and show their complementary benefits. Experiments show what is important to transform and how it should be transformed. What is more, actor-transformers achieve state-of-the-art results on two publicly available benchmarks for group activity recognition, outperforming the previous best published results by a considerable margin. <\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"766\" height=\"448\" src=\"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/03\/gavrilyuk-transformers-cvpr2020.png\" alt=\"\" class=\"wp-image-2159\" srcset=\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/03\/gavrilyuk-transformers-cvpr2020.png 766w, https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/03\/gavrilyuk-transformers-cvpr2020-300x175.png 300w\" sizes=\"auto, (max-width: 766px) 100vw, 766px\" \/><\/figure>\n\n\n\n<p><br><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The CVPR 2020 paper:\u00a0Actor-Transformers for Group Activity Recognition\u00a0by Kirill Gavrilyuk, Ryan Sanford, Mehrsan Javan and Cees Snoek is\u00a0now available. This paper strives to recognize individual actions and group activities from videos. While existing solutions for this challenging problem explicitly model spatial and temporal relationships based on location of individual actors, we propose an actor-transformer model [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-2158","post","type-post","status-publish","format-standard","hentry","category-science"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>CVPR 2\/4: Actor-Transformers for Group Activity Recognition - Cees Snoek<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"CVPR 2\/4: Actor-Transformers for Group Activity Recognition - Cees Snoek\" \/>\n<meta property=\"og:description\" content=\"The CVPR 2020 paper:\u00a0Actor-Transformers for Group Activity Recognition\u00a0by Kirill Gavrilyuk, Ryan Sanford, Mehrsan Javan and Cees Snoek is\u00a0now available. This paper strives to recognize individual actions and group activities from videos. While existing solutions for this challenging problem explicitly model spatial and temporal relationships based on location of individual actors, we propose an actor-transformer model [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/\" \/>\n<meta property=\"og:site_name\" content=\"Cees Snoek\" \/>\n<meta property=\"article:published_time\" content=\"2020-03-23T10:39:44+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-03-30T09:40:52+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/03\/gavrilyuk-transformers-cvpr2020.png\" \/>\n<meta name=\"author\" content=\"Cees\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Cees\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/\",\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/\",\"name\":\"CVPR 2\/4: Actor-Transformers for Group Activity Recognition - Cees Snoek\",\"isPartOf\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/03\/gavrilyuk-transformers-cvpr2020.png\",\"datePublished\":\"2020-03-23T10:39:44+00:00\",\"dateModified\":\"2020-03-30T09:40:52+00:00\",\"author\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/#primaryimage\",\"url\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/03\/gavrilyuk-transformers-cvpr2020.png\",\"contentUrl\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/03\/gavrilyuk-transformers-cvpr2020.png\",\"width\":766,\"height\":448},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.ceessnoek.info\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"CVPR 2\/4: Actor-Transformers for Group Activity Recognition\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.ceessnoek.info\/#website\",\"url\":\"https:\/\/www.ceessnoek.info\/\",\"name\":\"Cees Snoek\",\"description\":\"research on video and image ai\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.ceessnoek.info\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\",\"name\":\"Cees\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"caption\":\"Cees\"},\"sameAs\":[\"http:\/\/www.CeesSnoek.info\"],\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"CVPR 2\/4: Actor-Transformers for Group Activity Recognition - Cees Snoek","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/","og_locale":"en_US","og_type":"article","og_title":"CVPR 2\/4: Actor-Transformers for Group Activity Recognition - Cees Snoek","og_description":"The CVPR 2020 paper:\u00a0Actor-Transformers for Group Activity Recognition\u00a0by Kirill Gavrilyuk, Ryan Sanford, Mehrsan Javan and Cees Snoek is\u00a0now available. This paper strives to recognize individual actions and group activities from videos. While existing solutions for this challenging problem explicitly model spatial and temporal relationships based on location of individual actors, we propose an actor-transformer model [&hellip;]","og_url":"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/","og_site_name":"Cees Snoek","article_published_time":"2020-03-23T10:39:44+00:00","article_modified_time":"2020-03-30T09:40:52+00:00","og_image":[{"url":"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/03\/gavrilyuk-transformers-cvpr2020.png","type":"","width":"","height":""}],"author":"Cees","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Cees","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/","url":"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/","name":"CVPR 2\/4: Actor-Transformers for Group Activity Recognition - Cees Snoek","isPartOf":{"@id":"https:\/\/www.ceessnoek.info\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/#primaryimage"},"image":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/#primaryimage"},"thumbnailUrl":"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/03\/gavrilyuk-transformers-cvpr2020.png","datePublished":"2020-03-23T10:39:44+00:00","dateModified":"2020-03-30T09:40:52+00:00","author":{"@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1"},"breadcrumb":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/#primaryimage","url":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/03\/gavrilyuk-transformers-cvpr2020.png","contentUrl":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/03\/gavrilyuk-transformers-cvpr2020.png","width":766,"height":448},{"@type":"BreadcrumbList","@id":"https:\/\/www.ceessnoek.info\/index.php\/cvpr-2-4-actor-transformers-for-group-activity-recognition\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.ceessnoek.info\/"},{"@type":"ListItem","position":2,"name":"CVPR 2\/4: Actor-Transformers for Group Activity Recognition"}]},{"@type":"WebSite","@id":"https:\/\/www.ceessnoek.info\/#website","url":"https:\/\/www.ceessnoek.info\/","name":"Cees Snoek","description":"research on video and image ai","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.ceessnoek.info\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1","name":"Cees","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","caption":"Cees"},"sameAs":["http:\/\/www.CeesSnoek.info"],"url":"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/2158","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/comments?post=2158"}],"version-history":[{"count":2,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/2158\/revisions"}],"predecessor-version":[{"id":2168,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/2158\/revisions\/2168"}],"wp:attachment":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/media?parent=2158"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/categories?post=2158"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/tags?post=2158"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}