{"id":3803,"date":"2021-03-23T14:14:27","date_gmt":"2021-03-23T13:14:27","guid":{"rendered":"https:\/\/www.ceessnoek.info\/?p=3803"},"modified":"2021-03-23T14:15:07","modified_gmt":"2021-03-23T13:15:07","slug":"iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains","status":"publish","type":"post","link":"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/","title":{"rendered":"ICLR 2021: MetaNorm: Learning to Normalize Few-Shot Batches Across Domains"},"content":{"rendered":"\n

The cam-ready of the ICLR 2021 paper “MetaNorm: Learning to Normalize Few-Shot Batches Across Domains<\/em>” by Yingjun Du, Xiantong Zhen, Ling Shao and Cees Snoek is now available<\/a>. Batch normalization plays a crucial role when training deep neural networks. However, batch statistics become unstable with small batch sizes and are unreliable in the presence of distribution shifts. We propose MetaNorm, a simple yet effective meta-learning normalization. It tackles the aforementioned issues in a unified way by leveraging the meta-learning setting and learns to infer adaptive statistics for batch normalization. MetaNorm is generic, flexible and model-agnostic, making it a simple plug-and-play module that is seamlessly embedded into existing meta-learning approaches. It can be efficiently implemented by lightweight hypernetworks with low computational cost. We verify its effectiveness by extensive evaluation on representative tasks suffering from the small batch and domain shift problems: few-shot learning and domain generalization. We further introduce an even more challenging setting: few-shot domain generalization. Results demonstrate that MetaNorm consistently achieves better, or at least competitive, accuracy compared to existing batch normalization methods.<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n","protected":false},"excerpt":{"rendered":"

The cam-ready of the ICLR 2021 paper “MetaNorm: Learning to Normalize Few-Shot Batches Across Domains” by Yingjun Du, Xiantong Zhen, Ling Shao and Cees Snoek is now available. Batch normalization plays a crucial role when training deep neural networks. However, batch statistics become unstable with small batch sizes and are unreliable in the presence of […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-3803","post","type-post","status-publish","format-standard","hentry","category-science"],"yoast_head":"\nICLR 2021: MetaNorm: Learning to Normalize Few-Shot Batches Across Domains - Cees Snoek<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"ICLR 2021: MetaNorm: Learning to Normalize Few-Shot Batches Across Domains - Cees Snoek\" \/>\n<meta property=\"og:description\" content=\"The cam-ready of the ICLR 2021 paper “MetaNorm: Learning to Normalize Few-Shot Batches Across Domains” by Yingjun Du, Xiantong Zhen, Ling Shao and Cees Snoek is now available. Batch normalization plays a crucial role when training deep neural networks. However, batch statistics become unstable with small batch sizes and are unreliable in the presence of […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/\" \/>\n<meta property=\"og:site_name\" content=\"Cees Snoek\" \/>\n<meta property=\"article:published_time\" content=\"2021-03-23T13:14:27+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-03-23T13:15:07+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/03\/du-metanorm-iclr2021-1024x619.png\" \/>\n<meta name=\"author\" content=\"Cees\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Cees\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/\",\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/\",\"name\":\"ICLR 2021: MetaNorm: Learning to Normalize Few-Shot Batches Across Domains - Cees Snoek\",\"isPartOf\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/03\/du-metanorm-iclr2021-1024x619.png\",\"datePublished\":\"2021-03-23T13:14:27+00:00\",\"dateModified\":\"2021-03-23T13:15:07+00:00\",\"author\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/#primaryimage\",\"url\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/03\/du-metanorm-iclr2021.png\",\"contentUrl\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/03\/du-metanorm-iclr2021.png\",\"width\":1518,\"height\":918},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.ceessnoek.info\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"ICLR 2021: MetaNorm: Learning to Normalize Few-Shot Batches Across Domains\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.ceessnoek.info\/#website\",\"url\":\"https:\/\/www.ceessnoek.info\/\",\"name\":\"Cees Snoek\",\"description\":\"research on video and image ai\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.ceessnoek.info\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\",\"name\":\"Cees\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"caption\":\"Cees\"},\"sameAs\":[\"http:\/\/www.CeesSnoek.info\"],\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"ICLR 2021: MetaNorm: Learning to Normalize Few-Shot Batches Across Domains - Cees Snoek","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/","og_locale":"en_US","og_type":"article","og_title":"ICLR 2021: MetaNorm: Learning to Normalize Few-Shot Batches Across Domains - Cees Snoek","og_description":"The cam-ready of the ICLR 2021 paper “MetaNorm: Learning to Normalize Few-Shot Batches Across Domains” by Yingjun Du, Xiantong Zhen, Ling Shao and Cees Snoek is now available. Batch normalization plays a crucial role when training deep neural networks. However, batch statistics become unstable with small batch sizes and are unreliable in the presence of […]","og_url":"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/","og_site_name":"Cees Snoek","article_published_time":"2021-03-23T13:14:27+00:00","article_modified_time":"2021-03-23T13:15:07+00:00","og_image":[{"url":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/03\/du-metanorm-iclr2021-1024x619.png","type":"","width":"","height":""}],"author":"Cees","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Cees","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/","url":"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/","name":"ICLR 2021: MetaNorm: Learning to Normalize Few-Shot Batches Across Domains - Cees Snoek","isPartOf":{"@id":"https:\/\/www.ceessnoek.info\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/#primaryimage"},"image":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/#primaryimage"},"thumbnailUrl":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/03\/du-metanorm-iclr2021-1024x619.png","datePublished":"2021-03-23T13:14:27+00:00","dateModified":"2021-03-23T13:15:07+00:00","author":{"@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1"},"breadcrumb":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/#primaryimage","url":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/03\/du-metanorm-iclr2021.png","contentUrl":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/03\/du-metanorm-iclr2021.png","width":1518,"height":918},{"@type":"BreadcrumbList","@id":"https:\/\/www.ceessnoek.info\/index.php\/iclr-2021-metanorm-learning-to-normalize-few-shot-batches-across-domains\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.ceessnoek.info\/"},{"@type":"ListItem","position":2,"name":"ICLR 2021: MetaNorm: Learning to Normalize Few-Shot Batches Across Domains"}]},{"@type":"WebSite","@id":"https:\/\/www.ceessnoek.info\/#website","url":"https:\/\/www.ceessnoek.info\/","name":"Cees Snoek","description":"research on video and image ai","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.ceessnoek.info\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1","name":"Cees","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","caption":"Cees"},"sameAs":["http:\/\/www.CeesSnoek.info"],"url":"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3803","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/comments?post=3803"}],"version-history":[{"count":1,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3803\/revisions"}],"predecessor-version":[{"id":3805,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3803\/revisions\/3805"}],"wp:attachment":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/media?parent=3803"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/categories?post=3803"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/tags?post=3803"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}