{"id":3373,"date":"2020-08-07T11:17:21","date_gmt":"2020-08-07T10:17:21","guid":{"rendered":"http:\/\/www.ceessnoek.info\/?p=3373"},"modified":"2020-08-07T11:17:46","modified_gmt":"2020-08-07T10:17:46","slug":"learning-to-learn-kernels-with-variational-random-features","status":"publish","type":"post","link":"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/","title":{"rendered":"Learning to Learn Kernels with Variational Random Features"},"content":{"rendered":"\n<p>The ICML 2020 paper <em>Learning to Learn Kernels with Variational Random Features<\/em> by Xiantong Zhen, Haoliang Sun, Yingjun Du, Jun Xu, Yilong Yin, Ling Shaoand Cees Snoek is <a href=\"https:\/\/arxiv.org\/abs\/2006.06707\">now available<\/a>. In this work, we introduce kernels with random Fourier features in the meta-learning framework to leverage their strong few-shot learning ability. We propose meta variational random features (MetaVRF) to learn adaptive kernels for the base-learner, which is developed in a latent variable model by treating the random feature basis as the latent variable. We formulate the optimization of MetaVRF as a variational inference problem by deriving an evidence lower bound under the meta-learning framework. To incorporate shared knowledge from related tasks, we propose a context inference of the posterior, which is established by an LSTM architecture. The LSTM-based inference network can effectively integrate the context information of previous tasks with task-specific information, generating informative and adaptive features. The learned MetaVRF can produce kernels of high representational power with a relatively low spectral sampling rate and also enables fast adaptation to new tasks. Experimental results on a variety of few-shot regression and classification tasks demonstrate that MetaVRF delivers much better, or at least competitive, performance compared to existing meta-learning alternatives.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"571\" src=\"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/08\/meta-learning-kernels-random-features-1024x571.png\" alt=\"\" class=\"wp-image-3375\" srcset=\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/08\/meta-learning-kernels-random-features-1024x571.png 1024w, https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/08\/meta-learning-kernels-random-features-300x167.png 300w, https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/08\/meta-learning-kernels-random-features-768x428.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>The ICML 2020 paper Learning to Learn Kernels with Variational Random Features by Xiantong Zhen, Haoliang Sun, Yingjun Du, Jun Xu, Yilong Yin, Ling Shaoand Cees Snoek is now available. In this work, we introduce kernels with random Fourier features in the meta-learning framework to leverage their strong few-shot learning ability. We propose meta variational [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,1],"tags":[],"class_list":["post-3373","post","type-post","status-publish","format-standard","hentry","category-science","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Learning to Learn Kernels with Variational Random Features - Cees Snoek<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Learning to Learn Kernels with Variational Random Features - Cees Snoek\" \/>\n<meta property=\"og:description\" content=\"The ICML 2020 paper Learning to Learn Kernels with Variational Random Features by Xiantong Zhen, Haoliang Sun, Yingjun Du, Jun Xu, Yilong Yin, Ling Shaoand Cees Snoek is now available. In this work, we introduce kernels with random Fourier features in the meta-learning framework to leverage their strong few-shot learning ability. We propose meta variational [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/\" \/>\n<meta property=\"og:site_name\" content=\"Cees Snoek\" \/>\n<meta property=\"article:published_time\" content=\"2020-08-07T10:17:21+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-08-07T10:17:46+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/08\/meta-learning-kernels-random-features-1024x571.png\" \/>\n<meta name=\"author\" content=\"Cees\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Cees\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/\",\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/\",\"name\":\"Learning to Learn Kernels with Variational Random Features - Cees Snoek\",\"isPartOf\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/08\/meta-learning-kernels-random-features-1024x571.png\",\"datePublished\":\"2020-08-07T10:17:21+00:00\",\"dateModified\":\"2020-08-07T10:17:46+00:00\",\"author\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/#primaryimage\",\"url\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/08\/meta-learning-kernels-random-features.png\",\"contentUrl\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/08\/meta-learning-kernels-random-features.png\",\"width\":2152,\"height\":1200},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.ceessnoek.info\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Learning to Learn Kernels with Variational Random Features\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.ceessnoek.info\/#website\",\"url\":\"https:\/\/www.ceessnoek.info\/\",\"name\":\"Cees Snoek\",\"description\":\"research on video and image ai\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.ceessnoek.info\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\",\"name\":\"Cees\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"caption\":\"Cees\"},\"sameAs\":[\"http:\/\/www.CeesSnoek.info\"],\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Learning to Learn Kernels with Variational Random Features - Cees Snoek","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/","og_locale":"en_US","og_type":"article","og_title":"Learning to Learn Kernels with Variational Random Features - Cees Snoek","og_description":"The ICML 2020 paper Learning to Learn Kernels with Variational Random Features by Xiantong Zhen, Haoliang Sun, Yingjun Du, Jun Xu, Yilong Yin, Ling Shaoand Cees Snoek is now available. In this work, we introduce kernels with random Fourier features in the meta-learning framework to leverage their strong few-shot learning ability. We propose meta variational [&hellip;]","og_url":"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/","og_site_name":"Cees Snoek","article_published_time":"2020-08-07T10:17:21+00:00","article_modified_time":"2020-08-07T10:17:46+00:00","og_image":[{"url":"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/08\/meta-learning-kernels-random-features-1024x571.png","type":"","width":"","height":""}],"author":"Cees","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Cees","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/","url":"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/","name":"Learning to Learn Kernels with Variational Random Features - Cees Snoek","isPartOf":{"@id":"https:\/\/www.ceessnoek.info\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/#primaryimage"},"image":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/#primaryimage"},"thumbnailUrl":"http:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/08\/meta-learning-kernels-random-features-1024x571.png","datePublished":"2020-08-07T10:17:21+00:00","dateModified":"2020-08-07T10:17:46+00:00","author":{"@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1"},"breadcrumb":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/#primaryimage","url":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/08\/meta-learning-kernels-random-features.png","contentUrl":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2020\/08\/meta-learning-kernels-random-features.png","width":2152,"height":1200},{"@type":"BreadcrumbList","@id":"https:\/\/www.ceessnoek.info\/index.php\/learning-to-learn-kernels-with-variational-random-features\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.ceessnoek.info\/"},{"@type":"ListItem","position":2,"name":"Learning to Learn Kernels with Variational Random Features"}]},{"@type":"WebSite","@id":"https:\/\/www.ceessnoek.info\/#website","url":"https:\/\/www.ceessnoek.info\/","name":"Cees Snoek","description":"research on video and image ai","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.ceessnoek.info\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1","name":"Cees","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","caption":"Cees"},"sameAs":["http:\/\/www.CeesSnoek.info"],"url":"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3373","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/comments?post=3373"}],"version-history":[{"count":2,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3373\/revisions"}],"predecessor-version":[{"id":3376,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3373\/revisions\/3376"}],"wp:attachment":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/media?parent=3373"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/categories?post=3373"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/tags?post=3373"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}