{"id":3947,"date":"2021-12-27T14:37:34","date_gmt":"2021-12-27T13:37:34","guid":{"rendered":"https:\/\/www.ceessnoek.info\/?p=3947"},"modified":"2021-12-27T14:38:22","modified_gmt":"2021-12-27T13:38:22","slug":"ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior","status":"publish","type":"post","link":"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/","title":{"rendered":"IJCV: On Measuring and Controlling the Spectral Bias of the Deep Image Prior"},"content":{"rendered":"\n

The IJCV paper entitled On Measuring and Controlling the Spectral Bias of the Deep Image Prior<\/em>, by Zenglin Shi<\/a>, Pascal Mettes, Subhransu Maji and Cees Snoek is now available<\/a>. The deep image prior showed that a randomly initialized network with a suitable architecture can be trained to solve inverse imaging problems by simply optimizing it’s parameters to reconstruct a single degraded image. However, it suffers from two practical limitations. First, it remains unclear how to control the prior beyond the choice of the network architecture. Second, training requires an oracle stopping criterion as during the optimization the performance degrades after reaching an optimum value. To address these challenges we introduce a frequency-band correspondence measure to characterize the spectral bias of the deep image prior, where low-frequency image signals are learned faster and better than high-frequency counterparts. Based on our observations, we propose techniques to prevent the eventual performance degradation and accelerate convergence. We introduce a Lipschitz-controlled convolution layer and a Gaussian-controlled upsampling layer as plug-in replacements for layers used in the deep architectures. The experiments show that with these changes the performance does not degrade during optimization, relieving us from the need for an oracle stopping criterion. We further outline a stopping criterion to avoid superfluous computation. Finally, we show that our approach obtains favorable results compared to current approaches across various denoising, deblocking, inpainting, super-resolution and detail enhancement tasks. Code is available at https:\/\/github.com\/shizenglin\/Measure-and-Control-Spectral-Bias<\/a>.<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n","protected":false},"excerpt":{"rendered":"

The IJCV paper entitled On Measuring and Controlling the Spectral Bias of the Deep Image Prior, by Zenglin Shi, Pascal Mettes, Subhransu Maji and Cees Snoek is now available. The deep image prior showed that a randomly initialized network with a suitable architecture can be trained to solve inverse imaging problems by simply optimizing it’s […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-3947","post","type-post","status-publish","format-standard","hentry","category-science"],"yoast_head":"\nIJCV: On Measuring and Controlling the Spectral Bias of the Deep Image Prior - Cees Snoek<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"IJCV: On Measuring and Controlling the Spectral Bias of the Deep Image Prior - Cees Snoek\" \/>\n<meta property=\"og:description\" content=\"The IJCV paper entitled On Measuring and Controlling the Spectral Bias of the Deep Image Prior, by Zenglin Shi, Pascal Mettes, Subhransu Maji and Cees Snoek is now available. The deep image prior showed that a randomly initialized network with a suitable architecture can be trained to solve inverse imaging problems by simply optimizing it’s […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/\" \/>\n<meta property=\"og:site_name\" content=\"Cees Snoek\" \/>\n<meta property=\"article:published_time\" content=\"2021-12-27T13:37:34+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-12-27T13:38:22+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/shi-dip-spectral.png\" \/>\n<meta name=\"author\" content=\"Cees\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Cees\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/\",\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/\",\"name\":\"IJCV: On Measuring and Controlling the Spectral Bias of the Deep Image Prior - Cees Snoek\",\"isPartOf\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/shi-dip-spectral.png\",\"datePublished\":\"2021-12-27T13:37:34+00:00\",\"dateModified\":\"2021-12-27T13:38:22+00:00\",\"author\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/#primaryimage\",\"url\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/shi-dip-spectral.png\",\"contentUrl\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/shi-dip-spectral.png\",\"width\":648,\"height\":550},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.ceessnoek.info\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"IJCV: On Measuring and Controlling the Spectral Bias of the Deep Image Prior\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.ceessnoek.info\/#website\",\"url\":\"https:\/\/www.ceessnoek.info\/\",\"name\":\"Cees Snoek\",\"description\":\"research on video and image ai\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.ceessnoek.info\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\",\"name\":\"Cees\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"caption\":\"Cees\"},\"sameAs\":[\"http:\/\/www.CeesSnoek.info\"],\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"IJCV: On Measuring and Controlling the Spectral Bias of the Deep Image Prior - Cees Snoek","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/","og_locale":"en_US","og_type":"article","og_title":"IJCV: On Measuring and Controlling the Spectral Bias of the Deep Image Prior - Cees Snoek","og_description":"The IJCV paper entitled On Measuring and Controlling the Spectral Bias of the Deep Image Prior, by Zenglin Shi, Pascal Mettes, Subhransu Maji and Cees Snoek is now available. The deep image prior showed that a randomly initialized network with a suitable architecture can be trained to solve inverse imaging problems by simply optimizing it’s […]","og_url":"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/","og_site_name":"Cees Snoek","article_published_time":"2021-12-27T13:37:34+00:00","article_modified_time":"2021-12-27T13:38:22+00:00","og_image":[{"url":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/shi-dip-spectral.png","type":"","width":"","height":""}],"author":"Cees","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Cees","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/","url":"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/","name":"IJCV: On Measuring and Controlling the Spectral Bias of the Deep Image Prior - Cees Snoek","isPartOf":{"@id":"https:\/\/www.ceessnoek.info\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/#primaryimage"},"image":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/#primaryimage"},"thumbnailUrl":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/shi-dip-spectral.png","datePublished":"2021-12-27T13:37:34+00:00","dateModified":"2021-12-27T13:38:22+00:00","author":{"@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1"},"breadcrumb":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/#primaryimage","url":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/shi-dip-spectral.png","contentUrl":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/07\/shi-dip-spectral.png","width":648,"height":550},{"@type":"BreadcrumbList","@id":"https:\/\/www.ceessnoek.info\/index.php\/ijcv-on-measuring-and-controlling-the-spectral-bias-of-the-deep-image-prior\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.ceessnoek.info\/"},{"@type":"ListItem","position":2,"name":"IJCV: On Measuring and Controlling the Spectral Bias of the Deep Image Prior"}]},{"@type":"WebSite","@id":"https:\/\/www.ceessnoek.info\/#website","url":"https:\/\/www.ceessnoek.info\/","name":"Cees Snoek","description":"research on video and image ai","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.ceessnoek.info\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1","name":"Cees","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","caption":"Cees"},"sameAs":["http:\/\/www.CeesSnoek.info"],"url":"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3947","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/comments?post=3947"}],"version-history":[{"count":1,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3947\/revisions"}],"predecessor-version":[{"id":3948,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3947\/revisions\/3948"}],"wp:attachment":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/media?parent=3947"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/categories?post=3947"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/tags?post=3947"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}