{"id":3925,"date":"2021-10-24T12:20:02","date_gmt":"2021-10-24T11:20:02","guid":{"rendered":"https:\/\/www.ceessnoek.info\/?p=3925"},"modified":"2021-10-24T12:21:09","modified_gmt":"2021-10-24T11:21:09","slug":"bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias","status":"publish","type":"post","link":"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/","title":{"rendered":"BMVC 2021: Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias"},"content":{"rendered":"\n

The BMVC 2021 paper Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias<\/em> by William Thong<\/a> and Cees Snoek is now available<\/a>. This paper strives to address image classifier bias, with a focus on both feature and label embedding spaces. Previous works have shown that spurious correlations from protected attributes, such as age, gender, or skin tone, can cause adverse decisions. To balance potential harms, there is a growing need to identify and mitigate image classifier bias. First, we identify in the feature space a bias direction. We compute class prototypes of each protected attribute value for every class, and reveal an existing subspace that captures the maximum variance of the bias. Second, we mitigate biases by mapping image inputs to label embedding spaces. Each value of the protected attribute has its projection head where classes are embedded through a latent vector representation rather than a common one-hot encoding. Once trained, we further reduce in the feature space the bias effect by removing its direction. Evaluation on biased image datasets, for multi-class, multi-label and binary classifications, shows the effectiveness of tackling both feature and label embedding spaces in improving the fairness of the classifier predictions, while preserving classification performance.<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n","protected":false},"excerpt":{"rendered":"

The BMVC 2021 paper Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias by William Thong and Cees Snoek is now available. This paper strives to address image classifier bias, with a focus on both feature and label embedding spaces. Previous works have shown that spurious correlations from protected attributes, such as age, […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-3925","post","type-post","status-publish","format-standard","hentry","category-science"],"yoast_head":"\nBMVC 2021: Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias - Cees Snoek<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"BMVC 2021: Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias - Cees Snoek\" \/>\n<meta property=\"og:description\" content=\"The BMVC 2021 paper Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias by William Thong and Cees Snoek is now available. This paper strives to address image classifier bias, with a focus on both feature and label embedding spaces. Previous works have shown that spurious correlations from protected attributes, such as age, […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/\" \/>\n<meta property=\"og:site_name\" content=\"Cees Snoek\" \/>\n<meta property=\"article:published_time\" content=\"2021-10-24T11:20:02+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-10-24T11:21:09+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/10\/thong-bias.png\" \/>\n<meta name=\"author\" content=\"Cees\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Cees\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/\",\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/\",\"name\":\"BMVC 2021: Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias - Cees Snoek\",\"isPartOf\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/10\/thong-bias.png\",\"datePublished\":\"2021-10-24T11:20:02+00:00\",\"dateModified\":\"2021-10-24T11:21:09+00:00\",\"author\":{\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/#primaryimage\",\"url\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/10\/thong-bias.png\",\"contentUrl\":\"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/10\/thong-bias.png\",\"width\":1070,\"height\":636},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.ceessnoek.info\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"BMVC 2021: Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.ceessnoek.info\/#website\",\"url\":\"https:\/\/www.ceessnoek.info\/\",\"name\":\"Cees Snoek\",\"description\":\"research on video and image ai\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.ceessnoek.info\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1\",\"name\":\"Cees\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g\",\"caption\":\"Cees\"},\"sameAs\":[\"http:\/\/www.CeesSnoek.info\"],\"url\":\"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"BMVC 2021: Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias - Cees Snoek","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/","og_locale":"en_US","og_type":"article","og_title":"BMVC 2021: Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias - Cees Snoek","og_description":"The BMVC 2021 paper Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias by William Thong and Cees Snoek is now available. This paper strives to address image classifier bias, with a focus on both feature and label embedding spaces. Previous works have shown that spurious correlations from protected attributes, such as age, […]","og_url":"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/","og_site_name":"Cees Snoek","article_published_time":"2021-10-24T11:20:02+00:00","article_modified_time":"2021-10-24T11:21:09+00:00","og_image":[{"url":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/10\/thong-bias.png","type":"","width":"","height":""}],"author":"Cees","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Cees","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/","url":"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/","name":"BMVC 2021: Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias - Cees Snoek","isPartOf":{"@id":"https:\/\/www.ceessnoek.info\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/#primaryimage"},"image":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/#primaryimage"},"thumbnailUrl":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/10\/thong-bias.png","datePublished":"2021-10-24T11:20:02+00:00","dateModified":"2021-10-24T11:21:09+00:00","author":{"@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1"},"breadcrumb":{"@id":"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/#primaryimage","url":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/10\/thong-bias.png","contentUrl":"https:\/\/www.ceessnoek.info\/wp-content\/uploads\/2021\/10\/thong-bias.png","width":1070,"height":636},{"@type":"BreadcrumbList","@id":"https:\/\/www.ceessnoek.info\/index.php\/bmvc-2021-feature-and-label-embedding-spaces-matter-in-addressing-image-classifier-bias\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.ceessnoek.info\/"},{"@type":"ListItem","position":2,"name":"BMVC 2021: Feature and Label Embedding Spaces Matter in Addressing Image Classifier Bias"}]},{"@type":"WebSite","@id":"https:\/\/www.ceessnoek.info\/#website","url":"https:\/\/www.ceessnoek.info\/","name":"Cees Snoek","description":"research on video and image ai","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.ceessnoek.info\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/4bca975b7c432aeb5dced40bdbc204c1","name":"Cees","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ceessnoek.info\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/756ccb993852c1e8e3af39a228d11a7305b2a937750f26dc5799d5df019b0f51?s=96&d=mm&r=g","caption":"Cees"},"sameAs":["http:\/\/www.CeesSnoek.info"],"url":"https:\/\/www.ceessnoek.info\/index.php\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3925","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/comments?post=3925"}],"version-history":[{"count":1,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3925\/revisions"}],"predecessor-version":[{"id":3926,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/posts\/3925\/revisions\/3926"}],"wp:attachment":[{"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/media?parent=3925"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/categories?post=3925"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ceessnoek.info\/index.php\/wp-json\/wp\/v2\/tags?post=3925"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}