image2emoji

The ACM Multimedia paper Image2Emoji: Zero-shot Emoji Prediction for Visual Media by Spencer Cappallo, Thomas Mensink, and Cees Snoek is now available. We present Image2Emoji, a multi-modal approach for generating emoji labels for an image in a zero-shot manner. Different from existing zero-shot image-to-text approaches, we exploit both image and textual media to learn a semantic embedding for the new task of emoji prediction. We propose that the widespread adoption of emoji suggests a semantic universality which is well-suited for interaction with visual media. We quantify the efficacy of our proposed model on the MSCOCO dataset, and demonstrate the value of visual, textual and multi-modal prediction of emoji. We conclude the paper with three examples of the application potential of emoji in the context of multimedia retrieval.

Leave a Reply

Your email address will not be published. Required fields are marked *