Social-media Photo Appeal (SoPhoAppeal)

• Applicant: Professor Dr.-Ing. Alexander Raake
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Electronic Semiconductors, Components and Circuits, Integrated Systems, Sensor Technology, Theoretical Electrical Engineering
• Term: since 2020
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 437543412

The project "Social-media Photo Appeal" (SoPhoAppeal) addresses the aesthetic appeal of photographic images in the context of social media networks. Today, digital cameras are available in a multitude of devices such as smartphones or dedicated cameras of all types and grades. As a consequence, hundreds of millions of user-generated pictures are published via different social media platforms every day. Beyond textual information, images have become one of the key means for digital social communication. At the same time, images in social media enable a novel systematic form of combined user- and data-driven image aesthetic appeal research.In SoPhoAppeal, the ultimate goal is to extract the contribution of the intrinsic aesthetic quality of a picture to its liking and appreciation. It will be studied how aesthetic appeal relates to the picture properties, the image semantics, the presentation in the social network and the liking and viewing behaviour in the social media photo-sharing platform. To perform such aesthetics-oriented analysis, social media platforms dedicated to photography will primarily be studied, where photos are considered as an art rather than a way to share events of the photographer's life. This type of websites include, for example, 500px (https://500px.com), Flickr (https://www.flickr.com), or 1x (https://1x.com).Such social media sites provide a large amount of direct and indirect information on aspects such as the users' skills in photography, information about what people like, background knowledge on preferences in terms of types of photos liked or taken, a large database of annotated images with semantic information, and further anonymized social-media-related data including the popularity of individual photographers. One of the goals of the project is to study how valuable information about image appeal can be extracted from the large amount of meta-data, and how it can be disambiguated whether the popularity of an image comes from social-media aspects (interrelations, popularity of user, etc.), or from the intrinsic properties of the images. Here, aspects of data anonymization will need to be addressed, too. In addition, we will conduct tests in a controlled environment to collect ground-truth data on aesthetic appeal for a set of anchor images in a lab-viewing context.Moreover, we will conduct crowd-sourcing tests to collect further aesthetic-appeal-related ratings from users closer to their real-life usage contexts. The results will be analyzed in terms of their relation to the liking observed in the social media networks. Using the different data, the relationship between technical aspects, picture properties, background knowledge of the user, social network properties, aesthetic appeal ratings and social media liking will be studied. The results of this analysis will be used to study different models for automatic, machine-learning-based appeal and liking prediction (statistical, deep-learning-based and hybrid).

• DFG Programme: Research Grants