< Back to list
Extracted Abstract:
Recently, there has been an upsurge of activity in image-based non-photorealistic rendering (NPR), and in particular portrait image stylisation, due to the advent of neural style transfer (NST). However, the state of performance evaluation in this field is poor, especially compared to the norms in the computer vision and machine learning communities. Unfortunately, the task of evaluating image stylisation is thus far not well defined, since it involves subjective, perceptual, and aesthetic aspects. To make progress towards a solution, this paper proposes a new structured, three- level, benchmark dataset for the evaluation of stylised 1School of Computer Science and Informatics, Cardiff University, Cardiff, UK. E-mail: P. L. Rosin, RosinPL@ cardiff.ac.uk ( ); Y.-K. Lai, Yukun.Lai@cs.cardiff.ac.uk. 2 School of Computer Science, Carleton University, Ottawa, Canada. E-mail: D. Mould, mould@scs.carleton.ca; L. Doyle, larsdoyle@cmail.carleton.ca. 3Department of Computer Science and Technology, Tsinghua University, Beijing, China. E-mail: R. Yi, yr16@ mails.tsinghua.edu.cn; Y.-J. Liu, liuyongjin@tsinghua.edu.cn. 4 Reichman University (the Interdisciplinary Center), Herzliya, Israel. E-mail: I. Berger, berger.itamar@gmail.com; A. Shamir, arik@idc.ac.il. 5Department of Computer Science and Engineering, Pohang University of Science and Technology, Pohang, Republic of Korea. E-mail: leesy@postech.ac.kr. 6Lambda Labs, Inc., San Francisco, USA. E-mail: c@lambdal.com. 7 Hasso Plattner Institute, University of Potsdam, Potsdam, Germany. E-mail: Amir.Semmo@hpi.de. 8Multimedia Processing Laboratory, Samsung Advanced Institute of Technology, Suwon, Republic of Korea. E-mail: minjungs.son@samsung.com. 9Adobe Systems, Inc., San Jose, USA. E-mail: hwinnemo@adobe.com. Manuscript received: 2021-07-17; accepted: 2021-09-16 portrait images. Rigorous criteria were used for its construction, and its consistency was validated by user studies. Moreover, a new methodology has been developed for evaluating portrait stylisation algorithms, which makes use of the different benchmark levels as well as annotations provided by user studies regarding the characteristics of the faces. We perform evaluation for a wide variety of image stylisation methods (both portrait-specific and general purpose, and also both traditional NPR approaches and NST) using the new benchmark dataset. Keywordsnon-photorealistic rendering (NPR); image stylization;styletransfer;portrait; evaluation; benchmark 1