DOI:10.1109/VRW58643.2023.00101

Extracted Abstract:

, with complicated structure and inherent correlation. It is an interesting challenge to visually reveal the knowl- edge excavated from the massive data, and make it approachable, explorable, and usable to public audiences and humanities scholars. Over the past decades, cultural institutes have digitized a mas- sive amount of cultural heritage resources such as archaeological sites, buildings, and relics. These intangible resources, known as cultural memory, are attracting more and more attention. These include books, films, photographs, audio, and video recordings that are specified to fit the memories, knowledge, and information of a social group. [24]. Big data technologies including Linked Data, Resource Description Framework (RDF), Semantic Web, etc. are used to uniform, mark, and filter the data. As a result, platforms are 468 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) 979-8-3503-4839-2/23/$31.00 ©2023 IEEE DOI 10.1109/VRW58643.2023.00101 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) | 979-8-3503-4839-2/23/$31.00 ©2023 IEEE | DOI: 10.1109/VRW58643.2023.00101 Authorized licensed use limited to: TU Wien Bibliothek. Downloaded on October 26,2024 at 15:58:57 UTC from IEEE Xplore. Restrictions apply. created to expose information to researchers and the general public. Web-based services such as search engines, websites, and recommen- dation systems are developed for personal exploration and sharing over social media. Excavated data is open to the public and calls for creative usage. 3-dimensional visualization based on e.g. web3d is widely used to interactively visualize individual data sets [26]. The immersive narrative is introduced in the design of public me- dia space to incorporate artistic design, data-driven visualization, visual narration, knowledge exposition, and data communication in immersive environments [21]. With the development of 3D technologies, especially general- purpose GPU and real-time realistic rendering, real-time massive generative model rendering with high-resolution details becomes possible . However, there is still a considerable cost in render- ing large amounts of data-driven models and textures. Meanwhile, designing humanities data visualization [9] requires collaboration among artists, engineers, and humanities scholars, and a rapid pro- totyping and design framework with the capability of stable data extension is required [4]. In this paper, we contribute: • A design framework for massive data visualization, narrative design, generative design, interactive exploration, and artist control. • A novel generative modeling approach that integrates procedu- ral artistic design with data-driven modeling. • A correlation evaluation algorithm is propose to construct event relationship graph from the knowledge bases. • Interactive and immersive installations and applications that provide accessible, explorable, and inter-operable visualization of chronological cultural memory knowledge. 2B ACKGROUND 2.1 Cultural heritage and cultural memory While cultural heritage emphasis on material resources,cultural memory(CM) [2] is typically embedded in intangible resources including books, films, archives, photographs, and audio and video recordings. In 1992, UNESCO launched the Memory of the World Program to protect cultural memories. Since then, there have been continuous efforts to digitize cultural memory resources, and process the data derived from them [1]. In 2011, Shanghai Library started the project Shanghai Memory project to connect dozens of databases and knowledge bases about local chronology and geography, and provide a generalized platform service [6]. This platform provided direct searching services through the original databases of archives, photos, audio and videos, and knowledge bases were constructed on specific topics, such as the Old Film, Historical and Classical Archives, World Expo, etc. 1 . Despite the increasing amount of intangible cultural heritage data, there is a significant shortage of interfaces enabling its access, exploration and usage [26]. 2.2 Digital experience and interfaces After decades of digitization, massive cultural heritage (CH) data are open and available on the web. New types of interactive interfaces based on visualization technology are fostering novel cultural her- itage experiences, including web-based interfaces [14], augmented reality interfaces [20], virtual reality interfaces [27], and immersive installations [11]. The advantage of enhancing cultural heritage expe- rience with Augmented Reality and Virtual Reality lies in improving the quality of knowledge dissemination and enhancing the immer- sive experience of users [5]. An immersive interactive virtual tour application [1] is designed to provide users with an immersive and 1 http://scc.library.shanghai.cn interactive cultural heritage experience, by using 360-degree immer- sive video applications for Head Mounted Display (HMD) devices. Indraprastha et al used AR and HoloLens to provide a promising tool for 3D visualization and experience, wherein computer-generated ob- jects are enhanced into real and physical objects [8]. With these new interactive interfaces, different interaction methods can be used. For example, TombSeer, an augmented reality application, can vividly restore the cultural heritage in the museum into virtual space through gesture interaction. In addition, new and combined multimedia can also enhance the experience of cultural heritage [18]. Koeva et al integrates comprehensive information including high-resolution spherical panorama, various maps, global navigation satellite sys- tems, sound, video, and text information to display the cultural heritage data of spherical or cyclical images [12]. A more generous alternative is proposed: a rich and browsable interface that shows the scale and complexity of the digital heritage collection, including pictures, text, charts, and other forms [25]. 2.3 Data representation and visualization Visualization technology provides an effective way to explore the complex and comprehensive information space of cultural heritage and cultural memory [26]. Visualization methods for cultural her- itage and cultural memory include lists and slideshows, grids and mosaics, hierarchical diagrams, word clouds, charts, plots, clusters and sets, maps, and networks, such as a knowledge map based on paper data [10] Nishanbaev et al. transformed geospatial cultural heritage data into machine-readable and processable RDF data [17]. McKenna et al published the libraries’ metadata as Linked Data (LD) [13]. 3D models have gradually become popular in the field of cultural heritage, enriching the types and formats of data and com- bining spatial and non-spatial data wells, such as MayaArch3D [3]. 3D ESIGN FRAMEWORK Through years of digitization and research by cultural scholars, the General Knowledge Bases(GKB) are created with Big Data tech- nologies and laid the foundations with huge valuable humanities data(see Figure 2. Shanghai Memory is a database extracted and reconstructed from GKB [6]. Over 13000 cultural events are ex- tracted. Each event entity is defined by the title, keywords, influence, key entities and related entities. Key entities include figures and affiliations, geographical sites, locations and architectures, cultural field types and descriptions. Unstructured data are stored with other media archives including videos, audios, books, etc (see Table 1). Prior to the development of visualization systems for Shanghai Memory, we, a group of designers, artists, humanity researchers and software engineers reviewed the related research and design, and set up our design framework. • Immersive: In immersive environment, users construct their spacial imagination and understanding about data via naviga- tion, exploration and interaction. The process is a continuous and personalized experience. • Explorable: Free exploration promotes engagement and arouses interest. • Narrative: Storytelling draws the users to establishing the setting and helps the users understand the navigation logic. • Generative: The scene created is generated from existing database and knowledge base, that are responsive to data cor- rection, knowledge completion, information update and live news information. • Multiscale: Massive data viewing at different scale reveals dif- ferent structures and value. Hierarchical design with integrity and consistency cross scales provides informative experience. • Clear: When visualizing massive data, too much information visualized at one time without balanced design leads to over- whelming cognitive load and failure to deliver information effectively. 469 Authorized licensed use limited to: TU Wien Bibliothek. Downloaded on October 26,2024 at 15:58:57 UTC from IEEE Xplore. Restrictions apply. Figure 2: Design framework and data infrastructure supporting Culture Chronology 4G ENERATIVE DESIGN Based on the proposed framework, we designed and implemented a data visualization and storytelling system: Culture Chronology, exposing the contemporaneous events along a chronically timeline. In this section, we explain the design, the interaction and graphics algorithms. An