< Back to list

TimeQuestAR: Unfolding Cultural Narratives via Situated Visualizations

conferencePaper

DOI:10.1109/ISMAR-Adjunct60411.2023.00026
Authors: Xu Ningning / Liu Yu / Zhang Zhichao / Yu Lingyun

Extracted Abstract:

information with specific locations, facilitating analytical interactions that could be called “situated analytics” [1]. Moreover, Ens et al. [5] have shown the potential of utilizing physical objects with augmented information, such as tokens and a tabletop display, to enable inter- active data exploration and manipulation for complex collaborative visual analytics. We believe that the further combination of visual- ization and AR techniques would effectively demonstrate cultural features, trends, and differences that play a crucial role in cultural exchange and preservation. Additionally, AR educational games have emerged as a valu- able addition to CH learning, offering a wide range of interactive techniques and devices. The concept of gamification involves incor- porating game design elements such as interface mechanisms and principles into non-game contexts [17]. Interface design patterns and game mechanisms are employed to provide freedom of choice for unbound exploration while also setting restrictions and resource limitations for handheld AR analytics experiences. For example, Ramly and Neupane developed an AR game [19] for exploring ar- tifacts in an onsite museum. Users can acquire knowledge about CH preservation by collecting artifact fragments, collaborating on restoring missing artifacts, participating in virtual gallery design activities as well as contributing to crowdsourced research initiatives. In general terms, game design principles aim at achieving specific goals through rewards. Xu et al. [25] presented a tangible AR in- teraction device that integrated the visualization and gamification elements to trigger the collection and exploration for culture learning and museum gifting. The results confirmed the positive effects of playful interactions combined with visualization features on users’ learning motivation, experience and outcome levels. These find- ings propose that the integration of tangible AR and visualization in educational games has the potential to effectively enhance cultural learning performance. 3 TIMEQUESTAR: A PROTOTYPE OF THEARBOARDGAME Our goal is to design a playful learning tool that enables the public to gain insights into significant cultural places through augmented data visualization. To achieve this, we started the design process by shaping the learning context. First, we conducted the data analysis and processing to extract key information. Then, we formulated game mechanisms based on the data and tasks, and finally, we pre- sented the tacit knowledge into a physical map and a series of event cards, as well as digital game content. The detailed workflow of the prototype design is illustrated in Figure 2. To ensure the educational efficacy of our tool, we established the following learning goals [G]: G1.Experiencing urban development and local culture. G2.Exploring the collective creation and individual emotions. G3. Conveying the interweaving with historical timeline and cul- tural context. G4.Understanding CH preservation and innovation. 3.1 Data Analysis and Processing To extract key information, we first filter the source data, comprised of four datasheets (spot, character, event,andartwork), to exclude Figure 2: The workflow of the prototype design. entries that correspond to the empty keyword in the event sheet. Then, we categorized the analysis tasks into four aspects according to the sheet themes. This process is iterative, involving a gradual refinement of tasks to align with the criteria of being fundamental, pertinent, and comprehensive (refer to Appendix A), followed by a subsequent focus on narrowing the scope. Once this preliminary refinement was completed, we conducted an in-depth examination of the historical information pertaining toJinlingacross various dynasties. This phase equips us to identify key design directions. We mainly use Python to process the data. 3.2 Game Setup We incorporate four types of exploratory tasks in this prototype which link to the learning goals. The visualization elements related to these tasks will be presented in detail with vivid display after scan- ning the tangible interfaces via AR technology. Game mechanisms are illustrated using a flow diagram shown in Appendix B. 3.3 Visualization and Interactive Game Design Regarding the overall design, the game seamlessly integrates situ- ated visualization, tangible objects, and AR technology. The situated visualization superimposes additional visual information adjacent to, above, or around physical landmarks. This technique encourages users to delve deeper into key data, fostering a comprehensive grasp of information, facilitating exploration, and promoting effective communication. Tangible chess pieces and dice enhance interac- tivity and promote user engagement and enjoyment throughout the game process. Additionally, users can augment their experience by scanning the tangible map or AR cards, immersive themselves in a 3D view that provides detailed information, thereby enriching their understanding of cultural data. This integrated approach enhances the overall game experience and ignites a sense of curiosity and motivation for learning. 3.3.1 Physical Design The game’s physical design consists of a map, event cards, character chess pieces, and dice (Figure 1). Users can progress along the map route by rolling dice. Additionally, to delve into cultural knowledge, users are required to interact with event cards and accomplish the tasks they present. To present rich historical and cultural informa- tion, we have incorporated various visual elements to encode the information, as shown in Table 1. The inclusion of spot symbols, such as temples or lakes, allows users to easily identify the geo- graphical categories of different spots. Building upon this, we have 87 Authorized licensed use limited to: TU Wien Bibliothek. Downloaded on October 26,2024 at 15:58:19 UTC from IEEE Xplore. Restrictions apply. Table 1: Visual elements and coded information in map visualization. visual elementdata informationexample color of the connecting linedynasty thickness of the line number of the artworks in each dynasty spot symbolspot type color of the mountainlandscape type number of mountains number of relevant figures number of birds number of relevant artworks integrated other symbols that align with the geographical context (i.e., mountains, birds) to convey additional cultural data, resulting in a coherent and well-designed informative map. In addition, dis- tinct types of game cards also present the amount of tacit knowledge about events, figures and their artworks (examples are shown in Figure 3): - Event cardcontains event themes, sub-events, and related spots. For example, the theme “a forbidding strategic point” consists of six literary events, one activity, and two allusions, statistics by corresponding icons. - Character cardcontains the character’s identity, name, style name, dynasty, year of birth and death, portraits, artworks, and related spots. Heart icons represent the lifespan, while book icons rep- resent the number of artworks, and the card background color is matched to the dynasty connector line on the map. - Artwork cardcontains the artwork’s name, dynasty, genre, author, content, related events, and related spots. Furthermore, the poetic image is presented in the visualization. Figure 3: Physical design of the AR board game. 3.3.2 System Design Challenge-based Learning informed efficient and effective guidance for learning, also known as the challenge learning framework [15]. This framework comprises three interconnected stages:engage, investigate, andact. For system design, here we describe how to integrate the game features and visualization elements into these stages of CH learning [G4]. In theengagestage, we employ a map board and AR cards to motivate and guide users into the game, transforming the abstract knowledge into concrete and actionable challenge tasks (Figure 4a). Movement and interaction are required in the form of rolling dice and trigger the visualized augmented information and task lists via cards and map patterns (Figure 4b and 4c). Users are encouraged to explore various locations on the map, where they can engage with data and acquire cultural knowledge related to each spot [G1]. In theinvestigatestage, users follow the task and information cue, participating in a continuous collection journey to explore the solutions for learning challenges. This process requires the user’s “effort” (rolling the dice or using the jumping point prop reward) on (a)(b) (c)(d) (e)(f) Figure 4: Game design: (a) random event card draw; (b) AR details triggered by moving into a new spot; (c) task lists (left) and informa- tion cue (right); (d) character card acquisition by random drawing; (e) view of AR information and associated prop reward for reaching a jumping point; (f) gallery with artworks and historical figures. (a)(b) Figure 5: Artwork information display: (a) the development of dif- ferent types of artwork in each dynasty; (b) an overview of emotional tones with bubble charts in timeline visualization. moving to some specific locations or getting the chance for drawing the cards to collect the figures or artworks (see Figure 4d and 4e). We employ extra rewards in the gallery at the end of each sub-task to introduce new knowledge points (Figure 4f), such as artwork

Level 1: Include/Exclude

  • Papers must discuss situated information visualization* (by Willet et al.) in the application domain of CH.
    *A situated data representation is a data representation whose physical presentation is located close to the data’s physical referent(s).
    *A situated visualization is a situated data representation for which the presentation is purely visual – and is typically displayed on a screen.
  • Representation must include abstract data (e.g., metadata).
  • Papers focused solely on digital reconstruction without information visualization aspects are excluded.
  • Posters and workshop papers are excluded to focus on mature research contributions.
Show all meta-data