This demo paper presents a locative, site-specific, augmented-reality game for a WW1 historical site. The work constitutes the preliminary results of a research collaboration between two universities, one museum and a developer of serious games. The challenge was to produce an interactive, playful and educational digital experience based on the results of research into national policy and other cultural forces in relation to food and nutrition in Denmark during WW1. The resulting game needed to have a high technology readiness level and be deployed in the historical site for use by visitors. The research collaboration produced several innovations in location-based augmented-reality games for cultural heritage that are likely to be of interest to researchers and developers working in this space: (a) a way to present complex content that requires multiple perspectives in a geolocatable double structure; (b) the use of a visual aesthetic that resonates highly with the historical period in question in order to encourage reflection that relates the past to the present; (c) an approach to adding supplementary historical information in a fashion that aims not to overload the player with information during the play experience. While we have yet to evaluate the work through a user trial, this demo paper presents our design motivations and solutions that arose from the collaboration and the complex historical material.
INCLUDE
This paper focuses on the design parameters of mobile electronic guides for cultural sites and museums. We treat issues like user interaction and portability and present practical solutions for their installation and operation in uncontrolled indoor and outdoor environments. Initial experiences from the archaeological site of Pompeii, Italy, are reported. The major contribution of the paper is the proposal of a new concept, that of "light" Augmented Reality for portable guides which we believe will dominate the field of cultural guiding in the near future. Our implementation aims at setting the standards for such applications and initial qualitative results are presented from its test use.
INCLUDE
We describe a prototype system that combines the overlaid 3D graphics of augmented reality with the untethered freedom of mobile computing. The goal is to explore how these two technologies might together make possible wearable computer systems that can support users in their everyday interactions with the world. We introduce an application that presents information about our university's campus, using a head-tracked, see-through, head-worn, 3D display, and an untracked, opaque, hand-held, 2D display with stylus and trackpad. We provide an illustrated explanation of how our prototype is used, and describe our rationale behind designing its software infrastructure and selecting the hardware on which it runs.
INCLUDE
In the project SPIRIT, we developed a playable prototype of location-based Augmented Reality storytelling for museums. In order to develop a full experience, several aspects have been tackled by an interdisciplinary team, which are described here: the SPIRIT app catering to special location-based experiences, the formalized content structure STARML supporting authoring, a plot engine managing content based on STARML, integrated concepts for interaction as well as interactive storytelling. The result is an extensible system that constrains the design for a specialized experience. The paper describes each of the aspects and discusses potential extensions of the system for further application.
INCLUDE
INCLUDE
Can Augmented Reality (AR) techniques inform the design and implementation of a mobile multimedia guide for the museum setting? Drawing from our experience both on previous mobile museum guides projects and in AR technology, we present a fully functional prototype of an AR-enabled mobile multimedia museum guide, designed and implemented for the Museum of Fine Arts in Rennes, France. We report on the life cycle of the prototype and the methodology employed for the AR approach as well as on the selected mixed method evaluation process; finally, the first results emerging from quantitative evaluation are discussed, supported by evidence and findings from the qualitative part of the assessment process. We conclude with lessons learned during the full circle of conception, implementation, testing and assessment of the guide.
INCLUDE
An Augmented Reality (AR) prototype, CubeMuseum, is proposed in this paper to present an embodied experience with virtual museum collections. With a cost-effective cube and a smartphone application, users can view and interact with 3D museum objects embodied on the cube. Detailed design of the prototype is presented to illustrate the approaches to visualize, present, and interact with virtual objects. CubeMuseum has been evaluated by hundreds of users in both laboratory studies and public exhibitions. The results indicated that the prototype is simple yet effective. It demonstrates several benefits and potential implications in supporting user engagement and learning experience. This research provides insights to researchers and practitioners in designing interactive cultural heritage experiences using a cost-effective approach.
INCLUDE
Cultural heritage institutions store and manage large volumes of historical information, that have great material and humanitarian value. Their means of data organization, often obsolete, hinders the dissemination and reuse of information to be made in an effective way. In this sense, an application of Linked Open Data (LOD) technology is the possibility to extend the knowledge of a collection, with the use of open data already consolidated on the web to describe artworks or any type of physical object. This paper introduces a generic architecture based on an Extract, Transform, Load (ETL) methodology that connects LOD data from cultural heritage available on the web, generates descriptive content for physical objects and provides the mapping of research institutes engaged in studying them, highlighting their collaboration network. For instance, an implementation of the architecture was dedicated to the domain of Botany and can assist herbarium managers in creating exhibitions about medicinal plants. In this paper, we present a software architecture that provides an automatic method for creating dynamic pages from data stored in interconnected databases, and an application, which supports visitation systems with visualization and interaction mechanisms to encourage visitor learning. Keywords— Cultural Heritage; Linked Open Data; Georeferencing; Botany; RDFa. I.
INCLUDE
Abstract not found.
INCLUDE
In the SPIRIT research project, a location-based Augmented Reality (AR) storytelling application has been developed with the goal to support the imagination of lively historical events at places of cultural significance. We describe a showcase scenario and report on its quantitative and qualitative evaluation, conducted at the Saalburg Roman fort, an outdoor museum site near Bad Homburg in Germany. 107 random voluntary visitors were observed using the app, before filling questionnaires that were then analyzed with SPSS. Specifics of the app include a novel interaction pattern that uses positioning sensors of mobile devices and image recognition to trigger content, featuring transparent videos as ghost-like overlays on the camera image of the environment. Results presented in this paper show that in general, the app was effective and fun to use. Further, there have been differences in the experience of presence concerning the AR representation, as well as in the comprehension and appreciation of the story’s content. Concluding, we discuss influencing parameters on the results and draft hypotheses for future work.
INCLUDE
explorAR is a project that provides a new experience to learn the world of the past by exploring mixed reality with your phone. In this interactive experience, users engage with the museum and with each other by collecting artifacts which include fossils, paintings, statues, and other historical objects. Users will learn how to preserve historical objects by extracting fragments of artifacts, how to collaborate with each other by combining fragments of missing artifacts, how to express their creativity by designing their own virtual gallery, and how to participate in a crowdsourced research. We developed the concept using human-centered design approaches which includes interviews, personas, prototypes, and user testing.
INCLUDE
The Book of Songs is regarded as the origin of Chinese literature and has a prolonged impact on Chinese culture, aesthetics, and morality. In this work, we have analyzed the 305 poems in The Book of Songs from different dimensions. We aim to learn how various poetic imageries connect abstract themes and subjective emotions at the micro level, and how the poems connect people today and ancestors to understand the universal, everlasting, and poetical human lives at the macro level. Authors Keywords Digital Humanities; Cultural Heritage; Data Visualization; Data Art;
INCLUDE
We describe an experimental mobile augmented reality system (MARS) testbed that employs different user interfaces to allow outdoor and indoor users to access and manage information that is spatially registered with the real world. Outdoor users can experience spatialized multimedia presentations that are presented on a head-tracked, see-through, head-worn display used in conjunction with a hand-held pen-based computer. Indoor users can get an overview of the outdoor scene and communicate with outdoor users through a desktop user interface or a head- and hand-tracked immersive augmented reality user interface.
INCLUDE
Abstract This paper discusses the rationale for—and describes the methodology of—a new system of interactive storytelling being developed by the Ename Centre for Public Archaeology Heritage Presentation in Belgium. Based on 5 years' experience with multimedia heritage presentation systems, both on‐site and in museum contexts, this approach to the presentation of history and archaeology enables visitors to create their own ‘stories’ as they explore the information contained in a database. The collaborative virtual environment in this case is the visitor's exploration of a historical monument through a series of interactive panoramas and navigation options that allow the visitor to weave archaeological facts and historical information to larger narratives. Because the visitor can follow a number of different trajectories (of time, space and theme) through the monument, and can freely switch trajectories, hundreds of different narratives are possible. The creation of these interactive stories has a larger educational purpose. It has proved to be a uniquely flexible medium for the communication of personalized, interest‐oriented, and user‐driven heritage information for the general public. Copyright © 2003 John Wiley & Sons, Ltd.
INCLUDE
Cultural heritage (CH) plays an important role in realizing the Sustainable Development Goals (SDGs). In this paper, we focus on emerging technologies such as Augmented Reality (AR) and gamified learning to foster public understanding of cultural values in historical contexts. We design HeritageSite AR, an exploration game for onsite CH learning and visits with publics in Relics of Arhat Monastery and Twin Pagoda (also known as Shuangta). Based on research investigation of technical means, expert semi-structured interviews and online survey, we distill and incorporate four design goals using user journey map. The implemented game design is evaluated with respect to three design components (i.e., reality, meaning, play) and four stages (i.e., trigger, engage, consolidate, relate) in CH visits. We conclude our work with a discussion of contributions to SDGs.
INCLUDE
Museums and libraries store a huge number of data related to their exhibition objects. Often, these data cannot be presented in exhibitions because of limited space or limited time during guided tours. Interweaving these physical artifacts with data visualization on digital media can help to make these data visible and searchable for visitors independently. Concepts and best practices for such situated visualizations are still rare. Our approach is to utilize the visitor's own device to enhance the museum visit through interactive data visualization. Thus, the visitor gets the ability to interactively gain more information on topics of interest or can link different topics.
INCLUDE
Quality education and well-being have always been the focus of sustainable development as social concerns, with an increasing demand for enriched cultural activities. Shared experiences, especially in family setting, bring about a better understanding of culture and foster public interest. In this context, our research endeavors to promote the philosophies and historical insights of Yangmingism while concurrently seeking to uplift family well-being through the immersive cultural exposure offered by Yue Opera, a national opera in China. To intertwine the cultural content for a consistent and continued experience, we design LanternOperAR, a hybrid gift for cultural appreciation, learning and recreation. We fully consider and respect user interests and requirements among different target groups to provide playful interaction in cultural activity, thereby promoting cultural inheritance and strengthening family ties.
INCLUDE
In the SPIRIT project, a location-based Augmented Reality (AR) application has been developed to stimulate the imagination of historical personal life at outdoor places. This paper presents quantitative evaluation results concerning the app’s potential for educational purposes. They are based on 107 questionnaires filled by visitors of a Roman fort museum site, having used the AR app in a 40 min tour over several locations. At each place, users can turn into several directions and see keyed video representations of acting characters superimposed on their device’s camera image. The fictional events are made up based on historical facts specific to the location. Next to issues with the novelty of the interaction, there has been an interrelation of the subjects self-assessed gaining of knowledge and the appreciation of the story’s motivational factors.
INCLUDE
Mobile learning technologies can provide an important added-value to cultural tourism by supporting visitors in their direct field experience. Providing engaging experiences is a key factor to the success of educational and entertainment products. We have researched how mobile gaming can help tourists and citizens enhancing their experience when interacting with Cultural Heritage. GeoJoko aims at designing and implementing context-based mobile games that will allow users to enhance their experience about Cultural Heritage and collective memory. The prototype includes location technologies that provide the location of each of the players in real time. Contents can be both streamed in real time over the infrastructure provided by the Public Administration within the "wireless cities" concept or downloaded to the mobile device of the player. This approach is innovative since it exploits the challenge of location-based serious games in order to implement a mobile learning service that supports the user understanding the elements of Cultural Heritage.
INCLUDE
This paper describes a research and development project for a novel technology, making the conveyance of cultural heritage during a historic sightseeing tour a unique experience. The cornerstones of this system are mobile augmented reality, including a hybrid tracking approach, intelligent queries to pose complex questions about geographical and historical knowledge, as well as a story engine to interactively run a digital story. This system involves the user in a thrilling story while exploring the history and the attractions of the city.
INCLUDE
This paper describes the design and evaluation process of a location-based serious game in a heritage awareness context. Conveying knowledge regarding tangible cultural heritage with the help of video games is a well-established concept. Though many applications in this domain have proven to be effective, they always rely on restrictions regarding time, place and usage of specific hardware. In contrast to previous approaches, we have developed Memorial Quest, a serious game with the objective to convey knowledge regarding cultural heritage objects accessible without aforementioned constraints. We examined educational effects by conducting a user study (n = 40) in which we compared our game to a common learning method in cultural heritage. Statistical analysis of the results revealed that learning effects were significantly larger when playing the game instead of perceiving the same contents in a traditional way. With the help of questionnaires and qualitative data, we identified possible flaws and elaborated potential improvements for future iterations.
INCLUDE
Combining augmented reality (AR) and physicalization offers both opportunities and challenges when representing detailed historical data. In this article, we describe a framework where mobile AR supplements views of 3-D prints of historical locations with interactive functionality and small visual details that the prints alone cannot display. Since seeing certain details requires bringing the camera close to the physical objects, the resulting camera frames may lack the visual information necessary to determine objects’ positions and accurately superimpose the overlay. We address this by enhancing tracking of 3-D prints at close distances and employing visualization techniques that allow viewing small details in ways that do not interfere with tracking. To demonstrate these techniques, we apply our framework to the preservation of two heritage sites that represent large real-life areas containing smaller details of interest.
INCLUDE
We reflect on a four-year engagement with transport authorities and others involving a large dataset describing the use of a public bicycle-sharing scheme. We describe the role visualization of these data played in fostering engagement with policy makers, transport operators, the transport research community, the museum and gallery sector and the general public. We identify each of these as ‘channels’ – evolving relationships between producers and consumers of visualization – where traditional roles of the visualization expert and domain expert are blurred. In each case, we identify the different design decisions that were required to support each of these channels and the role played by the visualization process. Using chauffeured interaction with a flexible visual analytics system we demonstrate how insight was gained by policy makers into gendered spatio-temporal cycle behaviors, how this led to further insight into workplace commuting activity, group cycling behavior and explanations for street navigation choice. We demonstrate how this supported, and was supported by, the seemingly unrelated development of narrative-driven visualization via TEDx, of the creation and the setting of an art installation and the curating of digital and physical artefacts. We assert that existing models of visualization design, of tool/technique development and of insight generation do not adequately capture the richness of parallel engagement via these multiple channels of communication. We argue that developing multiple channels in parallel opens up opportunities for visualization design and analysis by building trust and authority and supporting creativity. This rich, non-sequential approach to visualization design is likely to foster serendipity, deepen insight and increase impact. Index Terms—Movement visualization, visual analytics, bikeshare, impact, visualization models, design study. 1
INCLUDE
On-site investigation is a natural efficient way to cognize tangible, while immersive visualization engages human senses to percept information from data. Hence on-site immersive visualization is able to provide a better stereo environment for enhancing the understanding of real scene and gathering insight. With the development of technology, optical see-through head-mounted devices can help researchers achieve the goals. However, worldwide ancient murals suffer from various degradations related to compound environmental factors, which requires interdisciplinary researches. And because of the importance of on-site investigation in heritage protection field, on-site visualization has to meet the requirements of relative professors. In this paper, we present a novel mixed reality, on-site immersive mural degradation visualization solution for analyzing relevance between degradation and environment data. And a rapid, simple, low-cost and interactive method of degradation contour annotation is provided to solve the data sparse problem of standard degradation data. To demonstrate the visualization solution and the annotation method, two application results with real data from Mogao Grottoes and domain-expert feedback are given. The solution is on-site, immersive oriented and integrates secure designs for heritages. Our solution is flexible and effective so that it can provide a better data analysis environment than conventional planar visualizations. It also provides a promising way to change work styles. KeywordsOn-site visualizationImmersive environmentHeritage degradationDegradation annotation 1
INCLUDE
Physical heritage objects and assets are related to a vast amount of digital information of different kinds, which are challenging to be communicated to visitors in understandable and engaging ways. Yet recent technological advances promise new opportunities to more tightly merge the digital with the physical world. This paper therefore introduces the concept of “phygital heritage”, the integration of digital technology ‘into’ physical reality, as a potential medium for more enriched and playful communication of heritage values and qualities. We propose that phygital heritage should enable the exploitation of typical advantages of both digital and physical reality, and that distinct categories of phygital can be recognized based on: 1) the level of physical affordance; and 2) in how far the technology is integrated into the physical reality. The paper also opens the discussion about the potential challenges and concerns which future explorations, scientific research and real-world applications of phygital heritage probably will encounter.
INCLUDE
Augmented Reality (AR) goes beyond conventional methods of engagement with the public. It offers an interactive method to expand visualization techniques in civic engagement necessary for museum exhibitions that are focused on cultural and social issues. This paper discusses four AR projects conducted by a partnership between a history museum and a university in the City of Charlotte, USA. These projects, which utilize a variety of AR technologies, support a series of community events that are aimed at expanding overall public participation with a goal of increasing awareness of neighborhood history through data visualization and storytelling. Keywords— Augmented Reality, immersive technologies, civic engagement, data visualization, Geographic Information System I.
INCLUDE
Museums offer an ideal environment for informal cultural learning on heritage artifacts, where visitors get engaged in learning due to an intrinsic motivation. Sharing the museum space among visitors allows for collective learning experiences and socializing with each other. Museums aim to design and deploy Tangible User Interfaces (TUIs) in order to embrace the physical materialities of artifacts in the visiting experience. TUIs are believed to be more collaborative, attract more visitors, and persuade them to explore further. Cultural learning on heritage artifacts is particularly meaningful from the early age when opinions and attitudes are shaped. Museums accordingly follow a gamification approach (i.e., using game elements in a non-game context) to provide a collaborative and entertaining learning experience to young visitors. In this study, we investigate the implications of merging these two approaches in order to take advantage of the qualities of both TUIs and gamification in an educational museum context. Accordingly, we present TouchTomb and its evaluation in a real-world museum environment. TouchTomb is a situated tangible gamification installation that aims to enhance informal cultural learning for young visitors and to foster engagement and collaboration among them. The basis of the installation is a shared progress bar and three games with different spatial configurations, embedded into a custom fabricated replica of an original ancient Egyptian tomb-chapel wall on a 1:1 scale. Our field study involved 14 school visits with a total number of 190 school pupils (from 10 to 14 years old). We deployed a mixed-method evaluation to investigate how such a tangible gamification approach entertains and educates 15 pupils collectively for a maximum of 15 minutes, including the evaluation procedures. We particularly investigated how the different spatial configurations of the game setups influenced the stages of pupils’ cultural learning, and the levels of engagement and collaboration among them. We conclude the article by discussing the qualities of tangible gamification and its role in facilitating cultural learning. For instance, cultural learning is enhanced by situating heritage artifacts in the experience, and embedding learning in the reward system. Engagement and collaboration among visitors are fostered by creating a sense of ownership and designing a diversity of goals.
INCLUDE
This paper describes the Smart Survey Tool, a novel multi-device application for museum visitor tracking and tracking data visualization. The application allows museum staff to capture detailed information describing how visitors move around an exhibition and interact with individual exhibits. They can then visualize the results of tracking either on a single mobile device or with multiple mobile devices connected to a large display. The platform uses orthogonal views of the exhibition space for tracking and visualization, with a chess-pieceicon to represent visitors during tracking, and curved semi-transparent lines with animated semi-circles to communicate the path and direction of visitor movement. Our visualization is novel in its use of an orthogonal projection for pedestrian tracking and animation to communicate the flow of visitors around the exhibition space, as well as allowing users to dynamically switch between views representing different groups of visitors. The design of our application was informed through an extensive requirements analysis study conducted with Nanjing Museum and evaluated by conducting expert interviews with museum managers who considered that the application allowed for more effective and efficient recording and analysis of visitor tracking data. Index Terms: Human-centered computing—Visualization— Visualization application domains—Information visualization; 1I NTRODUCTION There is currently a massive effort within the academic and commercial research communities to develop indoor tracking and indoor GPS type technologies that can locate pedestrians inside a building and track their movement [1,7,17,20,36]. One of the primary motivations behind this research is to give building designers, and the people who manage public spaces, a better understanding of how people move around a building so they can design the building or modify the design (by placing attractions, moving shops, or renovating) to better control the movement of people in order to improve their experience and maximize commercial outcomes. However, this outcome depends not only on the effectiveness of technologies to collect tracking data, but also the availability of suitable analysis software that can help users to find important patterns in the data. The people in charge of public spaces need to be able to know about the density and flow of different types of visitor, at different times and under different conditions, in order to be able to make informed decisions about building space design and management. The focus of this paper is to develop a mobile software platform that helps with the collection and proper analysis of visitor tracking data to support this requirement. We also look to learn from the design decisions taken during the development of our system and would * e-mail: p.craig@xjtlu.edu.cn hope that anyone working on a similar problem could learn from our findings as presented in this paper. As a case study for the general problem of indoor pedestrian tracking and information visualization, we have been working with staff at Nanjing Museum develop new techniques to help us understand how visitors move around different museum exhibitions. Our initial requirements analysis with Nanjing Museum revealed that that tracking data could be extremely valuable toward understanding how visitors moved around museum exhibitions. It also revealed that the analysis of this data was expensive and limited with regard to what they could find out from larger scale data collected over a longer period of time. The results of tracking-data analysis were found to be very useful, but the museum staff had the feeling that they could get a lot more from their data if they could ask more questions of the data and had access to a more powerful interface that was capable of answering those questions. This called for us to support them by developing a dedicated application to collect and visualize tracking data [32]. Prior to the development of our tracking and visualization application, the main procedure for tracking visitors involved either pen-and-paper manual tracking (see figure Fig. 1) or automatic tracking using RFID tagging technology. The manual pen-and-paper approach involved using a paper copy of the exhibition floor-plan to record; the time a visitor arrives at an exhibit, when they leave the exhibit, and what they do at the exhibit (with activities such as taking a photo, reading text, concentrating etc.). This method was found to be quite flexible, and allowed trackers to capture a good range of different types of data. The problem with this technique is that writing down times to capture timing information gets to be tedious and different trackers tend to drift towards using different notation. The data also needs to be entered into a computer in order to generate any sort of meaningful summary of the data or perform anything more than the most basic analysis. This data-entry could be somewhat of an arduous task when data is collected for larger numbers of visitors. The alternative to pen-and-paper tracking is to use RFID tagging technology to automatically record the time spent by each visitor at each exhibit. This allows staff to record the movement of more visitors (i.e. all the visitors who use an audio guide with an RFID tag reader fitted) but has the disadvantage of not being able to record information about visitor demographics or activity at different exhibits. After tracking data was collected, museum staff would commission a report to summarize the data. This would include some statistics and images to describe different aspects of the data such as the density of visitors or the general movement of visitors around the exhibition space. Statistics would include mostly demographic data, such as the proportion of visitors from different economic groups, but also things like the most popular exhibit by total visits or time spent. The images included in the report would be heat-maps to show the density of visitors in different areas of the exhibition and directed graph view with arrows showing the path of each user around the exhibition (see figure Fig. 2). These reports were found to work well as a summary of the data but were less effective for 267 2019 IEEE Pacific Visualization Symposium (PacificVis) 2165-8773/19/$31.00 ©2019 IEEE DOI 10.1109/PacificVis.2019.00039 Authorized licensed use limited to: TU Wien Bibliothek. Downloaded on October 26,2024 at 11:51:20 UTC from IEEE Xplore. Restrictions apply. more in-depth analysis of larger scale data. The primary limitation of relying on a printed report for the analysis of tracking data was described to us as being a lack of flexibility to explore less obvious patterns contained in the data. For example, a printed report might have a heat-map that shows the most popular areas of the exhibition for all visitors. It may even show separate heat-maps showing the most popular exhibits for different genders or different hours of the week. It would, however, be impractical to have separate heat-maps for every possible type of visitor (by gender, age group, type of visitor) or ever different time (different hours, days of the week, weeks of the year etc.). The report would end up being overloaded with too many images and many of the images would be redundant as the patterns of visitor activity for groups of visitors would be similar to the pattern for all visitors. It would also be difficult to detect patterns without comparing different groups. Here, it makes more sense to allow museum staff to interact with the data in a way that allows then to quickly switch between different groups of visitors to observe how patterns of movement are different. Another significant limitation of the graphics used in the printed report was that the graph view, showing the movement of visitors, suffered from the problem of clutter and crossing lines [33] and was ineffective at showing the direction of visitor movement for larger groups of visitors. Anything more than around ten or twenty visitors (depending on the layout of the exhibition) would render the graph view pretty much useless as it would just appear as a grey mass of crossing lines (see Figure Fig. 2 for an example of this). The focus of the project described in this paper was to develop an interactive visual interface to overcome the limitations of the existing method for visitor tracking and tracking data visualization. This led us to develop a new mobile software platform including a tracking application to allow museum staff to track a sample of visitors to a museum exhibition, and a visualization application to support more effective analysis of the results. Figure 1: An annotated print-out of the museum floor-plan used for pen-and-paper tracking. 2RELATEDWORK There is a long history of tracking visitors to museum and galleries exhibitions [26, 29, 34] with most systematic methods developed over the past 20 years. The main challenges of visitor tracking are managing the cost associated with gathering data, and performing effective analysis of the data once data is gathered. There are a number of different methods that can be can be used to track visitors to museums or other public spaces. The most common method is for people to be tracked manually with their time at each exhibit recorded using pen and paper [34]. Tracking can Figure 2: Examples of the type of visual representation used in printed reports for tracking data. These are heat-maps to show the density of visitors in different areas of the exhibition, and directed graph views with arrows showing the path of visitors around the exhibition. also be automated using pedestrian tracking systems that employ video camera footage, RFID tags [1,7,21,27], or applications that monitor WIFI signal strength on the visitor’s smart-phone [17,20]. While automated tracking has the advantage of potentially higher throughput with less investment in manpower, manual tracking is generally thought of as being more accurate and can record more detailed information such as the type of activity that each visitor is engaged in at each exhibit. Once visitor tracking data is collected it can be visualized in order to understand patterns of visitor movement. The most common visualizations for this use heat-map type displays that show the density of visitors in different areas and graph views that show the most common paths for visitors around the exhibits [24, 30]. Additional insights are often gleaned from the data when it is filtered according to sample attributes such as age or gender [30] or attributes of the data such as the door by which visitors enter the exhibition or the first turn they take when they enter [8]. The most challenging aspect of visualizing visitor tracking data is showing the path and movement of visitors around the exhibition space [24, 30]. While existing graphical representations worked well for smaller scale data [8,24,25], when the data was scaled up these techniques are found to suffer from the problems of clutter and crossing lines [33]. Certainly, more powerful analysis techniques are needed to properly exploit the potential of large-scale tracking data [2,33]. For the purposes of information visualization, this sort of data can be considered as a specialized type of large-scale directed graph data where the positions of nodes are fixed. Display techniques that can be applied to this sort of data are curved lines to distinguish 268 Authorized licensed use limited to: TU Wien Bibliothek. Downloaded on October 26,2024 at 11:51:20 UTC from IEEE Xplore. Restrictions apply. between paths [14,19], edge bundling to simplify the view by gather similar paths together [4, 18, 23], and different methods (such as arrow heads, tapering, curves, color, and animation) for showing the direction of edges [22]. The curved line techniques applied to parallel plots [19] can be thought of as particularly applicable to the problem of visualizing visitor paths as different the edges in parallel plots also form distinct paths and the curves should be capable of helping us follow visitor paths in a similar way to how they help us follow parallel-plot paths. 3R EQUIREMENTSANALYSIS Our initial requirements analysis for this project took place over approximately one month with different groups of urban planners, museum exhibition planners and museum staff based in the United Kingdom and China. The number of personnel involved in our requirements analysis study was relatively small, including just eleven key project stakeholders, but we felt it was important to involve people who had a genuine interest in the projected functionality of the software. These were the staff involved in the United Kingdom Arts and Humanities Research Council funded ’Romantic Scotland’ exhibition to be held at Nanjing Museum, and included a core team of three urban planners, five permanent Chinese museum staff, and three of the United Kingdom staff responsible for the design and management of the exhibition. The project team also included visualization experts, mobile device developers and personnel familiar with a variety of indoor pedestrian tracking technologies. Over the course of four months we held a series of eight meetings with various members of the project team to arrive at our final application design. This initial requirements analysis was supplemented with smallscale evaluations of early prototypes (involving twenty or so students at the XJTLU university museum) for tracking and visualization. This allowed us to iron out usability issues and refine the interface design before a final trial run at the Nanjing Museum, and the actual tracking and visualization of the Romantic Scotland exhibition. The first stage of requirements analysis was to gather requirements for visitor tracking. This included site trips to the museum and Skype meetings with the exhibition planners in the UK to determine what technology would be most appropriate for tracking the museum visitors. The main outcome of this stage of requirements analysis was the information that the personnel responsible for managing the exhibition required three types of data. These were data related to the visitor, the visitors behaviour in the exhibition and the timing of their movement through the exhibition. These data are described in table 1. The requirement of the exhibition managers to collect behavioural data in addition to timing data made it necessary for us to use human observers rather than an automated technique such as RFID tagging [1,7,27] or Wi-Fi tracking [17,20]. However, the range of behaviours that they wanted to record made pen-and-paper tracking impractical. This made it necessary to develop some way to make the process of manual tracking more manageable. As the layout of the exhibition made it necessary for trackers to actually follow visitors in order to observe their behaviour, the development of a mobile tracking application seemed the most plausible solution. The second stage of requirements analysis focused on the potential users’ objective for tracking data analysis. Here we held a number of meetings, initially using the reports commissioned by the museum to help guide our dialogue to determine how the data should be visualized and later using our own sketches of how the application should look. These meetings also allowed us to identify the main types of pattern that the users wanted to be able to find in the data (summarised in table 2), and the different ways in which they wanted to be able to analyses the data (table 3). A requirement that was stressed as particularly important for our users was to be able to view and compare patterns for different demographic groups. As well as providing clues as to why particular Table 1: Different types of data the users wanted to record. DataDescriptionCharacterization VisitorIncluding data about the visitor, such as their gender and age, as well as general information about their visit to the exhibition such as the type of group they belonged to or how busy the exhibition was when they arrived. We initially identified a total of nine classifications (gender, age group, etc.) and around one hundred individual classes that a person could belong to (male, female, child, teenager etc.). BehaviourWhat the visitor does at each exhibit, for example, taking a photo, reading text, or thinking about the exhibit. We initially identified a total of nine actions. The visitor could be recorded with multiple actions at each exhibit. The timing of these actions is not important and the exhibition managers were happy to rely on observer to judge if an action was taking place. TimingThe duration of the visitors stop at each exhibit and the order in which they visit exhibits. The exact path of visitors was not considered important. The time when a visitor arrives at an exhibit and when they leave the exhibit. parts of the exhibition may be visited more or less for different groups, data related to the behaviour of different demographics could also allow the analysts to test theories related to how different groups are attracted to different types of exhibit (looking at different genders [5, 6], types of visitor group [5], and age groups [16, 29] etc.). The requirement for some analysis to be done in locations where it would not be practical to use a laptop or desktop PC was also significant in that it made it necessary for us to design our visualization application to run on mobile devices. These were chosen as six eight inch tablets, to be used primarily for tracking, and a couple of larger ten inch tablets to be used mostly for visualization and analysis. It would also be necessary to connect mobile devices to synchronize the data collected on different devices, and these devices could use the same connection for collaboration on a large display. 4T HESMARTSURVEYTOOL The process of requirements analysis described above gave rise to our initial platform design. This was for a mobile application that allows us to track visitors (running on multiple devices so different tracking staff can work at the same time) and a mobile application that connects to a large display screen for visualization. We used an Android socket connection running over a shared Wi-Fi connection to synchronize data across devices and coordinate mobile devices with the large display for collaborative visualization. This connection could be initiated by using one device to scan a QR code encoding the IP address of the second device. Figure Fig. 3 shows the device configuration and outlines the basic functionality of our platform. 4.1 Visitor Tracking Figure Fig. 4 shows a screen-shot of the interface of our tracking application. The interface uses a military projection of the exhibition space using an image of the exhibition floor plan with the visitor currently being tracked represented as an icon resembling a pawn in chess. This icon is designed to fit the overall aesthetic of the interface without specifically representing any particular type of visitor. The military projection (also see figure Fig. 7) is an oblique projection 269 Authorized licensed use limited to: TU Wien Bibliothek. Downloaded on October 26,2024 at 11:51:20 UTC from IEEE Xplore. Restrictions apply. Table 2: Patterns the users wanted to be able to see in the data. PatternRationaleExisting analysis and limitations Density of visitors in different areas of the exhibition. To identify potential bottlenecks, where to space is too busy at times, or parts of the exhibition space that are underutilized. Heat-map representation is already quite effective for this type of pattern (see figure Fig. 2). Flow of visitors. How visitors generally move around the exhibition including their general around the exhibits and the direction of movement. This could tell them why parts of the exhibition are not being visited enough, or how their strategy to guide visitors towards certain exhibits is working. A line graph representation works well for smaller numbers of visitors but for larger numbers it becomes unusable due to the density of lines (see figure Fig. 2). Statistics for exhibits Including total and average number of visitors at an exhibit, time spent, number of photos taken, etc. to tell us how different exhibits appeal to visitors in different ways. Tables are used to present data for individual exhibits, but users wanted a more intuitive graphical representation (using the exhibition floorplan) that also allowed them to explore the data to view data for different demographic groups. Patterns for different demographics Above types of pattern for different demographic groups including different genders, age groups, and visitor groups (tour groups, couples families etc.) to see how different groups behaved (e.g. young people versus elderly people) in order to ensure that the exhibition caters to a wider demographic, or is able to target a particular demographic such as those who are more likely to become tourists in the location that the exhibition is promoting. Bar charts and pie-charts are used for broad overviews (such as male visitors versus female visitors) but users wanted to be able to explore the data to see more detailed data for smaller groups (such as the poplar exhibits for young couples or tour groups). Table 3: Different ways the users wanted to analyse the data. AnalysisDescription Mobile analysisThe users wanted to be able to analyse the data in different locations such as meeting rooms or the exhibition space itself. Using a desktop PC would not be possible in most of these locations and a laptop PC would be also impractical. Collaborative analysis Often the users would want to work together to analyse the data so they could combine their knowledge of the exhibition and discuss findings. Publication of results. The users also wanted to be able to share their findings in printed form, so the visualization needed to be aesthetically appealing. Figure 3: The device configuration and basic functionality of our platform. Tracking staff use a mobile device to track visitors moving around the exhibition. Devices used to track visitors and display the data are connected on the local Wi-Fi network by scanning a QR code. Data is automatically synchronized between connected devices and the data can be visualized on a mobile device or a mobile device connected to a large display. where the angles of the xand z-axes are at 45 degrees, meaning that the angle between the x-axis and the z-axis is 90 degrees. This allows vertical elements such as walls and doors to have a kind of 3D representation without having to skew the xz-plane. The interface is designed to run on a tablet to be held in landscape orientation and operated using the thumbs. Figure 4: The tracking application interface. The user presses on different exhibits to indicate the movement of a visitor and uses the thumb-wheel dials to specify visitor characteristics or actions. To operate the interface and track visitors moving around the exhibition space the tracking staff click on doors and exhibits to record visitor movement and uses the thumb-wheel dials at the bottom corners of the interface to record additional details about the visit (see figure Fig. 5). While a human tracker is certainly less efficient that automated technologies such as RFID for tracking motion, the advantage of employing a person to track visitors is that they can also record the activity of visitors at different exhibits. This indeed was one of the requirements of our project stakeholders who were interested not only in visitor movement but also how people behaved in front to exhibits by doing things like taking photos or 270 Authorized licensed use limited to: TU Wien Bibliothek. Downloaded on October 26,2024 at 11:51:20 UTC from IEEE Xplore. Restrictions apply. Figure 5: Using the tracking app to record the movement of a museum visitor around the Romantic Scotland exhibition in Nanjing Museum. reading exhibit text. When a visitor enters the exhibition space, tracking begins by the tracker clicking on the door that the visitor entered by. After clicking on the door an icon representing the visitor appears as if they are entering through the door. The tracker can then record how the visitor moves around the exhibition by clicking on each exhibit the person stops at. This is done in real time, and as the tracker clicks on exhibits the icon representing the visitor is animated to follow an approximation the path of the actual visitor. If the tracker makes a mistake they can hit an undo button to reverse their previous action. Clicking on a space away from an exhibit moves the visitor icon away from any current exhibit but not on to another exhibit. This whole process records the start and end time of the museum visitor’s attention being given to individual exhibits. Thumb-wheel dials at the bottom corners of the interface can be used to indicate details such as the type of visitor (approximate age group, gender, type of group etc.) and details about the visitor’s activity at each exhibit. So, if the visitor performs actions such as taking a photo, reading the exhibit text, or conversing with a friend, all this can be recorded. Figure Fig. 5 shows the tracking application running on a seven inch tablet. 4.2 Tracking Data Visualization The interface for our visualization application (see figure Fig. 6) uses different projected views of the exhibition space to show the density or flow of visitors for the tracking data. The different types of projections used are an isometric projection, a military projection, and top-down plan view (see figure Fig. 7). The interface aggregates data for all visitors or a selected group of visitors. The interface can be set to either show the density of visitors in different areas using a heat-map, the stats for different exhibits using a bubble-map, or the main path of visitors using a graph view with semi-transparent curved lines and animated arrowheads. Figure Fig. 6 shows the interface with the bubble map selected, figure Fig. 7 shows the path view for different projections and figure Fig. 8 shows the heat-map showing areas of the exhibition where visitors spent the most time. A bar-chart menu with icons at the left hand side of the screen (seen in figure Fig. 6) allows the user to select from a total of thirteen different attributes to display (number of visits, average visit time, number of photos taken etc.) and nine different data classifications (age group, gender, day-of-week, hour etc.) with a total of around one hundred different classes [32]. The different classes for the selected classification are shown in a horizontal bar chart together Figure 6: The main information visualization display with the graph view selected to show how visitors move around the exhibition. Figure 7: The different types of projections used in the Smart Survey Tool visualization interface. These are an isometric projection, a military projection, and a top-down plan view. 271 Authorized licensed use limited to: TU Wien Bibliothek. Downloaded on October 26,2024 at 11:51:20 UTC from IEEE Xplore. Restrictions apply. Figure 8: The heat-map view with an isometric projection. Figure 9: A heat-map with the 16-30 age group and average visit duration selected shows us that males (left) tend to spend more time at the bench, while females (right) spend more time in the interactive booth. with the values for the selected attribute. This allows the user to look at things like the most photographed exhibits, or the number of males and females or different types of group visiting the exhibition. The bar chart can also be used to select different groups in the main map view to look at popular exhibits or patterns of movement within these groups. The selection in figure Fig. 6 shows the number of visitors by group type (individual, friends, couple, family, school group or guided tour). Figure Fig. 9 shows a part of the heatmap for different genders in the sixteen to thirty year age bracket. Here males are found to spend more time than their female counterparts sitting on the bench (on average 1 minute 12 seconds versus 1 minute 6) but a lot less time in the interactive booth (33 seconds versus 59 second for females). This suggested to the museum staff that younger female visitors might be more outgoing and that more could be done to engage young male visitors in interactive exhibits. In addition to being able to select different types of visitor the user can also interact directly with the map to select different exhibits or paths and use the bar chart and icon to view statistics for that part of the data. Pinch, splay and drag actions, that were already familiar to our sample-users, can also be used to navigate around the map and the buttons at the right hand side of the screen allow the user to rotate the view 45 degrees in either direction or connect to another device to synchronize data or control a large display. The graph view of visualization interface uses curved lines to show the path of visitors between exhibits. Animated arrow heads communicate the direction of visitor movement, and color value is used to communicate the number of users moving along any given path. The curve used is a simple quadratic curve starting mid-way between exhibits with a control point at the exhibit. Transparency is used so that only the most popular paths are fully opaque and less popular paths are more transparent. This has the effect of reducing the problems of clutter and crossing lines, as less popular paths have less impact and less weight to occlude the more travelled paths that the analyst is likely to be interested in. The approach of making less common paths less visible made sense to our users as they only wanted to be able to see the main patterns of movement for each selected group. Outlying patterns of visitor movement would only be considered as significant if they Figure 10: Using the visualization interface with a mobile device controlling a large display. involved a significant number of visitors or a smaller numbers of users in a smaller portion of the data. In either of these cases the pattern would be evident as the opacity of a path is proportional to the total visitors taking that path and inversely proportional to the total number of selected visitors. Smaller numbers of visitors who took an unusual route within a larger group were not considered to be significant for the purpose of this analysis. Users can connect the application running on their own device to any other device running the application by scanning a QR code displayed on the top right of the screen. This connects the devices together using an Android socket connection running over a shared Wi-Fi connection. Typically, a mobile device will be used to connect to an Android box powering a large display so that the smaller mobile device can be used to control the large display (see figure Fig. 10). When multiple users connect to the same large display, they can take turns controlling the large display and collaborate to analyse the data. 4.3 Design Rationale and Methodology The design of our platform, and our platform components, were developed and refined through a series of meetings and small-scale prototype evaluations with potential users and project stakeholders. This proceeded over a period of roughly four months. The initial design was inspired by our initial requirements analysis (described in section 1) where staff at Nanjing were able tell us not just about what they needed, but also what they liked or didnt like about the existing method of analysis and what they wanted from the new system. This was followed by the development and evaluation of several small scale paper prototypes and consultation sessions with urban planning experts at Xi’an Jiaotong Liverpool University. Finally, we refined our design by testing specific functional components attached to early functional prototypes. 4.3.1 Bar chart widget The first thing we gathered from the requirements analysis was the importance of ranked lists for the visualization and analysis of the tracking data. The museum staff wanted to be able to view things like the most popular exhibits by number of visitors, average visit time, and things like the number of photos taken at different exhibits and by different types of visitor. They also wanted to be able to look at things like what different types of visitor they had, how much time different age groups spent in the exhibition and what where the most popular exhibits for different age groups. Written reports were good at presenting this sort of data in lists, but they were limited in the number of lists that could be included. Displaying each of the 272 Authorized licensed use limited to: TU Wien Bibliothek. Downloaded on October 26,2024 at 11:51:20 UTC from IEEE Xplore. Restrictions apply. thirteen different types of attribute for each of the nine classifications would give us a total of one hundred and seventeen different lists to display. This would take a lot of paper, even before we consider that we might want to display a filtered version of any of these lists with any combination of the hundred or so distinct classes (showing, for example, the most photographed exhibits for a particular gender or age group). This made it important to have some easy way for the users to be able to select the parameters for different lists and select different settings for the lists. Our interactive bar chart widget for looking at lists of tracking stats can be seen in figure Fig. 6. The motivation behind this design is to use icons to make a more efficient use of screen space but otherwise make the interface as clear as possible by having three distinct lists for views, attributes, classifications, and class filters. Attributes include number of visits, average visit time and behavioural data like number of photos taken. Classifications include timing data (day of the week, hour or day etc.) as well as information about the visitor (age-group, gender etc.) and the conditions (quiet, bust etc.). Selected attributes and classifications are highlighted using a red glow, applied filters are shown using icons, and a help mode is available to provide tool tips describing the function of the icons. 4.3.2 Heat-map and bubble-map One thing that was reported to work well for our sample users using the existing method of analysis was the heat-map view of the data. This was found to be effective at communicating the density of different visitors in different areas of the exhibition space. The only major limitation of this type of representation was that it could not show the density of different types of visitors or for different times, and it could be difficult to see which individual exhibits were popular if two exhibits were close together. To allow the user to be able to look at different heat-maps for different types of visitor of different times, it was easy to link the heat-map view in our visualization interface to the bar chart widget (described in section 4.3.1). Here the user could use the bar chart to select a class of visitor or a time (a week of the year, weekday or hour of the day) and show a heat-map for just that data. The user could also look at a different heat map for different data attributes such as average visit time or photos taken. This ability to view different aspects of the data in the heat-map view made it possible to replicate all the existing heat-map images included in the commissioned report and also generate any of the other types of heat-map that the museum staff told us they might want to see. To make it easier to distinguish between statistics for exhibits close to each other, we decided to supplement the heat-map with a bubble-map type display showing statistic values for individual exhibits. If this setting is selected a circle is shown next to each exhibit with the size of the circle proportionate to some statistic such as the average visit time or the total number of visitors. 4.3.3 Curved line graph view The museum staff felt that the graph views included in their printed reports, showing the direction and flow of visitors around the exhibition space, did not work well (see figure Fig. 1). These suffered from the problem of clutter and crossing lines citeyabushita2011summarization and were ineffective at showing the direction of visitor movement for larger groups of visitors. In order to resolve this issue we developed a new visualisation method using semi-transparent curved lines which could be combined to show the main patterns in each view allowing the user to view more detail by interacting with the interface to select different filters. Without filtering, this representation gives a simple overview of the data. If the user interacts by selecting exhibits in the map or classes of visitor in the bar-chart widget, they can drill down into the data to reveal more detail. This type of representation worked well in a small scale evaluation of an earlier prototype and was refined to be used in the latest prototype as shown in figures Fig. 6 and Fig. 9. We also tried various encodings to communicate the direction of movement (including arrow-heads, tapering, curves and colorcoding, from Holten et al. [22]). The most effective representation appeared to be animated semi-circle waves moving along the path. This had the advantages of not requiring us to change the shape of the path (which could result in a path that is confusing) and of allowing us to superimpose two paths in different directions without losing clarity. Animation and movement also has the advantage of being a pre attentive visual variable that grabs the users’ attention [13,31]. It can also be thought of as being quite a literal sort of representation as the graphic moves in the direction of movement of the actual visitors. The potential disadvantage of animation is that it can attract too much of the user’s attention and distract them from other important parts of the interface. In our case we did not find this to be a problem. Our small scale evaluation with our users told us that this representation gave them a better sense of how visitors would move around the exhibition and this did not detract in a negative way from any other aspects of the interface. The semi-circles showing the direction of motion can be seen in figure Fig. 7. 4.3.4 Military, top-down and isometric projections The military, top-down and isometric projections use in our tracking and visualization interfaces where sketched out and discussed at the very start of the project with a small group of urban planners based in the university. These are based on a style used for architectural drawings and it was felt that the 3D effect of the isometric and military projections could give the user a better feel for the exhibition space. In general 3D representations tend not to work well for information visualization due to the problem of occlusion where placing objects in a 3D space can mean that objects of interest get hidden behind other things [35]. Three dimensional visual cues are however proven to help users remember objects and can help with spatial orientation [3, 28]. As the task associated with this visualization was associated with a real physical space, we felt that the use of three dimensional visual cues to represent exhibits and doors was warranted. To reduce the problem of occlusion we made exhibits semi-transparent and allowed the users to rotate the display in increments of 45 degrees. Some initial testing of an early prototype of the tracking app indicated that the military projection would be preferred for tracking. Out of sixteen trackers, twelve expressed a preference for the military projection. The average rating for the military and isometric projections was significantly higher that the rating for the top-down view which scored badly. The users told us they preferred the military projection because exhibits where more easily recognizable with a more natural side-on perspective. They also considered it easier to use than the isometric projection because the spacing for exhibits is more regular on both x and z axes. This made it more convenient for the user to select different exhibits quickly and accurately regardless of whether the exhibits were spaced on the x or z axes. This finding led us to make the military projection the default projection for tracking. For the visualization interface users expressed a preference for isometric and top-down views. Out of the nine users involved in the evaluation seven preferred the isometric view, but they also scored the top-down view well and explained that they would like to be able to use both views. This was irrespective of whether analysis was done on a mobile device or mobile devices connected to the large display. The top-down view was felt to give a better overview of the data without the problem of occlusion and the isometric view gave a more natural perspective. There was less need to select items quickly during visualization, so this advantage of the military perspective 273 Authorized licensed use limited to: TU Wien Bibliothek. Downloaded on October 26,2024 at 11:51:20 UTC from IEEE Xplore. Restrictions apply. Figure 11: Thumb-wheel widget with help text (in Chinese) activated by pressing the question mark icon. to better support this was not important. As no single view was significantly preferred over the others, we gave the user the option of being able to switch between military, top-down and isometric projections while using the visualization interface. 4.3.5 Tracking application interface design For the tracking application interface design our principle concerns were ease of use and efficiency. The thumb-wheel dials of the tracking interface are designed so that trackers can specify this information as efficiently as possible with the minimum required effort while holding the tablet in both hands and maintaining a reasonable level of accuracy. The view of the interface also moves automatically to keep the tracked visitor in the centre of the screen so that the tracker does not need to adjust the view themselves. Both these features are aimed at reducing the effort needed from the tracker who may need to operate the app over extended periods of time. In a prototype evaluation with ten trackers we tested the tracking interface. During these tests a thumb-wheel menu with icons was compared with a more traditional list type menu. Our thumb-wheel menu (see figure Fig. 11) could be rotated using the users thumb with icons being tapped to specify functions. Icons where either reused from established iconic representation (for example the representations of gender), designed using symbols associated with the concept (for example a triangular flag for a tourist group, where in China it is common for the tour guide to carry a such flag), adapted from a physical representation (one person beside a painting for a quiet exhibition space), or designed using text when a small enough text label can fit into the button space (for ’age’ and different age ranges). The list type menu is simply stacked textual descriptions of each concept. Comparing the thumb-wheel and the text menu, the thumb-wheel was found to be more efficient as it required less movement and the user did not need to adjust their grip on the tablet. The icons also took up less space than text in the interface and could be recognised and pressed quickly if the user understood their meaning. The users told us that they wanted to be able to operate the interface without reading and that they found the thumb-wheels be more efficient than the text based menu. Unfortunately, some of the interface functions related to concepts that were not so easy to represent using an image that would be easily recognisable for the user. These included things like how busy the exhibition space was, or if the visitor was deep in thought. To help the user learn the icons for these actions we supplied an instruction manual describing the meaning of each icon and incorporated a help-text into the interface (see figure Fig. 11). Figure 12: Thumb-wheel widget with help text (in Chinese) activated by pressing the question mark icon. Figure 13: Thumb-wheel widget with help text (in Chinese) activated by pressing the question mark icon. Figure 14: Thumb-wheel widget with help text (in Chinese) activated by pressing the question mark icon. 5EXPERTINTERVIEWS ANDEVALUATION Our platform was used to track just over 750 individual visitors to Nanjing museum over a period of two months with each visitor viewing an average of seventeen individual exhibits each. The data successfully captured and visualized on mobile devices by individual museum staff and on mobile devices linked to a large display by multiple staff. Overall this was a success as the tracking and visualization proceeded smoothly with a small number of technical glitches that were easily resolved. Figures Fig. 12, Fig. 13 and Fig. 14 show a selection of the insights into the data learned from our visualization app that the users considered they would not be able to find using any other methods they had already tried. Our users were able to find a variety of patterns like these and it was evident through further investigation that they would not have been able, or would have found it significantly more difficult, to find most of these patterns using the previous analysis methods. Figure Fig. 12 shows how visitors tracked the left-hand-side wall or took a short-cut past an enclave to view the striking portrait of Sir Joshua Renolds in a kilt (or exhibition organizers considered that this type of image would be particularly attractive to Chinese tourists and it featured heavily in marketing material). This pattern is clear using the new interface, but less evident using the traditional technique where the volume of crossing lines and the abrupt change in direction 274 Authorized licensed use limited to: TU Wien Bibliothek. Downloaded on October 26,2024 at 11:51:20 UTC from IEEE Xplore. Restrictions apply. of straight lines makes it difficult to tell where the visitors come from before taking the short-cut or that most of the visitors who move along the wall move onto the portrait. Figure Fig. 13 shows how visitors tend to be attracted to the Wandering Shadows painting when they enter the exhibition then move to the left-hand-side wall. This is very clear from the new visualisation with curved lines and transparency, but not clear from the traditional representation where the volume of data makes it difficult to see any trend of this sort. Both figures Fig. 12 and Fig. 13 illustrate behaviour which is contrary to the analysts’ expectations, where visitor are known to have a tendency to follow the wall on the right hand side. While neither of these findings led the exhibition designers to change the layout of the exhibition (which would be very unusual for a temporary exhibition), they felt that it might change how they approach the layout of future exhibitions. Figure Fig. 14 shows how patterns of behaviour change when the exhibition is more or less busy with visitors tending to spend more time at interactive exhibitions near the exit when the museum is quieter. This pattern was revealed by alternating between different items in the bar-chart menu. This pattern would not be seen using the traditional method as a limited number of images need to be prerendered without giving the user the opportunity to make selections to filter and explore the data. Overall, the final evaluation was considered a success and expert interviews with museum managers revealed that they were satisfied that the software was more efficient than the existing technique employed and effective at supporting all their requirements for analysis. The main benefit of the tool was found to be that it allowed the users to interactively explore the data and allowed them quickly analyse the data without having to rely on data analysis experts. The users described the software as ’flexible’, ’interactive’, and said that it provided ’good access to the data’. The users were also impressed with the use of animation and semi-transparent lines to show visitor flow as this was felt to be clearer and give a better sense of how visitors moved around the exhibition. They also felt that the different projections each had advantages for different aspects of the data. The top down projection was felt to give a better overview of the data, while the isometric view gave a better feel for the exhibition space when zoomed-in. The limitation of the software was that museum staff could not change the mapping of the exhibition inside the software, and this is something they would like to add in future versions. Overall the user feedback was largely positive and as a result of their experience the museum have committed to using the software again for other temporary exhibitions. We also plan to adapt the platform for use with the museum’s permanent exhibitions. This will involve working to adapting the tracking software to work with RFID technology so that tracking can be automated. The system will be integrated with audio-guide equipment so we can incorporate information about the type of person being tracked. The disadvantage of using RFID tracking, rather than manual tracking, will be that it will not be able to record the users behaviour at individual exhibits unless we can use some other type of technology such as live-video object recognition and develop an additional system component to do this. This is being considered as a potential direction for further work. 6C ONCLUSION We have developed a software platform to support museum visitor tracking and data visualization. The software allows museum visitors to have their behaviour and movement recorded using a specialized tracking app running on a mobile device with the data visualized on a mobile device or mobile devices connected to a large screen display. The visualization uses a custom bar chart widget, and a novel combination of semi-transparent curved lines and animated semi-circle waves to show the flow of visitors around the exhibition. Our visualization is novel in its use of an orthogonal projection for pedestrian tracking and animation to communicate the flow of visitors around a physical space. Another significant advantage of our technique of established techniques was that it allowed users to dynamically switch between views representing different groups of visitors. Our study began by recording and gathering together user requirements (described in section 3) which informed the design of our software (section 4). The software was tested to refine the design and evaluated to gauge a positive response from sample users. It was also used to track just over 750 visitors to Nanjing museum over a period of two months and visualize the results. This led to positive results and our users who were able to find a significant number of patterns in the data that could not be found using the previous technique. A series of expert interviews revealed that our users felt the flexibility of the interface to allow them to be able to explore the data, rather than rely on data analysis experts, to be the main advantage. The design and successful evaluation of our platform demonstrates the power of a more interactive information visualization display to reveal patterns in large scale data, and also the potential of mobile devices for this sort of visualization task [9,11,12]. Our interface worked well by relying on the user to interact with the display to reveal patterns in the data, rather than us trying to show too much of the data in any single frame of the visualization [10,14,15]. This was particularly important for our graph view, where less popular visitor paths would stay in the background until the user selected a smaller portion of the data containing these paths so they became significant. It is also worth noting that the limitations of mobile devices (with less accurate interaction, less screen space, and reduced processing power) did not cause any significant issue for our users in the final evaluation. This was partially down to advancements in mobile device capabilities but also down to design decisions such as using a thumb-wheel menu and military projection for less accurate interaction in the tracking app, or the design of the bar-chart widget for more predictable reversible single touch actions. It is also very likely that allowing the mobile devices to control a large display, when the app was used in a meeting setting with multiple users, also helped us to overcome some of the limitations of mobile devices for information visualization . Reflecting on the complete design process the most important lessons for us, of potential interest to designers of similar applications, are as follows. • Having an interactive interface that allows the user to explore the data by selecting different facets of the data (e.g. different genders, age groups etc.) and different views (e.g. heat-map or curves for paths) can be very powerful. This was seen as the main advantage of this technique over existing methods. • Animation can be used to effectively to show the flow of visitors around an exhibition. • Different projections, including 2.5D projections such as isometric projections, can have value in the presentation of visitor movement data and heatmaps. Due on the success of this operation Nanjing Museum have committed to using the Smart Survey software for future tracking exercises and we will look to improve the software and adapt it for more challenging data (e.g. in multiple levels of a building or over longer periods of time) and data collected using different technologies such as RFID. More work could also be done looking at how to apply or adapt this sort of technique for other environments such as shopping centres and other public spaces where there may be a commercial advantage to tracking visitor movement but patterns of movement may be less predictable or more difficult to characterize. 275 Authorized licensed use limited to: TU Wien Bibliothek. Downloaded on October 26,2024 at 11:51:20 UTC from IEEE Xplore. Restrictions apply. ACKNOWLEDGMENTS The authors wish to thank the students who helped with the evaluations and development of prototype applications. These are Ke Wang, Chao Xiang, Kun Wang, Wenjun Chen, Yuxuan Zhao, Zifan Wang, Yuexuan Li, Zhejian Zhang, Xiaoyu Wu, Weiyi Zhang and Yuhao Wu. We would also like to thank the staff at Nanjing museum and our collaborators on the UK AHRC project ’Producing/Consuming Romantic Scotland’. R EFERENCES [1]S. Ahuja and P. Potti. An
INCLUDE
INCLUDE
INCLUDE
information with specific locations, facilitating analytical interactions that could be called “situated analytics” [1]. Moreover, Ens et al. [5] have shown the potential of utilizing physical objects with augmented information, such as tokens and a tabletop display, to enable interactive data exploration and manipulation for complex collaborative visual analytics. We believe that the further combination of visualization and AR techniques would effectively demonstrate cultural features, trends, and differences that play a crucial role in cultural exchange and preservation. Additionally, AR educational games have emerged as a valuable addition to CH learning, offering a wide range of interactive techniques and devices. The concept of gamification involves incorporating game design elements such as interface mechanisms and principles into non-game contexts [17]. Interface design patterns and game mechanisms are employed to provide freedom of choice for unbound exploration while also setting restrictions and resource limitations for handheld AR analytics experiences. For example, Ramly and Neupane developed an AR game [19] for exploring artifacts in an onsite museum. Users can acquire knowledge about CH preservation by collecting artifact fragments, collaborating on restoring missing artifacts, participating in virtual gallery design activities as well as contributing to crowdsourced research initiatives. In general terms, game design principles aim at achieving specific goals through rewards. Xu et al. [25] presented a tangible AR interaction device that integrated the visualization and gamification elements to trigger the collection and exploration for culture learning and museum gifting. The results confirmed the positive effects of playful interactions combined with visualization features on users’ learning motivation, experience and outcome levels. These findings propose that the integration of tangible AR and visualization in educational games has the potential to effectively enhance cultural learning performance. 3 TIMEQUESTAR: A PROTOTYPE OF THEARBOARDGAME Our goal is to design a playful learning tool that enables the public to gain insights into significant cultural places through augmented data visualization. To achieve this, we started the design process by shaping the learning context. First, we conducted the data analysis and processing to extract key information. Then, we formulated game mechanisms based on the data and tasks, and finally, we presented the tacit knowledge into a physical map and a series of event cards, as well as digital game content. The detailed workflow of the prototype design is illustrated in Figure 2. To ensure the educational efficacy of our tool, we established the following learning goals [G]: G1.Experiencing urban development and local culture. G2.Exploring the collective creation and individual emotions. G3. Conveying the interweaving with historical timeline and cultural context. G4.Understanding CH preservation and innovation. 3.1 Data Analysis and Processing To extract key information, we first filter the source data, comprised of four datasheets (spot, character, event,andartwork), to exclude Figure 2: The workflow of the prototype design. entries that correspond to the empty keyword in the event sheet. Then, we categorized the analysis tasks into four aspects according to the sheet themes. This process is iterative, involving a gradual refinement of tasks to align with the criteria of being fundamental, pertinent, and comprehensive (refer to Appendix A), followed by a subsequent focus on narrowing the scope. Once this preliminary refinement was completed, we conducted an in-depth examination of the historical information pertaining toJinlingacross various dynasties. This phase equips us to identify key design directions. We mainly use Python to process the data. 3.2 Game Setup We incorporate four types of exploratory tasks in this prototype which link to the learning goals. The visualization elements related to these tasks will be presented in detail with vivid display after scanning the tangible interfaces via AR technology. Game mechanisms are illustrated using a flow diagram shown in Appendix B. 3.3 Visualization and Interactive Game Design Regarding the overall design, the game seamlessly integrates situated visualization, tangible objects, and AR technology. The situated visualization superimposes additional visual information adjacent to, above, or around physical landmarks. This technique encourages users to delve deeper into key data, fostering a comprehensive grasp of information, facilitating exploration, and promoting effective communication. Tangible chess pieces and dice enhance interactivity and promote user engagement and enjoyment throughout the game process. Additionally, users can augment their experience by scanning the tangible map or AR cards, immersive themselves in a 3D view that provides detailed information, thereby enriching their understanding of cultural data. This integrated approach enhances the overall game experience and ignites a sense of curiosity and motivation for learning. 3.3.1 Physical Design The game’s physical design consists of a map, event cards, character chess pieces, and dice (Figure 1). Users can progress along the map route by rolling dice. Additionally, to delve into cultural knowledge, users are required to interact with event cards and accomplish the tasks they present. To present rich historical and cultural information, we have incorporated various visual elements to encode the information, as shown in Table 1. The inclusion of spot symbols, such as temples or lakes, allows users to easily identify the geographical categories of different spots. Building upon this, we have 87 Authorized licensed use limited to: TU Wien Bibliothek. Downloaded on October 26,2024 at 15:58:19 UTC from IEEE Xplore. Restrictions apply. Table 1: Visual elements and coded information in map visualization. visual elementdata informationexample color of the connecting linedynasty thickness of the line number of the artworks in each dynasty spot symbolspot type color of the mountainlandscape type number of mountains number of relevant figures number of birds number of relevant artworks integrated other symbols that align with the geographical context (i.e., mountains, birds) to convey additional cultural data, resulting in a coherent and well-designed informative map. In addition, distinct types of game cards also present the amount of tacit knowledge about events, figures and their artworks (examples are shown in Figure 3): Event cardcontains event themes, sub-events, and related spots. For example, the theme “a forbidding strategic point” consists of six literary events, one activity, and two allusions, statistics by corresponding icons. Character cardcontains the character’s identity, name, style name, dynasty, year of birth and death, portraits, artworks, and related spots. Heart icons represent the lifespan, while book icons represent the number of artworks, and the card background color is matched to the dynasty connector line on the map. Artwork cardcontains the artwork’s name, dynasty, genre, author, content, related events, and related spots. Furthermore, the poetic image is presented in the visualization. Figure 3: Physical design of the AR board game. 3.3.2 System Design Challenge-based Learning informed efficient and effective guidance for learning, also known as the challenge learning framework [15]. This framework comprises three interconnected stages:engage, investigate, andact. For system design, here we describe how to integrate the game features and visualization elements into these stages of CH learning [G4]. In theengagestage, we employ a map board and AR cards to motivate and guide users into the game, transforming the abstract knowledge into concrete and actionable challenge tasks (Figure 4a). Movement and interaction are required in the form of rolling dice and trigger the visualized augmented information and task lists via cards and map patterns (Figure 4b and 4c). Users are encouraged to explore various locations on the map, where they can engage with data and acquire cultural knowledge related to each spot [G1]. In theinvestigatestage, users follow the task and information cue, participating in a continuous collection journey to explore the solutions for learning challenges. This process requires the user’s “effort” (rolling the dice or using the jumping point prop reward) on (a)(b) (c)(d) (e)(f) Figure 4: Game design: (a) random event card draw; (b) AR details triggered by moving into a new spot; (c) task lists (left) and information cue (right); (d) character card acquisition by random drawing; (e) view of AR information and associated prop reward for reaching a jumping point; (f) gallery with artworks and historical figures. (a)(b) Figure 5: Artwork information display: (a) the development of different types of artwork in each dynasty; (b) an overview of emotional tones with bubble charts in timeline visualization. moving to some specific locations or getting the chance for drawing the cards to collect the figures or artworks (see Figure 4d and 4e). We employ extra rewards in the gallery at the end of each sub-task to introduce new knowledge points (Figure 4f), such as artwork
INCLUDE
To create knowledge from data, it is important to understand the characteristics and meaning of data, but the degree of understanding differs depending on the person's ability and knowledge. If there is a method that can visualize not only its values but also its characteristics and meaning, it could decrease the dependence on the user’s abilities and knowledge. This paper presents a study on a method to incorporate such design in the software development process. In particular, a design of a user interface of the software for museum curators to analyze data of museum visitors is presented and analyzed. The data contains various characteristics, such as technical limitations of sensor performance, installation locations, measurement range, and frequency. The methods used to communicate them are presented and their effects are evaluated through user evaluation. Keywords—Human-Computer-Interaction, Knowledge Acquisition, Experience Design I.
INCLUDE
.We present VEDI (Vision Exploitation for Data Interpretation), an integrated system to jointly assist the visitors of cultural sites and provide meaningful statistics about the visits to the managers of the sites. To address both goals, VEDI includes a wearable assistant (implemented through a wearable device such as HoloLens) which leverages Computer Vision algorithms to understand where the user is and what they are paying attention to. At the visitor’s end, such information is leveraged to augment the visit by displaying additional information on the observed points of interest, helping the visitors to navigate the site and suggesting what to see next. Concurrently, a back-end extracts high-level behavioral information from the captured video content which is used to provide the site manager with meaningful statistics and performance indexes on the cultural site. Experiments show that VEDI achieves good results on both the indoor and outdoor cultural sites considered for the experimentation. Keywords:First person vision ·Egocentric vision· AI for cultural sites 1
INCLUDE