DOI:10.1145/2786567.2801607

Extracted Abstract:

The evaluation of location-based mobile learning (LBML) concepts and technologies is typically performed using methods known from classical usability engineering, such as questionnaires or interviews. In this paper, we argue that many problems that may occur during LBML become apparent in th e learner’s spatio-temporal behavior (i.e., her trajectory). We systematically explore how location tracking and spatial analyses can be used for the evaluation of LBML. Examples with trajectories recorded during a real learning session are presented. Author Keywords Location-based mobile learning, trajectory analysis, learning evaluation ACM Classification Keywords Collaborative learning, computer-assisted instruction, computer-managed instruction, information systems education MOTIVATION The positioning and multimedia capabilities of current mobile devices have given rise to novel learning paradigms that integr ate th e learner’s position in th e didactical concept, thus enhancing learning by the discovery of phenomena in situ. We refer to this kind of learning as location-based mobile learning (LBML) [1]. Integrated LBML management systems, such as the one presented in [2] support the teacher in developing LBML lessons, as well as in the easy dissemination of these lessons to the learners’ devices. At the same time, the LBML management system stores content created by learners on a server, such as geo-tagged photos or Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third- party components of this work must be honored. For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s). MobileHCI '15 Adjunct, August 24-27, 2015, Copenhagen, Denmark ACM 978-1-4503-3653-6/15/08. http://dx.doi.org/10.1145/2786567.2801607 WorkshopsMobileHCI'15, August 24–27, Copenhagen, Denmark 1212 The following section reviews related work on the evaluation of LBML and trajectory analysis. The section after the review describes how trajectory analyses can contribute to the evaluation of LBML, as well as privacy issues. The paper finally concludes the findings with a discussion and outlook. RELATED WORK Evaluation of LBML Vavuola and Sharples proposed six challenges in evaluating mobile learning [6]: capturing and analysing learning in context and across contexts, measuring mobile learning processes and outcomes, respecting learner/participant privacy, assessing mobile technology utility and usability, considering the wider organisational and socio-cultural context of learning, and assessing (in)formality. We will discuss in the final section how the method proposed in this paper helps to approach these challenges. Several researchers have reported results of user evaluations of LBML, typically using methods known from classical usability engineering: Naismith et al. [7], for instance, describe their work with a complete context-aware educational resource system for outdoor tourist sites and educational centres (CAERUS) which was evaluated by a short questionnaire followed by a semi-structured interview on users’ experiences. Another study was performed by [8] on the augmented reality game “Environmental Detectives”, simulating a toxic spill for which students had to find the source. This study was evaluated through each team presenting their findings in front of the class, as well as through cross-team interviews and written short reports. A different evaluation approach was chosen by [9] who evaluated the “Augmenting the Visitor Experience” project through direct observation by the researchers and an analysis of in-field video diaries. Summarising, most evaluations of LBML were performed through questionnaires, interviews, or by evaluating the learning results. However, these evaluation methods require high effort and feedback to the learner has been hardly provided instantaneously. A specific group of LBML approaches targets the improvement of students’ spatial skills, such as understanding cartographic maps, improving orientation and wayfinding, or general spatial thinking. One example by [10], found in a study using a navigation game (Ori-Gami) that the interaction with the map was more intense (more touches) for children who made more errors in orientation and wayfinding. Those errors, as well as the average distance and speed, were determined by analysing GPS tracks. In this paper, we argue that spatio-temporal analyses can also help to evaluate LBML with learning objectives different from spatial thinking. The Ori-Gami example underlines the necessity to integrate spatio-temporal analysis functionality into LBML platforms. Trajectory analysis of moving objects Many scientists have conducted research on the physical activity and movement of human beings. Trajectory analysis has found particular interest in the fields of geographic data mining and wayfinding. Often, their goal is to understand the decision making processes, interests and activities of individual persons or crowds. Andrienko et al. [11] identified three different types of movement data and related analysis tasks: movements of a single object (e.g., one pedestrian’s navigation from A to B), movements of multiple unrelated objects (e.g., the daily commuting behavior of all inhabitants of one city), and movements of multiple related objects (e.g., an animal herd looking for food). The typical analysis goals, tasks, and methods for these three types differ, and most of the papers found in literature fall into exactly one of the three categories. We will use this categorization throughout our paper. For the movement of a single object, [12] describe the following typical analysis tasks: extracting significant places, times and durations of visits, typical trips, their times and durations, deviations, and their reasons. They distinguish between single events and trajectories (temporally ordered sequences of positions). For multiple unrelated objects [13] introduce the following analysis tasks: 1) studies of space use, accessibility, permeability, connectivity, major flows, typical routes between places, and 2) studies of emerging patterns of collective movement: concentration/dispersion, convergence/divergence, and propagation of movement characteristics. In our work, we consider a variety of spatio- temporal properties of both, single trajectories and multiple trajectories. Several approaches for spatial analyses of photo collections have been considered by [14]. Geo-tagged photos comprise a particularly interesting data source because, in addition to the photographer’s position, his or her object of interest can also be extracted from the data [15]. Photo tasks are common in LBML since they encourage learners to explore their environment and direct their attention to the real-world phenomena of interest. By uploading these photos to a server a (geo-referenced) collection becomes available for analysis. Attractions of interest to tourists were identified by [16] with different profiles who were visiting a tourist destination such as Hong Kong. Tourist managers are interested in what locations are preferred by different groups of tourists and what travel routes they are likely to take when visiting different locations. The authors presented a method for constructing a travel dataset from geotagged photos on Flickr (popular websites for sharing photos). A dataset containing thousands of photos with temporal and geographic information attached enabled them to capture the movement trajectories of tourists on a larger scale. Two techniques, a density-based clustering algorithm (P-DBSCAN) and Markov chains, were used to mine travel behavior patterns from this dataset. In addition, the third type of movement analysis – the analysis of relative movement of related objects (approaching, encountering, following, evading, etc.) – has WorkshopsMobileHCI'15, August 24–27, Copenhagen, Denmark 1213 been investigated. For instance, the RElative MOtion (REMO) approach proposed by [17] targets the analysis of motion based on geospatial lifelines of related moving objects. Motion patterns help to answer questions, such as the identification of an alpha animal in a pack of GPS- collared wolves, or the detection of strategic and game-play behavior of two football teams, where the trajectories of 22 players were recorded with a sampling rate of 1 second. The basic idea of the analysis is to compare the motion attributes of point objects over space and time, and thus to relate one object’s motion to the motion of all others. ANALYSING LEARNERS’ TRAJECTORIES As described in the previous section, we structure the following discussion based on the classification of movement analysis tasks by Andrienko & Andrienko [18]: analysis of the movement of single learners, analysis of the movement of multiple unrelated learners, and analysis of the movement of multiple related learners, i.e., learners moving in a group. The distinction between single users and group users plays an important role for LBML: single users learn alone and independently, and traverse the learning area on their own. If a teacher is involved in the LBML process, information sharing happens indirectly and delayed. Because phenomena perceived during the LBML process crucially contribute to a holistic understanding of the learning content, a single user might have difficulties to classify an impression as important or unimportant due to missing second opinions. Furthermore, it can be boring to fulfill vastly interactive location-based tasks alone. In this way, free exploration is highly constructivist and might increase motivation more than executing a learning module along a predefined path. In contrast, group users interact with each other and must take decisions together. Thus, group users can obtain social competence while finding solutions within a debate by compromise or by assertiveness. Often, self-assertive individuals try to act as map leaders. In contrast, other group members risk becoming followers by avoiding conflicts and by evading group decision-making. One advantage of groups is the direct and immediate share of impressions, which might contribute to the holistic understanding of the learning content. Groups may be put together randomly or based on common intrinsic or extrinsic motivational factors. In the following, we describe how to apply spatio-temporal analysis to learners’ trajectories. In addition, we discuss in how far spatio-temporal analysis might be useful to evaluate LBML classes post-hoc. Examples are mainly taken from an LBML project for architecture students at university level by [2]. Movement of single learners Two types of location tracking can be found in LBML projects and the type of tracking significantly influences the kind of analysis that can reasonably be applied: Seamless tracking, as done by a GPS logger, is typically implemented as a background service recording location data at a regular frequency, often chosen between one and several seconds. Sometimes, seamless tracking is considered as too privacy offending, too battery straining, or simply not possible due to missing hardware capabilities. In these cases, an alternative approach to location tracking consists in recording a position every time a specific function is called, such as when taking a picture [16] or solving a task. While the recording frequency of function-dependent location tracking is typically much lower, additional (task-related) data are recorded which can help in the semantic interpretation of the track point, i.e., finding the reason for the stop. The two tracking methods can also be combined. The logged data consist of information about location and time, from which additional spatio-temporal characteristics can be derived, depending on the recording rate (e.g., speed, acceleration, curviness, curvature, sinuosity, etc. [19]). These can be indicators for the reasons why decisions were made. Obviously, the higher the recording rate, the more valid conclusions can be drawn. For instance, acceleration or deceleration could provide evidence of the learner’s uncertainty, time pressure, or (missing) motivation. This, however, may be dependent on the learner’s social and cultural background. Another indicator for uncertainty could be a “zig-zag” path which could mean that the learner had problems finding the target or understanding the map [10]. However, in cases where exploration of the environment is part of the intended learning behavior, a “zig-zag” path is part of envisioned location-based learning. By correlating the path with spatial knowledge about the area, intended “zig-zag” or decelerations can be distinguished from those indicating problems. Often the teacher expects learners to follow a certain path and to take a certain means of transportation. In that case, a spatial analysis can reveal deviations from that path, or transportation mode respectively, for which several reasons may exist: wayfinding problems (see above), changes in the environment (e.g., a construction site or flooded area), unclear communication on the path to take, or physical activity avoidance behavior. For identifying the reason additional sources need to be used (including simply asking the student). A challenge, however, consists in determining whether learners who stayed on the intended path did really perceive the real-world phenomena the teacher expected them to pay attention to. As an example, Figure 1 shows a seamless track of one individual person who was supposed to visit as many shows and species as possible in the zoo in Nuremberg (Germany) within a four hour time limit. The trajectory in this case allows the teacher to investigate places in which the learner was moving rather fast, showing less interest in certain species (e.g., close to the lakes in the South), and which species or shows he spent more time with. In the example, WorkshopsMobileHCI'15, August 24–27, Copenhagen, Denmark 1214 the subject has visited the majority of the available shows, thus fulfilling at least part two of the learning goal. Analysing very carefully, the tracks also reveal stops when the visitor interrupted the task to eat, drink, or rest. Such information can be valuable to estimate the time a teacher should plan for the task. Thus, the evaluation of seamless tracking paths can provide helpful information for the improvement of lessons. Figure 1. Learner exploring the zoo in Nuremberg (basemap: © OpenStreetMap). Figure 2. Recorded track on the Former Nazi Party Rally Grounds, Nuremberg, Germany (today a museum site) (basemap: © OpenStreetMap). Another example is displayed in Figure 2. After a theoretical