1 Introduction
Mapping and modelling in the geosciences field have grown rapidly in the past 75 years (Mossa et al., 2019). The field has evolved from predominantly 2D cartography moving into 3D geovisualization (Kraak, 2003). Geovisualizations are created using spatial tools and algorithms with spatial data (Papadopoulou et al., 2021); it is a visual tool for data exploration (Forsythe et al., 2021). This paper identifies limitations in the research and commercial applications of geovisualization while offering an evolving framing of geovisualization and interfaces and how they may grow into opportunities for geoscience to progress. The paper offers a road map vision of future opportunity for geovisualization in the context of the geosciences.
The Earth is not a 2D object - its structures and systems are geometrically three-dimensional, and four-dimensional when one includes change over time. These propositions may seem obvious, but they provide a context that led to the emergence of ‘geovisualization’ and subsequently, ‘geovisual analytics’—fields that focused on developing multidimensional, interactive methods to support the analytical needs of 3- and 4-D spatial data analysis and visualization. (See (Robinson 2017) for an overview of this evolution).
The earliest definition of geovisualization comes from (MacEachren and Monmonier, 1992) and is defined as “direct depiction of movement and change, multiple views of the same data, user interaction with maps, realism (through three dimensional stereo views and other techniques), false realism (through fractal generation of landscapes), and the mixing of maps with other graphics, text and sound”.
As cartography and mapping evolved, the lines of definitions began to overlap and the boundaries of cartography blurred (Dykes et al., 2005). MacEachren and Kraak noted that “at one level, all mapping can be considered a kind of visualization - in the sense of making visible” (Maceachren and Menno-Jan Kraak, 1997), aligning with the MacEachren and Monmonier statement in 1992 that “all map use involves both visualization and communication”. As cartography evolved, so did geovisualization, as it is founded on cartography.
Propelled by the use of technology and increases of data, geovisualization found its foothold, drawing from the disciplines of cartography, scientific visualization, data analytics, and GIScience. The links between modern computing and visualization date back to the early 1990s (MacEachren and Monmonier, 1992) and continue to become more evident through the early 2000s as experience was acquired from geovisual software and new instruments (Dykes et al., 2005), and new advantages were gained through higher resolution data and faster computing (Mitasova et al., 2012). This has continued to grow as vast sets of data are acquired and computing continues to advance (Carbonell-Carrera and Hess-Medler, 2019).
Influenced by technological advances, and increasing volumes of data, geovisualization, and its definition, has evolved throughout the years. Evolving from the definition given by (MacEachren and Monmonier, 1992) geovisualization grew to be more about the process and the acquisition of knowledge, as opposed to the strict definition of information and graphics. (Dykes et al., 2005). defines geovisualization as being about “people, maps, process, and the acquisition of information and knowledge” and can “lead to enlightenment, decision making”. The boundaries of cartography, and the nature of map use and visualizations broaden and welcome in a new era and definition of geovisualization as experience through technology and new volumes of data is gained (Dykes et al., 2005).
The emergence of visual analytics (Wong and Thomas, 2004; Thomas and Cook, 2005; Keim, 2008) resonated with the field of geovisualization, and its scope from the early 2000s onwards. In the foundational volume on visual analytics (VA) Thomas and Cook envisioned VA as “the science of analytical reasoning facilitated by interactive visual interfaces.” (2005).
Much of what VA embodied and strived for, was familiar to the geovisualization community, but it helped to legitimize and integrate a set of burgeoning subfields of geovisualization research that were on the rise (such as immersive interfaces, and new directions in interface-based spatial cognition). It made total sense that geovisualization scholars initiated geovisual analytics (geoVA) to make a significant and distinct contribution to this movement (Andrienko and Andrienko, 2007).
The need for visualization to extend beyond traditional methods and to combine with computational analysis, and other computer based operations is noted in (Andrienko and Andrienko, 2007). Visualization is shown to play an increasingly important role, as it allows for background knowledge of the human analyst to be used (Andrienko and Andrienko, 2007). Reaching beyond just geovisualization, and into scientific visualizations, this importance of visualization is shown to aid in analysis and communication (Mitasova et al., 2012). These improvements continue into present day, where it is shown that geovisualization can improve spatial thinking and decision making (Carbonell-Carrera and Hess-Medler, 2019).
As boundaries are pushed and technology grows, the volumes of data increase and geovisualization has evolved to become a cornerstone to decision making and spatial thinking. Drawing from many disciplines, including cartography, it has grown beyond its roots of a “direct depiction of movement of change” and the “mixing of maps with other graphics” (MacEachren and Monmonier, 1992) to include the people, processes and decision making aspects (Dykes et al., 2005), and become founded on the evolution of computation and data (Andrienko and Andrienko, 2007; Mitasova et al., 2012).
The 30-plus year evolution of geovisualization has always been driven by a need for spatially analytical visualizations that support ideation, analysis, interpretation and communication., informed by both deep roots in analytical cartography, and spatial analysis with mainstream GIS. But what geovisualization offers now, versus 18 years ago, let alone at its inception 30 years ago, far exceeds the scope of its formative context. Advances in data acquisition/generation technologies have transformed our ability to record and reconstruct geographic spaces and phenomena. More recently, the latest wave of spatial interface technologies (virtual reality, augmented reality, mixed reality, extended reality, have delivered far more mature and durable platforms on which to iterate and develop meaningful geovisual information experiences.
As illustrated above, geovisualization has become far more than “simply a synthesis of the long-developed visual communication of cartography with current digital analytical technologies, principally GIS” (Smith, 2022). The preceding snapshot of the scope, context and evolution of geovisualization are not intended as an exhaustive review. Rather, our aim is to illustrate how contemporary geovisualization, and the many facets of it, are the result of an ongoing, dynamic evolution. And that the scope and definition of geovisualization are in constant flux. This quote is not intended to criticise the interest, but to offer an even deeper framing of geovisualization, that has utility for Earth sciences.
Advances in the capabilities, integration and application of enabling technologies have in turn led to an expansion in analytical methods. Perhaps one of the most exciting parts of this evolving geovisualization ‘landscape’, has been the way in which new opportunities with visualization technologies and methods, has led to exciting new vectors in spatial information design, spatial interface technology design, interaction design and information experience design applied to spatial phenomena (Lonergan and Hedley, 2014; Fenech et al., 2017; Rydvanskiy and Hedley, 2020; Lochhead and Hedley, 2021). These have all led to a new expanding Frontier of variously immersive and interactive geovisualizations (Coltekin et al., 2019). These developments have in turn elevated interest and research into the relationship between users and spatial phenomena - mediated by their perceptual experience with spatial data in new spaces of analytical visualization (Lochhead et al., 2022).
Geovisualization offers methods through which geoscientific research and communication may be enhanced–through their ability to deliver rigorous, analytical multidimensional visualizations of geoscience data and phenomena at any spatial or temporal scale. While legacy planar mapping methods may suit some applications, 3D/4D geovisualization approaches have considerable value for fields such as geological mapping, rock fall mechanics, and flood plain visualization–where an ability to analyze structure and dynamics in three and four dimensions is a core need.
In the context of this paper, the geosciences are framed as a research endeavour where physical Earth systems are being studied. It is important to note that these definitions and disciplines overlap frequently with geographical research domains, and in real life situations such as in riverbed morphology or natural hazard studies. This paper explores geovisualization in the geosciences in the context of characterizing and studying the Earth and Earth systems. As the data with which we characterize 3 and 4D has become more sophisticated, so too have the opportunities to utilize these data not just in visualizations but in analytical simulations (and that can be experienced using a range of interfaces). Exploratory work of this type is increasingly being conducted by practitioners at the overlapping borders of the geosciences and geography.
Geovisualization is critically valuable to geoscience modelling is necessary for analysis and communication of data, encompasses a broad range of formats and visual representations, and goes beyond data representation into the exploration of data through “interactive and dynamic interfaces” (Marmo et al., 2010). Geovisualizations are used to enhance visual thinking about geospatial relationships (Kraak, 2003). Geovisualization can reach beyond the formal definitions such as those listed above from Marmo, Cartwright and Yuille and Kraak. At its heart it is the visualization of geospatial data and works to enable simulation and exploration of said data. It has many defining characteristics and in the context of this paper, geovisualization as the general simulation and exploration of geospatial data to enhance thinking will be explored. It enables and develops spatial thinking which can furthermore benefit the analysis of complex geoscience situations such as rockslides, flood mitigation, and contamination migration (Carbonell-Carrera and Hess-Medler, 2019). Unlike generic use of spatial analytical software to output a standard visualizations of inputted data, 3D (and 4D) geovisualizations can be intentionally designed and tuned to the specific characteristics of a geoscience phenomenon, to the needs of users across the whole scientific process. Geovisualization outputs (and the information science behind their implementation) therefore, powerfully support the communication of data, geovisual analysis exchange of data and information between stakeholder groups including public officials, citizens, and experts in the field (Jacquinod and Julia Bonccorsi, 2019).
Traditionally, geoscience data were represented in the form of 2D maps, but has been evolving to different representations. Data collected for pseudo 3D is being represented in the form of digital elevation models (DEM), hillshades, slope maps, triangular irregular networks (TIN), among others. However, 3D data is now being represented with all three coordinates (x, y, z) in the form of point clouds, generally collected using Light Detection and Ranging (LiDAR) (Sarakinou et al., 2016).
The evolution of 3D/4D spatial data acquisition technologies (multibeam, LiDAR, MVS-SfM photogrammetry) has created immense opportunities for geoscientists. As a result, 4D spatial reality capture, data production and geovisualization (though not always labelled ‘geovisualization’) have become key pieces of geoscience characterization, analysis and communication.
Contemporary sensors and technologies offer methods that have the capability to record and reconstruct the 3D structure (and 4D change/dynamics) of geoscience phenomena with greater representational fidelity than the inferential dimensionality of pseudo 3D, and abstracted representations of historical 2D planar methods (that were largely inherited from cartography).
2D and pseudo 3D models have several shortcomings in the geosciences. Pseudo 3D is less effective in communicating complex terrain information than 3D (Forsythe et al., 2021), a shortcoming that occurs when the data is transformed. 3D features undergo distortion through transformation, which can affect accuracy and analysis of the results (Ahmed et al., 2021). Difficulties exist when presenting complex information using 2D methods (Juřík et al., 2020). Additionally, historic 2D data, both in maps and databases, can lack precision and accuracy. Previous cartographic models for geoscience applications can lack precision and the ability to communicate and analyze complex information; this has given rise to the popularity of 3D geovisualization.
Note that the authors recognize that the performance of contemporary methods over traditional ones, is not only the result of technology. It is also a function of the quality of information design, execution of the methods, as well as understanding the needs and capabilities of end users. These and other considerations must form part of a commensurate 3D/4D spatial information science that informs the integration and use of cutting-edge geovisual technology and methods in the geosciences.
The paper reviews the methods used to select and analyze the papers, provides an overview of the current state of practice and applications, reviews the limitations and key issues in the current state of practice and then moves to future developments in the field, followed by conclusions.
3 Results
3.1 Geoscience data acquisition
The geosciences acquire data from a number of sources including photogrammetry and LiDAR (Light Detection And Ranging) to analyze the morphological and geological landscapes (Guerin et al., 2020; Westoby, 2020; Romeo et al., 2021).
Geovisualization and modelling has come a considerable way in the past 70 years, particularly since GIS took hold in the 1990s and SAR monitoring gained popularity in the past 2 decades (Kraak, 2003). In direct linkages, geological modelling has followed this evolution to become more advanced. The advances in geosciences and geological modelling are greatly spurred by the evolvement of LiDAR, photogrammetry, and other sensing techniques, generating increased volumes of data to analyze (Havenith et al., 2019).
These sensors advance the field at large and provide new opportunities to manipulate and analyze data in 3D and 4D environments (Havenith et al., 2019). Geomodelling is frequently being used to understand and interpret geological hazards such as rockfalls, floods, and earthquakes. The majority of these models use data derived from a combination of SAR measurements and field work, then rely on numerical modelling and qualitative assessments to determine hazard risk (Jordan et al., 2017; Francioni et al., 2018; García et al., 2019; Yao et al., 2021). Though these solutions exist there is a need for more dynamic interpretation methods. The capabilities of GIS and numerical modelling are limited in both the temporal realm and 3D space (Havenith et al., 2019). GIS predominantly uses pseudo-3D modelling as opposed to real 3D and therefore, cannot capture the complexities of real 3D space and data, in instances such as flood mapping where the transformation of 3D data to pseudo-3D can affect the flow patterns and results. Similarly, numerical modelling is but a snapshot in time and fails to capture long term progression or time-lapse simulations the way a physics game engine can simulate. GIS and numerical modelling pose limitations when working with real 3D space and time lapsed activities such as rock falls or tsunamis.
3.2 Sensor mapping and integration
Autonomous vehicles have been adopting LiDAR to sense their surroundings in urban environments (Wang et al., 2019). However, sensor technology can also be used to map remote areas (such as by using a specialized off-road vehicle) and gather data to be used in geomorphological studies.
Studies are being conducted using Mobile LiDAR Sensors (MLS) that are ground-based, airborne, and water-based (e.g., boats). Several studies involve the mounting of MLS on boats to acquire topographic data for fluvial research, creating a spatially dense point cloud by integrating navigational and data acquisition sensors (Alho et al., 2009). MLS on boats can reach accuracies of 1–4 cm (Vaaja et al., 2013). MLS requires efficient and automatic processing, which can be done through noise point filtering, and automatic identification and point classification (Vaaja et al., 2013). Boat based LiDAR can also monitor coastal sites where airborne LiDAR does not offer adequate coverage. The near horizontal view of the laser allows for high coverage and accuracy of steep topographical surfaces (Kaminsky et al., 2014). An MLS has also been employed to map railways, as a specialized mobile mapping tool that also functions for general surveying.
Mounting sensors to vehicles on rail networks has proven to be a powerful way to generate datasets along a fixed, and repeatable, path. The RailMapper managed to execute its mapping with a high level of accuracy and a precise location identification system using a coupled Global Navigation Satellite System (GNSS)—Inertial Measurement Unit (IMU) system (Kremer and Grimm, 2012).
MLS is also being used in ground-based systems to evaluate slope mass rating and monitor rockfalls. Existing approaches involve the combination of multiple remote sensing methods, including LiDAR and photogrammetry, to scan slopes. While Terrestrial Lidar Sensors (TLS) and Airborne Lidar Sensors (ALS) are increasingly being used, minimal real time processing has been conducted, leading to gaps in the use of MLS and ALS for hazard monitoring. However, MLS is increasingly being used to monitor slopes and reach inaccessible areas through ground based exploration, to support development of a rockfall monitoring system (Romeo et al., 2021). Ground based systems are also being used to map forests (Pierzchała et al., 2018; Meadows et al., 2019; Mokroš et al., 2021). TLS is time-consuming, labour intensive, and computationally demanding when developing a forest inventory but the use of MLS in combination with SLAM is being explored to measure key variables, such as number and size of trees, and while immature, poses a strong future (Pierzchała, Giguère, and Astrup, 2018).
MLS is providing better opportunities for inaccessible areas in comparison to ALS and TLS by allowing movement not tethered to a static object or only gaining an airborne angle. TLS allows the collection of data at a higher resolution but poses spatial limitations, while ALS can cover a wide area but does not have the same high point density as TLS. MLS combines these strengths, measuring objects with an accuracy of a few cm to tens of meters and presenting the ability to cover large spaces (Alho et al., 2009). In addition, ALS has poor coverage on vertical surfaces, while MLS can cover high relief topography with greater ease (Kaminsky et al., 2014). While ALS is the most widely used method, and has good visibility, it also comes with a high price tag, MLS provides similar potential range capabilities with a lower cost attached (Kremer and Grimm, 2012). TLS can be time consuming and labour intensive when gathering a high point density, while MLS is immature and has less studies, it can maximize the density to the same level as a TLS survey (Pierzchała, Giguère, and Astrup, 2018). MLS has been developing rapidly, and while it has several issues with regards to its extensive processing and analysis due to high volumes of data and noise, as well as its object recognition, it provides promising opportunities for monitoring in fluvial and geomorphological environments (Vaaja et al., 2013; Kaminsky et al., 2014; Wang et al., 2019; Romeo et al., 2021).
In order to successfully integrate sensors with vehicles for effective mapping and modelling capabilities, the analytical data visualization must be agile, and be able to connect to different programs and operate in a variety of scenarios. Existing research demonstrates that interconnected visualization infrastructure can enable new forms of geovisualization analysis. Immersive data exploration on Mars (Lochhead and Hedley, 2021), coastal environment simulations (Lonergan and Hedley, 2015; Lonergan, Hedley, and Clague, 2015), and mixed reality flood simulations (Rydvanskiy and Hedley, 2020) present the diversity of scenarios where emerging geovisualization technology can aid in understanding. Sensors and data can prove key to the operator’s situational awareness in vehicle based sensing, as well as benefitting data collection and interpretation, as shown in the Mars rover project (Helmick, Angelova, and Matthies, 2009). Integration is essential to successful operation and providing elevated geovisual analysis methods.
3.3 Geoscience modelling
3.3.1 GIS-based
GIS-based software such as ArcGIS and QGIS can be used to create both 2D and 3D visualizations. GIS can take in multiple data types including point clouds collected from LiDAR, historical maps, shapefiles, among others. The simulations can be interactive or static and have a variety of applications. GIS-based 3D visualizations have been used for mapping alluvial sites, modelling elevation and topography, measuring landslide vulnerability, and mapping water depth (Masse and Christophe, 2015; Heitzler et al., 2017a; Mossa et al., 2019; Wahyudi et al., 2020; Ahmed, Mahmud, and Tuya, 2021). These techniques using GIS have demonstrated that 3D visualizations can provide new insight alongside their 2D counterparts (Mossa et al., 2019), can provide additional geographic data, and model virtual representations of real events (Wahyudi et al., 2020).
3.3.2 3D modelling software
3D modelling software is being used more frequently, moving beyond the realm of GIS. It takes in point cloud or image data through methods such as LiDAR and photogrammetry and generally produces interactive visualizations. There are many different types of 3D modelling software from game engines such as Unity, applications such as AutoDesk or SketchUp, FARV3DPT, Spaceyes 3D, and Agisoft Metashape. These tools provide a wide range of options to explore three-dimensional data.
A wider range of applications, particularly those with a temporal component, have been used in 3D modelling software, as opposed to GIS. Modelling software has been used for active events and simulations such as landslides, flooding mitigation, and rockfalls (Jacquinod and Julia Bonaccorsi, 2019; Forsythe et al., 2021; Nordvik et al., 2009; G; Zhang et al., 2020). It has also been used in static instances to create more in depth maps and 3D recreations of coastlines, bathymetry, and terrain (Sarakinou et al., 2016; Papakonstantinou et al., 2018; Papadopoulou et al., 2021; de Magalhaes et al., 2020).
There are numerous benefits to using 3D modelling software over existing GIS techniques. These include the use of a physics engine in game engines to model collisions and liquid behaviour (Sala, Hutchinson, and Harrap, 2019). In addition, they utilize real 3D over the use of pseudo 3D which can be beneficial in understanding complex shapes and generating spatial awareness (Juřík et al., 2020; Wahyudi, Ramdani, and Bachtiar, 2020). Both GIS-based and 3D-model software are viable options for generating geoscience models, and in combination they can be used even more powerfully.
3.4 Extended reality
The current state of practice in extended reality geoscience modelling has been advancing rapidly. Extended reality aids in the effective communication of complex three-dimensional data. Geological data is complex and layered and loses fidelity and becomes more difficult to understand when communicated in 2D. Extended reality helps communicate 3D information naturally, in an efficient way, to experts and laypeople alike (Janeras et al., 2022).
Virtual mapping and monitoring in conjunction with extended reality have been used in several studies. For instance, the RIMS system, Real-time Interactive Mapping Software, generates 3D models from DEM and texture data and allows measurements and geological reconstructions in an interactive 3D environment (Bernardin et al., 2006) and has been used to model the region surrounding the epicentre of the 2010 Haiti earthquake for practical fault analysis (Cowgill et al., 2012). Augmented reality has been used to model fluids in real time over the Fraser River watershed (Rydvanskiy and Hedley, 2020), and developing mobile augmented reality system for on site analysis of the Snowy River flood plains (Haynes, Hehl-Lange, and Lange, 2018). Augmented reality has also been used to model rock slopes and fuse geoinformation virtually into the slope environment, allowing identification of structural planes and potential rockfall hazard zones (Y. Zhang et al., 2019). Despite having the ability to map rock slopes, the majority of studies do not integrate a dynamic component, instead overlaying static visualizations onto the area.
Extended reality has also been used to communicate data for teaching and learning purposes. XR offers the ability to view remote and inaccessible locations such as volcanic sites, glaciers, and underground caverns or mines for teaching purposes (Janeras et al., 2022). For instance, at the University of Georgia they have developed the 3DIG, 3D Immersion and Geovisualization, to be used for Earth science teaching and learning. It uses a combination of image acquisition, VR, MR, and 2D panels. The 3DIG effectively communicated Earth science information to students and helped enrich their understanding of complex geological environments (Bernardes et al., 2018). In another instance, the Montserrat massif was studied through complex 3D models visualized using the Microsoft HoloLens 2 and Clino’s software (Janeras et al., 2022). The system effectively and naturally communicated complex 3D information to experts in the consulting field (Janeras et al., 2022).
XR offers a rich opportunity to facilitate the understanding of multi-faceted 3D environments. The world of geosciences is 3D, often multi-layered, with many complexities. XR provides the ability to communicate this information for mapping and monitoring, teaching and learning, and general understanding.