To assess the maturity of an ocean observing system the Framework of Ocean Observing has adapted the technical readiness level, a scheme developed by NASA (National Aeronotics and Space Administration) (Sadin et al., 1989), and introduced the ocean observing “readiness level” (Lindstrom et al., 2012). Following this framework, ocean observing should be seen as “[ … ] a chain of processes addressing “why to observe?” (requirement setting process), “what to observe?” (scoping of observational foci), “how to observe?” (coordination of observing elements), and “how to integrate, use and disseminate observational outcomes and understand their impacts?” (data management, analyses and creation and assessment of information products).” (Pearlman et al., 2019). The three pillars of this ocean observing value chain ¹ are: “Requirements”, “Observations” and “Data and Information”. For each of these pillars, FOO defined nine readiness levels and grouped these into the categories “Concept”, “Pilot” and “Mature”. A holistic approach enables the evaluation and classification of an entire ocean observing system in terms of feasibility, capacity, and impact. Here we only use the defined readiness levels for “Data Management and Information Products” ( Figure 1 ). We restrict ourselves to climate quality data since these are strongly tied to high-quality BGC EOV synthesis data products, especially to their quality control procedures.

The nine readiness levels (Lindstrom et al., 2012) are quite general, so to suit the aim of this work, we have developed a criteria catalog (Appendix 1) which forms an objective basis for the evaluation of the individual data products. Applying the catalog assigns (weighted) scores to typical characteristics of data products on a level-by-level scheme. Full compliance with the criteria yields a 100% score for a given level, with 80% being defined as a “pass”. For example, a product passes readiness level 5 if the data management practices are verified and validated through an existing data policy and archival plan. The criteria catalog (Appendix 1) assigns equally weighted scores to “Policy”, “Archival” and “QC Verification”. These, in turn, are linked to specific data product features, such as having a data usage statement for “Policy” ( Figure 2 ). Note that even though the order of levels is structured hierarchically, a data product can meet some requirements of higher levels before fully complying with all lower levels. Since the maturity of a data product is strongly tied to the FAIR guidelines, we have incorporated the guidelines into the criteria. Following Tanhua et al. (2019), a data product is FAIR if it has a unique persistent identifier with enriched and standardized metadata (findable), enabling access to the machine-readable data and metadata (accessible and interoperable), and can be integrated into other data sources (reusable). The degree of the implementation of the FAIR principles is reflected in the order of the FOO readiness levels. The degree of being “fit-for-purpose”, a requirement of the ocean observing value chain, is also incorporated into the criteria catalog.

Given the diverse nature of the data, the criteria have not been further specified and are kept generic on purpose. Workflows and tools used in different products might resemble one another but are tailored toward the specific requirements of the data products. In particular, the data upload (or ingestion) system and quality control methods differ as these are tailored towards the given observing platform, sampling method (continuous or discrete), analysis type, variable (e.g. Johnson et al., 2001; Dickson et al., 2007; Pierrot et al., 2009; Maurer et al., 2021) and stakeholder. Since many research groups and products implement different QC flagging schemes, we have applied a consistent set of quality levels (adapted from ICOS, https://www.icos-cp.eu/data-services/data-collection/data-levels-quality) to describe the data flow and QC of the different products ( Table 1 ). Typical QC examples of the different levels are range tests (level 1), the identification of spikes in space or time (level 2) and the adjustment of known biases (level 3).

The Surface Ocean CO₂ Atlas (Pfeil et al., 2013; Sabine et al., 2013) is an international community-driven effort. It synthesizes in-situ surface ocean fCO₂ (fugacity of carbon dioxide) measurements from ships, moored stations, autonomous and drifting surface platforms and yachts with an estimated accuracy better than 10 µatm. SOCAT increases ocean surface fCO₂ data availability and forms the basis of several other data products, such as the SeaFlux data set (Gregor and Fay, 2021) and diverse scientific applications and assessments. The latter range from ocean and climate model and sensor evaluation, regional process studies of surface ocean fCO₂, the detection and estimation of surface ocean acidification trends (Freeman and Lovenduski, 2015; Lauvset et al., 2015), to the quantification of the ocean carbon sink and its variation (Bakker et al., 2016; Friedlingstein et al., 2022). Thus, SOCAT represents a “[ … ] key step in the value chain based on in situ inorganic carbon measurements of the oceans, which provides policymakers in climate negotiations with essential information on ocean CO₂ uptake” (Bakker et al., 2020; Guidi et al., 2020). SOCAT’s first version (Pfeil et al., 2013; Sabine et al., 2013), was released in 2011 following a call from the international marine carbon community to create a quality-controlled, publicly available synthesis product of surface ocean CO₂ for the global oceans and coastal seas (IOCCP, 2007; Doney et al., 2009). SOCATv2 and SOCATv3 followed in 2013 (Bakker et al., 2014) and 2015 (Bakker et al., 2016), respectively. After the official launch of the SOCAT submission system in September 2015 (SOCAT and SOCOM, 2015), annual product releases have been accomplished. SOCATv2022 includes more than 40 million individual measurements from 1957 to 2021 from more than 100 data contributors (Bakker et al., 2022). The data product consists of 1) the collection of all individual data set files, 2) global and regional synthesis data products, 3) global (monthly, yearly and decadal) gridded products on a 1° latitude by 1° longitude grid and 4) a coastal monthly gridded product on a quarter degree grid. The main synthesis products (2, 3, 4) are based on surface water fCO₂ with an estimated accuracy of better than 5 µatm (33.7 million data points), while fCO₂ values with an accuracy of 5 to 10 µatm are made available separately (6.4 million data points). Recent SOCAT products contain searchable information on the organization where data providers are based, a step towards attributing data sets to funding agencies and countries.

While SOCAT synthesis products are made available via ERDDAP (Section 4.1.1.1), metadata of individual data sets in SOCAT are not yet machine-readable. Planned metadata automation will contribute to the initiative led by the Intergovernmental Oceanographic Commission of UNESCO towards a federated data system for the UN Sustainable Development Goal (SDG, UN, 2015) 14.3 (“Minimize and address the impacts of ocean acidification, including through enhanced scientific cooperation at all levels”). SOCAT also considers to include additional variables to the product, such as atmospheric CO₂, dissolved inorganic carbon (DIC), total alkalinity (TA), pH, nutrients, methane (CH₄) and nitrous oxide (N₂O) concentrations (SOCAT and SOCOM, 2015; Bakker et al., 2016).

The Global Ocean Data Analysis Project was initiated to enable the quantification of the anthropogenic ocean carbon sink (e.g. Key et al., 2004; Sabine et al., 2004; Gruber et al., 2019). To this end, GLODAP focuses on collecting and synthesizing interior ocean data from hydrographic cruises with carbon-relevant data. GLODAP defines carbon-relevant as data that includes at least one measurement of the following: inorganic carbon sub-variables (pH, DIC, TA and/or fCO₂), carbon isotopes (C14 and/or C13) or transient tracers (CFC11, CFC12, CFC113, CCl₄ and/or SF₆). Through multiple layers of quality control, aiming to remove biases between cruises, GLODAP makes cruise data from various sources, from individual projects to numerous larger campaigns, consistent and comparable. With its high internal consistency of the core variables (DIC, TA, pH, nutrients, O₂, salinity and transient tracers), particularly of DIC and TA (± 4 µmol kg^-1), GLODAP has also become a relevant source for other scientific applications and observing platforms. One prominent example is BGC Argo floats, which rely heavily on GLODAP’s high-quality data for validation. The first version of GLODAP, GLODAPv1 (Key et al., 2004), was released in 2004. It mainly included data from the World Ocean Circulation Experiment and Joint Global Ocean Flux Study campaigns as well as other historical cruise data from the Geochemical Ocean Sections Study, Transient Tracers in the Oceans, South Atlantic Ventilation Experiment, and INDIen Gaz Ocean expeditions. In combination with the CARbon dioxide IN the Atlantic Ocean (CARINA) product (Tanhua et al., 2009; Key et al., 2010) and the PACIFIc ocean Interior CArbon (PACIFICA, Suzuki et al., 2013) product, “[ … ] these products formed the natural basis for GLODAPv2” (Key et al., 2015; Olsen et al., 2016). Version 2 benefitted from advancements in data handling, which eventually enabled yearly updates starting in 2019. In addition to the annual updates, GLODAP plan to provide regular full decadal version releases, in concert with the GO-SHIP program (Olsen et al., 2019; Sloyan et al., 2019; Olsen et al., 2020; Lauvset et al., 2021). GLODAPv2.2022 (Lauvset et al., 2022) includes more than 1.4 million samples from 1085 cruises from 1972 to 2021. The data product consists of three pillars: 1) data from the individual cruises in a consistent format with coherent QC and unit conversion, 2) a bias-adjusted data product, and 3) a global 1°x1° mapped climatology. The latter is produced only for the full version releases (the last of which was in 2016).

For the future, “[the] GLODAP team now strive for advancements on two fronts towards a semi-automated system that reduces the work intensity and associated errors. Firstly, implementing a uniform, semi-automatic and standards-compliant data ingestion system that will facilitate the data submission and quality control (QC) procedures. [ … ] Secondly, upgrading to a modern and versatile data extraction system that provide users more flexibility and options [ … ] “(Tanhua et al., 2021).

The MarinE MethanE and NiTtrous Oxide database compiles N₂O and CH₄ measurements and - if available - associated data (such as atmospheric mole fractions, water temperature, salinity, dissolved O₂ and nutrients) from the open and coastal oceans. It provides calculated global and regional concentration fields for the surface and deep ocean in common units and estimates of the air-sea flux density of both gases. Initially starting with a database for N₂O only (Freing and Bange, 2007) a joint initiative between the Surface Ocean Lower Atmosphere Study and European CoOperation in Science and Technology Action 735 (European CoOperation in the Field of Scientific and Technical Research) resulted in the development of MEMENTO (Bange et al., 2009). MEMENTO’s main rationale is to help researchers to quantify the temporally and spatially variable N₂O and CH₄ oceanic distributions and their exchange with the atmosphere. N₂O and CH₄ are important atmospheric trace gases that act as strong greenhouse gases in the troposphere and as precursors of ozone depletion in the stratosphere (WMO, 2018; IPCC, 2021). The MEMENTO data product was used, for example, to model N₂O production and consumption processes on global and regional scales (Freing et al., 2012; Suntharalingam et al., 2012; Zamora et al., 2012). Recently, data from MEMENTO were also used to estimate the global N₂O and CH₄ emissions from the ocean (Weber et al., 2019; Yang et al., 2020). Being publicly available since 2009, MEMENTO cooperates with the Scientific Committee on Ocean Research working group 14.3 since 2014. By November 2021, MEMENTO included more than 120000 N₂O and more than 23000 CH₄ measurements from over 200 measurement campaigns covering the past 57 years of observations.

Besides the ongoing data update, MEMENTO wants to “continuously improve it by including additional meta-​information, allowing additional data formats, and implementing new data quality control criteria.” Further goals include the implementation of “[ … ] standard procedures that are developed within the [SCOR] working group for measuring N₂O and CH₄.” (Kock and Bange, 2015) and an enhanced data archive structure that is more user-friendly.

The main scientific rationale of the Global Ocean Oxygen Database and ATlas (GO₂DAT) lies in the understanding and prediction of ocean O₂ changes at daily to climate scales: “A better knowledge base of the spatial and temporal variations in marine O₂ will improve our understanding of the ocean O₂ budget, and allow for better quantification of the Earth’s carbon and heat budgets, net global primary production and for adopting sustainable fisheries and aquaculture management.” (Grégoire et al., 2021).

The first version of GO₂DAT is “under construction”, but in the recently published roadmap towards GO₂DAT (Grégoire et al., 2021), the GO₂DAT team envisions a consistent and FAIR cross-platform database that targets all available O₂ measurements from the coastal and open ocean from both Eulerian and Lagrangian platforms. Thus, GO₂DAT shall include O₂ measurements from ships (Winkler data and CTD-O₂ sensor data), Argo floats, gliders, moorings, underway sensors and benthic boundary layer data. To tackle the lack of uniformity in data treatments a key characteristic of GO₂DAT will be the definition of a “community-agreed, fully documented metadata format and a consistent quality control procedure and quality flagging (QF) system”. In addition to the database, several regularly updated “stacked” gridded products of O₂ concentration, O₂ partial pressure (pO₂) and the degree of saturation with respect to atmospheric O₂ for the coastal and global ocean with sub-seasonal to multi-decadal resolution, are planned.

GO₂DAT datasets and products will improve our understanding and estimation of the deoxygenation trend and mechanisms. Since 1950 the open ocean O₂ content has decreased (medium confidence) by a few percent (i.e. 0.5-3%) (IPCC, 2019) and the Oxygen Minimum Zones, which are permanent features of the open ocean, are expanding. However, models and observation-based products disagree on the amount and spatial distribution of deoxygenation. Different data sets and mapping procedures explain only part of these differences. In the global coastal ocean, the reference distribution of hypoxic sites is that assembled by Diaz and Rosenberg (2008), showing the worldwide distribution of regions affected by hypoxia at least once, as referred to in the literature. This effort has been valuable but should be updated and amended with the large volume of (sometimes disparate) quantitative information on coastal O₂ concentrations, including inventories of the frequency, timing, duration, intensity and spatial extension of the hypoxic events, and links to the original data contained in a globally accessible database.

The new criteria catalog and scoring system were successfully applied to the four selected data synthesis products. The so-determined readiness level scores and maturity of each product are listed in Table 2 . SOCAT is the most mature product, reaching the “Mature” status by being “Fit for purpose”. GLODAP passes the “Verification” level and represents the only product in the “Pilot” phase. MEMENTO and GO₂DAT are in the “Concept” phase. However, MEMENTO also complies with the “Proof of Concept” level. GO₂DAT is the most recent initiative with the publication of a community-agreed roadmap (Grégoire et al., 2021). At this stage, its maturity is capped at the “Documentation” level. Nonetheless, all living products provide consistent and comparable level 3 data.

During the assessment, we could identify some critical and common approaches, which all four products share, independent of their different foci and state of development. To begin with, it is a pre-requisite for the success of a product to follow a clear mandate, i.e. to have a clear mission. Since the four products are community-driven, this is implicitly fulfilled. All products recognize the importance of not only the synthesis itself but also the importance of accompanying original data and metadata. Also, the importance of known and common standards and a clearly outlined QC is reflected in the individual data products’ workflow. And even though the actual 2^nd QC methods of how to reach level 3 data differ from in-depth metadata checks (SOCAT) to bias corrections (GLODAP and MEMENTO), all products (in-) directly foster the usage of best practices by “rewarding” high-quality data in one way or another.

The diverging readiness levels of the products can mostly be linked to the varying implementations of critical features. Two themes that are reoccurring in the evaluation process are i) the extent to which the principles of FAIR and ii) the degree to which automation processes are incorporated at multiple readiness levels in the criteria catalog. Most prominently incorporated by SOCAT’s automated ingestion and extraction system. In particular its built-in version control, as well as interoperable data access for humans and machines (ERDDAP), fulfill multiple criteria throughout the readiness level catalog. Similarly, GLODAP’s cruise summary table and adjustment table provide good examples of how to increase a product’s maturity. These features should be used as blueprints for other synthesis products. Lastly, we want to stress one essential feature that no product has: long-term funding. This lack hinders the products from becoming fully sustainable and mature and directly puts the mandate of delivering comparable, consistent, high-quality ocean BGC observations at risk.

Generally, the readiness concept and the criteria catalog developed here provide - for the first time - an objective basis to assess the maturity of information and data products. The result of the assessment, i.e. the ranking, is in line with the number of citations of the different data products, serving as an independent proxy for the readiness of each product and proving the reliability of the FOO readiness levels. We have chosen to distribute the impact on the final scores equally among individual features and key characteristics, see Section 3.1. Of course, discussions of this equal weighting approach are appropriate, and we want to encourage the community to improve the scoring scheme. Also, we are aware of the risks associated with applying the readiness level approach to data products with clearly different foci. It is indeed easier for a product with a narrow focus, e.g. one type of observing platform and one key variable only, to obtain a mature level than for a product with multiple variables from multiple observing platforms. However, the latter product might tackle a bigger task or mandate. For this reason, we want to stress that the readiness level of a product should not be confused with the importance and utility of the product. The readiness should rather be used to identify steps a product needs to take to realize its full potential.

The assessment excluded further data management efforts related to EOV BGC data which do not provide consistent and synthesized data of multiple data sources, e.g. the highly advanced BGC Argo database. These efforts also display important elements of the marine BGC data landscape but the here applied readiness level assessment is tailored specifically towards data synthesis products. However, the capabilities of ERDDAP diffuse the delimitation between more general databases and synthesis products increasingly. Widening the scope of this assessment to also include BGC EOV databases such as BGC Argo displays a future challenge.

In our vision, the ocean observation system should be independent of project-based research funds along the entire ocean observing value chain. We are in support of a sustained financing model, which could be realized through “[ … ] an international entity with a subscription-based or a binding Nationally Defined Contributions model, with a backbone/core ocean observing capability [ … ]” (European Marine Board, 2021, page 14). Such a financing model would have to include the management of BGC data and would resolve the lack of long-term funding experienced by synthesis products.