As the first stage of a two-step process, we conducted a series of semi-structured interviews (Flick et al., 2004) online between April and May 2023 to identify the characteristics of successful monitoring design that were then used to inform the development of survey questions. The semi-structured interview is designed to present open-ended questions to participants, allowing participants to ask additional questions (Refer to Supplementary Material for interview questions). This flexibility is important to generate more in-depth information for complex issues (Young et al., 2018; Flick et al., 2004).

We reached out to collaborating parties within the program, including Selected Area leads, basin scale discipline leads, basin scale managers, representatives from the Commonwealth Environmental Water Holder, on-ground monitoring teams, local catchment management authorities and scientists outside the team. Sixteen interviewees were recruited mostly through email (with a response rate of approximately 70%), and 13 of these participants had worked directly on Flow-MER. Considering the potential different mindsets between managers and specialist scientists, we classified participants based on their level of management role in monitoring programs (Table 1). Given the primary aim of the interview was to identify general patterns to inform survey questions, the small sample size of interview responses is acceptable, and this aligns with literature in similar studies employing interview methodologies (Coreau et al., 2010; Meempatta et al., 2023). The focus is not solely on numerical representation but rather on the qualitative insights garnered from the responses, which can still offer valuable and meaningful contributions to the research objectives.

The AI transcription and recording tool, Otter.ai, was used to digitally record and transcribe interviews. Transcriptions were then manually reviewed by the lead author and were analysed through NVivo qualitative analysis software (NVivo 12 Pro, 2020). We employed a combination of deductive and inductive coding techniques (Fink, 2003; Refer to Supplementary Material for interview questions and coding framework). Based on their identified role as scientists or managers, participants’ views were subjected to an occurrence analysis through the Matrix Coding Query in NVivo to investigate the number of participants who mentioned specific criteria.

Based on the interview results, we developed an internet-based survey through Qualtrics to help gain a wider range of perspectives across biophysical scientists and managers. We recognize that our approach might not include all groups with an interest in the outcomes of the program (i.e., external stakeholders). However, the key stakeholders for the design of e-flows monitoring programs are prioritized because external groups are less likely to have an understanding of the monitoring program. We invited individuals featured on the Flow-MER website Team page (https://flow-mer.org.au/our-teams/). Recruitment was conducted through email distribution with a response rate of approximately 46%. The senior author also distributed an anonymous survey link during a major academic conference (the Freshwater Sciences Conference in Brisbane, Australia, June 2023), aiming to target people with more general environmental monitoring experience. Through the QR code dissemination at the conference, anyone attending the meeting (or informed about it by an attendee) had the opportunity to contribute to the survey, thus providing a greater breadth of stakeholder perspectives. The survey was completed by a total 60 people in June and July 2023. A relatively small number of managers, compared to scientists, responded to the survey, reflecting their proportionate involvement in the program. While this imbalance may limit the extent to which the survey findings can be generalized (Fink, 2003), it is a factor we have to accept given the structure of the program.

The survey included a series of ranking, Likert-type scale and open-ended questions related to roles of monitoring and criteria for successful monitoring design (Refer to the Supplementary Material for survey questions). Likert scale questions were used to measure self-perceptions and beliefs regarding a particular statement, ranging from varying degrees of agreement or disagreement in between (Fowler Jr and Cosenza, 2009). We chose the five-point Likert scale ranging from “Extremely well” to “Not well at all” (Brown, 2010; Chyung et al., 2017). We acknowledge that survey participants tend to be conservative while choosing extremes in responses (in this case “Extremely well” and “Not well at all”; Moors, 2008). This tendency can introduce a response bias that might not fully capture the range of participant views. However, the survey responses likely represent the opinions of those who are either highly engaged and enthusiastic about developing effective and efficient monitoring projects or those with grievances about the current Flow-MER programs. While these groups are not typically shy about expressing strong opinions, the formal context of the survey might still lead them to moderate their responses.

Respondents who had specific knowledge of the Flow-MER program were asked several follow-up questions relating to the overall performance of Flow-MER (Refer to Supplementary Material for survey question Q8). Forty-six out of 60 survey participants had worked directly in Flow-MER or self-identified as having sufficient understanding of the program. We acknowledge that participants directly involved in the program may be less critical in responses compared to external stakeholders. However, our aim is to understand the perceptions of those involved in the design and implementation of monitoring.

We conducted descriptive analysis to summarize and examine perceptions of monitoring programs by different stakeholder group (Fink, 2003) using R Studio statistical analysis software. Unlike analytic inferential analysis, descriptive analysis does not attempt to make predictions. Instead, it draws insights from existing data (Wolcott, 1994; Fink, 2003).

Five different roles of monitoring were identified based on the open-ended semi-structured interview question (Table 2). Among them, “support adaptive management” was mostly commonly mentioned during the interview, with all Selected Area lead scientists referring to this concept, while fewer participants cited community and social benefits as a motivation for monitoring. The interview results were corroborated by the larger survey results (Supplementary Figure S2).

In the survey, monitoring to support adaptive management was recognized as the primary role of monitoring by both managers and biophysical scientists, followed by “demonstrating the success of management interventions.” “Determining trends in the ecosystem,” the third ranked role, places a stronger emphasis on system condition and drawing insights from long-term datasets to trigger management actions (Supplementary Figure S2). Thus, the top three nominated roles of monitoring all share a connection to influencing environmental management actions and outcomes, with the fourth-ranked position encapsulating the scientific benefits of monitoring initiatives (“understanding holistic ecosystem dynamics”). When comparing the responses of biophysical scientists and managers, the survey did not reveal any major differences between these two groups (Supplementary Figure S2).

Interestingly, 69% of participants mentioned the utilization of long-term datasets during the interview, but not all were supportive of this idea. Some interviewees acknowledged the collection of long-term datasets as a key purpose of monitoring, but others pointed out that an emphasis on long-term datasets can impede the responsiveness of monitoring programs to changing knowledge.

This was further investigated through an open-ended question in the survey, exploring the major benefits of collecting long-term data in Flow-MER. The majority of respondents recognized the value of long-term data for observing and tracking environmental changes over time (32.8%), or for gaining a deeper understanding of ecological process (32.8%, refer to Supplementary Table S3). In contrast, a much smaller proportion of respondents noted the importance of monitoring for generating long-term datasets to support outcomes reporting (9.4%) or to facilitate adaptive management and decision-making (14.1%).

Five criteria for successful monitoring programs were raised during the interviews (Table 3). “Clear objectives” emerged as the most important characteristic of a successful program.

Data transferability concerns were frequently raised by managers, whereas Selected Area Leads more commonly discussed issues of detectability of ecological effects during monitoring. Whether monitoring was designed and structured adaptively was not raised by many interviewees as a successful design criterion.

Survey respondents provided their view on the level of importance for seven criteria identified through the interviews (Supplementary Figure S3). Similar to the interviews, survey results identified clear questions and objectives as the most important element for program success. This was followed by a need for good communication among all relevant stakeholders. However, in contrast to the interview results, the survey results found a low emphasis on the issue of detectability of effects (i.e., “High statistical power to detect the specified effects”), and data transferability (i.e., “Data can be extrapolated”).

We found very little difference between biophysical scientists and managers in their perceptions of what characterizes a successful monitoring program. The largest difference was for the question regarding importance of good communication, with scientists placing a greater premium on this characteristic, while all others were small (Supplementary Figure S3).

Participants in surveys generally expressed a positive assessment of Flow-MER’s performance for the three most important criteria previously identified: “clearly defined questions and objectives,” “good communication between all stakeholders” and “improved knowledge regarding biophysical variation” (Figure 3). Only 4% of biophysical scientists thought that the questions and objectives in Flow-MER are poorly defined. As with the criteria for successful monitoring programs, managers and biophysical scientists exhibited similar views on the success of Flow-MER.

Adaptive management is one element of Flow-MER for which participants were critical. Most interviewees expressed a similar understanding of the adaptive management process and its importance. In the context of the Flow-MER program, a total of 54 coding references to this topic were identified during the interviews (26 from managers, 23 from Selected Area Leads). However, notable differences emerged in how individuals perceived whether adaptive management is implemented effectively. While some scientists held negative opinions about the implementation of the adaptive management process, managers tended to view it as the best possible approach given the circumstances (Table 4). In general, interviewees agreed that the adaptive management process is implicit. This also led to some discussion around whether monitoring is adaptive within adaptive management framework.

Given the difficulty of obtaining general responses about whether monitoring is adaptive, survey respondents were asked a specific question—whether they have encountered the removal of monitoring indicators through time from a program or had been involved in such removal. Among the participants, 42 individuals responded in the affirmative (Figure 4). “There is enough understanding of this indicator” was the most frequent cited reason to decide not to continue monitoring a specific indicator. “Not a central management focus” was raised more often by managers than biophysical scientists, while scientists more often reported that a loss of funding or insufficient power to detect an effect was the main reason to discontinue monitoring.

The importance of clear objectives for monitoring was identified through both formative survey and subsequent larger interview process (Sections 4.2, 4.3). Survey respondents were asked to identify strategies to improve objective setting and alignment of monitoring to the objectives.

During both interviews and the survey, the concept of an objectives hierarchy (Keeney and McDaniels, 1992; Rogers and Biggs, 1999) was suggested as a potential solution to poorly defined objectives, and participants were asked about its potential utility. The majority of interview participants (60%) were favorable or demonstrated support for this approach, while 20% raised concerns or did not find this approach useful (Table 5).

In the survey, the majority of respondents were already familiar with the concept of an objectives hierarchy (Figure 6). Regardless of their previous experience, most participants had a positive attitude towards this concept. More than 50% of the survey respondents reported that this approach has contributed to the success of a program in which they have been involved. Even for those who had not used an objectives hierarchy before, they expressed a willingness to adopt it in future projects.

In keeping with existing literature (Clarke, 1999; Field et al., 2007; Prabhakar, 2009), the interviews and survey results show that clear objectives play a central role in the perception of success of environmental monitoring programs. Despite this common understanding that clearly articulated objectives are key to monitoring design and success, the vagueness of objectives was regarded as the second most pressing concern for the Flow-MER program. The Flow-MER program has two types of objectives, basin-scale, and selected area scale (Barbour et al., 2021). The basin-scale objectives focus on broad environmental concerns at the scale of the entire Murray-Darling Basin, while selected area scale objectives address more local target species or ecological responses. For example, “to sustain native fish at the Basin-scale” is one basin-scale objective. Meanwhile, the selected area-scale program in the lower Goulburn River aims to evaluate how Commonwealth Environmental water contributes to the recruitment of golden perch in the adult population (Gawne et al., 2020; Webb et al., 2022). A clear strength of the objectives within Flow-MER is that they exist across different spatial scales and multiple environmental responses. However, the objectives are often defined in comparison to a pre-environmental water baseline or benchmark. Moreover, the objectives are often vague, which makes evaluation difficult. For example, it is difficult to quantify the success of “sustain native fish” (Gawne et al., 2020) without a focal species and its target abundance, density or biomass. Thus, there needs to be a distinction between the “vision” of the system for upper-level management and “thresholds of probable concern” for the on-ground specific ecological endpoints (Rogers and Biggs, 1999).

To better align monitoring with objectives, the concept of an objectives hierarchy was introduced by the researchers. This idea was widely accepted among both interview and survey respondents. There was a willingness to endorse its use in future projects even among those previously unfamiliar with this approach.

Literature highlights the need to better integrate monitoring, management, and research in large-scale environmental projects (Yoccoz et al., 2001; Lindenmayer and Likens, 2009; Palmer, 2009). There is an emerging emphasis on delineating the purposes of monitoring between evaluating management programs and scientific research, as this distinction can enhance the overall efficiency of monitoring projects. However, our results reveal that improved scientific knowledge may not be perceived as equally important as monitoring for management purposes (Figure 3), and this opinion was surprisingly similar among scientists and managers. This might be explained by the strong legislative obligation of intervention monitoring programs to demonstrate success of the management action. The Flow-MER program also has significant legal obligations, necessitating frequent reporting to both the Commonwealth Environmental Water Holder and state government authorities (Barbour et al., 2021). These requirements may constrain the monitoring of monitoring to fill scientific knowledge gaps.

Adaptive management was a recurring theme in the interviews. Adaptive management is a central theme in literature and research (McDonald-Madden et al., 2010; Horne et al., 2017a; Webb et al., 2017; Williams and Brown, 2018), and is now often referred to in policy. However, people rarely consider the need to also revise monitoring adaptively (Lindenmayer and Likens, 2009). Adaptive monitoring provides a novel pathway for incorporating new questions into a monitoring approach while maintaining the integrity of the core measurements (Lindenmayer and Likens, 2009). An adaptive change to monitoring could include updated monitoring techniques, changing sampling frequency, or altering a monitoring endpoint due to change in priorities, and more.

A potential explanation for the lack of recognition of adaptive monitoring could be the persistent emphasis on the importance of consistent, long-term datasets. Multiple studies have highlighted the value of long-term datasets (Wolfe et al., 1987; Field et al., 2007; Bonney, 2019). Considering the rarity of some hydrological events (e.g., a major flood) and the life spans for long-lived species, monitoring may be required over an extended period to detect significant responses and reach a conclusion (Souchon et al., 2008). This mindset was also reflected in survey responses. Adaptive monitoring and the need for long term data sets are concepts that are at odds, and it will be challenging for any program to balance them appropriately.

There has been some adaptive monitoring and adaptive management within the Flow-MER program. Adaptive management in the LTIM and Flow-MER programs has focused on within-year decisions (e.g., flow release quantity, timing, etc.) based on monitoring results from previous years (Barbour et al., 2021). In contrast, changes to monitoring have occurred during project iterations. During the interviews and surveys, Interviewees widely acknowledged a shift in monitoring focus from LITM and EWKR to Flow-MER. Initial programs were primarily seeking scientific understanding of environmental water delivery, while later incarnations focused on evaluating management practices to maximize the associated benefits.

Both managers and scientists are actively driving the advancement of the next project—Flow-MER 2.0, which began monitoring in July 2024. The design phase of Flow-MER 2.0 involved a comprehensive evaluation of existing monitoring data and management outcomes, leading to several adjustments. These included reduced monitoring intensity for certain species responses (e.g., fish spawning in the Lower Goulburn River, Department of Climate Change, Energy, the Environment and Water, 2024), shifts in priorities such as changes to the main MER themes (DCCEEW, 2023), and an expansion of monitoring activities, including updating the MER selected areas to cover 10 regions (DCCEEW, 2023). Additionally, long-term monitoring data from LTIM and Flow-MER have been analyzed at the basin scale to refine water delivery strategies, aiming to enhance environmental outcomes ultimately.

Communication among those with an interest in the outcomes of e-flows was identified as an important characteristic of successful monitoring programs. A lack of effective stakeholder engagement, together with limited public acceptance, is often cited as a significant challenge for project implementation (Conallin et al., 2017; Horne et al., 2017b; Moore, 2020; Mussehl et al., 2023; Reis and Judd, 2000; Visser et al., 2000).

The interviews revealed a distinct disparity in how individuals in different roles within the Flow-MER project (e.g., Management sector, Selected Area Leads, etc.) perceive the characteristics of successful monitoring programs. This is understandable, as the Selected Area Leads’ responsibilities involve planning monitoring projects, as well as reporting data and acting based on monitoring outcomes. In comparison, discipline leads at the basin scale have less direct involvement in field sampling or monitoring activities. Therefore, the dual role of Selected Area Leads, which includes both planning and implementation, necessitates their increased consideration of the detectability issue in monitoring design (i.e., most selected area leads considers detectability important, while no basin-scale discipline leads mentioned this criterion, Table 5).

However, differences in viewpoints between scientists and managers during the survey were much smaller than initially anticipated (Poff et al., 2003; Roux et al., 2006; Robelia and Murphy, 2012; Taylor and de Loë, 2012). This observation serves as compelling evidence of the effectiveness of past communication and collaboration efforts in Flow-MER. These efforts have meant that scientists and managers have been able to exchange ideas, share information, and find common ground through regular information sharing events (e.g., Flow-MER Annual Forum, Flow-MER Fridays, engagement, and communications infrastructure; Cross-Cutting Theme: Stakeholder engagement and communications, 2023). This harmony between the two groups is indicative of a healthy working relationship and a shared understanding of project goals and objectives.