Incarcerated populations are consistently recognized as having some of the highest rates of mental illness of any population group (1–4). Studies consistently illustrate that psychosis is associated with criminal convictions, particularly for violence (5–9), however, reported associations depend on study design, sample size, different treatment and legal practices in a jurisdiction, and resources for treatment and support.

In 2016, we undertook a population-based data-linkage study to examine the association between psychosis and offending in the Australian state of New South Wales (NSW) which houses 33% of the nation's incarcerated population (10). Cases were all individuals diagnosed with psychosis between 2001 and 2012 (n = 86,461). For each case, two controls were matched by age and sex (11). In addition to endorsing the association between a diagnosis of psychosis and criminal convictions, including for violence, we found that increased treatment (defined as contact with community mental health services) (12), retention in treatment (13), early commencement of treatment following an offense (11), and diversion by the courts into treatment were associated with reduced reoffending (14). These findings indicate that treatment in the community rather than punitive sanctions is important, yet the attributes of care required to best achieve reduced reoffending is largely unexplored.

Community-based models of care for people with psychosis have tended to reflect either a “merged model” where forensic specialists work within community mental health service teams, or an “independent model” where forensic specialists work as a team independent of community health service teams (15). Regarding the later, the “Assertive Community Treatment” (ACT) model has become one of the prominent independent models (16). The key principles of ACT include outreach, direct delivery of treatments and services in the community including the patient's home by a specialized team, holistic and integrated services, and continuity of care (16–18). Although model fidelity concerns exist among some, there have been numerous adaptations of ACT including for forensic populations (17).

Complex inquiries regarding an optimal model of care for a specific population can perhaps be best answered by a panel of “experts,” including service providers and planners. In recent years, a focus on equity-orientated priority setting has emerged whereas input into decision-making have centered on the inclusion of diverse stakeholders rather than single-specialty experts (19, 20). Most studies have used the Delphi method to determine expert (21–23) or consumer (24, 25) consensus on psychosis care. A number of Delphi studies have included both consumer (and carer) and experts, with several using separate panels (26, 27). Very little Delphi studies include single panels made up of consumers and other experts such as researchers and clinicians who deliberate together to provide input into the development of the Delphi scoring survey (28). Additionally, a paucity of studies have sought consensus for models of care for forensic populations (15, 23).

The objective of this study was to determine the key attributes of an “optimal post-prison model of care” for those with a prior episode of psychosis, using the Delphi method to establish a consensus among experts and consumers (hereafter includes consumers as experts). The study was requested and funded by the NSW Department of Communities and Justice (DCJ) at a time when both the DCJ and NSW Health Ministry were implementing two independent pilot programs for people with serious mental illness leaving prison.

The Delphi method involved three rounds of deliberation by experts designed to inform survey construction, followed by two rounds of survey distribution to experts where model of care (hereafter “the model”) attributes and levels (i.e., attribute dimensions) were scored and commented on, to establish a convergence of opinion (Figure 1). Convergence was assessed by examining the similarity or central tendency of participants' responses to each question across rounds. Convergence was used as a measure of agreement or consensus (29).

A total of 25 experts participated in Meeting 1 (theme saturation), with 22 (88%) attending either or both Meetings 2a and 2b (attribute saturation). The participant retention rate for scoring was 96% for Round 1 and 84% for Round 2, where convergence of opinion was reached. Participants were from JH&FMHN (28%), CSNSW (16%), consumer/consumer support NGO (16%), consultant forensic psychiatrist (12%), NSW Mental Health Review Tribunal and former Mental Health Commissioner (8%), a non-government expert/academic (8%), NSW Ministry of Health (8%), and the NSW Police (4%).

Four model of care principles were identified: care integration and coordination; person-centered approach; evidence-based approach; and commitment, understanding and resourcing by government (Table 1). Four model themes where identified: pre-release care planning and coordination; treatments in community; diversion from prison; and evaluation (Table 1). A fifth theme that was originally identified—social and economic supports—was reconfigured as attributes under the pre-release care planning and treatment themes.

Of the 34 attributes and 168 attribute levels, 32 attributes and 72 attribute levels were included in the model (Table 2). Two rounds of scoring were conducted before consensus was reached for all attributes and levels except for two “diversion from prison” attributes. These related to the type of “offense committed” or “parole condition breached” that would prevent someone from automatically be considered for diversion from prison by the courts. Seventeen participants provided comments for these two attributes suggesting that most participants were reluctant to commit to a particular offense type or parole condition breach as criteria for diversion. Following discussion by the research team, it was decided not to ask participants to re-score these two attributes in a Round 3 survey due to face validity concerns and the unlikely prospect of reaching a consensus for these attributes. Consequently, these two attributes were excluded from the final model (see Supplementary Material A).

For the model stress test activity, participants did not identify any attribute of attribute level that could explain the model's failure. Instead, participants reiterated the importance of specific model principals or attributes, such as the model must be person-centered, encompass a trauma informed care and recovery approach, and that carers and peer support workers be central to care plan development and treatment decisions (see Table 1 footnotes). Findings from the consumer poll reinforced model thematic components and specific model attributes. However, poll participants did not comment on evaluation and evidence-based model themes as they were asked about the care components of the model and not on data aspects of a model (see Supplementary Material B for results).

In this Delphi study, a total of 32 attributes and 72 attribute levels across four themes were identified by experts to achieve an optimal model of care for people leaving prison who have experienced psychosis. Participant retention rates and level of input indicated a high level of their engagement in the process. Typically, consensus is reached in Delphi studies after three or more scoring rounds. For this study, consensus was reached after two scoring rounds. Although this may be interpreted as having a dominant scoring group or there being no contentious issues in this Delphi process, a probable explanation may lie in the scoring and attribute refinement activities participants engaged in as part of the online meetings. Thus, less relevant, or contentious model attributes were excluded from the scoring survey. Member checking processes overall reinforced the identified model of care principles and attributes, particularly underscoring the principal of a person-centered approach and involvement of peers.

A study limitation is potential affiliation and epistemic bias among experts. Although the participants' characteristics for the present study showed that a fairly even spread of affiliations and expertise was achieved, a higher proportion of participants affiliated with JH&FMHN (28%) took part compared to participants affiliated to CSNSW (16%) and consumer representatives (16%). To explore this potential bias, we examined the responses to specific questions which allowed respondents to select a specific agency to lead or undertake a responsibility attached to a model of care attribute. For example, responses to the attribute “consumer care planning lead” showed that 50% of CSNSW participants and 43% JH&FMHN participants gave the highest scores to agencies that they were not affiliated to theirs, and 50% of CSNSW participants and 57% JH&FMHN participants gave highest scores to the agency to which they were affiliated with (with some also scoring other agencies with the same highest score). These responses suggest participants' decisions can be informed by considerations beyond organizational allegiances. For the question: “who should supervise individuals assessed as medium to high reoffending risk,” 100% of CSNSW participants selected Community Corrections Officers (CCOs), and 100% of JH&FMHN participants selected the “case manager who implements the care plan” of which 57% of JH&FMHN participants selected that this case manager should not be JH&FMHN affiliated. Additionally, three participants not affiliated to either a DCJ agency or JH&FMHN selected that supervision should be undertaken by a CCO. It is difficult to determine whether these responses indicate (i) affiliation bias (for or against CSNSW), (ii) an objective and critical assessment of the attribute level, and/or—for participants preferring CCO supervision—(iii) current legislation requiring CSNSW to supervise this group and the substantial legislative reform required if this was altered to a non-CSNSW case manager. In reviewing, the comments provided by participants for this model attribute, factors (ii) and (iii), rather than affiliation bias, seem more likely to inform participant considerations. Although it is difficult to rule out potential affiliation and epistemic bias among experts as a study limitation, participants responses suggest that their decisions were informed by considerations beyond organizational allegiances.

Another limitation is that although experts included people with a lived experience of prison with or without a mental health condition and lived family experience of long-term mental illness, only one participant had both a lived experience of prison and psychosis. After consulting with our networks, it was assessed that it would likely be difficult recruiting people with both a lived experience of prison and psychosis. As reported in a review of Delphi studies (27), recruiting some minoritised consumer groups (e.g., specific refugee groups) can be difficult as relevant mental health advocacy and support NGOs for these groups are rare or do not exist. In response, we ensured that people affiliated with NGOs that provide support and advocacy for people in and leaving prison or for people with a lived experience of mental health systems were recruited. As people with serious mental health problems are overrepresented among incarcerated populations (2), mental health care of this population is typically a key advocacy area for the former NGOs.

A paucity of previous studies exists on stakeholder priorities of model of care attributes for forensic populations with psychosis with which to directly compare our findings. Nonetheless, many of the model attributes identified by participants in the present study are consistent with independent model attributes (15) such as the ACT (17), albeit with one notable departure. Although a high-fidelity ACT model instructs that an independent specialized team has full responsibility for, and directly provides, all treatments/services including psychiatry services, counseling, housing support, substance abuse treatment, and employment and rehabilitative services (16, 17), a hybrid case management approach was preferred by participants. This hybrid approach includes a specialized team directly delivering certain treatments/services while helping consumers to access and navigate external services including general medical practitioners, allied health professionals, pharmacists, Aboriginal health practitioners, and housing and income support providers (i.e., traditional brokerage model of case management). While some participants preferred the ACT model in terms of ensuring treatments and services align to the model's principle of care integration and coordination to prevent consumers “falling through the cracks,” most saw the hybrid model as a more appropriate balance given that individual consumer's level of need can vary over time and does not preclude an individual developing some level of independence. In this sense, the hybrid model was seen as better respecting and fostering consumer autonomy and independence and aligning to the model of care principle of being person-centered.

One issue often raised concerning the Delphi method is the generalizability of resulting outcomes. Sample size and sampling method are cited as two design features that can impact on the generalizability of Delphi findings. Given sample sizes of Delphi studies have ranged from 10 to 1,000, the sample size of the present study may be assessed as small or moderately sized. Additionally, we did not employ a random sampling method as this was not feasible to capture the diverse characteristics of our experts. As such, this likely reduces the generalizability of the results beyond NSW or Australia. However, we contend that the thematic components of the model and specific model attributes are likely to be generalizable to other jurisdictions, albeit with some modification based on the local resource, legal, and demographic realities. Modification to models of care for forensic populations. Despite fidelity concerns in modifying a model, there have been numerous adaptations of the ACT for example, including for forensic patients, to address the concerns with applying an “one-size-fits-all” model to different populations (17). The present Delphi study was commissioned by the DCJ to address the local NSW context. We noted previously that two separate pilot models of care were being implemented by DCJ and NSW Health Ministry before the results of the current study were released, and propose that future research examine the extent to which pilot model attributes align with the findings presented here.

Participants were asked to envisage an optimal model of care without having their thinking impeded by existing social or structural challenges. Participants agreed that an optimal model of care should include an independent specialized team directly delivering some treatments/services while helping consumers to access external services. Such a model may need to traverse traditional systems boundaries, require policy and legislative reform, as well as undergo rigorous evaluation to ensure iterative model of care enhancements as required. This may entail resource and budgetary implications for government. Studies assessing the ACT model for forensic populations indicate that sustainability can be a significant challenge with teams funded with time-limited seed money, and that once these funds are exhausted, teams were disestablished, or services diluted (17). These issues were acknowledged by participants and underscore the importance of the model's principle of commitment, understanding, and resourcing by government.

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

The studies involving human participants were reviewed and approved by University of New South Wales Sydney Human Research Ethics Committee (HC200339). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

PS, SS, AA, and TB contributed to the conception and design of the study. PS and SS performed the analysis. PS wrote the first draft of the manuscript. SS, AA, BT, and TB made revisions to the draft manuscript. All authors read and approved the submitted version.

This work was supported by a grant from the New South Wales Department of Communities and Justice. The authors declare that the research was designed, conducted, and findings analyzed and interpreted without input or influence by the funder. PS reports receiving support from a National Health and Medical Research Council grant APP1129816. TB reports receiving support from a National Health and Medical Research Council grant APP1124299. SS reports reviving support from the University of New South Wales. AA reports receiving support from a National Health and Medical Research Council grant APP1057492. BT reports receiving support from a grant from the New South Wales Department of Communities and Justice.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.