According to the Federal Statistical Office of Germany suicide led to more than 10,000 deaths in 2024 (1) and is the most common cause of death among young people between the age of 10–25 in 2023 (2). In most suicide cases, individuals suffered from mental health disorders with rates ranging from 87,3% (3) up to 96,8% (4). Preventing suicides or suicide attempts is the goal of every psychiatric patient treatment and requires an ongoing suicide risk assessment. Thereby, most people committing suicide consult clinicians in the days or weeks prior to death (5, 6). However, predictive markers are currently not sufficiently able to predict suicide risk. Clinicians rely on spoken self-reports to gain insights into the patient's psychological state. Speech comprises a variety of aspects which carry information about someone's psychological state, like e.g., the content, speaking rhythm, loudness, pitch, pace, or coherence (7, 8). The diversity of content-related linguistic features that have been shown to be relevant for diagnostic assessment (9) and for the identification of suicidal ideation (10) is so extensive that a comprehensive evaluation of these features during an ongoing clinical conversation is unrealistic. It is therefore plausible that, along other factors, this linguistic complexity contributes to the limited consistency observed in suicide risk assessments (11). Further, some of the aspects of speech are subtle and hard to assess reliably by a human listener during a conversation, like e.g., jitter, shimmer, or harmonics-to-noise ratio, but important for the understanding of psychiatric illness' like depression (12). Thus, to implement meaningful preventive measures, there is an urgent need for methods able to assess all important predictive markers related to suicide risk and mental disorders in general. Recent advances in Natural Language Processing (NLP), a field within artificial intelligence (AI), show promising results in mental disorder detection and an upward trend within this research area (13). Within NLP methods, linguistics and computer science enable large-scale analysis of language which are also applicable to the clinical psychological practice, where language plays a crucial role in patient care. NLP can be applied to various kinds of text data like social media posts (14–16), interviews (17, 18), and clinical notes (7, 19, 20). For the clinical psychiatric context, studies show promising results for the appliance of NLP methods. Regarding the field of suicide and suicide risk research, NLP models can successfully identify patients who have attempted suicide (20), predict suicidal ideation (17), and distinguish between suicidal and non-suicidal teenagers (21). Furthermore, suicidal language appears to be a stable classifier when compared to standard measurements. This stability is evident as suicidal language persists for up to 30 days after discharge from care, even when responses to other measurements change (18). Accordingly, there are multiple NLP-based methods which enable the exploration of previously unexamined linguistic features.

The aforementioned approaches have been somewhat overshadowed since the introduction of the Transformer architecture (22) and its fast development and huge impact, the evaluation of speech data has totally changed. Models have grown tremendously in a short period of time e.g., GPT3 is more than 100 times bigger than its predecessor GPT2 (23). Due to this rapid development, it is now possible to use even large amounts of speech data, such as clinical interviews, for evaluation with large language models (LLMs). Modern LLMs are not only ideal for general purposes, they also show capabilities when it comes to specific tasks in the domain of mental health (24). It could be demonstrated that LLMs like GPT-3.5 and GPT-4 are able to correctly identify patients with depression. However, they are known to show a considerable language bias and due to their nonspecialized training data, they are better at identifying control participants which could narrow their direct applicability for mental health applications (25). Models specialized on medical knowledge like Med-PaLM-2 can demonstrate human-like performance when predicting PHQ8 scores used for the assessment of the severity of depression (26). Besides spoken language many studies utilize written language like social media data and show promising results even with simple prompting strategies (27). Modern LLMs capabilities go way beyond this and offer numerous applications in addition to those mentioned before. Other studies indicate their usefulness in giving treatment suggestions (28) or creating simulated patients for psychotherapy training (29).

Apart from the content it is also important to take acoustic features accompanying the linguistic content in speech, also called prosodic and intonational features into account. These features are beyond the content related aspects of speech like phonemes and comprise patterns of pitch, duration, intensity, and rhythm, but are important in conveying meaning, emotion and nuance in speech (30). Recent studies focusing on acoustic features (31, 32) confirm that detection and severity assessment of mental health conditions like depression can be achieved with high accuracy based solely on acoustic information, demonstrating that vocal markers provide a reliable proxy for underlying affective states. In addition to research focusing on one modality, there are studies that consider both linguistic and acoustic features in a single study (33).

With multimodal models, large scale language and acoustics data can be processed simultaneously to get a better prediction of e.g., suicidality (34) and depression (35). Those multifaceted insights can greatly enrich existing knowledge about diseases like depression or schizophrenia and thus improve patient diagnosis and care. Moreover, a multimodal approach enables a much more realistic evaluation of speech data sets and is therefore useful in many counseling situations.

Data will be collected in the University hospital in Frankfurt a.M. Additionally, one clinician and project member will collect data at the private hospital Dr. Amelung in Königstein. Data access is regulated via a data security protocol.

We will include patients with ICD-10 diagnoses F20.x, F23, F25, F30-33, aged between 18 and 65 years, who are legally competent, without any issues in voice production, and have a German speaking proficiency of at least B2 (lower level in German would contain too little information content of linguistic utterances). Additionally, psychotherapists employed at the University Hospital Frankfurt will be included for the therapeutic conversations.

Informed consent will be contained by members of the study team who are not involved in the patients care. The patients are addressed at least one day prior to study participation and informed about the study. One day later, they will be addressed again and sign the consent if interested in participation.

n/a: No comparator treatment was conducted. All participants equally engaged in receiving therapy sessions and completing questionnaires.

The participation will consist of the recordings of the therapeutic clinical conversations that are carried out in the regular clinical practice. In addition, for our study a student assistant will place a recorder and microphones before the therapy session and pick it up afterwards. This student assistant will be absent during the therapy session and only install the recording hardware. Further, after the recorded therapy session the clinicians and participants are required to fill out questionnaires regarding the clinical state and suicide risk of the patient on the same day as the therapy session was recorded.

Study participation will be discontinued if the patients or their therapist request to discontinue participation or if the patient's clinical treatment is completed.

To maintain adherence as much as possible, we strive to integrate the participation process seamlessly into the clinical routine of both patients and therapists. The only additional effort required for participation is approximately 5 min to fill out questionnaires after a recorded therapy session.

n/a: we do not expect that patients suffer any harm from participation, since the recordings are conducted as part of their regular therapeutic care and participants additionally only fill out questionnaires.

As part of an initial exploratory analysis, the assessment of psychopathology by treating physicians is to be compared with the evaluation by a computer model. Therefore, using intraclass correlation coefficients or t- tests, the hypothesis “There is no difference in the assessment of patients regarding the severity of illness and suicidality” will be tested.

In-patients and out-patients will regularly be screened for participation eligibility. If the eligibility criteria are met and in consultation with their treating clinicians, patients will be informed about their study and will have at least one day to consider participation and sign the informed consent. The same procedure applies for clinicians who participate in the study, while they only must consent once for the recording of therapy sessions with several patients who gave informed consent. From the time point on where patients agreed to participate, up to five of their regularly held therapy sessions will be recorded. After the therapy session, patients and therapists will fill out questionnaires about the psychopathology of the patient (approx. 5 min. for each). They are asked to fill out the questionnaire on the same day the therapy session took place. Table 1 summarizes the procedure and when the respective questionnaires are used.

To determine the required sample size for comparing the accuracy of our models against the gold standard of a trained psychiatrist's evaluation, we conducted an a priori power analysis using G*Power software (43).

We based our effect size estimate on the accuracies reported in previous studies that used machine learning techniques to predict suicidality from clinical data (44). Pestian et al. (45) reported an accuracy of 85%, Bernert et al. (46) reported 90% among 87 studies in their meta-analysis, and Barak-Corren et al. (47) reported 94%. The average accuracy across these studies is 89.2%. Assuming a gold standard accuracy of 100%, we calculated an effect size (Cohen's g) of 0.11 using the formula: g = (P—Po)/√[Po(1—Po)], where P is the expected accuracy (0.892) and Po is the gold standard accuracy (1.0).

For our power analysis, we set the following parameters: a two-tailed test, an alpha level of 0.05, a power of 0.80, and a constant proportion of 0.5 (representing a balanced dataset). The results indicated that a total sample size of 162 is required to detect an effect size of 0.11 with 80% power at a significance level of 0.05. The critical values for the lower and upper bounds of the acceptance region are 68 and 94, respectively. The actual power achieved with this sample size is 0.8046757, and the actual alpha level is 0.0491726.

In summary, to compare the accuracy of our models against the gold standard of a trained psychiatrist's evaluation with 80% power and a 5% significance level, we require a total sample size of n = 162, assuming an effect size of 0.11 based on the accuracies reported in previous comparable studies. As we expect a drop-out rate of 15%–20% due to technical issues, quality issues etc., we expect to collect n = 200 interviews.

The participants are recruited from both inpatient and outpatient individuals at the Department of Psychiatry, Psychosomatics, and Psychotherapy at the University Hospital Frankfurt. Additionally, a physician collects data at the Dr. Amelung Private Clinic in Königstein. For ongoing enrollment, patients will be screened regularly for participation criteria by the study teams and by their treating clinicians. Study information will be given by members of the study team who are not involved with the patient's treatment to prevent participation out of a sense of obligation and to maintain voluntary participation. There will be no assignment of interventions since participation does not include differing intervention types.

n/a: since participation comprises the recording of up to five regularly held therapy sessions and subsequent filling out of questionnaires which will be equally done by all study participants, sequence generation is not needed.

The recording of the therapy sessions and the questionnaire data will be collected on the same day. An overview of all applied study instruments is depicted in Table 2. After giving the patient the study information and obtaining informed consent, audio recordings are conducted during routine medical activities. Up to five of their therapy sessions with a typical duration of 30–60 min are recorded. Audio recordings of therapeutic sessions are captured using high-quality recording equipment with carefully positioned microphones to minimize background noise and ensure clear speech capture. Recordings are made using a Tascam DR-40X audio recorder. Recordings are captured in stereo mode using the recorder's external left/right inputs. The audio is stored as uncompressed WAV files with a sampling rate of 48 kHz and a bit depth of 24 bit. The recorder input level is set to −12 dB, providing sufficient headroom to avoid clipping. Phantom power (48 V) is enabled to supply the external microphones. Audio signals are acquired using two external condenser microphones (the t.bone CC 75) with a frequency response of 20 Hz–20 kHz, corresponding to the standard human auditory range. Microphones are attached at the chest/collar area of the participants ensuring comparable distance to the mouth between the participants. Care is taken to ensure that the microphone grille did not come into contact with participants' clothing and is oriented toward the participant's head, in order to minimize clothing-rub noise and to enable more accurate recordings. These settings were chosen based on previous test runs using different setups of microphones and recording environments/parameters. We transcribed and diarized the recordings and manually evaluated the quality of these transcriptions with their corresponding audio recordings to find the best setup.

Additionally, data on general psychosocial functioning and the severity of the patient's psychopathological state are collected using questionnaires completed by both therapists and patients.

The Clinical Global Impression (CGI) is a short assessment of the clinician's view of the patient's global functioning and to track patient progress or treatment response over time and comprises three measures of which we use two: severity of illness, global improvement, and an efficacy index (which we leave out) (48). The instrument has been proven to be a readily understood and practical measurement tool which is well applicable in the clinical practice (49) and has been used in different mental disorders including affective and psychotic disorders (50, 51). Additionally, we use the CGI-SS which is a modified version of the CGI to evaluate severity of and change in suicidality according to the version of Meltzer et al. (52). We only changed the time period from “past 7 days” to “since the last assessment”.

The Global Assessment of Functioning (GAF) scale is an established instrument for measuring illness severity in the clinical context and introduced as a rating scale within the DSM-III-R (53). The instrument has shown to be a valid and reliable measure of psychiatric illness (54).

To promote retention of the participating patients and their therapists, we are integrating the participation progress in everyday clinical care with minimal extra effort for the participation process. If therapists or patients nevertheless want to end their participation, there will be no consequences or disadvantages.

During the data collection phase, the raw audio recordings, patient data, and questionnaires will be collected. In this phase, only clinicians and student assistants from the University hospital in Frankfurt as well as one doctoral student from the Privatklinik Dr. Amelung in Königstein are involved. The analysis of the data is conducted by the Ubiquitous Knowledge Processing (UKP) Lab at TU Darmstadt using natural language processing and multimodal methods and is mostly carried out in Darmstadt exclusively with pseudonymized data. The data accessed by the research group from TU Darmstadt will only have pseudonymized files and data without any personal information about the participants. The data analysis is complemented by a standardized assessment of the course by an independent board-certified psychiatrist or experienced assistant in psychiatry, serving as a “gold standard” for comparison with the AI analysis. Thus, the pseudonymized audio files are listened to and classified by a board-certified psychiatrist based on clinical features. The gold standard assessment covers three different areas. It consists of the BPRS as a broad psychopathology questionnaire, the five-stage severity item of the CGI-SS combined with an open question, asking the experts to briefly justify their rating within 2–3 sentences and the VTAS-R-Short Form, a five-item therapeutic alliance questionnaire (55).

The pseudonymized audio recordings will then be analyzed for acoustic and prosodic features using the open-source toolkit, openSMILE (56) and then transcribed to text with the freely available software WhisperX (57). Both datasets will then be analyzed on its own and later on fed to a multimodal neural network to generate a clinically relevant outcome score (e.g., suicidality prognosis score). Figure 1 shows an overview of our workflow.

Both, audio track and therapy transcript have to be pseudonymized to ensure data privacy. The sessions are transcribed to text with the freely available software WhisperX (57). In order to pseudonymize the data, the members of the study team listen to the therapy session and check the transcript at the same time. In this way it is not only possible to delete personal data but also keep track of transcription errors. Furthermore, this also enables the correction of errors in speaker diarization which will be relevant for data analysis. Transcripts are also edited according to predefined rules to ensure consistency, including standardized notation for stuttering and speech disfluencies, clear indication of overlapping speech, flagging of incomprehensible sections, and documentation of interruptions such as third parties entering. By doing this we are left with not only a pseudonymized audio track but also with a high quality pseudonymized and diarized transcript. Afterwards, the transcripts are ready for use with e.g., locally operating LLMs to assess their capability to distinguish between patient groups and evaluate symptom severity. The audio tracks are ready for feature extraction with the open-source toolkit openSMILE (56). Extracted audio features will serve as the basis for subsequent calculations, e.g., correlations with specific symptoms as well as general severity scores. Details of the unimodal and multimodal analyses are provided in the Statistical Methods section.

A dedicated data security concept for this study was developed in close collaboration with the local data protection officer, fully compliant with the EU General Data Protection Regulation [GDPR, Regulation (EU) 2016/679] and the German Federal Data Protection Act (Bundesdatenschutzgesetz, BDSG). The measures ensure both technical and organizational protection of personal data throughout the project. Key provisions include strict pseudonymization and, where feasible, anonymization of all personal data before processing. Access to sensitive data is restricted through a role-based authorization system, and all project staff are trained in privacy and IT security procedures. Data transfers are conducted via encrypted communication channels (TLS/SSL), and stored data is protected using state-of-the-art encryption algorithms. Routine system audits, monitoring, and regular penetration testing are implemented to detect and mitigate vulnerabilities. Redundant backups are encrypted and stored in physically secure environments, with access strictly controlled. Detailed data usage logs and audit trails are maintained to record all data access, modifications, and deletions. The network infrastructure is protected by firewalls and intrusion detection systems, and written informed consent is mandatory for all data subjects. These measures collectively ensure the confidentiality, integrity, and availability of research data, safeguarding data subjects' rights according to EU and German law. Specifically, we implemented the following measures to ensure data security. For every enrolled patient a participation ID will be generated via SecuTrial, a software for managing and conducting clinical trials, offering data collection, management, and analysis with high security and quality standards. One secured document which is saved locally in the University hospital Frankfurt contains the assigned patient identities to the SecuTrial-IDs. All confidential data will be stored exclusively in the university hospital Frankfurt. The questionnaire data will be transferred to SecuTrial and stored under the assigned participant ID. The raw audio files and their transcripts are then pseudonymized by a member of the study group. This involves deleting elements that can be clearly attributed to the patient or the clinician, such as names, addresses, institutions, or other personal information. All files are pseudonymized in compliance with data protection regulations and stored under the assigned participants ID. Password-protected Excel file (256-bit key) with the association of real names is stored in the hospital network. Only designated study personnel with identifiable names have knowledge of this password. Access to the system is logged in a log file.

We will employ a zero-shot prompting approach with OpenAI's recently released GPT-OSS (Generative Pretrained Transformer—Open Source Software) series models, which are available under the permissive Apache 2.0 license and have demonstrated strong performance on reasoning tasks relevant to clinical assessment. We have selected these models for several methodological and practical reasons:

First, their open-weight nature ensures full transparency and reproducibility of our analyses. Second, their mixture-of-experts architecture enables efficient deployment on clinical infrastructure with limited computational resources. Third, their documented performance on complex reasoning tasks, including healthcare-related queries, makes them well-suited for psychiatric assessment applications.

Our primary analytical comparison will involve two model sizes from the GPT-OSS family: GPT-OSS-120B (116.8 billion total parameters with 5.1 billion active parameters per token) and GPT-OSS-20B (20.9 billion total parameters with 3.6 billion active parameters per token) (58). This comparison directly addresses a critical question for clinical deployment: whether larger, more capable models provide sufficient performance advantages to justify their greater computational demands, or whether smaller, more efficient models can achieve comparable diagnostic accuracy. The GPT-OSS-120B model has been shown to achieve near-parity with state-of-the-art reasoning models on standard benchmarks while running efficiently on a single 80GB GPU, whereas GPT-OSS-20B delivers comparable performance to advanced compact models while requiring only 16GB of memory, making it feasible for deployment on standard clinical workstations or even edge devices.

For each model size, we will systematically evaluate two distinct inference configurations defined by the temperature hyperparameter, which controls the randomness of model outputs. We will test a conservative configuration (temperature = 0.1) designed to prioritize consistency and clinical reliability by producing more deterministic responses, and a rather liberal configuration (temperature = 0.8). In the NLP literature the effects of temperature are well represented but systematic clinical applications have yet to be done to our knowledge. Accuracy in multiple choice question answering tasks show high consistency in temperature ranging between 0 and 1, but an expansion to other problem types is recommended (59). This strategic selection of hyperparameters reflects the clinical tension between reproducibility critical for diagnostic applications where consistency across assessments is paramount and the risk of highly inconsistent outcomes that could potentially lead to incorrect diagnoses and treatment recommendations. In addition, we apply top-p (nucleus) sampling with a cumulative probability threshold of 0.9 to each analysis. This setting restricts sampling to the smallest set of tokens whose cumulative probability mass is at least 0.9, balancing output diversity with coherence.

This yields a 2 × 2 factorial design with four experimental conditions: 1. GPT-OSS-120B with temperature 0.1, 2. GPT-OSS-120B with temperature 0.8, 3. GPT-OSS-20B with temperature 0.1, and 4. GPT-OSS-20B with temperature 0.8. We will evaluate each condition's performance on primary outcomes including general diagnostic classification accuracy and suicidality detection, severity score prediction (correlation with standardized clinical measures), and clinical utility metrics such as sensitivity, specificity, and positive/negative predictive values. All model versions, hyperparameters, and inference code will be fully documented and made available to ensure complete reproducibility.

Concurrent with text-based analysis using GPT-OSS models, we will extract acoustic features from therapy session recordings using openSMILE (version 3.0) (56) with the eGeMAPSv02 feature set (60), which includes validated markers known to associate with psychiatric conditions. The eGeMAPSv02 feature set comprises 88 acoustic features encompassing spectral (Mel-frequency cepstral coefficients), glottal (fundamental frequency, jitter, shimmer), and prosodic characteristics (energy, speech rate, voice quality measures such as harmonics-to-noise ratio). These features will be extracted at the frame-level (10-millisecond intervals) and subsequently aggregated using statistical functions (mean, standard deviation, minimum, maximum, percentiles) to produce session-level representations comparable to clinical assessment timeframes.

Our overarching analytical strategy employs a three-tier statistical framework designed to characterize how textual, acoustic, and combined modalities contribute to clinical prediction:

Tier 1: Unimodal Baseline Analysis. We will analyze each modality independently to establish baseline predictive performance before considering fusion. For the text modality, we will conduct two-way mixed-effects logistic regression with outcomes (diagnostic classification or severity degree) regressed on the fixed effects of model size and temperature, with random intercepts for participant and session. Effect sizes will be reported as odds ratios with 95% confidence intervals. For the acoustic modality, we will conduct mixed-effects linear regression using a reduced set of the most clinically relevant acoustic features selected a priori based on prior literature associations with depression and suicidality, including pitch variability, energy measures, MFCC and jitter as predictors of continuously scored clinical outcomes (e.g., severity scores from standardized rating scales) (60, 61).

Statistical rigor and assumptions testing. For all mixed-effects models, we will verify assumptions including normality of residuals (Shapiro–Wilk test on standardized residuals), homogeneity of variance (Levene's test), and absence of multicollinearity (variance inflation factors < 5 for predictors). Violations of normality or homogeneity will be addressed through appropriate transformations (e.g., Box-Cox for acoustic features) or robust variance estimation. Missing data will be handled via maximum likelihood estimation under a missing-at-random assumption, standard for mixed-effects models.

Multiple comparisons within the 2 × 2 factorial framework will be controlled via Bonferroni correction applied within each outcome domain (text-based prediction, acoustic baseline, fusion). We will test all fixed effects (model size main effect, temperature main effect, Model Size × Temperature interaction) at α = 0.05 (two-tailed). post-hoc comparisons between specific factorial cells (e.g., testing whether GPT-OSS-120B at temperature 0.8 outperforms GPT-OSS-20B at temperature 0.1) will be pre-specified and corrected for multiple comparisons.

This simplified, focused approach ensures that our study adheres to pre-registration principles, prevents post-hoc methodological flexibility, and provides a reproducible framework that other research groups can directly replicate or build upon.

Access to the full protocol, participant-level data, and statistical code will be restricted and carefully controlled. Researchers interested in accessing these materials must submit a formal application, which will be subject to review by the Institutional Review Board (IRB) for local data processing. This process is designed to ensure the protection of sensitive information and maintain the integrity of the research. It is important to note that, in accordance with the decision of the data security board at the University of Frankfurt, no data can be shared outside of this controlled environment consisting of the study team in Frankfurt and selected members of the study team in Darmstadt.

Director of Studies are PD Dr med. Oliver Grimm (OG) & Prof. Dr. Andreas Reif (AR) from the Department of Psychiatry, Psychosomatics and Psychotherapy, University Hospital Frankfurt. Further, the UKP Lab headed by Prof. Dr. Iryna Gurevych (IG) will cooperate and is responsible for the processing of the data, feature extraction and building of an NLP/multimodal model. The trial steering committee consists of OG, AR, and IG, along with a senior statistician and an independent expert in psychiatry research. This committee will oversee the overall conduct of the study, monitor its progress, and make key decisions regarding the trial's direction and any necessary protocol amendments.

According to German law, no formal data monitoring committee is needed for a study without therapeutic intervention.

Protocol modifications will firstly be sent to the central and local ethics committees. Once changes are approved, currently enrolled patients will be asked to sign the newest version of the informed consent form accordingly.