The evaluation is based on the COVID-19 cohort study named Sektorenübergreifende Plattform (SÜP) (English: “cross-sector platform”), which was part of the National Pandemic Cohort Network (German: Nationales Pandemie Kohorten Netz, NAPKON) (11) for COVID-19 research. The SÜP participants were recruited in university hospitals, non-university hospitals, medical practices, and medical care centers, in a joint effort for COVID-19 research.

SÜP recruited SARS-CoV-2-positive study participants and persons who were SARS-CoV-2-negative as a control cohort. The inclusion criteria for the study participants were a positive polymerase chain reaction (PCR) test that detected SARS-CoV-2 and enrollment within 1 week after a positive PCR test. Medical staff in university and non-university clinics, medical practices (including GPs), and care centers actively invited eligible persons to participate in the SÜP cohort. The study participants received written information that described in detail how and which data would be collected, processed, and used scientifically (2). To provide informed consent, eligible persons filled out and signed either a paper-based CF or an electronic CF using a tablet PC. Both CF approaches had identical content.

Study participants were excluded if their CF was not signed, if their choice was not unambiguously stated, or if the paper form was not filled in correctly and the CF was not corrected within a certain amount of time.

All the SÜP-CFs were designed as modular CFs (12). This means that following introductory text that informs the participant about SÜP and the privacy policy in general, multiple modules (obligatory and optional) are offered that specifically address dimensions of the study, e.g., medical data, imaging data, and biological samples. At the end of the form, the signatures of the study participant and the medical professional are requested. Optional modules mean that a study participant can refuse or consent to their data being used for several specific purposes, e.g., whether health insurance data can be requested or whether the participant agrees to being re-contacted for defined purposes (for more information, refer to Appendix A). Study participants can indicate their choice for each module individually by ticking either “yes” (consent) or “no” (refusal). Two versions of the consent form were available to the study sites, one with the biomaterial collection option and one without, depending on the capabilities and requirements of the respective study site. Modified CF versions were available to meet the specific requirements of individual federal states or universities.

All the SÜP-CFs were managed by the Trusted Third Party of the University Medicine Greifswald (TTP) using the generic Informed Consent Service (gICS®) (12–15), a free-of-charge and open-source (AGPLv3) software solution (15). The term “management” refers to the generation of both the CF templates that are presented on a tablet or printed and the specific dataset items from the filled-in and signed CFs. The gICS® was integrated into both consent processes and supports paper-based and tablet-based (fully electronic) CF collection. For this, the gICS® provides interfaces for software that manage (study-) pseudonyms [e.g., gPAS (16)] and person-identifying data, including record linkage [e.g., E-PIX (17)]. In addition, interfaces for the SÜP’s infrastructural partners, such as Clinical Data Management (medical data), the imaging data management system, and the LIMS, have been implemented and used since November 2020.

The goal of IC management in complex research projects that integrate various data sources and systems is to provide readily available, structured, and machine-readable information on the consent status of each study participant for each CF module via interfacing (for automatic exchange between electronic systems) or a graphical user interface (GUI, for user-specific queries). Therefore, a structured version of each participant's CF information is a prerequisite for the electronic use of CF data.

Figure 1 illustrates different recruitment workflows with a focus on CF collection and the necessary additional manual work by the study staff who recruit the participants to provide structured and machine-readable consent information.

Recruitment workflows that include CF collection can be classified as follows: (1) a purely paper-and-pencil-based approach, (2) a paper-based approach with digital scan archiving, (3) a digitization approach, and (4) a fully electronic approach. For approaches 1 and 2, paper-based forms are filled in and are either directly filed in a paper-based patient record (approach 1) or are scanned and digitally archived as unstructured images in an electronic health record (approach 2). In both cases, only on-site source data quality assurance measures are possible, and no structured, digitized, and machine-readable information is available. Approaches 3 and 4, as depicted in Figure 1, allow for modular retrievable CF information but differ in the number of steps required. For example, the third approach includes digitization and structuring of information: informed consent is collected on a paper form and then manually documented in an electronic form, which means manually converting the CF form into machine-readable information (manual transcription or using optical character recognition techniques). In addition, the scan of the CF is attached as an unstructured image, and both the structured information and the unstructured image are subsequently used in the central QA measures (e.g., source data verification). As a result, quality-assured, structured, machine-readable, and queryable consent data are available. In the fully electronic approach (approach 4), CFs are filled in via a tablet PC using electronic case report forms. All the information is directly entered by the study staff and participant in a structured way, allowing for immediate plausibility checks. The participant's and the health professional's signatures are also obtained in a digital format. Thus, consent data are provided as structured and machine-readable information upon completion and saving of the CF to the database. Since all the information is shared with the local (patient) information system, no further (manual) work is needed to complete and document the CF of the participant and their consent status.

In SÜP, CFs were collected using approaches 3 (digitization approach) and 4 (fully electronic approach), as illustrated in Figure 1. In both cases, training material on how to document CFs and initial one-on-one training material were provided by the TTP staff.

The core workflow was very similar in both approaches: the clinician filled in the person-identifying information (first and last names, date, and place of birth) on the CF and handed the CF (in a paper form or on a tablet screen) to the eligible person. The potential study participant checked their person-identifying information and corrected it if there were any errors. Furthermore, the potential participant read the provided information, ticked the optional modules “yes” or “no” according to their choice, and signed the CF. Afterward, the CF (in a paper form or on a tablet screen) was returned to the clinician, who also signed the form and stored it (by filing it in the patient's record or by saving it on the tablet, which automatically sent it to the TTP via a secure connection). One copy of the filled-in CF remained with the study participant. In the case of the tablet-based CF, it was printed out after CF completion and provided to the participant.

Paper-based CFs were distributed to each recruiting institution by the SÜP study coordination staff as site-specific printable documents via a web portal. The documents were printed as needed on-site. Digitized CFs were provided as electronic forms by the TTP in the gICS®. The fully electronic consent collection was conducted using tablet PCs and the TTP's gICS® as a secure web application (a personalized TTP safety browser certificate was necessary). Thus, the digital CF collection operated independently of the tablet’s version, operating system, and brand. Therefore, tablet PCs could either be pre-installed with the software and provided by the TTP or the study sites could also use their own existing devices.

To evaluate the impact of the recruitment processes on the study’s inclusion rate and burden on the study staff, the participants who provided their IC via the paper-based digitization approach (Figure 1, approach 3) were used as the control group, because this was the most common way of obtaining consent in the clinical settings of the cohort study analyzed in this article. Those who provided IC via the fully electronic collection approach using a tablet PC (Figure 1, approach 4) were the intervention group. Please note that this is a process evaluation and not a systematic case-control study because the study sites were free to choose whether to collect consent via the paper-based or fully electronic approaches or by using both approaches.

To evaluate the fully electronic CF collection approach compared to the paper-based CF capture procedure, the following criteria were analyzed: (a)

initial CF validity (with/without possible quality issues);

“Time-to-initial availability of structured consent information” (criterion b) describes the machine-readable availability of structured consent information in other systems, such as an HIS, PACS, and LIMS. This is calculated as the duration between the timestamps for the “original CF signature” and “CF recorded digitally.”

Criterion c, “time-to-possible-research,” evaluates the time needed to confirm quality-assured CFs without quality issues, these being the legal basis to use participants’ datasets for research. This can differ according to the initial quality status (with/without quality issues), and, therefore, was divided into “time-to-possible-research” for (i) initially correct CFs and (ii) CFs with initial quality issues. The time was calculated using the timestamps for “original CF signature” and “successful completion of CF quality assurance.” The measurement units (hours, days, or weeks) of “time-to-usage” and “time-to-research” were determined based on the results. However, the “time-to-possible-research” may have differed from the actual time of availability of the datasets from the SÜP participants to researchers, as further quality criteria independent of consent needed to be observed before the datasets could be made available for research.

The practical usage of paper- or tablet-based CF collection approaches by several medical professions (e.g., data managers, doctors, and study nurses) resulted in feedback provided to the TTP during workshops, subsequent training, or support sessions. Therefore, the benefits and disadvantages of each approach, according to the feedback received from colleagues involved in patient care, were also evaluated for this article as criterion d.

This article only focuses on CFs; thus, withdrawals or other forms of study exclusion besides an invalid CF were not considered.

The concept and the processes of IC collection in SÜP were based on the different CFs used in the study. In principle, only CFs that had previously been approved by an ethics committee responsible for the respective site and/or federal state were used at the individual study sites. Therefore, it is possible that only certain versions or a subset of the CFs were available at a particular site. As of July 2024, 63 different versions of SUP-CFs were implemented for 15 federal states and 75 potential recruiting institutions (i.e., registered with the TTP).

Overall recruitment started in November 2020 and ended for COVID-19 patients on 31 July 2023. Control group recruitment ended on 31 December 2023, and follow-ups were possible up until 1 year after infection, i.e., 2024. The fully electronic recording of CFs ceased in June 2023. This was due to the WHO's declaration regarding COVID-19 on 5 May 2023 that lowered its classification from the highest alert to treating and managing COVID-19 like other (endemic) infectious diseases (20). Following this declaration, the number of specialized COVID-19 departments was reduced and the number of new COVID-19 patients at the respective study sites decreased. In the SÜP study, 57 study sites recruited 2,753 study participants (without duplicates and withdrawals) with a total of 3,509 CFs as of 29 July 2024. Among the study sites, 45 (79%) collected paper-based CFs exclusively, while another 11 sites (19%) collected both paper- and tablet-based CFs. Of those 11 sites, three sites collected 85% or more of the CFs via tablets. Only one site (2%) captured all CFs fully electronically using designated tablet PCs. Consequently, the majority of the CFs (3,136, 89.37%) were collected in a paper form, and 373 (10.63%) CFs were captured using the fully electronic approach as of 29 July 2024.

Figure 2 illustrates how the initial CF validity differed between the approaches. Of the CFs that had been quality-assured as of 29 July 2024, 2,113 (67.38%) paper-based and 371 (99.46%) tablet-based CFs were initially valid without any associated quality issues. The remaining 1,023 (32.62%) paper-based CFs and 2 (0.54%) tablet-based CFs showed quality issues and, consequently, were initially invalid.

Interestingly, 501 (15.98%) paper-based CFs showed quality issues related to the documentation process/completeness, and 521 CFs (16.61%) had quality issues related to the structured mapping of CF content/correctness as of July 2024 (see Figure 1, approach 3). Feedback on the quality issues was sent back to the respective study sites with specific requests for correction. In a cohort study, if an invalid CF is not corrected, the corresponding participant's data, including biological samples, must be deleted and are lost to the research.

Most of the quality issues with the paper-based CFs were related to missing scanned pages, missing signatures or dates, or using the wrong CF version. All of these issues were automatically averted in the tablet-based CFs due to the features of electronic data capture. Dates and signatures can be entered easily and are mandatory, i.e., the CF cannot be finalized and saved without entering them. The user is immediately made aware of any missing items by a prompt error message. In addition, correct CF versions are automatically provided, since the study site can only see and choose from their respective valid CF forms. The most common issues with the tablet-based CFs were implausible dates, e.g., the date of the signature is also the participant’s date of birth, or signature fields that were filled in incorrectly [e.g., only a dot or “TN” for “participant” (German: Teilnehmende) is provided in the signature field]. Thus, the fully electronic IC collection approach via a tablet prevented the majority of quality issues.

After QA was conducted, 935 CFs were corrected by the study sites, with six CFs requiring internal TTP corrections received from study sites (e.g., changing the electronic consent version). However, 82 (2.62%) paper-based CFs remained invalid (Figure 2).

As Figure 3 shows, the initial quality of the paper-based CFs varied greatly over time, whereas the quality of fully electronic CFs remained at a consistently high level.

In two cases, human error led to invalid electronic CFs. In the first, the wrong CF version was used, and in the second, the date of birth was incorrectly entered. In the first case, the caregivers of a pediatric patient filled in a CF used for people of legal age instead of a pediatric CF. In this case, the study site provided the correct CF version. The second quality issue occurred during the creation of the participant's record, as the current date was incorrectly used as their date of birth. In this case, the subsequent TTP QA process invalidated the electronic CF, and the study site corrected the CF for the patient.

The “time-to-initial availability of structured consent information,” calculated from the “original CF signature” and “CF recorded digitally” timestamps, was assessed in days.

A CF can only be used by other systems, such as a HIS, PACS, or LIMS, if it is in a structured digital format. When using tablet-based CFs, which are initially captured as structured digital information, no delay between CF collection and usage by other systems was detected. Therefore, the time-to-initial usage was approximately 0 days for the fully electronic consent form collection approach.

Paper-based forms have to be manually digitized and entered into electronic forms to provide structured information for other systems. Hence, the time-to-initial availability in other systems can include a delay. However, there is no delay (0 days) if the paper-based CF is digitized on the same day as it was signed by the study participant. In total, up to 357 days of delay were registered with a mean delay of 17 days. Long delays were usually detected during periods with high incidence rates. However, the median delay was only 1 day.

To calculate the “time-to-possible-research,” the difference between the “original CF signature” and “successful completion of CF quality assurance” timestamps was used for the (i) initially correct CFs and (ii) CFs with quality issues, respectively. The results were measured in days.

QA was conducted for all CFs, whether they were newly captured or corrected by the study site. Initially correct CFs can still result in delays if the CF digitization is delayed. The results show that the tablet-based CF collection approach did not result in any time lag (0 days), while the paper-based CF collection approach had a mean delay of 17 days (see previous section).

For CFs with initial quality issues, the main reason for a delay was the time taken to correct the CF at the study site. This usually required waiting for the patient’s next appointment.

Table 1 shows the calculated time-to-possible-research values for paper-based CFs in comparison to tablet-based CFs, with considerably higher values on average for the paper-based CFs. Since the median is less sensitive to outliers, the median value was used for this evaluation criterion.

Human factors in the recruitment process and user experiences are major contributors to successful recruitment and study inclusion. The qualitative feedback from clinicians, study staff, and TTP staff regarding the two different recruitment approaches is summarized in Table 2 according to the predefined evaluation criteria and is complemented by the benefits and disadvantages identified in the previous subchapters.