<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Archiving and Interchange DTD v2.3 20070202//EN" "archivearticle.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" article-type="methods-article" dtd-version="2.3" xml:lang="EN">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Med.</journal-id>
<journal-title>Frontiers in Medicine</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Med.</abbrev-journal-title>
<issn pub-type="epub">2296-858X</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fmed.2024.1378866</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Medicine</subject>
<subj-group>
<subject>Methods</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>OHDSI-compliance: a set of document templates facilitating the implementation and operation of a software stack for real-world evidence generation</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name><surname>Wirth</surname> <given-names>Felix N.</given-names></name>
<uri xlink:href="https://loop.frontiersin.org/people/2432467/overview"/>
<role content-type="https://credit.niso.org/contributor-roles/conceptualization/"/>
<role content-type="https://credit.niso.org/contributor-roles/resources/"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-original-draft/"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-review-editing/"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Abu Attieh</surname> <given-names>Hammam</given-names></name>
<role content-type="https://credit.niso.org/contributor-roles/resources/"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-original-draft/"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-review-editing/"/>
</contrib>
<contrib contrib-type="author" corresp="yes">
<name><surname>Prasser</surname> <given-names>Fabian</given-names></name>
<xref ref-type="corresp" rid="c001"><sup>&#x002A;</sup></xref>
<uri xlink:href="https://loop.frontiersin.org/people/2707450/overview"/>
<role content-type="https://credit.niso.org/contributor-roles/conceptualization/"/>
<role content-type="https://credit.niso.org/contributor-roles/resources/"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-original-draft/"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-review-editing/"/>
</contrib>
</contrib-group>
<aff><institution>Berlin Institute of Health at Charit&#x00E9; &#x2013; Universit&#x00E4;tsmedizin Berlin, Center of Health Data Science</institution>, <addr-line>Berlin</addr-line>, <country>Germany</country></aff>
<author-notes>
<fn fn-type="edited-by" id="fn0001">
<p>Edited by: Gokce Banu Laleci Erturkmen, Software Research and Development Consulting, T&#x00FC;rkiye</p>
</fn>
<fn fn-type="edited-by" id="fn0002">
<p>Reviewed by: Pantelis Natsiavas, Institute of Applied Biosciences, Greece</p>
<p>Martin Hofmann-Apitius, Fraunhofer Institute for Algorithms and Scientific Computing (FHG), Germany</p>
</fn>
<corresp id="c001">&#x002A;Correspondence: Fabian Prasser, <email>fabian.prasser@charite.de</email></corresp>
</author-notes>
<pub-date pub-type="epub">
<day>16</day>
<month>05</month>
<year>2024</year>
</pub-date>
<pub-date pub-type="collection">
<year>2024</year>
</pub-date>
<volume>11</volume>
<elocation-id>1378866</elocation-id>
<history>
<date date-type="received">
<day>30</day>
<month>01</month>
<year>2024</year>
</date>
<date date-type="accepted">
<day>02</day>
<month>05</month>
<year>2024</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x00A9; 2024 Wirth, Abu Attieh and Prasser.</copyright-statement>
<copyright-year>2024</copyright-year>
<copyright-holder>Wirth, Abu Attieh and Prasser</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/">
<p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p>
</license>
</permissions>
<abstract>
<sec id="sec1">
<title>Introduction</title>
<p>The open-source software offered by the Observational Health Data Science and Informatics (OHDSI) collective, including the OMOP-CDM, serves as a major backbone for many real-world evidence networks and distributed health data analytics platforms. While container technology has significantly simplified deployments from a technical perspective, regulatory compliance can remain a major hurdle for the setup and operation of such platforms. In this paper, we present OHDSI-Compliance, a comprehensive set of document templates designed to streamline the data protection and information security-related documentation and coordination efforts required to establish OHDSI installations.</p>
</sec>
<sec id="sec2">
<title>Methods</title>
<p>To decide on a set of relevant document templates, we first analyzed the legal requirements and associated guidelines with a focus on the General Data Protection Regulation (GDPR). Moreover, we analyzed the software architecture of a typical OHDSI stack and related its components to the different general types of concepts and documentation identified. Then, we created those documents for a prototypical OHDSI installation, based on the so-called Broadsea package, following relevant guidelines from Germany. Finally, we generalized the documents by introducing placeholders and options at places where individual institution-specific content will be needed.</p>
</sec>
<sec id="sec3">
<title>Results</title>
<p>We present four documents: (1) a record of processing activities, (2) an information security concept, (3) an authorization concept, as well as (4) an operational concept covering the technical details of maintaining the stack. The documents are publicly available under a permissive license.</p>
</sec>
<sec id="sec4">
<title>Discussion</title>
<p>To the best of our knowledge, there are no other publicly available sets of documents designed to simplify the compliance process for OHDSI deployments. While our documents provide a comprehensive starting point, local specifics need to be added, and, due to the heterogeneity of legal requirements in different countries, further adoptions might be necessary.</p>
</sec>
</abstract>
<kwd-group>
<kwd>health data analytics</kwd>
<kwd>real-world evidence</kwd>
<kwd>observational health data science</kwd>
<kwd>regulatory compliance</kwd>
<kwd>data protection</kwd>
</kwd-group>
<counts>
<fig-count count="2"/>
<table-count count="1"/>
<equation-count count="0"/>
<ref-count count="47"/>
<page-count count="7"/>
<word-count count="5125"/>
</counts>
<custom-meta-wrap>
<custom-meta>
<meta-name>section-at-acceptance</meta-name>
<meta-value>Regulatory Science</meta-value>
</custom-meta>
</custom-meta-wrap>
</article-meta>
</front>
<body>
<sec sec-type="intro" id="sec5">
<label>1</label>
<title>Introduction</title>
<sec id="sec6">
<label>1.1</label>
<title>Background</title>
<p>Collecting and analyzing data from real-world healthcare settings at a broad scale can provide new insights into patient outcomes, treatment efficacy, and healthcare practices (<xref ref-type="bibr" rid="ref1">1</xref>). This usually necessitates bringing together data from several healthcare institutions, which requires the implementation of or mapping to data standards, as well as approaches for ethical and data protection compliant access (<xref ref-type="bibr" rid="ref2">2</xref>). One common solution for the latter challenge is federation, where the analysis is brought to the data instead of bringing the data to the analysis (<xref ref-type="bibr" rid="ref3">3</xref>). This is, for example, implemented by SHRINE (<xref ref-type="bibr" rid="ref4">4</xref>), DataSHIELD (<xref ref-type="bibr" rid="ref5">5</xref>) and the Observational Health Data Sciences and Informatics (OHDSI) (<xref ref-type="bibr" rid="ref6">6</xref>) initiative. OHDSI is an international, multidisciplinary community of researchers and healthcare professionals to enable data standardization, analysis, and insight discovery from large-scale health datasets, launched in 2013. The community distributes a set of open-source software tools to represent and analyze data in the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM), which makes extensive use of terminologies and ontologies, such as Logical Observation Identifiers Names and Codes (LOINC) or Systematized Nomenclature of Medicine (SNOMED) Clinical Terms (CT) (<xref ref-type="bibr" rid="ref7">7</xref>). While the term OMOP describes the now discontinued collaboration that originally developed the CDM, the term OMOP-CDM refers to the further developed version that forms the current technical cornerstone of OHDSI. The EHDEN project has funded the deployment of the OMOP-CDM and the OHDSI software stack across Europe (<xref ref-type="bibr" rid="ref8">8</xref>). Moreover, the OMOP-CDM will also play an important role in the upcoming European Health Data Space (EHDS; see Section &#x201C;Discussion&#x201D;). The EHDS is planned as a large-scale ecosystem facilitating better exchange and access to different types of health data throughout the European Union (EU). EHDS pillar I focuses on primary healthcare data use, i.e., data sharing for healthcare delivery. EHDS pillar II focuses on secondary use of health data, e.g., analysis for research, policy-making or drug safety (<xref ref-type="bibr" rid="ref9">9</xref>).</p>
<p>Setting up an OHDSI node can involve significant efforts, in particular for the required mapping to standards. However, technical and data integration challenges are not the only obstacles faced when connecting to data sharing networks [for one example for the various technical challenges see (<xref ref-type="bibr" rid="ref10">10</xref>)]. Legal and regulatory compliance is another important issue (<xref ref-type="bibr" rid="ref11">11</xref>, <xref ref-type="bibr" rid="ref12">12</xref>). National and international data protection laws as well as ethical guidelines must be considered. Important examples include the US Health Insurance Portability and Accountability Act (HIPAA) (<xref ref-type="bibr" rid="ref13">13</xref>) and the European Union (EU) General Data Protection Regulation (GDPR) (<xref ref-type="bibr" rid="ref14">14</xref>). To fulfill central requirements, concepts need to be developed and documented for ensuring the confidentiality of the processed healthcare data. An important example is the so-called Record of Processing Activities (ROPA), which needs to be created according to the GDPR, but also according to laws in the United Kingdom (<xref ref-type="bibr" rid="ref15">15</xref>, <xref ref-type="bibr" rid="ref16">16</xref>), Australia (<xref ref-type="bibr" rid="ref17">17</xref>) or Thailand (<xref ref-type="bibr" rid="ref18">18</xref>). Amongst other aspects, a ROPA typically describes the processed categories of data and details information flows as well as the technical and organizational security measures implemented, although slight variations might exist between the requirements in different countries. Moreover, information security plays an important role, with relevant standards also requiring documentation of the measures taken (<xref ref-type="bibr" rid="ref19">19</xref>). Important examples include the International Standards Organization (ISO) Standard 27001 (<xref ref-type="bibr" rid="ref20">20</xref>), (<xref ref-type="bibr" rid="ref2">2</xref>) the US National Institute of Standards and Technology (NIST) Cybersecurity Framework (<xref ref-type="bibr" rid="ref21">21</xref>) or (<xref ref-type="bibr" rid="ref3">3</xref>) the Health Information Trust Alliance Common Security Framework (HITRUST CSF) (<xref ref-type="bibr" rid="ref22">22</xref>).</p>
</sec>
<sec id="sec7">
<label>1.2</label>
<title>Objective</title>
<p>It is well known that conceptualizing and documenting the secure operation of data processing platforms can be challenging (<xref ref-type="bibr" rid="ref23">23</xref>, <xref ref-type="bibr" rid="ref24">24</xref>). Research has shown that even reading and comprehending such documents can be difficult (<xref ref-type="bibr" rid="ref25 ref26 ref27">25&#x2013;27</xref>). As a result, different guidelines and templates have been developed (see Section <italic>Comparison with prior work</italic>). However, those are usually generic in nature and not directly applicable to the establishment of an OHDSI node. The objective of the work described in this paper, was to conceptualize an approach specifically for common OHDSI deployments. Moreover, we developed document templates that can be customized to local requirements. We focus on documents for a general OHDSI setup. Depending on the nature of projects that use this infrastructure as well as local requirements, additional documents might be needed for the individual studies performed.</p>
</sec>
</sec>
<sec sec-type="methods" id="sec8">
<label>2</label>
<title>Methods</title>
<sec id="sec9">
<label>2.1</label>
<title>Overview of the OHDSI tools</title>
<p>The main tools provided by OHDSI are focused on (1) establishing a common data model with clearly defined structure and semantics, as well as (2) assisting medical researchers and data scientists in extracting knowledge from this data. The OMOP-CDM is the central pillar of OHDSI, providing a standardized database schema and a set of terminologies with which heterogeneous data from different sources can be integrated to provide comparability across studies and institutions (<xref ref-type="bibr" rid="ref28">28</xref>). As a result, OHDSI forms a global network allowing for large-scale distributed studies to be performed. A common database management system for instances of the OMOP-CDM is <italic>PostgreSQL</italic> (<xref ref-type="bibr" rid="ref29">29</xref>). In addition, the following tools are provided for data mapping:</p>
<list list-type="bullet">
<list-item>
<p><italic>WhiteRabbit</italic> is a tool to scan and describe source data.</p>
</list-item>
<list-item>
<p><italic>Rabbit in a Hat</italic> supports structural mapping between source data and the OMOP-CDM.</p>
</list-item>
<list-item>
<p><italic>USAGI</italic> has been designed to support semantic standardization and terminology mapping.</p>
</list-item>
<list-item>
<p><italic>Athena</italic> is as a publicly available web service providing access to the vocabulary used by the OMOP-CDM.</p>
</list-item>
</list>
<p>We note that OHDSI does not provide a standard tool for extracting, transforming and loading (ETL) data, but focuses on tools for specifying the transformations and mappings needed. A common way of deploying a standard OHDSI stack is the container-based <italic>Broadsea</italic> distribution (<xref ref-type="bibr" rid="ref30">30</xref>). An overview of a typical set of components in Broadsea is provided in <xref ref-type="fig" rid="fig1">Figure 1</xref>.</p>
<fig position="float" id="fig1">
<label>Figure 1</label>
<caption>
<p>Common architecture of an OHDSI implementation.</p>
</caption>
<graphic xlink:href="fmed-11-1378866-g001.tif"/>
</fig>
<p>As can be seen, a common installation contains the following additional infrastructure components:</p>
<list list-type="bullet">
<list-item>
<p>A <italic>PostgreSQL</italic> database for storing configuration options and study designs.</p>
</list-item>
<list-item>
<p><italic>Apache SOLR</italic> for searching through the vocabulary.</p>
</list-item>
<list-item>
<p><italic>OpenLDAP</italic> for authentication and authorization.</p>
</list-item>
</list>
<p>Based on this basic infrastructure and the CDM, the Broadsea distribution offers further applications for accessing and analyzing the data:</p>
<list list-type="bullet">
<list-item>
<p><italic>WebAPI</italic> is a RESTful service layer for accessing and analyzing data stored in the OMOP-CDM.</p>
</list-item>
<list-item>
<p><italic>ATLAS</italic> is a web-based tool for conducting scientific analyses.</p>
</list-item>
<list-item>
<p><italic>ARES</italic> is a system facilitating data exploration, characterization, and quality assessments.</p>
</list-item>
<list-item>
<p><italic>RStudio</italic> for analyzing data using the statistical programming language R. Broadsea comes with a range of R-packages, such as Shiny for developing interactive web applications and HADES for analyzing data from the OMOP-CDM.</p>
</list-item>
</list>
<p>In summary, researchers can work with data stored in the OMOP-CDM through ATLAS and specific R packages. ATLAS provides graphical access to a variety of OHDSI tools and functions, trading usability off against the flexibility of the analyses that can be performed. In addition, analyses can be performed in R using a set of provided packages and APIs, providing more flexibility in working with the data but requiring programming and data science skills.</p>
</sec>
<sec id="sec10">
<label>2.2</label>
<title>Development process</title>
<p>We first identified a set of documents usually required to deploy and operate research systems at German university hospitals. As a basis, these include (1) a description of the processing activities and the technical and organizational measures taken in regards to data protection, (2) an analysis of information security risks and security-related measures taken, (3) a description of processes and responsibilities for maintaining and operating the system. We note that these documents need to be updated regularly following a continuous improvement process.</p>
<p>Next, we related those documents to the systems and processes covered by the common architecture described in the previous section. Data protection aspects were described with a specific focus on systems holding or processing individual-level health data, reflecting requirements by Article 30 GDPR on the content of the description of processing activities. Information security as well as operation of the stack was covered for the complete installation, oriented towards the information security basic protection methodology provided by the German government. Moreover, another document was developed to describe and implement governance processes for use of the data available in the CDM. Finally, we transformed the documents into customizable templates and uploaded them into a version-controlled repository.</p>
</sec>
</sec>
<sec sec-type="results" id="sec11">
<label>3</label>
<title>Results</title>
<sec id="sec12">
<label>3.1</label>
<title>Overview</title>
<p><xref ref-type="table" rid="tab1">Table 1</xref> provides an overview of the different document templates developed and provided through a GitHub repository (<xref ref-type="bibr" rid="ref31">31</xref>).</p>
<table-wrap position="float" id="tab1">
<label>Table 1</label>
<caption>
<p>Overview of the document templates.</p>
</caption>
<table frame="hsides" rules="groups">
<thead>
<tr>
<th align="left" valign="top">Document title</th>
<th align="left" valign="top">Short description</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="top">Record of processing activities</td>
<td align="left" valign="top">Description of the data processing activities and protection measures.</td>
</tr>
<tr>
<td align="left" valign="top">Information security concept</td>
<td align="left" valign="top">Description of information security measures.</td>
</tr>
<tr>
<td align="left" valign="top">Concept of operations</td>
<td align="left" valign="top">Description of processes and responsibilities when operating the installation.</td>
</tr>
<tr>
<td align="left" valign="top">Authorization concept</td>
<td align="left" valign="top">Description of groups of user roles and their permissions as well as a description of the process for requesting access to the database.</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec id="sec13">
<label>3.2</label>
<title>Record of processing activities</title>
<p>A general description of the software architecture, data flows and processing activities as well as protection measures taken forms the basis of most compliance framework for medical research systems. Thus, as a first component, we developed a template for a Record of Processing Activities (ROPA) for OHDSI installations. As outlined above, ROPAs or related documents are required in most jurisdictions. In this work, we base the content on the requirements outlined in Article 30 of the GDPR and provide information about the personal data processed, the purposes of the processing, retention periods and further relevant details. In the event of legal or data protection audits, the document can be used as a basis to demonstrate compliance and it can also serve as a communication measure for coordinating OHDSI-related activities with an institution&#x2019;s Data Protection Officer.</p>
</sec>
<sec id="sec14">
<label>3.3</label>
<title>Information security concept</title>
<p>While data protection and the ROPA template emphasizes the handling of personal data in a way that respects the rights and expectations of the data subjects, information security focuses on protecting data from unauthorized access and further threats more relevant to the organization itself than to the data subjects. The well-known ISO/IEC 27000 standard emphasizes confidentiality, integrity, and availability, but also adds further aspects, such as authenticity, accountability, non-repudiation, and reliability (<xref ref-type="bibr" rid="ref32">32</xref>).</p>
<p>To cover these aspects, we provide a template for describing information security-related properties of OHDSI installations. The template is pragmatic and designed to complement existing information security guidelines at the institution operating the installation. It contains a risk analysis of basic processes carried out with OHDSI installations, such as data transformation, loading, and usage, and systematically describes relevant information security measures. As an example, we use modules from the &#x201C;Basic Protection&#x201D; methodology of the Federal Office for Information Security in Germany. While there are some differences to the ISO 27000 set of standards, the &#x201C;Basic Protection&#x201D; methodology provides a solid foundation of security controls for achieving ISO 27001 compliance. An organization that already applies ISO 27000 can, for example, benefit from our documents through the included risk assessments and lists of relevant security controls that can inform local information security management processes. The document can also support coordination with an institutions Chief Information Security Officer (CISO).</p>
</sec>
<sec id="sec15">
<label>3.4</label>
<title>Concept of operations</title>
<p>In addition to a sound and secure setup of an OHDSI node, also the operation of the platform needs to be conceptualized and described. Relevant processes also include the continuous improvement process for data protection and information security-related aspects already described above. In addition, the installed components and their configurations need to be kept up to date, user accounts need to be managed and backups need to be performed. The template for an operational concept includes suggestions for those processes, tailored towards the OHDSI components.</p>
</sec>
<sec id="sec16">
<label>3.5</label>
<title>Authorization concept</title>
<p>How access requests by researchers to the OHDSI tools are handled and what governance rules are implemented is an important aspect of compliance. Consequently, we also developed a template for a guideline on how this is implemented. The template describing the access request process describes the duties of administrative personnel responsible for overseeing user access and processes for regular review and removal of outdated permissions. Additionally, it describes the steps researchers must follow to obtain access for conducting studies, including obtaining necessary approvals. In addition to researchers accessing the OHDSI tools, there are further types of personnel involved that need to access the installation for operational purposes. As this is a critical aspect, the proposed template describes all relevant roles, their responsibilities, and access permissions. The template outlines processes for nominating administrators, setting up user access and revoking them upon project completion or staff changes. Moreover, password guidelines and rules for timeouts of sessions are included.</p>
<p><xref ref-type="fig" rid="fig2">Figure 2</xref> illustrates how the developed document templates cover different components and aspects of a common OHDSI installation. As can be seen, the ROPA focuses on the general setup that processes personal data, while the information security concept and related templates cover all components. Access management focuses specifically on humans involved in the maintenance and use of an installation.</p>
<fig position="float" id="fig2">
<label>Figure 2</label>
<caption>
<p>Role of the different documents in a common OHDSI deployment.</p>
</caption>
<graphic xlink:href="fmed-11-1378866-g002.tif"/>
</fig>
</sec>
<sec id="sec17">
<label>3.6</label>
<title>Customization and document management</title>
<p>We have developed the templates as Markdown files and provide them in the form of a Git repository. Markdown is a lightweight markup language, designed to be easy to write and read, with the ability to present the document content in many different forms. For example, the documents provided can be compiled into PDF files using open-source tools, such as Pandoc. If visual editing is needed, tools like Pandoc can also be used to convert the markdown files into formats suited for word processors, such as the Open Document Format. We recommend to use the templates in their Markdown version, however, as this naturally enables keeping track of changes in versioned repositories, such as Git.</p>
</sec>
</sec>
<sec sec-type="discussion" id="sec18">
<label>4</label>
<title>Discussion</title>
<sec id="sec19">
<label>4.1</label>
<title>Principal results</title>
<p>We presented a set of templates for setting up and maintaining OHDSI installations in compliance with data protection and information security requirements, also covering data governance aspects. The document templates are public available under a permissive license. The templates are meant to provide a starting point and need to be filled out accordingly and potentially extended or modified to comply with local policies or legal requirements. We have successfully executed this process at Charit&#x00E9; &#x2013; Universit&#x00E4;tsmedizin Berlin.</p>
</sec>
<sec id="sec20">
<label>4.2</label>
<title>Comparison with related work</title>
<p>Several institutions or research groups have suggested compliance-oriented document templates for data processing in general or for medical research contexts. Examples include data protection guidelines, see (<xref ref-type="bibr" rid="ref33">33</xref>) for an example, and templates for institutional review board protocols, see (<xref ref-type="bibr" rid="ref34">34</xref>) for an example, and information security aspects, see (<xref ref-type="bibr" rid="ref35">35</xref>) as an example. Quite a lot of the documents are tailored towards specific jurisdictions and published in languages other than English [e.g., (<xref ref-type="bibr" rid="ref33">33</xref>, <xref ref-type="bibr" rid="ref36">36</xref>)]. Our work is different in that it focuses on a typical deployment of a common medical research platform and that its content has been, in large parts, abstracted away from country-specific requirements. Previous work has also focused on compliance for deployments of specific research systems (see the work by Wallace et al. (<xref ref-type="bibr" rid="ref37">37</xref>) and by Budin-Lj&#x00F8;sne et al. (<xref ref-type="bibr" rid="ref38">38</xref>) for an example on the DataSHIELD software). To the best of our knowledge, our work is the first to target OHDSI deployments. Governance models have also been studied in the literature. For example, Holmes et al. have presented an overview on governance models for federated research (<xref ref-type="bibr" rid="ref39">39</xref>). The authors propose a framework with which governance models can be assessed and compared considering different aspects. Pavlenko et al. have focused on data governance for health data warehouses (<xref ref-type="bibr" rid="ref40">40</xref>).</p>
<p>On a more general level, ethical and legal challenges in data-driven biomedical research have also been studied extensively. For instance, Wang et al. discussed several privacy-enhancing technologies and argue that accountability and informed consent are among the most relevant ethical challenges (<xref ref-type="bibr" rid="ref41">41</xref>). Arellano et al. conduct a review on privacy regulations, patient perspectives as well as consent practices and their interaction with technology (<xref ref-type="bibr" rid="ref42">42</xref>). They cover questions, such as under which circumstances consent can be considered ethical. Lamas et al. have argued that ethical and legal frameworks are often not fitting well to common scenarios in the secondary use of health data and the development of health data warehouses (<xref ref-type="bibr" rid="ref43">43</xref>).</p>
<p>Kalkman et al. have studied the sharing practices for compliance-related documentation (<xref ref-type="bibr" rid="ref44">44</xref>). The authors found that documents like the ones presented in this work is not common.</p>
<p>The OHDSI software stack addressed in the work described in this paper, is expected to play an important role in the upcoming EHDS and is promoted by a range of institutions. For example, the DARWIN initiative - an infrastructure built by the European Medicines Agency (EMA) to enable the secondary use of real-world data - is based on the OMOP-CDM and can be considered one of the first functional parts of the EHDS (<xref ref-type="bibr" rid="ref45">45</xref>). The Joint Action Towards the European Health Data Space (TEHDAS) is another project with significant contributions to the shaping of the EHDS. Recently, also Health Level Seven (HL7) International and OHDSI have started a collaboration to work on a joint common data model for sharing information for healthcare and research (<xref ref-type="bibr" rid="ref46">46</xref>).</p>
</sec>
<sec id="sec21">
<label>4.3</label>
<title>Limitations and future work</title>
<p>One limitation of our work is that it has been designed with European and German requirements in mind, although we aimed at generalizing and abstracting away specifics. We note, however, that there are many similarities between relevant laws and regulations in different parts of the world (<italic>cf.</italic> similarities between the California Consumer Privacy Act or the EU-US Data Privacy Framework and the GDPR). We stress again that our templates must hence be regarded as a starting point and might need adaptions. In future work, we hope to be able to extend and adjust our templates based on feedback from their application in different contexts and jurisdictions.</p>
<p>Another limitation of our work is that we currently did not explicitly include a document template for a Data Protection Impact Assessment (DPIA). Under the GDPR a DPIA is necessary for processing activities resulting in a high risk for the privacy of the data subjects. If an institution decides that this is needed for an OHDSI installation, tools, such as the one presented in (<xref ref-type="bibr" rid="ref47">47</xref>), can be used and information from the documents provided through our work can be reused.</p>
<p>One interested area for future work is to more thoroughly study the compliance of data sharing processes within the OHDSI network. For example, it is not trivial to decide when aggregated statistics can be considered to be anonymous data. The OHDSI collective could be supported by a guideline providing legal and technical assessments of commonly used methods.</p>
</sec>
</sec>
<sec id="sec22">
<label>5</label>
<title>Summary and conclusion</title>
<p>In this paper, we introduced a set of document templates designed to facilitate the implementation and operation of an OHDSI software stack for generating real-world evidence in compliance with data protection and information security requirements. These templates, tailored for typical OHDSI deployments, include crucial documents, such as a Record of Processing Activities, an Information Security Concept, and an Operational Concept. Our work addresses a significant gap by providing a framework adaptable to different institutional and legal requirements, thereby simplifying compliance processes for OHDSI deployments. Despite being primarily oriented towards European and German regulations, our templates can serve as an adaptable starting point for organizations worldwide. Future efforts will focus on refining these templates based on feedback received and extending their scope to further compliance aspects.</p>
</sec>
<sec sec-type="data-availability" id="sec23">
<title>Data availability statement</title>
<p>The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: The templates created can be found in the associated GitHub repository: <ext-link xlink:href="https://github.com/BIH-MI/ohdsi-compliance" ext-link-type="uri">https://github.com/BIH-MI/ohdsi-compliance</ext-link>.</p>
</sec>
<sec sec-type="author-contributions" id="sec24">
<title>Author contributions</title>
<p>FW: Conceptualization, Resources, Writing &#x2013; original draft, Writing &#x2013; review &#x0026; editing. HA: Resources, Writing &#x2013; original draft, Writing &#x2013; review &#x0026; editing. FP: Conceptualization, Resources, Writing &#x2013; original draft, Writing &#x2013; review &#x0026; editing.</p>
</sec>
</body>
<back>
<sec sec-type="funding-information" id="sec25">
<title>Funding</title>
<p>The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work has been partly funded by the German Federal Ministry of Education and Research under grant number 01ZZ2316B (PrivateAIM).</p>
</sec>
<sec sec-type="COI-statement" id="sec26">
<title>Conflict of interest</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec sec-type="disclaimer" id="sec27">
<title>Publisher&#x2019;s note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
<ref-list>
<title>References</title>
<ref id="ref1"><label>1.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sherman</surname> <given-names>RE</given-names></name> <name><surname>Anderson</surname> <given-names>SA</given-names></name> <name><surname>Dal Pan</surname> <given-names>GJ</given-names></name> <name><surname>Gray</surname> <given-names>GW</given-names></name> <name><surname>Gross</surname> <given-names>T</given-names></name> <name><surname>Hunter</surname> <given-names>NL</given-names></name> <etal/></person-group>. <article-title>Real-world evidence - what is it and what can it tell us?</article-title> <source>N Engl J Med</source>. (<year>2016</year>) <volume>375</volume>:<fpage>2293</fpage>&#x2013;<lpage>7</lpage>. doi: <pub-id pub-id-type="doi">10.1056/NEJMsb1609216</pub-id>, PMID: <pub-id pub-id-type="pmid">27959688</pub-id></citation></ref>
<ref id="ref2"><label>2.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Coorevits</surname> <given-names>P</given-names></name> <name><surname>Sundgren</surname> <given-names>M</given-names></name> <name><surname>Klein</surname> <given-names>GO</given-names></name> <name><surname>Bahr</surname> <given-names>A</given-names></name> <name><surname>Claerhout</surname> <given-names>B</given-names></name> <name><surname>Daniel</surname> <given-names>C</given-names></name> <etal/></person-group>. <article-title>Electronic health records: new opportunities for clinical research</article-title>. <source>J Intern Med</source>. (<year>2013</year>) <volume>274</volume>:<fpage>547</fpage>&#x2013;<lpage>60</lpage>. doi: <pub-id pub-id-type="doi">10.1111/joim.12119</pub-id></citation></ref>
<ref id="ref3"><label>3.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wirth</surname> <given-names>FN</given-names></name> <name><surname>Meurers</surname> <given-names>T</given-names></name> <name><surname>Johns</surname> <given-names>M</given-names></name> <name><surname>Prasser</surname> <given-names>F</given-names></name></person-group>. <article-title>Privacy-preserving data sharing infrastructures for medical research: systematization and comparison</article-title>. <source>BMC Med Inform Decis Mak</source>. (<year>2021</year>) <volume>21</volume>:<fpage>242</fpage>&#x2013;<lpage>55</lpage>. doi: <pub-id pub-id-type="doi">10.1186/s12911-021-01602-x</pub-id>, PMID: <pub-id pub-id-type="pmid">34384406</pub-id></citation></ref>
<ref id="ref4"><label>4.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>McMurry</surname> <given-names>AJ</given-names></name> <name><surname>Murphy</surname> <given-names>SN</given-names></name> <name><surname>MacFadden</surname> <given-names>D</given-names></name> <name><surname>Weber</surname> <given-names>G</given-names></name> <name><surname>Simons</surname> <given-names>WW</given-names></name> <name><surname>Orechia</surname> <given-names>J</given-names></name> <etal/></person-group>. <article-title>SHRINE: enabling nationally scalable multi-site disease studies</article-title>. <source>PLoS One</source>. (<year>2013</year>) <volume>8</volume>:<fpage>55811</fpage>. doi: <pub-id pub-id-type="doi">10.1371/journal.pone.0055811</pub-id></citation></ref>
<ref id="ref5"><label>5.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gaye</surname> <given-names>A</given-names></name> <name><surname>Marcon</surname> <given-names>Y</given-names></name> <name><surname>Isaeva</surname> <given-names>J</given-names></name> <name><surname>LaFlamme</surname> <given-names>P</given-names></name> <name><surname>Turner</surname> <given-names>A</given-names></name> <name><surname>Jones</surname> <given-names>EM</given-names></name> <etal/></person-group>. <article-title>DataSHIELD: taking the analysis to the data, not the data to the analysis</article-title>. <source>Int J Epidemiol</source>. (<year>2014</year>) <volume>43</volume>:<fpage>1929</fpage>&#x2013;<lpage>44</lpage>. doi: <pub-id pub-id-type="doi">10.1093/ije/dyu188</pub-id>, PMID: <pub-id pub-id-type="pmid">25261970</pub-id></citation></ref>
<ref id="ref6"><label>6.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hripcsak</surname> <given-names>G</given-names></name> <name><surname>Duke</surname> <given-names>JD</given-names></name> <name><surname>Shah</surname> <given-names>NH</given-names></name> <name><surname>Reich</surname> <given-names>CG</given-names></name> <name><surname>Huser</surname> <given-names>V</given-names></name> <name><surname>Schuemie</surname> <given-names>MJ</given-names></name> <etal/></person-group>. <article-title>Observational health data sciences and informatics (OHDSI): opportunities for observational researchers</article-title>. <source>Stud Health Technol Inform</source>. (<year>2015</year>) <volume>216</volume>:<fpage>574</fpage>&#x2013;<lpage>8</lpage>. doi: <pub-id pub-id-type="doi">10.3233/978-1-61499-564-7-574</pub-id> PMID: <pub-id pub-id-type="pmid">26262116</pub-id></citation></ref>
<ref id="ref7"><label>7.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Reich</surname> <given-names>C</given-names></name> <name><surname>Ostropolets</surname> <given-names>A</given-names></name> <name><surname>Ryan</surname> <given-names>P</given-names></name> <name><surname>Rijnbeek</surname> <given-names>P</given-names></name> <name><surname>Schuemie</surname> <given-names>M</given-names></name> <name><surname>Davydov</surname> <given-names>A</given-names></name> <etal/></person-group>. <article-title>OHDSI standardized vocabularies&#x2014;a large-scale centralized reference ontology for international data harmonization</article-title>. <source>J Am Med Inform Assoc</source>. (<year>2024</year>) <volume>31</volume>:<fpage>583</fpage>&#x2013;<lpage>90</lpage>. doi: <pub-id pub-id-type="doi">10.1093/jamia/ocad247</pub-id></citation></ref>
<ref id="ref8"><label>8.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Voss</surname> <given-names>EA</given-names></name> <name><surname>Blacketer</surname> <given-names>C</given-names></name> <name><surname>van Sandijk</surname> <given-names>S</given-names></name> <name><surname>Moinat</surname> <given-names>M</given-names></name> <name><surname>Kallfelz</surname> <given-names>M</given-names></name> <name><surname>van Speybroeck</surname> <given-names>M</given-names></name> <etal/></person-group>. <article-title>European Health Data &#x0026; Evidence Network-learnings from building out a standardized international health data network</article-title>. <source>J Am Med Inform Assoc JAMIA</source>. (<year>2023</year>) <volume>31</volume>:<fpage>209</fpage>&#x2013;<lpage>19</lpage>. doi: <pub-id pub-id-type="doi">10.1093/jamia/ocad214</pub-id>, PMID: <pub-id pub-id-type="pmid">37952118</pub-id></citation></ref>
<ref id="ref9"><label>9.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Shabani</surname> <given-names>M</given-names></name></person-group>. <article-title>Will the European health data space change data sharing rules?</article-title> <source>Science</source>. (<year>2022</year>) <volume>375</volume>:<fpage>1357</fpage>&#x2013;<lpage>9</lpage>. doi: <pub-id pub-id-type="doi">10.1126/science.abn4874</pub-id>, PMID: <pub-id pub-id-type="pmid">35324305</pub-id></citation></ref>
<ref id="ref10"><label>10.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Welten</surname> <given-names>S</given-names></name> <name><surname>Weber</surname> <given-names>S</given-names></name> <name><surname>Holt</surname> <given-names>A</given-names></name> <name><surname>Beyan</surname> <given-names>O</given-names></name> <name><surname>Decker</surname> <given-names>S</given-names></name></person-group>. <article-title>Will it run?-a proof of concept for smoke testing decentralized data analytics experiments</article-title>. <source>Front Med</source>. (<year>2023</year>) <volume>10</volume>:<fpage>1305415</fpage>. doi: <pub-id pub-id-type="doi">10.3389/fmed.2023.1305415</pub-id></citation></ref>
<ref id="ref11"><label>11.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Vis</surname> <given-names>DJ</given-names></name> <name><surname>Lewin</surname> <given-names>J</given-names></name> <name><surname>Liao</surname> <given-names>RG</given-names></name> <name><surname>Mao</surname> <given-names>M</given-names></name> <name><surname>Andre</surname> <given-names>F</given-names></name> <name><surname>Ward</surname> <given-names>RL</given-names></name> <etal/></person-group>. <article-title>Towards a global cancer knowledge network: dissecting the current international cancer genomic sequencing landscape</article-title>. <source>Ann Oncol</source>. (<year>2017</year>) <volume>28</volume>:<fpage>1145</fpage>&#x2013;<lpage>51</lpage>. doi: <pub-id pub-id-type="doi">10.1093/annonc/mdx037</pub-id>, PMID: <pub-id pub-id-type="pmid">28453708</pub-id></citation></ref>
<ref id="ref12"><label>12.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Khalil</surname> <given-names>R</given-names></name> <name><surname>Macdonald</surname> <given-names>JC</given-names></name> <name><surname>Gustafson</surname> <given-names>A</given-names></name> <name><surname>Aljuburi</surname> <given-names>L</given-names></name> <name><surname>Bisordi</surname> <given-names>F</given-names></name> <name><surname>Beakes-Read</surname> <given-names>G</given-names></name></person-group>. <article-title>Walking the talk in digital transformation of regulatory review</article-title>. <source>Front Med</source>. (<year>2023</year>) <volume>10</volume>:<fpage>1233142</fpage>. doi: <pub-id pub-id-type="doi">10.3389/fmed.2023.1233142</pub-id>, PMID: <pub-id pub-id-type="pmid">37564043</pub-id></citation></ref>
<ref id="ref13"><label>13.</label> <citation citation-type="journal"><person-group person-group-type="author"><collab id="coll1">Act Accountability</collab></person-group>. <article-title>Health insurance portability and accountability act of 1996</article-title>. <source>Public Law</source>. (<year>1996</year>) <volume>104</volume>:<fpage>191</fpage>.</citation></ref>
<ref id="ref14"><label>14.</label> <citation citation-type="journal"><person-group person-group-type="author"><collab id="coll2">Regulation Protection</collab></person-group>. <article-title>Regulation (EU) 2016/679 of the European Parliament and of the council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46</article-title>. <source>Off J Eur Union OJ</source>. (<year>2016</year>) <volume>59</volume>:<fpage>294</fpage>.</citation></ref>
<ref id="ref15"><label>15.</label> <citation citation-type="other"><person-group person-group-type="author"><collab id="coll3">UK GDPR</collab></person-group>. (<year>2020</year>). Available at: <ext-link xlink:href="https://www.legislation.gov.uk/eur/2016/679/contents" ext-link-type="uri">https://www.legislation.gov.uk/eur/2016/679/contents</ext-link></citation></ref>
<ref id="ref16"><label>16.</label> <citation citation-type="other"><person-group person-group-type="author"><collab id="coll4">Data Protection Act</collab></person-group> (<year>2018</year>) Available at: <ext-link xlink:href="https://www.legislation.gov.uk/ukpga/2018/12/contents" ext-link-type="uri">https://www.legislation.gov.uk/ukpga/2018/12/contents</ext-link></citation></ref>
<ref id="ref17"><label>17.</label> <citation citation-type="other"><person-group person-group-type="author"><collab id="coll5">Privacy Act</collab></person-group>. (<year>1988</year>). Available at: <ext-link xlink:href="https://www.legislation.gov.au/Details/C2022C00135" ext-link-type="uri">https://www.legislation.gov.au/Details/C2022C00135</ext-link></citation></ref>
<ref id="ref18"><label>18.</label> <citation citation-type="other"><person-group person-group-type="author"><collab id="coll6">OneTrust DataGuidance</collab></person-group>. Comparing privacy laws: GDPR v. Thai Personal Data Protection Act [Internet]. (<year>2024</year>). Available at: <ext-link xlink:href="https://www.dataguidance.com/sites/default/files/gdpr_v_thailand_updated.pdf" ext-link-type="uri">https://www.dataguidance.com/sites/default/files/gdpr_v_thailand_updated.pdf</ext-link></citation></ref>
<ref id="ref19"><label>19.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Azmi</surname> <given-names>R</given-names></name> <name><surname>Tibben</surname> <given-names>W</given-names></name> <name><surname>Win</surname> <given-names>KT</given-names></name></person-group>. <article-title>Review of cybersecurity frameworks: context and shared concepts</article-title>. <source>J Cyber Policy</source>. (<year>2018</year>) <volume>3</volume>:<fpage>258</fpage>&#x2013;<lpage>83</lpage>. doi: <pub-id pub-id-type="doi">10.1080/23738871.2018.1520271</pub-id></citation></ref>
<ref id="ref20"><label>20.</label> <citation citation-type="book"><person-group person-group-type="author"><collab id="coll7">ISO/IEC</collab></person-group>. <source>Information technology - security techniques - information security management systems - requirements (ISO/IEC 27001:2022)</source>. <publisher-loc>Geneva</publisher-loc>: <publisher-name>International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC)</publisher-name> (<year>2022</year>).</citation></ref>
<ref id="ref21"><label>21.</label> <citation citation-type="book"><person-group person-group-type="author"><name><surname>Barrett</surname> <given-names>MP</given-names></name></person-group>. <source>Framework for improving critical infrastructure cybersecurity</source>. <publisher-loc>Gaithersburg</publisher-loc>: <publisher-name>National Institute of Standards and Technology</publisher-name> (<year>2018</year>).</citation></ref>
<ref id="ref22"><label>22.</label> <citation citation-type="book"><person-group person-group-type="author"><collab id="coll8">HITRUST Alliance</collab></person-group>. <source>HITRUST common security framework (CSF version 9.0) [Internet]</source>. <publisher-loc>Frisco</publisher-loc>: <publisher-name>HITRUST Alliance</publisher-name> (<year>2021</year>).</citation></ref>
<ref id="ref23"><label>23.</label> <citation citation-type="book"><person-group person-group-type="author"><name><surname>Dierks</surname> <given-names>C</given-names></name> <name><surname>Kircher</surname> <given-names>P</given-names></name> <name><surname>Husemann</surname> <given-names>C</given-names></name> <name><surname>Kleinschmidt</surname> <given-names>J</given-names></name> <name><surname>Haase</surname> <given-names>M</given-names></name></person-group>. <source>Data privacy in european medical research: A contemporary legal opinion</source>. <publisher-loc>Berlin</publisher-loc>: <publisher-name>MWV Medizinisch Wissenschaftliche Verlagsgesellschaft</publisher-name> (<year>2021</year>).</citation></ref>
<ref id="ref24"><label>24.</label> <citation citation-type="other"><person-group person-group-type="author"><collab id="coll9">International Association of Privacy Professional</collab></person-group>. Measuring privacy operations [Internet] (<year>2019</year>). Available at: <ext-link xlink:href="https://iapp.org/media/pdf/resource_center/measuring_privacy_operations_2019.pdf" ext-link-type="uri">https://iapp.org/media/pdf/resource_center/measuring_privacy_operations_2019.pdf</ext-link></citation></ref>
<ref id="ref25"><label>25.</label> <citation citation-type="book"><person-group person-group-type="author"><name><surname>Becher</surname> <given-names>SI</given-names></name> <name><surname>Benoliel</surname> <given-names>U</given-names></name></person-group>. <article-title>Law in books and law in action: the readability of privacy policies and the GDPR</article-title> In: <person-group person-group-type="editor"><name><surname>Mathis</surname> <given-names>K</given-names></name> <name><surname>Tor</surname> <given-names>A</given-names></name></person-group>, editors. <source>Consumer law and economics</source>. <publisher-loc>Berlin</publisher-loc>: <publisher-name>Springer</publisher-name> (<year>2021</year>). <fpage>179</fpage>&#x2013;<lpage>204</lpage>. doi: <pub-id pub-id-type="doi">10.1007/978-3-030-49028-7_9</pub-id></citation></ref>
<ref id="ref26"><label>26.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>McDonald</surname> <given-names>AM</given-names></name> <name><surname>Cranor</surname> <given-names>LF</given-names></name></person-group>. <article-title>The cost of reading privacy policies</article-title>. <source>J Law Policy Inf Soc</source>. (<year>2008</year>) <volume>4</volume>:<fpage>543</fpage>&#x2013;<lpage>68</lpage>.</citation></ref>
<ref id="ref27"><label>27.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Benoliel</surname> <given-names>U</given-names></name> <name><surname>Becher</surname> <given-names>S</given-names></name></person-group>. <article-title>The duty to read the unreadable</article-title>. <source>Boston Coll Law Rev</source>. (<year>2019</year>) <volume>60</volume>:<fpage>2255</fpage>&#x2013;<lpage>96</lpage>. doi: <pub-id pub-id-type="doi">10.2139/ssrn.3313837</pub-id></citation></ref>
<ref id="ref28"><label>28.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Overhage</surname> <given-names>JM</given-names></name> <name><surname>Ryan</surname> <given-names>PB</given-names></name> <name><surname>Reich</surname> <given-names>CG</given-names></name> <name><surname>Hartzema</surname> <given-names>AG</given-names></name> <name><surname>Stang</surname> <given-names>PE</given-names></name></person-group>. <article-title>Validation of a common data model for active safety surveillance research</article-title>. <source>J Am Med Inform Assoc JAMIA.</source> (<year>2012</year>) <volume>19</volume>:<fpage>54</fpage>&#x2013;<lpage>60</lpage>. doi: <pub-id pub-id-type="doi">10.1136/amiajnl-2011-000376</pub-id>, PMID: <pub-id pub-id-type="pmid">22037893</pub-id></citation></ref>
<ref id="ref29"><label>29.</label> <citation citation-type="book"><person-group person-group-type="author"><name><surname>Obe</surname> <given-names>RO</given-names></name> <name><surname>Hsu</surname> <given-names>LS</given-names></name></person-group>. <source>PostgreSQL: Up and running: A practical guide to the advanced open source database</source>. <edition>3rd</edition> ed. <publisher-loc>Beijing, Boston, Farnham, Sebastopol, Tokyo</publisher-loc>: <publisher-name>O&#x2019;Reilly</publisher-name> (<year>2018</year>). <fpage>294</fpage> p.</citation></ref>
<ref id="ref30"><label>30.</label> <citation citation-type="other"><person-group person-group-type="author"><collab id="coll10">OHDSI</collab></person-group>. Software Tools (<year>2024</year>). Available at: <ext-link xlink:href="https://www.ohdsi.org/software-tools/" ext-link-type="uri">https://www.ohdsi.org/software-tools/</ext-link></citation></ref>
<ref id="ref31"><label>31.</label> <citation citation-type="other"><person-group person-group-type="author"><name><surname>Wirth</surname> <given-names>FN</given-names></name> <name><surname>Abu Attieh</surname> <given-names>H</given-names></name> <name><surname>Prasser</surname> <given-names>F</given-names></name></person-group>. OHDSI compliance. (<year>2024</year>) Available at: <ext-link xlink:href="https://github.com/BIH-MI/ohdsi-compliance" ext-link-type="uri">https://github.com/BIH-MI/ohdsi-compliance</ext-link></citation></ref>
<ref id="ref32"><label>32.</label> <citation citation-type="book"><person-group person-group-type="author"><collab id="coll11">ISO/IEC</collab></person-group>. <source>Information technology - security techniques - information security management systems - overview and vocabulary (ISO/IEC 27000:2020)</source>. <publisher-loc>Geneva</publisher-loc>: <publisher-name>International Organization for Standardization (ISO) and international Electrotechnical commission (IEC)</publisher-name> (<year>2020</year>).</citation></ref>
<ref id="ref33"><label>33.</label> <citation citation-type="book"><person-group person-group-type="author"><name><surname>Pommerening</surname> <given-names>K</given-names></name> <name><surname>Drepper</surname> <given-names>J</given-names></name> <name><surname>Helbing</surname> <given-names>K</given-names></name> <name><surname>Ganslandt</surname> <given-names>T</given-names></name></person-group>. <source>Leitfaden zum Datenschutz in medizinischen Forschungsprojekten: Generische L&#x00F6;sungen der TMF 2.0</source>. <publisher-loc>Berlin</publisher-loc>: <publisher-name>MWV Medizinisch Wissenschaftliche Verlagsgesellschaft</publisher-name> (<year>2014</year>).</citation></ref>
<ref id="ref34"><label>34.</label> <citation citation-type="other"><person-group person-group-type="author"><collab id="coll12">National Institutes of Health</collab></person-group>. Protocol template for secondary research with biospecimens, data and/or medical records only [Internet]. Available at: <ext-link xlink:href="https://ohsrp.nih.gov/confluence/download/attachments/67273200/Secondary%20Research%20Protocol%20Template.docx?api=v2" ext-link-type="uri">https://ohsrp.nih.gov/confluence/download/attachments/67273200/Secondary%20Research%20Protocol%20Template.docx?api=v2</ext-link></citation></ref>
<ref id="ref35"><label>35.</label> <citation citation-type="other"><person-group person-group-type="author"><collab id="coll13">Center for Internet Security</collab></person-group>. NIST Cybersecurity Framework Policy Template Guide [Internet]. Available at: <ext-link xlink:href="https://www.cisecurity.org/-/jssmedia/Project/cisecurity/cisecurity/data/media/files/uploads/2021/11/NIST-Cybersecurity-Framework-Policy-Template-Guide-v2111Online.pdf" ext-link-type="uri">https://www.cisecurity.org/-/jssmedia/Project/cisecurity/cisecurity/data/media/files/uploads/2021/11/NIST-Cybersecurity-Framework-Policy-Template-Guide-v2111Online.pdf</ext-link></citation></ref>
<ref id="ref36"><label>36.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Frielitz</surname> <given-names>F</given-names></name> <name><surname>Storm</surname> <given-names>N</given-names></name> <name><surname>Hiort</surname> <given-names>O</given-names></name> <name><surname>Katalinic</surname> <given-names>A</given-names></name> <name><surname>von Sengbusch</surname> <given-names>S</given-names></name></person-group>. <article-title>Die Erstellung eines Datenschutzkonzeptes: eine Anleitung f&#x00FC;r telemedizinische Versorgungsprojekte</article-title>. <source>Bundesgesundheitsblatt</source>. (<year>2019</year>) <volume>62</volume>:<fpage>479</fpage>:<fpage>485</fpage>. doi: <pub-id pub-id-type="doi">10.1007/s00103-019-02918-w</pub-id></citation></ref>
<ref id="ref37"><label>37.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wallace</surname> <given-names>SE</given-names></name> <name><surname>Gaye</surname> <given-names>A</given-names></name> <name><surname>Shoush</surname> <given-names>O</given-names></name> <name><surname>Burton</surname> <given-names>PR</given-names></name></person-group>. <article-title>Protecting personal data in epidemiological research: DataSHIELD and UK law</article-title>. <source>Public Health Genomics</source>. (<year>2014</year>) <volume>17</volume>:<fpage>149</fpage>&#x2013;<lpage>57</lpage>. doi: <pub-id pub-id-type="doi">10.1159/000360255</pub-id>, PMID: <pub-id pub-id-type="pmid">24685519</pub-id></citation></ref>
<ref id="ref38"><label>38.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Budin-Lj&#x00F8;sne</surname> <given-names>I</given-names></name> <name><surname>Burton</surname> <given-names>P</given-names></name> <name><surname>Isaeva</surname> <given-names>J</given-names></name> <name><surname>Gaye</surname> <given-names>A</given-names></name> <name><surname>Turner</surname> <given-names>A</given-names></name> <name><surname>Murtagh</surname> <given-names>MJ</given-names></name> <etal/></person-group>. <article-title>DataSHIELD: an ethically robust solution to multiple-site individual-level data analysis</article-title>. <source>Public Health Genomics</source>. (<year>2015</year>) <volume>18</volume>:<fpage>87</fpage>&#x2013;<lpage>96</lpage>. doi: <pub-id pub-id-type="doi">10.1159/000368959</pub-id></citation></ref>
<ref id="ref39"><label>39.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Holmes</surname> <given-names>JH</given-names></name> <name><surname>Elliott</surname> <given-names>TE</given-names></name> <name><surname>Brown</surname> <given-names>JS</given-names></name> <name><surname>Raebel</surname> <given-names>MA</given-names></name> <name><surname>Davidson</surname> <given-names>A</given-names></name> <name><surname>Nelson</surname> <given-names>AF</given-names></name> <etal/></person-group>. <article-title>Clinical research data warehouse governance for distributed research networks in the USA: a systematic review of the literature</article-title>. <source>J Am Med Inform Assoc JAMIA</source>. (<year>2014</year>) <volume>21</volume>:<fpage>730</fpage>&#x2013;<lpage>6</lpage>. doi: <pub-id pub-id-type="doi">10.1136/amiajnl-2013-002370</pub-id>, PMID: <pub-id pub-id-type="pmid">24682495</pub-id></citation></ref>
<ref id="ref40"><label>40.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Pavlenko</surname> <given-names>E</given-names></name> <name><surname>Strech</surname> <given-names>D</given-names></name> <name><surname>Langhof</surname> <given-names>H</given-names></name></person-group>. <article-title>Implementation of data access and use procedures in clinical data warehouses. A systematic review of literature and publicly available policies</article-title>. <source>BMC Med Inform Decis Mak</source>. (<year>2020</year>) <volume>20</volume>:<fpage>157</fpage>. doi: <pub-id pub-id-type="doi">10.1186/s12911-020-01177-z</pub-id></citation></ref>
<ref id="ref41"><label>41.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wang</surname> <given-names>S</given-names></name> <name><surname>Bonomi</surname> <given-names>L</given-names></name> <name><surname>Dai</surname> <given-names>W</given-names></name> <name><surname>Chen</surname> <given-names>F</given-names></name> <name><surname>Cheung</surname> <given-names>C</given-names></name> <name><surname>Bloss</surname> <given-names>CS</given-names></name> <etal/></person-group>. <article-title>Big data privacy in biomedical research</article-title>. <source>IEEE Trans Big Data</source>. (<year>2016</year>) <volume>6</volume>:<fpage>296</fpage>&#x2013;<lpage>308</lpage>. doi: <pub-id pub-id-type="doi">10.1109/TBDATA.2016.2608848</pub-id>, PMID: <pub-id pub-id-type="pmid">32478127</pub-id></citation></ref>
<ref id="ref42"><label>42.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Arellano</surname> <given-names>AM</given-names></name> <name><surname>Dai</surname> <given-names>W</given-names></name> <name><surname>Wang</surname> <given-names>S</given-names></name> <name><surname>Jiang</surname> <given-names>X</given-names></name> <name><surname>Ohno-Machado</surname> <given-names>L</given-names></name></person-group>. <article-title>Privacy policy and technology in biomedical data science</article-title>. <source>Annu Rev Biomed Data Sci</source>. (<year>2018</year>) <volume>1</volume>:<fpage>115</fpage>&#x2013;<lpage>29</lpage>. doi: <pub-id pub-id-type="doi">10.1146/annurev-biodatasci-080917-013416</pub-id>, PMID: <pub-id pub-id-type="pmid">31058261</pub-id></citation></ref>
<ref id="ref43"><label>43.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lamas</surname> <given-names>E</given-names></name> <name><surname>Barh</surname> <given-names>A</given-names></name> <name><surname>Brown</surname> <given-names>D</given-names></name> <name><surname>Jaulent</surname> <given-names>MC</given-names></name></person-group>. <article-title>Ethical, legal and social issues related to the health data-warehouses: re-using health data in the research and public health research</article-title>. <source>Stud Health Technol Inform</source>. (<year>2015</year>) <volume>210</volume>:<fpage>719</fpage>&#x2013;<lpage>23</lpage>. doi: <pub-id pub-id-type="doi">10.3233/978-1-61499-512-8-719</pub-id> PMID: <pub-id pub-id-type="pmid">25991247</pub-id></citation></ref>
<ref id="ref44"><label>44.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kalkman</surname> <given-names>S</given-names></name> <name><surname>Mostert</surname> <given-names>M</given-names></name> <name><surname>Udo-Beauvisage</surname> <given-names>N</given-names></name> <name><surname>van Delden</surname> <given-names>JJ</given-names></name> <name><surname>van Thiel</surname> <given-names>GJ</given-names></name></person-group>. <article-title>Responsible data sharing in a big data-driven translational research platform: lessons learned</article-title>. <source>BMC Med Inform Decis Mak</source>. (<year>2019</year>) <volume>19</volume>:<fpage>283</fpage>. doi: <pub-id pub-id-type="doi">10.1186/s12911-019-1001-y</pub-id>, PMID: <pub-id pub-id-type="pmid">31888593</pub-id></citation></ref>
<ref id="ref45"><label>45.</label> <citation citation-type="journal"><person-group person-group-type="author"><name><surname>Arlett</surname> <given-names>P</given-names></name> <name><surname>Kj&#x00E6;r</surname> <given-names>J</given-names></name> <name><surname>Broich</surname> <given-names>K</given-names></name> <name><surname>Cooke</surname> <given-names>E</given-names></name></person-group>. <article-title>Real-world evidence in EU medicines regulation: enabling use and establishing value</article-title>. <source>Clin Pharmacol Ther</source>. (<year>2022</year>) <volume>111</volume>:<fpage>21</fpage>&#x2013;<lpage>3</lpage>. doi: <pub-id pub-id-type="doi">10.1002/cpt.2479</pub-id>, PMID: <pub-id pub-id-type="pmid">34797920</pub-id></citation></ref>
<ref id="ref46"><label>46.</label> <citation citation-type="other"><person-group person-group-type="author"><collab id="coll14">OHDSI</collab></person-group>. HL7 International and OHDSI announce collaboration to provide single common data model for sharing information in clinical care and observational research (<year>2024</year>). Available at: <ext-link xlink:href="https://www.ohdsi.org/ohdsi-hl7-collaboration/" ext-link-type="uri">https://www.ohdsi.org/ohdsi-hl7-collaboration/</ext-link></citation></ref>
<ref id="ref47"><label>47.</label> <citation citation-type="other"><person-group person-group-type="author"><collab id="coll15">CNIL</collab></person-group> The open source PIA software helps to carry out data protection impact assessment (<year>2023</year>). Available at: <ext-link xlink:href="https://www.cnil.fr/en/open-source-pia-software-helps-carry-out-data-protection-impact-assessment" ext-link-type="uri">https://www.cnil.fr/en/open-source-pia-software-helps-carry-out-data-protection-impact-assessment</ext-link></citation></ref>
</ref-list>
</back>
</article>