<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Microbiol.</journal-id>
<journal-title>Frontiers in Microbiology</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Microbiol.</abbrev-journal-title>
<issn pub-type="epub">1664-302X</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fmicb.2017.01443</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Microbiology</subject>
<subj-group>
<subject>Original Research</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Quantification by qPCR of Pathobionts in Chronic Periodontitis: Development of Predictive Models of Disease Severity at Site-Specific Level</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<name><surname>Tom&#x00E1;s</surname> <given-names>Inmaculada</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="author-notes" rid="fn001"><sup>&#x002A;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/193301/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Regueira-Iglesias</surname> <given-names>Alba</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/460292/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>L&#x00F3;pez</surname> <given-names>Maria</given-names></name>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/392900/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Arias-Bujanda</surname> <given-names>Nora</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/425459/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Novoa</surname> <given-names>Lourdes</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
</contrib>
<contrib contrib-type="author">
<name><surname>Balsa-Castro</surname> <given-names>Carlos</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/457008/overview"/>
</contrib>
<contrib contrib-type="author" corresp="yes">
<name><surname>Tom&#x00E1;s</surname> <given-names>Maria</given-names></name>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
<xref ref-type="author-notes" rid="fn001"><sup>&#x002A;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/389159/overview"/>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Oral Sciences Research Group, Department of Surgery and Medical Surgical Specialties, School of Medicine and Dentistry, Universidade de Santiago de Compostela, Health Research Institute of Santiago (IDIS)</institution> <country>Santiago de Compostela, Spain</country></aff>
<aff id="aff2"><sup>2</sup><institution>Department of Microbiology, Complejo Hospitalario Universitario A Coru&#x00F1;a-Instituto de Investigaci&#x00F3;n Biom&#x00E9;dica de A Coru&#x00F1;a</institution> <country>A Coru&#x00F1;a, Spain</country></aff>
<author-notes>
<fn fn-type="edited-by"><p>Edited by: <italic>Fumito Maruyama, Kyoto University, Japan</italic></p></fn>
<fn fn-type="edited-by"><p>Reviewed by: <italic>Takahiko Shiba, Tokyo Medical and Dental University, Japan; Hidetomo Iwano, Rakuno Gakuen University, Japan; Ching-Yuan Chen, Columbia University Medical Center, United States</italic></p></fn>
<fn fn-type="corresp" id="fn001"><p>&#x002A;Correspondence: <italic>Inmaculada Tom&#x00E1;s, <email>inmaculada.tomas@usc.es</email> Maria Tom&#x00E1;s, <email>ma.del.mar.tomas.carmona@sergas.es</email></italic></p></fn>
<fn fn-type="other" id="fn002"><p>This article was submitted to Infectious Diseases, a section of the journal Frontiers in Microbiology</p></fn></author-notes>
<pub-date pub-type="epub">
<day>09</day>
<month>08</month>
<year>2017</year>
</pub-date>
<pub-date pub-type="collection">
<year>2017</year>
</pub-date>
<volume>08</volume>
<elocation-id>1443</elocation-id>
<history>
<date date-type="received">
<day>22</day>
<month>03</month>
<year>2017</year>
</date>
<date date-type="accepted">
<day>17</day>
<month>07</month>
<year>2017</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x00A9; 2017 Tom&#x00E1;s, Regueira-Iglesias, L&#x00F3;pez, Arias-Bujanda, Novoa, Balsa-Castro and Tom&#x00E1;s.</copyright-statement>
<copyright-year>2017</copyright-year>
<copyright-holder>Tom&#x00E1;s, Regueira-Iglesias, L&#x00F3;pez, Arias-Bujanda, Novoa, Balsa-Castro and Tom&#x00E1;s</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p></license>
</permissions>
<abstract>
<p>Currently, there is little evidence available on the development of predictive models for the diagnosis or prognosis of chronic periodontitis based on the qPCR quantification of subgingival pathobionts. Our objectives were to: (1) analyze and internally validate pathobiont-based models that could be used to distinguish different periodontal conditions at site-specific level within the same patient with chronic periodontitis; (2) develop nomograms derived from predictive models. Subgingival plaque samples were obtained from control and periodontal sites (probing pocket depth and clinical attachment loss &#x003C;4 mm and >4 mm, respectively) from 40 patients with moderate-severe generalized chronic periodontitis. The samples were analyzed by qPCR using TaqMan probes and specific primers to determine the concentrations of <italic>Actinobacillus actinomycetemcomitans (Aa)</italic>, <italic>Fusobacterium nucleatum (Fn)</italic>, <italic>Parvimonas micra (Pm)</italic>, <italic>Porphyromonas gingivalis (Pg)</italic>, <italic>Prevotella intermedia (Pi)</italic>, <italic>Tannerella forsythia (Tf)</italic>, and <italic>Treponema denticola (Td)</italic>. The pathobiont-based models were obtained using multivariate binary logistic regression. The best models were selected according to specified criteria. The discrimination was assessed using receiver operating characteristic curves and numerous classification measures were thus obtained. The nomograms were built based on the best predictive models. Eight bacterial cluster-based models showed an area under the curve (AUC) &#x2265;0.760 and a sensitivity and specificity &#x2265;75.0%. The <italic>PiTfFn</italic> cluster showed an AUC of 0.773 (sensitivity and specificity = 75.0%). When <italic>Pm</italic> and <italic>AaPm</italic> were incorporated in the <italic>TdPiTfFn</italic> cluster, we detected the two best predictive models with an AUC of 0.788 and 0.789, respectively (sensitivity and specificity = 77.5%). The <italic>TdPiTfAa</italic> cluster had an AUC of 0.785 (sensitivity and specificity = 75.0%). When <italic>Pm</italic> was incorporated in this cluster, a new predictive model appeared with better AUC and specificity values (0.787 and 80.0%, respectively). Distinct clusters formed by species with different etiopathogenic role (belonging to different Socransky&#x2019;s complexes) had a good predictive accuracy for distinguishing a site with periodontal destruction in a periodontal patient. The predictive clusters with the lowest number of bacteria were <italic>PiTfFn</italic> and <italic>TdPiTfAa</italic>, while <italic>TdPiTfAaFnPm</italic> had the highest number. In all the developed nomograms, high concentrations of these clusters were associated with an increased probability of having a periodontal site in a patient with chronic periodontitis.</p>
</abstract>
<kwd-group>
<kwd>chronic periodontitis</kwd>
<kwd>multivariate modeling techniques</kwd>
<kwd>paired design</kwd>
<kwd>periopathogens</kwd>
<kwd>predictive ability</kwd>
<kwd>qPCR</kwd>
<kwd>site-specific</kwd>
<kwd>subgingival plaque</kwd>
</kwd-group>
<counts>
<fig-count count="6"/>
<table-count count="5"/>
<equation-count count="0"/>
<ref-count count="87"/>
<page-count count="16"/>
<word-count count="0"/>
</counts>
</article-meta>
</front>
<body>
<sec><title>Introduction</title>
<p>Periodontal diseases are among the most common conditions affecting human beings (<xref ref-type="bibr" rid="B29">Dentino et al., 2013</xref>). In 2010, severe periodontitis was estimated to be the sixth most prevalent disease globally, affecting 743 million people worldwide and with an age-standardized incidence of 701 cases per 100,000 person-years (<xref ref-type="bibr" rid="B46">Kassebaum et al., 2014</xref>). Periodontitis is an inflammatory condition of the gingivae. It causes the destruction of the ligament and alveolar bone supporting the teeth, resulting in oral malodour and tooth loss (<xref ref-type="bibr" rid="B29">Dentino et al., 2013</xref>). In addition to its impact on the oral health status and quality of life of patients (<xref ref-type="bibr" rid="B73">Shanbhag et al., 2012</xref>; <xref ref-type="bibr" rid="B2">Al-Harthi et al., 2013</xref>), periodontitis is currently being connected bidirectionally to the pathogenesis of various systemic diseases and conditions such as diabetes (<xref ref-type="bibr" rid="B23">Chapple et al., 2013</xref>), coronary heart disease (<xref ref-type="bibr" rid="B80">Tonetti et al., 2013</xref>), rheumatoid arthritis (<xref ref-type="bibr" rid="B26">de Pablo et al., 2009</xref>), respiratory diseases (<xref ref-type="bibr" rid="B8">Bansal et al., 2013</xref>), and dementia (<xref ref-type="bibr" rid="B1">Abbayya et al., 2015</xref>).</p>
<p>Although traditional clinical measures are informative for evaluating the severity of periodontitis and the response to therapy (<xref ref-type="bibr" rid="B49">Korte and Kinney, 2016</xref>), these clinical criteria are only partially able to determine current disease activity or the future risk of structure loss (<xref ref-type="bibr" rid="B35">Giannobile et al., 2009</xref>; <xref ref-type="bibr" rid="B86">Zhang et al., 2009</xref>). As a result, one of the major challenges in the field of periodontology is to determine biomarkers for screening and predicting the early onset of periodontitis or evaluating disease activity and the efficacy of therapy (diagnostic or prognostic tests) (<xref ref-type="bibr" rid="B86">Zhang et al., 2009</xref>; <xref ref-type="bibr" rid="B16">Buduneli and Kinane, 2011</xref>).</p>
<p>Periodontal diseases are multifactorial in origin; their initiation and progression require the involvement of several factors, including bacteria that contribute to the formation of a polymicrobial biofilm at the subgingival level. Consequently, periodontal microbiology has been an area of intense research for decades (<xref ref-type="bibr" rid="B78">Teles et al., 2013</xref>).</p>
<p>In order to overcome the well-known limitations of bacterial cultures, a series of molecular techniques were developed, including DNA hybridization and the polymerase chain reaction (PCR) (<xref ref-type="bibr" rid="B74">Socransky et al., 1998</xref>; <xref ref-type="bibr" rid="B55">Loomer, 2004</xref>). <xref ref-type="bibr" rid="B74">Socransky et al. (1998)</xref> used genomic hybridization techniques to identify three species that were strongly associated with the clinical parameters of chronic periodontitis: <italic>Treponema denticola</italic>, <italic>Porphyromonas gingivalis</italic>, and <italic>Tannerella forsythia</italic>. These species together constituted the so-called &#x2018;Red Complex.&#x2019; This complex was related to other bacteria such as <italic>Fusobacterium nucleatum</italic>, <italic>Prevotella intermedia</italic>, <italic>Prevotella nigrescens</italic>, <italic>Peptostreptococcus micros (Parvimonas micra)</italic>, <italic>Campylobacter rectus</italic>, <italic>Campylobacter showae</italic>, <italic>Campylobacter gracilis</italic>, <italic>Eubacterium nodatum</italic>, <italic>Streptococcus constellatus</italic>, and <italic>Fusobacterium periodonticum</italic>. All of these species formed the &#x2018;Orange Complex.&#x2019;</p>
<p>Quantitative PCR, qPCR or real-time PCR are variants of the conventional PCR technique that is used to amplify and simultaneously quantify the amplification product obtained from a sample (<xref ref-type="bibr" rid="B70">Sakamoto et al., 2005</xref>). There are several studies in the scientific literature in which the diagnostic accuracy of the qPCR technique in samples from healthy subjects and periodontitis patients was assessed using the conventional bacterial culture as a reference (<xref ref-type="bibr" rid="B13">Boutaga et al., 2005</xref>, <xref ref-type="bibr" rid="B14">2006</xref>, <xref ref-type="bibr" rid="B12">2007</xref>; <xref ref-type="bibr" rid="B45">Jervoe-Storm et al., 2005</xref>; <xref ref-type="bibr" rid="B7">Atieh, 2008</xref>; <xref ref-type="bibr" rid="B50">Kotsilkov et al., 2015</xref>). The conclusion that can be drawn from these investigations is that the qPCR is a sensitive and specific method which has a high positive predictive value (PPV) when it comes to the detection of target bacteria (<xref ref-type="bibr" rid="B13">Boutaga et al., 2005</xref>, <xref ref-type="bibr" rid="B14">2006</xref>, <xref ref-type="bibr" rid="B12">2007</xref>; <xref ref-type="bibr" rid="B45">Jervoe-Storm et al., 2005</xref>; <xref ref-type="bibr" rid="B7">Atieh, 2008</xref>; <xref ref-type="bibr" rid="B50">Kotsilkov et al., 2015</xref>). In fact, <xref ref-type="bibr" rid="B7">Atieh (2008)</xref> performed a meta-analysis in 2008 in which it was demonstrated that this molecular technique is associated with a high diagnostic accuracy in detecting (specifically) <italic>Aggregatibacter actinomycetemcomitans</italic> and <italic>P. gingivalis</italic> compared to conventional cultures.</p>
<p>Until now, research on periodontal microbiology using qPCR has focused on the study of periopathogens such as: <italic>A. actinomycetemcomitans</italic>, <italic>P. gingivalis</italic>, <italic>T. forsythia</italic>, <italic>P. intermedia</italic>, <italic>T. denticola</italic>, <italic>F. nucleatum</italic>, and <italic>P. micra</italic> (<xref ref-type="bibr" rid="B15">Braga et al., 2010</xref>; <xref ref-type="bibr" rid="B71">Saygun et al., 2011</xref>; <xref ref-type="bibr" rid="B27">Decat et al., 2012</xref>; <xref ref-type="bibr" rid="B65">Polonyi et al., 2013</xref>; <xref ref-type="bibr" rid="B34">Gatto et al., 2014</xref>; <xref ref-type="bibr" rid="B48">Kinney et al., 2014</xref>; <xref ref-type="bibr" rid="B3">Al-hebshi et al., 2015</xref>; <xref ref-type="bibr" rid="B50">Kotsilkov et al., 2015</xref>; <xref ref-type="bibr" rid="B58">Milne et al., 2015</xref>; <xref ref-type="bibr" rid="B72">Scapoli et al., 2015</xref>; <xref ref-type="bibr" rid="B81">Torrungruang et al., 2015</xref>; <xref ref-type="bibr" rid="B37">Golynska et al., 2016</xref>). In most of these q-PCR studies, the authors applied a &#x2018;simple analytical approach&#x2019; consisting of univariate comparisons of the periopathogen levels quantified in different periodontal conditions (<xref ref-type="bibr" rid="B15">Braga et al., 2010</xref>; <xref ref-type="bibr" rid="B65">Polonyi et al., 2013</xref>; <xref ref-type="bibr" rid="B50">Kotsilkov et al., 2015</xref>; <xref ref-type="bibr" rid="B58">Milne et al., 2015</xref>; <xref ref-type="bibr" rid="B72">Scapoli et al., 2015</xref>; <xref ref-type="bibr" rid="B37">Golynska et al., 2016</xref>). As a result, there are very few studies based on a multivariate modeling approach (<xref ref-type="bibr" rid="B71">Saygun et al., 2011</xref>; <xref ref-type="bibr" rid="B34">Gatto et al., 2014</xref>; <xref ref-type="bibr" rid="B48">Kinney et al., 2014</xref>; <xref ref-type="bibr" rid="B3">Al-hebshi et al., 2015</xref>; <xref ref-type="bibr" rid="B81">Torrungruang et al., 2015</xref>).</p>
<p>Consequently, when assuming that chronic periodontitis is orchestrated by a bacterial consortium rather than by a single pathogen (<xref ref-type="bibr" rid="B78">Teles et al., 2013</xref>), further evidence is required on the development of predictive models for diagnosing periodontitis or its prognosis based on the qPCR quantification of subgingival pathobionts using appropriate multivariate analytical techniques. Accordingly, the objectives of the present cross-sectional study (paired design) were to:</p>
<list list-type="simple" prefix-word="simple">
<list-item><label>(1)</label><p> Compare the detection frequencies and levels of seven well-known pathobionts detected in subgingival sites with different periodontal conditions (control site vs. periodontal site) within the same patient with chronic periodontitis.</p></list-item>
<list-item><label>(2)</label><p> Obtain pathobiont-based predictive models that could be used to distinguish different periodontal conditions at site-specific level within the same periodontal patient.</p></list-item>
<list-item><label>(3)</label><p> Develop nomograms derived from pathobiont-based predictive models.</p></list-item></list>
</sec>
<sec id="s1" sec-type="materials|methods">
<title>Materials and Methods</title>
<sec><title>Selection of Study Group and Clinical Examination</title>
<p>A total of 40 subjects affected by moderate to severe generalized chronic periodontitis were recruited among consecutive patients who were referred between 2014 and 2016 to the Periodontology and Patients with Special Needs units at the School of Medicine and Dentistry (Universidade de Santiago de Compostela, Spain) for an assessment of their oral health status. Patients were selected if they fulfilled the following inclusion criteria: (1) age 30 to 65 years; (2) good general health, no pregnancy or breastfeeding; (3) no intake of systemic antimicrobials during the previous 6 months; (4) no intake of antiinflammatory medication in the previous 4 months; (5) no routine use of oral antiseptics; (6) no presence of implants or orthodontic appliances; (7) no previous periodontal treatment; and (8) the presence of at least 18 natural teeth.</p>
<p>Two experienced dentists and previously calibrated performed all the periodontal diagnoses. The probing pocket depth (PPD) and clinical attachment loss (CAL) (=PPD+ gingival recession) were recorded throughout the mouth on six sites per tooth (excluding third molars) using a PCP-UNC 15 probe. Bleeding on probing (BOP) and bacterial plaque levels (BPL) were recorded for the full mouth on a binary scale (presence/absence) on six sites per tooth. Standardized radiographs of all teeth were obtained to assess the alveolar bone status. Patients were diagnosed as suffering from moderate to severe generalized chronic periodontitis based on the previously established criteria (<xref ref-type="bibr" rid="B6">Armitage, 1999</xref>; <xref ref-type="bibr" rid="B64">Page and Eke, 2007</xref>). Smoking status was also recorded (<bold>Figure <xref ref-type="fig" rid="F1">1</xref></bold>).</p>
<fig id="F1" position="float">
<label>FIGURE 1</label>
<caption><p>Selection of study groups, collection of subgingival samples and construction of dataframe.</p></caption>
<graphic xlink:href="fmicb-08-01443-g001.tif"/>
</fig>
<p>Patients who agreed to participate in the study provided their written informed consent. The study protocol was approved by the Ethics Committee of Clinical Investigations of Galicia (registration number 2015/006). The &#x2018;transparent reporting of a multivariable prediction model for individual prognosis or diagnosis&#x2019; (TRIPOD) guidelines were considered (<xref ref-type="bibr" rid="B60">Moons et al., 2015</xref>).</p>
</sec>
<sec><title>Collection of Subgingival Plaque Samples</title>
<p>Control and periodontal sites were defined within the same periodontal patient as follows: (1) the control sites had a PPD and CAL &#x003C;4 mm and no radiographic evidence of alveolar bone loss; and (2) the periodontal sites had a PPD and CAL >4 mm and radiographic evidence of bone loss (<bold>Figure <xref ref-type="fig" rid="F1">1</xref></bold>).</p>
<p>All the subgingival plaque samples were collected in the afternoon, approximately 5&#x2013;7 h after toothbrushing. The samples were obtained from both a control site and a periodontal site (two or three adjacent locations in each site) in each periodontal patient by inserting a total of five sterile endodontic paper points into each location (size 30) for 10 s following isolation and supragingival plaque removal. The samples were placed in 1.5 ml microcentrifuge tubes with 300 &#x03BC;l of a phosphate buffer and frozen at -80&#x00B0;C until further analysis.</p>
</sec>
<sec><title>DNA Extraction and qPCR</title>
<p>Two investigators blinded to the clinical data performed the bacterial analyses by qPCR. The bacteria were separated from the paper points before the DNA extraction by washing with 500 &#x03BC;L of a phosphate buffer solution and vortexing. DNA extraction from the samples was performed in the MagNA Pure Compact Instrument (Roche, Germany) following the manufacturer&#x2019;s instructions. We used it in combination with the MagNA Pure Bacteria Lysis Buffer (Roche) and the DNA Bacteria purification protocol (Roche) to work with the bacterial DNA isolated from subgingival plaque samples.</p>
<p>The levels of seven pathobionts were quantified: <italic>A. actinomycetemcomitans (Aa); F. nucleatum (Fn); P. micra (Pm); P. gingivalis (Pg); P. intermedia (Pi); T. forsythensis (Tf);</italic> and <italic>T. denticola (Td).</italic> Real-time PCR (LightCycler<sup>&#x00AE;</sup>480 system, Roche) was performed using TaqMan probes (Universal Probe Library, Roche) and primers were designed to amplify specific target genes from each bacterium (<bold>Table <xref ref-type="table" rid="T1">1</xref></bold>) (<xref ref-type="bibr" rid="B32">Fern&#x00E1;ndez-Pinero et al., 2013</xref>; <xref ref-type="bibr" rid="B33">Fraczyk et al., 2016</xref>). The specificity of the probes was analyzed among all the bacteria from this study. We worked with uracil-<italic>N</italic>-glycosylase, Cod-UNG (ArcticZymes, Norway) (<xref ref-type="bibr" rid="B11">Bou et al., 2014</xref>) to remove contaminating amplicons from the previous qPCR. To confirm this, we used the water from the reaction of the qPCR as a negative control (<xref ref-type="bibr" rid="B22">Champlot et al., 2010</xref>).</p>
<table-wrap position="float" id="T1">
<label>Table 1</label>
<caption><p>DNA regions amplified, the targeted gene name, TaqMan probes and primers used in the present study for the detection of the seven bacteria analyzed.</p></caption>
<table cellspacing="5" cellpadding="5" frame="hsides" rules="groups">
<thead>
<tr>
<th valign="top" align="left">Bacteria</th>
<th valign="top" align="left">DNA region amplified (bp)</th>
<th valign="top" align="left">Genes (Genbank; Acc. number)</th>
<th valign="top" align="left">UPL probes (Number)</th>
<th valign="top" align="left">Primers (5&#x2032;&#x2013;3&#x2032;)</th>
<th valign="top" align="left">Controls</th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left"><italic>Aa</italic></td>
<td valign="top" align="left">CAATACTACGGTGGTGCAGTATC TGCACCGTTGTTCTCCAGCATTA TGGGTTACGCATTACGCGCCAAC AATAT</td>
<td valign="top" align="left"><italic>ftsI</italic> (gb&#x007C;CP001733.1&#x007C; : 702960&#x2013;704783)</td>
<td valign="top" align="left">CTCCAGCA (number 67)</td>
<td valign="top" align="left">5&#x2032;-CAATACTACGGTGGTGCAGTATCT-3&#x2032; 5&#x2032;-ATATTGTTGGCGCGTAATGC-3&#x2032;</td>
<td valign="top" align="left">ATCC<sup>&#x00AE;</sup> 700685<sup>TM</sup></td>
</tr>
<tr>
<td valign="top" align="left"><italic>Fn</italic></td>
<td valign="top" align="left">TCCCAGCAAATGTTGGAAGAAT TGAATATGCTGAAGAAGAAGAT GAAGACTATGACGAATTTGATG ATGAA</td>
<td valign="top" align="left"><italic>ftsX</italic> (gb&#x007C;AE009951.2&#x007C; : 891544&#x2013;892470)</td>
<td valign="top" align="left">AGAAGAAGA (number 143)</td>
<td valign="top" align="left">5&#x2032;- TCCCAGCAAATGTTGGAAG-3&#x2032; 5&#x2032;-TTCATCATCAAATTCGTCATAGTCT-3&#x2032;</td>
<td valign="top" align="left">ATCC<sup>&#x00AE;</sup> 25611<sup>TM</sup></td>
</tr>
<tr>
<td valign="top" align="left"><italic>Pm</italic></td>
<td valign="top" align="left">AGAATCAATTTCTCAAGGTGCA GAAGCTGAAGGAAAATACATCA GACAGTCTGGTGGTAGTGGACA ATATGGACATTG</td>
<td valign="top" align="left"><italic>fusA</italic> (gb&#x007C;CP009761.1&#x007C; : 310535&#x2013;312610)</td>
<td valign="top" align="left">GCTGAAGG (number 161)</td>
<td valign="top" align="left">5&#x2032;-AGAATCAATTTCTCAAGGTGCAG-3&#x2032; 5&#x2032;-CAATGTCCATATTGTCCACTACCA-3&#x2032;</td>
<td valign="top" align="left">ATCC<sup>&#x00AE;</sup> 33270<sup>TM</sup></td>
</tr>
<tr>
<td valign="top" align="left"><italic>Pg</italic></td>
<td valign="top" align="left">ATAGTAGCGTGTCCGGCTTCGTG GATGGCGATGCTGCGACGCTCCT CCTCTGTGGTGATTTTGTTCTTTT TCTCCAATCCGCCTACGAT</td>
<td valign="top" align="left"><italic>ftsH</italic> (gi&#x007C;188593544&#x007C; : 54171&#x2013;56192)</td>
<td valign="top" align="left">CTCCTCTG (number 82)</td>
<td valign="top" align="left">5&#x2032;-ATAGTAGCGTGTCCGGCTTC-3&#x2032; 5&#x2032;-ATCGTAGGCGGATTGGAGA-3&#x2032;</td>
<td valign="top" align="left">ATCC<sup>&#x00AE;</sup> 33277<sup>TM</sup></td>
</tr>
<tr>
<td valign="top" align="left"><italic>Pi</italic></td>
<td valign="top" align="left">TGGTATCAAAATCAGCAAGGAA ACAACACCAGAGATTTATAAGC TTGTGCTGAATATGCGCGAAGACG</td>
<td valign="top" align="left"><italic>piACP</italic> (gb&#x007C;CP019300.1&#x007C; : 638473&#x2013;639264)</td>
<td valign="top" align="left">ACCAGAGA (number 126)</td>
<td valign="top" align="left">5&#x2032;- TGGTATCAAAATCAGCAAGGAA-3&#x2032; 5&#x2032;- CGTCTTCGCGCATATTCAG-3&#x2032;</td>
<td valign="top" align="left">ATCC<sup>&#x00AE;</sup> 25611<sup>TM</sup></td>
</tr>
<tr>
<td valign="top" align="left"><italic>Tf</italic></td>
<td valign="top" align="left">AAACATCGTGGATACCCTCCTTA TACATATGTGTGACAGCGTTTCC GCCGCCACCACCGACACCGACC ACTTTGAT</td>
<td valign="top" align="left"><italic>ftsZ</italic> (gb&#x007C;CP003191.1&#x007C; : 422801&#x2013;424174)</td>
<td valign="top" align="left">CCGCCGCC (number 70)</td>
<td valign="top" align="left">5&#x2032;-AAACATCGTGGATACCCTCCT-3&#x2032; 5&#x2032;-ATCAAAGTGGTCGGTGTCG-3&#x2032;</td>
<td valign="top" align="left">ATCC<sup>&#x00AE;</sup> 43037<sup>TM</sup></td>
</tr>
<tr>
<td valign="top" align="left"><italic>Td</italic></td>
<td valign="top" align="left">GCAGATATACAGGTAGACATAG GAAGCGCAGCTCCTGAAGAATC CAAAAAAAGCATTATAGAATCC CTTGTCGTAATAGAAGATATAA ACACTGAACAGGACTTAGAAGAGC</td>
<td valign="top" align="left"><italic>ftsK</italic> (gb&#x007C;AE017226.1&#x007C; : 204110&#x2013;206650)</td>
<td valign="top" align="left">AGCTCCTG (number 128)</td>
<td valign="top" align="left">5&#x2032;-CAGATATACAGGTAGACATAGGAAGC-3&#x2032; 5&#x2032;-GCTCTTCTAAGTCCTGTTCAGTGTT-3&#x2032;</td>
<td valign="top" align="left">VPI 4355</td></tr>
</tbody></table>
<table-wrap-foot>
<attrib><italic>Aa, A. actinomycetemcomitans; Fn, F. nucleatum; Pm, P. micra; Pg, P. gingivalis; Pi, P. intermedia; Tf, T. forsythia; Td, T. denticola.</italic></attrib>
</table-wrap-foot>
</table-wrap>
<p>Absolute quantification of the DNA was performed using the standard curve method. A standard dilution series of known amounts of genomic DNA (from 100 pg to 1 fg) were assessed in the same assay. Cp-values obtained from these standards were used to generate a standard curve from which the amount of DNA in the unknown samples was calculated. Each standard contained the same target region that was amplified in the sample. Strains from each microorganism, obtained from the ATCC Bacteriology Collection, were used both to generate the standard curves and as positive controls. A regression analysis was performed to obtain the equation used to interpolate the Cp-values from healthy and diseased sites and thus quantify the corresponding concentrations of genomic DNA of each pathobiont in each sample (pg/&#x03BC;l). The value of <italic>R</italic><sup>2</sup> was used as a measure of the goodness-of-fit of the regression analysis.</p>
</sec>
<sec><title>Statistical Analysis</title>
<p>The unit of analysis in the study was the sampled subgingival site. The &#x2018;<italic>a priori</italic>&#x2019; sample size calculation was performed using the program G<sup>&#x2217;</sup>Power 3.1.5 (<xref ref-type="bibr" rid="B31">Faul et al., 2007</xref>). The following statistical criteria were established: (1) an effect size of 0.5 for bacterial concentrations between the control and periodontal sites; (2) an alpha error of 0.05; and (3) a statistical power of 90%. A sample size of 36 subjects was required on the basis of these criteria and the application of the Wilcoxon test to evaluate the differential bacterial concentrations.</p>
<p>The independent variables were: (1) the quantitative levels of the species individually (7 variables); and (2) due to its biological significance, the quantitative levels of all possible bacterial clusters (120 variables) (<bold>Figure <xref ref-type="fig" rid="F1">1</xref></bold>) (<bold>Supplementary Data Sheet <xref ref-type="supplementary-material" rid="SM1">S1</xref></bold>). The statistical analyses were performed using the R software (<xref ref-type="bibr" rid="B67">R Core Team, 2016</xref>).</p>
<sec><title>Univariate Analysis of the Detection Frequencies and Levels of Pathobionts in Relation to the Severity of Chronic Periodontitis: Control Sites vs. Periodontal Sites</title>
<p>Contingency tables were used and the McNemar test was applied to compare the detection frequencies between the control and periodontal sites. The Shapiro&#x2013;Wilk test was performed to analyze the distribution of the quantitative variables. As the clinical parameters associated with chronic periodontitis, as well as the subgingival concentrations of the pathobionts, showed a non-normal distribution, the contrast between the two types of subgingival site (control vs. periodontal) was determined using the Wilcoxon test.</p>
<p>The Student&#x2019;s <italic>t</italic>-test or Mann&#x2013;Whitney <italic>U</italic> test for quantitative variables (according to the type of distribution of variables) and the Fisher&#x2019;s exact test for categorical variables were used to compare the clinical characteristics between smokers and non-smokers. The influence of smoking (smokers vs. non-smokers) in the detection frequencies and bacterial concentrations for each type of subgingival site were studied using the Fisher&#x2019;s exact test and the Mann&#x2013;Whitney <italic>U</italic> test, respectively.</p>
<p>The Benjamini&#x2013;Hochberg correction was applied to control the false discovery rate (FDR) (<xref ref-type="bibr" rid="B10">Benjamini and Hochberg, 1995</xref>) in the comparative analysis, establishing a Q parameter of 0.05, which corresponds to a FDR &#x003C; 5%. A significance level of <italic>P</italic> &#x003C; 0.05 was applied.</p>
<p>Correlations of bacterial levels with clinical parameters (BOP, PPD, and CAL) were assessed using Spearman&#x2019;s correlation coefficients.</p>
</sec>
<sec><title>Multivariate Predictive Modeling of Chronic Periodontitis at Site-Specific Level Based on Subgingival Pathobiont Levels: Selection of Models</title>
<p>Spearman correlations between pathobionts were calculated and used as an orientation for model building, in order to prevent redundancies and possible collinearity between bacteria with similar biological effects. Pathobiont-based models were selected for their biological significance, their capacity to predict chronic periodontitis at site-specific level and their statistical validity.</p>
<p>Models were constructed by initially selecting one bacterium or bacterial cluster as predictor variable. In order to test whether the predictive ability of bacterial clusters can be increased by incorporating others, two-variable models combining different bacterial clusters were analyzed. A methodological advantage of the matched case-control sample approach within the same periodontal patient used in this study is that the subject-associated variables (such as age, gender, and smoking status) are the same for the control and periodontal sites and so do not need to be incorporated in the predictive models.</p>
<p>The statistical criteria applied for model selection were:</p>
<list list-type="simple" prefix-word="simple">
<list-item><label>(1)</label><p> The capacity of each pathobiont-based model to discriminate the severity of chronic periodontitis at site-specific level, that was assessed with the Epi package and using the receiver operating characteristic (ROC) curve (<xref ref-type="bibr" rid="B20">Carstensen et al., 2017</xref>). The models with an area under the curve (AUC) value &#x2265;0.76 were selected, as these are typically considered to be acceptable predictive models (<xref ref-type="bibr" rid="B43">Hosmer et al., 2013</xref>). The calculation of the AUC values and their corresponding 95% confidence intervals (CIs) by bootstrapping was performed using the pROC package (<xref ref-type="bibr" rid="B69">Robin et al., 2011</xref>).</p></list-item>
<list-item><label>(2)</label><p> The capacity of each pathobiont-based model to classify the severity of chronic periodontitis at site-specific level, that was assessed with the pROC package and bootstrapping. Numerous classification measures such as accuracy (ACC), sensitivity, specificity, the PPV, and the negative predictive value (NPV) were obtained by setting an optimal threshold, as well as their corresponding 95% CIs (<xref ref-type="bibr" rid="B69">Robin et al., 2011</xref>). The best cut-off value for each model was determined so that the percentage of correct predictions was the maximum. The models with sensitivity and specificity values &#x2265;75% were selected.</p></list-item>
<list-item><p> As a single indicator of diagnostic performance, the diagnostic odds ratio (DOR) was calculated as the ratio of the odds of positivity in the diseased patients relative to the odds of positivity in those with no disease (<xref ref-type="bibr" rid="B36">Glas et al., 2003</xref>). The value of the DOR ranges from 0 to infinity, with higher values indicating a better discriminatory test performance (higher accuracy). A DOR of 1.0 indicates that a test does not discriminate between subjects with the disease and those without it. DOR values &#x003C;1.0 suggest improper test interpretation (a proportion of negative test results in the group with disease) (<xref ref-type="bibr" rid="B36">Glas et al., 2003</xref>).</p></list-item>
<list-item><label>(3)</label><p> In models with at least two independent variables, the variance inflation factor (VIF; for models which only include continuous variables) was &#x003C;2.28. This value allowed us to assume that the model did not present evidence of multicollinearity between the independent variables (<xref ref-type="bibr" rid="B63">O&#x2019;brien, 2007</xref>).</p></list-item>
<list-item><label>(4)</label><p> The Hosmer&#x2013;Lemeshow test (a calibration measure) was not significant (<italic>P</italic> > 0.05) when using the Resource Selection package (<xref ref-type="bibr" rid="B53">Lele et al., 2017</xref>).</p></list-item></list>
<p>Any model that did not fulfill any of these inclusion criteria was discarded for the posterior analysis.</p>
</sec>
<sec><title>Multivariate Predictive Modeling of Chronic Periodontitis at Site-Specific Level Based on Subgingival Pathobiont Levels: Validation of Selected Models, and the Development of Nomograms</title>
<p>Bootstrap methods were used to test for possible overfitting by determining optimism values on the discrimination and classification measures. The bootstrap analysis was replicated on 1000 different samples of the same sample size drawn with replacements from the original sample. Optimism, which is a measurement of internal model validation that refers to the absolute magnitude of bias, equals the difference between respective statistics of the bootstrap sample (bootstrap performance) and the bootstrap model in the original sample (test performance) (<xref ref-type="bibr" rid="B30">Efron and Tibshirani, 1994</xref>; <xref ref-type="bibr" rid="B76">Steyerberg et al., 2001b</xref>). Bias-corrected (bc) AUC values and all the classification measures (bc-ACC, bc-sensitivity, bc-specificity, bc-PPV, and bc-NPV) were calculated as their corresponding apparent measures derived from the entire original sample minus optimism (<xref ref-type="bibr" rid="B30">Efron and Tibshirani, 1994</xref>; <xref ref-type="bibr" rid="B76">Steyerberg et al., 2001b</xref>). In terms of the bc-DOR, these ratios were calculated from the values of bc-sensitivity and bc-specificity.</p>
<p>The nomograms were built based on the selected models using the RMS package (<xref ref-type="bibr" rid="B40">Harrel, 2016</xref>). A nomogram maps the predicted probabilities into points on a scale from zero to 100 in a user-friendly graphical interface. The total points accumulated by the various covariates correspond to the predicted probability of having a subgingival site with periodontal destruction in a periodontal patient (<xref ref-type="bibr" rid="B44">Iasonos et al., 2008</xref>).</p>
<p><bold>Figure <xref ref-type="fig" rid="F2">2</xref></bold> shows the flow chart of the statistical analysis: binary logistic regression and diagnostic nomograms.</p>
<fig id="F2" position="float">
<label>FIGURE 2</label>
<caption><p>Flow chart of the statistical analysis: binary logistic regression and diagnostic nomograms.</p></caption>
<graphic xlink:href="fmicb-08-01443-g002.tif"/>
</fig>
</sec>
</sec></sec>
<sec><title>Results</title>
<sec><title>Characteristics of the Study Group</title>
<p>The study group consisted of 40 individuals with a diagnosis of moderate-severe generalized chronic periodontitis and an average age of 51.35 years. Of these patients, 18 were male and 22 female. With regard to the comparative analysis of clinical parameters between the control and periodontal sites, the latter showed significantly higher BOP, PPD, and CAL values (<bold>Table <xref ref-type="table" rid="T2">2</xref></bold>).</p>
<table-wrap position="float" id="T2">
<label>Table 2</label>
<caption><p>Demographic and clinical characteristics of the patients with chronic periodontitis included in the present study, as well as the clinical characteristics of the sampled subgingival sites (control site vs. periodontal site).</p></caption>
<table cellspacing="5" cellpadding="5" frame="hsides" rules="groups">
<thead>
<tr>
<th valign="top" align="left" colspan="4">Clinical characteristics of patients (<italic>n</italic> = 40)</th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left" colspan="2">Age, years; Mean &#x00B1; SD</td>
<td valign="top" align="center">51.35 &#x00B1; 8.31</td>
<td valign="top" align="center">&#x2013;</td>
</tr>
<tr>
<td valign="top" align="left">Gender; No. and %</td>
<td valign="top" align="left">Male</td>
<td valign="top" align="center">18</td>
<td valign="top" align="center">45.00%</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left">Female</td>
<td valign="top" align="center">22</td>
<td valign="top" align="center">55.00%</td>
</tr>
<tr>
<td valign="top" align="left">Smoking habit<sup>1</sup>; No. and %</td>
<td valign="top" align="left">Never</td>
<td valign="top" align="center">19</td>
<td valign="top" align="center">47.50%</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left">Current</td>
<td valign="top" align="center">21</td>
<td valign="top" align="center">52.50%</td>
</tr>
<tr>
<td valign="top" align="left" colspan="2">Teeth present, No.; Mean &#x00B1; SD</td>
<td valign="top" align="center">25.80 &#x00B1; 3.93</td>
<td valign="top" align="center">&#x2013;</td>
</tr>
<tr>
<td valign="top" align="left" colspan="2">BPL full mouth, % sites; Mean &#x00B1; SD</td>
<td valign="top" align="center">&#x2013;</td>
<td valign="top" align="center">55.43% &#x00B1; 29.25%</td>
</tr>
<tr>
<td valign="top" align="left" colspan="2">BOP full mouth, % sites; Mean &#x00B1; SD</td>
<td valign="top" align="center">&#x2013;</td>
<td valign="top" align="center">47.15% &#x00B1; 21.32%</td>
</tr>
<tr>
<td valign="top" align="left" colspan="2">PPD full mouth, mm; Mean &#x00B1; SD</td>
<td valign="top" align="center">3.65 &#x00B1; 0.63</td>
<td valign="top" align="center">&#x2013;</td>
</tr>
<tr>
<td valign="top" align="left" colspan="2">CAL full mouth, mm; Mean &#x00B1; SD</td>
<td valign="top" align="center">4.55 &#x00B1; 1.08</td>
<td valign="top" align="center">&#x2013;</td>
</tr>
<tr>
<td valign="top" align="left">PPD full mouth, % sites</td>
<td valign="top" align="left">&#x003C;4 mm</td>
<td valign="top" align="center">&#x2013;</td>
<td valign="top" align="center">55.62%</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left">4&#x2013;6 mm</td>
<td valign="top" align="center">&#x2013;</td>
<td valign="top" align="center">39.76%</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left">>6 mm</td>
<td valign="top" align="center">&#x2013;</td>
<td valign="top" align="center">4.62%</td>
</tr>
<tr>
<td valign="top" align="left">CAL full mouth, % sites</td>
<td valign="top" align="left">&#x003C;4 mm</td>
<td valign="top" align="center">&#x2013;</td>
<td valign="top" align="center">38.43%</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left">4&#x2013;6 mm</td>
<td valign="top" align="center">&#x2013;</td>
<td valign="top" align="center">46.49%</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left">>6 mm</td>
<td valign="top" align="center">&#x2013;</td>
<td valign="top" align="center">15.08%</td>
</tr>
<tr>
<th valign="top" align="center" colspan="4"><hr/></th>
</tr>
<tr>
<td valign="top" align="left" colspan="2"><bold>Clinical characteristics of sampled subgingival sites (<italic>n</italic> = 80)</bold></td>
<td valign="top" align="center"><bold>Control site (<italic>n</italic> = 40)</bold></td>
<td valign="top" align="center"><bold>Periodontal site (<italic>n</italic> = 40)</bold></td>
</tr>
<tr>
<th valign="top" align="center" colspan="4"><hr/></th>
</tr>
<tr>
<td valign="top" align="left" colspan="2">BOP, % sites; Mean &#x00B1; SD</td>
<td valign="top" align="center">44.17% &#x00B1; 41.61%<sup>&#x2217;</sup></td>
<td valign="top" align="center">70.83% &#x00B1; 34.75%<sup>&#x2217;</sup></td>
</tr>
<tr>
<td valign="top" align="left" colspan="2">BOP, % sites; Median (IQR)</td>
<td valign="top" align="center">50% (100.00%)<sup>&#x2217;</sup></td>
<td valign="top" align="center">83.34% (66.67%)<sup>&#x2217;</sup></td>
</tr>
<tr>
<td valign="top" align="left" colspan="2">PPD, mm; Mean &#x00B1; SD</td>
<td valign="top" align="center">2.68 &#x00B1; 0.32<sup>&#x2217;&#x2217;</sup></td>
<td valign="top" align="center">5.90 &#x00B1; 1.49<sup>&#x2217;&#x2217;</sup></td>
</tr>
<tr>
<td valign="top" align="left" colspan="2">PPD, mm; Median (IQR)</td>
<td valign="top" align="center">2.67 (0.50)<sup>&#x2217;&#x2217;</sup></td>
<td valign="top" align="center">5.50 (1.00)<sup>&#x2217;&#x2217;</sup></td>
</tr>
<tr>
<td valign="top" align="left" colspan="2">CAL, mm; Mean &#x00B1; SD</td>
<td valign="top" align="center">2.96 &#x00B1; 0.61<sup>&#x2217;&#x2217;</sup></td>
<td valign="top" align="center">6.28 &#x00B1; 1.79<sup>&#x2217;&#x2217;</sup></td>
</tr>
<tr>
<td valign="top" align="left" colspan="2">CAL, mm; Median (IQR)</td>
<td valign="top" align="center">2.84 (1.00)<sup>&#x2217;&#x2217;</sup></td>
<td valign="top" align="center">5.84 (1.75)<sup>&#x2217;&#x2217;</sup></td></tr>
</tbody></table>
<table-wrap-foot>
<attrib><sup>1</sup><italic>In terms of smoking habit, a patient was considered to be: a current smoker if he/she currently smoked and had a smoking history of at least 8 years; a non-smoker if he/she had never smoked or had stopped smoking at least 5 years ago; and a former smoker if he/she had stopped smoking less than 5 years ago</italic>.</attrib>
<attrib><italic>BPL, bacterial plaque levels; BOP, bleeding on probing; PPD, probing pocket depth; CAL, clinical attachment loss; SD, standard deviation; IQR, interquartile range;</italic></attrib>
<attrib><italic><sup>&#x2217;</sup><italic>P</italic> = 0.001, <sup>&#x2217;&#x2217;</sup><italic>P</italic> &#x003C; 0.001</italic>.</attrib>
</table-wrap-foot>
</table-wrap>
<p>In relation to their smoking habits, 19 patients were non-smokers and 21 current smokers, and both patient groups showed similar age and gender characteristics, as well as similar values in terms of the periodontal indices recorded in the full mouth and the sampled subgingival sites.</p>
</sec>
<sec><title>Detection Frequencies and Levels of Pathobionts in the Chronic Periodontitis: Control Sites vs. Periodontal Sites</title>
<p>The detection frequencies of the seven species evaluated were similar between the control and periodontal sites. The median total bacterial concentration of the periodontal sites was four times higher than that detected in the control sites (<italic>P</italic> &#x003C; 0.001). Five pathobionts showed significantly higher concentrations in the periodontal sites than the control sites. These species were: <italic>Fn, Pm, Pg</italic>, <italic>Tf</italic>, and <italic>Td</italic> (<italic>P</italic> &#x003C; 0.001 for all the bacteria except <italic>Fn</italic>, <italic>P</italic> = 0.023). In terms of the 127 bacterial clusters, all except <italic>PiAaFn</italic>, <italic>PiFn</italic>, and <italic>PiAa</italic> showed significantly higher concentrations in the periodontal sites than the control sites (<italic>P</italic>-values ranged from &#x003C;0.001 to 0.009). After applying the Benjamini&#x2013;Hochberg correction, all the significant <italic>P</italic>-values were maintained (<bold>Table <xref ref-type="table" rid="T3">3</xref></bold> and <bold>Supplementary Data Sheet <xref ref-type="supplementary-material" rid="SM1">S1</xref></bold>). With regard to smoking habit, no significant differences were found in the detection frequencies or the bacterial concentrations for each subgingival site between the smokers and non-smokers.</p>
<table-wrap position="float" id="T3">
<label>Table 3</label>
<caption><p>Comparison of the detection frequencies and concentrations (pg/&#x03BC;l) of seven periopathogens between the two subgingival sites (control and periodontal sites) in the periodontal patient group.</p></caption>
<table cellspacing="5" cellpadding="5" frame="hsides" rules="groups">
<thead>
<tr>
<th valign="top" align="left">Bacteria</th>
<th valign="top" align="center" colspan="2">Detection frequency<sup>1</sup>; No. of sites (Percentage)<hr/></th>
<td valign="top" align="left"></td>
<th valign="top" align="center" colspan="2">Concentration (pg/&#x03BC;l); Mean &#x00B1; SD, Median (IQR)<hr/></th>
<td valign="top" align="left"></td>
</tr>
<tr>
<td valign="top" align="left"></td>
<th valign="top" align="center">Control site</th>
<th valign="top" align="center">Periodontal site</th>
<th valign="top" align="center"><italic>P</italic>-value</th>
<th valign="top" align="center">Control site</th>
<th valign="top" align="center">Periodontal site</th>
<th valign="top" align="center"><italic>P</italic>-value<sup>3</sup></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Total bacteria<sup>2</sup></td>
<td valign="top" align="left">40 (100)</td>
<td valign="top" align="left">40 (100)</td>
<td valign="top" align="left">NA</td>
<td valign="top" align="left">318.46 &#x00B1; 427.68</td>
<td valign="top" align="left">1006.05 &#x00B1; 915.18</td>
<td valign="top" align="left">&#x003C;0.001</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left">184.05 (429.89)</td>
<td valign="top" align="left">773.88 (1066.76)</td>
</tr>
<tr>
<td valign="top" align="left"><italic>Aa</italic></td>
<td valign="top" align="left">7 (17.5)</td>
<td valign="top" align="left">7 (17.5)</td>
<td valign="top" align="left">NS</td>
<td valign="top" align="left">1.17 &#x00B1; 5.35</td>
<td valign="top" align="left">2.27 &#x00B1; 9.95</td>
<td valign="top" align="left">NS</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left">0.00 (0.00)</td>
<td valign="top" align="left">0.00 (0.00)</td>
</tr>
<tr>
<td valign="top" align="left"><italic>Fn</italic></td>
<td valign="top" align="left">37 (92.5)</td>
<td valign="top" align="left">39 (97.5)</td>
<td valign="top" align="left">NS</td>
<td valign="top" align="left">9.32 &#x00B1; 29.82</td>
<td valign="top" align="left">22.04 &#x00B1; 49.73</td>
<td valign="top" align="left">0.023</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left">0.02 (0.05)</td>
<td valign="top" align="left">0.03 (5.71)</td>
</tr>
<tr>
<td valign="top" align="left"><italic>Pm</italic></td>
<td valign="top" align="left">40 (100)</td>
<td valign="top" align="left">40 (100)</td>
<td valign="top" align="left">NA</td>
<td valign="top" align="left">22.08 &#x00B1; 23.16</td>
<td valign="top" align="left">47.69 &#x00B1; 41.32</td>
<td valign="top" align="left">&#x003C;0.001</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left">13.16 (31.55)</td>
<td valign="top" align="left">35.83 (45.59)</td>
</tr>
<tr>
<td valign="top" align="left"><italic>Pg</italic></td>
<td valign="top" align="left">36 (90)</td>
<td valign="top" align="left">36 (90)</td>
<td valign="top" align="left">NS</td>
<td valign="top" align="left">113.25 &#x00B1; 69.34</td>
<td valign="top" align="left">373.13 &#x00B1; 1.23</td>
<td valign="top" align="left">&#x003C;0.001</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left">7.74 (79.67)</td>
<td valign="top" align="left">209.71 (498.89)</td>
</tr>
<tr>
<td valign="top" align="left"><italic>Pi</italic></td>
<td valign="top" align="left">29 (72.5)</td>
<td valign="top" align="left">26 (65)</td>
<td valign="top" align="left">NS</td>
<td valign="top" align="left">25.62 &#x00B1; 57.27</td>
<td valign="top" align="left">37.23 &#x00B1; 81.54</td>
<td valign="top" align="left">NS</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left">0.01 (5.62)</td>
<td valign="top" align="left">0 (38.22)</td>
</tr>
<tr>
<td valign="top" align="left"><italic>Tf</italic></td>
<td valign="top" align="left">40 (100)</td>
<td valign="top" align="left">40 (100)</td>
<td valign="top" align="left">NA</td>
<td valign="top" align="left">139.04 &#x00B1; 204.93</td>
<td valign="top" align="left">486.04 &#x00B1; 5.75</td>
<td valign="top" align="left">&#x003C;0.001</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left">82.21 (198.00)</td>
<td valign="top" align="left">251.79 (562.65)</td>
</tr>
<tr>
<td valign="top" align="left"><italic>Td</italic></td>
<td valign="top" align="left">33 (82.5)</td>
<td valign="top" align="left">32 (80)</td>
<td valign="top" align="left">NS</td>
<td valign="top" align="left">7.99 &#x00B1; 24.28</td>
<td valign="top" align="left">37.66 &#x00B1; 58.82</td>
<td valign="top" align="left">&#x003C;0.001</td>
</tr>
<tr>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left"></td>
<td valign="top" align="left">0.69 (4.06)</td>
<td valign="top" align="left">9.63 (49.54)</td></tr>
</tbody></table>
<table-wrap-foot>
<attrib><italic>IQR, interquartile range; NA, not applicable; NS, not significant; <italic>Aa, A. actinomycetemcomitans; Fn, F. nucleatum; Pm, P. micra; Pg, P. gingivalis; Pi, P. intermedia; Tf, T. forsythia; Td, T. denticola.</italic> <sup>1</sup>The threshold value for detection frequency was 10&#x2013;100 copies of DNA (3&#x2013;5 femtograms/10<sup>-<italic>15</italic></sup> grams). <sup>2</sup>The variable &#x2018;total bacteria&#x2019; represents the sum of the concentrations of the seven bacteria analyzed. <sup>3</sup>After applying the Benjamini&#x2013;Hochberg correction, all the <italic>P</italic>-values that showed significance were maintained</italic>.</attrib>
</table-wrap-foot>
</table-wrap>
</sec>
<sec><title>Multivariate Predictive Modeling of Chronic Periodontitis at Site-Specific Level Based on Subgingival Pathobiont Levels: Selection and Validation of Models, and the Development of Nomograms</title>
<p>A first description of the relation between pathobiont levels, as well as with the clinical parameters (PPD, CAL) is given in <bold>Supplementary Data Sheet <xref ref-type="supplementary-material" rid="SM1">S1</xref></bold>, by means of their Spearman correlations. Almost all correlations between bacteria and with the PPD and CAL values were positive. The interpretation is that when a subgingival site with periodontal destruction was present, the majority of pathobionts presented larger values, which can be verified in the comparative analysis between control and periodontal sites shown in <bold>Table <xref ref-type="table" rid="T3">3</xref></bold>. However, the correlation values between the bacteria and these with the clinical parameters were not very high, the strongest associations being observed between <italic>Pg</italic> and <italic>Tf</italic> (Rho = 0.699) and several clusters with &#x2265;4 bacteria and CAL (Rho > 0.540) (<bold>Supplementary Data Sheet <xref ref-type="supplementary-material" rid="SM1">S1</xref></bold>).</p>
<p>Applying the statistical criteria set out in the section &#x2018;Materials and Methods,&#x2019; a total of eight models were selected (<bold>Table <xref ref-type="table" rid="T4">4</xref></bold> and <bold>Supplementary Data Sheet <xref ref-type="supplementary-material" rid="SM2">S2</xref></bold>). In all of them, the predictor variables were represented by one bacterial cluster composed of at least three species. The most common bacteria were: <italic>Tf</italic> and <italic>Pi</italic> (both were present in eight models), followed by <italic>Td</italic> and <italic>Fn</italic> (both were present in six models) and <italic>Aa</italic> and <italic>Pm</italic> (both were present in five and three models, respectively). The concentration values of the optimal threshold ranged from 218.86 pg/&#x03BC;l for the <italic>PiTfFn</italic> model to 264.16 pg/&#x03BC;l for the <italic>TdPiTfAaFnPm</italic> model. There was no predictive model formed by two bacterial clusters that fulfilled the established statistical criteria.</p>
<table-wrap position="float" id="T4">
<label>Table 4</label>
<caption><p>Description of the eight models based on bacterial clusters, including the apparent and bias-corrected AUC values, as well as optimal threshold values.</p></caption>
<table cellspacing="5" cellpadding="5" frame="hsides" rules="groups">
<thead>
<tr>
<th valign="top" align="left">Cluster-based model</th>
<th valign="top" align="center">AUC</th>
<th valign="top" align="center">bc-AUC</th>
<th valign="top" align="center">Optimal threshold<sup>1</sup></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">-0.98148 + 0.00329<italic>PiTfFn</italic></td>
<td valign="top" align="center">0.773</td>
<td valign="top" align="center">0.772</td>
<td valign="top" align="center">218.86</td>
</tr>
<tr>
<td valign="top" align="left">-1.0115 + 0.00331<italic>TdPiTfAa</italic></td>
<td valign="top" align="center">0.785</td>
<td valign="top" align="center">0.784</td>
<td valign="top" align="center">220.61</td>
</tr>
<tr>
<td valign="top" align="left">-1.03208 + 0.00325<italic>TdPiTfFn</italic></td>
<td valign="top" align="center">0.783</td>
<td valign="top" align="center">0.781</td>
<td valign="top" align="center">218.27</td>
</tr>
<tr>
<td valign="top" align="left">-0.98534 + 0.00328<italic>PiTfAaFn</italic></td>
<td valign="top" align="center">0.773</td>
<td valign="top" align="center">0.774</td>
<td valign="top" align="center">220.71</td>
</tr>
<tr>
<td valign="top" align="left">-1.03627 + 0.00324<italic>TdPiTfAaFn</italic></td>
<td valign="top" align="center">0.783</td>
<td valign="top" align="center">0.781</td>
<td valign="top" align="center">220.14</td>
</tr>
<tr>
<td valign="top" align="left">-1.10445 + 0.00326<italic>TdPiTfAaPm</italic></td>
<td valign="top" align="center">0.787</td>
<td valign="top" align="center">0.784</td>
<td valign="top" align="center">264.45</td>
</tr>
<tr>
<td valign="top" align="left">-1.12387 + 0.00321<italic>TdPiTfFnPm</italic></td>
<td valign="top" align="center">0.788</td>
<td valign="top" align="center">0.786</td>
<td valign="top" align="center">262.26</td>
</tr>
<tr>
<td valign="top" align="left">-1.12778 + 0.00320<italic>TdPiTfAaFnPm</italic></td>
<td valign="top" align="center">0.789</td>
<td valign="top" align="center">0.790</td>
<td valign="top" align="center">264.16</td></tr>
</tbody></table>
<table-wrap-foot>
<attrib><sup>1</sup><italic>The best cut-off value of bacterial concentration for each model (optimal threshold) was determined so that the percentage of correct predictions was the maximum</italic>.</attrib>
</table-wrap-foot>
</table-wrap>
<p>The main results on the apparent and bc-measures of discrimination and classification of eight pathobiont-based models are detailed in <bold>Table <xref ref-type="table" rid="T5">5</xref></bold> and <bold>Supplementary Data Sheet <xref ref-type="supplementary-material" rid="SM2">S2</xref></bold>. The apparent AUC values of these models ranged from 0.773 to 0.789. The <italic>PiTfFn</italic> cluster showed an apparent AUC value of 0.773 (sensitivity and specificity = 75.0%; DOR = 9.0). When <italic>Td</italic> was incorporated in this cluster, a new predictive model emerged with better apparent AUC, sensitivity and DOR values (0.783, 77.5%, and 10.3, respectively). When <italic>Pm</italic> and <italic>AaPm</italic> were incorporated in the <italic>TdPiTfFn</italic> cluster, we obtained the two best predictive models with apparent AUC values of 0.788 and 0.789, respectively, as well as sensitivity and specificity values of 77.5% and DOR of 11.8 for both models. However, when <italic>Aa</italic> and <italic>TdAa</italic> were incorporated in the <italic>PiTfFn</italic> cluster, the two predictive models that appeared did not have better discrimination and classification measures than those detected in clusters of lower numbers of bacteria, such as <italic>PiTfFn</italic> and <italic>TdPiTfFn.</italic></p>
<table-wrap position="float" id="T5">
<label>Table 5</label>
<caption><p>Measures of discrimination and classification of the eight models based on bacterial clusters.</p></caption>
<table cellspacing="5" cellpadding="5" frame="hsides" rules="groups">
<thead>
<tr>
<th valign="top" align="left">Model</th>
<th valign="top" align="left">ACC (%)</th>
<th valign="top" align="left">Sensitivity (%)</th>
<th valign="top" align="left">Specificity (%)</th>
<th valign="top" align="left">Positive predictive value (%)</th>
<th valign="top" align="left">Negative predictive value (%)</th>
<th valign="top" align="left">Diagnostic OR</th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left"><italic>PiTfFn</italic></td>
<td valign="top" align="left">75.0 72.4</td>
<td valign="top" align="left">75.0 72.6</td>
<td valign="top" align="left">75.0 72.8</td>
<td valign="top" align="left">75.0 72.4</td>
<td valign="top" align="left">75.0 73.0</td>
<td valign="top" align="left">9.0 7.1</td>
</tr>
<tr>
<td valign="top" align="left"><italic>TdPiTfAa</italic></td>
<td valign="top" align="left">75.0 71.9</td>
<td valign="top" align="left">75.0 72.0</td>
<td valign="top" align="left">75.0 72.6</td>
<td valign="top" align="left">75.0 72.1</td>
<td valign="top" align="left">75.5 73.0</td>
<td valign="top" align="left">9.0 6.8</td>
</tr>
<tr>
<td valign="top" align="left"><italic>TdPiTfFn</italic></td>
<td valign="top" align="left">76.2 73.4</td>
<td valign="top" align="left">77.5 74.8</td>
<td valign="top" align="left">75.0 72.8</td>
<td valign="top" align="left">75.6 73.0</td>
<td valign="top" align="left">76.9 74.5</td>
<td valign="top" align="left">10.3 7.9</td>
</tr>
<tr>
<td valign="top" align="left"><italic>PiTfAaFn</italic></td>
<td valign="top" align="left">75.0 72.6</td>
<td valign="top" align="left">75.0 72.8</td>
<td valign="top" align="left">75.0 73.1</td>
<td valign="top" align="left">75.0 72.3</td>
<td valign="top" align="left">75.5 73.8</td>
<td valign="top" align="left">9.0 7.2</td>
</tr>
<tr>
<td valign="top" align="left"><italic>TdPiTfAaFn</italic></td>
<td valign="top" align="left">76.2 73.4</td>
<td valign="top" align="left">77.5 74.5</td>
<td valign="top" align="left">75.0 73.1</td>
<td valign="top" align="left">75.6 73.1</td>
<td valign="top" align="left">77.2 74.8</td>
<td valign="top" align="left">10.3 7.9</td>
</tr>
<tr>
<td valign="top" align="left"><italic>TdPiTfAaPm</italic></td>
<td valign="top" align="left">77.5 75.0</td>
<td valign="top" align="left">75.0 72.5</td>
<td valign="top" align="left">80.0 78.1</td>
<td valign="top" align="left">79.0 76.6</td>
<td valign="top" align="left">76.1 74.1</td>
<td valign="top" align="left">12.0 9.4</td>
</tr>
<tr>
<td valign="top" align="left"><italic>TdPiTfFnPm</italic></td>
<td valign="top" align="left">77.5 74.9</td>
<td valign="top" align="left">77.5 74.9</td>
<td valign="top" align="left">77.5 75.6</td>
<td valign="top" align="left">77.7 75.3</td>
<td valign="top" align="left">77.7 75.5</td>
<td valign="top" align="left">11.8 9.2</td>
</tr>
<tr>
<td valign="top" align="left"><italic>TdPiTfAaFnPm</italic></td>
<td valign="top" align="left">77.5 75.3</td>
<td valign="top" align="left">77.5 75.2</td>
<td valign="top" align="left">77.5 75.9</td>
<td valign="top" align="left">77.7 75.6</td>
<td valign="top" align="left">77.5 75.8</td>
<td valign="top" align="left">11.8 9.6</td></tr>
</tbody></table>
<table-wrap-foot>
<attrib><italic>In each cell, the first value is referred to the apparent performance measures and the second, to the corrected performance measures (bc-ACC, bc-sensitivity, bc-specificity, bc-PPV, bc-NPV, and bc-DOR) by the level of optimism calculated using a bootstrap procedure. The 95% CIs of the different performances measures, excepting DOR, are detailed in <bold>Supplementary Data Sheet <xref ref-type="supplementary-material" rid="SM2">S2</xref></bold></italic>.</attrib>
</table-wrap-foot>
</table-wrap>
<p>The <italic>TdPiTfAa</italic> cluster showed an apparent AUC value of 0.785 (sensitivity and specificity = 75.0%; DOR = 9.0). When <italic>Pm</italic> was incorporated in this cluster, a new predictive model appeared with better apparent AUC, specificity and DOR values (0.787, 80.0%, and 12.0, respectively). The optimism values obtained by the bootstrap methods in relation to the performance measures of these models ranged from -0.001 to 0.003 in the AUC values and 1.52&#x2013;3.07% in the classification indices.</p>
<p><bold>Figures <xref ref-type="fig" rid="F3">3</xref></bold>, <bold><xref ref-type="fig" rid="F4">4</xref></bold> show the ROC curves of the most predictive clusters with the lowest number of species &#x2013; <italic>PiTfFn</italic> and <italic>TdPiTfAa</italic> &#x2013; and the highest number of bacteria &#x2013; <italic>TdPiTfAaFnPm</italic> and <italic>TdPiTfAaPm</italic>. <bold>Figures <xref ref-type="fig" rid="F5">5</xref></bold>, <bold><xref ref-type="fig" rid="F6">6</xref></bold> show the diagnostic nomograms of the four previous models. As noted previously, the discrimination and classification performance values of all the nomograms were high, indicating their good accuracy. Overall, in eight nomograms, the high concentration levels of the different bacterial clusters were associated with an increased probability of having a subgingival site with periodontal destruction in a patient with chronic periodontitis. Supplementary Files show these graphics of the eight models with an AUC &#x2265;0.76 and sensitivity and specificity values &#x2265; 75% (<bold>Supplementary Data Sheets <xref ref-type="supplementary-material" rid="SM3">S3</xref></bold>, <bold><xref ref-type="supplementary-material" rid="SM4">S4</xref></bold>, and Figures S1&#x2013;S8).</p>
<fig id="F3" position="float">
<label>FIGURE 3</label>
<caption><p>Receiver operating characteristic (ROC) curves of the two predictive models based on the <italic>PiTfFn</italic> and <italic>TdPiTfAa</italic> clusters, including the apparent and bias-corrected measures by bootstrapping. <bold>(A)</bold> <italic>PiTfFn</italic> cluster; <bold>(B)</bold> <italic>TdPiTfAa</italic> cluster.</p></caption>
<graphic xlink:href="fmicb-08-01443-g003.tif"/>
</fig>
<fig id="F4" position="float">
<label>FIGURE 4</label>
<caption><p>Receiver operating characteristic (ROC) curves of the two predictive models based on the <italic>TdPiTfAaPm</italic> and <italic>TdPiTfAaFnPm</italic> clusters, including the apparent and bias-corrected measures by bootstrapping. <bold>(A)</bold> <italic>TdPiTfAaPm</italic> cluster; <bold>(B)</bold> <italic>TdPiTfAaFnPm</italic> cluster.</p></caption>
<graphic xlink:href="fmicb-08-01443-g004.tif"/>
</fig>
<fig id="F5" position="float">
<label>FIGURE 5</label>
<caption><p>Diagnostic nomograms based on the <italic>PiTfFn</italic> and <italic>TdPiTfAa</italic> models for predicting the probability of having a periodontal site. The probability of having a subgingival site with periodontal destruction is calculated by drawing a line to the point on the axis for each of the following variables: <bold>(A)</bold> <italic>PiTfFn</italic> cluster; <bold>(B)</bold> <italic>TdPiTfAa</italic> cluster. The points for each variable are located on the total point line. Next, a vertical line is projected from the total point line to the predicted probability bottom scale to obtain the individual probability of periodontitis at the site-specific level.</p></caption>
<graphic xlink:href="fmicb-08-01443-g005.tif"/>
</fig>
<fig id="F6" position="float">
<label>FIGURE 6</label>
<caption><p>Diagnostic nomograms based on the <italic>TdPiTfAaPm</italic> and <italic>TdPiTfAaFnPm</italic> models for predicting the probability of having a periodontal site. The probability of having a subgingival site with periodontal destruction is calculated by drawing a line to the point on the axis for each of the following variables: <bold>(A)</bold> <italic>TdPiTfAaPm</italic> cluster; <bold>(B)</bold> <italic>TdPiTfAaFnPm</italic> cluster. The points for each variable are located on the total point line. Next, a vertical line is projected from the total point line to the predicted probability bottom scale to obtain the individual probability of periodontitis at the site-specific level.</p></caption>
<graphic xlink:href="fmicb-08-01443-g006.tif"/>
</fig>
</sec>
</sec>
<sec><title>Discussion</title>
<sec><title>Methodological Aspects of the qPCR Studies</title>
<p>In terms of study design, a control group (healthy subjects) is compared to a group of periodontitis patients in numerous qPCR studies of periodontal pathogens at the subgingival level (<xref ref-type="bibr" rid="B15">Braga et al., 2010</xref>; <xref ref-type="bibr" rid="B51">Kretschmar et al., 2012</xref>). The control group may also include gingivitis patients (<xref ref-type="bibr" rid="B72">Scapoli et al., 2015</xref>), or they may constitute a separate group (<xref ref-type="bibr" rid="B57">Masunaga et al., 2010</xref>). In addition, there are other authors who have studied subgingival samples microbiologically using the qPCR from different periodontal patient groups according to the degree of periodontal affection (mild, moderate, or severe) (<xref ref-type="bibr" rid="B87">Zorina et al., 2011</xref>) or the type of disease (chronic vs. aggressive) (<xref ref-type="bibr" rid="B21">Casarin et al., 2010</xref>). There are also qPCR studies that have been performed exclusively on subjects with periodontitis (<xref ref-type="bibr" rid="B50">Kotsilkov et al., 2015</xref>; <xref ref-type="bibr" rid="B58">Milne et al., 2015</xref>).</p>
<p>Although certain subject-associated factors can increase an individual&#x2019;s overall susceptibility to periodontal diseases, it appears that not all sites within a subject are equally susceptible at one time (<xref ref-type="bibr" rid="B38">Griffiths, 2003</xref>). This site-specific clinical pattern can be attributed to differences in the bacterial species composition of subgingival plaque (<xref ref-type="bibr" rid="B17">Byrne et al., 2009</xref>). However, there is little evidence in the literature of the influence of pathobionts analyzed by the qPCR in relation to the site-specific severity of chronic periodontitis (<xref ref-type="bibr" rid="B59">Mineoka et al., 2008</xref>; <xref ref-type="bibr" rid="B77">Teixeira et al., 2009</xref>; <xref ref-type="bibr" rid="B4">Al-hebshi et al., 2014</xref>). As a consequence, we propose using a qPCR technique to study the relationship between pathobiont levels from subgingival sites with different degrees of disease (control sites vs. periodontal sites) within the same periodontal patient. This methodology (paired design) means that the effects of inter-individual variations in factors other than bacterial composition are avoided.</p>
<p>Regarding the species studied, we decided to quantify them initially according to Socransky&#x2019;s bacterial complexes (<xref ref-type="bibr" rid="B74">Socransky et al., 1998</xref>), with those most frequently studied being: three species of the Red Complex (<italic>Pg, Tf</italic>, and <italic>Td</italic>), the main three species of the Orange Complex (<italic>Fn</italic>, <italic>Pi</italic>, and <italic>Pm</italic>) and a species from the Green Complex (<italic>Aa</italic>). Consequently, our intention was to focus on various well-known pathobionts that could be used as biomarkers in a possible diagnostic tool for dental practitioners in order to improve disease control and reduce the future risk of progression seen in everyday clinical practice.</p>
<p>In numerous qPCR papers, the authors report the number of DNA copies as a measure of the quantification of the bacterial load rather than cell numbers, because the copy number of the marker gene, particularly the 16S rRNA gene, varies from one species to another and is unknown for some of them (<xref ref-type="bibr" rid="B4">Al-hebshi et al., 2014</xref>, <xref ref-type="bibr" rid="B3">2015</xref>; <xref ref-type="bibr" rid="B85">Yang et al., 2016</xref>). This implies that available bacterial counts for some of the tested genera/species have probably been over-estimated (<xref ref-type="bibr" rid="B4">Al-hebshi et al., 2014</xref>). In the present series, we used a new qPCR technique with TaqMan probes and Cod-UNG enzymes for the molecular detection of these pathobionts. This patented technique allows the easy design of probes for the detection of any pathogen while maintaining excellent sensitivity and specificity (<xref ref-type="bibr" rid="B11">Bou et al., 2014</xref>). In the present study, the absolute quantification of the DNA of each species was performed using the standard curve method. Moreover, the target genes studied were different single-copy genes per genome, which may allow absolute counts of DNA copies to be reported in terms of cell numbers.</p>
<p>From an analytical point of view, the majority of authors in their respective qPCR studies applied a &#x2018;simple approach.&#x2019; As a result, the subgingival levels of pathobionts in control volunteers and periodontal patients were compared for testing using q-PCR, whether or not a particular bacterium is absent/present and/or diminished/elevated at the individual level (<xref ref-type="bibr" rid="B15">Braga et al., 2010</xref>; <xref ref-type="bibr" rid="B65">Polonyi et al., 2013</xref>; <xref ref-type="bibr" rid="B50">Kotsilkov et al., 2015</xref>; <xref ref-type="bibr" rid="B58">Milne et al., 2015</xref>; <xref ref-type="bibr" rid="B72">Scapoli et al., 2015</xref>; <xref ref-type="bibr" rid="B37">Golynska et al., 2016</xref>). In parallel with current trends in big data analytics for genomic medicine (<xref ref-type="bibr" rid="B41">He et al., 2017</xref>), further evidence is required about applying multivariate analytical approaches to qPCR data. This would enable there to be a better understanding of the complex bacterial interactions present in chronic periodontitis. This type of analytical approach was used in our study.</p>
</sec>
<sec><title>Detection Frequencies and Levels of Pathobionts in the Severity of Chronic Periodontitis: Control Sites vs. Periodontal Sites</title>
<p><xref ref-type="bibr" rid="B4">Al-hebshi et al. (2014)</xref> used the qPCR to investigate seven pathobionts (the species of the Red Complex, the genera <italic>Fusobacterium</italic> spp. and <italic>Prevotella</italic> spp., as well as <italic>Aa</italic>) in healthy and periodontal subgingival sites within the same periodontal patient. This study was similar to ours methodologically. The Al-hebshi&#x2019;s series revealed detection percentages of 100% for <italic>Pm</italic>, <italic>Td</italic>, <italic>Tf</italic>, <italic>Fusobacterium</italic> spp. and <italic>Prevotella</italic> spp., 97.5% for <italic>Pg</italic>, and 67.5% for <italic>Aa</italic>. We agree with them on the high detection rates found for the Red Complex species, <italic>Fn</italic> and <italic>Pm</italic> (80&#x2013;100%), confirming that the mere presence of the bacteria is not an indicator of the clinical situation (<xref ref-type="bibr" rid="B17">Byrne et al., 2009</xref>; <xref ref-type="bibr" rid="B24">Charalampakis et al., 2013</xref>). However, in contrast to <xref ref-type="bibr" rid="B4">Al-hebshi et al. (2014)</xref>, our series of periodontal patients showed lower detection frequencies for <italic>Pi</italic> (69%) and, especially, <italic>Aa</italic> (17.5%), although those values were not conditioned by the subgingival site tested. Initially, our results on the low prevalence of <italic>Aa</italic> confirmed its marginal role in chronic periodontitis in our study population, which reflects the findings of other authors (<xref ref-type="bibr" rid="B34">Gatto et al., 2014</xref>); however, we will later see how its role changes in the multivariate analysis.</p>
<p>In Al-hebshi&#x2019;s study (<xref ref-type="bibr" rid="B4">Al-hebshi et al., 2014</xref>), the levels of <italic>Pm</italic>, <italic>Prevotella</italic> spp<italic>., Pg</italic>, <italic>Tf</italic>, and <italic>Td</italic> were higher in subgingival sites with periodontal destruction than the control sites. However, after correcting for multiple comparisons, these differences only remained statistically significant for <italic>Pm</italic> and <italic>Tf.</italic> This was probably due to the small sample size (only 20 periodontal patients). In the present study, our results revealed that periodontal sites had significantly higher concentrations for the three Red Complex species (concentration increments of 27, 14, and 3 times for <italic>Pg</italic>, <italic>Td</italic>, and <italic>Tf</italic>, respectively) and <italic>Pm</italic> and <italic>Fn</italic> (concentration increments of 2.7 and 1.5 times, respectively).</p>
<p>Our findings demonstrated that all the pathobionts (except <italic>Aa</italic> and <italic>Pi</italic>) were present at the control sites (non-active sites) and that an increase in their concentrations was required to trigger activity in the periodontal site; <italic>Pg</italic> was the bacteria that had a higher increment. The relevance of the Red Complex species together and <italic>Pg</italic> alone in the pathogenesis of periodontitis has been proved in numerous <italic>in vitro</italic> studies (<xref ref-type="bibr" rid="B9">Bao et al., 2014</xref>; <xref ref-type="bibr" rid="B54">Lin et al., 2014</xref>; <xref ref-type="bibr" rid="B84">Willi et al., 2014</xref>). A bacterial consortium named the Orange Complex, which includes <italic>Fn</italic> and <italic>Pm</italic>, was indicated as preceding the Red Complex with respect to colonization and proliferation (<xref ref-type="bibr" rid="B74">Socransky et al., 1998</xref>). In relation to <italic>Fn</italic> and <italic>Pm</italic>, there is growing evidence based on <italic>in vitro</italic> research on their role as periodontal pathogens (<xref ref-type="bibr" rid="B62">Nonnenmacher et al., 2003</xref>; <xref ref-type="bibr" rid="B52">Lee and Baek, 2013</xref>; <xref ref-type="bibr" rid="B82">Wang et al., 2013</xref>; <xref ref-type="bibr" rid="B47">Kim and Lee, 2014</xref>; <xref ref-type="bibr" rid="B56">Marchesan et al., 2016</xref>).</p>
<p>Although evaluating the influence of smoking was not a major objective of the present study, our findings in this regard are consistent with those of <xref ref-type="bibr" rid="B79">Tomita et al. (2013)</xref>, as we did not detect significant differences in the profiles of the seven species present in the sampled subgingival sites (control and periodontal sites). These sites showed similar clinical characteristics between smokers and non-smokers. Nevertheless, these results should be interpreted with caution since, in larger series, other authors have found significantly higher prevalences of <italic>Td</italic> and <italic>Tf</italic> and significantly higher levels of <italic>Aa</italic>, <italic>Pg</italic>, and <italic>Tf</italic> in smokers with chronic periodontitis (<xref ref-type="bibr" rid="B34">Gatto et al., 2014</xref>; <xref ref-type="bibr" rid="B39">Guglielmetti et al., 2014</xref>).</p>
</sec>
<sec><title>Multivariate Predictive Modeling of the Subgingival Levels of Pathobionts: Selection of the Best Models and the Development of Nomograms</title>
<p>In periodontology, traditional clinical criteria are often inadequate for determining sites of active disease, measuring the degree of susceptibility to future progression, and monitoring the response to therapy (<xref ref-type="bibr" rid="B35">Giannobile et al., 2009</xref>; <xref ref-type="bibr" rid="B49">Korte and Kinney, 2016</xref>). In this sense, there is a need for research on innovative diagnostic tests based on biomarkers that focus on the early recognition of the microbial challenge to the host, providing a benefit in terms of disease control at the site-specific level.</p>
<p>The first step to assess whether a test could be clinically useful is the analysis of its predictive ability (<xref ref-type="bibr" rid="B28">Deeks, 1999</xref>). In this sense, there are some papers in the literature that have focused on the analysis of the predictive ability of the pathobiont levels quantified by the qPCR to distinguish different degrees of periodontal affection in subgingival samples from different individuals (<xref ref-type="bibr" rid="B68">Ramseier et al., 2009</xref>; <xref ref-type="bibr" rid="B71">Saygun et al., 2011</xref>; <xref ref-type="bibr" rid="B48">Kinney et al., 2014</xref>; <xref ref-type="bibr" rid="B3">Al-hebshi et al., 2015</xref>; <xref ref-type="bibr" rid="B81">Torrungruang et al., 2015</xref>). Several of these studies coincide in terms of detecting that the subgingival levels of certain bacteria, such as the Red Complex species, <italic>C. rectus</italic>, <italic>Pi</italic>, <italic>Pm</italic> and even a new bacterial phylum of oral Synergistetes, showed a good diagnostic power for distinguishing between healthy/gingivitis patients and those with periodontitis (AUC values ranged from >0. 74 to >0.80) (<xref ref-type="bibr" rid="B68">Ramseier et al., 2009</xref>; <xref ref-type="bibr" rid="B3">Al-hebshi et al., 2015</xref>).</p>
<p>However, we have found very few papers like ours on bacteria-based predictive models for diagnosing periodontitis severity or progression using a paired design (<xref ref-type="bibr" rid="B17">Byrne et al., 2009</xref>; <xref ref-type="bibr" rid="B61">Nomura et al., 2012</xref>; <xref ref-type="bibr" rid="B24">Charalampakis et al., 2013</xref>; <xref ref-type="bibr" rid="B34">Gatto et al., 2014</xref>). <xref ref-type="bibr" rid="B24">Charalampakis et al. (2013)</xref> identified sites at risk of future progression during 2 years of maintenance in 50 patients with chronic periodontitis. This research was based on longitudinal clinical and microbiological monitoring using DNA&#x2013;DNA hybridization involving 25 bacterial species. In this series, both members of the Red Complex as well as of the &#x2018;B Complex&#x2019; (<italic>Pi</italic>, <italic>Fn</italic> and <italic>C. rectus</italic>) were equally able to identify periodontitis from non-periodontitis sites and severe periodontitis sites from non-severe periodontitis sites; these bacterial complexes had AUC values above 0.75. In accordance with these results (<xref ref-type="bibr" rid="B24">Charalampakis et al., 2013</xref>), in our study the best predictive models with AUC values &#x2265;0.76 (sensitivity and specificity &#x2265;75%), which could be regarded as good (<xref ref-type="bibr" rid="B43">Hosmer et al., 2013</xref>), included the concentration levels of different bacterial clusters formed by at least three bacteria. These findings were ratified by the analysis of the internal validation that was performed, since the optimism values obtained by the bootstrap techniques were not high. Therefore, in accordance with the findings of previous authors (<xref ref-type="bibr" rid="B17">Byrne et al., 2009</xref>; <xref ref-type="bibr" rid="B24">Charalampakis et al., 2013</xref>), our results confirm that the use of bacterial quantitative data rather than dichotomous data (presence/absence) provides more statistical power for detecting any predictive relationship with disease severity.</p>
<p>Interestingly, and unlike Charalampakis&#x2019;s study, the bacteria present in all the predictive clusters were <italic>Tf</italic> and <italic>Pi</italic> (both in eight models), followed by <italic>Td</italic> and <italic>Fn</italic> (both in six models) and <italic>Aa</italic> and <italic>Pm</italic> (both in five and three models, respectively). All these clusters were constituted by bacteria that belong to different Socransky&#x2019;s complexes and have different pathogenic roles in disease severity in terms of individual concentration levels. Such an outcome was previously observed in our own series. In fact, although <italic>Pi</italic> and <italic>Aa</italic> had non-significant, higher concentrations in the periodontal sites than the control sites, their levels contributed to increasing the predictive ability of the cluster in which they were incorporated. On the other hand, although <italic>Pg</italic> was the species that showed the highest concentration increment in the periodontal sites, surprisingly this species did not have a high power for predicting periodontitis severity in comparison with other pathobionts, since it was not included in any of the best predictive models. From a predictive perspective, these findings contribute to <italic>Pg</italic> being regarded as a &#x2018;keystone&#x2019; pathobiont which is a microorganism that can change the environment to alter proportions or levels of other symbionts and pathobionts within the ecological niche; these microorganisms are the main ones responsible for triggering the destructive cascade that provokes the activation of inflammation and subsequent bone destruction (<xref ref-type="bibr" rid="B25">Costalonga and Herzberg, 2014</xref>).</p>
<p>In accordance with previous authors (<xref ref-type="bibr" rid="B17">Byrne et al., 2009</xref>; <xref ref-type="bibr" rid="B24">Charalampakis et al., 2013</xref>), we demonstrated that the detection of a specific threshold of concentration levels of certain bacterial clusters may serve as a predictor of periodontitis severity at site-specific level. Two main predictive clusters were identified: <italic>PiTfFn</italic> (AUC = 0.773; sensitivity and specificity = 75.0%; DOR = 9.0) and <italic>TdPiTfAa</italic> (AUC = 0.785; sensitivity and specificity = 75.0%; DOR = 9.0). In relation to <italic>PiTfFn</italic>, when other bacteria or clusters such as <italic>Td</italic>, <italic>Pm</italic>, and <italic>AaPm</italic> were incorporated in this cluster, new predictive models emerged with better predictive parameters. The two best predictive clusters were <italic>TdPiTfFnPm</italic> and <italic>TdPiTfAaFnPm</italic>, with AUC values of 0.788 and 0.789, respectively (sensitivity and specificity values of 77.5% and DOR of 11.8 for both models). In relation to <italic>TdPiTfAa</italic>, when <italic>Pm</italic> was incorporated in this cluster, a new predictive model appeared with better AUC, specificity and DOR values (0.787, 80.0%, and 12.0, respectively).</p>
<p>To the best of our knowledge, this study presents the first evidence that these new bacterial clusters are capable of predicting the severity of chronic periodontitis, with several species detected outside the Red Complex that contribute to the detection of periodontal destruction at the specific-site level. To date, the findings obtained by open-ended molecular approaches support the hypothesis that chronic periodontitis is initiated by polymicrobial synergy and the dysbiosis of the entire microbial community (PSD model). The condition is characterized not only by a greater involvement of the &#x2018;established&#x2019; pathobionts, as studied in this series, but also by the coexistence of other pathobionts with an unknown role such as <italic>Anaeroglobus</italic>, <italic>Bulleidia</italic>, <italic>Desulfobulbus</italic>, <italic>Filifactor alocis</italic>, <italic>Mogibacterium</italic>, <italic>Phocaeicola</italic>, <italic>Schwartzia, or TM7</italic> (<xref ref-type="bibr" rid="B18">Camelo-Castillo et al., 2015a</xref>,<xref ref-type="bibr" rid="B19">b</xref>). Given this new pathogenic approach of the disease, it would be very interesting to study the predictive ability of these new pathobionts and other symbionts in chronic periodontitis, as well as their relationship with host mediators.</p>
<p>Nomograms are simplified representations of complicated statistical models, and their clinical value relates to the fact that they map the predicted probabilities into points on a scale from 0 to 100 in a user-friendly graphical interface (<xref ref-type="bibr" rid="B44">Iasonos et al., 2008</xref>). To our knowledge, this is the first study providing several nomograms based on the concentration levels of certain bacterial clusters to predict the probability of having a periodontal site in a patient with chronic periodontitis.</p>
<p>Our nomograms, which are derived from the eight best-fitting models, fulfilled the requirements of discrimination. Overall, in all the nomograms, higher concentration levels of different bacterial clusters were associated with an increased probability of having a periodontal site in a periodontal patient. The use of only a few variables is desirable in nomograms to increase their utility in clinical practice (<xref ref-type="bibr" rid="B5">Altman and Royston, 2000</xref>). Consequently, of the nomograms developed in the present study, we highlight those based on the clusters formed by the lower number of species, <italic>PiTfFn</italic> and <italic>TdPiTfAa</italic>. The application of these tools in the field of clinical activity would improve the identification of sites with periodontal destruction that are at possible risk of future progression, contributing to the decision-making process and treatment planning dilemmas.</p>
<p>Our research has some limitations. The most important weakness is that the prediction of the study&#x2019;s accuracy is only measured in the samples that generated the model equations. As a consequence, to evaluate the reproducibility of the models and control for the possibility of overfitting, we validated the prediction rule internally (discrimination and classification measures) using bootstrap methods on the original derivation dataset by sampling with replacements for 1000 iterations (<xref ref-type="bibr" rid="B75">Steyerberg et al., 2001a</xref>,<xref ref-type="bibr" rid="B76">b</xref>). This internal validation method revealed optimal results on the control of overfitting in the models.</p>
<p>The evaluation of these cluster-based predictive models and nomograms is a potential future research direction. Firstly, it would greatly benefit the strength of our study if the predictive accuracy of the predictive models derived from our series could be measured in a large &#x2018;external&#x2019; or independent cohort of patients to verify whether our findings are universally applicable or if they are conditioned by differences associated with geographic regions and ethnic groups (<xref ref-type="bibr" rid="B42">Herrera et al., 2008</xref>; <xref ref-type="bibr" rid="B83">Wara-aswapati et al., 2009</xref>; <xref ref-type="bibr" rid="B66">Psoter et al., 2011</xref>). An appropriate calibration analysis of the predictive models should also be performed. Secondly, the potential prognostic value at the site-specific level of these cluster-based predictive models with regard to disease progression and the response to treatment in periodontal patients should be exploited in longitudinal studies, as should their relationship with host mediators and their potential predictive accuracy in saliva samples.</p>
</sec>
</sec>
<sec><title>Conclusion</title>
<p>We corroborated the important etiopathogenic role in quantitative terms, not only of Socransky&#x2019;s Red Complex pathobionts, but also of other species such as <italic>Parvimonas micra</italic>, in the severity of chronic periodontitis at site-specific level. Several statistically validated models based on the bacterial concentration levels with a good predictive accuracy demonstrated that some species are good biomarkers when it comes to distinguishing a site with periodontal destruction in a periodontal patient. All these models consist of different clusters formed by several bacteria, at least three, which belong to different Socransky&#x2019;s complexes. The most predictive clusters with the lowest number of species were <italic>PiTfFn</italic> and <italic>TdPiTfAa</italic>, and those with the highest number were <italic>TdPiTfFnPm</italic> and <italic>TdPiTfAaFnPm.</italic> In all the nomograms, higher concentration levels of these clusters were associated with an increased probability of finding a periodontal site in a patient with chronic periodontitis. The clinical implications of these predictive tools could include improved patient monitoring and the control of disease activity at the site-specific level. However, additional evidence is needed to test the external validity of these bacterial cluster-based models for predicting chronic periodontitis severity at the site-specific level and confirming the clinical value of the proposed nomograms.</p>
</sec>
<sec><title>Author Contributions</title>
<p>IT, AR-I, NA-B, and LN participated in the recruitment and diagnosis of patients, the collection of oral samples, in reviewing the literature and writing the paper. ML and MT participated in the quantification by qPCR of pathobionts and in the interpretation of the obtained results. IT and CB-C participated in the statistical design of the models, in the internal validation analysis and the development of the graphics. All authors made substantial contributions to all of the following: (1) the conception and design of the study, or acquisition of data, or analysis and interpretation of data, (2) drafting the article or revising it critically for important intellectual content, (3) the review and approval of the manuscript in its submitted form.</p>
</sec>
<sec><title>Conflict of Interest Statement</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
</body>
<back>
<fn-group>
<fn fn-type="financial-disclosure">
<p><bold>Funding.</bold> This work was supported by the EM2014/025 project from the Regional Ministry of Culture, Education and University (regional government of Galicia, Spain), which is integrated in the Regional Plan of Research, Innovation and Development 2011&#x2013;2015; and grants PI13/02390-PI16/01163 awarded to MT within the State Plan for R+D+I 2013&#x2013;2016 (National Plan for Scientific Research, Technological Development and Innovation 2008&#x2013;2011) and co-financed by the ISCIII-Deputy General Directorate of evaluation and Promotion of Research-European Regional Development Fund &#x201C;A way of Making Europe&#x201D; and Instituto de Salud Carlos III FEDER.</p>
</fn>
</fn-group>
<sec sec-type="supplementary material">
<title>Supplementary Material</title>
<p>The Supplementary Material for this article can be found online at: <ext-link ext-link-type="uri" xlink:href="http://journal.frontiersin.org/article/10.3389/fmicb.2017.01443/full#supplementary-material">http://journal.frontiersin.org/article/10.3389/fmicb.2017.01443/full#supplementary-material</ext-link></p>
<supplementary-material xlink:href="Data_Sheet_1.XLSX" id="SM1" mimetype="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" xmlns:xlink="http://www.w3.org/1999/xlink">
<label>DATA SHEET S1</label>
<caption><p>Dataframe and univariate analysis.</p></caption>
</supplementary-material>
<supplementary-material xlink:href="Data_Sheet_1.XLSX" id="S1" mimetype="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" xmlns:xlink="http://www.w3.org/1999/xlink"/>
<supplementary-material xlink:href="Data_Sheet_2.XLSX" id="SM2" mimetype="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" xmlns:xlink="http://www.w3.org/1999/xlink">
<label>DATA SHEET S2</label>
<caption><p>Description of cluster-based models with an AUC value &#x2265;0.76 and sensitivity and specificity values &#x2265;75%, and their principal characteristics. Results of the analysis on the internal validation by bootstrap methods of these predictive models.</p></caption>
</supplementary-material>
<supplementary-material xlink:href="Data_Sheet_2.XLSX" id="S2" mimetype="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" xmlns:xlink="http://www.w3.org/1999/xlink"/>
<supplementary-material xlink:href="Data_Sheet_3.pdf" id="SM3" mimetype="application/pdf" xmlns:xlink="http://www.w3.org/1999/xlink">
<label>DATA SHEET S3</label>
<caption><p>(Figures S1&#x2013;S8): Receiver operating characteristic (ROC) curves of the cluster-based models with an AUC &#x2265;0.76 and sensitivity and specificity values &#x2265;75%.</p></caption>
</supplementary-material>
<supplementary-material xlink:href="Data_Sheet_3.pdf" id="S3" mimetype="application/pdf" xmlns:xlink="http://www.w3.org/1999/xlink"/>
<supplementary-material xlink:href="Data_Sheet_4.pdf" id="SM4" mimetype="application/pdf" xmlns:xlink="http://www.w3.org/1999/xlink">
<label>DATA SHEET S4</label>
<caption><p>(Figures S1&#x2013;S8): Diagnostic nomograms derived from the cluster-based models with an AUC &#x2265;0.76 and sensitivity and specificity values &#x2265;75%.</p></caption>
</supplementary-material>
<supplementary-material xlink:href="Data_Sheet_4.pdf" id="S4" mimetype="application/pdf" xmlns:xlink="http://www.w3.org/1999/xlink"/>
</sec>
<ref-list>
<title>References</title>
<ref id="B1"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Abbayya</surname> <given-names>K.</given-names></name> <name><surname>Puthanakar</surname> <given-names>N. Y.</given-names></name> <name><surname>Naduwinmani</surname> <given-names>S.</given-names></name> <name><surname>Chidambar</surname> <given-names>Y. S.</given-names></name></person-group> (<year>2015</year>). <article-title>Association between periodontitis and alzheimer&#x2019;s disease.</article-title> <source><italic>N. Am. J. Med. Sci.</italic></source> <volume>7</volume> <fpage>241</fpage>&#x2013;<lpage>246</lpage>. <pub-id pub-id-type="doi">10.4103/1947-2714.159325</pub-id></citation></ref>
<ref id="B2"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Al-Harthi</surname> <given-names>L. S.</given-names></name> <name><surname>Cullinan</surname> <given-names>M. P.</given-names></name> <name><surname>Leichter</surname> <given-names>J. W.</given-names></name> <name><surname>Thomson</surname> <given-names>W. M.</given-names></name></person-group> (<year>2013</year>). <article-title>The impact of periodontitis on oral health-related quality of life: a review of the evidence from observational studies.</article-title> <source><italic>Aust. Dent. J.</italic></source> <volume>58</volume> <fpage>274</fpage>&#x2013;<lpage>277</lpage>. <pub-id pub-id-type="doi">10.1111/adj.12076</pub-id></citation></ref>
<ref id="B3"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Al-hebshi</surname> <given-names>N. N.</given-names></name> <name><surname>Al-Alimi</surname> <given-names>A.</given-names></name> <name><surname>Taiyeb-Ali</surname> <given-names>T.</given-names></name> <name><surname>Jaafar</surname> <given-names>N.</given-names></name></person-group> (<year>2015</year>). <article-title>Quantitative analysis of classical and new putative periodontal pathogens in subgingival biofilm: a case-control study.</article-title> <source><italic>J. Periodontal Res.</italic></source> <volume>50</volume> <fpage>320</fpage>&#x2013;<lpage>329</lpage>. <pub-id pub-id-type="doi">10.1111/jre.12210</pub-id></citation></ref>
<ref id="B4"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Al-hebshi</surname> <given-names>N. N.</given-names></name> <name><surname>Shuga-Aldin</surname> <given-names>H. M.</given-names></name> <name><surname>Al-Sharabi</surname> <given-names>A. K.</given-names></name> <name><surname>Ghandour</surname> <given-names>I.</given-names></name></person-group> (<year>2014</year>). <article-title>Subgingival periodontal pathogens associated with chronic periodontitis in Yemenis.</article-title> <source><italic>BMC Oral Health</italic></source> <volume>14</volume>:<issue>13</issue>. <pub-id pub-id-type="doi">10.1186/1472-6831-14-13</pub-id></citation></ref>
<ref id="B5"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Altman</surname> <given-names>D. G.</given-names></name> <name><surname>Royston</surname> <given-names>P.</given-names></name></person-group> (<year>2000</year>). <article-title>What do we mean by validating a prognostic model?</article-title> <source><italic>Stat. Med.</italic></source> <volume>19</volume> <fpage>453</fpage>&#x2013;<lpage>473</lpage>.</citation></ref>
<ref id="B6"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Armitage</surname> <given-names>G. C.</given-names></name></person-group> (<year>1999</year>). <article-title>Development of a classification system for periodontal diseases and conditions.</article-title> <source><italic>Ann. Periodontol.</italic></source> <volume>4</volume> <fpage>1</fpage>&#x2013;<lpage>6</lpage>. <pub-id pub-id-type="doi">10.1902/annals.1999.4.1.1</pub-id></citation></ref>
<ref id="B7"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Atieh</surname> <given-names>M. A.</given-names></name></person-group> (<year>2008</year>). <article-title>Accuracy of real-time polymerase chain reaction <italic>versus</italic> anaerobic culture in detection of Aggregatibacter actinomycetemcomitans and <italic>Porphyromonas gingivalis</italic>: a meta-analysis.</article-title> <source><italic>J. Periodontol.</italic></source> <volume>79</volume> <fpage>1620</fpage>&#x2013;<lpage>1629</lpage>. <pub-id pub-id-type="doi">10.1902/jop.2008.070668</pub-id></citation></ref>
<ref id="B8"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bansal</surname> <given-names>M.</given-names></name> <name><surname>Khatri</surname> <given-names>M.</given-names></name> <name><surname>Taneja</surname> <given-names>V.</given-names></name></person-group> (<year>2013</year>). <article-title>Potential role of periodontal infection in respiratory diseases - a review.</article-title> <source><italic>J. Med. Life</italic></source> <volume>6</volume> <fpage>244</fpage>&#x2013;<lpage>248</lpage>.</citation></ref>
<ref id="B9"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bao</surname> <given-names>K.</given-names></name> <name><surname>Belibasakis</surname> <given-names>G. N.</given-names></name> <name><surname>Thurnheer</surname> <given-names>T.</given-names></name> <name><surname>Aduse-Opoku</surname> <given-names>J.</given-names></name> <name><surname>Curtis</surname> <given-names>M. A.</given-names></name> <name><surname>Bostanci</surname> <given-names>N.</given-names></name></person-group> (<year>2014</year>). <article-title>Role of <italic>Porphyromonas gingivalis</italic> gingipains in multi-species biofilm formation.</article-title> <source><italic>BMC Microbiol.</italic></source> <volume>14</volume>:<issue>258</issue>. <pub-id pub-id-type="doi">10.1186/s12866-014-0258-7</pub-id></citation></ref>
<ref id="B10"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Benjamini</surname> <given-names>Y.</given-names></name> <name><surname>Hochberg</surname> <given-names>Y.</given-names></name></person-group> (<year>1995</year>). <article-title>Controlling the false discovery rate: a practical and powerful approach to multiple testing.</article-title> <source><italic>J. R. Stat. Soc. Series B (Methodological)</italic></source> <volume>57</volume> <fpage>289</fpage>&#x2013;<lpage>300</lpage>.</citation></ref>
<ref id="B11"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bou</surname> <given-names>G.</given-names></name> <name><surname>Fern&#x00E1;ndez</surname> <given-names>A.</given-names></name> <name><surname>Tom&#x00E1;s</surname> <given-names>M.</given-names></name></person-group> (<year>2014</year>). <article-title>Procedure for clinical diagnosis of an infectious disease using quantitative PCR, oligonucleotides marked from 8 to 9 nucleotides in length and the uracil-N- glycosylase of the atlantic cod. Patent No ES2449665</article-title>. <publisher-loc>Washington, DC</publisher-loc>: <publisher-name>U.S. Patent and Trademark Office</publisher-name>.</citation></ref>
<ref id="B12"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Boutaga</surname> <given-names>K.</given-names></name> <name><surname>Savelkoul</surname> <given-names>P. H.</given-names></name> <name><surname>Winkel</surname> <given-names>E. G.</given-names></name> <name><surname>Van Winkelhoff</surname> <given-names>A. J.</given-names></name></person-group> (<year>2007</year>). <article-title>Comparison of subgingival bacterial sampling with oral lavage for detection and quantification of periodontal pathogens by real-time polymerase chain reaction.</article-title> <source><italic>J. Periodontol.</italic></source> <volume>78</volume> <fpage>79</fpage>&#x2013;<lpage>86</lpage>. <pub-id pub-id-type="doi">10.1902/jop.2007.060078</pub-id></citation></ref>
<ref id="B13"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Boutaga</surname> <given-names>K.</given-names></name> <name><surname>Van Winkelhoff</surname> <given-names>A. J.</given-names></name> <name><surname>Vandenbroucke-Grauls</surname> <given-names>C. M.</given-names></name> <name><surname>Savelkoul</surname> <given-names>P. H.</given-names></name></person-group> (<year>2005</year>). <article-title>Periodontal pathogens: a quantitative comparison of anaerobic culture and real-time PCR.</article-title> <source><italic>FEMS Immunol. Med. Microbiol.</italic></source> <volume>45</volume> <fpage>191</fpage>&#x2013;<lpage>199</lpage>. <pub-id pub-id-type="doi">10.1016/j.femsim.2005.03.011</pub-id></citation></ref>
<ref id="B14"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Boutaga</surname> <given-names>K.</given-names></name> <name><surname>Van Winkelhoff</surname> <given-names>A. J.</given-names></name> <name><surname>Vandenbroucke-Grauls</surname> <given-names>C. M.</given-names></name> <name><surname>Savelkoul</surname> <given-names>P. H.</given-names></name></person-group> (<year>2006</year>). <article-title>The additional value of real-time PCR in the quantitative detection of periodontal pathogens.</article-title> <source><italic>J. Clin. Periodontol.</italic></source> <volume>33</volume> <fpage>427</fpage>&#x2013;<lpage>433</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-051X.2006.00925.x</pub-id></citation></ref>
<ref id="B15"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Braga</surname> <given-names>R. R.</given-names></name> <name><surname>Carvalho</surname> <given-names>M. A.</given-names></name> <name><surname>Bruna-Romero</surname> <given-names>O.</given-names></name> <name><surname>Teixeira</surname> <given-names>R. E.</given-names></name> <name><surname>Costa</surname> <given-names>J. E.</given-names></name> <name><surname>Mendes</surname> <given-names>E. N.</given-names></name><etal/></person-group> (<year>2010</year>). <article-title>Quantification of five putative periodontal pathogens in female patients with and without chronic periodontitis by real-time polymerase chain reaction.</article-title> <source><italic>Anaerobe</italic></source> <volume>16</volume> <fpage>234</fpage>&#x2013;<lpage>239</lpage>. <pub-id pub-id-type="doi">10.1016/j.anaerobe.2010.02.007</pub-id></citation></ref>
<ref id="B16"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Buduneli</surname> <given-names>N.</given-names></name> <name><surname>Kinane</surname> <given-names>D. F.</given-names></name></person-group> (<year>2011</year>). <article-title>Host-derived diagnostic markers related to soft tissue destruction and bone degradation in periodontitis.</article-title> <source><italic>J. Clin. Periodontol.</italic></source> <volume>38(Suppl. 11)</volume>, <fpage>85</fpage>&#x2013;<lpage>105</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-051X.2010.01670.x</pub-id></citation></ref>
<ref id="B17"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Byrne</surname> <given-names>S. J.</given-names></name> <name><surname>Dashper</surname> <given-names>S. G.</given-names></name> <name><surname>Darby</surname> <given-names>I. B.</given-names></name> <name><surname>Adams</surname> <given-names>G. G.</given-names></name> <name><surname>Hoffmann</surname> <given-names>B.</given-names></name> <name><surname>Reynolds</surname> <given-names>E. C.</given-names></name></person-group> (<year>2009</year>). <article-title>Progression of chronic periodontitis can be predicted by the levels of <italic>Porphyromonas gingivalis</italic> and <italic>Treponema denticola</italic> in subgingival plaque.</article-title> <source><italic>Oral Microbiol. Immunol.</italic></source> <volume>24</volume> <fpage>469</fpage>&#x2013;<lpage>477</lpage>. <pub-id pub-id-type="doi">10.1111/j.1399-302X.2009.00544.x</pub-id></citation></ref>
<ref id="B18"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Camelo-Castillo</surname> <given-names>A.</given-names></name> <name><surname>Mira</surname> <given-names>A.</given-names></name> <name><surname>Pico</surname> <given-names>A.</given-names></name> <name><surname>Nibali</surname> <given-names>L.</given-names></name> <name><surname>Henderson</surname> <given-names>B.</given-names></name> <name><surname>Donos</surname> <given-names>N.</given-names></name><etal/></person-group> (<year>2015a</year>). <article-title>Subgingival microbiota in health compared to periodontitis and the influence of smoking.</article-title> <source><italic>Front. Microbiol.</italic></source> <volume>6</volume>:<issue>119</issue>. <pub-id pub-id-type="doi">10.3389/fmicb.2015.00119</pub-id></citation></ref>
<ref id="B19"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Camelo-Castillo</surname> <given-names>A.</given-names></name> <name><surname>Novoa</surname> <given-names>L.</given-names></name> <name><surname>Balsa-Castro</surname> <given-names>C.</given-names></name> <name><surname>Blanco</surname> <given-names>J.</given-names></name> <name><surname>Mira</surname> <given-names>A.</given-names></name> <name><surname>Tom&#x00E1;s</surname> <given-names>I.</given-names></name></person-group> (<year>2015b</year>). <article-title>Relationship between periodontitis-associated subgingival microbiota and clinical inflammation by 16S pyrosequencing.</article-title> <source><italic>J. Clin. Periodontol.</italic></source> <volume>42</volume> <fpage>1074</fpage>&#x2013;<lpage>1082</lpage>. <pub-id pub-id-type="doi">10.1111/jcpe.12470</pub-id></citation></ref>
<ref id="B20"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Carstensen</surname> <given-names>B.</given-names></name> <name><surname>Plummer</surname> <given-names>M.</given-names></name> <name><surname>Laara</surname> <given-names>E.</given-names></name> <name><surname>Hills</surname> <given-names>M.</given-names></name></person-group> (<year>2017</year>). <source><italic>Epi: A Package for Statistical Analysis in Epidemiology. R Package Version</italic></source> 2.10. Available at: <ext-link ext-link-type="uri" xlink:href="http://CRAN.R-project.org/package=Epi">http://CRAN.R-project.org/package=Epi</ext-link></citation></ref>
<ref id="B21"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Casarin</surname> <given-names>R. C.</given-names></name> <name><surname>Ribeiro Edel</surname> <given-names>P.</given-names></name> <name><surname>Mariano</surname> <given-names>F. S.</given-names></name> <name><surname>Nociti</surname> <given-names>F. H.</given-names> <suffix>Jr.</suffix></name> <name><surname>Casati</surname> <given-names>M. Z.</given-names></name> <name><surname>Goncalves</surname> <given-names>R. B.</given-names></name></person-group> (<year>2010</year>). <article-title>Levels of Aggregatibacter actinomycetemcomitans, <italic>Porphyromonas gingivalis</italic>, inflammatory cytokines and species-specific immunoglobulin G in generalized aggressive and chronic periodontitis.</article-title> <source><italic>J. Periodontal Res.</italic></source> <volume>45</volume> <fpage>635</fpage>&#x2013;<lpage>642</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-0765.2010.01278.x</pub-id></citation></ref>
<ref id="B22"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Champlot</surname> <given-names>S.</given-names></name> <name><surname>Berthelot</surname> <given-names>C.</given-names></name> <name><surname>Pruvost</surname> <given-names>M.</given-names></name> <name><surname>Bennett</surname> <given-names>E. A.</given-names></name> <name><surname>Grange</surname> <given-names>T.</given-names></name> <name><surname>Geigl</surname> <given-names>E. M.</given-names></name></person-group> (<year>2010</year>). <article-title>An efficient multistrategy DNA decontamination procedure of PCR reagents for hypersensitive PCR applications.</article-title> <source><italic>PLoS ONE</italic></source> <volume>5</volume>:<issue>e13042</issue>. <pub-id pub-id-type="doi">10.1371/journal.pone.0013042</pub-id></citation></ref>
<ref id="B23"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chapple</surname> <given-names>I. L.</given-names></name> <name><surname>Genco</surname> <given-names>R.</given-names></name></person-group> <collab>Working group 2 of joint Efp/Aap workshop</collab> (<year>2013</year>). <article-title>Diabetes and periodontal diseases: consensus report of the joint EFP/AAP workshop on periodontitis and systemic diseases.</article-title> <source><italic>J. Clin. Periodontol.</italic></source> <volume>84(4 Suppl.)</volume>, <fpage>S106</fpage>&#x2013;<lpage>S112</lpage>. <pub-id pub-id-type="doi">10.1111/jcpe.12077</pub-id></citation></ref>
<ref id="B24"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Charalampakis</surname> <given-names>G.</given-names></name> <name><surname>Dahlen</surname> <given-names>G.</given-names></name> <name><surname>Carlen</surname> <given-names>A.</given-names></name> <name><surname>Leonhardt</surname> <given-names>A.</given-names></name></person-group> (<year>2013</year>). <article-title>Bacterial markers vs. clinical markers to predict progression of chronic periodontitis: a 2-yr prospective observational study.</article-title> <source><italic>Eur. J. Oral Sci.</italic></source> <volume>121</volume> <fpage>394</fpage>&#x2013;<lpage>402</lpage>. <pub-id pub-id-type="doi">10.1111/eos.12080</pub-id></citation></ref>
<ref id="B25"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Costalonga</surname> <given-names>M.</given-names></name> <name><surname>Herzberg</surname> <given-names>M. C.</given-names></name></person-group> (<year>2014</year>). <article-title>The oral microbiome and the immunobiology of periodontal disease and caries.</article-title> <source><italic>Immunol. Lett.</italic></source> <volume>162</volume> <fpage>22</fpage>&#x2013;<lpage>38</lpage>. <pub-id pub-id-type="doi">10.1016/j.imlet.2014.08.017</pub-id></citation></ref>
<ref id="B26"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>de Pablo</surname> <given-names>P.</given-names></name> <name><surname>Chapple</surname> <given-names>I. L.</given-names></name> <name><surname>Buckley</surname> <given-names>C. D.</given-names></name> <name><surname>Dietrich</surname> <given-names>T.</given-names></name></person-group> (<year>2009</year>). <article-title>Periodontitis in systemic rheumatic diseases.</article-title> <source><italic>Nat. Rev. Rheumatol.</italic></source> <volume>5</volume> <fpage>218</fpage>&#x2013;<lpage>224</lpage>. <pub-id pub-id-type="doi">10.1038/nrrheum.2009.28</pub-id></citation></ref>
<ref id="B27"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Decat</surname> <given-names>E.</given-names></name> <name><surname>Cosyn</surname> <given-names>J.</given-names></name> <name><surname>De Bruyn</surname> <given-names>H.</given-names></name> <name><surname>Miremadi</surname> <given-names>R.</given-names></name> <name><surname>Saerens</surname> <given-names>B.</given-names></name> <name><surname>Van Mechelen</surname> <given-names>E.</given-names></name><etal/></person-group> (<year>2012</year>). <article-title>Optimization of quantitative polymerase chain reactions for detection and quantification of eight periodontal bacterial pathogens.</article-title> <source><italic>BMC Res. Notes</italic></source> <volume>5</volume>:<issue>664</issue>. <pub-id pub-id-type="doi">10.1186/1756-0500-5-664</pub-id></citation></ref>
<ref id="B28"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Deeks</surname> <given-names>J. J.</given-names></name></person-group> (<year>1999</year>). <article-title>Using evaluations of diagnostic tests: understanding their limitations and making the most of available evidence.</article-title> <source><italic>Ann. Oncol.</italic></source> <volume>10</volume> <fpage>761</fpage>&#x2013;<lpage>768</lpage>. <pub-id pub-id-type="doi">10.1023/A:1008359805260</pub-id></citation></ref>
<ref id="B29"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dentino</surname> <given-names>A.</given-names></name> <name><surname>Lee</surname> <given-names>S.</given-names></name> <name><surname>Mailhot</surname> <given-names>J.</given-names></name> <name><surname>Hefti</surname> <given-names>A. F.</given-names></name></person-group> (<year>2013</year>). <article-title>Principles of periodontology.</article-title> <source><italic>Periodontology</italic></source> <volume>2000</volume> <fpage>16</fpage>&#x2013;<lpage>53</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-0757.2011.00397.x</pub-id></citation></ref>
<ref id="B30"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Efron</surname> <given-names>B.</given-names></name> <name><surname>Tibshirani</surname> <given-names>R. J.</given-names></name></person-group> (<year>1994</year>). <source><italic>An Introduction to the Bootstrap.</italic></source> <publisher-loc>Boca Raton, FL</publisher-loc>: <publisher-name>CRC Press</publisher-name>.</citation></ref>
<ref id="B31"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Faul</surname> <given-names>F.</given-names></name> <name><surname>Erdfelder</surname> <given-names>E.</given-names></name> <name><surname>Lang</surname> <given-names>A. G.</given-names></name> <name><surname>Buchner</surname> <given-names>A.</given-names></name></person-group> (<year>2007</year>). <article-title>G&#x2217;Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences.</article-title> <source><italic>Behav. Res. Methods</italic></source> <volume>39</volume> <fpage>175</fpage>&#x2013;<lpage>191</lpage>. <pub-id pub-id-type="doi">10.3758/BF03193146</pub-id></citation></ref>
<ref id="B32"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fern&#x00E1;ndez-Pinero</surname> <given-names>J.</given-names></name> <name><surname>Gallardo</surname> <given-names>C.</given-names></name> <name><surname>Elizalde</surname> <given-names>M.</given-names></name> <name><surname>Robles</surname> <given-names>A.</given-names></name> <name><surname>Gomez</surname> <given-names>C.</given-names></name> <name><surname>Bishop</surname> <given-names>R.</given-names></name><etal/></person-group> (<year>2013</year>). <article-title>Molecular diagnosis of african swine fever by a new real-time PCR using universal probe library.</article-title> <source><italic>Transbound. Emerg. Dis.</italic></source> <volume>60</volume> <fpage>48</fpage>&#x2013;<lpage>58</lpage>. <pub-id pub-id-type="doi">10.1111/j.1865-1682.2012.01317.x</pub-id></citation></ref>
<ref id="B33"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fraczyk</surname> <given-names>M.</given-names></name> <name><surname>Wozniakowski</surname> <given-names>G.</given-names></name> <name><surname>Kowalczyk</surname> <given-names>A.</given-names></name> <name><surname>Niemczuk</surname> <given-names>K.</given-names></name> <name><surname>Pejsak</surname> <given-names>Z.</given-names></name></person-group> (<year>2016</year>). <article-title>Development of cross-priming amplification for direct detection of the african swine fever virus, in pig and wild boar blood and sera samples.</article-title> <source><italic>Lett. Appl. Microbiol.</italic></source> <volume>62</volume> <fpage>386</fpage>&#x2013;<lpage>391</lpage>. <pub-id pub-id-type="doi">10.1111/lam.12569</pub-id></citation></ref>
<ref id="B34"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gatto</surname> <given-names>M. R.</given-names></name> <name><surname>Montevecchi</surname> <given-names>M.</given-names></name> <name><surname>Paolucci</surname> <given-names>M.</given-names></name> <name><surname>Landini</surname> <given-names>M. P.</given-names></name> <name><surname>Checchi</surname> <given-names>L.</given-names></name></person-group> (<year>2014</year>). <article-title>Prevalence of six periodontal pathogens in subgingival samples of italian patients with chronic periodontitis.</article-title> <source><italic>New Microbiol.</italic></source> <volume>37</volume> <fpage>517</fpage>&#x2013;<lpage>524</lpage>.</citation></ref>
<ref id="B35"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Giannobile</surname> <given-names>W. V.</given-names></name> <name><surname>Beikler</surname> <given-names>T.</given-names></name> <name><surname>Kinney</surname> <given-names>J. S.</given-names></name> <name><surname>Ramseier</surname> <given-names>C. A.</given-names></name> <name><surname>Morelli</surname> <given-names>T.</given-names></name> <name><surname>Wong</surname> <given-names>D. T.</given-names></name></person-group> (<year>2009</year>). <article-title>Saliva as a diagnostic tool for periodontal disease: current state and future directions.</article-title> <source><italic>Periodontology</italic></source> <volume>2000</volume> <fpage>52</fpage>&#x2013;<lpage>64</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-0757.2008.00288.x</pub-id></citation></ref>
<ref id="B36"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Glas</surname> <given-names>A. S.</given-names></name> <name><surname>Lijmer</surname> <given-names>J. G.</given-names></name> <name><surname>Prins</surname> <given-names>M. H.</given-names></name> <name><surname>Bonsel</surname> <given-names>G. J.</given-names></name> <name><surname>Bossuyt</surname> <given-names>P. M.</given-names></name></person-group> (<year>2003</year>). <article-title>The diagnostic odds ratio: a single indicator of test performance.</article-title> <source><italic>J. Clin. Epidemiol.</italic></source> <volume>56</volume> <fpage>1129</fpage>&#x2013;<lpage>1135</lpage>. <pub-id pub-id-type="doi">10.1016/S0895-4356(03)00177-X</pub-id></citation></ref>
<ref id="B37"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Golynska</surname> <given-names>M.</given-names></name> <name><surname>Polkowska</surname> <given-names>I.</given-names></name> <name><surname>Bartoszcze-Tomaszewska</surname> <given-names>M.</given-names></name></person-group> (<year>2016</year>). <article-title>Molecular evaluation of selected periodontal pathogens from subgingival region in canine and human periodontal disease.</article-title> <source><italic>J. Vet. Sci.</italic></source> <volume>18</volume> <fpage>51</fpage>&#x2013;<lpage>58</lpage>. <pub-id pub-id-type="doi">10.4142/jvs.2017.18.1.51</pub-id></citation></ref>
<ref id="B38"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Griffiths</surname> <given-names>G. S.</given-names></name></person-group> (<year>2003</year>). <article-title>Formation, collection and significance of gingival crevice fluid.</article-title> <source><italic>Periodontology</italic></source> <volume>2000</volume> <fpage>32</fpage>&#x2013;<lpage>42</lpage>. <pub-id pub-id-type="doi">10.1034/j.1600-0757.2003.03103.x</pub-id></citation></ref>
<ref id="B39"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Guglielmetti</surname> <given-names>M. R.</given-names></name> <name><surname>Rosa</surname> <given-names>E. F.</given-names></name> <name><surname>Lourencao</surname> <given-names>D. S.</given-names></name> <name><surname>Inoue</surname> <given-names>G.</given-names></name> <name><surname>Gomes</surname> <given-names>E. F.</given-names></name> <name><surname>De Micheli</surname> <given-names>G.</given-names></name><etal/></person-group> (<year>2014</year>). <article-title>Detection and quantification of periodontal pathogens in smokers and never-smokers with chronic periodontitis by real-time polymerase chain reaction.</article-title> <source><italic>J. Periodontol.</italic></source> <volume>85</volume> <fpage>1450</fpage>&#x2013;<lpage>1457</lpage>. <pub-id pub-id-type="doi">10.1902/jop.2014.140048</pub-id></citation></ref>
<ref id="B40"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Harrel</surname> <given-names>F. E.</given-names> <suffix>Jr.</suffix></name></person-group> (<year>2016</year>). <source><italic>Rms: Regression Modeling Strategies. R Package Version</italic></source> 4.<fpage>5</fpage>&#x2013;<lpage>0</lpage>. Available at: <ext-link ext-link-type="uri" xlink:href="http://CRAN.R-project.org/package=rms">http://CRAN.R-project.org/package=rms</ext-link></citation></ref>
<ref id="B41"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>He</surname> <given-names>K. Y.</given-names></name> <name><surname>Ge</surname> <given-names>D.</given-names></name> <name><surname>He</surname> <given-names>M. M.</given-names></name></person-group> (<year>2017</year>). <article-title>Big data analytics for genomic medicine.</article-title> <source><italic>Int. J. Mol. Sci.</italic></source> <volume>18</volume>:<issue>E412</issue>. <pub-id pub-id-type="doi">10.3390/ijms18020412</pub-id></citation></ref>
<ref id="B42"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Herrera</surname> <given-names>D.</given-names></name> <name><surname>Contreras</surname> <given-names>A.</given-names></name> <name><surname>Gamonal</surname> <given-names>J.</given-names></name> <name><surname>Oteo</surname> <given-names>A.</given-names></name> <name><surname>Jaramillo</surname> <given-names>A.</given-names></name> <name><surname>Silva</surname> <given-names>N.</given-names></name><etal/></person-group> (<year>2008</year>). <article-title>Subgingival microbial profiles in chronic periodontitis patients from Chile, Colombia and Spain.</article-title> <source><italic>J. Clin. Periodontol.</italic></source> <volume>35</volume> <fpage>106</fpage>&#x2013;<lpage>113</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-051X.2007.01170.x</pub-id></citation></ref>
<ref id="B43"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hosmer</surname> <given-names>D. W.</given-names> <suffix>Jr.</suffix></name> <name><surname>Lemeshow</surname> <given-names>S.</given-names></name> <name><surname>Sturdivant</surname> <given-names>R. X.</given-names></name></person-group> (<year>2013</year>). <source><italic>Applied Logistic Regression.</italic></source> <publisher-loc>Hoboken, NJ</publisher-loc>: <publisher-name>John Wiley &#x0026; Sons, Inc</publisher-name>.</citation></ref>
<ref id="B44"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Iasonos</surname> <given-names>A.</given-names></name> <name><surname>Schrag</surname> <given-names>D.</given-names></name> <name><surname>Raj</surname> <given-names>G. V.</given-names></name> <name><surname>Panageas</surname> <given-names>K. S.</given-names></name></person-group> (<year>2008</year>). <article-title>How to build and interpret a nomogram for cancer prognosis.</article-title> <source><italic>J. Clin. Oncol.</italic></source> <volume>26</volume> <fpage>1364</fpage>&#x2013;<lpage>1370</lpage>. <pub-id pub-id-type="doi">10.1200/JCO.2007.12.9791</pub-id></citation></ref>
<ref id="B45"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Jervoe-Storm</surname> <given-names>P. M.</given-names></name> <name><surname>Koltzscher</surname> <given-names>M.</given-names></name> <name><surname>Falk</surname> <given-names>W.</given-names></name> <name><surname>Dorfler</surname> <given-names>A.</given-names></name> <name><surname>Jepsen</surname> <given-names>S.</given-names></name></person-group> (<year>2005</year>). <article-title>Comparison of culture and real-time PCR for detection and quantification of five putative periodontopathogenic bacteria in subgingival plaque samples.</article-title> <source><italic>J. Clin. Periodontol.</italic></source> <volume>32</volume> <fpage>778</fpage>&#x2013;<lpage>783</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-051X.2005.00740.x</pub-id></citation></ref>
<ref id="B46"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kassebaum</surname> <given-names>N. J.</given-names></name> <name><surname>Bernabe</surname> <given-names>E.</given-names></name> <name><surname>Dahiya</surname> <given-names>M.</given-names></name> <name><surname>Bhandari</surname> <given-names>B.</given-names></name> <name><surname>Murray</surname> <given-names>C. J.</given-names></name> <name><surname>Marcenes</surname> <given-names>W.</given-names></name></person-group> (<year>2014</year>). <article-title>Global burden of severe periodontitis in 1990-2010: a systematic review and meta-regression.</article-title> <source><italic>J. Dent. Res.</italic></source> <volume>93</volume> <fpage>1045</fpage>&#x2013;<lpage>1053</lpage>. <pub-id pub-id-type="doi">10.1177/0022034514552491</pub-id></citation></ref>
<ref id="B47"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kim</surname> <given-names>Y. J.</given-names></name> <name><surname>Lee</surname> <given-names>S. H.</given-names></name></person-group> (<year>2014</year>). <article-title>Reducing the bioactivity of <italic>Tannerella forsythia</italic> lipopolysaccharide by <italic>Porphyromonas gingivalis</italic>.</article-title> <source><italic>J. Microbiol. (Seoul, Korea)</italic></source> <volume>52</volume> <fpage>702</fpage>&#x2013;<lpage>708</lpage>. <pub-id pub-id-type="doi">10.1007/s12275-014-4324-5</pub-id></citation></ref>
<ref id="B48"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kinney</surname> <given-names>J. S.</given-names></name> <name><surname>Morelli</surname> <given-names>T.</given-names></name> <name><surname>Oh</surname> <given-names>M.</given-names></name> <name><surname>Braun</surname> <given-names>T. M.</given-names></name> <name><surname>Ramseier</surname> <given-names>C. A.</given-names></name> <name><surname>Sugai</surname> <given-names>J. V.</given-names></name><etal/></person-group> (<year>2014</year>). <article-title>Crevicular fluid biomarkers and periodontal disease progression.</article-title> <source><italic>J. Clin. Periodontol.</italic></source> <volume>41</volume> <fpage>113</fpage>&#x2013;<lpage>120</lpage>. <pub-id pub-id-type="doi">10.1111/jcpe.12194</pub-id></citation></ref>
<ref id="B49"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Korte</surname> <given-names>D. L.</given-names></name> <name><surname>Kinney</surname> <given-names>J.</given-names></name></person-group> (<year>2016</year>). <article-title>Personalized medicine: an update of salivary biomarkers for periodontal diseases.</article-title> <source><italic>Periodontology</italic></source> <volume>2000</volume> <fpage>26</fpage>&#x2013;<lpage>37</lpage>. <pub-id pub-id-type="doi">10.1111/prd.12103</pub-id></citation></ref>
<ref id="B50"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kotsilkov</surname> <given-names>K.</given-names></name> <name><surname>Popova</surname> <given-names>C.</given-names></name> <name><surname>Boyanova</surname> <given-names>L.</given-names></name> <name><surname>Setchanova</surname> <given-names>L.</given-names></name> <name><surname>Mitov</surname> <given-names>I.</given-names></name></person-group> (<year>2015</year>). <article-title>Comparison of culture method and real-time PCR for detection of putative periodontopathogenic bacteria in deep periodontal pockets.</article-title> <source><italic>Biotechnol. Biotechnol. Equip.</italic></source> <volume>29</volume> <fpage>996</fpage>&#x2013;<lpage>1002</lpage>. <pub-id pub-id-type="doi">10.1080/13102818.2015.1058188</pub-id></citation></ref>
<ref id="B51"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kretschmar</surname> <given-names>S.</given-names></name> <name><surname>Yin</surname> <given-names>L.</given-names></name> <name><surname>Roberts</surname> <given-names>F.</given-names></name> <name><surname>London</surname> <given-names>R.</given-names></name> <name><surname>Flemmig</surname> <given-names>T. T.</given-names></name> <name><surname>Arushanov</surname> <given-names>D.</given-names></name><etal/></person-group> (<year>2012</year>). <article-title>Protease inhibitor levels in periodontal health and disease.</article-title> <source><italic>J. Periodontal Res.</italic></source> <volume>47</volume> <fpage>228</fpage>&#x2013;<lpage>235</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-0765.2011.01425.x</pub-id></citation></ref>
<ref id="B52"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lee</surname> <given-names>S. H.</given-names></name> <name><surname>Baek</surname> <given-names>D. H.</given-names></name></person-group> (<year>2013</year>). <article-title>Characteristics of <italic>Porphyromonas gingivalis</italic> lipopolysaccharide in co-culture with <italic>Fusobacterium nucleatum</italic>.</article-title> <source><italic>Mol. Oral Microbiol.</italic></source> <volume>28</volume> <fpage>230</fpage>&#x2013;<lpage>238</lpage>. <pub-id pub-id-type="doi">10.1111/omi.12020</pub-id></citation></ref>
<ref id="B53"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lele</surname> <given-names>S. R.</given-names></name> <name><surname>Keim</surname> <given-names>J. L.</given-names></name> <name><surname>Solymos</surname> <given-names>P.</given-names></name></person-group> (<year>2017</year>). <source><italic>Package &#x201C;ResourceSelection&#x201D;. R Package Version</italic></source> 0.<fpage>3</fpage>&#x2013;<lpage>2</lpage>. Available at: <ext-link ext-link-type="uri" xlink:href="https://github.com/psolymos/ResourceSelection">https://github.com/psolymos/ResourceSelection</ext-link>.</citation></ref>
<ref id="B54"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lin</surname> <given-names>F. Y.</given-names></name> <name><surname>Hsiao</surname> <given-names>F. P.</given-names></name> <name><surname>Huang</surname> <given-names>C. Y.</given-names></name> <name><surname>Shih</surname> <given-names>C. M.</given-names></name> <name><surname>Tsao</surname> <given-names>N. W.</given-names></name> <name><surname>Tsai</surname> <given-names>C. S.</given-names></name><etal/></person-group> (<year>2014</year>). <article-title><italic>Porphyromonas gingivalis</italic> GroEL induces osteoclastogenesis of periodontal ligament cells and enhances alveolar bone resorption in rats.</article-title> <source><italic>PLoS ONE</italic></source> <volume>9</volume>:<issue>e102450</issue>. <pub-id pub-id-type="doi">10.1371/journal.pone.0102450</pub-id></citation></ref>
<ref id="B55"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Loomer</surname> <given-names>P. M.</given-names></name></person-group> (<year>2004</year>). <article-title>Microbiological diagnostic testing in the treatment of periodontal diseases.</article-title> <source><italic>Periodontology 2000</italic></source> <volume>34</volume> <fpage>49</fpage>&#x2013;<lpage>56</lpage>. <pub-id pub-id-type="doi">10.1046/j.0906-6713.2002.003424.x</pub-id></citation></ref>
<ref id="B56"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Marchesan</surname> <given-names>J.</given-names></name> <name><surname>Jiao</surname> <given-names>Y. Z.</given-names></name> <name><surname>Schaff</surname> <given-names>R. A.</given-names></name> <name><surname>Hao</surname> <given-names>J.</given-names></name> <name><surname>Morelli</surname> <given-names>T.</given-names></name> <name><surname>Kinney</surname> <given-names>J. S.</given-names></name><etal/></person-group> (<year>2016</year>). <article-title>TLR4, NOD1 and NOD2 mediate immune recognition of putative newly identified periodontal pathogens.</article-title> <source><italic>Mol. Oral Microbiol.</italic></source> <volume>31</volume> <fpage>243</fpage>&#x2013;<lpage>258</lpage>. <pub-id pub-id-type="doi">10.1111/omi.12116</pub-id></citation></ref>
<ref id="B57"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Masunaga</surname> <given-names>H.</given-names></name> <name><surname>Tsutae</surname> <given-names>W.</given-names></name> <name><surname>Oh</surname> <given-names>H.</given-names></name> <name><surname>Shinozuka</surname> <given-names>N.</given-names></name> <name><surname>Kishimoto</surname> <given-names>N.</given-names></name> <name><surname>Ogata</surname> <given-names>Y.</given-names></name></person-group> (<year>2010</year>). <article-title>Use of quantitative PCR to evaluate methods of bacteria sampling in periodontal patients.</article-title> <source><italic>J. Oral Sci.</italic></source> <volume>52</volume> <fpage>615</fpage>&#x2013;<lpage>621</lpage>. <pub-id pub-id-type="doi">10.2334/josnusd.52.615</pub-id></citation></ref>
<ref id="B58"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Milne</surname> <given-names>T. J.</given-names></name> <name><surname>Coates</surname> <given-names>D. E.</given-names></name> <name><surname>Leichter</surname> <given-names>J. W.</given-names></name> <name><surname>Soo</surname> <given-names>L.</given-names></name> <name><surname>Williams</surname> <given-names>S. M.</given-names></name> <name><surname>Seymour</surname> <given-names>G. J.</given-names></name><etal/></person-group> (<year>2015</year>). <article-title>Periodontopathogen levels following the use of an er:YAG laser in the treatment of chronic periodontitis.</article-title> <source><italic>Aust. Dent. J.</italic></source> <pub-id pub-id-type="doi">10.1111/adj.12306</pub-id> <comment>[Epub ahead of print]</comment>.</citation></ref>
<ref id="B59"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mineoka</surname> <given-names>T.</given-names></name> <name><surname>Awano</surname> <given-names>S.</given-names></name> <name><surname>Rikimaru</surname> <given-names>T.</given-names></name> <name><surname>Kurata</surname> <given-names>H.</given-names></name> <name><surname>Yoshida</surname> <given-names>A.</given-names></name> <name><surname>Ansai</surname> <given-names>T.</given-names></name><etal/></person-group> (<year>2008</year>). <article-title>Site-specific development of periodontal disease is associated with increased levels of <italic>Porphyromonas gingivalis</italic>, <italic>Treponema denticola</italic>, and <italic>Tannerella forsythia</italic> in subgingival plaque.</article-title> <source><italic>J. Periodontol.</italic></source> <volume>79</volume> <fpage>670</fpage>&#x2013;<lpage>676</lpage>. <pub-id pub-id-type="doi">10.1902/jop.2008.070398</pub-id></citation></ref>
<ref id="B60"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Moons</surname> <given-names>K. G.</given-names></name> <name><surname>Altman</surname> <given-names>D. G.</given-names></name> <name><surname>Reitsma</surname> <given-names>J. B.</given-names></name> <name><surname>Ioannidis</surname> <given-names>J. P.</given-names></name> <name><surname>Macaskill</surname> <given-names>P.</given-names></name> <name><surname>Steyerberg</surname> <given-names>E. W.</given-names></name><etal/></person-group> (<year>2015</year>). <article-title>Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration.</article-title> <source><italic>Ann. Intern. Med.</italic></source> <volume>162</volume> <fpage>W1</fpage>&#x2013;<lpage>W73</lpage>. <pub-id pub-id-type="doi">10.7326/M14-0698</pub-id></citation></ref>
<ref id="B61"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Nomura</surname> <given-names>Y.</given-names></name> <name><surname>Shimada</surname> <given-names>Y.</given-names></name> <name><surname>Hanada</surname> <given-names>N.</given-names></name> <name><surname>Numabe</surname> <given-names>Y.</given-names></name> <name><surname>Kamoi</surname> <given-names>K.</given-names></name> <name><surname>Sato</surname> <given-names>T.</given-names></name><etal/></person-group> (<year>2012</year>). <article-title>Salivary biomarkers for predicting the progression of chronic periodontitis.</article-title> <source><italic>Arch. Oral Biol.</italic></source> <volume>57</volume> <fpage>413</fpage>&#x2013;<lpage>420</lpage>. <pub-id pub-id-type="doi">10.1016/j.archoralbio.2011.09.011</pub-id></citation></ref>
<ref id="B62"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Nonnenmacher</surname> <given-names>C.</given-names></name> <name><surname>Dalpke</surname> <given-names>A.</given-names></name> <name><surname>Zimmermann</surname> <given-names>S.</given-names></name> <name><surname>Flores-De-Jacoby</surname> <given-names>L.</given-names></name> <name><surname>Mutters</surname> <given-names>R.</given-names></name> <name><surname>Heeg</surname> <given-names>K.</given-names></name></person-group> (<year>2003</year>). <article-title>DNA from periodontopathogenic bacteria is immunostimulatory for mouse and human immune cells.</article-title> <source><italic>Infect. Immun.</italic></source> <volume>71</volume> <fpage>850</fpage>&#x2013;<lpage>856</lpage>. <pub-id pub-id-type="doi">10.1128/IAI.71.2.850-856.2003</pub-id></citation></ref>
<ref id="B63"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>O&#x2019;brien</surname> <given-names>R. M.</given-names></name></person-group> (<year>2007</year>). <article-title>A caution regarding rules of thumb for variance inflation factors.</article-title> <source><italic>Qual. Quant.</italic></source> <volume>41</volume> <fpage>673</fpage>&#x2013;<lpage>690</lpage>. <pub-id pub-id-type="doi">10.1007/s11135-006-9018-6</pub-id></citation></ref>
<ref id="B64"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Page</surname> <given-names>R. C.</given-names></name> <name><surname>Eke</surname> <given-names>P. I.</given-names></name></person-group> (<year>2007</year>). <article-title>Case definitions for use in population-based surveillance of periodontitis.</article-title> <source><italic>J. Periodontol.</italic></source> <volume>78</volume> <fpage>1387</fpage>&#x2013;<lpage>1399</lpage>. <pub-id pub-id-type="doi">10.1902/jop.2007.060264</pub-id></citation></ref>
<ref id="B65"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Polonyi</surname> <given-names>M.</given-names></name> <name><surname>Prenninger</surname> <given-names>N.</given-names></name> <name><surname>Arweiler</surname> <given-names>N. B.</given-names></name> <name><surname>Haririan</surname> <given-names>H.</given-names></name> <name><surname>Winklehner</surname> <given-names>P.</given-names></name> <name><surname>Kierstein</surname> <given-names>S.</given-names></name></person-group> (<year>2013</year>). <article-title>Assessment of viable periodontal pathogens by reverse transcription quantitative polymerase chain reaction.</article-title> <source><italic>J. Periodontal Res.</italic></source> <volume>48</volume> <fpage>671</fpage>&#x2013;<lpage>676</lpage>. <pub-id pub-id-type="doi">10.1111/jre.12052</pub-id></citation></ref>
<ref id="B66"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Psoter</surname> <given-names>W. J.</given-names></name> <name><surname>Ge</surname> <given-names>Y.</given-names></name> <name><surname>Russell</surname> <given-names>S. L.</given-names></name> <name><surname>Chen</surname> <given-names>Z.</given-names></name> <name><surname>Katz</surname> <given-names>R. V.</given-names></name> <name><surname>Jean-Charles</surname> <given-names>G.</given-names></name><etal/></person-group> (<year>2011</year>). <article-title>PCR detection of <italic>Streptococcus mutans</italic> and Aggregatibacter actinomycetemcomitans in dental plaque samples from Haitian adolescents.</article-title> <source><italic>Clin. Oral Investig.</italic></source> <volume>15</volume> <fpage>461</fpage>&#x2013;<lpage>469</lpage>. <pub-id pub-id-type="doi">10.1007/s00784-010-0413-y</pub-id></citation></ref>
<ref id="B67"><citation citation-type="journal"><collab>R Core Team</collab> (<year>2016</year>). <source><italic>R: A Language and Environment for Statistical Computing.</italic></source> <publisher-loc>Vienna</publisher-loc>: <publisher-name>R foundation for statistical computing</publisher-name>.</citation></ref>
<ref id="B68"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ramseier</surname> <given-names>C. A.</given-names></name> <name><surname>Kinney</surname> <given-names>J. S.</given-names></name> <name><surname>Herr</surname> <given-names>A. E.</given-names></name> <name><surname>Braun</surname> <given-names>T.</given-names></name> <name><surname>Sugai</surname> <given-names>J. V.</given-names></name> <name><surname>Shelburne</surname> <given-names>C. A.</given-names></name><etal/></person-group> (<year>2009</year>). <article-title>Identification of pathogen and host-response markers correlated with periodontal disease.</article-title> <source><italic>J. Periodontol.</italic></source> <volume>80</volume> <fpage>436</fpage>&#x2013;<lpage>446</lpage>. <pub-id pub-id-type="doi">10.1902/jop.2009.080480</pub-id></citation></ref>
<ref id="B69"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Robin</surname> <given-names>X.</given-names></name> <name><surname>Turck</surname> <given-names>N.</given-names></name> <name><surname>Hainard</surname> <given-names>A.</given-names></name> <name><surname>Tiberti</surname> <given-names>N.</given-names></name> <name><surname>Lisacek</surname> <given-names>F.</given-names></name> <name><surname>Sanchez</surname> <given-names>J. C.</given-names></name><etal/></person-group> (<year>2011</year>). <article-title>pROC: An open-source package for R and S+ to analyze and compare ROC curves.</article-title> <source><italic>BMC Bioinformatics</italic></source> <volume>12</volume>:<issue>77</issue>. <pub-id pub-id-type="doi">10.1186/1471-2105-12-77</pub-id></citation></ref>
<ref id="B70"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sakamoto</surname> <given-names>M.</given-names></name> <name><surname>Umeda</surname> <given-names>M.</given-names></name> <name><surname>Benno</surname> <given-names>Y.</given-names></name></person-group> (<year>2005</year>). <article-title>Molecular analysis of human oral microbiota.</article-title> <source><italic>J. Periodontal Res.</italic></source> <volume>40</volume> <fpage>277</fpage>&#x2013;<lpage>285</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-0765.2005.00793.x</pub-id></citation></ref>
<ref id="B71"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Saygun</surname> <given-names>I.</given-names></name> <name><surname>Nizam</surname> <given-names>N.</given-names></name> <name><surname>Keskiner</surname> <given-names>I.</given-names></name> <name><surname>Bal</surname> <given-names>V.</given-names></name> <name><surname>Kubar</surname> <given-names>A.</given-names></name> <name><surname>Acikel</surname> <given-names>C.</given-names></name><etal/></person-group> (<year>2011</year>). <article-title>Salivary infectious agents and periodontal disease status.</article-title> <source><italic>J. Periodontal Res.</italic></source> <volume>46</volume> <fpage>235</fpage>&#x2013;<lpage>239</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-0765.2010.01335.x</pub-id></citation></ref>
<ref id="B72"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Scapoli</surname> <given-names>L.</given-names></name> <name><surname>Girardi</surname> <given-names>A.</given-names></name> <name><surname>Palmieri</surname> <given-names>A.</given-names></name> <name><surname>Martinelli</surname> <given-names>M.</given-names></name> <name><surname>Cura</surname> <given-names>F.</given-names></name> <name><surname>Lauritano</surname> <given-names>D.</given-names></name><etal/></person-group> (<year>2015</year>). <article-title>Quantitative analysis of periodontal pathogens in periodontitis and gingivitis.</article-title> <source><italic>J. Biol. Regul. Homeost. Agents</italic></source> <volume>29</volume> <fpage>101</fpage>&#x2013;<lpage>110</lpage>.</citation></ref>
<ref id="B73"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Shanbhag</surname> <given-names>S.</given-names></name> <name><surname>Dahiya</surname> <given-names>M.</given-names></name> <name><surname>Croucher</surname> <given-names>R.</given-names></name></person-group> (<year>2012</year>). <article-title>The impact of periodontal therapy on oral health-related quality of life in adults: a systematic review.</article-title> <source><italic>J. Clin. Periodontol.</italic></source> <volume>39</volume> <fpage>725</fpage>&#x2013;<lpage>735</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-051X.2012.01910.x</pub-id></citation></ref>
<ref id="B74"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Socransky</surname> <given-names>S. S.</given-names></name> <name><surname>Haffajee</surname> <given-names>A. D.</given-names></name> <name><surname>Cugini</surname> <given-names>M. A.</given-names></name> <name><surname>Smith</surname> <given-names>C.</given-names></name> <name><surname>Kent</surname> <given-names>R. L.</given-names> <suffix>Jr.</suffix></name></person-group> (<year>1998</year>). <article-title>Microbial complexes in subgingival plaque.</article-title> <source><italic>J. Clin. Periodontol.</italic></source> <volume>25</volume> <fpage>134</fpage>&#x2013;<lpage>144</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-051X.1998.tb02419.x</pub-id></citation></ref>
<ref id="B75"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Steyerberg</surname> <given-names>E. W.</given-names></name> <name><surname>Eijkemans</surname> <given-names>M. J. C.</given-names></name> <name><surname>Habbema</surname> <given-names>J. D. F.</given-names></name></person-group> (<year>2001a</year>). <article-title>Application of shrinkage techniques in logistic regression analysis: a case study.</article-title> <source><italic>Stat. Neerl.</italic></source> <volume>55</volume> <fpage>76</fpage>&#x2013;<lpage>88</lpage>. <pub-id pub-id-type="doi">10.1111/1467-9574.00157</pub-id></citation></ref>
<ref id="B76"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Steyerberg</surname> <given-names>E. W.</given-names></name> <name><surname>Harrell</surname> <given-names>F. E.</given-names> <suffix>Jr.</suffix></name> <name><surname>Borsboom</surname> <given-names>G. J.</given-names></name> <name><surname>Eijkemans</surname> <given-names>M. J.</given-names></name> <name><surname>Vergouwe</surname> <given-names>Y.</given-names></name> <name><surname>Habbema</surname> <given-names>J. D.</given-names></name></person-group> (<year>2001b</year>). <article-title>Internal validation of predictive models: efficiency of some procedures for logistic regression analysis.</article-title> <source><italic>J. Clin. Epidemiol.</italic></source> <volume>54</volume> <fpage>774</fpage>&#x2013;<lpage>781</lpage>. <pub-id pub-id-type="doi">10.1016/S0895-4356(01)00341-9</pub-id></citation></ref>
<ref id="B77"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Teixeira</surname> <given-names>S. R.</given-names></name> <name><surname>Mattarazo</surname> <given-names>F.</given-names></name> <name><surname>Feres</surname> <given-names>M.</given-names></name> <name><surname>Figueiredo</surname> <given-names>L. C.</given-names></name> <name><surname>de Faveri</surname> <given-names>M.</given-names></name> <name><surname>Simionato</surname> <given-names>M. R.</given-names></name><etal/></person-group> (<year>2009</year>). <article-title>Quantification of <italic>Porphyromonas gingivalis</italic> and fimA genotypes in smoker chronic periodontitis.</article-title> <source><italic>J. Clin. Periodontol.</italic></source> <volume>36</volume> <fpage>482</fpage>&#x2013;<lpage>487</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-051X.2009.01411.x</pub-id></citation></ref>
<ref id="B78"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Teles</surname> <given-names>R.</given-names></name> <name><surname>Teles</surname> <given-names>F.</given-names></name> <name><surname>Frias-Lopez</surname> <given-names>J.</given-names></name> <name><surname>Paster</surname> <given-names>B.</given-names></name> <name><surname>Haffajee</surname> <given-names>A.</given-names></name></person-group> (<year>2013</year>). <article-title>Lessons learned and unlearned in periodontal microbiology.</article-title> <source><italic>Periodontology 2000</italic></source> <volume>62</volume> <fpage>95</fpage>&#x2013;<lpage>162</lpage>. <pub-id pub-id-type="doi">10.1111/prd.12010</pub-id></citation></ref>
<ref id="B79"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tomita</surname> <given-names>S.</given-names></name> <name><surname>Komiya-Ito</surname> <given-names>A.</given-names></name> <name><surname>Imamura</surname> <given-names>K.</given-names></name> <name><surname>Kita</surname> <given-names>D.</given-names></name> <name><surname>Ota</surname> <given-names>K.</given-names></name> <name><surname>Takayama</surname> <given-names>S.</given-names></name><etal/></person-group> (<year>2013</year>). <article-title>Prevalence of Aggregatibacter actinomycetemcomitans, <italic>Porphyromonas gingivalis</italic> and <italic>Tannerella forsythia</italic> in japanese patients with generalized chronic and aggressive periodontitis.</article-title> <source><italic>Microb. Pathog.</italic></source> <volume>6</volume> <fpage>11</fpage>&#x2013;<lpage>15</lpage>. <pub-id pub-id-type="doi">10.1016/j.micpath.2013.04.006</pub-id></citation></ref>
<ref id="B80"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tonetti</surname> <given-names>M. S.</given-names></name> <name><surname>Van Dyke</surname> <given-names>T. E.</given-names></name></person-group> <collab>Working group 1 of the joint Efp/Aap workshop</collab> (<year>2013</year>). <article-title>Periodontitis and atherosclerotic cardiovascular disease: consensus report of the joint EFP/AAP workshop on periodontitis and systemic diseases.</article-title> <source><italic>J. Clin. Periodontol.</italic></source> <volume>40(Suppl. 14)</volume>, <fpage>S24</fpage>&#x2013;<lpage>S29</lpage>. <pub-id pub-id-type="doi">10.1111/jcpe.12089</pub-id></citation></ref>
<ref id="B81"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Torrungruang</surname> <given-names>K.</given-names></name> <name><surname>Jitpakdeebordin</surname> <given-names>S.</given-names></name> <name><surname>Charatkulangkun</surname> <given-names>O.</given-names></name> <name><surname>Gleebbua</surname> <given-names>Y.</given-names></name></person-group> (<year>2015</year>). <article-title><italic>Porphyromonas gingivalis</italic>, Aggregatibacter actinomycetemcomitans, and <italic>Treponema denticola</italic>/Prevotella intermedia co-infection are associated with severe periodontitis in a thai population.</article-title> <source><italic>PLoS ONE</italic></source> <volume>10</volume>:<issue>e0136646</issue>. <pub-id pub-id-type="doi">10.1371/journal.pone.0136646</pub-id></citation></ref>
<ref id="B82"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wang</surname> <given-names>Q.</given-names></name> <name><surname>Wright</surname> <given-names>C. J.</given-names></name> <name><surname>Dingming</surname> <given-names>H.</given-names></name> <name><surname>Uriarte</surname> <given-names>S. M.</given-names></name> <name><surname>Lamont</surname> <given-names>R. J.</given-names></name></person-group> (<year>2013</year>). <article-title>Oral community interactions of <italic>Filifactor alocis</italic> in vitro.</article-title> <source><italic>PLoS ONE</italic></source> <volume>8</volume>:<issue>e76271</issue>. <pub-id pub-id-type="doi">10.1371/journal.pone.0076271</pub-id></citation></ref>
<ref id="B83"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wara-aswapati</surname> <given-names>N.</given-names></name> <name><surname>Pitiphat</surname> <given-names>W.</given-names></name> <name><surname>Chanchaimongkon</surname> <given-names>L.</given-names></name> <name><surname>Taweechaisupapong</surname> <given-names>S.</given-names></name> <name><surname>Boch</surname> <given-names>J. A.</given-names></name> <name><surname>Ishikawa</surname> <given-names>I.</given-names></name></person-group> (<year>2009</year>). <article-title>Red bacterial complex is associated with the severity of chronic periodontitis in a Thai population.</article-title> <source><italic>Oral Dis.</italic></source> <volume>15</volume> <fpage>354</fpage>&#x2013;<lpage>359</lpage>. <pub-id pub-id-type="doi">10.1111/j.1601-0825.2009.01562.x</pub-id></citation></ref>
<ref id="B84"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Willi</surname> <given-names>M.</given-names></name> <name><surname>Belibasakis</surname> <given-names>G. N.</given-names></name> <name><surname>Bostanci</surname> <given-names>N.</given-names></name></person-group> (<year>2014</year>). <article-title>Expression and regulation of triggering receptor expressed on myeloid cells 1 in periodontal diseases.</article-title> <source><italic>Clin. Exp. Immunol.</italic></source> <volume>178</volume> <fpage>190</fpage>&#x2013;<lpage>200</lpage>. <pub-id pub-id-type="doi">10.1111/cei.12397</pub-id></citation></ref>
<ref id="B85"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yang</surname> <given-names>X.</given-names></name> <name><surname>Li</surname> <given-names>C.</given-names></name> <name><surname>Pan</surname> <given-names>Y.</given-names></name></person-group> (<year>2016</year>). <article-title>The influences of periodontal status and periodontal pathogen quantity on salivary 8-hydroxydeoxyguanosine and interleukin-17 levels.</article-title> <source><italic>J. Periodontol.</italic></source> <volume>87</volume> <fpage>591</fpage>&#x2013;<lpage>600</lpage>. <pub-id pub-id-type="doi">10.1902/jop.2015.150390</pub-id></citation></ref>
<ref id="B86"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zhang</surname> <given-names>L.</given-names></name> <name><surname>Henson</surname> <given-names>B. S.</given-names></name> <name><surname>Camargo</surname> <given-names>P. M.</given-names></name> <name><surname>Wong</surname> <given-names>D. T.</given-names></name></person-group> (<year>2009</year>). <article-title>The clinical value of salivary biomarkers for periodontal disease.</article-title> <source><italic>Periodontology</italic></source> <volume>2000</volume> <fpage>25</fpage>&#x2013;<lpage>37</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-0757.2009.00315.x</pub-id></citation></ref>
<ref id="B87"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zorina</surname> <given-names>O. A.</given-names></name> <name><surname>Kulakov</surname> <given-names>A. A.</given-names></name> <name><surname>Boriskina</surname> <given-names>O. A.</given-names></name> <name><surname>Rebrikov</surname> <given-names>D. V.</given-names></name></person-group> (<year>2011</year>). <article-title>Relationship between the pathogenic representatives of periodontal pockets microbiocenosis in patients with periodontitis with varying degrees of severity.</article-title> <source><italic>Acta Naturae</italic></source> <volume>3</volume> <fpage>99</fpage>&#x2013;<lpage>102</lpage>.</citation></ref>
</ref-list>
</back>
</article>