<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="review-article">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Med.</journal-id>
<journal-title>Frontiers in Medicine</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Med.</abbrev-journal-title>
<issn pub-type="epub">2296-858X</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fmed.2021.642380</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Medicine</subject>
<subj-group>
<subject>Review</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Molecular Biomarkers for Melanoma Screening, Diagnosis and Prognosis: Current State and Future Prospects</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<name><surname>Deacon</surname> <given-names>Dekker C.</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="corresp" rid="c001"><sup>&#x0002A;</sup></xref>
<xref ref-type="author-notes" rid="fn002"><sup>&#x02020;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/1218882/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Smith</surname> <given-names>Eric A.</given-names></name>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
<xref ref-type="author-notes" rid="fn002"><sup>&#x02020;</sup></xref>
</contrib>
<contrib contrib-type="author" corresp="yes">
<name><surname>Judson-Torres</surname> <given-names>Robert L.</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="aff" rid="aff3"><sup>3</sup></xref>
<xref ref-type="corresp" rid="c002"><sup>&#x0002A;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/881794/overview"/>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Department of Dermatology, University of Utah</institution>, <addr-line>Salt Lake City, UT</addr-line>, <country>United States</country></aff>
<aff id="aff2"><sup>2</sup><institution>Department of Pathology, University of Utah</institution>, <addr-line>Salt Lake City, UT</addr-line>, <country>United States</country></aff>
<aff id="aff3"><sup>3</sup><institution>Huntsman Cancer Institute</institution>, <addr-line>Salt Lake City, UT</addr-line>, <country>United States</country></aff>
<author-notes>
<fn fn-type="edited-by"><p>Edited by: Mitchell Stark, The University of Queensland, Australia</p></fn>
<fn fn-type="edited-by"><p>Reviewed by: Oleg E. Akilov, University of Pittsburgh, United States; Nicola Pimpinelli, University of Florence, Italy</p></fn>
<corresp id="c001">&#x0002A;Correspondence: Dekker C. Deacon <email>Dekker.Deacon&#x00040;hsc.utah.edu</email></corresp>
<corresp id="c002">Robert L. Judson-Torres <email>Robert.judson-torres&#x00040;hci.utah.edu</email></corresp>
<fn fn-type="other" id="fn001"><p>This article was submitted to Dermatology, a section of the journal Frontiers in Medicine</p></fn>
<fn fn-type="other" id="fn002"><p>&#x02020;These authors have contributed equally to this work</p></fn></author-notes>
<pub-date pub-type="epub">
<day>16</day>
<month>04</month>
<year>2021</year>
</pub-date>
<pub-date pub-type="collection">
<year>2021</year>
</pub-date>
<volume>8</volume>
<elocation-id>642380</elocation-id>
<history>
<date date-type="received">
<day>15</day>
<month>12</month>
<year>2020</year>
</date>
<date date-type="accepted">
<day>17</day>
<month>03</month>
<year>2021</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x000A9; 2021 Deacon, Smith and Judson-Torres.</copyright-statement>
<copyright-year>2021</copyright-year>
<copyright-holder>Deacon, Smith and Judson-Torres</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p></license> </permissions>
<abstract><p>Despite significant progress in the development of treatment options, melanoma remains a leading cause of death due to skin cancer. Advances in our understanding of the genetic, transcriptomic, and morphologic spectrum of benign and malignant melanocytic neoplasia have enabled the field to propose biomarkers with potential diagnostic, prognostic, and predictive value. While these proposed biomarkers have the potential to improve clinical decision making at multiple critical intervention points, most remain unvalidated. Clinical validation of even the most commonly assessed biomarkers will require substantial resources, including limited clinical specimens. It is therefore important to consider the properties that constitute a relevant and clinically-useful biomarker-based test prior to engaging in large validation studies. In this review article we adapt an established framework for determining minimally-useful biomarker test characteristics, and apply this framework to a discussion of currently used and proposed biomarkers designed to aid melanoma detection, staging, prognosis, and choice of treatment.</p></abstract>
<kwd-group>
<kwd>melanoma</kwd>
<kwd>biomarkers</kwd>
<kwd>diagnostics</kwd>
<kwd>prognostics</kwd>
<kwd>likelihood ratio</kwd>
</kwd-group>
<counts>
<fig-count count="5"/>
<table-count count="4"/>
<equation-count count="0"/>
<ref-count count="127"/>
<page-count count="26"/>
<word-count count="17821"/>
</counts>
</article-meta>
</front>
<body>

<sec sec-type="intro" id="s1">
<title>Introduction</title>
<p>At a fundamental level, clinical tests are designed to aid clinicians in determining a best course of action when there is clinical uncertainty. These tests aim to improve certainty by ruling out a disease state, confirming a disease state, delineating prognosis, and/or identifying the most appropriate treatment regimen. Toward these goals, specific laboratory tests interrogate biomarkers: biological molecules that have clinically-relevant associations with a specific disease state. However, even a biomarker with an established disease state association may not be useful if it does not influence clinical decision-making. There are numerous factors that determine clinical utility: pre-test certainty of disease state, risks associated with making a clinical decision, cost, turn-around time, and the unique circumstances of each patient. Since biomarker development, from discovery to validation, requires substantial resources, investigators should carefully consider what characteristics are necessary for a useful clinical test at each stage of development.</p>
<p>With melanoma and other neoplasms, clinicians seek to balance the efficacy and potential side effects of an intervention with the estimated risk of disease progression. This risk has been traditionally inferred using prognostic tumor staging criteria, which takes into account tumor size, extent, nodal involvement, distant metastases, and histopathological grade and type. In more recent revisions of National Comprehensive Cancer Network (NCCN) and American Joint Committee on Cancer (AJCC) guidelines, select biomarkers have augmented traditional staging algorithms. The level of serum LDH is currently incorporated as a staging (prognostic) factor for malignant melanoma (<xref ref-type="bibr" rid="B1">1</xref>), and the mutational status of the <italic>BRAF</italic> proto-oncogene is predictive of therapeutic response (<xref ref-type="bibr" rid="B2">2</xref>&#x02013;<xref ref-type="bibr" rid="B4">4</xref>). However, many critical intervention points in melanoma care remain fraught with uncertainty. Critical shortcomings in diagnosing, staging, and recurrence monitoring for melanoma could be alleviated with appropriate biomarker development. Toward this goal, many biomarker tests have been proposed, some of which are commercially available or are performed in academic referral centers. Presently, none of these tests have been compared to the current standard of care with large scale, randomized, prospective, multi-center, and independently validated studies with long-term clinical follow up.</p>
<p>For this literature review, we summarize the established theoretical framework for determining the minimal test characteristics required to potentially alter clinical decision making at different stages of melanoma care. We then apply this theory to evaluate currently utilized and proposed melanoma biomarkers. A recent review from the Melanoma Prevention Working Group critiqued the rigor in which commercially available prognostic tests have been validated and provided recommendations for care providers (<xref ref-type="bibr" rid="B5">5</xref>). Although we touch upon similar analyses here, our goal is to review a larger spectrum of biomarkers for melanoma care&#x02014;preliminary to practiced&#x02014;from the perspective of prioritizing further development. Given the substantial resources required for clinical test validation, we hope this discussion will aid the field in deciding which existing tests warrant further validation and which decision points are in most need of further biomarker discovery.</p>
<sec>
<title>The Case for Biomarkers in Melanoma Diagnosis and Treatment</title>
<p>Despite recent advances in treatment of advanced stage melanoma, particularly with the expansion of the use of immune checkpoint inhibitors (<xref ref-type="bibr" rid="B6">6</xref>), melanoma continues to confer significant morbidity and mortality. In 2015, there were 59,782 deaths attributed to melanoma worldwide (<xref ref-type="bibr" rid="B7">7</xref>). Nearly 1.6% of all cancer diagnoses are melanoma, and the disease accounts for 0.64% of cancer deaths (<xref ref-type="bibr" rid="B8">8</xref>). In the United States specifically, it is expected that there will be over 100,000 new diagnoses of melanoma in 2020 with almost 7,000 deaths attributed to melanoma (<xref ref-type="bibr" rid="B9">9</xref>). In Europe, while cost of treatment varies widely based on country and stage, melanoma treatment can range from several thousand to tens of thousands of Euros per patient on average, and these costs are expected to increase with wider adoption of immune checkpoint inhibitors and targeted kinase inhibitors (<xref ref-type="bibr" rid="B10">10</xref>).</p>
<p>Varied and unique uncertainties surround clinical decision-making during the detection, diagnosis, and treatment of melanocytic tumors. We have reviewed and analyzed four critical decision points during the identification and treatment of melanoma for which biomarkers that reduce uncertainty have been investigated. These decision points are: (i) deciding whether to biopsy a melanocytic neoplasm, (ii) confirming histopathologic diagnosis, (iii) stratifying risk for lymphatic spread with consideration for SLNB and, (iv) selecting systemic therapy. While this review focuses on cutaneous melanoma, the considerations discussed could be adapted to potential biomarkers for mucosal and uveal melanoma as well.</p>
</sec>
<sec>
<title>Histopathology: the 14-Karat Gold Standard for Diagnosing Melanocytic Lesions</title>
<p>One poignant example of where biomarkers could dramatically enhance melanoma care is with the histopathologic diagnosis of melanoma. Histopathologic diagnosis is the gold standard for melanoma diagnosis, but, despite the advent of immunohistochemistry (IHC) and standardization of diagnostic criteria (<xref ref-type="bibr" rid="B11">11</xref>&#x02013;<xref ref-type="bibr" rid="B13">13</xref>), it remains a subjective medical art constrained by significant intra- and interobserver variability. While nuances exist, melanocytic proliferations largely exist on a spectrum from histologically benign to malignant, as described in the MPATH-Dx Classification scheme (<xref ref-type="table" rid="T1">Table 1</xref>) (<xref ref-type="bibr" rid="B11">11</xref>). This classification scheme breaks melanocytic lesions into benign (class I), moderately dysplastic nevus and Spitz nevus (class II), severely dysplastic nevus, atypical Spitz nevi (class III), AJCC T1a and T1b invasive melanomas (class IV), and AJCC T1b-T4a invasive melanomas (class V). In the absence of metastasis identified during sentinel lymph node biopsy (SLNB) or further staging, these primary melanomas are considered Stage I-II using the AJCC 8th Edition Pathological Staging Criteria (<xref ref-type="bibr" rid="B14">14</xref>).</p>
<table-wrap position="float" id="T1">
<label>Table 1</label>
<caption><p>Abbreviated MPATH-DX classification and AJCC 8th edition staging criteria for cutaneous melanoma.</p></caption>
<graphic xlink:href="fmed-08-642380-i0001.tif"/>
</table-wrap>
<p>Although lesions at the extremes of the spectrum (class I and class V lesions) are most likely to be reproducibly diagnosed as such, there remains significant diagnostic discordance among dermatopathologists for each class. In one study evaluating 187 pathologists, MPATH-Dx class I and V have intraobserver concordance rates of 77 and 67%, respectively, and interobserver concordance rates of 71 and 55%, respectively. When compared to consensus diagnosis, the accuracies of individual pathologists were 92% for class I and 72% for class V (<xref ref-type="bibr" rid="B15">15</xref>). Separating class II, class III, and class IV is a greater diagnostic challenge. Class II have the poorest concordance (35% intraobserver, 25% interobserver) and consensus accuracy (25%), followed closely by classes III (60% intra- and 45% interobserver consensus with 40% accuracy) and IV (63% intra- and 46% interobserver consensus with 43% accuracy) (<xref ref-type="bibr" rid="B15">15</xref>). Similar results have been seen in smaller studies (<xref ref-type="bibr" rid="B16">16</xref>, <xref ref-type="bibr" rid="B17">17</xref>). To summarize, these studies concluded that pathologists concurred with their own previous evaluations in only two-thirds to three-quarters of cases, and, for most stages, reached the same diagnosis as their colleagues less than half the time.</p>
<p>Given the poor concordance in distinguishing class II through IV melanocytic lesions, several reports have sought to identify barriers to reproducibility. Unsurprisingly, those pathologists who are trained in dermatopathology, have a significant caseload of melanocytic lesions, and have more than 5 years of post-training experience have better reproducibility and accuracy (<xref ref-type="bibr" rid="B18">18</xref>). Dermatopathology training reduced misclassification rates of class II lesions by 10% and class III-IV lesions by 20%. While second and third opinions from trained dermatopathologists can reduce the misclassification rate of a general pathologist, additional peer review has a smaller, but still significant, benefit when a dermatopathologist performs the initial read (<xref ref-type="bibr" rid="B19">19</xref>). If these findings reflect the pathology community, then three highly trained dermatopathologists reviewing the same class II, III, or IV case, would be expected to have a consensus accuracy of 35, 52, and 62%, respectively (<xref ref-type="bibr" rid="B19">19</xref>). These studies indicate that training and experience are not sufficient to reach high levels of accuracy for intermediate lesions. The most beneficial, albeit modest, improvement in concordance was obtained by updating from the AJCC 7th edition diagnostic criteria to the 8th edition (<xref ref-type="bibr" rid="B12">12</xref>), suggesting that further refinement of diagnostic criteria could yet improve accuracy. However, diagnostic threshold variability between observers and a tendency to over call diagnoses due to medicolegal concerns (diagnostic drift) are expected to limit the histopathological diagnosis of melanocytic lesions regardless of diagnostic criteria improvements (<xref ref-type="bibr" rid="B20">20</xref>&#x02013;<xref ref-type="bibr" rid="B22">22</xref>).</p>
</sec>
<sec>
<title>The Promise and Perils of Developing Biomarkers</title>
<p>The above observations highlight the need for objective biomarkers to improve diagnostic accuracy. Biomarkers that bring clarity to the MPATH-Dx classifications would improve patient care by more accurately matching intervention risk to lesion prognosis and save patients and medical systems from expense and potential adverse outcomes related to unnecessary procedures. However, the poor concordance and accuracy of histologic melanoma diagnosis highlight the substantial challenge in developing such biomarkers. The development and validation of clinically useful biomarkers require a gold standard, or a true state, for comparison. If only unambiguous cases are selected for study, then the biomarker, by definition, provides no clinical value to augment current standard of care. Alternatively, if the gold standards are sufficiently ambiguous (for example, a consensus accuracy of 35%), then a theoretically perfect biomarker is expected to fail validation. Thus, for prospective biomarkers to yield useful test characteristics and address a clinically useful decision-making process, careful selection of the gold standard is required. In addition to validated clinical utility, practical considerations such as cost, invasiveness, tissue requirement, and ease of performing the test may influence biomarker use by clinicians.</p>
<p>The resources required to validate a candidate biomarker are substantial in terms of cost, time, and, most critically, patient volunteers. For example, an international coalition of melanoma experts reported that a randomized clinical trial evaluating the added value of prognostic gene-expression based biomarkers would require cohorts of 1,000 to 9,000 patients, depending on the trial design and hypothesis being tested (<xref ref-type="bibr" rid="B5">5</xref>). Therefore, it is critical to carefully consider all of the potential pitfalls of biomarker discovery, development, and implementation prior to committing limited resources to validation of any specific test. Considerations include not only the performance of a proposed biomarker, but also the size and diversity of the cohorts from which it was discovered and validated, and its potential to integrate with current practice and positively influence clinical decision-making. Prior to development, investigators should consider what test characteristics are required for a test to be useful, so that the field can determine how best to distribute limited resources for biomarker discovery and validation. These characteristics are described next.</p>
</sec>
</sec>
<sec id="s2">
<title>Application of Fundamental Theory of Biomarker Utility To Current Melanoma Biomarkers</title>
<sec>
<title>Clinical Uncertainty in Melanoma Diagnosis and Treatment</title>
<p>The diversity of clinical circumstances surrounding individual melanomas ensures that there is no &#x0201C;one-size-fits-all&#x0201D; approach to the diagnosis and management of melanoma. Often, the first decision for a clinician in the evaluation of a pigmented skin lesion is to determine whether it is likely benign or has atypical clinical characteristics that warrant biopsy for further evaluation (<xref ref-type="fig" rid="F1">Figure 1A</xref>). If a biopsy is performed, a pathologist must evaluate the likelihood of the lesion being malignant or benign (<xref ref-type="fig" rid="F1">Figure 1B</xref>). For invasive malignant melanomas, a surgical oncologist must decide whether to offer SLNB to evaluate for early metastatic disease (<xref ref-type="fig" rid="F1">Figure 1C</xref>), and selection of systemic therapy becomes necessary with local (e.g., a positive SLNB) or distant metastasis. Each of these decisions represent a junction where the clinician is faced with potential uncertainty and is forced to balance the odds of disease with a significant intervention. Clarification of uncertainty at these decision points might be achieved with objective biomarkers, as discussed in detail below. While these points do not comprise an exhaustive set of decision points in melanoma diagnosis and management that could benefit from biomarker development, tests that aid these decision points would lead to dramatic improvements in patient care. To build a foundation for this discussion, a summary of the fundamental theory of biomarker accuracy and utility is presented.</p>
<fig id="F1" position="float">
<label>Figure 1</label>
<caption><p>Examples of pigmented skin lesion clinical decision points and their outcomes, which could be aided by validated biomarkers. <bold>(A)</bold> Assessing a melanocytic lesion of unknown significance will result in a decision to either monitor or biopsy that lesion. <bold>(B)</bold> After biopsy, the histopathological diagnosis made by a pathologist or dermatopathologist will identify the lesion as a benign, atypical or malignant melanocytic proliferation. This determination will dictate whether the region is monitored or additional surgeries are performed. <bold>(C)</bold> Recommendations for additional therapy, such as potential sentinel lymph node biopsy and systemic adjuvant therapy, are made based on the estimated risk of metastasis, occurrence of metastasis, and evidence of treatment failure.</p></caption>
<graphic xlink:href="fmed-08-642380-g0001.tif"/>
</fig>
</sec>
<sec>
<title>The Theoretical Framework of a Clinically Useful Biomarker</title>
<p>A clinically useful test must be capable of changing clinical decision making. In other words, usefulness is determined by the clinician ordering the test. Clinicians routinely encounter patients presenting with an ambiguous disease state. An illustrative example would be a sore throat that could be due to either <italic>Streptococcus pyogenes</italic> or a viral etiology. In this scenario, a rapid &#x0201C;strep&#x0201D; test or bacterial cultures may be ordered to resolve this ambiguity by increasing the clinician&#x00027;s certainty on whether the intervention of antibiotics would benefit the patient. A clinical test could resolve this uncertainty by decreasing the perceived likelihood of the disease state (exclusion), such as with a negative point-of-care rapid &#x0201C;strep&#x0201D; test. Alternatively, a clinical test could increase the likelihood of a disease state (confirmation), such as with a positive bacterial culture. If the test result shifts the clinician&#x00027;s certainty on the patient&#x00027;s diagnosis above or below their self-determined intervention threshold (e.g., a positive rapid &#x0201C;strep&#x0201D; test resulting in an antibiotic prescription), the test will have usefully influenced a clinical decision.</p>
<p>Similarly, predictive and prognostic tests may aim to determine disease trajectory and can assist in directing further intervention, such as follow up imaging or adjuvant therapy selection.</p>
<p>The potential utility of a biomarker can be estimated by first considering clinicians&#x00027; certainty of a disease state using current standard of care (&#x0201C;prior odds&#x0201D;) and the certainty threshold that must be breached for intervention (&#x0201C;threshold&#x0201D;) (<xref ref-type="fig" rid="F2">Figure 2A</xref>). It is important to recognize that since both values are derived from individual clinicians&#x00027; perceptions, they are inherently imprecise and inconsistent. However, ranges for these values can be estimated through analysis of collective clinical decisions or surveying and these ranges are informative when considering the potential utility of a candidate test. When prior odds and intervention thresholds have been estimated, the slope of the line that connects them represents the change in the odds (also called the likelihood ratio or LR) required to change a decision. A clinical test that decreases the perceived likelihood of the disease state results in a negative likelihood ratio or LR- (<xref ref-type="fig" rid="F2">Figure 2A</xref>, green line). A clinical test that increases the perceived likelihood of the disease state results in a positive likelihood ratio or LR&#x0002B; (<xref ref-type="fig" rid="F2">Figure 2A</xref>, blue line).</p>
<fig id="F2" position="float">
<label>Figure 2</label>
<caption><p>Theoretical framework for biomarker utility analysis. <bold>(A)</bold> A clinician may be challenged with a state of uncertainty, where the perceived probability of the disease (prior odds, purple) does not surpass the clinician&#x00027;s personal thresholds for recommending either major intervention or no intervention (decision thresholds, red dotted lines). A clinician may hope to reduce uncertainty based upon a biomarker test result. A positive result must alter the probability of the disease sufficiently to breach the decision threshold to recommend major intervention (blue line). A negative result must alter the probability of the disease sufficiently to breach the decision threshold to recommend no intervention (green line). In this example, the prior odds are an equal probability of disease or no disease. Based upon this clinician&#x00027;s decision thresholds, an actionable biomarker test result would require either a 10-fold increase or 10-fold decrease in this perceived probability (posterior odds). The extent by which a biomarker test might change the perceived odds can be calculated as a likelihood ratio (LR). A positive LR (LR&#x0002B;, blue line) increases the likelihood of the disease state being present and a negative LR (LR&#x02013;, green line) decreases the likelihood of the disease state being present. The posterior odds following a biomarker test can be calculated as the prior odds multiplied by the likelihood ratio. <bold>(B)</bold> LR&#x0002B; and LR&#x02013; can be plotted against sensitivity and specificity on the axes of a receiver operator characteristic (ROC) curve graph, which represents the sensitivity and specificity of the test at different cut-off values where an &#x0201C;abnormal&#x0201D; result flag would be generated (red curved line). The LR values represent the slope at any given point on the ROC curve, and the LR lines for point V are represented. The area under the ROC curve (AUC) describes the overall sensitivity and specificity of a test at all cut-off points. AUC of 1.0 represents 100% sensitivity and specificity, and a useless test (gray line) has an AUC of 0.5. The red ROC curve has an approximate AUC of &#x0003E;0.9. <bold>(C)</bold> In the example shown in <bold>(A)</bold>, a minimum LR&#x0002B; of 10 is required to generate posterior odds that breach the clinician&#x00027;s decision threshold for major intervention. The area (shaded in light blue) left of this LR&#x0002B; slope (dark blue line) represents the sensitivity and specificity characteristics of a test useful for confirming or ruling in a disease state that has an LR&#x0002B; &#x02265;10. A biomarker test with performance represented by the red ROC curve has a greater range of cut-off values that can produce an LR&#x0002B; &#x02265; 10, and has a higher maximal potential sensitivity (point W) compared to the purple ROC curve (point X). This translates to a greater odds of achieving a true test value that has the potential to change the clinician&#x00027;s decision for the red ROC curve, especially with test cut-off values at point W. <bold>(D)</bold> Similar to <bold>(C)</bold>, the area (shaded in green) above the LR&#x02013; slope (dark green line) represents the sensitivity and specificity characteristics of a test useful for excluding or ruling out the example disease state from <bold>(A)</bold>. The red ROC curve represents a hypothetically useful test with a higher maximal sensitivity capable of achieving a LR&#x02013; &#x02264;0.1 (point Y), and the purple ROC curve represents a test that is less likely to influence clinical decision making with a lower maximal sensitivity capable of achieving a LR&#x02013; &#x02264; 0.1 (point Z). Figure adapted from (<xref ref-type="bibr" rid="B23">23</xref>).</p></caption>
<graphic xlink:href="fmed-08-642380-g0002.tif"/>
</fig>
<p>When determining whether a decision point might benefit from biomarker development, it is useful to estimate what values of LR would be required of a theoretical test for posterior odds to surpass the intervention threshold (<xref ref-type="fig" rid="F2">Figure 2A</xref>). Likelihood ratios are calculated from the sensitivity and specificity of the test at one reference range or cut-off value [For information on how to calculate these and other established clinical accuracy metrics see (<xref ref-type="bibr" rid="B24">24</xref>) and (<xref ref-type="bibr" rid="B23">23</xref>)]. Sensitivity describes the ability of the test to discern the presence of a disease and specificity quantifies how often it correctly identifies disease absence. There is often a trade-off between sensitivity and specificity, particularly with test outcomes reported as a continuous variable. Since clinical accuracy is based on assignment of the test cut-off value (a value outside of the &#x0201C;normal&#x0201D; reference range that is flagged as an abnormal result), the sensitivity and specificity of a test can vary wildly based on where a laboratory assigns the cut-off values for normal. To address this issue, test characteristics can be plotted as a receiver operating characteristic (ROC) curve, which plots the sensitivity and specificity at different cut-off points. This also allows one to compare different tests/biomarkers by calculating the area under the curve (AUC). In general, the higher area under the curve reflects a better performing test, with an AUC of 1.0 having perfect sensitivity and specificity and an AUC of 0.5 being useless (<xref ref-type="fig" rid="F2">Figure 2B</xref>). Since utility is determined by the clinician ordering and interpreting the test, there is no standardized AUC at which a test becomes clinically useful. Instead, positive and negative likelihood ratios can be derived from the slope between any ROC point and the (0,0) and (1,1) vertices, respectively (<xref ref-type="fig" rid="F2">Figure 2B</xref>, point V). If a biomarker test application requires at least a LR&#x0002B; of 10 and/or LR&#x02013; of 0.1 to change clinical decision (as in the example presented in <xref ref-type="fig" rid="F2">Figure 2A</xref>), then theoretically any point on the ROC curve that lies to the left (for LR&#x0002B;) or above (LR&#x02013;) of the LR lines could be considered tests worth further development (<xref ref-type="fig" rid="F2">Figures 2C,D</xref>). However, ROC curves with higher AUC will provide a larger range of potential test cut-offs that will meet these minimal LRs and will consequently have a higher sensitivity and/or specificity at those cut-offs (<xref ref-type="fig" rid="F2">Figures 2C,D</xref>, ROC curves at point W and Y).</p>
<p>In the following sections we apply this theory to estimate the test characteristics required to change clinical decision making in the diagnosis and treatment of melanoma. Where available, we use a combination of published studies and surveys to estimate ranges for the prior odds and intervention thresholds. However, every providers&#x00027; internal threshold of certainty for clinical decision making will vary in the context of patient-specific factors in addition to test characteristics. Thus, these ranges should not be considered exact quantifications but rather estimates to help determine which biomarkers may be worth investing community resources for further discovery and development.</p>
</sec>
<sec>
<title>Toward Enhancing Pigmented Lesion Screening With Biomarkers</title>
<p>Many factors are considered when assessing a pigmented lesion in clinic prior to recommending a biopsy. These include but are not limited to clinical impression of the lesion, history of the lesion (either as reported by the patient or through comparison to previous clinical images), the patient&#x00027;s history and risk for melanoma, total number of lesions, and dermatoscopic features. Since the decision to biopsy represents a gestalt of these different factors, it is not surprising that the level of specialty training and experience of the physician substantially alter the number of lesions biopsied for each melanoma identified. A recent meta-analysis of 351 studies determined the sensitivity and specificity of identifying melanomas during skin exams was 87.5 and 81.4%, respectively, for dermatologists, but only 79.9 and 70.9% for primary care physicians (<xref ref-type="bibr" rid="B25">25</xref>). Unfortunately, access to dermatologists for regular skin exams is neither uniform nor comprehensive, with over half of U.S. citizens having insufficient access to dermatologic care (<xref ref-type="bibr" rid="B26">26</xref>). Dermoscopy and reflectance confocal microscopy (RCM) are two non-invasive approaches that increase the number of melanomas biopsied, while reducing the number of benign lesions biopsied (<xref ref-type="bibr" rid="B27">27</xref>, <xref ref-type="bibr" rid="B28">28</xref>). However, these techniques also require highly skilled interpretation, and use of RCM is largely limited to dermatologists who specialize in pigmented lesions. Dermoscopy is more broadly used in clinical practice, though it does not appear to increase diagnostic accuracy in the practice of non-experts (<xref ref-type="bibr" rid="B29">29</xref>) and has even been reported to reduce sensitivity of melanoma detection compared to clinical assessment alone (<xref ref-type="bibr" rid="B30">30</xref>). An objective biomarker could therefore improve the odds to biopsy a true melanoma, especially in situations where a specialized dermatologist and auxiliary equipment/techniques are not available. In contrast, any increases in certainty might be minimal when used by a highly specialized and experienced pigmented lesion dermatologist.</p>
<p>Ultimately, the cost of a skin biopsy to the individual patient, in terms of both morbidity and finance, is relatively low, but the cost of leaving an early-stage melanoma undiagnosed is exceptionally high. By applying the above theory of biomarker utility, the prior odds of malignancy can be estimated by considering the number of lesions biopsied to diagnose one melanoma. This ranges from &#x0007E;2 for some dermatologists to &#x0007E;30 for primary care physicians (<xref ref-type="bibr" rid="B31">31</xref>). Thus, according to current clinical practices, any lesions with prior odds lower than 1/2&#x02013;1/30 of being malignant are less likely to be biopsied. A biomarker introduced at this stage could therefore provide utility if it conferred a significant degree of certainty that a lesion was not malignant and thus avoided biopsy. For certainty to be sufficiently increased such that the perceived odds of malignancy is reduced from 1/2 and 1/30 prior to the test to &#x0003C;1/30 after the test (posterior odds), an estimated likelihood ratio of 0.1 (LR-) is required (<xref ref-type="fig" rid="F3">Figures 3A,B</xref>, green line). Conversely, for a melanocytic tumor eliciting a high suspicion of malignancy from conventional means (high prior odds), a biomarker is unlikely to change decision making, as the clinical threshold to perform the outcome (biopsy) has already been met (LR&#x0002B; 1.0, <xref ref-type="fig" rid="F3">Figures 3A,B</xref>, blue line). Although this analysis suggests that molecular biomarkers are unlikely to change clinical behavior when assessing solitary lesions, there are other clinical scenarios that challenge the assumptions and conclusions of this model. For example, when individual lesions present in sensitive areas, such as the face or genitals, the morbidity of biopsy and the clinical suspicion threshold required to biopsy are increased. In these cases, a modest LR&#x02013; may suffice to shift the clinical decision from &#x0201C;biopsy&#x0201D; to &#x0201C;monitor.&#x0201D; Alternatively, due to increased morbidity and decreased patient tolerance of multiple skin biopsies, a patient with many suspicious lesions may benefit from a combination &#x0201C;rule in&#x0201D; and &#x0201C;rule out&#x0201D; molecular test (necessary characteristics estimated as LR&#x0002B; = 5.0 and LR&#x02013; 0.2, respectively, <xref ref-type="fig" rid="F3">Figures 3C,D</xref>) to prioritize lesions to biopsy (<xref ref-type="bibr" rid="B32">32</xref>).</p>
<fig id="F3" position="float">
<label>Figure 3</label>
<caption><p>Hypothetical biomarker test characteristics to assist in deciding to biopsy. <bold>(A)</bold> Solitary lesions with perceived prior odds of malignancy estimated at 1:2 to 1:30 are typically biopsied due to the minimal pain and expense of biopsy. Therefore, a test&#x00027;s LR&#x0002B; characteristic is not generally valuable when evaluating a single lesion. In contrast, a LR&#x02013; of 0.1 indicates a 10-fold decrease in likelihood of malignancy, and may inform the decision to not biopsy. <bold>(B)</bold> LR&#x0002B; and LR- slopes from <bold>(A)</bold> plotted on a receiver operating characteristic (ROC) curve, indicate the hypothetical test characteristics required of a biomarker to provide clinically meaningful decision to biopsy a suspicious lesion, with the area shaded in green representing estimated test characteristics useful in ruling out biopsy. <bold>(C)</bold> The estimations and assumptions change when a single patient presents with numerous borderline lesions. Due to limited patient tolerance for multiple biopsies, there may be an increased threshold to biopsy each additional lesion. A test designed to address these parameters would have different test characteristics. For instance, a LR&#x02013; &#x02264; 0.2 might convince a provider to defer biopsy on a lesion, and a LR&#x0002B; &#x02265;5 could prompt a biopsy for a lesion that would have otherwise been monitored. <bold>(D)</bold> In the setting of multiple lesions, the question of which lesions to biopsy may be more readily influenced with biomarker testing, as indicated by the light blue shaded area (shift from indecision to biopsy) or light green shaded area (shift from indecision to monitor). These assumptions representing a higher burden of biopsy and altered threshold may also apply to biopsies of sensitive areas such as face, genitals, or nail bed.</p></caption>
<graphic xlink:href="fmed-08-642380-g0003.tif"/>
</fig>
<p>The pigmented lesion assay (PLA), developed by DermTech, is the only biomarker currently commercially available for clinical use addressing this decision (<xref ref-type="bibr" rid="B33">33</xref>, <xref ref-type="bibr" rid="B34">34</xref>). This test is an adhesive patch-based gene expression test measuring expression of <italic>PRAME</italic> and <italic>LINC00518</italic>, with reported sensitivity of 91% and specificity of 69% (<xref ref-type="bibr" rid="B34">34</xref>). The non-invasive nature and high sensitivity of this test has allowed for clinical incorporation in certain specialized scenarios (<xref ref-type="bibr" rid="B32">32</xref>). However, a recent study concluded the test is neither widely nor routinely used by pigmented lesion experts, which is consistent with predictions of the above model (<xref ref-type="bibr" rid="B35">35</xref>). Beyond the test&#x00027;s performance characteristics, another limitation is the delay between sampling and result, which necessitates a subsequent biopsy visit if the test is positive. If a similarly sensitive point-of-care biomarker test could be developed as a rapid in-office test, it may be more widely adopted.</p>
<p>In summary, because the clinical default in a setting of uncertainty will usually be to biopsy an indeterminate lesion, biomarkers with a large LR&#x02013; represent the opportunity to change clinical management by avoiding biopsy. For this purpose, a biomarker with maximum specificity, thus minimizing false negatives, is critical since the adverse consequences of missing the diagnosis is high and the morbidity and cost of a skin biopsy is relatively low. There are situations within a dermatology clinic where a biomarker with high sensitivity (LR&#x0002B;) may also prove clinically useful, but these are likely to be more specialized scenarios such as prioritizing lesions for biopsy in a patient with dysplastic nevus syndrome. Combined with practical limitations for such widespread use of non-invasive biomarkers used at this stage, including the ease of use, cost, and the timing of results, we speculate that current technologies for molecular profiling are unlikely to be influential in the routine evaluation of single lesions.</p>
</sec>
<sec>
<title>Current Biomarkers for the Histopathological Diagnosis of Pigmented Lesions</title>
<p>The established challenges of histopathologic analysis has motivated the search for biomarkers to assist in melanoma diagnosis. If prior odds based on original histology favor a more benign dysplasia, such as a dysplastic nevus, then it is possible that a moderate LR&#x02013;, estimated at 0.2, may convince a clinician to monitor a biopsy site rather than pursuing an excision with margins (<xref ref-type="fig" rid="F4">Figure 4</xref>). Conversely, if a biomarker applied to the same lesion demonstrated a significant LR&#x0002B; of 10, then the result may convince a clinician to excise the area with larger margins (10&#x02013;20 mm instead of 5 mm), despite relatively low prior odds. For a lesion with higher prior odds of possessing malignant potential but lacking <italic>bona fide</italic> features of melanoma, often termed an atypical melanocytic proliferation, a less compelling LR&#x0002B; of 2 might be sufficient to upgrade a diagnosis to melanoma, whereas a significantly lower LR- of 0.1&#x02013;0.01 might be required to decrease excision margins or avoid excision altogether. These analyses demonstrate that there could be great clinical utility of biomarkers examining the malignant potential of intermediate melanocytic lesions, particularly when the biomarker holds a significant likelihood ratio that modifies the prior odds by an order of magnitude or more.</p>
<fig id="F4" position="float">
<label>Figure 4</label>
<caption><p>Biomarkers have the potential to clarify the risk of melanoma in the histopathologic interpretation of borderline pigmented lesions. <bold>(A)</bold> Melanocytic lesions exist on a histologic spectrum from benign-appearing to malignant. At the extremes of this spectrum (e.g., unambiguous benign melanocytic nevus (MPATH-Dx class I) or unambiguous invasive melanoma (MPATH-Dx class IV-V) no biomarker test is likely to change management (gray dots, <xref ref-type="table" rid="T1">Table 1</xref>). For borderline melanocytic lesions, especially MPATH-Dx class II-III lesions, pathologists are generally conservative and recommend excision of ambiguous lesions that may have no malignant potential to avoid potential adverse consequences of underdiagnosis (<xref ref-type="bibr" rid="B36">36</xref>). This results in excision using 5 mm margins of many atypical and dysplastic melanocytic lesions without full knowledge of malignant potential. A LR&#x02013; of 0.2, corresponding to a 5-fold reduction in estimated malignant potential, may be sufficient to avoid a recommendation of excision in a more benign-appearing atypical/dysplastic specimen if threshold for treatment is set at &#x0007E;1:200. At the more dysplastic end of the spectrum, a biomarker test with LR&#x0002B; of 2 may be sufficient to increase recommendation from excision with 5 mm margins to excision with 10 mm margins. This results in a significant increase in size of total excision and repair that is commensurate with the higher estimated increased risk of malignancy. Due to the inherently ambiguous nature of MPATH-Dx class II&#x02013;III lesions, it is challenging to estimate the decision thresholds for treatment. Depending on the pathologist&#x00027;s level of suspicion and the clinician&#x00027;s threshold for intervention (which may be influenced by patient specific-factors including prior odds and morbidity of procedure), different LR values are almost certainly necessary to justify changing intervention, such as deferring excision or increasing excision margins. <bold>(B)</bold> The estimated minimum necessary likelihood ratios indicate that there is a significant range of biomarker characteristics that provide clinical utility in deferring excision (light green shaded area) and/or increasing excision margins (light blue shaded area).</p></caption>
<graphic xlink:href="fmed-08-642380-g0004.tif"/>
</fig>
<p>Numerous biomarker tests have been developed to address this issue. <xref ref-type="table" rid="T2">Table 2</xref> reports biomarker tests that are currently commercially available or used in selected referral centers and <xref ref-type="table" rid="T3">Table 3</xref> (discussed below) reports additional candidate biomarker tests reported in the literature. Immunohistochemistry (IHC) biomarkers are usually the most easily adopted test with regards to tissue requirement, cost, and required laboratory equipment and expertise. One promising IHC marker is PRAME (Preferentially expressed Antigen in Melanoma), which has demonstrated an LR&#x0002B; of 29&#x02013;62.5 and an LR- of 0.13&#x02013;0.253 when assessed with an independent validation cohort (<xref ref-type="bibr" rid="B39">39</xref>, <xref ref-type="bibr" rid="B40">40</xref>). Considering that it shares 90% concordance with fluorescent <italic>in situ</italic> hybridization (FISH) and 92.7% agreement with the final diagnostic interpretation (<xref ref-type="bibr" rid="B40">40</xref>), adopting PRAME IHC could be an economical option for most laboratories to use for routine differentiation of nevi and melanoma. Comparatively, p16 IHC, which is used at some institutions to help distinguish between nevi and melanoma in challenging cases, has no published validation cohorts to identify the clinical characteristics of this test. As a result, the accuracy, reliability, and versatility of p16 is unclear. In addition, the utility of p16 expression depends on the type of lesion since it may help distinguish desmoplastic Spitz nevi from desmoplastic melanoma (<xref ref-type="bibr" rid="B37">37</xref>), but fails at distinguishing Spitz nevi from Spitzoid melanoma (<xref ref-type="bibr" rid="B38">38</xref>).</p>
<table-wrap position="float" id="T2">
<label>Table 2</label>
<caption><p>Melanoma biomarkers used by reference centers and dermatopathologists.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th valign="top" align="left"><bold>Category</bold></th>
<th valign="top" align="center"><bold>Method class</bold></th>
<th valign="top" align="center"><bold>Method specifics</bold></th>
<th valign="top" align="center"><bold>Total study number</bold></th>
<th valign="top" align="center"><bold>Study breakdown</bold></th>
<th valign="top" align="center"><bold>Estimated tissue requirement</bold></th>
<th valign="top" align="center"><bold>Study objective</bold></th>
<th valign="top" align="center"><bold>Number of pathologists confirming cohort</bold></th>
<th valign="top" align="center"><bold>AUC</bold></th>
<th valign="top" align="center"><bold>S/Sp/PPV/NPV, LR&#x0002B;/LR&#x02013;<sup><bold><xref ref-type="table-fn" rid="TN3">&#x02021;</xref></bold></sup></bold></th>
<th valign="top" align="center"><bold>Independent cohorts assessed<xref ref-type="table-fn" rid="TN2"><sup>&#x0002A;&#x0002A;</sup></xref></bold></th>
<th valign="top" align="center"><bold>Qualitative vs. quantitative</bold></th>
<th valign="top" align="center"><bold>Citation</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left" colspan="13"><bold>Individual biomarker assays</bold></td>
</tr>
<tr>
<td valign="top" align="center">Protein expression</td>
<td valign="top" align="center">IHC</td>
<td valign="top" align="center">Loss of p16 protein expression</td>
<td valign="top" align="center">26</td>
<td valign="top" align="center">11 desmoplastic melanomas, 15 Spitz nevi</td>
<td valign="top" align="center">Sectioned FFPE slide</td>
<td valign="top" align="center">Differentiate desmoplastic Spitz nevus and desmoplastic melanoma</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">82<xref ref-type="table-fn" rid="TN1"><sup>&#x0002A;</sup></xref>/100<xref ref-type="table-fn" rid="TN1"><sup>&#x0002A;</sup></xref>/NR/NR, &#x0003E;80/0.18</td>
<td valign="top" align="center">No</td>
<td valign="top" align="center">Qualitative, requires expert interpretation</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B37">37</xref>)</td>
</tr>
<tr>
<td/>
<td/>
<td/>
<td valign="top" align="center">37</td>
<td valign="top" align="center">19 Spitzoid melanomas, 18 Spitz nevi</td>
<td valign="top" align="center">Sectioned FFPE slide</td>
<td valign="top" align="center">Does not differentiate Spitzoid melanomas from Spitz nevi</td>
<td valign="top" align="center">Initial diagnosis plus consensus conference with 3&#x0002B; dermatopathologists</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">21<xref ref-type="table-fn" rid="TN1"><sup>&#x0002A;</sup></xref>/83<xref ref-type="table-fn" rid="TN1"><sup>&#x0002A;</sup></xref>/NR/NR, 1.2/0.95</td>
<td/>
<td/>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B38">38</xref>)</td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="center">PRAME protein expression (4&#x0002B; staining)</td>
<td valign="top" align="center">400</td>
<td valign="top" align="center">255 primary and metastatic melanomas, 145 melanocytic nevi</td>
<td valign="top" align="center">Sectioned FFPE slide</td>
<td valign="top" align="center">Support diagnosis of melanoma by PRAME positivity</td>
<td valign="top" align="center">2&#x0002B;</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">87<xref ref-type="table-fn" rid="TN1"><sup>&#x0002A;</sup></xref>/97<xref ref-type="table-fn" rid="TN1"><sup>&#x0002A;</sup></xref>/NR/NR, 29/0.13</td>
<td valign="top" align="center">Yes</td>
<td valign="top" align="center">Qualitative, requires expert interpretation</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B39">39</xref>)</td>
</tr>
<tr style="border-bottom: thin solid #000000;">
<td/>
<td/>
<td/>
<td valign="top" align="center">110</td>
<td valign="top" align="center">110 ambiguous cases reviewed by 2&#x0002B; dermato-pathologists</td>
<td valign="top" align="center">Sectioned FFPE slide</td>
<td valign="top" align="center">Validate and compare PRAME IHC to FISH for differentiating ambiguous cases</td>
<td valign="top" align="center">2&#x0002B;</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">75.0/98.8/NR/NR, &#x0003E;62.5, 0.253</td>
<td/>
<td/>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B40">40</xref>)</td>
</tr> <tr style="border-bottom: thin solid #000000;">
<td valign="top" align="center">Copy Number Variation</td>
<td valign="top" align="center">CGH</td>
<td valign="top" align="center">Copy number variation by comparative genomic hybridization</td>
<td valign="top" align="center">186</td>
<td valign="top" align="center">132 melanomas, 54 benign nevi (27 Spitz nevi, 19 blue nevi, 7 congenital nevi)</td>
<td valign="top" align="center">High tissue requirement; remainder of FFPE block</td>
<td valign="top" align="center">Differentiate melanoma from benign nevi on basis of copy number variation</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">96<xref ref-type="table-fn" rid="TN1"><sup>&#x0002A;</sup></xref>/87<xref ref-type="table-fn" rid="TN1"><sup>&#x0002A;</sup></xref>/NR/NR, 7.4/0.046</td>
<td valign="top" align="center">No</td>
<td valign="top" align="center">Qualitative, requires expert interpretation</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B41">41</xref>)</td>
</tr> <tr>
<td valign="top" align="center">FISH</td>
<td valign="top" align="center">FISH</td>
<td valign="top" align="center">Four probes targeting chromosome 6p23, 6p25, 11q13, and centromere 6</td>
<td valign="top" align="center">22</td>
<td valign="top" align="center">12 ambiguous lesions, 10 unequivocal lesions</td>
<td valign="top" align="center">Medium to High tissue requirement</td>
<td valign="top" align="center">Validate FISH on histologically ambiguous lesions based on clinical behavior of lesion</td>
<td valign="top" align="center">Initial diagnosis plus one dermatopathologist review, ambiguous tumors reviewed by two pathologists</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">60/50/NR/NR, 1.2/0.80</td>
<td valign="top" align="center">Yes</td>
<td valign="top" align="center">Qualitative, requires expert interpretation</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B42">42</xref>)</td>
</tr>
<tr style="border-bottom: thin solid #000000;">
<td/>
<td/>
<td valign="top" align="center">Four probes targeting chromosome 9p21, 6p25, 11q13, and 8q24</td>
<td valign="top" align="center">424</td>
<td valign="top" align="center">Training set: 152 melanoma and 170 nevi. Validation set: 51 melanoma and 51 nevi.</td>
<td valign="top" align="center">Medium to High tissue requirement</td>
<td valign="top" align="center">Distinguish between melanoma and nevi on basis of chromosomal changes</td>
<td valign="top" align="center">1</td>
<td valign="top" align="center">0.94&#x0002B;</td>
<td valign="top" align="center">94/98/NR/NR, 47/0.061</td>
<td valign="top" align="center">Yes</td>
<td/>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B43">43</xref>)</td>
</tr> <tr>
<td valign="top" align="center">Gene Expression with Myriad myPath</td>
<td valign="top" align="center">23 GEP qRT-PCR</td>
<td valign="top" align="center">Weighted 23 gene expression algorithm by qRT-PCR</td>
<td valign="top" align="center">901</td>
<td valign="top" align="center">Training set: 254 melanoma and 210 nevi. Validation set: 437 independent lesions</td>
<td valign="top" align="center">Medium tissue requirement, macro-dissection from sections</td>
<td valign="top" align="center">Differentiate benign nevi from malignant melanoma</td>
<td valign="top" align="center">Initial diagnosis, one study dermatopathologist review, if discordance a third dermatopathologist</td>
<td valign="top" align="center">0.96</td>
<td valign="top" align="center">90/91/NR/NR, 10/0.11</td>
<td valign="top" align="center">Yes</td>
<td valign="top" align="center">Quantitative algorithm, Qualitative output</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B44">44</xref>)</td>
</tr>
<tr>
<td/>
<td/>
<td/>
<td valign="top" align="center">1,400</td>
<td valign="top" align="center">204 melanoma, 656 nevi</td>
<td/>
<td valign="top" align="center">Prospective validation of differentiating between benign and malignant melanocytic lesions</td>
<td valign="top" align="center">Concordance between 3 experienced dermatopathologists</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">91.5/92/NR/NR, 11.4/0.0924</td>
<td/>
<td/>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B45">45</xref>)</td>
</tr>
<tr>
<td/>
<td/>
<td/>
<td valign="top" align="center">182</td>
<td valign="top" align="center">99 melanomas, 83 nevi</td>
<td/>
<td valign="top" align="center">Correlate long-term clinical outcomes with gene signature</td>
<td valign="top" align="center">3</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">93.8/96.2/NR/NR, 24.7/0.0644</td>
<td/>
<td/>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B46">46</xref>)</td>
</tr>
<tr>
<td/>
<td/>
<td/>
<td valign="top" align="center">50</td>
<td valign="top" align="center">23 melanomas, 27 nevi</td>
<td/>
<td valign="top" align="center">Categorize potential desmoplastic melanomas as likely malignant or likely benign</td>
<td valign="top" align="center">2 dermatopathologists, independent review</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">Sensitivity is &#x0201C;about 80%, better than FISH&#x0201D;</td>
<td/>
<td/>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B47">47</xref>)</td>
</tr>
<tr style="border-bottom: thin solid #000000;">
<td/>
<td/>
<td/>
<td valign="top" align="center">181</td>
<td valign="top" align="center">125 diagnostically uncertain lesions, 56 diagnostically certain lesions</td>
<td/>
<td valign="top" align="center">Test accuracy of GEP to diagnosis and outcomes (gold standard) in cases with uncertain histo-pathological diagnoses</td>
<td valign="top" align="center">3 dermatopathologists</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">90.4/95.5/NR/NR, 20.1/0.101</td>
<td/>
<td/>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B48">48</xref>)</td>
</tr>
<tr style="border-bottom: thin solid #000000;">
<td valign="top" align="center"><bold>Compared techniques</bold></td>
<td valign="top" align="center"><bold>Total study number</bold></td>
<td valign="top" align="center"><bold>Study breakdown</bold></td>
<td valign="top" align="center"><bold>&#x0201C;Gold Standard&#x0201D;</bold></td>
<td valign="top" align="center" colspan="2"><bold>Number of pathologists confirming cohort</bold></td>
<td valign="top" align="center"><bold>Unequivocal histology: S/Sp/PPV/NPV, LR+/LR&#x02013;<xref ref-type="table-fn" rid="TN3"><sup>&#x02021;</sup></xref></bold></td>
<td valign="top" align="center"><bold>Unequivocal Concordance</bold></td>
<td valign="top" align="center" colspan="2"><bold>Challenging histology: S/Sp/PPV/NPV, LR+/LR&#x02013;<xref ref-type="table-fn" rid="TN3"><sup>&#x02021;</sup></xref></bold></td>
<td valign="top" align="center"><bold>Challenging Concordance</bold></td>
<td valign="top" align="center" colspan="2"><bold>Citation</bold></td>
</tr>
<tr>
<td valign="top" align="left" colspan="13"><bold>Comparison of biomarker assays</bold></td>
</tr>
<tr style="border-bottom: thin solid #000000;">
<td valign="top" align="center">CGH vs. FISH</td>
<td valign="top" align="center">30</td>
<td valign="top" align="center">25 melanomas, 5 nevi</td>
<td valign="top" align="center">Histology</td>
<td valign="top" align="center" colspan="2">Confirmation of initial diagnosis by 1 dermatopathologist</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center">FISH vs. CGH: 90%</td>
<td valign="top" align="center" colspan="2">&#x02013;</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center" colspan="2">(<xref ref-type="bibr" rid="B49">49</xref>)</td>
</tr> <tr style="border-bottom: thin solid #000000;">
<td valign="top" align="center">Histology vs. FISH vs. myPath</td>
<td valign="top" align="center">117</td>
<td valign="top" align="center">Histologically unequivocal: 15 malignant and 24 benign. 78 Histologically challenging cases</td>
<td valign="top" align="center">Histology</td>
<td valign="top" align="center" colspan="2">1 pathologist for unequivocal and 3 for challenging cases</td>
<td valign="top" align="center">FISH: 93/100/NR/NR, &#x0003E;90/0.07 <break/> myPath: 62/95/NR/NR, 12.4/0.40</td>
<td valign="top" align="center">FISH vs. histology: 97%, myPath vs. histology: 83% <break/> FISH vs. myPath: 80%</td>
<td valign="top" align="center" colspan="2">FISH: 56/83/NR/NR, 3.3/0.53 <break/> myPath: 52/80/NR/NR, 2.6/0.60</td>
<td valign="top" align="center">FISH vs. histology: 70%, myPath vs. histology: 64% <break/> FISH vs. myPath: 70%</td>
<td valign="top" align="center" colspan="2">(<xref ref-type="bibr" rid="B50">50</xref>)</td>
</tr> <tr>
<td valign="top" align="center">Histology vs. myPath vs. FISH vs. (discordant cases only) CGH</td>
<td valign="top" align="center">268</td>
<td valign="top" align="center">Histologically unequivocal: 198. Histologically challenging: 70</td>
<td valign="top" align="center">Histology and SNP-array</td>
<td valign="top" align="center" colspan="2">Challenging cases reviewed by 1 study and 2 independent dermatopathologists</td>
<td valign="top" align="center">myPath: 67/81/NR/NR, 3.5/0.41</td>
<td valign="top" align="center">myPath vs. histology: 75%</td>
<td valign="top" align="center" colspan="2">FISH: 61/100/NR/NR, &#x0003E;60/0.41 <break/> myPath: 50/93/NR/NR, 7.1/0.53</td>
<td valign="top" align="center">FISH vs. histology: 84%, myPATH vs. histology: 74% <break/> FISH vs. myPath: 69%<underline>Agreement with nine</underline><break/><underline> pathologists in discordant</underline><break/> <underline>cases:</underline> <break/> CGH vs. histology: 71% <break/> FISH vs. histology: 54% <break/> MyPath vs. histology: 14%</td>
<td valign="top" align="center" colspan="2">(<xref ref-type="bibr" rid="B51">51</xref>)</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p><italic>AUC, area under (receiver operating characteristic) curve; CGH, comparative genomic hybridization; FFPE, formalin-fixed paraffin embedded; FISH, Fluorescence in situ hybridization; IHC, immunohistochemistry; LR&#x0002B;, positive likelihood radio; LR&#x02013;, negative likelihood radio; NPV, negative predictive value; NR, not reported; PPV, positive predictive value; qRT-PCR, quantitative real time polymerase chain reaction; S, sensitivity; Sp, specificity.</italic></p> 
<fn id="TN1"><label>&#x0002A;</label><p><italic>Calculated from manuscript data, no validation cohort.</italic></p></fn>
<fn id="TN2"><label>&#x0002A;&#x0002A;</label><p><italic>Tests are considered to have independent cohort validation if an independent clinical cohort was included in the original study or at least one follow-up study used an independent cohort with the same diagnoses or a cohort with ambiguous diagnoses.</italic></p></fn>
<fn id="TN3"><label>&#x02021;</label><p><italic>All LR values are calculated based on reported sensitivity and specificity</italic>.</p></fn>
</table-wrap-foot>
</table-wrap>
<table-wrap position="float" id="T3">
<label>Table 3</label>
<caption><p>Proposed melanoma diagnosis biomarkers and prospective uses.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th valign="top" align="center"><bold>Category</bold></th>
<th valign="top" align="center"><bold>Method class</bold></th>
<th valign="top" align="center"><bold>Method specifics</bold></th>
<th valign="top" align="center"><bold>Total study number</bold></th>
<th valign="top" align="center"><bold>Study breakdown</bold></th>
<th valign="top" align="center"><bold>Estimated tissue requirement</bold></th>
<th valign="top" align="center"><bold>Study objective</bold></th>
<th valign="top" align="center"><bold>Number of pathologists confirming cohort</bold></th>
<th valign="top" align="center"><bold>AUC</bold></th>
<th valign="top" align="center"><bold>S/Sp/PPV/NPV (%), LR&#x0002B;/LR-<sup><bold><xref ref-type="table-fn" rid="TN5">&#x02021;</xref></bold></sup></bold></th>
<th valign="top" align="center"><bold>Qualitative vs. quantitative</bold></th>
<th valign="top" align="center"><bold>Citation</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="center">DNA Methylation</td>
<td valign="top" align="center">40-CpG classifier</td>
<td valign="top" align="center">Next generation sequencing</td>
<td valign="top" align="center">162</td>
<td valign="top" align="center">89 melanoma, 73 nevi</td>
<td valign="top" align="center">250&#x02013;300 ng of DNA from microdissected slides</td>
<td valign="top" align="center">Classify uncertain samples as melanoma or nevi</td>
<td valign="top" align="center">4</td>
<td valign="top" align="center">0.996</td>
<td valign="top" align="center">96.6/100/ 100/96.2, &#x0003E;90/0.03</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B52">52</xref>)</td>
</tr>
<tr>
<td/>
<td valign="top" align="center">Promoter methylation</td>
<td valign="top" align="center">1,505 CpG site microarray, analyzed 12 CpG loci highly predictive of melanoma</td>
<td valign="top" align="center">49</td>
<td valign="top" align="center">22 melanoma, 27 nevi</td>
<td valign="top" align="center">250 ng of DNA from FFPE tissue</td>
<td valign="top" align="center">Differentiate nevi and malignant melanoma</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">0.89&#x02013;1.0</td>
<td valign="top" align="center"> &#x02264; 100/ &#x02264; 100/NR/NR</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B53">53</xref>)</td>
</tr>
<tr>
<td/>
<td/>
<td valign="top" align="center">CpG island hypermethylation in promoter of <italic>CLDN11</italic></td>
<td valign="top" align="center">405</td>
<td valign="top" align="center">199 melanoma, 208 nevi</td>
<td valign="top" align="center">Not reported, extraction from tissue</td>
<td valign="top" align="center">Differentiate dysplastic nevi and malignant melanoma</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">0.806</td>
<td valign="top" align="center">52/94/ /NR/NR, 8.67/0.51</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B54">54</xref>)</td>
</tr>
<tr style="border-bottom: thin solid #000000;">
<td/>
<td/>
<td valign="top" align="center"><italic>RASSF1A</italic> promoter methylation in serum cell-Free DNA</td>
<td valign="top" align="center">152</td>
<td valign="top" align="center">84 melanoma, 68 nevi</td>
<td valign="top" align="center">Serum draw</td>
<td valign="top" align="center">Differentiate healthy vs. <italic>in-situ</italic>, invasive, and metastatic melanoma</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">0.905</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B55">55</xref>)</td>
</tr> <tr>
<td valign="top" align="center">Gene expression</td>
<td valign="top" align="center">microRNA ratio</td>
<td valign="top" align="center">Sequencing or qRT-PCR to obtain microRNA Ratio Trained Model score</td>
<td valign="top" align="center">179</td>
<td valign="top" align="center">106 melanoma, 73 nevi</td>
<td valign="top" align="center">Single tissue section</td>
<td valign="top" align="center">Differentiate nevi and malignant melanoma</td>
<td valign="top" align="center">5&#x0002B;</td>
<td valign="top" align="center">0.9</td>
<td valign="top" align="center">81/88/ NR/NR, 6.75/0.22</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B56">56</xref>)</td>
</tr>
<tr>
<td/>
<td valign="top" align="center">RT-PCR</td>
<td valign="top" align="center">qRT-PCR for <italic>SILV</italic></td>
<td valign="top" align="center">193</td>
<td valign="top" align="center">47 melanoma, 48 nevi</td>
<td valign="top" align="center">3&#x02013;12 tissue sections</td>
<td valign="top" align="center">Differentiate dysplastic nevi and malignant melanoma</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">0.94</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B57">57</xref>)</td>
</tr>
<tr style="border-bottom: thin solid #000000;">
<td/>
<td valign="top" align="center">Microarray</td>
<td valign="top" align="center">DNA microarray assay using 14 genes</td>
<td valign="top" align="center">120</td>
<td valign="top" align="center">62 melanoma, 58 nevi</td>
<td valign="top" align="center">8 &#x003BC;M tissue section, microdissected with laser capture</td>
<td valign="top" align="center">Differentiate nevi and malignant melanoma</td>
<td valign="top" align="center">2&#x02013;4</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">90/86/NR/ NR<xref ref-type="table-fn" rid="TN4"><sup>&#x0002A;</sup></xref>, 6.43/0.10</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B58">58</xref>)</td>
</tr> <tr style="border-bottom: thin solid #000000;">
<td valign="top" align="center">Serum nucleic acid</td>
<td valign="top" align="center">Circulating miRNA panel</td>
<td valign="top" align="center">qRT-PCR for six microRNAs in serum</td>
<td valign="top" align="center">65</td>
<td valign="top" align="center">Serum from 50 uveal melanoma, 5 metastatic uveal melanoma, 10 uveal nevi</td>
<td valign="top" align="center">Serum draw</td>
<td valign="top" align="center">Differentiate uveal melanoma from nevi</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">NR for panel</td>
<td valign="top" align="center">93/100<xref ref-type="table-fn" rid="TN4"><sup>&#x0002A;</sup></xref>/100/76, 20/0.07</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B59">59</xref>)</td>
</tr> <tr>
<td valign="top" align="center">Protein Expression</td>
<td valign="top" align="center">Ciliation index</td>
<td valign="top" align="center">Immunofluorescence for acetylated alpha-Tubulin</td>
<td valign="top" align="center">124</td>
<td valign="top" align="center">26 melanoma, 42 nevi</td>
<td valign="top" align="center">Single tissue section</td>
<td valign="top" align="center">Classification of Spitzoid tumors as malignant or benign</td>
<td valign="top" align="center">3&#x0002B;</td>
<td valign="top" align="center">0.84</td>
<td valign="top" align="center">81/65/NR/NR, 2.31/0.29</td>
<td valign="top" align="center">Semi- quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B60">60</xref>)</td>
</tr>
<tr>
<td/>
<td valign="top" align="center">IHC</td>
<td valign="top" align="center">IHC for 5-hydroxymethyl-cystosine</td>
<td valign="top" align="center">190</td>
<td valign="top" align="center">126 melanoma, 45 nevi</td>
<td valign="top" align="center">Single tissue section</td>
<td valign="top" align="center">Differentiate nevi from melanoma</td>
<td valign="top" align="center">2</td>
<td valign="top" align="center">0.78</td>
<td valign="top" align="center">93/98/NR/NR, 46.5/0.07</td>
<td valign="top" align="center">Semi-quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B61">61</xref>)</td>
</tr>
<tr>
<td/>
<td valign="top" align="center">Multi-IHC semi-quantitative scoring system</td>
<td valign="top" align="center">IHC scoring of Ki-67, p16, and HMB45</td>
<td valign="top" align="center">308</td>
<td valign="top" align="center">234 melanoma, 74 nevi</td>
<td valign="top" align="center">Three tissue sections</td>
<td valign="top" align="center">Differentiate nevi and malignant melanoma</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">0.987</td>
<td valign="top" align="center">97.4/97.3/NR/NR, 36.1/0.03</td>
<td valign="top" align="center">Semi-quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B62">62</xref>)</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p><italic>AUC, area under (receiver operating characteristic) curve; FFPE, formalin-fixed paraffin embedded; LR&#x0002B;, positive likelihood radio; LR&#x02013;, negative likelihood radio; NPV, negative predictive value; NR, not reported; PPV, positive predictive value; S, sensitivity; Sp, specificity; qRT-PCR, quantitative real time polymerase chain reaction.</italic></p> 
<fn id="TN4"><label>&#x0002A;</label><p><italic>Leave 1-out validation.</italic></p></fn>
<fn id="TN5"><label>&#x02021;</label><p><italic>All LR values are calculated based on reported sensitivity and specificity</italic>.</p></fn>
</table-wrap-foot>
</table-wrap>
<p>FISH is another tissue staining modality that has been used for over a decade to help distinguish nevi from melanoma. It has well-developed scoring criteria, several validation studies, has undergone quality improvement to reduce the number of erroneous Spitzoid classifications, and has been directly compared to both the myPath gene expression panel (GEP) and copy number variation (CNV) assay by comparative genome hybridization (CGH) (<xref ref-type="bibr" rid="B42">42</xref>, <xref ref-type="bibr" rid="B43">43</xref>, <xref ref-type="bibr" rid="B49">49</xref>&#x02013;<xref ref-type="bibr" rid="B51">51</xref>). The clinical characteristics of FISH in melanoma diagnosis are highly competitive and cost-effective when compared to the other molecular techniques (<xref ref-type="table" rid="T2">Table 2</xref>) (<xref ref-type="bibr" rid="B43">43</xref>, <xref ref-type="bibr" rid="B49">49</xref>&#x02013;<xref ref-type="bibr" rid="B51">51</xref>). However, evaluating FISH staining requires expert interpretation and expensive microscopy equipment, which limits the adoption of the technique outside of reference laboratories.</p>
<p>The CNV by CGH assay has been used clinically for almost two decades and assesses large genomic mutations and deletions across the genome. Melanomas usually possess multiple chromosomal losses and gains (96.2% frequency), Spitzoid nevi frequently have a unique 11p amplification, and non-Spitzoid nevi usually have no detectable chromosomal changes (<xref ref-type="bibr" rid="B41">41</xref>). While a powerful technique to distinguish nevi from melanomas, there is no clear CNV-based definition of melanoma. As a result, the CNV cut-off that separates a benign vs. malignant lesion is often a gestalt decision from the dermatopathologists, which makes quantifying the clinical accuracy characteristics challenging. However, the price of CGH has fallen in recent years, the technology can be readily adopted by laboratories with experience in molecular diagnostics, and, it has 90% concordance with FISH in cases with unequivocal histopathology (<xref ref-type="bibr" rid="B49">49</xref>). In particularly challenging cases, CGH also outperformed both FISH and myPath in reaching concordance with a consensus histopathological diagnosis (<xref ref-type="bibr" rid="B51">51</xref>).</p>
<p>More recently, the Myriad myPath quantitative gene expression panel (GEP) and scoring algorithm was reported to have favorable test characteristics (AUC of 0.96, sensitivity of 90% and specificity of 91%) (<xref ref-type="bibr" rid="B44">44</xref>). This platform has subsequently been studied across prospectively submitted samples (<xref ref-type="bibr" rid="B45">45</xref>) in the evaluation of desmoplastic melanocytic proliferations (<xref ref-type="bibr" rid="B47">47</xref>) and in conjunction with long-term clinical outcomes (<xref ref-type="bibr" rid="B46">46</xref>). The most recent study is well-designed to assess function in histopathologically uncertain lesions and exemplifies how to use clinical outcome data as the gold standard instead of histopathology (<xref ref-type="bibr" rid="B48">48</xref>). Despite its impressive clinical characteristics, when assessed individually, the myPath methodology is not as competitive in comparison studies to FISH and CGH when histopathology is the gold standard (<xref ref-type="bibr" rid="B50">50</xref>, <xref ref-type="bibr" rid="B51">51</xref>). Additional studies of the myPath test compared directly against FISH and CGH with long term follow up of clinical outcomes as the gold standard are warranted to determine the true relative utility of these tests.</p>
<p>In summary, the existing literature most highly supports the use of PRAME IHC, FISH, and CGH to aid dermatopathologists in routinely stratifying melanoma risk. While the clinical characteristics of CGH are uncertain, the presence of amplifications and deletions would likely prompt a conservative practitioner to upgrade the diagnosis to melanoma (presumed LR&#x0002B; &#x0003E;2). FISH and PRAME IHC easily reach the modeled threshold of utility, even in challenging cases, but would not be sufficient to rule out melanoma (LR&#x02013; is not &#x0003C;0.1&#x02013;0.01) (<xref ref-type="bibr" rid="B50">50</xref>, <xref ref-type="bibr" rid="B51">51</xref>, <xref ref-type="bibr" rid="B63">63</xref>), and it is unclear if a CGH result revealing no amplifications or deletions would have a sufficient LR- to overturn a conservative melanoma diagnosis. The only available biomarker that could potentially have such a robust LR&#x02013; is myPath, but such performance would need to be proven in a head-to-head comparison with the other testing modalities. As it stands, there is no currently used biomarker with a robust enough LR&#x0002B; and LR&#x02013; to be definitive for including or excluding a melanoma diagnosis with certainty, which leaves room for the further development of novel tests and more rigorous validation of current tests.</p>
</sec>
<sec>
<title>Potential Future Biomarkers for the Diagnosis of Pigmented Lesions</title>
<p>Since distinguishing early invasive melanoma from pre-malignant melanocytic lesions by routine histopathology remains a diagnostic challenge (<xref ref-type="bibr" rid="B15">15</xref>), numerous research groups have attempted to develop ancillary tests to augment melanoma diagnosis. Presented below and summarized in <xref ref-type="table" rid="T3">Table 3</xref> are a set of studies that examined promising biomarkers. All of these studies utilize methodologies obtainable by academic reference laboratories, all have at least one test characteristic defined, and most have been assessed in at least one independent test cohort.</p>
<p>One study used IHC to develop a semi-quantitative scoring system based on the percent expression of Ki-67 (MIB-1), p16, and HMB45 expression in melanocytes. Of these commonly available stains, Ki-67 has the best individual sensitivity and specificity, but the addition of p16 and HMB45 increased this to 97.4% sensitivity and 97.3% specificity with an AUC of 0.987. Notably, the IHC panel accurately predicted the metastatic evolution of tumors that were initially classified as melanocytic or Spitzoid nevi, and it appropriately downgraded one melanoma to a benign nevus (<xref ref-type="bibr" rid="B62">62</xref>). While these results are impressive, this test requires stain scoring which is notoriously subjective. Therefore, maintaining a high interobserver concordance with three IHC stains could be challenging, especially with scoring borderline p16 and HMB45 cases. Another proposed diagnostic marker is 5-hydroxymethylcytosine (5-hmC), which is decreased in melanoma compared to nevi, keratinocytes, and lymphocytes. Staining variation limits the sensitivity and specificity of 5-hmC scoring system to 92.7 and 97.7%, respectively, though there is a characteristic biphasic staining pattern between the melanoma and nevi components of a transformed nevus that could be clinically useful according to a pilot study (<xref ref-type="bibr" rid="B61">61</xref>).</p>
<p>Melanomas lose primary cilia after transformation whereas they are retained in benign nevi (<xref ref-type="bibr" rid="B64">64</xref>&#x02013;<xref ref-type="bibr" rid="B66">66</xref>). The presence of primary cilia have been shown to differentiate typical and atypical Spitzoid nevi from malignant tumors with a sensitivity of 81% and specificity of 65% (AUC 0.84), but, with the incorporation of cytologic atypia, hyperchromatism, and asymmetry in making a diagnosis, the combined specificity and AUC were increased (sensitivity 72%, specificity 89%, AUC 0.92) (<xref ref-type="bibr" rid="B60">60</xref>). This is an improvement over standard diagnosis of Spitzoid tumors, which are classically difficult to distinguish between benign and malignant subtypes. Recent pilot studies have also suggested visualization of primary cilia using IHC may also provide similar utility in other melanoma types (<xref ref-type="bibr" rid="B67">67</xref>&#x02013;<xref ref-type="bibr" rid="B69">69</xref>).</p>
<p>Discrimination between nevus and melanoma may also be accomplished using differences in gene expression. The most accessible method of analysis is to perform quantitative real time polymerase chain reaction (qRT-PCR) on a single transcript target from tissue or serum. For instance, silver homolog (also called <italic>SILV, PMEL, gp100</italic>, or <italic>ME20M</italic>) expression from FFPE tissue is significantly lower in melanomas compared to nevi, and it has favorable characteristics (AUC 0.94) as a single biomarker (<xref ref-type="bibr" rid="B57">57</xref>). Similarly, a qRT-PCR panel for 6 microRNAs (miRNA) in serum demonstrated positive and negative likelihood ratios of 20 and 0.07, respectively, in distinguishing uveal melanoma and nevi (<xref ref-type="bibr" rid="B59">59</xref>). Additional work remains to be completed to confirm clinical utility of these studies as neither contained validation cohorts and both had small sample sizes. The importance of a validation cohort is highlighted by our 2019 study where we developed a miRNA ratio trained model from miRNAs enriched in microdissected melanoma cells compared to nevus cells. While the initial training model had an AUC of 0.98, the clinical accuracy of the qRT-PCR validation cohort containing whole FFPE sections dropped to a sensitivity of 81%, specificity of 88%, and AUC of 0.90 (<xref ref-type="bibr" rid="B56">56</xref>). Likewise, a microarray study on microdissected FFPE tissue found that the expression profiles of 36 genes could be used to identify melanoma with sensitivity (90%) and specificity (88%) in the validation cohort (<xref ref-type="bibr" rid="B58">58</xref>). While the gene expression panels described above have test characteristics that are theoretically compatible with a useful clinical test, their reliability is uncertain until larger studies confirm clinical utility.</p>
<p>Of the different classes of proposed biomarkers, DNA methylation testing is not widely available outside of reference laboratory testing for X chromosome disorders and specific genes, such as <italic>MLH1</italic> and <italic>MGMT</italic>. In melanoma, promoter methylation status of two genes have been examined as potential biomarkers. The Ras associated domain family 1 isoform A (<italic>RASSF1A</italic>) gene is a tumor suppressor found to be methylated in melanomas compared to healthy subjects (AUC of 0.905) by cell-free DNA (cfDNA) analysis (<xref ref-type="bibr" rid="B55">55</xref>). While cfDNA allows for a non-invasive blood draw and detects <italic>in situ</italic> melanomas with an AUC of 0.945, no studies have been published validating the <italic>RASSF1A</italic> marker. In comparison, the promoter for <italic>CLDN11</italic>, a component of the tight junction, undergoes CpG island methylation in approximately 50% of primary melanomas compared to 3&#x02013;6% of dysplastic nevi. <italic>CLDN11</italic> promoter methylation status was determined by extraction from FFPE tissue and methylation-specific PCR in a large cohort of melanomas and nevi. This yielded a 52% sensitivity and 94% specificity (AUC of 0.806), suggesting utility for ruling in melanoma in the context of a suspicious dysplastic nevus. Another group developed a CpG site microarray panel and, ultimately, a 40-CpG next generation sequencing (NGS) panel to assess the methylation of status of multiple genes from microdissected FFPE slides. While both of these studies had characteristics compatible with a useful clinical test (sensitivity up to 96.6%, specificity up to 100%, AUC up to 0.996), they are limited by small cohort sizes and lack of long-term patient outcome data (<xref ref-type="bibr" rid="B52">52</xref>, <xref ref-type="bibr" rid="B53">53</xref>).</p>
<p>Overall, there are several potential biomarkers that could be used to distinguish early melanoma from dysplastic nevi. All of the above tests have LRs (<xref ref-type="table" rid="T3">Table 3</xref>) that could be beneficial in distinguishing atypical melanocytic proliferations from melanoma based on our model (<xref ref-type="fig" rid="F4">Figure 4</xref>). Of these biomarkers, we speculate that the most accessible, least complex, minimally invasive, and relatively affordable test that could be readily adopted into the laboratory is 5-hmC IHC (<xref ref-type="bibr" rid="B61">61</xref>). Prior to its adoption, however, large cohort studies are required to confirm the promising initial results. Of all of the tests included in <xref ref-type="table" rid="T2">Tables 2</xref>, <xref ref-type="table" rid="T3">3</xref>, the 40-CpG classifier next generation sequencing test for DNA methylation has the best overall combined characteristics (<xref ref-type="bibr" rid="B52">52</xref>). If future multi-institutional studies show good concordance and validation of the test characteristics, the 40-CpG classifier test could be useful for assessing the most challenging cases of atypical melanocytic proliferations. However, the major limitation of this test is the NGS approach: ample tumor tissue will be required, the method is expensive, and turn-around-time will likely be at least 8&#x02013;14 days. Thus, other tests that require less tissue, cost, time, and expert interpretation, such as the multi-IHC panel or ciliation index, also warrant further consideration (<xref ref-type="bibr" rid="B62">62</xref>, <xref ref-type="bibr" rid="B68">68</xref>).</p>
</sec>
<sec>
<title>Potential Biomarkers for Staging, Prognosis, and Treatment Effect Monitoring</title>
<p>In addition to indications for biopsy and diagnostic accuracy, potential biomarkers were assessed for other important clinical decision points related to the staging and treatment of invasive melanomas (<xref ref-type="fig" rid="F1">Figure 1C</xref>, <xref ref-type="table" rid="T4">Table 4</xref>).</p>
<table-wrap position="float" id="T4">
<label>Table 4</label>
<caption><p>Proposed prognostic, staging, and treatment-monitoring biomarkers.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th valign="top" align="center"><bold>Category</bold></th>
<th valign="top" align="center"><bold>Method class</bold></th>
<th valign="top" align="center"><bold>Method specifics</bold></th>
<th valign="top" align="center"><bold>Total study number</bold></th>
<th valign="top" align="center"><bold>Study breakdown</bold></th>
<th valign="top" align="center"><bold>Estimated tissue requirement</bold></th>
<th valign="top" align="center"><bold>Study objective</bold></th>
<th valign="top" align="center"><bold>Number of pathologists confirming cohort</bold></th>
<th valign="top" align="center"><bold>AUC</bold></th>
<th valign="top" align="center"><bold>S/Sp/PPV/NPV (%), LR&#x0002B;/LR-<sup><bold><xref ref-type="table-fn" rid="TN6">&#x02021;</xref></bold></sup></bold></th>
<th valign="top" align="center"><bold>Qualitative vs. quantitative</bold></th>
<th valign="top" align="center"><bold>Citation</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left" colspan="12"><bold>Prediction of sentinel lymph node biopsy status</bold></td>
</tr>
<tr style="border-bottom: thin solid #000000;">
<td valign="top" align="center">Castle DecisionDx</td>
<td valign="top" align="center">31 GEP</td>
<td valign="top" align="center">TaqMan qRT-PCR from FFPE tissue</td>
<td valign="top" align="center">1421</td>
<td valign="top" align="center">By T: 613 T1, 452 T2, 240 T3, 116 T4</td>
<td valign="top" align="center">macrodissected FFPE sections</td>
<td valign="top" align="center">Identify T1&#x02013;T2 melanomas at low risk for positive SLNB</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B70">70</xref>)</td>
</tr> <tr>
<td valign="top" align="center">Skyline</td>
<td valign="top" align="center">GEP and clinico-pathologic features</td>
<td valign="top" align="center">TaqMan qRT-PCR from FFPE tissue</td>
<td valign="top" align="center">506</td>
<td valign="top" align="center">160 discovery, 360 development, 146 validation</td>
<td valign="top" align="center">FFPE blocks</td>
<td valign="top" align="center">Develop and validate a GEP panel to identify positive SLNB</td>
<td valign="top" align="center">2&#x0002B; dermato-pathologists</td>
<td valign="top" align="center">0.78</td>
<td valign="top" align="center">89/76/NR/NR, 3.71/0.14</td>
<td valign="top" align="center">Quantitative GEP, qualitative clinicopathologic features</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B71">71</xref>)</td>
</tr>
<tr style="border-bottom: thin solid #000000;">
<td/>
<td/>
<td/>
<td valign="top" align="center">754</td>
<td valign="top" align="center">128 positive SLNB, 626 negative SLNB</td>
<td valign="top" align="center">FFPE blocks</td>
<td valign="top" align="center">Identify patients who can forgo SLNB</td>
<td valign="top" align="center">Initial diagnosis made with 2&#x0002B; dermato-pathologists. No other review</td>
<td valign="top" align="center">0.82</td>
<td valign="top" align="center">TIb: 41/82/7/98, 2.28/0.72; TIIa: 80/53/21/95, 1.7/0.24; TIIb: 94/27/21/96, 1.29/0.22; TIIIa: 99/12/33/98, 1.13/0.08; TIIIb: 100/7/45/100, 1.08/ &#x0003C;0.01</td>
<td/>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B72">72</xref>)</td>
</tr> <tr>
<td valign="top" align="left" colspan="12"><bold>Metastasis identification</bold></td>
</tr>
<tr style="border-bottom: thin solid #000000;">
<td valign="top" align="left">Circulating Tumor Cells</td>
<td valign="top" align="center">Flow and fluorescent microscopy</td>
<td valign="top" align="center">CD146 cell sorting. DAPI&#x0002B;, HMW-MAA&#x0002B;, CD45/CD34&#x02013; Immuno-fluorescence</td>
<td valign="top" align="center">15</td>
<td valign="top" align="center">Tumor: 15, 0. Metastasis 6, 9</td>
<td valign="top" align="center">Whole blood</td>
<td valign="top" align="center">Identification of early mets</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">CTC: 33/100/100/69, &#x0003E;30/0.67; CTC&#x0002B;5-S-cysteinyldopa: 67/100/100/82, &#x0003E;60/0.33</td>
<td valign="top" align="center">Semi-quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B73">73</xref>)</td>
</tr> <tr style="border-bottom: thin solid #000000;"> 
<td valign="top" align="left">Serum nucleic acid</td>
<td valign="top" align="center">Circulating non-coding RNA by qRT-PCR</td>
<td valign="top" align="center">hsa-miR-1246 miScript assay for RNU2-1f</td>
<td valign="top" align="center">122</td>
<td valign="top" align="center">33 training, 16 distant metastasis validation, 16 nevi, 57 healthy control</td>
<td valign="top" align="center">Serum draw</td>
<td valign="top" align="center">Use RNU2-1f non-coding RNA serum level to diagnose melanoma mets</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">0.9375 (validation)</td>
<td valign="top" align="center">Regional mets: 69.6/87.2/NR/NR Distant mets: 70.0/87.2/NR/NR</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B74">74</xref>)</td>
</tr> <tr style="border-bottom: thin solid #000000;">
<td valign="top" align="left">Protein Expression</td>
<td valign="top" align="center">ELISA</td>
<td valign="top" align="center">ELISA for S100 and OPN</td>
<td valign="top" align="center">106</td>
<td valign="top" align="center">6 squamous cell carcinomas, 76 melanomas, 24 metastatic mela-nomas, 3 normal</td>
<td valign="top" align="center">Plasma draw</td>
<td valign="top" align="center">Distinguish metastatic melanoma from local disease with OPN &#x0002B;/&#x02013; S100 and LDH</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">OPN and S100: 0.97</td>
<td valign="top" align="center">95.5/85.9/NR/NR, 6.77/0.05</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B75">75</xref>)</td>
</tr> <tr>
<td valign="top" align="left" colspan="12"><bold>Prognosis</bold></td>
</tr>
<tr>
<td valign="top" align="left">Castle DecisionDx</td>
<td valign="top" align="center">31 GEP</td>
<td valign="top" align="center">TaqMan qRT-PCR from FFPE tissue</td>
<td valign="top" align="center">479</td>
<td valign="top" align="center">107 discovery, 268 development, 104 validation</td>
<td valign="top" align="center">macrodissected FFPE sections</td>
<td valign="top" align="center">Identification of stage I/II tumors with worse prognosis</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">0.93</td>
<td valign="top" align="center">100/78/NR/NR, 4.54/ &#x0003C;0.01</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B76">76</xref>)</td>
</tr>
<tr>
<td/>
<td/>
<td/>
<td valign="top" align="center">217</td>
<td valign="top" align="center">All post-SLNB: 58 positive SLN, 159 negative SLN</td>
<td valign="top" align="center">macrodissected FFPE sections</td>
<td valign="top" align="center">Prognostic accuracy of GEP and SLNB in predicting RFS, distant mets, and OS</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">N.R./N.R./50/82</td>
<td/>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B77">77</xref>)</td>
</tr>
<tr style="border-bottom: thin solid #000000;">
<td/>
<td/>
<td/>
<td valign="top" align="center">205</td>
<td valign="top" align="center">By stage: 2 I, 68 IA, 39 IB, 1 II, 40 IIA, 41 IIB, 14 IIC</td>
<td valign="top" align="center">Biopsy</td>
<td valign="top" align="center">Compare GEP to AJCC in predicting 5 yr RFS, distant mets, and OS</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">GEP&#x0002B;AJCC for Recurrence: 90/71/NR/NR, 3.10/0.14; Distant Mets: 88/63/NR/NR., 2.38/0.19; Death: 82/62/NR/NR, 2.16/0.29</td>
<td/>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B78">78</xref>)</td>
</tr> <tr style="border-bottom: thin solid #000000;">
<td valign="top" align="left">Gene Expression</td>
<td valign="top" align="center">RT-PCR</td>
<td valign="top" align="center">qRT-PCR for PAX3d to monitor recurrent in stage II-IV disease</td>
<td valign="top" align="center">198</td>
<td valign="top" align="center">111 melanoma, 87 healthy controls</td>
<td valign="top" align="center">Plasma draw</td>
<td valign="top" align="center">Identify recurrence in stage II-IV disease with plasma PAX3d mRNA</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">0.823</td>
<td valign="top" align="center">Stage II&#x02013;III relapse: 67/75/67/75, 2.68/0.44; Stage IV relapse: 75/93/43/98, 10.7/0.27</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B79">79</xref>)</td>
</tr> <tr style="border-bottom: thin solid #000000;">
<td valign="top" align="left">Serum nucleic acid</td>
<td valign="top" align="center">Circulating tumor DNA</td>
<td valign="top" align="center">PCR of BRAF and NRAS mutants</td>
<td valign="top" align="center">125</td>
<td valign="top" align="center">20 with progression, 9 with pseudo-progression</td>
<td valign="top" align="center">Plasma draw</td>
<td valign="top" align="center">Differentiate radiologic pseudo-progression and true progression</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">90/100/100/82, &#x0003E;90/0.1</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B80">80</xref>)</td>
</tr> <tr>
<td valign="top" align="left" colspan="12"><bold>Monitoring of treatment effect</bold></td>
</tr>
<tr>
<td valign="top" align="left">Serum nucleic acid</td>
<td valign="top" align="center">Circulating tumor DNA</td>
<td valign="top" align="center">BRAF V600mut ctDNA by qRT-PCR</td>
<td valign="top" align="center">36</td>
<td valign="top" align="center">16 before therapy, 20 after therapy</td>
<td valign="top" align="center">Plasma draw</td>
<td valign="top" align="center">Monitor patient response to BRAF/MEK inhibition therapy with BRAF V600mut ctDNA</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">NR</td>
<td valign="top" align="center">70/100/NR/NR., &#x0003E;70/0.30</td>
<td valign="top" align="center">Quantitative</td>
<td valign="top" align="center">(<xref ref-type="bibr" rid="B81">81</xref>)</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p><italic>AUC, area under (receiver operating characteristic) curve; CTC, circulating tumor cells; ctDNA, circulating tumor DNA; ELISA, enzyme-linked immunosorbent assay. FFPE, formalin-fixed paraffin embedded; GEP, gene expression panel; LR&#x0002B;, positive likelihood radio; LR&#x02013;: negative likelihood radio; Mets, metastasis; NPV, negative predictive value; NR, not reported; OS, overall survival; PPV, positive predictive value; RFS, recurrence-free survival; S, sensitivity; qRT-PCR, quantitative real time polymerase chain reaction; SLNB, sentinel lymph node biopsy; Sp, specificity.</italic></p> 
<fn id="TN6"><label>&#x02021;</label><p><italic>All LR values are calculated based on reported sensitivity and specificity</italic>.</p></fn>
</table-wrap-foot>
</table-wrap>
<sec>
<title>Prognostic Biomarkers</title>
<p>Prognostic biomarkers compose the broadest category of biomarkers. In terms of clinical decision impact, they can range from nebulous indicators of survival odds to potential heralds of disease recurrence. The principal issue with survival indicators is that they may require adoption by the AJCC or other analogous staging protocols prior to widespread utilization. Otherwise, the ordering clinician may be unsure how an unfavorable result will affect their patient&#x00027;s stage, choice of intervention, and ultimately outcome. A biomarker with excellent test characteristics may therefore be effectively useless if it is not incorporated into a staging framework from which clinical decisions can be made.</p>
<p>One commercially available prognostic test is the Castle DecisionDx assay. While the DecisionDx assay is not included in AJCC staging nor NCCN treatment guidelines, the test aims to identify patients with stage I and II disease who may be at increased risk of metastasis and death. Analytically, this gene expression profile (GEP) has demonstrated reproducibility across instruments in separating &#x0201C;class 1&#x0201D; (low-risk) from &#x0201C;class 2&#x0201D; (high-risk) probability scores and in technical concordance between runs (<xref ref-type="bibr" rid="B82">82</xref>). Results from small retrospective and prospective clinical cohorts have suggested improved ability to stratify risk of relapse and distant metastasis independent of SLNB status (<xref ref-type="bibr" rid="B83">83</xref>&#x02013;<xref ref-type="bibr" rid="B87">87</xref>). However, each of these studies are limited by the lack of stage or substage breakdown of test performance, rendering it difficult to determine how much additional information is imparted by the biomarker beyond existing staging criteria. Podlipnik et al. studied the additive utility of DecisionDx probability scores in conjunction with combined AJCC stages (&#x0201C;low-risk&#x0201D;: IB-IIA, &#x0201C;high-risk&#x0201D;: IIB-IIC) and found disease free survival was lower in melanoma patients with DecisionDx high-risk &#x0201C;class 2&#x0201D; scores, regardless of grouped AJCC staging (<xref ref-type="bibr" rid="B87">87</xref>). Furthermore, in a 205 patient cohort, comprised of combined Stage I and II melanoma cases, where the DecisionDx GEP was combined with the AJCC Individualized Melanoma Patient Outcome Prediction Tool, the authors reported improved sensitivity but worse specificity for predicting disease recurrence, distant metastasis, and death compared to either metric alone (<xref ref-type="bibr" rid="B78">78</xref>). However, because neither positive predictive value/negative predictive values (PPV/NPV), nor AJCC substage breakdown of DecisionDx test performance were reported, it is challenging to determine the utility of this test beyond AJCC staging criteria, and it remains to be determined if and how this test can be integrated into current standard of care.</p>
<p>The AJCC melanoma staging and substaging criteria are built upon robust collection of patient clinicopathologic features and have been refined over decades using long-term outcomes data for more than 45,000 patients with melanoma (<xref ref-type="bibr" rid="B1">1</xref>). As such, this is the standard to which prognostic biomarkers must be measured, and full transparency of clinical characteristics and AJCC substages of patients in future studies will be necessary to meaningfully compare prognostic outcomes to AJCC staging (<xref ref-type="bibr" rid="B88">88</xref>, <xref ref-type="bibr" rid="B89">89</xref>). Because of these limitations, a recent consensus group statement from the melanoma prevention working group members recommended against routine use of prognostic GEPs, including the DecisionDx, outside of clinical trials, without thorough discussion of testing limitations with the patient, or outside of multidisciplinary conference recommendations (<xref ref-type="bibr" rid="B5">5</xref>).</p>
</sec>
<sec>
<title>Stratifying Risk of Lymph Node Metastasis</title>
<p>A major clinical decision point is to perform a sentinel lymph node biopsy (SLNB) for patients with T1b tumors or in AJCC stages IB and II, which will upstage the patient to stage III if positive (<xref ref-type="table" rid="T1">Table 1</xref>). At present, the benefit of SLNB is in determining who is eligible for adjuvant systemic therapy (<xref ref-type="bibr" rid="B90">90</xref>). Examples of adjuvant therapy include the combination of dabrafenib plus trametinib in resected Stage III BRAFV600E/K mutant melanoma, which improves 5 year overall survival (<xref ref-type="bibr" rid="B91">91</xref>), and pembrolizumab which improves relapse free survival in high risk, resected, Stage III melanoma patients (<xref ref-type="bibr" rid="B92">92</xref>). Since current NCCN guidelines recommend SLNB if the odds of a positive node are &#x0003E;5% (<xref ref-type="bibr" rid="B90">90</xref>), the positive rate of SLNB is low (<xref ref-type="bibr" rid="B93">93</xref>). In theory, there may be significant clinical utility for risk-stratifying biomarkers to determine which patients may benefit from this procedure (<xref ref-type="fig" rid="F5">Figure 5</xref>). Because of the large degree of uncertainty as to which patients will benefit from SLNB, minimal changes in LR of LN metastasis may be sufficient to alter clinical decision making in T1b melanomas (<xref ref-type="fig" rid="F5">Figures 5A,D</xref>). Further, the risk to the patient of an undetected LN metastasis (a missed opportunity to treat Stage III disease before progression) must be incorporated into the decision. Indeed, improved efficacy of adjuvant systemic therapy alters the risk-benefit analysis of SLNB, potentially shifting the decision to favor surgery in some patients.</p>
<fig id="F5" position="float">
<label>Figure 5</label>
<caption><p>Utility of biomarkers to augment Breslow depth in determining which patients are offered sentinel lymph node biopsy (SLNB). <bold>(A)</bold> As depth of melanoma increases, from <italic>in situ</italic> to greater than one millimeter, the probability of sentinel lymph node positivity, and thereby stage and treatment selection, increases. Sentinel lymph node biopsy (SLNB) is positive in &#x0007E;5% of patients with T1b melanoma, resulting in a decision threshold posterior odds of &#x0007E;1:20 for many surgical oncologists to offer SLNB and reflecting NCCN guideline recommendations (<xref ref-type="bibr" rid="B90">90</xref>). As SLNB is not routinely offered for melanoma <italic>in situ</italic> or T1a melanomas, the incidence of SLNB positivity is unknown. We therefore estimate that these are substantially and progressively less positive as tumor thickness decreases. SLNB is standard of care for T2&#x0002B; melanomas (gray dot) and thus no biomarker is likely to alter this recommendation without exceptionally low LR- test characteristics. <bold>(B)</bold> Currently patients with melanoma <italic>in situ</italic> are not generally offered SLNB since the likelihood of positivity (prior odds) is very low. A very high LR&#x0002B;, estimated at 25, would be necessary to recommend SLNB. <bold>(C)</bold> As the melanoma Breslow depth increases, the prior odds of sentinel lymph node metastasis increases (becomes closer to 1:1). Therefore, a lower LR&#x0002B;, such as LR&#x0002B; &#x02265;10, would be necessary to prompt SLNB. <bold>(D)</bold> Biomarkers may have the greatest utility for T1b malignant melanomas. Current NCCN guidelines recommend thoughtful consideration of SLNB based on patient-specific factors, and any test that could offer relatively small changes in certainty could be of great clinical utility. Thus, a modest LR&#x0002B; of 2.0 or LR&#x02013; of 0.5 may be sufficient for a clinically-actionable biomarker.</p></caption>
<graphic xlink:href="fmed-08-642380-g0005.tif"/>
</fig>
<p>In theory, a prognostic assay with high LR- or NPV could rule out SNLB eligible patients who would not benefit from the procedure. Conversely, a test with high LR&#x0002B; or PPV could identify patients who may benefit from SLNB but otherwise would not have been offered (i.e., melanoma <italic>in situ</italic>; <xref ref-type="fig" rid="F5">Figure 5B</xref>, or T1a melanoma; <xref ref-type="fig" rid="F5">Figure 5C</xref>). In reality, NPV and PPV are often inversely related, and as such, the goals of the test must be clearly defined. For a molecular assay to replace the gold standard SLN procedure in patients without clinically positive nodes (<xref ref-type="bibr" rid="B94">94</xref>), which at present is the only determinant of patient eligibility for systemic therapy (<xref ref-type="bibr" rid="B90">90</xref>), the assay must have an NPV higher than the SLN procedure [NPV of SLNB reported up to 94% (<xref ref-type="bibr" rid="B95">95</xref>)]. While the value of the assay remains uncertain, SkylineDx has conducted validation studies for this indication. In their initial study, SkylineDx combined established prognostic factors, such as patient age and Breslow depth, with their 11 GEP to predict SLN metastasis (<xref ref-type="bibr" rid="B71">71</xref>), thereby convoluting the contribution of the GEP to the classification. Concerningly, the initial gene set was defined using a cohort of only seven patients&#x02014;three with and four without regional metastasis (<xref ref-type="bibr" rid="B72">72</xref>)&#x02014;calling into question the likelihood the GEP will perform well across a greater diversity of patients.</p>
<p>A recent consensus statement expressed concern about the lack of test performance metrics in SkylineDx&#x00027;s studies (<xref ref-type="bibr" rid="B5">5</xref>). These values have since been released (<xref ref-type="bibr" rid="B72">72</xref>), demonstrating a specificity of 53% and LR- of 0.24 for TIIa disease with otherwise unremarkable test characteristics (<xref ref-type="table" rid="T4">Table 4</xref>). Based on these results and our model (<xref ref-type="fig" rid="F5">Figure 5</xref>), there is a narrow window of utility for determining which TIIa tumors would benefit from a SLNB and a larger range of utility in determining which patients could safely forego SLNB using this test. However, the combined low number of positive SLNB (128 of 754 biopsies) and small discovery cohort (<italic>n</italic> = 7) are concerning given the heterogeneity of melanoma. Extensive validation on diverse cohorts that report a NPV higher than SLNB are required to determine whether the SkylineDx GEP is capable of answering the clinical questions for which these tests are ordered.</p>
<p>Similarly, a 2019 Castle DecisionDx study claimed the 31 GEP assay distinguished individuals &#x0003E;55 years of age with T1-T2 disease who have a &#x0003C;5% chance of SLN involvement. However, no measures of accuracy or predictive value were presented, making it challenging to assess the clinical utility of the test for this application (<xref ref-type="bibr" rid="B70">70</xref>). Of greater concern, the Castle DecisionDx test was developed to assess for metastatic risk associated with a primary melanoma, not to predict SLN involvement. Although the repurposing of a clinical test developed as an indicator of one condition to another is possible, successful performance would be fortuitous, especially since previous validation studies assessing metastasis risk groups are irrelevant when considering this test&#x00027;s utility for assessing the value of SLNB. Thus, while the Castle DecisionDx test may eventually improve the NPV of SLNB in prognosticating relapse or distant metastasis (<xref ref-type="bibr" rid="B83">83</xref>&#x02013;<xref ref-type="bibr" rid="B85">85</xref>), without further study this test should not be considered to supplant SLNB, which remains the most important prognostic staging element guiding further treatment (<xref ref-type="bibr" rid="B90">90</xref>).</p>
<p>To determine the impact of these GEPs, and other tests, on decision to biopsy SLNs and their role in staging (see below), a large retrospective study and/or longitudinal studies will be necessary to determine how these biomarkers predict clinically meaningful outcomes like distant metastasis free-survival and all-cause mortality. Meta-analysis of existing studies has demonstrated that currently available GEP tests have limited ability to prognosticate recurrence in early stage melanomas (<xref ref-type="bibr" rid="B96">96</xref>). Due to the cost of conducting such trials, in terms of economic considerations and irreplaceable patient samples, the melanoma community will need to identify and prioritize the coordinated validation of only the most promising biomarker assays. Considerations for test selection need to include not only test characteristics, but also the details by which those characteristics were generated (size and diversity of cohorts, independence of studies, selection bias) and the likelihood to change clinical decision making when considering current guidelines.</p>
</sec>
<sec>
<title>Monitoring for Disease Recurrence</title>
<p>Monitoring for disease recurrence via serial imaging is costly and, depending on modality, can lead to significant cumulative radiation exposure in patients. Use of blood- or serum-based biomarker assays (termed liquid biopsies) may be able to identify residual or recurrent disease prior to demonstrable evidence of such disease with imaging studies (<xref ref-type="bibr" rid="B97">97</xref>). Prediction or early identification of distant metastasis would enable clinicians to begin adjuvant systemic therapy prior to confirmation of macroscopic metastatic disease by radiology.</p>
<p>Current AJCC staging criteria and NCCN guidelines incorporate one serum-based prognostic biomarker in the treatment of melanoma: lactate dehydrogenase (LDH), a proxy for tumor lysis and disease burden (<xref ref-type="bibr" rid="B1">1</xref>, <xref ref-type="bibr" rid="B90">90</xref>). Several studies have demonstrated that a baseline LDH level &#x0003E;2 times above the laboratory-established upper reference range is associated with poor survival outcomes in patients with stage IV melanoma, despite the advent of targeted immunological agents (<xref ref-type="bibr" rid="B98">98</xref>&#x02013;<xref ref-type="bibr" rid="B101">101</xref>). However, due to lack of sensitivity and biological specificity for melanoma, LDH has been relegated to advanced metastatic disease staging and treatment effect monitoring (<xref ref-type="bibr" rid="B1">1</xref>, <xref ref-type="bibr" rid="B90">90</xref>). Other studies have proposed biomarkers with test characteristics superior to LDH. These include U2 small nuclear RNA, which is reported to correlate with increased risk of regional (AUC 0.80) and distant (AUC 0.94) metastasis (<xref ref-type="bibr" rid="B74">74</xref>), and serum <italic>PAX3d</italic> mRNA which has potential to monitor recurrence in stage IV disease, with specificity up to 93% and NPV up to 98% (<xref ref-type="bibr" rid="B79">79</xref>). However, further prospective testing is necessary to determine if and how these tests could be incorporated into current treatment recommendations for the prediction of early metastasis.</p>
<p>In a proof-of-concept study, circulating tumor cells (CTC) quantification from peripheral blood was a specific indicator of peripheral metastasis with a PPV of 100% (<xref ref-type="bibr" rid="B73">73</xref>). Besides the small cohort size (<italic>n</italic> = 15), one major limitation of this study was the unusual combination of immunomagnetic cell sorting and four-color immunofluorescence microscopy. A second challenge for CTC approaches in general is the propensity for melanoma cells to de-differentiate and thereby down-regulate lineages-specific cell markers used for selecting CTCs (S100, MelanA, SOX10). If CTC quantification could be adapted to a more widely available and objective platform, such as flow cytometry, and invariant melanoma markers identified, then it might find utility as a routine liquid biopsy. Alternatively, plasma concentrations of proteins other than LDH, such as osteopontin and S100 (<xref ref-type="bibr" rid="B75">75</xref>), may have functionality in predicting early metastasis, but have yet to be validated.</p>
<p>Identifying and validating biomarkers for minimal residual disease burden (MRD) have the potential to reduce imaging frequency, assess post-treatment disease burden, and identify microscopic disease relapse that is undetectable with standard imaging. One promising assay type is the quantification of common melanoma mutations as circulating tumor DNA (ctDNA) from peripheral blood. While there are ctDNA assays approved to identify actionable mutations in a variety of solid tumor types, these assays usually have a limit of detection (LOD) between 0.1 and 1.0% of ctDNA, which corresponds to 1 tumor sequence per 100&#x02013;1,000 normal sequences (<xref ref-type="bibr" rid="B102">102</xref>). Current molecular techniques including digital droplet PCR, allele-specific oligonucleotide primer PCR, and NGS have the ability to further increase sensitivity by detecting known mutant alleles, resulting in an LOD between 0.001 and 0.1% of sequences for hematologic malignancies such as multiple myeloma (<xref ref-type="bibr" rid="B103">103</xref>). Detecting solid tumor ctDNA in peripheral blood has been more challenging due to the lower estimated amount of ctDNA present in non-metastatic cancer patients (<xref ref-type="bibr" rid="B102">102</xref>), but recent advances using techniques such as targeted digital sequencing (TARDIS) have improved the LOD of tumor-specific sequences to 0.01% of circulating DNA (<xref ref-type="bibr" rid="B104">104</xref>), likely representing the absolute minimum sensitivity required to detect a 5 mm primary tumor in breast cancer (<xref ref-type="bibr" rid="B102">102</xref>). Since the amount of ctDNA in patients with melanoma is on average 2 orders of magnitude less compared to breast cancer (<xref ref-type="bibr" rid="B105">105</xref>, <xref ref-type="bibr" rid="B106">106</xref>), it is expected that a more sensitive assay with a lower LOD will be required to assess MRD in melanoma.</p>
<p>Several pilot studies have assessed the use of ctDNA as a MRD marker to monitor disease progression and relapse, with a few achieving a competitive LOD. One example demonstrated that monitoring <italic>BRAF</italic> V600 mutant sequences could identify disease progression prior to radiographic evidence in 44% of cases and was co-identified with imaging in an additional 26% (sensitivity 70%, specificity 100%, LOD 0.01%) (<xref ref-type="bibr" rid="B81">81</xref>). Similarly, a second study examining the most common <italic>NRAS</italic> and <italic>BRAF</italic> mutants found that detectable levels of ctDNA did not occur in cases with radiologic pseudoprogression (sensitivity 90%, specificity 100%, PPV 100%, NPV 82%, LOD &#x0003C;10 copies/ml or &#x0007E;1%) (<xref ref-type="bibr" rid="B80">80</xref>). The only study with a prospective validation cohort (<italic>n</italic> = 36) found that increasing mutant <italic>TERT</italic>/<italic>NRAS</italic>/<italic>BRAF</italic> ctDNA preceded radiologic evidence of progression by an average of 3.5 months (AUC of 0.85&#x02013;0.87, LOD: BRAF<sup>V600E</sup> and NRAS<sup><italic>Q</italic>61</sup> 0.01%, TERT<sup>C250T</sup> 1 copy/reaction and TER<sup>C228T</sup> 10 copies/reaction) and were superior to LDH and S100 serology for this indication (<xref ref-type="bibr" rid="B107">107</xref>). Additional studies are needed to confirm these findings in a larger cohort, to determine the impact of observed mutational heterogeneity on test performance (<xref ref-type="bibr" rid="B107">107</xref>), and to find surrogates for melanomas that do not bear a <italic>TERT, NRAS</italic>, or <italic>BRAF</italic> mutation.</p>
</sec>
</sec>
<sec>
<title>The Wild West of Predictive Biomarkers for Systemic Therapy</title>
<p>For Stage III melanomas that have metastasized to sentinel lymph nodes and for those in which nodal or in transit metastatic disease has been surgically resected, adjuvant therapy using anti PD-1 immunotherapy or BRAF/MEK inhibition is currently recommended for consideration based on recent clinical trial data showing improved relapse-free survival (RFS) (<xref ref-type="bibr" rid="B91">91</xref>, <xref ref-type="bibr" rid="B108">108</xref>). There is significant interest in using adjuvant therapy in Stage II high-risk melanomas, as evidenced by a large, double-blind, prospective, randomized clinical trial currently evaluating the safety and efficacy of immune checkpoint blockade in resected Stage II melanoma (KEYNOTE-716: <ext-link ext-link-type="uri" xlink:href="https://clinicaltrials.gov/ct2/show/NCT03553836">https://clinicaltrials.gov/ct2/show/NCT03553836</ext-link>). Recently an 11-gene GEP was published that demonstrated the ability to stratify patients with stage II melanoma into high GEP score groups and low GEP score groups, with the latter having significantly higher melanoma specific survival and relapse free survival rates (<xref ref-type="bibr" rid="B109">109</xref>). The authors of this study speculate that patients with higher GEP scores may be better candidates for adjuvant therapy due to the increased risk for relapse and death, potentially reflecting ambitions to test this hypothesis in a future study (<xref ref-type="bibr" rid="B109">109</xref>).</p>
<p>If an actionable <italic>BRAF</italic> V600 mutation is identified by sensitive and specific molecular testing (<xref ref-type="bibr" rid="B110">110</xref>), then BRAF plus MEK inhibitor therapy is an option for first-line therapy in resected Stage III and in Stage IV melanoma patients (<xref ref-type="bibr" rid="B90">90</xref>). Regardless of BRAF mutation status, immune checkpoint inhibitors (ICI) against PD-1 and CTLA-4 are frequently used due to improved progression-free and overall survival compared to both chemotherapy and combined BRAF/MEK inhibition (<xref ref-type="bibr" rid="B111">111</xref>&#x02013;<xref ref-type="bibr" rid="B113">113</xref>). However, &#x0007E;65% of Stage IV metastatic melanomas do not significantly respond to ICI therapy (<xref ref-type="bibr" rid="B114">114</xref>&#x02013;<xref ref-type="bibr" rid="B116">116</xref>), indicating an unfilled niche for future biomarkers to discriminate responders from non-responders prior to therapy initiation. Furthermore, accurately predicting which Stage III, Stage II, and even Stage I patients, who comprise the majority of cases of melanoma specific mortality (<xref ref-type="bibr" rid="B117">117</xref>), have the potential to benefit from adjuvant therapy is a major goal of predictive biomarkers in melanoma.</p>
<p>With the development of new ICIs, kinase inhibitors, combination therapy regimens, and other novel treatment modalities, there will be an increasing need for biomarkers to match patients to optimal adjuvant therapies. Successful predictive biomarkers will have the potential to decrease patient morbidity by avoiding side-effects of less helpful treatments, boost response rates by identifying the most appropriate therapy upfront, and increase the cost-effectiveness of treatment by avoiding expensive ICIs when not indicated. When there are multiple treatment options considered first- or second-line treatments, there is even more value in the role of biomarkers guiding therapy selection, however the dichotomous framework that we have presented for other key decision-making points in the treatment of melanoma is less applicable. The complex decisions surrounding adjuvant therapy selection would necessitate multi-dimensional LR&#x0002B;/LR- analyses not attempted in this review due to exponentially increasing complexity. We await the results of ongoing studies and clinical trials in this field and refer the reader to several recent reviews that cover the impressive breadth of proposed biomarkers for use with ICI (<xref ref-type="bibr" rid="B118">118</xref>&#x02013;<xref ref-type="bibr" rid="B121">121</xref>).</p>
</sec>
</sec>
<sec sec-type="discussion" id="s3">
<title>Discussion</title>
<p>Many studies have addressed the potential utility of biomarkers in the diagnosis and treatment of melanoma, highlighting the drive to improve melanoma patient care through novel technologies. In this review, we have discussed several key clinical decision points in the diagnosis and treatment of melanoma including determining when to biopsy, aiding in the histopathologic diagnosis of melanoma, and stratification of prognosis of malignant melanoma. We have applied fundamental theory of clinical test utility to model these decision points and identify test characteristics required for a biomarker to influence decision making. This discussion is not intended to be an ultimate determination of the utility of any specific biomarker but rather to highlight how clinical-use-assumptions affect the characteristics required of a test to be potentially incorporated into practice. We hope this framework and analysis will assist basic science researchers in designing investigations with a greater likelihood of true translation and aid clinical researchers in identifying which candidate biomarker tests are worth allocating resources for validation.</p>
<p>Similar to other malignancies, melanoma is best treated when diagnosed early. When a melanoma is identified prior to invasion, treatment has high rates of cure with wide local excision alone (<xref ref-type="bibr" rid="B1">1</xref>, <xref ref-type="bibr" rid="B122">122</xref>) but as invasion depth increases toward eventual metastasis, treatment becomes increasingly morbid and survival rate decreases. An important observation that arises from the modeling applied here is that across this spectrum, the characteristics of an ideal biomarker for melanoma change to reflect the changing clinical considerations surrounding treatment morbidity and efficacy. For early melanomas (T0-T1a), specificity (and thus LR&#x02013;) is emphasized for diagnosis to avoid missing melanomas that are treatable with excision alone. At this stage of diagnosis, an intermediate lesion will default to treatment given the relatively low morbidity of excision, and as a result, sensitivity is generally less important. However, for primary malignant melanomas with a more significant invasive component (T1b-T4), the question of staging and assessing SLNB utility becomes important. Here the utility of a biomarker with high sensitivity (LR&#x0002B;), specificity (LR&#x02013;) or both become quite useful. With Stage III&#x0002B; melanomas demonstrating metastatic behavior, biomarkers with optimized specificity characteristics (LR&#x0002B;) suggesting response to specific treatment regimens will potentially be more useful given the higher morbidity and cost associated with adjuvant therapies.</p>
<p>Ideal biomarkers would prioritize neoplasia with true malignant potential for appropriate treatment, while providing confidence that intermediate neoplasia with reassuring biomarker findings can safely avoid further treatment. However, due to a lack of an ideal gold standard for verification, rigorous clinical validation of candidate tests by conventional standards is difficult for ambiguous lesions. Practical considerations of cost, ease of use, and tissue requirement will also factor into which biomarkers can be incorporated into clinical practice. Thus, the field should take great care when considering which biomarkers to prioritize for validation. While immunohistochemistry, copy number variation, and FISH represented the mainstay of biomarker use in this space for many years, more recently reported techniques including gene expression profiling, DNA methylation, and ciliation index have the potential to provide superior test characteristics and objective interpretation at lower cost and tissue requirement and thus represent exciting new potential avenues for further improvement in the diagnosis of melanoma (<xref ref-type="bibr" rid="B44">44</xref>, <xref ref-type="bibr" rid="B52">52</xref>, <xref ref-type="bibr" rid="B60">60</xref>).</p>
<p>The challenges of developing novel biomarkers for use in melanoma diagnosis are significant. One of the most substantial obstacles is the inherent subjectivity of histopathologic diagnosis of melanoma. When validation studies rely on histopathologic diagnosis, ambiguous lesions are often excluded, though these are the exact cases most in need of adjudication with an objective biomarker. Because of the imperfect nature of this gold standard, several studies have used consensus diagnosis from three or more experienced dermatopathologists (<xref ref-type="bibr" rid="B45">45</xref>, <xref ref-type="bibr" rid="B46">46</xref>), which we would recommend of future studies. At a minimum, reporting the number of pathologists diagnosing each case, and whether or not they are experts in pigmented lesions, is helpful to determine the reliability of the study. Given the subjectivity of histopathologic diagnosis, we should strive to review the performance of biomarkers compared to long-term patient follow up data, despite the large sample sizes, cost, and time necessary to conduct such studies. Recurrence-free survival and melanoma-specific survival are perhaps the true gold standards, and gathering these data is essential to incorporate biomarkers into the standard of care and meaningfully improve patient outcomes.</p>
<p>Because of the high level of uncertainty surrounding the determination of which patients will benefit from SLNB, relatively small changes in certainty imparted by a biomarker may result in large changes in clinical management. The models applied here suggest that biomarkers with high LR&#x0002B;/sensitivity characteristics have the potential to confirm the utility of SLNB, while biomarkers with high LR-/specificity characteristics may prove useful in determining which patients can safely forego SLNB. Due to the high cost and morbidity associated with SLNB compared to wide local excision alone, and the potential for biomarkers to prove clinically useful with either high LR&#x0002B;, LR&#x02013; or both, significant attention has rightfully been paid to biomarkers that address the question of which patients should undergo SLNB. Compared to diagnostic biomarkers, those targeting the necessity of SLNB require a relatively smaller change in certainty (smaller LR&#x0002B; or LR&#x02013;) to be clinically actionable. This, coupled with the costs of SLNB, financial and otherwise, suggest that biomarker tests in this space are worthy of the significant resources needed for further development.</p>
<p>Regardless of the purpose of a candidate biomarker, large prospective validation studies conducted on diverse and relevant patient populations that demonstrate clinically meaningful outcomes over time are needed to propel biomarkers into the mainstay of treatment. The time, large sample sizes, potential coordination across multiple trial sites, and significant expense of such studies present major barriers to generalizable validation studies. Because of these challenges, the vast majority of published biomarkers do not have multiple validation studies. Even those that do (such as Myriad myPath Melanoma and Castle Biosciences DecisionDx-Melanoma) (<xref ref-type="bibr" rid="B44">44</xref>, <xref ref-type="bibr" rid="B76">76</xref>) have yet to be incorporated into NCCN guidelines due to lack of evidence in ambiguous melanocytic neoplasms and lack of evidence of actionable prognostic information across the melanoma spectrum (<xref ref-type="bibr" rid="B90">90</xref>). Due to these lengthy development and validation phases, combined with the pressing need for clinical tests, novel biomarkers are sometimes ordered by clinicians prior to robust establishment of clinical utility (<xref ref-type="bibr" rid="B35">35</xref>, <xref ref-type="bibr" rid="B123">123</xref>). It is generally recommended that clinical tests should not be ordered that do not have the potential to change management or prognosis. Unnecessary testing can result in unanticipated results that complicate an otherwise straightforward diagnosis, cause undue physical and emotional hardship on the patient, lead to additional unwarranted testing, and increases costs borne by the patient and medical infrastructure (<xref ref-type="bibr" rid="B124">124</xref>&#x02013;<xref ref-type="bibr" rid="B127">127</xref>).</p>
</sec>
<sec sec-type="conclusions" id="s4">
<title>Conclusions</title>
<p>The challenges of clinically validating a biomarker are significant. Given the substantial resources necessary to comprehensively validate a biomarker, clarity regarding the clinical question being addressed and the practical considerations that would confer clinical utility should be fundamental considerations prior to initiating a study. This review applies the fundamental theory of biomarker utility to estimate the necessary test characteristics required to influence clinical decision making in the diagnosis and treatment of melanoma, and summarizes the test characteristics and rigor of existing and candidate tests. It is clear that there is significant room for improvement in the validation of biomarkers in this space, and we hope that the models and considerations summarized here will help guide the community in study design and investment that will yield the maximum clinical utility.</p>
</sec>
<sec id="s5">
<title>Author Contributions</title>
<p>DD and RJ-T conceptualized this work. DD, ES, and RJ-T performed review of the literature and wrote the manuscript. All authors contributed to the article and approved the submitted version.</p>
</sec>
<sec sec-type="COI-statement" id="conf1">
<title>Conflict of Interest</title>
<p>RJ-T is an inventor on international patent application no. PCT/US2019/023834 concerning the use of microRNA ratios as molecular diagnostics for melanoma. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
</body>
<back>
<ack><p>We thank Dr. Allie Grossmann, Dr. John Hyngstrom, Dr. Robert Schmidt, Dr. Douglas Grossman, and Dr. Scott Florell for helpful discussions and critical editing during the preparation of this manuscript.</p>
</ack>
<ref-list>
<title>References</title>
<ref id="B1">
<label>1.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gershenwald</surname> <given-names>JE</given-names></name> <name><surname>Scolyer</surname> <given-names>RA</given-names></name> <name><surname>Hess</surname> <given-names>KR</given-names></name> <name><surname>Sondak</surname> <given-names>VK</given-names></name> <name><surname>Long</surname> <given-names>GV</given-names></name> <name><surname>Ross</surname> <given-names>MI</given-names></name> <etal/></person-group>. <article-title>Melanoma staging: evidence-based changes in the American Joint Committee on Cancer eighth edition cancer staging manual</article-title>. <source>CA Cancer J Clin.</source> (<year>2017</year>) <volume>67</volume>:<fpage>472</fpage>&#x02013;<lpage>92</lpage>. <pub-id pub-id-type="doi">10.3322/caac.21409</pub-id><pub-id pub-id-type="pmid">29028110</pub-id></citation></ref>
<ref id="B2">
<label>2.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chapman</surname> <given-names>PB</given-names></name> <name><surname>Hauschild</surname> <given-names>A</given-names></name> <name><surname>Robert</surname> <given-names>C</given-names></name> <name><surname>Haanen</surname> <given-names>JB</given-names></name> <name><surname>Ascierto</surname> <given-names>P</given-names></name> <name><surname>Larkin</surname> <given-names>J</given-names></name> <etal/></person-group>. <article-title>Improved survival with vemurafenib in melanoma with BRAF V600E mutation</article-title>. <source>N Engl J Med.</source> (<year>2011</year>) <volume>364</volume>:<fpage>2507</fpage>&#x02013;<lpage>16</lpage>. <pub-id pub-id-type="doi">10.1056/NEJMoa1103782</pub-id><pub-id pub-id-type="pmid">22646765</pub-id></citation></ref>
<ref id="B3">
<label>3.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hauschild</surname> <given-names>A</given-names></name> <name><surname>Grob</surname> <given-names>JJ</given-names></name> <name><surname>Demidov</surname> <given-names>LV</given-names></name> <name><surname>Jouary</surname> <given-names>T</given-names></name> <name><surname>Gutzmer</surname> <given-names>R</given-names></name> <name><surname>Millward</surname> <given-names>M</given-names></name> <etal/></person-group>. <article-title>Dabrafenib in BRAF-mutated metastatic melanoma: a multicentre, open-label, phase 3 randomised controlled trial</article-title>. <source>Lancet.</source> (<year>2012</year>) <volume>380</volume>:<fpage>358</fpage>&#x02013;<lpage>65</lpage>. <pub-id pub-id-type="doi">10.1016/S0140-6736(12)60868-X</pub-id><pub-id pub-id-type="pmid">22735384</pub-id></citation></ref>
<ref id="B4">
<label>4.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Menzer</surname> <given-names>C</given-names></name> <name><surname>Menzies</surname> <given-names>AM</given-names></name> <name><surname>Carlino</surname> <given-names>MS</given-names></name> <name><surname>Reijers</surname> <given-names>I</given-names></name> <name><surname>Groen</surname> <given-names>EJ</given-names></name> <name><surname>Eigentler</surname> <given-names>T</given-names></name> <etal/></person-group>. <article-title>Targeted therapy in advanced melanoma with rare BRAF mutations</article-title>. <source>J Clin Oncol.</source> (<year>2019</year>) 3142&#x02013;51. <pub-id pub-id-type="doi">10.1200/JCO.19.00489</pub-id><pub-id pub-id-type="pmid">31580757</pub-id></citation></ref>
<ref id="B5">
<label>5.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Grossman</surname> <given-names>D</given-names></name> <name><surname>Okwundu</surname> <given-names>N</given-names></name> <name><surname>Bartlett</surname> <given-names>E</given-names></name> <name><surname>Marchetti</surname> <given-names>M</given-names></name> <name><surname>Othus</surname> <given-names>M</given-names></name> <name><surname>Coit</surname> <given-names>D</given-names></name> <etal/></person-group>. <article-title>Prognostic gene expression profiling in cutaneous melanoma: identifying the knowledge gaps and assessing the clinical benefit</article-title>. <source>JAMA Dermatol.</source> (<year>2020</year>) <volume>156</volume>:<fpage>1004</fpage>&#x02013;<lpage>011</lpage>. <pub-id pub-id-type="doi">10.1001/jamadermatol.2020.1729</pub-id><pub-id pub-id-type="pmid">32725204</pub-id></citation></ref>
<ref id="B6">
<label>6.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kwak</surname> <given-names>M</given-names></name> <name><surname>Farrow</surname> <given-names>NE</given-names></name> <name><surname>Salama</surname> <given-names>AKS</given-names></name> <name><surname>Mosca</surname> <given-names>PJ</given-names></name> <name><surname>Hanks</surname> <given-names>BA</given-names></name> <name><surname>Slingluff</surname> <given-names>CL</given-names></name> <etal/></person-group>. <article-title>Updates in adjuvant systemic therapy for melanoma</article-title>. <source>J Surg Oncol.</source> (<year>2019</year>) <volume>119</volume>:<fpage>222</fpage>&#x02013;<lpage>31</lpage>. <pub-id pub-id-type="doi">10.1002/jso.25298</pub-id><pub-id pub-id-type="pmid">30481375</pub-id></citation></ref>
<ref id="B7">
<label>7.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Karimkhani</surname> <given-names>C</given-names></name> <name><surname>Green</surname> <given-names>AC</given-names></name> <name><surname>Nijsten</surname> <given-names>T</given-names></name> <name><surname>Weinstock</surname> <given-names>MA</given-names></name> <name><surname>Dellavalle</surname> <given-names>RP</given-names></name> <name><surname>Naghavi</surname> <given-names>M</given-names></name> <etal/></person-group>. <article-title>The global burden of melanoma: results from the Global Burden of Disease Study 2015</article-title>. <source>Br J Dermatol.</source> (<year>2017</year>) <volume>177</volume>:<fpage>134</fpage>&#x02013;<lpage>40</lpage>. <pub-id pub-id-type="doi">10.1111/bjd.15510</pub-id><pub-id pub-id-type="pmid">28369739</pub-id></citation></ref>
<ref id="B8">
<label>8.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ferlay</surname> <given-names>J</given-names></name> <name><surname>Colombet</surname> <given-names>M</given-names></name> <name><surname>Soerjomataram</surname> <given-names>I</given-names></name> <name><surname>Mathers</surname> <given-names>C</given-names></name> <name><surname>Parkin</surname> <given-names>DM</given-names></name> <name><surname>Pi&#x000F1;eros</surname> <given-names>M</given-names></name> <etal/></person-group>. <article-title>Estimating the global cancer incidence and mortality in 2018: GLOBOCAN sources and methods</article-title>. <source>Int J Cancer.</source> (<year>2019</year>) <volume>144</volume>:<fpage>1941</fpage>&#x02013;<lpage>53</lpage>. <pub-id pub-id-type="doi">10.1002/ijc.31937</pub-id><pub-id pub-id-type="pmid">30350310</pub-id></citation></ref>
<ref id="B9">
<label>9.</label>
<citation citation-type="web"><person-group person-group-type="author"><name><surname>USCSWG</surname> <given-names>(United States Cancer Statistics Working Group)</given-names></name></person-group>. <article-title>U.S. Cancer Statistics Data Visualizations Tool, Based on 2019 Submission Data</article-title> (1999-2017). U.S. Department of Health and Human Services, Centers for Disease Control and Prevention and National Cancer Institute (<year>2020</year>). Available online at: <ext-link ext-link-type="uri" xlink:href="http://www.cdc.gov/cancer/dataviz">www.cdc.gov/cancer/dataviz</ext-link></citation></ref>
<ref id="B10">
<label>10.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Krensel</surname> <given-names>M</given-names></name> <name><surname>Sch&#x000E4;fer</surname> <given-names>I</given-names></name> <name><surname>Augustin</surname> <given-names>M</given-names></name></person-group>. <article-title>Cost-of-illness of melanoma in Europe &#x02013; a systematic review of the published literature</article-title>. <source>J Eur Acad Dermatology Venereol.</source> (<year>2019</year>) <volume>33</volume>:<fpage>504</fpage>&#x02013;<lpage>10</lpage>. <pub-id pub-id-type="doi">10.1111/jdv.15315</pub-id><pub-id pub-id-type="pmid">30408246</pub-id></citation></ref>
<ref id="B11">
<label>11.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Piepkorn</surname> <given-names>MW</given-names></name> <name><surname>Barnhill</surname> <given-names>RL</given-names></name> <name><surname>Elder</surname> <given-names>DE</given-names></name> <name><surname>Knezevich</surname> <given-names>SR</given-names></name> <name><surname>Carney</surname> <given-names>PA</given-names></name> <name><surname>Reisch</surname> <given-names>LM</given-names></name> <etal/></person-group>. <article-title>The MPATH-Dx reporting schema for melanocytic proliferations and melanoma</article-title>. <source>J Am Acad Dermatol.</source> (<year>2014</year>) <volume>70</volume>:<fpage>131</fpage>&#x02013;<lpage>41</lpage>. <pub-id pub-id-type="doi">10.1016/j.jaad.2013.07.027</pub-id><pub-id pub-id-type="pmid">24176521</pub-id></citation></ref>
<ref id="B12">
<label>12.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Elmore</surname> <given-names>JG</given-names></name> <name><surname>Elder</surname> <given-names>DE</given-names></name> <name><surname>Barnhill</surname> <given-names>RL</given-names></name> <name><surname>Knezevich</surname> <given-names>SR</given-names></name> <name><surname>Longton</surname> <given-names>GM</given-names></name> <name><surname>Titus</surname> <given-names>LJ</given-names></name> <etal/></person-group>. <article-title>Concordance and reproducibility of melanoma staging according to the 7th vs 8th edition of the AJCC cancer staging manual</article-title>. <source>JAMA Netw Open.</source> (<year>2018</year>) <volume>1</volume>:<fpage>e180083</fpage>. <pub-id pub-id-type="doi">10.1001/jamanetworkopen.2018.0083</pub-id><pub-id pub-id-type="pmid">30556054</pub-id></citation></ref>
<ref id="B13">
<label>13.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dinehart</surname> <given-names>MS</given-names></name> <name><surname>Dinehart</surname> <given-names>SM</given-names></name> <name><surname>Sukpraprut-Braaten</surname> <given-names>S</given-names></name> <name><surname>High</surname> <given-names>WA</given-names></name></person-group>. <article-title>Immunohistochemistry utilization in the diagnosis of melanoma</article-title>. <source>J Cutan Pathol.</source> (<year>2020</year>) <volume>47</volume>:<fpage>446</fpage>&#x02013;<lpage>50</lpage>. <pub-id pub-id-type="doi">10.1111/cup.13648</pub-id><pub-id pub-id-type="pmid">31955450</pub-id></citation></ref>
<ref id="B14">
<label>14.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gershenwald</surname> <given-names>JE</given-names></name> <name><surname>Scolyer</surname> <given-names>RA</given-names></name></person-group>. <article-title>Melanoma staging: American Joint Committee on Cancer (AJCC) 8th edition and beyond</article-title>. <source>Ann Surg Oncol.</source> (<year>2018</year>) <volume>25</volume>:<fpage>2105</fpage>&#x02013;<lpage>10</lpage>. <pub-id pub-id-type="doi">10.1245/s10434-018-6513-7</pub-id><pub-id pub-id-type="pmid">30105437</pub-id></citation></ref>
<ref id="B15">
<label>15.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Elmore</surname> <given-names>JG</given-names></name> <name><surname>Barnhill</surname> <given-names>RL</given-names></name> <name><surname>Elder</surname> <given-names>DE</given-names></name> <name><surname>Longton</surname> <given-names>GM</given-names></name> <name><surname>Pepe</surname> <given-names>MS</given-names></name> <name><surname>Reisch</surname> <given-names>LM</given-names></name> <etal/></person-group>. <article-title>Pathologists&#x00027; diagnosis of invasive melanoma and melanocytic proliferations: observer accuracy and reproducibility study</article-title>. <source>BMJ.</source> (<year>2017</year>) <volume>357</volume>:<fpage>2813</fpage>. <pub-id pub-id-type="doi">10.1136/bmj.j2813</pub-id><pub-id pub-id-type="pmid">28790163</pub-id></citation></ref>
<ref id="B16">
<label>16.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Carney</surname> <given-names>PA</given-names></name> <name><surname>Reisch</surname> <given-names>LM</given-names></name> <name><surname>Piepkorn</surname> <given-names>MW</given-names></name> <name><surname>Barnhill</surname> <given-names>RL</given-names></name> <name><surname>Elder</surname> <given-names>DE</given-names></name> <name><surname>Knezevich</surname> <given-names>S</given-names></name> <etal/></person-group>. <article-title>Achieving consensus for the histopathologic diagnosis of melanocytic lesions: use of the modified Delphi method</article-title>. <source>J Cutan Pathol.</source> (<year>2016</year>) <volume>43</volume>:<fpage>830</fpage>&#x02013;<lpage>7</lpage>. <pub-id pub-id-type="doi">10.1111/cup.12751</pub-id></citation>
</ref>
<ref id="B17">
<label>17.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lott</surname> <given-names>JP</given-names></name> <name><surname>Elmore</surname> <given-names>JG</given-names></name> <name><surname>Zhao</surname> <given-names>GA</given-names></name> <name><surname>Knezevich</surname> <given-names>SR</given-names></name> <name><surname>Frederick</surname> <given-names>PD</given-names></name> <name><surname>Reisch</surname> <given-names>LM</given-names></name> <etal/></person-group>. <article-title>Evaluation of the Melanocytic Pathology Assessment Tool and Hierarchy for Diagnosis (MPATH-Dx) classification scheme for diagnosis of cutaneous melanocytic neoplasms: results from the International Melanoma Pathology Study Group</article-title>. <source>J Am Acad Dermatol.</source> (<year>2016</year>) <volume>75</volume>:<fpage>356</fpage>&#x02013;<lpage>63</lpage>. <pub-id pub-id-type="doi">10.1016/j.jaad.2016.04.052</pub-id></citation>
</ref>
<ref id="B18">
<label>18.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Elder</surname> <given-names>DE</given-names></name> <name><surname>Piepkorn</surname> <given-names>MW</given-names></name> <name><surname>Barnhill</surname> <given-names>RL</given-names></name> <name><surname>Longton</surname> <given-names>GM</given-names></name> <name><surname>Nelson</surname> <given-names>HD</given-names></name> <name><surname>Knezevich</surname> <given-names>SR</given-names></name> <etal/></person-group>. <article-title>Pathologist characteristics associated with accuracy and reproducibility of melanocytic skin lesion interpretation</article-title>. <source>J Am Acad Dermatol.</source> (<year>2018</year>) <volume>79</volume>:<fpage>52</fpage>&#x02013;<lpage>9</lpage>.e5. <pub-id pub-id-type="doi">10.1016/j.jaad.2018.02.070</pub-id></citation>
</ref>
<ref id="B19">
<label>19.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Piepkorn</surname> <given-names>MW</given-names></name> <name><surname>Longton</surname> <given-names>GM</given-names></name> <name><surname>Reisch</surname> <given-names>LM</given-names></name> <name><surname>Elder</surname> <given-names>DE</given-names></name> <name><surname>Pepe</surname> <given-names>MS</given-names></name> <name><surname>Kerr</surname> <given-names>KF</given-names></name> <etal/></person-group>. <article-title>Assessment of second-opinion strategies for diagnoses of cutaneous melanocytic lesions</article-title>. <source>JAMA Netw Open.</source> (<year>2019</year>) <volume>2</volume>:<fpage>e1912597</fpage>. <pub-id pub-id-type="doi">10.1001/jamanetworkopen.2019.12597</pub-id><pub-id pub-id-type="pmid">31603483</pub-id></citation></ref>
<ref id="B20">
<label>20.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Geller</surname> <given-names>BM</given-names></name> <name><surname>Frederick</surname> <given-names>PD</given-names></name> <name><surname>Knezevich</surname> <given-names>SR</given-names></name> <name><surname>Lott</surname> <given-names>JP</given-names></name> <name><surname>Nelson</surname> <given-names>HD</given-names></name> <name><surname>Titus</surname> <given-names>LJ</given-names></name> <etal/></person-group>. <article-title>Pathologists&#x00027; use of second opinions in interpretation of melanocytic cutaneous lesions: policies, practices, and perceptions</article-title>. <source>Dermatol Surg.</source> (<year>2018</year>) <volume>44</volume>:<fpage>177</fpage>&#x02013;<lpage>85</lpage>. <pub-id pub-id-type="doi">10.1097/DSS.0000000000001256</pub-id><pub-id pub-id-type="pmid">28858936</pub-id></citation></ref>
<ref id="B21">
<label>21.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Titus</surname> <given-names>LJ</given-names></name> <name><surname>Reisch</surname> <given-names>LM</given-names></name> <name><surname>Tosteson</surname> <given-names>ANA</given-names></name> <name><surname>Nelson</surname> <given-names>HD</given-names></name> <name><surname>Frederick</surname> <given-names>PD</given-names></name> <name><surname>Carney</surname> <given-names>PA</given-names></name> <etal/></person-group>. <article-title>Malpractice concerns, defensive medicine, and the histopathology diagnosis of melanocytic skin lesions</article-title>. <source>Am J Clin Pathol Oct.</source> (<year>2018</year>) <volume>150</volume>:<fpage>338</fpage>&#x02013;<lpage>45</lpage>. <pub-id pub-id-type="doi">10.1093/ajcp/aqy057</pub-id><pub-id pub-id-type="pmid">30007278</pub-id></citation></ref>
<ref id="B22">
<label>22.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gardner</surname> <given-names>LJ</given-names></name> <name><surname>Strunck</surname> <given-names>JL</given-names></name> <name><surname>Wu</surname> <given-names>YP</given-names></name> <name><surname>Grossman</surname> <given-names>D</given-names></name></person-group>. <article-title>Current controversies in early-stage melanoma: questions on incidence, screening, and histologic regression</article-title>. <source>J Am Acad Dermatol.</source> (<year>2019</year>) <volume>80</volume>:<fpage>1</fpage>&#x02013;<lpage>12</lpage>. <pub-id pub-id-type="doi">10.1016/j.jaad.2018.03.053</pub-id><pub-id pub-id-type="pmid">30553298</pub-id></citation></ref>
<ref id="B23">
<label>23.</label>
<citation citation-type="web"><person-group person-group-type="author"><name><surname>Schmidt</surname> <given-names>R</given-names></name></person-group>. <article-title>Testing a Test: Beyond Sensitivity and Specificity</article-title>. (<year>2015</year>). Available online at: <ext-link ext-link-type="uri" xlink:href="https://arup.utah.edu/education/testing_test.php/">https://arup.utah.edu/education/testing_test.php/</ext-link>.</citation></ref>
<ref id="B24">
<label>24.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bewick</surname> <given-names>V</given-names></name> <name><surname>Cheek</surname> <given-names>L</given-names></name> <name><surname>Ball</surname> <given-names>J</given-names></name></person-group>. <article-title>Statistics review 13: receiver operating characteristics curves</article-title>. <source>Crit Care.</source> (<year>2004</year>) <volume>8</volume>:<fpage>508</fpage>&#x02013;<lpage>12</lpage>. <pub-id pub-id-type="doi">10.1186/cc3000</pub-id></citation></ref>
<ref id="B25">
<label>25.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Phillips</surname> <given-names>M</given-names></name> <name><surname>Greenhalgh</surname> <given-names>J</given-names></name> <name><surname>Marsden</surname> <given-names>H</given-names></name> <name><surname>Palamaras</surname> <given-names>I</given-names></name></person-group>. <article-title>Detection of malignant melanoma using artificial intelligence: an observational study of diagnostic accuracy</article-title>. <source>Dermatol Pract Concept.</source> (<year>2019</year>) <volume>10</volume>:<fpage>e2020011</fpage>. <pub-id pub-id-type="doi">10.5826/dpc.1001a11</pub-id><pub-id pub-id-type="pmid">31921498</pub-id></citation></ref>
<ref id="B26">
<label>26.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Glazer</surname> <given-names>AM</given-names></name> <name><surname>Rigel</surname> <given-names>DS</given-names></name></person-group>. <article-title>Analysis of trends in geographic distribution of us dermatology workforce density</article-title>. <source>JAMA Dermatol.</source> (<year>2017</year>) <volume>153</volume>:<fpage>472</fpage>&#x02013;<lpage>3</lpage>. <pub-id pub-id-type="doi">10.1001/jamadermatol.2016.6032</pub-id><pub-id pub-id-type="pmid">28296988</pub-id></citation></ref>
<ref id="B27">
<label>27.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Guitera</surname> <given-names>P</given-names></name> <name><surname>Menzies</surname> <given-names>SW</given-names></name> <name><surname>Longo</surname> <given-names>C</given-names></name> <name><surname>Cesinaro</surname> <given-names>AM</given-names></name> <name><surname>Scolyer</surname> <given-names>RA</given-names></name> <name><surname>Pellacani</surname> <given-names>G</given-names></name></person-group>. <article-title><italic>In vivo</italic> confocal microscopy for diagnosis of melanoma and basal cell carcinoma using a two-step method: analysis of 710 consecutive clinically equivocal cases</article-title>. <source>J Invest Dermatol.</source> (<year>2012</year>) <volume>132</volume>:<fpage>2386</fpage>&#x02013;<lpage>94</lpage>. <pub-id pub-id-type="doi">10.1038/jid.2012.172</pub-id><pub-id pub-id-type="pmid">22718115</pub-id></citation></ref>
<ref id="B28">
<label>28.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Pampena</surname> <given-names>R</given-names></name> <name><surname>Borsari</surname> <given-names>S</given-names></name> <name><surname>Lai</surname> <given-names>M</given-names></name> <name><surname>Benati</surname> <given-names>E</given-names></name> <name><surname>Longhitano</surname> <given-names>S</given-names></name> <name><surname>Mirra</surname> <given-names>M</given-names></name> <etal/></person-group>. <article-title>External validation and comparison of four confocal microscopic scores for melanoma diagnosis on a retrospective series of highly suspicious melanocytic lesions</article-title>. <source>J Eur Acad Dermatol Venereol.</source> (<year>2019</year>) <volume>33</volume>:<fpage>1541</fpage>&#x02013;<lpage>6</lpage>. <pub-id pub-id-type="doi">10.1111/jdv.15617</pub-id><pub-id pub-id-type="pmid">30974506</pub-id></citation></ref>
<ref id="B29">
<label>29.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kittler</surname> <given-names>H</given-names></name> <name><surname>Pehamberger</surname> <given-names>H</given-names></name> <name><surname>Wolff</surname> <given-names>K</given-names></name> <name><surname>Binder</surname> <given-names>M</given-names></name></person-group>. <article-title>Diagnostic accuracy of dermoscopy</article-title>. <source>Lancet Oncol.</source> (<year>2002</year>) <volume>3</volume>:<fpage>159</fpage>&#x02013;<lpage>65</lpage>. <pub-id pub-id-type="doi">10.1016/S1470-2045(02)00679-4</pub-id></citation></ref>
<ref id="B30">
<label>30.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Binder</surname> <given-names>M</given-names></name></person-group>. <article-title>Epiluminescence microscopy. A useful tool for the diagnosis of pigmented skin lesions for formally trained dermatologists</article-title>. <source>Arch Dermatol.</source> (<year>1995</year>) <volume>131</volume>:<fpage>286</fpage>&#x02013;<lpage>91</lpage>. <pub-id pub-id-type="doi">10.1001/archderm.1995.01690150050011</pub-id><pub-id pub-id-type="pmid">7887657</pub-id></citation></ref>
<ref id="B31">
<label>31.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Nelson</surname> <given-names>KC</given-names></name> <name><surname>Swetter</surname> <given-names>SM</given-names></name> <name><surname>Saboda</surname> <given-names>K</given-names></name> <name><surname>Chen</surname> <given-names>SC</given-names></name> <name><surname>Curiel-Lewandrowski</surname> <given-names>C</given-names></name></person-group>. <article-title>Evaluation of the number-needed-to-biopsy metric for the diagnosis of cutaneous melanoma: a systematic review and meta-analysis</article-title>. <source>JAMA Dermatol.</source> (<year>2019</year>) <volume>155</volume>:<fpage>1167</fpage>&#x02013;<lpage>74</lpage>. <pub-id pub-id-type="doi">10.1001/jamadermatol.2019.1514</pub-id></citation>
</ref>
<ref id="B32">
<label>32.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Shah</surname> <given-names>A</given-names></name> <name><surname>Hyngstrom</surname> <given-names>J</given-names></name> <name><surname>Florell</surname> <given-names>SR</given-names></name> <name><surname>Grossman</surname> <given-names>D</given-names></name></person-group>. <article-title>Use of the pigmented lesion assay to rapidly screen a patient with numerous clinically atypical pigmented lesions</article-title>. <source>JAAD Case Rep.</source> (<year>2019</year>) <volume>5</volume>:<fpage>1048</fpage>&#x02013;<lpage>50</lpage>. <pub-id pub-id-type="doi">10.1016/j.jdcr.2019.10.004</pub-id></citation>
</ref>
<ref id="B33">
<label>33.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yao</surname> <given-names>Z</given-names></name> <name><surname>Allen</surname> <given-names>T</given-names></name> <name><surname>Oakley</surname> <given-names>M</given-names></name> <name><surname>Samons</surname> <given-names>C</given-names></name> <name><surname>Garrison</surname> <given-names>D</given-names></name> <name><surname>Jansen</surname> <given-names>B</given-names></name></person-group>. <article-title>Analytical characteristics of a noninvasive gene expression assay for pigmented skin lesions</article-title>. <source>Assay Drug Dev Technol.</source> (<year>2016</year>) <volume>14</volume>:<fpage>355</fpage>&#x02013;<lpage>63</lpage>. <pub-id pub-id-type="doi">10.1089/adt.2016.724</pub-id><pub-id pub-id-type="pmid">27505074</pub-id></citation></ref>
<ref id="B34">
<label>34.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gerami</surname> <given-names>P</given-names></name> <name><surname>Yao</surname> <given-names>Z</given-names></name> <name><surname>Polsky</surname> <given-names>D</given-names></name> <name><surname>Jansen</surname> <given-names>B</given-names></name> <name><surname>Busam</surname> <given-names>K</given-names></name> <name><surname>Ho</surname> <given-names>J</given-names></name> <etal/></person-group>. <article-title>Development and validation of a noninvasive 2-gene molecular assay for cutaneous melanoma</article-title>. <source>J Am Acad Dermatol.</source> (<year>2017</year>) <volume>76</volume>:<fpage>114</fpage>&#x02013;<lpage>120</lpage>.e2. <pub-id pub-id-type="doi">10.1016/j.jaad.2016.07.038</pub-id><pub-id pub-id-type="pmid">27707590</pub-id></citation></ref>
<ref id="B35">
<label>35.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Varedi</surname> <given-names>A</given-names></name> <name><surname>Gardner</surname> <given-names>LJ</given-names></name> <name><surname>Kim</surname> <given-names>CC</given-names></name> <name><surname>Chu</surname> <given-names>EY</given-names></name> <name><surname>Ming</surname> <given-names>ME</given-names></name> <name><surname>Leachman</surname> <given-names>SA</given-names></name> <etal/></person-group>. <article-title>Use of new molecular tests for melanoma by pigmented-lesion experts</article-title>. <source>J Am Acad Dermatol.</source> (<year>2020</year>) <volume>82</volume>:<fpage>245</fpage>&#x02013;<lpage>7</lpage>. <pub-id pub-id-type="doi">10.1016/j.jaad.2019.08.022</pub-id><pub-id pub-id-type="pmid">31415835</pub-id></citation></ref>
<ref id="B36">
<label>36.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lee</surname> <given-names>KC</given-names></name> <name><surname>Peacock</surname> <given-names>S</given-names></name> <name><surname>Weinstock</surname> <given-names>MA</given-names></name> <name><surname>Zhao</surname> <given-names>GA</given-names></name> <name><surname>Knezevich</surname> <given-names>SR</given-names></name> <name><surname>Elder</surname> <given-names>DE</given-names></name> <etal/></person-group>. <article-title>Variation among pathologists&#x00027; treatment suggestions for melanocytic lesions: a survey of pathologists</article-title>. <source>J Am Acad Dermatol.</source> (<year>2017</year>) <volume>76</volume>:<fpage>121</fpage>&#x02013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.1016/j.jaad.2016.07.029</pub-id><pub-id pub-id-type="pmid">27692732</pub-id></citation></ref>
<ref id="B37">
<label>37.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hilliard</surname> <given-names>NJ</given-names></name> <name><surname>Krahl</surname> <given-names>D</given-names></name> <name><surname>Sellheyer</surname> <given-names>K</given-names></name></person-group>. <article-title>P16 Expression differentiates between desmoplastic Spitz nevus and desmoplastic melanoma</article-title>. <source>J Cutan Pathol.</source> (<year>2009</year>) <volume>36</volume>:<fpage>753</fpage>&#x02013;<lpage>9</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-0560.2008.01154.x</pub-id><pub-id pub-id-type="pmid">19519606</pub-id></citation></ref>
<ref id="B38">
<label>38.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mason</surname> <given-names>A</given-names></name> <name><surname>Wititsuwannakul</surname> <given-names>J</given-names></name> <name><surname>Klump</surname> <given-names>VR</given-names></name> <name><surname>Lott</surname> <given-names>J</given-names></name> <name><surname>Lazova</surname> <given-names>R</given-names></name></person-group>. <article-title>Expression of p16 alone does not differentiate between Spitz nevi and Spitzoid melanoma</article-title>. <source>J Cutan Pathol.</source> (<year>2012</year>) <volume>39</volume>:<fpage>1062</fpage>&#x02013;<lpage>74</lpage>. <pub-id pub-id-type="doi">10.1111/cup.12014</pub-id><pub-id pub-id-type="pmid">23005921</pub-id></citation></ref>
<ref id="B39">
<label>39.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lezcano</surname> <given-names>C</given-names></name> <name><surname>Jungbluth</surname> <given-names>AA</given-names></name> <name><surname>Nehal</surname> <given-names>KS</given-names></name> <name><surname>Hollmann</surname> <given-names>TJ</given-names></name> <name><surname>Busam</surname> <given-names>KJ</given-names></name></person-group>. <article-title>PRAME expression in melanocytic tumors</article-title>. <source>Am J Surg Pathol.</source> (<year>2018</year>) <volume>42</volume>:<fpage>1456</fpage>&#x02013;<lpage>65</lpage>. <pub-id pub-id-type="doi">10.1097/PAS.0000000000001134</pub-id></citation>
</ref>
<ref id="B40">
<label>40.</label>
<citation citation-type="web"><person-group person-group-type="author"><name><surname>Lezcano</surname> <given-names>C</given-names></name> <name><surname>Jungbluth</surname> <given-names>AA</given-names></name> <name><surname>Busam</surname> <given-names>KJ</given-names></name></person-group>. <article-title>Comparison of Immunohistochemistry for PRAME With Cytogenetic Test Results in the Evaluation of Challenging Melanocytic Tumors</article-title>. (<year>2020</year>). Available online at: <ext-link ext-link-type="uri" xlink:href="http://www.ajsp.com">www.ajsp.com</ext-link></citation>
</ref>
<ref id="B41">
<label>41.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bastian</surname> <given-names>BC</given-names></name> <name><surname>Olshen</surname> <given-names>AB</given-names></name> <name><surname>LeBoit</surname> <given-names>PE</given-names></name> <name><surname>Pinkel</surname> <given-names>D</given-names></name></person-group>. <article-title>Classifying melanocytic tumors based on DNA copy number changes</article-title>. <source>Am J Pathol</source>. (<year>2003</year>) <volume>163</volume>:<fpage>1765</fpage>&#x02013;<lpage>70</lpage>. <pub-id pub-id-type="doi">10.1016/S0002-9440(10)63536-5</pub-id></citation>
</ref>
<ref id="B42">
<label>42.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gaiser</surname> <given-names>T</given-names></name> <name><surname>Kutzner</surname> <given-names>H</given-names></name> <name><surname>Palmedo</surname> <given-names>G</given-names></name> <name><surname>Siegelin</surname> <given-names>MD</given-names></name> <name><surname>Wiesner</surname> <given-names>T</given-names></name> <name><surname>Bruckner</surname> <given-names>T</given-names></name> <etal/></person-group>. <article-title>Classifying ambiguous melanocytic lesions with FISH and correlation with clinical long-term follow up</article-title>. <source>Mod Pathol.</source> (<year>2010</year>) <volume>23</volume>:<fpage>413</fpage>&#x02013;<lpage>9</lpage>. <pub-id pub-id-type="doi">10.1038/modpathol.2009.177</pub-id><pub-id pub-id-type="pmid">20081813</pub-id></citation></ref>
<ref id="B43">
<label>43.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gerami</surname> <given-names>P</given-names></name> <name><surname>Li</surname> <given-names>G</given-names></name> <name><surname>Pouryazdanparast</surname> <given-names>P</given-names></name> <name><surname>Blondin</surname> <given-names>B</given-names></name> <name><surname>Beilfuss</surname> <given-names>B</given-names></name> <name><surname>Slenk</surname> <given-names>C</given-names></name> <etal/></person-group>. <article-title>A highly specific and discriminatory FISH assay for distinguishing between benign and malignant melanocytic neoplasms</article-title>. <source>Am J Surg Pathol.</source> (<year>2012</year>) <volume>36</volume>:<fpage>808</fpage>&#x02013;<lpage>17</lpage>. <pub-id pub-id-type="doi">10.1097/PAS.0b013e31824b1efd</pub-id><pub-id pub-id-type="pmid">22588064</pub-id></citation></ref>
<ref id="B44">
<label>44.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Clarke</surname> <given-names>LE</given-names></name> <name><surname>Warf</surname> <given-names>BM</given-names></name> <name><surname>Flake</surname> <given-names>DD</given-names></name> <name><surname>Hartman</surname> <given-names>AR</given-names></name> <name><surname>Tahan</surname> <given-names>S</given-names></name> <name><surname>Shea</surname> <given-names>CR</given-names></name> <etal/></person-group>. <article-title>Clinical validation of a gene expression signature that differentiates benign nevi from malignant melanoma</article-title>. <source>J Cutan Pathol.</source> (<year>2015</year>) <volume>42</volume>:<fpage>244</fpage>&#x02013;<lpage>52</lpage>. <pub-id pub-id-type="doi">10.1111/cup.12475</pub-id><pub-id pub-id-type="pmid">25727210</pub-id></citation></ref>
<ref id="B45">
<label>45.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Clarke</surname> <given-names>LE</given-names></name> <name><surname>Flake</surname> <given-names>DD</given-names></name> <name><surname>Busam</surname> <given-names>K</given-names></name> <name><surname>Cockerell</surname> <given-names>C</given-names></name> <name><surname>Helm</surname> <given-names>K</given-names></name> <name><surname>McNiff</surname> <given-names>J</given-names></name> <etal/></person-group>. <article-title>An independent validation of a gene expression signature to differentiate malignant melanoma from benign melanocytic nevi</article-title>. <source>Cancer.</source> (<year>2017</year>) <volume>123</volume>:<fpage>617</fpage>&#x02013;<lpage>28</lpage>. <pub-id pub-id-type="doi">10.1002/cncr.30385</pub-id><pub-id pub-id-type="pmid">27768230</pub-id></citation></ref>
<ref id="B46">
<label>46.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ko</surname> <given-names>JS</given-names></name> <name><surname>Matharoo-Ball</surname> <given-names>B</given-names></name> <name><surname>Billings</surname> <given-names>SD</given-names></name> <name><surname>Thomson</surname> <given-names>BJ</given-names></name> <name><surname>Tang</surname> <given-names>JY</given-names></name> <name><surname>Sarin</surname> <given-names>KY</given-names></name> <etal/></person-group>. <article-title>Diagnostic distinction of malignant melanoma and benign nevi by a gene expression signature and correlation to clinical outcomes</article-title>. <source>Cancer Epidemiol Biomarkers Prev.</source> (<year>2017</year>) <volume>26</volume>:<fpage>1107</fpage>&#x02013;<lpage>13</lpage>. <pub-id pub-id-type="doi">10.1158/1055-9965.EPI-16-0958</pub-id><pub-id pub-id-type="pmid">28377414</pub-id></citation></ref>
<ref id="B47">
<label>47.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Clarke</surname> <given-names>LE</given-names></name> <name><surname>Pimentel</surname> <given-names>JD</given-names></name> <name><surname>Zalaznick</surname> <given-names>H</given-names></name> <name><surname>Wang</surname> <given-names>L</given-names></name> <name><surname>Busam</surname> <given-names>KJ</given-names></name></person-group>. <article-title>Gene expression signature as an ancillary method in the diagnosis of desmoplastic melanoma</article-title>. <source>Hum Pathol.</source> (<year>2017</year>) <volume>70</volume>:<fpage>113</fpage>&#x02013;<lpage>20</lpage>. <pub-id pub-id-type="doi">10.1016/j.humpath.2017.10.005</pub-id><pub-id pub-id-type="pmid">29079183</pub-id></citation></ref>
<ref id="B48">
<label>48.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Clarke</surname> <given-names>LE</given-names></name> <name><surname>Mabey</surname> <given-names>B</given-names></name> <name><surname>Flake</surname> <given-names>DD</given-names></name> <name><surname>Meek</surname> <given-names>S</given-names></name> <name><surname>Cassarino</surname> <given-names>DS</given-names></name> <name><surname>Duncan</surname> <given-names>LM</given-names></name> <etal/></person-group>. <article-title>Clinical validity of a gene expression signature in diagnostically uncertain neoplasms</article-title>. <source>Per Med.</source> (<year>2020</year>) <volume>17</volume>:<fpage>361</fpage>&#x02013;<lpage>71</lpage>. <pub-id pub-id-type="doi">10.2217/pme-2020-0048</pub-id><pub-id pub-id-type="pmid">32915688</pub-id></citation></ref>
<ref id="B49">
<label>49.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wang</surname> <given-names>L</given-names></name> <name><surname>Rao</surname> <given-names>M</given-names></name> <name><surname>Fang</surname> <given-names>Y</given-names></name> <name><surname>Hameed</surname> <given-names>M</given-names></name> <name><surname>Viale</surname> <given-names>A</given-names></name> <name><surname>Busam</surname> <given-names>K</given-names></name> <etal/></person-group>. <article-title>A genome-wide high-resolution array-cgh analysis of cutaneous melanoma and comparison of array-cgh to fish in diagnostic evaluation</article-title>. <source>J Mol Diagn.</source> (<year>2013</year>) <volume>15</volume>:<fpage>581</fpage>&#x02013;<lpage>91</lpage>. <pub-id pub-id-type="doi">10.1016/j.jmoldx.2013.04.001</pub-id><pub-id pub-id-type="pmid">23800576</pub-id></citation></ref>
<ref id="B50">
<label>50.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Minca</surname> <given-names>EC</given-names></name> <name><surname>Al-Rohil</surname> <given-names>RN</given-names></name> <name><surname>Wang</surname> <given-names>M</given-names></name> <name><surname>Harms</surname> <given-names>PW</given-names></name> <name><surname>Ko</surname> <given-names>JS</given-names></name> <name><surname>Collie</surname> <given-names>AM</given-names></name> <etal/></person-group>. <article-title>Comparison between melanoma gene expression score and fluorescence <italic>in situ</italic> hybridization for the classification of melanocytic lesions</article-title>. <source>Mod Pathol.</source> (<year>2016</year>) <volume>29</volume>:<fpage>832</fpage>&#x02013;<lpage>43</lpage>. <pub-id pub-id-type="doi">10.1038/modpathol.2016.84</pub-id><pub-id pub-id-type="pmid">27174586</pub-id></citation></ref>
<ref id="B51">
<label>51.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Reimann</surname> <given-names>JDR</given-names></name> <name><surname>Salim</surname> <given-names>S</given-names></name> <name><surname>Velazquez</surname> <given-names>EF</given-names></name> <name><surname>Wang</surname> <given-names>L</given-names></name> <name><surname>Williams</surname> <given-names>KM</given-names></name> <name><surname>Flejter</surname> <given-names>WL</given-names></name> <etal/></person-group>. <article-title>Comparison of melanoma gene expression score with histopathology, fluorescence <italic>in situ</italic> hybridization, and SNP array for the classification of melanocytic neoplasms</article-title>. <source>Mod Pathol.</source> (<year>2018</year>) <volume>31</volume>:<fpage>1733</fpage>&#x02013;<lpage>43</lpage>. <pub-id pub-id-type="doi">10.1038/s41379-018-0087-6</pub-id><pub-id pub-id-type="pmid">29955141</pub-id></citation></ref>
<ref id="B52">
<label>52.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Conway</surname> <given-names>K</given-names></name> <name><surname>Edmiston</surname> <given-names>SN</given-names></name> <name><surname>Parker</surname> <given-names>JS</given-names></name> <name><surname>Kuan</surname> <given-names>PF</given-names></name> <name><surname>Tsai</surname> <given-names>YH</given-names></name> <name><surname>Groben</surname> <given-names>PA</given-names></name> <etal/></person-group>. <article-title>Identification of a robust methylation classifier for cutaneous melanoma diagnosis</article-title>. <source>J Invest Dermatol.</source> (<year>2019</year>) <volume>139</volume>:<fpage>1349</fpage>&#x02013;<lpage>61</lpage>. <pub-id pub-id-type="doi">10.1016/j.jid.2018.11.024</pub-id><pub-id pub-id-type="pmid">30529013</pub-id></citation></ref>
<ref id="B53">
<label>53.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Conway</surname> <given-names>K</given-names></name> <name><surname>Edmiston</surname> <given-names>SN</given-names></name> <name><surname>Khondker</surname> <given-names>ZS</given-names></name> <name><surname>Groben</surname> <given-names>PA</given-names></name> <name><surname>Zhou</surname> <given-names>X</given-names></name> <name><surname>Chu</surname> <given-names>H</given-names></name> <etal/></person-group>. <article-title>DNA-methylation profiling distinguishes malignant melanomas from benign nevi</article-title>. <source>Pigment Cell Melanoma Res.</source> (<year>2011</year>) <volume>24</volume>:<fpage>352</fpage>&#x02013;<lpage>60</lpage>. <pub-id pub-id-type="doi">10.1111/j.1755-148X.2011.00828.x</pub-id><pub-id pub-id-type="pmid">21375697</pub-id></citation></ref>
<ref id="B54">
<label>54.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gao</surname> <given-names>L</given-names></name> <name><surname>Van Den Hurk</surname> <given-names>K</given-names></name> <name><surname>Moerkerk</surname> <given-names>PTM</given-names></name> <name><surname>Goeman</surname> <given-names>JJ</given-names></name> <name><surname>Beck</surname> <given-names>S</given-names></name> <name><surname>Grui</surname> <given-names>NA</given-names></name> <etal/></person-group>. <article-title>Promoter CpG Island hypermethylation in dysplastic nevus and melanoma: CLDN11 as an epigenetic biomarker for malignancy</article-title>. <source>J Invest Dermatol</source>. (<year>2014</year>) <volume>134</volume>:<fpage>2957</fpage>&#x02013;<lpage>66</lpage>. <pub-id pub-id-type="doi">10.1038/jid.2014.270</pub-id><pub-id pub-id-type="pmid">24999589</pub-id></citation></ref>
<ref id="B55">
<label>55.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Salvianti</surname> <given-names>F</given-names></name> <name><surname>Orlando</surname> <given-names>C</given-names></name> <name><surname>Massi</surname> <given-names>D</given-names></name> <name><surname>De Giorgi</surname> <given-names>V</given-names></name> <name><surname>Grazzini</surname> <given-names>M</given-names></name> <name><surname>Pazzagli</surname> <given-names>M</given-names></name> <etal/></person-group>. <article-title>Tumor-related methylated cell-free DNA and circulating tumor cells in melanoma</article-title>. <source>Front Mol Biosci.</source> (<year>2016</year>) <volume>2</volume>:<fpage>76</fpage>. <pub-id pub-id-type="doi">10.3389/fmolb.2015.00076</pub-id><pub-id pub-id-type="pmid">26779490</pub-id></citation></ref>
<ref id="B56">
<label>56.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Torres</surname> <given-names>R</given-names></name> <name><surname>Lang</surname> <given-names>UE</given-names></name> <name><surname>Hejna</surname> <given-names>M</given-names></name> <name><surname>Shelton</surname> <given-names>SJ</given-names></name> <name><surname>Joseph</surname> <given-names>NM</given-names></name> <name><surname>Shain</surname> <given-names>AH</given-names></name> <etal/></person-group>. <article-title>MicroRNA ratios distinguish melanomas from nevi</article-title>. <source>J Invest Dermatol.</source> (<year>2019</year>) <volume>140</volume>:<fpage>164</fpage>&#x02013;<lpage>73</lpage>.e7. <pub-id pub-id-type="doi">10.1016/j.jid.2019.06.126</pub-id><pub-id pub-id-type="pmid">31580842</pub-id></citation></ref>
<ref id="B57">
<label>57.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Alexandrescu</surname> <given-names>DT</given-names></name> <name><surname>Kauffman</surname> <given-names>CL</given-names></name> <name><surname>Jatkoe</surname> <given-names>TA</given-names></name> <name><surname>Hartmann</surname> <given-names>DP</given-names></name> <name><surname>Vener</surname> <given-names>T</given-names></name> <name><surname>Wang</surname> <given-names>H</given-names></name> <etal/></person-group>. <article-title>Melanoma-specific marker expression in skin biopsy tissues as a tool to facilitate melanoma diagnosis</article-title>. <source>J Invest Dermatol.</source> (<year>2010</year>) <volume>130</volume>:<fpage>1887</fpage>&#x02013;<lpage>92</lpage>. <pub-id pub-id-type="doi">10.1038/jid.2010.61</pub-id></citation>
</ref>
<ref id="B58">
<label>58.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Koh</surname> <given-names>SS</given-names></name> <name><surname>Opel</surname> <given-names>ML</given-names></name> <name><surname>Wei</surname> <given-names>JPJ</given-names></name> <name><surname>Yau</surname> <given-names>K</given-names></name> <name><surname>Shah</surname> <given-names>R</given-names></name> <name><surname>Gorre</surname> <given-names>ME</given-names></name> <etal/></person-group>. <article-title>Molecular classification of melanomas and nevi using gene expression microarray signatures and formalin-fixed and paraffin-embedded tissue</article-title>. <source>Mod Pathol.</source> (<year>2009</year>) <volume>22</volume>:<fpage>538</fpage>&#x02013;<lpage>46</lpage>. <pub-id pub-id-type="doi">10.1038/modpathol.2009.8</pub-id></citation>
</ref>
<ref id="B59">
<label>59.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Stark</surname> <given-names>MS</given-names></name> <name><surname>Gray</surname> <given-names>ES</given-names></name> <name><surname>Isaacs</surname> <given-names>T</given-names></name> <name><surname>Chen</surname> <given-names>FK</given-names></name> <name><surname>Millward</surname> <given-names>M</given-names></name> <name><surname>McEvoy</surname> <given-names>A</given-names></name> <etal/></person-group>. <article-title>A panel of circulating microRNAs detects uveal melanoma with high precision</article-title>. <source>Transl Vis Sci Technol</source>. (<year>2019</year>) <volume>8</volume>:<fpage>12</fpage>. <pub-id pub-id-type="doi">10.1167/tvst.8.6.12</pub-id></citation>
</ref>
<ref id="B60">
<label>60.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lang</surname> <given-names>UE</given-names></name> <name><surname>Torres</surname> <given-names>R</given-names></name> <name><surname>Cheung</surname> <given-names>C</given-names></name> <name><surname>Vladar</surname> <given-names>EK</given-names></name> <name><surname>Mccalmont</surname> <given-names>TH</given-names></name> <name><surname>Kim</surname> <given-names>J</given-names></name> <etal/></person-group>. <article-title>Ciliation index is a useful diagnostic tool in challenging spitzoid melanocytic neoplasms</article-title>. <source>J Invest Dermatol.</source> (<year>2020</year>) <volume>140</volume>:<fpage>1401</fpage>&#x02013;<lpage>9</lpage>.e2. <pub-id pub-id-type="doi">10.1016/j.jid.2019.11.028</pub-id><pub-id pub-id-type="pmid">31978411</pub-id></citation></ref>
<ref id="B61">
<label>61.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rodic</surname> <given-names>N</given-names></name> <name><surname>Zampella</surname> <given-names>J</given-names></name> <name><surname>Sharma</surname> <given-names>R</given-names></name> <name><surname>Burns</surname> <given-names>KH</given-names></name> <name><surname>Taube</surname> <given-names>JM</given-names></name></person-group>. <article-title>Diagnostic utility of 5-hydroxymethylcytosine immunohistochemistry in melanocytic proliferations</article-title>. <source>J Cutan Pathol.</source> (<year>2015</year>) <volume>42</volume>:<fpage>807</fpage>&#x02013;<lpage>14</lpage>. <pub-id pub-id-type="doi">10.1111/cup.12564</pub-id><pub-id pub-id-type="pmid">26239102</pub-id></citation></ref>
<ref id="B62">
<label>62.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Uguen</surname> <given-names>A</given-names></name> <name><surname>Talagas</surname> <given-names>M</given-names></name> <name><surname>Costa</surname> <given-names>S</given-names></name> <name><surname>Duigou</surname> <given-names>S</given-names></name> <name><surname>Bouvier</surname> <given-names>S</given-names></name> <name><surname>De Braekeleer</surname> <given-names>M</given-names></name> <etal/></person-group>. <article-title>A p16-Ki-67-HMB45 immunohistochemistry scoring system as an ancillary diagnostic tool in the diagnosis of melanoma</article-title>. <source>Diagn Pathol.</source> (<year>2015</year>) <volume>10</volume>:<fpage>195</fpage>. <pub-id pub-id-type="doi">10.1186/s13000-015-0431-9</pub-id><pub-id pub-id-type="pmid">26503349</pub-id></citation></ref>
<ref id="B63">
<label>63.</label>
<citation citation-type="web"><person-group person-group-type="author"><name><surname>Lezcano</surname> <given-names>C</given-names></name> <name><surname>Pulitzer</surname> <given-names>M</given-names></name> <name><surname>Moy</surname> <given-names>AP</given-names></name> <name><surname>Hollmann</surname> <given-names>TJ</given-names></name> <name><surname>Jungbluth</surname> <given-names>AA</given-names></name> <name><surname>Busam</surname> <given-names>KJ</given-names></name></person-group>. <article-title>Immunohistochemistry for PRAME in the Distinction of Nodal Nevi From Metastatic Melanoma</article-title>. (<year>2019</year>). Available online at: <ext-link ext-link-type="uri" xlink:href="http://www.ajsp.com">www.ajsp.com</ext-link><pub-id pub-id-type="pmid">31633488</pub-id></citation></ref>
<ref id="B64">
<label>64.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kim</surname> <given-names>W</given-names></name> <name><surname>Han</surname> <given-names>TH</given-names></name> <name><surname>Kim</surname> <given-names>HJ</given-names></name> <name><surname>Park</surname> <given-names>MY</given-names></name> <name><surname>Kim</surname> <given-names>KS</given-names></name> <name><surname>Park</surname> <given-names>RW</given-names></name></person-group>. <article-title>An automated measurement of ciliary beating frequency using a combined optical flow and peak detection</article-title>. <source>Healthc Inform Res.</source> (<year>2011</year>) <volume>17</volume>:<fpage>111</fpage>&#x02013;<lpage>9</lpage>. <pub-id pub-id-type="doi">10.4258/hir.2011.17.2.111</pub-id><pub-id pub-id-type="pmid">21886872</pub-id></citation></ref>
<ref id="B65">
<label>65.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Snedecor</surname> <given-names>ER</given-names></name> <name><surname>Sung</surname> <given-names>CC</given-names></name> <name><surname>Moncayo</surname> <given-names>A</given-names></name> <name><surname>Rothstein</surname> <given-names>BE</given-names></name> <name><surname>Mockler</surname> <given-names>DC</given-names></name> <name><surname>Tonnesen</surname> <given-names>MG</given-names></name> <etal/></person-group>. <article-title>Loss of primary cilia in melanoma cells is likely independent of proliferation and cell cycle progression</article-title>. <source>J Invest Dermatol.</source> (<year>2015</year>) <volume>135</volume>:<fpage>1456</fpage>&#x02013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.1038/jid.2015.22</pub-id><pub-id pub-id-type="pmid">25629424</pub-id></citation></ref>
<ref id="B66">
<label>66.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Love</surname> <given-names>NR</given-names></name> <name><surname>Lang</surname> <given-names>UE</given-names></name> <name><surname>Cheung</surname> <given-names>C</given-names></name> <name><surname>Kim</surname> <given-names>J</given-names></name></person-group>. <article-title>Depletion of primary cilium in acral melanoma</article-title>. <source>J Cutan Pathol.</source> (<year>2019</year>) <volume>46</volume>:<fpage>665</fpage>&#x02013;<lpage>71</lpage>. <pub-id pub-id-type="doi">10.1111/cup.13484</pub-id><pub-id pub-id-type="pmid">31020686</pub-id></citation></ref>
<ref id="B67">
<label>67.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kim</surname> <given-names>J</given-names></name> <name><surname>Dabiri</surname> <given-names>S</given-names></name> <name><surname>Seeley</surname> <given-names>ES</given-names></name></person-group>. <article-title>Primary cilium depletion typifies cutaneous melanoma <italic>in situ</italic> and malignant melanoma</article-title>. <source>PLoS ONE.</source> (<year>2011</year>) <volume>6</volume>:<fpage>e27410</fpage>. <pub-id pub-id-type="doi">10.1371/journal.pone.0027410</pub-id><pub-id pub-id-type="pmid">22096570</pub-id></citation></ref>
<ref id="B68">
<label>68.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lang</surname> <given-names>UE</given-names></name> <name><surname>Cheung</surname> <given-names>C</given-names></name> <name><surname>Vladar</surname> <given-names>EK</given-names></name> <name><surname>Swetter</surname> <given-names>SM</given-names></name> <name><surname>Kim</surname> <given-names>J</given-names></name></person-group>. <article-title>Loss of primary cilia correlates with cytologic severity in dysplastic melanocytic nevi</article-title>. <source>J Cutan Pathol.</source> (<year>2016</year>) <volume>43</volume>:<fpage>113</fpage>&#x02013;<lpage>9</lpage>. <pub-id pub-id-type="doi">10.1111/cup.12612</pub-id><pub-id pub-id-type="pmid">26272818</pub-id></citation></ref>
<ref id="B69">
<label>69.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lang</surname> <given-names>UE</given-names></name> <name><surname>Love</surname> <given-names>NR</given-names></name> <name><surname>Cheung</surname> <given-names>C</given-names></name> <name><surname>McCalmont</surname> <given-names>TH</given-names></name> <name><surname>Kim</surname> <given-names>J</given-names></name></person-group>. <article-title>Use of the ciliation index to distinguish invasive melanoma from associated conventional melanocytic nevi</article-title>. <source>Am J Dermatopathol.</source> (<year>2020</year>) <volume>42</volume>:<fpage>11</fpage>&#x02013;<lpage>5</lpage>. <pub-id pub-id-type="doi">10.1097/DAD.0000000000001459</pub-id><pub-id pub-id-type="pmid">31145105</pub-id></citation></ref>
<ref id="B70">
<label>70.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Vetto</surname> <given-names>JT</given-names></name> <name><surname>Hsueh</surname> <given-names>EC</given-names></name> <name><surname>Gastman</surname> <given-names>BR</given-names></name> <name><surname>Dillon</surname> <given-names>LD</given-names></name> <name><surname>Monzon</surname> <given-names>FA</given-names></name> <name><surname>Cook</surname> <given-names>RW</given-names></name> <etal/></person-group>. <article-title>Guidance of sentinel lymph node biopsy decisions in patients with T1-T2 melanoma using gene expression profiling</article-title>. <source>Futur Oncol.</source> (<year>2019</year>) <volume>15</volume>:<fpage>1207</fpage>&#x02013;<lpage>17</lpage>. <pub-id pub-id-type="doi">10.2217/fon-2018-0912</pub-id><pub-id pub-id-type="pmid">30691297</pub-id></citation></ref>
<ref id="B71">
<label>71.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Meves</surname> <given-names>A</given-names></name> <name><surname>Nikolova</surname> <given-names>E</given-names></name> <name><surname>Heim</surname> <given-names>JB</given-names></name> <name><surname>Squirewell</surname> <given-names>EJ</given-names></name> <name><surname>Cappel</surname> <given-names>MA</given-names></name> <name><surname>Pittelkow</surname> <given-names>MR</given-names></name> <etal/></person-group>. <article-title>Tumor cell adhesion as a risk factor for sentinel lymph node metastasis in primary cutaneous melanoma</article-title>. <source>J Clin Oncol.</source> (<year>2015</year>) <volume>33</volume>:<fpage>2509</fpage>&#x02013;<lpage>15</lpage>. <pub-id pub-id-type="doi">10.1200/JCO.2014.60.7002</pub-id><pub-id pub-id-type="pmid">26150443</pub-id></citation></ref>
<ref id="B72">
<label>72.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bellomo</surname> <given-names>D</given-names></name> <name><surname>Arias-Mejias</surname> <given-names>SM</given-names></name> <name><surname>Ramana</surname> <given-names>C</given-names></name> <name><surname>Heim</surname> <given-names>JB</given-names></name> <name><surname>Quattrocchi</surname> <given-names>E</given-names></name> <name><surname>Sominidi-Damodaran</surname> <given-names>S</given-names></name> <etal/></person-group>. <article-title>Model combining tumor molecular and clinicopathologic risk factors predicts sentinel lymph node metastasis in primary cutaneous melanoma</article-title>. <source>JCO Precis Oncol.</source> (<year>2020</year>) <volume>4</volume>:<fpage>319</fpage>&#x02013;<lpage>34</lpage>. <pub-id pub-id-type="doi">10.1200/PO.19.00206</pub-id><pub-id pub-id-type="pmid">32405608</pub-id></citation></ref>
<ref id="B73">
<label>73.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hida</surname> <given-names>T</given-names></name> <name><surname>Yoneta</surname> <given-names>A</given-names></name> <name><surname>Wakamatsu</surname> <given-names>K</given-names></name> <name><surname>Yanagisawa</surname> <given-names>K</given-names></name> <name><surname>Ishii-Osai</surname> <given-names>Y</given-names></name> <name><surname>Kan</surname> <given-names>Y</given-names></name> <etal/></person-group>. <article-title>Circulating melanoma cells as a potential biomarker to detect metastasis and evaluate prognosis</article-title>. <source>Australas J Dermatol.</source> (<year>2016</year>) <volume>57</volume>:<fpage>145</fpage>&#x02013;<lpage>9</lpage>. <pub-id pub-id-type="doi">10.1111/ajd.12455</pub-id><pub-id pub-id-type="pmid">26931184</pub-id></citation></ref>
<ref id="B74">
<label>74.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kuhlmann</surname> <given-names>JD</given-names></name> <name><surname>Wimberger</surname> <given-names>P</given-names></name> <name><surname>Wilsch</surname> <given-names>K</given-names></name> <name><surname>Fluck</surname> <given-names>M</given-names></name> <name><surname>Suter</surname> <given-names>L</given-names></name> <name><surname>Brunner</surname> <given-names>G</given-names></name></person-group>. <article-title>Increased level of circulating U2 small nuclear RNA fragments indicates metastasis in melanoma patients</article-title>. <source>Clin Chem Lab Med.</source> (<year>2015</year>) <volume>53</volume>:<fpage>605</fpage>&#x02013;<lpage>11</lpage>. <pub-id pub-id-type="doi">10.1515/cclm-2014-1064</pub-id><pub-id pub-id-type="pmid">25741740</pub-id></citation></ref>
<ref id="B75">
<label>75.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Maier</surname> <given-names>T</given-names></name> <name><surname>Laubender</surname> <given-names>RP</given-names></name> <name><surname>Sturm</surname> <given-names>RA</given-names></name> <name><surname>Klingenstein</surname> <given-names>A</given-names></name> <name><surname>Korting</surname> <given-names>HC</given-names></name> <name><surname>Ruzicka</surname> <given-names>T</given-names></name> <etal/></person-group>. <article-title>Osteopontin expression in plasma of melanoma patients and in melanocytic tumours</article-title>. <source>J Eur Acad Dermatol Venereol.</source> (<year>2012</year>) <volume>26</volume>:<fpage>1084</fpage>&#x02013;<lpage>91</lpage>. <pub-id pub-id-type="doi">10.1111/j.1468-3083.2011.04210.x</pub-id><pub-id pub-id-type="pmid">21838826</pub-id></citation></ref>
<ref id="B76">
<label>76.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gerami</surname> <given-names>P</given-names></name> <name><surname>Cook</surname> <given-names>RW</given-names></name> <name><surname>Wilkinson</surname> <given-names>J</given-names></name> <name><surname>Russell</surname> <given-names>MC</given-names></name> <name><surname>Dhillon</surname> <given-names>N</given-names></name> <name><surname>Amaria</surname> <given-names>RN</given-names></name> <etal/></person-group>. <article-title>Development of a prognostic genetic signature to predict the metastatic risk associated with cutaneous melanoma</article-title>. <source>Clin Cancer Res.</source> (<year>2015</year>) <volume>21</volume>:<fpage>175</fpage>&#x02013;<lpage>83</lpage>. <pub-id pub-id-type="doi">10.1158/1078-0432.CCR-13-3316</pub-id></citation>
</ref>
<ref id="B77">
<label>77.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gerami</surname> <given-names>P</given-names></name> <name><surname>Cook</surname> <given-names>RW</given-names></name> <name><surname>Russell</surname> <given-names>MC</given-names></name> <name><surname>Wilkinson</surname> <given-names>J</given-names></name> <name><surname>Amaria</surname> <given-names>RN</given-names></name> <name><surname>Gonzalez</surname> <given-names>R</given-names></name> <etal/></person-group>. <article-title>Gene expression profiling for molecular staging of cutaneous melanoma in patients undergoing sentinel lymph node biopsy</article-title>. <source>J Am Acad Dermatol.</source> (<year>2015</year>) <volume>72</volume>:<fpage>780</fpage>&#x02013;<lpage>5</lpage>.e3. <pub-id pub-id-type="doi">10.1016/j.jaad.2015.01.009</pub-id></citation>
</ref>
<ref id="B78">
<label>78.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ferris</surname> <given-names>LK</given-names></name> <name><surname>Farberg</surname> <given-names>AS</given-names></name> <name><surname>Middlebrook</surname> <given-names>B</given-names></name> <name><surname>Johnson</surname> <given-names>CE</given-names></name> <name><surname>Lassen</surname> <given-names>N</given-names></name> <name><surname>Oelschlager</surname> <given-names>KM</given-names></name> <etal/></person-group>. <article-title>Identification of high-risk cutaneous melanoma tumors is improved when combining the online American Joint Committee on Cancer Individualized Melanoma Patient Outcome Prediction Tool with a 31-gene expression profile&#x02013;based classification</article-title>. <source>J Am Acad Dermatol.</source> (<year>2017</year>) <volume>76</volume>:<fpage>818</fpage>&#x02013;<lpage>25</lpage>.e3. <pub-id pub-id-type="doi">10.1016/j.jaad.2016.11.051</pub-id><pub-id pub-id-type="pmid">28110997</pub-id></citation></ref>
<ref id="B79">
<label>79.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Autilio</surname> <given-names>C</given-names></name> <name><surname>Paolillo</surname> <given-names>C</given-names></name> <name><surname>Lavieri</surname> <given-names>MM</given-names></name> <name><surname>Pocino</surname> <given-names>K</given-names></name> <name><surname>De Paolis</surname> <given-names>E</given-names></name> <name><surname>Di Stasio</surname> <given-names>E</given-names></name> <etal/></person-group>. <article-title>PAX3d mRNA over 2.76 copies/&#x003BC;L in the bloodstream predicts cutaneous malignant melanoma relapse</article-title>. <source>Oncotarget.</source> (<year>2017</year>) <volume>8</volume>:<fpage>85479</fpage>&#x02013;<lpage>91</lpage>. <pub-id pub-id-type="doi">10.18632/oncotarget.20177</pub-id></citation></ref>
<ref id="B80">
<label>80.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lee</surname> <given-names>JH</given-names></name> <name><surname>Long</surname> <given-names>GV</given-names></name> <name><surname>Menzies</surname> <given-names>AM</given-names></name> <name><surname>Lo</surname> <given-names>S</given-names></name> <name><surname>Guminski</surname> <given-names>A</given-names></name> <name><surname>Whitbourne</surname> <given-names>K</given-names></name> <etal/></person-group>. <article-title>Association between circulating tumor DNA and pseudoprogression in patients with metastatic melanoma treated with anti&#x02013;programmed cell death 1 antibodies</article-title>. <source>JAMA Oncol.</source> (<year>2018</year>) <volume>4</volume>:<fpage>717</fpage>&#x02013;<lpage>21</lpage>. <pub-id pub-id-type="doi">10.1001/jamaoncol.2017.5332</pub-id><pub-id pub-id-type="pmid">29423503</pub-id></citation></ref>
<ref id="B81">
<label>81.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Schreuer</surname> <given-names>M</given-names></name> <name><surname>Meersseman</surname> <given-names>G</given-names></name> <name><surname>Van Den Herrewegen</surname> <given-names>S</given-names></name> <name><surname>Jansen</surname> <given-names>Y</given-names></name> <name><surname>Chevolet</surname> <given-names>I</given-names></name> <name><surname>Bott</surname> <given-names>A</given-names></name> <etal/></person-group>. <article-title>Quantitative assessment of BRAF V600 mutant circulating cell-free tumor DNA as a tool for therapeutic monitoring in metastatic melanoma patients treated with BRAF/MEK inhibitors</article-title>. <source>J Transl Med.</source> (<year>2016</year>) <volume>14</volume>:<fpage>1</fpage>&#x02013;<lpage>11</lpage>. <pub-id pub-id-type="doi">10.1186/s12967-016-0852-6</pub-id><pub-id pub-id-type="pmid">27095081</pub-id></citation></ref>
<ref id="B82">
<label>82.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cook</surname> <given-names>RW</given-names></name> <name><surname>Middlebrook</surname> <given-names>B</given-names></name> <name><surname>Wilkinson</surname> <given-names>J</given-names></name> <name><surname>Covington</surname> <given-names>KR</given-names></name> <name><surname>Oelschlager</surname> <given-names>K</given-names></name> <name><surname>Monzon</surname> <given-names>FA</given-names></name> <etal/></person-group>. <article-title>Analytic validity of DecisionDx-Melanoma, a gene expression profile test for determining metastatic risk in melanoma patients</article-title>. <source>Diagn Pathol.</source> (<year>2018</year>) <volume>13</volume>:<fpage>13</fpage>. <pub-id pub-id-type="doi">10.1186/s13000-018-0690-3</pub-id><pub-id pub-id-type="pmid">29433548</pub-id></citation></ref>
<ref id="B83">
<label>83.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zager</surname> <given-names>JS</given-names></name> <name><surname>Gastman</surname> <given-names>BR</given-names></name> <name><surname>Leachman</surname> <given-names>S</given-names></name> <name><surname>Gonzalez</surname> <given-names>RC</given-names></name> <name><surname>Fleming</surname> <given-names>MD</given-names></name> <name><surname>Ferris</surname> <given-names>LK</given-names></name> <etal/></person-group>. <article-title>Performance of a prognostic 31-gene expression profile in an independent cohort of 523 cutaneous melanoma patients</article-title>. <source>BMC Cancer.</source> (<year>2018</year>) <volume>18</volume>:<fpage>130</fpage>. <pub-id pub-id-type="doi">10.1186/s12885-018-4016-3</pub-id><pub-id pub-id-type="pmid">29402264</pub-id></citation></ref>
<ref id="B84">
<label>84.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gastman</surname> <given-names>BR</given-names></name> <name><surname>Zager</surname> <given-names>JS</given-names></name> <name><surname>Messina</surname> <given-names>JL</given-names></name> <name><surname>Cook</surname> <given-names>RW</given-names></name> <name><surname>Covington</surname> <given-names>KR</given-names></name> <name><surname>Middlebrook</surname> <given-names>B</given-names></name> <etal/></person-group>. <article-title>Performance of a 31-gene expression profile test in cutaneous melanomas of the head and neck</article-title>. <source>Head Neck.</source> (<year>2019</year>) <volume>41</volume>:<fpage>871</fpage>&#x02013;<lpage>9</lpage>. <pub-id pub-id-type="doi">10.1002/hed.25473</pub-id><pub-id pub-id-type="pmid">30694001</pub-id></citation></ref>
<ref id="B85">
<label>85.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gastman</surname> <given-names>BR</given-names></name> <name><surname>Gerami</surname> <given-names>P</given-names></name> <name><surname>Kurley</surname> <given-names>SJ</given-names></name> <name><surname>Cook</surname> <given-names>RW</given-names></name> <name><surname>Leachman</surname> <given-names>S</given-names></name> <name><surname>Vetto</surname> <given-names>JT</given-names></name></person-group>. <article-title>Identification of patients at risk of metastasis using a prognostic 31-gene expression profile in subpopulations of melanoma patients with favorable outcomes by standard criteria</article-title>. <source>J Am Acad Dermatol.</source> (<year>2019</year>) <volume>80</volume>:<fpage>149</fpage>&#x02013;<lpage>57</lpage>.e4. <pub-id pub-id-type="doi">10.1016/j.jaad.2018.07.028</pub-id><pub-id pub-id-type="pmid">30081113</pub-id></citation></ref>
<ref id="B86">
<label>86.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Keller</surname> <given-names>J</given-names></name> <name><surname>Schwartz</surname> <given-names>TL</given-names></name> <name><surname>Lizalek</surname> <given-names>JM</given-names></name> <name><surname>Chang</surname> <given-names>E</given-names></name> <name><surname>Patel</surname> <given-names>AD</given-names></name> <name><surname>Hurley</surname> <given-names>MY</given-names></name> <etal/></person-group>. <article-title>Prospective validation of the prognostic 31-gene expression profiling test in primary cutaneous melanoma</article-title>. <source>Cancer Med.</source> (<year>2019</year>) <volume>8</volume>:<fpage>2205</fpage>&#x02013;<lpage>12</lpage>. <pub-id pub-id-type="doi">10.1002/cam4.2128</pub-id><pub-id pub-id-type="pmid">30950242</pub-id></citation></ref>
<ref id="B87">
<label>87.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Podlipnik</surname> <given-names>S</given-names></name> <name><surname>Carrera</surname> <given-names>C</given-names></name> <name><surname>Boada</surname> <given-names>A</given-names></name> <name><surname>Richarz</surname> <given-names>NA</given-names></name> <name><surname>L&#x000F3;pez-Estebaranz</surname> <given-names>JL</given-names></name> <name><surname>Pinedo-Moraleda</surname> <given-names>F</given-names></name> <etal/></person-group>. <article-title>Early outcome of a 31-gene expression profile test in 86 AJCC stage IB-II melanoma patients. A prospective multicentre cohort study</article-title>. <source>J Eur Acad Dermatol Venereol.</source> (<year>2019</year>) <volume>33</volume>:<fpage>857</fpage>&#x02013;<lpage>62</lpage>. <pub-id pub-id-type="doi">10.1111/jdv.15454</pub-id><pub-id pub-id-type="pmid">30702163</pub-id></citation></ref>
<ref id="B88">
<label>88.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wong</surname> <given-names>SL</given-names></name> <name><surname>Kattan</surname> <given-names>MW</given-names></name> <name><surname>McMasters</surname> <given-names>KM</given-names></name> <name><surname>Coit</surname> <given-names>DG</given-names></name></person-group>. <article-title>A nomogram that predicts the presence of sentinel node metastasis in melanoma with better discrimination than the American joint committee on cancer staging system</article-title>. <source>Ann Surg Oncol.</source> (<year>2005</year>) <volume>12</volume>:<fpage>282</fpage>&#x02013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.1245/ASO.2005.05.016</pub-id><pub-id pub-id-type="pmid">15827679</pub-id></citation></ref>
<ref id="B89">
<label>89.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lo</surname> <given-names>SN</given-names></name> <name><surname>Ma</surname> <given-names>J</given-names></name> <name><surname>Scolyer</surname> <given-names>RA</given-names></name> <name><surname>Haydu</surname> <given-names>LE</given-names></name> <name><surname>Stretch</surname> <given-names>JR</given-names></name> <name><surname>Saw</surname> <given-names>RPM</given-names></name> <etal/></person-group>. <article-title>Improved risk prediction calculator for sentinel node positivity in patients with melanoma: the melanoma institute australia nomogram</article-title>. <source>J Clin Oncol.</source> (<year>2020</year>) <volume>38</volume>:<fpage>2719</fpage>&#x02013;<lpage>27</lpage>. <pub-id pub-id-type="doi">10.1200/JCO.19.02362</pub-id><pub-id pub-id-type="pmid">32530761</pub-id></citation></ref>
<ref id="B90">
<label>90.</label>
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Swetter</surname> <given-names>SM</given-names></name> <name><surname>Thompson</surname> <given-names>JA</given-names></name> <name><surname>Albertini</surname> <given-names>MR</given-names></name> <name><surname>Barker</surname> <given-names>CA</given-names></name> <name><surname>Boland</surname> <given-names>G</given-names></name> <name><surname>Carson</surname> <given-names>WE</given-names></name> <etal/></person-group>. <article-title>NCCN Guidelines Version 1.2021</article-title> <publisher-loc>Melanoma: Cutaneous. National Comprehensive Cancer Network</publisher-loc> (<year>2020</year>).</citation></ref>
<ref id="B91">
<label>91.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dummer</surname> <given-names>R</given-names></name> <name><surname>Hauschild</surname> <given-names>A</given-names></name> <name><surname>Santinami</surname> <given-names>M</given-names></name> <name><surname>Atkinson</surname> <given-names>V</given-names></name> <name><surname>Mandal&#x000E0;</surname> <given-names>M</given-names></name> <name><surname>Kirkwood</surname> <given-names>JM</given-names></name> <etal/></person-group>. <article-title>Five-year analysis of adjuvant dabrafenib plus trametinib in stage III melanoma</article-title>. <source>N Engl J Med.</source> (<year>2020</year>) <volume>383</volume>:<fpage>1139</fpage>&#x02013;<lpage>48</lpage>. <pub-id pub-id-type="doi">10.1056/NEJMoa2005493</pub-id><pub-id pub-id-type="pmid">32877599</pub-id></citation></ref>
<ref id="B92">
<label>92.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Eggermont</surname> <given-names>AMM</given-names></name> <name><surname>Blank</surname> <given-names>CU</given-names></name> <name><surname>Mandala</surname> <given-names>M</given-names></name> <name><surname>Long</surname> <given-names>GV</given-names></name> <name><surname>Atkinson</surname> <given-names>V</given-names></name> <name><surname>Dalle</surname> <given-names>S</given-names></name> <etal/></person-group>. <article-title>Adjuvant pembrolizumab versus placebo in resected stage III melanoma</article-title>. <source>N Engl J Med.</source> (<year>2018</year>) <volume>378</volume>:<fpage>1789</fpage>&#x02013;<lpage>801</lpage>. <pub-id pub-id-type="doi">10.1056/NEJMoa1802357</pub-id><pub-id pub-id-type="pmid">31870188</pub-id></citation></ref>
<ref id="B93">
<label>93.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Morton</surname> <given-names>DL</given-names></name> <name><surname>Thompson</surname> <given-names>JF</given-names></name> <name><surname>Cochran</surname> <given-names>AJ</given-names></name> <name><surname>Mozzillo</surname> <given-names>N</given-names></name> <name><surname>Nieweg</surname> <given-names>OE</given-names></name> <name><surname>Roses</surname> <given-names>DF</given-names></name> <etal/></person-group>. <article-title>Final trial report of sentinel-node biopsy versus nodal observation in melanoma</article-title>. <source>N Engl J Med.</source> (<year>2014</year>) <volume>370</volume>:<fpage>599</fpage>&#x02013;<lpage>609</lpage>. <pub-id pub-id-type="doi">10.1056/NEJMoa1310460</pub-id><pub-id pub-id-type="pmid">24521106</pub-id></citation></ref>
<ref id="B94">
<label>94.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Karakousis</surname> <given-names>GC</given-names></name> <name><surname>Gimotty</surname> <given-names>PA</given-names></name> <name><surname>Czerniecki</surname> <given-names>BJ</given-names></name> <name><surname>Elder</surname> <given-names>DE</given-names></name> <name><surname>Elenitsas</surname> <given-names>R</given-names></name> <name><surname>Ming</surname> <given-names>ME</given-names></name> <etal/></person-group>. <article-title>Regional nodal metastatic disease is the strongest predictor of survival in patients with thin vertical growth phase melanomas: a case for SLN staging biopsy in these patients</article-title>. <source>Ann Surg Oncol.</source> (<year>2007</year>) <volume>14</volume>:<fpage>1596</fpage>&#x02013;<lpage>603</lpage>. <pub-id pub-id-type="doi">10.1245/s10434-006-9319-y</pub-id><pub-id pub-id-type="pmid">17285396</pub-id></citation></ref>
<ref id="B95">
<label>95.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Saltman</surname> <given-names>BE</given-names></name> <name><surname>Ganly</surname> <given-names>I</given-names></name> <name><surname>Patel</surname> <given-names>SG</given-names></name> <name><surname>Coit</surname> <given-names>DG</given-names></name> <name><surname>Brady</surname> <given-names>MS</given-names></name> <name><surname>Wong</surname> <given-names>RJ</given-names></name> <etal/></person-group>. <article-title>Prognostic implication of sentinel lymph node biopsy in cutaneous head and neck melanoma</article-title>. <source>Head Neck.</source> (<year>2010</year>) <volume>32</volume>:<fpage>1686</fpage>&#x02013;<lpage>92</lpage>. <pub-id pub-id-type="doi">10.1002/hed.21390</pub-id><pub-id pub-id-type="pmid">20848412</pub-id></citation></ref>
<ref id="B96">
<label>96.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Marchetti</surname> <given-names>MA</given-names></name> <name><surname>Coit</surname> <given-names>DG</given-names></name> <name><surname>Dusza</surname> <given-names>SW</given-names></name> <name><surname>Yu</surname> <given-names>A</given-names></name> <name><surname>McLean</surname> <given-names>LT</given-names></name> <name><surname>Hu</surname> <given-names>Y</given-names></name> <etal/></person-group>. <article-title>Performance of gene expression profile tests for prognosis in patients with localized cutaneous melanoma: a systematic review and meta-analysis</article-title>. <source>JAMA Dermatol.</source> (<year>2020</year>) <volume>156</volume>:<fpage>953</fpage>&#x02013;<lpage>62</lpage>. <pub-id pub-id-type="doi">10.1001/jamadermatol.2020.1731</pub-id><pub-id pub-id-type="pmid">32745161</pub-id></citation></ref>
<ref id="B97">
<label>97.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Pantel</surname> <given-names>K</given-names></name> <name><surname>Alix-Panabi&#x000E8;res</surname> <given-names>C</given-names></name></person-group>. <article-title>Liquid biopsy and minimal residual disease &#x02014; latest advances and implications for cure</article-title>. <source>Nat Rev Clin Oncol.</source> (<year>2019</year>) <volume>16</volume>:<fpage>409</fpage>&#x02013;<lpage>24</lpage>. <pub-id pub-id-type="doi">10.1038/s41571-019-0187-3</pub-id><pub-id pub-id-type="pmid">30796368</pub-id></citation></ref>
<ref id="B98">
<label>98.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Finck</surname> <given-names>SJ</given-names></name> <name><surname>Giuliano</surname> <given-names>AE</given-names></name> <name><surname>Morton</surname> <given-names>DL</given-names></name></person-group>. <article-title>LDH and melanoma</article-title>. <source>Cancer.</source> (<year>1983</year>) <volume>51</volume>:<fpage>840</fpage>&#x02013;<lpage>3</lpage>.</citation></ref>
<ref id="B99">
<label>99.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Weide</surname> <given-names>B</given-names></name> <name><surname>Els&#x000E4;sser</surname> <given-names>M</given-names></name> <name><surname>B&#x000FC;ttner</surname> <given-names>P</given-names></name> <name><surname>Pflugfelder</surname> <given-names>A</given-names></name> <name><surname>Leiter</surname> <given-names>U</given-names></name> <name><surname>Eigentler</surname> <given-names>TK</given-names></name> <etal/></person-group>. <article-title>Serum markers lactate dehydrogenase and S100B predict independently disease outcome in melanoma patients with distant metastasis</article-title>. <source>Br J Cancer.</source> (<year>2012</year>) <volume>107</volume>:<fpage>422</fpage>&#x02013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.1038/bjc.2012.306</pub-id><pub-id pub-id-type="pmid">22782342</pub-id></citation></ref>
<ref id="B100">
<label>100.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Long</surname> <given-names>GV</given-names></name> <name><surname>Grob</surname> <given-names>JJ</given-names></name> <name><surname>Nathan</surname> <given-names>P</given-names></name> <name><surname>Ribas</surname> <given-names>A</given-names></name> <name><surname>Robert</surname> <given-names>C</given-names></name> <name><surname>Schadendorf</surname> <given-names>D</given-names></name> <etal/></person-group>. <article-title>Factors predictive of response, disease progression, and overall survival after dabrafenib and trametinib combination treatment: a pooled analysis of individual patient data from randomised trials</article-title>. <source>Lancet Oncol.</source> (<year>2016</year>) <volume>17</volume>:<fpage>1743</fpage>&#x02013;<lpage>54</lpage>. <pub-id pub-id-type="doi">10.1016/S1470-2045(16)30578-2</pub-id><pub-id pub-id-type="pmid">27864013</pub-id></citation></ref>
<ref id="B101">
<label>101.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Nosrati</surname> <given-names>A</given-names></name> <name><surname>Tsai</surname> <given-names>KK</given-names></name> <name><surname>Goldinger</surname> <given-names>SM</given-names></name> <name><surname>Tumeh</surname> <given-names>P</given-names></name> <name><surname>Grimes</surname> <given-names>B</given-names></name> <name><surname>Loo</surname> <given-names>K</given-names></name> <etal/></person-group>. <article-title>Evaluation of clinicopathological factors in PD-1 response: derivation and validation of a prediction scale for response to PD-1 monotherapy</article-title>. <source>Br J Cancer.</source> (<year>2017</year>) <volume>116</volume>:<fpage>1141</fpage>&#x02013;<lpage>7</lpage>. <pub-id pub-id-type="doi">10.1038/bjc.2017.70</pub-id><pub-id pub-id-type="pmid">28324889</pub-id></citation></ref>
<ref id="B102">
<label>102.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Diamandis</surname> <given-names>EP</given-names></name> <name><surname>Fiala</surname> <given-names>C</given-names></name></person-group>. <article-title>Can circulating tumor DNA be used for direct and early stage cancer detection?</article-title> <source>F1000Research.</source> (<year>2017</year>) <volume>6</volume>:<fpage>2129</fpage>. <pub-id pub-id-type="doi">10.12688/f1000research.13440.1</pub-id><pub-id pub-id-type="pmid">29333251</pub-id></citation></ref>
<ref id="B103">
<label>103.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Romano</surname> <given-names>A</given-names></name> <name><surname>Palumbo</surname> <given-names>GA</given-names></name> <name><surname>Parrinello</surname> <given-names>NL</given-names></name> <name><surname>Conticello</surname> <given-names>C</given-names></name> <name><surname>Martello</surname> <given-names>M</given-names></name> <name><surname>Terragna</surname> <given-names>C</given-names></name></person-group>. <article-title>Minimal residual disease assessment within the bone marrow of multiple myeloma: a review of caveats, clinical significance and future perspectives</article-title>. <source>Front Oncol.</source> (<year>2019</year>) <volume>9</volume>:<fpage>699</fpage>. <pub-id pub-id-type="doi">10.3389/fonc.2019.00699</pub-id><pub-id pub-id-type="pmid">31482061</pub-id></citation></ref>
<ref id="B104">
<label>104.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>McDonald</surname> <given-names>BR</given-names></name> <name><surname>Contente-Cuomo</surname> <given-names>T</given-names></name> <name><surname>Sammut</surname> <given-names>SJ</given-names></name> <name><surname>Odenheimer-Bergman</surname> <given-names>A</given-names></name> <name><surname>Ernst</surname> <given-names>B</given-names></name> <name><surname>Perdigones</surname> <given-names>N</given-names></name> <etal/></person-group>. <article-title>Personalized circulating tumor DNA analysis to detect residual disease after neoadjuvant therapy in breast cancer</article-title>. <source>Sci Transl Med.</source> 11:eaax7392. <pub-id pub-id-type="doi">10.1126/scitranslmed.aax7392</pub-id><pub-id pub-id-type="pmid">31391323</pub-id></citation></ref>
<ref id="B105">
<label>105.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chin</surname> <given-names>RI</given-names></name> <name><surname>Chen</surname> <given-names>K</given-names></name> <name><surname>Usmani</surname> <given-names>A</given-names></name> <name><surname>Chua</surname> <given-names>C</given-names></name> <name><surname>Harris</surname> <given-names>PK</given-names></name> <name><surname>Binkley</surname> <given-names>MS</given-names></name> <etal/></person-group>. <article-title>Detection of solid tumor molecular Residual Disease (MRD) using circulating tumor DNA (ctDNA)</article-title>. <source>Mol Diagn Ther.</source> (<year>2019</year>) <volume>23</volume>:<fpage>311</fpage>. <pub-id pub-id-type="doi">10.1007/s40291-019-00390-5</pub-id><pub-id pub-id-type="pmid">30941670</pub-id></citation></ref>
<ref id="B106">
<label>106.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bratman</surname> <given-names>SV</given-names></name> <name><surname>Yang</surname> <given-names>SYC</given-names></name> <name><surname>Iafolla</surname> <given-names>MAJ</given-names></name> <name><surname>Liu</surname> <given-names>Z</given-names></name> <name><surname>Hansen</surname> <given-names>AR</given-names></name> <name><surname>Bedard</surname> <given-names>PL</given-names></name> <etal/></person-group>. <article-title>Personalized circulating tumor DNA analysis as a predictive biomarker in solid tumor patients treated with pembrolizumab</article-title>. <source>Nat Cancer.</source> (<year>2020</year>) <volume>1</volume>:<fpage>873</fpage>&#x02013;<lpage>81</lpage>. <pub-id pub-id-type="doi">10.1038/s43018-020-0096-5</pub-id></citation></ref>
<ref id="B107">
<label>107.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>V&#x000E1;raljai</surname> <given-names>R</given-names></name> <name><surname>Wistuba-Hamprecht</surname> <given-names>K</given-names></name> <name><surname>Seremet</surname> <given-names>T</given-names></name> <name><surname>Diaz</surname> <given-names>JMS</given-names></name> <name><surname>Nsengimana</surname> <given-names>J</given-names></name> <name><surname>Sucker</surname> <given-names>A</given-names></name> <etal/></person-group>. <article-title>Application of circulating cell-free tumor DNA profiles for therapeutic monitoring and outcome prediction in genetically heterogeneous metastatic melanoma</article-title>. <source>JCO Precis Oncol.</source> (<year>2019</year>) <volume>3</volume>:<fpage>PO</fpage>.18.00229. <pub-id pub-id-type="doi">10.1200/PO.18.00229</pub-id><pub-id pub-id-type="pmid">32914028</pub-id></citation></ref>
<ref id="B108">
<label>108.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Eggermont</surname> <given-names>AMM</given-names></name> <name><surname>Blank</surname> <given-names>CU</given-names></name> <name><surname>Mandala</surname> <given-names>M</given-names></name> <name><surname>Long</surname> <given-names>GV</given-names></name> <name><surname>Atkinson</surname> <given-names>VG</given-names></name> <name><surname>Dalle</surname> <given-names>S</given-names></name> <etal/></person-group>. <article-title>Longer follow-up confirms recurrence-free survival benefit of adjuvant pembrolizumab in high-risk stage III melanoma: updated results from the EORTC 1325-MG/KEYNOTE-054 trial</article-title>. <source>J Clin Oncol.</source> (<year>2020</year>) <volume>38</volume>:<fpage>3925</fpage>&#x02013;<lpage>36</lpage>. <pub-id pub-id-type="doi">10.1200/JCO.20.02110</pub-id><pub-id pub-id-type="pmid">32946353</pub-id></citation></ref>
<ref id="B109">
<label>109.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Amaral</surname> <given-names>TMS</given-names></name> <name><surname>Hoffmann</surname> <given-names>MC</given-names></name> <name><surname>Sinnberg</surname> <given-names>T</given-names></name> <name><surname>Niessner</surname> <given-names>H</given-names></name> <name><surname>S&#x000FC;lberg</surname> <given-names>H</given-names></name> <name><surname>Eigentler</surname> <given-names>TK</given-names></name> <etal/></person-group>. <article-title>Clinical validation of a prognostic 11-gene expression profiling score in prospectively collected FFPE tissue of patients with AJCC v8 stage II cutaneous melanoma</article-title>. <source>Eur J Cancer.</source> (<year>2020</year>) <volume>125</volume>:<fpage>38</fpage>&#x02013;<lpage>45</lpage>. <pub-id pub-id-type="doi">10.1016/j.ejca.2019.10.027</pub-id><pub-id pub-id-type="pmid">31838403</pub-id></citation></ref>
<ref id="B110">
<label>110.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ritterhouse</surname> <given-names>LL</given-names></name> <name><surname>Barletta</surname> <given-names>JA</given-names></name></person-group>. <article-title>BRAF V600E mutation-specific antibody: a review</article-title>. <source>Semin Diagn Pathol.</source> (<year>2015</year>) <volume>32</volume>:<fpage>400</fpage>&#x02013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.1053/j.semdp.2015.02.010</pub-id><pub-id pub-id-type="pmid">25744437</pub-id></citation></ref>
<ref id="B111">
<label>111.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Larkin</surname> <given-names>J</given-names></name> <name><surname>Chiarion-Sileni</surname> <given-names>V</given-names></name> <name><surname>Gonzalez</surname> <given-names>R</given-names></name> <name><surname>Grob</surname> <given-names>J-J</given-names></name> <name><surname>Rutkowski</surname> <given-names>P</given-names></name> <name><surname>Lao</surname> <given-names>CD</given-names></name> <etal/></person-group>. <article-title>Five-year survival with combined nivolumab and ipilimumab in advanced melanoma</article-title>. <source>N Engl J Med.</source> (<year>2019</year>) <volume>381</volume>:<fpage>1535</fpage>&#x02013;<lpage>46</lpage>. <pub-id pub-id-type="doi">10.1056/NEJMoa1910836</pub-id><pub-id pub-id-type="pmid">31562797</pub-id></citation></ref>
<ref id="B112">
<label>112.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Luke</surname> <given-names>JJ</given-names></name></person-group>. <article-title>Comprehensive clinical trial data summation for BRAF-MEK inhibition and checkpoint immunotherapy in metastatic melanoma</article-title>. <source>Oncologist.</source> (<year>2019</year>) <volume>24</volume>:<fpage>e1197</fpage>. <pub-id pub-id-type="doi">10.1634/theoncologist.2018-0876</pub-id><pub-id pub-id-type="pmid">31064886</pub-id></citation></ref>
<ref id="B113">
<label>113.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Robert</surname> <given-names>C</given-names></name> <name><surname>Grob</surname> <given-names>JJ</given-names></name> <name><surname>Stroyakovskiy</surname> <given-names>D</given-names></name> <name><surname>Karaszewska</surname> <given-names>B</given-names></name> <name><surname>Hauschild</surname> <given-names>A</given-names></name> <name><surname>Levchenko</surname> <given-names>E</given-names></name> <etal/></person-group>. <article-title>Five-Year Outcomes with dabrafenib plus trametinib in metastatic melanoma</article-title>. <source>N Engl J Med.</source> (<year>2019</year>) <volume>381</volume>:<fpage>626</fpage>&#x02013;<lpage>36</lpage>. <pub-id pub-id-type="doi">10.1056/NEJMoa1904059</pub-id><pub-id pub-id-type="pmid">31166680</pub-id></citation></ref>
<ref id="B114">
<label>114.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Topalian</surname> <given-names>SL</given-names></name> <name><surname>Sznol</surname> <given-names>M</given-names></name> <name><surname>McDermott</surname> <given-names>DF</given-names></name> <name><surname>Kluger</surname> <given-names>HM</given-names></name> <name><surname>Carvajal</surname> <given-names>RD</given-names></name> <name><surname>Sharfman</surname> <given-names>WH</given-names></name> <etal/></person-group>. <article-title>Survival, durable tumor remission, and long-term safety in patients with advanced melanoma receiving nivolumab</article-title>. <source>J Clin Oncol.</source> (<year>2014</year>) <volume>32</volume>:<fpage>1020</fpage>&#x02013;<lpage>30</lpage>. <pub-id pub-id-type="doi">10.1200/JCO.2013.53.0105</pub-id><pub-id pub-id-type="pmid">24590637</pub-id></citation></ref>
<ref id="B115">
<label>115.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ribas</surname> <given-names>A</given-names></name> <name><surname>Hamid</surname> <given-names>O</given-names></name> <name><surname>Daud</surname> <given-names>A</given-names></name> <name><surname>Hodi</surname> <given-names>FS</given-names></name> <name><surname>Wolchok</surname> <given-names>JD</given-names></name> <name><surname>Kefford</surname> <given-names>R</given-names></name> <etal/></person-group>. <article-title>Association of pembrolizumab with tumor response and survival among patients with advanced melanoma</article-title>. <source>JAMA.</source> (<year>2016</year>) <volume>315</volume>:<fpage>1600</fpage>&#x02013;<lpage>9</lpage>. <pub-id pub-id-type="doi">10.1001/jama.2016.4059</pub-id><pub-id pub-id-type="pmid">27092830</pub-id></citation></ref>
<ref id="B116">
<label>116.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Betof Warner</surname> <given-names>A</given-names></name> <name><surname>Palmer</surname> <given-names>JS</given-names></name> <name><surname>Shoushtari</surname> <given-names>AN</given-names></name> <name><surname>Goldman</surname> <given-names>DA</given-names></name> <name><surname>Panageas</surname> <given-names>KS</given-names></name> <name><surname>Hayes</surname> <given-names>SA</given-names></name> <etal/></person-group>. <article-title>Long-term outcomes and responses to retreatment in patients with melanoma treated with PD-1 blockade</article-title>. <source>J Clin Oncol.</source> (<year>2020</year>) <volume>38</volume>:<fpage>1655</fpage>&#x02013;<lpage>63</lpage>. <pub-id pub-id-type="doi">10.1200/JCO.19.01464</pub-id><pub-id pub-id-type="pmid">32053428</pub-id></citation></ref>
<ref id="B117">
<label>117.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lo</surname> <given-names>SN</given-names></name> <name><surname>Scolyer</surname> <given-names>RA</given-names></name> <name><surname>Thompson</surname> <given-names>JF</given-names></name></person-group>. <article-title>Long-term survival of patients with thin (T1) cutaneous melanomas: a breslow thickness cut point of 0.8 mm separates higher-risk and lower-risk tumors</article-title>. <source>Ann Surg Oncol.</source> (<year>2018</year>) <volume>25</volume>:<fpage>894</fpage>&#x02013;<lpage>902</lpage>. <pub-id pub-id-type="doi">10.1245/s10434-017-6325-1</pub-id><pub-id pub-id-type="pmid">29330716</pub-id></citation></ref>
<ref id="B118">
<label>118.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Buder-Bakhaya</surname> <given-names>K</given-names></name> <name><surname>Hassel</surname> <given-names>JC</given-names></name></person-group>. <article-title>Biomarkers for clinical benefit of immune checkpoint inhibitor treatment-a review from the melanoma perspective and beyond</article-title>. <source>Front Immunol.</source> (<year>2018</year>) <volume>9</volume>:<fpage>1474</fpage>. <pub-id pub-id-type="doi">10.3389/fimmu.2018.01474</pub-id><pub-id pub-id-type="pmid">30002656</pub-id></citation></ref>
<ref id="B119">
<label>119.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Morrison</surname> <given-names>C</given-names></name> <name><surname>Pabla</surname> <given-names>S</given-names></name> <name><surname>Conroy</surname> <given-names>JM</given-names></name> <name><surname>Nesline</surname> <given-names>MK</given-names></name> <name><surname>Glenn</surname> <given-names>ST</given-names></name> <name><surname>Dressman</surname> <given-names>D</given-names></name> <etal/></person-group>. <article-title>Predicting response to checkpoint inhibitors in melanoma beyond PD-L1 and mutational burden</article-title>. <source>J Immunother Cancer.</source> (<year>2018</year>) <volume>6</volume>:<fpage>32</fpage>. <pub-id pub-id-type="doi">10.1186/s40425-018-0344-8</pub-id><pub-id pub-id-type="pmid">29743104</pub-id></citation></ref>
<ref id="B120">
<label>120.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yi</surname> <given-names>M</given-names></name> <name><surname>Jiao</surname> <given-names>D</given-names></name> <name><surname>Xu</surname> <given-names>H</given-names></name> <name><surname>Liu</surname> <given-names>Q</given-names></name> <name><surname>Zhao</surname> <given-names>W</given-names></name> <name><surname>Han</surname> <given-names>X</given-names></name> <etal/></person-group>. <article-title>Biomarkers for predicting efficacy of PD-1/PD-L1 inhibitors</article-title>. <source>Mol Cancer.</source> (<year>2018</year>) <volume>17</volume>:<fpage>129</fpage>. <pub-id pub-id-type="doi">10.1186/s12943-018-0864-3</pub-id><pub-id pub-id-type="pmid">30139382</pub-id></citation></ref>
<ref id="B121">
<label>121.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Arora</surname> <given-names>S</given-names></name> <name><surname>Velichinskii</surname> <given-names>R</given-names></name> <name><surname>Lesh</surname> <given-names>RW</given-names></name> <name><surname>Ali</surname> <given-names>U</given-names></name> <name><surname>Kubiak</surname> <given-names>M</given-names></name> <name><surname>Bansal</surname> <given-names>P</given-names></name> <etal/></person-group>. <article-title>Existing and emerging biomarkers for immune checkpoint immunotherapy in solid tumors</article-title>. <source>Adv Ther.</source> (<year>2019</year>) <volume>36</volume>:<fpage>2638</fpage>&#x02013;<lpage>78</lpage>. <pub-id pub-id-type="doi">10.1007/s12325-019-01051-z</pub-id><pub-id pub-id-type="pmid">31410780</pub-id></citation></ref>
<ref id="B122">
<label>122.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Siegel</surname> <given-names>RL</given-names></name> <name><surname>Miller</surname> <given-names>KD</given-names></name> <name><surname>Jemal</surname> <given-names>A</given-names></name></person-group>. <article-title>Cancer statistics, 2020</article-title>. <source>CA Cancer J Clin.</source> (<year>2020</year>) <volume>70</volume>:<fpage>7</fpage>&#x02013;<lpage>30</lpage>. <pub-id pub-id-type="doi">10.3322/caac.21590</pub-id><pub-id pub-id-type="pmid">31912902</pub-id></citation></ref>
<ref id="B123">
<label>123.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Grossman</surname> <given-names>D</given-names></name> <name><surname>Kim</surname> <given-names>CC</given-names></name> <name><surname>Hartman</surname> <given-names>RI</given-names></name> <name><surname>Berry</surname> <given-names>E</given-names></name> <name><surname>Nelson</surname> <given-names>KC</given-names></name> <name><surname>Okwundu</surname> <given-names>N</given-names></name> <etal/></person-group>. <article-title>Prognostic gene expression profiling in melanoma: necessary steps to incorporate into clinical practice</article-title>. <source>Melanoma Manag.</source> (<year>2019</year>) <volume>6</volume>:<fpage>MMT32</fpage>. <pub-id pub-id-type="doi">10.2217/mmt-2019-0016</pub-id><pub-id pub-id-type="pmid">31871621</pub-id></citation></ref>
<ref id="B124">
<label>124.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Plebani</surname> <given-names>M</given-names></name></person-group>. <article-title>Exploring the iceberg of errors in laboratory medicine</article-title>. <source>Clin Chim Acta.</source> (<year>2009</year>) <volume>404</volume>:<fpage>16</fpage>&#x02013;<lpage>23</lpage>. <pub-id pub-id-type="doi">10.1016/j.cca.2009.03.022</pub-id><pub-id pub-id-type="pmid">19302995</pub-id></citation></ref>
<ref id="B125">
<label>125.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Houben</surname> <given-names>PHH</given-names></name> <name><surname>Winkens</surname> <given-names>RAG</given-names></name> <name><surname>Van Der Weijden</surname> <given-names>T</given-names></name> <name><surname>Vossen</surname> <given-names>RCRM</given-names></name> <name><surname>Naus</surname> <given-names>AJM</given-names></name> <name><surname>Grol</surname> <given-names>RPTM</given-names></name></person-group>. <article-title>Reasons for ordering laboratory tests and relationship with frequency of abnormal results</article-title>. <source>Scand J Prim Health Care.</source> (<year>2010</year>) <volume>28</volume>:<fpage>18</fpage>&#x02013;<lpage>23</lpage>. <pub-id pub-id-type="doi">10.3109/02813430903281758</pub-id><pub-id pub-id-type="pmid">20121652</pub-id></citation></ref>
<ref id="B126">
<label>126.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fang</surname> <given-names>C</given-names></name> <name><surname>Otero</surname> <given-names>HJ</given-names></name> <name><surname>Greenberg</surname> <given-names>D</given-names></name> <name><surname>Neumann</surname> <given-names>PJ</given-names></name></person-group>. <article-title>Cost-utility analyses of diagnostic laboratory tests: a systematic review</article-title>. <source>Value Heal.</source> (<year>2011</year>) <volume>14</volume>:<fpage>1010</fpage>&#x02013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.1016/j.jval.2011.05.044</pub-id><pub-id pub-id-type="pmid">22152169</pub-id></citation></ref>
<ref id="B127">
<label>127.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kelly</surname> <given-names>SG</given-names></name> <name><surname>Yarrington</surname> <given-names>M</given-names></name> <name><surname>Zembower</surname> <given-names>TR</given-names></name> <name><surname>Sutton</surname> <given-names>SH</given-names></name> <name><surname>Silkaitis</surname> <given-names>C</given-names></name> <name><surname>Postelnick</surname> <given-names>M</given-names></name> <etal/></person-group>. <article-title>Inappropriate <italic>Clostridium difficile</italic> testing and consequent overtreatment and inaccurate publicly reported metrics</article-title>. <source>Infect Control Hosp Epidemiol.</source> (<year>2016</year>) <volume>37</volume>:<fpage>1395</fpage>&#x02013;<lpage>400</lpage>. <pub-id pub-id-type="doi">10.1017/ice.2016.210</pub-id><pub-id pub-id-type="pmid">27666285</pub-id></citation></ref>
</ref-list>
<fn-group>
<fn fn-type="financial-disclosure"><p><bold>Funding.</bold> This work was supported by the P30 Cancer Center Support Grant (P30CA040214) to the University of Utah and Huntsman Cancer Institute.</p>
</fn>
</fn-group>
</back>
</article> 