In 2013, the NIH Common Fund launched the Extracellular RNA Communication Consortium to stimulate research into the fundamental biology of exRNA and its clinical applications in disease diagnosis and treatment. One product of the first stage, ERCC1 (2013–2018), is the exRNA Atlas, a database of small RNA-sequencing and RT-qPCR data. To date, the Atlas holds over 7,700 samples from 14 biofluids and 16 disease conditions (Subramanian et al., 2015; Murillo et al., 2019). The current stage, ERCC2, focuses on development of technologies to characterize exRNA carriers and to isolate and characterize the contents of individual extracellular vesicles (EVs). A key strength of the exRNA Atlas is that the small RNA-seq datasets are uniformly processed by the extracellular RNA processing toolkit (exceRpt) (Rozowsky et al., 2019). A hard-learned lesson from ERCC1 was the difficulty of eradicating systematic biases from the data, which makes it difficult to compare exRNA profiles across conditions. The April 2021 online workshop was held to address these and other problems in exRNA data analysis (Figure 1).

Rob Kitchen opened the workshop by outlining open problems in exRNA data analysis. One key challenge is that exRNA data quality varies widely and systematically between different experimental methods. Isolating and purifying exRNA and extracellular vesicles (EVs) from experimental samples is itself difficult, and RNA isolation kits used for these tasks are known to be a major source of variability in the resulting exRNA data. Compensating for this variation is a central challenge, as each kit and RNA sequencing method has different sequence biases that must accounted for when doing larger analyses (Murillo et al., 2019; Srinivasan et al., 2019). Dr. Kitchen stressed that comparison of the relative amounts of exRNAs across samples should be attempted in samples prepared using identical methods wherever possible. Even then, large sample-to-sample variation in exRNA carrier abundance persists, obscuring biological signal in case-control studies (Murillo et al., 2019).

Another central question of interest in exRNA and EV biology is determining tissue and cell type of origin of different clusters of exRNAs in a biofluid. Dr. Kitchen argues that it might be possible in peripheral biofluids like urine and saliva, but more challenging in circulating blood serum and plasma, where the exRNA complement may be too diverse to parse. For vesicular exRNAs, the problem should be made simpler by improvements in experimental techniques to isolate EV sub-fractions, such as selecting for EVs with cell-type-specific surface proteins. As fractionation techniques improve, however, it will be necessary to compensate for variable enrichment efficiency.

Juan Pablo Tosar focused specifically on the quality of non-coding RNA annotations. He outlined how the mechanisms of biogenesis of miRNA and piRNA serve as the basis of existing annotations like miRbase. The problem is that such databases often lack strict curation, resulting in many sequences that are not miRNAs or piRNAs under any reasonable definition. Tosar described two examples from miRbase annotations, showing that miR-1202 is, in fact the small nucleolar RNA, SNORD126, and miR-1246, a microRNA enriched in EVs, is likely a contaminant from fetal bovine serum (FBS) in the cell culture media and likely fragment of the small nuclear RNA RNU2-1 (Sakha et al., 2016; Tosar et al., 2017). This is not a problem of miRbase itself, which is designed as a community-driven repository of putative miRNA sequences, providing minimal quality control at the point of submission (Kozomara et al., 2019). His proposed solution to the problem of mis-annotation is to use a curated miRNA database like MirGeneDB (Fromm et al., 2015; Fromm et al., 2020), resulting in a smaller number of higher quality miRNA calls.

Tosar emphasized that piRNAs have a complex biogenesis that imposes a strong bias to start with U or to have A in the 10th position, and they are expressed from genomic clusters with a high density of piRNA sequences (Czech et al., 2018). PiRNA are mostly expressed in gonads and early embryos where their main role is to dampen the expression of transposable elements. However, existing piRNA databases include a very small number (<1%) of contaminating sequences that do not match these criteria and have 100% overlap with other ncRNA families (Tosar et al., 2018a). Findings of piRNA expression in cancers and biofluids are very often highly enriched in sequences from that set of false positive contaminants (Tosar et al., 2018a). For example, the level of piR-54265 in the serum of colorectal cancer patients has been found to be predictive of tumor relapse after surgery. Its value as a biomarker notwithstanding, piR-54265 is, in fact, a mis-annotated full-length snoRNA, SNORD57 (Tosar et al., 2021). Going back to the cell type of origin problem described above, while piRNA expression might be cancer-specific, snoRNAs are ubiquitously expressed. Consequently, annotation accuracy affects biological interpretation of the data. The final message is that it is important to realize our bioinformatics is only as strong as our RNA annotations.

In a session on exRNA data sources, Matt Roth gave an overview of the ERCC’s exRNA Atlas (Subramanian et al., 2015; Murillo et al., 2019), a curated catalog of exRNA sequencing and qPCR data generated from a wide array of biofluids and disease states. Roth outlined features of the exRNA Atlas that facilitate accessing, querying, interpreting, and reusing experimental data and sample metadata. He also described ongoing efforts to expand Atlas content to include data and metadata from additional exRNA technologies being developed as part of ERCC2, and to integrate exRNA Atlas data into the NIH Common Fund Data Ecosystem (https://app.nih-cfde.org/). Justin Chang gave a preview of the exRNA Explorer tool, a data exploration and visualization tool that will soon be integrated into the public exRNA Atlas. Joel Rozowsky later outlined the exceRpt pipeline used to process short exRNA sequencing data in the exRNA Atlas (Rozowsky et al., 2019).

Pieter Mestdagh presented the Human Biofluid RNA Atlas (Hulstaert et al., 2020), which characterizes and compares exRNA transcriptome profiles in a wide variety of biological fluids (n = 20) using both small RNA-sequencing and mRNA-capture sequencing. Introducing short and long synthetic spike-in RNAs before and after RNA extraction, enabled comparison of the absolute miRNA and mRNA content between samples, revealing large differences between fluids. Mestdagh also summarized efforts to identify the relative contribution of different tissues to exRNA transcriptomes and how this varied across biofluids. Deconstruction of exRNA profiles into contributing tissues can be achieved through computational deconvolution, a topic explored in-depth later in the workshop. He showed that the accuracy of deconvolution depends on several factors, including 1) proper transformation and normalization of the exRNA-seq read count, 2) choice of deconvolution algorithm and 3) quality and completeness of the reference data. Mestdagh also presented evidence suggesting that circular RNAs (circRNAs) are present in biofluids, potentially at a higher proportional abundance relative to linear transcripts than they are in cells and tissues.

Klaas Max discussed healthy reference profiles of extracellular miRNA in serum and plasma (Max et al., 2018). Based on an initial cohort of 13 individuals and a second larger cohort of over 200 individuals, they found very little variation between males and females, but more noticeable differences between serum and plasma samples. Interestingly, they found that the exRNA profiles of pregnant women were distinct from non-pregnant women, and that the variant miRNAs could predict not only whether a woman were pregnant, but in which stage of pregnancy they were in. Among the most variable miRNAs was cluster-miR-498(46), which was upregulated in pregnant versus non-pregnant women, increasing between 50- and 250-fold from first to third trimester. Max noted that many of the most variable miRNAs in healthy subjects were cell-lineage-specific miRNAs of the liver, neuroendocrine organs, adrenal glands, epithelial cells and muscle. Abundance of several such miRNAs sharing a common origin were moderately correlated. Although they identified additional sets of variably expressed miRNAs, Max noted that few miRNAs are known to be cell-lineage-specific, which complicates methods to deconvolute and identify tissues of origin. Plasma subfractionation by ultracentrifugation to enrich for non-hematopoietic miRNAs did not result in a strong enrichment of organ- or cell-type-specific miRNAs.

Extracellular RNA that circulates in biofluids must be shielded from the harsh environment, particularly from enzymes that digest RNA (RNases). Some exRNAs are resistant to RNase digestion, e.g., Gly/Glu tRNA fragments that can form stable homo- and hetero-dimers (Tosar et al., 2018b). Other exRNAs are protected inside vesicles or by association with RNA-binding proteins (RBPs). Recent work shows that some cell-surface exRNAs are protected by glycosylation (Flynn et al., 2021). Vesicular exRNAs are the best studied class of exRNAs. RBP-associated exRNAs have been difficult to study because of the delicate protein biochemistry required to isolate and characterize RNA binding sites for each of the hundreds of RBPs in the human genome (Gerstberger et al., 2014). Eric Van Nostrand outlined resources from the Encyclopedia of RNA Elements (ENCORE) to aid in this effort, including validated antibodies and shRNA reagents (Sundararaman et al., 2016). ENCORE experiments systematically characterized aspects of RBP regulation including RBP in vitro motifs and RNA interactions in K562 and HepG2 cell lines for over 350 RNA binding proteins (Van Nostrand et al., 2020).

Emily LaPlante described initial work scanning the exRNA Atlas for ENCORE RNA binding sites and establishing the infrastructure to allow Atlas users to study their own regions of interest. Bogdan Mateescu outlined current knowledge about exRNA-associated RBPs (exRBPs) (Fabbiano et al., 2020) as well as experimental challenges to discovering new ones. Then he laid out the ERCC2 PRISM (Purification of exRNA by Immuno-capture and Sorting using Microfluidics) project designed to meet those challenges. The exRBP-HIT bioinformatic pipeline identifies candidate exRBPs by overlapping peak calls from exRNA-seq and RBP eCLIP (enhanced CrossLinking and ImmunoPrecipitation) experiments. Permutation analysis of the data yields a ranked list of the most likely candidates. Initial analysis of data from plasma, saliva, and urine samples from the Van Keuren Jensen lab reassuringly identifies exRBPs known to be associated with particular classes of exRNA, for example RNA silencing factors with miRNA and Ro60 with YRNA. More interesting were new candidate exRBPs targeting extracellular mRNA fragments.

The next step in the PRISM group’s strategy is experimental validation of candidate exRBPs in a model system. After using CRISPR to create knockout strains in the 293T cell line for each candidate exRBP gene, exRNA profiles from wild-type and knockout conditioned media are compared. A case study including knockout lines for 10 genes in the RNA silencing pathway led to changes in expression in classes of exRNA beyond miRNA: tRNA, snRNA, snoRNA, and YRNA. Whereas the bioinformatic pipeline identifies individual sites of exRNA interaction with RBPs, the model system provides a global perspective on the impact of each RBP on expression of all exRNAs and perhaps gives insight into exRBP function. The ERCC2 PRISM group plans to identify over 100 candidate exRBPs and create an atlas of exRNA profiles from the model system exRBP knockout strains.

Two major open problems in the field are identifying tissue of origin of the exRNAs in a biofluid and associating them with their molecular carrier, whether that be an RNA binding protein, a lipid like HDL or LDL, or a variety of classes of extracellular vesicle. Computational deconvolution, a method for partitioning a heterogeneous dataset into contributions from different independent constituents, can complement experimental approaches to addressing such challenges. There are two broad classes of deconvolution algorithms—reference-based and reference-free. Reference-based algorithms require a known signature matrix of gene expression profiles from all the cell types or molecular carriers to be separated. Reference-free methods estimate simultaneously both the signature matrix and the relative ratios of each cell type or carrier in the mixture.

In a session on deconvolution, Brian White described a Tumor Deconvolution DREAM Challenge (White et al., 2019) to assess existing and inspire novel methods for deconvolving bulk RNA expression data. DREAM (Dialogue on Reverse Engineering Assessment and Methods) Challenges use crowd-sourcing to address fundamental questions in biomedical research. Challenges are posed by domain experts in concert with DREAM organizers, who are responsible for curating data, defining objective evaluation criteria, and engineering a computational framework for method submission and execution. Recently, Challenges have leveraged a “model-to-data” paradigm (Guinney and Saez-Rodriguez 2018) in which models are executed in the cloud, facilitating reproducible deployment of methods and ensuring that those methods are not overfit to data. The DREAM community includes over 30,000 cross-disciplinary participants who have contributed to more than 60 Challenges resulting in over 100 publications (http://DREAMchallenges.org/).

In the Tumor Deconvolution DREAM Challenge, the organizers provided teams RNA expression profiles from in vitro and in silico admixtures spanning 14 different immune, stromal, and cancer cell types and asked teams to predict the relative ratios of each cell type in the admixtures. To help assess its potential relevance for deconvolving exRNA data, Dr. White described CIBERSORTx, one of the baseline reference methods used in the Challenge. CIBERSORTx is based on the earlier Cell-type Identification By Estimating Relative Subsets Of RNA Transcripts (CIBERSORT) algorithm (Newman et al., 2015). Both CIBERSORT and CIBERSORTx are reference-based. Given an input matrix of reference gene expression signatures from all cell types expected to be present in a mixture, CIBERSORT uses support-vector regression (SVR) to estimate from bulk RNA sequencing data the ratios of each cell type in the mixture. CIBERSORTx (Newman et al., 2019) is a two-stage deconvolution algorithm, including a batch correction step to reduce technical variation across the single-cell RNA-seq datasets used to develop the signature matrix. Results, including those from CIBERSORTx, are summarized on the DREAM Challenge website (https://www.synapse.org/tumorDeconvolutionChallenge).

Rongshan Yu described the DAISM-DNN algorithm (Data Augmentation through In Silico Mixing and Deep Neural Network) used to win the Tumor Deconvolution DREAM Challenge. Dr. Yu explained that the choice of a neural network method was guided by the non-linearity of the input data. He showed that expression levels of different genes vary across cell types in a non-linear way. Neural networks perform better than linear regression in that case. The problem with using DNN is that it requires a very large number of training datasets, on the order of ten thousand. The team’s solution was to augment the existing training datasets by shuffling them together in silico with expression data from target cells, either bulk RNAseq from purified cell samples or single-cell RNAseq. Finally, although DNN models are generally considered to be difficult to interpret black boxes, it is possible to apply methods such as the SHapley Additive explanation (SHAP) model from game theory (Lundberg and Lee 2017) to output a ranked list of the genes that contribute most to the deconvolution results from DAISM-DNN (Lin et al., 2021).

Finally, Aleks Milosavljevic described the XDec algorithm for analyzing small RNA-seq datasets in the exRNA Atlas. XDec is a two-stage reference-free deconvolution algorithm that separates exRNAs in a biofluid sample into sets associated with different molecular carriers—extracellular vesicles, lipoproteins HDL and LDL, and three categories of RNA binding proteins (Murillo et al., 2019).

A major goal of the workshop was to lay the foundation for creating an exRNA-themed data analysis challenge. The DREAM challenge framework is ideal for presenting problems in data analysis to the wider scientific community. The last session of the workshop was chaired by Gustavo Stolovitzky, co-founder of the DREAM challenges. He gave a talk asking the question: “Can we use a crowdsourcing challenge to benchmark exRNA transcriptomics analyses?” Building on that foundation, Roger Alexander discussed specific classes of challenges that the community might put forth. The ensuing discussion and a post-workshop survey narrowed the field to two classes of challenge: first, a biomarker discovery challenge, and second, a deconvolution challenge to address two problems: identifying cell type of origin and molecular carrier of groups of exRNAs within a biofluid.

The workshop introduced experimental and data scientists to the field of exRNA data analysis. The standard for small RNA-seq data is uniform processing by the exceRpt pipeline and storage in the exRNA Atlas. Long extracellular RNA is less thoroughly studied, and methods for its analysis are still under development. It is important to be mindful that standard RNA-seq methods are blind to many RNA base modifications, and mis-annotated RNAs can lead to misinterpretation. Biomarker and other studies requiring comparison across datasets would benefit greatly from improved algorithms to compensate for batch-to-batch variation and other systematic errors.

With the discovery of new classes of exRNA (Flynn et al., 2021) and development of new types of exRNA-based disease therapies (Segel et al., 2021), now is an exciting time in the field of extracellular RNA research. To better our understanding of exRNA biology and improve our ability to use exRNA in the clinic, we wish to issue a call to action to the scientific community. We are seeking a validation dataset that will enable us to launch an exRNA biomarker discovery challenge. Specifically, we seek small exRNA sequencing data for several hundred cases and controls that can be shared prior to publication. The DREAM challenge “model to data” paradigm will ensure that the data remains private in a secure computing environment and is not shared with challenge participants (Ellrott et al., 2019).