Edited by: Wai Lim, University of Western Australia, Australia
Reviewed by: Josefina M. Alberu, Tecnológico de Monterrey, Mexico; Noble Johan, Centre Hospitalier Universitaire de Grenoble, France; Cécile Couchoud, Agence de la Biomédecine, France
*Correspondence: S. Joseph Kim,
This article was submitted to Alloimmunity and Transplantation, a section of the journal Frontiers in Immunology
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Clinical research is about asking and answering questions. Before solutions relevant to clinical problems can be sought, clinicians must frame questions in ways that are answerable using the methods of clinical research. Different types of questions are best answered using specific study designs. Each design has inherent strengths and limitations. In this review article, we provide an approach to asking answerable clinical research questions, review the major study designs, describe their strengths and weaknesses, and link the study designs to their intended purposes.
Clinical transplantation has developed in parallel with many companion sciences and has been the initial testing ground for numerous novel surgical techniques, medications, and clinical practice. This close association with medical advancement has required an almost continuous relationship between clinical transplantation and clinical research. In order to generate appropriate data summaries and inferences about any population, it is necessary to utilize the appropriate methods (
Before solutions relevant to clinical problems can be sought, one must frame questions in ways that are answerable, using the methods of clinical research. A well formulated question is the half the battle. Thereafter, an approach to answering the question has to be formulated. Different types of questions are best answered using specific study designs. Each design has its inherent strengths and limitations for addressing the objectives of the study. However, recognizing different study designs and choosing the most appropriate one for a given question is not always straightforward.
The purpose of this article is to outline an approach to asking answerable clinical research questions, to review the major study designs, to describe the strengths and limitations of each design, and to link the study designs to their intended purposes. Reviewing specific statistical approaches to analyzing clinical research studies is beyond the scope of this article but we have provided several excellent resources to which the reader can refer (
The first step in discovering new insights on an issue relevant to patient care is framing the question so that it is answerable using the methods of clinical research (
Patient, population, or problem (P): Which characteristics, like the target clinical condition, ethnicity, and age group, define the patients or population?
Intervention (I): Which intervention or exposure (e.g., form of treatment, diagnostic test, or educational program) is being applied to the patient/population/problem?
Comparator or control (C): Is there an alternative to the main intervention, for example, treatment with placebo or the standard of care? This category is only applicable to studies with a comparator group (i.e., analytical studies).
Outcomes or effects (O): Which outcomes or effects relating to the intervention or exposure (e.g., mortality, morbidity, quality of life, cost-effectiveness) are being studied?
Framing the clinical question determines the question type (etiology, diagnosis, prognosis, intervention), which then determines the most appropriate study design to answer the question (
Appropriate study design for addressing different clinical questions.
| Question type | Example | Best study design | Other possible designs |
|---|---|---|---|
| Drug vs. control |
Randomized controlled trial | Cohort study |
|
| Test A vs. reference test |
Cross-sectional study | – | |
| Exposure present vs. absent | Cohort study | Case-control study | |
| Exposure present vs. absent | Cohort study | – | |
| Period prevalence* | Cohort study | – | |
| Point prevalence** | Cross-sectional | – | |
| Incidence | Cohort | – |
Adapted from Cross NB, Craig JC, Webster AC. Asking the right question and finding the right answers. Nephrology (Carlton). 2010 Feb;15(1):8-11. doi: 10.1111/j.1440-1797.2009.01264.x. PMID: 20377764.
*Period prevalence refers to prevalence measured over an interval of time.
**Point prevalence refers to the prevalence measured at a particular point in time. It is the proportion of persons with a particular disease or attribute on a particular date.
Along with clarifying the objectives of a research project, the PICO method directly supports electronic search strategies on platforms such as PubMed. An extension of PICO adds a “T” for time frame and “S” for setting (i.e., PICOTS). The latter version may be particularly useful for observational studies that use existing data sources. Alternative approaches to framing research questions exist and may be appropriate for specific circumstances (e.g., ECLIPSE for qualitative research) (
As in biology, anatomy dictates physiology. The anatomy of a study determines what it can and cannot do. Biology has animal and plant kingdoms. Similarly, clinical research has two large kingdoms: experimental and observational research.1 A useful classification system for the different types of clinical research study designs is depicted in
Adapted from
Once a study has been deemed experimental in design, the next step is to decide on the mechanism by which the intervention or exposure will be allocated (
The structure of different study designs.
The main outcome of randomization is that it reduces the likelihood that prognostic characteristics of the study patients will be unequally distributed between the intervention and comparator arms. As the sample size increases, the probability that important factors will be imbalanced across treatment groups will further decrease. Moreover, this uncoupling of the link between treatment allocation and patient prognosis ensures that, both
RCTs are the ideal study type to investigate the benefit or harm of an intervention, such as a drug therapy (
Although RCTs are powerful tools, they also have some weaknesses. RCTs are not a panacea (
Numerous important health exposures simply cannot be randomized, either for practical or ethical reasons. For instance, exposure to radiation and cigarette smoke cannot be randomized. Ethical objections may prevent interventions to be tested within an RCT setting when a well-accepted best practice is compared with treatment with an unknown or potentially less favorable outcome (
The problem of generalizability has practical implications the design and interpretation of RCTs. Unlike the observational study, the RCT often includes only volunteers who pass through a screening process before inclusion. Those who volunteer for trials tend to be different from those who do not; for example, their health might be better. In addition, it is well known that people may act differently when they are being observed (i.e., Hawthorne effect). The results from a closely monitored trial population may not accurately reflect what will happen when an intervention is moved into a general population. To mitigate this problem, knowledge of the level of exposure assigned to each group should be withheld from subjects and their providers (they are “blinded”), when possible (
Finally, RCTs are generally more expensive to conduct than observational studies (
Experimental studies: Randomized Controlled Trial. Rabbit-ATG or Basiliximab Induction for Rapid Steroid Withdrawal After Renal Transplantation (Harmony): An Open-label, Multicentre, Randomized Controlled Trial.
Observational studies on the effects of therapy are usually the first step to generate and test hypotheses, but they may be more prone to biases than RCTs such that causal inferences must be made carefully. However, many observational studies do not focus on the effects of therapy but rather seek to answer research questions about etiology, diagnosis, prognosis, or adverse effects, areas where RCTs may be inappropriate or even impossible. The following the main types of observational study designs and a schema to classify them relative to each other is depicted in
An observational study with no comparison group, for example patient(s) exposed to a novel treatment, is considered a
Case reports and case series can provide the basis for more rigorous, hypothesis-driven, analytical studies to examine the mechanism of disease, effect of a novel intervention, or the emergence of a new/rare adverse event from exposure to a specific drug or risk factor. Other potential roles of case reports and case series include the recognition and description of new diseases, medical education, and highlighting rare manifestations of common conditions (
An important caveat of descriptive studies is that it does not allow for assessments of associations between treatment/exposure and disease since there is no comparison group. Only comparative studies (both experimental and analytical observational) enable assessments of possible causal relationships (
A descriptive study that has led to the design of more definitive analytical studies is the case report by Locke et al. (
Observational studies: Case Report. The Use of Antibody to Complement Protein C5 for Salvage Treatment of Severe Antibody-Mediated Rejection.
There are three major types of analytical observational study designs in clinical research: cohort, case-control, and cross-sectional studies (
In a cohort study, patients from a population are recruited into the study using clearly defined and prespecified inclusion/exclusion criteria. At the time of recruitment, an exposure or risk factor, such as delayed graft function or current smoking, is measured in each patient and the development of the outcome or disease in the “exposed” group is examined against a comparable group of patients who are “unexposed” (
Since all patients entering a cohort study are free of the outcome or disease of interest at the time of study recruitment, the
Cohorts of patients may be assembled in the present and followed into the future for the event(s) of interest. This is known as a
Retrospective cohort studies are often assumed to be inherently inferior to prospective cohort studies in terms of study validity. This is not necessarily true. The key issue is the quality and breadth of data collection and adherence to good study design principles. An existing dataset may not have been meant to answer a specific scientific question, but if the relevant data have been comprehensively captured, the rigor and quality of a retrospective cohort study can rival any prospective cohort study (
Cohort studies are the best way to ascertain both the incidence and natural history of a disorder. Cohort studies are analytical studies that have the potential to provide answers to research questions on interventions, etiology (e.g., smoking, alcohol, or genetic factors), diagnosis, and prognosis. They are useful in the investigation of multiple outcomes that might arise after a single exposure. A prototype would be cigarette smoking (the exposure) and stroke, emphysema, oral cancer, and heart disease (the outcomes). The cohort design is also useful in the study of rare exposures, such as the health effects of ionizing radiation or chemicals in the workplace (
Cohort studies can reduce the risk of survivor bias if study entry criteria are equally applied across exposure groups (
The strengths of cohort studies further lie in their potentially larger sample sizes, as a result of their lower cost compared to RCTs and in their broader patient populations resulting in a higher generalizability of their results. Recent work has highlighted design features in cohort studies that may allow for the emulation of a target trial (
Cohort studies have important limitations too. Selection of certain patient population is built into cohort studies. For example, in a cohort study investigating effects of jogging on cardiovascular disease, those who choose to jog probably differ in other important ways (such as diet and smoking) from those who do not exercise (
Cohort studies can be inefficient because it may take a long time before an outcome occurs and, as a result, these studies can be expensive (
Observational studies: Cohort. Pre-transplant HLA Antibodies and Delayed Graft Function in the Current Era of Kidney Transplantation.
Case-control studies are less common in the transplant literature but are widely used in other areas such as genetic epidemiology. Case-control studies initially assemble patients based on their outcome (i.e., diseased) and then group of patients who did not have the outcome (i.e., non-diseased) are sampled from the population of interest. Subsequently, the exposure status of diseased patients at some time point in the past is ascertained and compared to the exposure status among non-diseased patients. Inherently, case-control studies are retrospective in the sense that both the exposure and outcome have already occurred by the time the study is conceived. Note that the case-control study is still an analytical observational study since the investigator did not assign the exposure or risk factor and there are at least two comparison groups (
Five main notions guide investigators who do, or readers who assess, case-control studies. First, investigators must explicitly define the criteria for ascertainment of a case and any eligibility criteria used for selection. Second, controls should come from the same population as the cases, and their selection should be independent of the exposures of interest. Third, investigators should blind the data gatherers to the case or control status of participants or, if not possible, at least blind them to the main hypothesis of the study. Fourth, data gatherers need to be thoroughly trained to elicit or collect data on exposures in a similar manner from cases and controls. Finally, investigators should address the potential for confounding bias at both the design and analysis stages (
The case-control study is the most efficient design for evaluating rare diseases or outcomes that take many years to develop since it takes advantage of existing datasets with exposures and outcomes that have already been captured (
Although easier to do, they are also easier to do wrong. The Achilles heel of case-control studies is choosing an appropriate control group. Control patients should be individuals who are eligible to develop the disease in question but were disease-free at the time of control selection. This pool of patients is also known as the
Observational studies: Case Control. HLA-DR and -DQ Eplet Mismatches and Transplant Glomerulopathy: A Nested Case–Control Study.
In cross-sectional studies, the exposure and outcome is usually assessed at the same time. This hampers the interpretation of associations because the temporality of exposure and outcome may be uncertain. Therefore, this study design can draw limited causal inferences about relationships between exposure and outcome. Having said this, since patients can be categorized into exposure groups, and the proportion of patients with the outcome of interest can be calculated among exposed and unexposed patients, this design qualifies as an analytical observational study.
Cross-sectional studies can be thought of as providing a ‘snapshot’ of the frequency and characteristics of an outcome at a particular point in time. In general, cross-sectional studies are prevalence studies (number of cases existing per 1000 population) but cannot describe incidence (number of new cases per unit time). As a result, their findings can be used to estimate the burden of diseases in populations for the planning of health services delivery (
Cross-sectional studies can be performed quickly, since no follow-up is necessary, and at little expense. It can also provide clues to scientifically interesting associations that may be later confirmed in cohort studies or an RCT. As a result, cross-sectional analyses have been typically referred to as
The major weakness of cross-sectional studies is that the temporal relation between exposure and outcome may not be clearly delineated and thus associations derived from these studies may be susceptible to
Cross-sectional analyses of baseline characteristics are commonly performed in clinical trial populations.
Observational studies: Cross Sectional. Baseline Characteristics of Participants in the Folic Acid for Vascular Outcome Reduction in Transplantation (FAVORIT) Trial.
Meta-analysis is a statistical method that quantitatively combines results from different studies to come up with a pooled estimate of the treatment or exposure effect. It is typically conducted on clinical trials that address the same intervention but can also be used to pool other types of data, such as studies on diagnostic accuracy (e.g., pooled estimates on sensitivity and specificity) and epidemiologic studies (e.g., pooled incidence or prevalence rates) (
Classifying studies by their design characteristics is helpful in highlighting the methodologic features that each one possesses to support causal inferences, while outlining features that may influence their susceptibility to bias. However, sometimes the problems that clinicians face make the linkage between a given question and the appropriate study design less than obvious. Herein, several clinical scenarios are described that lead to specific research questions. The types of study designs that most appropriately address the questions are also provided.
Ms. Gonzalez has end-stage secondary to lupus nephritis. She received a kidney transplant 28 days ago and has developed antibody-mediated rejection treated with plasmapheresis, intravenous immunoglobulin (IVIg) and corticosteroids (i.e., standard of care). Her father has read that eculizumab is a drug that potentially may help her daughter. However, the use of this medication might be associated with severe infections and is rather expensive.
The patient’s physician asks the following question: In kidney transplant recipients, does the addition of eculizumab (vs. no eculizumab), in patients treated with plasmapheresis, IVIg, and corticosteroids for antibody-mediated rejection, improve kidney transplant outcomes? Given that this is classic question of treatment efficacy, the optimal study design would be an RCT. However, a multi-center cohort study may also be pursued if eculizumab has been variably used across different transplant centers for the indication of antibody-mediated rejection, along with a background of standard of care for all patients. Theoretically, a case-control study could also be conducted, although they are used less commonly for questions of treatment efficacy.
Cardiovascular mortality is the leading cause of death in patients with end-stage kidney disease. In addition to traditional risk factors, proinflammatory cytokines, C-reactive protein, and T cell-mediated immunity have been shown to relate to cardiovascular morbidity and mortality in dialysis patients. A transplant nephrologist has two patients on hemodialysis waiting for a kidney transplant, with the same age, race, and dialysis vintage. Neither patient has diabetes, prior transplants, nor autoimmune diseases, but the first patient has cPRA of 0%, while the second patient has cPRA of 98%.
The patient’s physician asks the following question: Among end-stage kidney disease patients on hemodialysis awaiting a kidney transplant, does a higher level of cPRA (vs. a lower level) independently predict the future risk of cardiovascular mortality? This is a question of disease etiology. Since cPRA cannot be randomized, an RCT would not be appropriate. Cardiovascular mortality occurs over follow-up. As a result, the cohort design is most suitable for answering this question. A case-control study may be preferred over a cohort study if additional data collection was necessary (e.g., abstracting cPRA data from HLA reports), since data collection would only have to performed for cases and a sample of controls.
Fibrillary glomerulonephritis (FGN) is a rare primary glomerular disease. Histologic and histochemical features of FGN overlap with those of other glomerular diseases and no unique histologic biomarkers for diagnosing FGN have been identified. A novel FGN-specific diagnostic marker called DNAJB9 has been touted as a potentially useful clinical tool to diagnose FGN. Mr. Jones originally kidney disease is unknown but has now developed a glomerular disease in his kidney transplant. The histologic features are suggestive of FGN but there remains some diagnostic uncertainty.
The patient’s physician asks the following question: In kidney transplant recipients with suspected FGN, what are the operating characteristics (i.e., sensitivity, specificity, predictive value) of DNAJB9 in diagnosing FGN? Diagnostic studies are cross-sectional since the biomarker being evaluated is applied to a group of patients, some with and some without disease (as per the gold standard test of disease).
Mr. Smith developed end-stage kidney disease secondary to autosomal dominant polycystic kidney disease and is undergoing a living donor kidney transplant next week. He has an impaired glucose tolerance test and has been informed that he may develop post-transplant diabetes. He would like to know the probability of developing posttransplant diabetes over the first post-transplant year, given his age, oral glucose tolerance test result, and other characteristics.
The patient’s physician asks the following question: In patients with a history of autosomal dominant polycystic kidney disease who undergo a kidney transplant, what is the likelihood of being diagnosed with post-transplant diabetes in the first year after kidney transplantation, as a function of their glucose tolerance and other baseline factors? Studies of prognosis are typically evaluating outcomes that occur at some follow-up time after patients come under risk for the outcome in question. As a result, all prognosis studies require cohort designs.
Frailty, a measure of physiologic reserve, is associated with poor outcomes and mortality among kidney transplant candidates and recipients. The director of a kidney transplant program would like to offer all her patients the best and most comprehensive care possible. When she requests for the allocation of resources to rehabilitate frail patients, she is asked how often this problem occurs in her transplant center.
The physician asks the following question: Among kidney transplant candidates and recipients followed at our center, how frequently are patients diagnosed with frailty? Depending on its intended uses, the best measure of disease frequency can either be prevalence or incidence. The former may be estimated cross-sectionally in a group of kidney transplant candidates and recipients at one point in time. However, if the director would like to understand the future needs of patients being followed in her program, a measure of incidence (i.e., new cases of frailty) would be most appropriate. The latter can only be estimated from a cohort study.
The quality of a clinical study depends on internal and external factors. Studies have internal validity when, random error apart, reported differences between exposed and unexposed individuals can be attributed only to the exposure under investigation. Internal validity can be affected by two types of error: random error and systematic error. Random error depends on chance and can be minimized by increasing the sample size. Systematic errors are flaws in study design and/or analysis that can lead to an over- or under-estimation of the association of interest. This type of error is independent of sample size. Systematic error is also known as bias (
Selection, information, and confounding biases are the three major forms of bias in clinical research. Although these biases are typically discussed in the context of observational studies, similar problems can arise in experimental studies. Selection bias stems from an absence of comparability between groups being studied. Information bias results from errors in the measurement of exposure, outcome, and/or confounders. Confounding is a mixing or confusion of effects; a researcher attempts to relate an exposure to an outcome but actually measures (at least in part) the effect of a third factor, i.e., the confounding variable. Bias can be prevented at two levels: (1) by choosing the appropriate study design for addressing the study hypothesis and (2) by carefully establishing the procedures of data handling and the definitions of exposures and outcomes.
How to look for bias in observational studies.
Were the same criteria used for study entry among groups defined by exposure? Were all patients accounted for over follow-up? Was one exposure group more likely to have losses to follow-up that were driven by factors impacting the likelihood of developing the outcome? |
Were controls selected independent of their exposure status? Were controls sampled from the underlying cohort from which cases were derived? |
In recent years, new observational study design methods, that take advantage of the evolving causal inference literature (
Clinical research can be divided into experimental and observational; observational studies are further categorized into those with and without a comparison group. Only studies with comparison groups allow investigators to assess possible causal associations. The RCT remains the gold standard study design to evaluate the effects of therapies. In contrast, studies of etiology, diagnosis, prognosis, or disease burden rely heavily on observational designs. Usually, results from observational studies are needed to generate hypothesis that can subsequently be tested within an RCT. They can also be used to corroborate the results of RCTs in real-world settings and ascertain long-term outcomes that could not be observed over the usual follow-up duration of an RCT. Both observational studies and RCTs fulfill a complementary and valuable role in transplant.
SR-R wrote the first draft of manuscript. SR-R and SK performed the literature review. SR-R designed the figures and tables. SK revised the manuscript. All authors contributed to the article and approved the submitted version.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The handling Editor [WL] declared a past collaboration with the author [SK].
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.