experimental study randomized controlled trial (rct)

What is a Randomized Control Trial (RCT)?

Julia Simkus

Editor at Simply Psychology

BA (Hons) Psychology, Princeton University

Julia Simkus is a graduate of Princeton University with a Bachelor of Arts in Psychology. She is currently studying for a Master's Degree in Counseling for Mental Health and Wellness in September 2023. Julia's research has been published in peer reviewed journals.

Learn about our Editorial Process

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

A randomized control trial (RCT) is a type of study design that involves randomly assigning participants to either an experimental group or a control group to measure the effectiveness of an intervention or treatment.

Randomized Controlled Trials (RCTs) are considered the “gold standard” in medical and health research due to their rigorous design.

Control Group

A control group consists of participants who do not receive any treatment or intervention but a placebo or reference treatment. The control participants serve as a comparison group.

The control group is matched as closely as possible to the experimental group, including age, gender, social class, ethnicity, etc.

Because the participants are randomly assigned, the characteristics between the two groups should be balanced, enabling researchers to attribute any differences in outcome to the study intervention.

Since researchers can be confident that any differences between the control and treatment groups are due solely to the effects of the treatments, scientists view RCTs as the gold standard for clinical trials.

Random Allocation

Random allocation and random assignment are terms used interchangeably in the context of a randomized controlled trial (RCT).

Both refer to assigning participants to different groups in a study (such as a treatment group or a control group) in a way that is completely determined by chance.

The process of random assignment controls for confounding variables , ensuring differences between groups are due to chance alone.

Without randomization, researchers might consciously or subconsciously assign patients to a particular group for various reasons.

Several methods can be used for randomization in a Randomized Control Trial (RCT). Here are a few examples:

• Simple Randomization: This is the simplest method, like flipping a coin. Each participant has an equal chance of being assigned to any group. This can be achieved using random number tables, computerized random number generators, or drawing lots or envelopes.
• Block Randomization: In this method, participants are randomized within blocks, ensuring that each block has an equal number of participants in each group. This helps to balance the number of participants in each group at any given time during the study.
• Stratified Randomization: This method is used when researchers want to ensure that certain subgroups of participants are equally represented in each group. Participants are divided into strata, or subgroups, based on characteristics like age or disease severity, and then randomized within these strata.
• Cluster Randomization: In this method, groups of participants (like families or entire communities), rather than individuals, are randomized.
• Adaptive Randomization: In this method, the probability of being assigned to each group changes based on the participants already assigned to each group. For example, if more participants have been assigned to the control group, new participants will have a higher probability of being assigned to the experimental group.

Computer software can generate random numbers or sequences that can be used to assign participants to groups in a simple randomization process.

For more complex methods like block, stratified, or adaptive randomization, computer algorithms can be used to consider the additional parameters and ensure that participants are assigned to groups appropriately.

Using a computerized system can also help to maintain the integrity of the randomization process by preventing researchers from knowing in advance which group a participant will be assigned to (a principle known as allocation concealment). This can help to prevent selection bias and ensure the validity of the study results .

Allocation Concealment

Allocation concealment is a technique to ensure the random allocation process is truly random and unbiased.

RCTs use allocation concealment to decide which patients get the real medicine and which get a placebo (a fake medicine)

It involves keeping the sequence of group assignments (i.e., who gets assigned to the treatment group and who gets assigned to the control group next) hidden from the researchers before a participant has enrolled in the study.

This helps to prevent the researchers from consciously or unconsciously selecting certain participants for one group or the other based on their knowledge of which group is next in the sequence.

Allocation concealment ensures that the investigator does not know in advance which treatment the next person will get, thus maintaining the integrity of the randomization process.

Blinding (Masking)

Binding, or masking, refers to withholding information regarding the group assignments (who is in the treatment group and who is in the control group) from the participants, the researchers, or both during the study .

A blinded study prevents the participants from knowing about their treatment to avoid bias in the research. Any information that can influence the subjects is withheld until the completion of the research.

Blinding can be imposed on any participant in an experiment, including researchers, data collectors, evaluators, technicians, and data analysts.

Good blinding can eliminate experimental biases arising from the subjects’ expectations, observer bias, confirmation bias, researcher bias, observer’s effect on the participants, and other biases that may occur in a research test.

In a double-blind study , neither the participants nor the researchers know who is receiving the drug or the placebo. When a participant is enrolled, they are randomly assigned to one of the two groups. The medication they receive looks identical whether it’s the drug or the placebo.

Figure 1 . Evidence-based medicine pyramid. The levels of evidence are appropriately represented by a pyramid as each level, from bottom to top, reflects the quality of research designs (increasing) and quantity (decreasing) of each study design in the body of published literature. For example, randomized control trials are higher quality and more labor intensive to conduct, so there is a lower quantity published.

Resesearch Designs

The choice of design should be guided by the research question, the nature of the treatments or interventions being studied, practical considerations (like sample size and resources), and ethical considerations (such as ensuring all participants have access to potentially beneficial treatments).

The goal is to select a design that provides the most valid and reliable answers to your research questions while minimizing potential biases and confounds.

1. Between-participants randomized designs

Between-participant design involves randomly assigning participants to different treatment conditions. In its simplest form, it has two groups: an experimental group receiving the treatment and a control group.

With more than two levels, multiple treatment conditions are compared. The key feature is that each participant experiences only one condition.

This design allows for clear comparison between groups without worrying about order effects or carryover effects.

It’s particularly useful for treatments that have lasting impacts or when experiencing one condition might influence how participants respond to subsequent conditions.

A study testing a new antidepressant medication might randomly assign 100 participants to either receive the new drug or a placebo.

The researchers would then compare depression scores between the two groups after a specified treatment period to determine if the new medication is more effective than the placebo.

Use this design when:

• You want to compare the effects of different treatments or interventions
• Carryover effects are likely (e.g., learning effects or lasting physiological changes)
• The treatment effect is expected to be permanent
• You have a large enough sample size to ensure groups are equivalent through randomization

2. Factorial designs

Factorial designs investigate the effects of two or more independent variables simultaneously. They allow researchers to study both main effects of each variable and interaction effects between variables.

These can be between-participants (different groups for each combination of conditions), within-participants (all participants experience all conditions), or mixed (combining both approaches).

Factorial designs allow researchers to examine how different factors combine to influence outcomes, providing a more comprehensive understanding of complex phenomena.

They’re more efficient than running separate studies for each variable and can reveal important interactions that might be missed in simpler designs.

A study examining the effects of both exercise intensity (high vs. low) and diet type (high-protein vs. high-carb) on weight loss might use a 2×2 factorial design.

Participants would be randomly assigned to one of four groups: high-intensity exercise with high-protein diet, high-intensity exercise with high-carb diet, low-intensity exercise with high-protein diet, or low-intensity exercise with high-carb diet.

• You want to study the effects of multiple independent variables simultaneously
• You’re interested in potential interactions between variables
• You want to increase the efficiency of your study by testing multiple hypotheses at once

3. Cluster randomized designs

In cluster randomized trials, groups or “clusters” of participants are randomized to treatment conditions, rather than individuals.

This is often used when individual randomization is impractical or when the intervention is naturally applied at a group level.

It’s particularly useful in educational or community-based research where individual randomization might be disruptive or lead to treatment diffusion.

A study testing a new teaching method might randomize entire classrooms to either use the new method or continue with the standard curriculum.

The researchers would then compare student outcomes between the classrooms using the different methods, rather than randomizing individual students.

• You have a smaller sample size available
• Individual differences are likely to be large
• The effects of the treatment are temporary
• You can effectively control for order and carryover effects

4. Within-participants (repeated measures) designs

In these designs, each participant experiences all treatment conditions, serving as their own control.

Within-participants designs are more statistically powerful as they control for individual differences. They require fewer participants, making them more efficient.

However, they’re only appropriate when the treatment effects are temporary and when you can effectively counterbalance to control for order effects.

A study on the effects of caffeine on cognitive performance might have participants complete cognitive tests on three separate occasions: after consuming no caffeine, a low dose of caffeine, and a high dose of caffeine.

The order of these conditions would be counterbalanced across participants to control for order effects.

5. Crossover designs

Crossover designs are a specific type of within-participants design where participants receive different treatments in different time periods.

This allows each participant to serve as their own control and can be more efficient than between-participants designs.

Crossover designs combine the benefits of within-participants designs (increased power, control for individual differences) with the ability to compare different treatments.

They’re particularly useful in clinical trials where you want each participant to experience all treatments, but need to ensure that the effects of one treatment don’t carry over to the next.

A study comparing two different pain medications might have participants use one medication for a month, then switch to the other medication for another month after a washout period.

Pain levels would be measured during both treatment periods, allowing for within-participant comparisons of the two medications’ effectiveness.

• You want to compare the effects of different treatments within the same individuals
• The treatments have temporary effects with a known washout period
• You want to increase statistical power while using a smaller sample size
• You want to control for individual differences in response to treatment

Prevents bias

In randomized control trials, participants must be randomly assigned to either the intervention group or the control group, such that each individual has an equal chance of being placed in either group.

This is meant to prevent selection bias and allocation bias and achieve control over any confounding variables to provide an accurate comparison of the treatment being studied.

Because the distribution of characteristics of patients that could influence the outcome is randomly assigned between groups, any differences in outcome can be explained only by the treatment.

High statistical power

Because the participants are randomized and the characteristics between the two groups are balanced, researchers can assume that if there are significant differences in the primary outcome between the two groups, the differences are likely to be due to the intervention.

This warrants researchers to be confident that randomized control trials will have high statistical power compared to other types of study designs.

Since the focus of conducting a randomized control trial is eliminating bias, blinded RCTs can help minimize any unconscious information bias.

In a blinded RCT, the participants do not know which group they are assigned to or which intervention is received. This blinding procedure should also apply to researchers, health care professionals, assessors, and investigators when possible.

“Single-blind” refers to an RCT where participants do not know the details of the treatment, but the researchers do.

“ Double-blind ” refers to an RCT where both participants and data collectors are masked of the assigned treatment.

Limitations

Costly and timely.

Some interventions require years or even decades to evaluate, rendering them expensive and time-consuming.

It might take an extended period of time before researchers can identify a drug’s effects or discover significant results.

Requires large sample size

There must be enough participants in each group of a randomized control trial so researchers can detect any true differences or effects in outcomes between the groups.

Researchers cannot detect clinically important results if the sample size is too small.

Change in population over time

Because randomized control trials are longitudinal in nature, it is almost inevitable that some participants will not complete the study, whether due to death, migration, non-compliance, or loss of interest in the study.

This tendency is known as selective attrition and can threaten the statistical power of an experiment.

Randomized control trials are not always practical or ethical, and such limitations can prevent researchers from conducting their studies.

For example, a treatment could be too invasive, or administering a placebo instead of an actual drug during a trial for treating a serious illness could deny a participant’s normal course of treatment. Without ethical approval, a randomized control trial cannot proceed.

Fictitious Example

An example of an RCT would be a clinical trial comparing a drug’s effect or a new treatment on a select population.

The researchers would randomly assign participants to either the experimental group or the control group and compare the differences in outcomes between those who receive the drug or treatment and those who do not.

Real-life Examples

• Preventing illicit drug use in adolescents: Long-term follow-up data from a randomized control trial of a school population (Botvin et al., 2000).
• A prospective randomized control trial comparing medical and surgical treatment for early pregnancy failure (Demetroulis et al., 2001).
• A randomized control trial to evaluate a paging system for people with traumatic brain injury (Wilson et al., 2009).
• Prehabilitation versus Rehabilitation: A Randomized Control Trial in Patients Undergoing Colorectal Resection for Cancer (Gillis et al., 2014).
• A Randomized Control Trial of Right-Heart Catheterization in Critically Ill Patients (Guyatt, 1991).
• Berry, R. B., Kryger, M. H., & Massie, C. A. (2011). A novel nasal excitatory positive airway pressure (EPAP) device for the treatment of obstructive sleep apnea: A randomized controlled trial. Sleep , 34, 479–485.
• Gloy, V. L., Briel, M., Bhatt, D. L., Kashyap, S. R., Schauer, P. R., Mingrone, G., . . . Nordmann, A. J. (2013, October 22). Bariatric surgery versus non-surgical treatment for obesity: A systematic review and meta-analysis of randomized controlled trials. BMJ , 347.
• Streeton, C., & Whelan, G. (2001). Naltrexone, a relapse prevention maintenance treatment of alcohol dependence: A meta-analysis of randomized controlled trials. Alcohol and Alcoholism, 36 (6), 544–552.

How Should an RCT be Reported?

Reporting of a Randomized Controlled Trial (RCT) should be done in a clear, transparent, and comprehensive manner to allow readers to understand the design, conduct, analysis, and interpretation of the trial.

The Consolidated Standards of Reporting Trials ( CONSORT ) statement is a widely accepted guideline for reporting RCTs.

Further Information

• Cocks, K., & Torgerson, D. J. (2013). Sample size calculations for pilot randomized trials: a confidence interval approach. Journal of clinical epidemiology, 66(2), 197-201.
• Kendall, J. (2003). Designing a research project: randomised controlled trials and their principles. Emergency medicine journal: EMJ, 20(2), 164.

Akobeng, A.K., Understanding randomized controlled trials. Archives of Disease in Childhood , 2005; 90: 840-844.

Bell, C. C., Gibbons, R., & McKay, M. M. (2008). Building protective factors to offset sexually risky behaviors among black youths: a randomized control trial. Journal of the National Medical Association, 100 (8), 936-944.

Bhide, A., Shah, P. S., & Acharya, G. (2018). A simplified guide to randomized controlled trials. Acta obstetricia et gynecologica Scandinavica, 97 (4), 380-387.

Botvin, G. J., Griffin, K. W., Diaz, T., Scheier, L. M., Williams, C., & Epstein, J. A. (2000). Preventing illicit drug use in adolescents: Long-term follow-up data from a randomized control trial of a school population. Addictive Behaviors, 25 (5), 769-774.

Demetroulis, C., Saridogan, E., Kunde, D., & Naftalin, A. A. (2001). A prospective randomized control trial comparing medical and surgical treatment for early pregnancy failure. Human Reproduction, 16 (2), 365-369.

Gillis, C., Li, C., Lee, L., Awasthi, R., Augustin, B., Gamsa, A., … & Carli, F. (2014). Prehabilitation versus rehabilitation: a randomized control trial in patients undergoing colorectal resection for cancer. Anesthesiology, 121 (5), 937-947.

Globas, C., Becker, C., Cerny, J., Lam, J. M., Lindemann, U., Forrester, L. W., … & Luft, A. R. (2012). Chronic stroke survivors benefit from high-intensity aerobic treadmill exercise: a randomized control trial. Neurorehabilitation and Neural Repair, 26 (1), 85-95.

Guyatt, G. (1991). A randomized control trial of right-heart catheterization in critically ill patients. Journal of Intensive Care Medicine, 6 (2), 91-95.

MediLexicon International. (n.d.). Randomized controlled trials: Overview, benefits, and limitations. Medical News Today. Retrieved from https://www.medicalnewstoday.com/articles/280574#what-is-a-randomized-controlled-trial

Wilson, B. A., Emslie, H., Quirk, K., Evans, J., & Watson, P. (2005). A randomized control trial to evaluate a paging system for people with traumatic brain injury. Brain Injury, 19 (11), 891-894.

An official website of the United States government

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Publications
• Account settings
• Advanced Search
• Journal List

Randomized Controlled Trials

Emily c zabor , drph, alexander m kaizer , phd, brian p hobbs , phd.

• Author information
• Article notes
• Copyright and License information

CORRESPONDENCE TO: Emily C. Zabor, DrPH, Department of Quantitative Health Sciences & Taussig Cancer Institute, Cleveland Clinic, 9500 Euclid Ave, CA-60, Cleveland, OH 44195 [email protected]

Issue date 2020 Jul.

Randomized controlled trials (RCTs) are considered the highest level of evidence to establish causal associations in clinical research. There are many RCT designs and features that can be selected to address a research hypothesis. Designs of RCTs have become increasingly diverse as new methods have been proposed to evaluate increasingly complex scientific hypotheses. This article reviews the principles and general concepts behind many common RCT designs and introduces newer designs that have been proposed, such as adaptive and cluster randomized trials. A focus on the many choices for randomization within an RCT is described, along with their potential tradeoffs. To illustrate their diversity, examples of RCTs from the literature are provided. Statistical considerations, such as power and type I error rates, are discussed with the intention of providing practical guidance about how to specify study hypotheses that address the scientific question while being statistically appropriate. Finally, the freely available Consolidated Standards of Reporting Trials guidelines and US Food and Drug Administration guidance documents are introduced, along with a set of guidelines one should consider when planning an RCT or reviewing RCTs submitted for publication in peer-reviewed academic journals.

Key Words: biostatistics, clinical trials, randomized controlled trials, study design

Abbreviation: RCT, randomized controlled trial

General Overview of Study Design

Clinical studies are conducted among human participants to generate new knowledge through describing the impact of interventions devised to improve the diagnosis, prevention, and treatment of human disorders. There are many types of designs for clinical studies, but they all aim to obtain objective data to evaluate interventions with respect to an associated outcome in a target population. The two main types of clinical study are: (1) clinical trials, in which participants are assigned to receive a certain intervention according to a prespecified research plan and are then followed up prospectively to observe the outcome of interest; and (2) observational studies, in which the study investigators do not assign the exposure or intervention that participants receive. The quality of evidence generated by any study is determined by its experimental design. In all clinical studies, bias may be introduced due to misclassification of interventions or outcomes and missing data. In nonrandomized studies, bias may also be introduced through selection of the included participants and confounding due to differences in prognostic characteristics. These sources of bias may make it difficult or impossible to measure precisely the impact of the intervention under study.

The strongest evidence for causality between the exposure or intervention under study and the outcome observed comes from prospective experimental designs that randomly assign interventions to trial participants. Randomized controlled trials (RCTs) have traditionally been viewed as the gold standard of clinical trial design, residing at the top of the hierarchy of levels of evidence in clinical study; this is because the process of randomization can minimize differences in characteristics of the groups that may influence the outcome, thus providing the most definitive evidence regarding the impact of the exposure or intervention on the outcome. 1 , 2 In an RCT, one or more treatments are compared vs a control group, and patients are assigned to treatment or control by chance, such as by rolling a die or flipping a coin. Each group in an RCT is called an “arm,” so that, for example, a two-arm study may compare an experimental treatment vs a control group, and these would then be referred to as the “treatment arm” and the “control arm,” respectively.

Description of Subtypes of Study Design

Active control.

In an RCT, the control arm can take a variety of forms. If there is a well-established treatment for the disease under consideration, this standard-of-care treatment could then be used as the control arm, against which the novel experimental treatment is compared. In an active control trial, the goal may be to show that the experimental treatment is superior to the standard-of-care treatment (ie, superiority study), to show that the experimental treatment is similar to the standard-of-care treatment (ie, equivalence study), or simply to show that the experimental treatment is not much less effective than the standard-of-care treatment (ie, noninferiority study). If there is already a known treatment for the condition under study that will be used as the active control arm, then it is very important, for ethical reasons, to ensure that there is sound scientific rationale that the experimental treatment will be at least as effective. 3

Placebo Control

In the absence of an effective treatment for a disease, the control arm may consist of a group of patients receiving no treatment or receiving sham treatment, known as a placebo group. In a placebo group, inactive medication is given to patients in a way that is indistinguishable from the active treatment. For example, if patients assigned to the experimental treatment receive a small white capsule that they take three times a day, then the placebo group would receive a small white capsule with similar appearance, although containing no active ingredients, to take three times a day. In this way, effects of taking medication in and of itself, known as the placebo effect, are controlled between the two groups to reduce bias. Placebo-controlled trials seek to show that the experimental treatment is superior to the placebo. If no effective treatment is available for the condition being studied, then there are generally minimal ethical problems with a placebo-controlled trial. However, it is generally inappropriate and unethical to use a placebo if a treatment that improves outcomes is available for the condition under study. 3

Multiple Arm

There are two main types of multiple-arm trials. The first includes multiple dose levels or regimens of the experimental treatment all compared vs a single-control arm. In these so-called dose-response studies, it is ideal to include a zero-dose, or placebo, arm to avoid a situation in which all doses show similar activity and to establish whether any of the doses was superior to no treatment. 3 The second involves a single treatment arm with multiple control arms (eg, both an active control and a placebo control arm).

Cluster Randomized

In a cluster randomized trial, groups of subjects are randomized as opposed to individual subjects. There are several reasons for randomizing clusters as opposed to individuals, including administrative convenience, ethical considerations, and ease of application at the cluster level. This trial design is more common in health services and policy research as opposed to studies of drug interventions. For example, it may be of interest to randomize hospitals in a study of a new educational initiative for physicians both for ease of implementation of the intervention at the hospital level, as well as to avoid within-hospital contamination across individual physicians receiving different interventions. In a cluster randomized trial, it is important to carefully consider the impact of both the number of clusters and the cluster size on the power of the study. 4 A pragmatic trial design known as the stepped wedge cluster randomized trial has been gaining in popularity. In this trial design, aimed at eliminating logistical constraints, each cluster undergoes a period, or “step,” with no intervention followed by a step with exposure to the intervention. 5

Adaptive Design

In traditional RCTs, most trial elements, such as randomization allocations and the number of study arms, are fixed throughout the trial. However, adaptive designs, in which accumulating information during the trial is used to modify some aspect of the trial, are becoming increasingly popular. For example, accumulating information may inform the randomization assignment of the next patient to enroll, which represents a form of adaptive randomization. These designs may allow more patients to be accrued to the arm that is showing more promise, thus reducing ethical concerns about continuing enrollment on fixed randomization designs in the face of possibly increasing evidence that one of the treatments under study is superior and allowing more patients in the course of the trial to be given more effective treatments or doses. 6 , 7 We note that to maintain the established scientific and ethical standards of randomized comparative trials with the acquisition of evidence that is both prospective and objective, it is essential to prespecify potential avenues for adaptation as well as establish corresponding statistical criteria in advance of implementing the trial.

Platform trials describe multiple-arm studies with the potential to include control and experimental arms that can be opened or closed for enrollment throughout the course of the trial based on decision rules regarding efficacy. 8 , 9 In this way, ineffective treatments can be discontinued before many patients have been treated with them, and newly emerging treatments that show promise can be added at any point. These designs allow investigation into more experimental treatments in a shorter period of time. In addition, compared with a series of stand-alone, concurrent two-arm designs, platform designs allow more patients to be assigned to experimental treatment arms as opposed to control arms. 10

Use Cases of Study Design

The medical literature contains a multitude of examples of RCTs across many disease and intervention types. Some examples of recent two-arm randomized controlled trials published in CHEST are presented in Table 1 11 , 12 , 13 , 14 , 15 to demonstrate various applications of RCTs, although these examples are not exhaustive.

Examples of Randomized Controlled Trials Published in CHEST

Benefits of Study Design

The primary benefit of RCTs comes from the randomization itself, which greatly reduces confounding from both known and unknown sources. In nonrandomized studies, it may be possible to control for known confounders, but it is much more difficult to control for unknown or unmeasured confounders, although some methods that attempt to do so are available. With randomization, causal conclusions regarding the exposure or intervention and outcome can be made. Additional benefits stem from the controlled and prospective nature of an RCT. The dosage, timing, frequency, and duration of treatment can be controlled, and blinding may be possible. Blinding refers to a treatment assignment being unknown. In a single-blind study, patients do not know which treatment they are receiving. Blinding the patient prevents outcomes from being influenced by knowledge of treatment assignment. This is particularly important if any outcomes are self-reported. In a double-blind study, neither the patient nor the provider knows the treatment assignment. This additionally ensures that any care given by the provider or provider-assessed outcomes are not biased by knowledge of treatment assignment. Blinding is typically not possible in other study designs.

Downsides of Study Design

There are also disadvantages to RCTs. Because RCTs are highly controlled, the inclusion and exclusion criteria may lead to a homogeneous patient population, thus limiting the generalizability of the results to the broader population of patients with the condition being studied. RCTs also tend to study treatments in idealized environments, which may not be perfectly in-line with real-world usage of the treatment. Due to the complexities of design and conduct, RCTs are expensive and can take a long time to complete. RCTs are also not always feasible, for example, if a disease is very rare or if there are special considerations surrounding a disease that make randomized allocation either impractical or unethical.

Study Subject Considerations

An important consideration in the design of an RCT is the subject inclusion and exclusion criteria. These criteria will affect a variety of aspects of the conduct and interpretability of the study results and are primarily meant to ensure patient safety. 16 If eligibility criteria are too strict, it could be difficult to enroll the planned number of patients because of a potentially cumbersome screening process, as many screened patients may prove to be ineligible. In addition, very strict eligibility criteria could result in a study population that is not reflective of the broader target population, thus limiting the generalizability of the study result and the ability to establish the effectiveness of the treatment. These considerations need to be balanced with the fact that more strict exclusion criteria may be necessary to establish an intervention’s efficacy. It is especially important to ensure there are medically valid reasons for excluding any commonly underrepresented groups, such as women, racial and ethnic minority groups, pregnant and breastfeeding women, and children. 17

Equally important as the factors an investigator must consider when establishing patient eligibility criteria are the factors that potential study subjects consider when deciding whether to participate in a given trial. Most patients choose to participate in clinical trials with the hope of receiving a novel therapy from which they may benefit, yet the chance of receiving placebo or standard of care is also likely (often 50% or 33%). For these patients, the effort to participate in the trial is an altruistic act often driven by a desire to further scientific knowledge that may benefit future patients, if not themselves. With this in mind, investigators should additionally consider the burden placed on patients’ time and energy throughout the course of the trial, and weigh that against the scientific importance of additional follow-up visits and invasive or time-consuming testing procedures.

Statistical Considerations

End point definition.

There are several statistical considerations that an investigator must recognize when designing a RCT. The first is a clear and specific definition of the study end point. The end point needs to be an event or outcome that can be measured objectively so that the experimental group can be compared with the control group. For example, a study may wish to compare overall survival, rates of myocardial infarction, or improvements in quality of life. When selecting an end point for an RCT, the investigator must consider how the end point will be measured so that this will be standardized across all patients enrolled in the trial. The timing of assessment of the end point is also important to consider. For example, if interest is in overall survival, patients may need to be followed up for a long time before enough patients have died to determine whether there is a difference between the study arms. Alternatively, if interest is in rates of myocardial infarction, a time frame for occurrence of the event could be defined, such as myocardial infarction within 1 year of treatment start. It is common to have a primary end point in a study that is used as the basis for determining the needed sample size and ultimately making a decision regarding the efficacy of the experimental treatment, and then to include secondary end points as well; these secondary end points are more exploratory in nature and would not be used to make decisions about whether the experimental treatment is better than the control.

Sometimes in a clinical study, it may be of interest to determine whether the experimental treatment is efficacious with respect to more than one end point. In such cases, it is possible to have co-primary end points or to use composite end points. Co-primary end points require that all primary end points indicate that the experimental treatment is superior to control to conclude efficacy, and they require special statistical considerations (discussed later in more detail). Composite end points are a single end point composed of more than one measure. For example, disease-free survival is a composite end point defined as recurrence of disease or death. Use of a composite end point can increase the number of events observed in a study; this use must be considered carefully, however, to ensure that the true outcome of interest will be captured by the composite end point and that the individual components of the composite end point align.

One area of caution in selecting an end point for an RCT relates to the use of surrogate end points. Surrogate end points are used to represent an end point that is either difficult to measure or takes too long to occur. For example, interest may be in death from heart disease but a surrogate end point of change in cholesterol level from baseline may be used. To confidently use a surrogate end point, an investigator must be certain that the effect of the intervention on the surrogate predicts the effect of the intervention on the true clinical end point of interest, which is often difficult or impossible to establish.

Effect Size

Once the end point is specifically defined, the investigators must then establish the meaningful difference between groups that they seek to detect. This is a clinical decision that has statistical implications for the design with respect to the number of patients that will be needed in the study. For example, in a study of an experimental treatment for lung cancer, in which overall survival is the primary end point, we know that 1-year overall survival on the standard-of-care treatment is 70%. Interest is in improving overall survival to 80% in patients on the experimental treatment. These rates can now be used to define the study hypotheses and determine the sample size required to conduct the study.

Power, Error Rates, and Sample Size

The traditional statistical approach considers two possible outcomes of the trial. Consider a two-arm RCT in which p t represents the rate of a binary outcome in the treatment arm and p c represents the rate in the control arm. The null hypothesis then represents the scenario in which the treatment has no benefit and is denoted as H 0 : p t = p c to indicate that the rates are equivalent in the treatment and control arms. The alternative hypothesis represents the scenario in which the treatment and control differ and is denoted as H A : p t ≠ p c to indicate that the rates are not equivalent in the treatment and control arms. Note that in this example we have used what is termed a “two-sided alternative,” meaning that we are looking for the two rates to not be equal, but the rate in the treatment group could be either higher or lower than the rate in the control group. A two-sided alternative such as this provides the most definitive evidence about an experimental treatment. However, the use of a two-sided alternative will require that a larger number of patients be enrolled in the trial, and at times, a “one-sided alternative” (choosing either H A : p t > p c or H A : p t < p c as the specified alternative hypothesis) could be appropriate.

Now that the end point has been defined, the effect size of interest has been established, and the null and alternative hypotheses are fixed, the sample size needed for analysis can be calculated. Sample size is traditionally based on error rates. A type I error is rejecting a null hypothesis when it is true, and a type II error is failing to reject the null hypothesis when it is false, in which case the alternative hypothesis is assumed to be true. The type I error rate is conventionally set to 0.05, meaning that we are willing to accept a 1 in 20 chance that we will claim there is a difference between groups when in truth there is no difference. The complement of type II error (ie, 1 – type II error) is known as statistical power. Statistical power represents the probability of rejecting the null hypothesis when the alternative hypothesis is true. It is commonplace to ensure that statistical power is ≥ 0.8 in an RCT. The specific formula used to calculate the sample size will depend on many things, including the type of end point (eg, binary, continuous, time-to-event) as well as the study design, the type I and type II error rates, the variance of the end point, and allocation ratios to the various study arms. The calculation should also account for patients who drop out or who are lost to follow-up during the study. The details of all possibilities are outside the scope of this commentary, and a statistician should be consulted when designing an RCT.

For trials designed to examine multiple end points, such as the co-primary end points described previously, the possibility of committing a type I error may occur within each end point, yielding two types of false conclusions. 18 Marginal type I error rates estimate the type I error for each end point separately, whereas family-wise type I error rates consider the entire trial in violation if the null hypothesis is falsely rejected for a single end point. A family-wise type I error represents stronger control against false-positive findings for individual end points when directly compared with a marginal type I error. Marginal and family-wise errors are identical when testing a single end point. When multiple end points are being examined in a trial, control of family-wise error can be accomplished through adjustment for multiple comparisons. Common techniques include the Bonferroni procedure, 19 the Šidák procedure, 20 or the Holm’s procedure. 21 There are many others, however, and consultation with a statistician is advised when designing a study that requires adjustment for multiple comparisons.

Methods for Randomization

Figure 1 depicts a flowchart for selecting from among the following methods for randomization.

Flowchart for selecting a method of randomization.

Simple Randomization

In simple randomization, no restrictions are placed on the randomization sequence other than the final sample size. The randomization can be conducted with the flip of a coin or any other procedure that assigns each enrolling patient to an arm with an equal probability (eg, 1/2 for a two-arm study). However, because only the total sample size (and not the per-arm sample sizes) is being controlled in simple randomization, imbalances in the numbers assigned to the two or more arms are possible; this could lead to imbalances in subject characteristics, especially when the total sample size is small.

Permuted Block Randomization

To overcome the possible imbalances that can arise from simple randomization, the permuted-block design divides patients into blocks over time and balances the randomization to each arm within each block. If the total sample size is a multiple of the block size, balance is then guaranteed at the end of the study. 22 It is also possible to use unequal block sizes throughout the study, which would serve to further obscure any predictability of future treatment assignments.

Minimization

Although simple randomization will frequently control imbalance in prognostic factors between arms, it is still possible to have imbalances, especially if the sample size is relatively small (ie, ≤ 100). Often this is overcome through stratified randomization, in which simple randomization is conducted within groups based on important prognostic characteristics to ensure balance within those features. Stratified randomization can become cumbersome as the number of prognostic factors increases, and the strata must be accounted for when analyzing the resulting study data. Another approach, minimization, was introduced as an alternative to stratified randomization. With the minimization approach, assignment to a study arm is done with the intention of achieving balance between randomization groups with respect to prognostic factors of interest. 23 Minimization can greatly reduce prognostic imbalances between groups but at the cost of truly random treatment assignment, as assignment to a study arm is determined by the characteristics of patients who have already been assigned and not according to chance alone.

Outcome Adaptive Randomization

Adaptive randomization refers to randomization procedures that adjust the allocation ratio as the study progresses. In outcome adaptive randomization, the goal is to assign more patients to the more promising treatments based on the accumulating data in the trial to minimize the expected number of treatment failures and to overcome the questionable ethics of continuing to assign patients to treatment arms for which there is evidence of poorer efficacy. 6 The most commonly used outcome adaptive randomization designs are based on the Bayesian statistical paradigm and include Bayesian adaptive randomization 24 and response adaptive randomization. 25

Analytic Considerations

Prior to conducting an RCT, the analysis plan should be detailed. There are several ways that RCT data can be analyzed to account for lack of adherence. Consider a patient who is randomized to the experimental treatment arm but for whatever reason discontinues use of the treatment before completing the trial-specified regimen. How should this patient be incorporated into the statistical analysis at the end of the trial? One approach is “intention-to-treat,” in which all patients are analyzed in the group to which they were randomized, regardless of their adherence to the treatment regimen. Intention-to-treat analysis is recommended in superiority trials to reduce bias, as the original randomization assignment is maintained. An alternative is a “per-protocol” analysis. In this approach, only patients who completed the treatment to which they were originally assigned are analyzed. A per-protocol analysis may lead to a purer estimate of the treatment effect, but the results of the analysis should be interpreted cautiously because bias can be introduced by the reasons subjects did not adhere to the planned treatment regimen. Often in RCTs, intention-to-treat will be the primary analysis approach but a per-protocol analysis will additionally be performed as a secondary analysis.

Another analytic consideration is how to accommodate the potential for treatment effect heterogeneity, which is the possibility that the treatment has different effects in different subgroups of patients. The heterogeneity of treatment effect, in its extreme, can make the overall population effect seem clinically insignificant when there are certain subpopulations that would benefit from the treatment and other subpopulations that do not benefit. To avoid potential inflation of the type I error rate, any subgroup analyses should be specified prior to the trial, rather than as a post hoc attempt at salvaging a trial with a null result. Traditionally, the approach has consisted of a priori subgroup analyses that consider subgroups “one variable at a time,” with results graphically presented in forest plots. However, consensus building research has sought to identify more efficient methods to consider all relevant patient attributes simultaneously with the Predictive Approaches to Treatment Effect Heterogeneity statement. 26

Reporting Considerations

The Consolidated Standards of Reporting Trials guidelines ( http://www.consort-statement.org/ ) were established to guide investigators in appropriate reporting of results of RCTs. These guidelines consist of a 25-item checklist for use in putting together a report on an RCT, as well as a template for a flow diagram to include in the trial report indicating the breakdown of sample size at various stages of the study.

Available Standards

The most comprehensive guidelines regarding the design and conduct of clinical trials are published by the US Food and Drug Administration ( https://www.fda.gov/regulatory-information/search-fda-guidance-documents ). In addition to documents referenced earlier that provide guidance on selection of an appropriate control group, information regarding the use of placebos and blinding is also available. The International Conference on Harmonisation of Technical Requirements for Pharmaceuticals for Human Use have also created guidelines for the conduct of clinical trials that cover the selection of end points, various design options, trial monitoring, and analytic considerations. 27

Short List of Questions to Guide the Researcher

Consult with a statistician early and often. It is never too early to involve a statistician in planning the design of an RCT.

Select an intervention(s). For treatment trials, dosage and duration of treatment should also be determined.

Select an appropriate control arm and consider the practical and ethical considerations of using a placebo or standard of care.

Define the study end point as specifically as possible, including when it will be assessed.

Establish the effect size of interest. How much improvement are you hoping to see in the experimental treatment arm?

Write down the study hypotheses and determine whether a one-sided or two-sided alternative hypothesis is most appropriate.

Select acceptable type I and type II error rates and calculate the sample size needed to detect the desired effect size. If multiple primary end points are being used, be sure to consider marginal vs family-wise error rates.

Determine the feasibility of accruing the number of patients needed according to the calculated sample size. Can this be accomplished at a single institution, or are multiple institutions needed? If multiple institutions are needed, how does this affect study implementation?

Write down the analysis plan in detail before beginning the study.

Short List of Questions to Guide the Reviewer

When reviewing a manuscript describing a randomized controlled trial, consider commenting on the following:

The exposure or intervention in the treatment arm and control arm. Was there justification for the exposure or intervention in the treatment arm? If the control arm received standard of care, was an appropriate standard of care applied? If there was a placebo control arm, was there a possibility of distinguishing the treatment and control arms by the nature of the intervention?

Key features of the study methodology. Were appropriate study end point(s) chosen? Was their measurement accurate and consistent? Was the randomization procedure appropriate? Were details of the sample size calculation, including the anticipated effect of the intervention, provided? Was drop out handled appropriately in the planning and analysis of the trial? Is the Consolidated Standards of Reporting Trials guidelines flowchart included in the report?

The reported results and their interpretation. Were the reported results in line with the planned analyses? Was the interpretation of the results made based on the planned primary end point? Was the interpretation of the results appropriate?

Acknowledgments

Financial/nonfinancial disclosures: The authors have reported to CHEST the following: B. P. H. is scientific advisor to Presagia; consultant for SimulStat; and receives research funds from Amgen. None declared (E. C. Z., A. M. K.).

Role of sponsors: The sponsor had no role in the design of the study, the collection and analysis of the data, or the preparation of the manuscript.

FUNDING/SUPPORT: B. P. H. was supported in part by the Case Comprehensive Cancer Center [P30 CA043703].

• 1. The periodic health examination. Canadian Task Force on the Periodic Health Examination. Can Med Assoc J. 1979;121(9):1193–1254. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 2. Sackett D.L. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest. 1989;95(suppl 2):2s–4s. [ PubMed ] [ Google Scholar ]
• 3. US Department of Health and Human Services Food and Drug Administration. Guidance for Industry: Choice of Control Group and Related Issues in Clinical Trials. 2001. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/e10-choice-control-group-and-related-issues-clinical-trials . Accessed May 15, 2020.
• 4. Hemming K., Eldridge S., Forbes G., Weijer C., Taljaard M. How to design efficient cluster randomised trials. BMJ. 2017;358:j3064. doi: 10.1136/bmj.j3064. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 5. Hemming K., Haines T.P., Chilton P.J., Girling A.J., Lilford R.J. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ. 2015;350:h391. doi: 10.1136/bmj.h391. [ DOI ] [ PubMed ] [ Google Scholar ]
• 6. Hobbs B.P., Berry D.A., Coombes K.R. Biostatistics and bioinformatics in clinical trials. In: Niederhuber J.E., Armitage J.O., Doroshow J.H., Kastan M.B., Tepper J.E., editors. Abeloff's Clinical Oncology, Sixth Edition. Elsevier; Philadelphia, PA: 2020. pp. 284–295.e2. [ Google Scholar ]
• 7. Hobbs B.P., Lee J.J. Adaptive trial designs. In: Crowley J., Hoering A., editors. Handbook of Statistics in Clinical Oncology. Third Edition. Taylor & Francis/CRC Press; Boca Raton, FL: 2012. pp. 265–292. [ Google Scholar ]
• 8. Renfro L.A., Sargent D.J. Statistical controversies in clinical research: basket trials, umbrella trials, and other master protocols: a review and examples. Ann Oncol. 2017;28(1):34–43. doi: 10.1093/annonc/mdw413. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 9. Woodcock J., LaVange L.M. Master protocols to study multiple therapies, multiple diseases, or both. N Engl J Med. 2017;377(1):62–70. doi: 10.1056/NEJMra1510062. [ DOI ] [ PubMed ] [ Google Scholar ]
• 10. Hobbs B.P., Chen N., Lee J.J. Controlled multi-arm platform design using predictive probability. Stat Methods Med Res. 2018;27(1):65–78. doi: 10.1177/0962280215620696. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 11. Pépin J.L., Jullian-Desayes I., Sapéne M. Multimodal remote monitoring of high cardiovascular risk patients with OSA initiating CPAP: a randomized trial. Chest. 2019;155(4):730–739. doi: 10.1016/j.chest.2018.11.007. [ DOI ] [ PubMed ] [ Google Scholar ]
• 12. Tanner N.T., Yarmus L., Chen A. Standard bronchoscopy with fluoroscopy vs thin bronchoscopy and radial endobronchial ultrasound for biopsy of pulmonary lesions: a multicenter, prospective, randomized trial. Chest. 2018;154(5):1035–1043. doi: 10.1016/j.chest.2018.08.1026. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 13. Furian M., Lichtblau M., Aeschbacher S.S. Efficacy of dexamethasone in preventing acute mountain sickness in COPD patients: randomized trial. Chest. 2018;154(4):788–797. doi: 10.1016/j.chest.2018.06.006. [ DOI ] [ PubMed ] [ Google Scholar ]
• 14. Semler M.W., Janz D.R., Russell D.W. A multicenter, randomized trial of ramped position vs sniffing position during endotracheal intubation of critically ill adults. Chest. 2017;152(4):712–722. doi: 10.1016/j.chest.2017.03.061. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 15. Alansari K., Sayyed R., Davidson B.L., Al Jawala S., Ghadier M. IV magnesium sulfate for bronchiolitis: a randomized trial. Chest. 2017;152(1):113–119. doi: 10.1016/j.chest.2017.03.002. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 16. Kim E.S., Bruinooge S.S., Roberts S. Broadening eligibility criteria to make clinical trials more representative: American Society of Clinical Oncology and Friends of Cancer Research Joint Research Statement. J Clin Oncol. 2017;35(33):3737–3744. doi: 10.1200/JCO.2017.73.7916. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 17. Van Spall H.G.C., Toren A., Kiss A., Fowler R.A. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA. 2007;297(11):1233–1240. doi: 10.1001/jama.297.11.1233. [ DOI ] [ PubMed ] [ Google Scholar ]
• 18. Hobbs B.P., Thall P.F., Lin S.H. Bayesian group sequential clinical trial design using total toxicity burden and progression-free survival. J R Stat Soc Ser C Appl Stat. 2016;65(2):273–297. doi: 10.1111/rssc.12117. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 19. Dunn O.J. Multiple comparisons among means. J Am Stat Assoc. 1961;56(293):52–64. [ Google Scholar ]
• 20. Šidák Z. Rectangular confidence regions for the means of multivariate normal distributions. J Am Stat Assoc. 1967;62(318):626–633. [ Google Scholar ]
• 21. Holm S. A simple sequentially rejective multiple test procedure. Scand J Stat. 1979;6(2):65–70. [ Google Scholar ]
• 22. Matts J.P., Lachin J.M. Properties of permuted-block randomization in clinical trials. Control Clin Trials. 1988;9(4):327–344. doi: 10.1016/0197-2456(88)90047-5. [ DOI ] [ PubMed ] [ Google Scholar ]
• 23. Pocock S.J., Simon R. Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. Biometrics. 1975;31(1):103–115. [ PubMed ] [ Google Scholar ]
• 24. Thall P.F., Wathen J.K. Practical Bayesian adaptive randomisation in clinical trials. Eur J Cancer. 2007;43(5):859–866. doi: 10.1016/j.ejca.2007.01.006. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 25. Lee J.J., Xuemin G., Suyu L. Bayesian adaptive randomization designs for targeted agent development. Clin Trials. 2010;7(5):584–596. doi: 10.1177/1740774510373120. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 26. Kent DM, Paulus JK, van Klaveren D, et al. The Predictive Approaches to Treatment effect Heterogeneity (PATH) statement [published online ahead of print November 12, 2019]. Ann Intern Med . https://doi.org/10.7326/M18-3667 . [ DOI ] [ PubMed ]
• 27. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use, Statistical Principles for Clinical Trials. 1998. https://database.ich.org/sites/default/files/E9_Guideline.pdf . Accessed May 15, 2020.
• View on publisher site
• PDF (172.2 KB)
• Collections

Similar articles

Cited by other articles, links to ncbi databases.

• Download .nbib .nbib
• Format: AMA APA MLA NLM

Add to Collections