Understanding Internal and External Validity

How These Concepts Are Applied in Research

Plan your study well to ensure internal validity.

 Getty / Andresr

Internal and external validity are concepts that reflect whether or not the results of a study are trustworthy and meaningful. While internal validity relates to how well a study is conducted (its structure), external validity relates to how applicable the findings are to the real world.

Internal Validity

Internal validity is the extent to which a study establishes a trustworthy cause-and-effect relationship between a treatment and an outcome. It also reflects that a given study makes it possible to eliminate alternative explanations for a finding. For example, if you implement a smoking cessation program with a group of individuals, how sure can you be that any improvement seen in the treatment group is due to the treatment that you administered?

Internal validity depends largely on the procedures of a study and how rigorously it is performed.

Internal validity is not a "yes or no" type of concept. Instead, we consider how confident we can be with the findings of a study, based on whether it avoids traps that may make the findings questionable.

The less chance there is for "confounding" in a study, the higher the internal validity and the more confident we can be in the findings. Confounding refers to a situation in which other factors come into play that confuses the outcome of a study. For instance, a study might make us unsure as to whether we can trust that we have identified the above "cause-and-effect" scenario.

In short, you can only be confident that your study is internally valid if you can rule out alternative explanations for your findings. As a brief summary, you can only assume cause-and-effect when you meet the following three criteria in your study:

  1. The cause preceded the effect in terms of time.
  2. The cause and effect vary together.
  3. There are no other likely explanations for this relationship that you have observed.

Factors That Improve Internal Validity

If you are looking to improve the internal validity of a study, you will want to consider aspects of your research design that will make it more likely that you can reject alternative hypotheses. There are many factors that can improve internal validity.

  • Randomization refers to randomly assigning participants to treatment and control groups, and ensures that there is not any systematic bias between groups.
  • Random selection of participants refers to choosing your participants at random or in a manner in which they are representative of the population that you wish to study.
  • Blinding in a study refers to participants—and sometimes researchers—being unaware of what intervention they are receiving (such as by using a placebo in a medication study) to avoid this knowledge biasing their perceptions and behaviors and thus the outcome of the study.
  • Experimental manipulation refers to manipulating an independent variable in a study (for instance, giving smokers a cessation program) instead of just observing an association without conducting any intervention (examining the relationship between exercise and smoking behavior).
  • Study protocol refers to following specific procedures for the administration of a treatment so as not to introduce any effects of, for example, doing things differently with one group of people versus another group of people.

    Factors That Threaten Internal Validity

    Just as there are many ways to ensure that a study is internally valid, there is also a list of potential threats to internal validity that should be considered when planning a study.

    • Confounding refers to a situation in which changes in an outcome variable can be thought to have resulted from some third variable that is related to the treatment that you administered.
    • Historical events may influence the outcome of studies that occur over a period of time. Examples of these events might include a change in political leader or natural disaster that influences how study participants feel and act.
    • Maturation refers to the impact of time as a variable in a study. If a study takes place over a period of time in which it is possible that participants naturally changed in some way (grew older, became tired), then it may be impossible to rule out whether effects seen in the study were simply due to the effect of time.
    • Testing refers to the effect of repeatedly testing participants using the same measures. If you give someone the same test three times, isn't it likely that they will do better as they learn the test or become used to the testing process so that they answer differently?
    • Instrumentation refers to the impact of the actual testing instruments used in a study on how participants respond. While it may sound unusual, its possible to "prime" participants in a study in certain ways with the measures that you use, which causes them to react in a way that is different than they would have otherwise.
    • Statistical regression refers to the natural effect of participants at extreme ends of a measure falling in a certain direction just due to the passage of time rather than the effect of an intervention.
    • Attrition refers to participants dropping out or leaving a study, which means that the results are based on a biased sample of only the people who did not choose to leave (and possibly who all have something in common, such as higher motivation).
    • Diffusion refers to the treatment in a study spreading from the treatment group to the control group through the groups interacting and talking with or observing one another. This can also lead to another issue called resentful demoralization, in which a control group tries less hard because they feel resentful over the group that they are in.
    • Experimenter bias refers to an experimenter behaving in a different way with different groups in a study, which leads to an impact on the results of this study (and is eliminated through blinding).

    External Validity

    External validity refers to how well the outcome of a study can be expected to apply to other settings. In other words, this type of validity refers to how generalizable the findings are. For instance, do the findings apply to other people, settings, situations, and time periods?

    Ecological validity, an aspect of external validity, refers to whether a study's findings can be generalized to the real world.

    While rigorous research methods can ensure internal validity, external validity, on the other hand, may be limited by these methods.

    Another term called transferability relates to external validity and refers to the qualitative research design. Transferability refers to whether results transfer to situations with similar characteristics.

    Factors that Improve External Validity

    What can you do to improve the external validity of your study?

    • Inclusion and exclusion criteria should be used to ensure that you have clearly defined the population that you are studying in your research.
    • Psychological realism refers to making sure that participants are experiencing the events of a study as a real event and can be achieved by telling them a "cover story" about the aim of the study. Otherwise, in some cases, participants might behave differently than they would in real life if they know what to expect or know what the aim of the study is.
    • Replication refers to conducting the study again with different samples or in different settings to see if you get the same results. When many studies have been conducted, meta-analysis can also be used to determine if the effect of an independent variable is reliable (based on examining the findings of a large number of studies on one topic).
    • Field experiments can also be used in which you conduct a study outside the laboratory in a natural setting.
    • Reprocessing or calibration refers to using statistical methods to adjust for problems related to external validity. For example, if a study had uneven groups for some characteristic (such as age), reweighting might be used.

      Factors That Threaten External Validity

      External validity is threatened when a study does not take into account the interactions of variables in the real world.

      • Situational factors such as time of day, location, noise, researcher characteristics, and how many measures are used may affect the generalizability of findings.
      • Pre- and post-test effects refer to the situation in which the pre- or post-test is in some way related to the effect seen in the study, such that the cause-and-effect relationship disappears without these added tests.
      • Sample features refer to the situation in which some feature of the particular sample was responsible for the effect (or partially responsible), leading to limited generalizability of the findings.
      • Selection bias refers to the problem of differences between groups in a study that may relate to the independent variable (once again, something like motivation or willingness to take part in the study, specific demographics of individuals being more likely to take part in an online survey). This can also be considered a threat to internal validity.

      Similarities and Differences

      Internal and external validity are like two sides of the same coin. You can have a study with good internal validity, but overall it could be irrelevant to the real world. On the other hand, you could conduct a field study that is highly relevant to the real world, but that doesn't have trustworthy results in terms of knowing what variables caused the outcomes that you see.

      Similarities

      What are the similarities between internal and external validity? They are both factors that should be considered when designing a study, and both have implications in terms of whether the results of a study have meaning. Both are not "either/or" concepts, and so you will always be deciding to what degree your study performs in terms of both types of validity.

      Each of these concepts is typically reported in a research article that is published in a scholarly journal. This is so that other researchers can evaluate the study and make decisions about whether the results are useful and valid.

      Differences

      The essential difference between internal and external validity is that internal validity refers to the structure of a study and its variables while external validity relates to how universal the results are. There are further differences between the two as well.

      Internal Validity

      • Focus on accuracy and strong research methods

      • Controls extraneous variables

      • Conclusions are warranted

      • Eliminates alternative explanations

      External Validity

      • Results translate to world at large

      • Findings are generalizable

      • Outcomes apply to practical situations

      • Results can be translated into another context

      Internal validity focuses on showing a difference that is due to the independent variable alone, whereas external validity results can be translated to the world at large.

      Examples

      An example of a study with good internal validity would be if a researcher hypothesizes that using a particular mindfulness app will reduce negative mood. To test this hypothesis, the researcher randomly assigns a sample of participants to one of two groups: those who will use the app over a defined period, and those who engage in a control task.

      The researcher ensures that there is no systematic bias in how participants are assigned to the groups, and also blinds his research assistants to the groups the students are in during experimentation.

      A strict study protocol is used that outlines the procedures of the study. Potential confounding variables are measured along with mood, such as the participants socioeconomic status, gender, age, among other factors. If participants drop out of the study, their characteristics are examined to make sure there is no systematic bias in terms of who stays in the study.

      An example of a study with good external validity would be in the above example, the researcher also ensured that the study had external validity by having participants use the app at home rather than in the laboratory. The researcher clearly defines the population of interest and choosing a representative sample, and he/she replicates the study for different technological devices.

      A Word From Verywell

      Setting up an experiment so that it has sound internal and external validity involves being mindful from the start about factors that can influence each aspect of your research. It's best to spend extra time designing a structurally sound study that has far-reaching implications rather than to quickly rush through the design phase only to discover problems later on. Only when both internal and external validity are high can strong conclusions be made about your results.

      Was this page helpful?
      Article Sources
      • Indiana University. Threats to internal and external validity.

      • Slack MK, Draugalis JR. Establishing the internal and external validity of experimental studies. Am J Health Syst Pharm. 2001 Nov 15;58(22):2173–81; quiz 2182–3.

      • Behi R, Nolan M. Causality and control: threats to internal validity. Br J Nurs. 1996 Apr 28;5(6):374–7. DOI: 10.12968/bjon.1996.5.6.374 

      • Andrade C. Internal, external, and ecological validity in research design, conduct, and evaluation. Indian J Psychol Med. 2018;40(5):498–9. DOI: 10.4103/ijpsym.ijpsym_334_18