What Is the Hawthorne Effect? | Definition & Examples

The Hawthorne effect refers to people’s tendency to behave differently when they become aware that they are being observed. As a result, what is observed may not represent “normal” behavior, threatening the internal and external validity of your research.

The Hawthorne effect is also known as the observer effect and is closely linked with observer bias.

Example: Hawthorne effect
You are researching the smoking rates among bank employees as part of a smoking cessation program. You collect your data by watching the employees during their work breaks.

If employees are aware that you are observing them, this can affect your study’s results. For example, you may record higher or lower smoking rates than are genuinely representative of the population under study.

Like other types of research bias, the Hawthorne effect often occurs in observational and experimental study designs in fields like medicine, organizational psychology, and education.

Continue reading: What Is the Hawthorne Effect? | Definition & Examples

What Is Confirmation Bias? | Definition & Examples

Confirmation bias is the tendency to seek out and prefer information that supports our preexisting beliefs. As a result, we tend to ignore any information that contradicts those beliefs.

Confirmation bias is often unintentional but can still lead to poor decision-making in (psychology) research and in legal or real-life contexts.

Example: Confirmation bias
During presidential elections, people tend to seek information that paints the candidate they support in a positive light, while dismissing any information that paints them in a negative light.

This type of research bias is more likely to occur while processing information related to emotionally charged topics, values, or deeply held beliefs.

Continue reading: What Is Confirmation Bias? | Definition & Examples

Inclusion and Exclusion Criteria | Examples & Definition

Inclusion and exclusion criteria determine which members of the target population can or can’t participate in a research study. Collectively, they’re known as eligibility criteria, and establishing them is critical when seeking study participants for clinical trials.

This allows researchers to study the needs of a relatively homogeneous group (e.g., people with liver disease) with precision. Examples of common inclusion and exclusion criteria are:

  • Demographic characteristics: Age, gender identity, ethnicity
  • Study-specific variables: Type and stage of disease, previous treatment history, presence of chronic conditions, ability to attend follow-up study appointments, technological requirements (e.g., internet access)
  • Control variables: Fitness level, tobacco use, medications used

Failure to properly define inclusion and exclusion criteria can undermine your confidence that causal relationships exist between treatment and control groups, affecting the internal validity of your study and the generalizability (external validity) of your findings.

Continue reading: Inclusion and Exclusion Criteria | Examples & Definition

What Is Predictive Validity? | Examples & Definition

Predictive validity refers to the ability of a test or other measurement to predict a future outcome. Here, an outcome can be a behavior, performance, or even disease that occurs at some point in the future.

Example: Predictive validity
A pre-employment test has predictive validity when it can accurately identify the applicants who will perform well after a given amount of time, such as one year on the job.

Predictive validity is a subtype of criterion validity. It is often used in education, psychology, and employee selection.

Continue reading: What Is Predictive Validity? | Examples & Definition

What Is Concurrent Validity? | Definition & Examples

Concurrent validity shows you the extent of the agreement between two measures or assessments taken at the same time. It compares a new assessment with one that has already been tested and proven to be valid.

Concurrent validity is a subtype of criterion validity. It is called “concurrent” because the scores of the new test and the criterion variables are obtained at the same time.

Example: Concurrent validity
You want to assess the concurrent validity of a new survey measuring employee commitment. To do so, you can either:

  1. Ask the same sample of employees to fill in both an existing (validated) survey and your new survey. Then compare the results.
  2. Ask a sample of employees to fill in your new survey. Then, compare their responses to the results of a common measure of employee performance, such as a performance review.

If the results of the two measurement procedures are similar, you can conclude that they are measuring the same thing (i.e., employee commitment). This demonstrates concurrent validity.

Establishing concurrent validity is particularly important when a new measure is created that claims to be better in some way than existing measures: more objective, faster, cheaper, etc.

Continue reading: What Is Concurrent Validity? | Definition & Examples

What Is Ecological Validity? | Definition & Examples

Ecological validity measures how generalizable experimental findings are to the real world, such as situations or settings typical of everyday life. It is a subtype of external validity.

If a test has high ecological validity, it can be generalized to other real-life situations, while tests with low ecological validity cannot.

Example: Ecological validity
You are researching whether airline passengers pay attention to in-flight safety videos. You are interested in whether or not they can recall specific information from them. You recruit a sample of 100 people and send them a safety video, asking them to watch it on their own time. Afterwards, you send them a questionnaire to find out what they can recall from the video.

Using this approach, your findings would have low ecological validity. The experience of watching the video at home is vastly different from watching it on a plane.

To achieve high ecological validity, the best approach would be to conduct the experiment on an actual flight.

Ecological validity is often applied in experimental studies of human behavior and cognition, such as in psychology and related fields.

Continue reading: What Is Ecological Validity? | Definition & Examples

What Is Criterion Validity? | Definition & Examples

Criterion validity (or criterion-related validity) evaluates how accurately a test measures the outcome it was designed to measure. An outcome can be a disease, behavior, or performance. Concurrent validity measures tests and criterion variables in the present, while predictive validity measures those in the future.

To establish criterion validity, you need to compare your test results to criterion variables. Criterion variables are often referred to as a “gold standard” measurement. They comprise other tests that are widely accepted as valid measures of a construct.

Example: Criterion validity
A researcher wants to know whether a college entrance exam is able to predict future academic performance. First-semester GPA can serve as the criterion variable, as it is an accepted measure of academic performance.

The researcher can then compare the college entry exam scores of 100 students to their GPA after one semester in college. If the scores of the two tests are close, then the college entry exam has criterion validity.

When your test agrees with the criterion variable, it has high criterion validity. However, criterion variables can be difficult to find.

Continue reading: What Is Criterion Validity? | Definition & Examples

What Is Discriminant Validity? | Definition & Example

Discriminant validity refers to the extent to which a test is not related to other tests that measure different constructs. Here, a construct is a behavior, attitude, or concept, particularly one that is not directly observable.

The expectation is that two tests that reflect different constructs should not be highly related to each other. If they are, then you cannot say with certainty that they are not measuring the same construct. Thus, discriminant validity is an indication of the extent of the difference between constructs.

Discriminant validity is assessed in combination with convergent validity. In some fields, discriminant validity is also known as divergent validity.

Example: Discriminant validity (divergent validity)
You are researching extroversion as a personality trait among marketing students. To establish discriminant validity, you must also measure an unrelated construct, such as intelligence.

You have developed a questionnaire to measure extroversion, but you also ask your respondents to fill in a second questionnaire measuring intelligence in order to test the discriminant validity of your questionnaire.

Since the two constructs are unrelated, there should be no significant relationship between the scores of the two tests.

If there is a correlation, then you may be measuring the same construct in both tests. This is an indication of poor discriminant validity.

Continue reading: What Is Discriminant Validity? | Definition & Example

What Is Convergent Validity? | Definition & Examples

Convergent validity refers to how closely a test is related to other tests that measure the same (or similar) constructs. Here, a construct is a behavior, attitude, or concept, particularly one that is not directly observable.

Ideally, two tests measuring the same construct, such as stress, should have a moderate to high correlation. High correlation is evidence of convergent validity, which, in turn, is an indication of construct validity.

Example: Convergent validity
Suppose you use two different methods to collect data about anger: observation and a self-report questionnaire. If the scores of the two methods are similar, this suggests that they indeed measure the same construct. A high correlation between the two test scores suggests convergent validity.

Continue reading: What Is Convergent Validity? | Definition & Examples

What Is Content Validity? | Definition & Examples

Content validity evaluates how well an instrument (like a test) covers all relevant parts of the construct it aims to measure. Here, a construct is a theoretical concept, theme, or idea: in particular, one that cannot usually be measured directly.

Content validity is one of the four types of measurement validity. The other three are:

  • Face validity: Does the content of the test appear to be suitable for its aims?
  • Criterion validity: Do the results accurately measure the concrete outcome they are designed to measure?
  • Construct validity: Does the test measure the concept that it’s intended to measure?
Example: Content validity in exams
A written exam tests whether individuals have enough theoretical knowledge to acquire a driver’s license. The exam would have high content validity if the questions asked cover every possible topic in the course related to traffic rules. At the same time, it should also exclude all other questions that aren’t relevant for the driver’s license.

Continue reading: What Is Content Validity? | Definition & Examples