The terms reproducibility, repeatability, and replicability are sometimes used interchangeably, but they mean different things.
A research study is reproducible when the existing data is reanalysed using the same research methods and yields the same results. This shows that the analysis was conducted fairly and correctly.
A research study is replicable (or repeatable) when the entire research process is conducted again, using the same methods but new data, and still yields the same results. This shows that the results of the original study are reliable.
NoteA study may be reproducible but not replicable.
A survey of 60 children between the ages of 12 and 16 shows that football and hockey are the most popular sports. Football received 20 votes and hockey 18.
An independent researcher reanalyses the survey data and also finds that 20 children chose football and 18 children chose hockey. This makes the research reproducible.
The researcher then decides to conduct the study all over again. Another 60 children between the ages of 12 and 16 take part in the study. This time the results show that tennis is the most popular sport, chosen 25 times. The research is therefore not replicable.
Snowball sampling is a non-probability sampling method where new units are recruited by other units to form part of the sample. Snowball sampling can be a useful way to conduct research about people with specific traits who might otherwise be difficult to identify (e.g., people with a rare disease).
Also known as chain sampling or network sampling, snowball sampling begins with one or more study participants. It then continues on the basis of referrals from those participants. This process continues until you reach the desired sample, or a saturation point.
Example: Snowball samplingYou are studying changes over time in romantic relationships. You are relatively new to the city, so you are finding it difficult to find participants yourself. You first look for interviewees within your own circle of friends and acquaintances.
A number of criteria are used for the selection:
The couple must have been together for a period of at least five years.
The couple must live together now.
The couple must live within a certain geographic area.
The couple must have examples of changes or challenges they have experienced together (e.g., long-distance, illness or loss of a loved one).
After the first interview with a couple from your own circle, you ask them if they know of another couple who may be interested in taking part in the research. Once you have interviewed the second couple, you ask again for a referral. You continue until you have conducted interviews with 30 couples, which is your desired sample size.
Quota sampling is a non-probability sampling method that relies on the non-random selection of a predetermined number or proportion of units. This is called a quota.
You first divide the population into mutually exclusive subgroups (called strata) and then recruit sample units until you reach your quota. These units share specific characteristics, determined by you prior to forming your strata.
The aim of quota sampling is to control what or who makes up your sample. Your design may:
Replicate the true composition of the population of interest
Include equal numbers of different types of respondents
Over-sample a particular type of respondent, even if population proportions differ
Example: Quota sampling Suppose you want to gauge consumer interest in a new meal kit delivery service in Washington, D.C.
Depending on your research goals, you can divide your population into several strata, such as:
Dietary preferences
Age group
Zip code
Let’s say you want to focus on dietary preferences. You divide the population into meat eaters, vegetarians, and vegans, drawing a sample of 600 people. Since the company wants to cater to all consumers, you set a quota of 200 people for each dietary group. In this way, all dietary preferences are equally represented in your research, and you can easily compare these groups.
You continue recruiting until you reach the quota of 200 participants for each subgroup.
Purposive sampling refers to a group of non-probability sampling techniques in which units are selected because they have characteristics that you need in your sample. In other words, units are selected “on purpose” in purposive sampling.
Also called judgmental sampling, this sampling method relies on the researcher’s judgment when identifying and selecting the individuals, cases, or events that can provide the best information to achieve the study’s objectives.
Convenience sampling is a non-probability sampling method where units are selected for inclusion in the sample because they are the easiest for the researcher to access.
This can be due to geographical proximity, availability at a given time, or willingness to participate in the research. Sometimes called accidental sampling, convenience sampling is a type of non-random sampling.
Example: Convenience samplingSuppose you are researching public perception towards the city of Seattle. You have determined that a sample of 100 people is sufficient to answer your research question.
To collect your data, you stand at a subway station and approach passersby, asking them whether they want to participate in your research. You continue to ask until the sample size is reached.
Note: Make sure not to confuse random selection with stopping passersby at random.
In probability (or random) sampling, random selection means that each unit has an equal chance of being selected.
In convenience sampling, stopping people at random means that not everyone has an equal chance of forming part of your sample. For instance, here you have excluded people who did not pass through that subway station on the day and time you were collecting your data.
Non-probability sampling is a sampling method that uses non-random criteria like the availability, geographical proximity, or expert knowledge of the individuals you want to research in order to answer a research question.
Non-probability sampling is used when the population parameters are either unknown or not possible to individually identify. For example, visitors to a website that doesn’t require users to create an account could form part of a non-probability sample.
Note that this type of sampling is at higher risk for research biases than probability sampling, particularly sampling bias.
In non-probability sampling, each unit in your target population does not have an equal chance of being included. Here, you can form your sample using other considerations, such as convenience or a particular characteristic.
In probability sampling, each unit in your target population must have an equal chance of selection.
Probability sampling is a sampling method that involves randomly selecting a sample, or a part of the population that you want to research. It is also sometimes called random sampling.
To qualify as being random, each research unit (e.g., person, business, or organization in your population) must have an equal chance of being selected. This is usually done through a random selection process, like a drawing.
Social desirability bias occurs when respondents give answers to questions that they believe will make them look good to others, concealing their true opinions or experiences. It often affects studies that focus on sensitive or personal topics, such as politics, drug use, or sexual behavior.
Social desirability bias is a type of response bias. Here, study participants have a tendency to answer questions in such a way as to present themselves in socially acceptable terms, or in an attempt to gain the approval of others.
It is especially likely in self-report questionnaires, but it can also affect the validity of any type of behavioral research, particularly if the participants know they’re being observed. However, there are ways to detect and reduce research bias in your research design if you know what to look for.
Example: Social desirability biasYou are conducting a study about the relationship between gambling and drug use. You ask participants to fill out a survey where you ask about their habits regarding the use of cocaine and casino gambling. These types of questions require participants to admit to attitudes, beliefs, or behaviors that may violate social norms.
Due to this, participants may downplay how often they visit casinos or use cocaine. In other words, they may give answers they consider to be socially desirable in order to project a favorable image of themselves, or to avoid being perceived negatively.
