How to Choose the Best Sampling Method

The best sampling method is the sampling method that most effectively meets the particular goals of the study in question.

The effectiveness of a sampling method depends on many factors. Because these factors interact in complex ways, the "best" sampling method is seldom obvious. Good researchers use the following strategy to identify the best sampling method.

  • List the research goals (usually some combination of accuracy, precision, and/or cost).
  • Identify potential sampling methods that might effectively achieve those goals.
  • Test the ability of each method to achieve each goal.
  • Choose the method that does the best job of achieving the goals.

The next section presents an example that illustrates this strategy.

Sample Size Calculator

The computations involved in comparing different sampling methods can be complex and time-consuming. Stat Trek's Sample Size Calculator can help. The calculator computes survey precision, sample size requirements, costs, etc., allowing you to compare alternative sampling methods quickly, easily, and accurately. The calculator creates a summary report that lists key findings and documents analytical techniques. And the calculator is free. You can find the Sample Size Calculator in Stat Trek's main menu under the Stat Tools tab. Or you can tap the button below.

Sample Size Calculator

How to Choose the Best Sampling Method

In this section, we illustrate how to choose the best sampling method by working through a sample problem. Here is the problem:

Problem Statement

At the end of every school year, the state administers a reading test to a sample of third graders. The school system has 20,000 third graders, half boys and half girls. There are 1000 third-grade classes, each with 20 students.

The maximum budget for this research is $3600. The only expense is the cost to proctor each test session. This amounts to $100 per session.

The purpose of the study is to estimate the reading proficiency of third graders, based on sample data. School administrators want to maximize the precision of this estimate without exceeding the $3600 budget. What sampling method should they use?

As noted earlier, finding the "best" sampling method is a four-step process. We work through each step below.

  • List goals. This study has two main goals: (1) maximize precision and (2) stay within budget.
  • Identify potential sampling methods. This tutorial has covered three basic sampling methods - simple random sampling, stratified sampling, and cluster sampling. In addition, we've described some variations on the basic methods (e.g., proportionate vs. disproportionate stratification, one-stage vs. two-stage cluster sampling, sampling with replacement vs. sampling without replacement).

    Because one of the main goals is to maximize precision, we can eliminate some of these alternatives. Sampling without replacement always provides equal or better precision than sampling with replacement, so we will focus only on sampling without replacement. Also, as long as the same clusters are sampled, one-stage cluster sampling always provides equal or better precision than two-stage cluster sampling, so we will focus only on one-stage cluster sampling. (Note: For cluster sampling in this example, the cost is the same whether we sample all students or only some students from a particular cluster; so in this example, two-stage sampling offers no cost advantage over one-stage sampling.)

    This leaves us with four potential sampling methods - simple random sampling, proportionate stratified sampling, disproportionate stratified sampling, and one-stage cluster sampling. Each of these uses sampling without replacement. Because of the need to maximize precision, we will use Neyman allocation with our disproportionate stratified sample.
  • Test methods. A key part of the analysis is to test the ability of each potential sampling method to satisfy the research goals. Specifically, we will want to know the level of precision and the cost associated with each potential method. For our test, we use the standard error to measure precision. The smaller the standard error, the greater the precision.

    To avoid getting bogged down in the computational details of the analysis, we will use results from sample problems that have appeared in previous lessons. Those results are summarized in the table below. (To review the analyses that produced this output, click the "See analysis" links in the last column of the table.)

    Cost Standard error Sample size Analytical details
    Simple random sampling
    $3600 1.66 36 See analysis
    Proportionate stratified sampling
    $3600 1.45 36 See analysis
    Disproportionate stratified sampling
    $3600 1.41 36 See analysis
    One-stage cluster sampling
    $3600 1.10 720 See analysis
    Because the budget is $3600 and because each test session costs $100 (for the proctor), there can be at most 36 test sessions. For the first three methods, students in the sample might come from 36 different schools, which would mean that each test session could have only one student. Thus, for simple random sampling and stratified sampling, the sample size might be only 36 students. For cluster sampling, in contrast, each of the 36 test sessions will have a full class of 20 students; so the sample size will be 36 * 20 = 720 students.
  • Choose best method. In this example, the cost of each sampling method is identical, so none of the methods has an advantage on cost. However, the methods do differ with respect to precision (as measured by standard error). Cluster sampling provides the most precision (i.e., the smallest standard error); so cluster sampling is the best method.

Although cluster sampling was "best" in this example, it may not be the best solution in other situations. Other sampling methods may be best in other situations. Use the four-step process described above to determine which method is best in any situation.