Join us for conversations that inspire, recognize, and encourage innovation and best practices in the education profession.
Available on Apple Podcasts, Spotify, Google Podcasts, and more.
Private: Learning Math: Data Analysis, Statistics, and Probability
Statistics As Problem Solving Part C: Bias in Measurement (20 minutes)
In This Part: Testing Your Measurement Bias
Measurement is never perfect, and we can always expect measurement errors in our data. Our goal, of course, is to keep these errors to a minimum. For this reason, we need to be aware of the various sources and causes of measurement error. See Note 3.
Random error is a nonsystematic measurement error that is beyond our control, though its effects average out over a set of measurements. For example, a scale may be properly calibrated but give inconsistent weights (sometimes too high, sometimes too low). Over repeated uses, however, the effects of these random errors average out to zero. The errors are random rather than biased: They neither understate nor overstate the actual measurement.
In contrast, measurement bias, or systematic error, favors a particular result. A measurement process is biased if it systematically overstates or understates the true value of the measurement. Consider our scale example again. If a scale is not properly calibrated, it might consistently understate weight. In this case, the measuring device — the scale — produces the bias. Human observation can also produce bias. The important thing to keep in mind is that biased measurements invariably produce unreliable results.
In any statistical investigation, we can always attribute some of the variation in data to measurement error, part of which can result from the measurement instrument itself. But human mistakes, especially recording errors (e.g., misreading a dial, incorrectly writing a number, not observing an important event, misjudging a particular behavior), can also often contribute to the variability of the measurement and thus to the results of a study.
Video Segment
In this video segment, Norm Abram discusses measurement error and bias in carpentry. What examples of bias, or systematic error, were present in this discussion? How could these be avoided?
You can find this segment on the session video approximately 23 minutes and 31 seconds after the Annenberg Media logo.
Video Segment
In this video segment, Professor Kader and participants discuss the presence of bias in two surveys about nuclear power, including the one presented in Problem B8. How did the surveys’ designs bias the studies’ findings?
You can find this segment on the session video approximately 16 minutes and 51 seconds after the Annenberg Media logo.
In This Part: Measurement Error
The difference between what individuals think they see (and how they see it) and the objective reality of what they have observed is called “visual error.” These errors in perception, which can significantly misrepresent reality, are a natural consequence of being human.
In this Interactive Activity, you will have an opportunity to see how well you make two visual judgements. On a piece of paper, try drawing each of the graphics below, but with the length of segment BC equal to the length of AB. Then measure your segments to see how accurate your drawings are.
Problem C1
Would you say that the errors you made in the visual judgments of the Interactive Activity were due to random error, or to bias? Why or why not?
Note
Measurement processes may be biased due to human error. For example, error can result from our visual perceptions — the difference between what we think we see and the objective reality of what we’ve actually observed.
The Interactive Activity is based on a visual illusion known as the Muller-Lyer illusion, a phenomenon thoroughly studied by behavioral scientists. In this illusion, two arrows with different arrowheads — one pointing out and one pointing in — are placed next to each other. Although the shafts are of equal lengths, the arrow with the outgoing head looks longer than the other arrow.
If you are working with a group, recording the responses of each group member will make the bias demonstrated in the activity more clear. There are two ways you might record results. The simplest would be to record “+” if a person overstates the length and “-” if he or she understates it. Your results should show an obvious difference between pluses and minuses. With a little more effort you could measure the deviation of perceived length and actual length (e.g., by using a tape measure on your monitor) and then look at the average deviation as well as pluses and minuses.
In the second part of the Interactive Activity, you are asked to judge the length of two lines with vertical bars at each end. Judgments in this case tend to be closer to the actual length; any errors are “random.”
Solutions
Problem C1
The errors are mainly due to the bias in the presentation. A random error is just as likely to be an overestimate as an underestimate, whereas bias is a systematic error that consistently overestimates or underestimates the true length. Most people who compare the lengths overestimate the length of BC, with its outward-pointing arrow, because of the optical illusion caused by the direction of the arrowheads. They are less likely to make this error when the arrows are replaced by perpendicular lines.
