Private: Learning Math: Data Analysis, Statistics, and Probability
Statistics As Problem Solving Part C: Bias in Measurement (20 minutes)
In This Part: Testing Your Measurement Bias
Measurement is never perfect, and we can always expect measurement errors in our data. Our goal, of course, is to keep these errors to a minimum. For this reason, we need to be aware of the various sources and causes of measurement error. See Note 3.
Random error is a nonsystematic measurement error that is beyond our control, though its effects average out over a set of measurements. For example, a scale may be properly calibrated but give inconsistent weights (sometimes too high, sometimes too low). Over repeated uses, however, the effects of these random errors average out to zero. The errors are random rather than biased: They neither understate nor overstate the actual measurement.
In contrast, measurement bias, or systematic error, favors a particular result. A measurement process is biased if it systematically overstates or understates the true value of the measurement. Consider our scale example again. If a scale is not properly calibrated, it might consistently understate weight. In this case, the measuring device — the scale — produces the bias. Human observation can also produce bias. The important thing to keep in mind is that biased measurements invariably produce unreliable results.
In any statistical investigation, we can always attribute some of the variation in data to measurement error, part of which can result from the measurement instrument itself. But human mistakes, especially recording errors (e.g., misreading a dial, incorrectly writing a number, not observing an important event, misjudging a particular behavior), can also often contribute to the variability of the measurement and thus to the results of a study.
In this video segment, Norm Abram discusses measurement error and bias in carpentry. What examples of bias, or systematic error, were present in this discussion? How could these be avoided?
You can find this segment on the session video approximately 23 minutes and 31 seconds after the Annenberg Media logo.
In this video segment, Professor Kader and participants discuss the presence of bias in two surveys about nuclear power, including the one presented in Problem B8. How did the surveys’ designs bias the studies’ findings?
You can find this segment on the session video approximately 16 minutes and 51 seconds after the Annenberg Media logo.
In This Part: Measurement Error
The difference between what individuals think they see (and how they see it) and the objective reality of what they have observed is called “visual error.” These errors in perception, which can significantly misrepresent reality, are a natural consequence of being human.
In this Interactive Activity, you will have an opportunity to see how well you make two visual judgements. On a piece of paper, try drawing each of the graphics below, but with the length of segment BC equal to the length of AB. Then measure your segments to see how accurate your drawings are.
Would you say that the errors you made in the visual judgments of the Interactive Activity were due to random error, or to bias? Why or why not?
Measurement processes may be biased due to human error. For example, error can result from our visual perceptions — the difference between what we think we see and the objective reality of what we’ve actually observed.
The Interactive Activity is based on a visual illusion known as the Muller-Lyer illusion, a phenomenon thoroughly studied by behavioral scientists. In this illusion, two arrows with different arrowheads — one pointing out and one pointing in — are placed next to each other. Although the shafts are of equal lengths, the arrow with the outgoing head looks longer than the other arrow.
If you are working with a group, recording the responses of each group member will make the bias demonstrated in the activity more clear. There are two ways you might record results. The simplest would be to record “+” if a person overstates the length and “-” if he or she understates it. Your results should show an obvious difference between pluses and minuses. With a little more effort you could measure the deviation of perceived length and actual length (e.g., by using a tape measure on your monitor) and then look at the average deviation as well as pluses and minuses.
In the second part of the Interactive Activity, you are asked to judge the length of two lines with vertical bars at each end. Judgments in this case tend to be closer to the actual length; any errors are “random.”
The errors are mainly due to the bias in the presentation. A random error is just as likely to be an overestimate as an underestimate, whereas bias is a systematic error that consistently overestimates or underestimates the true length. Most people who compare the lengths overestimate the length of BC, with its outward-pointing arrow, because of the optical illusion caused by the direction of the arrowheads. They are less likely to make this error when the arrows are replaced by perpendicular lines.
Session 1 Statistics As Problem Solving
Consider statistics as a problem-solving process and examine its four components: asking questions, collecting appropriate data, analyzing the data, and interpreting the results. This session investigates the nature of data and its potential sources of variation. Variables, bias, and random sampling are introduced.
Session 2 Data Organization and Representation
Explore different ways of representing, analyzing, and interpreting data, including line plots, frequency tables, cumulative and relative frequency tables, and bar graphs. Learn how to use intervals to describe variation in data. Learn how to determine and understand the median.
Session 3 Describing Distributions
Continue learning about organizing and grouping data in different graphs and tables. Learn how to analyze and interpret variation in data by using stem and leaf plots and histograms. Learn about relative and cumulative frequency.
Session 4 Min, Max and the Five-Number Summary
Investigate various approaches for summarizing variation in data, and learn how dividing data into groups can help provide other types of answers to statistical questions. Understand numerical and graphic representations of the minimum, the maximum, the median, and quartiles. Learn how to create a box plot.
Session 5 Variation About the Mean
Explore the concept of the mean and how variation in data can be described relative to the mean. Concepts include fair and unfair allocations, and how to measure variation about the mean.
Session 6 Designing Experiments
Examine how to collect and compare data from observational and experimental studies, and learn how to set up your own experimental studies.
Session 7 Bivariate Data and Analysis
Analyze bivariate data and understand the concepts of association and co-variation between two quantitative variables. Explore scatter plots, the least squares line, and modeling linear relationships.
Session 8 Probability
Investigate some basic concepts of probability and the relationship between statistics and probability. Learn about random events, games of chance, mathematical and experimental probability, tree diagrams, and the binomial probability model.
Session 9 Random Sampling and Estimation
Learn how to select a random sample and use it to estimate characteristics of an entire population. Learn how to describe variation in estimates, and the effect of sample size on an estimate's accuracy.
Session 10 Classroom Case Studies, Grades K-2
Explore how the concepts developed in this course can be applied through a case study of a K-2 teacher, Ellen Sabanosh, a former course participant who has adapted her new knowledge to her classroom.
Session 11 Classroom Case Studies, Grades 3-5
Explore how the concepts developed in this course can be applied through case studies of a grade 3-5 teacher, Suzanne L'Esperance and grade 6-8 teacher, Paul Snowden, both former course participants who have adapted their new knowledge to their classrooms.