Culture Wars
Making the Wrong Point
by Tim Lien
An article written by Alan Fram (“One in Four Read No Books Last Year,” AP), was printed in many newspapers on August 21, 2007. The Washington Post purchased it and printed it, as did many other papers. You can find the online version here.
As a great reader and a protectorate of all things bibliophilic, Jimmy Hopper alerted me to the discouraging news: Americans are reading less. But more saline was to be sprayed on the wound: the article asserted that (in general) Southerners don’t interact with anything intellectual. To boot, religious people don’t see a need to read as much. Plus, old women, minorities, poor people, Southerners, country-dwellers, and conservative Republicans have an insatiable craving for religious books. Read a section for yourself:
People from the West and Midwest are more likely to have read at least one book in the past year. Southerners who do read, however, tend to read more books, mostly religious books and romance novels, than people from other regions. Whites read more than blacks and Hispanics, and those who said they never attend religious services read nearly twice as many as those who attend frequently…Those likeliest to read religious books included older and married women, lower earners, minorities, lesser educated people, Southerners, rural residents, Republicans and conservatives.
In a recent presentation to a community group, I even cited the article. My point was this: The self-satisfied, non-religious, non-Southerner, library-card-carrier can easily mutter, “That just proves those Red State Christians are crazy and it definitely shows their growing separation from the global cultural conversation.” But, I pointed out, based on the article, (and my presupposition that all mankind craves truth) the non-religious person has to ingest “more crumbs of truth,” whereas the believer enjoys a pipeline of Truth vis a vis his respective religious service.
Hindsight: What a careless and stupid point.
Why?
I failed to read the fine print underneath the article:
The AP-Ipsos poll was conducted from August 6 to 8 and involved telephone interviews with 1,003 adults. It had a margin of sampling error of plus or minus 3 percentage points.
Where Clay Staggs might say, “I am at a loss for words,” I, on the other hand have a barrelful of words at the ready. Some, that I will choose not to use. According to the US Census Bureau, the United Sates has approximately 303,036,000 people living within its borders. 1,003 respondents is hardly representative. That would be like taking “a poll” of Bryant-Denny Stadium and asking only one person a question—if Bryant-Denny held over 3,000,000 people, that is. Out of pure Christian charity, let’s assume the AP/Ipsos Poll was methodologically responsible in the conduct of the poll. Let’s assume that they asked the proper amount of people from each state—based on population (similar to how each state gets representatives to Congress). Thus, (if conducted properly) the pollsters would have called 16 Alabama persons. (MS, 9; GA, 16;SC, 13;LA, 16, AK, 9; TN, 20) California, by contrast, would have had 121 respondents. Sixteen people represent how the state of Alabama reads? That’s not just irresponsible. That’s lunacy. How can a poll like that be the subject for an AP article and also printed in the Washington Post? I haven’t even begun to point out the inherent flaws in modern phone polls, either. What if your more “edgimicated” types subscribe to a “no-call” list? What if college grads are migrating to VOIP and cell phone usage?
Benjamin Disraeli (made popular by Mark Twain) had it right:
“There are three kinds of lies: lies, damned lies, and statistics.”
Mea Culpa. Shame on me for not reading critically.
Posted by Tim Lien at October 3, 2007 01:56 PM
Being a science-loving (worshipping) culture, almost anyone can back up almost anything by inserting a few numbers and we fall for it. The Disraeli quote is one to file away and recall anytime I’m reading “survey” oriented assertions like the one you referenced here. I was tickled by the force with which the assertions were presented in this article, just as if it were Gospel truth…shot out machine gun style.
One statistic that I’ve come across is that 62% of all statistics are incorrect. However, even though I may joke about the use of statistics (and agree that they can be used to show anything), they can be useful. I would agree, Tim, that there seem to be very few people polled in Alabama and maybe a larger sample should have been used when looking at regional trends. This poll would probably be quite accurate if it simply measured the number of books Americans read, on average (which is what a poll measures). However, I must disagree with your argument on the flaws of phone polls. First, statisticians know that certain groups, by and large, are more likely to either be on no-call lists or not have a landline. However, this does not mean that they won’t find an individual of a certain group. While it may be harder to find this type of person, it is not impossible. How these polls work is that they call and you answer some demographic questions and then the question at hand (which is the major source of the flaws in polls). They keep calling until each demographic quota has been filled and then analyze the data. Thus, they are able to match the demographics in the poll to that of the country. The poll has a margin of error of 3 percentage points. Two things can be said about this. First, the pollsters sampled enough people to get a relatively low margin of error. The larger the sample, the lower the margin of error (this comes from the law of large numbers). Now someone might say, why don’t they keep asking people until the margin of error is 2%, 1%, or even 0%. Well, that would be nice but you would have to poll more people to get the margin of error down to 2% from 3% than you would have to in order to reduce the margin of error from 4% to 3%. In other words, there are increasing costs for reducing the margin of error below a certain point. Second, a margin of error of 3 points is not bad. Most political polls have a MoE of 3.5 to 5 percentage points.
Ok, this where the poll should be dismissed: According to a Stanford researcher, small numbers used to represent large numbers are valid insamuch as they are monochromatic (i.e., one variable) Sixteen respondents from the state for the State of Alabama might be barely passable, if the only variable was reading. However, additional variables were added within specific demographics. Example: In addition to asking the respondents how much they read, they were also asked what genre/types of books they read. Thus, you would need a new pool sampling of each specific demographic, detailing types of prefererred genres within each demographic. Tacking on subsequent variables (religious participation, ethnicity, occupation, pay-scale, etc…..AND cross comparing variables such as regionality) should increase your overall poll numbers substantially. I allowed the assumption that they used proper methods, so let’s assume that is correct. What I continue to question along with others, is the validity of the “phone poll” as it simply misses segments of society that have changed due to cell/internet usage and the increasingly varied ways how Americans now communicate/absorb/relay information. What is difficult to believe is this: If they allowed for proper weighting due to population condensation, how did they fulfill regional demographcs for states like Montana and Idaho? One person is hardly representative for genre taste among all readers.