Discussion about this post

User's avatar
Anne Johnson's avatar

And having read the study, I am even more skeptical. First of all, the differences are miniscule, despite the authors' claims of statistical significance. I work with pharmaceutical clinical studies all the time, and I caution my coworkers that there is a big difference between a statistically significant outcome and a clinically significant outcome.

Second, the manipulation of the voice samples was extensive--to the extent that no words could be discerned. And the snippets were taken after the initial encounter of driver with officer--meaning that the demeanor of the driver when the officer approached the car is not taken into account--only the officer's tone (no words) after the first impression.

Finally, 47% of the officers were white. This means that 53% of the supposed "racism" comes from non-whites. That is something to chew on, isn't it?

Expand full comment
JimInNashville's avatar

I just subscribed to this forum, and this is my first post here.

As a long-time reader/listener of John McWhorter, and as one who admires him and his writing, and has a very extensive academic/professional background in social science statistics, I was struck by the headline of this article. I felt compelled to sign up and offer a mild rebuttal. In my view, John's conclusion is correct, but his evaluation of the article is way off the mark.

The article John cites "The Thin Blue Waveform" (hereafter referred to as TTBW21) does not *begin* to show "racism," at least in the classic sense. Do to its methodological shortcomings, it is difficult to say with precision exactly what TTBW21 does show. John's reaction to the article demonstrates the very human tendency to overgeneralize a set of very minor (possibly meaningless) results into something of genuine significance.

Let's take a close look at the methodology. In any such study, the key questions are (a) what is the population, (b) what is the sampling procedure, (c) what measures were taken, (d) what procedures were used to protect against experimenter bias, and (e) how strong were the results. I'll be selective in analyzing these.

Start with the research population. This can be crucial in a study of this type. How was the population selected, and what can we say about it?

Here we have to be doubly careful. There is a tendency to snooze a bit when reading an article's "method section," especially when the topic is one of great substantive interest. It is easy to fall into "benefit of the doubt reading," i.e., the reader assumes without evidence that the experimenter did the correct thing. Classic examples: (a) the reader assumes random sampling when it did not in fact occur, and (b) the reader assumes that the selection of experimental subjects ("participants") was done by people who were unaware of the purpose of the experiment.

Quite often, when something is not stated in a social psychology article, it is because the experimenters did *not* do the right thing, often because the right thing takes more work. True random sampling by workers who are unaware of the purpose of the experiment takes more work.

On page 3 of TTBW21, we are given virtually no information about the research population. They say they grabbed conversations "in a medium-sized U.S. city." Note, we have no idea what the racial mixture of this unnamed city is. That turns out to be important.

The authors started with a pool of conversations from "routine" traffic stops not resulting in an arrest." There were 180 White males and 433 Black males stopped. 70.6% of the stops were Black males. What do those numbers tell us? Without knowing more about the city and its population, we're completely at sea. Suppose, for example, this is a city that is 70% Black, 30% White. Then there is a kind of proportionality. On the other hand, suppose this city is 70% White and only 30% Black. Then Black drivers are heavily overrepresented. We are told the racial composition of the police officers. Less than half are White, and less than 1 in 5 is Black.

Suppose Black drivers are overrepresented. Is it because they were, on average, driving faster than White male drivers? (Some previous studies have found this.) If a White driver is doing 69 in a 55 mph zone, and the Black driver is doing 90, perhaps the greater tension in the officer's voice has nothing to do with race.

All we know is that 70% of the drivers stopped were Black. We are then told that a subset of the 180 White and 433 Black conversations were selected to be in the study. 100 Black and 100 White. WE ARE NOT TOLD HOW THEY WERE SELECTED. The temptation is to believe the sampling was random. Suppose it wasn't? Anyone with experience in the social sciences can tell you myriad ways that the person doing the selecting might have biased the study, unless certain precautions

were taken.

What about the measures? There were a couple of items using a 6 point Likert scale, ranging from 1 to 6. (very cold to very warm). So a mean score of 3.5 would be right in the middle, i.e., lukewarm.

What about the results? From a practical standpoint, the results are miniscule! The average response to a Black driver is judged as lukewarm (about 3.5) and the average response to a White driver is judges slightly warmer than lukewarm (about 3.7). This is judged statistically significant. If the study had been done properly, we would still conclude that this is not microaggression, it is nanoaggression.

Unfortunately, this study eliminates all information about what took place prior to the interactions between police and drivers. What were the drivers stopped for? How did they respond when first approached? Without this information, here is what we can conclude:

A truly trivial difference in "prosody" was found in a study that did not control for severity of offense or race of officer. On average, police respond with "lukewarm" prosody to both White and Black drivers, slightly warmer toward Whites. Without the proper controls, we have no idea what this means.

Expand full comment
240 more comments...

No posts