American News Media bias chart - is it reliable? How accurate is it?
I've been seeing this chart and variations on it floating around for a while, most recently on Facebook with an overlay that shows the "range" in which these media corporations present content.
From the site's methodology page, they say "During this project, nearly 1800 individual articles and TV news shows were rated by at least three analysts with different political views (left, right and center). We had 20 analysts, each analyst having analyzed about 370 articles and about 17 TV shows. Each analyst rated approximately three articles from each of the over 100 news sources available for viewing on the Chart. As a result, we have nearly 7,000 individual ratings."
That ... doesn't sound like a sufficiently large sample size to me? But I'm not a stats guy. Is this just an eye chart that makes people go "yeah, that looks about right" or is there some real insight here?
Answers ( 1 )
Yes and yes.
Yeah, anyone will look at the chart and spot their favorite extremist media outlets way over on the edges and agree that that's where they belong. Things do get a little blurry in the middle, but generally, it all looks pretty much how you'd imagine. This is partly because of the way the bias chart collects its information and the analysts who do the analyzing having their own biases, consciously or not. People can't help but be people.
And people being people will see an outlet closer to the middle than expected and immediately gain more trust, which further strengthens confirmation bias. Strangely, this chart also works on the media outlets themselves. The journalists know where they land on the chart and where they want to be, so subsequent articles might actually push their content in one direction or another. Good? Bad? I don't know. People are too people.
There's a little bit of rigor. For one thing, Ad Fontes never stops updating. They're constantly moving things around, reevaluating results, throwing out bad data, and removing loaded words from their polls and charts. You're right, though, 7000 ratings is practically zero when we're talking about something as enormous as media bias.
Which brings us to AllSides. You'll immediately notice there are some glaring inconsistencies between it and the Ad Fontes model. There are also quite a few media outlets that do line up, giving credence to their spot. I tend to prefer the AllSides model because they don't just have one team of analysts but use multiple methodologies to collect their information. It's an older research chart with "hundreds of thousands of community votes" too, not just their own research, too. More data = more accuracy. It's not as pretty as the Ad Fontes model and hasn't been as viral on social media, but I trust this one more.
Hah!
tl;dr - Yeah it's biased, but that's because we're biased.