Friday, April 29, 2011

The PolitiFact Truth Index

In December, I published an analysis of all the statements analyzed by, the preeminent fact-checking website. I looked at all the statements rated by the PolitiFact national editorial team and found that Republicans lied a lot more than Democrats. I found that almost 70% of the statements rated by as "false" or "pants-on-fire" since the 2008 election came from Republicans.

But my interest in PolitiFact ratings didn't end there. I wondered if there were more lies now than there used to be. I wondered what subjects get lied about the most. I wondered if Senators lied more than members of the House of Representatives. I also wondered what the ratings were like for all the different state newspapers that PolitiFact partners with to see if there were any biases and to see if local Republicans seemed to lie as much as their national counterparts.

So I started a database. I recorded every statement analyzed by PolitiFact National and every statement analyzed by each of its eight partner state newspapers since the beginning of March. Then I looked at when the statement was made, what subject the statement was about, what occupation the statement-maker had, how the statement was delivered, and to what audience the statement was delivered, among other characteristics.

The information I have found is astounding.

First of all, in order to interpret the "truthiness" of each statement, I had to come up with a system that would convert the rankings into a numeric system. PolitiFact assigns one of six rankings to each of their statements: "True", "Mostly True", "Half True", "Barely True", "False", and "Pants on Fire". How they assign these rankings is detailed on their website. True and Mostly True statements are not misleading, Half True and Barely True statements occur when there is a nugget of truth inside some important omitted details, and False and Pants on Fire statements are straight up lies.

I tried a couple of different ways of counting up truth points, but the one that seems to reveal the most is a simple +1 / 0 / -1 system.

- True or Mostly True statements get +1
- False or Pants on Fire statements get -1
- Half True or Barely True statements get 0

With this system, if an individual or organization has a positive average, that means that individual or organization tells more truths than lies. Likewise, if there is a negative average, that means that an individual lies more than he/she/it/they tell the truth.

With that in mind, here is the latest graph showing the weekly breakdown of the statements by political party. The four-week average is less volatile than the weekly average, so I have included it as well.

As you can see, the trend of Democrats telling the truth more often than Republicans is still going strong. Check out the last numbers for this week. Of the last nine statements of Donald Trump's that are in this database, seven of them have been lies. With this week's birth certificate release and subsequent press conference, the Donald is driving the Republican numbers down.

This will be an ongoing project for me, and I hope to continue this project through the 2012 elections at the very least. I hope to update this graph every week.

Next: The political issues that get lied about the most.

1 comment:

Karen S. said...

Hi Steve-- I do the same type of PolitiFact analysis as you, altho I am not yet into standard deviations. Here is my site:
I also look at Bryan White's blog and respond to his criticisms. He ignores me entirely. Since PF now has a Truth Index scoring, I compare PF analyses to other fact-checking sites. But I enjoyed reading your blog!