PolitiFact Truth Index, Part 3: Breakdown by State. Selection Bias?

My attempt with the PolitiFact Truth Index is to sort out some objective measure of the truthiness of statements made by politicians, journalists, activists and advocacy groups. I have shown that the statements made by Republicans tend to be rated false by PolitiFact and the statements made by Democrats tend to be rated true by PolitiFact. I have jumped to the conclusion that this is because Republicans lie more than Democrats do. But the whole process of rating statements on a six-point truth scale necessarily involves a selection bias, a subjective process that picks which statements to select for analysis and which to ignore.

There is a universe of statements out there composed of verifiable facts that are significant to some swath of Americans, newsworthy in some way, and likely to be repeated by others. PolitiFact sees its Truth-o-Meter™ as a somewhat-objective measure of this universe of statements, like dunking a test tube into a lake and analyzing the contents under a microscope back at the lab. This would be highly useful to settle political debates with scientific evidence about the relative truthiness of each party’s statements. But fact-checking isn’t like sampling a lake: each individual would probably have a different definition about what constitutes a newsworthy or significant statement. Even if one assumes that PolitiFact can objectively pass judgment on the truthiness of any given statement, it’s hard to make the case that their statements are fully representative of all political discourse. The lake could be as big as an ocean, and your test tube may not be a representative sample of the water in it.

This is a point made over and over again on the many sites that criticize PolitiFact, sites that are written mostly from a conservative perspective. One of the most thorough smackdowns of PolitiFact since the beginning of this year is the blog PolitiFact Bias whose writers have a beef not only with the selection bias, but also with the rating process, the final rating itself, and the selection of the “lie of the year”. In one post, blogger Bryan White tries to make an objective case for partisan bias (“bolstered by methods of science”) by compiling a list of errors in PolitiFact ratings going back to 2008 and pointing out that the overwhelming majority of them are unfavorable to Republicans or favorable to Democrats. But how does he determine which PolitiFact ratings are errors? He used his own (conservative-leaning) judgment, fraught with even more partisan bias than PolitiFact.

The fact is that it’s pretty easy to claim that PolitiFact has a partisan bias based on anecdotal evidence, but it’s pretty much impossible to prove such a claim unless PolitiFact and a bunch of other fact checking websites analyze a given control set of statements that we the public could use to determine who is more biased, who is more nitpicky, etc. (actually, this would be a great idea).

But until that happens, one can only compare PolitiFact with itself. Specifically with the affiliated state sites that are run by different newspaper editorial boards and may exhibit a degree of independence from the national PolitiFact site. I was interested in knowing which newspapers’ editors exhibited the most political bias towards or against one party or the other. So I averaged the PolitiFact Truth Index across each state by political party and came up with the following results.



PolitiFact Texas and PolitiFact Ohio both exhibited an average Truth Index value above one standard deviation for Democrats, while PolitiFact Wisconsin showed a negative value for its Democrats (the only state with more Democratic lies than truths since the end of February). It could mean that local and state Democratic politicians in Texas and Ohio really do tell the truth more often than their counterparts in other states, but it could also be a measure of partisan bias towards Democrats by the editorial board of the Cleveland Plain Dealer and the Austin American Statesman.



PolitiFact Georgia and PolitiFact Virginia both exhibited an average Truth Index value above one standard deviation for Republicans, (the only states where Republicans have made more true statements than false ones) while Oregon and Wisconsin’s averages are lower than one standard deviation. Again, this could mean that Georgia’s and Virginia’s Republicans are more honest than everywhere else in the nation. But it could also mean that the Atlanta Journal Constitution and the Richmond Times-Dispatch employ fact-checkers with a Republican bias.

I thought it was interesting that both Wisconsin’s Democratic statements and Republican statements averaged below one standard deviation compared to statements made by fellow Democrats and Republicans in other states. I think this is at least partially due to the heated rhetoric due to the labor fight in that state, and specifically to the local and judicial elections that followed. As I published in Part 2, statements about elections (workings of government) and personal attacks tend to be rated false across the board, no matter which party makes the statement.

How does the national version of PolitiFact rank against the states in terms of the Truth Index?



For Democrats, PolitiFact National averages very close to the state average. For Republicans, PolitiFact National scores much lower than the state average. Since the National branch of PolitiFact tends to analyze statements with a national policy scope more than the state branches do, one can look at this fact a couple of different ways. One is that Republicans with a national scope lie more often than their state and local counterparts do. Or, PolitiFact National has an anti-Republican partisan bias. Or perhaps it could be due to some other factor. Maybe Republican pants-on-fire ratings drive more traffic to the website through links on Facebook and liberal blogs, and PolitiFact’s editors could be motivated to sample more of these statements than other types.

The debate about PolitiFact won’t subside, but I do think that the website is a useful mythbusting tool even if it does have a selection bias.

Next: Politicians versus political commentators

Politifact Truth Index, Part 2: Breakdown by Subject

Republicans lie. On average, more than Democrats. At least according to the statements analyzed by PolitiFact.com. But do they lie at a uniform rate across any given policy statement? Are there some subject matters where Democrats are more likely to stretch the truth than Republicans are? I had to find out.

I categorized each statement into a fairly broad policy subject based not only on the words of the statement, but also on the point that the statement is trying to make. If the politician is saying something about Medicare, for instance, the intent of the comment might be to urge hospitals to improve care (in which case the statement would be in the Health Care category), or the intent might be to reduce the cost to the government (in which case the statement would be placed in the Government Spending category).

My PolitiFact database breaks each statement down into one of twenty subjects. The distribution of these subjects is shown in the pie chart below.



Recently the biggest subject of discussion has been issues related to government spending as both sides have waged budget battles over the amount of federal money distributed to government programs. Similar subjects have also been popular such as debt/deficit, taxes and the economy. (If a statement refers to both taxes and spending, it is counted as a statement about the debt/deficit).

Labor issues are also popular subjects in this database due to the recent protests in Wisconsin (and also due to the fact that Wisconsin is one of the eight states where PolitiFact partners with a local newspaper for state and local fact-checking). Labor issues incorporate collective bargaining issues, statements made about the benefits of public employees, and statements made about unions.

Which types of statements are usually true across all parties? Government spending issues. Both Republicans and Democrats score highly in the PolitiFact Truth Index when it comes to issues relating to government spending, which is great because government spending issues are the largest category of statements. Politicians can be generally trusted when it comes to statements made about government spending.

Second Amendment issues also score highly across both parties, but only nine statements (five by Republicans, three by Democrats) have been recorded in my database. The dataset isn't high, so the numbers are subject to wild changes. Public safety issues (crime statistics mostly) also score highly, although there has only been one identifiably Republican statement made in regard to this subject. Most public safety statements are made by local police officers or other non-partisan figures.

Which types of statements are usually lies? Personal attacks or boasts are more often false than true no matter which party makes the claim. Personal attacks are most often made in the heat of campaigns and encompass claims about an opponent's background or some statement he or she made in the past. It is the category of all the birther claims, but also of the accusations that certain politicians said they wanted to get rid of Medicare or other government programs.

Statements about the workings of government are also usually lies across all parties. This category includes statements about elections, including polls, demography, gerrymandering, vote counting and allegations of fraud; and it also includes statements about congressional rules and bureaucracy. Don't trust a politician when he/she says something like a recent poll shows him/her statistically tied with Obama for the 2012 presidency.

Which types of statements show Republicans lying as Democrats tell the truth? Labor issues, taxes, Obamacare, social issues, and environmental issues. Democrats can point to these subjects as areas where Democrats are on the right side of the truth and Republicans are on the wrong side, usually.

Are there any subjects where Democrats lie as Republicans tell the truth? So far just one: transportation. And this cannot be relied on as there have only been two Republican and four Democratic statements made.

Here's the chart:



Here's the main point - Republicans do score better than Democrats in some sizeable subjects, namely government spending and education. But Democrats do better in almost everything else: the economy, labor issues, the workings of government, debt/deficit, taxes, personal attacks, Obamacare, social issues, energy issues, environmental issues, and health care issues. The difference is vast when it comes to social issues (abortion, gay rights), environmental issues, Obamacare and taxes. When it comes to these policy issues, one can not only not trust Republicans, but also one can usually trust Democrats.

Next: PolitiFact's state newspaper affiliates: are there biases?

The PolitiFact Truth Index

In December, I published an analysis of all the statements analyzed by PolitiFact.com, the preeminent fact-checking website. I looked at all the statements rated by the PolitiFact national editorial team and found that Republicans lied a lot more than Democrats. I found that almost 70% of the statements rated by PolitiFact.com as "false" or "pants-on-fire" since the 2008 election came from Republicans.

But my interest in PolitiFact ratings didn't end there. I wondered if there were more lies now than there used to be. I wondered what subjects get lied about the most. I wondered if Senators lied more than members of the House of Representatives. I also wondered what the ratings were like for all the different state newspapers that PolitiFact partners with to see if there were any biases and to see if local Republicans seemed to lie as much as their national counterparts.

So I started a database. I recorded every statement analyzed by PolitiFact National and every statement analyzed by each of its eight partner state newspapers since the beginning of March. Then I looked at when the statement was made, what subject the statement was about, what occupation the statement-maker had, how the statement was delivered, and to what audience the statement was delivered, among other characteristics.

The information I have found is astounding.

First of all, in order to interpret the "truthiness" of each statement, I had to come up with a system that would convert the rankings into a numeric system. PolitiFact assigns one of six rankings to each of their statements: "True", "Mostly True", "Half True", "Barely True", "False", and "Pants on Fire". How they assign these rankings is detailed on their website. True and Mostly True statements are not misleading, Half True and Barely True statements occur when there is a nugget of truth inside some important omitted details, and False and Pants on Fire statements are straight up lies.

I tried a couple of different ways of counting up truth points, but the one that seems to reveal the most is a simple +1 / 0 / -1 system.

- True or Mostly True statements get +1
- False or Pants on Fire statements get -1
- Half True or Barely True statements get 0

With this system, if an individual or organization has a positive average, that means that individual or organization tells more truths than lies. Likewise, if there is a negative average, that means that an individual lies more than he/she/it/they tell the truth.

With that in mind, here is the latest graph showing the weekly breakdown of the statements by political party. The four-week average is less volatile than the weekly average, so I have included it as well.



As you can see, the trend of Democrats telling the truth more often than Republicans is still going strong. Check out the last numbers for this week. Of the last nine statements of Donald Trump's that are in this database, seven of them have been lies. With this week's birth certificate release and subsequent press conference, the Donald is driving the Republican numbers down.

This will be an ongoing project for me, and I hope to continue this project through the 2012 elections at the very least. I hope to update this graph every week.

Next: The political issues that get lied about the most.

Senator Brian Bingman = Admiral Ackbar

The federal health care bill, lovingly referred to as Obamacare by most Republicans, stipulates that every state needs to set up a health insurance exchange where consumers can access information about cost and coverage of every insurance carrier in the state. It is a free-market idea, one that makes comparing different plans easier and expands consumer choice. But a fully functioning exchange would require some additional investment in information technology infrastructure, which is why the federal government is giving away millions of dollars worth of grants to the states to help them set their exchanges up.

Enter Senator Brian Bingman (R-Sapulpa), the president pro-tem of the Oklahoma Senate. He told reporters that a bill passed by the state House of Representatives that helps establish a state health insurance exchange would not be considered by the Senate. "We're concerned about actually falling into the trap ... of adopting the Obamacare, which is really what we're against," said the Senator, who last month released a joint statement with Governor Mary Fallin and House Speaker Kris Steele (Republicans all) saying he supported the federal grant. Steele and Fallin both still support getting free money from the federal government to better the lives of their fellow Oklahomans. And it's not like these people could ever be described as liberals.

"My hope is that the federal health care bill will either be ruled unconstitutional by the Supreme Court or that Congress might take up legislation to repeal it and replace it with other free-market health care reform initiatives," Fallin said. "But whether that happens or not, the federal health care bill is still the law of the land."

If Oklahoma fails to take action to establish its own health insurance exchange, Fallin warned the federal government will impose its own system on the state.

"I don't think it's helpful to the state of Oklahoma not to have a plan in place," Fallin said. "I also think it puts us on a dangerous path of the Obama administration being able to come in and force a federal exchange on our state."

So Fallin fears a trap just as much as Bingman does. She is just responding a little bit more proactively than he is.

Oklahomans passed a bill last November allowing Oklahoma to opt out of any federal health care law due to a fear of government intrusion (and therefore socialism). The Senate president pro-tem thinks that accepting money from the federal government would trap the state into accepting government intrusion (and therefore socialism). The Governor thinks that not creating an exchange would lead to government intrusion (and therefore socialism).

Our governor, our state leaders, and the majority of state voters are all a bunch of Admiral Ackbars, too blinded by right-wing dogma to see that the whole point of Obamacare is to offer choices to consumers to get us a step closer to universal health insurance coverage.

Lies, and the Lying Liars ...

"Reality has a well-known liberal bias" - Stephen Colbert

So I was thinking about that recent study that found that watching Fox News makes you stupid, as the headlines put it. And it got me thinking about how it's a really insulting thing to say that those who watch Fox News are actually idiots. I mean, the study is actually about misinformation and the 2010 election (that's actually its title) and not just about Fox News viewers. Sure, 63% of everyday Fox News viewers thought that it was unclear where the president was born, but that might be because 64% of Republicans share the same view. Fox News may lie directly to their customers, but they may also just be broadcasting and repeating Republican lies.

So this got me thinking about Republican lies. I'm pretty sure there are a lot of them. Like, a whole lot. It seems like Republicans lie more often than do Democrats. But I recognize that I am a liberal and I am a Democrat, and so when I hear about statements made by conservative Republicans, I notice the lies more prominently than your average American. So I decided to go to politifact.com (the non-partisan fact-checking project of the St. Petersburg Times that won a Pulitzer Prize in 2009) and count up all the lies to see who lies more: Republicans or Democrats.

The editors of Politifact.com spend a lot of time sorting out controversial statements by how much truth they contain, always recognizing that, in politics, truth is never black-and-white. So they place statements into the following truth categories: "true", "mostly true", "half true", "barely true", "false", and "pants on fire."

Politifact.com has been around since the early stages of the 2008 presidential campaign, so that meant there were a lot of statements to count. I counted each statement made by a human or organization, and I did not count any chain emails or statements made by "bloggers". Journalists and pundits counted towards partisan totals only if it was clear which party they sympathized with (i.e. Rachel Maddow = Democrat, Glenn Beck = Republican). Joe Liebermann counted as a Democrat, even when he was bashing Obama; and Arlen Spector counted as a Democrat only if he made the statement after he switched parties. Bernie Sanders and Ralph Nader were put in the blue category: Bob Barr and the Cato Institute in the red.

There is surprising parity in the number of statements analyzed by Politifact.com. Politifact.com has chosen an almost equal number of Democratic and Republican statements over the years.

But of those statements, Republicans have told more lies than Democrats. And more Democratic statements have been rated "true" and "mostly true" than Republican statements.

A statement made by a Democrat is most likely going to be in the "true" category (25%). A statement made by a Republican is most likely going to be in the "false" category (23%).

A true or mostly true statement is more likely to come from a Democrat than a Republican by a margin of 54% to 44%.

And throughout Barack Obama's presidency and all the policy debates about health care, the stimulus and everything else, the margin has been fairly consistent: 54% for Democrats to 46% for Republicans.

However, a "false" or "pants-on-fire" statement is much more likely to come from a Republican source than a Democratic source.

And the margin has only increased during Obama's tenure as president. Nearly 7 out of 10 lies categorized by Politifact.com have been Republican lies. Think back to the policy debates during this time. The health care debate (death panels), the stimulus package (no jobs created), all the tax policy stuff (tax hikes); of all the lies that were produced during this time, almost 70% of them came from Republicans.

When one isolates the worst, most egregious lies, the "pants-on-fire" lies, one finds that the liars are overwhelmingly Republican.

And 87% of the worst, foulest, stankiest lies of the past two years originate from members of the Republican Party.

My Letter to the Senators: An Exercise in Futility

Below is my letter to Oklahoma's U.S. Senators Tom Coburn and Jim Inhofe. In it, I try to lay out a conservative argument in favor of the DREAM Act. I probably should have made the word "taxpayers" bold and capitalized.

_____________________________


Dear Senator (Coburn / Inhofe),

I am writing you today to please consider voting in favor of the DREAM Act. You may have heard many arguments both for and against the proposed bill, as have I, but one argument that has not been given a lot of play is this one:

We taxpayers have been subsidizing the education of immigrants without status for years in the public education system. The whole point of public financing of education is that we all share a common belief that an educated population is an economically productive population; it is worth taxpayer funding for the betterment of the common good. And we hear stories all the time about the many immigrants without status in our society who are honor students destined for success in college and beyond.

But our current immigration policy mandates that we deport these students right at the time in their lives when our nation would start to be able to reap the benefits of the education we taxpayers have financed for so long.

How can it make sense to give away to some other country the future economic power generated by these individuals? Why would we want to prevent graduates of our colleges and universities from using their knowledge to form a better society right here in the United States?

Please don’t let trillions of dollars of taxable income go to waste. Please don’t exacerbate “brain drain” by forcing new graduates and young professionals to leave the country.

Please vote in favor of the DREAM Act.

Don't Believe the Pledge: Social Issues Still Dominate the Republican Agenda

One of the sites that I visit several times a day is the Slatest, Slate.com's picks for the dozen most important or newsworthy items at any given time. Today, this morning, I went there and discovered that item number one was this story about how all Senate Republicans have now signed a pledge stating that they will block all Democratic legislation unrelated to tax cuts or government spending in the current "lame duck" session of congress, according to the AP.

To an idiot, this pledge would seem to fit right in line with the Republican Party's current focus on economic issues rather than social issues. Ever since the economy tanked and the Tea Party gained national prominence, Republicans have made it seem like they are more eager to talk about taxing and spending rather than divisive social issues. But the real idea behind the Republicans' pledge isn't a call for congress to focus more on the economy: it's a call to stop progress that Democrats are trying to make towards ending discriminatory government policies.

There is not and has never been much of a threat that congress will refuse to act on the sunsetting of the Bush tax cuts. News agencies and the public have been talking about this for months, even since before the election, and every move in the negotiations between Democrats and Republicans has been high-profile. And also, congress is not full of idiots: our elected senators and representatives do have the mental ability to discuss more than one issue per session of congress. So the whole point of the pledge is that Republicans have vowed to block the Democrats' two important social policy bills: the DREAM Act and repeal of Don't Ask Don't Tell. The whole point of this pledge therefore is not to advance discussion on the economy, but rather to advance a social agenda that discriminates against gay people in the military and perpetuates the injustice of deporting non-citizen U.S. college grads and military servicepeople.

The hypocrisy doesn't stop there.

I noticed some of the other items on the Slatest were also about Republicans. The number 4 item was a story about an art exhibit at the Smithsonian that was pulled because Republican lawmakers were offended at its content. The number 10 item was a story about how the chief of staff of incoming House Speaker John Boehner had a meeting with a rabid right-wing anti-abortion activist named Randall Terry, who has called for the murder of abortion doctors. And the number 6 item was a story about Julian Assange, who Republican Representative Peter King has labeled as a terrorist.

Censorship of art, abortion activism, calling people terrorists: all of these things are social issues, and all of these things are foremost on the minds of prominent Republicans.

You would think the newly fiscally-focused GOP would be able to put aside their social views and seriously discuss a measure that would generate $3.6 trillion dollars over the next 40 years of new taxable income for America. One would think Republicans would be open to hearing about ways the military could retain thousands of honorable soldiers while simultaneously increasing the number of college campuses in which ROTC groups will be allowed to operate, not to mention the cost and efficiency savings that would come from not having to investigate the personal lives of soldiers. And one would think that any politician would think twice about blocking measures that have the support of 58% (DADT repeal) and 70% (DREAM Act) of Americans.

By signing the pledge of non-cooperation, Senate Republicans have made a choice. They have chosen to allow social issues to continue to dominate their party, even if those social policies would create significant economic good. They've placed social issues above economic issues, even as they claim that they are doing the exact opposite.