Thursday, November 01, 2012

Pollster House Effects

I've talked about house effects against my median poll spread in brief before, but I haven't revealed my calculated list in full until now.  But first, some methodology.

The median poll spread is the difference between the median Obama number and the median Romney number.  These numbers may come from two separate polls; it is not the most central spread of any single poll.  When I first started keeping track of polls in September, I didn't really think much about how I'd choose the polls to put into my database.  I just used whatever polls were in the RealClearPolitics database, though I did not include polls marked with the partisan tag (except for PPP polls).  As I got into it more, I learned that RCP didn't always include PPP polls if they weren't independent; that is, if they were produced for left-leaning organizations.  But aside from maybe the wording of some of the questions that occurred after the presidential survey, these polls were no different than the usual ones PPP included, and I decided it wouldn't be fair to state that PPP's latest guess was a poll produced two weeks ago when really they had just done one yesterday.

And then I discovered all the polls in Huffpost Pollster, and then Pharos Research Group, and then Steve Singiser's daily roundup of polls that I had never heard of or considered before, and so I realized I needed a better methodology to selecting the polls.  Here's what I've got so far:

Include all polls except:
  1. Polls produced for political parties or individual campaigns
  2. Polls produced for advocacy organizations (except PPP) (so, no Grove, Lake or Mellman polls)
  3. Internet polls (no real reason, just that RCP doesn't use them)
  4. Class assignment academic polls (with long period of time in the field and/or small number of responses) (the Old Dominion poll and the U. of Iowa poll are the only two to fall under this exception)
  5. Polls that only report registered voters instead of likely voters after September (one High Point poll in North Carolina was excluded).
  6. Any poll with the word "Newsmax" in it.
I wish I could say there was more science behind the methodology, but it's been more ad hoc and hunch than scientific.  More sophisticated poll averagers use weighting, emphasizing the quality polls and minimizing partisan polls, but since I use a median instead of a mean, each poll has to count the same.  What this method does provide is a quick and easy way to gauge the current margin in any given state using the knowledge base of every pollster's latest guess in that state.
We can't know until after the election which polls are nearest to the final result, but we can know which polls are nearest to the median poll spread and calculate house effects based on this.  I have calculated the median spread for every day since September 1 in 19 state presidential contests and 14 senate contests.  The average distance each poll's spread is from the calculated median spread provides the house effect.

The following are my calculated house effects for each pollster with three or more presidential polls in my limited database.

Pollster No. of
No. of
House effect (D-R)
Susquehanna 4 4 -7.1
Gravis Marketing 25 13 -3.5
Baydoun/Foster 3 3 -2.8
ARG 15 0 -2.2
Rasmussen 63 46 -2.1
U. of Cincinnati 3 3 -1.8
Muhlenberg Coll. 4 4 -1.6
Mason-Dixon 10 8 -1.3
Civitas 3 0 -1.2
WeAskAmerica 15 9 -1.0
Survey USA 16 10 -0.8
Opinion Research 3 0 +0.0
MassINC 4 4 +0.1
Purple Strategies 5 0 +0.4
Fox News 6 5 +0.9
Suffolk U. 7 6 +1.2
PPP 57 32 +1.2
Marist 22 14 +1.3
Pharos Research Group 16 16 +1.4
Quinnipiac 19 17 +1.7
Marquette U. 5 5 +2.6
Detroit News 3 3 +3.8
Washington Post 4 4 +4.1
Epic-MRA 4 3 +4.4
UNH 4 0 +4.9

So if you read this morning that Rasmussen found the race in Wisconsin tied, you can guess that given Rasmussen's 2.1 point slant towards Republicans, Obama is actually up by 2.1 points there. 

I have found similar house effects to those published elsewhere, like these from Simon Jackman, which gives me confidence that I'm not just finding a bunch of hooey.


As of yesterday's polls, the median poll spread in the 9 battleground states plus Romney's Extended Map States (MI, PA, MN) are as follows:

State Obama Romney Spread EV
Minnesota 51 44 O+7 201
Wisconsin 51 46 O+5 211
Pennsylvania 49 45 O+4 231
Ohio 49 45 O+4 249
Michigan 48 45 O+3 265
Iowa 49 46 O+3 271
Nevada 50 47 O+3 277
New Hampshire 49 47 O+2 281
Colorado 47 47.5 R+0.5 257
Virginia 47 47.5 R+0.5 248
Florida 47 48 R+1 235
North Carolina 46 49 R+3 206

  •  The North Carolina number has grown more and more Romneyish this week, and a big part of that was a poll from the centrist Survey USA that showed Romney up 5. 
  •  Romney's margin in Virginia changed with the Roanoke College poll that was released yesterday. Roanoke College showed an 8 point Obama lead right before the first debate, and now shows a 5 point Romney lead. More impressively, Roanoke College went from a 10-point Tim Kaine lead over George Allen (highly implausible) to a 5-point George Allen lead over Tim Kaine (also highly implausible).  They swung from being a blue outlier to a red outlier in one of the largest poll-to-poll shifts I've seen in my database.
  • It almost doesn't matter about any other state besides Ohio at this point, and Obama's Ohio numbers constantly show him in the lead by a few percentage points
  • This Michigan number (Obama +3) is why my method of calculating a median spread may not be the best.  It's calculated from seven polls that show a margin of 0, 3, 4, 6, 6, 7, 8, but since the Obama numbers are 47, 48, 48, 48, 50, 52, 53 and the Romney numbers are 42, 42, 45, 45, 45, 46, 47, the spread is 3 instead of 6. 

No comments: