Site Meter

Monday, July 14, 2008

In Which I give Pollster.com unsolicited advice.

Rasmussen weights responses to make the shares of Democrats, Republicans and independents in their tracking poll equal to a long run average (3 month average it seems). This reduces noise due to sampling. I think it also makes for numbers which are less useful to pollster.com

Mark Blumenthal discusses the issue here.

I put in my 0.2 cents worth in comments.

I believe two things. First that by weighting by party Rasmussen achieves better estimates of the true state of public opinion. Second that you should use the unweighted results to estimate your smoothed summary of polls (the pollster.com trend line).

No contradiction. Party weighting reduces variance due to sampling. It also introduces bias when the true population distribution of party affiliation changes. For plausible rates of true change in party affiliation and a sample size of around 3,000 I (like the Rasmussen team) would guess that weighting by party reduces mean squared error.

However, when you average over a bunch of polls, you reduce the effect of sampling error. If all polls were party weighted the same way, the average would have the same bias as each poll. A larger sample makes avoiding bias more important relative to achieving precision. You at pollster.com effectively have a very large sample, since you average over many polls.

I think for your purposes, Rasmussen's party weighting creates more problems than it solves, that is increases the MSE of your trend forecast. I really would suggest that you use the raw unweighted data if you can get them. I'd also guess that you can. I don't think that pollsters can afford to quarrel with you.

No comments: