Comments posted organically
SelectSmart.com Homepage
Display Order:

Trump's sacrilegious Bible scam. If people don't recognize Trump as a phony now, they never will.
Religion by Curt_Anderson     March 27, 2024 1:54 pm (Rating: 0.0) Last comment by: oldedude (24 comments) [366 views]


Baltimore Bridge Collapse Victims Were Working to Support Families, Co-Worker Says
News by Curt_Anderson     March 26, 2024 7:31 pm (Rating: 0.0) Last comment by: oldedude (21 comments) [373 views]


James Comer pens a pathetically desperate letter to Joe Biden
Government by Curt_Anderson     March 28, 2024 3:10 pm (Rating: 0.0) Last comment by: (0 comments) [168 views]


Anonymous comments regarding the Presidential Candidate Selector
President by Curt_Anderson     March 19, 2024 10:10 am (Rating: 0.0) Last comment by: Curt_Anderson (9 comments) [88 views]


People who live in CPAC houses shouldn't grab cajones
Gay & Lesbian by Curt_Anderson     March 28, 2024 11:33 am (Rating: 0.0) Last comment by: Curt_Anderson (2 comments) [38 views]


Trump's lawyer should be ashamed for making a preposterous First Amendment argument.
Law by Curt_Anderson     March 28, 2024 11:04 am (Rating: 0.0) Last comment by: (0 comments) [35 views]


Supreme Court Gets Jan 6. Defendant Out of Jail
Dungeons & Dragons by oldedude     March 27, 2024 8:55 pm (Rating: 0.0) Last comment by: oldedude (4 comments) [39 views]


Anonymous comment regarding the City Selector
Travel by Curt_Anderson     March 28, 2024 10:33 am (Rating: 0.0) Last comment by: (0 comments) [12 views]


Trump's co-conspirators face disbarment.
Law by Curt_Anderson     March 27, 2024 8:29 pm (Rating: 0.0) Last comment by: Indy! (1 comments) [9 views]


J6 investigative committee recommends that Trump be charged with four crimes.
Law by Curt_Anderson     December 19, 2022 12:05 pm (Rating: 0.0) Last comment by: oldedude (20 comments) [397 views]


Education selectors, pages, etc.
Can we trust what polls are telling us?
By Curt Anderson
January 27, 2020 6:21 pm
Category: Education

(5.0 from 1 vote)
Rules of the Post

SelectSmart.com SelectSmart.com SelectSmart.com


Rate this article
5 Stars
4 Stars
3 Stars
2 Stars
1 Star
0 Stars
(5=best, 0=poor)

I usually defend polling organizations. Polls, when properly done, are the best tool we have to measure public opinion on a particular issue or candidate. However, not all polls are created equal.

Open-access polls vs. scientific polls
We have all seen online polls in which we are asked to opine on various topics. These are known as open-access polls. The participants are self-selected, and anything but a random cross-section of voters or of any other identified group. They are fun, but not very meaningful.

For polls to be considered meaningful they must be scientific polls. The most glaring difference between an open-access poll and a scientific poll is that scientific polls randomly select their samples. Scientific polls use statistical weights to make them demographically representative of the target population.

People who are critical of polls (often when they don't like the results) sometimes ask how can a relatively small number of polled participants, often around 1000 people or so, be representative of a state or even all of America. There is a soup making analogy that answers that question. If you are making a large amount of a cream soup and want to know if it is properly seasoned, you don't need to consume the entire cauldron of soup. If you stir the soup sufficiently for minute or so, you can get a pretty good idea how much pepper, salt or other spices are needed by tasting a single spoonful. In other words, that spoonful you tasted represents a randomized sampling of all the soup in that cauldron. That is what a scientific poll endeavors to accomplish.

One of the polls that caught my attention is the Emerson poll. You might think I am picking on Emerson. My criticism might be considered unfair since I have not scrutinized other polls in the same way. I am less than confident in how Emerson achieves their results, and I will explain why.

In their defense, I will note that Emerson College Polling is well-regarded. Emerson polls are cited by major news organizations. FiveThirtyEight.com gives Emerson an A- grade. That said, being naturally skeptical, I tend to look at the methodology of polls. Here is how Emerson says they do their polling:


The National Emerson College Poll was conducted January 21-23, 2020 under the supervision of Assistant Professor Spencer Kimball. The sample consisted of registered voters, n= 1,128, with a Credibility Interval (CI) similar to a poll's margin of error (MOE) of +/- 2.8 percentage points. The data was weighted based on 2016 voter model of gender, education, age, ethnicity, and region. It is important to remember that subsets based on gender, age, party breakdown, ethnicity, and region carry with them higher margins of error, as the sample size is reduced. Data was collected using both an Interactive Voice Response (IVR) system of landlines only (n= 741) and an online panel provided by Amazon Turk (n= 387).


The terminology used in Emerson's description of their methodology caused me to search the Internet for definitions. Here is my glossary of the bold type terms above.

Credibility Interval (CI)
AAPOR [American Association for Public Opinion Research] urges caution in the interpretation of a new quantity that is appearing with some nonprobability opt-in, online polling results – the credibility interval. The credibility interval is not the margin of sampling error (MOE) that the public has come to understand as the statistical uncertainty of probability based scientific polls. Instead, credibility intervals are being used when reporting on some nonprobability polls (typically opt-in online polling), or for model-based inferences such as arise in small-area estimation.

Margin of error
The margin of error is a statistic expressing the amount of random sampling error in a survey's results.

Interactive Voice Response
Interactive voice response (IVR) is a technology that allows a computer to interact with humans through the use of voice and DTMF tones input via a keypad.

Online panel provided by Amazon Turk
This term in particular had me flummoxed. I went to the website that explains it. Here is what it says:

Make money in your spare time
Get paid for completing simple tasks
Want to make money in your spare time? Become an Amazon Mechanical Turk (MTurk) Worker and start applying your skills to the thousands of available tasks in the MTurk marketplace.

What type of work is available on Amazon Mechanical Turk?
Workers help companies collect and edit information by:
Filling out market research or survey data on a variety of topics.


IVR and Amazon Turk respondents may not be like the rest of us
I am suspicious of Emerson's methodology not their sincerity. I suspect that only a certain sort of person has a landline and the time and patience to answer the questions of a robotic voice. I would be excluded from that group. I don't answer the phone unless I recognize the caller's number. Maybe if I were lonely I would pickup the phone whenever it rings. Maybe if I had more time in my life I'd be willing to "talk" with a robot.

I also doubt that Amazon Turk workers who are paid to respond to pollsters are representative of all registered voters. I assume that they are low paid workers without many benefits. I can't say if the IVR or the Amazon Turk respondents are any more liberal or any more conservative than the entire voting population. It certainly seems that those groups would have characteristics that not all of us share.

Cited and related links:

  1. emersonpolling.reportablenews.com
  2. aapor.org
  3. en.wikipedia.org
  4. mturk.com

Comments Start Below


The views and claims expressed by contributors are their own and do not necessarily reflect the views and beliefs of SelectSmart.com. Not every statement made here can be assumed to be a fact.
Comments on "Can we trust what polls are telling us?":

Be the first to comment on this article.


Comment on: "Can we trust what polls are telling us?"


* Anonymous comments are subject to approval before they appear. Cookies Consent Policy & Privacy Statement. All Rights Reserved. SelectSmart® is a registered trademark. | Contact SelectSmart.com | Advertise on SelectSmart.com | This site is for sale!

Find old posts & articles

Articles by category:

SelectSmart.com
Report spam & abuse
SelectSmart.com home page