Consider the following experimental results:
People who became vegetarians for ethical reasons were found to be more committed to their diet choice and remained vegetarians for longer than those who did so for health reasons.
Loyalty to expert advisers (doctors, financial advisors, etc.) leads to higher prices but not necessarily better services.
Smokers who viewed ads featuring messages about “how” to quit smoking were substantially more likely to quit than those who viewed ads with reasons “why” to quit.
In adults, creativity was substantially inhibited during and shortly after walking (either outdoors or on a treadmill) as compared to sitting.
Answer each question before scrolling down and reading the next, because of SPOILERS:
1. How do you explain these effects?
2. How would you have gone about uncovering them?
3. These are all reversed, and the actual findings were the opposite of what I said. How do you explain the opposite, correct effects?
4. Actually, none of these results could be replicated. Why and how were non-null effects detected in the first place? Answers using your designs from (2) are preferable.
Final spoilers below.
For the real findings, see Useful Science (, ,,), which is Useful as a source of further exercises, at least. Some of the four are indeed reversed, but as far as I know I made up the part about replication. Reflect on the quality of your explanations and on any feelings of confusion you noticed or failed to notice. I apologize for lying; it was for your own good.
Extra credit: Follow the links to find the original papers. Compare your proposed test, and determine whether your alternative explanations were ruled out.
Thanks, that was cool.
For me the marker of a poor explanation was only considering one explanation, the one that fit the evidence.
It never ceases to amaze me, my ability to rationalise seems foolproof.