As if the educational research landscape were not hard enough to navigate, a new article in July's Perspectives on Psychological Science threatens to make it much harder.

The authors contend that many "active control" experiments -- where the control group does "something", a standard treatment not expected to impact the outcome measure(s) -- do not control for the expectations the group may have of the effectiveness of the treatment.

They conducted surveys to measure the participants' expectations on the effectiveness of playing different types of video games on different experimental measures. After just 30 seconds of exposure to a video game, those surveyed had expectations that matched the claims in the video-game literature. In short, the measured improvements in the video gaming literature are almost certainly picking up placebo effects -- the participants expect certain game types to impact certain measurement types but not others.

This is not limited to studies of video games -- the authors provide a few examples from the research of cognitive science, psychotherapy, and education. 

All is not lost. In addition to providing a useful flowchart to determine the merits of causal claims in intervention research (below), the authors note that experimenters can overcome this by (a) directly measuring expectations for each group and using those measurements in the analysis; (b) empirically proving that expectations cannot influence the outcomes; or, (c) selecting other experimental designs that minimize, manipulate, or demonstrate placebo effects.