Understanding the Research Slogans

Demonstrate your understanding of the research slogans by matching each research slogan with it’s meaning. Record your answers at the end of this document by linking numbers with letters, such as 7J, 9B, 10E, etc.

1. Motivated by truth, with no hope of Proof A. Translate all of the terms in a hypothesis into concrete things you can measure. We can’t directly measure concepts like “sadness.” We have no choice but to measure things using imperfect rulers.
2. The best research invites failure. B. All concepts are inherently enigmatic and fuzzy. Terms like “melody,” “listen” or “note” can never be pinned-down. It is impossible to provide comprehensive definitions or grasp the essence of some concept. We are forced to approximate or estimate concepts through operational definitions — but don’t confuse the operational definition with the concept itself, and don’t assume concepts are “real.”
3. We invite failure by testing predictions. C. Exploratory and descriptive studies are important, but you can’t invite failure without testing predictions.
4. We recognize failure by drawing a line in the sand. D. The scholar who only offers theories after looking at the evidence is a scholar who is never wrong. Post hoc theorists don’t allow the world to tell them when their ideas are problematic.
5. Aim not to be right, but to be not not right. E. People are most impressed when someone accurately foretells the future. Science is a form of rhetoric whose persuasive power resides in the testing of predictions. The rhetorical power of science comes not from scholars assembling evidence, but from scholars testing predictions.
6. Test hypotheses by operationalizing terms. F. Test an idea by making a prediction, and then determine whether the observations are consistent with the prediction.
7. Operationalize, but don’t essentialize. G. Give the world an opportunity to tell you that you’re wrong. (This is the essence of good research.)
8. Compare, compare, compare. H. Most things seem obvious in retrospect. When the results aren’t obvious, humans are enormously gifted at coming up with explanatory accounts. We can make up a story for just about any set of data. Post hoc theories don’t have the same plausibility as a priori theories. The true test is making up the story first (i.e., prediction)! Prefer theorizing first, then collect your data.
9. The rhetoric of science is the rhetoric of prophecy. I. Contrast a “treatment” condition with one or more “control” conditions.
10. Hindsight is 20/20. J. There is no inductive proof. We are not in the business of proving something to be true. We would love to know the truth (if that exists), but we understand that we could never be sure of the truth, even if we had it. The best we can hope for is that what we observe is consistent with our theories.
11. Reductionism is a method, not a belief. K. We simplify problems, not because we believe problems to be simple, but because we believe problems to be complex. Restricting our gaze is a useful strategy for discovery.
12. Don’t try to explain the whole world at once. L. When presenting your results, frame them narrowly rather than broadly.
13. Generalize, but don’t universalize. M. Manipulate one variable at a time. Seek simplicity, even as you distrust it.
14. Avoid chronic hypothesislessness. N. Start with a question, propose an explanatory theory, derive a conjecture, refine the conjecture into a hypothesis, then operationalize the terms of the hypothesis into a protocol. The protocol provides an action plan for how to carry out the research.
15. Beware of the post hoc theory. O. In order to make failure obvious, establish a criterion in advance that says, “If the evidence doesn’t cross this line, then I’ll admit failure.” In statistics, the line is referred to as the confidence level.
16. From Question to Theory to Conjecture to Hypothesis to Protocol. P. Instead of establishing The Truth, our more modest aim is to be not obviously wrong. When our observations turn out to be consistent with our hypothesis, we don’t claim that we are right; instead the observations suggest that our hypothesis may not be wrong.