Opinions as Operationalizations

Some terms are easier to operationalize than others. It may be straightforward to count the number of people attending a concert, but usually some refinement of the counting procedure is necessary. Suppose for example that you are counting “the number of notes in a work” from a printed score. You might decide to exclude repeats signs but include a Da Capo return. You might reasonably exclude “tied-to noteheads,” but what about groupettos and grace notes? And what about trills and other ornaments? Some clarification will be necessary. How you settle on an operationalization will depend on the research goal and on the amount of labor required.

When operationalizing terms, try to anticipate the kinds of problems that might arise, and define your terms accordingly. Even so, once the measuring begins, you are apt to encounter situations which will require further refinement to the operational definitions.

In some cases, it will seem impossible to define a “mechanical” procedure that will always produce unambiguous results. In these cases, it may be necessary to rely on opinion. For example, in several studies we’ve done, we have contrasted works in major and minor keys. Most of the time, musicians will have no difficulty identifying whether a score is in the major or minor mode. There are some difficulties, however. It is not uncommon for a minor-mode work to modulate to the relative major for a period of time. Minor-mode works may end with a Picardy Third (where the final tonic chord is major). Some works don’t end in the same key in which they began. There are atonal works, whole-tone works, and works written in medieval modes like Dorian or Phrygian. Despite these and other complications, for common-practice period music, 9 out of 10 musicians will agree on the key. It’s not outrageous for an experimenter to judge the key of most works. In many research projects, we’ve relied on the opinion of the researcher to determine key. Since some pieces can be ambiguous or questionable, we typically use three categories: (1) obviously major, (2) obviously minor, and (3) not obviously major or minor. In effect, we operationalize a work’s mode as “the opinion of the experimenter when examining the score, as to whether the work is obviously major or obviously minor.”

In other cases, it may be much less clear, and so opinion may be easily skewed by the beliefs of the experimenter. Suppose, for example, that an experimenter was testing the hypothesis that “composer A wrote shorter phrases than composer B.” Suppose further that neither composer took pains to write explicit phrase markings over every phrase. Opinions about “what’s a phrase” are much less unanimous than opinions about major or minor mode. Here, experimenter bias is more likely to be a problem. Nevertheless, an independent musician—not familiar with the aims of the project or the hypothesis being tested—may prove useful. Rather than trying to create a mechanical procedure for defining a phrase, we might simple rely on an independent opinion. In this case, one could operationalize phrases as “whatever an independent musician marks in the score as a phrase.”

For some tasks, opinions can diverge considerably. For example, although it is relatively easy to determine whether pieces in the 18th century exhibit a “sonata-allegro” form, this becomes much more contentious in music from the 19th century. If a researcher needs to know whether a given movement is in sonata-allegro form, it may be necessary to poll the opinions of several theorists. In thise case, one could operationalize “in sonata-allegro form” as “those movements that a minimum of 3 of 5 music theorists claim to conform to sonata-allegro form.”

All concepts ultimately prove to be difficult to “pin-down.” In operationally defining a term, the researcher should first aim to produce a mechanical procedure that anyone could follow, resulting in a clear and repeatable measurement. If that’s not possible, “opinions” can be used. In some cases, opinions are sufficiently uncontentious that the experimenter’s own opinion can be used. In order to avoid experimenter bias, independent opinions from people unfamiliar with the aims and hypothesis of the study might be used. It may be necessary to recruit “experts”—so that the opinions are more informed. If the term is especially fuzzy, the experimenter might employ several experts, and establish a minimum criterion of unanimity.

The important point is that opinions can provide useful operationalizations for especially intractable terms or concepts.