Debating the merits of large- and small-N studies
Sample size does more than determine the sort of methodology appropriate for a given study; theorists of social science have long pointed out that the number of case studies considered determines the sorts of questions researchers can analyze and the structure of their causal claims.
A 2003 paper by PETER HALL takes these debates further. In the context of comparative political science, Hall argues that the sort of methods researchers use should be consistent with their beliefs about the nature of historical development. From the paper:
"Ontology is crucial to methodology because the appropriateness of a particular set of methods for a given problem turns on assumptions about the nature of the causal relations they are meant to discover. It makes little sense to apply methods designed to establish the presence of functional relationships, for instance, if we confront a world in which causal relationships are not functional. To be valid, the methodologies used in a field must be congruent with its prevailing ontologies. There has been a postwar trend in comparative politics toward statistical methods, based preeminently on the standard regression model. Over the same period, the ontologies of the field have moved in a different direction: toward theories, such as those based on path dependence or strategic interaction, whose conceptions of the causal structures underlying outcomes are at odds with the assumptions required for standard regression techniques.
The types of regression analyses commonly used to study comparative politics provide valid support for causal inferences only if the causal relations they are examining meet a rigorous set of assumptions. In general, this method assumes unit homogeneity, which is to say that, other things being equal, a change in the value of a causal variable x will produce a corresponding change in the value of the outcome variable y of the same magnitude across all the cases. It assumes no systematic correlation between the causal variables included in the analysis and other causal variables. And most regression analyses assume that there is no reciprocal causation, that is, that the causal variables are unaffected by the dependent variable. The problem is that the world may not have this causal structure.
Small-N comparison is therefore far more useful for assessing causal theories than conventional understandings of the 'comparative method' imply. Precisely because such research designs cover small numbers of cases, the researcher can investigate causal processes in each of them in detail, thereby assessing the relevant theories against especially diverse kinds of observations. Reconceptualized in these terms, the comparative method emerges not as a poor substitute for statistical analysis, but as a distinctive approach that offers a much richer set of observations, especially about causal processes, than statistical analyses normally allow."
Link to the piece.
- "Except for probabilistic situations that approach 1 or 0 (in other words, those that are almost deterministic), studies based on a small number of cases have difficulty in evaluating probabilistic theories." Stanley Lieberson's 1991 overview of the causal assumptions inherent to small-N studies. Link.
- Theda Skocpol and Margaret Somers on "The Uses of Comparative History in Macrosocial Inquiry." Link.
- Jean Lachapelle, Lucan A. Way, and Steven Levitsky use small-N process tracing to "examine the role of the coercive apparatus in responding to crises triggered by mass anti-regime protest in Iran and Egypt." Link. Andrey V. Korotayev, Leonid M. Issaev, Sergey Yu. Malkov and Alisa R. Shishkina present a quantitative analysis of destabilization factors in 19 countries during the Arab Spring. Link.