I have been reading an interesting internet discussion on the analytical efficacy of in-depth interviews, based around this paper by Samuel Lucas. Two other sociologists — Andrew Perrin here, and Fabio Rojas here and here — provide responses. I feel no need to disagree with anything yet said. However, I feel the need to add an additional perspective on the value of in-depth interviews untouched by Lucas or the others.
The part of Lucas’s argument most interesting to me goes like this: Studies that rely on in-depth interviews often begin with qualifiers about how the study’s sample cannot be used to generalize to a population, but then proceed to draw generalized conclusions anyway. This is a problem because studies featuring in-depth interview data tend to use non-probabilistic sampling techniques — most notably, snowball and purposive sampling. I rely heavily on both these techniques, indeed almost exclusively, so his argument is of particular interest to me.
Why do snowball samples and purposive sampling lead directly to unrepresentative samples? Because of what Lucas calls the “lumpy” nature of the social world. Meaning: similar facts, variables, mindsets, beliefs tend to congregate, or at the very least, are not randomly distributed among different locales. A snowball sample, where you rely on respondents to give you the names of further respondents, seems especially prone to this problem. Lucas puts it this way:
“[I]n the large-dimensioned social space there are concentrations of entities, and sparse locales; some constellations of characteristics are common, others rare; hills and mountains rise from some spots on the social terrain, valleys and ravines mark others”
This point sounds theoretically solid. In fact, I think Lucas’s paper empirically demonstrates it. One of the characteristics of this discussion on in-depth interviews is it is being done by academic sociologists with academic sociology as the vantage point. No mention has been made that a growing amount of sociological research is being done in the private sector — most of this is quantitative analysis, and irrelevant to the discussion on in-depth interviews. But a growing minority is indeed qualitative and ethnographic. And, in my case, based on in-depth interviews.
This distinction — between academic and privatized sociology — matters, because privatized sociology is usually done for particular clients with particular needs, interests, products, and audiences. Academic sociology has qualitatively different bases of funding. Often the source of funding is government, or departmental. In both these cases, generalizing from samples to populations is very important. In the case of particular private clients, however, generalizing to populations is not important, indeed it is counter-productive. In Lucas’s terms, clients want to live within the lumpiness, not transcend it. In my terms, clients want to be experts on particularized audiences. Why do people from the upscale neighborhood shop at our competitor and not us? Why do black audiences in the area immediately surrounding our store ignore us?
These are a kind of question clearly not on the mind of Lucas, and I don’t blame him. By raising these questions I do not mean to criticize. His mind is elsewhere, and his paper deserves to be dealt with on its terms, not mine.
But when Lucas writes that “non-probability sampling is of extremely limited utility, providing grounds, at best, only for existence proofs,” he is writing from within his social lump, not mine.
So let me make three particular points.
1. A growing amount of sociology is private, not academic, for private clients not government or departmental, and in these private circles generalizability is less essential and particularistic analysis (e.g. case studies) more so.
2. Interview data based on snowball and purposive sampling do a good job of raising and sometimes providing an answer to the most important question private clients need to know: Why? Why do people behave the way they do? I could probably agree that representative samples better answer what: e.g., how many people are behaving that way? But snowball samples provide me a network of people, and purposive sampling provides me especially important and articulate “informants.” It is from these people, and because of these techniques, that I can start to answer why.
Now, indeed, any “why” answer relies heavily on theory. I can’t get around that.
3. Finally, the closest thing I have to a criticism of Lucas’s approach to this question is that his “lumpy” vantage point puts too much stock in government-based sources of funding. While governments have every interest in generalizing to populations, I think one can argue that where sociological methods are growing is in the private sector, for clients who have less interest in, or need for, generalizing. They need particular knowledge, they need it now, and even if you don’t have a perfectly formed dataset, you need to make a decision using sound theory and intuition.
Of course, I might be wrong about where sociology is and is not experiencing growth. My own lumpiness may be telling me things that aren’t broadly true.