14 June 2022Recently I read a blog post by the syntactician Omer Preminger about why ecological validity is undesirable and “would undermine research into the structure of human language.” Indeed, Preminger argues that when scientists are confronted with complex natural phenomena that arise out of messy interactions between multiple factors, the typical approach has been to simplify matters by isolating or eliminating factors through highly artificial setups.
An example of one such setup in physics would be the Large Hadron Collider in Switzerland, which shoots beams of particles at high speed into each other such that they collide. Particle physicists study the outcomes of such collisions. Implicitly then, the analogue in linguistics might be something like acceptability judgement tasks or controlled psycholinguistic experiments.
There’s nothing wrong with this approach. Preminger is right that this is what science does to study the structure of some naturally-occurring phenomenon. However, we must be careful here not to get confused. Controlling and isolating factors is vital to rigourous scientific experimentation, sure. But is that in any way mutually exclusive with ecological validity? Preminger along with many other linguists seem to assume yes. Yet, that essentially reduces ecological validity to the claim that we should not control or isolate factors before investigating a phenomenon. This is, of course, a mischaracterisation.
Ecological validity merely demands that experimental conditions be naturalistic, not uncontrolled. It is well known that showing participants lists of sentences with no continuous discursive context would grant an experimenter great freedom to isolate hypothesised grammatical factors, but the unnaturalness of the stimuli could damage the generalisability of the experiment’s results as a claim about natural language use. And generalisability of results is key.
On the other hand, taking ecological validity into account would require the experimenter to design stimuli that better emulate language in natural communicative contexts, perhaps by tying together blocks of stimuli into a consistent narrative (see Trevisan & García 2019), but that doesn’t mean the experimenter sacrifices any control over relevant experimental factors.
Of course, critics of ecological validity as a notion in linguistics will, no doubt, note that I used the term natural language use in the previous paragraph. We remember that Chomsky attempted to defend his own ecologically-challenged theories with the notion of the competence/performance distinction. According to this line of thought, ecological validity would be desirable for researching about linguistic performance, that is, how language is performed in real-time contexts, but not competence.
Chomsky claimed that his linguistics was concerned with the cognitive system underlying linguistic competence, an innate naturally-occurring phenomenon of human life. Competence, as the potential to perform, is then understood to be invariant (universal) across contexts. It is the common base which performance systems draw upon to perform speech acts in any sociopragmatic context.
Because it is context-free, the study of linguistic competence has been touted as the primary object of the truly “scientific” study of language. It is deemed scientific because it is neatly isolated, like the particles in the Large Hadron Collider. Where particle physicists are concerned with discovering the subatomic properties of the isolated particles in the collider to learn more about the structure of the physical world, linguists should be concerned with the properties of the conceptually isolated competence system to learn more about the structure of the cognitive faculty for language. This seems like a reasonable analogy at first glance, but if you think a little deeper, you might notice an important philosophical assumption.
Namely, the assumption that physical matter can exist in a context-free state. We might think that the particles being blasted in Geneva’s Large Hadron Collider exist context-free because they’re in a vacuum. If the particles are particles of matter, and vacuum is defined as the lack of matter, then one particle floating in a vacuum must be rather isolated, right?
But consider this: the vacuum is the context for that particle’s existence at that point in spacetime. It is isolated from other matter, but not isolated from vacuum. Hence, isolation and context-freeness becomes a matter of perspective. Applying the same logic to Chomsky’s competence/performance dichotomy, just because the competence system is invariant across contexts of performance doesn’t mean that it is objectively context-free.
Let’s use another, more grounded analogy (one that comes from Chomsky himself and was mentioned by Preminger). Imagine some leaves on the street being blown about by natural winds. A number of factors play a role in the trajectory of each leaf as it flies in the air: the weight of the leaf, its surface area, the direction of the wind, how many holes in the leaf some caterpillar may have made, whether it is currently raining or not, etc.
Now, if we say that the leaves represent the structure of linguistic knowledge, we might say the leaves blowing in the wind (the whole phenomenon) represents language in use. If the wind is of a natural origin, then this represents natural language use. As experimenters, we might use a leafblower to create artificial wind, and hence manipulate language use in an experimental context.
Essentially, what Chomsky and others seem to be doing is studying the structure of the leaves with little regard for how that structure coordinates with other factors such that the leaves blow in some certain way when wind occurs in some configuration. But the leaves only constitute one part of the blowing-leaves phenomenon. Furthermore, if we took the leaves, put them in a vacuum (much like in the Large Hadron Collider) and studied them then, can we really be sure we would be gaining much useful insight about how the leaves blow in the wind?
Likewise, if we studied linguistic competence as an isolated context-free phenomenon, what useful conclusions might we be able to draw about language, the naturally-occuring phenomenon that unfolds in real-time contexts?
Again, some Chomskyans may argue that it is simply natural that the study of linguistic competence would yield little about the nature of linguistic performance in context. And the goal, after all, is the study of competence, not performance. But we also have to question whether we can really understand the competence system without studying performance.
Let us return to the anology of leaves blowing in the wind, but replace the leaves with helicopter seeds (also known as winged seeds). I remember seeing many of these near my college in Thailand when I was an undergraduate. The seeds are like little round balls with 3 or 4 large leaf-like wings which allow them to spin and fall slowly from trees.
Now, this is a naturally-occurring phenomenon. So, imagine we took some of these winged seeds and placed them in the vacuum of space. Now further imagine we were martians scientists with no knowledge of the atmospheric conditions on Earth’s surface, attempting to understand the seed’s physical structure.
A dipterocarp winged seed from Sarawak, Malaysia. Photo by: Bernard Dupont.
It is likely we would fail to even recognise the 3-4 protrusions out of the seed’s body as wings, let alone be able to deduce any realistic theory of why the helicopter seed is shaped the way it is. We might discover some facts about the molecular and material structure of the seed, but there would be no way to answer any questions about the significance of the structural arrangement of the seed’s matter, such as its evolutionary origin (assuming we know something about evolution).
From this perspective, what we have been calling a seed looks more like a weirdly arranged and segmented blob of particles. We might falsely conclude that the wings serve some fictional function to help the seed drift in space.
In other words, questions of structure rely interdependently with questions of function! And since function cannot be decoupled from contexts, neither can structure. Ecological validity, then, simply recognises this and attempts to work within the constraints of context. If language comprehension is subject to the constraints of working memory, then that’s fine. Rather than try to abstract a linguistic system (Chomsky’s competence) independent of working memory, we can study and theorise about a linguistic system that is, well, subject to working memory, just as in natural contexts.
Preminger’s claim that linguistic research should be “ecologically invalid” in order to isolate truly linguistic cognitive phenomena seems explanatorily vacuous given that any theories that follow from this way of thinking would be describing something that doesn’t exist in nature. On the other hand, the particle collisions that physicists test with their fancy colliders do likely occur in nature (maybe not on Earth, but at least elsewhere in the universe), hence making their experiments ecologically valid.
In sum, the fact of the matter is is that language is a naturally-occuring phenomenon that unfolds in time. This unfolding is modulated by sociopragmatic contexts (occasion, interlocutor relationships, discursive aims, etc.) on the behavioural plane of things and by neuropsychological states and structures (memories, neurological conditions, attention, etc.) on the cognitive plane.
For successful language use, each of these factors work together in tandem. We may focus on one or a few of these factors in our scientific pursuits, but to study leaves as a factor in the phenomenon of leaf-blowing, it would make the most sense to study the leaves in contexts where other factors enabling leaf-blowing are also present.
Linguistic competence and linguistic performance are not two distinct parts of the phenomenon that is language. Perhaps they would be best thought of as a singular competence-performance spectrum, in the spirit of Sydney Lamb’s remark that “competence really means competence to perform” (Language & Reality 2004, chapter 8). And perhaps, this way, linguistics could be like the leaves themselves, moving in the air to new places and directions, not drifting to nowhere in the vacuum of space.
(And perhaps us linguists can then actually be human scientists, not martian ones!)