Previously I blogged about an experiment which used the time it takes people to make decisions to try and elucidate something about the underlying mechanisms of information processing (Stafford, Ingram & Gurney, 2011) . This post is about the companion paper to that experiment, reporting some computational modelling inspired by the experiment (Stafford & Gurney, 2011).
The experiment contained a surprising result, or at least a result that I claim should surprise some decision theorists. We has asked people to make a simple judgement – to name out loud the ink colour of a word stimulus, the famous Stroop Task (Stroop, 1935). We found that two factors which affected the decision time had independent effects – the size of the effect of each factors was not effected by the other factor. (The factors were the strength of the colour, in terms of how pale vs deep it was, and how the word was related to the colour, matching it, contradicting it or being irrelevant). This type of result is known as “additive factors” (because they add independently of each other. On a graph of results this looks like parallel lines).
There’s a long tradition in psychology of making an inference from this pattern of experimental results to saying something about the underlying information processing that must be going on. Known as the additive factors methodology (Donders, 1868–1869/1969; Sternberg, 1998), the logic is this: if we systematically vary two things about a decision and they have independent effects on response times, then the two things are operating on separate loci in the decision making architecture – thus proving that there are separate loci in the decision making architecture. Therefore, we can use experiments which measure only outcomes – the time it takes to respond – to ask questions about cognitive architecture; i.e. questions about how information is transformed and combined as it travels between input and output.
The problem with this approach is that it commits a logical fallacy. True separate information processing modules can produce additive factors in response data (A -> B), but that doesn’t mean that additive factors in response time data imply separate information processing modules (B -> A). My work involved taking a widely used model of information processing in the Stroop task (Cohen et al, 1990) and altering it so it contained discrete processing stages, or not. This allowed me to simulate response times in a situation where I knew for certain the architecture – because I’d built the information processing system. The result was surprising. Yes, a system of discrete stages could generate the pattern of data I’d observed experimentally and reported in Stafford, Ingram & Gurney (2011), but so could a single stage system in which all information was continuously processed in parallel, with no discrete information processing modules. Even stranger, both of these kind of systems could be made to produce either additive or non-additive factors without changing their underlying architecture.
The conclusion is straightforward. Although inferring different processing stages (or ‘modules’) from additive factors in data is a venerable tradition in psychology, and one that remains popular (Sternberg, 2011), it is a mistake. As Henson (2011) points out, there’s too much non-linearity in cognitive processing, so that you need additional constraints if you want to make inferences about cognitive modules.
Thanks to Jon Simons for spotting the Sternberg and Henson papers, and so inadvertantly promting this bit of research blogging
References
Cohen, J. D., Dunbar, K., and McClelland, J. L. (1990). On the control of automatic processes – a parallel distributed-processing account of the Stroop effect. Psychol. Rev. 97, 332–361.
Donders, F. (1868–1869/1969). “Over de snelheid van psychische processen. onderzoekingen gedann in het physiologish laboratorium der utrechtsche hoogeshool,” in Attention and Performance, Vol. II, ed. W. G. Koster (Amsterdam: North-Holland).
Henson, R. N. (2011). How to discover modules in mind and brain: The curse of nonlinearity, and blessing of neuroimaging. A comment on Sternberg (2011). Cognitive Neuropsychology, 28(3-4), 209-223. doi:10.1080/02643294.2011.561305
Stafford, T. and Gurney, K. N.(2011), Additive factors do not imply discrete processing stages: a worked example using models of the Stroop task, Frontiers in Psychology, 2:287.
Stafford, T., Ingram, L., and Gurney, K. N. (2011), Pieron’s Law holds during Stroop conflict: insights into the architecture of decision making, Cognitive Science 35, 1553–1566.
Sternberg, S. (1998). “Discovering mental processing stages: the method of additive factors,” in An Invitation to Cognitive Science: Methods, Models, and Conceptual Issues, 2nd Edn, eds D. Scarborough, and S. Sternberg (Cambridge, MA: MIT Press), 702–863.
Sternberg, S. (2011). Modular processes in mind and brain. Cognitive Neuropsychology, 28(3-4), 156-208. doi:10.1080/02643294.2011.557231
Stroop, J. (1935). Studies of interference in serial verbal reactions. J. Exp. Psychol. 18, 643–662.
7 replies on “It isn’t simple to infer cognitive modules from behaviour”
It is argued above that “True separate information processing modules can produce additive factors in response data (A -> B), but that doesn’t mean that additive factors in response time data imply separate information processing modules (B -> A).” This reasoning is intended to apply only when the modules are stages, in the Sternbergian sense of stage. (i.e. processing in module A has to finish before processing in module B begins, and there is no feedback from B to A). The model in Fig 1 of Stafford et al, (2001) is not a stage model because there is continuous information flow between its components. Therefor the inference “additive factors in response time data imply separate information processing modules” is not intended to apply to this model.
As far as I know, the position you are criticising, which is “additive factors in response time data imply separate information processing modules even when these models are not stages” has never been adopted by anyone. Isn’t it therefore a Straw Man position?
For more on this: Coltheart, M. (2011). Methods for modular modelling: Additive factors and cognitive neuropsychology. Cognitive Neuropsychology, 28, 224-240.
Best,
Max C.
Hi Max
The model in this paper does have Sternbergian stages, in that processing completes in one stage before information is passed to another and there is no feedback. The code is available in the supplementary material in case you want to check. Part of the lesson of this modelling, for me, is that although it is easy to define what a stage is in words, the issue begins to look a lot more murky when you try and make a system that actually does information processing.
I am not criticising the position you quote in your second paragraph. It is possible I made it look like that with some loose talk of modules, stages, loci, etc in this blog post. My apologies. Hopefully the paper does a better job of being precise.
Your paper sounds completely relevant, thanks for letting me know about it
Dear Tom,
Agree completely that having an actual computational model is essential if issues like these are to be unambiguously dealt with.
If you weren’t criticising the position I quoted, then when you say in the blog sidebar “my paper in Frontiers tells you why one of the oldest claims in cognitive science is wrong”, what is the claim that you are referring to here?
And I don’t think I followed what you meant by “The model in this paper does have Sternbergian stages, in that processing completes in one stage before information is passed to another”. Consider the unlocked version of the Figure 2 model. Is processing at the Input encoding stage completed before anything is transmitted to the Decision stage? That seems to be denied by what page 3 says: “This model is a continuous processing model. Activity in all parts of the model is continuously updated as the effect of the change in inputs (representing stimulus presentation) propagates through”.
Are you saying that the Figure 2 model is a single-stage model? But the caption refers to the Decision component of the model as a stage, so presumably Inout encoding is also thought of as a stage, so there are two stages, not one.
Best,
Max
Max
My error – the caption for Figure 2 is misleading, since it uses the word ‘stage’ in a non-Sternbergian way to describe a component of the model. If you look at Figure 5 this shows the model with true Sternbergian stages – processing in the “detection” component finishes before any information is passed on to the “decision” component.
Manipulation of these models – those with and without Sternbegian stages – shows that both produce, or not produce, additive factors. This suggests that any inference from additive factors to Sternbergian stages is invalid.
Does that clear things up?
Dear Tom,
Has anyone ever claimed that additivity necessitates Sternbergian stages? I don’t think so. Sternberg himself denies this. Roberts & Sternberg (1993) discuss how a nonstage model (the McClelland-Ashby model) can yield additivity. So here they are pointing out an example where additivity does not necessitate Sternbergian stages
The general Sternbergian approach is the claim is that IF the system is a Sternbergian staged one, THEN additivity of two factors implies that the factors affect separate stages. But if one has a non-stage model, this inference from additivity does not follow. And as I have said, additivity does not imply Sternbergian stages because, as Roberts and Sternberg show, additivity is compatible with cascaded i.e. nonstage models.
So given that no one, including Sternberg, argues that additivity implies staged processing, I am not quite sure what you see as the the new point your paper is making.
Max
Dear Max
The contribution of the paper is to offer a worked example, in a specific domain, of an important point – that additive factors do not imply discrete processing stages. You are quite correct that making inferences from additive factors requires that you assume discrete stages. Hitherto I had not sufficiently appreciated the extent of qualification made by AFM theorists. Prompted by your comments I have revisited the papers I cite in Stafford & Gurney (2011) as being examples of the employment of the AFM (Pins & Bonnet, 1996; Woodman et al., 2008). Looking at them now they are more circumspect than I thought, or than I recall during conference presentation, in the claims they make using the AFM. (There is a strong possibility that this memory is coloured by my own ignorance at the time of encounter, of course).
All this said, the remaining enthusiasm for discrete stage processing accounts surprises me. That you need to assume Sternbergian stages before you can employ the AFM to make inferences about those stages increases that surprise. I would hope that connectionist modelling, in the tradition begun by McClelland (1979) and continued by our paper, helps to erode confidence that this is a useful assumption.
Yours
Tom
Thanks, Tom. I am not sure that there is much contemporary enthusiasm for models that are stage models in the Sternbergian sense. Most current computational models of any domain of cognition posit cascaded processing (and so are not stage models). Many of these also posit feedback (and so are doubly not stage models). For this reason, I think we’d find if we looked that the AFM is not much used in contemporary cognitive psycholog research.
Best,
Max