A Psychological Model of Learning in Action
Summary:
Are you really conscious of the actions you make, or just of the effects you intend? The Action-Concept model suggests the latter. Instead of consciously planning muscle movements to hit a baseball, you are actually planning desired action effects and subconsciously backtracking to figure out how to achieve them. According to the model, the brain tightly connects representations of actions and action-effects unconsciously to enable learning and the capacity for goal-directed action.
Actions and Effects: Learn the latter to plan the former
When a newborn baby moves about, he seems to wave his arms randomly, blinks crazily, drools handsomely, and is behaviorally akin to a beached squid. However, over time his actions look less and less erratic (and more and more thirsty). Though random, his actions may have been very deliberate: they helped him to consciously or subconsciously learn how to interact with his adoring environment. If as a mother you were to carelessly offer your supple breast to this clueless infant, he may not yet know that he should grasp it for dear life.{1} The first few times you attempt to feed him, he clumsily baby-handles your breast until he gains a concept of how to work you. But with enough experience of failed and successful grasping, sucking, and general baby behavior, the baby will have enough information to suckle skillfully and perhaps even ravenously.
So goes the theory of the Action-Concept model, devised by Bernhard Hommel, head of the Cognitive Psychology Unit at Leiden University in the Netherlands. According to the model, the brain creates representations of an action and its "action-effects" and bidirectionally associates them with each other in structures called action-concepts. An example of an action is extending your arm, and an example of an action-effect is causing your baby brother to fall over. But other action-effects that may be bundled into the action-concept are the internal sensation of your muscles flexing, the feeling of baby skin on your fingers, and the external sound of baby-hitting-floor. An action-concept is therefore a perception-action unit that forms the basic building block of the brain's ability to process actions.
Per the action-concept model, once the baby learns the relevant action-effects, he has the ability to carry out intentions, so he can consciously choose to suckle a particular nipple and his brain will do the behind-the-scenes work of activating appropriate motor codes.
Two Phases: Automatic Acquisition and Voluntary Action
Associated with the action-concept model is a two-phase model of action control. Action-effects are discovered in the first phase and employed in the second. This two-phase process looks like this:
Phase 1: Action-concept acquistion. The baby performs an action, observes the outcome, and automatically associates the motor code with the action-effect code.
Motor System Cognitive System
+-----------+ +-----------+
| * - * | | * - * |
| / \ / \ | | / \ / \ |
| * - / - * | -> outcome -> | * - / - * |
| \ / \ / | | \ / \ / |
| * - * | | * - * |
+-----------+ +-----------+
^ ^
action code action-effect code
Phase 2: Action-effect-based movement. The baby performs an action by selecting the desired action-effect, and the brain automatically activates the associated action code.
Motor System Cognitive System
+-----------+ +-----------+
| * - * | | * - * |
| / \ / \ | | / \ / \ |
| * - / - * | <------------ | * - / - * |
| \ / \ / | | \ / \ / |
| * - * | | * - * |
+-----------+ +-----------+
^ ^
action code action-effect code
It is important to note that the perceptual part of the structure represents what sensory information the agent (the baby) should perceive (or has empirically perceived) based on the performed action. Therefore in Phase 1, action-effect codes are created by observing the real world, but in Phase 2, desired action-effects are selected to change the real world. Furthermore, an action-effect code might be compared to new observed effect codes to determine whether or not an action is successful or complete.{2}
So not only are actions linked to action-effects, but action-effects are linked to actions. If you carry out an action, the brain can use the associated action-effect codes to anticipate the results of that action: you execute the action "touch the pot on the side" and you automatically anticipate a sensory event equal to the action-effect "ouch, that's hot a little bit." Alternatively, the brain can use action-effects to plan an action: you desire the action-effect of "a warm hand" and you execute the associated action action of "grasping the pot of coffee." Because of its bidirectional quality, the Action-Concept model provides a foundation for talking about not only learning and instinctive response, but also voluntary action and performance of practiced events.
Is there a difference betwen kinesthetic action-effects (e.g. the internal sensation of moving your finger) and externally generated action-effects? This is one aspect of the model that distinguished it from other models: there is no qualitative difference between proximal and distal action-effects. If the action of flipping a light switch will produce a feeling of pressure on your index finger as well as the sensation of more light in the room, these feature codes of the action-effect are considered qualitatively identical.
But perhaps more importantly, this models differs from other models of action and intention in that it makes action-effects inherent to the learning process and gives them a starring role in voluntary action. Some other models suggest that action-effects are not integral to choosing actions, but are rather used to determine if responses are good or not via a feedback system. In other words, the quality of the action-effect perceived--annoying, pleasureful, etc.--strengthens or weakens the connection between a stimulus and a response. It's a one phase process where: you are prompted to act, you act, and you determine if you acted well or not:
stimulus --> response (action) --> action-effect
^ |
| |
\------------------------------/
Following this model, if a baby is presented with a nipple (stimulus) and it bites it (response) and the mother spanks the baby (action-effect), the fact that the action-effect was painful would weaken the connection between the stimulus and poorly chosen response.
Contrast this with the action-concept model:
Acquisition: action --> action-effect
Intention: action-effect --> action
So, presented with a stimulus, the baby considers action-effects it would like to achieve, and then carries out the action. The following diagram shows that the action-effect doesn't strengthen the connection between stimulus and action; rather, it further develops the association between action-effect and action (the action-concept):
/-------------------\
| |
v |
stimulus --> action-effect <---> action --> action-effect
^ ^
(desired/associated) (perceived)
The baby learns that biting results in a
spanking and sucking results in less
hunger. After it acquires these associations, it can in the future perform actions by activating the desired action-effect rather than using on a stimulus-to-response connection that has been augmented or diminished by an action-effect.
Interesting Concurring Experiments
So what experiments lend validity to this model? Here are a few interesting ones (in order of the complexity of my explanation):
The goal-oriented rat experiment. Rats are famous for their maze-solving experiments. They can learn to solve mazes (given a cheesy incentive) and the mazes and rats can be relatively easily "adjusted" to see how learning works. But do rats memorize a sequence of specific actions requires to complete their maze, or do they remember goals? Experiments showed that rats are goal-oriented and do not need to relearn a maze, for example, when a walkable path became only crossable by swimming. In other words, a breakdown in a key feature of the maze if rats were action-oriented (no longer being able to walk a route) did not befuddle the rats. This suggested that rats learn mazes by goals or action-effects. After turning a corner, instead of thinking "I remember walking 10 little rat paces," the rat thinks "I remember needing to get to that cheese around the corner." If there's a solid path, that desired action-effect is turned into a happy rat gait; if there's water, that desired action-effect is turned into a silly rate stroke.
The (not-so) skilled typist experiment. According to the model, actions are automatically associated with their action-effect features. But contrary to the model, one would think that skilled typists, for example, should be able to type F or J on command without considering what side of the keyboard the letter is on. The action-effect of typing F or J and the action of pressing the left or right index finger should be directly linked because the action is so routine and simple. (In E2 speak, F should be hardlinked to pressing the left index finger.)
But the action-concept model predicts that, when learning typing and typing normally, a feature such as on what side of the keyboard a letter appears is automatically associated with the action of pressing that letter, not just what finger is used to type the letter. To test this, subjects were asked to type F's and J's which would appear on the left- or right-hand side of the screen in an attempt to activate an action-effect feature that would either be consistent or inconsistent with the desired action. If the model was wrong, skilled typists would type the letters just as fast regardless of what side of the screen prompts them to type the letter. If the model was right, skilled typists perform worse if the letter appeared on the side of the screen opposite to the hand used to type them.
Interestingly, skilled typists were slower and less accurate at typing F's and J's when they appeared on the side of the screen opposite to the hand used to type them. The results suggests that typists have to suppress an action-effect feature (that the letter is on a particular side) in order to appropriately execute the typing action. "Thus," concludes Hommel, "practice neither eliminates spatial codes from action concepts nor seems to permit stimulus information to circumvent the stage where action concepts reside" (Hommel 1998).
The free acquistion, forced action tests. This experiment directly tested the two stages of the Action-Concept model. In the first part of the experiment, subjects were asked to randomly press the right or left keys when they were prompted by a white rectangle. They were informed that they would hear a tone after pressing each key, but that this was irrelevant to the test and should be ignored. One half of the subjects would hear a low tone after pressing the left key, and a high tone after pressing the right key (map A) and the other half would hear the opposite key-tone mapping (map B). They performed 200 (boring!) acquisition trials to become familiar with the action of pressing the right and left keys.
In the second part of the experiment, subjects were instructed to respond to 100 tones as instructed. The subjects, however, were divided into two groups: the reversal and the non-reversal group, each consisting of equal numbers of map A and map B subjects. In the non-reversal group, subjects were asked to respond to a tone with the key consistent with their practiced mapping. (For example, map A subjects were asked to press the left key after a low tone and the right key after a high tone.) In the reversal group, subjects were asked to respond to a tone with the opposite key. (For example, map A subjects were asked to press the left key after a high tone and the right key after a low tone.)
The nonreversal group responded faster with less errors in the test phase, as expected. It makes sense that they had associated the keys with the same tones that were used in the test phase. However, the experiment had another aspect. In one series of test phases, after hearing the prompt tone and pressing the key, the subjects would hear the practiced key tone. (For example, a reversal map A subject would hear a high tone, press the left key as instructed, then hear the low tone due to pressing left.) This way they could not ignore the action-effect. In another series, the subjects would not hear the practiced tone after each key press. This way they had a chance of "practicing out" the previously acquired action-effect. Non-reversal subjects always responded in the test phase within about 350 ms, with or without the after-tone. With the after tone, reversal subjects responded within about 500 ms without much improvement over time. Without the inconsistent after-tone, however, reversal subjects quickly started responding within about 400 ms., a considerable improvement, but suggesting the earlier acquired inconsistent action-effect is not easily erased.
The result of the experiment indicated not only that the unconscious acquisition of action-effects help a subject choose actions (as predicted by the action-concept model's bidirectionality, but not by other models), but also that the consistency of a known action-effect has a considerable impact even when the subjects if forced to choose.
Conclusion and Ponderables
The key ideas of the Action-Concept model are that action-effects are automatically associated with actions, the association is bidirectional, and we can plan our actions by activating desired action-effect codes instead of motor codes (relying on the birectionality of the mapping). Action concepts are generated subconsciously: you will associate effects with actions even if you are told not to, and those associations will improve or inhibit your actions in the future (see the second experiment). Furthermore, even given a routine, simple task, action-effects will still be automatically gathered and practice does not erase action-effects features (see the first experiment).
Action-effects may be used to plan action in a few ways:
- to determine if an action is successful - we can compare the observed effect code of an action to the desired action effect code
- to determine if an action is complete - (same as above)
- to plan complex actions - by activating action-effect codes in sequence (see the rat experiment)
The great thing about this model, as simplistic as it is, is that it makes sense and (a source a great relief for me) still allows for free will. (Unfortunately, it does nothing to even suggest an explanation for free will--how we might choose action-effects to activate actions. But isn't free will more wondrous if it can't be explained?) It makes sense that a baby looks around randomly without purpose and slowly starts to make more sensical movements as it gains experience. It makes sense that we cannot perform many tasks well the first time, as simple as they are, because we have not yet acquired experience in executing them. And it makes sense that rather than consciously planning our muscle movements to type an email, we may instead think of desired action-effects and use the bidirectionality of the action-concept to subconsciously link our action-effects to our muscle movements.
Where I leave you, the reader, stranded is with regards to the moral implications of the Action-Concept model. If action-effects and actions are automatically linked based on experience, then what if someone's well-meaning action-effects lead to bad actions because of bad experience? Should a police academy be faulted instead of the police officer for a wrongful shooting? Can video game companies be sued for violence by children supposedly conditioned by those games? How much weight should we give intentions versus actions if experience rather than thinking has primacy, and our consciousness, for which we are culpable, plans action-effects rather than actions?
References:
- Hommel, B. (1998). Perceiving One's Own Actions. In J. S. Jordan (Ed.), Systems Theories and A Priori Aspects of Perception (pp. 143-179). Elsevier Science B.V.
- Elsner, B., & Hommel B. (2001). Effect Anticipation and Action Control. Journal of Experimental Psychology: Human Perception and Performance, 25, 229-240.
Notes:
- He probably also wouldn't know he'll never get this opportunity again for at least another fifteen years.
- Coders, think of a simple while loop comparing what is observed to what is desired.