Connectionist Models Of Development Research Paper

1. Techniques

There are several neural network techniques that have been applied to developmental issues. Virtually all of them employ layered, feed-forward networks in which activation is passed forward from input units to hidden units and on to output units. Sometimes there are multiple layers of hidden units, other times no hidden units at all. These networks learn by processing examples of input–output pairs. Presentation of an input vector of activation results in a vector of output activation. Hidden and output unit activations are typically determined by summing the products of sending-unit activations and incoming weights and passing this weighted sum through a nonlinear activation function, such as a sigmoid or hyperbolic-tangent function. A linear activation function typically is used for continuous outputs.

In supervised learning, error at the output units is computed as the sum of squared differences between outputs and targets. Weights are adjusted to reduce this discrepancy, using the delta rule in which weight change is proportional to error. In multilayer net-works, error is typically propagated backward to earlier layers of weights, a technique known as back-propagation. Unsupervised learning can also occur, without output targets, as networks learn to group together similar input patterns.

Networks can be designed by hand and remain static, or generated by a learning algorithm. One such generative algorithm, called cascade-correlation, grows by recruiting new hidden units that have learned to be active when the network is experiencing a high degree of error. Back-propagation of error is un-necessary here because incoming weights to a recruited hidden unit are frozen when training of the single-layer of output-unit weights resumes. Unit recruitment corresponds roughly to the formation of new neurons and new synapses.

A few developmental models employ auto-associator networks, in which units are fully interconnected (no layers) and learning occurs with either the delta rule or Hebb rule (in which weights are strengthened in proportion to the product of the activations of the sending and receiving units).

2. Cognitive Stages And Perceptual Effects

One of the first developmental connectionist models concerned a task that has become a benchmark for modeling developmental stages—the balance beam. Here a child is presented with a rigid beam on which a number of equal-sized weights are placed at certain positions to the left and right of a central fulcrum. The child is asked to predict which way the beam will tip when supporting blocks are removed, or whether the beam will remain balanced. A back-propagation model trained on examples of balance beam problems captured the first three stages observed with children (the use of only weight information, distance in-formation when both sides have equal weight, both weight and distance information without resolving conflicts), but failed to settle into the final stage of mostly correct solutions (McClelland 1989).

A cascade-correlation model progressed through all four stages of balance beam performance and also captured the effect that problems with small weight distance differences from one side to the other are relatively difficult at each stage (Shultz et al. 1994). Such perceptual effects are pervasive in cognitive tasks, but have not been comprehensively explained. Connectionist models produce these effects naturally in their method of passing activations.

There are several of these perceptual effects in the vast literature on conservation acquisition: the problem size effect (small number problems are easier than large number problems), length bias (nonconservers choose the longer row as having more), and the screening effect (even nonconservers assume conservation when the transformation is hidden from view). A cascade-correlation model of conservation captured these phenomena, along with natural conservation acquisition and sudden spurts in performance, while showing how perceptual and cognitive information, although initially in conflict, are eventually integrated (Shultz 1998).

A modular cascade-correlation model of seriation (sorting) captured all four stages that children go through (random performance, partial sorts, complete trial-and-error sorts, and systematic sorts) as well as the fact that smaller differences between items make the task more difficult (Mareschal and Shultz 1999).

3. Transition Mechanisms

Perhaps the most difficult issue in developmental psychology concerns how children progress from one stage to the next. The longstanding debate on nativism vs. empiricism has been enlightened by connectionist analyses of different ways in which innate influences can be implemented—as neural representations, architectures, or timing of development (Elman et al. 1996). Interactions between learning and inheritance have been modeled by having successful neural networks reproduce over generations, showing that evolutionary progress is enhanced by learning ability (Nolfi et al. 1994). This belies the common assumption that connectionist approaches to development are strictly empiricist.

A constructivist approach to development has been implemented in several simulations using the cascade-correlation algorithm. These simulations show that it is possible to escape from Fodor’s paradox of cognitive development—that experiential accounts of development are incoherent because it is computationally impossible to learn anything genuinely new. Generative networks escape from this paradox by in-creasing their computational power when needed (Mareschal and Shultz 1996).

4. Non-Normative Stages

A curious feature of development is the presence of early nonnormative stages. Why would children regularly construct incorrect solutions to a problem? In one example, length bias in nonconservation resulted from a correlation between number and length for transformations that preserve density (Shultz 1998).

A cascade-correlation model of the integration of velocity, time, and distance cues for moving objects progressed through the non-normative stages observed with children on their way to the normative integration based on products and ratios of cues (Buckingham and Shultz 2000). For example, children first view velocity as being proportional to distance traveled, then as the difference between distance and time, and finally as the ratio of distance to time. Such stages are a natural consequence of the growth in computational power in cascade-correlation net-works. Initially, these networks lack the power to represent the nonlinear relations among velocity, time, and distance and the ability to simultaneously rep-resent the various directions of inferences based on these equations. Static feed-forward networks that adjust their weights, but do not grow, are unable to capture these stages.

5. Developmental Lags

Different methods of assessing stages in children sometimes produce different results. One example concerns the age at which infants develop the notion of permanent objects, objects that continue to exist even when they are no longer being experienced. Piaget had found that infants below about 9 months would not retrieve an object from under a screen, but Baillargeon has found that infants as young as 3.5 months look longer at events in which a moving object violates permanency, such as failing to emerge between occluding screens. A modular network model of infants’ visual tracking of moving objects learned to predict the emergence of an object from behind an occluding screen (Mareschal et al. 1999). An unsupervised module learned to recognize objects, a recurrent back-propagation module learned to predict object trajectories, and another back-propagation module learned to retrieve objects by integrating information from the other two modules. The developmental lag was explained by noting that reaching requires integration of information about both the location and the nature of the object, whereas prediction requires only information about its location.

6. Modularity

With verbal theorizing, it is difficult to know when to claim that the operation of distinct modules is required for cognitive processing. Computational modeling suggests modularity whenever a homogenous processor fails to learn the task or capture the psycho-logical data. Modular systems are known to simplify learning and processing of particularly difficult tasks.

Another example of modularity is a model of word learning using an unsupervised network to learn categories and an autoassociator network to learn category names (Schyns 1991). The output of each module served as input to the other. This model captured several phenomena in children’s concept learning: strong recognition of a previously unseen prototype, semantic overextension, interword competition, faster learning of a label for a distinctive category, and faster learning of a category with a distinctive label.

The seriation model presented earlier provides a third example of modularity, with one network module selecting a stick to move and another selecting a destination. Homogeneous networks did not progress through the proper psychological stages and did not learn to sort successfully.

7. Self-Organization

Self-organization refers to spontaneous ordering tendencies in complex systems. Both brains and artificial neural networks, because of their complex dynamics, parallelism, and responsiveness to feedback are likely candidates for self-organization. Indeed, neural net-work models can be used to investigate self-organizing properties of the brain.

An example is a model with only input and output layers, a Hebb learning rule, and competition among output units that simulated the formation of ocular dominance columns in visual cortex (Miller et al. 1989). Groups of neurons in visual cortex eventually organize themselves into columns responding to in-formation coming from one eye or the other.

8. Integration Of Diverse Findings

Some connectionist models have been able to integrate large areas of psychological literature that have resisted the explanatory efforts of verbal theories. Examples of conservation and seriation were noted earlier. Another example is a cascade-correlation model of shift learning (Sirois and Shultz 1998). In shift learning, subjects learn a discrimination problem and are then shifted to another related problem to determine what was learned on the first problem. The network model integrated diverse developmental findings on a variety of shifts that could not be accounted for by any one of several verbal theories. The model was based on the idea that older children and adults spontaneously overlearn these problems.

9. Explanation Of Mystery Effects

In smaller, contemporary literatures, there are often results that are difficult to explain. One curious finding concerns asymmetric exclusivity effects in infant habituation to categories of pictures, namely that dogs are not included in the category of cats, but cats are included in the category of dogs. This was captured by

a back-propagation encoder network, and subsequently explained by determining how these effects arose within the model (Mareschal and French 1997). Encoder networks learn to reproduce their inputs on their output units using a smaller number of hidden units. A stimulus is recognized as familiar if it can be encoded onto the hidden units and then decoded accurately onto the output units. Habituation to repeated stimuli results in a decrease in network error; recovery of error to a new stimulus is interpreted as an indicator of novelty. The asymmetric exclusivity effects were explained in terms of the distribution of stimulus features. Because some cat-feature values are inside the range of dog-feature values but not the reverse, a system learning these statistics would naturally view cats as a subset of dogs.

10. Resolution Of Theoretical Disputes

Theoretical disputes are often difficult to resolve at the level of verbal theories. One such recurrent debate concerns whether human cognition is based on symbolic rules and variables, or on sub-symbolic connections. Infant habituation and recovery to sentences in a simple artificial grammar have been covered by variety of standard neural models, without explicit rules and variables that others had argued were required (e.g., Shultz 1999, Sirois et al. 2000). Similar theoretical disputes have been addressed by models of conservation, seriation, and the balance beam presented earlier.

11. Conclusions

Connectionist models have simulated large varieties and amounts of developmental data while addressing important and longstanding developmental issues. Connectionist approaches provide a novel view of how knowledge is represented in children and a compelling picture of how and why developmental transitions occur. Like other modeling techniques, connectionism has increased the precision of theorizing and thus clarified theoretical debates.

As these models become more widely known, it is likely that many more of their predictions will be tested with children. It is also likely that connectionist models will be extended to a wider range of developmental phenomena. Although it is not yet clear whether these models will be able to cover phenomena in social development, there is a promising connectionist model of imprinting (O’Reilly and Johnson 1994). Much of the connectionist developmental literature concerns language acquisition, which is covered in another article. Some features lacking in current models will continue to receive attention: explicit rule use, genotypes, multitask learning, impact of knowledge on learning, embodiment, and neurological realism.

Bibliography:

1. Buckingham D, Shultz T R 2000 The developmental course of distance, time, and velocity concepts: A generative connectionist model. Journal of Cognition and Development 1: 305–45
2. Elman J L, Bates E, Johnson M H, Karmiloff-Smith A, Parisi D, Plunkett K 1996 Rethinking Innateness: A Connectionist Perspective on Development. MIT, Cambridge, MA
3. Mareschal D, French R M 1997 A connectionist account of interference effects in early infant memory and categorization. Proceedings of the 19th Annual Conference of the Cognitive Science Society. Erlbaum, Mahwah, NJ, pp. 484–9
4. Mareschal D, Plunkett K, Harris P 1999 A computational and neuropsychologial account of object-oriented behaviours in infancy. Developmental Science 2: 306–17
5. Mareschal D, Shultz T R 1996 Generative connectionist net-works and constructivist cognitive development. Cognitive Development 11: 571–603
6. Mareschal D, Shultz T R 1999 Development of children’s seriation: A connectionist approach. Connection Science 11: 149–86
7. McClelland J L 1989 Parallel distributed processing: Implications for cognition and development. In: Morris R G M (ed.) Parallel Distributed Processing: Implications for Psychology and Neurobiology. Oxford University Press, Oxford, UK
8. Miller K D, Keller J B, Stryker M P 1989 Ocular dominance and column development: Analysis and simulation. Science 245: 605–15
9. Nolfi S, Parisi D, Elman J L 1994 Learning and evolution in neural networks. Adaptive Behavior 3: 5–28
10. O’Reilly R C, Johnson M H 1994 Object recognition and sensitive periods: A computational analysis of visual imprinting. Neural Computation 6: 357–89
11. Schyns P 1991 A modular neural network model of concept acquisition. Cognitive Science 15: 461–508
12. Shultz T R 1998 A computational analysis of conservation. Developmental Science 1: 103–26
13. Shultz T R 1999 Rule learning by habituation can be simulated in neural networks. Proceedings of the 21st Annual Conference of the Cognitive Science Society. Erlbaum, Hillsdale, NJ, pp. 665–70
14. Shultz T R, Mareschal D, Schmidt W C 1994 Modeling cognitive development on balance scale phenomena. Machine Learning 16: 57–86
15. Sirois S, Buckingham D, Shultz T R 2000 Artificial grammar learning by infants: An auto-associator perspective. Developmental Science 4: 442–56
16. Sirois S, Shultz T R 1998 Neural network modeling of developmental effects in discrimination shifts. Journal of Experimental Child Psychology 71: 235–74