Learning as self organization

The greatest wonder of organic life is that it develops, in seeming defiance of all the laws of probability, from the simple to the complicated, from systems of lower to systems of higher harmony. However, infractions of the second law of thermodynamics simply do not occur, and, to achieve what it does, life is dependent on a gradient in the general, all-pervading flow of energy dissipation. Life lives on negative entropy. Any living species is a system that, very much like a prairie fire, greedily gathers energy and, in a positive feedback cycle, becomes able to gather the more energy, and to do it the quicker, the more it has already acquired

The Behavioral Level: Learning

1 Respondents, Operants, and Emergents

Duane M Rumbaugh, David A Washburn & William A Hillix

2 Analysis of Behavioral Selection by Consequences and Its Potential: Contributions toUnderstanding Brain-Behavior Relations

William J McIlvane, William V Dube & Richard W Serna

3 Mechanics of the Animate

Peter R Killeen

4 The Response Dimension

Peter R Killeen and Lewis A Bizo

5 Nonlinear Phenomena in Learning Processes

Michael Stadler, Günter Vetter, John D Haynes & Peter Kruse

6 The Attractor of the Intentional Learning System

Lillian Greeley

II

The Network Level: Self-Organization

7 The Three Languages of the Brain: Quantum, Reorganizational, And Associative

Subhash C Kak

8 Automatic Formation of Wavelet- and Gabor-Type Filters in an Adaptive-Subspace SOM

Teuvo Kohonen

9 Democratic Reinforcement: Learning via Self-Organization

Dimitris Stassinopoulos and Per Bak

10

Biological Plausibility of Synaptic Associative Memory Models

Daniel L Alkon, Kim T Blackwell, Garth S Barbour, Susan A Werness, and Thomas P Vogl

Topographically Different Regional Networks Impose Structural Limitations on Both Sexes

in Early Postnatal Development

Harriett W Hanlon

14

Brain Regions Associated With Retrieval of Structurally Coherent Visual Information

Daniel L Schacter, Eric Reiman, Anne Uecker, Michael R Polster, Lang Sheng Yun & Lynn A Cooper

Emotion and the Self-Organization of Semantic Memory

Daniel D de Grandpre and Don M Tucker

IV

The Social Level: The Organization of Self in Society

Self-Organization and the Social Collective

Raymond Trevor Bradley and Karl H Pribram

Joseph S King and Karl H Pribram

The Behavioral Level: Learning

1.

Respondents, Operants, and Emergents

Duane M Rumbaugh

Toward an Integrated Perspective on Behavior

Duane M Rumbaugh, David A Washburn, and William A HillixGeorgia State University and San Diego State University

Key Words: respondents, operants, emergents, competence, comparative

A triarchic organization of behavior, building on Skinner’s description of respondents andoperants, is proposed by introducing a third class of behavior called “emergents.” Emergents arenew responses, never specifically reinforced, that require operations more complex thanassociation Some of these operations occur naturally only in animals above a minimum level ofbrain complexity, and are developed in an interaction between treatment and organismicvariables (Here complexity is defined in terms of relative levels of hierarchical integration madepossible both by the amount of brain, afforded both by brain-body allometric relationships and

by encephalization, and, also, the elaboration of dendritic and synaptic connections within thecortex and connections between various parts/regions of the brain.) Examples of emergents arediscussed to advance this triarchic view of behavior the prime example is language Thistriarchic view reflects both the common goals and the cumulative nature of psychologicalscience

Respondents, Operants, and Emergents: Toward an Integrated View of Behavior

Scientific psychology has been accused of failure to grow theoretically Its critics claim that we

do not integrate prior findings and explanations into contemporary perspectives (see, forexample, the discussion and rebuttal by Posner, 1982) A goal of good science is progress,whether reflected in cumulative theoretical development, or through Kuhn’s (1962) paradigmaticrevolutions, cyclic and dramatic changes that are likely to exclude many central tenets of theprevious theoretical regime in favor of “more enlightened” or “more accurate” approaches

Science may have moved beyond the phase Kuhn described, in which paradigmatic developmentand rejection were the primary modes of change Kuhn’s unflattering claim that exponents ofdifferent paradigms could not communicate may have been a self- subverting law; scientists whoknew about it may have tried harder to eliminate their intellectual provincialism Technologicaladvances like the “information highway” have countered most of the contribution thatgeographical distance made to intellectual distance In any case, the present article is an attempt

to circumvent revolution to achieve cumulative progress

Psychology was (R I Watson, 1967), and perhaps still is, in a preparadigmatic stagecharacterized by a failure to agree sufficiently on the fundamentals to qualify for a Kuhnianparadigm If we are right about the progressive substitution of cumulative science forparadigmatic revolutions, psychology may move smoothly from pre–paradigmatic to post–paradigmatic status without ever clearly having a Kuhnian paradigm

Whether or not it is philosophically justifiable, it is trendy to discuss the “cognitive revolution”kindled in the 1950’s and 1960’s and evident in the current popularity of cognitive science.Behaviorism may not have been a true paradigm, but in any event cognitivists tended tochallenge, discount, or ignore five decades of research in the behaviorist tradition Conditioning,schedules of reinforcement, and similar topics once esteemed by behaviorists are rarelydiscussed in treatments of human cognition; rather, they receive limited attention in introductoryand animal learning texts Ironically, if behaviorism did have a kingly paradigmatic head thatcognitivism has chopped off, its crown of objective methodology remains firmly in place

It is true that behaviorism’s metatheoretical commitment to associationism (Marx & Hillix,1987) has been challenged by the camp of cognitivists most closely related to traditionalcomputer science and artificial intelligence However, the parallel distributed processing camp,technologically advanced and sophisticated though it is, relies on a connectionism that isfundamentally the same as that of Edward Thorndike (1898) or John B Watson (1919).(Connectionists frequently do try to identify within hidden layers the rule-like patterns thatmediate stimulus-response associations patterns that are consistent with the thesis advancedhere.) The historical roots of the connectionistic movement are often overlooked; even the verydirect ancestors of parallel distributed processing (Selfridge, 1955, 1959; Rosenblatt 1958, 1962)are seldom cited.

Although there is thus a recidivistic/modern side of cognitive psychology, the present thesis isthat the rise of cognitive psychology represents substantial progress—not just change As oneway of recognizing this progress, we suggest a trichotomous classification of behavior thatrecognizes and adds to Skinner’s (1938) distinction between respondent and operantconditioning, while continuing to acknowledge the importance of antecedents, behavior, andconsequences in psychological research At the same time, we assert that there exist complexprocesses and determinants of behavior that go beyond those involved in operant or respondentbehaviors These emergent processes should not be confused with species–typical behaviors(instincts) that are fundamentally unlearned adaptations, such as imprinting, taste aversion (i.e.,bait shyness), and courtship and migration patterns (see Alcock, 1979)

Consequently, we propose that a third category of behavior, emergents, be defined to extend thedomain of inquiry for those who espouse an experimental analysis of behavior The recognition

of emergents will provide a unifying link connecting the several camps (e.g., behaviorist andcognitivist) that try to understand behavior through empirical, systematic research that identifiesthe antecedents and consequences responsible for the appearance, morphology, anddisappearance of responses This “new” class of behaviors is particularly likely to appear inorganisms possessing cerebral complexity (see earlier definition) and encephalization (i.e., theextraordinary elaboration of the cortex relative to the rest of the brain; see Stephan, Bauchot, &Andy, 1970), as within the order Primates

Emergents include alterations in the nature of the learning process (e.g., in the ability to learnrelationally as well as associatively, to form both natural and arbitrary concepts, to recognizeequivalence relations between stimuli that are not specifically trained/reinforced, and to developthe ability to solve novel problems in a single trial) Emergent abilities also enable an organism

to learn to use symbols as representations of things and events not necessarily present, tocomprehend and to use language, to speak and sing, to be able to learn vicariously fromsecondary records (e.g., written materials and other records), and to reflect upon past experiencesand events projected in the future—to mention a few of the salient ones

From a behavioral perspective, these alterations can be properly viewed as emergent responsemodes; from a cognitive perspective, they can be viewed as cognitive operations and structures.Either way, however, these alterations have properties that reflect the neuroarchitecture,neurophysiology, and neuropsychology of specific organisms as affected by specific experiences,treatments, or rearing conditions

Precedents in the history of thought have led the present authors to label this third category oflearning “emergent.” In the 19th century, John Stuart Mill postulated a “mental chemistry” thatcoalesced simple ideas into complex ideas (see Heidbreder, 1933) Emergent complex ideas hadtheir own distinguishing structures and properties and, hence, were more than just a composite ofthe simpler ideas on which they were based In the 20th century, Nissen’s (1954) discussions ofpossibly new and qualitatively different processes emerging as products of quantitativeelaborations of the primate brain directed the senior author of the present report into researchregarding their etiologies.

An interesting question that arises in this connection is whether phylogeny to some extentrecapitulates the ontogeny of human development with respect to emergent behaviors Thesebehaviors, like all behaviors, depend on an interaction of organismic and experiential factors;thus, the full complement of emergents is available only to normal adult humans It may be thatsome animals never get beyond the first stage of human development according to Piaget, thesensory-motor stage Higher stages may emerge in more complex animals An argument can bemade that linguistically trained chimpanzees, orangutans, and gorillas have manifested in rarecases some properties of Piaget’s highest stage, the formal operational stage Some aspects of theintermediate stages are almost certainly seen in nonhuman primates

Another fascinating question is how precisely the fundamental elements of behavior should bedescribed It is well accepted that the formation of associations is one basic mental capacity Thisinvolves one type of memory The ability to compare stimuli with respect to various properties size, color, shape, and desirability, for example seems to be an emergent capacity Severalresearchers, from Krechevsky (1932) to Levine (1971) have presented evidence that animalsfrom rats to humans are able to generate and test hypotheses about the relationships betweenstimuli and reinforcers These are only two of many possible emergent capabilities that might besuggested

Before distinguishing emergents from Skinner’s respondents and operants, consider theirimportant dimensions of commonality First, they are all forms of behavior Second, thebehaviors are observable and measurable Third, all three are taxonomic groups of behaviors Assuch, they categorize behaviors so that they can be better understood and studied with tacticsappropriate to their defining features It is important to note, however, that, as categories ofadaptive behaviors, they are not to be confused with scientific explanations Fourth, eachcategory has antecedents and consequences that must be defined as parameters of behavior ifvalid scientific descriptions and explanations of the form and continuance of behavior are to beobtained Fifth, none of the three categories can be accounted for satisfactorily by, or reduced to,the operations of any two of the other categories Generally, respondents and operants providethe foundation for emergents; stimulus equivalence relationships, or expectancies (Tolman,1959) may also be considered part of this foundation; alternatively, means-end readinesses andthe expectancies on which they are based can themselves be regarded as emergents

Brief definitions of each category of the behavioral trichotomy are as follows:

I Respondents

Respondents are responses that are elicited, without prior training, by the presentation of specificstimuli, called “unconditional stimuli” (UCS) or their conditional associates It is reasonable toview respondents as being basically unlearned, reflexive responses elicited by specific stimulithat organisms encounter in the natural world All other things being equal, one can predict withconsiderable confidence the form and continuance of a respondent upon its initial elicitationgiven the identity of the subject’s species, its state and context, and the specifics of the UCS For

a given species, set of circumstances, and UCS, a respondent is very likely to recur time aftertime in the same form Generally, a respondent requires only the impact of the UCS upon a givenspecimen, not upon that specimen’s history of reinforcement with the UCS Pavlovianconditioning involves respondents; the reinforcer is a stimulus, the UCS, that is correlated with

an initially neutral stimulus, the conditional stimulus (CS) The UCS both elicits the respondent

to be conditioned and serves as the reinforcer After repeated presentations of the CS–UCS pair,the CS will tend to elicit a response similar to, though not in detail identical to, the responseelicited initially by the UCS

II Operants

In contrast to respondents, operants are responses that are emitted by the organism and that aremodified by their consequences There is no readily definable UCS that elicits the operant to beconditioned Rather, the response is initially emitted with apparent spontaneity by the subject and

is not directly produced by specific operations of the experimenter The operant can come to beoccasioned by an initially neutral stimulus—a discriminative stimulus (SD)—that functionssomewhat analogously to the CS in respondent conditioning Operants function by operatingupon the environment and are selected by the reinforcing properties of the environment (e.g., thelocations of nourishment, contrasted with sources of pain and trauma) Reinforcers for operantscan be any external stimuli that increase the probability that the operant will be emitted.Consequently, by contrast to respondent conditioning, where the reinforcer is a rather specificUCS, in operant conditioning any of a number of consequences (e.g., things and events) mightsustain the acquisition and continuance of an operant

In the case of both respondent and operant conditioning, the presentation of the antecedentstimulus may provide a necessary context for the conditioned response to be manifested (i.e, for

a discriminated operant, or for a respondent) Their learning entails reinforcers as consequences.There are several different types of procedures for both respondent and operant conditioning, andfor schedule–of–reinforcement effects, that are beyond the scope and purpose of this paper.III Emergents

Emergents are new competencies and/or new patterns of responding that were never specificallyreinforced by operations of the experimenter They are not relatively simple, unitary responses(e.g., salivating, eye blinking, jumping over a hurdle, pecking a target, pressing a bar, or evenchains of such behaviors) as in the case of respondents and operants Several good examples ofwhat we call emergents are presented by Sidman (1994; see Rumbaugh, 1995, for a review)stimulus-equivalence paradigm, in which as a result of a few specifically reinforced responses torelationships between specific stimuli, a substantially larger number of unreinforced relations can

be obtained that, in turn, demonstrate “stimulus equivalence,” defined by the properties

of reflexivity, symmetry, and transitivity These associations have been described by Sidman ashaving “emerged”; hence, their classification here as emergents is congruent with Sidman’s view

of them

Emergents occur in a variety of contexts, in addition to that of Sidman’s stimulus equivalenceparadigm These examples of emergents will be discussed subsequently, but each of them has incommon the following attributes: (1) All emergents are forms of silent learning—by which it ismeant that learning or acquisition of new response patterns or the cultivation of newcompetencies (i.e., the emergents) might progress with no obvious manifestation (In reference tovarious aspects of inhibition, excitation, second order conditioning, and so on, Flaherty, 1985,pages 126–127, uses the term “silent” in his discussions of kinds of learning that go unnoticedunless special tests are instituted.) Emergent behaviors/competencies may go unmeasured, if notanticipated, unless the subject is tested in unique/altered contexts for transfer of learning andnovel patterns of behavioral adaptation However, subjects may spontaneously manifestemergents if, during training, they markedly alter their responses in a way that is both novel andextraordinarily adaptive (2) The emergent behaviors/skills were never intentionally orsystematically reinforced as part of the experimenter’s treatment procedures (3) The emergentbehaviors/skills are established through induction, so it would appear, by the organism Again itshould be noted that emergents sometimes surprise the observer when they first appear—aconsequence of the fact that they were not specifically reinforced or trained by the experimenter.(4) Emergents are noted for their apparent appropriateness to new situations Emergents canmake their appearance in new contexts which only in principle are similar to those in which theyformed They generalize between contexts not on the basis of the of specific stimulusdimension, as in stimulus generalization, but rather on the basis of relationships between stimuliand/or rules The relationships and/or rules referenced here can be between any kind or number

of elements (stimuli, responses, reinforcers, etc.) that are shared by two or more contexts

Interim Summary Although emergents, like operants and respondents, provide for adaptationand generally gain in strength with time and experience, only emergents are characterized bytheir complexity (e.g., heirarchical integration and creativity) and by their adaptive value inhighly novel contexts These contexts must be novel enough that, as posited above,generalization on traditional stimulus and response dimensions cannot provide a sufficientaccount for the response Additionally, whereas operants, respondents, and emergents all depend

on antecedents and consequences, and are sensitive to contingencies, emergents are not asreadily accessible to the experimenter for specific shaping by consequences as are operants.Hence, emergents are distinguished from respondents and operants in that they can appear innovel, unanticipated forms that frequently appear to be clever, creative, and, indeed, smart.Emergents differ from respondents and operants in still other important characteristics: Whereasboth respondents and operants are relatively specific responses that can become conditioned toinitially neutral stimuli, emergents are modes of responding or solving problems that are not

“forced” by specific antecedents/stimuli, such as a UCS or SD Also, the overt motoric responseentailed in the conditioning of respondents and emergents is fundamentally the same as theresultant conditioned response, whereas an emergent response might be strikingly different fromthe behavior manifested by the subject during the training experiences that generated theemergent response Whereas overt motor responses are generally required by the subject for the

conditioning of respondents and operants (sensory preconditioning is a notableexception), emergent responses can be learned silently by an apparent passive subject throughobservation Finally, the learning of respondents or operants can be easily charted, for example

by a cumulative recorder, whereas the formation of emergent response modes may not bediscernible, because neither their formation nor their probability of later emission necessarily areindexed by concomitant behaviors

These distinctions between respondents, operants, and emergents are summarized in Table 1.Most important, however, is that emergents are much more likely to be revealed in treatment Xorganismic interactions, where “organismic” refers to both between– and within–speciesvariables, than are either respondents or operants Some species are able to benefit fromtreatment conditions that hinder others, or to which the latter species are oblivious For example,although stimulus equivalence training can generate reflexive, symmetric, and transitive relations

in normal 4-year- old children, it did not in rhesus macaques (Macaca mulatta; Sidman, Rauzin,Lazar, & Cunningham, 1982)—though that is not to conclude that macaques are incapable ofstimulus–equivalence relations After appropriate training on other pairs of numerals, Rhesusmacaques can choose the larger of two numerals, never before encountered as a pair, and,thereby, obtain the greater number of reinforcers (Washburn & Rumbaugh, 1991, p 191; seedetails below) This behavioral skill, like the acquisition of symmetric relationships, requires anadvanced brain, but not one so advanced as that of the human child

in Table 2 do not exhaust those available from the literature, and future research will surelydefine additional ones

See Table 2.

Examples of emergents

Learning set, defined by Harlow in his classic paper of 1949, operationalized procedures whichresulted in the transformation of rhesus subjects from trial–and–error associative learners to one–trial, seemingly insightful, problem solvers Complexity of the brain across species and integrity

of the brain within species, along with levels of maturation, were demonstrated to bepowerful organismic variables which, in interaction with the treatment of learning–set training,affected the probability that one-trial learning capabilities would emerge The ability to choosethe correct (reinforced) one of a pair of novel stimuli at nearly the 100% level after a single

“testing” trial was the terminal point of learning set formation From the cognitivist perspective,the organisms capable of learning set formation had learned an emergent strategy, “win– stay,lose–shift,” that they applied to each new pair of stimuli From a connectionist perspective, they

had learned to strengthen or reduce associative strength to stimulus cues enough in a single trial

so that they could choose the stronger association at near 100% levels after that trial Part of thereason for that might be that all increments or decrements in associative strength were attached

to the cues offered by the discriminanda rather than to other “error factor” cues like right vs leftposition

Transfer of learning research has a long and rich history Transfer of learning is quantified on acontinuum that extends from strongly negative (e.g., transfer slows learning), through null (e.g.,

no transfer), to strongly positive (e.g., transfer facilitates new learning) Brain complexity, asrepresented within the array of species that comprise the order, Primates, is also a continuum thatextends across several levels When one examines transfer–of–training effects in reversallearning as a function of amount learned prior to the test for transfer, one finds a remarkableeffect—transfer for prosimians with their relatively smooth, small brains becomes increasinglynegative as pre–test trials increase, whereas the more encephalized, large-brained primates’transfer can become increasingly positive The interactive effect between treatment (i.e., amountlearned prior to transfer) and the organismic variable of brain complexity qualitatively alters theessence of the transfer effect (Rumbaugh & Pate, 1984) This phenomenon may be related to,and certainly confirms, the connection between brain complexity and the ability to form learningsets In both cases, organisms with more complex brains are better able to “escape the bonds”formed by previous learning in order to form new associations quickly Cognitively speaking,more complex organisms learn to identify “relevant” and discount “irrelevant” cues better thanless complex organisms

Learning processes also vary in relation to levels of brain complexity within the order, Primates.Primates with relatively smaller and simpler brains learn in accordance with the traditionalstimulus–response associative models that apply best to the establishment of habits of responding

to reinforced stimulus choices and of not responding to unreinforced stimulus choices in amultiple-problem, two-choice, discrimination- learning situation Whereas some primates withrelatively larger brains and cortical elaborations apparently learn as stimulus– response learners,others can learn in accordance with a mediational or relational model which enables the subject

to take, for example, discrimination–reversal test trials seemingly as a continuance of the initialdiscrimination task (Rumbaugh and Pate, 1984) In other words, they discount the fact that thecue values of the discriminanda have been exchanged and continue to improve in the execution

of choices These emergent response modes alter transfer–of–learning effects, and the essence ofthe discrimination learning process itself They are not a consequence of procedures used by theexperimenter to establish such modes Rather, they emerge as a consequence of how brains ofgreater and lesser degrees of complexity respond to the same treatments (e.g., the discriminationtasks and tests for transfer)

Of course, organisms that learn relationally do not cease to learn associatively In fact, it seemslikely that, for species with the capacity for relational learning, the propensity for relationalversus associative learning improves both with phylogeny and ontogeny Thus, rhesus monkeyshave demonstrated the capacity for relational learning, but have also failed to extract rule-likerelations from other tasks, responding stubbornly (but generally successfully) according tostimulus-response associations (Filion, Washburn, & Fragaszy, 1995)

Stimulus–equivalence training experiences result in differential outcomes depending upon thespecies (humans are markedly superior to nonhuman primates that, at best, have less ability tomanifest equivalence, symmetry, and transitive relations) and, within humans, upon whether ornot language is operative (Sidman, 1994).

Concept learning for both natural and arbitrary things and events varies markedly as a function

of species and age level, when treatment variables (e.g., tasks) are held constant Emergentbehaviors may include generalized identity matching-to-sample, symbolic matching, andsameness-difference concepts

The representational use of symbols, as an ability, is strongly controlled by brain complexity andthe age at which such training/learning experiences are given to the subject Chimpanzees areclearly capable of using symbols to represent things not present, as indicated by their ability toclassify symbols into appropriate categories (for example, whether the symbol represents a tool

or food; see Savage–Rumbaugh, 1986, for a review of relevant research)

Speech comprehension and the invention of proto–grammar appear to be strongly related both tothe variables of brain complexity (e.g., monkeys, chimpanzees, and children) and rearing (i.e.,treatment) conditions for the subject Kanzi, a bonobo (Pan paniscus) has manifested the ability

to understand novel requests, conveyed to him via sentences spoken by humans, at a level thatcompares favorably with a child whose mental age was 2½ years He also has employed whatwould be termed grammar, if he were a human of 1–1/2 years, in the productive combinations ofgestures and symbols that he uses to communicate complex messages/requests to his caretakers(for details see Savage–Rumbaugh, Murphy, Sevcik, Brakke, Williams, & Rumbaugh, 1993;Savage–Rumbaugh & Lewin, 1994; Greenfield and Savage–Rumbaugh, 1991) It is alsosignificant that Kanzi did not develop his language skills as a result of specific, discrete–trial,reinforced training Rather, his skills were acquired quite indirectly—through observation ofefforts to teach his mother, Matata, to learn the appropriate use of word–lexigrams (i.e.,geometric symbols) and use a “talking” lexigram board Matata, who was then more than 15years old, failed to learn any language skills, quite possibly because she was a feral animal untilthe age of about 6 years For her, the years for the optimal learning of language had long passed.For Kanzi, however, they had not, for he played about in the context within which Matatareceived her scheduled language training from soon after birth to the age of 2½ years Here wehave, then, a prime example of the organismic variable of age (Matata was too old to learnlanguage skills, while Kanzi was precisely the right age, as it turned out) interacting with thetreatment condition that consisted not of language training, but, rather, of exposure to languageusage

Numerical cognition by nonhuman animals provides an additional example of emergents Asmentioned earlier, Washburn and Rumbaugh (1991) reported that rhesus monkeys learnsubstantially more than which of two numerals is the one that pays off the most in food pellets.Two monkeys were trained with all but seven combinations of pairs of numerals 0 through 9;seven pairs were chosen to be used later as novel test pairs to determine whether, during training,the monkeys had learned only to pick one of each specific pair of numerals, or whether theylearned something about the “value” of each numeral If, for example, on a given training trialthey were presented with a 5 paired with a 3, the selection of the 3 would result in the automatic

delivery of 3 food pellets, whereas the selection of the 5 would result in the delivery of 5 pellets.During test trials on the seven new pairs in which the numerals 6, 7, and 9 were eachused twice (i.e., 6:4, 6:5, 7:5, 7:6, 8:5, 9:7, and 9:8), one monkey made no errors on their firstpresentation, and the other made only two errors If they had learned only which numeral tochoose in the context of each training pair, they would not have been able to perform abovechance on the novel pairings Thus the monkeys performed significantly above chance—theymay have learned something like a matrix of relative values.

Alternatively, the animals could have learned a comparison strategy: they could have attached avalue to each numeral as a result of the original training, and learned that they profited most bycomparing each pair of numerals and choosing the one with the larger value In Hull’s theory,these “values” for each stimulus would be called “reaction potential.” In contemporary cognitiveterms, these would be representations of quantities corresponding to the meaning of thenumerals In either case, this type of comparison is a different process from immediatelyresponding to any stimulus that has been reinforced, or even to the stimulus that had the greatesthabit strength

Such an altered response mode was not specifically trained— nor could it have been demanded

It may, however, have been prepared through evolutionary selection for animals that try to obtainbetter nutrients, rather than selecting whatever food is available Notwithstanding, the training, ininteraction with the brain/learning capacity of the rhesus subjects, allowed the ability to executeordinal judgments accurately to emerge, as reflected in their choice of the larger numeral innovel pairings presented for test

Other examples of phenomena from the history of psychology that exemplify emergent responsemodes include latent learning (Blodgett, 1929; Tolman, 1948) and the effects of early rearingenvironments (Riesen, 1982; Bryan & Riesen, 1989; Stell & Riesen, 1987) upon patterns of braindevelopment and complex learning skills, and still others that are listed in Table 2 In theseexperiments, treatment effects interacted with developmental, hence organismic, variables todetermine whether or not learning was manifested subsequent to explorations of mazes withoutspecific reinforcement, or whether learning, language, and speech were compromised as aconsequence of deprivation of appropriate stimulation or of the opportunity to learn atappropriate levels of maturation

Even Epstein’s (Epstein, 1985; Epstein, Kirshnit, Lanza, & Rubin, 1984) simulation of “insight”

in the pigeon illustrates what we call an emergent response mode in this paper Epstein’s pigeon,

in a final test, moved a box into position, then stood on it in order to access a target that wasotherwise out of reach His account detailed the antecedents, but it was the pigeon’s brain thatprocessed the prior training and blended it to allow for a chimpanzee–like solution to a classicproblem (Ellen & Pate, 1986) The importance of experiences relevant to task demands has beenrecognized by researchers with chimpanzees from the days of Köhler’s (1925) classic studies

Notwithstanding, it is the subject, be it pigeon or chimpanzee, whose brain operations generate anew response mode, an emergent, that allows for problem solution That individual and specificprior conditioning of operants is part of the subject’s training history is certainly relevant, indeedcritical, to the emergent response mode; but it is the subject’s brain’s processes, contingent as

they are upon the organization and complexity of the brain, that generate the new, emergent,response modes The most salient attribute of those modes is that, in novel tests/contexts, theyprovide for adaptive novel behaviors that are substantially extended in form and organizationbeyond those manifested during “training.”

Summary

Do operants and respondents operate in the manifestation of emergents? Most certainly they do,but it is the novel blending of them, their varied orchestration and patterning, their immediatemanifestation, that reveals the emergents present in the brain’s operations; it is not specificreflections of antecedents and contingencies provided by the environment or the experimenter

Are emergents reducible to either operants or respondents? It is the argument of this paper thatthey are not, though, as stated above, operants and respondents surely are the behavioralelements and indicants of emergents Indeed it is through behaviors that by tradition might betermed respondent or operant that emergents are manifested Notwithstanding, it is precisely thenon–respondent, non–operant nature that makes certain behavior an emergent Emergents maketheir appearance as novel patterns of responding or choosing between alternatives, and they do

so with some element of surprise to the observer By contrast, both respondents and operantsmake their appearance as improved forms of what they were at the very beginning of training orconditioning Their basic forms are not altered Again, and by contrast, emergents do not havespecific training histories There is no reason to assert that they were there in some minisculeform that either became stronger or was shaped across time, as is the case of operants This is not

to contradict the argument, however, that emergents have their etiology in the experienceswhereby organisms, particularly those with complex brains, acquired respondents and operants.Emergents are new competencies, new patterns of behavior, based in experience, that areproduced by novel generative operations of the subject’s brain—a brain whose operationsdepend on age, absence of trauma, experience, tasks, and species

The category “emergents” encourages the behavioral researcher to use time-tested tactics thatemphasize antecedents and consequences to study behaviors that are new patterns anddemonstrate competence for adapting Alternatively, one may study the same behavioral modesusing a cognitivist point of view, but that is neither necessary nor necessarily advantageouscompared to use of the framework herein advanced

Science moves with the times and new findings We here argue that it is timely for behaviorallyoriented psychologists to evaluate the merits of extending Skinner’s “respondent and operant”dichotomy to a trichotomy that includes the new category of emergent The category “emergent”can facilitate the integration of large corpuses of comparative, developmental, and brain researchinto the behavioral framework, and thereby substantively enhance the science generated by therich tradition of psychological research

Author Notes

Preparation of this paper was supported by HD-06016 from the National Institute of Child Healthand Human Development to Georgia State University Additional support was provided by the

College of Arts and Sciences of Georgia State University, and by a grant (NAG2–438) from theNational Aeronautics and Space Administration Dr Shelly Williams and Dr Daniel Ceruttigave helpful comments on early drafts of this paper, which is based on a presentation by the firstauthor at the Association for Behavior Analysis, Atlanta, Georgia, 1991.

Epstein, R (1985) The spontaneous interconnection of three repertoires The PsychologicalRecord, 35, 131–141

Epstein, R., Kirshnit, C., Lanza, R., & Rubin, L (1984) “Insight” in the pigeon: Antecedentsand determinants of an intelligent performance Nature, 308, 1 March, 61–62

Ellen, P & Pate, J L Is insight merely response chaining?: A reply to Epstein PsychologicalRecord, 1986, 36, 155–160

Ellen, P., Sotores, B J., & Wages, C (1984) Problem solving in the rat: Piecemeal acquisition

of cognitive maps Learning and Motivation, 12, 2, 232–237

Filion, C., Washburn, D A., & Fragaszy, D M (1995, June) Trajectory estimation by rhesusmacaques Poster presented at the annual meeting of the American Psychological Society, NewYork, NY

Flaherty, C F (1985) Animal learning and cognition New York: Alfred A Knopf

Gallup, G G (1987) Self awareness In J Erwin (Ed.), Comparative primate biology: Volume2B Behavior, cognition and motivation, (pp 3–16) New York: Alan R Liss

Greenfield, P M & Savage-Rumbaugh, E S (1991) Imitation, grammatical development, andthe invention of protogrammar by an ape (Pan paniscus) In N Krasnegor, D M Rumbaugh, M.Studdert-Kennedy, & R L Schiefelbusch (Eds.), Biobehavioral Foundations of LanguageDevelopment Hillsdale, NH: Erlbaum

Harlow, H F (1949) The formation of learning sets Psychological Review, 56, 51–65

Heidbreder, E (1933) Seven psychologies New York: Appleton-Century-Crofts, Inc

Koehler, W (1925) The mentality of apes New York: Routledge & Kegan Paul.

Krechevsky, I “Hypotheses” in rats Psychological Review, 39, 516–532

Kuhn, T S (1962) The structure of scientific revolutions Chicago: University of Chicago Press

Levine, M (1971) Hypothesis theory and nonlearning despite ideal S-R-reinforcementcontingencies Psychological Review, 78, 130–140

Nissen, H W (1951) Phylogenetic comparison In S S Stevens (Ed.), Handbook ofexperimental psychology (pp 347–386) New York: Wiley

Posner, M I (1982) Cumulative development of attentional theory, American Psychologist, 37,168–179

Riesen, A H (1982) Effects of environments on development in sensory systems In W D Neff(Ed.), Contributions to sensory physiology, (Vol 6, pp 45–77) New York: Academic Press

Rosenblatt, F (1958) The perceptron: A probabilistic model for information storage andorganization in the brain Psychological Review, 65, 386–407

Rosenblatt, F (1962) Principles of neurodynamics New York: Spartan

Rumbaugh, D M (1995) Emergence of relations and the essence of learning: A review ofSidman’s Equivalence Relations and Behavior: A Research Story The Behavior Analyst, 18,367–375

Rumbaugh, D M., Hopkins, W D., Washburn, D A., & Savage-Rumbaugh, E S (1989) Lanachimpanzee learns to count by “Numath”: A summary of a videotaped experimental report ThePsychological Record, 39, 459–470

Rumbaugh, D M., & Pate, J L (1984) The evolution of cognition in primates: A comparativeperspective In H L Roitblat, T G Bever, & H S Terrace (Eds.), Animal cognition (pp.403–420) Hillsdale, N J.: Lawrence Erlbaum Associates

Savage-Rumbaugh, E S (1986) Ape language: From conditioned responses to symbols NewYork: Columbia University Press

Savage-Rumbaugh, E S & Lewin, R Kanzi: At the brink of the human mind New York: JohnWiley

Savage-Rumbaugh, E S, Murphy, J., Sevcik, R A., Rumbaugh, D., Brakke, K E., & Williams,

S (1993) Language comprehension in ape and child Monographs of the Society for Research inChild Development, Serial No 233, Vol 58, Nos 3–4, pp 1 - 242

Schrier, A M., Harlow, H F., & Stollnitz, F (1965) Behavior of nonhuman primates NewYork: Academic Press.

Selfridge, O G (1955) Pattern recognition by a machine Proceedings of the Western JointComputer Conference

Selfridge, O G (1959), Pandemonium: A paradigm for learning In D V Blake & A M Uttley(Eds.), The Mechanisation of Thought Processes London: H M Stationary Office

Sidman, M (1994) Equivalence relations and behavior: A research story (pp.606) Boston:Authors Cooperative, Inc

Sidman, M., Rauzin, R., Lazar, R., & Cunningham, S (1982) A search for symmetry in theconditional discrimination of Rhesus monkeys, baboons, and children Journal of theExperimental Analysis of Behavior, 37, 23–44

Skinner, B F (1938) The behavior of organisms: An experimental analysis New York:Appleton-Century-Crofts

Stell, M., & Riesen, A (1987) Effects of early environments on motor cortex neuroanatomicalchanges following somatosensory experience: Effects of Layer III pyramidal cells in monkeycortex Behavioral Neuroscience, 101, 341–346

Stephen, H., Bauchot, R., & Andy, O J (1970) Data on size of the brain and of various brainparts in insectivores and primates In C R Noback & W Montagna (Eds.), The primatebrain (pp.289–297) New York: Appleton-Century-Crofts

Tolman, E C (1948) Cognitive maps in rats and men Psychological Review, 55, 189–208

Tolman, E C (1959) Principles of purposive behavior In S Koch (Ed.) Psychology: A study

of a science Vol.2 pgs 92–157 New York: McGraw-Hill

Washburn, D A & Rumbaugh, D M (1991) Ordinal judgments of numerical symbols bymacaques (Macaca mulatta) Psychological Science, 2, (3), 1991

Table 1 A Summary of Similarities and Differences between Respondents, Operants, andEmergents

Table 2 Research Areas that Produce Emergents

2.

Analysis of Behavioral Selection by Consequences and Its Potential: Contributions to Understanding Brain-Behavior Relations

William J McIlvane

Analysis of Behavioral Selection by Consequences and Its Potential Contributions to Understanding Brain-Behavior Relations

William J McIlvane, William V Dube, & Richard W Serna

Harvard Medical School, & Northeastern University

Among the best known of these campaigns occurred during the middle part of this century, whenlearning theorists like B F Skinner were competing with those involved in the “cognitiverevolution,” notably Noam Chomsky As in most such competitions, each group concerned itselfwith some fundamental, essential truth about behavior, and thus each could justify its continuedexistence and relative prosperity The 4th Annual Behavioral Neurodynamics Conference, infact, can be seen as a celebration of the success of the many subdisciplines of behavioral science

In Pribram’s expansive vision, each has a critical role to play in understanding the activities andoperational processes of the brain The program included not only learning theorists, but alsoaccomplished representatives of the cognitive, biological, and engineering sciences

Concerning the subdisciplines of behavior science, our hope is that the individuals involved havebegun to see at least the rough outline of their future and that this future does not keep alive theold battles Rather, an essential component will be an alliance with the brain sciences that usesthe many advances in science and technology to help unravel the relationship between brain and

behavior An important part of forging this alliance will be articulating what each subdisciplinehas learned that might contribute to this grand enterprise This chapter will endeavor to elucidatesome of the contributions of our field, behavior analysis a field that is based on principlesarticulated by Skinner and one that has contributed a large body of relevant theoretical andempirical work.

Behavior Analysis and Brain Science

A common misperception is that behavior analysis does not concern itself with the biologicalprocesses that underlie behavior This misperception reflects a fundamental misunderstanding ofthe behavior-analytic position on the relationship between behavioral science and brain science

Perhaps the clearest expression of that position was offered in Skinner’s 1989 American Psychologist paper: “There are two unavoidable gaps in any behavioral account: one between the

stimulating action of the environment and the response of the organism and one between

consequences and the resulting change in behavior Only brain science can fill those gaps In

doing so, it completes the account; it does not give a different account of the same thing” (italicsours, p 18) From its early days, the literature of behavior analysis has included studies ofrelationships between behavior and brain processes (e.g., see Mogenson & Cioe´, 1977) Whythen have behavior analysts been labeled “black box” psychologists who were uninterested in theoperations of the nervous system?

Perhaps behavior analysis has been misunderstood in part because of Skinner’s frequentadmonitions about studies of the “conceptual nervous system.” Skinner was much concernedabout efforts that used very limited data sets to guess how the biological nervous system worked.Central also to Skinner’s thinking was the possibility of developing an independent science ofbehavior Even if little progress were made in the brain sciences, he reasoned, one could stilldevelop a science of behavior with its own body of data, principles, and theories Moreover,Skinner understood that a well developed science of behavior would be needed even if everyimportant operation of the brain was discovered In order to relate brain and behavior, it would

be essential to have a cohesive, integrated, intellectually rigorous account of behavior

In essence, Skinner and his followers proposed a division of labor Behavioral scientists wouldconcern themselves mainly with delineating precise, quantitative relations between behavior andthe controlling variables that could be directly observed or reasonably inferred from empiricalanalysis Such relations would emerge in part from methodologies with increasing capabilities topredict and influence the behavior of both humans and nonhumans Brain scientists wouldconcern themselves with increasing understanding of the operations of the nervous systems And

as each discipline developed its knowledge base, it would become increasingly possible to definesecure empirical relationships between brain and behavior

As things have turned out, this division of labor has been rejected by many behavioral scientists,most notably those who have turned to cognitive psychology Cognitivists were inspired, ofcourse, by the development of the serial digital computer One may characterize their goal asattempting to discern the mind/brain’s computer program and its symbol system This enterprise,while productive, has not been entirely satisfactory, and that dissatisfaction has given rise toconnectionism and neural network models that do not rely on symbolic processes Connectionists

seek to account for complex behavior via simpler processes that are not dissimilar from thosethat concerned Skinner and other behavior analysts (cf Donahoe, Burgos, & Palmer, 1993) This

is not to say that connectionists have espoused behavior analysis, but merely that there has beensome realization that behavior analysis does touch nature in some fundamental way

Many behavior analysts, in turn, have begun to take on problems that have been the traditionalsubject matter of cognitive psychology For example, the burgeoning field of stimulus

equivalence research is now addressing the phenomena of stimulus-stimulus relations, the sine qua non of cognitive analyses Moreover, there is growing appreciation within behavior analysis

that advances in the brain sciences for example, imaging technologies like PET, MRI, EEG(Posner & Raichle, 1994; Tucker, this volume) have made it possible to observe directlyevents within the skin This growing capability is rapidly removing barriers to experimentalanalysis of private events, a longstanding interest of behavior analysis (e.g., Skinner, 1974) Forthe remainder of this chapter, we shall briefly review some noteworthy accomplishments ofbehavior analysis over the last thirty years and endeavor to articulate a rationale for itsparticipation in interdisciplinary studies of brain-behavior relations

Overview of Behavior Analysis

From the outset, behavior analysis made a radical departure from methodological behaviorismand the S-R psychologies that dominated the first half of this century Perhaps one of the mostimportant distinctions is the behavior-analytic acceptance of private events (e.g., Posner’s [1980]covert shifts of attention) as perfectly acceptable subject matter (cf McIlvane, Dube, &Callahan, 1996) Also, behavior analysts tend to agree with cognitively oriented scientists thatwhat is learned is contingencies relations or regularities among environmental events andnot specific stimulus-response connections

The fundamental analytical method is termed contingency analysis, which at its most basic levelconcerns itself with three types of events: antecedent events outside and within the skin thatoccasion behaving; behaviors such as listening, talking, moving, thinking, and so forth; andconsequences, the events that influence the future probability of the behaviors occasioned byantecedents (i.e., positive and negative reinforcers; see below) Important variables ofcontingency analysis include: (1) antecedent stimulus variables (complexity, duration, intensity,modality, salience, etc.); (2) behavioral variables (type, latency, force, duration, etc.); and (3)consequential stimulus variables (type, magnitude, schedule, etc.) Potentially modulating theeffects of one or more of these classes of variables are: (4) subject variables (age, sex, clinicaldiagnosis, behavioral history, etc.); and (5) state variables (biological establishing operations,disease, drug, etc.) The main goals of behavior analysis are to describe qualitative and ultimatelyquantitative functional relations among these variables

It is important to note here that contingency analysis is not merely applied to study nonhumananimal behavior in “Skinner boxes.” There is a large contingent who do experimental analyses ofhuman behavior and not just its simple forms Behavior analysis takes on subjects of varyingcomplexity from individual responses to larger behavioral episodes (see McIlvane et al., 1996,for further illustrations) Among the more illustrative examples of this scope is the researchprogram of Allan Neuringer and his students (e.g., Neuringer & Voss, 1993) This group studies

reinforcement procedures that teach both humans and nonhumans to behave in a highly variablefashion, that is, to continually emit new behaviors that have not previously occurred in theexperimental context Humans’ performance becomes statistically indistinguishable from thatwhich would be predicted from a random number generator, and nonhumans approach that level.

The Riddle of Reinforcement

The problem of specifying the nature of the reinforcement process is significant enough thatmany behavioral scientists both within behavior analysis and outside it deal with it mainly

by ignoring the complexities Within behavior analysis, the most common approach has been todefine reinforcers through an empirical analysis of their effects A stimulus is reinforcing if itsdelivery contingent upon a given behavior is followed by a reliable change in the probability ofthat behavior

This functional definition has long been recognized as circular, most notably by Paul Meehl(1950) Continued reliance on it is often justified on practical grounds; both experimental andapplied behavior analysis have made useful contributions to basic science and clinical practice

by defining reinforcers in this functional manner Nevertheless, many behavior analysts haveendeavored to define reinforcement in a more informative fashion For example, John Donahoe(Donahoe & Palmer, 1994) has recently defined a reinforcer as a stimulus that elicits reflexivebehavior that differs from the behavior currently ongoing, which in turn alters the probability ofbehavior that precedes the interruption

This “unified reinforcement principle” is a very attractive definition in certain respects It notonly defines reinforcement functionally, but also begins to suggest a way out of the circularityproblem and allows one to make reinforcement a central feature of both respondent and operantconditioning Although a comprehensive presentation of Donahoe’s thinking is beyond the scope

of this chapter, it seems particularly noteworthy in that he offers this account as part of a broadereffort to connect behavior analysis not only with connectionism but also with facts ofdevelopmental neurobiology

Many years ago, there were other efforts to eliminate the circularity in the definition ofreinforcement Reinforcement was conceptualized in terms of drive states, drive reduction, and

so on Subsequent work proved devastating for these formulations It was found, for example,that lesions of the ventral medial hypothalamus would cause rats to eat voraciously, suggesting ahigh drive state If food was made contingent upon an operant response, however, the foodwould not function as a reinforcer, suggesting a low drive state How could animalssimultaneously exist in a high and low drive state? (See Pribram 1995, for additional discussion

of this issue)

David Premack (1965) helped to circumvent this problem, when he articulated his now-famousPremack Principle Briefly, he suggested that organisms tended to prefer to engage in certainbehaviors over others, and that at any given moment there was a hierarchy of valued activities.Those behaviors with higher preference values, like eating if food had not been recentlyavailable, could serve as reinforcers for lower probability behaviors, like pressing a bar in anoperant chamber Notice that in Premack’s formulation there is no reference to basic biological

needs or drives that impel behavior In their place, one talks about events that have preference orvalue for the organism, thus implicitly expanding the range of events that might serve asreinforcers.

Theoretical and empirical work subsequent to Premack’s original contribution has refined andmodified the value-based account One such account, for example, suggests that what is critical

to establish a given behavior as reinforcing is that access to that behavior is restricted, so that itsordinarily preferred (or baseline) level is not achievable (Timberlake & Allison, 1974) A relatedone suggests that animals have certain levels of the behaviors that they characteristically engage

in a preferred behavioral organization if you will (Allison, Miller, & Wozny, 1979).Experimental operations that alter the preferred levels create organismic efforts to restore thoselevels, thus rendering opportunities to do so reinforcing Each of these accounts may be seen assimilar in that behaviors are reinforcers not because they reduce a drive or restore biologicalhomeostasis but rather because behaviors or behavioral organizations have established value forthe organism

These new ways to conceptualize reinforcement were further enriched when Jack Michael (1982)helped to redefine operations that would establish events as reinforcers He defined the

“establishing operation” as an environmental event, operation, or stimulus condition that affectsthe organism by momentarily altering (a) the reinforcing effectiveness of other events and (b) thefrequency of occurrence of that part of the organism’s repertoire relevant to those events asconsequences The notion that operations can momentarily render events reinforcing is attractivefor many reasons, but perhaps especially because it gives us a more plausible framework fortalking about reinforcement processes, especially in humans For example, imagine that you arethe audiovisual specialist responsible at the 4th Annual Behavioral Neurodynamics conference,and it is your job to make sure that the speakers’ slides can be read by the audience Imaginefurther that the projector bulb fails Failure of the old bulb is the establishing operation thatmakes finding a new bulb strongly reinforcing although that same spare bulb would have been

at best a weak reinforcer or even a neutral stimulus just a moment before Note that theestablishing operation formulation applies to all three conceptions of the reinforcer describedabove

These newer conceptions of reinforcement and reinforcement operations have been extremelyuseful for behavior analysis They help the field to shed criticisms that have been leveled atreinforcement-based accounts of behavior In the 1950s, the drive-based accounts ofreinforcement stressed “primary” reinforcers According to such accounts, the ongoing stream ofbehavior was largely motivated by a search for biologically significant reinforcers like food, sex,warmth, and secondary factors related to those reinforcers, such as status within the group Thus,the drive reductionists were offering a sort of Rabelaisian view of existence that was not widelyseen as an attractive picture of the human condition The more recent formulations allow us toarticulate the principles of reinforcement in new terms of values, preferred activities, orreducing behavioral discrepancies By contrast with drive theorists, these modern formulationssuggest a broader view that also outlines a research agenda: What are the circumstances underwhich activities become preferred?

The Nature of Selection by Consequences

Two longstanding, somewhat controversial issues in the analysis of behavior are (1) under whatcircumstances does selection by consequences occur and (2) what is selected? With respect to theformer, behavior analysis has typically asserted that reinforcement does not require awareness orother intervening processes to operate Among the most compelling demonstrations of thisphenomenon was one reported by Hefferline and colleagues at Columbia (e.g., Hefferline,Keenan, & Harford, 1959) They showed that appropriate reinforcement contingencies couldincrease the frequency of minute, imperceptible muscle contractions events clearly out of theawareness of the humans who were their subjects Thus, selection by consequences has beenassumed to operate in a fairly broad fashion that emphasizes environment-behavior contingency,not awareness, attention, or some obvious selective process.

More recently, there have emerged apparent challenges to this assumption Peter Killeen (1994),for example, has offered a mathematical model of reinforcement that suggested an important rolefor short-term memory processes Based on certain characteristic delay functions obtained fromempirical data, he argues that reinforcement selects not behavior directly but rather itsrepresentation in memory It seems difficult to reconcile such analyses with findings likeHefferline’s that suggest broader operation of reinforcement processes, and this seems to be afertile area for future research and theoretical development

With respect to what is selected by consequences, both early (Ray & Sidman, 1970) and recent(e.g., McIlvane & Dube, 1992) work from our group suggests that reinforcement selects notmerely individual behaviors but rather environment-behavior relations as a unit (see Sidman[1986] and Donahoe and Palmer [1994] for further development of this issue) Central to thisargument is the question of whether every behavior or behavioral episode has a potentiallyidentifiable antecedent and conversely whether behavior can be emitted in the absence of anoccasioning environmental event (see Shull, 1995, for a comprehensive discussion)

Outcomes of Selection by Consequences

The process of selection by consequences leads to a number of outcomes that seem directlyrelevant to the emerging field of behavioral neurodynamics Three aspects of our currentresearch program seem particularly relevant, and we shall describe them to exemplify research inthis area

Stimulus equivalence If reinforcement is about value for the organism, the antecedent or

discriminative stimuli that set the occasion for the reinforcer should have a detectable value.Further, stimuli discriminative for the same reinforcer should have equal values, that is, theyshould be equivalent to one another From a great many examples available to us, we shallbriefly describe an illustrative, recently reported study that sought to define a minimalexperimental history for relating stimuli on the basis of common relations to reinforcers (Dube &McIlvane, 1995)

Figure 1 illustrates the procedures (see fig 1) During baseline training, four differentnonrepresentational forms served as samples and comparisons on identity-matching trials.Correct selections of two of the forms were always followed by a specific reinforcer, SR1 (e.g., asip of juice), and those of the other two forms were followed by a second reinforcer, SR2 (e.g., a

pretzel) Unreinforced test trials then asked whether this history was sufficient to establish thearbitrary-matching performances shown in the lower portion of Figure 1 That is, were stimulithat were antecedents for the same reinforcer equivalent to each other? If so, then they should besubstitutable for one another in the matching task Such performances, in fact, did emerge in anumber of individuals with mental retardation, thus suggesting that the stimuli matched to eachother had the same value (cf Edwards, Jagielo, Zentall, & Hogan, 1982; Schenk, 1995).

Based in part on research such as this, Murray Sidman (1994) has argued that stimulusequivalence is not a derived phenomenon, as has been widely suggested (e.g., Horne & Lowe, inpress) Rather, it may be a direct product of the reinforcement process If so, then equivalencerelations may turn out to be fundamental building blocks of cognition rather than its products.Although the equivalence work thus far has been mainly descriptive, there are now thebeginnings of a structural and quantitative analysis of equivalence relations, including suchvariables as nodal distance how the number of connecting relations interacts with a subject’sability to demonstrate equivalence relations (Fields, Adams, Verhave, & Newman, 1990) Therehave also been recent efforts to develop neural network models of stimulus equivalence (e.g.,Donahoe & Palmer, 1994), which may ultimately prove relevant to behavioral neurodynamics

Studies of choice If reinforcement establishes value, one should be also able to allocate

behavior according to value to choose among available reinforcement sources analytic researchers have been studying choice situations for over 30 years (for reviews, seeBaum, 1974, 1979; de Villiers, 1977; McDowell, 1982; Williams, 1988) Research in this area isextensive, quantitatively rigorous, and seems to compel the attention of those interested inbehavioral neurodynamics Because even cursory coverage of this area would consume manypages, we shall merely introduce it

Behavior-Richard Herrnstein (1970) has offered formal mathematical statements of the relation betweenreinforcement and behavior According to this matching law, behavior among concurrentlyavailable response alternatives is distributed in the same proportion as the reinforcements aredistributed among the alternatives Matching has been investigated most widely with concurrentschedules paradigms, that is, those in which two independent reinforcement schedules areavailable The following equation describes results of quantitative analyses in two-responsesituations:

where B1 and B2 are response rates or durations for behavioral alternatives 1 and 2, and r1 andr2 are the rates of reinforcement obtained from alternatives 1 and 2 An alternate form of theequation describes response-reinforcer relations in a single-response situation, where the choice

is between the measured response and any other behavior In a later section, we will suggest thatthe matching analysis may be applicable to an even broader range of problems than it has beenheretofore

Behavioral momentum Not only does reinforcement establish value and permit choice, but we

are now learning that it establishes what Tony Nevin has termed behavioral momentum(reviewed in Nevin, 1992) The momentum analysis makes analogies between the relationships

described in the physics of motion and the psychology of behavioral persistence The momentum

of a moving body is defined in classical mechanics as the product of mass and velocity Thedegree to which an outside force can perturb the motion depends upon momentum; increasingmass while holding velocity constant increases the resistance to change

As a starting point for Nevin’s analogy, Newton’s Second Law (Equation 2) states that a change

in the velocity of a body (UV) will be the product of the force applied (f) and the reciprocal ofthe body’s mass (m)

Thus, given equal force applied to two bodies, the ratio of changes in their velocities will beinversely proportional to the ratio of their masses (Equation 3)

Nevin suggests a direct parallel in the domain of behavior He argues that rate of responding isanalogous to velocity, and the resistance of that rate to change by a perturbing operation(prefeeding, alternative reinforcement, punishment, etc.) can be used to index the analogue ofmass As expressed in Equation 4, if the same perturbing operation is applied to two responses,the ratio of changes in response rates (UR) will be inversely proportional to the ratio of thebehavioral masses (b)

Nevin and others have shown that behavioral mass is determined by the rate of reinforcement(Nevin, Mandell, & Atak, 1983; Nevin, Tota, Torquato, & Shull, 1990) Higher rates ofreinforcement produce greater behavioral mass, and this occurs independently of the responserates As suggested in Figure 2 (see fig.2), if two responses are maintained at differentreinforcement rates, then a perturbing force will have less effect on the response with the higherrate of reinforcement (filled points) than on the response with the lower rate of reinforcement(open points)

Figure 3 presents some findings from our laboratory that illustrate the momentum analysis Ayoung woman with mental retardation (chronological age 17 years, mental age 2.8 years1) wasgiven two discrimination tasks that she could perform accurately Sessions alternated betweenblocks of trials with each task, that is, she was not asked to choose between the two, but rather towork on each at different times On one task, every correct response produced a reinforcer (asmall bit of cookie), and on the other task, every fourth response on average did so Figure

3 shows the rate of trial completion, which was under her control Baseline response rates for thetwo tasks are shown in Condition A, and they were roughly comparable According to themomentum analysis, the richer reinforcement schedule (filled points in Figure 3) would haveestablished greater behavioral mass, and thus rendered the behavior less vulnerable to

perturbation (see fig 3).

Condition B in Figure 3 shows the effect of our perturbing force, a concurrently availabledistraction, videotapes shown on a television next to the discrimination apparatus Note that rate

of behavior fell proportionally more with the leaner schedule After a return to baseline, in aneffort to increase the perturbing force, we added prefeeding to the test sessions (two cookies justbefore the session), which led to the more pronounced momentum effect shown in Condition B’.Although still in an early stage, the analysis of behavioral momentum addresses a previouslyunder-appreciated dynamic characteristic of the reinforcement process In particular, the findingthat behavioral persistence is largely independent of response rate, but rather related toreinforcement rate, may have important implications for the design of behavioral training andtesting procedures For example, one somewhat counterintuitive implication of momentumtheory is that under some conditions a decrease in reinforcement rate may facilitate the transfer

of behavioral control from instructional supports to the stimuli that are the training goal

Integrating Behavior Analysis and Brain Science: Potential Benefits

Benefits for Behavior Analysis

As noted earlier, behavior analysis has the challenge of developing a coherent, broadlyapplicable theory of reinforcement One possible benefit of integrating behavior analysis andbrain science has been articulated by respondents to Killeen’s (1994) target article on

mathematical modeling of reinforcement in Behavioral and Brain Sciences C M Bradshaw, for

example, mentioned a problem that plagues mathematical modeling: Behavioral data canfrequently be modeled by more than one, and sometimes many, equations Suggesting a solution,

he wrote, “Skinner’s eschewal of neurobiological explanations of behavior … was in partpragmatic; neuroscience was not ready … to provide a satisfactory account of… environment-behavior interactions Such accounts are now conceivable, and it may behoove us to placeincreasing reliance on physiological plausibility as a criterion for selecting equations to modeloperant behavior.” (p 137)

Bradshaw’s suggestion also appears pertinent to behavior analysts such as those in our researchgroup at the Shriver Center, who are interested in studying antecedent stimulus control ofbehavior and the nature of environment-behavior relations selected by the reinforcement process.(For those more comfortable with the older language of experimental psychology, the problem ofstimulus selection.) Skinner was not much interested in this very difficult problem, perhapsbecause he felt that descriptive and quantitative analysis of response-reinforcer relations wasmore tractable

Our group’s efforts, however, have been directed toward defining and perhaps amelioratingproblems of individuals with intellectual disabilities like mental retardation, autism, and otherproblems of clinical significance With individuals who have clinical problems, merelyestablishing reinforcer control is typically not challenging What is challenging, however, isestablishing and maintaining the kinds of environment-behavior relations that comprise anadaptive behavioral repertoire In responding to this challenge, we have become less concernedwith operant response topographies Rather, we have been drawn to the analysis of variations instimulus control, which we have termed “stimulus control topographies” (see Dube & McIlvane

[in press] for a comprehensive presentation of this concept) For the purposes of this chapter,suffice it to say that stimulus control topography corresponds roughly to “representation” or “thestimulus as encoded.”

Other behavior analysts have begun to confront the same problems For example, Mark Rilling,

who authored the chapter on “Stimulus Control” in the widely read Handbook of Operant Behavior (1977), recently wrote, “When the stimulus is a complex event (e.g., a color slide of a

scene in the real world) … the experimenter is forced to confront two problems: perception andrepresentation … A theory of discrimination learning that cannot specify precisely the nature ofthe event about which [the subject] has learned will be unable to predict the events that controlbehavior in the future” (Rilling, 1992) To redress this potential problem, Rilling was attracted to

J J Gibson’s (1979) perspective, which seems to offer much to behavior analysis (cf Green,Mackay, McIlvane, Saunders, & Soraci, 1990)

In addition to ecological perspectives, behavior analysis will likely benefit from the brainsciences in the analysis of stimulus variables For example, consider the extensive analysis of

visual perception that Pribram (1991) has accomplished in Brain and Perception and the

preceding three volumes of this series This analysis addresses the gap between the stimulatingaction of the environment and those aspects of it that would be available to guide behavior Byunderstanding how the brain decomposes complex sensory information, we may advance a steptoward solving what Skinner called “the problem of the first instance” (Skinner, 1957) Hereferred to the logical problem of identifying the variables responsible for the first instance ofnew behavior, before that behavior is reinforced.2 A related question asks, what are the variablesresponsible for noticing, for example, a new feature of a complex stimulus for the first time? Webelieve that the answer must be some interaction between our sensory abilities and capacities(products of phylogenic [survival] contingencies), and the cumulative experience that builds andtunes them (i.e., ontogenic [reinforcement] contingencies) (cf Skinner, 1969)

There are two other areas in which behavior analysis might benefit from closing the gap betweenitself and the brain sciences Above, we discussed several ways in which reinforcementprocesses have been defined in behavior analysis Harkening back to Bradshaw’s point,connecting with the neurology and neurobiology of the brain may give us clues that will help tofurther refine our understanding of reinforcement from the psychological perspective Such workmay also help to illuminate the nature of behavioral selection by consequences Perhaps onereason that there remains some issue about what reinforcement selects responses orenvironment-behavior relations is that the immediate antecedents to some behavior are withinthe skin and thus difficult to study directly Such antecedent events may be studied only viamethods that render them detectable (e.g., appropriately targeted EEG recordings)

In addition, we think it unlikely that behavior analysis can, by itself, do an adequate job ofcoming to terms with how subject variables influence environment-behavior interactions Whilepsychological and ethological analyses have been reasonably successful in revealing species-specific behaviors selected by phylogenic contingencies, there are limits to what unguidedstudies can accomplish in this area For example, our own studies of stimulus control andreinforcement processes in people with intellectual and other developmental disabilities require

us to make daily contact with individuals who have an appalling variety of neurological

problems, many of which are at best poorly understood In our view, such problems cannot befully understood without looking at the brain operations, defects, or dysfunctions that subsumethem.

Benefits for Brain Science

While behavior analysis needs the brain sciences to complete the account, it offers a number ofworthwhile contributions to brain science, two of which merit special attention here The first is

in developing and/or applying methodologies that will allow the brain scientist to study researchsubjects who do not have well-developed language skills and do not share a common frame ofreference with the experimenter To make this point, we will relay something of the history ofour research group The Shriver Center Behavioral Sciences Division exists today becauseRaymond D Adams, Chief of Neurology at the Massachusetts General Hospital and the HarvardMedical School, realized in the 1950s the many limits of behavioral testing to evaluateneurological dysfunction and disease He looked to the behavioral sciences for help, and this ledhim to recruit several behavior analysts, most notably Murray Sidman, to the MassachusettsGeneral Hospital In collaboration with several others, Sidman initiated a program that sought toapply the methods of behavior analysis to the problem of conducting neurological evaluations ofindividuals who had limited language abilities particularly those who had suffered strokes orwere severely mentally retarded

Stimulus control shaping Sidman and his colleagues chose to begin with visual perception, a

sensible choice given that so much of the brain’s activities are devoted to acquiring andintegrating visual information They needed a procedure that could evaluate visual perception inpeople with very limited language The product of their effort is shown in Figure 4 (see fig 4)

To communicate without using words, Sidman and Stoddard adapted a methodology called

“fading” which had been suggested by Skinner and had recently been demonstrated by Terrace(1963) at Columbia in an animal behavior model They sought to use fading to establish adiscrimination of a circle (S+) from ellipses of various sizes (S-) (Sidman & Stoddard, 1966)

Their fading program proceeded as follows: Initially, the circle was displayed on one of eightresponse keys The key was lit, and the remaining keys were blank and dark (Figure 4, a) Aftertypically rapid acquisition of this very easy circle vs dark key discrimination, the blank keyswere illuminated gradually (b-c) When the S+ and S- keys were illuminated equally, the subjectwas required to make a form vs no form discrimination (d) When this discrimination wasmastered, flat ellipses were faded onto the S- keys (e-g) until the subject was ultimately required

to make a circle-ellipse discrimination (h) Finally, the ellipses were made gradually rounder 1), thus requiring a progressively finer circle-ellipse discrimination and providing a means fordetermining a circle-ellipse discrimination threshold

(i-The circle-ellipse program was a truly remarkable achievement With it, they were able toevaluate visual perception in a variety of individuals with profound neurological problems Thetest was successful, for example, with Cosmo, a microcephalic man with profound retardation,who became quite famous as a result of his participation in these studies A similar program wasused with another famous neurological patient, H.M., and the methods helped clarify the nature

of his behavioral deficits (Sidman, Stoddard, & Mohr, 1968) These successes exemplify manymore that have been achieved when behavior analysts and brain scientists collaborate.

Research since Sidman and Stoddard’s seminal work has greatly expanded the scope andcomplexity of performances that can be taught by stimulus control shaping methods The workhas advanced to the point where one can reasonably entertain the possibility of rendering shapingless of an art and more of a science, hopefully a quantitative science One use of stimulus controlshaping, for example, is to establish matching performances based on arbitrary stimulus-stimulusrelations, where the stimuli that are matched are not physically identical (e.g., matching “2” and

“II”) Subjects begin by selecting a comparison stimulus that is identical to a sample stimulus.Then, the features of the comparison stimulus gradually change, over a series of graded steps,and ultimately become those of the new stimulus that is the training goal We envisionmethodology whereby a computer program might be able to determine in advance what teachingsteps are needed, by using the learner’s judgments of stimulus similarity in the intended teachingsequence (see below) More generally, we believe that many barriers to effective stimuluscontrol shaping could be removed by understanding better the determinants of stimulus controltransfer For example, what accounts for the smooth transition between steps in a successfulstimulus control transfer program?

Recently, we have been pursuing a stimulus-class analysis of stimulus control transfer Ouranalysis predicts the following: The optimal series of stimuli for a successful stimulus shapingprogram is one in which the learner regards the stimuli from adjacent steps as members of thesame feature class A feature class is one in which stimuli are grouped on the basis of similar,though not necessarily identical, features If a pretest revealed, for example, that an experimentalsubject regarded the stimuli in the first three steps of a shaping program series as similar, wewould predict that the second step would be superfluous Conversely, if the subject did notregard the stimuli from fourth and fifth steps as similar, we would predict a breakdown inperformance at that point in a shaping program Most importantly, we would expect that specificfeature-class judgments would vary from individual to individual

Thus far, our studies have focused on methods for assessing the feature-class boundaries withingraded series of stimuli like those used in stimulus-control transfer programs (examples of suchseries are shown in the rows in Figure 5) (see fig 5) In one study, three individuals with mentalretardation were trained to make “yes/no” similarity judgments (Serna & Wilkinson, 1995; cf.McIlvane, Kledaras, Lowry, & Stoddard, 1992) After extensive training, the following baselineperformance was established with nonrepresentational form stimuli: Delayed matching-to-sample trials presented a sample stimulus, one comparison stimulus, and a “blank” comparisonstimulus (a black mask large enough to cover any stimulus) If the sample and displayedcomparison were very similar, but not identical, the subject made a “yes” response by selectingthe displayed stimulus; if not, he/she made a “no” response by selecting the blank comparison.When this yes/no performance baseline was firmly established, stimuli from two new series ofgraded forms were presented on unreinforced test trials that were interspersed among thebaseline trials

Results of the test trials can be illustrated with individual “feature-class profiles” for eachsubject, shown in Figure 5 Feature classes within the two tested stimulus series are indicated by

enclosing boundaries A noteworthy characteristic of the three profiles is that no two areidentical; judgments about stimulus similarity varied across individual subjects It is alsonoteworthy that the graded steps within the series, constructed according to the experimenter’sbest guess as to equalsized steps, did not yield tidy judgments by the subjects Instead, eachprofile shows classes of varying sizes, as well as adjacent steps in which the participants did notregard the stimuli as similar.

If these series were subsequently used in programs designed to transfer control from the firststimulus in the series to the last, these profiles would lead to clear predictions aboutperformance For example, for Subject TFR (upper profiles in Figure 5), we would expect abreakdown in performance between E-4 and E-5 because the feature classes do not overlap atthat point No such breakdowns would be predicted for the G-H series Further, we would predictthat stimuli E-2 and E-3 are superfluous because E-1, E-2, E-3, and E-4 were all classed together.Similar, though idiosyncratic, predictions could be made for the other two participants

The goal of making shaping less of an art and more of a science seems within our reach Many ofthe building blocks for a quantitative stimulus control transfer technology already exist; forexample, potential stimulus series of various gradations could be generated by applyingalgorithms like those that produce visual “morphing” effects in computer graphics Mostimportantly, the feature-class analysis described above can provide a theoretical framework tohelp organize technology so that it will make effective contact with the behavior of theindividual subject

High-probability environment-behavior relations A second, related contribution is the

production of behavioral baselines of unusual purity, such that brain-behavior interactions might

be more clearly revealed in individuals with neurological disease For example, assume that onehas the goal of using our ever increasing ability to image brain activity to trace the way in whichthe brain performs some simple discrimination task like detecting a single salient feature thatdistinguished two otherwise identical visual stimuli Ideally, one would prefer to see crisp, high-probability stimulus control baselines, so that the relevant stimuli and corresponding brain eventscan be related Such baselines are assumed when one tests adults with normal capabilities, butthe same assumption does not hold when testing a young child or someone with neurologicaldisease Their baselines are often very messy, and high-probability stimulus control is a rarity.These and related observations, in fact, have led to gating theories of neurological deficit anddysfunction that the behavioral deficits result because the individual is unable to screen outirrelevant information (Hasher & Zacks, 1988)

Behavior analytic methods allow one to show, for example, that such gating or screeningproblems are due at least in part to contingencies of reinforcement that encourage too wide agate As such, contingencies can also narrow it For example, Figure 6 illustrates a task that we

have used for several years to study variables responsible for intermediate accuracy (see fig 6).

The left half illustrates our method for analyzing imperfect accuracy scores In Condition A,every trial-initiation response is followed by a discrimination between visual stimuli that areidentical except for one feature The difference is obvious to individuals without developmentallimitations; the positive form flashes, alternates with a black field, or appears on a coloredbackground Selecting the positive stimulus is followed by a reinforcer and selecting negative

stimuli are not In individuals with developmental limitations accuracy scores may stabilize atlevels above chance but short of perfection, as shown for Condition A in the right portion

of Figure 6

In Condition B, the contingencies are altered such that each trial proceeds through two “trialstates.” Every trial begins with trial state 1 (TS1) by presenting negative stimuli on all keys Theappropriate response is to wait a few seconds until TS2, where one stimulus begins to flash, forexample, and thus becomes S+ Any failures to wait merely extend TS1 Note that the stimulusdisplay in TS2 is identical to that in Condition A Characteristic data for Condition B are shown

in the rightmost portion of Figure 6 When the subject waits, selections in TS2 may be alwayscorrect Over sessions, competing control reflected in the TS1 responses gradually declines

as it is extinguished

The upper portion of Figure 7 shows data from four individuals with severe mental retardation

(see fig 7) An essentially flat, near “chance-level” function in Condition A is followed

immediately or nearly so by close to 100% TS2 accuracy scores and declining TS1 responding inCondition B Sometimes, however, we obtain stable, intermediate accuracy scores in Condition

B like those shown for three other subjects in the lower portion of Figure 7 These scores are farenough above chance levels to indicate that the relevant stimulus control occurs with somefrequency, but they do not improve with continued training

Herrnstein’s equation (Equation 1, above) and quantitative analyses of behavioral choice mayoffer one way to understand intermediate accuracy scores like these When one viewsdiscrimination procedures from a perspective that permits multiple stimulus controltopographies, they can be seen to present the subject with choices between the discriminativestimuli for concurrently available response options One option is the relevant form of stimuluscontrol that the experimenter is trying to establish in our case, select the flashing stimulus Ifthe subject’s performance reflected only this type of stimulus control, the resulting accuracyscore would be 100%, as illustrated by the shaded portion of the circle in the top left of Figure

8 (see fig 8) The probability of reinforcement for behavior under the relevant stimulus control,B1, is 1.0 The other options are irrelevant forms of stimulus control, and a performanceconsisting only of them would produce 50% accuracy scores in a two-choice task, as shown inthe top right of Figure 8 The probability of reinforcement for behavior under irrelevant forms ofstimulus control, B2, is 0.5

According to Herrnstein’s equation, and as illustrated in Figure 8, if the proportions ofB1 responding matched the proportion of reinforcers obtainable by the R1 reinforcementcontingency, then B1 would occur on two-thirds of the trials If so, then the resulting accuracyscore is shown at the bottom of Figure 8, 67% for perfect accuracy on two-thirds of the trials,plus 17% for chance accuracy on one-third of the trials, resulting in an overall accuracy score of84% If our analysis is correct, the problem of stable, intermediate accuracy may not be the result

of faulty perception, poor gating of attention, or other such problems Rather, such scores may bethe ordinary outcome of known reinforcement processes and may be correctable, for example, byteaching subjects to maximize reinforcers rather than to match reinforcement probabilities inchoice situations

Summary of contributions of behavior analysis to brain science To summarize our basic

points, behavior analysis can contribute to brain science by helping to solve problems incommunicating with and testing individuals with neurological disease The methodology mayalso be helpful with nonhuman populations that participate in more invasive aspects of brainresearch This methodological contribution, we believe, is one that behavior analysis canuniquely make Yet another contribution is that behavior analysts are studying important subjectmatter that is not currently under study by other branches of the behavioral sciences The focus

on reinforcement processes such as equivalence, choice, and momentum, has producedimpressive, quantitative analyses of a critical dimension of behavior Data reviewed in thepreceding section, for example, suggest that what have traditionally been interpreted as problems

of attention may be due in part to failure to appreciate important aspects of reinforcementprocesses If so, much unproductive and/or misdirected research might be avoidable by increasedunderstanding of such processes

Perspectives on Discipline Integration

From certain perspectives, the arguments that we have made thus far may seem curious Some

will ask whether behavior analysis-brain science disciplinary integration is not already a fait accompli Those interested in the neurobiology and neuropsychology of learning and memory,

for example, make routine use of behavioral testing methods in their work and have done so foryears Consider neurobiologicai studies of learning processes, which often use very simpleorganisms and respondent and/or operant methods Consider also the long tradition inneuropsychology of studying the impact of naturally occurring or artificially created lesions onsimple performances such as delayed matching (or nonmatching) to sample, which areextensively studied in behavior analysis This tradition notwithstanding, we believe it fair to saythat work thus far has only begun to scratch the surface of much deeper possibilities (seeDonohoe, et al [1993] and Schull [1995] for additional relevant commentary and illustrativeanalyses)

Discipline isolation seems particularly persistent, for example, in the experimental analysis ofhuman behavior, and doubly so in the growing field of neural imaging Imaging work isdominated by cognitive neuroscientists, who are interested in such topics as tracking theinformation flow that occurs in complex behavioral repertoires like reading (Posner & Raichle,1994) Work in this area has not been much concerned with looking at reinforcement processes,that is, issues of stimulus equivalence, choice, and behavioral persistence, as exemplified by themomentum analysis

In our view, there are at least two reasons for behavior analysts and brain scientists to jointogether to take on such problems First, the problems are scientifically interesting, and theirrespective sciences have advanced to the point that such studies now appear to be feasible.Second, practical and humanitarian concerns dictate this path It is being increasingly recognizedthat disabling conditions like autism and attention deficit hyperactivity disorder are in partrelated to reinforcement processes (Barkley, 1996) In sum, it seems that behavior analysis andbrain science have developed common cause, and this in the final analysis is what sustainseffective interdisciplinary efforts

Learning as Self-Organization

To conclude, we shall speak to the overarching theme of this conference: Learning as selforganization rather than self gratification As an example supporting this interpretation, Pribramoften relates his observation of a monkey who was performing a learning task apparently forpeanut reinforcers As the task progressed, the animal tended to store earned peanuts in hispouch, rather than eating them immediately In addition, when erroneous responses occurred, theanimal often ate stored peanuts shortly afterwards, apparently “self-reinforcing” the errors.Finally, Pribram observed that the animal sometimes responded incorrectly for long series oftrials, apparently indifferent to the absence of peanut deliveries Each of these phenomena seeminconsistent on their face with the suggestion that the animal was performing the task merely forthe sake of gratifying its desires for peanuts Indeed, observations such as these are not unusual,particularly with humans, and made remarkable only by taking an impoverished view ofreinforcement and its role in supporting and organizing behavioral activity

To develop our argument, we shall relate another observation from our own research program.Some time ago, we studied a severely autistic, mentally retarded boy on the circle-ellipsethreshold task (Figure 4, h-1) As the boy proceeded through the task, he initially respondedslowly, carefully scanning the display matrix to find the single circle among the many similarellipses Each success, like all others before it, resulted in an M&M, which the boy ate avidly.The M&Ms and auditory stimuli accompanying their delivery served as the only instructions Noone else was physically present during the testing, which was accomplished in an automatedteaching environment (see McIlvane, Kledaras, Dube, & Stoddard [1989] and Stoddard [1982]for descriptions)

As the discrimination became more difficult, the boy encountered circle-ellipse differences thatwere beyond his ability to discriminate The result was a characteristic shift in stimulus control;

if his first response was not correct, he selected adjacent keys in rapid succession, “circling” thedisplay matrix in search of the correct key Because of a correction procedure, incorrectresponses had no effect and every trial ended with a correct response One consequence was thatthe boy came to earn chocolates even faster than before, because circling can be accomplishedrapidly, while careful scanning typically requires more time Another feature of the testprocedure a backup following errors ultimately returned the boy to circle-ellipse differencesthat he could discriminate Under such circumstances, the careful scanning and slowerresponding returned, despite the fact that the slower rate led to fewer chocolates

On their face, such observations are not easily reconciled with a view of reinforcement as gratification That is, one must explain why the boy abandoned the more lucrative circling in

self-favor of more careful responding Clearly, the chocolates per se were not the only factors

influencing performance This seems doubly interesting, given that this was a boy with autismworking alone in an automated test environment What other variables might account for hisbehavior? Perhaps one might argue that the prior, more extensive programmed training hadestablished greater momentum in the more careful approach Also, the careful approach led to areduction in unreinforced responding (i.e., errors) which past research has shown may beaversive (Stoddard & Sidman, 1967; Terrace, 1971) Going beyond known effects ofreinforcement procedures, we speculate that well-organized behavioral sequences may be

inherently reinforcing, perhaps through the operation of phylogenic rather ontogeniccontingencies (Skinner, 1969) One characteristic, in fact, of behaviors that are reinforcing is thatthey are inherently well-organized (e.g., eating, drinking, sex, etc.; cf Pribram, 1995) Takingthis line, one may think of learning as extending and elaborating biologically determined patterns

of organized activity Consistent with our overall argument, it seems likely that determiningwhether our speculations have merit will require not only comprehensive understanding ofbehavior but also of the brain processes that underlie it

to fully understand the relationship between brain and behavior Address correspondence to W J.McIlvane, Shriver Mental Retardation/Developmental Disabilities Center, 200 Trapelo Road,Waltham, MA 02254

Baum, W M (1979) Matching, undermatching, and overmatching in studies of choice Journal

of the Experimental Analysis of Behavior, 32, 231–242

Bradshaw, C M (1994) Variations in behavioral equations: Can neurobiology help? Behavioraland Brain Sciences, 17, 136–137

de Villiers, P A (1977) Choice in concurrent schedules and a quantitative formulation of thelaw of effect In W K Honig & J E R Staddon (Eds.), Handbook of operant behavior (pp.233–287) Englewood Cliffs, NJ: Prentice-Hall

Donahoe, J W & Palmer, D C (1994) Learning and complex behavior Boston, MA Allyn &Bacon

Donahoe, J W Burgos, J E., & Palmer, D C (1993) A selectionist approach toreinforcement Journal of the Experimental Analysis of Behavior, 60, 17–40

Dube, W V & McIlvane, W J (1995) Stimulus-reinforcer relations and emergent matching tosample The Psychological Record, 45, 591–612.

Dube, W V & McIlvane, W J (in press) Some implications of a stimulus control topographyanalysis for emergent stimulus classes In T R Zentall & P M Smeets (Eds.), Stimulusclass formation in humans and animals North Holland

Edwards, C A., Jagielo, J A., Zentall, T R., & Hogan, D E (1982) Acquired equivalence anddistinctiveness in matching to sample by pigeons: Mediation by reinforcer-specificexpectancies Journal of Experimental Psychology: Animal Behavior Processes, 8, 244–259.Fields, L., Adams, B J., Verhave, T., & Newman, S (1990) The effects of nodality on theformation of equivalence classes Journal of the Experimental Analysis of Behavior, 53, 345–358

Galbicka, G (1994) Shaping in the 21st century: Moving percentile schedules into appliedsettings Journal of Applied Behavior Analysis, 27, 739–760

Gibson, J J (1979) The ecological approach to visual perception Boston: Houghton-Mifflin

Green, G., Mackay, H A., McIlvane, W J., Saunders, R R., & Soraci, S A (1990).Perspectives on relational learning in mental retardation American Journal on MentalRetardation, 95, 249–259

Hasher L & Zacks, R T (1988) Working memory, comprehension, and aging: A review and anew view In G H Bower (Ed.), The psychology of learning and motivation (Vol 22 pp 193–225) San Diego, CA: Academic Press

Hefferline, R F., Keenan, B., & Harford, R A (1959) Escape and avoidance conditioning inhuman subjects without their observation of the response Science, 130,1338–1339

Herrnstein, R J (1970) On the law of effect Journal of the Experimental Analysis ofBehavior, 13, 243–266

Horne, F & Lowe, P (in press) On the origins of naming and other symbolic behavior Journal

of the Experimental Analysis of Behavior

Killeen, P R (1994) Mathematical principles of reinforcement Behavioral and BrainSciences, 17, 105–135

McDowell, J J (1982) The importance of Herrnstein’s mathematical statement of the Law ofEffect for behavior therapy American Psychologist, 37, 771–779

McIlvane, W J., & Dube, W V (1992) Stimulus control shaping and stimulus controltopographies The Behavior Analyst, 15, 89–94

McIlvane, W J., Dube, W V., & Callahan, T D (1996) Attention: A behavior analyticperspective In G R Lyon & N A Krasnegor (Eds.), Attention, memory, & executivefunction (pp 97–117) Baltimore: Brookes.

McIlvane, W J., Kledaras, J B., Dube, W V., & Stoddard, L T (1989) Automated instruction

of severely and profoundly retarded individuals In J Mulick and R Antonak (Eds.), Transitions

in Mental Retardation (Vol 4, pp 15–76) Norwood, NJ: Ablex

McIlvane, W J., Kledaras, J B., Lowry, M J & Stoddard, L T (1992) Studies of exclusion inindividuals with severe mental retardation Research in Developmental Disabilities, 13, 509–532.Meehl, P E (1950) On the circularity of the law of effect Psychological Bulletin, 47, 52–75

Michael, J (1982) Distinguishing between discriminative and motivational functions ofstimuli Journal of the Experimental Analysis of Behavior, 37, 149–155

Mogenson, G & Cioe΄, J (1977) Central reinforcement In W K Honig & J E R Staddon(Eds.), Handbook of operant behavior (pp 570–595) Englewood Cliffs, NJ: Prentice-Hall.Neuringer, A., & Voss, C Approximating chaotic behavior Psychological Science, 4, 113–119

Nevin, J A (1992) An integrative model for the study of behavioral momentum Journal ofthe Experimental Analysis of Behavior, 57, 301–316

Nevin, J A., Mandell, C., & Atak, J R (1983) The analysis of behavioral momentum Journal

of the Experimental Analysis of Behavior, 39, 49–59

Nevin, J A., Tota, M E., Torquato, R D., & Shull, R L (1990) Alternative reinforcementincreases resistance to change: Pavlovian or operant contingencies? Journal of theExperimental Analysis of Behavior, 53, 359–379

Posner, M I (1980) Orienting of attention The VIIth Sir Frederic Bartlett Lecture TheQuarterly Journal of Experimental Psychology, 32, 3–25

Posner, M I., & Raichle, M E (1994) Images of mind New York: Scientific American Library

Premack, D (1965) Reinforcement theory In D Levine (ed.), Nebraska Symposium onMotivation Lincoln: University of Nebraska Press

Pribram, K H (1991) Brain and perception: Holonomy and structure in figural processing.Hillsdale, NJ: Lawrence Erlbaum Associates

Pribram, K.H (1995) The Enigma of Reinforcement In Neurobehavioral Plasticity: Learning,Development and Response to Brain Insults (pp 381–403) Proceedings of the Bob IsaacsonSymposium in Clearwater, FL NJ: Lawrence Erlbaum Associates

Ray, B A., & Sidman, M (1970) Reinforcement schedules and stimulus control In W N.Schoenfeld (Ed.), The theory of reinforcement schedules (pp 187–214) New York: Appleton-Century-Crofts.

Rilling, M (1977) Stimulus control and inhibitory processes In W K Honig & J E R Staddon(Eds.), Handbook of operant behavior (pp 432–480) Englewood Cliffs, NJ: Prentice-Hall.Rilling, M (1992) An ecological approach to stimulus control and tracking In W.K Honig &J.G Fetterman (Eds.), Cognitive aspects of stimulus control (pp 347–366) New York: Erlbaum

Schenk, J J (1995) Emergent relations of equivalence by outcome-specific consequences inconditional discrimination The Psychological Record, 44, 537–558

Serna, R W., & Wilkinson, K M (1995) Methods for assessing feature-class membership inseries of graded stimuli Proceedings of the 28th Annual Gatlinburg Conference on Researchand Theory in Mental Retardation and Developmental Disabilities 78

Shull, R L (1995) Interpreting cognitive phenomena: Review of Donahoe andPalmer’s Learning and Complex Behavior Journal of the Experimental Analysis ofBehavior, 63, 347–358

Sidman, M (1986) Functional analysis of emergent verbal classes In T Thompson & M D.Zeiler (Eds.), Analysis and integration of behavioral units (pp 213–245) Hillsdale, NJ: Erlbaum

Sidman (1994) Equivalence relations and behavior: A research story Boston: AuthorsCooperative

Sidman, M., & Stoddard, L T (1966) Programming perception and learning for retardedchildren In N R Ellis (Ed.), International Review of Research in Mental Retardation Vol

2 (pp 151–208) New York: Academic Press

Sidman, M., Stoddard, L T., & Mohr, J P (1968) Some additional quantitative observations ofimmediate memory in a patient with bilateral hippocampal lesions Neuropsychologia, 6, 245–254

Skinner, B F (1957) Verbal behavior New York: Appleton-Century-Crofts

Skinner, B F (1969) Contingencies of reinforcement New York: Appleton-Century-Crofts.Skinner, B F (1974) About behaviorism New York: Knopf

Skinner, B F (1989) The origins of cognitive thought American Psychologist, 44, 13–18

Stoddard, L T (1982) An investigation of automated methods for teaching severely retardedindividuals In N R Ellis (Ed.), International review of research in mental retardation (pp 163–207) New York: Academic Press

Stoddard, L T., & Sidman, M (1967) The effects of errors on children’s performance on acircle-ellipse discrimination Journal of the Experimental Analysis of Behavior, 10, 261–270.Terrace, H S (1963) Discrimination learning with and without “errors” Journal of theExperimental Analysis of Behavior, 6, 1–27.

Terrace, H S (1971) Escape from S- Learning and Motivation, 2, 148–163

Timberlake, W & Allison, J (1974) Response deprivation: An empirical approach toinstrumental performance Psychological Review, 81, 146–164

Tucker, D (this volume) «Need reference when available»

Williams, B A (1988) Reinforcement, choice, and response strength In R C Atkinson, R J.Herrnstein, G Lindzey, & R D Luce (Eds.), Stevens’ handbook of experimentalpsychology Volume 2: Perception and motivation (pp 167–244) New York: J Wiley & Sons

Figure 1 Identity-matching baseline and arbitrary-matching test trials for an investigation of theminimal experimental history necessary for emergent matching based on stimulus-reinforcerrelations

Figure 2 Hypothetical data illustrating effects of different reinforcement schedules on behavioralresistance to change (momentum).

Figure 3 Actual data illustrating effects of different reinforcement schedules on behavioralmomentum in an individual with severe mental retardation