The Evolution of Cooperation

The Evolution of Cooperation in a Hierarchically Structured, Two-Issue Iterated Prisoner’s Dilemma Game: An Agent-Based Simulation of Trade and War October 2004 Abstract: The study of the emergence of cooperation in anarchy has neglected to address the implications of multiple issues linked within a hierarchical structure In many social situations, actors simultaneously (or sequentially) interact in several issue areas For example, countries typically interact with their neighbors on both security issues and economic issues In this situation, there is a hierarchy among the issues in that the costs and benefits in one issue (war) surpass the costs and benefits in the other issue area (trade) This article explores the impact of multiple, hierarchically structured issue areas on the level of cooperation in an iterated prisoner’s dilemma game using a computer simulation Three findings emerge from the analysis First, while altering the payoffs in only one issue area has a marginal impact on overall cooperation, sustaining high levels of cooperation across the population requires improving incentives to cooperate in both issue areas Second, cooperation can be sustained even when the payoffs are based on relative power Third, the rate of learning is critical for the emergence and spread of cooperative strategies David L Rousseau Assistant Professor Department of Political Science 235 Stiteler Hall University of Pennsylvania Philadelphia, PA 19104 E-mail: rousseau@sas.upenn.edu Phone: (215) 898-6187 Fax: (215) 573-2073 and Max Cantor Department of Political Science and the School of Engineering and Applied Science University of Pennsylvania Philadelphia, PA 19104 E-mail: mxcantor@sas.upenn.edu INTRODUCTION Since Axelrod’s (1984) landmark study, we have learned a lot about the evolution of cooperation and the utility of agent-based computer simulations to explore the evolutionary process.1 One issue that has been neglected in this growing literature is the impact of multiple issue areas on the emergence of cooperation In many social situations, actors simultaneously (or sequentially) interact in several issue areas For example, countries typically interact with their neighbors on both security issues and economic issues In this example, there is a hierarchy among the issues in that the costs and benefits in one issue (war) surpass the costs and benefits in the other issue area (trade) This article explores the impact of multiple, hierarchically structured issue areas on the level of cooperation in an iterated prisoner’s dilemma game using a computer simulation Three findings emerge from the analysis First, while altering the payoffs in only one issue area has a marginal impact on overall cooperation, sustaining high levels of cooperation across the population requires improving incentives to cooperate in both issue areas Second, cooperation can be sustained even when the payoffs are based on relative power Third, the rate of learning is critical for the emergence and spread of cooperative strategies While our findings are generalizable to any hierarchically linked issue areas, we focus on the implications of multiple issue areas in international relations Over the last several centuries the sovereign state has emerged as the dominant organizational unit in the international system (Spruyt 1994) During this evolutionary period, sovereign states and their competitors have struggled to identify an optimal strategy for maximizing economic growth and prosperity In general, states have pursued some combination of three general strategies: (1) war, (2) trade, or (3) isolation For example, realists such as Machiavelli (1950) argued that military force is an effective instrument for extracting wealth and adding productive territory In contrast, liberals such as Cobden (1850, 518) argued that international trade was the optimal strategy for maximizing economic efficiency and national wealth Finally, economic nationalists such as Hamilton (Earle 1986) rejected this liberal hypothesis and argued that isolation from the leading trading states rather than integration with them would enhance economic development The three policies are not mutually exclusive For example, List promoted both military expansion and economic isolation for Germany in the 19th century (Earle 1986) However, in general, trade and war are viewed as hierarchical because states rarely trade with enemies during military conflict While some instances have been identified (e.g., Barbieri and Levy 1999), many systematic studies indicated that military conflict reduces international trade (e.g., Kim and Rousseau 2004) The importance of issue linkage has long been recognized Axelrod and Keohane (1986, 239) argued that issue linkage could be used to alter incentive structures by increasing rewards or punishments Although this idea of issue linkage has been explored with rational choice models (e.g., Morgan 1990; Lacy and Niou 2004) and case studies (e.g., Oye 1986), modeling issue linkage as a two-stage game within an N-person setting has largely been neglected Conceptualizing trade and war as generic two stage game raises a number of interesting questions that will be explored in this article If trade and war are the two most important issues in international relations, how does the hierarchical linkage between the two issues influence the amount of cooperation in the international system? What factors encouraged (or discouraged) the rise of a cooperative order? What strategies might evolve to facilitate the maximization of state wealth in a two issue world? Finally, how stable are systems characterized by hierarchical linkages? COOPERATION IN ANARCHY Many interactions among states are structured as “Prisoner’s Dilemmas” due to order of preferences and the anarchical environment of the international system The Prisoner’s Dilemma is a non-zero-sum game in which an actor has two choices: cooperate (C) with the other or defect (D) on them The 2x2 game yields four possible outcomes that can be ranked from best to worst: 1) I defect and you cooperate (DC or the Temptation Payoff "T"); 2) we both cooperate (CC or the Reward Payoff "R"); 3) we both defect (DD or the Punishment Payoff "P"); and 4) I cooperate and you defect (CD or the Sucker's Payoff "S") The preference order coupled with the symmetrical structure of the game implies that defection is a dominant strategy for both players in a single play game because defecting always yields a higher payoff regardless of the strategy selected by the opponent Therefore, the equilibrium or expected outcome for the single play prisoner’s dilemma game is “defect-defect” (i.e., no cooperation) This collectively inferior equilibrium is stable despite the fact that both actors would be better off under mutual cooperation The problem is neither actor has an incentive to unilaterally alter their selected strategy because they fear exploitation.3 Many situations in international economic and security affairs have been models as iterated Prisoner’s Dilemmas In the trade arena, researcher tend to assume states have the following preference order from “best” to “worst”: 1) I raise my trade barriers and you keep yours low; 2) we both keep our trade barriers low; 3) we both raise our trade barriers; and 4) I keep my trade barriers low and you keep yours high In the security sphere, scholars typically assume that states possess the following preference order: 1) you make security concessions, but I not; 2) neither of us makes security concessions; 3) both of us make security concessions; and 4) I make security concessions but you not While scholars have identified particular instances in which preference orders may deviate from these norms (e.g., Conybeare 1986 and Martin 1992), the Prisoner’s Dilemma preference order is typical of many, if not most, dyadic interactions in the two issue areas.4 Students of game theory have long known that iteration offers a possible solution to the dilemma (Axelrod 1984, 12) If two agents can establish a cooperative relationship, the sum of a series of small "Reward Payoffs" (CC) can be larger than a single "Temptation Payoff" followed by a series of "Punishment Payoffs" (DD) The most common solution for cooperation within an iterated game involves rewards for cooperative behavior and punishments for non-cooperative behavior I will only cooperate if you cooperate The strategy of Tit-For-Tat nicely captures the idea of conditional cooperation A player using a Tit-For-Tat strategy cooperates on the first move and reciprocates on all subsequent Axelrod (1984) argues that the strategy is superior to others because it is nice (i.e., cooperates on first move allowing a CC relationship to emerge), firm (i.e., punishes the agent's opponent for defecting), forgiving (i.e., if a defector returns to cooperation, the actor will reciprocate), and clear (i.e., simple enough for the agent's opponent to quickly discover the strategy) Leng (1993) and Huth (1988) provide empirical evidence from international relations to support the claim that states have historically used reciprocity oriented strategies to increase cooperation Although subsequent work has demonstrated that Tit-For-Tat is not a panacea for a variety of reasons (e.g., noise, spirals of defection, sensitive to the composition of the initial population), reciprocity oriented strategies are still viewed as viable mechanisms for encouraging cooperation under anarchy Although the emergence of cooperation can be studied using a variety of techniques (e.g., case studies, regression analysis, laboratory experiments), in this article we employ an approach that has been widely used in this literature: an agent-based computer simulation (Axelrod 1997; Macy and Skvoretz 1998; Rousseau 2005) Agent-based simulations are “bottom-up” models that probe the micro-foundations of observable macro-patterns Agent-based models make four important assumptions (Macy and Willer 2002, 146) First, agents are autonomous in that no hierarchical structure exists in the environment Second, agents are interdependent in that their welfare depends on interactions with other agents Third, agent choices are based on simple rules utilizing limited information rather than complex calculations requiring large amounts of data Finally, agents are adaptive or backward looking (e.g., if doing poorly in last few rounds, change behavior) as they alter characteristics or strategies in order to improve performance Agent-based models are ideally suited for complex, non-linear, self-organizing situations involving many actors Macy and Willer claim that agent-based models are “most appropriate for studying processes that lack central coordination, including the emergence of organizations that, once established, impose order form the top down” (2002, 148) As with all methods of investigation, computer simulations have strengths and weaknesses.6 On the positive side of the ledger, five strengths stand out First, as with formal mathematical models, simulations compel the researcher to be very explicit about assumptions and decision rules Second, simulations allow us to explore extremely complex systems that often have no analytical solution Third, simulations resemble controlled experiments in that the researcher can precisely vary a single independent variable (or isolate a particular interaction between two or more variables) Fourth, while other methods of inquiry primarily focus on outcomes (e.g., democratic dyads engage in war?), simulations allow us to explore the processes underlying the broader causal claim (e.g., how does joint democracy decrease the likelihood of war?) Fifth, simulations provide a nice balance between induction and deduction While the developer must construct a logically consistent model based on theory and history, the output of the model is explored inductively by assessing the impact of varying assumptions and decision rules On the negative side of the ledger, two important weaknesses stand out First, simulations have been criticized because they often employ arbitrary assumptions and decision rules (Johnson 1999, 1512) In part, this situation stems from the need to explicitly operationalize each assumption and decision rule However, it is also due to the reluctance of many simulation modelers to empirically test assumptions using alternative methods of inquiry Second, critics often question the external validity of computer simulations While one of the strengths of the method is its internal consistency, it is often unclear if the simulation captures enough of the external world to allow us to generalize from the artificial system we have created to the real world we inhabit Given that we are primarily interested in testing the logical consistency and completeness of arguments, the weaknesses are less problematic in our application of agent-based model That is, we are interested in probing whether traditional claims (e.g., increasing the reward payoff increases cooperation or relative payoffs decrease cooperation) produce expected patterns when confronting multiple, hierarchically structured issues While we use the trade/war setting to illustrate the applicability of the model, we not claim to be precisely modeling the international trade or interstate conflict This is not to say that using agent-based simulations to model real world interactions is impossible or unimportant We simply wish to use the technique to evaluate general claims about factors promoting or inhibiting cooperation OVERVIEW OF THE MODEL In our agent-based model, the world or "landscape" is populated with agents possessing strategies that are encoded on a string or “trait set” (e.g., 00010010100110011001) Over time the individual traits in the trait set (e.g., the “0” at the start of the string) change as less successful agents emulate more successful agents The relatively simple trait set with twenty individual traits employed in the model allows for over million possible strategies Presumably, only a small subset of these possible combinations produces coherent strategies that maximize agent wealth The structure of our model was inspired by the agent-based model developed by Macy and Skvoretz (1998) They use a genetic algorithm to model the evolution of trust and strategies of interaction in a prisoner’s dilemma game with an exit option Like us, they are interested in the relative payoff of the exit option, the location of interaction, and the conditionality of strategies Our model, however, differs from theirs in many important respects First, unlike their single stage game, our model is a two stage prisoner’s dilemma game that links two issues in a hierarchical fashion (i.e., both a “war” game and a “trade” game) Second, our trait set differs from theirs because we have tailored it to conform to standard assumptions about trade and war In contrast, their trait set is more akin to “first encounter” situations (e.g., you greet partner? you display marker? you distrust those that display marker?) Third, our model allows for complex strategies such as Tit-For-Tat to evolve across time Our simulation model consists of a population of agents that interact with each other in one of three ways: 1) trade with each other; 2) fight with each other; or 3) ignore each other Figure illustrates the logic of each of these encounters The game is symmetrical so each actor has the same decision tree and payoff matrices (i.e., the right side of the figure is the mirror image of the left side of the figure) Each agent begins by assessing the geographic dimension of the relationship: if the agents are not immediate neighbors, then the agent skips directly to the trade portion of the decision tree If the two agents are neighbors, the agent must ask a series of questions in order to determine if it should enter the war game If it chooses not to fight, it asks a similar series of questions to determine if it should enter the trade game If it chooses neither war nor trade, it simply exits or "ignores" the other agent Both the war and the trade games are structured as prisoner's dilemma games **** insert Figure about here **** The model focuses on learning from one's environment In many agent-based simulations, agents change over time through birth, reproduction, and death (Epstein and Axtell 1996) In such simulations, unsuccessful agents die as their wealth or power declines to zero These agents are replaced by the offspring of successful agents that mate with other successful agents In contrast, our model focuses on social learning.7 Unsuccessful agents compare themselves to agents in their neighborhood If they are falling behind, they look around for an agent to emulate Given that agents lack insight into why other agents are successful, they simply imitate decision rules (e.g., don't initiate war against stronger agents) selected at random from more successful agents Over time repeatedly unsuccessful agents are likely to copy more and more of the strategies of their more successful counterparts Thus, the agents are “boundedly rational” in that they use short cuts in situations of imperfect information in order to improve their welfare (Simon 1982).8 The fitness of a particular strategy is not absolute because its effectiveness depends on the environment in which it inhabits Unconditional defection in trade and war is a very effective strategy in a world populated by unconditional cooperators However, such an easily exploited environment begins to disappear as more and more agents emulate the more successful (and more coercive) unconditional defectors While this implies that cycling is possible, it does not mean it is inevitable As the simulation results demonstrate, some populations are more stable across time because they are not easily invaded by new strategies In the simulation, trade and war are linked in four ways First, agents cannot trade with other agents if they are in a crisis or at war A crisis emerges when one side exploits the other in the war game (i.e., DC or CD outcome in the war game) A war occurs when both sides defect in the war game (i.e., DD) Second, agents can adopt strategies that either mandate or prohibit going to war with current trading partners Third, while agents can fight and trade with immediate neighbors, they can only trade with non-contiguous states Although this rule is obviously a simplification because great powers are able to project power, it captures the fact that most states are capable to trading but not fighting at a great distance Fourth, while an "exit" option exists with respect to trade (i.e., you can choose not to enter a relationship with someone you don't trust), an agent cannot exit from a war relationship because agents can be a victim of war whether or not they wanted to participate THE WAR-TRADE-ISOLATION SIMULATION Figure illustrates the basic structure of the war-trade-isolation simulation The population consists of a wrapping 20 x 20 landscape or grid of agents Each of the 400 individual agents is surrounded by eight immediate neighbors (referred to as the Moore neighborhood in the simulation literature) Each agent in the landscape has a set of traits that determine the agent’s characteristics and rules for interacting with other agents In Figure 2, we see the distribution of the Trait #1 shown in black over the entire landscape The structure of the actor and its northeastern neighbor are highlighted in the Figure in order to illustrate the properties of the agents In the Figure, the actor's War Character (Trait #1) is "Cooperate" (or "0" shown in white in the figure) and the neighbor's War Character is "Defect" (or "1" shown in black) The Figure indicates that war-like agents tend to cluster spatially and are in a minority in this particular landscape The simulation is executed for a number of iterations or "runs." In each iteration, the actor interacts with neighbors, updates their power based on outcomes of interactions, and (possibly) updates traits by adopting those of more successful neighbors Therefore, the trait landscapes update slowly over time as the more successful agents are emulated by their less successful neighbors **** insert Figure about here **** Figure displays the payoff structure for each interaction During each iteration of the simulation, the agent interacts with the eight members of its Moore neighborhood Nine outcomes of the interactions are possible: ignoring (exiting prior to trade), trading (DC, CC, DD, CD), or fighting (DC, CC, DD, CD) For example, if both states cooperate in the trade game they each receive a payoff of "1" Conversely, if they both defect in the war game they each receive a payoff of "-5" The exit payoff always falls between the DD payoff and the CC payoff in the trade game The ExitPayoff parameter, which varies from to 1, determines the exit payoff between these endpoints So a setting of 0.50, which is the default in the simulation, sets the exit payoff as the midpoint between the DD minimum (0) and the CC maximum (1) As with all parameters in the model, the user is free to manipulate the parameter in order to explore the model.10 **** insert Figure about here **** Four points about the payoff structure should be highlighted First, the payoffs in the model must be symmetrical because the user specifies a single matrix for all actors While this is an obvious simplification, it is a standard assumption in most of the formal and simulation work employing a prisoner's dilemma.11 Second, the gains from the temptation payoff and the losses from the sucker's payoff in the trade game are always less than for the analogous payoffs in the war game This captures the hierarchical nature of the relationship between trade and war Moreover, it reflects the widely held belief that the stakes are lower in the economic realm compared to the military realm (Lipson 1984; Stein 1990, 135; Rousseau 2002) Third, the simulations in this paper always set the DD outcome in the trade game to zero and the CC outcome in the war game to zero While this is not necessary, it implies that peacetime and no 10 Location: Are you a neighbor? No Location: Are you a neighbor? Figure 1: Overview of Trade-War Model Yes Yes Situation: What is the current state of relations? Crisis or War War Choice: Should I fight with the other? Yes War Strategy: Cooperate or Defect? Cooperate Defect No Trade Choice: Should I trade with the other? No R R S T T S P P Cooperate War Strategy: Cooperate or Defect? Peace Yes Defect War Choice: Should I fight with the other? No Exit Exit Yes Trade Strategy: Cooperate or Defect? Situation: What is the current state of relations? Crisis or War Peace No No Trade Choice: Should I trade with the other? Yes Cooperate Defect R R S T T S P P 31 Cooperate Defect Trade Strategy: Cooperate or Defect? Figure 2: Landscapes and Actors 32 Figure 3: Payoff Structure for Agents Trade Payoffs Other Cooperate Defect Cooperate Self Defect -4 War Payoffs Other Cooperate Defect Cooperate Self Defect -4 10 -11 -5 -11 10 -5 Notes: Self payoff is located in the lower left of each cell Other payoff is located in the upper right of each cell The “exit” payoff is equal to the Reward payoff plus the Punishment payoff times the ExitPayoff parameter set by the user In the default simulation the ExitPayoff is 0.50, implying an exit payoff of 0.50 33 Figure 4: The War Choice Module Is my war Yes (G2=1) strategy unconditional? What is my strategy? Cooperate (G1=0) Defect (G1=1) No Add # of Defections Fear Bin How often has the other defected in war with “me” during the last X (G23) rounds? (G2=0) Yes (G3=1) Con tinu e Add # of Defections Fear Bin Add # of Defections Fear Bin No (G3=0) How often has the other Yes (G4=1) defected in war with “common neighbors” in the last X (G23) rounds? Conti nu Do I use war behavior with “me” marker? No (G4=0) e How often has the other defected in war with “all others” in the last X (G23) rounds? Con tin ue Do I use war behavior with “neighbors” marker? Yes (G5=1) War Choice Module Do I use war behavior with “all others” marker? No (G5=0) Calculate Probability of Fear to continue (continues to War Strategy Module) 34 Defect With Probability 1-p (Trust) Figure 5: The War Strategy Module War Strategy Module No Did I trade with the other state last round? Yes Is the other agent my type? Yes No No Do I treat weaker states differently? Do I treat Yes (G7=1) trade partners differently? 8= 1) No Yes G =0 G8 s( Ye Do I treat non-trade partners differently? No Is the other state weaker than me? same ) Do I treat other types differently? No No No Do I treat my type differently? Ye s( 6= G Ye s( Yes (G7=0) No ) =1 (G If Cooperator (G1=0), then defect If Defector (G1=1), then cooperate Do I treat stronger states differently? s Ye 0) (continues from War Choice Module) Not Sure If Defector (G1=1), then defect What is my war strategy? 35 If Cooperator (G1=0), then cooperate If Cooperator (G1=0), then defect If Defector (G1=1), then cooperate Figure 6: The Trade Choice Module Is my trade Yes (G13=1) strategy unconditional? No What is my trade strategy? Cooperate (G12=0) Defect (G12=1) (G13=0) Add # of Defections Trust Bin How often has the other defected in trade with “me” during the last X (G23) rounds? Yes (G14=1) Do I use trade behavior with “me” marker? Con tinu e Add # of Defections Trust Bin How often has the other Yes (G15=1) defected with “common neighbors” in the last X (G23) rounds? Conti nu Add # of Defections Trust Bin No (G14=0) e How often has the other defected with “all others” in the last X (G23) rounds? Con tin Yes (G16=1) Do I use trade behavior with “neighbors” marker? No (G15=0) Trade Choice Module Do I use trade behavior with “all others” marker? No (G16=0) ue Calculate Probability of Trust To continue (continues to Trade Strategy Module) 36 Exit With Probability 1-p (Trust) Figure 7: The Trade Strategy Module ) =0 19 (G No Yes Do I treat neighbors differently in trade? Yes Do I treat my type differently in trade? No Am I a neighbor of the other state? No Yes (G18=1) 1) Do I treat other types differently in trade? No Do I treat weaker states differently in trade? 19 = s Ye No Yes Is the other agent my type? No Not Sure What is my trade strategy? 37 No Ye s( G G Ye s( No Do I treat non-neighbors differently in trade? If Defector (G12=1), then defect Is the other state weaker than me? ) =1 Yes (G18=0) No If Cooperator (G12=0), then defect If Defector (G12=1), then cooperate Do I treat stronger states differently in trade? Trade Strategy Module G1 s( Ye 17 = 0) (continues from Trade Choice Module) If Cooperator (G12=0), then cooperate If Cooperator (G12=0), then defect If Defector (G12=1), then cooperate Figure 8: Output From The Baseline Model A) Distribution of traits in the landscape at iteration 5000 (Attribute=1 shown in black) B) Distribution of War shown by black lines connecting nodes C) Distribution of Trade shown by green lines connecting nodes E) Change in the Percentage of States Having a Traits #1 (red), #2 (blue), #12 (aqua), & #13 (black) Across Time D) Spatial Distribution of Wealth (Blue above 10000 and Red Below 10000 Power Units) F) Average Number of Wars (blue), Trade (red), and Gini Coefficient (aqua) Across Time 38 Figure 9: H1: Increasing the Gains From Trade A) Distribution of traits in the landscape at iteration 5000 (Attribute=1 shown in black) B) Distribution of War shown by black lines connecting nodes C) Distribution of Trade shown by green lines connecting nodes E) Change in the Percentage of States Having a Traits #1 (red), #2 (blue), #12 (aqua), & #13 (black) Across Time D) Spatial Distribution of Wealth (Blue above 10000 and Red Below 10000 Power Units) F) Average Number of Wars (blue), Trade (red), and Gini Coefficient (aqua) Across Time 39 Figure 10: Rapid Learning Representative Runs A) Distribution of traits in the landscape at iteration 5000 (Attribute=1 shown in black) B) Distribution of War shown by black lines connecting nodes C) Distribution of Trade shown by green lines connecting nodes E) Change in the Percentage of States Having a Traits #1 (red), #2 (blue), #12 (aqua), & #13 (black) Across Time 40 D) Spatial Distribution of Wealth (Blue above 10000 and Red Below 10000 Power Units) F) Average Number of Wars (blue), Trade (red), and Gini Coefficient (aqua) Across Time Figure 11: Relative Gains, Rapid Learning, Increased Trade Benefits, & Increased War Costs A) Distribution of traits in the landscape at iteration 5000 (Attribute=1 shown in black) B) Distribution of War shown by black lines connecting nodes C) Distribution of Trade shown by green lines connecting nodes E) Change in the Percentage of States Having a Traits #1 (red), #2 (blue), #12 (aqua), & #13 (black) Across Time 41 D) Spatial Distribution of Wealth (Blue above 10000 and Red Below 10000 Power Units) F) Average Number of Wars (blue), Trade (red), and Gini Coefficient (aqua) Across Time Table 1: Trait Set for Each Agent Trait Name War Character Trait # Attribute 1 War Unconditional War Behavior w/ Me War Behavior w/ Common Neighbors War Behavior w/ All War Power War Interdependence War Type blank blank 10 blank 11 Trade Character 12 Trade Unconditional 13 Trade Behavior w/ Me 14 Trade Behavior w/ 15 Common Neighbors Trade Behavior w/ All 16 Trade Power 17 Trade Neighbors 18 Trade Type 19 blank 20 Display Type 21 Detect Type 22 Memory Length 23 Description Cooperate in war Defect in war Do not use unconditional approaches in war Use unconditional strategies in war Don't use past behavior in war with me Use past behavior in war with me Don't use past behavior in war with common neighbor Use past behavior in war with common neighbor Don't use past behavior in war with all others Use past behavior in war with all others Treat stronger states differently in war Treat weaker states differently in war Treat non-trading partners differently in war Treat trading partners differently in war Treat other type differently in war Treat similar type differently in war not currently in use Military Propensity Continuous variable measuring probability of initiating conflict 24 not currently in use not currently in use Cooperate in trade Defect in trade Do not use unconditional approaches in trade Use unconditional strategies in trade Don't use past behavior in trade with me Use past behavior in trade with me Don't use past behavior in trade with common neighbor Use past behavior in trade with common neighbor Don't use past behavior in trade with all others Use past behavior in trade with all others Treat stronger states differently in trade Treat weaker states differently in trade Treat non-neighbors differently in trade Treat neighbors differently in trade Treat other type differently in trade Treat your type differently in trade not currently in use Do not display type marker Display type marker Ignore marker Attend to marker Integer recording number of previous rounds used by the agent 42 ENDNOTES 43 For a review of this literature, see Hoffman (2000) and Axelrod and D’Ambrosio (1994) Bearce and Fisher (2002) model trade and war using an agent-based model However, our model differs from theirs in terms of the number of actors, the structure of interaction, the role of learning, the evolution of strategies, the number of stages, and basic interaction rules Defect-Defect (DD) is a Nash equilibrium because no player can unilaterally switch strategies and reap a higher payoff The simulation model described below allows the user to explore any preference order buy simply altering the payoffs in the input screen (e.g., a Stag Hunt in stage and a Prisoner’s Dilemma in stage 2) However, we focus on a two-stage Prisoner’s Dilemma game because we consider it the modal case in a two issue area situation For a critique of Axelrod’s conclusions, see Binmore (1998) For a more positive evaluation of Axelrod’s contribution, see Hoffmann (2000) For a more extensive discussion of strengths and weakness of agent-based modeling, see Lustick, Miodownik, and Eidelson (2004), Rousseau (2005), Johnson (1999), and Axtell (2000) Many authors prefer to restrict the use of the term “gene” to situations involving death and reproduction For these individuals, the learning model proposed here would be more appropriately labeled a “meme” structure (Dawkins 1976) In order to make the model as accessible as possible, we will simply talk about the evolution of traits A fully rational agent would be able to select a strategy through optimization In our model, agents “satisfice” based on the mean of the neighborhood (i.e., if power if below the mean, copy from anyone above the mean) This implies that agents simply have to rank order neighbors in terms of wealth Moreover, agents have imperfect information about the strategies employed by others and the effectiveness of individual strategies The model allows both local and long distance interaction However, the results presented here involve local learning and interaction only 10 The model can be downloaded from www.xxx.edu 11 For an interesting exception, see the "trade war" analysis by Conybeare (1986), the sanctions game by Martin (1992), and some of the games explored by Snyder and Diesing (1977) 12 These strategies have typically been discussed with respect to the "echo effect." If both players are using a Tit-For-Tat strategy, it is quite easy to become trapped in a punishment spiral (i.e., you defect because I punished you, I defect because you punished me, you defect because I punished you, ) Axelrod (1984, 38) argues that more lenient strategies such as Tit-For-2-Tats and 90% Tit-For-Tat can reduce the likelihood of spirals, particularly under uncertainty 13 Research has shown that “uncertainty” or noise can sharply reduce cooperation (Bendor 1993; Axelrod 1997) Many different sources of uncertainty exist within the iterated prisoner’s dilemma, including (1) uncertainty over payoffs, (2) uncertainty over relative power, (3) uncertainty over specific plays, and (4) uncertainty over strategies We plan to explore the relative impact of different types of uncertainty in future analysis 14 If the UpdateProbability parameter is set to 1.0, the agents will copy all of the traits of the more successful agent Thus, the most rapid learning occurs with learning rule (a) and update probability 1.0 15 Rogowski (1989, 21) claims that railroads decreased the cost of land transportation by 85-95 percent and steam ships decreased the cost of water transportation by 50 percent 16 The four traits that are currently not used in the model not appear in the figure Therefore, the number of the traits runs from left to right: Row 1: {1, 2, 3, 4, and 5}; Row 2: {6, 7, 8, 12, and 13}; Row 3: {14, 15, 16, 17, and 18}; Row 4: {19, 21, 22, 23, and 24 (not shown)} 17 Hoffmann (1999) examines the interaction between the location of learning (local versus distant) and the location of interaction (again local versus distant) He concludes that cooperation is enhanced by local learning and local interaction In his simulation of the democratic peace, Cederman (2001) found that the democratic peace tended to appear in clusters that grew as alliances and collective security were added to the baseline simulation In sum, the theoretical literature and the empirical analysis to date supports the expectation of clustering 18 Pahre focuses on clusters of negotiations rather than spatial clustering However, the MFN treaties were regionally clustered in Western Europe ... in the trade game they each receive a payoff of "1" Conversely, if they both defect in the war game they each receive a payoff of "-5" The exit payoff always falls between the DD payoff and the. .. trait (#23) of 10, then the agent looks to the last ten iterations between the agent and the other If the other has defected of the last 10 times with the agent, there is a 50% change that the agent... located in the lower left of each cell Other payoff is located in the upper right of each cell The “exit” payoff is equal to the Reward payoff plus the Punishment payoff times the ExitPayoff parameter