CT Scheme and ERA Scheme
The Cross-Target (CT) Method
To develop our agent based experiments, we introduce thefollowing general hypothesis (GH): an agent, acting in aneconomic environment, must develop and adapt her capability ofevaluating, in a coherent way, (1) what she has to do in order toobtain a specific result and (2) how to foresee the consequencesof her actions. The same is true if the agent is interacting withother agents. Beyond this kind of internal consistency (IC),agents can develop other characteristics, for example thecapability of adopting actions (following external proposals, EPs)or evaluations of effects (following external objectives, EOs)suggested from the environment (for example, following rules) orfrom other agents (for examples, imitating them). Thoseadditional characteristics are useful for a better tuning of theagents in making experiments.
To apply the GH we are employing here artificial neuralnetworks; we observe, anyway, that the GH can be applied usingother algorithms and tools, reproducing the experience-learning-consistency-behaviorcycle with or without neural networks.
An introductory general remark: in all the cases to which wehave applied our GH, the preliminary choice of classifyingagents' output in actions and effects has been useful (i) toclarify the role of the agents, (ii) to develop modelplausibility and results, (iii) to avoid the necessity of priorstatements about economic rational optimizing behavior (Beltrattiet al. 1996).
Economic behavior, simple or complex, can appear directly as aby-product of IC, EPs and EOs. To an external observer, ourArtificial Adaptive Agents (AAAs) are apparently operating withgoals and plans. Obviously, they have no such symbolic entities,which are inventions of the observer. The similarity that werecall here is that the observations and analyses about realworld agents' behavior can suffer from the same bias. Moreover,always to an external observer, AAAs can appear to apply therationality paradigm, with maximizing behavior.
The main problem is: obviously agents, with their action, havethe goal of increasing or decreasing something, but it is notcorrect to deduce from this statement any formal apparatusencouraging the search for complexity within agents, not even inthe as if perspective. With our GH, and hereafter with theCross Target (CT) method, we work at the edge of Alife techniquesto develop Artificial Worlds of simple bounded rationality AAAs:from their interaction, complexity, optimizing behavior andOlympic rationality can emerge, but externally to the agents.
In order to implement this ideal target without falling in thetrap of creating models that are too complicated to be managed,we consider artificially intelligent agents founded uponalgorithms which can be modified by a trial and error process. Inone sense our agents are even simpler than those considered inneoclassical models, as their targets and instruments are not aspowerful as those assumed in those models. From another point ofview, however, our agents are much more complex, due to theircontinuous effort to learn the main features of the environmentwith the available instruments.
The name cross-targets (CTs) comes from the technique used tofigure out the targets necessary to train the ANNs representingthe artificial adaptive agents (AAAs) that populate ourexperiments.
Following the GH, the main characteristic of these AAAs isthat of developing internal consistency between what to do andthe related consequences. Always according to the GH, in many (economic)situations, the behavior of agents produces evaluations that canbe split in two parts: data quantifying actions (what to do) andforecasts of the outcomes of the actions. So we specify two typesof outputs of the ANN and, identically, of the AAA: (i) actionsto be performed and (ii) guesses about the effects of thoseactions.
Both the targets necessary to train the network from the pointof view of the actions and those connected with the effects arebuilt in a crossed way, originating the name Cross Targets. Theformer are built in a consistent way with the outputs of thenetwork concerning the guesses of the effects, in order todevelop the capability to decide actions close to the expectedresults. The latter, similarly, are built in a consistent way withthe outputs of the network concerning the guesses of the actions,in order to improve the agent's capability of estimating theeffects emerging from the actions that the agent herself isdeciding.
CTs, as a fulfillment of the GH, can reproduce economicsubjects' behavior, often in internally ingenuous ways, butexternally with complex results.
The method of CTs, introduced to develop economic subjects'autonomous behavior, can also be interpreted as a generalalgorithm useful for building behavioral models without usingconstrained or unconstrained optimization techniques. The kernelof the method, conveniently based upon ANNs (but it could also beconceivable with the aid of other mathematical tools), islearning by guessing and doing: the subject control capabilitiescan be developed without defining either goals or maximizingobjectives.
We choose the neural networks approach to develop CTs, mostlyas a consequence of the intrinsic adaptive capabilities of neuralfunctions. Here we will use feed forward multilayer networks.
Figure 1 describes an AAA learning and behaving in a CT scheme.The AAA has to produce guesses about its own actions and relatedeffects, on the basis of an information set (the input elementsare I1,...,Ik). Remembering the requirement of IC, targets inlearning process are: (i) on one side, the actual effects -measured through accounting rules - of the actions made by thesimulated subject; (ii) on the other side, the actions needed tomatch guessed effects. In the last case we have to use inverserules, even though some problems arise when the inverse isindeterminate.
A first remark, about learning and CT: analyzing the changesof the weights during the process we can show that the matrix ofweights linking input elements to hidden ones has little or nochanges, while the matrix of weights from hidden to output layerchanges in a relevant way. Only hidden-output weight changesdetermine the continuous adaptation of ANN responses to theenvironment modifications, as the output values of hidden layerelements stay almost constant. This situation is the consequenceboth of very small changes in targets (generated by CT method)and of a reduced number of learning cycles.
The resulting network is certainly under trained:consequently, the simulated economic agent develops a localability to make decisions, but only by adaptations of outputs tothe last targets, regardless to input values. This is short termlearning as opposed to long term learning.
Some definitions: we have (i) short term learning, in theacting phase, when agents continuously modify their weights (mainlyfrom the hidden layer to the output one), to adapt to the targetsself-generated via CT; (ii) long term learning, ex post, when weeffectively map inputs to targets (the same generated in theacting phase) with a large number of learning cycles, producingANNs able to definitively apply the rules implicitly developed inthe acting and learning phase.
A second remark, about both external objectives (EOs) andexternal proposals (EPs): if used, these values substitute thecross targets in the acting and adapting phase and areconsistently included in the data set for ex post learning.Despite the target coming from actions, the guess of an effectcan be trained to approximate a value suggested by a simple rule,for example increasing wealth. This is an EO in CT terminology.Its indirect effect, via CT, will modify actions, making themmore consistent with the (modified) guesses of effects. Viceversa, the guess about an action to be accomplished can bemodified via an EP, affecting indirectly also the correspondingguesses of effects. If EO, EP and IC conflict in determiningbehavior, complexity may emerge also within agents, but in abounded rationality perspective, always without the optimizationand full rationality apparatus.
Figure 1
The Environment-Rules-Agents (ERA) scheme
Swarm is a natural candidate to this kind of structures, butwe need some degree of standardization, mainly when we go fromsimple models to complex results. Here a crucial role for theusefulness and the acceptability of the experiments is played bythe structure of the underlying models. For this reason, weintroduce here a general scheme that can be employed in buildingagent-based simulations.
The main value of the Environment-Rules-Agents (ERA) scheme,introduced in Gilbert and Terna (2000) and shown in Figure 2, isthat it keeps both the environment, which models the context bymeans of rules and general data, and the agents, with theirprivate data, at different conceptual levels. To simplify thecode, we suggest that agents should not communicate directly, butalways through the environment; as an example, the environmentallows each agent to know the list of its neighbors. This is notmandatory, but if we admit direct communication between agents,the code becomes more complex.
With the aim of simplifying the code design, agent behavior isdetermined by external objects, named Rule Masters, that can beinterpreted as abstract representations of the cognition of theagent. Production systems, classifier systems, neural networksand genetic algorithms are all candidates for the implementationof Rule Masters.
We may also need to employ meta-rules, i.e., rules used tomodify rules (for example, the training side of a neural network).The Rule Master objects are therefore linked to Rule Makerobjects, whose role is to modify the rules mastering agentbehavior, for example by means of a simulated learning process.Rule Masters obtain the information necessary to apply rules fromthe agents themselves or from special agents charged with thetask of collecting and distributing data. Similarly, Rule Makersinteract with Rule Masters, which are also responsible forgetting data from agents to pass to Rule Makers.
Agents may store their data in a specialized object, theDataWarehouse, and may interact both with the environment andother agents via another specialized object, the Interface (DataWarehouseand Interface are not represented in Figure 2, having a simpleone to one link with their agent; they are used in the bp-ctpackage and in its derivatives).
Although this code structure appears to be complex, there is abenefit when we have to modify a simulation. The rigidity of thestructure then becomes a source of invaluable clarity. An exampleof the use of this structure can be found in the code of theSwarm application bp-ct in which the agents are neural networks;also the specialized code related to the hayekian experimentreported in ct-hayek code, being built upon bp-ct, uses the samestructure
A second advantage of using the ERA structure is itsmodularity, which allows model builders to modify only the RuleMaster and Rule Maker modules or objects whenever one wants toswitch from agents based on neural networks, to alternatives suchas production systems, classifier systems or genetic algorithms.
In Swarm terms, the Environment coincides with the ModelSwarmobject, while the ObserverSwarm object is external to thisstructure.
Figure 2
More details about CT and ERA can be found in Terna (2000).
Beltratti, A., Margarita, S., Terna, P.
Gilbert N., Terna P. (2000), How to build anduse agent-based models in social science, Mind & Society,no. 1, 2000.
Terna P. (2000), Economic Experiments withSwarm: a Neural Network Approach to the Self-Development ofConsistency in Agents' Behavior, in F. Luna and B. Stefansson (eds.),Economic Simulations in Swarm: Agent-Based Modelling and ObjectOriented Programming. Dordrecht and London, Kluwer Academic,2000.