Behaviorist approaches to investigating memory and learning: A primer for synthetic biology and bioengineering

Motivation is generally not an issue with habituation, sensitization, alpha conditioning and classical conditioning experiments. The motivation in these experiments comes from the termination of the unconditioned stimuli themselves. Motivation is important in instrumental and operant experiments; as these experiments are based on “the control of behavior by its consequences”, the consequences have to be “coveted.”

The use of aversive stimuli has much to recommend it at the initial stages of experimentation, and can be more effective than rewards. An aversive stimulus such as electric shock is easy to administer and control. It can be precisely turned on and off without after-effects such as those associated with changes in temperature. Shock can also be easily incorporated into instrumental and operant conditioning experimental designs associated with escape, punishment, and avoidance. One decision the researcher must make is whether to use alternating current (AC) or direct current (DC) shock. We recommend DC current as it is readily produced by batteries and therefore easier to use than AC current. Direct current is also less dangerous to use than AC. Whether AC or DC current is used, the researcher should run pilot experiments to determine the minimum amount of shock that can be used to elicit a response from the organism.

Of all the available instrumental and operant conditioning procedures, we recommend the study of escape during the pilot phase of experimentation. Escape can be considered as an example of appetitive conditioning, with the difference being that the reinforcement is the termination of the aversive event rather than the receipt of an appetitive stimulus. In this case, the appetitive reinforcement is “shock free time.” The escape paradigm can be used to investigate many phenomena of appetitive conditioning including the delay between the response and the cessation of shock, reinforcement schedules, reinforcement magnitude, and rate of reinforcement.

Punishment is another associative learning paradigm that can easily be studied with synthetic organisms. In punishment, a specific response (such as crawling) produces the delivery of an aversive event. If punishment is effective, the organism will stop making the response relative to unpaired controls. As in the escape paradigm, punishment can be used to study the delay between the response and the cessation of shock, reinforcement schedules, reinforcement magnitude, and rate of reinforcement. Perhaps the easiest punishment paradigm to use is to first determined if the synthetic organism is unconditionally attracted to some stimulus such as light. When the organism moves toward the light, it is punished with a shock. After a number of light-shock pairings, the organism should stop moving toward the light. If a group is included that receives unpaired light and shock (these animals should continue to move toward the light) and a statistical difference is revealed between the two groups, punishment is demonstrated. Motivation can easily be studied in this paradigm by just varying the light intensity. A pilot experiment should first be run to determine the speed in which the designer organism is attracted to light of various intensities (or some other stimulus the organism is attracted to). Motivation can then be studied as a function of the light intensity.

The most common example of avoidance conditioning is known as signaled avoidance. In the signaled avoidance paradigm, a cue or CS is presented and if the organism responds to the signal the aversive event is omitted or postponed. It is important to note that until the organism begins to respond to the signal, the first few trials resemble the classical conditioning paradigm (the CS is paired with the shock). However, if a response is made to the CS the response avoids or postpones the aversive event. The avoidance paradigm is an interesting blend of classical and instrumental conditioning. It resembles classical conditioning in that a CS and US are presented, yet it contains an instrumental component in that a response to the CS avoids or postpones the US.

It often goes unrecognized that until the organism responds to the CS, the avoidance paradigm is a straightforward application of classical conditioning. General activity conditioning is one of the simplest of the classical conditioning paradigms. In this paradigm a stimulus (CS) is paired with a brief electrical shock (US). The amount of general reactivity to the CS is measured on each trial. If conditioning is evident, the organism will increase its reactivity to the CS until some asymptote is reached compared to unpaired or discrimination control groups. To modify this paradigm for avoidance learning all that is needed is a contingency where, if the organism responds to the CS, the aversive event avoided or postponed.

This section provides an overview of some factors that must be considered in the design of an initial demonstration of Pavlovian and operant/instrumental experiments. Additional details on experimental design can be found in [59,71,77]. A low cost automated system for programming conditioning experiments is available [78,79]. As apparatus for many species have not yet been developed, we recommend using 3d printing [80]. We also recommend object orientated modeling (OOM) for analyzing the data [81,82]. OOM has the advantage that no statistical assumptions are necessary and P values are not used.

Pavlovian conditioning (Forward conditioning) Select conditioning protocol [83]Find a CS that is neutral. If none can be found use an alpha conditioning procedure (US-US conditioning)Find a US that provides a consistent and reliable unconditioned response.Choose CS and US intensity, intertrial interval (ITI) and interstimulus interval (ISI). The ITI (time from the end of the US to the beginning of the next CS) should be relatively long to avoid effector fatigue and sensory adaptation. In contrast the ISI (time from the end of the CS to the next US) should as short as possible.Select whether you will measure conditioning on each trial (trial by trial method) or on a select number of trials (test trial method). We recommend the trial by trial method.Select whether the experiment will end after a fixed number of trials or if some conditioning criteria is met (ex. 10 consecutive conditioned responses).Determine if an extinction procedure will be used following acquisition training. There are two methods of extinction: the most common is to omit the US (CS only), and the other is to unpair the CS and US (unpaired method). The unpaired method of extinction is seldom used. It is important to include an extinction component for two reasons. First, the extinction data will provide important information on the persistence of the CR (conditioned response), and second, it will provide data for future experiments on inhibitory conditioning.Incorporate control groups. We strongly recommend that during the initial demonstration multiple control groups be used.

These control groups are CS only (provides information whether the organism’s response to the CS increases over the course of the experiment and sets the stage for possible experiments on latent inhibition), US only (provides information that the unconditioned response remains robust and stable throughout the experiment and sets the stage for possible experiments on the US pre-exposure effect) and “blank.” The blank group serves as an activity control. In addition to the CS only, US only, and blank groups, unpaired and discrimination groups should also be included. If using a group design, the unpaired group is crucial as it contains both the CS and US and can be directly compared to the CS-US paired group.

When using the unpaired group, a decision must be made as to the order of presentations of the unpaired stimuli. We recommend sequences of “ABBA BAAB” where A is the CS and B is the US. If additional trials are needed the sequence is repeated. It is important to note that the unpaired group will have twice as many trials as the paired group because, in contrast to paired animals, the CS and US are presented individually on each trial. For example, on trial 1 the CS is presented and on trial 2, the US is presented (CS, US, US, CS, US, CS, CS, US).

When designing the unpaired control group, a decision must be made as to the time between the CS and US. We recommend that the time between stimuli be half the ITI of the paired group. This will ensure that the time between CS presentations closely approximates the ITI used for the paired group. For example, if the ITI is 10 minutes for the paired animals, it will be 5 minutes for the unpaired animals. An additional rationale for using an unpaired control group is that, if this group subsequently receives paired training, conditioned inhibition can be studied. While not typically used during an initial demonstration of Pavlovian conditioning, we recommend using a discrimination group. In this case two CSs are used. The CS paired with a US is called CS+ and a CS not paired with a US is called the CS-. The ABBA BAAB pseudorandom sequence can also be used to present the CS+ and CS- (CS+, CS-, CS-, CS+, CS-, CS+, CS+, CS-). Once again, the ITI should be half that used for the paired animals. The discrimination group provides all of the advantages offered by a single subject design (i.e., each subject serves as its own control). Moreover, if the CS+ and CS- are reversed after initial training (the CS+ now becomes CS- and CS+ now becomes CS-) reversal learning can be studied. Differences between paired and unpaired groups and between CS+ and CS- provide the strongest evidence of conditioning.

After the initial demonstration of classical conditioning as recommended here, the researcher can then explore other conditioning protocols. Some of the more interesting include:

a. Trace conditioning. In trace conditioning the CS terminates before the onset of the US. The gap between the end of the CS and onset of the US is called the trace interval. This procedure can be used to study memory. The non-overlap procedure can be considered as trace conditioning with the time between the end of the CS and onset of the US as “0”.

b. Temporal conditioning. In temporal conditioning there is no explicit CS. Rather, the US is presented at regular intervals and the presence of a conditioned response is noted as the presentation of the US approaches. This procedure is useful to determine whether the organism shows timing behavior.

(B) Instrumental/operant conditioning Determine if you are studying instrumental or operant conditioning. If the response is arbitrary, standard control groups are not necessary as no animal will make the required response without specific training. In effect, an animal that can make an arbitrary response is serving as its own control.Select reinforcers and discriminative stimuli. If no positive reinforcers can be found, escape conditioning can be substituted. In escape conditioning, the reinforcer is time away from the aversive event. The discriminative stimuli (Sd) will serve as a cue to set the occasion for the response. The use of discriminative stimuli is not required during an initial demonstration but if responding can be restricted to the presence of the Sd, this provides strong evidence of conditioning.Select the training apparatus. There are several choices for apparatus including shuttle boxes, runways, mazes, and operant chambers. Many of these apparatus can be 3d-printed.Incorporate control groups. The control groups will be like those used in Pavlovian conditioning assuming that an arbitrary response cannot be found. We would recommend: 1) response only (no consequence) and 2) reinforcement only. In addition, there must be a control group that receives an unpaired presentation of response and reinforcer.

When determining whether a novel organism exhibits associative learning, it is best to employ an unpaired control group. For example, if a researcher is interested in demonstrating classical conditioning (also known as Pavlovian conditioning), the control group should receive unpaired presentations of the conditioned (CS) and unconditioned stimulus (US) (Figure 5). The pairing of the CS and US is known as a training trial and the time between the training trials is known as the intertrial interval (ITI). When using the unpaired control group, the ITI should be half of that used for a paired training group.

The rationale for using an ITI that is half that used in the paired group is to keep the time between CS presentations roughly equal. If the control group uses an ITI that is the same as that used for the paired group, the control group will have an ITI that is twice as long as that used for the paired group. This is because the paired group received both the CS and US on each trial (i.e., they are paired). In contrast, only a CS or a US is presented during a trial (i.e., they are unpaired). For example, if a researcher has 10 training trials for the paired group, there will be 20 trials in the unpaired group (10 CS trials and 10 US trials).

The use of the unpaired control is also necessary to determine whether alpha conditioning is learned. Alpha conditioning is an example of conditioned (i.e., learned) sensitization and is seldom studied in the contemporary learning literature. We feel this is unfortunate because when working with novel organisms, the researcher may find that the organism does not respond to neutral stimuli. Classical conditioning involves pairing a previously neutral stimulus with a response producing stimulus [55]. In alpha conditioning, the association is between two non-neutral stimuli. In this case, “non-neutral stimuli” refers to stimuli that elicit a response resembling a conditioned response (i.e., a learned response) without the benefit of training. Consider the case where two different light intensities are paired and both intensities elicit eye closure albeit one elicits a slight twitch and the other a full closure. The first presentation of the low intensity stimulus (US1) elicits a response resembling the conditioned response (CR) without the benefit of training. Following the presentation of the low intensity light, a high intensity light is presented (US2). This high intensity light should elicit a vigorous response. After a number of US1-US2 pairings, the organism elicits a vigorous response to US1. Whether this is learned is demonstrated with the unpaired control group.

The logic of the unpaired control group is also used in instrumental runway experiments. A reinforcement is placed at the end of the runway and over successive runs, the organism speeds up to some asymptote. Without an unpaired control group that receives reinforcement in different parts of the runway, such as in a “goal box” on some trials and in an alley on others, it cannot be unequivocally concluded that reinforcement in the goal box produces the increased running speed. The increase in running speed can easily be interpreted as simple escape behavior. However, the increase in running speed would be an example of learning if the running speed of the unpaired control group is significantly less than the experimental group.

The ability to construct an endless variety of novel “model systems” provides a rich opportunity to improve the state of the art in behavioral science.

The confusion related to the various definitions of conditioning is problematic for a more inclusive science of behavior. This confusion is becoming more pronounced as behavioral neuroscientists interested in learning begin to enter the field. As the current zeitgeist is to interpret all behavior in terms of cognitive processes, they are not exposed to the alternatives provided by behaviorist interpretations [49]. We believe that one way to solve this problem is to do away with behaviorist and cognitive interpretations and focus on the description of paradigms used to generate the behavior in which the researcher is interested (i.e., behavioral taxonomies). As Bitterman noted almost 50 years ago, “Classification is not merely a matter of taste” [84] (pg. 81). We believe that many of the problems associated with definitions of conditioning phenomena can be avoided if researcher link a conditioning procedure to a behavioral taxonomy. Several such have been proposed [83–86] but none are in consensus use. The addition of novel, engineered life forms to this field is sure to trigger additional discussion aimed at defining truly inclusive and general taxonomic frameworks for wide-ranging types of behavior.

The “operant conditioning journal” known as the Journal of the Experimental Analysis of Behavior historically contains many examples of the individual performance of various vertebrates including monkeys, pigeons and rats, which can be used as inspiration for designing novel experimental designs. The book Schedules of Reinforcement [87] contains hundreds of examples of individual performance of pigeons in the form of cumulative records (a curve showing the number of responses across time). This book is also unique in that not only is the focus on individual performance but the book describes many types of schedules of reinforcement that go far beyond the simple schedules of reinforcement described in this paper. For example, in a conjunctive schedule, a reinforcement is delivered after the requirements of both a ratio and interval schedule of reinforcement are satisfied. Schedules of Reinforcement is an excellent source of ideas and protocols for the study of operant conditioning in synthetic organisms and provides the researcher with many graphic examples of the importance of including examples of individual data.

In contrast to vertebrate studies of learning – especially in operant conditioning – there are few reports containing individual data in invertebrate models. Many invertebrate learning experiments present data in the form of group curves. Such aggregate data provides little information regarding an individual’s performance or variation among animals. Moreover, in some cases the data of individual animals that appear to be outliers are discarded. We view this as inappropriate because much information about functional heterogeneity, the role of noise, and the relationship between genetics and behavior can be obtained from individuals considered outliers.

With some exceptions there are few examples of individual data for the Limax, Aplysia, Apis, and Hermissenda learning models [60]. Examples of individual data are available for the classical conditioning of proboscis conditioning in honey bees and operant conditioning in both crabs and honey bees [71,88,89]. We recommend that researchers studying the learning of novel organisms publish examples of individual data and upload all of the data as a Supplement to enable others to mine it in new ways.

Table 1 is a bit misleading in that it presents the various conditioning paradigms as unequivocally distinct examples. In fact, there are no consistent, universally agreed-upon definitions of conditioning phenomena. The problems associated with inconsistent definitions have stimulated debate related to, for example, the definitions of species, sexual selection, eusociality, and tool use [90].

Surprisingly, among contemporary behavioral scientists, with few exceptions, there has been little debate or recognition of the inconsistent definitions of conditioning procedures. One of the most egregious examples is classical conditioning. Researchers working with novel organisms must understand that contemporary accounts of classical conditioning often fail to mention that there are at least four different methods to produce classical conditioning. These methods can be distinguished based on the degree of experimental control and the relationship between the conditioned and unconditioned response [83]. There is no research directly comparing these procedures and it is doubtful that these four methods all produce the same behavioral phenomena – i.e., classical conditioning. The conditioning methods range from general activity conditioning, suppression of ongoing behavior, autoshaping (learning an operant response through classical conditioning), and situations where the unconditioned stimulus is directly injected into the organism.

Another definitional issue concerns operant conditioning. For many researchers, operant conditioning is any “behavior controlled by its consequences.” Seldom discussed, the hallmark of operant conditioning is whether the organism can not only demonstrate the use of a manipulandum such as a lever, but that it can also use that manipulandum in novel ways. Novel behaviors can be demonstrated by training the organism to manipulate a device with different degrees of force, moving the device up or down or from side to side. Such behavior is easily observed in most vertebrates but has never been demonstrated in an invertebrate.

Examples of “operant” conditioning in invertebrates include using punishment to manipulate eye position in a crab, negotiating a maze, running down an alley, and running against a preference [59]. It is often forgotten that the rationale for using a lever or some other manipulandum is that species-typical behavior is minimized. If a manipulandum cannot be used, there is set of runway procedures that are analogous to the operant methods [64]. For example, if a researcher is interested in studying a situation where a reinforcement becomes available after a specified number of responses since the previous reinforcement (known as a Fixed Ratio schedule of reinforcement), all that needs to be done is to present the reinforcement after the required number of trips through the runway. Even here, care must be taken to ensure that what is being manipulated is operant behavior rather than a simple instrumental response.

While it has been repeatedly demonstrated that an invertebrate can increase its running speed to some asymptote in the pursuit of food [59], it has not been shown that an invertebrate can adjust its speed to procure the food. It is this adjustment in response to the contingencies of reinforcement which transforms the instrumental behavior of running down the alley into an operant behavior.

Consider the head turning response in Aplysia (sea slug). If turning its head to the right is punished, the Aplysia will quickly learn to keep its head to the left. To truly make it an example of operant behavior, what would be needed is a demonstration that the Aplysia can vary some aspect of its head turning response such as speed or duration. In situations such as these we refer to the behavior as instrumental conditioning.