Behaviorist approaches to investigating memory and learning: A primer for synthetic biology and bioengineering

The fields of developmental biology, biomedicine, and artificial life are being revolutionized by advances in synthetic morphology. The next phase of synthetic biology and bioengineering is resulting in the construction of novel organisms (biobots), which exhibit not only morphogenesis and physiology but functional behavior. It is now essential to begin to characterize the behavioral capacity of novel living constructs in terms of their ability to make decisions, form memories, learn from experience, and anticipate future stimuli. These synthetic organisms are highly diverse, and often do not resemble familiar model systems used in behavioral science. Thus, they represent an important context in which to begin to unify and standardize vocabulary and techniques across developmental biology, behavioral ecology, and neuroscience. To facilitate the study of behavior in novel living systems, we present a primer on techniques from the behaviorist tradition that can be used to probe the functions of any organism – natural, chimeric, or synthetic – regardless of the details of their construction or origin. These techniques provide a rich toolkit for advancing the fields of synthetic bioengineering, evolutionary developmental biology, basal cognition, exobiology, and robotics.

KEYWORDS: Behaviorism, synthetic morphology, biobot, memory, learning, basal cognition

One of the most salient and interesting aspects of living things is their ability to learn from experience, exhibit preferences, and adaptively solve a diverse range of problems. The dynamic behavioral capacity of life forms is a central aspect of understanding evolutionary change and of efforts to control biological processes for beneficial applications in biomedicine and technology. The repertoire of behavior science is rapidly being expanded beyond the typical workhorse organisms (rats, etc.) to include a range of unconventional systems with rich behavioral capacities, including single cells, slime molds, plants, bio-hybrid robotics, and many others [1–18]. The sensors, effectors, and internal structures of these organisms may be quite different from those of typical animals studied by neuroscientists; thus, these systems present a challenge to the conventional approaches for characterizing intelligence and learning capacity, and to conceptual frameworks formed in the context of a fixed set of evolved, brainy creatures that have been produced in the biosphere to date [19–24].

A plethora of novel biological systems are being produced by efforts in synthetic biology, artificial life, chimeric technology, and bioengineering [25–27] (Figure 1). Biobots, motile organoids, hybrots, cyborgs, chimeras, and other categories of living systems are now being made in laboratories, by combining organic cells and tissues from diverse species and incorporating inorganic components such as scaffolds, closed-loop software components, and electronic interfaces [28–30]. These organisms may contain various cell types (muscle, skin, etc.) and/or a range of smart materials and active matter [31–35], and each level of organization of such a system can be engineered, modified, or evolved independently. The demonstrated interoperability and plasticity of life gives rise to a huge option space of possible beings (Figure 2), which may be evolved, designed, or any combination thereof [32,36–41]. Remarkably, many such constructs are not merely passive tissues that implement self-assembly and physiology, but in fact exhibit various degrees of functionality such as motility, spontaneously-initiated behavior, and responsiveness to external stimuli. We refer to the whole class of possible active constructs, in whatever implementation, as “novel organisms” in our methodological discussion below.

For example, Xenobots [42,43] are self-propelled, autonomous proto-organisms made of epithelial and/or muscle cells that can navigate their environments and interact with each other in swarms, and perform actions that individuals could not do alone (Figure 3). It is imperative to begin to understand the degree and type of intelligence of such novel living beings, which are giving rise to an emerging interdisciplinary field at the intersection of cell biology, neuroscience, and engineering. How much sensing, decision-making, learning, and problem-solving do such systems exhibit? Developing a framework for empirically answering these questions, which could place any given new life form on a scale such as Rosenblueth et al.’s continuum (Figure 4) [44], is essential to progress in fields ranging from evolutionary ethology to soft robotics and machine learning.

A number of disciplines will be strongly impacted by the developments of a rigorous science of behavior not tied to familiar organisms and brain structures, and freed from contingent assumptions about the material components essential for various degrees of functional sophistication.

The field of “basal cognition” [23,45–47] seeks to understand the phylogenetic origins of behavioral complexity, and is greatly enriched by the ability to make novel living beings in arbitrary configurations (for example, varying the amount or organization of neural components, or even producing entirely aneural systems) to more broadly probe structure-function relationships. Similarly, developmental neuroscience is concerned with the earliest stages of sensing and behavior during embryonic development, which shed light on functionality that is possible prior to the development of a complex brain. Soft robotics and Artificial Intelligence will greatly benefit from understanding engineering principles, inspired by emergent properties of novel life forms, that can be used to design and implement constructs with intelligent and problem-solving behavior. The characterization of degree of learning capacity is also of interest to exobiology (putative life found outside of Earth), as it may be a key criterion by which truly alien life forms could be recognized as such. Finally, learning capacity is fundamental to the ethics of organoids and synthetic biology, in terms of determining the degree of behavioral sophistication and thus framing our relationship to novel life forms, whether evolved, discovered, or engineered.

A major roadblock to the characterization of learning capacity in novel constructs is that they often do not resemble any known model species used in neuroscience. Given a lack of precedent for existing training protocols within an astronomical option space of novel organisms, laboratories with expertise in bioengineering but not behavioral science often face a barrier for exploring the learning and behavioral capacity of new kinds of living organisms. Maximizing the positive impact of new bioengineering technologies requires a flexible, portable set of conceptual tools that focus on the essential functionality of learning, and that are not dependent on any assumptions about the structure or origin of the subject. Fortunately, there is an ideally appropriate formalism for this new field: behaviorism.

Behaviorist approaches are ideally suited to the new science of synthetic and chimeric organisms because they focus on observable functionality and are fundamentally agnostic about the internal construction of the subject, thus freeing researchers from brain-related assumptions that can constrain the study of novel creatures. Unlike the behaviorist tradition, cognitive approaches focus on inferring internal processes associated with information processing and are currently strongly associated with specific brain architectures, which many natural or bioengineered creatures will not possess [48–58]. Our goal here is not to review the large literature debating the relative merits of behaviorist vs. cognitivist traditions in neuroscience. Nor do we claim that this is the only approach to understanding novel living systems. Here, we offer another tool for the bioengineer’s toolbox, which facilitates focus on practical, functional analysis of capabilities of novel constructs.

The behaviorist approach avoids thorny philosophical issues of defining “cognition” in the context of sometimes minimal biological systems, or attempts to map their capacities onto familiar neural concepts, paradigms, and architectures developed for standard model species. We provide an overview of the conceptual and methodological tools that classical behaviorism has to offer the field of functional synthetic morphology, referring the reader to in-depth discussions of neglected aspects of invertebrate learning and the learning of plants [59–63] as precedents for even more profound extensions. We also discuss several methodological and conceptual issues that a bioengineer will face when designing learning experiments with novel organisms and provide practical strategies to help design a research program with novel organisms.

One of the most important and interesting aspects of behavior is learning; thus, we begin with a taxonomy of concepts useful in the design of experiments to see how the behavior of a given living construct changes as a function of past experiences. Learning is classified as non-associative or associative. Nonassociative learning involves changes in the response to a single type of event, such as when the repeated presentation of a light alters the probability or strength of an orientation response to that light. It is considered the most basic learning process and serves as a building block for more complex learning. The two types of nonassociative learning that have received the most analyses are habituation and sensitization, discussed below.

Associative learning is a form of behavior modification involving the association of two or more events such as between two stimuli, a stimulus and response, or a chain of responses. In associative learning, the organism does learn to do something new or better. The three types of associative learning that have received the most attention are classical, instrumental, and operant conditioning.

Table 1 shows the type of conditioning and its relationship between nonassociative and associative learning. It is arranged from the simplest (habituation and sensitization) to the most complex (operant conditioning). When considering the table, it should be noted that each of the six conditioning categories can be made more or less complex. Consider the case of habituation. Here, an organism receives a repeated presentation of some stimulus until a behavioral response is no longer elicited. However, this situation can be made more complex if the experimenter simply adds context to the situation. For example, to design a habituation assay with “context”, one could perform the experiment in a chamber that contains some background stimulus such as a specific light intensity, temperature, and/or apparatus configuration (round vs. square). These specific background stimuli represent the context. When habituation is complete under the original context, a second context can be introduced (i.e., changes in temperature, light intensity, and/or apparatus configuration) and a comparison of habituation rates made between the two contexts. If the organism in question can process contextual information, habituation will be demonstrated in one context and re-learned in another.

Not all animals show all types of learning, and it is advised to check new systems for all of them to survey its capabilities, starting with the simpler ones. In general, the more complex the organism the wider variety of learning it will exhibit. During an early survey phase, preliminary experiments can be performed without controls, to determine training parameters such as stimulus intensity and calibrate the assay. For experiments to be conclusive during the formal phase of the research process, appropriate control groups must be employed.

Several fundamental learning paradigms can be used to study nonassociative and associative learning.

To carry out a behavioral experiment, one must make a decision early in the design phase whether the experimental design will employ a single subject design or a group design. In a typical single-subject design the subject serves as its own control. The single subject design has two benefits: it uses fewer organisms, and individual differences in a sample can more easily be controlled. If a single subject design is used, the organism receives two stimuli, one of which is followed by an event such as a reinforcement or by a US. In the case of classical conditioning, a CS followed by the US is known as CS+, and a CS not followed by the US is known as CS-. In the case of instrumental and operant conditioning, the stimuli are known as Sd and S-delta (SΔ), respectively. Statistical differences between the CS+ and CS- (or Sd and SΔ) serve as evidence for learning. The strongest evidence for demonstrating learning is obtained when the experimenter can employ both group and single subject designs.

A bioengineer considering employing situations in which the behavior of the synthetic organism is manipulated by the consequences of its actions must make a decision whether to employ instrumental or operant techniques. While instrumental and operant behavior are considered “behavior controlled by its consequences” the apparatus and research strategies are different. Apparatus associated with instrumental conditioning include such well known devices as the maze, runway, and shuttle box. In contrast, operant conditioning is most often studied in some version of the “Skinner box” (also known as an operant chamber). Unlike the apparatus used in instrumental conditioning, the Skinner box requires the experimenter to first train the organism to make some manipulative response such as pressing a lever. Only after the organism is trained to make such a manipulative response can the experiment proper begin. In general, how the experimenter trains the organism to press a lever (or make some other manipulative response) is not the primary interest of the experimenter. In contrast, all that is required to use the instrumental conditioning apparatus is that the designer organism be able to move from one place to another.

In addition to differences in apparatus, another difference between instrumental and operant behavior are the strategies used to examine learning. If a researcher is interested in how the organism learns, than the instrumental apparatus should be used. The reason for this is that the instrumental apparatus beaks down the behavior into parts or discrete units. Consider the case of a runway – a maze without choice points. The runway contains a start box, alley, and goal box segments. The behavior of the organism can be analyzed in each of the three segments in terms of such dependent variables as the time needed to leave the start box, the time required to transverse the alley and time required to consume the reinforcer in the goal box. In contrast, the Skinner box is best suited to measure changes in response rates. How the organism learns to press the lever is not the main interest. The main interest is how an independent variable influences the rate of a response – the dependent variable. The independent variable can either increase response rate, decrease response rate, or leave the response rate unchanged.

A bioengineer reading a textbook on learning will generally find no distinction between instrumental and operant behavior as both are considered “behavior controlled by its consequences.” Indeed, the terms are often used interchangeably. We believe this is not correct. Although both instrumental and operant conditioning are behavior controlled by its consequences, the term operant behavior should be restricted to arbitrary behavior in which a manipulative response or skilled movement are used. On the assumption that developing a lever press or other manipulandum for a synthetic organism will be problematic, the question naturally arises whether the bioengineer can create a situation where an instrumental response is arbitrary.

An instrumental response can be shown to be arbitrary if some property of the response can be manipulated. For example, if a novel organism can be shown to increase or decrease its speed of movement as a result of the contingency of reinforcement or learn a series of correct turns in a complex maze, this would turn movement into an arbitrary behavior. The key phrase to help the bioengineer to distinguish instrumental and operant behavior is whether the organism in any given apparatus “can show you it knows how to use it.”

A good illustration of the distinction between instrumental and operant behavior is the “hunt and peck” method of typing on a keyboard. The reinforcement is typing the correct letter – an instrumental response. However, with training, the “hunt and peck” method is replaced with a series of rapid and coordinated movements – an operant response. If a synthetic organism cannot make a manipulative response, there are a series of procedures that use the runway to study operant schedules of reinforcement. For example, rather than pressing a lever, an organism might receive reinforcement on the 5th run (i.e., trial) through the runway. This is known as a “fixed ratio – 5 schedule of reinforcement [64].

How can the experimenter explore learning when one does not know in advance what the animal perceives of the outside world, or which stimuli will have salience to a creature that does not have eons of evolutionary pressure behind it for specific behaviors? Many of the synthetic living organisms have unusual sensory or effector capabilities. They may have natural or artificial (bioengineered) sensors of magnetic fields, light in unusual wavelengths, vibration, novel chemical receptors, or may be instrumentized by electrodes to respond to stimuli that exist in a virtual world (for example, like sensory substitution [65–68], but the stimuli could come from unconventional spaces such as stock market data or physiological parameters of another life form). Their behavioral output may be muscle activity, or it may be electric signals that are amplified and used to drive a vehicle, control some other animal’s habitat, or run a 3D printer to modify the environment.

Picking appropriate stimuli and testing paradigms for entirely novel organisms is largely a matter of trial and exploration. The stimuli selected during the initial pilot experiments will be based on knowledge of its component cells and tissues, but it can be very hard to extrapolate from that to system-level behavioral capacities. As the study of animal learning has a long and rich tradition [69], we strongly recommend that the researcher collaborate with a comparative psychologist at least in the initial phase of the experiments [70] as analogies might be useful to unconventional models such as various invertebrates, plants, etc.

When working with novel organisms, the researcher must be aware that the organism may be less sensitive to environmental contingencies. Such decreased sensitivity may, for example, result in an inability to associate conditioned and unconditioned stimuli in a classical conditioning situation or a response and reinforcer in an instrumental or operant situation. Such results may be deceptive. In a lever press situation developed for crabs, it was found that restraining the crab with clamps produced poor results but enclosing them in a small box produced effective lever pressing [71]. As much as possible, organisms should be allowed to interact with the world and choose which signals are salient.

A similar situation was found with the proboscis conditioning of stingless bees. Restraining stingless bees in tubes did not produce any proboscis conditioning but putting them in small bottles where they made contact with the stimuli through a screen produced rapid learning [72]. We recommend that a catalog of stimuli, responses, and training situations be created for synthetic organisms and that this catalog be shared with the scientific community.

In our view, creating such a “behavioral catalog” is possibly one of the more exciting aspects of the work with new organisms. The researcher should try a variety of stimuli such as light, magnetic field, and vibration. The stimuli initially selected will be based upon the design of the organism. Each stimulus should be tested systematically at a range of intensities similar to that used in psychophysics experiments. Moreover, we recommend that detailed behavioral records be kept describing the organism’s reactions to the stimuli. The researcher, thus furnished with an empirically based data-set complete with detailed observations, will then be sufficiently informed to design non-associative and associative learning experiments.

We recommend beginning with habituation experiments (i.e., nonassociative learning). The rationale for this is fivefold. First, only one stimulus is needed. The use of only one stimulus, repeatedly presented, reduces the complexity of the experiment. Second, nonassociative learning shares many properties of associative learning including spontaneous recovery, generalization, and stimulus intensity effects [73]. Third, the habituation experiment can guide the researcher in the selection of training variables for associative learning experiments. For example, if it takes the synthetic organism 40 trials to reach some habituation criteria to light, then the researcher knows that light can be used as an unconditioned stimulus for at least 30 trials (assuming that the criterion for habituation is ten consecutive no responses). It makes no sense to design a classical conditioning experiment until the researcher knowns how effective the unconditioned stimulus is. These data can be obtained in a habituation experiment. Fourth, habituation has been studied in both invertebrates and vertebrates for decades and there are literally hundreds of published experiments to which the results from the new organism can be compared. Fifth, the habituation paradigm can be converted to an associative learning paradigm by the addition of context. When the synthetic organism demonstrates habituation in one context, such as color, temperature, or shape of apparatus, and is then placed in a second context, is habituation maintained or does the organism need to re-learn to habituate in the new context? We discuss habituation in context below.

The other widely studied non-associative learning paradigm is sensitization. Sensitization experiments also require the presentation of only a single stimulus. The results of repeatedly presenting the stimulus is that a reaction will develop as the number of stimulus presentations is increased. Like habituation, sensitization also has advantages for the design of associative learning experiments. One principal advantage is the design of alpha conditioning experiments also known as conditioned sensitization. As discussed previously, here, instead of associating a neutral stimulus (CS) with a non-neutral stimulus (US), two US’s are associated. The two US’s can be from the same sensory modality (example, a low intensity vibration followed by a strong intensity vibration) or a different modality (example, light and vibration). The intensity of the first is lower than the intensity of the second. After a number of pairings, the first stimulus should evoke a reaction similar to the second stimulus. As long as appropriate controls are used, alpha conditioning is an example of associative learning. Alpha conditioning is especially important if a novel organism does not respond to neutral events.

Related to the question “How do you explore learning when you do not know in advance what the animal can perceive?” is the key question of how to motivate a new type of organism. A rich literature addresses the conceptual issues around preferences and valence in minimal living [74] and even non-living (e.g., AI) [75,76] systems; here, given the focus on novel biological constructs, we use a functional definition of motivation with respect to substances, states, or signals that are necessary or contrary to the system’s longevity and well-being. An organism’s motivational state is manipulated by depriving it of some commodity, using a preferred commodity, or by varying the intensity of an aversive event. If a behavioral catalog is created and shared, the answer(s) will readily be revealed. The question of motivation is especially important for instrumental and operant experiments because behavior in these two paradigms is controlled by its consequence. The consequences will be the contingent application of either appetitive stimuli (such as food) or aversive stimuli (such as electric shock).

One way to search for appetitive stimuli is to determine what the organism “covets.” This can be done in the course of constructing a behavioral catalog. For the vast majority of learning experiments, food is the appetitive stimulus of choice. The use of food has several difficulties, including satiation effects and the need to use apparatus to deliver the food. For food to be effective, the organism must be food-deprived. Even synthetic organisms will have metabolic limitations, and often can be deprived of nutritional resources. An alternative to food deprivation is to use a preferred food. One interesting challenge of working with synthetic organisms is the search for unique and novel appetitive stimuli. Until such stimuli are found, we recommend the use of aversive stimuli.