Learning in Transcriptional Network Models: Computational Discovery of Pathway-Level Memory and Effective Interventions

Trainability, in any substrate, refers to the ability to change future behavior based on past experiences. An understanding of such capacity within biological cells and tissues would enable a particularly powerful set of methods for prediction and control of their behavior through specific patterns of stimuli. This top-down mode of control (as an alternative to bottom-up modification of hardware) has been extensively exploited by computer science and the behavioral sciences; in biology however, it is usually reserved for organism-level behavior in animals with brains, such as training animals towards a desired response. Exciting work in the field of basal cognition has begun to reveal degrees and forms of unconventional memory in non-neural tissues and even in subcellular biochemical dynamics. Here, we characterize biological gene regulatory circuit models and protein pathways and find them capable of several different kinds of memory. We extend prior results on learning in binary transcriptional networks to continuous models and identify specific interventions (regimes of stimulation, as opposed to network rewiring) that abolish undesirable network behavior such as drug pharmacoresistance and drug sensitization. We also explore the stability of created memories by assessing their long-term behavior and find that most memories do not decay over long time periods. Additionally, we find that the memory properties are quite robust to noise; surprisingly, in many cases noise actually increases memory potential. We examine various network properties associated with these behaviors and find that no one network property is indicative of memory. Random networks do not show similar memory behavior as models of biological processes, indicating that generic network dynamics are not solely responsible for trainability.

Rational control of dynamic pathway function using stimuli derived from computational models opens the door to empirical studies of proto-cognitive capacities in unconventional embodiments and suggests numerous possible applications in biomedicine, where behavior shaping of pathway responses stand as a potential alternative to gene therapy.

Keywords: biological network, training, memory, association, pharmacoresistance, sensitization

Brains enable remarkable behavioral capacities and are required for the complex cognitive abilities and first-person experience that evolved human and other animals possess. However, it is critical to remember that each of us has taken the journey across the Cartesian Cut [1]—smoothly and slowly transforming from a bag of biochemical reactions (a quiescent oocyte) into an adult form, capable of rational thought, metacognition, and a sense of selfhood as distinct from “insentient objects”. To the extent that “ontogeny recapitulates phylogeny” [2], taking evolution and developmental biology seriously means seeking to understand primitive forms of cognition as a spectrum that could extend to unconventional substrates besides mature complex brains. Recent progress in the field of basal cognition seeks phylogenetic antecedents to memory and other cognitive functions and asks what minimal dynamics are sufficient for implementing simple functional building blocks that underlie more advanced minds [3,4,5,6,7]. A key aspect of this research axis, related to not only evolutionary biology but also synthetic bioengineering, exobiology, and robotics/AI, is the development of frameworks for recognizing proto-cognitive activity in unfamiliar embodiments [8,9].

Morgan’s Canon [10] urges us to err on the side of mechanistic, over cognitive, models of biological systems. However, in the case of systems that are capable of learning, having preferences, or even more advanced functions, treating them as simple machines that can only be modified at the hardware level is often a sub-optimal strategy. Humans have discovered, by training animals for millennia (despite knowing no neuroscience), that effective prediction and control in some systems can be achieved by taking advantage of the system’s competencies, such as learning. This allows manipulation through an interface different than hardware rewiring: that of stimuli. Recently, we proposed an explicit framework for the empirical investigation of systems (of evolutionary or engineered origin) based on what level of cognitive model avails the engineer the most efficient rational control over the system’s function [11]. We proposed that the same efficiency and causal power gains that result from training animals in such a way, rather than micromanaging their neural states, can be reaped in biomedicine [12,13,14]. Moreover, we suggested that behavior takes place not only in 3D space by canonical muscle motion, but also in metabolic, transcriptional, and anatomical spaces [15]. Thus, it may be possible to achieve high levels of control in manipulating health and disease by behavior-shaping the dynamic navigation of living systems within any of these spaces. These strategies offer a key to the inverse problem (i.e., what low-level rules to alter to achieve a system-level change [16]) plaguing genome-editing and molecular medicine. They may also enable the development of strategies that take synthetic biology beyond simple circuits and into complex bioengineering [17,18,19,20,21].

How many of the tools of behavior science [22] can be brought to bear on cells, tissues, and other body components that are not brains, is an empirical question.

Protein pathways and gene regulatory networks (GRNs), as well as many other chemical pathways, are key drivers of embryogenesis [23], cell behavior, and complex physiology [24,25]. Understanding the dynamics of these pathways is important not only for the study of evolutionary developmental biology [26,27,28,29,30], but also for the prediction and management of numerous disease states including cancer [31,32,33]. Thus, effort has gone into computational inference of both protein pathways and GRN models [34,35,36], and the development of algorithms for predicting their behavior [37]. In general, these pathways are conventionally treated as a simple machine with most strategies focusing around rewiring their structure to achieve a desired outcome. While dynamical systems approaches have made great strides in understanding how these systems settle on specific stable states [38,39], the prospects for designing interventions have lagged: it is often unclear how to edit the structure of the pathway to result in the appropriate dynamic behavior (or in the case of developmental GRNs, the desired anatomy). Indeed, some approaches have aimed at control of these networks at the structural level [40], others, using Boolean network models, have shown control measures based on perturbations of specific nodes identified through network measures [41,42]. While this prior work examined changes to the physical structure of networks as learning, here we specifically focus on dynamical systems learning that requires no alteration to the properties of the network components [43]—a desirable property from the therapeutic perspective. Likewise, there exist formalisms that address the control of certain biological networks towards larger scale phenotypic goals, such as cell fates [44,45,46,47].

However, frameworks are still needed that facilitate the discovery of interventions for altering system-level dynamics (such as those provided efficiently by training regimens in behavioral science).

An alternative hypothesis concerns the utility of applying a computational lens to GRNs and other biological networks [48], seeing them as agents which convert activation levels of certain genes (inputs) to those of effector genes (outputs), with layers of other nodes between them. A view of pathways as an information-processing device of unknown computational capacity, as opposed to a mechanical clockwork, would therefore suggest strategies to control network behavior via inputs. Under this view, spatiotemporally regulated patterns of stimuli could remodel the landscape of attractors corresponding to a system’s “memory”. Many systems, from molecular networks [43,49] to physiological networks in somatic organs [50,51] exhibit plasticity and history-based remodeling of stable dynamical states. Consistent with the basal cognition paradigm, we define memory in functionalist terms—a system in which future behavior is specifically modulated by past experience. This definition holds regardless of the system’s physical implementation and is not limited to canonical views of behavioral memory, such as changes at the epigenetic and protein levels [52,53,54,55], or the requirement of synapses and neurons. It is an engineering perspective, in which memory in diverse media is a broader phenomenon and is not restricted to its common manifestations. Such a substrate-agnostic view enables the question: could trainability be found in molecular networks?

Several prior studies have tested specific memory phenomena in GRN models [56,57,58,59,60,61,62,63,64,65]. Others have addressed harnessing dynamical systems approaches to control networks through the structure of the network interactions such as feedback loops [66]. However, it is still largely unclear what kind of memories various pathways can possess, and whether it is possible to use this knowledge to drive the discovery of interventions. In past work [67], we formalized the notion of training a chemical pathway via pulsed input stimuli (node activation or suppression drugs), and defined a taxonomy of memory types appropriate to the GRN system. We demonstrated how paradigms from behavior science can be naturally applied to cell-biological contexts (Figure 1A). Specifically, we developed an algorithm that does not rewire an existing network but interrogates the natural system to discover the optimal perspective from which it can be controlled with existing ‘training tools’, such as Pavlovian conditioning. This, for example, was achieved by computationally identifying which transcriptional loci can serve as efficient neutral stimulus (NS) and unconditioned stimulus (UCS) nodes in the GRN network, such that the desired response (R) can be seen in a chosen gene’s expression level after training regimes such as Pavlovian conditioning. We therefore can investigate any network of size N by testing every permutation of NS, UCS, and R (Figure 1B) and evaluating memory of each possible permutation (Figure 1C).

Here, we sought the following extensions of prior work. First, we explore continuous (ODE) models to show that the phenomena we found were not due to a Boolean nature of previously explored models. Second, we analyze and compare continuous models of not only transcriptional networks, but both protein signaling pathways and GRNs, with randomized models to show that evolved systems possess different learning properties than those that can be obtained by chance using similar network properties [68,69]. Third, we test the long-term stability of memories by assessing their behavior over long time scales, to determine whether memory declines over time. Fourth, we characterize their robustness to stochastic noise, and in general to different strengths of input stimuli. Finally, having found examples of behavior such as habituation (decreased response to repeated stimulation) and sensitization (increased response to repeated stimulation), graded properties that are not detectable in binary or Boolean networks, we demonstrate the existence of specific stimuli regimes that abolish these memories. These results provide a possible strategy to combat pharmacoresistance and other prevalent problems limiting use of pharmacological agents in biomedicine [70,71,72,73]. Importantly however, the memory properties of these networks are not meant to be only a road toward therapeutics—they are expected to be an important part of their function in normal scenarios, making them interesting to both evolutionary and developmental biology. Overall, we show that a computational perspective on a minimal model of memory and learning can contribute toward a roadmap for the development of biomedical interventions that do not require genomic editing, gene therapy, nor any structural modification of the network itself.

Biological pathways represent mutual interactions among molecular entities such as genes, proteins, and metabolites within a given cellular context and link to potentially many different biological processes. Here, we survey 35 published biological pathway networks from BioModels (see Table S1) [74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96], a mathematical model repository [97,98,99,100,101,102,103,104]. Each model is comprised of genes, proteins, or metabolites which are represented as nodes and the mutual chemical reactions are defined by edges. The dynamics of the chemical reactions themselves are comprised within an ordinary differential equation (ODE) that gives the rate of change of the expression level of a node over time. Depending on the given initial expression level and current time instance, the expression level of a node is evaluated by solving these ODEs. To model an external input to a given node, we can mathematically alter the expression of the biomolecule, allowing its use as a stimulus (conceptually similar to applying sensory input to an animal for training), applying either stimulation (increased expression level) or inhibition (decreased expression level). Then, considering a separate node’s expression level as a sensory response, we analyze whether it was up- or downregulated by the applied stimulus. We built a memory ‘test suite’ to run across all biological pathway models to determine whether, and which, memory phenotypes (transfer memory, consolidation memory, etc.) are found [65]. In short, for each memory model three nodes are selected to be an unconditional stimulus (UCS), a potentially conditioned stimulus (CS), and a response (R). For each of the 35 models, a set of potential memory models is built out of all possible combinations of the n-node model.

For example, a three-node network has six potential memory models. We then tested each potential memory network for each surveyed biological pathway, evaluating the different combinations of nodes in different roles and recorded the types of memories found in the dynamics [67] (Figure 1, see Section 4 for more details).

We define the existence of memory in an ODE modeled biological network as altered, persistent future expression of a given biomolecule in response to the stimulation of another biomolecule in the network. Our definition of memory in ODE-modeled biological networks resembles memory defined in the study of Boolean-modeled GRNs with a few differences [67] (see Section 3). Most crucially, binary network investigation considers ‘states’ of the node as binary variables, either off or on. While ODE models have theoretically infinite states, as they are continuous equations, we define them as having three potential modes: upregulated, downregulated, and non-regulated. Importantly, these modes are not discrete (i.e., −1, 0, or 1), but can have various degrees of regulation. Overall, this allows for more robust characterization and leads to four key relationships between nodes that are associated with our definition of memory: upregulation of a stimulus (ST) can upregulate a response (R), upregulation of ST can downregulate R, downregulation of ST can upregulate R, and downregulation of ST can downregulate R (Figure 2). We consider two types of stimuli: (1) unconditioned stimulus (UCS), which unconditionally regulates R, and (2) a neutral stimulus, which does not regulate R but can be converted to a conditioned stimulus (CS) upon proper training. It is important to note that we refer to this third stimulus always as CS for clarity, as a neutral stimulus has no effect on memory.

We evaluate five kinds of ODE network memory: UCS-based memory [65], pairing memory (PM), transfer memory [74], associative memory (AM), and consolidation memory (CM) (see Figure 2 for a visual description and Methods for explicit description). A UCS memory is when a strong training stimulus is delivered, causing a long-term stable response of R. While this is the simplest memory we evaluated, its usefulness from a bioengineering standpoint may be invaluable as a simple latch. A paired memory is similar to the UCS memory, but in this case the UCS and CS are stimulated simultaneously to see if a long-term stable response is caused in R. This is different from UCS memory, as this memory only needs one node to be stimulated, whereas a paired memory requires both. Paired memory can be viewed as a ‘fail-safe’ system or AND gate; a response is only generated when two nodes are stimulated, and therefore R is partially protected from noise (i.e., if one node is noisily regulated). Transfer memory occurs when the original UCS stimulation drives the network dynamics into a slightly different configuration, allowing a previously non-effecting node to now effect the response node. An associative memory is similar to the classical Pavlovian conditioning model wherein a previously non-effective stimulus is delivered alongside a stimulus which causes a response in R. One striking possibility for such a memory phenotype would be to associate a desired response from a potentially dangerous stimulus (as in a drug with a hazardous side effect) with a more accessible or less hazardous stimulus that previously did not affect the circuit. Finally, consolidation memory is the same paradigm as in associative memory, but there is a consolidation period before the CS creates a response on R.

All of these types of memories have implications for how cells bearing such GRNs would behave in healthy physiology and disease states (in which case they can be exploited by therapeutic strategies). These provide a context-sensitive response to internal and external (environmental) stimuli that are sure to have impacts on evolution and systemic function.

Overall, we found memory models in all surveyed biological networks except four models, of which three had a low number of nodes (<4) suggesting that too small networks either cannot, or are very unlikely to, have memory (Networks 3, 14, 17 and 33, and Table S2). Additionally, we tested to see if certain network properties, such as mean number of nodes, edges, degree, and hubness were associated with more or less memory and found there were no significant correlations with any of these measures and any memory (see Discussion).

Before testing for memory, a pretest is performed to evaluate if the chosen UCS node can indeed cause a response on the R node while the chosen CS node elicits no response on the R node (Figure 2). The remaining circuits are then tested for the five kinds of memory as well as no memory. Each kind of memory is associated with a specific paradigm of training, i.e., sequences of stimulation which results in stable, long-lasting, predictable, and specific changes in network behavior that map nicely onto concepts of training in behavioral science. Stimulation can be used to train these networks, i.e., to shape their future behavior, just as is achieved in traditional biological systems. Visual examples of the five memories, their training, and tests can be seen in Figure 2 and details found within Methods. A ‘no memory’ circuit is defined as a circuit that has a UCS-R relationship, but no memory component (i.e., when UCS stimulation is removed, R reverts its expression to pre-stimulus expression levels). We show the evaluation process with an example three-node biological network [75] with two out of six potential memory circuits passing the pretest. We demonstrate that this model has UCS-based memory in one case and no memory in the other (Figure 3). To quantify this, the number of circuits showing a particular memory type are summed and divided by the number of potential memory circuits, giving the lower bound of 0% (no circuits with a given type of memory) and an upper bound of 100% (every circuit demonstrating that type of memory). Therefore, the example network shows a 50% memory capacity (one out of two networks) and a 50% no memory capacity (one out of two networks).

The idea of self-sustaining feedback loops in biochemical reaction networks and their relationship with hysteresis may be one explanation for simple memories [105]. Figure 3 shows the dual phosphorylation-dephosphorylation cycle of MAPK, which has been characterized as having positive feedback loop architecture [75]. This does not diminish, however, the ideas that memories have been formed through stimulation of key nodes in this feedback loop. For example, while simple, a traditional latch circuit, common in electrical engineering, works in a similar fashion but still can be thought of as a memory. Overall, we suggest that feedback loop network architecture may be conducive to memories but may be only one way of many to support them. In a different biological network [75], we show an example of AM, wherein a previously neutral stimulus is converted to a CS through training (Figure 4). Again, while feedback loop architecture is present, our results indicate that memories can be formed through dynamical interactions alone, without changing network structure/architecture. Additional examples of other memory types are presented in Figures S4–S7.

We also sought to examine the effects of varying stimulation strengths during the training period, to determine whether therapeutic triggers would have to be used in a very narrow range of concentrations. We examined the persistence of memories given smaller stimulations, namely 10X and 50X of baseline, whereas normal training was completed with 100X (see S8A). While most models (52%) kept the same number of memories, 10% of models lost memories, with only one losing more than 50% of the memories found at 100X. Interestingly, four models demonstrated a 50% increase in memory with weaker simulation, which suggests that some models are more sensitive to strong stimulations (i.e., too strong and no memory is formed). We conclude that the memory properties are robust to a wide range of stimulus strengths, a desirable property for therapeutics, and also for functional linkages with other cell pathways during evolution.

To determine the sensitivity of the results to noise, which represents a more realistic view of how biology functions [106], we re-ran the analyses while adding stochasticity throughout the simulation (see S8B). Interestingly, while almost all models retained the memories created (83%), there was an overall 20% increase in memory in the presence of noise (averaged across models and memory phenotypes) most notably in the associative memory paradigm. We conclude that biological noise should not inhibit the trainability of most networks, and indeed actually seems to be helpful, a phenomenon familiar in many biological systems, including signaling pathways [107,108].

We examined the stability of memories of long periods of time, relative to the model, after training to determine how quickly memories might decay after experience. We investigate the long-term presence of UCS memory in two ways: (1) by testing R nodes after 10X the initial waiting period and then again after 100X times the initial waiting period, and (2) by testing the R nodes after 100X times the initial waiting period. This paradigm investigates both the stability of memories (2), as well as whether testing can potentially destroy the formed memory (1). Out of 30 networks showing UCS memory, a near majority did not lose significant memory capacity (defined as <50% loss) with any delay. Twelve networks did lose more than 50% of memories over long timeframes, including 1, 21, and 25, each of which experienced loss of 73%, 60%, and 53%, respectively. Of those losses, most occurred between 1 and 10X timepoints but were constant after that; only eight networks lost all memories across both 10X and 100X tests (S9). Interestingly, in two cases the memories increased after testing at 10X, suggesting that memory recall could potentially strengthen certain dynamics, but we emphasize that this effect was rare. Overall, we conclude that while some networks have a limit on how long they retain training, the majority of networks can retain the bulk of their possible memories over very long time periods and that in general, memory ‘access’ does not destroy the memory but can, in some rare cases, increase memory.

The memory capacity of neural networks, such as Hopfield networks [109], have been thoroughly examined [110] and reveal that even small networks could have capacities larger than one. However, in prior work it was not known whether GRNs or protein pathways could be trained in a way that did not wipe out the effects of prior experience. Could a single system, or network, keep more than one memory?

Thus, we tested all networks to determine whether a second memory could be added in addition to the first. Two paradigms of adding a secondary memory were used: (1) memories were added serially by giving training stimulations one after another allowing a short relaxation period and then giving a second training stimulation, and (2) memories were added in parallel by giving two training stimulations simultaneously and then tested. It is important to note that each memory requires a separate UCS-CS-R permutation, and therefore the functional roles of some nodes may be different between the two memories, may be the same, or may be partially overlapping. It however is never the case that the response node of one permutation is the UCS or CS of another. Interestingly, we find that both biological and random networks were able to add an additional UCS to a UCS memory in both cases of serial or parallel memories (Figure 5 and Figure 6). This may be due to the ‘simple’ nature of UCS memory, a stimulus creating a longer-term response. When more complex memories were tested, here AM-AM memory pairs, when created serially all biological and random models showed catastrophic forgetting wherein both are lost. When created in parallel, two biological networks retained both memories in some of their potential memory models while roughly half of random networks retained theirs. The case for UCS-AM memory pairs was similar, in that when created serially, both biological and random networks either forgot one or both, and when created in parallel random networks showed a stronger remembering of both memories (see Section 4).

While it may be surprising that random networks are able to better form paired memories, especially in parallel, it may be that biological models have emerged to use certain dynamics towards a typical biological function and therefore may not have the capacity for additional memories such as we propose.

Both biological and random networks were tested for the capability of forming two memories, serially, or one after the other after a brief period of relaxation. We tested UM-UM memories, AM-AM memories, and UM-AM across all models. Above are bar graphs representing the percentage of potential memory models that either retain both memories, forget the first, or forget both. To test this, we followed a similar procedure as in memory testing. However, in this case once a memory was formed, another was formed following the same procedure (see Section 4). Note there was no pretest for the second memory, as this potential memory circuit had already passed.

Both biological and random networks were tested for the capability of forming two memories in parallel, or at the same time. We tested UM–UM memories, AM–AM memories, and UM–AM across all models. Above are bar graphs representing the percentage of potential memory models that either retain both memories, forget the first, or forget both. To test this, we followed a similar procedure as in memory testing. However, in this case two memories were attempted to be formed at the same time (see Section 4).

Finally, we attempted categorizing memory in the surveyed networks on the basis of species in the evolutionary strata. We considered eight categories, namely, bacteria, slime mold, yeast, plant, lower vertebrates, mammals, and general. Here, general corresponds to the networks not associated with any species but related to generalized biological processes. We found no privileged categorization of memory over classes of organisms—memory seemed distributed uniformly over all strata of life (see Figure S3). This may suggest that memory capabilities are not unique to a particular stratum, reinforcing the hypothesis that proto-cognitive capacity may be found across a variety of processes.

We sought to test whether trainability in these networks is a mathematically inherent feature of network dynamics or if biological networks are unique in the space of possible networks with respect to the memories they exhibit. To this end, we created 500 random synthetic biology networks for comparison (see Section 4). Each network created has a network size of 10, equal to both the mean and median size of the surveyed biological networks. We also controlled for other network measures when creating the random models, namely, number of nodes, number of edges, indegree, outdegree, betweenness, PageRank, and hubness, such that the values of these measures fell within a similar distribution of the biomodels. A comparison of 35 randomly selected random models to 35 biomodels used can be found in Figure S10, showing that they represent statistically very similar kinds of networks to the biological ones. We tested three aspects of the memory profiles across both biological and random networks to search for potential unique properties among biological networks that resulted from selection in the biosphere.

First, we checked the prevalence of each type of memory in biological vs. random networks, as it may be that certain memories are more inherent to network dynamics themselves, rather than something developed through evolution and development. To do this, we computed the average occurrence for a memory type in both biological networks and the random networks. We observed for most memory types, there is an increase in the prevalence of memory in biological networks over random networks. Interestingly, the average number of no memory cases (no memory or trivial S-R relationships) is much larger in random networks, providing an important null baseline. Second, we examined if any type of memory is indicative of other memory types, i.e., if memories are correlated. We calculated the correlation matrix across all types of memories in biological networks as well as random networks and observed that memories in biological networks are more correlated to each other as compared to random networks. In addition, we computed an asymmetry measure between lower and upper half of the correlation matrices of biological and random networks to examine if there are non-trivial relations between memory types. We found that the matrix corresponding to biological network has a significantly higher of asymmetry measure (Asym = 5.73) compared to that of the random models (Asym = 0.91). The third aspect tested was the robustness of each type of memory obtained in both types of networks. For robustness, we first considered the initial memory capacity found in a biological network as maximal capacity. We then altered the connectivity of the network by both randomly deleting and randomly adding an edge and measure the memory capacity as compared to the maximal capacity. This altered network was again altered, and capacity measured.