The scaling of goals from cellular to anatomical homeostasis: an evolutionary simulation, experiment and analysis
Complex living agents consist of cells, which are themselves competent sub-agents navigating physiological and metabolic spaces. Behaviour science, evolutionary developmental biology and the field of machine intelligence all seek to understand the scaling of biological cognition: what enables individual cells to integrate their activities to result in the emergence of a novel, higher-level intelligence with large-scale goals and competencies that belong to it and not to its parts? Here, we report the results of simulations based on the TAME framework, which proposes that evolution pivoted the collective intelligence of cells during morphogenesis of the body into traditional behavioural intelligence by scaling up homeostatic competencies of cells in metabolic space. In this article, we created a minimal in silico system (two-dimensional neural cellular automata) and tested the hypothesis that evolutionary dynamics are sufficient for low-level setpoints of metabolic homeostasis in individual cells to scale up to tissue-level emergent behaviour. Our system showed the evolution of the much more complex setpoints of cell collectives (tissues) that solve a problem in morphospace: the organization of a body-wide positional information axis (the classic French flag problem in developmental biology). We found that these emergent morphogenetic agents exhibit a number of predicted features, including the use of stress propagation dynamics to achieve the target morphology as well as the ability to recover from perturbation (robustness) and long-term stability (even though neither of these was directly selected for). Moreover, we observed an unexpected behaviour of sudden remodelling long after the system stabilizes. We tested this prediction in a biological system—regenerating planaria—and observed a very similar phenomenon.
We propose that this system is a first step towards a quantitative understanding of how evolution scales minimal goal-directed behaviour (homeostatic loops) into higher-level problem-solving agents in morphogenetic and other spaces.
Keywords: artificial embryogeny, active inference, artificial life, teleonomy, modelling, morphogenesis
How does the complex anatomical structure of the body form reliably from the activity of individual cells? More than emergent morphogenesis, many organisms are able to reach the same specific target morphology despite surgical injury, changes of cell size or number, organ rearrangement, genetic defects and many other insults [1]. How does the collective decision-making of cell groups enable regeneration, regulative, development, metamorphosis and cancer suppression? Answers to these questions are critical to many fields, including evolutionary developmental biology, cell biology and complexity science [2,3]. Beyond their importance for basic science, the ability to control the collective behaviour of cells toward specific anatomical shapes has massive implications for transformative biomedicine. How can we best control morphogenetic systems composed of several levels of organization from cell to tissue, to achieve desired outcomes in regenerative medicine and synthetic bioengineering [4]?
All of these questions raise the problem of understanding how the competencies of cells relate to those of the emergent tissue-level swarm, and what evolutionary dynamics enable the shift between those states to occur. No individual cell knows what a finger is or how many a salamander is supposed to have, and yet when amputated, the collective will actively build until precisely the right number, size, shape and position of the fingers is restored [5]. What is the origin of the dynamics that enable a cellular collective to work toward large-scale anatomical target states that are too complex and big for individual cells to represent?
Some frameworks, motivated by cybernetics and behaviour science, seek a deep symmetry among homeostatic behaviour across scales and material implementations [6,7]. It has been suggested [8] that evolution pivoted some of the same strategies for navigating problem spaces across metabolic, anatomical and behavioural domains. For example (reviewed in [9,10]), the same mechanisms using ion channels, electrical synapses and computations via bioelectrical networks underlie problem-solving in physiological space by bacterial communities [11,12] and plants [13,14]; navigation of morphospace by a wide range of embryonic and regenerative systems [15–17]; and classical behaviour by organisms with neurons and brains [18,19].
Traditional behaviour science focuses on the cognitive capacities of an ‘individual’—memory, perception, learning, anticipation, decision-making, goal-directed behaviour—and makes two assumptions. First, the body is usually conceived as a fixed structure, following the mainstream paradigm that the genome codes for specific bodyplans. Second, brain structure is thought of as stable—the individuality of its neuronal cells is gone for good. Our primary experience is that of a centralized, coherent self. For organisms like caterpillar, which undergo metamorphosis, with drastic changes of body, brain and behaviour, studies usually focus on their separate phases of life. The transitional states are usually not studied [20].
However, all intelligences are collective intelligences, they are made of parts that are themselves biological agents. Regenerative biology and the development of new organisms like biobots [21] reveal that studies of cognition in intact bodies present just a narrow slice of a much bigger picture: that of the multi-scale interface between body and mind. Indeed, cognition is tightly linked to the physical structure of the body, from the anatomical level to the molecular and bioelectric levels that store its information [20]. And structure and function are both highly plastic. Cognition continues to function despite important changes to the body/brain and the corresponding modification of its information at the cellular, molecular, or bioelectric level [22].
In the age of regenerative medicine, and of hybridization between evolved and designed subsystems [23], it seems essential to understand how mind and body coevolve [24], in order to explore the origin of multicellularity and the scaling of basal cognition of individual cells into larger organisms. The algorithms that allow cells to perform goal-directed decision-making and the transition to larger selves are still poorly understood [20,25]. To integrate and advance the fields of developmental/evolutionary biology, synthetic bioengineering, artificial intelligence and cognitive science, it is fundamental to begin to develop computational frameworks that allow us to ask how cognitive capacities arise in agents made up of parts, and how these emergent cognitive agents scale up during evolution.
In the scale-free cognition framework [20], we rely on ideas from control and information theory to identify multi-scale information processing principles. That theoretical framework proposed that the scaling of homeostatic behaviour in morphogenetic space is the direct result of group dynamics of homeostatic subunits in metabolic space [20]. The work we present here is a quantitative in silico embodiment of that theory, generating testable predictions. It provides, to our knowledge, the first exploration of the emergent morphogenetic consequences of a scaling that does not treat cells as simple automata following static rules, but instead as entities with information-processing homeostatic capacity. Earlier studies showed that individuals can be outperformed by collectives [26], and that a collective’s overall performance is dependent on several characteristics including the organizational or network structure [27], the information aggregation and communication system among individuals, and the diversity of members [28]. These studies focused mainly on human networks and the associated wisdom of the crowd. We focus here on morphogenetic systems.
The mechanistic and algorithmic invariance between scales of organization has enabled numerous tools and approaches from neuroscience to be ported to developmental biology [4,29,30] resulting in novel capabilities in regeneration, cancer reprogramming, and repair of birth defects. To move this field forward, and to better understand the role of evolution, it is essential to develop quantitative, generative models that reveal what dynamics are sufficient for the scaling of competencies and their generalization to new problem spaces. Thus, we sought to create a minimal in silico system in which we could observe whether, and how, evolutionary dynamics could drive the shift from cell-level metabolic goals to the much larger, body-wide goal of patterning a positional information axis. The scope of this work applies to any organism capable of axial patterning including not only metazoans but also plants, corals and sponges [31–33]. Goals are here defined from the perspective of a cybernetic approach, focused on error minimization with respect to a specified homeostatic setpoint [34,35]. Here, we present the details and analysis of a simple artificial embryogeny system (two-dimensional cellular automata with each cell’s behaviour controlled by an evolutionary artificial neural network) which we initialize with cell-level physiological homeostasis and observe the emergence of competence to reach a target morphology (solving the classic French flag problem [36]) in an entirely new problem domain: anatomical morphospace. While a number of models have been proposed for solving the French flag problem [37–40], our model uniquely shows how the capability to robustly self-organize a spatial axis at the tissue level emerges from unicellular metabolic competencies (cellular homeostasis in the absence of local instruction).
In this work, we used the French flag problem as a measure of morphogenetic competency to flesh out a specific theory of how collective intelligence scales up across problem spaces in evolution. Our data show that evolutionary forces drive the emergence of several higher-level competencies, including error-minimization to reach an anatomical goal state and robustness to perturbation. This multi-scale homeostasis is driven by shared (non-local) stress dynamics and biologically plausible cell–cell interactions. Our analysis reveals how minimal low-level metabolic requirements can scale up into morphogenetic competency.
Our model is based on the following background assumptions as defined in detail in the TAME framework [7]. The first is that individual cells already have the competency to stay alive via homeostatic pursuit of metabolic resources. This is a minimal requirement for all living forms (already present in microbes, both unicellular and multicellular), but we assume its presence—we are not modelling the origin of life or the initial formation of the very first reproductive unit, but rather seeking to understand the collective behaviour of subunits that were once independent organisms themselves.
Second, we model two scales: a developmental phase where cells are alive and interact with each other, and an evolutionary wrapper which changes the frequency of different behavioural policies of the cells based on the fitness of the collective (how well the resulting ‘embryo’ matches a very simple criterion: having a single, three-valued positional information axis).
Third, our use of the term ‘goal’ is not meant in any kind of meta-cognitive, self-aware complex sense. Goals here are synonymous with the cybernetic approach [34,35], in which certain systems can expend effort to reduce error from a specified homeostatic setpoint. Our aim is to show how such goals can scale in size and arise spontaneously in new problem spaces under evolutionary selection, given very minimal and realistic assumptions about the properties of the components.
Finally, while we do not explicitly invoke bioelectric signalling mechanisms, our cell interaction model is quite compatible with that mode of communication (spelled out in detail in [20]). We do provide the cells with gap junctions (electrochemical synapses [41,42]), through which they can share state signals. We also include a mechanism for stress signals (initially, intracellular detectors of out-of-homeostasis states) to be propagated outward from cells, experiencing geometric frustration (a measure of how much the neighbours of a cell share the same state) as predicted by the TAME framework [7].
Multicellular morphogenetic algorithms or set of built-in behavioural and signalling policies that allow cells to cooperate and compete to reliably construct complex body pattern are still incompletely understood [20,25]. One relevant approach is amorphous computing, which refers to systems of many identical simple processors that have limited computational power and that interact locally. Typically, such systems have a more irregular structure in contrast to the regular structure of cell networks as they are used in our study [43]. Several authors have investigated pattern formation in developmental biology using the same task we use in this article, the French flag problem. Herman and Liu solved it through the simulation of linear iterative arrays of cells [44]. Several other models have been developed to resolve the same problem, including a reaction–diffusion model, use of computational primitives and genetic programming [37–40]. Miller evolved a development program that follows a predefined grammar that connects developmental encodings to genetic programming. He used a feed-forward Boolean circuit to implement a cell program [38]. Chavoya and Duthen used a genetic algorithm to evolve cellular automata that produced different two-dimensional and three-dimensional shapes [45] and evolved an artificial regulatory network (ARN) for cell pattern generation, resolving the French flag problem [46]. While others have simulated evolutionary growth of neural network-controlled cellular automata with hardwired mechanistic rules, this is the first exploration of the emergent consequences of evolutionary scaling based on dynamic interactions of cells with information-processing homeostatic capacity. The combination of neural networks and cellular automata which grow under evolutionary control is described in the work of Elmenreich & Mordvintsev [47,48].
However, their neural cellular automata do not include any cellular signalling or any use of stress capable of linking the different homeostatic levels for the scaling of goals (metabolic in individual cells and morphogenetic for the tissue). A key difference in our work is that the artificial neural network (ANN) inside each cell here models the behaviour of a gene-regulatory network and can control gap junctions to deliver the morphogen and the ability to reduce stress (tendency to reduce geometric frustration) to the other cells (figure 1).
Past studies have also investigated the multi-scale interface of body and mind, notably with ‘morphological computation’ in artificial life and soft evolutionary robotics [49–53]. These studies model and exploit the fact that brains, like other developing organs, are not hardwired but are able to ascertain the structure of the body and adjust their functional programs accordingly. Similar to scale-free cognition, morphological computation is about connecting the body and cognition—showing how optimal control policies span different scales, mechanisms and problem spaces. However, to our knowledge, no prior model has explicitly investigated how cell-level metabolic competencies scale up into tissue-level morphological ones.
We developed an evolutionary simulation system composed of cells that forms a tissue on a two-dimensional grid (figure 1). The system is an evolutionary, agent-based, spatialized model that uses two main time-scale loops, with different temporal scales. The outside loop is an evolutionary (phylogenetic) long time scale where genomes are mutated (here the genome is the one corresponding to the evolutionary algorithm and it generates the ANN controlling each cell in the tissue and agents are selected). The inner loop is a short time-scale ontogenetic loop that constructs each agent, simulates development, and then tests the phenotype for fitness. Each agent is one cell, that encodes some very simple metabolic processes and interacts with the other cells.
We integrated a homeostatic loop that enables it to optimize for the levels of energy (we are not trying to model the origin of life itself—we start with cells that can keep themselves alive via basic homeostasis, e.g cells that need to keep a level of energy superior to 0). We allow cells to attach to each other using gap junctions, to send intracellular signals. These signals, and their properties—such as speed of spread, ability to propagate gap junctions, gating properties of gap junctions based on signal and cell state etc.—are all controlled by the ANN inside each cell through evolution. Cells do not know the origin of a given signal once it is inside the cell; it has been suggested that this ‘wiping property’ enables a kind of shared memory that is essential for the scaling of small agents into larger intelligences [7]. The cells have a minimal memory of the past and have access to their stress and energy levels, and their states, at time t and t − 1 (see algorithm 1).
The general approach is related to cellular neural networks [54] and more particularly to the growing neural cellular automata [48] where each cell has access to the states of its neighbours and contains a neural network to drive actions (neural networks are simply a convenient formalism for encoding cell behavioural properties, equal in power to gene-regulatory networks [55]). In our case, the number of cells is fixed and does not grow with time and the tissue formed by those cells has borders (i.e. it is not a torus). The order of execution of cells is shuffled at each step.
Each cell contains an ANN (representing the computational functions carried out by gene-regulatory networks and pathways operating within cells [55,56]) which controls the opening and closure of gap junctions in four directions: up, down, left, right. ANNs are commonly used to model genetic regulatory networks [57]; in that line of research, the ANN inside each cell can be interpreted as a genetic regulatory network controlling the metabolic processes. Each cell has one type of morphogen molecules that can go through the gap junctions depending on their openings and that will trigger different kinds of genomic expression leading to three different cell states: blue, white, or red (see figure 1a). These states depend only on the level of morphogen inside the cells (red if the level of morphogen is between 0 and 5, white if between 5 and 10, blue if greater than 10). We also imposed an energy cost on state change. Each cell also has a stress molecule and an antistress counterpart that can be sent to its neighbours (see algorithm 2). To summarize, each cell can contain the following: the energy state, necessary for survival and for state change, morphogens that can go through gap junctions to define the cell state; and stress and stress-reduction molecules, which can also move between cells via gap junctions, and which define the stress state.
Our model does not enforce the use of the stress system: it is completely evolved since the number of stress and stress-reduction molecules to be transferred is chosen by the embedded neural network. In addition, evolution can decide to make the stress molecule instructive or not. We have chosen to refer to it as the stress molecule but, before evolution, it is just another communication system for cell signalling that does not directly affect the cell state (see §4.1.1.5). The neural network, which is identical for each cell, has 11 inputs: the internal levels of morphogens, energy at time t, energy at time t − 1, stress at time t, stress at time t − 1, the internal state at time t and t − 1, the size of the collective the cell is part of (number of cells of the same state connected by opened gap junctions in the tissue), the error between the size of the collective the cell is part of and the target size in percentage—e.g. the size of one stripe—(|100 − [size of sub-collective/(total number of cells in the tissue/3 × 100)]|), the perception of the cell neighbourhood (geometrical frustration or how similar the cell is to its neighbours) and a bias set at 0.5. The four outputs of the ANN are: the number of morphogen molecules to send to neighbours, the number of stress molecules to send to neighbours and to be applied to the cell itself, the number of stress-reduction molecules to send to neighbours and to be applied to the cell itself, and the opening of gap junctions in the four directions. The level of stress of one cell is bounded between 0 and 100.
The learning task is the French flag problem [58,59] (see figure 1b). It was originally developed by Wolpert [58] to formalize discussions about how spatial gradients might specify patterns of cell fates in a tissue.
We tied the single-cell and the anatomical homeostatic loops. Each cell has only one goal—to survive—and that corresponds to being in the appropriate state in order to receive energy (with the other members of the collective). On the other hand, the collective/tissue has a morphogenetic goal, which is to reach the French flag.
At each step, each cell receives a reward in the form of an amount of energy which is proportional to how close its corresponding sub-collective (corresponding to one stripe and therefore bigger that the immediate neighbourhood of one cell) is to reaching the appropriate anatomical goal. Each cell therefore has a reward uncertainty, because the reward depends not only on that cell’s own behaviour but also on the behaviour of the (sub)collective; in other words, the reward is affected by the decisions of distant cells. Therefore, the environment of the cells is of high uncertainty. In a sense, this scheme can be understood as a problem of multi-agent reinforcement learning under reward uncertainty.
Goal-directed systems have the ability to focus on relevant information and ignore distracting information. To do so, they rely on selective attention and/or interference suppression. Selective attention would rely on top-down biasing mechanisms as proposed by Desimone & Duncan [60]. In our case, the top-down biasing mechanism is represented by the reward in energy that ties the two homeostatic levels. We also imposed an energy cost for communication and state changes (respectively, of 0.8 and 0.25 per step).
Stress can be defined as three related concepts—the external and internal stimuli that cause stress, the emergency physiological and behavioural responses activated in response to those stimuli, and the pathological consequences of over-stimulation of the emergency responses [61,62]. Here, we implemented a communication system that enables diffusion of a stress molecule through the tissue to allow other cells to feel stress that was not caused by their own internal state. When the ANN of a cell makes the decision to send stress, that cell will diffuse a molecule to its neighbours, equally increasing its own stress level and that of its neighbours. In the same manner, the ANN can also cause the cell to send a stress-reduction molecule (functionally equivalent to an anxiolytic intervention) to the neighbouring cells that will decrease the cell stress levels (figure 1f). Importantly, whether or not (and how) the stress system is used to solve the patterning problem is not determined in the code. Evolution can produce agents that do or do not use this set of signals as a communication system. In other words, before evolution, what we call the stress system here is just another communication system in the tissue where a specific cell signalling will increase (or decrease) the level of a molecule inside the cell. This cell signalling works exactly like the morphogen communication system, except that it does not have any effect on the cell state. Evolution can choose whether or not to add an ‘interpreter’ to these specific molecules (e.g. the ones we defined in this section as stress and stress-reduction molecules) to make their levels informative for the cells or by triggering a specific behaviour (like biological stress).
We use ES-HyperNEAT [63] to simulate the evolutionary cycle. ES-HyperNEAT (evolving substrates HyperNEAT) is an extension of the original HyperNEAT method for evolving large-scale artificial neural networks using the NEAT method (neuroevolution of augmenting topologies). HyperNEAT evolves large-scale neural networks using the geometric regularities of the task domain. It uses compositional pattern producing networks (CPPNs) to generate the structure of the ANN. ES-HyperNEAT allows the positions and number of the neurons to be evolved instead of being defined before evolution as in HyperNEAT [63]. The fitness function is the percentage of good states the tissue has: