The scaling of goals from cellular to anatomical homeostasis: an evolutionary simulation, experiment and analysis
The whole system has been coded in Python programming language, using the agent-based modelling framework Mesa [64] and the MultiNEAT package. It is freely available on https://github.com/LPioL/scalefreecognition.
All experiments in this article used the same parameters and have been repeated over 20 runs. For each evolution, we used 250 generations, a population size of 350 individuals, a division and variance thresholds of 0.03. The energy cost for state change is 0.25 and at each step, cells lose 0.8 in energy. The minimum and maximum number of species are, respectively, 5 and 15. The number of generations without improvement (stagnation) allowed for a species is 10. The depth for the neural net is four hidden layers (increasing the number of hidden layers to five did not change the general fitness score) and three for the quadtree.
The available CPPN activation functions for the French flag task domain were sigmoid, Gaussian, linear, sine and step. The band-pruning threshold for all ES-HyperNEAT experiments was set to 0.3. The bias value for the CPPN queries is −1. The cells starts with energy levels initialized at 70. The energy cost of communication is set at 0.8. We also applied an energy cost of −0.25 to the state change for all cells.
Information theory [65] is a very useful tool to understand the information dynamics in complex systems. We used two information-theoretic measures to analyse the information dynamics of the results: active information storage (AIS) [66] and transfer entropy [67].
The amount of information in the past of one agent that is relevant to predict its future state is defined as the information storage. In this article, we focus on the AIS component, which is the stored information that is currently in use for computing the next state of the agent [66]. Formally, the AIS of an agent Q is defined as the local (or unaveraged) mutual information between its semi-infinite past qn(k) as k → ∞ and its next state qn+1 at time step n + 1:
aQ(n, k) represents an approximation of history length k. The average over time (or equivalently weighted by the distribution of (qn(k),qn+1)): AQ(k) = 〈aQ(n, k)〉. With AIS, an agent can store information regardless of whether it is causally connected with itself [66].
Transfer entropy is the information provided by the agent source about the destination’s next state that was not contained in the past of the destination agent. In this article, we use the local transfer entropy introduced by Lizier [66]. The local transfer entropy from a source agent Z to a destination agent Q is the local mutual information between the previous state of the source zn and the next state of the destination agent qn+1, conditioned on the semi-infinite past of the destination qn(k) (as k → ∞):
Transfer entropy TQ(n, k) is the (time or distribution) average: TQ(k) = 〈tQ(n, k)〉 and tQ(n, k) represents an approximation of history length k. While mutual information measures correlation only, the transfer entropy measures a directed and dynamic flow of information in the network of agents.
For the analysis of our information dynamics, we computed a different kind of transfer entropy: the transfer entropy from stress (by discretizing it by chunks of 10) to the states of the neighbours (blue, white, red). We computed the local transfer entropy pairwise with all the neighbours of one cell and averaged it. In the same manner, we computed the transfer entropy from the energy state to the state of the cells and conversely the transfer entropy from the state of the cells to the energy state.
We also defined the transfer entropy from one stripe to another as the sum of all transfer entropy of one cell of one stripe to all cells of the other stripes computed pairwise on the time series of the internal states. Formally, we define the averaged transfer entropy over a range of source–destination pairs in the spatial locations of different stripes S1 and S2:
The transfer entropy defined for specific subsets of tissue processes is useful in considering distributed communications across agents with specific roles.
Dugesia japonica were maintained in Poland Spring water at 20°C, fed calf liver paste once a week and cleaned twice a week, as described in [68]. Animals were starved for one week prior to usage in amputation and pharmacological treatment experiments and were not fed for the duration of all experiments.
Cutting of planaria was performed on a cooling plate using scalpel fragments. For generating headless animals, pre-tail fragments were cut by placing a cut at the pharynx opening and another cut narrowly above the tail tip. For tracking tail regeneration, animals were cut halfway between the head and tail. For tracking regeneration along the anterior/posterior axis, animals were either decapitated narrowly by cutting just below the auricles, cut halfway between the base of the head and the top of the pharynx, or cut directly at the top of the pharynx.
Headless worms were produced through transient pharmacological treatment of freshly cut pre-tail fragments in 18 μM U0126 (a well-known blocker of ERK/MAP kinase [69]) dissolved in DMSO (Sigma). No more than 40 fragments were treated per 10 cm Petri dish. Fragments were incubated in U0126 for 3 days at 20°C before washing out the drug solution. At 14 days post-amputation, the fragments were scored for regenerative phenotype.
Headless animals were maintained in individual wells of 12-well plates and their phenotypes were scored weekly for the duration of the experiment. Any significant changes in morphology, such as head regeneration, fissioning or ectopic tissue development, were noted. Time of repatterning was determined once one eye was visible in the forming head structure. Worms were labelled as ‘Polarity Flip’ when a headless worm fissioned and a head regenerated at the posterior wound site. Worms were labelled as ‘Dorsal/Ventral repatterning’ when an outgrowth of tissue occurred in the dorsal/ventral (D/V) plane of the headless worm. Worms were labelled as ‘Lateral growth’ when significant changes in morphology occurred with growth on the lateral area of the animal, resulting in stable morphology which did not produce a head.
We analysed a number of experiments performed with this system, tracking key physiological parameters over time in each experiment (see figures), including gap junctional communication, stress levels and cell types as a function of position. We focused on the most biological evolved tissue: the one where stress emerged during evolution as an instructive signal and increases and decreases as a function of the homeostasis of the tissue.
We first tested the ability of the cellular collective to solve the French flag problem, that is, to organize the two-dimensional tissue into a one-dimensional axis of positional information with respect to cell type identity. Each cell received energy according to is location on the tissue, and its energy reward was proportional to how well the other cells of one stripe of the French flag were resolving the French flag pattern (in other words, evolution gives partial credit for imperfect primary axial patterning, selecting for embryos with optimal morphogenesis). The cell colours in all figures represent cell fate, as in the original definition of the problem in which an embryo must spontaneously pattern itself into three ordered regions of cell fate [58]. All cells started in the blue state (homogeneous tissue corresponding to an un-patterned nascent blastoderm). To solve the problem, cells had to dynamically cooperate with, and send the appropriate signals to their neighbours. The fitness function was computed for 100 steps, defining the time course of development in this virtual embryo.
We observed that the ANN inside each cell evolved, and that this system was able to solve the problem (figure 2). A typical tissue behaviour had the following features. First, the regions corresponding to future white and red stripes began to be stressed and then it was mainly the red stripe that was stressed. All gap junctions were open, the upper and right gap junctions were fully open and the low and left gap junctions were half open. The gap junction states weighted the flow of molecules so that the streams from right to left and left to right were equal as was also the case for the streams in the vertical axis. The diffusion of morphogen molecules from left to right occurred because the cells of the blue stripes acted as a reservoir of morphogens and the cells decided to send more or fewer of these molecules to their neighbours depending on the dynamics.
In the representative example we show here, the stress increased at 55 steps and it then decreased as the French flag morphogenetic problem was resolved. At 90 steps, the tissue reached 95.1% of the French flag target morphology, and there were four remaining blue cells in the red stripe (an almost perfect solution). The same result was seen over 20 repeat runs, solving the problem in 94.4±0.84% of over 20 runs. The sharp change in morphogenetic activity around step 55 appears because the cells had learned to be stressed at a specific energy level. This converts continuous metrics to sharp discontinuities—the perception and decision-making that cells have learned over their history. This is demonstrated by our finding that when the stress level at initialization is increased, the spike in stress is delayed proportionally (see electronic supplementary material, figure S1).
Thus, the tissue learned to minimize error between its current state and the target morphology during its lifetime in order to stay alive; by doing so, the tissue—starting from a homogeneous state—resolved the French flag problem. It seems to work less well with random initializations, suggesting that robust development is optimized when cells start with a muted, homogeneous ‘baseline’ state, not highly diverse starting states, which is indeed what is observed in real development, where embryos do not begin as a collection of randomly differentiated cells, but as a homogeneous mass of undifferentiated ones (see electronic supplementary material, figure S3). We conclude that this minimal system shows how cell behaviours (that can be tuned by evolution or learning) enable the collective to harness individual metabolic homeostatic loops (the pursuit of energy) towards a global patterning goal.
The cells had learned error-minimization in order to reach the anatomical goal, but did they follow a hardwired plan that could only enable a feed-forward emergent pattern? Or had they in fact acquired an ability for homeostasis that would allow the tissue to reach the target morphology even in the presence of perturbations beyond the ab initio morphogenesis?
To determine the degree of plasticity of this process, in this next set of simulations, we perturbed the tissue by artificially changing at 110 steps the states of the last two columns of the red stripe to white cells. The cells had never been evolved on more than 100 steps. However, after the perturbation at 110 steps, the tissue corrected the red stripe in 10 steps and a few blue cells appeared in the white stripe (figure 3). The stress increased and decreased in parallel to the perturbation and its resolution, as it did to resolve the French flag problem. This capacity is similar to that observed in biological systems, many of which are able to regain normal morphology despite a wide range of perturbations [1]. The system tried to get rid of the remaining blue cells left in the white stripe. At 200 steps, the tissue reached 95.1% of the French flag. The dynamics of the gap junctions stayed the same as during the French flag resolution as seen in figure 2 without perturbation.
We also examined the effect of changing the size of the perturbation domain (see electronic supplementary material, figure S2). The tissue can adapt sufficiently for survival following a range of smaller perturbations (from two cells to two columns within a stripe) made at step 110. However, it was unable to recover from a much larger perturbed region (e.g. converting the entire red stripe into a white one). In this case, it attempted to adapt but dies at 200 steps. For smaller perturbations, the tissue reached a French flag compatible with survival. Interestingly, biological data support this surprising result: in the case of transformation of normal melanocytes in tadpoles to body-wide melanoma [70], and in the case of transplantation of healthy tissue into a deformed tadpole to repair its brain development [71], a very few cells are necessary to effect system-wide morphogenetic change.
Thus, we conclude that even though we did not specifically reward for anything other than the single self-organizing property, the cells were able to repair to the target setpoint from multiple starting configurations. In this sense, the tissue is excitable as a perturbation will spread and change the morphology.
We discovered that the tissue can resolve the French flag problem and is robust to perturbation, but does the tissue reach long-term survival and maintenance of an adult phase? To answer this question, we performed simulations running for 1000 steps. We observed that the tissue maintained its morphology during the whole lifetime. In addition, at 1000 steps, the morphology was even better than in the developmental phase, with the tissue reaching 96.3% of the French flag target morphology on this simulation (figure 4a). The tissue was observed to spontaneously become stressed several times during its lifetime; these stress increases were not the consequence of any external perturbation, and were due to the intrinsic dynamics of the tissue. The first stress increase, as described above, happened prior to step 90 in order to reach enough of the target morphology to stay alive. We observed (e.g. in the representative individual shown in figure 4b) four additional stress increases at steps 285, 322, 395 and 896. At these steps, the intrinsic dynamics of the tissue created deviations from the target morphology that increased the level of stress, after which the tissue corrected the new anatomical trajectory in order to reach homeostasis, at which point stress decreased. Once the tissue reached a morphology compatible with life, stress was reduced to 0. On average for this task, the collective reached 87.7±13% of the target morphology over 20 runs. These deviations represent spontaneous remodelling as no external perturbation has been applied to trigger them. Remodelling first begins after a time window three times greater than the time needed to reach the target morphology during development. In other words, the morphogenesis can be said to have been completed and ceased long before spontaneous morphogenetic activity suddenly resumes.
This phenomenon is similar to that described by catastrophe theory [72], and suggests self-induced missing-tissue response (MTR) [73,74], a process similar to the one we observed in the planarian experiments (see §6). In our simulations, a dramatic spike in stress immediately precedes the sharp onset of extensive morphogenetic activity (see electronic supplementary material, figure S1a), suggesting that stress drives the morphogenetic changes in the tissue.
Interestingly, the cells had never been evolved over a period of time greater than 100 steps—there was no selection pressure for long-term stability or survival. However, they were able to maintain the tissue more than 10 times longer than their initial developmental period (the only aspect on which they were evolved). We were also surprised to find they also learned allostasis. Following the definition of McEwen & Wingfield [75]: allostasis is the process of maintaining stability (homeostasis) through change in both environmental stimuli and physiological mechanisms. In this simulation, the tissue changed regularly as it tried to get rid of remaining, inappropriately located blue cells; sometimes this remodelling process took the collective a sufficient distance from the French flag target morphology to activate homeostatic mechanisms which then drove it back to normal, allowing long-term maintenance and survival of the tissue.
Thus, we conclude that long-term stable survival does not need to be specifically selected for, as intrinsic dynamics and emergent anatomical regenerative capacity are enough to maintain order through an adult phase. Homeostasis is an active process which is mirrored in this case by the level of stress that increases and decreases with the distance between the current tissue state and the target morphology.
The morphogenetic process exhibits interesting stress dynamics, consistent with the proposals [7] that homeostatic loops are driven by stress as a reflection of delta from setpoint, and that complex setpoints such as tissue-level morphogenetic patterns could arise from cells sharing stress information to optimize plasticity and coordinate in more complex problem spaces. Thus, we next studied the functional role of stress in the emergent morphogenesis we observed.
We first sought to determine whether stress was instructive or merely a by-product of the various dynamics. To answer this question, we simulated a loss-of-function of the stress system in the tissue (e.g. as the action of an anxiolytic drug), which forcibly reduced the level of stress to 0 during the whole lifetime of the embryo (figure 5).