Cellular Competency during Development Alters Evolutionary Dynamics in an Artificial Embryogeny Model

Biological genotypes do not code directly for phenotypes; developmental physiology is the control layer that separates genomes from capacities ascertained by selection. A key aspect is cellular competency, since cells are not passive materials but descendants of unicellular organisms with complex context-sensitive behavioral capabilities. To probe the effects of different degrees of cellular competency on evolutionary dynamics, we used an evolutionary simulation in the context of minimal artificial embryogeny. Virtual embryos consisted of a single axis of positional information values provided by cells’ ‘structural genes’, operated upon by an evolutionary cycle in which embryos’ fitness was proportional to monotonicity of the axial gradient. Evolutionary dynamics were evaluated in two modes: hardwired development (genotype directly encodes phenotype), and a more realistic mode in which cells interact prior to evaluation by the fitness function (“regulative” development). We find that even minimal ability of cells with to improve their position in the embryo results in better performance of the evolutionary search. Crucially, we observed that increasing the behavioral competency masks the raw fitness encoded by structural genes, with selection favoring improvements to its developmental problem-solving capacities over improvements to its structural genome. This suggests the existence of a powerful ratchet mechanism: evolution progressively becomes locked in to improvements in the intelligence of its agential substrate, with reduced pressure on the structural genome. This kind of feedback loop in which evolution increasingly puts more effort into the developmental software than perfecting the hardware explains the very puzzling divergence of genome from anatomy in species like planaria.

In addition, it identifies a possible driver for scaling intelligence over evolutionary time, and suggests strategies for engineering novel systems in silico and in bioengineering.

One critical aspect of real biology which is not always taken into account in evolutionary computation and theoretical biology efforts, is that the mapping between genotype and phenotype is not direct [1,2,3,4,5,6,7,8,9,10,11,12,13]. Genes generally do not directly encode for structure and function of the organism. Instead, it has become increasingly clear that developmental physiology provides a critical layer of control that sits between genomes (on which mutation operates) and anatomy (the phenotype which is the subject of selection). During development, organisms emerge as the result of a complex set of interactions among cells, with anatomical order and functionality being the result of cellular activities. While genomes specify the cellular hardware (proteins), it is the software (cellular activity) studied by developmental biologists that is ultimately responsible for the organism’s overall structure and behavior [14,15,16,17,18].

The simple story of genomes determining anatomy is shown to be incomplete by examples such as the highly regenerative planaria [19]: due to reproduction by fissioning and regeneration, they retain mutations made in the parent body and pass them on to their offspring (somatic inheritance). As a result, planaria have an incredibly messy genome (indeed, worms are mixoploid—different numbers of chromosomes in each cell). Despite this, they have the most reliable anatomy: every fragment of a planarian regenerates a perfect worm each time. They are essentially immortal, and highly resistant to cancer [20]. How can the animal with the most chaotic genome have the most reliable, robust anatomy? Fundamental knowledge gaps in this area not only impede our understanding of basic evolutionary developmental biology but also limit our ability to make desired system-level changes to complex anatomy in the context of regenerative medicine [21,22,23].

The indirect relationship between genotype and phenotype has a number of important implications. For example, it is currently impossible to guess the anatomy of an organism by examining its genome—overall symmetry type, number and kinds of organs, size, regenerative capacity, etc.—can only be estimated if one compares a genome to that of another organism for which all of these are already known. Likewise, even when one has access to complete genomes, for example of the frog and axolotl, one cannot guess the shape of a chimeric embryo: will a “frogolotl”, consisting of 50% of each kind of cells, make legs (like an axolotl larva) or not (like a tadpole)? This is because, while much research has shed light on molecular mechanisms necessary for morphogenesis, the field still largely lacks an understanding of the key dynamics that determine form and function: large-scale anatomical decision-making by cellular collectives [19,24,25].

Importantly, the cells that make up these collectives evolved from independent unicellular organisms with extensive capabilities for sensing their environments and responding. The key role of these capabilities in morphogenesis reframes cells as an agential [26], not a passive, material. There are numerous examples: one of the most remarkable qualities of morphogenesis is its competency in reaching an adaptive anatomical outcome despite novel starting states and perturbations [24,27,28]. For example, mammalian embryos can be split into pieces, and each piece gives rise to a complete organism (monozygotic twinning). Some animals retain these regenerative capacities into adulthood—salamanders whose limbs (or eyes, jaws, tails, etc.) are amputated will re-grow exactly the missing portion and then stop when the correct structure is complete [29]. Tadpoles with scrambled faces become largely normal frogs, as the craniofacial organs move in novel paths until the correct configuration is achieved [30,31,32]. Tadpoles with eyes placed on their tails (and not in their heads) can see [33]. All of this means that mutations resulting in noise or changes in initial positions of the organs, which would have been disastrous for a hardwired architecture, will not have a strong effect on survival because the tissues will make needed reconfigurations to compensate for errors in initial state. It is clear that this rapid, built-in capacity for anatomical homeostasis and problem-solving must have implications for the evolutionary process, but this has not been extensively explored.

We previously proposed that the competency of the developmental layer results from the navigation policies of a collective intelligence of cells in anatomical morphospace—an evolutionary precursor to the intelligence of neural cells which are well known to navigate 3-dimensional and other problem spaces [24,27]. We refer to these navigation policies as competency of the cellular collective—the capacity to sense their environment and each other, and to communicate to effectively solve problems in morphospace during development (i.e., reach appropriate target morphology despite perturbations and changing internal and external conditions). Such collective cellular intelligence is an essential aspect of morphogenesis during development, regeneration, and cancer suppression, and is central to the genome-form-function relationship. As has been claimed for learning [22,34,35], these collective competencies could greatly smooth the evolutionary landscape and enable access to regions of the phenotype space that would otherwise have been hard or impossible to reach.

The evolutionary importance of information not encoded in the genome directly has been studied previously by evolutionary biologists and in the field of artificial life, primarily in the context of learning [24,27]. Many organisms can modify their behavior in light of experience, improving their fitness via information that was not provided by the genome. As early as 1896, Baldwin proposed that such adaptive behaviors, while not encoded by the genome, are assimilated into the genome (hardcoded) over evolutionary timescales, a phenomenon known as the Baldwin Effect [24,27]. Classic simulations of the interplay between learning and evolution demonstrated that the Baldwin Effect does indeed change the evolutionary landscape of organisms capable of learning [24,27]. This dynamic offers competing tendencies, in which learning takes pressure off the genome via a shielding effect [24,27] at first, by providing behaviors that do not need to be discovered by evolution of the genome. Subsequently, genetic assimilation takes the pressure off the need for learning, by eventually finding ways to hardcode those behaviors. This learning paradigm emphasizes two different kinds of fitness [24,27]. Genotypic fitness is the quality of the structural genome—what the organism would have been able to accomplish given only the static information in its genome. Phenotypic fitness is what selection actually ‘sees’—the performance of the organism after both hardwired and learned repertoires have had their chance to shine.

Here, we introduce cellular competency as another important contributor to phenotypic fitness. Cellular competency [25,34,35] has a number of important differences from learning. First, the ability of cells and tissues to attain and maintain setpoints in morphological spaces is independent of learning at the level of the individual. Second, learning at the level of the organism needs a mechanism (e.g., nervous system architecture) that must itself be painstakingly evolved—evolution must discover, maintain, and pay the costs of new capabilities such as brains and exploratory behaviors. In contrast, cellular competencies come “by default” because organisms consist of cellular components that already have many capabilities evolved during their ancestral lifetime as independent organisms. Thus, it is important to begin to study how cellular competencies affect evolution, to complement approaches focused on learning, evolvability mechanisms [36,37,38,39,40,41], and the material properties of morphological computation [42,43,44,45,46]. All of these factors interact in vivo, and will need to be studied separately and together.

The question we address here is: How do diverse levels of competency in the cellular collective during morphogenesis impact the rate and course of the evolutionary process? We undertook a quantitative investigation of this question using a minimal model of artificial embryogeny. Our model did not include learning or classical behavior at the individual level, but instead solved a problem in morphogenetic space. Our system simulates an animal with a single axis of positional information values (such as the anterior-posterior axis) [47,48]. Virtual embryos consist of a 1-dimensional array of integers, with their evolutionary fitness being proportional to the degree of monotonicity of those values. In the baseline case, we use a direct encoding where the phenotype is a direct consequence of the genotype—the values of the array directly specify the order of values in each embryo. Under these conditions, a genetic algorithm eventually produces structural genes in which all the values are in the correct (monotonic) order. We compare these outcomes to a more realistic case, in which the mapping is not direct: we introduce a development algorithm in which individual cells have some degree of competency to rearrange themselves based on their local environment. Cells can move to numerically more-advantageous positions before evaluation of phenotypic fitness. We accomplish this through a restricted bubble sort procedure [49], highlighting the conceptual similarity between sorting algorithms and navigation in a geometric problem space. This corresponds to embryogenesis in vivo, in which cells act before the mature animal’s fitness is ascertained in the environment. Importantly, our system does not include Lamarckian inheritance.

Instead, it features a strong barrier between soma and germline: the rearrangements occur for each individual but the only thing that gets passed on to their offspring is their original pre-swap structural genome [50,51].

We varied the degree of cellular competency, and tracked the dynamics of the resulting evolution, both in terms of raw genotypic fitness and phenotypic fitness. We also explored the effects of adding a competency cost. We observed a number of interesting outcomes. First, including a developmental layer that models a range of cellular competencies improves evolutionary efficiency in proportion to the degree of cellular competency. Second, in mixed populations, competent individuals tend to eventually dominate the population. Third, when the degree of competency is itself allowed to evolve, populations settle on a specific, sub-maximal level of competency. Finally, and most critically, we observed that because competency hides genetic deficiencies from selection, pressure to improve the structural genome is released, while pressure to improve the morphogenetic competency of cells is increased. These dynamics establish a positive feedback loop in which populations advance by progressively improving cellular capacities, not just the genes dictating the actual initial structure of each embryo. This provides an explanation for the otherwise mysterious disconnect between planarian genomes and their amazing anatomical robustness, and suggests the existence of an evolutionary ratchet working to optimize intelligence in even very basal forms [52,53,54,55,56,57].

We simulate the evolution of artificial 1-dimensional embryos in silico. The following sections describe the structure of each embryo, our paradigm for modeling developmental morphogenesis towards a target adult anatomy, and the process of selection employed to study their dynamics over time.

A population consists of a number of embryos. Each embryo is represented as a one-dimensional array of fixed size (matching the cell count in the 1-dimensional embryo). Each cell of this array is initialized with a different integer value representing the positional value gene for the corresponding cell of the embryonic axis (see Supplement S1). In this minimal model, there is no further chromosomal structure or transcriptional change, and we simply refer to the structural genes as directly specifying the positional preference of a given cell. Each embryo undergoes a developmental cycle (described below) to become a mature “individual”. We model evolution in three kinds of populations: a “hardwired” population consisting of only hardwired embryos, a “competent” population of only competent embryos, and a “mixed” population which contains both kinds of embryos, in varying proportions. Our mixed populations have 200 embryos, the rest have 100.

We define two types of embryos, a “hardwired embryo” and a “competent embryo” (Figure 1A,B). The difference between them lies in the way they develop during the evolutionary cycle. A competent embryo consists of cells capable of sensing neighboring cells and adapting morphology by moving around prior to the adult stage in which fitness is evaluated. “Competency” is the capability of these embryos to carry out such reorganization, and they carry a gene that dictates their degree of motility (fixed, in some experiments, but free to evolve in others). Our competent embryos leverage sensing and motility to reorganize their cells during ‘development’ in a way that boosts fitness (see below and Supplement S1). We vary the degree to which they can reorganize (competency level). A hardwired embryo lacks this capability; its structure from birth to maturity is constant.

Soon after initialization, embryos undergo a developmental cycle. During this process, competent (but not hardwired) embryos undergo a restricted bubble-sort procedure (see Supplement S1) to rearrange their cells in a way that boosts fitness (i.e., to increase ascending order of its array of integers). At the end of the developmental cycle, embryos are considered “individuals”.

We define fitness as the degree to which an embryo’s array of integers is in ascending order. Individuals with cells arranged in ascending order by value are attributed a fitness of 1.0 (maximum), those whose cells are randomly ordered are attributed a fitness of 0.5. We calculate the fitness (the degree of order) of an array by counting the number of non-inversions present (see Supplement S1). At the beginning of each evolutionary cycle, all embryos are considered “just born”; their morphological structure determined by their parents from the previous generation. Therefore, we call their fitness at the start of each cycle the genotypic fitness. At the end of the developmental cycle, the fitness of each resulting individual is calculated again, which we call the phenotypic fitness. For hardwired individuals, phenotypic and genotypic fitnesses are always identical. For competent individuals, however, phenotypic fitness reflects the reorganization that occurs based on their competency level.

At the start of each developmental cycle, a competent embryo is assigned an integer representing its competency level. This integer determines how many successive bubble-sort swaps will take place during its developmental cycle. Usually, competency levels are much lower than the total number of bubble-sort swaps required by an embryo to attain maximum fitness, for this reason it is called “restricted” bubble-sort.

To evolve populations (hardwired or competent), we iteratively pass them through three stages (Figure 1C):Selection: The fittest 10% of individuals in a population are selected to move on to the next generation. Selection in a population is based on its individuals’ phenotypic fitness.Cross-Over: In order to repopulate a population back to its original strength, we carry out a process of reproduction called cross-over. It occurs as follows: Two individuals are involved, each of these are split at a random location along their length. One half of Individual 1 is swapped with the same half of Individual 2 to give rise to two children. Figure 1 contains an illustration of this process.Mutation: The repopulated population is subjected to random point mutations. We set the probability of an individual receiving a point mutation to be 0.6.

We built a virtual embryogeny model in which fitness was defined by the degree of monotonicity of a 1D array of numbers, simulating a minimal metazoan bodyplan—a single axis of positional information (Figure 1). The initial sequence of numbers for each embryo was assigned randomly. Since these sequences decided the embryo’s structure (cell order), they are referred to as its structural genes. As described above, in hardwired embryos, that sequence is fixed: their genome directly encoded their phenotype. For competent embryos, we implemented different degrees of competency during a developmental period in which cells were allowed some degree of movement relative to their neighbors, allowing them to reorganize to improve monotonicity prior to evaluation of phenotypic fitness. This enabled phenotypic fitness for competent individuals to diverge from raw genotypic fitness, with the extent of divergence depending on how much cell movement was permitted. This corresponds to different degrees of capacity for cells in vivo to optimize homeostatically preferred local conditions with respect to informational signals such as positional cues and polling of neighboring cell states [58]. An evolutionary cycle was implemented around these developmental events [58]. In the initial experiments, the competency gene is fixed across the evolutionary run, enabling study of the evolutionary dynamics over time as a function of different degrees of cellular competency.

We first compared, over 250 generations, the time-course of evolutionary search towards a fully ordered axis in hardwired vs. competent individuals. After 100 generations, the hardwired population had the least fitness compared to populations with varying degrees of competency (Figure 2 and Table 1). Table 1 provides a summary of the generation number at which each population crossed different fitness thresholds. We compared fitness of the best individual in competent and hardwired populations at generations 2, 10, and 20 (because these points exhibited the greatest sample variances.) At each of these, the difference in fitness between hardwired and competent populations was significant (p-values << 1 × 10−3 for all points, Student’s t-test; for details see Supplement S2.1.1).

Figure 2 also shows that the 95% confidence interval bands over 100 repeat runs decreased with increasing competency level, suggesting that more competent architectures are also more consistent in performance over time. Note that hardwired individuals gradually improved to reach peak fitness, taking well over 200 generations to do so, whereas the most competent individuals (with a competency level of 400) did so in under 6 generations. These data demonstrate the role competency plays in non-linearly improving the rate of fitness of a population and support a clear conclusion: the higher the competency, the better the performance.

Based on the impact of competency, one could hypothesize that progressively increasing competency would lead to a progressive decrease in selective pressure for good structural genes to appear. An embryo with high competency would have no selective pressure to improve its structural genes beyond a certain level because it can rely on its competency to re-order its cells to reach peak fitness. This is in fact what we observed (Figure 3). We compared the genotypic fitnesses of the best individual in three populations with different levels of competency (20, 100, and 400) to that of a hardwired population. In all three competent populations, genotypic fitness rose with that of the hardwired population for a few generations, after which it plateaued, indicating that at this point, the structural genes were good enough for competency to achieve a phenotypic fitness that insured selection. Further, with increased competency, the 95% confidence interval bands for genotypic fitness grew wider. Thus, as hypothesized, increasing competency in our simulation enabled excellent performance but reduced selective pressure on the embryo’s structural genes.

Given these tradeoffs, we next asked how mixed populations (200 embryos per population) of competent and hardwired embryos would evolve (Figure 4). We varied both the level of competency and the percentage of competent embryos in the hybrid population at the start of the simulation. To probe the levels of competency required for embryos to dominate the population over the evolutionary simulation, competent embryos were always initialized as a minority of the starting population. Relationships between competency, initial population proportion and dominance were observed over several runs.

When competent embryos constituted just 2.5% of the initial population, they failed to dominate even at the highest level of competency tested: embryos with a competency level of 95 merely reached equal percentages with hardwired embryos. As their initial proportion in the population increased, competent embryos required progressively less competency to dominate over their hardwired competitors. At 10%, embryos with a competency level of 75 could dominate; at 20%, the competency level required for domination decreased to 40; and at 30%, competent embryos dominated with a competency as low as 10 (Figure 4). In all starting conditions that resulted in dominance of competent embryos, it occurred rapidly, in just two or at most three generations (Table 2).

To determine how competency might spontaneously evolve over generations, we introduced competency as an evolvable trait by letting each embryo’s competency level be determined by a single ‘competency gene’ with value in the range [1, 500]. During initialization, the competency genes of all embryos were set randomly to low values in the range [1, 15]. Then, during evolution cycles, we allowed each competency gene to be mutated, potentially taking values across the range of [1, 500], and tracked the competency gene values of the best individual over 1000 generations (Figure 5). The prevalence of the competency allele rapidly rose, meandering and exploring values up to 485 during evolution (shaded area in Figure 5A) before plateauing at ~470. We provide a possible explanation for this outcome in the Discussion section.

To understand how allowing the competency gene to evolve over 1000 generations affects genotypic fitness, we looked at the phenotypic and genotypic values for the fittest individual in each generation (Figure 5B). Values for the fittest individual quickly settled at consistent configurations in which the phenotypic and genotypic fitnesses diverged considerably. This is a fascinating outcome because it suggests that a certain level of competency reduces the pressure for improvements in an embryo’s structural genes. Once selection can no longer distinguish whether fitness is achieved by a set of good structural genes or by a high competency level that compensates for a poor set of structural genes, it can only improve the population by increasing competency, not by selecting better genetics.

To quantify this effect and determine how well selection, which ‘sees’ phenotypic fitness only, selects for genotypes when competency is allowed to evolve, we plotted the degree of correlation between genotypic and phenotypic fitness for all individuals in these populations (Figure 5C). Correlation dropped to 0 within about 20 generations as individuals who succeeded because of their developmental competencies rapidly dominated the population. We conclude that allowing competency to evolve disrupts the ability to select for the best structural genes. We further validated this by examining the frequency, over 1000 cycles of evolution, with which positional changes to a single ‘cell’ resulted from tweaks to the competency gene vs. from tweaks to one of the structural genes. Figure 6 shows that the frequency of changes to the competency gene was much higher than the average of all fifty structural genes across 1000 generations in our simulation.

The Baldwin Effect [59] is the now broadly accepted phenomenon in which individual organisms can achieve greater reproductive success based on behavioral adaptations, and that these adaptations can eventually become hardwired into the genome in subsequent evolutionary cycles. Hinton and Nowlan [58] found that introducing an algorithm for behavioral learning into evolutionary simulations produced outcomes consistent with the Baldwin Effect. Our initial simulations of the evolutionary impact of cellular competency did not exhibit the Baldwin Effect. This could have been due to the fact that our minimal model did not simulate any cost associated with increasing cellular competency, and thus there was no selective pressure towards genomic changes. Although the actual energetic (or other) costs of cellular competencies are not known for any living model system, it is possible that the cellular computations required for axial patterning require additional resources over and above developmental events (competent or not) that are essential for any embryo. Thus, we next studied the effects of introducing a competency cost by penalizing the fitness of embryos in our model by a factor of their competency-value. Using penalty factors in the range of [1 × 10−7, 0.5], we did see a Baldwin effect: the rate of rise of genotypic fitness corresponded positively with the increase in penalty factors. For penalty factors over 0.5, the genotypic fitness rose well above the phenotypic fitness, leading to disappearance of the Baldwin effect.

The results of simulation using a penalty factor of 1 × 10−4 over 3000 generations are shown in Figure 7. As described above for simulations with no competency cost, phenotypic fitness reached its maximum in under 20 generations. However, unlike the previous experiment, the fitness of the structural genes did not plateau after a brief increase, but continued to improve over the course of evolution (Figure 7A). Further, as the genotypic fitness rose, selection preferred progressively lower competency values (Figure 7B). Phenotypic fitness was maintained at the maximum level, but the way in which embryos achieved phenotypic fitness evolved to value structural genes over the competency gene. Over time, selection ensured that the genotype improved to a stage where competency became redundant—the Baldwin effect [59]. We conclude that in the context of expensive competencies, selection is faced with a tradeoff between competency and the structural genome: it can either pick high competencies and bear subsequent penalties, or, it can pick low competencies and improve its structural genome. Since improving the structural genome does not bear a cost, selection prioritizes improvement of the structural genome, and over time, nullifies the effect of competency. Thus early gains based on the competency gene are later assimilated into the structural genes, paralleling what has been described previously in the context of organism-level learning [59].