A second-generation device for automated training and quantitative behavior analyses of molecularly-tractable model organisms

During the development phase, a number of problems were solved, including the following:

A high sensitivity current detector was designed to operate in the presence of high common-mode voltages for shock sensing; this is key, since a feedback needs to tell the device that a given shock actually occurred (some organisms secrete slime or change the salt content of the chamber's medium, which in turn impacts the amperage of the shock current).

The lighting system required extensive illumination head sealing for 72 hrs continuous run time, and a high-performance LED cooling system for 50,000 hour life requirement.

A system of signal switching was implemented, to allow shock to be administered in a shifting hexapole configuration, producing a uniform electric field in the dish and insuring the animal receives an appropriate shock regardless of orientation relative to the electrodes. This required finite element analysis modeling to design a chamber geometry, electrode position, and voltage profile which ensured a homogenous-current distribution during punishment that gives no dead-spots in which to hide, nor hotspots that will be avoided independently of the task being learned. All electronics were designed so as to minimize vibration and noise that could be confounding factors for sensitive aquatic animals.

Electrode materials were chosen in a shape, size, and material that balances nonlinear tradeoffs of low capacitance, lack of hot-spots (non-uniformity of field), and avoidance of electrolysis products (toxic metal ions) being shed into the medium during shock. The electrode holder was also chosen for inertness (low toxicity) and optimal compatibility with the electrodes, including also a spring-pin connection system for easy disassembly.

Image processing algorithms were designed and optimized to allow organisms to be tracked reliably despite significant variation in size and position relative to the walls and divider. This is a very challenging task because of the many shadows, edges, and occasional bubbles, the ability of animals to move within the Z plane (vertically), and the huge range of brightness levels (gain) between quadrants in which no blue light is on and those in which it is on at full strength. Moreover, the software had to continue tracking even when water levels changed over 3-day experiments, which is difficult because as the meniscus moves due to microevaporation, additional shadows and shifting brightness levels impact the use of a background image captured without the animal at the start of the experiment. An automatically-healing background capture algorithm was developed to derive background images from any quadrant vacated by an animal, as well as an intelligent image-processing component that adjusts parameters as a function of organism size. Figures S3, S4 and Videos S1, S2 contain examples of observing real animals, showing successful tracking under normal and difficult conditions.

Software was written, containing a flexible Graphical User Interface (to allow the operator to define the type of trial and controls he or she wishes to perform, Fig. 2) and front-end code, and firmware to track the animals (image processing and machine vision tasks) and control the light- and shock-emitting electronics to the chambers. This allowed individual shock and light levels (each animal needs light and shock inputs specific to its own position and performance in the past time intervals, for true flexibility of training paradigms). This included the development of an IR light source and cameras, a staggered processing and synchronization scheme, registration of images to a small tolerance to allow background subtraction, high-gain image analysis (to allow detection of animals in both dark quadrants and lit-up quadrants), diffraction and light leakage due to quadrant separators, implementation in firmware on digital signal processor chips, and much more. Video S2 contains an example of an animal's tracks re-created after the experiment (from saved logfile data) using one of the utilities developed for this system.

The work was performed in 3 phases. First, a single-channel prototype was created and tested to ensure that environmental controls and tracking were optimized. Next, a 3-channel system was multiplexed to test parallelized independent function and lack of “leakage” of stimuli between adjacent cells. Finally, the whole 12-channel system was assembled and fully tested for compliance with the original specification document. Below, we illustrate the device's use through proof-of-principle analysis of behavior in 2 different organisms.

A goal of all automated behavior analysis systems is to be able to rapidly screen basic behaviors of organisms, including preferences for locations, lighting conditions, and general activity levels. A useful feature for analysis of trial data is the “curiosity plot,” a pseudocolored heatmap showing the relative frequency of occupancy for each position in the dish. To illustrate, a 14-day-old Xenopus tadpole was placed in the device with the top half of the dish illuminated with red light and the bottom half of the dish illuminated with blue light (Fig. 4A). The curiosity plot software reads in the position of the tadpole over the 30 minute trial and places a blue spot (size determined by user) on a map of the dish for each timepoint. Where dots overlap (indicating multiple visits to this location by the animal during the trial), the color increases in intensity, from blue to green, yellow, then red. The output shows that over the course of a 30 minute trial the tadpole spends the majority of the time at the edges of the dish with no obvious preference or blue or red illuminated halves (Fig. 4B). To screen for effective levels of shock (those that illicit a behavioral response but do not injure or kill the organism over long periods of exposure) in different organisms, heat plots can be used to determine if tadpoles avoid quadrants that are punished. Tadpoles receiving a 1.4 mA shock when located in the blue half of the dish quickly learn to spend the majority of their time on the non-punished red side (Fig. 4C). Heat plots can be generated within minutes of trial completion and provide a convenient way to evaluate light preference as well as responses to varying intensities and durations of shock.

Planarian learning and memory, and the mechanisms underlying it, have been studied since the beginning of the 20th century, mainly through the 50's and 60's by McConnell and his associates [105], [107], [108]. Although the extensive research led to some remarkable discoveries [105], [107], the field of planaria learning and memory has been limited in impact, suffering from controversies mainly due to the lack of sufficient standardization of experimental procedures [108], [109]. Our platform was designed to overcome these problems, which will resolve the controversy through quantitative rigorous behavioral analyses and standardization of experimental procedures (greatly facilitating the reproduction of experimental results by other labs).

Here we describe preliminary studies characterizing suitable planaria species for automated training. We examined two planaria species, D. japonica and S. mediterranea. Both species are easy to breed and maintain in the lab, possess outstanding regenerative capabilities and have been well studied in recent years using modern cell and molecular biology techniques [45], [110], [111]. In contrast to D. japonica, S. mediterranea has a stable diploid genome making it more amenable to genetic approaches [112].

In each of the experiment trials, we utilized the device to track locomotor and exploration behavior, as well as light preference, of 12 planarians simultaneously. A total of 24 worms from each species (0.5–1 cm in length) were tested in 4 separate trials (results are displayed as the average of the entire group for each species). Each trial lasted 122 minutes. During the initial 2 minutes, the environment was illuminated with a homogenous red light background. Following this initial period, one half of the training environment was illuminated with blue light and the other half remained red. After one hour the illumination halves were swapped.

The results show that both species prefer red background illumination over blue (Fig. 5). At the end of the first two minutes of the trial (where the whole environment was illuminated with red light), the worms were found scattered uniformly (Fig. 5A,B, starting state). Shortly after half of the dish was illuminated with the blue light, worms started moving from the blue light to the half illuminated with red light; at the end of the first hour most worms (23 out of 24) from each group were located in the red half. Examination of the last 10 minutes of the first hour reveal that both species spent approximately 95% of their time in the red illuminated half (Fig. 5,A,B). After the illumination quadrants were exchanged, the D. japonica worms slowly moved to the red half, however the majority of the S. mediterranea worms stayed in their original locale— previously illuminated with red but now illuminated with blue (Fig. 5.A,B). At the end of the second hour, 13/24 D. japonica but only 2/24 S. mediterranea were found in the red illuminated half (as a group, D. japonica spent 55% of the time in the red half while S. mediterranea spent only 8.17% during the last 10 minutes of the trial, Fig. 5 A,B).

These light preferences were further examined by looking at exploration behavior (Fig. 5.C–F). The data on movement rate and total area explored during the trial showed that both species exhibited an exploratory period during the early phase of the trial when they were first introduced to the new environment (Fig. 5C,D). However, S. mediterranea displayed a significantly lower movement rate and shorter exploratory phase (10–30 minutes) compared to D. japonica (20–50 minutes). At the end of this exploratory phase, the worms settled down in their preferred place (the red illuminated half, usually on the edges of the dish). When the blue and red illuminated halves were exchanged after the end of this exploratory period, it caused the D. japonica worms to move into the new red half, while most of the S. mediterranea animals stayed where they were (despite now being in the blue half). The curiosity maps (Fig. 5E&F) of the one most active worms from each species show that D. japonica is much more active and its exploration behavior is more extensive. It also reveals that both species spend most of their time at the edges of the dish (red spots in Fig. 5 E&F).

These data show that the system is capable of accurately tracking animals as small as planarians 0.5 cm in length. As has been observed in manual studies, planaria indeed show negative phototaxis [103], although they have a lower sensitivity to red light [104]. Thus, combinations of faint red background and bright blue light in our device can be used for planarian training paradigms such as training (against their normal preference) to move toward the quadrants illuminated with blue light or instead illuminating the entire environment with blue and using its removal as a reward for correct behavior. S. mediterranea locomotor and exploration behavior are markedly less pronounced than those of D. japonica and as such D. japonica is likely to be a better candidate for learning studies. Using D. japonica, which can readily be manipulated even after the initial exploratory phase, will allow long (up to 72 hours) trials and flexible training procedures uniquely suited to this automated system.

Applicability of the system to a range of aquatic organisms enables comparisons between related or divergent species, as well as the same species at different ages. As an example, we compared the preference for blue or red light and movement rates of 14-day-old Xenopus tadpoles and 21-day-old juvenile zebrafish. These ages were chosen based on the relative size similarities between the organisms at this stage. For the comparison, both organisms were put in a testing environment where half of the dish was illuminated with red light, while the other half was illuminated with blue light. Over the course of 30 minutes, quadrant location and movement rate were recorded at 10 frames per second (10 Hz) for each organism and averaged over 5 minute intervals.

The data reveal that while Xenopus tadpoles showed no preference for either color light, spending roughly half of the time in both blue and red halves, zebrafish spent approximately 70% of the time under the blue light (Fig. 6A, t test p = 0.032). In addition, zebrafish also appeared more active, moving around the dish at greater speeds than the tadpoles during the course of the 30 minute evaluation (Fig. 6B). Interestingly both organisms showed a 15–20 minute exploratory phase during the beginning of the trial, with movement rates starting high while the tadpoles and fry adjusted to the training environment. After this initial period, movement rates remained steady (up to 2 hours of evaluation, data not shown).

These data demonstrate the system is effective for comparing the behavior of different organisms of varying shapes (for example, to compare nootropic compounds in multiple species to show conservation of effect across clades and rule out or identify species-specific effects).

While zebrafish have been shown to be good learners in classical conditioning experiments, learning and memory in Xenopus has been far less studied. Our learning trial was broken into three separate blocks: an initial preference phase, a training phase, and a testing phase. During initial preference evaluation, half of the dish was illuminated with low intensity red light and half with high intensity blue light. During training optimization experiments, we found that pairing wavelength and intensity as training stimuli resulted in more robust learning than using either individually. Every ten minutes, the pattern of light rotated 90° in a clockwise direction, for a total of 3 rotations over the course of the evaluation (with the lighting conditions at the end of the trial being the inverse of those at the beginning). Location of the tadpole was recorded 10 times a second (10 Hz capture rate) and data was averaged over 5 minutes for ease of analysis and presentation.

The training phase proceeded exactly as did the initial preference evaluation, with the exception of a 1.2 mA AC shock being delivered if the tadpole was in the low intensity red half of the dish, thus conditioning the organism to stay under the blue light. The light was rotated 90° clockwise every 10 minutes during the trial to prevent tadpoles remaining motionless in the red half of the dish without actually experiencing an electric shock, and thus only appearing to learn. During this phase, four identical training sessions of 30 minutes were executed, separated by 90 minute rest periods where the entire dish was illuminated with blue light. Following the final 90 minute rest session, tadpoles were tested for light preference with the exact same setup as the initial phase, where occupancy in neither the red nor in the blue half of the dish was punished.

Results showed that similar to the Xenopus/zebrafish comparison, tadpoles initially possess no preference for either low intensity red or high intensity blue light at the beginning of the trial (Fig. 7, pre-training). However, following four training sessions in which the red quadrants were punished, tadpoles showed a significant change in behavior, spending more time in the high intensity blue half of the dish (Fig. 7, post training, 2way repeated measure ANOVA P<0.001) of their time in the high intensity blue half of the dish. This behavior was not due to individuals remaining motionless during the trial as overall blue preference was maintained even after light rotation at 10 and 20 minutes respectively.

Contrary to our initial expectation of a characteristic ‘learning curve’ across time, we observed that tadpoles appear to avoid shock very quickly and efficiently. Examination of each of the 30 minute training sessions revealed that tadpoles spend near 100% of the time in the blue half of the dish (Fig. 7B). During first 5 minutes of the initial training session (analyzed in 30 second intervals), all individuals moved to non-punishing quadrants within the first 30 seconds (Fig. 7C, six representative individuals plotted for clarity), and this was reflected across all training periods.

Movement rates did not change as a result of training. During the pre-training period, individuals showed the previously documented “exploratory phase”, with a gradual decrease in movement rate as the trial proceeds (Fig. 7D, compare with 6D). After leveling out, it remained steady across training and post-training phases (Fig. 7D). For comparison, we also performed a sham training trial which proceeded exactly as above but in which no shocks were delivered. Movement rates between the sham and training trial were not significantly different (data not shown) suggesting our punishment regime per se did not significantly affect movement rates in Xenopus tadpoles.

These data, and the fact that no learning is observed when lights are not rotated, suggest the following description of tadpole learning dynamics in this kind of trial. When tadpoles are punished, they move until punishment ceases, then remain in one area while moving slowly until they receive another punishment. Given the speed at which tadpoles can move (they can circle the dish in 2–3 seconds), this occurs extremely rapidly. In the case of non-rotating light trials, individuals then remain relatively still throughout the entire training session and thus receive few punishing ‘experiences’. However, when the lights are rotated during a training trial, it forces tadpoles to occasionally occupy punishing quadrants throughout the trial, which increases the number of incidents in which individuals are presented with the shock-red light pairing, resulting in more robust learning results. Thus, in non-rotating-light trials, tadpoles are learning a “don't-move” behavior, and in post-training evaluations could simply stay in one quadrant even in the event of changing light conditions. In contrast, an association is clearly made in the training outlined above using rotating light: if tadpoles were simply learning ‘don't move’, the preference line would not stay significantly below 50% while light conditions reversed continuously.

These results are promising considering the minimal training period used in this example. It is likely that repeated training regimes over the multiple days would yield enhanced learning, as would pre-screening animals for ‘strong’ or ‘weak’ learners. Future efforts should test training regimes involving a variety of parameters such as position at the center or edge of the dish, moving at a greater or slower speed than baseline rates, or training to intensity of light rather than wavelength. Training to multiple parameters will likely give insight into the functional constraints of tadpole memory. Can a tadpole learn to not move under red light and avoid the edge under blue light? How quickly can preferences for red or blue light be acquired and reversed through training? Will tadpoles with artificially-expanded forebrains, duplicated CNS structures, increased proportion of serotonergic neurons, or ectopic eyes exhibit faster learning, better sensory acuity, or different baseline behavior than wild-type siblings? It is likely that such memory assays can be used as a powerful screening tool for nootropic compounds (cognitive enhancers). All of these questions can be addressed in a quantitative, non-biased manner using the automated training paradigms described above.

The current challenge of modern cognitive science is to understand the processes that span from developmental genetics to the information processing mechanisms that give rise to behavior and thought. The biomedical aspect of this program includes the search for useful neuromodulatory drugs, as well as the restoration of normal cognition as part of regenerative medicine targeting injuries of the CNS. Fundamental advances thus require the characterization of behavior in a variety of genetically and pharmacologically-modified organisms. Manual analyses of animal behavior places significant limitations on experimental progress. These restrictions include the limited number of animals that can realistically be analyzed by hand, the confounding experimenter effects inherent in manual handling and observation by different individuals, and the difficulty of documenting raw results completely enough to enable other groups to analyze all of the primary data and potentially uncover trends missed (or not even recorded) by the experimenters.

Indeed, these problems have been central to a number of controversies in neurobiology. For example, the lack of consensus on the learning ability of planarian flatworms was due in large part to the small sample sizes necessitated by the tedium of training worms by hand, as well as the inevitable but often important small differences in handling by different experimenters (observer bias, oversensitization from handling), inconsistencies in protocols and controls, and difficulties in making every aspect of the data available to other groups in the field [24], [61], [105], [113], [114], [115], [116], [117], [118], [119], [120], [121], [122], [123]. The use of an automated system would have allowed other labs to reproduce even complex behavioral experiments precisely, and analyze the data without bias.

Molecularly-tractable model organisms promise the greatest insight into cognitive function. Xenopus laevis larvae have been a popular behavioral system for the investigation of responses to light and gravity, and in individual behaviors and schooling [62], [124], [125], [126], [127], [128], [129]. Because of optical, developmental, and genetic accessibility of neurons and their embryonic precursor cells, zebrafish are an excellent system for investigating how neural circuits give rise to behavior [130], [131], [132]. They have been used to study circadian rhythms [133], prey tracking [134], social interactions [135], and vision [136]. Comparisons of behavior in wild-type and genetic mutant zebrafish have been initiated [130], [131], [134], [135], [136], [137], [138], [139], [140], as have analyses of drug effects on fish behavior, including ethanol [32], [130], [141], [142], [143]. One of the most exciting next frontiers is the synthetic modeling of how neural function gives rise to behavior [144], [145], [146], [147], [148], [149], and an automated paradigm for analysis will greatly speed up this effort. This is of particular relevance to non-rodent model organisms popular in neurobiological studies [150], [151], [152].

We describe the prototype of a modular, highly-flexible, second-generation system that allows quantitative characterization (behavioral and sensory phenotyping) as well as individual feedback (memory and learning studies) using several different modalities. It can be extended to work with almost any other animal type within the relevant size range. Our goal is to ensure that the prototype system becomes widely available and accessible to any lab wishing to perform quantitative behavioral analysis. We are working with engineering firms to (1) modify the design to achieve a polished, user-friendly, relatively bullet-proof, optimized version of this system that can be deployed readily outside our lab as a standard piece of equipment (including for use with students in educational institutions), and (2) set up a distribution channel within which several versions of this system, operating manuals, construction kits, and technical support can be obtained.

It is important to note that while the price for end-users (now that many engineering problems have been solved) will be reasonable (approximately the same as a laser microscopy system), the capabilities of our prototype system were constrained almost entirely by initial R&D costs. Thus, we reached no fundamental limits, and increasing performance along almost every dimension of this system can be readily achieved with further development. Extensions of this system may include: 1) placement of transparent mazes within dishes, and various haptic coatings on the dish surfaces, 2) use of smaller model systems (C. elegans), or tracing of specific behaviors (subtle tail twitches or specific kinds of motions) with higher-resolution cameras, 3) tracking animals by fluorescent tags and control of optogenetic reagents [153], [154], [155], 4) addition of microvibration as a stimulus modality, and 5) microfluidic ports for training to odors or exchanges of pharmacological compounds during experiments. The system is readily scalable, and core facilities would be able to obtain large units for high-throughput studies. A significant addition to be developed for this application would be automated animal loading. Solutions for automated dispersing of small organisms exist already, such as the COPAS system [156], and the next generation of screening robots will be integrated with such delivery platforms to enable large numbers of animals to be efficiently placed into the device without human intervention.