A second-generation device for automated training and quantitative behavior analyses of molecularly-tractable model organisms
A deep understanding of cognitive processes requires functional, quantitative analyses of the steps leading from genetics and the development of nervous system structure to behavior. Molecularly-tractable model systems such as Xenopus laevis and planaria offer an unprecedented opportunity to dissect the mechanisms determining the complex structure of the brain and CNS. A standardized platform that facilitated quantitative analysis of behavior would make a significant impact on evolutionary ethology, neuropharmacology, and cognitive science. While some animal tracking systems exist, the available systems do not allow automated training (feedback to individual subjects in real time, which is necessary for operant conditioning assays). The lack of standardization in the field, and the numerous technical challenges that face the development of a versatile system with the necessary capabilities, comprise a significant barrier keeping molecular developmental biology labs from integrating behavior analysis endpoints into their pharmacological and genetic perturbations. Here we report the development of a second-generation system that is a highly flexible, powerful machine vision and environmental control platform. In order to enable multidisciplinary studies aimed at understanding the roles of genes in brain function and behavior, and aid other laboratories that do not have the facilities to undergo complex engineering development, we describe the device and the problems that it overcomes. We also present sample data using frog tadpoles and flatworms to illustrate its use. Having solved significant engineering challenges in its construction, the resulting design is a relatively inexpensive instrument of wide relevance for several fields, and will accelerate interdisciplinary discovery in pharmacology, neurobiology, regenerative medicine, and cognitive science.
Fundamental understanding of the mechanisms of cognition, behavior, and memory require the synthesis of insights from genetics, developmental neuroscience, and ethology in molecularly-tractable model organisms. While some groups are beginning to add analyses of cognitive endpoints to their pharmacological and genetic perturbations [1], [2], [3], [4], a significant barrier prevents the ready investigation of the causal chain leading from genetics, through the embryonic establishment of CNS structure, to behavior. Similarly, developing the next generation of nootropic drugs (e.g., for enhancement of intelligence or memory) requires the introduction of high-throughput screening technologies beyond single-cell in vitro systems or automated detection of behavioral defects.
Performing behavior experiments in a fully automated, computer-controlled, quantitative platform has several advantages. First, bias and experimenter effects inherent in manual procedures are minimized, allowing more consistent, quantitative, rigorous behavioral analyses [5]. Second, it becomes feasible to train and quantify the behavior of large numbers of individuals simultaneously. Third, automation allows for essential control conditions such as yoked controls, in which control animals receive rewards and punishments based on the behavior of other animals being trained, and ensures that consistent training protocols can be maintained for as long as needed (e.g., continually across multiple days, ensuring more robust learning). Fourth, it makes it possible for investigators at distant locations to reexamine experiments using identical training protocols in similar environment, increasing the cross-compatibility of datasets and validity of analyses.
The difficulties of ensuring accessible, standardized, quantitative behavior analysis in labs specializing in molecular genetics or developmental neurobiology is a significant barrier to progress in many fields. While automated analysis of specific metrics in mouse behavior is now routine [6], [7], [8], [9], [10], [11], [12], some of the most exciting progress in linking cell dynamics to cognitive functions are coming from systems such as zebrafish, Drosophila, C. elegans, and Xenopus laevis. A few systems have been developed for small species that are amenable to screening and large population studies such as crustaceans [13], zebrafish [14], [15], [16], Drosophila [17], C. elegans [18], [19], and planaria [20], [21], [22], [23], [24], [25], [26]. The development of unique, highly-specialized devices has had limited impact on labs without expertise in engineering, access to an electronics facility, and programming knowledge, since reproduction and adaptation of an existing system from another group can present formidable challenges.
Several commercial efforts have attempted to fill the gap. The most popular off-the-shelf solutions for automated tracking systems include Ethovision (Noldus Information Technology) and Videotrack/Zebralab (Viewpoint Life Sciences). Both systems offer automated behavior analysis from video feeds or stored video files and can track multiple organisms, typically from a single camera imaging a number of individual dishes or 96 well plates. Viewpoint software has been used to study circadian systems and sleep [27], [28] as well as neuroactive drugs and addiction [29], [30] in zebrafish. Software from Noldus has been used both molecularly, to study alcohol effects and anti epileptic drugs [31], [32], [33], as well as ecologically to examine how fish respond to allopatric/sympatric/animated predators, conspecifics, or alarm substances [34], [35], [36], [37] and has produced similar results to standard manual recording procedures [38].
A variety of custom software has also been developed for specific tasks, including tracking multiple organisms that cross paths or following individuals in three-dimensional space [39], [40]. However, all existing commercially-available systems lack a key feature: the ability to perform automated training and learning tasks in real time. To address a more complete repertoire of questions about behavior (as well as perform screens of mutations or drugs that affect learning and memory), it is necessary to be able to perform operant conditioning as well as simpler associative learning. Operant (instrumental) conditioning requires that a system not only be able to track multiple individuals in parallel but must also give immediate individual feedback to each organism based on behavior. For example, if lights or shock are used as training stimuli in a task where an animal is taught to avoid a moving light or stay in a particular region of the dish, they must be executed in real time in response to the potentially different actions of each individual, and the stimuli must be insulated from all surrounding members undergoing training. Table 1 lists the rich variety of behavioral paradigms and illustrates which are and are not possible using current commercial systems.
While salesmen generally tell customers that, for example, “shock effectors could easily be added” to their system, the reality is that numerous engineering, fabrication, and software issues must be overcome in order to adapt an existing tracking system and enable it to give positive and negative feedback to each animal in real time. Moreover, the various requirements are highly interdependent, and multiple cost-feature tradeoffs have to be managed. It is difficult for an investigator who wants to perform behavioral experiments to test numerous choices of cameras, lenses, electrodes, and other components. Creating individual animal chambers (amenable to differential light levels and spatially-homogenous shock with no “special points” due to electrodes breaking radial symmetry of the dish) and modifying software to control them at high speeds in real time is a very difficult task.
Our group sought to investigate the effects of duplication of brain and CNS structures on behavior [41], [42]. Key requirements for such research programs include the ability to: (1) characterize multiple animals in any one run (for statistical power despite individual variability), (2) provide differential environmental cues and feedback (reward/punishment) to individual animals in real time (enable analysis of memory and learning, which proceeds differently for each animal), (3) gather separate behavior data timelines on each animal (4) afford a high level of automation and rich data acquisition (to provide comprehensive analysis while minimizing experimenter effects and optimizing efficiency and reproducibility of protocols among labs), (5) be usable with several model organisms, (6) allow a large parameter space of possible training paradigms and stimuli covering several different sensory modalities (flexibility), and (7) acquire and use the system rapidly, without a long period of expensive custom modification by each lab (affordability and turn-key operation). Since no satisfactory system could be commercially obtained, we addressed these challenges to produce a system [43] with the following properties.
The platform was to contain individual “Skinner chambers” sized to fit standard Petri dishes, within which single animals could be provided with light and mild electric shock stimuli. It must allow for the programming of individualized combinations of environmental cues and feedback (reward/punishment) in real time to accommodate different rates of memory and learning among animals within a sample group. The system should be modular, flexible, convenient, and powerful, allowing use by operators without computer programming skills, while using off-the-shelf components and costing end-users no more than a mid-range fluorescence microscope system. Its design should be flexible enough for the convenient inclusion of essential control conditions such as yoked controls, in which control animals receive rewards and punishments based on the behavior of other animals being trained. Primary data must be recorded over long time periods and be available for subsequent review, analysis and transfer to other laboratories. It was important to incorporate scaleability to increase throughput, expandability to include other model organisms and additional assays (e.g., maze studies), and ease of use and maintenance. Our prototype was tested in two species specifically chosen for their wide-spread use in several fields, amenability to molecular-genetic, developmental, pharmacological, and behavioral analyses, and different physical and optical properties (to maximally stretch the applicability and flexibility of the system).
Free-living flatworms, planarians, can regenerate a whole worm from only a small section of the adult [42], [44], [45]. Besides regenerative biology, they are also a popular system for the study of cancer [46], [47], the neuropharmacology of drug addiction [48], [49], [50], [51], [52], [53], and memory and learning [23], [54], [55], [56], [57], [58], [59], [60]. They are a critical breakthrough in the evolution of the animal body plan, representing the “first” organism to have both bilateral symmetry and a centralized brain with true synaptic transmission [61]. Planarians exhibit much of the complexity of vertebrate systems, including a well-differentiated nervous system (with most of the same neurotransmitters as human brains), eyes, brain, three body layers, and bilateral symmetry. Thus, planaria are a unique model organism for studying the mechanisms that underlie the regeneration of a functional nervous system and the restoration of cognitive structures after injury by adult stem cells.
The tadpoles of the African clawed frog Xenopus possess a complex CNS and exhibit rich social behaviors (schooling), kin recognition, and the ability to modulate growth rate based on visual appraisal of conspecific density [62], [63]. While some reports have noted difficulty when training tadpoles manually [64], a number of species have been shown to learn efficiently in simple paradigms using electric shock [65], [66], [67], [68], [69], [70], [71], vibrations [72], [73], [74], odor avoidance [75], and conditioned place preference [76]. Molecular manipulations currently allow the study of Xenopus larvae with extra eyes [77], [78], altered brain compartment sizing along anterior-posterior and medio-lateral axes [79], [80], [81], reversed laterality [82], [83], multiple brains [41], [84], [85], hyper-innervated muscles, and altered complements of neurotransmitter receptors throughout the CNS [86], [87], [88]. Given their sensory and behavioral repertoires, learning capabilities, and prominent role as a model system in developmental biology, Xenopus larvae are an excellent subject for systems biology approaches to cognitive function. An automated system, increasing both the statistical power and the number of different kinds of assays to be tested, will be essential to establishing a paradigm to ask in a rigorous and quantitative way how the brain of various model species handles alterations in somatic and sensory organs, as well as changes in nervous system structure.
Comparatively more is known about behavior, learning, and memory in the zebrafish Danio rario. Fry and adult fish have been studied in a variety of ecological contexts including shoaling, predation, feeding, mate choice, and social transmission of information through release of alarm pheromone [34], [35], [36], [37], [89], [90], [91], [92], [93]. In addition, this organism is positioned at the forefront of molecular inquiries in the field of human health given its rich history of forward and reverse genetic approaches, as well as its amenability to pharmaceutical screening in 96-well plates [31], [32], [94], [95]. Thus, zebafish are a well-established model system for investigating the basis of of addiction, memory consolidation, eye disease, and sleep [27], [28], [32], [33], [94], [96], [97], [98], [99], [100], [101], [102]. Combining the ability to generate hundreds of mutant animals with a device capable of automating learning and memory assays is a particularly attractive match; it has the potential to generate huge amounts of data on the genetic mechanisms of normal cognitive phenomena as well as disorders. In addition, comparing data between frog and fish (within the same apparatus under identical conditions) offers a powerful “evo-devo” perspective in which conservation and divergence of results between vertebrates can be studied.
We built a system that can be readily applied to studies in Xenopus, planaria, zebrafish, and similar model organisms in any laboratory. Here we describe this platform and present sample data illustrating its use.
The automated behavior machine consists of a multi-channel experiment environment that is comprised of modules, with each module containing three experimental channels. The system is connected to a single PC running Microsoft Windows™, and the top-level system schematic is shown in Fig. 1A. A single channel is fundamentally a Skinner Chamber, in which one animal receives stimuli, is observed, and is given feedback in the form of light and/or weak electric shock. Our current system has four banks of 3 modules (12 units total), although additional banks can be readily added for higher throughput. The modules are mounted in an aluminum frame, bolted in place, and raised off the ground for ease of access. Each bank of channels contains three Petri-Dish holders (designed to snugly fit standard 6 cm Petri Dishes), three Machine Vision Cameras (Insight-Micro 1400, Cognex Corporation, Natick MA) and three channels of control electronics (Fig. 1B,C).
Each Petri-dish is illuminated by an Illumination Control Module (ICM) that sits 8 cm above the dish, mounted to the top of the Illumination Head, and provides lighting to the Petri-dish in four separate quadrants. The ICM provides blue and red illumination through high-brightness Light Emitting Diodes (LEDs, which allows bright light with minimal heat output, Osram Semiconductors, blue LED; 470 nm, red LED, 635 nm). Between the petri dish and the illumination control module is a “Diffuser/Divider Assembly” attached to the bottom of the Illumination Head. The assembly houses a diffuser plate which ensures even light distribution throughout the dish and supports two stainless steel vanes mounted at right angles to each other to allow independent light status in each of the quadrants of the dish and minimize the leakage of light from neighboring quadrants within the same Petri dish (Fig. 1D). The Petri dish holder insert also contains electrodes (see below). Much attention was given to optical and electrical separation of dishes from each other, to ensure that each animal perceived only the stimuli intended for it. Thus, the dishes have a high degree of isolation with respect to stray light and electromagnetic interference to ensure that the light and shock conditions in one dish do not impact the animals in adjacent dishes. Similarly, high-performance electronics ensure that the timing between behavior and outcome was as small as possible and uniform among all of the dishes (synchronization). A detailed spec of all of the tolerance limits and performance characteristics of the device is given in Figure S1.
The workflow is shown in Fig. 1E. In a front-end application running on a PC (see Figure 2 for a sample of the graphical user interface), the user describes a set of stimuli and a set of outcomes that will occur if the animal behaves in specific ways. The user interface is flexible, so that almost any conceivable relationship between lights and position of the animal can cause a change in light conditions or a mild electric shock. Thus for example, the user can select for a single lit quadrant out of four, and shock the animal if it does not stay within it; if the lit quadrant moves every so often, the animal is trained continually across days, with no intervention on the part of the experimenter, to follow the moving light. Pauses, yoked controls, and many complex variants of this can be accommodated. Samples of types of trials that can be performed are in Table 1.
Once the parameters of the trial are set and the animals loaded into the machine, the trial begins. Its progress can be monitored on the PC, but it runs unattended, continuously providing stimuli as instructed, observing the position of the animals, changing the light/shock as needed, and recording all of the data in a logfile. An email is sent at the completion of the trial. At this point, the dataset is processed using a custom Excel™ script, which identifies any irregularities (e.g., instances of failed tracking) and produces summary statistics on all of the main characteristics of the experiment. The dataset also contains tracking information as QuickTime movies (so that specific behaviors in any channel can be re-analyzed by hand), as well as static curiosity maps (pseudocolored occupancy plots showing at a glance where each animal spent most of its time during an entire trial).
Software allows light to be controlled in individual quadrants (Fig. 3A shows the efficiency of the top-mounted divider in establishing distinct light and dark quadrants). Red light is always used as background illumination and is typically set homogonously between all quadrants, while blue lights are turned on as a training or punishment stimulus. A red filter (632 nm narrowband) rests just above each camera effectively removing any blue light from the video feed, even though it is perceived by the animals clearly. This allows the camera to be sensitive to subtle changes in shadow (e.g., a planarian moving along the dish's edge) despite large variations in overall light levels perceived by the animals in different quadrants and dishes. The use of two different colors was driven by the need of the camera to be able to see animals without providing an unpleasant environment for organisms that are negatively phototactic. For example, planaria generally avoid light [103], but have very low sensitivity to light in the red range of the spectrum [104].
Both blue and red lights can vary in intensity from completely off to full brightness in 10 even steps, and the blue lights can be controlled so as to vary as a result of animal behavior (for example, have the blue lights on at a weak intensity when the organism is at the center of the dish, and become brighter when it nears the edge). This allows gradual response in behavior shaping experiments. In addition, the software can cause the lighting conditions to “rotate” 90° or 180° at any interval set by the user. This functionality allows (for example) “chase the light” experiments, where the organism must continually follow a moving quadrant of light to avoid punishment.
Electric shock can be applied to individual dishes based on organism position or movement rate over a given period (the latter is designed to facilitate “keep moving” trials). Each plastic insert contains a set of 6 iridium oxide-coated titanium electrodes extending the entire height of the insert (Fig. 3A'). When a shock command is issued by the device, two adjacent electrodes act as a source for the current while the two opposite electrodes act as sinks. Every 8 ms, the sources rotate one electrode to the right, meaning that after 48 ms the source has rotated around the entire dish (this occurs “virtually” via solid-state electronics – no actual movement takes place). The use of multiple electrodes (not simply 2 poles) and the rotation scheme was designed to maximize the homogeneity of the field (Figure S2 contains the models and data needed to perform this analysis on electrode configurations). The electric field produced by a pair of electrodes (used in most such studies in the past) is anisotropic, introducing significant variability in the perceived strength of the aversive stimulus depending on where the animal happens to be. Our scheme minimizes the ‘dead zones’ and ‘hot zones’ (Fig. 3D). Interestingly, some organisms including planaria respond differently depending upon the directionality and polarity of current relative to their own orientation [105]. The rotating scheme ensures that, even within short shock periods, the field is applied across multiple angles ensuring efficient punishment regardless of where the animal is or which way it is facing. The electronics also precisely match the cycles of alternating current flow during rotation, ensuring that it is net balanced at the end of each shock, thus avoiding accumulation of ionic gradients in the dish (e.g., excess of positive or negative charges at any electrode vicinity).
Current can range from 0.2 mA to 20 mA in 0.2 mA increments, and the duration can be any number divisible by 48 ms (the amount of time it takes for the source electrode to rotate completely around the dish). All tadpole training used a 1.4-1.6 mA electric shock, as this was the minimum current which induced a behavioral response. All shocks are delivered as a square waveform with a frequency selected by the user between 10 and 1000 Hz. In addition, punishment can be delayed (for example: when the lights come on, wait 5 seconds and then issue a shock) or inhibited for a given interval (shock under blue quadrants, but after shocking wait 10 seconds before shocking again so the organism has time to move to a different location before the next punishment).
This system offers consistent, balanced shocks to individual dishes based on behavior, with flexible control over duration, intensity, and AC frequency of electric shock. Parameters can easily be adjusted, making the system usable for many aquatic species (current control ensures that shock can be kept constant despite variability in salt concentration of the media). Many different shock paradigms (e.g., continuous weak shock vs. short strong shock) can be tested to determine which one results in the most robust learning for a particular type of experiment. Rounded inserts provide a continuous smooth surface, reducing edges in which organisms can become caught, and easing the cleaning process.
The system itself runs an embedded Linux operating system that communicates with the front-end PC through an Ethernet connection. Each of the modules continuously produce message packets containing the X,Y coordinates of each animal (the centroid of its shape) in each dish. These packets are received by a thread within the front-end application, which logs the data, makes decisions based on the position of each animal and the defined trial type, and issues commands to control light and shock as needed. This process proceeds at a rate of up to 25 Hz (25 complete cycles of observe-decide-punish per second), fast enough to train rapidly-moving organisms like Xenopus.
Tracking of animal position and speed from the camera images proceeds through 3 stages. The first stage (occurring in firmware on-board the processor at each dish unit) analyzes all pixels in the complete image from the camera, one frame at a time. The second stage performs quality control, rejecting frames that may be erroneous according to several criteria. The third stage calculates the final best estimate of animal position and speed, thus producing the instantaneous coordinate values of the animal in the current frame. It then combines this latest estimate with the history of coordinate values from preceding frames, through double exponential smoothing, to derive the best estimate of position and vector velocity. The second and third stages are performed in the main control code running on the front-end PC.
The camera (Insight-Micro 1400 from Cognex Corporation) contains an internal Digital Signal Processor that executes the image processing algorithms - a sequence of image processing operations to acquire, enhance and analyze the raw image from the camera as follows. The background is removed, leaving the organism as image “blobs” (arbitrary clusters of bright pixels representing an object found from subsequent processing). This system contains an intelligent background update algorithm that automatically captures the backgrounds of empty quadrants as the organism moves about the image. The contrast of the resulting image is adjusted to enhance the brightness level of the organism blobs while generally reducing the brightness of background noise. This system also contains an intelligent contrast adjustment algorithm that uses a “histogram stretch” technique to select an optimal amount of pixels to make bright based on the typical size of the selected organism. A binary threshold is applied to the grayscale result of the contrast adjustment, producing a black & white (binary) image that shows blobs representing the outline of the organisms. The system selects threshold level such that the threshold is higher than the level of background noise. The level is selected based on the statistics of the grayscale image input. A blob selection algorithm is applied to binary result of the threshold operation. This selects only continuous groups of pixels that are above a certain area, and ignores small groups of pixels that are typically noise artifacts. The resulting blobs are sorted and the resulting geometrical parameters (location, size, perimeter, etc) are placed as message packets on the TCP/IP network.
Standard 60×15 mm petri-dishes (Fisher Scientific) are inserted into each of the channel depressions and secured by the electrode insert, clamped mechanically at opposite ends. Fresh media is added to the individual channels (13 ml for Xenopus and Danio, 11 ml for planaria), after which each bank of channels is closed and locked; upon completion, background acquisition is initiated by the training software. Following background acquisition, each bank of channels is opened and an individual animal is placed into the center of each dish. When all channels are loaded, the illumination heads are again locked in place and accurate tracking of each animal is inspected by eye using a direct video feed (observed on the monitor of the front-end PC), after which the experiment is executed by the training program.
Data for each trial are saved to a single log file in text format, containing positional, lighting, and punishment information for each channel at the specified frame rate. Log files are first processed into individual excel spreadsheets for each channel (animal) in an easy-to-read format, and then loaded into a custom Excel workbook which automates data analysis (e.g, average time in each quadrant by the animal, average speed, proportion of time punished, time spent at the edge vs. center of the dish, and area of the dish explored).
The workbook also calculates “time-segment” data to the user's specification. This time segment function bins the data into any interval specified by the user. For example, it is possible to produce the average movement rate or quadrant location of the organism in five minute, ten minute, or thirty minute blocks to look at changes in behavior over time. These statistics are useful to compare a number of variables including rates of learning, quenching of memory, and exploration/habituation to the testing environment. Time-segment analysis also allows the user to make adjustments based on the organism being tested; the same workbook could be used to evaluate fast moving tadpoles in five-minute increments or slow moving planaria in one-hour blocks. In addition, the time-segment function greatly aids in re-analysis of data. If it is necessary to re-examine particular time periods of a trial in greater detail, the same workbook can be used to generate a new output without the need of creating, copying, and pasting new macros into multiple excel files, greatly reducing the time required for analysis.
Colonies of planaria, Dugesia japonica and Schmidtea mediterranea, were stored in 1 L rectangular glass containers, filled with Poland Springtm natural spring water for D. japonica and 1X Montjuïch salts [106] for S. mediterranea. Worms were kept on a 10 h/14 h light/dark cycle at 16–17°C. Planarians were fed once per week with organic beef liver and the water was changed three times a week. Planarian were starved a week before use in order to increase activity and to inhibit spontaneous fission. Xenopus laevis larvae and Danio rerio fry were raised in 100×20 mm petri dishes (Fisher Scientific) at 18° and 24°C respectively, under a 12 h/12 h light/dark cycle and fed daily. Planarian (invertebrate) experiments do not require animal committee approval. All vertebrate experiments were conducted in accordance to accepted NIH and university guidelines and approved protocols (Tufts IACUC protocol number M2008-08).
We first identified the functional needs of this system by careful analysis of typical projects in the field, as well as the biggest roadblocks keeping behavioral analysis out of reach of typical molecular biology/development laboratories. We examined commercially-available systems and determined their limitations and capabilities. We worked closely with a local engineering firm, Wireless Techniques, to develop the necessary software, hardware, optics, electronics, and programming components. It was crucial to use one single entity for the construction of the device; although the individual subsystems can be outsourced, the necessary tight integration cannot be achieved unless the whole system is designed to work as a coherent platform from the outset.