Tracking Animals in Motion: SLEAP Article Published in Nature Methods

A timely collaboration between several laboratories across the Princeton community, including the Murthy Lab in the Princeton Neuroscience Institute and the Shaevitz Lab in the Lewis-Sigler Institute for Integrative Genomics, resulted in the creation of a novel open-source program called SLEAP. This program can simultaneously track and quantify the movement and behavior of multiple organisms at once. SLEAP is short for Social LEAP Estimates Animal Poses and builds on a previous program designed by the Murthy and Shaevitz Labs, called LEAP, which could only track one animal at a time. An article describing this software was recently published in the journal, Nature Methods.

SLEAP is a deep learning-based framework for motion capture and tracking multiple animals that are socially interacting. It is designed to be accessible to a non-technical biologist or experimentalist user base. The program works by dropping a video of the subject(s) into the software and defining the landmarks (which can be tails, heads, paws, wings etc.) across a few images to train the neural network (users usually need to train around 5-10 frames). Then, SLEAP will further train a neural network that can automatically identify landmarks and group them to individual animals over millions of frames. SLEAP will then convert that data into an accessible form for analysis, depending on what movements or behaviors the researcher would like to study. This program has become widely used among researchers; SLEAP has been downloaded over 40,000 times and is used by at least 70 laboratories across 58 universities and 15 countries. To design this program, members of the Murthy and Shaevitz Labs, among others, needed to solve three basic methodological obstacles.

Designing a program for simultaneous tracking of multiple animal movements requires finding solutions to three obstacles: 1) to localize the body part of interest across frames, 2) to group those body parts to the same animal across frames, and 3) to identify, track and differentiate individual animals and their landmarks across frames. SLEAP solves each of these obstacles using a smaller and more focused neural network than has been previously used to remain “light-weight”, faster, and more accessible.

Additionally, tracking programs can be designed in two ways. The first approach involves the network finding landmarks (or body parts) and then grouping these landmarks into animals. This approach is termed “bottom up”. Another approach to design an automated tracking system can be “top down”. In this approach, the network identifies an animal and then assigns the respective landmarks to that animal. SLEAP utilizes both “bottom up” and “top down” approaches to efficiently and accurately track animal movement.

SLEAP was spearheaded by Talmo Pereira, a PhD student co-advised by Mala Murthy and Joshua Shaevitz at Princeton and the first author of the Nature Methods article. Since graduating, Talmo is now a Salk Fellow at the Salk Institute for Biological Studies in La Jolla California, where he runs his own laboratory and continues to maintain SLEAP. Talmo’s new lab is also adding new features to SLEAP, such as 3D pose-tracking and a web-based extension to make SLEAP a more portable platform. This pivotal new feature would allow the program to be used without dedicated high-throughput computers and further increase the accessibility of the program to laboratories who don’t typically use such software to track animal behavior. To utilize SLEAP and keep up to date with new features, please visit the SLEAP website.

by Christos Suriano