This dataset provides human-teleoperated socially compliant navigation demonstrations.
Social navigation is the capability of an autonomous agent to navigate in a socially compliant manner such that it recognizes and reacts to the objectives of other navigating agents, at least somewhat adjusting its own path in response, while also projecting signals that can help the other agents reciprocate. Some examples of socially compliant navigation behavior include sticking to the right of the road, following a crowd, overtaking a crowd, etc.
The image above shows two socially compliant navigation strategies around human crowds: a) moving with traffic, and b) sticking to the right of the road.
SCAND contains 25 miles and 8.7 hours of robot driven trajectories through a variety of social environments around the University of Texas at Austin campus.
The image below shows the area within the UT Austin campus where data was collected by teleoperating the robots.
SCAND CONTENTS
- 138 robot-driven trajectories.
- 15 days of social navigation data on 2 robots: a wheeled Clearpath Jackal, and a legged Boston Dynamics Spot.
- Indoor and outdoor environments @ UT Austin campus.
- 2 highly crowded football game days (including a concert at the same time!)
- Mild to heavily crowded environments.
- 4 different human demonstrators.
Each SCAND trajectory includes: RGB Azure Kinect camera, velodyne pointclouds, stereo camera, visual odometry (Spot), wheel odometry (Jackal), leg positions (Spot), joystick commands issued by the demonstrator, and monocular cameras (Spot). All data is stored in the ROSBAG data format http://wiki.ros.org/Bags/Format.
For each trajectory the accompanying video contains a speed up version of the robot's camera view, intended to provide a quick overlook of the scene in a particular trajectory for the end user.
SCAND files follow the following file naming convention : {Demonstrator tag}_{Robot name}_{Start location}_{End location}_{Day}_{Month}_{Date}_{Trajectory number}.bag/.mp4/.avi
The wheeled Jackal and legged Spot robots used to collect the data are shown below along with the multi-modal sensors implemented in both.
The image below illustrates five example scenarios from SCAND showing the RGB image and accompanying LIDAR. The scenarios have tags indicating the characteristics present in the trajectories such as: street crossing, narrow doorway, navigating through large crowds, vehicle interaction and crossing stationary queue.
In SCAND, we provide twelve different labels of socially compliant navigation events, along an entire trajectory for all the trajectories in the dataset. To label the dataset, the trajectories were manually annotated by monitoring the camera information. The labels are included in the AB_README.txt file alongside their frequency of appearance and in the metadata corresponding to each file published in this dataset. Note that the labels provided are coarse because they are not associated with a timestamp.
Detailed information about the trajectories, the human demonstrators, the robots, the sensors, and the software used to collect and process the data is also included in the AB_README.txt file published along with the data.
In Related Materials we point to software and hardware resources utilized in this project.
DATA USAGE EXAMPLES
In Related Publications, users can find references to publications describing this study including examples of experiments conducted with the dataset.
The objective behind the SCAND dataset is to provide a rich set of socially compliant robot navigation demonstrations. An imitative policy was trained using the behavior cloning algorithm. Below is an example figure of how the behavior cloned policy trained using SCAND was tested using real-world deployments in a human participant study. Results were perceived to be socially compliant in comparison to the classical move_base navigation stack.
This dataset can be used to develop autonomous mobile robots that can navigate within human crowds in a socially compliant manner, and to analyze human reactions and behaviors in the presence of mobile robots of different morphologies.
DOWNLOADING LARGE DATASETS
The entire SCAND is ~ 400 gig of size. A few individual rosbag files are larger than 4 gig. These characteristics present limitations for downloading the entire dataset as well as individual large files through a web interface. Users that want to download bulks of data and have problems downloading large files can send an email to maria@tacc.utexas.edu or to haresh.miriyala@utexas.edu requesting other types of access to the dataset. In the email include your institution and your download needs.
