5.0 KiB
GDC_ATRIUM
Repository for robust, adaptive 3D person tracking and reidentification in busy areas. Please contact adipu@utexas.edu with questions. Property of AMRL - UT Austin.
Environment Setup
Your environment MUST have MMPOSE and the tracking_re_id ROS2 package properly installed Follow the instructions below to set it up
# wherever you want your workspace root to be (anywhere upstream of tracking_re_id):
colcon build --symlink-install
source install/setup.bash
# 3.10 is the latest supported by MMPOSE
conda create -n mmpose python=3.10 -y
conda activate mmpose
# find your max. cuda version
nvidia-smi
# We need torch 2.1.x for mmcv 2.1.0, so here's an example using cuda 12.1
# this is what was used on sentry laptop
pip install torch==2.1.2 torchvision==0.16.2 --index-url https://download.pytorch.org/whl/cu121
# mmpose dependencies
pip install -U openmim
mim install mmengine
mim install "mmcv==2.1.0"
mim install "mmdet==3.2.0"
mim install "mmpose==1.2.0"
pip install "numpy<2"
pip install "opencv-python==4.11.0.86"
This installation is referenced in each launchfile by finding the location of your conda base environment, then finding an environment named "mmpose" - if you use a different name, you must change each launch file.
Camera Driver / Calibration
Many nodes in the tracking_re_id package rely on the /stereo/left/camera_info and /stereo/right/camera_info topics publishing calibration data for the camera. This requires calibration to be performed, replacing left.yaml and right.yaml in tracking_re_id/calibration.
Visit https://docs.ros.org/en/jazzy/p/camera_calibration/doc/tutorial_stereo.html for instructions on running the calibration. The left and right yaml files always had camera_name=narrow_stereo/left and camera_name=narrow_stereo/right which have to be changed to stereo/left and stereo/right, respectively.
To run the camera driver after calibration has been conducted, simply execute ros2 launch tracking_re_id start_cameras_only.launch.py. Running colcon build again is NOT required, as you should have run it with --symlink-install previously.
The following must only be run after the camera driver is started and once mmpose is installed:
single_person_loc_node
This node forms the basis for 3D pose tracking methods, taking raw image feeds and camera_info from both cameras, generating joint keypoints using MMPOSE, projecting these points onto the image plane, then tracing rays from each camera's copy of the keypoint through the respective focal center to their intersection, the 3D keypoint.
ros2 launch tracking_re_id single_person_demo.launch.py will start a visualization of the keypoints and show the average normal distance and average coordinate relative to the left camera.
Other launchfiles run single_person_loc_node in headless mode, drawing upon the published 3D keypoints.
ground_plane_node
Listens to keypoints from single_person_loc_node, waiting for ankle keypoints that stay in a 3D location within a 5cm radius for more than 5 frames. Once at least 3 such keypoints are found with at least one off of a line connecting the other two by at least 25 cm, a plane is formed via a least-squares method. The plane is published as a large, disc-shaped marker, along with the keypoints it was based off of.
overlay_node
Provides visualization of the ground plane, transformed back into the camera feed and overlaid on the undistorted image along with the 3d keypoints
ros2 launch tracking_re_id full_pipeline.launch.py will launch single_person_loc_node, ground_plane_node, and overlay_node, allowing for visualization of the ground plane while people are walking through.
reid_utils and reid_node
Work together to deliver KeyRe-ID functionality (from KeyRe_ID_model.py) based on weights stored in tracking_re_id/weights. Currently, the pipeline uses the weights I finetuned from ViT-base on the iLIDSVID dataset.
The Re-ID system works on top of MMPOSE, utilizing generated keypoints to create attention heatmaps and use these to reidentify subjects.
Running ros2 launch reid_pipeline.launch.py will return a visualization of the live camera feed, along with reidentification bounding boxes (the confidence value on the first identification is always 1.0, the confidence on a new box with the same ID is the similarity between the current subject and the subject they were matched with, and the confidence on a new box with a different ID is the highest confidence KeyRe-ID had when attempting to match to an existing subject.)
The Re-ID implementation is based on KeyRe-ID:
@article{kim2025keyreid,
title = {KeyRe‑ID: Keypoint‑Guided Person Re‑Identification using Part‑Aware Representation in Videos},
author = {Jinseong Kim and Jeonghoon Song and Gyeongseon Baek and Byeongjoon Noh},
journal = {arXiv preprint arXiv:2507.07393},
year = {2025}
}