IR Marks

The IR Marks video data set is a face motion data set of three video sequences. Prior to filming the three sequences, several points were drawn on the subject’s face using an infrared marking pen. Each video sequence was filmed simultaneously using 4 visible-light cameras (in which the drawn points are not visible) and 3 infrared-sensitive (IR) cameras (in which the drawn points are visible).

For several frames in each sequence, the point locations in all three IR cameras were hand-labeled in order to obtain the groundtruth 3D locations of each of the points. The data set includes the groundtruth 3D locations of these points, as well as their groundtruth 2D locations in the images from the main visible-light camera.

Citation and Additional Information

Please give appropriate acknowledgement when you use any portion of this data set.

The IR Marks video data set was introduced in the following journal paper. The paper, as well as the online Supplemental material for the paper, contain additional information about the IR Marks data set. Please cite this paper when you use any portion of the data set:

Tim K. Marks, John R. Hershey, Javier R. Movellan, “Tracking Motion, Deformation, and Texture Using Conditionally Gaussian Processes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 348-363, February, 2010.

Bibtex:
@article{Marks_TPAMI_2010,
author = {Tim K. Marks and John R. Hershey and Javier R. Movellan},
title = {Tracking Motion, Deformation, and Texture Using Conditionally Gaussian Processes},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {32},
number = {2},
year = {2010},
pages = {348–363},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2008.278},
publisher = {IEEE Computer Society},
}

You can download the paper here:

click to download Marks_Hershey_Movellan_2010.pdf

The online Supplemental Material for the paper includes more detailed information about the IR Marks video data set in Appendix II, as well as a demo video of the G-flow algorithm tracking the Talk2 video sequence.

You can download the supplemental material here:

click to download Marks_Hershey_Movellan_2010_supplemental.zip

Download the IR Marks video data set

There are two parts of the dataset available for download: The Basic Download (2.3 GB) and the Extra Data (6.9 GB). For most users, the Basic Download will be all that is necessary. The Basic Download includes all of the video data (including the groundtruth vertex locations) that were used in the paper (see Citation and Additional Information, above).

click to download the Basic Download (IR_Marks.zip)

click to download the Extra Data (IR_Marks_Extra_Data.zip)

The Basic Download contains:

• Training frames as filmed by the main visible-light camera, including groundtruth vertex locations:
• 3D groundtruth location (in mm) of each vertex in all of the training frames
• 2D groundtruth location (in image/pixel coordinates) of each vertex in all of the training frames
• Three test video sequences (called Emote, Talk1, and Talk2), from the main visible-light camera. All video frames are provided both in color and in grayscale
• For several selected frames of each test sequence from the main visible-light camera, groundtruth vertex locations are provided:
• 3D groundtruth location (in mm) of each vertex in these selected frames from each test sequence
• 2D groundtruth location (in image/pixel coordinates) of each vertex in these selected frames from each test sequence
• IR_Marks_README.pdf , which is a detailed explanation of the contents of both the Basic Download and the Extra Data

  The Extra Data includes:

• The training frames as filmed by the other three visible-light cameras
• The three test video sequences as filmed by the other three visible-light cameras
• Vertex locations that were hand-marked in the IR camera frames, which were used to compute groundtruth locations in the training frames and selected testing frames
• MATLAB code that computes the groundtruth vertex locations, in 3D and in the 2D image from the main visible-light camera, from the hand-marked locations in the IR frames
• Calibration checkerboard images, from all 7 cameras (3 IR cameras and 4 visible-light cameras), for 12 calibration frames.
• IR_Marks_README.pdf , which is a detailed explanation of the contents of both the Basic Download and the Extra Data.

  Contact
  Contact: Tim K. Marks