This challenge is part of The second International Symposium on 3DPVT (3D Data Processing, Visualization, and Transmission) which will be held on September 6 to 9, 2004 in the city of Thessaloniki, Greece. We will make available to the public three sets of up to 33 synchronized and calibrated image sequences that observe non-rigidly moving objects such as faces and hands in action. The goal is to make an accurate three-dimensional reconstruction of the shape and motion observed in the video sequences. The submitted reconstructions will be judged by a panel of established researchers and the winner will receive an award.

These datasets were captured by Patrick Baker and Jan Neumann from the Computer Vision Lab at the University of Maryland.The data sets consist of a collection of compressed tar files containing the individual frames of the image sequences in png format and the corresponding calibration data which was computed by Patrick Baker using a novel fully automatic multi-camera calibration method.

If you have any questions please email us at challenge@3d-photography.org. We also kindly request that you drop us a note when you download any of the data sets, so that we can judge the interest of the community.

The three data sets are as follows:

Sequence I: Patrick putting on his glasses (50 frames, 22 cameras 54fps, 9 cameras 6 fps)

Video summary: Windows Media or DIVX

Image Files (10 frames per tar file): 160-169, 170-179, 180-189, 190-199, 200-209, HighRes

Calibration Data: www.cfar.umd.edu/~pbaker/calibrate/3dpvt

Sequence II: Yiannis talking while rotating his head (80 frames, 22 cameras 54fps, 8 cameras 6 fps) )

Video summary: Windows Media or DIVX

Image Files (10 frames per tar file): 180-189, 190-199, 200-209, 210-219, 220-229, 230-239, 240-249, 250-259, 260-269, HighRes

Sequence III: Jan covering his face (70 frames, 22 cameras 54fps, 9 cameras 6 fps)

Video summary: Windows Media or DIVX

Image Files (10 frames per tar file): 170-179, 180-189, 190-199, 200-209, 210-219, 220-229, 230-239, HighRes

General Information about the data sets:

The filename of each image is constructed as:

sprintf(filename,"f%03d/c%02d.png",frameIdx,cameraIdx);

There are three types of cameras:

1. 11 gray-scale 644x484 image sequences captured by a Kodak ES-310 camera at 54 fps
2. 11 Bayer color-filter 644x484 image sequences captured by a Kodak ES-310 camera at 54 fps where each 2x2 macro-pixel is of the form
   

   Red	Green
   Green	Blue

3. 8 or 9 gray-scale 1024x768 image sequences captured by a Sony X-700 camera at 6 fps (HiRes files).

Things to note:

1. Since the sony cameras were captured at a lower frame rate, they only captured a view of the scene for every 9th frame captured by the Kodak cameras (frames where mod(framesIdx,9) = 8), thus
   • for sequence I there are 33 synchronized views of the object captured at frames 161,170, 179, ...,
   • for sequence II there are 32 synchronized views captured at frames 188, 197, 206, ...,
   • for sequence III there are 33 synchronized views captured at frames 170, 179, 188, ..., and
   • for all the other frames, there are 22 synchronized views.
2. for each camera background images are also available.
3. All the cameras are intrinsically and extrinsically calibrated. The calibration data and more information about the calibration process can be found at www.cfar.umd.edu/~pbaker/calibrate/3dpvt

    

The cameras were placed on the edges of a wooden octahedron that is placed on one of its triangular sides. The spatial arrangement can be seen here in the pdf file. T and B refer to the vertices of the triangles at the T(op) or B(ottom) of the octahedron.

As can be seen on most rungs pairs or triples of cameras are set up in a small-baseline stereo configuration.

The mapping between camera indices and the individual videos in the 6x6 video summaries above is as follows :

• gray scale kodak cameras columns 1 and 4
• sony cameras columns 2 and 5
• bayer filter kodak cameras columns 3 and 6