Upgrade & Performance Evaluation of Video Frame Ranking Toolbox

[ Up ]

 

FRT Upgrade

FRT is upgraded to:

  1. produce video summarization based on luminance (Y only). This feature is available for both basic and smoothing techniques of creating feature vector.
  2. to combine color and motion based ranking technique
  3. output feature vectors in a file. All types of feature vector can be stored in file (YUVBasic, YUVSmooth, YOnlyBasic, YOnlySmooth and Motion). The feature is implemented to help the analysis for comparative evaluation of different ranking techniques against user-defined ground truth.
  4. to split the input video into different clips and then produce the ranking for the entire video as summation of individual clips. The feature is implemented to evaluate the influence of number frames being processed on summarization quality. (Ref: Section 5.3 of the proposal)

VFRT library is updated to contain 3 command line programs. Various command line arguments are added to the earlier release of VFRT. This will help to evaluate the ranking result with different options and to perform user study. These programs can be built on Windows/Unix/Linux.

1. rankYUV

Purpose: To generate YUV based frame ranking

Usage Format: rankYUV [-b] [-s] [-y] [mpeg file name] [-v] [clips]

Option Description
-b Basic YUV
-s Basic YUV + smoothing
To use Y information alone for frame ranking (no color) use:
-by or -yb Basic Y
-sy or -ys Basic Y + smoothing
You must specify one of the above options
mpeg file name Complete path and name of the MPEG file. Please ensure "write" permission for the specified path 
-v

Generate feature vector file.

Feature vectors are stored in a separate file having following convention:

Filename - Same as MPEG file

File Extension - 

".bcv" for -b option (Basic Color Vector)

".scv" for -s option  (Smooth Color Vector)

".bv" for -by option  (Basic Vector)

".sv" for -sy option  (Smooth Vector)

Format of feature vector file is available here.

NOTE: This option cannot be used with clips.

clips

The mpeg file can be split into N clips. Each clip can be ranked independently. Ranking for the entire MPEG is then obtained by combining the results of individual clips. This option is to evaluate the influence of number of frames being processed on summarization quality.

 

For example, sport.mpg having 900 frames can be ranked as summary of 4 clips by command

 

rankYUV -s sport.mpg 0 200 500 700

 

Here,

clip1 - frame 0 to 200
clip2 - frame 200 to 500
clip3 - frame 500 to 700
clip4 - frame 700 to 899

NOTE: This option cannnot be used with -v.

 

Results of ranking are stored in file having following convention

 

Filename: Same as MPEG file

File Extension: 

"REL0" for -b option

"REL" for -s option

"yREL0" for -by option

"yREL" for -sy option

"sNREL0" for -b option with N clips

"sNREL" for -s option with N clips

"sNyREL0" for -by option with N clips

"sNyREL" for -sy option with N clips

2. rankMotion

Purpose: To generate motion based frame ranking

Usage Format: rankMotion [mpeg_stat file name] [-v]

Option Description
mpeg_stat file name Complete path and name of the file storing the output of mpeg_stat.

For example: To obtain motion based ranking for sports.mpg, we do,

1. mpeg_stat -quiet -block_info sports.stat sports.mpg > sports.txt

2. rankMotion sports.stat

The program uses sports.stat to get motion vectors and sports.txt to get information on macro block size. Please note that .stat and .txt files should have the same name as the MPEG file.
-v

Generate feature vector file.

Feature vectors are stored in a separate file having following convention:

Filename - Same as .stat file

File Extension - ".mfv"

Format of feature vector file is available here.

 

3. Combine

 

Purpose: To generate ranking based on combination of color and motion information.

Usage Format:  Combine [input file 1] [input file 2] [optional output]

Option Description
input file 1 Complete path and name of the file storing feature vectors obtained from rankYUV program.
input file 2 Complete path and name of the file storing feature vectors obtained from rankMotion program.
optional output Complete path and name of the file to store the result. This is optional. If not specified, please ensure that input file 1 and 2 differ only in file extension. The result is then store in a file with ".cmrel" extension. This extension is recognized by the Java video browser.

Format of feature vector file:

line#1 total number of feature vectors / frame numbers in MPEG (integer)

line#2 number of dimensions of each vector (integer)

line#3 space separated vector data for the first frame (double)

line#4 space separated vector data for the second frame (double)

....

and so on till the last frame.

 

For problems or questions regarding this web contact [Ayesh Mahajan].
Last updated: January 14, 2003.