On clustering and retrieval of video shots

Clustering of video data is an important issue in video abstraction, browsing and retrieval. In this paper, we propose a two-level hierarchical clustering approach by aggregating shots with similar motion and color features. Motion features are computed directly from 2D tensor histograms, while color features are represented by 3D color histograms. Cluster validity analysis is further applied to automatically determine the number of clusters at each level. Video retrieval can then be done directly based on the result of clustering. The proposed approach is found to be useful particularly for sports games, where motion and color are important visual cues when searching and browsing the desired video shots. Since most games involve two teams, classification and retrieval of teams becomes an interesting topic. To achieve these goals, nevertheless, an initial as well as critical step is to isolate team players from background regions. Thus, we also introduce approach to segment foreground objects (players) prior to classification and retrieval.