International organisation for standardisation organisation internationale de normalisation

CE2: Global descriptor

14.1.97.1.1.1.1.1.28m32261 Peking University Response to CE2: Improved Global Descriptor [Jie Lin, Ling-Yu Duan, Zhe Wang, Tiejun Huang, Wen Gao]

Integration of SCFV global descriptor into TM8, in particular working with the ALP detector adopted at the last meeting. TM8 is still using 14 parameters for a Fourier curve fitting mechanism adopted for pairwise comparison. As proposed during the last meeting, new training datasets balancing the number of 2D and 3D objects for PCA projection matrix and GMM model are used. 4K images were used for the training dataset, taken from Flickr, internal CADAL project dataset and some other 3D objects of pictures randomly selected from TM8 training datasets.

1 byte representation for the elements of PCA matrix and GMM tables integrated. Bit selection module, taken from the proposal of University of Surrey/Visualatoms presented at last meeting integrated into the solution (this costs slight increase of fixed memory occupancy to overall 21KB for the 256 configuration).

Single threshold adopted for pairwise matching.

Significant improvements achieved keeping 128 gaussian components for fair comparison to TM8: +1.45% TPR on average, 3.6 on 4-5 datasets. mAP results not reported but stated to drop around 0.3: full results and associated TM8 parameters to be presented during the week.

Further improvements were obtained introducing RootSIFT and 256 Gaussian components.

Overall performance was reported as a significant improvement (+1.33 %TPR and +%1.77mAP), with a slight drop in localization accuracy (~1%).

Details about data training were requested (e.g. how many images from Flickr, how many from CADAL, ...) and regarding the images that can be shared (from Flickr and TM8), which were asked to be provided by Wednesday.

An issue about rights for usage of the CADAL dataset had not been solved yet.

Conclusions: The full set of results on 128 configuration was asked to be disclosed during the week. Proposed changes in the text for the Study of CD were to be released by Wednesday.

The proposal was conditionally adopted into TM9, provided that the full training dataset and training tool would be available by one week after the meeting. The Flickr and TM8 images were to be released by Wednesday.

The group was willing to establish this collaborative CE2 to favor integration of presented technology in order to improve performance.

14.1.97.1.1.1.1.1.29m32315 Cross check of "Peking University Response to CE2: Improved Global Descriptor" m32261 [Emanuele Plebani, Danilo Pau]

Linux cross check; results O.K..

14.1.97.1.1.1.1.1.30m32212 CDVS: Cross-Check of Peking University’s Proposal m32261 [Zheng Liu, Bin Chen, Giovanni Cordara]

Windows cross check; results O.K..

14.1.97.1.1.1.1.1.31m32639 PKU Response to CE2: information on training dataset [Jie Lin, Ling-Yu Duan, Zhe Wang, Tiejun Huang, Wen Gao]

Discussion in Video plenary Wednesday:

The data set used for re-training is to be delivered within a week; otherwise, m32261 will not be adopted and will not be further considered in the CE work (i.e., the CE will only be on testing m32330 against the current technology in TM8).

14.1.97.1.1.1.1.1.32m32330 Improved RVD in TM8 - CE 2 Response from University of Surrey and Visual Atoms [Miroslaw Bober, Syed Husain, Stavros Paschalakis, Karol Wnukowicz]

Further investigation in a CE; either versus current TM8, or TM8+m32261.

Improvement of the RVD descriptor was presented (m30311) in Vienna, increasing the SIFT projection matrix size to 64 and increasing the codebook size to 256. RVD uses a single threshold for pair-wise matching. Memory allocation size is around 28K. Significant improvements were reportedly achieved: TPR +0.84%, +1.44% mAP, PTM 0.62%. Drop of localization ~1.7.

It was agreed that this would not be adopted if the issue with the m32261 training dataset and tools is resolved (provided within a week after the MPEG meeting ends).

This would be adopted into TM9 and Study text of CD otherwise.

14.1.97.1.1.1.1.1.33m32466 CDVS: crosscheck of m32330 [Massimo Balestri, Gianluca Francini, Skjalg Lepsoy]

Cross-check; results O.K.