International organisation for standardisation organisation internationale de normalisation

6.2Discussions about dataset clean-up

Discussions aobut dataset cleanup considered several options

• 
  Replacing the second image in non-matching pairs with:
  
  • 
    Distractor set
    
  • 
    Category 1-4
    
  • 
    Images from the same dataset.
    
  • 
    Removing all non-matching pairs.
    

The decision of the group is to replace the current list with images arbitrarily taken from the following datasets:

• 
  Business cards
  
• 
  Paintings
  
• 
  Video Frames
  
• 
  Newspapers
  
• 
  Buildings
  

Any error in the new list will be reported on the reflector during the AhG period between the current meeting and the 100th WG 11 meeting and possible decisions will be taken during the 100th meeting.

Gabriel Takacs will collect the new non matching pairs for category 5 and new bounding box annotations. He will prepare a package with all the current datasets (except from distracters sets) and upload the data on the TI Gforge repository.
For the distractor set, the same procedure reported in the evaluation framework document will be followed.
Regarding matching images in the distractor set – it was planned that for next meeting cycle we will use the distractor set as it is currently. Participants were encouraged in providing input contributions to next meeting reporting lists of incorrect images. These images will be removed from the distractor set by consensus of the 100^th meeting.