Image Annotation Result 图像标注结果

Some experiment results of image annotation according to the papers.

Result

　　　　　　 Corel-5K ESP Game IAPRTC-12 Method Year P R F1 N+ P R F1 N+ P R F1 N+ MBRM 2004 24 25 24.5 122 18 19 18.5 209 24 23 23.5 223 JEC 2008 27 32 29.3 139 22 25 23.4 224 28 29 28.5 250 Group Sparsity 2010 30 33 31.4 146 32 29 30.4 252 CNN-R 2015 32 41.3 36.1 166 44.5 28.5 34.7 248 49 31 38.0 272 FastTag 2013 32 43 36.7 166 46 22 29.8 247 47 26 33.5 280 TagProp(σML) 2009 33 42 37.0 160 39 27 31.9 239 46 35 39.8 266 2PKNN 2012 39 40 39.5 177 51 23 31.7 245 49 32 38.7 274 GLKNN 2015 36 47 40.8 184 41 36 38.3 282 34 31 32.4 255 SVM-DMBRM 2014 36 48 41.1 197 55 25 34.4 259 56 29 38.2 283 SKL-CRM 2014 39 46 42.2 184 41 26 31.8 248 47 32 38.1 274 KCCA-2PKNN 2014 42 46 43.9 179 59 30 39.8 259 KCCA 2015 39 53 44.9 184 30 36 32.7 252 38 39 38.5 273 2PKNN+ML 2012 44 46 45.0 191 53 27 35.8 252 54 37 43.9 278 NMF-KNN 2014 38 56 45.3 150 33 26 29.1 238 CCA-KNN 2015 42 52 46.5 201 46 36 40.4 260 45 38 41.2 278 context-RM-B 2015 61 24 34.4 242 61 20 30.1 234 SLED 2015 35 51 41.5 49.82 47.36 48.6 NSIDML 2016 44.12 51.76 47.76 194 49.8 29.5 37.05 253 56.9 36.5 45.21 282

Dataset

• Corel 5K
  Paper: P. Duygulu, K. Barnard, J. F. de Freitas, and D. A. Forsyth. Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In ECCV, 2002.

• ESP Game
  Paper: L. Von Ahn and L. Dabbish. Labeling images with a computer game. In SIGCHI Conference on Human Factors in Computing Systems, 2004.
  http://hunch.net/~learning/ESP-ImageSet.tar.gz

• IAPRTC-12
  http://www.imageclef.org/photodata

Large Scale Dataset

• NUS-WIDE
  Paper: Chua T S, Tang J, Hong R, et al. NUS-WIDE: a real-world web image database from National University of Singapore[C]//Proceedings of the ACM international conference on image and video retrieval. ACM, 2009: 48.
  http://lms.comp.nus.edu.sg/research/NUS-WIDE.htm

Dataset Corel 5K ESP Game IAPR TC-12 NUS-WIDE No. of images 5000 20770 19627 269648 (209347 annotated) No. of labels 260 268 291 81 Train images 4500 18689 17665 110K (not fixed) Test images 500 2081 1962 4K (not fixed) labels per image 3.4, 4, 5 4.7, 5, 15 5.7, 5, 23 2.4, 2 images per label 58.6, 22, 1004 326.7, 172, 4553 347.7, 153, 4999 5701.3, 1682 No. of labels < mean-freq 195 (75.0%) 201 (75.0%) 217 (74.6%)

(entry format: mean, median, maximum)

Features and annotations from INRIA

http://lear.inrialpes.fr/people/guillaumin/data.php

• gen_annotation.m
  input: files provided in the website
  output: train_annot.txt, test_annot.txt foreach dataset folder

  datasets = { 'corel5k', 'iaprtc12', 'espgame' };sets = { 'test' , 'train' };for db=1:length(datasets),    ds = datasets{db};    for s=1:length(sets),        str = sets{s};        list = textread([ds '/' ds '_' str '_list.txt'],'%s');        annot = logical(vec_read([ds '/' ds '_' str '_annot.hvecs']));        fid = fopen([ds '/' str '_list.txt'], 'w');        for i=1:length(list),            annotation = annot(i,:);            fprintf(fid, '%d', annot(1));            for j=2:length(annotation),                fprintf(fid, '\t%d', annot(j));            end            fprintf(fid, '\n');        end        fclose(fid);    endend

Reference

Method Year Conference Reference Paper MBRM 2004 CVPR Multiple bernoulli relevance models for image and video annotation JEC 2008 ECCV A new baseline for image annotation TagProp(σML) 2009 ICCV Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation Group Sparsity 2010 CVPR Automatic image annotation using group sparsity 2PKNN 2012 ECCV Image annotation using metric learning in semantic neighbourhoods 2PKNN+ML 2012 ECCV Image annotation using metric learning in semantic neighbourhoods FastTag 2013 ICML Fast image tagging NMF-KNN 2014 CVPR NMF-KNN: image annotation using weighted multi-view non-negative matrix factorization SVM-DMBRM 2014 ICMR A Hybrid Model for Automatic Image Annotation KCCA-2PKNN 2014 ICMR A cross-media model for automatic image annotation SKL-CRM 2014 MIR A sparse kernel relevance model for automatic image annotation context-RM-B 2015 CVPR Feature-Independent Context Estimation for Automatic Image Annotation CNN-R 2015 ICMR Automatic Image Annotation using Deep Learning Representations KCCA 2015 ICMR Automatic Image Annotation using Deep Learning Representations CCA-KNN 2015 ICMR Automatic Image Annotation using Deep Learning Representations GLKNN 2015 ICMR Graph Learning on K Nearest Neighbours for Automatic Image Annotation SLED 2015 J. TIP SLED: Semantic Label Embedding Dictionary Representation for Multilabel Image Annotation NSIDML 2016 J. VCIR Image distance metric learning based on neighborhood sets for automatic image annotation

Large Scale Dataset

Year Conference Reference Paper 2014 ICLR Deep Convolutional Ranking for Multilabel Image Annotation 2015 ICCV Love Thy Neighbors Image Annotation by Exploiting Image Metadata 2015 ICMR Large Scale Image Annotation via Deep Representation Learning and Tag Embedding Learning

Some reference links

Method Link TagProp(σML) http://lear.inrialpes.fr/people/guillaumin/code.php#tagprop Group Sparsity http://ranger.uta.edu/~huang/codes/annotation_corel.zip 2PKNN(+ML) http://cvit.iiit.ac.in/projects/imageAnnotation/ FastTag http://www.cse.wustl.edu/~mchen/ NMF-KNN http://crcv.ucf.edu/people/phd_students/mahdi/ SKL-CRM https://github.com/sjmoran/sklcrm