Exploiting kernel-based feature weighting and instance clustering to transfer knowledge across domains

: Learning invariant features across domains is of vital importance to unsupervised domain adaptation, where classifiers trained on the training examples (source domain) need to adapt to a different set of test examples (target domain) in which no labeled examples are available. In this paper, we propose a novel approach to find the invariant features in the original space and transfer the knowledge across domains. We extract invariant features of input data by a kernel-based feature weighting approach, which exploits distribution difference and instance clustering to find desired features. The proposed method is called the kernel-based feature weighting (KFW) approach and benefits from the maximum mean discrepancy to measure the difference between domains. KFW uses condensed clusters in the reduced domains, the domains that do not contain variant features, to enhance the classification performance. Simultaneous use of feature weighting and instance clustering increases the adaptation and classification performance. Our approach automatically discovers the invariant features across domains and employs them to bridge between source and target domains. We demonstrate the effectiveness of our approach in the task of artificial and real world dataset examinations. Empirical results show that the proposed method outperforms other state-of-the-art methods on the standard transfer learning benchmark datasets.

PDF

___

[1] Long M, Wang J, Ding G, Sun J, Yu PS. Transfer joint matching for unsupervised domain adaptation. In: IEEE 2014 Computer Vision and Pattern Recognition (CVPR); 2427 June 2014; Columbus, OH, USA: IEEE. pp. 1410-1417.
[2] Gong B, Grauman K, Sha F. Learning kernels for unsupervised domain adaptation with applications to visual object recognition. Int J Comput Vision 2014; 109: 3-27.
[3] Lu J, Behbood V, Hao P, Zuo H, Xue S, Zhang G. Transfer learning using computational intelligence: a survey. Knowl-Based Syst 2015; 80: 14-23.
[4] Pan SJ, Yang Q. A survey on transfer learning. IEEE T Knowl Data En 2010; 22:1345-1359.
[5] Russell BC, Torralba A, Murphy KP, Freeman WT. LabelMe: a database and web-based tool for image annotation. Int J Comput Vision 2008; 77:157-173.
[6] Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. Imagenet: a large-scale hierarchical image database. In: IEEE 2009 Computer Vision and Pattern Recognition (CVPR); 2025 June 2009; Miami Beach, FL, USA: IEEE. pp. 248-255.
[7] Borgwardt KM, Gretton A, Rasch MJ, Kriegel HP, Sch¨olkopf B, Smola AJ. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 2006; 22: 49-57.
[8] Gopalan R, Li R, Chellappa R. Domain adaptation for object recognition: an unsupervised approach. In: IEEE 2011 International Conference on Computer Vision; 613 November 2011; Barcelona, Spain: IEEE. pp. 999-1006.
[9] Ben-David S, Blitzer J, Crammer K, Pereira F. Analysis of representations for domain adaptation. Adv Neur In 2007; 19: 137-144.
[10] Blitzer J, McDonald R, Pereira F. Domain adaptation with structural correspondence learning. In: Conference on empirical methods in natural language processing; 2223 July 2006; Sydney, Australia. pp. 120-128.
[11] Pan SJ, Kwok JT, Yang Q. Transfer learning via dimensionality reduction. In: Association for the Advancement of Artificial Intelligence (AAAI) Conference; 1317 July 2008; Chicago, IL, USA. pp. 677-682.
[12] Pan SJ, Tsang IW, Kwok JT, Yang Q. Domain adaptation via transfer component analysis. IEEE T Neural Networ 2011; 22: 199-210.
[13] Uguroglu S, Carbonell J. Feature selection for transfer learning. In: Machine Learning and Knowledge Discovery in Databases; 59 September 2011; Athens, Greece: Springer. pp. 430-442.
[14] Pan SJ, Ni X, Sun JT, Yang Q, Chen Z. Cross-domain sentiment classification via spectral feature alignment. In: Proceedings of the 19th international conference on World wide web; 2630 April 2010; Raleigh, NC, USA. pp. 751-760.
[15] Huang J, Gretton A, Borgwardt K, Karsten M, Sch¨olkopf B, Smola AJ. Correcting sample selection bias by unlabeled data. In: Advances in neural i305 TAHMORESNEZHAD and HASHEMI/Turk J Elec Eng & Comp Scinformation processing systems; 47 December 2006; Vancouver, BC, Canada; pp. 601-608.
[16] Duan L, Tsang IW, Xu D, Maybank SJ. Domain transfer svm for video concept detection. In: IEEE 2009 Conference on Computer Vision and Pattern Recognition; 2025 June 2009; Florida, USA: IEEE; pp. 1375-1381.
[17] Gong B, Shi Y, Sha F, Grauman K. Geodesic flow kernel for unsupervised domain adaptation. In: IEEE 2012 Conference on Computer Vision and Pattern Recognition; 1621 June 2012; Rhode Island, USA: IEEE; pp. 2066- 2073.
[18] Gopalan R, Li R, Chellappa R. Unsupervised adaptation across domain shifts by generating intermediate data representations. IEEE T Pattern Anal 2014; 36: 2288-2302.
[19] Saenko K, Kulis B, Fritz M, Darrell T. Adapting visual category models to new domains. In: European Conference on Computer Vision; 511 September 2010; Heraklion, Crete, Greece: Springer. pp. 213-226.
[20] Jiang J, Zhai C. Instance weighting for domain adaptation in nlp. ACL 2007; 7: 264-271.
[21] Zhong E, Fan W, Peng J, Zhang K, Ren J, Turaga D, Verscheure O. Cross domain distribution adaptation via kernel mapping. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 28 June1 July 2009; Paris, France; pp. 1027-1036.
[22] Duan L, Tsang IW, Xu D, Chua TS. Domain adaptation from multiple sources via auxiliary classifiers. In: Proceedings of the 26th Annual International Conference on Machine Learning; 14 18 June 2009; Montreal, Canada; pp. 289-296.
[23] Long M, Wang J, Ding G, Pan SJ, Yu PS. Adaptation regularization: a general framework for transfer learning. IEEE T Knowl Data En 2014; 26: 1076-1089.
[24] Bruzzone L, Marconcini M. Domain adaptation problems: A dasvm classification technique and a circular validation strategy. IEEE T Pattern Anal 2010; 32: 770-787.
[25] Satpal S, Sarawagi S. Domain adaptation of conditional probability models via feature subsetting. In: Knowledge Discovery in Databases 2007; 1721 September 2007; Warsaw, Poland; pp. 224-235.
[26] Si S, Tao D, Geng B. Bregman divergence-based regularization for transfer subspace learning. IEEE T Knowl Data En 2010; 22: 929-942.
[27] Jhuo IH, Liu D, Lee DT, Chang SF. Robust visual domain adaptation with low-rank reconstruction. In: IEEE 2012 Computer Vision and Pattern Recognition (CVPR); 1621 June 2012; Rhode Island, USA: IEEE; pp. 2168-2175.
[28] Qiu Q, Patel VM, Turaga P, Chellappa R. Domain adaptive dictionary learning. In: 12th European Conference on Computer Vision; 713 October 2012; Firenze, Italy; pp. 631-645.
[29] Roy SD, Mei T, Zeng W, Li S. Socialtransfer: cross-domain transfer learning from social streams for media applications. In: Proceedings of the 20th ACM International Conference on Multimedia; 29 October2 November 2012; Nara, Japan; pp. 649-658.
[30] Gretton A, Borgwardt KM, Rasch M, Sch¨olkopf B, Smola AJ. A kernel method for the two-sample-problem. In: Advances in Neural Information Processing Systems; 49 December; Vancouver, Canada; pp. 513-520.
[31] Baktashmotlagh M, Harandi MT, Lovell B, Salzmann M. Unsupervised domain adaptation by domain invariant projection. In: IEEE 2013 International Conference on Computer Vision; 18 December 2013; Sydney, Australia: IEEE. pp. 769-776.
[32] Grant M, Boyd S, Ye Y. CVX: Matlab software for disciplined convex programming. 2008.
[33] Shiraishi J, Katsuragawa S, Ikezoe J, Matsumoto T, Matsumoto T, Komatsu KI, Matsui M, Fujita H, Kodera Y, Doi K. Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists detection of pulmonary nodules. AJR Am J Roentgenol 2000; 174: 71-74.
[34] Koenderink JJ, van-Doorn AJ. Representation of local geometry in the visual system. Biol Cybern 1987; 55: 367-375.
[35] Dinh CV, Duin RPW, Piqueras-Salazar I, Loog M. FIDOS: a generalized Fisher based feature extraction method for domain shift. Pattern Recogn 2013; 46: 2510-2518.