A novel cross-modal hashing algorithm based on multimodal deep learning

IMR OpenIR

	A novel cross-modal hashing algorithm based on multimodal deep learning
其他题名	A novel cross-modal hashing algorithm based on multimodal deep learning
	Qu Wen 1; Wang Daling 1; Feng Shi 1; Zhang Yifei 1; Yu Ge 1
	2017
发表期刊	SCIENCE CHINA-INFORMATION SCIENCES
ISSN	1674-733X
卷号	60 期号:9
摘要	With the growing popularity of multimodal data on the Web, cross-modal retrieval on large-scale multimedia databases has become an important research topic. Cross-modal retrieval methods based on hashing assume that there is a latent space shared by multimodal features. To model the relationship among heterogeneous data, most existing methods embed the data into a joint abstraction space by linear projections. However, these approaches are sensitive to noise in the data and are unable to make use of unlabeled data and multimodal data with missing values in real-world applications. To address these challenges, we proposed a novel multimodal deep-learning-based hash (MDLH) algorithm. In particular, MDLH uses a deep neural network to encode heterogeneous features into a compact common representation and learns the hash functions based on the common representation. The parameters of the whole model are fine-tuned in a supervised training stage. Experiments on two standard datasets show that the method achieves more effective results than other methods in cross-modal retrieval.
其他摘要	With the growing popularity of multimodal data on the Web, cross-modal retrieval on large-scale multimedia databases has become an important research topic. Cross-modal retrieval methods based on hashing assume that there is a latent space shared by multimodal features. To model the relationship among heterogeneous data, most existing methods embed the data into a joint abstraction space by linear projections. However, these approaches are sensitive to noise in the data and are unable to make use of unlabeled data and multimodal data with missing values in real-world applications. To address these challenges, we proposed a novel multimodal deep-learning-based hash (MDLH) algorithm. In particular, MDLH uses a deep neural network to encode heterogeneous features into a compact common representation and learns the hash functions based on the common representation. The parameters of the whole model are fine-tuned in a supervised training stage. Experiments on two standard datasets show that the method achieves more effective results than other methods in cross-modal retrieval.
关键词	hashing cross-modal retrieval cross-modal hashing multimodal data analysis deep learning
收录类别	CSCD
语种	英语
资助项目	[National Natural Science Foundation of China] ; [Fundamental Research Funds for the Central Universities of China]
CSCD记录号	CSCD:6087845
引用统计	被引频次：7[CSCD] [CSCD记录]
文献类型	期刊论文
条目标识符	http://ir.imr.ac.cn/handle/321006/151631
专题	中国科学院金属研究所
作者单位	1.东北大学 2.中国科学院金属研究所
推荐引用方式 GB/T 7714	Qu Wen,Wang Daling,Feng Shi,et al. A novel cross-modal hashing algorithm based on multimodal deep learning[J]. SCIENCE CHINA-INFORMATION SCIENCES,2017,60(9).
APA	Qu Wen,Wang Daling,Feng Shi,Zhang Yifei,&Yu Ge.(2017).A novel cross-modal hashing algorithm based on multimodal deep learning.SCIENCE CHINA-INFORMATION SCIENCES,60(9).
MLA	Qu Wen,et al."A novel cross-modal hashing algorithm based on multimodal deep learning".SCIENCE CHINA-INFORMATION SCIENCES 60.9(2017).