| A novel cross-modal hashing algorithm based on multimodal deep learning | |
| Alternative Title | A novel cross-modal hashing algorithm based on multimodal deep learning |
| Qu Wen1; Wang Daling1; Feng Shi1; Zhang Yifei1; Yu Ge1 | |
| 2017 | |
| Source Publication | SCIENCE CHINA-INFORMATION SCIENCES
 |
| ISSN | 1674-733X |
| Volume | 60Issue:9 |
| Abstract | With the growing popularity of multimodal data on the Web, cross-modal retrieval on large-scale multimedia databases has become an important research topic. Cross-modal retrieval methods based on hashing assume that there is a latent space shared by multimodal features. To model the relationship among heterogeneous data, most existing methods embed the data into a joint abstraction space by linear projections. However, these approaches are sensitive to noise in the data and are unable to make use of unlabeled data and multimodal data with missing values in real-world applications. To address these challenges, we proposed a novel multimodal deep-learning-based hash (MDLH) algorithm. In particular, MDLH uses a deep neural network to encode heterogeneous features into a compact common representation and learns the hash functions based on the common representation. The parameters of the whole model are fine-tuned in a supervised training stage. Experiments on two standard datasets show that the method achieves more effective results than other methods in cross-modal retrieval. |
| Other Abstract | With the growing popularity of multimodal data on the Web, cross-modal retrieval on large-scale multimedia databases has become an important research topic. Cross-modal retrieval methods based on hashing assume that there is a latent space shared by multimodal features. To model the relationship among heterogeneous data, most existing methods embed the data into a joint abstraction space by linear projections. However, these approaches are sensitive to noise in the data and are unable to make use of unlabeled data and multimodal data with missing values in real-world applications. To address these challenges, we proposed a novel multimodal deep-learning-based hash (MDLH) algorithm. In particular, MDLH uses a deep neural network to encode heterogeneous features into a compact common representation and learns the hash functions based on the common representation. The parameters of the whole model are fine-tuned in a supervised training stage. Experiments on two standard datasets show that the method achieves more effective results than other methods in cross-modal retrieval. |
| Keyword | hashing cross-modal retrieval cross-modal hashing multimodal data analysis deep learning |
| Indexed By | CSCD |
| Language | 英语 |
| Funding Project | [National Natural Science Foundation of China] ; [Fundamental Research Funds for the Central Universities of China] |
| CSCD ID | CSCD:6087845 |
| Citation statistics |
Cited Times:5[CSCD]
[CSCD Record]
|
| Document Type | 期刊论文 |
| Identifier | http://ir.imr.ac.cn/handle/321006/151634 |
| Collection | 中国科学院金属研究所 |
| Affiliation | 1.东北大学 2.中国科学院金属研究所 |
| Recommended Citation GB/T 7714 | Qu Wen,Wang Daling,Feng Shi,et al. A novel cross-modal hashing algorithm based on multimodal deep learning[J]. SCIENCE CHINA-INFORMATION SCIENCES,2017,60(9). |
| APA | Qu Wen,Wang Daling,Feng Shi,Zhang Yifei,&Yu Ge.(2017).A novel cross-modal hashing algorithm based on multimodal deep learning.SCIENCE CHINA-INFORMATION SCIENCES,60(9). |
| MLA | Qu Wen,et al."A novel cross-modal hashing algorithm based on multimodal deep learning".SCIENCE CHINA-INFORMATION SCIENCES 60.9(2017). |
| Files in This Item: | There are no files associated with this item. | |||||
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment