Dataset Construction and Deep Learning Models for Cross-Modal Retrieval for Text-Color Image in SNS 


Vol. 44,  No. 9, pp. 1742-1753, Sep.  2019
10.7840/kics.2019.44.9.1742


PDF
  Abstract

Recently, various types of large dataset give the great help for the deveopment of AI technology as the same way as ImageNet does for the computer vision area. Unfortunately, however, there is no such large dataset for cross-modal retrieval between the text of Korean language and color images. This paper proposes the method how to easily collect and arrange to construct such dataset for cross-modal retrieval from Instagram, a kind of SNS(Social Network Service). Then we construct the dataset according to the methos, and the applicability has been proven by performing the cross-modal retrieval experiments. In the experiment, several methods for cross-modal retrievals are adopted including attention-based deep learning approach to compare the performances. The dataset in the study can be used to various multi-modal machine learning which requires the analysis of short Korean sentences, and the attention-based deep learning model which provides the best performance can be applied to automatically generate a Korean sentence from a color image, or a color image from a Korean sentence in SNS.

  Statistics
Cumulative Counts from November, 2022
Multiple requests among the same browser session are counted as one view. If you mouse over a chart, the values of data points will be shown.


  Cite this article

[IEEE Style]

K. Kim and J. Lee, "Dataset Construction and Deep Learning Models for Cross-Modal Retrieval for Text-Color Image in SNS," The Journal of Korean Institute of Communications and Information Sciences, vol. 44, no. 9, pp. 1742-1753, 2019. DOI: 10.7840/kics.2019.44.9.1742.

[ACM Style]

KangSub Kim and Joonwhoan Lee. 2019. Dataset Construction and Deep Learning Models for Cross-Modal Retrieval for Text-Color Image in SNS. The Journal of Korean Institute of Communications and Information Sciences, 44, 9, (2019), 1742-1753. DOI: 10.7840/kics.2019.44.9.1742.

[KICS Style]

KangSub Kim and Joonwhoan Lee, "Dataset Construction and Deep Learning Models for Cross-Modal Retrieval for Text-Color Image in SNS," The Journal of Korean Institute of Communications and Information Sciences, vol. 44, no. 9, pp. 1742-1753, 9. 2019. (https://doi.org/10.7840/kics.2019.44.9.1742)

Search
(IN TITLE, AUTHOR, ABSTRACT,KEYWORDS)

Advanced Search ▼

POPULAR KEYWORDS
(TOP 10 KEYWORDS)

Recent Publications
(LAST 3 YEARS)






pISSN : 1226 - 4717
eISSN : 2287 - 3880