Transformer-Based Detection and Facial Expression Recognition of a Pansori Singer 


Vol. 47,  No. 12, pp. 2156-2167, Dec.  2022
10.7840/kics.2022.47.12.2156


PDF
  Abstract

Our work aims to recognize the singer’s motions and facial expression of Pansori, a traditional Korean music. In order to achieve our goal, the region of a singer is at first detected from every video frame, then the motion and facial expression are classified based on still image analysis in the detected region. In order to overcome the difficulty due to insufficient labelled data, the person class of MS COCO data is mixed up with the collected Pansori dataset for detecting Pansori singer, and ViT(Visual Transformer) pre-trained backbone is adopted to construct a singer’s facial expression classifier after self-supervised training in MAE model with ImageNet and RAF-DB dataset. For singer detection, 91.2% of AP has been achieved under 75% of IoU threshold. Also, 78.44% of accuracy has been obtained in Pansori singer’s facial expression classified into 4 categories. The results of this study are expected to be meaningful for preservation or education of cultural content.

  Statistics
Cumulative Counts from November, 2022
Multiple requests among the same browser session are counted as one view. If you mouse over a chart, the values of data points will be shown.


  Related Articles
  Cite this article

[IEEE Style]

W. Wu, H. Lee, J. Lee, "Transformer-Based Detection and Facial Expression Recognition of a Pansori Singer," The Journal of Korean Institute of Communications and Information Sciences, vol. 47, no. 12, pp. 2156-2167, 2022. DOI: 10.7840/kics.2022.47.12.2156.

[ACM Style]

Wenqin Wu, Hyejeong Lee, and Joonwhoan Lee. 2022. Transformer-Based Detection and Facial Expression Recognition of a Pansori Singer. The Journal of Korean Institute of Communications and Information Sciences, 47, 12, (2022), 2156-2167. DOI: 10.7840/kics.2022.47.12.2156.

[KICS Style]

Wenqin Wu, Hyejeong Lee, Joonwhoan Lee, "Transformer-Based Detection and Facial Expression Recognition of a Pansori Singer," The Journal of Korean Institute of Communications and Information Sciences, vol. 47, no. 12, pp. 2156-2167, 12. 2022. (https://doi.org/10.7840/kics.2022.47.12.2156)
Vol. 47, No. 12 Index

Search
(IN TITLE, AUTHOR, ABSTRACT,KEYWORDS)

Advanced Search ▼

POPULAR KEYWORDS
(TOP 10 KEYWORDS)

Recent Publications
(LAST 3 YEARS)






pISSN : 1226 - 4717
eISSN : 2287 - 3880