SWNQ: Scaled Weight Normalization Based Post-Training Quantization Method 


Vol. 46,  No. 3, pp. 583-590, Mar.  2021
10.7840/kics.2021.46.3.583


PDF
  Abstract

Post-training quantization has an advantage that it does not require any training process and thus does not depend on the training data, but has a disadvantage in that its performance severely degrades especially at low precision. To solve aforementioned problem, in this paper, we propose a novel SWNQ(Scaled Weight Normalization based post-training Quantization) method that reduces quantization errors arising from the long-tailed weight distribution by introducing a scaling factor to the weight normalization technique used for the existing quantization methods. Experimental results demonstrate that the SWNQ can perform an immediate quantization while increasing quantization performance compared to the state-of-the-art weight normalization-based quantization with no further training or fine-tuning. Moreover, the SWNQ proves that it can effectively solve the performance degradation problem of post-training quantization by showing that the proposed method can be quantized by the performance gap of only 1.2% compared with a full-precision model in the 4-bit-based mixed-precision quantization.

  Statistics
Cumulative Counts from November, 2022
Multiple requests among the same browser session are counted as one view. If you mouse over a chart, the values of data points will be shown.


  Cite this article

[IEEE Style]

G. Ban and J. Yoo, "SWNQ: Scaled Weight Normalization Based Post-Training Quantization Method," The Journal of Korean Institute of Communications and Information Sciences, vol. 46, no. 3, pp. 583-590, 2021. DOI: 10.7840/kics.2021.46.3.583.

[ACM Style]

Geun-Woo Ban and Joonhyuk Yoo. 2021. SWNQ: Scaled Weight Normalization Based Post-Training Quantization Method. The Journal of Korean Institute of Communications and Information Sciences, 46, 3, (2021), 583-590. DOI: 10.7840/kics.2021.46.3.583.

[KICS Style]

Geun-Woo Ban and Joonhyuk Yoo, "SWNQ: Scaled Weight Normalization Based Post-Training Quantization Method," The Journal of Korean Institute of Communications and Information Sciences, vol. 46, no. 3, pp. 583-590, 3. 2021. (https://doi.org/10.7840/kics.2021.46.3.583)

Search
(IN TITLE, AUTHOR, ABSTRACT,KEYWORDS)

Advanced Search ▼

POPULAR KEYWORDS
(TOP 10 KEYWORDS)

Recent Publications
(LAST 3 YEARS)






pISSN : 1226 - 4717
eISSN : 2287 - 3880