간단하게 이전에 읽은 논문 정리 아주 간단간단
논문은 링크에서 볼 수 있다.
activation quantization을 accuracy degradation 없이하는 방법에 관한 논문이다.
activation quantization에 적합한 함수를 학습시점에 학습한다.
Quantization of weights is equivalent to discretizing the hypothesis space of the loss function with respect to the weight variables
we achieve ∼14× improvement in density when the bit-precisions of both activations and weights are uniformly reduced from 16 bits to 2 bits
quantizing from 16 to 4 bits leads to a 4× increase in peak FLOPs but a 4.5× improvement in performance.
we see a near-linear increase in performance for up-to 4 bits and a small drop at extreme quantization levels (2 bits)