Matches in SemOpenAlex for { <https://semopenalex.org/work/W3215741372> ?p ?o ?g. }
Showing items 1 to 85 of
85
with 100 items per page.
- W3215741372 abstract "Quantization is one of the most effective methods to compress neural networks, which has achieved great success on convolutional neural networks (CNNs). Recently, vision transformers have demonstrated great potential in computer vision. However, previous post-training quantization methods performed not well on vision transformer, resulting in more than 1% accuracy drop even in 8-bit quantization. Therefore, we analyze the problems of quantization on vision transformers. We observe the distributions of activation values after softmax and GELU functions are quite different from the Gaussian distribution. We also observe that common quantization metrics, such as MSE and cosine distance, are inaccurate to determine the optimal scaling factor. In this paper, we propose the twin uniform quantization method to reduce the quantization error on these activation values. And we propose to use a Hessian guided metric to evaluate different scaling factors, which improves the accuracy of calibration with a small cost. To enable the fast quantization of vision transformers, we develop an efficient framework, PTQ4ViT. Experiments show the quantized vision transformers achieve near-lossless prediction accuracy (less than 0.5% drop at 8-bit quantization) on the ImageNet classification task." @default.
- W3215741372 created "2021-12-06" @default.
- W3215741372 creator A5010203669 @default.
- W3215741372 creator A5043763518 @default.
- W3215741372 creator A5066056671 @default.
- W3215741372 creator A5069921473 @default.
- W3215741372 creator A5089720861 @default.
- W3215741372 date "2021-11-24" @default.
- W3215741372 modified "2023-09-23" @default.
- W3215741372 title "PTQ4ViT: Post-Training Quantization Framework for Vision Transformers" @default.
- W3215741372 cites W2117539524 @default.
- W3215741372 cites W2786771851 @default.
- W3215741372 cites W2884150179 @default.
- W3215741372 cites W2963341956 @default.
- W3215741372 cites W2963403868 @default.
- W3215741372 cites W2963576971 @default.
- W3215741372 cites W2981751377 @default.
- W3215741372 cites W2998183051 @default.
- W3215741372 cites W2998218113 @default.
- W3215741372 cites W3038470071 @default.
- W3215741372 cites W3098873988 @default.
- W3215741372 cites W3098903812 @default.
- W3215741372 cites W3108516375 @default.
- W3215741372 cites W3119786062 @default.
- W3215741372 cites W3131500599 @default.
- W3215741372 cites W3138516171 @default.
- W3215741372 cites W3168649818 @default.
- W3215741372 cites W3169793979 @default.
- W3215741372 cites W3170874841 @default.
- W3215741372 cites W3178702014 @default.
- W3215741372 cites W3211787299 @default.
- W3215741372 cites W3122499249 @default.
- W3215741372 hasPublicationYear "2021" @default.
- W3215741372 type Work @default.
- W3215741372 sameAs 3215741372 @default.
- W3215741372 citedByCount "0" @default.
- W3215741372 crossrefType "posted-content" @default.
- W3215741372 hasAuthorship W3215741372A5010203669 @default.
- W3215741372 hasAuthorship W3215741372A5043763518 @default.
- W3215741372 hasAuthorship W3215741372A5066056671 @default.
- W3215741372 hasAuthorship W3215741372A5069921473 @default.
- W3215741372 hasAuthorship W3215741372A5089720861 @default.
- W3215741372 hasConcept C11413529 @default.
- W3215741372 hasConcept C119599485 @default.
- W3215741372 hasConcept C121332964 @default.
- W3215741372 hasConcept C127413603 @default.
- W3215741372 hasConcept C154945302 @default.
- W3215741372 hasConcept C163716315 @default.
- W3215741372 hasConcept C165801399 @default.
- W3215741372 hasConcept C188441871 @default.
- W3215741372 hasConcept C28855332 @default.
- W3215741372 hasConcept C41008148 @default.
- W3215741372 hasConcept C50644808 @default.
- W3215741372 hasConcept C62520636 @default.
- W3215741372 hasConcept C66322947 @default.
- W3215741372 hasConcept C81363708 @default.
- W3215741372 hasConceptScore W3215741372C11413529 @default.
- W3215741372 hasConceptScore W3215741372C119599485 @default.
- W3215741372 hasConceptScore W3215741372C121332964 @default.
- W3215741372 hasConceptScore W3215741372C127413603 @default.
- W3215741372 hasConceptScore W3215741372C154945302 @default.
- W3215741372 hasConceptScore W3215741372C163716315 @default.
- W3215741372 hasConceptScore W3215741372C165801399 @default.
- W3215741372 hasConceptScore W3215741372C188441871 @default.
- W3215741372 hasConceptScore W3215741372C28855332 @default.
- W3215741372 hasConceptScore W3215741372C41008148 @default.
- W3215741372 hasConceptScore W3215741372C50644808 @default.
- W3215741372 hasConceptScore W3215741372C62520636 @default.
- W3215741372 hasConceptScore W3215741372C66322947 @default.
- W3215741372 hasConceptScore W3215741372C81363708 @default.
- W3215741372 hasOpenAccess W3215741372 @default.
- W3215741372 hasRelatedWork W10809924 @default.
- W3215741372 hasRelatedWork W12405394 @default.
- W3215741372 hasRelatedWork W2433769 @default.
- W3215741372 hasRelatedWork W3781714 @default.
- W3215741372 hasRelatedWork W4136762 @default.
- W3215741372 hasRelatedWork W792754 @default.
- W3215741372 hasRelatedWork W8067959 @default.
- W3215741372 hasRelatedWork W819137 @default.
- W3215741372 hasRelatedWork W8208422 @default.
- W3215741372 hasRelatedWork W9190101 @default.
- W3215741372 isParatext "false" @default.
- W3215741372 isRetracted "false" @default.
- W3215741372 magId "3215741372" @default.
- W3215741372 workType "article" @default.