Matches in SemOpenAlex for { <https://semopenalex.org/work/W3196407858> ?p ?o ?g. }
Showing items 1 to 80 of
80
with 100 items per page.
- W3196407858 abstract "Network quantization, which aims to reduce the bitlengths of the network weights and activations, has emerged for their deployments to resource-limited devices. Although recent studies have successfully discretized a full-precision network, they still incur large quantization errors after training, thus giving rise to a significant performance gap between a full-precision network and its quantized counterpart. In this work, we propose a novel quantization method for neural networks, Cluster-Promoting Quantization (CPQ) that finds the optimal quantization grids while naturally encouraging the underlying full-precision weights to gather around those quantization grids cohesively during training. This property of CPQ is thanks to our two main ingredients that enable differentiable quantization: i) the use of the categorical distribution designed by a specific probabilistic parametrization in the forward pass and ii) our proposed multi-class straight-through estimator (STE) in the backward pass. Since our second component, multi-class STE, is intrinsically biased, we additionally propose a new bit-drop technique, DropBits, that revises the standard dropout regularization to randomly drop bits instead of neurons. As a natural extension of DropBits, we further introduce the way of learning heterogeneous quantization levels to find proper bit-length for each layer by imposing an additional regularization on DropBits. We experimentally validate our method on various benchmark datasets and network architectures, and also support a new hypothesis for quantization: learning heterogeneous quantization levels outperforms the case using the same but fixed quantization levels from scratch." @default.
- W3196407858 created "2021-09-13" @default.
- W3196407858 creator A5070302452 @default.
- W3196407858 creator A5086698569 @default.
- W3196407858 creator A5087337598 @default.
- W3196407858 creator A5090379383 @default.
- W3196407858 date "2021-10-01" @default.
- W3196407858 modified "2023-09-27" @default.
- W3196407858 title "Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Loss" @default.
- W3196407858 cites W1686810756 @default.
- W3196407858 cites W2095705004 @default.
- W3196407858 cites W2112796928 @default.
- W3196407858 cites W2194775991 @default.
- W3196407858 cites W2242818861 @default.
- W3196407858 cites W2300242332 @default.
- W3196407858 cites W2402144811 @default.
- W3196407858 cites W2405920868 @default.
- W3196407858 cites W2547875792 @default.
- W3196407858 cites W2548228487 @default.
- W3196407858 cites W2785916643 @default.
- W3196407858 cites W2786771851 @default.
- W3196407858 cites W2884150179 @default.
- W3196407858 cites W2887005264 @default.
- W3196407858 cites W2908510526 @default.
- W3196407858 cites W2962706989 @default.
- W3196407858 cites W2962768518 @default.
- W3196407858 cites W2963114950 @default.
- W3196407858 cites W2963163009 @default.
- W3196407858 cites W2963480671 @default.
- W3196407858 cites W2963735467 @default.
- W3196407858 cites W2963813662 @default.
- W3196407858 cites W2963828549 @default.
- W3196407858 cites W2964297791 @default.
- W3196407858 cites W2970971581 @default.
- W3196407858 cites W2972918064 @default.
- W3196407858 cites W2982041622 @default.
- W3196407858 cites W2982479999 @default.
- W3196407858 cites W2994840239 @default.
- W3196407858 cites W2994955197 @default.
- W3196407858 cites W2995329031 @default.
- W3196407858 cites W2995738369 @default.
- W3196407858 cites W3004061291 @default.
- W3196407858 doi "https://doi.org/10.1109/iccv48922.2021.00532" @default.
- W3196407858 hasPublicationYear "2021" @default.
- W3196407858 type Work @default.
- W3196407858 sameAs 3196407858 @default.
- W3196407858 citedByCount "0" @default.
- W3196407858 crossrefType "proceedings-article" @default.
- W3196407858 hasAuthorship W3196407858A5070302452 @default.
- W3196407858 hasAuthorship W3196407858A5086698569 @default.
- W3196407858 hasAuthorship W3196407858A5087337598 @default.
- W3196407858 hasAuthorship W3196407858A5090379383 @default.
- W3196407858 hasBestOaLocation W31964078582 @default.
- W3196407858 hasConcept C11413529 @default.
- W3196407858 hasConcept C28855332 @default.
- W3196407858 hasConcept C41008148 @default.
- W3196407858 hasConcept C93372532 @default.
- W3196407858 hasConceptScore W3196407858C11413529 @default.
- W3196407858 hasConceptScore W3196407858C28855332 @default.
- W3196407858 hasConceptScore W3196407858C41008148 @default.
- W3196407858 hasConceptScore W3196407858C93372532 @default.
- W3196407858 hasFunder F4320322120 @default.
- W3196407858 hasLocation W31964078581 @default.
- W3196407858 hasLocation W31964078582 @default.
- W3196407858 hasOpenAccess W3196407858 @default.
- W3196407858 hasPrimaryLocation W31964078581 @default.
- W3196407858 hasRelatedWork W10675490 @default.
- W3196407858 hasRelatedWork W12405394 @default.
- W3196407858 hasRelatedWork W12646621 @default.
- W3196407858 hasRelatedWork W13775620 @default.
- W3196407858 hasRelatedWork W14007254 @default.
- W3196407858 hasRelatedWork W2191154 @default.
- W3196407858 hasRelatedWork W2711915 @default.
- W3196407858 hasRelatedWork W335157 @default.
- W3196407858 hasRelatedWork W7407951 @default.
- W3196407858 hasRelatedWork W7694933 @default.
- W3196407858 isParatext "false" @default.
- W3196407858 isRetracted "false" @default.
- W3196407858 magId "3196407858" @default.
- W3196407858 workType "article" @default.