Matches in SemOpenAlex for { <https://semopenalex.org/work/W4309803817> ?p ?o ?g. }
Showing items 1 to 79 of
79
with 100 items per page.
- W4309803817 abstract "Vector Quantization (VQ) is an appealing model compression method to obtain a tiny model with less accuracy loss. While methods to obtain better codebooks and codes under fixed clustering dimensionality have been extensively studied, optimizations of the vectors in favour of clustering performance are not carefully considered, especially via the reduction of vector dimensionality. This paper reports our recent progress on the combination of dimensionality compression and vector quantization, proposing a Low-Rank Representation Vector Quantization ($text{LR}^2text{VQ}$) method that outperforms previous VQ algorithms in various tasks and architectures. $text{LR}^2text{VQ}$ joins low-rank representation with subvector clustering to construct a new kind of building block that is directly optimized through end-to-end training over the task loss. Our proposed design pattern introduces three hyper-parameters, the number of clusters $k$, the size of subvectors $m$ and the clustering dimensionality $tilde{d}$. In our method, the compression ratio could be directly controlled by $m$, and the final accuracy is solely determined by $tilde{d}$. We recognize $tilde{d}$ as a trade-off between low-rank approximation error and clustering error and carry out both theoretical analysis and experimental observations that empower the estimation of the proper $tilde{d}$ before fine-tunning. With a proper $tilde{d}$, we evaluate $text{LR}^2text{VQ}$ with ResNet-18/ResNet-50 on ImageNet classification datasets, achieving 2.8%/1.0% top-1 accuracy improvements over the current state-of-the-art VQ-based compression algorithms with 43$times$/31$times$ compression factor." @default.
- W4309803817 created "2022-11-29" @default.
- W4309803817 creator A5007052694 @default.
- W4309803817 creator A5018218343 @default.
- W4309803817 creator A5070295826 @default.
- W4309803817 date "2022-11-21" @default.
- W4309803817 modified "2023-09-26" @default.
- W4309803817 title "Learning Low-Rank Representations for Model Compression" @default.
- W4309803817 doi "https://doi.org/10.48550/arxiv.2211.11397" @default.
- W4309803817 hasPublicationYear "2022" @default.
- W4309803817 type Work @default.
- W4309803817 citedByCount "0" @default.
- W4309803817 crossrefType "posted-content" @default.
- W4309803817 hasAuthorship W4309803817A5007052694 @default.
- W4309803817 hasAuthorship W4309803817A5018218343 @default.
- W4309803817 hasAuthorship W4309803817A5070295826 @default.
- W4309803817 hasBestOaLocation W43098038171 @default.
- W4309803817 hasConcept C111030470 @default.
- W4309803817 hasConcept C11413529 @default.
- W4309803817 hasConcept C114614502 @default.
- W4309803817 hasConcept C115961682 @default.
- W4309803817 hasConcept C127413603 @default.
- W4309803817 hasConcept C13481523 @default.
- W4309803817 hasConcept C153180895 @default.
- W4309803817 hasConcept C154945302 @default.
- W4309803817 hasConcept C159985019 @default.
- W4309803817 hasConcept C164226766 @default.
- W4309803817 hasConcept C171146098 @default.
- W4309803817 hasConcept C180016635 @default.
- W4309803817 hasConcept C192562407 @default.
- W4309803817 hasConcept C199833920 @default.
- W4309803817 hasConcept C25797200 @default.
- W4309803817 hasConcept C28855332 @default.
- W4309803817 hasConcept C33923547 @default.
- W4309803817 hasConcept C41008148 @default.
- W4309803817 hasConcept C511840579 @default.
- W4309803817 hasConcept C70518039 @default.
- W4309803817 hasConcept C73555534 @default.
- W4309803817 hasConcept C78548338 @default.
- W4309803817 hasConcept C9417928 @default.
- W4309803817 hasConceptScore W4309803817C111030470 @default.
- W4309803817 hasConceptScore W4309803817C11413529 @default.
- W4309803817 hasConceptScore W4309803817C114614502 @default.
- W4309803817 hasConceptScore W4309803817C115961682 @default.
- W4309803817 hasConceptScore W4309803817C127413603 @default.
- W4309803817 hasConceptScore W4309803817C13481523 @default.
- W4309803817 hasConceptScore W4309803817C153180895 @default.
- W4309803817 hasConceptScore W4309803817C154945302 @default.
- W4309803817 hasConceptScore W4309803817C159985019 @default.
- W4309803817 hasConceptScore W4309803817C164226766 @default.
- W4309803817 hasConceptScore W4309803817C171146098 @default.
- W4309803817 hasConceptScore W4309803817C180016635 @default.
- W4309803817 hasConceptScore W4309803817C192562407 @default.
- W4309803817 hasConceptScore W4309803817C199833920 @default.
- W4309803817 hasConceptScore W4309803817C25797200 @default.
- W4309803817 hasConceptScore W4309803817C28855332 @default.
- W4309803817 hasConceptScore W4309803817C33923547 @default.
- W4309803817 hasConceptScore W4309803817C41008148 @default.
- W4309803817 hasConceptScore W4309803817C511840579 @default.
- W4309803817 hasConceptScore W4309803817C70518039 @default.
- W4309803817 hasConceptScore W4309803817C73555534 @default.
- W4309803817 hasConceptScore W4309803817C78548338 @default.
- W4309803817 hasConceptScore W4309803817C9417928 @default.
- W4309803817 hasLocation W43098038171 @default.
- W4309803817 hasOpenAccess W4309803817 @default.
- W4309803817 hasPrimaryLocation W43098038171 @default.
- W4309803817 hasRelatedWork W1540466154 @default.
- W4309803817 hasRelatedWork W2032282729 @default.
- W4309803817 hasRelatedWork W2089758490 @default.
- W4309803817 hasRelatedWork W2113807068 @default.
- W4309803817 hasRelatedWork W2129693240 @default.
- W4309803817 hasRelatedWork W2153592792 @default.
- W4309803817 hasRelatedWork W2367682000 @default.
- W4309803817 hasRelatedWork W2545137258 @default.
- W4309803817 hasRelatedWork W2766076811 @default.
- W4309803817 hasRelatedWork W4220868150 @default.
- W4309803817 isParatext "false" @default.
- W4309803817 isRetracted "false" @default.
- W4309803817 workType "article" @default.