Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385328101> ?p ?o ?g. }
Showing items 1 to 67 of
67
with 100 items per page.
- W4385328101 abstract "Vision transformers (ViT) have been of broad interest in recent theoretical and empirical works. They are state-of-the-art thanks to their attention-based approach, which boosts the identification of key features and patterns within images thanks to the capability of avoiding inductive bias, resulting in highly accurate image analysis. Meanwhile, neoteric studies have reported a ``sparse double descent'' phenomenon that can occur in modern deep-learning models, where extremely over-parametrized models can generalize well. This raises practical questions about the optimal size of the model and the quest over finding the best trade-off between sparsity and performance is launched: are Vision Transformers also prone to sparse double descent? Can we find a way to avoid such a phenomenon? Our work tackles the occurrence of sparse double descent on ViTs. Despite some works that have shown that traditional architectures, like Resnet, are condemned to the sparse double descent phenomenon, for ViTs we observe that an optimally-tuned $ell_2$ regularization relieves such a phenomenon. However, everything comes at a cost: optimal lambda will sacrifice the potential compression of the ViT." @default.
- W4385328101 created "2023-07-28" @default.
- W4385328101 creator A5022572651 @default.
- W4385328101 creator A5046851981 @default.
- W4385328101 creator A5067519673 @default.
- W4385328101 date "2023-07-26" @default.
- W4385328101 modified "2023-09-26" @default.
- W4385328101 title "Sparse Double Descent in Vision Transformers: real or phantom threat?" @default.
- W4385328101 doi "https://doi.org/10.1007/978-3-031-43153-1_41" @default.
- W4385328101 hasPublicationYear "2023" @default.
- W4385328101 type Work @default.
- W4385328101 citedByCount "0" @default.
- W4385328101 crossrefType "posted-content" @default.
- W4385328101 hasAuthorship W4385328101A5022572651 @default.
- W4385328101 hasAuthorship W4385328101A5046851981 @default.
- W4385328101 hasAuthorship W4385328101A5067519673 @default.
- W4385328101 hasBestOaLocation W43853281011 @default.
- W4385328101 hasConcept C111472728 @default.
- W4385328101 hasConcept C11413529 @default.
- W4385328101 hasConcept C116149140 @default.
- W4385328101 hasConcept C119857082 @default.
- W4385328101 hasConcept C121332964 @default.
- W4385328101 hasConcept C138885662 @default.
- W4385328101 hasConcept C153258448 @default.
- W4385328101 hasConcept C153294291 @default.
- W4385328101 hasConcept C154945302 @default.
- W4385328101 hasConcept C165801399 @default.
- W4385328101 hasConcept C2776135515 @default.
- W4385328101 hasConcept C2776637919 @default.
- W4385328101 hasConcept C41008148 @default.
- W4385328101 hasConcept C50335755 @default.
- W4385328101 hasConcept C50644808 @default.
- W4385328101 hasConcept C62520636 @default.
- W4385328101 hasConcept C66322947 @default.
- W4385328101 hasConceptScore W4385328101C111472728 @default.
- W4385328101 hasConceptScore W4385328101C11413529 @default.
- W4385328101 hasConceptScore W4385328101C116149140 @default.
- W4385328101 hasConceptScore W4385328101C119857082 @default.
- W4385328101 hasConceptScore W4385328101C121332964 @default.
- W4385328101 hasConceptScore W4385328101C138885662 @default.
- W4385328101 hasConceptScore W4385328101C153258448 @default.
- W4385328101 hasConceptScore W4385328101C153294291 @default.
- W4385328101 hasConceptScore W4385328101C154945302 @default.
- W4385328101 hasConceptScore W4385328101C165801399 @default.
- W4385328101 hasConceptScore W4385328101C2776135515 @default.
- W4385328101 hasConceptScore W4385328101C2776637919 @default.
- W4385328101 hasConceptScore W4385328101C41008148 @default.
- W4385328101 hasConceptScore W4385328101C50335755 @default.
- W4385328101 hasConceptScore W4385328101C50644808 @default.
- W4385328101 hasConceptScore W4385328101C62520636 @default.
- W4385328101 hasConceptScore W4385328101C66322947 @default.
- W4385328101 hasLocation W43853281011 @default.
- W4385328101 hasOpenAccess W4385328101 @default.
- W4385328101 hasPrimaryLocation W43853281011 @default.
- W4385328101 hasRelatedWork W1970131234 @default.
- W4385328101 hasRelatedWork W1998295157 @default.
- W4385328101 hasRelatedWork W2044847021 @default.
- W4385328101 hasRelatedWork W2076165463 @default.
- W4385328101 hasRelatedWork W2157414975 @default.
- W4385328101 hasRelatedWork W2335120442 @default.
- W4385328101 hasRelatedWork W2349940749 @default.
- W4385328101 hasRelatedWork W2363378255 @default.
- W4385328101 hasRelatedWork W3035991101 @default.
- W4385328101 hasRelatedWork W4291309967 @default.
- W4385328101 isParatext "false" @default.
- W4385328101 isRetracted "false" @default.
- W4385328101 workType "article" @default.