Matches in SemOpenAlex for { <https://semopenalex.org/work/W4367189403> ?p ?o ?g. }
Showing items 1 to 81 of
81
with 100 items per page.
- W4367189403 abstract "In the last few years, the success of Transformers in computer vision has stimulated the discovery of many alternative models that compete with Transformers, such as the MLP-Mixer. Despite their weak induced bias, these models have achieved performance comparable to well-studied convolutional neural networks. Recent studies on modern Hopfield networks suggest the correspondence between certain energy-based associative memory models and Transformers or MLP-Mixer, and shed some light on the theoretical background of the Transformer-type architectures design. In this paper we generalize the correspondence to the recently introduced hierarchical Hopfield network, and find iMixer, a novel generalization of MLP-Mixer model. Unlike ordinary feedforward neural networks, iMixer involves MLP layers that propagate forward from the output side to the input side. We characterize the module as an example of invertible, implicit, and iterative mixing module. We evaluate the model performance with various datasets on image classification tasks, and find that iMixer reasonably achieves the improvement compared to the baseline vanilla MLP-Mixer. The results imply that the correspondence between the Hopfield networks and the Mixer models serves as a principle for understanding a broader class of Transformer-like architecture designs." @default.
- W4367189403 created "2023-04-28" @default.
- W4367189403 creator A5047076115 @default.
- W4367189403 creator A5084053028 @default.
- W4367189403 date "2023-04-25" @default.
- W4367189403 modified "2023-10-18" @default.
- W4367189403 title "iMixer: hierarchical Hopfield network implies an invertible, implicit and iterative MLP-Mixer" @default.
- W4367189403 doi "https://doi.org/10.48550/arxiv.2304.13061" @default.
- W4367189403 hasPublicationYear "2023" @default.
- W4367189403 type Work @default.
- W4367189403 citedByCount "0" @default.
- W4367189403 crossrefType "posted-content" @default.
- W4367189403 hasAuthorship W4367189403A5047076115 @default.
- W4367189403 hasAuthorship W4367189403A5084053028 @default.
- W4367189403 hasBestOaLocation W43671894031 @default.
- W4367189403 hasConcept C11413529 @default.
- W4367189403 hasConcept C119599485 @default.
- W4367189403 hasConcept C127413603 @default.
- W4367189403 hasConcept C133731056 @default.
- W4367189403 hasConcept C134306372 @default.
- W4367189403 hasConcept C153180895 @default.
- W4367189403 hasConcept C154945302 @default.
- W4367189403 hasConcept C159423971 @default.
- W4367189403 hasConcept C165801399 @default.
- W4367189403 hasConcept C177148314 @default.
- W4367189403 hasConcept C197352929 @default.
- W4367189403 hasConcept C201995342 @default.
- W4367189403 hasConcept C202444582 @default.
- W4367189403 hasConcept C2780451532 @default.
- W4367189403 hasConcept C28006648 @default.
- W4367189403 hasConcept C33923547 @default.
- W4367189403 hasConcept C38858127 @default.
- W4367189403 hasConcept C41008148 @default.
- W4367189403 hasConcept C46421273 @default.
- W4367189403 hasConcept C50644808 @default.
- W4367189403 hasConcept C53442348 @default.
- W4367189403 hasConcept C66322947 @default.
- W4367189403 hasConcept C80444323 @default.
- W4367189403 hasConcept C81363708 @default.
- W4367189403 hasConcept C96442724 @default.
- W4367189403 hasConceptScore W4367189403C11413529 @default.
- W4367189403 hasConceptScore W4367189403C119599485 @default.
- W4367189403 hasConceptScore W4367189403C127413603 @default.
- W4367189403 hasConceptScore W4367189403C133731056 @default.
- W4367189403 hasConceptScore W4367189403C134306372 @default.
- W4367189403 hasConceptScore W4367189403C153180895 @default.
- W4367189403 hasConceptScore W4367189403C154945302 @default.
- W4367189403 hasConceptScore W4367189403C159423971 @default.
- W4367189403 hasConceptScore W4367189403C165801399 @default.
- W4367189403 hasConceptScore W4367189403C177148314 @default.
- W4367189403 hasConceptScore W4367189403C197352929 @default.
- W4367189403 hasConceptScore W4367189403C201995342 @default.
- W4367189403 hasConceptScore W4367189403C202444582 @default.
- W4367189403 hasConceptScore W4367189403C2780451532 @default.
- W4367189403 hasConceptScore W4367189403C28006648 @default.
- W4367189403 hasConceptScore W4367189403C33923547 @default.
- W4367189403 hasConceptScore W4367189403C38858127 @default.
- W4367189403 hasConceptScore W4367189403C41008148 @default.
- W4367189403 hasConceptScore W4367189403C46421273 @default.
- W4367189403 hasConceptScore W4367189403C50644808 @default.
- W4367189403 hasConceptScore W4367189403C53442348 @default.
- W4367189403 hasConceptScore W4367189403C66322947 @default.
- W4367189403 hasConceptScore W4367189403C80444323 @default.
- W4367189403 hasConceptScore W4367189403C81363708 @default.
- W4367189403 hasConceptScore W4367189403C96442724 @default.
- W4367189403 hasLocation W43671894031 @default.
- W4367189403 hasOpenAccess W4367189403 @default.
- W4367189403 hasPrimaryLocation W43671894031 @default.
- W4367189403 hasRelatedWork W1931653978 @default.
- W4367189403 hasRelatedWork W1936509470 @default.
- W4367189403 hasRelatedWork W1987836535 @default.
- W4367189403 hasRelatedWork W2089251960 @default.
- W4367189403 hasRelatedWork W2104533351 @default.
- W4367189403 hasRelatedWork W2121614139 @default.
- W4367189403 hasRelatedWork W2128271402 @default.
- W4367189403 hasRelatedWork W2218872326 @default.
- W4367189403 hasRelatedWork W2368008317 @default.
- W4367189403 hasRelatedWork W2540809067 @default.
- W4367189403 isParatext "false" @default.
- W4367189403 isRetracted "false" @default.
- W4367189403 workType "article" @default.