Matches in SemOpenAlex for { <https://semopenalex.org/work/W3172704865> ?p ?o ?g. }
Showing items 1 to 92 of
92
with 100 items per page.
- W3172704865 endingPage "6758" @default.
- W3172704865 startingPage "6748" @default.
- W3172704865 abstract "Descent methods for deep networks are notoriously capricious: they require careful tuning of step size, momentum and weight decay, and which method will work best on a new benchmark is a priori unclear. To address this problem, this paper conducts a combined study of neural architecture and optimisation, leading to a new optimiser called Nero: the neuronal rotator. Nero trains reliably without momentum or weight decay, works in situations where Adam and SGD fail, and requires little to no learning rate tuning. Also, Nero's memory footprint is ~ square root that of Adam or LAMB. Nero combines two ideas: (1) projected gradient descent over the space of balanced networks; (2) neuron-specific updates, where the step size sets the angle through which each neuron's hyperplane turns. The paper concludes by discussing how this geometric connection between architecture and optimisation may impact theories of generalisation in deep learning." @default.
- W3172704865 created "2021-06-22" @default.
- W3172704865 creator A5028724419 @default.
- W3172704865 creator A5058303526 @default.
- W3172704865 creator A5067399219 @default.
- W3172704865 creator A5085826758 @default.
- W3172704865 date "2021-07-18" @default.
- W3172704865 modified "2023-09-23" @default.
- W3172704865 title "Learning by Turning: Neural Architecture Aware Optimisation" @default.
- W3172704865 hasPublicationYear "2021" @default.
- W3172704865 type Work @default.
- W3172704865 sameAs 3172704865 @default.
- W3172704865 citedByCount "1" @default.
- W3172704865 countsByYear W31727048652021 @default.
- W3172704865 crossrefType "proceedings-article" @default.
- W3172704865 hasAuthorship W3172704865A5028724419 @default.
- W3172704865 hasAuthorship W3172704865A5058303526 @default.
- W3172704865 hasAuthorship W3172704865A5067399219 @default.
- W3172704865 hasAuthorship W3172704865A5085826758 @default.
- W3172704865 hasConcept C10138342 @default.
- W3172704865 hasConcept C108583219 @default.
- W3172704865 hasConcept C111472728 @default.
- W3172704865 hasConcept C111919701 @default.
- W3172704865 hasConcept C11577676 @default.
- W3172704865 hasConcept C123657996 @default.
- W3172704865 hasConcept C13280743 @default.
- W3172704865 hasConcept C138885662 @default.
- W3172704865 hasConcept C142362112 @default.
- W3172704865 hasConcept C153258448 @default.
- W3172704865 hasConcept C153349607 @default.
- W3172704865 hasConcept C154945302 @default.
- W3172704865 hasConcept C162324750 @default.
- W3172704865 hasConcept C185798385 @default.
- W3172704865 hasConcept C205649164 @default.
- W3172704865 hasConcept C2524010 @default.
- W3172704865 hasConcept C33923547 @default.
- W3172704865 hasConcept C41008148 @default.
- W3172704865 hasConcept C50644808 @default.
- W3172704865 hasConcept C60718061 @default.
- W3172704865 hasConcept C68693459 @default.
- W3172704865 hasConcept C74912251 @default.
- W3172704865 hasConcept C75553542 @default.
- W3172704865 hasConceptScore W3172704865C10138342 @default.
- W3172704865 hasConceptScore W3172704865C108583219 @default.
- W3172704865 hasConceptScore W3172704865C111472728 @default.
- W3172704865 hasConceptScore W3172704865C111919701 @default.
- W3172704865 hasConceptScore W3172704865C11577676 @default.
- W3172704865 hasConceptScore W3172704865C123657996 @default.
- W3172704865 hasConceptScore W3172704865C13280743 @default.
- W3172704865 hasConceptScore W3172704865C138885662 @default.
- W3172704865 hasConceptScore W3172704865C142362112 @default.
- W3172704865 hasConceptScore W3172704865C153258448 @default.
- W3172704865 hasConceptScore W3172704865C153349607 @default.
- W3172704865 hasConceptScore W3172704865C154945302 @default.
- W3172704865 hasConceptScore W3172704865C162324750 @default.
- W3172704865 hasConceptScore W3172704865C185798385 @default.
- W3172704865 hasConceptScore W3172704865C205649164 @default.
- W3172704865 hasConceptScore W3172704865C2524010 @default.
- W3172704865 hasConceptScore W3172704865C33923547 @default.
- W3172704865 hasConceptScore W3172704865C41008148 @default.
- W3172704865 hasConceptScore W3172704865C50644808 @default.
- W3172704865 hasConceptScore W3172704865C60718061 @default.
- W3172704865 hasConceptScore W3172704865C68693459 @default.
- W3172704865 hasConceptScore W3172704865C74912251 @default.
- W3172704865 hasConceptScore W3172704865C75553542 @default.
- W3172704865 hasOpenAccess W3172704865 @default.
- W3172704865 hasRelatedWork W1482651362 @default.
- W3172704865 hasRelatedWork W1486687522 @default.
- W3172704865 hasRelatedWork W1605158001 @default.
- W3172704865 hasRelatedWork W2030040561 @default.
- W3172704865 hasRelatedWork W2753160622 @default.
- W3172704865 hasRelatedWork W2789423257 @default.
- W3172704865 hasRelatedWork W2794949442 @default.
- W3172704865 hasRelatedWork W2803911450 @default.
- W3172704865 hasRelatedWork W2951189778 @default.
- W3172704865 hasRelatedWork W3008314020 @default.
- W3172704865 hasRelatedWork W3035486480 @default.
- W3172704865 hasRelatedWork W3045356209 @default.
- W3172704865 hasRelatedWork W3111657519 @default.
- W3172704865 hasRelatedWork W3126611227 @default.
- W3172704865 hasRelatedWork W3129487256 @default.
- W3172704865 hasRelatedWork W3163969090 @default.
- W3172704865 hasRelatedWork W3209305110 @default.
- W3172704865 hasRelatedWork W623534045 @default.
- W3172704865 hasRelatedWork W772458362 @default.
- W3172704865 hasRelatedWork W824121112 @default.
- W3172704865 isParatext "false" @default.
- W3172704865 isRetracted "false" @default.
- W3172704865 magId "3172704865" @default.
- W3172704865 workType "article" @default.