Matches in SemOpenAlex for { <https://semopenalex.org/work/W1864754470> ?p ?o ?g. }
Showing items 1 to 88 of
88
with 100 items per page.
- W1864754470 abstract "Recently, we proposed to transform the outputs of each hidden neuron in a multi-layer perceptron network to have zero output and zero slope on average, and use separate shortcut connections to model the linear dependencies instead. We continue the work by firstly introducing a third transformation to normalize the scale of the outputs of each hidden neuron, and secondly by analyzing the connections to second order optimization methods. We show that the transformations make a simple stochastic gradient behave closer to second-order optimization methods and thus speed up learning. This is shown both in theory and with experiments. The experiments on the third transformation show that while it further increases the speed of learning, it can also hurt performance by converging to a worse local optimum, where both the inputs and outputs of many hidden neurons are close to zero." @default.
- W1864754470 created "2016-06-24" @default.
- W1864754470 creator A5001226970 @default.
- W1864754470 creator A5008894339 @default.
- W1864754470 creator A5049927688 @default.
- W1864754470 creator A5050971487 @default.
- W1864754470 date "2013-01-15" @default.
- W1864754470 modified "2023-09-27" @default.
- W1864754470 title "Pushing Stochastic Gradient towards Second-Order Methods -- Backpropagation Learning with Transformations in Nonlinearities" @default.
- W1864754470 cites W1811843574 @default.
- W1864754470 cites W196761320 @default.
- W1864754470 cites W2100495367 @default.
- W1864754470 cites W2139398462 @default.
- W1864754470 cites W2152424459 @default.
- W1864754470 cites W2164273299 @default.
- W1864754470 hasPublicationYear "2013" @default.
- W1864754470 type Work @default.
- W1864754470 sameAs 1864754470 @default.
- W1864754470 citedByCount "9" @default.
- W1864754470 countsByYear W18647544702013 @default.
- W1864754470 countsByYear W18647544702015 @default.
- W1864754470 countsByYear W18647544702016 @default.
- W1864754470 crossrefType "posted-content" @default.
- W1864754470 hasAuthorship W1864754470A5001226970 @default.
- W1864754470 hasAuthorship W1864754470A5008894339 @default.
- W1864754470 hasAuthorship W1864754470A5049927688 @default.
- W1864754470 hasAuthorship W1864754470A5050971487 @default.
- W1864754470 hasConcept C104317684 @default.
- W1864754470 hasConcept C111472728 @default.
- W1864754470 hasConcept C11413529 @default.
- W1864754470 hasConcept C126255220 @default.
- W1864754470 hasConcept C138885662 @default.
- W1864754470 hasConcept C154945302 @default.
- W1864754470 hasConcept C155032097 @default.
- W1864754470 hasConcept C185592680 @default.
- W1864754470 hasConcept C204241405 @default.
- W1864754470 hasConcept C2780586882 @default.
- W1864754470 hasConcept C2780813799 @default.
- W1864754470 hasConcept C33923547 @default.
- W1864754470 hasConcept C41008148 @default.
- W1864754470 hasConcept C41895202 @default.
- W1864754470 hasConcept C50644808 @default.
- W1864754470 hasConcept C55493867 @default.
- W1864754470 hasConcept C60908668 @default.
- W1864754470 hasConceptScore W1864754470C104317684 @default.
- W1864754470 hasConceptScore W1864754470C111472728 @default.
- W1864754470 hasConceptScore W1864754470C11413529 @default.
- W1864754470 hasConceptScore W1864754470C126255220 @default.
- W1864754470 hasConceptScore W1864754470C138885662 @default.
- W1864754470 hasConceptScore W1864754470C154945302 @default.
- W1864754470 hasConceptScore W1864754470C155032097 @default.
- W1864754470 hasConceptScore W1864754470C185592680 @default.
- W1864754470 hasConceptScore W1864754470C204241405 @default.
- W1864754470 hasConceptScore W1864754470C2780586882 @default.
- W1864754470 hasConceptScore W1864754470C2780813799 @default.
- W1864754470 hasConceptScore W1864754470C33923547 @default.
- W1864754470 hasConceptScore W1864754470C41008148 @default.
- W1864754470 hasConceptScore W1864754470C41895202 @default.
- W1864754470 hasConceptScore W1864754470C50644808 @default.
- W1864754470 hasConceptScore W1864754470C55493867 @default.
- W1864754470 hasConceptScore W1864754470C60908668 @default.
- W1864754470 hasLocation W18647544701 @default.
- W1864754470 hasOpenAccess W1864754470 @default.
- W1864754470 hasPrimaryLocation W18647544701 @default.
- W1864754470 hasRelatedWork W1533861849 @default.
- W1864754470 hasRelatedWork W1573715633 @default.
- W1864754470 hasRelatedWork W1981275876 @default.
- W1864754470 hasRelatedWork W2019322993 @default.
- W1864754470 hasRelatedWork W2029821122 @default.
- W1864754470 hasRelatedWork W2050485765 @default.
- W1864754470 hasRelatedWork W2130984546 @default.
- W1864754470 hasRelatedWork W2152424459 @default.
- W1864754470 hasRelatedWork W2418791326 @default.
- W1864754470 hasRelatedWork W2593634001 @default.
- W1864754470 hasRelatedWork W2809600276 @default.
- W1864754470 hasRelatedWork W2881426224 @default.
- W1864754470 hasRelatedWork W2892899012 @default.
- W1864754470 hasRelatedWork W2912648029 @default.
- W1864754470 hasRelatedWork W2963763411 @default.
- W1864754470 hasRelatedWork W2975491454 @default.
- W1864754470 hasRelatedWork W2976426989 @default.
- W1864754470 hasRelatedWork W3021623424 @default.
- W1864754470 hasRelatedWork W3042063146 @default.
- W1864754470 hasRelatedWork W6908809 @default.
- W1864754470 isParatext "false" @default.
- W1864754470 isRetracted "false" @default.
- W1864754470 magId "1864754470" @default.
- W1864754470 workType "article" @default.