Matches in SemOpenAlex for { <https://semopenalex.org/work/W2474828585> ?p ?o ?g. }
Showing items 1 to 65 of
65
with 100 items per page.
- W2474828585 abstract "The performance surfaces of large neural networks contain ravines, flat spots, non-convex regions, and other features that make weight optimization difficult. Although a variety of sophisticated alternatives are available, the simple on-line backpropagation procedure remains the most popular method for adapting the weights of these systems. This approach performs stochastic (or incremental) steepest descent, and is significantly hampered by the character of the performance surface. Backpropagation's principal advantage over alternate methods rests in its ability to perform an update after each pattern presentation, while maintaining time and space demands that grow only linearly with the number of adaptive weights.In this dissertation, we explore new stochastic methods that improve on the learning speed of the backpropagation algorithm, while retaining its linear complexity. We begin by examining the convergence properties of two deterministic steepest descent methods. Corresponding scaled stochastic algorithms are then developed from an analysis of the neural network's Expected Mean Square Error (EMSE) sequence in the neighborhood of a local minimum of the performance surface. To maintain stable behavior under broad conditions, this development uses a general statistical model for the neural network's instantaneous Hessian matrix. For theoretical performance comparisons, however, we require a more specialized statistical framework. The corresponding analysis helps reveal the complementary convergence properties of the two updates--a relationship we exploit by combining the updates to form a family of dual-update procedures.Effective methods are established for generating a slowly varying sequence of search direction vectors and all required scaling information. The result is a practical algorithm which performs robustly when the weight vector of a large neural network is placed at arbitrary initial positions. The two weight updates are scaled by parameters computed from recursive estimates of five scalar sequences: the first and second moments of the trace of the instantaneous Hessian matrix, the first and second moments of the instantaneous gradient vector's along the search direction, and the first moment of the instantaneous Hessian's projection along the same direction." @default.
- W2474828585 created "2016-07-22" @default.
- W2474828585 creator A5031750471 @default.
- W2474828585 creator A5044542575 @default.
- W2474828585 date "1996-01-01" @default.
- W2474828585 modified "2023-09-23" @default.
- W2474828585 title "Scaled stochastic methods for training neural networks" @default.
- W2474828585 hasPublicationYear "1996" @default.
- W2474828585 type Work @default.
- W2474828585 sameAs 2474828585 @default.
- W2474828585 citedByCount "2" @default.
- W2474828585 crossrefType "journal-article" @default.
- W2474828585 hasAuthorship W2474828585A5031750471 @default.
- W2474828585 hasAuthorship W2474828585A5044542575 @default.
- W2474828585 hasConcept C11413529 @default.
- W2474828585 hasConcept C126255220 @default.
- W2474828585 hasConcept C153258448 @default.
- W2474828585 hasConcept C154945302 @default.
- W2474828585 hasConcept C155032097 @default.
- W2474828585 hasConcept C162324750 @default.
- W2474828585 hasConcept C206688291 @default.
- W2474828585 hasConcept C2777303404 @default.
- W2474828585 hasConcept C33923547 @default.
- W2474828585 hasConcept C41008148 @default.
- W2474828585 hasConcept C50522688 @default.
- W2474828585 hasConcept C50644808 @default.
- W2474828585 hasConceptScore W2474828585C11413529 @default.
- W2474828585 hasConceptScore W2474828585C126255220 @default.
- W2474828585 hasConceptScore W2474828585C153258448 @default.
- W2474828585 hasConceptScore W2474828585C154945302 @default.
- W2474828585 hasConceptScore W2474828585C155032097 @default.
- W2474828585 hasConceptScore W2474828585C162324750 @default.
- W2474828585 hasConceptScore W2474828585C206688291 @default.
- W2474828585 hasConceptScore W2474828585C2777303404 @default.
- W2474828585 hasConceptScore W2474828585C33923547 @default.
- W2474828585 hasConceptScore W2474828585C41008148 @default.
- W2474828585 hasConceptScore W2474828585C50522688 @default.
- W2474828585 hasConceptScore W2474828585C50644808 @default.
- W2474828585 hasLocation W24748285851 @default.
- W2474828585 hasOpenAccess W2474828585 @default.
- W2474828585 hasPrimaryLocation W24748285851 @default.
- W2474828585 hasRelatedWork W1489444018 @default.
- W2474828585 hasRelatedWork W1813485996 @default.
- W2474828585 hasRelatedWork W2115031564 @default.
- W2474828585 hasRelatedWork W2199117338 @default.
- W2474828585 hasRelatedWork W2225988436 @default.
- W2474828585 hasRelatedWork W2299367673 @default.
- W2474828585 hasRelatedWork W2758053331 @default.
- W2474828585 hasRelatedWork W2766293931 @default.
- W2474828585 hasRelatedWork W2852105405 @default.
- W2474828585 hasRelatedWork W2922424155 @default.
- W2474828585 hasRelatedWork W2945880513 @default.
- W2474828585 hasRelatedWork W2950203249 @default.
- W2474828585 hasRelatedWork W2951527401 @default.
- W2474828585 hasRelatedWork W2963293084 @default.
- W2474828585 hasRelatedWork W2963483869 @default.
- W2474828585 hasRelatedWork W3012404247 @default.
- W2474828585 hasRelatedWork W3171138071 @default.
- W2474828585 hasRelatedWork W3179632780 @default.
- W2474828585 hasRelatedWork W3181226302 @default.
- W2474828585 hasRelatedWork W7934506 @default.
- W2474828585 isParatext "false" @default.
- W2474828585 isRetracted "false" @default.
- W2474828585 magId "2474828585" @default.
- W2474828585 workType "article" @default.