FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2924131876> ?p ?o ?g. }

• W2924131876 abstract "Mini-batch sub-sampling is likely here to stay, due to growing data demands, memory-limited computational resources such as graphical processing units (GPUs), and the dynamics of on-line learning. Sampling a new mini-batch at every loss evaluation brings a number of benefits, but also one significant drawback: The loss function becomes discontinuous. These discontinuities are generally not problematic when using fixed learning rates or learning rate schedules typical of subgradient methods. However, they hinder attempts to directly minimize the loss function by solving for critical points, since function minimizers find spurious minima induced by discontinuities, while critical points may not even exist. Therefore, finding function minimizers and critical points in stochastic optimization is ineffective. As a result, attention has been given to reducing the effect of these discontinuities by means such as gradient averaging or adaptive and dynamic sampling. This paper offers an alternative paradigm: Recasting the optimization problem to rather find Non-Negative Associated Gradient Projection Points (NN-GPPs). In this paper, we demonstrate the NN-GPP interpretation of gradient information is more robust than critical points or minimizers, being less susceptible to sub-sampling induced variance and eliminating spurious function minimizers. We conduct a visual investigation, where we compare function value and gradient information for a variety of popular activation functions as applied to a simple neural network training problem. Based on the improved description offered by NN-GPPs over minimizers to identify true optima, in particular when using smooth activation functions with high curvature characteristics, we postulate that locating NN-GPPs can contribute significantly to automating neural network training." @default.
• W2924131876 created "2019-04-01" @default.
• W2924131876 creator A5023733265 @default.
• W2924131876 creator A5048063937 @default.
• W2924131876 date "2019-03-20" @default.
• W2924131876 modified "2023-09-27" @default.
• W2924131876 title "Visual interpretation of the robustness of Non-Negative Associative Gradient Projection Points over function minimizers in mini-batch sampled loss functions" @default.
• W2924131876 cites W1169135902 @default.
• W2924131876 cites W1491622225 @default.
• W2924131876 cites W1566309885 @default.
• W2924131876 cites W196761320 @default.
• W2924131876 cites W1994616650 @default.
• W2924131876 cites W20283819 @default.
• W2924131876 cites W2032969363 @default.
• W2924131876 cites W2096840748 @default.
• W2924131876 cites W2126590018 @default.
• W2924131876 cites W2140303265 @default.
• W2924131876 cites W2146502635 @default.
• W2924131876 cites W2176412452 @default.
• W2924131876 cites W2564807118 @default.
• W2924131876 cites W2619219755 @default.
• W2924131876 cites W2771005692 @default.
• W2924131876 cites W2799042347 @default.
• W2924131876 cites W2892675615 @default.
• W2924131876 cites W2963317585 @default.
• W2924131876 cites W2963589953 @default.
• W2924131876 cites W2964121744 @default.
• W2924131876 cites W2964160102 @default.
• W2924131876 cites W3022467278 @default.
• W2924131876 hasPublicationYear "2019" @default.
• W2924131876 type Work @default.
• W2924131876 sameAs 2924131876 @default.
• W2924131876 citedByCount "0" @default.
• W2924131876 crossrefType "posted-content" @default.
• W2924131876 hasAuthorship W2924131876A5023733265 @default.
• W2924131876 hasAuthorship W2924131876A5048063937 @default.
• W2924131876 hasConcept C104317684 @default.
• W2924131876 hasConcept C106131492 @default.
• W2924131876 hasConcept C11413529 @default.
• W2924131876 hasConcept C119857082 @default.
• W2924131876 hasConcept C126255220 @default.
• W2924131876 hasConcept C134306372 @default.
• W2924131876 hasConcept C14036430 @default.
• W2924131876 hasConcept C140779682 @default.
• W2924131876 hasConcept C154945302 @default.
• W2924131876 hasConcept C15627037 @default.
• W2924131876 hasConcept C158968445 @default.
• W2924131876 hasConcept C185592680 @default.
• W2924131876 hasConcept C186633575 @default.
• W2924131876 hasConcept C31972630 @default.
• W2924131876 hasConcept C33923547 @default.
• W2924131876 hasConcept C41008148 @default.
• W2924131876 hasConcept C50644808 @default.
• W2924131876 hasConcept C55493867 @default.
• W2924131876 hasConcept C63479239 @default.
• W2924131876 hasConcept C78458016 @default.
• W2924131876 hasConcept C86803240 @default.
• W2924131876 hasConcept C97256817 @default.
• W2924131876 hasConceptScore W2924131876C104317684 @default.
• W2924131876 hasConceptScore W2924131876C106131492 @default.
• W2924131876 hasConceptScore W2924131876C11413529 @default.
• W2924131876 hasConceptScore W2924131876C119857082 @default.
• W2924131876 hasConceptScore W2924131876C126255220 @default.
• W2924131876 hasConceptScore W2924131876C134306372 @default.
• W2924131876 hasConceptScore W2924131876C14036430 @default.
• W2924131876 hasConceptScore W2924131876C140779682 @default.
• W2924131876 hasConceptScore W2924131876C154945302 @default.
• W2924131876 hasConceptScore W2924131876C15627037 @default.
• W2924131876 hasConceptScore W2924131876C158968445 @default.
• W2924131876 hasConceptScore W2924131876C185592680 @default.
• W2924131876 hasConceptScore W2924131876C186633575 @default.
• W2924131876 hasConceptScore W2924131876C31972630 @default.
• W2924131876 hasConceptScore W2924131876C33923547 @default.
• W2924131876 hasConceptScore W2924131876C41008148 @default.
• W2924131876 hasConceptScore W2924131876C50644808 @default.
• W2924131876 hasConceptScore W2924131876C55493867 @default.
• W2924131876 hasConceptScore W2924131876C63479239 @default.
• W2924131876 hasConceptScore W2924131876C78458016 @default.
• W2924131876 hasConceptScore W2924131876C86803240 @default.
• W2924131876 hasConceptScore W2924131876C97256817 @default.
• W2924131876 hasLocation W29241318761 @default.
• W2924131876 hasOpenAccess W2924131876 @default.
• W2924131876 hasPrimaryLocation W29241318761 @default.
• W2924131876 hasRelatedWork W1039514347 @default.
• W2924131876 hasRelatedWork W1638543427 @default.
• W2924131876 hasRelatedWork W1772464306 @default.
• W2924131876 hasRelatedWork W2034577990 @default.
• W2924131876 hasRelatedWork W2520915071 @default.
• W2924131876 hasRelatedWork W2789621689 @default.
• W2924131876 hasRelatedWork W2949205035 @default.
• W2924131876 hasRelatedWork W2952132225 @default.
• W2924131876 hasRelatedWork W2963622172 @default.
• W2924131876 hasRelatedWork W2963959597 @default.
• W2924131876 hasRelatedWork W2964190872 @default.
• W2924131876 hasRelatedWork W2995941914 @default.
• W2924131876 hasRelatedWork W2998438489 @default.
• W2924131876 hasRelatedWork W3000052276 @default.
• W2924131876 hasRelatedWork W3042684314 @default.
• W2924131876 hasRelatedWork W3148100789 @default.
• W2924131876 hasRelatedWork W3186092959 @default.

• next