Matches in SemOpenAlex for { <https://semopenalex.org/work/W4300155862> ?p ?o ?g. }
Showing items 1 to 89 of
89
with 100 items per page.
- W4300155862 abstract "Vanishing (and exploding) gradients effect is a common problem for recurrent neural networks with nonlinear activation functions which use backpropagation method for calculation of derivatives. Deep feedforward neural networks with many hidden layers also suffer from this effect. In this paper we propose a novel universal technique that makes the norm of the gradient stay in the suitable range. We construct a way to estimate a contribution of each training example to the norm of the long-term components of the target function s gradient. Using this subroutine we can construct mini-batches for the stochastic gradient descent (SGD) training that leads to high performance and accuracy of the trained network even for very complex tasks. We provide a straightforward mathematical estimation of minibatch s impact on for the gradient norm and prove its correctness theoretically. To check our framework experimentally we use some special synthetic benchmarks for testing RNNs on ability to capture long-term dependencies. Our network can detect links between events in the (temporal) sequence at the range approx. 100 and longer." @default.
- W4300155862 created "2022-10-03" @default.
- W4300155862 creator A5006823112 @default.
- W4300155862 creator A5039085915 @default.
- W4300155862 date "2016-06-24" @default.
- W4300155862 modified "2023-10-16" @default.
- W4300155862 title "Sampling-based Gradient Regularization for Capturing Long-Term Dependencies in Recurrent Neural Networks" @default.
- W4300155862 doi "https://doi.org/10.48550/arxiv.1606.07767" @default.
- W4300155862 hasPublicationYear "2016" @default.
- W4300155862 type Work @default.
- W4300155862 citedByCount "0" @default.
- W4300155862 crossrefType "posted-content" @default.
- W4300155862 hasAuthorship W4300155862A5006823112 @default.
- W4300155862 hasAuthorship W4300155862A5039085915 @default.
- W4300155862 hasBestOaLocation W43001558621 @default.
- W4300155862 hasConcept C111919701 @default.
- W4300155862 hasConcept C11413529 @default.
- W4300155862 hasConcept C115680565 @default.
- W4300155862 hasConcept C121332964 @default.
- W4300155862 hasConcept C127413603 @default.
- W4300155862 hasConcept C133731056 @default.
- W4300155862 hasConcept C147168706 @default.
- W4300155862 hasConcept C153258448 @default.
- W4300155862 hasConcept C154945302 @default.
- W4300155862 hasConcept C155032097 @default.
- W4300155862 hasConcept C158622935 @default.
- W4300155862 hasConcept C159985019 @default.
- W4300155862 hasConcept C17744445 @default.
- W4300155862 hasConcept C191795146 @default.
- W4300155862 hasConcept C192562407 @default.
- W4300155862 hasConcept C199360897 @default.
- W4300155862 hasConcept C199539241 @default.
- W4300155862 hasConcept C204323151 @default.
- W4300155862 hasConcept C206688291 @default.
- W4300155862 hasConcept C2776135515 @default.
- W4300155862 hasConcept C2780801425 @default.
- W4300155862 hasConcept C38365724 @default.
- W4300155862 hasConcept C38858127 @default.
- W4300155862 hasConcept C41008148 @default.
- W4300155862 hasConcept C50644808 @default.
- W4300155862 hasConcept C55439883 @default.
- W4300155862 hasConcept C61797465 @default.
- W4300155862 hasConcept C62520636 @default.
- W4300155862 hasConcept C96147967 @default.
- W4300155862 hasConceptScore W4300155862C111919701 @default.
- W4300155862 hasConceptScore W4300155862C11413529 @default.
- W4300155862 hasConceptScore W4300155862C115680565 @default.
- W4300155862 hasConceptScore W4300155862C121332964 @default.
- W4300155862 hasConceptScore W4300155862C127413603 @default.
- W4300155862 hasConceptScore W4300155862C133731056 @default.
- W4300155862 hasConceptScore W4300155862C147168706 @default.
- W4300155862 hasConceptScore W4300155862C153258448 @default.
- W4300155862 hasConceptScore W4300155862C154945302 @default.
- W4300155862 hasConceptScore W4300155862C155032097 @default.
- W4300155862 hasConceptScore W4300155862C158622935 @default.
- W4300155862 hasConceptScore W4300155862C159985019 @default.
- W4300155862 hasConceptScore W4300155862C17744445 @default.
- W4300155862 hasConceptScore W4300155862C191795146 @default.
- W4300155862 hasConceptScore W4300155862C192562407 @default.
- W4300155862 hasConceptScore W4300155862C199360897 @default.
- W4300155862 hasConceptScore W4300155862C199539241 @default.
- W4300155862 hasConceptScore W4300155862C204323151 @default.
- W4300155862 hasConceptScore W4300155862C206688291 @default.
- W4300155862 hasConceptScore W4300155862C2776135515 @default.
- W4300155862 hasConceptScore W4300155862C2780801425 @default.
- W4300155862 hasConceptScore W4300155862C38365724 @default.
- W4300155862 hasConceptScore W4300155862C38858127 @default.
- W4300155862 hasConceptScore W4300155862C41008148 @default.
- W4300155862 hasConceptScore W4300155862C50644808 @default.
- W4300155862 hasConceptScore W4300155862C55439883 @default.
- W4300155862 hasConceptScore W4300155862C61797465 @default.
- W4300155862 hasConceptScore W4300155862C62520636 @default.
- W4300155862 hasConceptScore W4300155862C96147967 @default.
- W4300155862 hasLocation W43001558621 @default.
- W4300155862 hasOpenAccess W4300155862 @default.
- W4300155862 hasPrimaryLocation W43001558621 @default.
- W4300155862 hasRelatedWork W2086999410 @default.
- W4300155862 hasRelatedWork W2117725694 @default.
- W4300155862 hasRelatedWork W2460601950 @default.
- W4300155862 hasRelatedWork W2767206870 @default.
- W4300155862 hasRelatedWork W2846499288 @default.
- W4300155862 hasRelatedWork W2949203910 @default.
- W4300155862 hasRelatedWork W3159389381 @default.
- W4300155862 hasRelatedWork W3170244987 @default.
- W4300155862 hasRelatedWork W3199510319 @default.
- W4300155862 hasRelatedWork W4300155862 @default.
- W4300155862 isParatext "false" @default.
- W4300155862 isRetracted "false" @default.
- W4300155862 workType "article" @default.