FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W1583776211> ?p ?o ?g. }

• W1583776211 abstract "Stochastic neurons can be useful for a number of reasons in deep learning models, but in many cases they pose a challenging problem: how to estimate the gradient of a loss function with respect to the input of such s tochastic neurons, i.e., can we “back-propagate” through these stochastic neurons? We examine this question, existing approaches, and present two novel families of solutions, applicable in different settings. In particular, it is demonstrate d that a simple biologically plausible formula gives rise to an an unbiased (but noisy) estimator of the gradient with respect to a binary stochastic neuron firing proba bility. Unlike other estimators which view the noise as a small perturbation in order to estimate gradients by finite differences, this estimator is unbiased even w ithout assuming that the stochastic perturbation is small. This estimator is also in teresting because it can be applied in very general settings which do not allow gradient back-propagation, including the estimation of the gradient with respect to futur e rewards, as required in reinforcement learning setups. We also propose an approach to approximating this unbiased but high-variance estimator by learning to predict it using a biased estimator. The second approach we propose assumes that an estimator of the gradient can be back-propagated and it provides an unbiased estimator of the gradient, but can only work with non-linearities unlike the hard threshold, but like the rectifier, that are not flat for all of their range. This is similar to trad itional sigmoidal units but has the advantage that for many inputs, a hard decision (e.g., a 0 output) can be produced, which would be convenient for conditional computation and achieving sparse representations and sparse gradients." @default.
• W1583776211 created "2016-06-24" @default.
• W1583776211 creator A5086198262 @default.
• W1583776211 date "2013-05-14" @default.
• W1583776211 modified "2023-09-27" @default.
• W1583776211 title "Estimating or Propagating Gradients Through Stochastic Neurons" @default.
• W1583776211 cites W1498436455 @default.
• W1583776211 cites W1613249581 @default.
• W1583776211 cites W1665214252 @default.
• W1583776211 cites W1904365287 @default.
• W1583776211 cites W2025768430 @default.
• W1583776211 cites W2041176801 @default.
• W1583776211 cites W2072128103 @default.
• W1583776211 cites W2099257174 @default.
• W1583776211 cites W2124289529 @default.
• W1583776211 cites W2136922672 @default.
• W1583776211 cites W2156387975 @default.
• W1583776211 cites W2163605009 @default.
• W1583776211 cites W2294059674 @default.
• W1583776211 cites W2913932916 @default.
• W1583776211 cites W60493759 @default.
• W1583776211 hasPublicationYear "2013" @default.
• W1583776211 type Work @default.
• W1583776211 sameAs 1583776211 @default.
• W1583776211 citedByCount "45" @default.
• W1583776211 countsByYear W15837762112013 @default.
• W1583776211 countsByYear W15837762112014 @default.
• W1583776211 countsByYear W15837762112015 @default.
• W1583776211 countsByYear W15837762112016 @default.
• W1583776211 countsByYear W15837762112017 @default.
• W1583776211 countsByYear W15837762112018 @default.
• W1583776211 countsByYear W15837762112019 @default.
• W1583776211 countsByYear W15837762112020 @default.
• W1583776211 countsByYear W15837762112021 @default.
• W1583776211 crossrefType "posted-content" @default.
• W1583776211 hasAuthorship W1583776211A5086198262 @default.
• W1583776211 hasConcept C105795698 @default.
• W1583776211 hasConcept C121332964 @default.
• W1583776211 hasConcept C121955636 @default.
• W1583776211 hasConcept C126255220 @default.
• W1583776211 hasConcept C134962040 @default.
• W1583776211 hasConcept C14036430 @default.
• W1583776211 hasConcept C144133560 @default.
• W1583776211 hasConcept C154945302 @default.
• W1583776211 hasConcept C164172150 @default.
• W1583776211 hasConcept C165646398 @default.
• W1583776211 hasConcept C177918212 @default.
• W1583776211 hasConcept C185429906 @default.
• W1583776211 hasConcept C191393472 @default.
• W1583776211 hasConcept C196083921 @default.
• W1583776211 hasConcept C2779880469 @default.
• W1583776211 hasConcept C28826006 @default.
• W1583776211 hasConcept C33923547 @default.
• W1583776211 hasConcept C41008148 @default.
• W1583776211 hasConcept C48372109 @default.
• W1583776211 hasConcept C50644808 @default.
• W1583776211 hasConcept C62520636 @default.
• W1583776211 hasConcept C78458016 @default.
• W1583776211 hasConcept C81388566 @default.
• W1583776211 hasConcept C8272713 @default.
• W1583776211 hasConcept C86803240 @default.
• W1583776211 hasConcept C94375191 @default.
• W1583776211 hasConcept C97541855 @default.
• W1583776211 hasConceptScore W1583776211C105795698 @default.
• W1583776211 hasConceptScore W1583776211C121332964 @default.
• W1583776211 hasConceptScore W1583776211C121955636 @default.
• W1583776211 hasConceptScore W1583776211C126255220 @default.
• W1583776211 hasConceptScore W1583776211C134962040 @default.
• W1583776211 hasConceptScore W1583776211C14036430 @default.
• W1583776211 hasConceptScore W1583776211C144133560 @default.
• W1583776211 hasConceptScore W1583776211C154945302 @default.
• W1583776211 hasConceptScore W1583776211C164172150 @default.
• W1583776211 hasConceptScore W1583776211C165646398 @default.
• W1583776211 hasConceptScore W1583776211C177918212 @default.
• W1583776211 hasConceptScore W1583776211C185429906 @default.
• W1583776211 hasConceptScore W1583776211C191393472 @default.
• W1583776211 hasConceptScore W1583776211C196083921 @default.
• W1583776211 hasConceptScore W1583776211C2779880469 @default.
• W1583776211 hasConceptScore W1583776211C28826006 @default.
• W1583776211 hasConceptScore W1583776211C33923547 @default.
• W1583776211 hasConceptScore W1583776211C41008148 @default.
• W1583776211 hasConceptScore W1583776211C48372109 @default.
• W1583776211 hasConceptScore W1583776211C50644808 @default.
• W1583776211 hasConceptScore W1583776211C62520636 @default.
• W1583776211 hasConceptScore W1583776211C78458016 @default.
• W1583776211 hasConceptScore W1583776211C81388566 @default.
• W1583776211 hasConceptScore W1583776211C8272713 @default.
• W1583776211 hasConceptScore W1583776211C86803240 @default.
• W1583776211 hasConceptScore W1583776211C94375191 @default.
• W1583776211 hasConceptScore W1583776211C97541855 @default.
• W1583776211 hasLocation W15837762111 @default.
• W1583776211 hasOpenAccess W1583776211 @default.
• W1583776211 hasPrimaryLocation W15837762111 @default.
• W1583776211 hasRelatedWork W1496559305 @default.
• W1583776211 hasRelatedWork W1597459461 @default.
• W1583776211 hasRelatedWork W1686810756 @default.
• W1583776211 hasRelatedWork W1904365287 @default.
• W1583776211 hasRelatedWork W1909320841 @default.
• W1583776211 hasRelatedWork W1959608418 @default.
• W1583776211 hasRelatedWork W2064675550 @default.

• next