Matches in SemOpenAlex for { <https://semopenalex.org/work/W3034730115> ?p ?o ?g. }
- W3034730115 abstract "Training neural networks with binary weights and activations is a challenging problem due to the lack of gradients and difficulty of optimization over discrete weights. Many successful experimental results have been achieved with empirical straight-through (ST) approaches, proposing a variety of ad-hoc rules for propagating gradients through non-differentiable activations and updating discrete weights. At the same time, ST methods can be truly derived as estimators in the stochastic binary network (SBN) model with Bernoulli weights. We advance these derivations to a more complete and systematic study. We analyze properties, estimation accuracy, obtain different forms of correct ST estimators for activations and weights, explain existing empirical approaches and their shortcomings, explain how latent weights arise from the mirror descent method when optimizing over probabilities. This allows to reintroduce, once empirical, ST methods as sound approximations, apply them with clarity and develop further improvements." @default.
- W3034730115 created "2020-06-19" @default.
- W3034730115 creator A5068080640 @default.
- W3034730115 creator A5076376532 @default.
- W3034730115 date "2021-05-04" @default.
- W3034730115 modified "2023-09-27" @default.
- W3034730115 title "Reintroducing Straight-Through Estimators as Principled Methods for Stochastic Binary Networks" @default.
- W3034730115 cites W1505731132 @default.
- W3034730115 cites W1677182931 @default.
- W3034730115 cites W1836465849 @default.
- W3034730115 cites W1999085092 @default.
- W3034730115 cites W2074078071 @default.
- W3034730115 cites W2083380015 @default.
- W3034730115 cites W2095705004 @default.
- W3034730115 cites W2097268041 @default.
- W3034730115 cites W2108677974 @default.
- W3034730115 cites W2119717200 @default.
- W3034730115 cites W2135354436 @default.
- W3034730115 cites W2187669537 @default.
- W3034730115 cites W2242818861 @default.
- W3034730115 cites W2300242332 @default.
- W3034730115 cites W2314470091 @default.
- W3034730115 cites W2396976214 @default.
- W3034730115 cites W2469490737 @default.
- W3034730115 cites W2547875792 @default.
- W3034730115 cites W2602076750 @default.
- W3034730115 cites W2604700561 @default.
- W3034730115 cites W2740797857 @default.
- W3034730115 cites W2740840795 @default.
- W3034730115 cites W2749241159 @default.
- W3034730115 cites W2753301142 @default.
- W3034730115 cites W2887447938 @default.
- W3034730115 cites W2890984855 @default.
- W3034730115 cites W2907087576 @default.
- W3034730115 cites W2909637611 @default.
- W3034730115 cites W2924522543 @default.
- W3034730115 cites W2937923353 @default.
- W3034730115 cites W2948482439 @default.
- W3034730115 cites W2949593890 @default.
- W3034730115 cites W2951829782 @default.
- W3034730115 cites W2952165242 @default.
- W3034730115 cites W2962706989 @default.
- W3034730115 cites W2962919781 @default.
- W3034730115 cites W2963114950 @default.
- W3034730115 cites W2963162885 @default.
- W3034730115 cites W2963619462 @default.
- W3034730115 cites W2963851840 @default.
- W3034730115 cites W2963891249 @default.
- W3034730115 cites W2963901583 @default.
- W3034730115 cites W2964121744 @default.
- W3034730115 cites W2970328156 @default.
- W3034730115 cites W2970971581 @default.
- W3034730115 cites W2977484217 @default.
- W3034730115 cites W2996538923 @default.
- W3034730115 cites W3004061291 @default.
- W3034730115 cites W3015508253 @default.
- W3034730115 cites W3026289162 @default.
- W3034730115 cites W3042978870 @default.
- W3034730115 cites W3102288316 @default.
- W3034730115 cites W3106323762 @default.
- W3034730115 cites W3114371569 @default.
- W3034730115 cites W3159700743 @default.
- W3034730115 cites W3174185214 @default.
- W3034730115 cites W3203323829 @default.
- W3034730115 cites W3022571137 @default.
- W3034730115 cites W3034950653 @default.
- W3034730115 cites W3122010520 @default.
- W3034730115 hasPublicationYear "2021" @default.
- W3034730115 type Work @default.
- W3034730115 sameAs 3034730115 @default.
- W3034730115 citedByCount "1" @default.
- W3034730115 countsByYear W30347301152021 @default.
- W3034730115 crossrefType "journal-article" @default.
- W3034730115 hasAuthorship W3034730115A5068080640 @default.
- W3034730115 hasAuthorship W3034730115A5076376532 @default.
- W3034730115 hasConcept C105795698 @default.
- W3034730115 hasConcept C11413529 @default.
- W3034730115 hasConcept C126255220 @default.
- W3034730115 hasConcept C127413603 @default.
- W3034730115 hasConcept C134306372 @default.
- W3034730115 hasConcept C136197465 @default.
- W3034730115 hasConcept C146978453 @default.
- W3034730115 hasConcept C152361515 @default.
- W3034730115 hasConcept C154945302 @default.
- W3034730115 hasConcept C185429906 @default.
- W3034730115 hasConcept C202615002 @default.
- W3034730115 hasConcept C33923547 @default.
- W3034730115 hasConcept C41008148 @default.
- W3034730115 hasConcept C46802686 @default.
- W3034730115 hasConcept C48372109 @default.
- W3034730115 hasConcept C50644808 @default.
- W3034730115 hasConcept C94375191 @default.
- W3034730115 hasConceptScore W3034730115C105795698 @default.
- W3034730115 hasConceptScore W3034730115C11413529 @default.
- W3034730115 hasConceptScore W3034730115C126255220 @default.
- W3034730115 hasConceptScore W3034730115C127413603 @default.
- W3034730115 hasConceptScore W3034730115C134306372 @default.
- W3034730115 hasConceptScore W3034730115C136197465 @default.
- W3034730115 hasConceptScore W3034730115C146978453 @default.
- W3034730115 hasConceptScore W3034730115C152361515 @default.