FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3037397743> ?p ?o ?g. }

• W3037397743 abstract "Abstract Recent work in signal propagation theory has shown that dropout limits the depth to which information can propagate through a neural network. In this paper, we investigate the effect of initialisation on training speed and generalisation for ReLU networks within this depth limit. We ask the following research question: given that critical initialisation is crucial for training at large depth, if dropout limits the depth at which networks are trainable, does initialising critically still matter? We conduct a large-scale controlled experiment, and perform a statistical analysis of over 12 000 trained networks. We find that (1) trainable networks show no statistically significant difference in performance over a wide range of non-critical initialisations; (2) for initialisations that show a statistically significant difference, the net effect on performance is small; (3) only extreme initialisations (very small or very large) perform worse than criticality. These findings also apply to standard ReLU networks of moderate depth as a special case of zero dropout. Our results therefore suggest that, in the shallow-to-moderate depth setting, critical initialisation provides zero performance gains when compared to off-critical initialisations and that searching for off-critical initialisations that might improve training speed or generalisation, is likely to be a fruitless endeavour." @default.
• W3037397743 created "2020-07-02" @default.
• W3037397743 creator A5017043594 @default.
• W3037397743 creator A5020437129 @default.
• W3037397743 creator A5040305929 @default.
• W3037397743 creator A5046382731 @default.
• W3037397743 creator A5047706306 @default.
• W3037397743 creator A5068297734 @default.
• W3037397743 creator A5076529776 @default.
• W3037397743 creator A5080963336 @default.
• W3037397743 creator A5080981345 @default.
• W3037397743 date "2020-10-01" @default.
• W3037397743 modified "2023-09-23" @default.
• W3037397743 title "If dropout limits trainable depth, does critical initialisation still matter? A large-scale statistical analysis on ReLU networks" @default.
• W3037397743 cites W1533861849 @default.
• W3037397743 cites W1565746575 @default.
• W3037397743 cites W1974075673 @default.
• W3037397743 cites W1974758710 @default.
• W3037397743 cites W2000950277 @default.
• W3037397743 cites W2036893104 @default.
• W3037397743 cites W2095705004 @default.
• W3037397743 cites W2107031757 @default.
• W3037397743 cites W2112796928 @default.
• W3037397743 cites W2165466912 @default.
• W3037397743 cites W2169284845 @default.
• W3037397743 cites W2294567968 @default.
• W3037397743 cites W2423689290 @default.
• W3037397743 cites W2597852078 @default.
• W3037397743 cites W2750384547 @default.
• W3037397743 cites W2753358588 @default.
• W3037397743 cites W2890166761 @default.
• W3037397743 cites W2891962546 @default.
• W3037397743 cites W2949608135 @default.
• W3037397743 cites W2951595529 @default.
• W3037397743 cites W2952088488 @default.
• W3037397743 cites W2952316226 @default.
• W3037397743 cites W2952825952 @default.
• W3037397743 cites W2962804662 @default.
• W3037397743 cites W2963504252 @default.
• W3037397743 cites W2963679562 @default.
• W3037397743 cites W2964059111 @default.
• W3037397743 cites W2964065616 @default.
• W3037397743 cites W3002842489 @default.
• W3037397743 cites W3037590790 @default.
• W3037397743 cites W3118608800 @default.
• W3037397743 cites W35527955 @default.
• W3037397743 cites W4919037 @default.
• W3037397743 cites W767037412 @default.
• W3037397743 doi "https://doi.org/10.1016/j.patrec.2020.06.025" @default.
• W3037397743 hasPublicationYear "2020" @default.
• W3037397743 type Work @default.
• W3037397743 sameAs 3037397743 @default.
• W3037397743 citedByCount "0" @default.
• W3037397743 crossrefType "journal-article" @default.
• W3037397743 hasAuthorship W3037397743A5017043594 @default.
• W3037397743 hasAuthorship W3037397743A5020437129 @default.
• W3037397743 hasAuthorship W3037397743A5040305929 @default.
• W3037397743 hasAuthorship W3037397743A5046382731 @default.
• W3037397743 hasAuthorship W3037397743A5047706306 @default.
• W3037397743 hasAuthorship W3037397743A5068297734 @default.
• W3037397743 hasAuthorship W3037397743A5076529776 @default.
• W3037397743 hasAuthorship W3037397743A5080963336 @default.
• W3037397743 hasAuthorship W3037397743A5080981345 @default.
• W3037397743 hasBestOaLocation W30373977432 @default.
• W3037397743 hasConcept C11413529 @default.
• W3037397743 hasConcept C119857082 @default.
• W3037397743 hasConcept C121332964 @default.
• W3037397743 hasConcept C125611927 @default.
• W3037397743 hasConcept C127413603 @default.
• W3037397743 hasConcept C134306372 @default.
• W3037397743 hasConcept C138885662 @default.
• W3037397743 hasConcept C146978453 @default.
• W3037397743 hasConcept C151201525 @default.
• W3037397743 hasConcept C154945302 @default.
• W3037397743 hasConcept C185544564 @default.
• W3037397743 hasConcept C204323151 @default.
• W3037397743 hasConcept C2776145597 @default.
• W3037397743 hasConcept C2778755073 @default.
• W3037397743 hasConcept C2780813799 @default.
• W3037397743 hasConcept C33923547 @default.
• W3037397743 hasConcept C41008148 @default.
• W3037397743 hasConcept C41895202 @default.
• W3037397743 hasConcept C50644808 @default.
• W3037397743 hasConcept C62520636 @default.
• W3037397743 hasConceptScore W3037397743C11413529 @default.
• W3037397743 hasConceptScore W3037397743C119857082 @default.
• W3037397743 hasConceptScore W3037397743C121332964 @default.
• W3037397743 hasConceptScore W3037397743C125611927 @default.
• W3037397743 hasConceptScore W3037397743C127413603 @default.
• W3037397743 hasConceptScore W3037397743C134306372 @default.
• W3037397743 hasConceptScore W3037397743C138885662 @default.
• W3037397743 hasConceptScore W3037397743C146978453 @default.
• W3037397743 hasConceptScore W3037397743C151201525 @default.
• W3037397743 hasConceptScore W3037397743C154945302 @default.
• W3037397743 hasConceptScore W3037397743C185544564 @default.
• W3037397743 hasConceptScore W3037397743C204323151 @default.
• W3037397743 hasConceptScore W3037397743C2776145597 @default.
• W3037397743 hasConceptScore W3037397743C2778755073 @default.
• W3037397743 hasConceptScore W3037397743C2780813799 @default.
• W3037397743 hasConceptScore W3037397743C33923547 @default.

• next