FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3099488075> ?p ?o ?g. }

• W3099488075 abstract "Deep learning is often criticized by two serious issues which rarely exist in natural nervous systems: overfitting and catastrophic forgetting. It can even memorize randomly labelled data, which has little knowledge behind the instance-label pairs. When a deep network continually learns over time by accommodating new tasks, it usually quickly overwrites the knowledge learned from previous tasks. Referred to as the {it neural variability}, it is well-known in neuroscience that human brain reactions exhibit substantial variability even in response to the same stimulus. This mechanism balances accuracy and plasticity/flexibility in the motor learning of natural nervous systems. Thus it motivates us to design a similar mechanism named {it artificial neural variability} (ANV), which helps artificial neural networks learn some advantages from ``natural'' neural networks. We rigorously prove that ANV plays as an implicit regularizer of the mutual information between the training data and the learned model. This result theoretically guarantees ANV a strictly improved generalizability, robustness to label noise, and robustness to catastrophic forgetting. We then devise a {it neural variable risk minimization} (NVRM) framework and {it neural variable optimizers} to achieve ANV for conventional network architectures in practice. The empirical studies demonstrate that NVRM can effectively relieve overfitting, label noise memorization, and catastrophic forgetting at negligible costs. footnote{Code: url{https://github.com/zeke-xie/artificial-neural-variability-for-deep-learning}." @default.
• W3099488075 created "2020-11-23" @default.
• W3099488075 creator A5001819736 @default.
• W3099488075 creator A5017703463 @default.
• W3099488075 creator A5032483871 @default.
• W3099488075 creator A5060421432 @default.
• W3099488075 creator A5066773635 @default.
• W3099488075 creator A5072744508 @default.
• W3099488075 date "2020-11-12" @default.
• W3099488075 modified "2023-10-03" @default.
• W3099488075 title "Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting" @default.
• W3099488075 cites W104184427 @default.
• W3099488075 cites W1522301498 @default.
• W3099488075 cites W1570448133 @default.
• W3099488075 cites W1682403713 @default.
• W3099488075 cites W1971735090 @default.
• W3099488075 cites W1984367183 @default.
• W3099488075 cites W1988245436 @default.
• W3099488075 cites W1998794714 @default.
• W3099488075 cites W2004132309 @default.
• W3099488075 cites W2014384147 @default.
• W3099488075 cites W2029029543 @default.
• W3099488075 cites W2040499723 @default.
• W3099488075 cites W2041540302 @default.
• W3099488075 cites W2047229728 @default.
• W3099488075 cites W2099111195 @default.
• W3099488075 cites W2103496339 @default.
• W3099488075 cites W2105380935 @default.
• W3099488075 cites W2108677974 @default.
• W3099488075 cites W2113839990 @default.
• W3099488075 cites W2124136621 @default.
• W3099488075 cites W2124227140 @default.
• W3099488075 cites W2132211083 @default.
• W3099488075 cites W2149479912 @default.
• W3099488075 cites W2164411961 @default.
• W3099488075 cites W2167433878 @default.
• W3099488075 cites W2171776652 @default.
• W3099488075 cites W2194775991 @default.
• W3099488075 cites W2257979135 @default.
• W3099488075 cites W2263490141 @default.
• W3099488075 cites W2560647685 @default.
• W3099488075 cites W2592929672 @default.
• W3099488075 cites W2605372163 @default.
• W3099488075 cites W2737492962 @default.
• W3099488075 cites W2768267830 @default.
• W3099488075 cites W2788388592 @default.
• W3099488075 cites W2804386825 @default.
• W3099488075 cites W2886067286 @default.
• W3099488075 cites W2890638274 @default.
• W3099488075 cites W2891160178 @default.
• W3099488075 cites W2899748887 @default.
• W3099488075 cites W2900832763 @default.
• W3099488075 cites W2902227449 @default.
• W3099488075 cites W2908510526 @default.
• W3099488075 cites W2912811302 @default.
• W3099488075 cites W2919115771 @default.
• W3099488075 cites W2938101602 @default.
• W3099488075 cites W2946668020 @default.
• W3099488075 cites W2947518183 @default.
• W3099488075 cites W2951266961 @default.
• W3099488075 cites W2962712513 @default.
• W3099488075 cites W2962720772 @default.
• W3099488075 cites W2962835968 @default.
• W3099488075 cites W2963096987 @default.
• W3099488075 cites W2963163009 @default.
• W3099488075 cites W2963376662 @default.
• W3099488075 cites W2963588172 @default.
• W3099488075 cites W2963735582 @default.
• W3099488075 cites W2963739978 @default.
• W3099488075 cites W2963862692 @default.
• W3099488075 cites W2963959597 @default.
• W3099488075 cites W2970330753 @default.
• W3099488075 cites W2970490659 @default.
• W3099488075 cites W3006315234 @default.
• W3099488075 cites W3008788102 @default.
• W3099488075 cites W3035600444 @default.
• W3099488075 cites W3037278715 @default.
• W3099488075 cites W3109394096 @default.
• W3099488075 cites W3118431319 @default.
• W3099488075 cites W3118608800 @default.
• W3099488075 cites W3137695714 @default.
• W3099488075 cites W3146803896 @default.
• W3099488075 cites W3157424867 @default.
• W3099488075 cites W3172895807 @default.
• W3099488075 cites W3200038001 @default.
• W3099488075 doi "https://doi.org/10.48550/arxiv.2011.06220" @default.
• W3099488075 hasPublicationYear "2020" @default.
• W3099488075 type Work @default.
• W3099488075 sameAs 3099488075 @default.
• W3099488075 citedByCount "5" @default.
• W3099488075 countsByYear W30994880752020 @default.
• W3099488075 countsByYear W30994880752021 @default.
• W3099488075 crossrefType "posted-content" @default.
• W3099488075 hasAuthorship W3099488075A5001819736 @default.
• W3099488075 hasAuthorship W3099488075A5017703463 @default.
• W3099488075 hasAuthorship W3099488075A5032483871 @default.
• W3099488075 hasAuthorship W3099488075A5060421432 @default.
• W3099488075 hasAuthorship W3099488075A5066773635 @default.
• W3099488075 hasAuthorship W3099488075A5072744508 @default.
• W3099488075 hasBestOaLocation W30994880751 @default.

• next