Matches in SemOpenAlex for { <https://semopenalex.org/work/W3131036299> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W3131036299 abstract "Negative pretraining is a prominent sequential learning effect of neural networks where a pretrained model obtains a worse generalization performance than a model that is trained from scratch when either are trained on a target task. We conceptualize the ingredients of this problem setting and examine the negative pretraining effect experimentally by providing three interventions to remove and fix it. First, acting on the learning process, altering the learning rate after pretraining can yield even better results than training directly on the target task. Second, on the learning task-level, we intervene by increasing the discretization of data distribution changes from start to target task instead of “jumping” to a target task. Finally at the model-level, resetting network biases to larger values likewise removes negative pretraining effects, albeit to a smaller degree. With these intervention experiments, we aim to provide new evidence to help understand the subtle influences that neural network training and pretraining can have on final generalization performance on a target task in the context of negative pretraining." @default.
- W3131036299 created "2021-03-01" @default.
- W3131036299 creator A5020127805 @default.
- W3131036299 creator A5029069358 @default.
- W3131036299 creator A5033094522 @default.
- W3131036299 creator A5057460079 @default.
- W3131036299 creator A5081580957 @default.
- W3131036299 date "2021-05-04" @default.
- W3131036299 modified "2023-10-01" @default.
- W3131036299 title "The Negative Pretraining Effect in Sequential Deep Learning and Three Ways to Fix It" @default.
- W3131036299 hasPublicationYear "2021" @default.
- W3131036299 type Work @default.
- W3131036299 sameAs 3131036299 @default.
- W3131036299 citedByCount "0" @default.
- W3131036299 crossrefType "journal-article" @default.
- W3131036299 hasAuthorship W3131036299A5020127805 @default.
- W3131036299 hasAuthorship W3131036299A5029069358 @default.
- W3131036299 hasAuthorship W3131036299A5033094522 @default.
- W3131036299 hasAuthorship W3131036299A5057460079 @default.
- W3131036299 hasAuthorship W3131036299A5081580957 @default.
- W3131036299 hasConcept C134306372 @default.
- W3131036299 hasConcept C151730666 @default.
- W3131036299 hasConcept C154945302 @default.
- W3131036299 hasConcept C15744967 @default.
- W3131036299 hasConcept C162324750 @default.
- W3131036299 hasConcept C177148314 @default.
- W3131036299 hasConcept C180747234 @default.
- W3131036299 hasConcept C187736073 @default.
- W3131036299 hasConcept C2779343474 @default.
- W3131036299 hasConcept C2780451532 @default.
- W3131036299 hasConcept C33923547 @default.
- W3131036299 hasConcept C41008148 @default.
- W3131036299 hasConcept C50644808 @default.
- W3131036299 hasConcept C86803240 @default.
- W3131036299 hasConceptScore W3131036299C134306372 @default.
- W3131036299 hasConceptScore W3131036299C151730666 @default.
- W3131036299 hasConceptScore W3131036299C154945302 @default.
- W3131036299 hasConceptScore W3131036299C15744967 @default.
- W3131036299 hasConceptScore W3131036299C162324750 @default.
- W3131036299 hasConceptScore W3131036299C177148314 @default.
- W3131036299 hasConceptScore W3131036299C180747234 @default.
- W3131036299 hasConceptScore W3131036299C187736073 @default.
- W3131036299 hasConceptScore W3131036299C2779343474 @default.
- W3131036299 hasConceptScore W3131036299C2780451532 @default.
- W3131036299 hasConceptScore W3131036299C33923547 @default.
- W3131036299 hasConceptScore W3131036299C41008148 @default.
- W3131036299 hasConceptScore W3131036299C50644808 @default.
- W3131036299 hasConceptScore W3131036299C86803240 @default.
- W3131036299 hasLocation W31310362991 @default.
- W3131036299 hasOpenAccess W3131036299 @default.
- W3131036299 hasPrimaryLocation W31310362991 @default.
- W3131036299 hasRelatedWork W1211899705 @default.
- W3131036299 hasRelatedWork W1511140346 @default.
- W3131036299 hasRelatedWork W2011248157 @default.
- W3131036299 hasRelatedWork W2013716719 @default.
- W3131036299 hasRelatedWork W2030722779 @default.
- W3131036299 hasRelatedWork W2044640966 @default.
- W3131036299 hasRelatedWork W2065155634 @default.
- W3131036299 hasRelatedWork W2074847955 @default.
- W3131036299 hasRelatedWork W2085268669 @default.
- W3131036299 hasRelatedWork W2315493793 @default.
- W3131036299 hasRelatedWork W2398714964 @default.
- W3131036299 hasRelatedWork W2607412107 @default.
- W3131036299 hasRelatedWork W2624418577 @default.
- W3131036299 hasRelatedWork W2944525093 @default.
- W3131036299 hasRelatedWork W2977701070 @default.
- W3131036299 hasRelatedWork W2980892059 @default.
- W3131036299 hasRelatedWork W2997306128 @default.
- W3131036299 hasRelatedWork W3048509029 @default.
- W3131036299 hasRelatedWork W3098079787 @default.
- W3131036299 hasRelatedWork W323589100 @default.
- W3131036299 isParatext "false" @default.
- W3131036299 isRetracted "false" @default.
- W3131036299 magId "3131036299" @default.
- W3131036299 workType "article" @default.