Matches in SemOpenAlex for { <https://semopenalex.org/work/W3135512773> ?p ?o ?g. }
- W3135512773 abstract "Traditional statistics forbids use of test data (a.k.a. holdout data) during training. Dwork et al. 2015 pointed out that current practices in machine learning, whereby researchers build upon each other's models, copying hyperparameters and even computer code -- amounts to implicitly training on the test set. Thus error rate on test data may not reflect the true population error. This observation initiated {em adaptive data analysis}, which provides evaluation mechanisms with guaranteed upper bounds on this difference. With statistical query (i.e. test accuracy) feedbacks, the best upper bound is fairly pessimistic: the deviation can hit a practically vacuous value if the number of models tested is quadratic in the size of the test set. In this work, we present a simple new estimate, {em Rip van Winkle's Razor}. It relies upon a new notion of textquotedblleft information contenttextquotedblright of a model: the amount of information that would have to be provided to an expert referee who is intimately familiar with the field and relevant science/math, and who has been just been woken up after falling asleep at the moment of the creation of the test data (like textquotedblleft Rip van Winkletextquotedblright of the famous fairy tale). This notion of information content is used to provide an estimate of the above deviation which is shown to be non-vacuous in many modern settings." @default.
- W3135512773 created "2021-03-15" @default.
- W3135512773 creator A5012440924 @default.
- W3135512773 creator A5079951047 @default.
- W3135512773 date "2021-02-25" @default.
- W3135512773 modified "2023-09-27" @default.
- W3135512773 title "Rip van Winkle's Razor: A Simple Estimate of Overfit to Test Data" @default.
- W3135512773 cites W1486367002 @default.
- W3135512773 cites W1982723861 @default.
- W3135512773 cites W2077641783 @default.
- W3135512773 cites W2079145130 @default.
- W3135512773 cites W2117539524 @default.
- W3135512773 cites W2163605009 @default.
- W3135512773 cites W2194775991 @default.
- W3135512773 cites W2225981128 @default.
- W3135512773 cites W2761453842 @default.
- W3135512773 cites W2806021783 @default.
- W3135512773 cites W2911109671 @default.
- W3135512773 cites W2914334103 @default.
- W3135512773 cites W2946813150 @default.
- W3135512773 cites W2946947608 @default.
- W3135512773 cites W2949117887 @default.
- W3135512773 cites W2951865872 @default.
- W3135512773 cites W2953937638 @default.
- W3135512773 cites W2963446712 @default.
- W3135512773 cites W2991486856 @default.
- W3135512773 cites W2991719478 @default.
- W3135512773 cites W3035131109 @default.
- W3135512773 cites W3099878876 @default.
- W3135512773 hasPublicationYear "2021" @default.
- W3135512773 type Work @default.
- W3135512773 sameAs 3135512773 @default.
- W3135512773 citedByCount "1" @default.
- W3135512773 countsByYear W31355127732021 @default.
- W3135512773 crossrefType "posted-content" @default.
- W3135512773 hasAuthorship W3135512773A5012440924 @default.
- W3135512773 hasAuthorship W3135512773A5079951047 @default.
- W3135512773 hasConcept C105795698 @default.
- W3135512773 hasConcept C111472728 @default.
- W3135512773 hasConcept C11413529 @default.
- W3135512773 hasConcept C121332964 @default.
- W3135512773 hasConcept C126255220 @default.
- W3135512773 hasConcept C127705205 @default.
- W3135512773 hasConcept C134306372 @default.
- W3135512773 hasConcept C138885662 @default.
- W3135512773 hasConcept C151730666 @default.
- W3135512773 hasConcept C154945302 @default.
- W3135512773 hasConcept C16910744 @default.
- W3135512773 hasConcept C169903167 @default.
- W3135512773 hasConcept C177264268 @default.
- W3135512773 hasConcept C17744445 @default.
- W3135512773 hasConcept C179254644 @default.
- W3135512773 hasConcept C199360897 @default.
- W3135512773 hasConcept C199539241 @default.
- W3135512773 hasConcept C22019652 @default.
- W3135512773 hasConcept C2776760102 @default.
- W3135512773 hasConcept C2777267654 @default.
- W3135512773 hasConcept C2779151265 @default.
- W3135512773 hasConcept C2780586882 @default.
- W3135512773 hasConcept C33923547 @default.
- W3135512773 hasConcept C41008148 @default.
- W3135512773 hasConcept C50644808 @default.
- W3135512773 hasConcept C74650414 @default.
- W3135512773 hasConcept C77553402 @default.
- W3135512773 hasConcept C86803240 @default.
- W3135512773 hasConceptScore W3135512773C105795698 @default.
- W3135512773 hasConceptScore W3135512773C111472728 @default.
- W3135512773 hasConceptScore W3135512773C11413529 @default.
- W3135512773 hasConceptScore W3135512773C121332964 @default.
- W3135512773 hasConceptScore W3135512773C126255220 @default.
- W3135512773 hasConceptScore W3135512773C127705205 @default.
- W3135512773 hasConceptScore W3135512773C134306372 @default.
- W3135512773 hasConceptScore W3135512773C138885662 @default.
- W3135512773 hasConceptScore W3135512773C151730666 @default.
- W3135512773 hasConceptScore W3135512773C154945302 @default.
- W3135512773 hasConceptScore W3135512773C16910744 @default.
- W3135512773 hasConceptScore W3135512773C169903167 @default.
- W3135512773 hasConceptScore W3135512773C177264268 @default.
- W3135512773 hasConceptScore W3135512773C17744445 @default.
- W3135512773 hasConceptScore W3135512773C179254644 @default.
- W3135512773 hasConceptScore W3135512773C199360897 @default.
- W3135512773 hasConceptScore W3135512773C199539241 @default.
- W3135512773 hasConceptScore W3135512773C22019652 @default.
- W3135512773 hasConceptScore W3135512773C2776760102 @default.
- W3135512773 hasConceptScore W3135512773C2777267654 @default.
- W3135512773 hasConceptScore W3135512773C2779151265 @default.
- W3135512773 hasConceptScore W3135512773C2780586882 @default.
- W3135512773 hasConceptScore W3135512773C33923547 @default.
- W3135512773 hasConceptScore W3135512773C41008148 @default.
- W3135512773 hasConceptScore W3135512773C50644808 @default.
- W3135512773 hasConceptScore W3135512773C74650414 @default.
- W3135512773 hasConceptScore W3135512773C77553402 @default.
- W3135512773 hasConceptScore W3135512773C86803240 @default.
- W3135512773 hasLocation W31355127731 @default.
- W3135512773 hasOpenAccess W3135512773 @default.
- W3135512773 hasPrimaryLocation W31355127731 @default.
- W3135512773 hasRelatedWork W1511364348 @default.
- W3135512773 hasRelatedWork W16302041 @default.
- W3135512773 hasRelatedWork W2013984052 @default.
- W3135512773 hasRelatedWork W2057287519 @default.