Matches in SemOpenAlex for { <https://semopenalex.org/work/W2946813150> ?p ?o ?g. }
Showing items 1 to 98 of
98
with 100 items per page.
- W2946813150 endingPage "1900" @default.
- W2946813150 startingPage "1892" @default.
- W2946813150 abstract "Excessive reuse of holdout data can lead to overfitting. However, there is little concrete evidence of significant overfitting due to holdout reuse in popular multiclass benchmarks today. Known results show that, in the worst-case, revealing the accuracy of $k$ adaptively chosen classifiers on a data set of size $n$ allows to create a classifier with bias of $Theta(sqrt{k/n})$ for any binary prediction problem. We show a new upper bound of $tilde O(max{sqrt{klog(n)/(mn)},k/n})$ on the worst-case bias that any attack can achieve in a prediction problem with $m$ classes. Moreover, we present an efficient attack that achieve a bias of $Omega(sqrt{k/(m^2 n)})$ and improves on previous work for the binary setting ($m=2$). We also present an inefficient attack that achieves a bias of $tildeOmega(k/n)$. Complementing our theoretical work, we give new practical attacks to stress-test multiclass benchmarks by aiming to create as large a bias as possible with a given number of queries. Our experiments show that the additional uncertainty of prediction with a large number of classes indeed mitigates the effect of our best attacks. Our work extends developments in understanding overfitting due to adaptive data analysis to multiclass prediction problems. It also bears out the surprising fact that multiclass prediction problems are significantly more robust to overfitting when reusing a test (or holdout) dataset. This offers an explanation as to why popular multiclass prediction benchmarks, such as ImageNet, may enjoy a longer lifespan than what intuition from literature on binary classification suggests." @default.
- W2946813150 created "2019-05-29" @default.
- W2946813150 creator A5039915143 @default.
- W2946813150 creator A5054675941 @default.
- W2946813150 creator A5069963995 @default.
- W2946813150 date "2019-05-24" @default.
- W2946813150 modified "2023-09-26" @default.
- W2946813150 title "The advantages of multiple classes for reducing overfitting from test set reuse" @default.
- W2946813150 hasPublicationYear "2019" @default.
- W2946813150 type Work @default.
- W2946813150 sameAs 2946813150 @default.
- W2946813150 citedByCount "10" @default.
- W2946813150 countsByYear W29468131502019 @default.
- W2946813150 countsByYear W29468131502020 @default.
- W2946813150 countsByYear W29468131502021 @default.
- W2946813150 crossrefType "proceedings-article" @default.
- W2946813150 hasAuthorship W2946813150A5039915143 @default.
- W2946813150 hasAuthorship W2946813150A5054675941 @default.
- W2946813150 hasAuthorship W2946813150A5069963995 @default.
- W2946813150 hasConcept C11413529 @default.
- W2946813150 hasConcept C119857082 @default.
- W2946813150 hasConcept C121332964 @default.
- W2946813150 hasConcept C12267149 @default.
- W2946813150 hasConcept C134306372 @default.
- W2946813150 hasConcept C154945302 @default.
- W2946813150 hasConcept C16910744 @default.
- W2946813150 hasConcept C169903167 @default.
- W2946813150 hasConcept C177264268 @default.
- W2946813150 hasConcept C18903297 @default.
- W2946813150 hasConcept C199360897 @default.
- W2946813150 hasConcept C206588197 @default.
- W2946813150 hasConcept C22019652 @default.
- W2946813150 hasConcept C2777212361 @default.
- W2946813150 hasConcept C2779557605 @default.
- W2946813150 hasConcept C33923547 @default.
- W2946813150 hasConcept C41008148 @default.
- W2946813150 hasConcept C48372109 @default.
- W2946813150 hasConcept C50644808 @default.
- W2946813150 hasConcept C62520636 @default.
- W2946813150 hasConcept C66905080 @default.
- W2946813150 hasConcept C77553402 @default.
- W2946813150 hasConcept C86803240 @default.
- W2946813150 hasConcept C94375191 @default.
- W2946813150 hasConcept C95623464 @default.
- W2946813150 hasConceptScore W2946813150C11413529 @default.
- W2946813150 hasConceptScore W2946813150C119857082 @default.
- W2946813150 hasConceptScore W2946813150C121332964 @default.
- W2946813150 hasConceptScore W2946813150C12267149 @default.
- W2946813150 hasConceptScore W2946813150C134306372 @default.
- W2946813150 hasConceptScore W2946813150C154945302 @default.
- W2946813150 hasConceptScore W2946813150C16910744 @default.
- W2946813150 hasConceptScore W2946813150C169903167 @default.
- W2946813150 hasConceptScore W2946813150C177264268 @default.
- W2946813150 hasConceptScore W2946813150C18903297 @default.
- W2946813150 hasConceptScore W2946813150C199360897 @default.
- W2946813150 hasConceptScore W2946813150C206588197 @default.
- W2946813150 hasConceptScore W2946813150C22019652 @default.
- W2946813150 hasConceptScore W2946813150C2777212361 @default.
- W2946813150 hasConceptScore W2946813150C2779557605 @default.
- W2946813150 hasConceptScore W2946813150C33923547 @default.
- W2946813150 hasConceptScore W2946813150C41008148 @default.
- W2946813150 hasConceptScore W2946813150C48372109 @default.
- W2946813150 hasConceptScore W2946813150C50644808 @default.
- W2946813150 hasConceptScore W2946813150C62520636 @default.
- W2946813150 hasConceptScore W2946813150C66905080 @default.
- W2946813150 hasConceptScore W2946813150C77553402 @default.
- W2946813150 hasConceptScore W2946813150C86803240 @default.
- W2946813150 hasConceptScore W2946813150C94375191 @default.
- W2946813150 hasConceptScore W2946813150C95623464 @default.
- W2946813150 hasLocation W29468131501 @default.
- W2946813150 hasOpenAccess W2946813150 @default.
- W2946813150 hasPrimaryLocation W29468131501 @default.
- W2946813150 hasRelatedWork W1555759181 @default.
- W2946813150 hasRelatedWork W1798749056 @default.
- W2946813150 hasRelatedWork W1982723861 @default.
- W2946813150 hasRelatedWork W2153864077 @default.
- W2946813150 hasRelatedWork W2183889313 @default.
- W2946813150 hasRelatedWork W2225981128 @default.
- W2946813150 hasRelatedWork W2273315233 @default.
- W2946813150 hasRelatedWork W2571630279 @default.
- W2946813150 hasRelatedWork W2593576259 @default.
- W2946813150 hasRelatedWork W2599133678 @default.
- W2946813150 hasRelatedWork W2800996583 @default.
- W2946813150 hasRelatedWork W2806021783 @default.
- W2946813150 hasRelatedWork W2949340671 @default.
- W2946813150 hasRelatedWork W2950644037 @default.
- W2946813150 hasRelatedWork W2951609955 @default.
- W2946813150 hasRelatedWork W2962955763 @default.
- W2946813150 hasRelatedWork W3022893496 @default.
- W2946813150 hasRelatedWork W3173020161 @default.
- W2946813150 hasRelatedWork W3184051314 @default.
- W2946813150 hasRelatedWork W3196588968 @default.
- W2946813150 isParatext "false" @default.
- W2946813150 isRetracted "false" @default.
- W2946813150 magId "2946813150" @default.
- W2946813150 workType "article" @default.