Matches in SemOpenAlex for { <https://semopenalex.org/work/W4286978605> ?p ?o ?g. }
Showing items 1 to 89 of
89
with 100 items per page.
- W4286978605 abstract "For machine learning models trained with limited labeled training data, validation stands to become the main bottleneck to reducing overall annotation costs. We propose a statistical validation algorithm that accurately estimates the F-score of binary classifiers for rare categories, where finding relevant examples to evaluate on is particularly challenging. Our key insight is that simultaneous calibration and importance sampling enables accurate estimates even in the low-sample regime (< 300 samples). Critically, we also derive an accurate single-trial estimator of the variance of our method and demonstrate that this estimator is empirically accurate at low sample counts, enabling a practitioner to know how well they can trust a given low-sample estimate. When validating state-of-the-art semi-supervised models on ImageNet and iNaturalist2017, our method achieves the same estimates of model performance with up to 10x fewer labels than competing approaches. In particular, we can estimate model F1 scores with a variance of 0.005 using as few as 100 labels." @default.
- W4286978605 created "2022-07-25" @default.
- W4286978605 creator A5004353237 @default.
- W4286978605 creator A5006412542 @default.
- W4286978605 creator A5027115836 @default.
- W4286978605 creator A5037444018 @default.
- W4286978605 creator A5069897950 @default.
- W4286978605 creator A5070318397 @default.
- W4286978605 creator A5088661550 @default.
- W4286978605 date "2021-09-13" @default.
- W4286978605 modified "2023-09-27" @default.
- W4286978605 title "Low-Shot Validation: Active Importance Sampling for Estimating Classifier Performance on Rare Categories" @default.
- W4286978605 doi "https://doi.org/10.48550/arxiv.2109.05720" @default.
- W4286978605 hasPublicationYear "2021" @default.
- W4286978605 type Work @default.
- W4286978605 citedByCount "0" @default.
- W4286978605 crossrefType "posted-content" @default.
- W4286978605 hasAuthorship W4286978605A5004353237 @default.
- W4286978605 hasAuthorship W4286978605A5006412542 @default.
- W4286978605 hasAuthorship W4286978605A5027115836 @default.
- W4286978605 hasAuthorship W4286978605A5037444018 @default.
- W4286978605 hasAuthorship W4286978605A5069897950 @default.
- W4286978605 hasAuthorship W4286978605A5070318397 @default.
- W4286978605 hasAuthorship W4286978605A5088661550 @default.
- W4286978605 hasBestOaLocation W42869786051 @default.
- W4286978605 hasConcept C105795698 @default.
- W4286978605 hasConcept C119857082 @default.
- W4286978605 hasConcept C121955636 @default.
- W4286978605 hasConcept C12267149 @default.
- W4286978605 hasConcept C124101348 @default.
- W4286978605 hasConcept C140779682 @default.
- W4286978605 hasConcept C144133560 @default.
- W4286978605 hasConcept C149635348 @default.
- W4286978605 hasConcept C154945302 @default.
- W4286978605 hasConcept C165838908 @default.
- W4286978605 hasConcept C185429906 @default.
- W4286978605 hasConcept C185592680 @default.
- W4286978605 hasConcept C196083921 @default.
- W4286978605 hasConcept C198531522 @default.
- W4286978605 hasConcept C2780513914 @default.
- W4286978605 hasConcept C33923547 @default.
- W4286978605 hasConcept C41008148 @default.
- W4286978605 hasConcept C43617362 @default.
- W4286978605 hasConcept C48372109 @default.
- W4286978605 hasConcept C66905080 @default.
- W4286978605 hasConcept C76155785 @default.
- W4286978605 hasConcept C94375191 @default.
- W4286978605 hasConcept C94915269 @default.
- W4286978605 hasConcept C95623464 @default.
- W4286978605 hasConceptScore W4286978605C105795698 @default.
- W4286978605 hasConceptScore W4286978605C119857082 @default.
- W4286978605 hasConceptScore W4286978605C121955636 @default.
- W4286978605 hasConceptScore W4286978605C12267149 @default.
- W4286978605 hasConceptScore W4286978605C124101348 @default.
- W4286978605 hasConceptScore W4286978605C140779682 @default.
- W4286978605 hasConceptScore W4286978605C144133560 @default.
- W4286978605 hasConceptScore W4286978605C149635348 @default.
- W4286978605 hasConceptScore W4286978605C154945302 @default.
- W4286978605 hasConceptScore W4286978605C165838908 @default.
- W4286978605 hasConceptScore W4286978605C185429906 @default.
- W4286978605 hasConceptScore W4286978605C185592680 @default.
- W4286978605 hasConceptScore W4286978605C196083921 @default.
- W4286978605 hasConceptScore W4286978605C198531522 @default.
- W4286978605 hasConceptScore W4286978605C2780513914 @default.
- W4286978605 hasConceptScore W4286978605C33923547 @default.
- W4286978605 hasConceptScore W4286978605C41008148 @default.
- W4286978605 hasConceptScore W4286978605C43617362 @default.
- W4286978605 hasConceptScore W4286978605C48372109 @default.
- W4286978605 hasConceptScore W4286978605C66905080 @default.
- W4286978605 hasConceptScore W4286978605C76155785 @default.
- W4286978605 hasConceptScore W4286978605C94375191 @default.
- W4286978605 hasConceptScore W4286978605C94915269 @default.
- W4286978605 hasConceptScore W4286978605C95623464 @default.
- W4286978605 hasLocation W42869786051 @default.
- W4286978605 hasOpenAccess W4286978605 @default.
- W4286978605 hasPrimaryLocation W42869786051 @default.
- W4286978605 hasRelatedWork W1533581613 @default.
- W4286978605 hasRelatedWork W1966331501 @default.
- W4286978605 hasRelatedWork W2017052259 @default.
- W4286978605 hasRelatedWork W2061707367 @default.
- W4286978605 hasRelatedWork W2109135198 @default.
- W4286978605 hasRelatedWork W2313211764 @default.
- W4286978605 hasRelatedWork W2323231699 @default.
- W4286978605 hasRelatedWork W2961085424 @default.
- W4286978605 hasRelatedWork W567739382 @default.
- W4286978605 hasRelatedWork W2173022174 @default.
- W4286978605 isParatext "false" @default.
- W4286978605 isRetracted "false" @default.
- W4286978605 workType "article" @default.