Matches in SemOpenAlex for { <https://semopenalex.org/work/W3172887788> ?p ?o ?g. }
- W3172887788 abstract "A few years ago, the first CNN surpassed human performance on ImageNet. However, it soon became clear that machines lack robustness on more challenging test cases, a major obstacle towards deploying machines the wild and towards obtaining better computational models of human visual perception. Here we ask: Are we making progress in closing the gap between human and machine vision? To answer this question, we tested human observers on a broad range of out-of-distribution (OOD) datasets, adding the missing human baseline by recording 85,120 psychophysical trials across 90 participants. We then investigated a range of promising machine learning developments that crucially deviate from standard supervised CNNs along three axes: objective function (self-supervised, adversarially trained, CLIP language-image training), architecture (e.g. vision transformers), and dataset size (ranging from 1M to 1B). Our findings are threefold. (1.) The longstanding robustness gap between humans and CNNs is closing, with the best models now matching or exceeding human performance on most OOD datasets. (2.) There is still a substantial image-level consistency gap, meaning that humans make different errors than models. In contrast, most models systematically agree in their categorisation errors, even substantially different ones like contrastive self-supervised vs. standard supervised models. (3.) In many cases, human-to-model consistency improves when training dataset size is increased by one to three orders of magnitude. Our results give reason for cautious optimism: While there is still much room for improvement, the behavioural difference between human and machine vision is narrowing. In order to measure future progress, 17 OOD datasets with image-level human behavioural data are provided as a benchmark here: this https URL" @default.
- W3172887788 created "2021-06-22" @default.
- W3172887788 creator A5006111171 @default.
- W3172887788 creator A5043113119 @default.
- W3172887788 creator A5043258699 @default.
- W3172887788 creator A5058294765 @default.
- W3172887788 creator A5061457780 @default.
- W3172887788 creator A5077628073 @default.
- W3172887788 creator A5086801305 @default.
- W3172887788 date "2021-06-14" @default.
- W3172887788 modified "2023-10-16" @default.
- W3172887788 title "Partial success in closing the gap between human and machine vision" @default.
- W3172887788 cites W1673923490 @default.
- W3172887788 cites W1945616565 @default.
- W3172887788 cites W1985394992 @default.
- W3172887788 cites W2053154970 @default.
- W3172887788 cites W2058616551 @default.
- W3172887788 cites W2081580037 @default.
- W3172887788 cites W2099001231 @default.
- W3172887788 cites W2108598243 @default.
- W3172887788 cites W2117539524 @default.
- W3172887788 cites W2162950292 @default.
- W3172887788 cites W2163605009 @default.
- W3172887788 cites W2176287621 @default.
- W3172887788 cites W2194321275 @default.
- W3172887788 cites W2230740169 @default.
- W3172887788 cites W2502949459 @default.
- W3172887788 cites W2582187633 @default.
- W3172887788 cites W2612573399 @default.
- W3172887788 cites W2612637113 @default.
- W3172887788 cites W2618235498 @default.
- W3172887788 cites W2731168235 @default.
- W3172887788 cites W2774616426 @default.
- W3172887788 cites W2782812883 @default.
- W3172887788 cites W2790548233 @default.
- W3172887788 cites W2798878556 @default.
- W3172887788 cites W2842511635 @default.
- W3172887788 cites W2883386984 @default.
- W3172887788 cites W2888339491 @default.
- W3172887788 cites W2902617128 @default.
- W3172887788 cites W2910992787 @default.
- W3172887788 cites W2943152387 @default.
- W3172887788 cites W2945359720 @default.
- W3172887788 cites W2946619452 @default.
- W3172887788 cites W2947707615 @default.
- W3172887788 cites W2951065015 @default.
- W3172887788 cites W2951506741 @default.
- W3172887788 cites W2951934944 @default.
- W3172887788 cites W2952984539 @default.
- W3172887788 cites W2961540362 @default.
- W3172887788 cites W2963060032 @default.
- W3172887788 cites W2963305465 @default.
- W3172887788 cites W2964096266 @default.
- W3172887788 cites W2965470910 @default.
- W3172887788 cites W2966900272 @default.
- W3172887788 cites W2968993450 @default.
- W3172887788 cites W2970001856 @default.
- W3172887788 cites W2970104209 @default.
- W3172887788 cites W2970131854 @default.
- W3172887788 cites W2970692043 @default.
- W3172887788 cites W2970971581 @default.
- W3172887788 cites W2972831213 @default.
- W3172887788 cites W2972970334 @default.
- W3172887788 cites W2976501124 @default.
- W3172887788 cites W2980620660 @default.
- W3172887788 cites W2989576588 @default.
- W3172887788 cites W2999044305 @default.
- W3172887788 cites W3005680577 @default.
- W3172887788 cites W3009561768 @default.
- W3172887788 cites W3015146382 @default.
- W3172887788 cites W3016954138 @default.
- W3172887788 cites W3026092005 @default.
- W3172887788 cites W3034672496 @default.
- W3172887788 cites W3034781633 @default.
- W3172887788 cites W3035160371 @default.
- W3172887788 cites W3035258717 @default.
- W3172887788 cites W3035475567 @default.
- W3172887788 cites W3035494189 @default.
- W3172887788 cites W3035524453 @default.
- W3172887788 cites W3036076479 @default.
- W3172887788 cites W3046535886 @default.
- W3172887788 cites W3092138487 @default.
- W3172887788 cites W3092472394 @default.
- W3172887788 cites W3093070351 @default.
- W3172887788 cites W3094502228 @default.
- W3172887788 cites W3098018961 @default.
- W3172887788 cites W3104911444 @default.
- W3172887788 cites W3104962541 @default.
- W3172887788 cites W3105558400 @default.
- W3172887788 cites W3131930251 @default.
- W3172887788 cites W3135367836 @default.
- W3172887788 cites W3146670357 @default.
- W3172887788 cites W3153606705 @default.
- W3172887788 cites W3157521173 @default.
- W3172887788 cites W3157790104 @default.
- W3172887788 cites W3162316477 @default.
- W3172887788 cites W3164024107 @default.
- W3172887788 cites W3164688203 @default.
- W3172887788 cites W3174962775 @default.
- W3172887788 cites W3176732771 @default.