Matches in SemOpenAlex for { <https://semopenalex.org/work/W3097055006> ?p ?o ?g. }
- W3097055006 abstract "In this paper, we propose EqCo (Equivalent Rules for Contrastive Learning) to make self-supervised learning irrelevant to the number of negative samples in the contrastive learning framework. Inspired by the InfoMax principle, we point that the margin term in contrastive loss needs to be adaptively scaled according to the number of negative pairs in order to keep steady mutual information bound and gradient magnitude. EqCo bridges the performance gap among a wide range of negative sample sizes, so that for the first time, we can use only a few negative pairs (e.g., 16 per query) to perform self-supervised contrastive training on large-scale vision datasets like ImageNet, while with almost no accuracy drop. This is quite a contrast to the widely used large batch training or memory bank mechanism in current practices. Equipped with EqCo, our simplified MoCo (SiMo) achieves comparable accuracy with MoCov2 on ImageNet (linear evaluation protocol) while only involves 16 negative pairs per query instead of 65536, suggesting that large quantities of negative samples is not a critical factor in contrastive learning frameworks." @default.
- W3097055006 created "2020-11-09" @default.
- W3097055006 creator A5027961969 @default.
- W3097055006 creator A5041750427 @default.
- W3097055006 creator A5064029351 @default.
- W3097055006 creator A5088702914 @default.
- W3097055006 creator A5091093335 @default.
- W3097055006 date "2020-10-05" @default.
- W3097055006 modified "2023-09-23" @default.
- W3097055006 title "EqCo: Equivalent Rules for Self-supervised Contrastive Learning" @default.
- W3097055006 cites W1598866093 @default.
- W3097055006 cites W2108598243 @default.
- W3097055006 cites W2122925692 @default.
- W3097055006 cites W2138621090 @default.
- W3097055006 cites W2148349024 @default.
- W3097055006 cites W2170413022 @default.
- W3097055006 cites W2194775991 @default.
- W3097055006 cites W2565639579 @default.
- W3097055006 cites W2622263826 @default.
- W3097055006 cites W2757910899 @default.
- W3097055006 cites W2798991696 @default.
- W3097055006 cites W2842511635 @default.
- W3097055006 cites W2935908327 @default.
- W3097055006 cites W2944828972 @default.
- W3097055006 cites W2949117887 @default.
- W3097055006 cites W2949517790 @default.
- W3097055006 cites W2949759968 @default.
- W3097055006 cites W2951585248 @default.
- W3097055006 cites W2953070460 @default.
- W3097055006 cites W2962898354 @default.
- W3097055006 cites W2963016543 @default.
- W3097055006 cites W2964420626 @default.
- W3097055006 cites W2969985801 @default.
- W3097055006 cites W2971155163 @default.
- W3097055006 cites W3005680577 @default.
- W3097055006 cites W3009561768 @default.
- W3097055006 cites W3026092005 @default.
- W3097055006 cites W3027758526 @default.
- W3097055006 cites W3034781633 @default.
- W3097055006 cites W3035014997 @default.
- W3097055006 cites W3035060554 @default.
- W3097055006 cites W3035524453 @default.
- W3097055006 cites W3036982689 @default.
- W3097055006 cites W3039465721 @default.
- W3097055006 cites W3041919418 @default.
- W3097055006 cites W3080420240 @default.
- W3097055006 cites W3093929102 @default.
- W3097055006 cites W3095121901 @default.
- W3097055006 cites W3106005682 @default.
- W3097055006 cites W3107668149 @default.
- W3097055006 doi "https://doi.org/10.48550/arxiv.2010.01929" @default.
- W3097055006 hasPublicationYear "2020" @default.
- W3097055006 type Work @default.
- W3097055006 sameAs 3097055006 @default.
- W3097055006 citedByCount "1" @default.
- W3097055006 countsByYear W30970550062021 @default.
- W3097055006 crossrefType "posted-content" @default.
- W3097055006 hasAuthorship W3097055006A5027961969 @default.
- W3097055006 hasAuthorship W3097055006A5041750427 @default.
- W3097055006 hasAuthorship W3097055006A5064029351 @default.
- W3097055006 hasAuthorship W3097055006A5088702914 @default.
- W3097055006 hasAuthorship W3097055006A5091093335 @default.
- W3097055006 hasBestOaLocation W30970550061 @default.
- W3097055006 hasConcept C112972136 @default.
- W3097055006 hasConcept C119857082 @default.
- W3097055006 hasConcept C120317606 @default.
- W3097055006 hasConcept C127162648 @default.
- W3097055006 hasConcept C153180895 @default.
- W3097055006 hasConcept C153402090 @default.
- W3097055006 hasConcept C154945302 @default.
- W3097055006 hasConcept C185592680 @default.
- W3097055006 hasConcept C198531522 @default.
- W3097055006 hasConcept C2776502983 @default.
- W3097055006 hasConcept C31258907 @default.
- W3097055006 hasConcept C41008148 @default.
- W3097055006 hasConcept C43617362 @default.
- W3097055006 hasConcept C774472 @default.
- W3097055006 hasConceptScore W3097055006C112972136 @default.
- W3097055006 hasConceptScore W3097055006C119857082 @default.
- W3097055006 hasConceptScore W3097055006C120317606 @default.
- W3097055006 hasConceptScore W3097055006C127162648 @default.
- W3097055006 hasConceptScore W3097055006C153180895 @default.
- W3097055006 hasConceptScore W3097055006C153402090 @default.
- W3097055006 hasConceptScore W3097055006C154945302 @default.
- W3097055006 hasConceptScore W3097055006C185592680 @default.
- W3097055006 hasConceptScore W3097055006C198531522 @default.
- W3097055006 hasConceptScore W3097055006C2776502983 @default.
- W3097055006 hasConceptScore W3097055006C31258907 @default.
- W3097055006 hasConceptScore W3097055006C41008148 @default.
- W3097055006 hasConceptScore W3097055006C43617362 @default.
- W3097055006 hasConceptScore W3097055006C774472 @default.
- W3097055006 hasLocation W30970550061 @default.
- W3097055006 hasOpenAccess W3097055006 @default.
- W3097055006 hasPrimaryLocation W30970550061 @default.
- W3097055006 hasRelatedWork W2948853804 @default.
- W3097055006 hasRelatedWork W2961085424 @default.
- W3097055006 hasRelatedWork W3046775127 @default.
- W3097055006 hasRelatedWork W4205958290 @default.
- W3097055006 hasRelatedWork W4286629047 @default.
- W3097055006 hasRelatedWork W4288333949 @default.