Matches in SemOpenAlex for { <https://semopenalex.org/work/W3153482424> ?p ?o ?g. }
Showing items 1 to 83 of
83
with 100 items per page.
- W3153482424 abstract "BERT-based Neural Ranking Models (NRMs) can be classified according to how the query and document are encoded through BERT's self-attention layers - bi-encoder versus cross-encoder. Bi-encoder models are highly efficient because all the documents can be pre-processed before the query time, but their performance is inferior compared to cross-encoder models. Both models utilize a ranker that receives BERT representations as the input and generates a relevance score as the output. In this work, we propose a method where multi-teacher distillation is applied to a cross-encoder NRM and a bi-encoder NRM to produce a bi-encoder NRM with two rankers. The resulting student bi-encoder achieves an improved performance by simultaneously learning from a cross-encoder teacher and a bi-encoder teacher and also by combining relevance scores from the two rankers. We call this method TRMD (Two Rankers and Multi-teacher Distillation). In the experiments, TwinBERT and ColBERT are considered as baseline bi-encoders. When monoBERT is used as the cross-encoder teacher, together with either TwinBERT or ColBERT as the bi-encoder teacher, TRMD produces a student bi-encoder that performs better than the corresponding baseline bi-encoder. For P@20, the maximum improvement was 11.4%, and the average improvement was 6.8%. As an additional experiment, we considered producing cross-encoder students with TRMD, and found that it could also improve the cross-encoders." @default.
- W3153482424 created "2021-04-26" @default.
- W3153482424 creator A5007022993 @default.
- W3153482424 creator A5052541016 @default.
- W3153482424 creator A5056032525 @default.
- W3153482424 creator A5088150787 @default.
- W3153482424 date "2021-07-11" @default.
- W3153482424 modified "2023-09-27" @default.
- W3153482424 title "Improving Bi-encoder Document Ranking Models with Two Rankers and Multi-teacher Distillation" @default.
- W3153482424 cites W2194775991 @default.
- W3153482424 cites W2610935556 @default.
- W3153482424 cites W2743289088 @default.
- W3153482424 cites W2747909401 @default.
- W3153482424 cites W2782157559 @default.
- W3153482424 cites W3021397474 @default.
- W3153482424 cites W3094444847 @default.
- W3153482424 cites W3102286003 @default.
- W3153482424 cites W4244536387 @default.
- W3153482424 doi "https://doi.org/10.1145/3404835.3463076" @default.
- W3153482424 hasPublicationYear "2021" @default.
- W3153482424 type Work @default.
- W3153482424 sameAs 3153482424 @default.
- W3153482424 citedByCount "6" @default.
- W3153482424 countsByYear W31534824242022 @default.
- W3153482424 countsByYear W31534824242023 @default.
- W3153482424 crossrefType "proceedings-article" @default.
- W3153482424 hasAuthorship W3153482424A5007022993 @default.
- W3153482424 hasAuthorship W3153482424A5052541016 @default.
- W3153482424 hasAuthorship W3153482424A5056032525 @default.
- W3153482424 hasAuthorship W3153482424A5088150787 @default.
- W3153482424 hasBestOaLocation W31534824242 @default.
- W3153482424 hasConcept C101738243 @default.
- W3153482424 hasConcept C111368507 @default.
- W3153482424 hasConcept C111919701 @default.
- W3153482424 hasConcept C118505674 @default.
- W3153482424 hasConcept C12725497 @default.
- W3153482424 hasConcept C127313418 @default.
- W3153482424 hasConcept C154945302 @default.
- W3153482424 hasConcept C158154518 @default.
- W3153482424 hasConcept C17744445 @default.
- W3153482424 hasConcept C178790620 @default.
- W3153482424 hasConcept C185592680 @default.
- W3153482424 hasConcept C189430467 @default.
- W3153482424 hasConcept C199539241 @default.
- W3153482424 hasConcept C204030448 @default.
- W3153482424 hasConcept C37488316 @default.
- W3153482424 hasConcept C41008148 @default.
- W3153482424 hasConcept C50644808 @default.
- W3153482424 hasConceptScore W3153482424C101738243 @default.
- W3153482424 hasConceptScore W3153482424C111368507 @default.
- W3153482424 hasConceptScore W3153482424C111919701 @default.
- W3153482424 hasConceptScore W3153482424C118505674 @default.
- W3153482424 hasConceptScore W3153482424C12725497 @default.
- W3153482424 hasConceptScore W3153482424C127313418 @default.
- W3153482424 hasConceptScore W3153482424C154945302 @default.
- W3153482424 hasConceptScore W3153482424C158154518 @default.
- W3153482424 hasConceptScore W3153482424C17744445 @default.
- W3153482424 hasConceptScore W3153482424C178790620 @default.
- W3153482424 hasConceptScore W3153482424C185592680 @default.
- W3153482424 hasConceptScore W3153482424C189430467 @default.
- W3153482424 hasConceptScore W3153482424C199539241 @default.
- W3153482424 hasConceptScore W3153482424C204030448 @default.
- W3153482424 hasConceptScore W3153482424C37488316 @default.
- W3153482424 hasConceptScore W3153482424C41008148 @default.
- W3153482424 hasConceptScore W3153482424C50644808 @default.
- W3153482424 hasLocation W31534824241 @default.
- W3153482424 hasLocation W31534824242 @default.
- W3153482424 hasOpenAccess W3153482424 @default.
- W3153482424 hasPrimaryLocation W31534824241 @default.
- W3153482424 hasRelatedWork W195777573 @default.
- W3153482424 hasRelatedWork W2530090885 @default.
- W3153482424 hasRelatedWork W2589098947 @default.
- W3153482424 hasRelatedWork W2848186146 @default.
- W3153482424 hasRelatedWork W3085258535 @default.
- W3153482424 hasRelatedWork W3153482424 @default.
- W3153482424 hasRelatedWork W3170968215 @default.
- W3153482424 hasRelatedWork W3198715852 @default.
- W3153482424 hasRelatedWork W4287119736 @default.
- W3153482424 hasRelatedWork W4307933309 @default.
- W3153482424 isParatext "false" @default.
- W3153482424 isRetracted "false" @default.
- W3153482424 magId "3153482424" @default.
- W3153482424 workType "article" @default.