Matches in SemOpenAlex for { <https://semopenalex.org/work/W4367046962> ?p ?o ?g. }
- W4367046962 abstract "Previous researches demonstrate that users’ actions in search interaction are associated with relative gains and losses to reference points, known as the reference dependence effect. However, this widely confirmed effect is not represented in most user models underpinning existing search evaluation metrics. In this study, we propose a new evaluation metric framework, namely Reference Dependent Metric (ReDeM), for assessing query-level search by incorporating the effect of reference dependence into the modelling of user search behavior. To test the overall effectiveness of the proposed framework, (1) we evaluate the performance, in terms of correlation with user satisfaction, of ReDeMs built upon different reference points against that of the widely-used metrics on three search datasets; (2) we examine the performance of ReDeMs under different task states, like task difficulty and task urgency; and (3) we analyze the statistical reliability of ReDeMs in terms of discriminative power. Experimental results indicate that: (1) ReDeMs integrated with a proper reference point achieve better correlations with user satisfaction than most of the existing metrics, like Discounted Cumulative Gain (DCG) and Rank-Biased Precision (RBP), even though their parameters have already been well-tuned; (2) ReDeMs reach relatively better performance compared to existing metrics when the task triggers a high-level cognitive load; (3) the discriminative power of ReDeMs is far stronger than Expected Reciprocal Rank (ERR), slightly stronger than Precision and similar to DCG, RBP and INST. To our knowledge, this study is the first to explicitly incorporate the reference dependence effect into the user browsing model and offline evaluation metrics. Our work illustrates a promising approach to leveraging the insights about user biases from cognitive psychology in better evaluating user search experience and enhancing user models." @default.
- W4367046962 created "2023-04-27" @default.
- W4367046962 creator A5007524571 @default.
- W4367046962 creator A5023595778 @default.
- W4367046962 creator A5088558868 @default.
- W4367046962 date "2023-04-30" @default.
- W4367046962 modified "2023-10-17" @default.
- W4367046962 title "A Reference-Dependent Model for Web Search Evaluation" @default.
- W4367046962 cites W1968927634 @default.
- W4367046962 cites W1983164881 @default.
- W4367046962 cites W1987831316 @default.
- W4367046962 cites W1991774506 @default.
- W4367046962 cites W2000270827 @default.
- W4367046962 cites W2009485604 @default.
- W4367046962 cites W2014721055 @default.
- W4367046962 cites W2017292914 @default.
- W4367046962 cites W2031858317 @default.
- W4367046962 cites W2041946752 @default.
- W4367046962 cites W2043787485 @default.
- W4367046962 cites W2053100920 @default.
- W4367046962 cites W2058896506 @default.
- W4367046962 cites W2059120814 @default.
- W4367046962 cites W2069870183 @default.
- W4367046962 cites W2071446250 @default.
- W4367046962 cites W2090035194 @default.
- W4367046962 cites W2106404250 @default.
- W4367046962 cites W2113640060 @default.
- W4367046962 cites W2143331230 @default.
- W4367046962 cites W2165893637 @default.
- W4367046962 cites W2624553223 @default.
- W4367046962 cites W2740517242 @default.
- W4367046962 cites W2808234013 @default.
- W4367046962 cites W2810030148 @default.
- W4367046962 cites W2895782815 @default.
- W4367046962 cites W2896022497 @default.
- W4367046962 cites W2952667223 @default.
- W4367046962 cites W2998372272 @default.
- W4367046962 cites W3034645139 @default.
- W4367046962 cites W3043886961 @default.
- W4367046962 cites W3117895913 @default.
- W4367046962 cites W3125065975 @default.
- W4367046962 cites W3125913709 @default.
- W4367046962 cites W3146722042 @default.
- W4367046962 cites W3155638432 @default.
- W4367046962 cites W3183835286 @default.
- W4367046962 cites W4220728222 @default.
- W4367046962 cites W4233760227 @default.
- W4367046962 cites W4243617875 @default.
- W4367046962 cites W4281624652 @default.
- W4367046962 cites W4283762145 @default.
- W4367046962 cites W4284689681 @default.
- W4367046962 cites W4284713556 @default.
- W4367046962 doi "https://doi.org/10.1145/3543507.3583551" @default.
- W4367046962 hasPublicationYear "2023" @default.
- W4367046962 type Work @default.
- W4367046962 citedByCount "0" @default.
- W4367046962 crossrefType "proceedings-article" @default.
- W4367046962 hasAuthorship W4367046962A5007524571 @default.
- W4367046962 hasAuthorship W4367046962A5023595778 @default.
- W4367046962 hasAuthorship W4367046962A5088558868 @default.
- W4367046962 hasConcept C114614502 @default.
- W4367046962 hasConcept C119857082 @default.
- W4367046962 hasConcept C124101348 @default.
- W4367046962 hasConcept C154945302 @default.
- W4367046962 hasConcept C162324750 @default.
- W4367046962 hasConcept C164226766 @default.
- W4367046962 hasConcept C176217482 @default.
- W4367046962 hasConcept C187736073 @default.
- W4367046962 hasConcept C21547014 @default.
- W4367046962 hasConcept C23123220 @default.
- W4367046962 hasConcept C2780451532 @default.
- W4367046962 hasConcept C33923547 @default.
- W4367046962 hasConcept C41008148 @default.
- W4367046962 hasConcept C97931131 @default.
- W4367046962 hasConceptScore W4367046962C114614502 @default.
- W4367046962 hasConceptScore W4367046962C119857082 @default.
- W4367046962 hasConceptScore W4367046962C124101348 @default.
- W4367046962 hasConceptScore W4367046962C154945302 @default.
- W4367046962 hasConceptScore W4367046962C162324750 @default.
- W4367046962 hasConceptScore W4367046962C164226766 @default.
- W4367046962 hasConceptScore W4367046962C176217482 @default.
- W4367046962 hasConceptScore W4367046962C187736073 @default.
- W4367046962 hasConceptScore W4367046962C21547014 @default.
- W4367046962 hasConceptScore W4367046962C23123220 @default.
- W4367046962 hasConceptScore W4367046962C2780451532 @default.
- W4367046962 hasConceptScore W4367046962C33923547 @default.
- W4367046962 hasConceptScore W4367046962C41008148 @default.
- W4367046962 hasConceptScore W4367046962C97931131 @default.
- W4367046962 hasLocation W43670469621 @default.
- W4367046962 hasOpenAccess W4367046962 @default.
- W4367046962 hasPrimaryLocation W43670469621 @default.
- W4367046962 hasRelatedWork W1556916750 @default.
- W4367046962 hasRelatedWork W2026121273 @default.
- W4367046962 hasRelatedWork W2102106825 @default.
- W4367046962 hasRelatedWork W2752271443 @default.
- W4367046962 hasRelatedWork W2801772698 @default.
- W4367046962 hasRelatedWork W2961085424 @default.
- W4367046962 hasRelatedWork W2983744209 @default.
- W4367046962 hasRelatedWork W4306674287 @default.
- W4367046962 hasRelatedWork W66955737 @default.