Matches in SemOpenAlex for { <https://semopenalex.org/work/W3186281040> ?p ?o ?g. }
- W3186281040 abstract "Content moderation is often performed by a collaboration between humans and machine learning models. However, it is not well understood how to design the collaborative process so as to maximize the combined moderator-model system performance. This work presents a rigorous study of this problem, focusing on an approach that incorporates model uncertainty into the collaborative process. First, we introduce principled metrics to describe the performance of the collaborative system under capacity constraints on the human moderator, quantifying how efficiently the combined system utilizes human decisions. Using these metrics, we conduct a large benchmark study evaluating the performance of state-of-the-art uncertainty models under different collaborative review strategies. We find that an uncertainty-based strategy consistently outperforms the widely used strategy based on toxicity scores, and moreover that the choice of review strategy drastically changes the overall system performance. Our results demonstrate the importance of rigorous metrics for understanding and developing effective moderator-model systems for content moderation, as well as the utility of uncertainty estimation in this domain." @default.
- W3186281040 created "2021-08-02" @default.
- W3186281040 creator A5015180445 @default.
- W3186281040 creator A5030217791 @default.
- W3186281040 creator A5060252172 @default.
- W3186281040 creator A5063684082 @default.
- W3186281040 date "2021-01-01" @default.
- W3186281040 modified "2023-10-05" @default.
- W3186281040 title "Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimation" @default.
- W3186281040 cites W1976526581 @default.
- W3186281040 cites W2024425679 @default.
- W3186281040 cites W2119769989 @default.
- W3186281040 cites W2161813894 @default.
- W3186281040 cites W2209227144 @default.
- W3186281040 cites W2254249950 @default.
- W3186281040 cites W2303654018 @default.
- W3186281040 cites W2462906003 @default.
- W3186281040 cites W2522957395 @default.
- W3186281040 cites W2531327146 @default.
- W3186281040 cites W2540646130 @default.
- W3186281040 cites W2557283755 @default.
- W3186281040 cites W2588334501 @default.
- W3186281040 cites W2612077511 @default.
- W3186281040 cites W2791170418 @default.
- W3186281040 cites W2920807444 @default.
- W3186281040 cites W2920893575 @default.
- W3186281040 cites W2954992865 @default.
- W3186281040 cites W2963177662 @default.
- W3186281040 cites W2963238274 @default.
- W3186281040 cites W2963341956 @default.
- W3186281040 cites W2963351448 @default.
- W3186281040 cites W2964006987 @default.
- W3186281040 cites W2964059111 @default.
- W3186281040 cites W2964212410 @default.
- W3186281040 cites W2970281030 @default.
- W3186281040 cites W2970859221 @default.
- W3186281040 cites W3035139434 @default.
- W3186281040 cites W3098528040 @default.
- W3186281040 cites W3099111334 @default.
- W3186281040 cites W3100691980 @default.
- W3186281040 cites W3100895823 @default.
- W3186281040 cites W3102559011 @default.
- W3186281040 cites W3104169042 @default.
- W3186281040 cites W3104448368 @default.
- W3186281040 cites W3104668038 @default.
- W3186281040 cites W3105957474 @default.
- W3186281040 cites W3118929067 @default.
- W3186281040 cites W3152911627 @default.
- W3186281040 cites W3155742828 @default.
- W3186281040 cites W3170037207 @default.
- W3186281040 cites W3172908301 @default.
- W3186281040 cites W3173351576 @default.
- W3186281040 doi "https://doi.org/10.18653/v1/2021.woah-1.5" @default.
- W3186281040 hasPublicationYear "2021" @default.
- W3186281040 type Work @default.
- W3186281040 sameAs 3186281040 @default.
- W3186281040 citedByCount "0" @default.
- W3186281040 crossrefType "proceedings-article" @default.
- W3186281040 hasAuthorship W3186281040A5015180445 @default.
- W3186281040 hasAuthorship W3186281040A5030217791 @default.
- W3186281040 hasAuthorship W3186281040A5060252172 @default.
- W3186281040 hasAuthorship W3186281040A5063684082 @default.
- W3186281040 hasBestOaLocation W31862810401 @default.
- W3186281040 hasConcept C111919701 @default.
- W3186281040 hasConcept C112930515 @default.
- W3186281040 hasConcept C119857082 @default.
- W3186281040 hasConcept C124101348 @default.
- W3186281040 hasConcept C127413603 @default.
- W3186281040 hasConcept C13280743 @default.
- W3186281040 hasConcept C134306372 @default.
- W3186281040 hasConcept C13736549 @default.
- W3186281040 hasConcept C154945302 @default.
- W3186281040 hasConcept C185798385 @default.
- W3186281040 hasConcept C201995342 @default.
- W3186281040 hasConcept C205649164 @default.
- W3186281040 hasConcept C33923547 @default.
- W3186281040 hasConcept C36503486 @default.
- W3186281040 hasConcept C41008148 @default.
- W3186281040 hasConcept C71924100 @default.
- W3186281040 hasConcept C93225998 @default.
- W3186281040 hasConcept C96250715 @default.
- W3186281040 hasConcept C98045186 @default.
- W3186281040 hasConceptScore W3186281040C111919701 @default.
- W3186281040 hasConceptScore W3186281040C112930515 @default.
- W3186281040 hasConceptScore W3186281040C119857082 @default.
- W3186281040 hasConceptScore W3186281040C124101348 @default.
- W3186281040 hasConceptScore W3186281040C127413603 @default.
- W3186281040 hasConceptScore W3186281040C13280743 @default.
- W3186281040 hasConceptScore W3186281040C134306372 @default.
- W3186281040 hasConceptScore W3186281040C13736549 @default.
- W3186281040 hasConceptScore W3186281040C154945302 @default.
- W3186281040 hasConceptScore W3186281040C185798385 @default.
- W3186281040 hasConceptScore W3186281040C201995342 @default.
- W3186281040 hasConceptScore W3186281040C205649164 @default.
- W3186281040 hasConceptScore W3186281040C33923547 @default.
- W3186281040 hasConceptScore W3186281040C36503486 @default.
- W3186281040 hasConceptScore W3186281040C41008148 @default.
- W3186281040 hasConceptScore W3186281040C71924100 @default.
- W3186281040 hasConceptScore W3186281040C93225998 @default.
- W3186281040 hasConceptScore W3186281040C96250715 @default.