Matches in SemOpenAlex for { <https://semopenalex.org/work/W4285075985> ?p ?o ?g. }
Showing items 1 to 65 of
65
with 100 items per page.
- W4285075985 abstract "Automatically evaluating the quality of dialogue responses for unstructured domains is a challenging problem. ADEM(Lowe et al. 2017) formulated the automatic evaluation of dialogue systems as a learning problem and showed that such a model was able to predict responses which correlate significantly with human judgements, both at utterance and system level. Their system was shown to have beaten word-overlap metrics such as BLEU with large margins. We start with the question of whether an adversary can game the ADEM model. We design a battery of targeted attacks at the neural network based ADEM evaluation system and show that automatic evaluation of dialogue systems still has a long way to go. ADEM can get confused with a variation as simple as reversing the word order in the text! We report experiments on several such adversarial scenarios that draw out counterintuitive scores on the dialogue responses. We take a systematic look at the scoring function proposed by ADEM and connect it to linear system theory to predict the shortcomings evident in the system. We also devise an attack that can fool such a system to rate a response generation system as favorable. Finally, we allude to future research directions of using the adversarial attacks to design a truly automated dialogue evaluation system." @default.
- W4285075985 created "2022-07-13" @default.
- W4285075985 creator A5050036814 @default.
- W4285075985 creator A5065555497 @default.
- W4285075985 creator A5073570662 @default.
- W4285075985 creator A5087678154 @default.
- W4285075985 date "2019-02-23" @default.
- W4285075985 modified "2023-09-27" @default.
- W4285075985 title "Re-evaluating ADEM: A Deeper Look at Scoring Dialogue Responses" @default.
- W4285075985 doi "https://doi.org/10.48550/arxiv.1902.08832" @default.
- W4285075985 hasPublicationYear "2019" @default.
- W4285075985 type Work @default.
- W4285075985 citedByCount "0" @default.
- W4285075985 crossrefType "posted-content" @default.
- W4285075985 hasAuthorship W4285075985A5050036814 @default.
- W4285075985 hasAuthorship W4285075985A5065555497 @default.
- W4285075985 hasAuthorship W4285075985A5073570662 @default.
- W4285075985 hasAuthorship W4285075985A5087678154 @default.
- W4285075985 hasBestOaLocation W42850759851 @default.
- W4285075985 hasConcept C101097943 @default.
- W4285075985 hasConcept C111472728 @default.
- W4285075985 hasConcept C119857082 @default.
- W4285075985 hasConcept C138885662 @default.
- W4285075985 hasConcept C154945302 @default.
- W4285075985 hasConcept C204321447 @default.
- W4285075985 hasConcept C2775852435 @default.
- W4285075985 hasConcept C2779530757 @default.
- W4285075985 hasConcept C2780586882 @default.
- W4285075985 hasConcept C37736160 @default.
- W4285075985 hasConcept C38652104 @default.
- W4285075985 hasConcept C41008148 @default.
- W4285075985 hasConcept C41065033 @default.
- W4285075985 hasConcept C41895202 @default.
- W4285075985 hasConcept C90805587 @default.
- W4285075985 hasConceptScore W4285075985C101097943 @default.
- W4285075985 hasConceptScore W4285075985C111472728 @default.
- W4285075985 hasConceptScore W4285075985C119857082 @default.
- W4285075985 hasConceptScore W4285075985C138885662 @default.
- W4285075985 hasConceptScore W4285075985C154945302 @default.
- W4285075985 hasConceptScore W4285075985C204321447 @default.
- W4285075985 hasConceptScore W4285075985C2775852435 @default.
- W4285075985 hasConceptScore W4285075985C2779530757 @default.
- W4285075985 hasConceptScore W4285075985C2780586882 @default.
- W4285075985 hasConceptScore W4285075985C37736160 @default.
- W4285075985 hasConceptScore W4285075985C38652104 @default.
- W4285075985 hasConceptScore W4285075985C41008148 @default.
- W4285075985 hasConceptScore W4285075985C41065033 @default.
- W4285075985 hasConceptScore W4285075985C41895202 @default.
- W4285075985 hasConceptScore W4285075985C90805587 @default.
- W4285075985 hasLocation W42850759851 @default.
- W4285075985 hasOpenAccess W4285075985 @default.
- W4285075985 hasPrimaryLocation W42850759851 @default.
- W4285075985 hasRelatedWork W1508636238 @default.
- W4285075985 hasRelatedWork W2095577883 @default.
- W4285075985 hasRelatedWork W2120486996 @default.
- W4285075985 hasRelatedWork W2585881251 @default.
- W4285075985 hasRelatedWork W2610321374 @default.
- W4285075985 hasRelatedWork W2952919291 @default.
- W4285075985 hasRelatedWork W2953920146 @default.
- W4285075985 hasRelatedWork W2994725226 @default.
- W4285075985 hasRelatedWork W3124408655 @default.
- W4285075985 hasRelatedWork W4297785512 @default.
- W4285075985 isParatext "false" @default.
- W4285075985 isRetracted "false" @default.
- W4285075985 workType "article" @default.