Matches in SemOpenAlex for { <https://semopenalex.org/work/W4384268538> ?p ?o ?g. }
Showing items 1 to 95 of
95
with 100 items per page.
- W4384268538 abstract "Large vision-language models have recently achieved remarkable progress, exhibiting great perception and reasoning abilities concerning visual information. However, how to effectively evaluate these large vision-language models remains a major obstacle, hindering future model development. Traditional benchmarks like VQAv2 or COCO Caption provide quantitative performance measurements but suffer from a lack of fine-grained ability assessment and non-robust evaluation metrics. Recent subjective benchmarks, such as OwlEval, offer comprehensive evaluations of a model's abilities by incorporating human labor, but they are not scalable and display significant bias. In response to these challenges, we propose MMBench, a novel multi-modality benchmark. MMBench methodically develops a comprehensive evaluation pipeline, primarily comprised of two elements. The first element is a meticulously curated dataset that surpasses existing similar benchmarks in terms of the number and variety of evaluation questions and abilities. The second element introduces a novel CircularEval strategy and incorporates the use of ChatGPT. This implementation is designed to convert free-form predictions into pre-defined choices, thereby facilitating a more robust evaluation of the model's predictions. MMBench is a systematically-designed objective benchmark for robustly evaluating the various abilities of vision-language models. We hope MMBench will assist the research community in better evaluating their models and encourage future advancements in this domain. Project page: https://opencompass.org.cn/mmbench." @default.
- W4384268538 created "2023-07-14" @default.
- W4384268538 creator A5003834229 @default.
- W4384268538 creator A5010087030 @default.
- W4384268538 creator A5028468431 @default.
- W4384268538 creator A5029762841 @default.
- W4384268538 creator A5031836077 @default.
- W4384268538 creator A5034995105 @default.
- W4384268538 creator A5044677327 @default.
- W4384268538 creator A5048500768 @default.
- W4384268538 creator A5055183049 @default.
- W4384268538 creator A5072694881 @default.
- W4384268538 creator A5075722024 @default.
- W4384268538 creator A5081255558 @default.
- W4384268538 date "2023-07-12" @default.
- W4384268538 modified "2023-10-14" @default.
- W4384268538 title "MMBench: Is Your Multi-modal Model an All-around Player?" @default.
- W4384268538 doi "https://doi.org/10.48550/arxiv.2307.06281" @default.
- W4384268538 hasPublicationYear "2023" @default.
- W4384268538 type Work @default.
- W4384268538 citedByCount "0" @default.
- W4384268538 crossrefType "posted-content" @default.
- W4384268538 hasAuthorship W4384268538A5003834229 @default.
- W4384268538 hasAuthorship W4384268538A5010087030 @default.
- W4384268538 hasAuthorship W4384268538A5028468431 @default.
- W4384268538 hasAuthorship W4384268538A5029762841 @default.
- W4384268538 hasAuthorship W4384268538A5031836077 @default.
- W4384268538 hasAuthorship W4384268538A5034995105 @default.
- W4384268538 hasAuthorship W4384268538A5044677327 @default.
- W4384268538 hasAuthorship W4384268538A5048500768 @default.
- W4384268538 hasAuthorship W4384268538A5055183049 @default.
- W4384268538 hasAuthorship W4384268538A5072694881 @default.
- W4384268538 hasAuthorship W4384268538A5075722024 @default.
- W4384268538 hasAuthorship W4384268538A5081255558 @default.
- W4384268538 hasBestOaLocation W43842685381 @default.
- W4384268538 hasConcept C107457646 @default.
- W4384268538 hasConcept C119857082 @default.
- W4384268538 hasConcept C13280743 @default.
- W4384268538 hasConcept C134306372 @default.
- W4384268538 hasConcept C136197465 @default.
- W4384268538 hasConcept C154945302 @default.
- W4384268538 hasConcept C169760540 @default.
- W4384268538 hasConcept C17744445 @default.
- W4384268538 hasConcept C185798385 @default.
- W4384268538 hasConcept C199360897 @default.
- W4384268538 hasConcept C199539241 @default.
- W4384268538 hasConcept C205649164 @default.
- W4384268538 hasConcept C2522767166 @default.
- W4384268538 hasConcept C26760741 @default.
- W4384268538 hasConcept C2776650193 @default.
- W4384268538 hasConcept C33923547 @default.
- W4384268538 hasConcept C36503486 @default.
- W4384268538 hasConcept C41008148 @default.
- W4384268538 hasConcept C43521106 @default.
- W4384268538 hasConcept C48044578 @default.
- W4384268538 hasConcept C77088390 @default.
- W4384268538 hasConcept C86803240 @default.
- W4384268538 hasConceptScore W4384268538C107457646 @default.
- W4384268538 hasConceptScore W4384268538C119857082 @default.
- W4384268538 hasConceptScore W4384268538C13280743 @default.
- W4384268538 hasConceptScore W4384268538C134306372 @default.
- W4384268538 hasConceptScore W4384268538C136197465 @default.
- W4384268538 hasConceptScore W4384268538C154945302 @default.
- W4384268538 hasConceptScore W4384268538C169760540 @default.
- W4384268538 hasConceptScore W4384268538C17744445 @default.
- W4384268538 hasConceptScore W4384268538C185798385 @default.
- W4384268538 hasConceptScore W4384268538C199360897 @default.
- W4384268538 hasConceptScore W4384268538C199539241 @default.
- W4384268538 hasConceptScore W4384268538C205649164 @default.
- W4384268538 hasConceptScore W4384268538C2522767166 @default.
- W4384268538 hasConceptScore W4384268538C26760741 @default.
- W4384268538 hasConceptScore W4384268538C2776650193 @default.
- W4384268538 hasConceptScore W4384268538C33923547 @default.
- W4384268538 hasConceptScore W4384268538C36503486 @default.
- W4384268538 hasConceptScore W4384268538C41008148 @default.
- W4384268538 hasConceptScore W4384268538C43521106 @default.
- W4384268538 hasConceptScore W4384268538C48044578 @default.
- W4384268538 hasConceptScore W4384268538C77088390 @default.
- W4384268538 hasConceptScore W4384268538C86803240 @default.
- W4384268538 hasLocation W43842685381 @default.
- W4384268538 hasOpenAccess W4384268538 @default.
- W4384268538 hasPrimaryLocation W43842685381 @default.
- W4384268538 hasRelatedWork W112744582 @default.
- W4384268538 hasRelatedWork W1897333786 @default.
- W4384268538 hasRelatedWork W1992807924 @default.
- W4384268538 hasRelatedWork W2051963567 @default.
- W4384268538 hasRelatedWork W2789601449 @default.
- W4384268538 hasRelatedWork W2792109275 @default.
- W4384268538 hasRelatedWork W2999356315 @default.
- W4384268538 hasRelatedWork W3123920941 @default.
- W4384268538 hasRelatedWork W4288364295 @default.
- W4384268538 hasRelatedWork W4304194246 @default.
- W4384268538 isParatext "false" @default.
- W4384268538 isRetracted "false" @default.
- W4384268538 workType "article" @default.