Matches in SemOpenAlex for { <https://semopenalex.org/work/W4387355666> ?p ?o ?g. }
Showing items 1 to 69 of
69
with 100 items per page.
- W4387355666 abstract "Recently, a myriad of conditional image generation and editing models have been developed to serve different downstream tasks, including text-to-image generation, text-guided image editing, subject-driven image generation, control-guided image generation, etc. However, we observe huge inconsistencies in experimental conditions: datasets, inference, and evaluation metrics - render fair comparisons difficult. This paper proposes ImagenHub, which is a one-stop library to standardize the inference and evaluation of all the conditional image generation models. Firstly, we define seven prominent tasks and curate high-quality evaluation datasets for them. Secondly, we built a unified inference pipeline to ensure fair comparison. Thirdly, we design two human evaluation scores, i.e. Semantic Consistency and Perceptual Quality, along with comprehensive guidelines to evaluate generated images. We train expert raters to evaluate the model outputs based on the proposed metrics. Our human evaluation achieves a high inter-worker agreement of Krippendorff's alpha on 76% models with a value higher than 0.4. We comprehensively evaluated a total of around 30 models and observed three key takeaways: (1) the existing models' performance is generally unsatisfying except for Text-guided Image Generation and Subject-driven Image Generation, with 74% models achieving an overall score lower than 0.5. (2) we examined the claims from published papers and found 83% of them hold with a few exceptions. (3) None of the existing automatic metrics has a Spearman's correlation higher than 0.2 except subject-driven image generation. Moving forward, we will continue our efforts to evaluate newly published models and update our leaderboard to keep track of the progress in conditional image generation." @default.
- W4387355666 created "2023-10-05" @default.
- W4387355666 creator A5003850660 @default.
- W4387355666 creator A5015926318 @default.
- W4387355666 creator A5023599556 @default.
- W4387355666 creator A5036525093 @default.
- W4387355666 creator A5046134983 @default.
- W4387355666 creator A5063070498 @default.
- W4387355666 creator A5085839455 @default.
- W4387355666 date "2023-10-02" @default.
- W4387355666 modified "2023-10-06" @default.
- W4387355666 title "ImagenHub: Standardizing the evaluation of conditional image generation models" @default.
- W4387355666 doi "https://doi.org/10.48550/arxiv.2310.01596" @default.
- W4387355666 hasPublicationYear "2023" @default.
- W4387355666 type Work @default.
- W4387355666 citedByCount "0" @default.
- W4387355666 crossrefType "posted-content" @default.
- W4387355666 hasAuthorship W4387355666A5003850660 @default.
- W4387355666 hasAuthorship W4387355666A5015926318 @default.
- W4387355666 hasAuthorship W4387355666A5023599556 @default.
- W4387355666 hasAuthorship W4387355666A5036525093 @default.
- W4387355666 hasAuthorship W4387355666A5046134983 @default.
- W4387355666 hasAuthorship W4387355666A5063070498 @default.
- W4387355666 hasAuthorship W4387355666A5085839455 @default.
- W4387355666 hasBestOaLocation W43873556661 @default.
- W4387355666 hasConcept C111472728 @default.
- W4387355666 hasConcept C115961682 @default.
- W4387355666 hasConcept C119857082 @default.
- W4387355666 hasConcept C124101348 @default.
- W4387355666 hasConcept C138885662 @default.
- W4387355666 hasConcept C154945302 @default.
- W4387355666 hasConcept C199360897 @default.
- W4387355666 hasConcept C23123220 @default.
- W4387355666 hasConcept C2776214188 @default.
- W4387355666 hasConcept C2776436953 @default.
- W4387355666 hasConcept C2779530757 @default.
- W4387355666 hasConcept C41008148 @default.
- W4387355666 hasConcept C43521106 @default.
- W4387355666 hasConcept C55020928 @default.
- W4387355666 hasConceptScore W4387355666C111472728 @default.
- W4387355666 hasConceptScore W4387355666C115961682 @default.
- W4387355666 hasConceptScore W4387355666C119857082 @default.
- W4387355666 hasConceptScore W4387355666C124101348 @default.
- W4387355666 hasConceptScore W4387355666C138885662 @default.
- W4387355666 hasConceptScore W4387355666C154945302 @default.
- W4387355666 hasConceptScore W4387355666C199360897 @default.
- W4387355666 hasConceptScore W4387355666C23123220 @default.
- W4387355666 hasConceptScore W4387355666C2776214188 @default.
- W4387355666 hasConceptScore W4387355666C2776436953 @default.
- W4387355666 hasConceptScore W4387355666C2779530757 @default.
- W4387355666 hasConceptScore W4387355666C41008148 @default.
- W4387355666 hasConceptScore W4387355666C43521106 @default.
- W4387355666 hasConceptScore W4387355666C55020928 @default.
- W4387355666 hasLocation W43873556661 @default.
- W4387355666 hasOpenAccess W4387355666 @default.
- W4387355666 hasPrimaryLocation W43873556661 @default.
- W4387355666 hasRelatedWork W1200423363 @default.
- W4387355666 hasRelatedWork W1603736412 @default.
- W4387355666 hasRelatedWork W2055243143 @default.
- W4387355666 hasRelatedWork W2171591485 @default.
- W4387355666 hasRelatedWork W2252100032 @default.
- W4387355666 hasRelatedWork W2380685755 @default.
- W4387355666 hasRelatedWork W2734796617 @default.
- W4387355666 hasRelatedWork W2963436428 @default.
- W4387355666 hasRelatedWork W3037187668 @default.
- W4387355666 hasRelatedWork W3171253712 @default.
- W4387355666 isParatext "false" @default.
- W4387355666 isRetracted "false" @default.
- W4387355666 workType "article" @default.