Matches in SemOpenAlex for { <https://semopenalex.org/work/W4378765257> ?p ?o ?g. }
Showing items 1 to 71 of
71
with 100 items per page.
- W4378765257 abstract "Stable Diffusion revolutionised image creation from descriptive text. GPT-2, GPT-3(.5) and GPT-4 demonstrated astonishing performance across a variety of language tasks. ChatGPT introduced such language models to the general public. It is now clear that large language models (LLMs) are here to stay, and will bring about drastic change in the whole ecosystem of online text and images. In this paper we consider what the future might hold. What will happen to GPT-{n} once LLMs contribute much of the language found online? We find that use of model-generated content in training causes irreversible defects in the resulting models, where tails of the original content distribution disappear. We refer to this effect as Model Collapse and show that it can occur in Variational Autoencoders, Gaussian Mixture Models and LLMs. We build theoretical intuition behind the phenomenon and portray its ubiquity amongst all learned generative models. We demonstrate that it has to be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web. Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of content generated by LLMs in data crawled from the Internet." @default.
- W4378765257 created "2023-05-31" @default.
- W4378765257 creator A5018809423 @default.
- W4378765257 creator A5029186201 @default.
- W4378765257 creator A5046983053 @default.
- W4378765257 creator A5048483915 @default.
- W4378765257 creator A5069844959 @default.
- W4378765257 creator A5087214686 @default.
- W4378765257 date "2023-05-27" @default.
- W4378765257 modified "2023-09-24" @default.
- W4378765257 title "The Curse of Recursion: Training on Generated Data Makes Models Forget" @default.
- W4378765257 doi "https://doi.org/10.48550/arxiv.2305.17493" @default.
- W4378765257 hasPublicationYear "2023" @default.
- W4378765257 type Work @default.
- W4378765257 citedByCount "0" @default.
- W4378765257 crossrefType "posted-content" @default.
- W4378765257 hasAuthorship W4378765257A5018809423 @default.
- W4378765257 hasAuthorship W4378765257A5029186201 @default.
- W4378765257 hasAuthorship W4378765257A5046983053 @default.
- W4378765257 hasAuthorship W4378765257A5048483915 @default.
- W4378765257 hasAuthorship W4378765257A5069844959 @default.
- W4378765257 hasAuthorship W4378765257A5087214686 @default.
- W4378765257 hasBestOaLocation W43787652571 @default.
- W4378765257 hasConcept C110875604 @default.
- W4378765257 hasConcept C111472728 @default.
- W4378765257 hasConcept C132010649 @default.
- W4378765257 hasConcept C136764020 @default.
- W4378765257 hasConcept C138885662 @default.
- W4378765257 hasConcept C144024400 @default.
- W4378765257 hasConcept C154945302 @default.
- W4378765257 hasConcept C15744967 @default.
- W4378765257 hasConcept C167966045 @default.
- W4378765257 hasConcept C188147891 @default.
- W4378765257 hasConcept C19165224 @default.
- W4378765257 hasConcept C2522767166 @default.
- W4378765257 hasConcept C2780273121 @default.
- W4378765257 hasConcept C39890363 @default.
- W4378765257 hasConcept C41008148 @default.
- W4378765257 hasConcept C50335755 @default.
- W4378765257 hasConceptScore W4378765257C110875604 @default.
- W4378765257 hasConceptScore W4378765257C111472728 @default.
- W4378765257 hasConceptScore W4378765257C132010649 @default.
- W4378765257 hasConceptScore W4378765257C136764020 @default.
- W4378765257 hasConceptScore W4378765257C138885662 @default.
- W4378765257 hasConceptScore W4378765257C144024400 @default.
- W4378765257 hasConceptScore W4378765257C154945302 @default.
- W4378765257 hasConceptScore W4378765257C15744967 @default.
- W4378765257 hasConceptScore W4378765257C167966045 @default.
- W4378765257 hasConceptScore W4378765257C188147891 @default.
- W4378765257 hasConceptScore W4378765257C19165224 @default.
- W4378765257 hasConceptScore W4378765257C2522767166 @default.
- W4378765257 hasConceptScore W4378765257C2780273121 @default.
- W4378765257 hasConceptScore W4378765257C39890363 @default.
- W4378765257 hasConceptScore W4378765257C41008148 @default.
- W4378765257 hasConceptScore W4378765257C50335755 @default.
- W4378765257 hasLocation W43787652571 @default.
- W4378765257 hasOpenAccess W4378765257 @default.
- W4378765257 hasPrimaryLocation W43787652571 @default.
- W4378765257 hasRelatedWork W1968421027 @default.
- W4378765257 hasRelatedWork W1999935863 @default.
- W4378765257 hasRelatedWork W2884815824 @default.
- W4378765257 hasRelatedWork W2963286442 @default.
- W4378765257 hasRelatedWork W3003214776 @default.
- W4378765257 hasRelatedWork W3014074531 @default.
- W4378765257 hasRelatedWork W3017062960 @default.
- W4378765257 hasRelatedWork W3127737296 @default.
- W4378765257 hasRelatedWork W3215252950 @default.
- W4378765257 hasRelatedWork W4286656090 @default.
- W4378765257 isParatext "false" @default.
- W4378765257 isRetracted "false" @default.
- W4378765257 workType "article" @default.