Matches in SemOpenAlex for { <https://semopenalex.org/work/W4387156741> ?p ?o ?g. }
Showing items 1 to 71 of
71
with 100 items per page.
- W4387156741 abstract "The use of machine learning (ML) models to assess and score textual data has become increasingly pervasive in an array of contexts including natural language processing, information retrieval, search and recommendation, and credibility assessment of online content. A significant disruption at the intersection of ML and text are text-generating large-language models such as generative pre-trained transformers (GPTs). We empirically assess the differences in how ML-based scoring models trained on human content assess the quality of content generated by humans versus GPTs. To do so, we propose an analysis framework that encompasses essay scoring ML-models, human and ML-generated essays, and a statistical model that parsimoniously considers the impact of type of respondent, prompt genre, and the ML model used for assessment model. A rich testbed is utilized that encompasses 18,460 human-generated and GPT-based essays. Results of our benchmark analysis reveal that transformer pretrained language models (PLMs) more accurately score human essay quality as compared to CNN/RNN and feature-based ML methods. Interestingly, we find that the transformer PLMs tend to score GPT-generated text 10-15% higher on average, relative to human-authored documents. Conversely, traditional deep learning and feature-based ML models score human text considerably higher. Further analysis reveals that although the transformer PLMs are exclusively fine-tuned on human text, they more prominently attend to certain tokens appearing only in GPT-generated text, possibly due to familiarity/overlap in pre-training. Our framework and results have implications for text classification settings where automated scoring of text is likely to be disrupted by generative AI." @default.
- W4387156741 created "2023-09-30" @default.
- W4387156741 creator A5011529786 @default.
- W4387156741 creator A5016943900 @default.
- W4387156741 creator A5020278091 @default.
- W4387156741 creator A5045501818 @default.
- W4387156741 creator A5085246509 @default.
- W4387156741 creator A5092535966 @default.
- W4387156741 creator A5092964203 @default.
- W4387156741 creator A5092964204 @default.
- W4387156741 date "2023-09-25" @default.
- W4387156741 modified "2023-09-30" @default.
- W4387156741 title "When Automated Assessment Meets Automated Content Generation: Examining Text Quality in the Era of GPTs" @default.
- W4387156741 doi "https://doi.org/10.48550/arxiv.2309.14488" @default.
- W4387156741 hasPublicationYear "2023" @default.
- W4387156741 type Work @default.
- W4387156741 citedByCount "0" @default.
- W4387156741 crossrefType "posted-content" @default.
- W4387156741 hasAuthorship W4387156741A5011529786 @default.
- W4387156741 hasAuthorship W4387156741A5016943900 @default.
- W4387156741 hasAuthorship W4387156741A5020278091 @default.
- W4387156741 hasAuthorship W4387156741A5045501818 @default.
- W4387156741 hasAuthorship W4387156741A5085246509 @default.
- W4387156741 hasAuthorship W4387156741A5092535966 @default.
- W4387156741 hasAuthorship W4387156741A5092964203 @default.
- W4387156741 hasAuthorship W4387156741A5092964204 @default.
- W4387156741 hasBestOaLocation W43871567411 @default.
- W4387156741 hasConcept C119599485 @default.
- W4387156741 hasConcept C119857082 @default.
- W4387156741 hasConcept C127413603 @default.
- W4387156741 hasConcept C137293760 @default.
- W4387156741 hasConcept C148524875 @default.
- W4387156741 hasConcept C154945302 @default.
- W4387156741 hasConcept C165801399 @default.
- W4387156741 hasConcept C17744445 @default.
- W4387156741 hasConcept C199539241 @default.
- W4387156741 hasConcept C204321447 @default.
- W4387156741 hasConcept C2780224610 @default.
- W4387156741 hasConcept C2985684807 @default.
- W4387156741 hasConcept C41008148 @default.
- W4387156741 hasConcept C66322947 @default.
- W4387156741 hasConceptScore W4387156741C119599485 @default.
- W4387156741 hasConceptScore W4387156741C119857082 @default.
- W4387156741 hasConceptScore W4387156741C127413603 @default.
- W4387156741 hasConceptScore W4387156741C137293760 @default.
- W4387156741 hasConceptScore W4387156741C148524875 @default.
- W4387156741 hasConceptScore W4387156741C154945302 @default.
- W4387156741 hasConceptScore W4387156741C165801399 @default.
- W4387156741 hasConceptScore W4387156741C17744445 @default.
- W4387156741 hasConceptScore W4387156741C199539241 @default.
- W4387156741 hasConceptScore W4387156741C204321447 @default.
- W4387156741 hasConceptScore W4387156741C2780224610 @default.
- W4387156741 hasConceptScore W4387156741C2985684807 @default.
- W4387156741 hasConceptScore W4387156741C41008148 @default.
- W4387156741 hasConceptScore W4387156741C66322947 @default.
- W4387156741 hasLocation W43871567411 @default.
- W4387156741 hasOpenAccess W4387156741 @default.
- W4387156741 hasPrimaryLocation W43871567411 @default.
- W4387156741 hasRelatedWork W2359001871 @default.
- W4387156741 hasRelatedWork W2947903144 @default.
- W4387156741 hasRelatedWork W3033862527 @default.
- W4387156741 hasRelatedWork W3033942572 @default.
- W4387156741 hasRelatedWork W3097571385 @default.
- W4387156741 hasRelatedWork W3177920269 @default.
- W4387156741 hasRelatedWork W3196747313 @default.
- W4387156741 hasRelatedWork W3214697120 @default.
- W4387156741 hasRelatedWork W4287761227 @default.
- W4387156741 hasRelatedWork W4362451017 @default.
- W4387156741 isParatext "false" @default.
- W4387156741 isRetracted "false" @default.
- W4387156741 workType "article" @default.