Matches in SemOpenAlex for { <https://semopenalex.org/work/W4378498706> ?p ?o ?g. }
Showing items 1 to 51 of
51
with 100 items per page.
- W4378498706 abstract "Large Language Models (LLMs) have successfully been applied to code generation tasks, raising the question of how well these models understand programming. Typical programming languages have invariances and equivariances in their semantics that human programmers intuitively understand and exploit, such as the (near) invariance to the renaming of identifiers. We show that LLMs not only fail to properly generate correct Python code when default function names are swapped, but some of them even become more confident in their incorrect predictions as the model size increases, an instance of the recently discovered phenomenon of Inverse Scaling, which runs contrary to the commonly observed trend of increasing prediction quality with increasing model size. Our findings indicate that, despite their astonishing typical-case performance, LLMs still lack a deep, abstract understanding of the content they manipulate, making them unsuitable for tasks that statistically deviate from their training data, and that mere scaling is not enough to achieve such capability." @default.
- W4378498706 created "2023-05-27" @default.
- W4378498706 creator A5030363604 @default.
- W4378498706 creator A5030503109 @default.
- W4378498706 creator A5030546839 @default.
- W4378498706 creator A5050436152 @default.
- W4378498706 date "2023-05-24" @default.
- W4378498706 modified "2023-09-25" @default.
- W4378498706 title "The Larger They Are, the Harder They Fail: Language Models do not Recognize Identifier Swaps in Python" @default.
- W4378498706 doi "https://doi.org/10.48550/arxiv.2305.15507" @default.
- W4378498706 hasPublicationYear "2023" @default.
- W4378498706 type Work @default.
- W4378498706 citedByCount "0" @default.
- W4378498706 crossrefType "posted-content" @default.
- W4378498706 hasAuthorship W4378498706A5030363604 @default.
- W4378498706 hasAuthorship W4378498706A5030503109 @default.
- W4378498706 hasAuthorship W4378498706A5030546839 @default.
- W4378498706 hasAuthorship W4378498706A5050436152 @default.
- W4378498706 hasBestOaLocation W43784987061 @default.
- W4378498706 hasConcept C154504017 @default.
- W4378498706 hasConcept C154945302 @default.
- W4378498706 hasConcept C165696696 @default.
- W4378498706 hasConcept C199360897 @default.
- W4378498706 hasConcept C38652104 @default.
- W4378498706 hasConcept C41008148 @default.
- W4378498706 hasConcept C519991488 @default.
- W4378498706 hasConcept C80444323 @default.
- W4378498706 hasConceptScore W4378498706C154504017 @default.
- W4378498706 hasConceptScore W4378498706C154945302 @default.
- W4378498706 hasConceptScore W4378498706C165696696 @default.
- W4378498706 hasConceptScore W4378498706C199360897 @default.
- W4378498706 hasConceptScore W4378498706C38652104 @default.
- W4378498706 hasConceptScore W4378498706C41008148 @default.
- W4378498706 hasConceptScore W4378498706C519991488 @default.
- W4378498706 hasConceptScore W4378498706C80444323 @default.
- W4378498706 hasLocation W43784987061 @default.
- W4378498706 hasOpenAccess W4378498706 @default.
- W4378498706 hasPrimaryLocation W43784987061 @default.
- W4378498706 hasRelatedWork W1999473061 @default.
- W4378498706 hasRelatedWork W2018535394 @default.
- W4378498706 hasRelatedWork W2327204559 @default.
- W4378498706 hasRelatedWork W2529681551 @default.
- W4378498706 hasRelatedWork W2964604098 @default.
- W4378498706 hasRelatedWork W3000979607 @default.
- W4378498706 hasRelatedWork W3017187763 @default.
- W4378498706 hasRelatedWork W4232504361 @default.
- W4378498706 hasRelatedWork W4245752324 @default.
- W4378498706 hasRelatedWork W4297497426 @default.
- W4378498706 isParatext "false" @default.
- W4378498706 isRetracted "false" @default.
- W4378498706 workType "article" @default.