Matches in SemOpenAlex for { <https://semopenalex.org/work/W4387561321> ?p ?o ?g. }
Showing items 1 to 87 of
87
with 100 items per page.
- W4387561321 abstract "While code generation has been widely used in various software development scenarios, the quality of the generated code is not guaranteed. This has been a particular concern in the era of large language models (LLMs)- based code generation, where LLMs, deemed a complex and powerful black-box model, is instructed by a high-level natural language specification, namely a prompt, to generate code. Nevertheless, effectively evaluating and explaining the code generation capability of LLMs is inherently challenging, given the complexity of LLMs and the lack of transparency. Inspired by the recent progress in causality analysis and its application in software engineering, this paper launches a causality analysis-based approach to systematically analyze the causal relations between the LLM input prompts and the generated code. To handle various technical challenges in this study, we first propose a novel causal graph-based representation of the prompt and the generated code, which is established over the fine-grained, human-understandable concepts in the input prompts. The formed causal graph is then used to identify the causal relations between the prompt and the derived code. We illustrate the insights that our framework can provide by studying over 3 popular LLMs with over 12 prompt adjustment strategies. The results of these studies illustrate the potential of our technique to provide insights into LLM effectiveness, and aid end-users in understanding predictions. Additionally, we demonstrate that our approach provides actionable insights to improve the quality of the LLM-generated code by properly calibrating the prompt." @default.
- W4387561321 created "2023-10-12" @default.
- W4387561321 creator A5006901857 @default.
- W4387561321 creator A5048399617 @default.
- W4387561321 creator A5070959037 @default.
- W4387561321 creator A5072001009 @default.
- W4387561321 date "2023-10-10" @default.
- W4387561321 modified "2023-10-13" @default.
- W4387561321 title "Benchmarking and Explaining Large Language Model-based Code Generation: A Causality-Centric Approach" @default.
- W4387561321 doi "https://doi.org/10.48550/arxiv.2310.06680" @default.
- W4387561321 hasPublicationYear "2023" @default.
- W4387561321 type Work @default.
- W4387561321 citedByCount "0" @default.
- W4387561321 crossrefType "posted-content" @default.
- W4387561321 hasAuthorship W4387561321A5006901857 @default.
- W4387561321 hasAuthorship W4387561321A5048399617 @default.
- W4387561321 hasAuthorship W4387561321A5070959037 @default.
- W4387561321 hasAuthorship W4387561321A5072001009 @default.
- W4387561321 hasBestOaLocation W43875613211 @default.
- W4387561321 hasConcept C111472728 @default.
- W4387561321 hasConcept C115903868 @default.
- W4387561321 hasConcept C11671645 @default.
- W4387561321 hasConcept C121332964 @default.
- W4387561321 hasConcept C132525143 @default.
- W4387561321 hasConcept C133162039 @default.
- W4387561321 hasConcept C138885662 @default.
- W4387561321 hasConcept C142724271 @default.
- W4387561321 hasConcept C144133560 @default.
- W4387561321 hasConcept C154945302 @default.
- W4387561321 hasConcept C162853370 @default.
- W4387561321 hasConcept C177264268 @default.
- W4387561321 hasConcept C195324797 @default.
- W4387561321 hasConcept C199360897 @default.
- W4387561321 hasConcept C2522767166 @default.
- W4387561321 hasConcept C26517878 @default.
- W4387561321 hasConcept C2776187449 @default.
- W4387561321 hasConcept C2776760102 @default.
- W4387561321 hasConcept C2779530757 @default.
- W4387561321 hasConcept C38652104 @default.
- W4387561321 hasConcept C41008148 @default.
- W4387561321 hasConcept C62520636 @default.
- W4387561321 hasConcept C64357122 @default.
- W4387561321 hasConcept C71924100 @default.
- W4387561321 hasConcept C80444323 @default.
- W4387561321 hasConcept C86251818 @default.
- W4387561321 hasConceptScore W4387561321C111472728 @default.
- W4387561321 hasConceptScore W4387561321C115903868 @default.
- W4387561321 hasConceptScore W4387561321C11671645 @default.
- W4387561321 hasConceptScore W4387561321C121332964 @default.
- W4387561321 hasConceptScore W4387561321C132525143 @default.
- W4387561321 hasConceptScore W4387561321C133162039 @default.
- W4387561321 hasConceptScore W4387561321C138885662 @default.
- W4387561321 hasConceptScore W4387561321C142724271 @default.
- W4387561321 hasConceptScore W4387561321C144133560 @default.
- W4387561321 hasConceptScore W4387561321C154945302 @default.
- W4387561321 hasConceptScore W4387561321C162853370 @default.
- W4387561321 hasConceptScore W4387561321C177264268 @default.
- W4387561321 hasConceptScore W4387561321C195324797 @default.
- W4387561321 hasConceptScore W4387561321C199360897 @default.
- W4387561321 hasConceptScore W4387561321C2522767166 @default.
- W4387561321 hasConceptScore W4387561321C26517878 @default.
- W4387561321 hasConceptScore W4387561321C2776187449 @default.
- W4387561321 hasConceptScore W4387561321C2776760102 @default.
- W4387561321 hasConceptScore W4387561321C2779530757 @default.
- W4387561321 hasConceptScore W4387561321C38652104 @default.
- W4387561321 hasConceptScore W4387561321C41008148 @default.
- W4387561321 hasConceptScore W4387561321C62520636 @default.
- W4387561321 hasConceptScore W4387561321C64357122 @default.
- W4387561321 hasConceptScore W4387561321C71924100 @default.
- W4387561321 hasConceptScore W4387561321C80444323 @default.
- W4387561321 hasConceptScore W4387561321C86251818 @default.
- W4387561321 hasLocation W43875613211 @default.
- W4387561321 hasOpenAccess W4387561321 @default.
- W4387561321 hasPrimaryLocation W43875613211 @default.
- W4387561321 hasRelatedWork W2018580387 @default.
- W4387561321 hasRelatedWork W2152921782 @default.
- W4387561321 hasRelatedWork W2249392076 @default.
- W4387561321 hasRelatedWork W2575772232 @default.
- W4387561321 hasRelatedWork W3135588948 @default.
- W4387561321 hasRelatedWork W3170261037 @default.
- W4387561321 hasRelatedWork W382594479 @default.
- W4387561321 hasRelatedWork W4210420802 @default.
- W4387561321 hasRelatedWork W4312269093 @default.
- W4387561321 hasRelatedWork W2470045054 @default.
- W4387561321 isParatext "false" @default.
- W4387561321 isRetracted "false" @default.
- W4387561321 workType "article" @default.