Matches in SemOpenAlex for { <https://semopenalex.org/work/W4387360079> ?p ?o ?g. }
Showing items 1 to 56 of
56
with 100 items per page.
- W4387360079 abstract "Although efficiency is one of the core metrics in programming, recent large-scale language models often face the issue of “inefficient code” generation, which struggles to meet the real-time requirements of algorithms. However, there is relatively little research on evaluating the selection of efficient algorithms, and it is not easy to rigorously assess a model’s ability to correctly choose efficient algorithm solutions. Furthermore, the selection of efficient algorithm solutions often relies on the appropriate application of problem-solving skills, necessitating more in-depth research on algorithm reasoning. To address this challenge, we introduce the Generation of Efficient Code (GEC) benchmark, which aims to evaluate the ability to select efficient algorithm solutions. Unlike code generation, our benchmark focuses on a model’s ability to generate satisfactory efficient code when given a natural language description and inefficient code. We propose two novel metrics to examine the efficiency of the generated code and assess the model’s ability to generate efficient code. Our benchmark includes 3,712 problems, 31,577 combinations of efficient and inefficient code pairs, and 13,092 alternative efficient codes. We evaluate the performance of mainstream code generation models on the GEC benchmark. As the societal importance of code efficiency increases in the coming years, our benchmark will provide an essential measurement standard for tracking research progress. Our dataset and models are open-source and can be accessed at https://github.com/CodeGeneration2/Efficient-Code-Generation-with-GEC." @default.
- W4387360079 created "2023-10-06" @default.
- W4387360079 creator A5045065594 @default.
- W4387360079 creator A5093006956 @default.
- W4387360079 date "2023-08-04" @default.
- W4387360079 modified "2023-10-06" @default.
- W4387360079 title "Measuring Efficient Code Generation with GEC" @default.
- W4387360079 cites W4229675450 @default.
- W4387360079 cites W4311887664 @default.
- W4387360079 doi "https://doi.org/10.1145/3609437.3609459" @default.
- W4387360079 hasPublicationYear "2023" @default.
- W4387360079 type Work @default.
- W4387360079 citedByCount "0" @default.
- W4387360079 crossrefType "proceedings-article" @default.
- W4387360079 hasAuthorship W4387360079A5045065594 @default.
- W4387360079 hasAuthorship W4387360079A5093006956 @default.
- W4387360079 hasConcept C13280743 @default.
- W4387360079 hasConcept C133162039 @default.
- W4387360079 hasConcept C151578736 @default.
- W4387360079 hasConcept C177264268 @default.
- W4387360079 hasConcept C185798385 @default.
- W4387360079 hasConcept C199360897 @default.
- W4387360079 hasConcept C205649164 @default.
- W4387360079 hasConcept C26517878 @default.
- W4387360079 hasConcept C2776760102 @default.
- W4387360079 hasConcept C38652104 @default.
- W4387360079 hasConcept C41008148 @default.
- W4387360079 hasConcept C43126263 @default.
- W4387360079 hasConceptScore W4387360079C13280743 @default.
- W4387360079 hasConceptScore W4387360079C133162039 @default.
- W4387360079 hasConceptScore W4387360079C151578736 @default.
- W4387360079 hasConceptScore W4387360079C177264268 @default.
- W4387360079 hasConceptScore W4387360079C185798385 @default.
- W4387360079 hasConceptScore W4387360079C199360897 @default.
- W4387360079 hasConceptScore W4387360079C205649164 @default.
- W4387360079 hasConceptScore W4387360079C26517878 @default.
- W4387360079 hasConceptScore W4387360079C2776760102 @default.
- W4387360079 hasConceptScore W4387360079C38652104 @default.
- W4387360079 hasConceptScore W4387360079C41008148 @default.
- W4387360079 hasConceptScore W4387360079C43126263 @default.
- W4387360079 hasLocation W43873600791 @default.
- W4387360079 hasOpenAccess W4387360079 @default.
- W4387360079 hasPrimaryLocation W43873600791 @default.
- W4387360079 hasRelatedWork W116913286 @default.
- W4387360079 hasRelatedWork W1977905671 @default.
- W4387360079 hasRelatedWork W2356102990 @default.
- W4387360079 hasRelatedWork W2407476586 @default.
- W4387360079 hasRelatedWork W2750654225 @default.
- W4387360079 hasRelatedWork W2763045826 @default.
- W4387360079 hasRelatedWork W3081644756 @default.
- W4387360079 hasRelatedWork W3122369893 @default.
- W4387360079 hasRelatedWork W3162146186 @default.
- W4387360079 hasRelatedWork W632311702 @default.
- W4387360079 isParatext "false" @default.
- W4387360079 isRetracted "false" @default.
- W4387360079 workType "article" @default.