Matches in SemOpenAlex for { <https://semopenalex.org/work/W4383473924> ?p ?o ?g. }
Showing items 1 to 85 of
85
with 100 items per page.
- W4383473924 abstract "Large-scale code generation models such as Codex and CodeT5 have achieved impressive performance. However, libraries are upgraded or deprecated very frequently and re-training large-scale language models is computationally expensive. Therefore, Continual Learning (CL) is an important aspect that remains underexplored in the code domain. In this paper, we introduce a benchmark called CodeTask-CL that covers a wide range of tasks, including code generation, translation, summarization, and refinement, with different input and output programming languages. Next, on our CodeTask-CL benchmark, we compare popular CL techniques from NLP and Vision domains. We find that effective methods like Prompt Pooling (PP) suffer from catastrophic forgetting due to the unstable training of the prompt selection mechanism caused by stark distribution shifts in coding tasks. We address this issue with our proposed method, Prompt Pooling with Teacher Forcing (PP-TF), that stabilizes training by enforcing constraints on the prompt selection mechanism and leads to a 21.54% improvement over Prompt Pooling. Along with the benchmark, we establish a training pipeline that can be used for CL on code models, which we believe can motivate further development of CL methods for code models. Our code is available at https://github.com/amazon-science/codetaskcl-pptf" @default.
- W4383473924 created "2023-07-07" @default.
- W4383473924 creator A5000542080 @default.
- W4383473924 creator A5001906708 @default.
- W4383473924 creator A5001987532 @default.
- W4383473924 creator A5002609733 @default.
- W4383473924 creator A5006771539 @default.
- W4383473924 creator A5022974076 @default.
- W4383473924 creator A5024782538 @default.
- W4383473924 creator A5059197338 @default.
- W4383473924 creator A5067518142 @default.
- W4383473924 creator A5077875900 @default.
- W4383473924 creator A5078881867 @default.
- W4383473924 creator A5091676855 @default.
- W4383473924 date "2023-07-05" @default.
- W4383473924 modified "2023-10-16" @default.
- W4383473924 title "Exploring Continual Learning for Code Generation Models" @default.
- W4383473924 doi "https://doi.org/10.48550/arxiv.2307.02435" @default.
- W4383473924 hasPublicationYear "2023" @default.
- W4383473924 type Work @default.
- W4383473924 citedByCount "0" @default.
- W4383473924 crossrefType "posted-content" @default.
- W4383473924 hasAuthorship W4383473924A5000542080 @default.
- W4383473924 hasAuthorship W4383473924A5001906708 @default.
- W4383473924 hasAuthorship W4383473924A5001987532 @default.
- W4383473924 hasAuthorship W4383473924A5002609733 @default.
- W4383473924 hasAuthorship W4383473924A5006771539 @default.
- W4383473924 hasAuthorship W4383473924A5022974076 @default.
- W4383473924 hasAuthorship W4383473924A5024782538 @default.
- W4383473924 hasAuthorship W4383473924A5059197338 @default.
- W4383473924 hasAuthorship W4383473924A5067518142 @default.
- W4383473924 hasAuthorship W4383473924A5077875900 @default.
- W4383473924 hasAuthorship W4383473924A5078881867 @default.
- W4383473924 hasAuthorship W4383473924A5091676855 @default.
- W4383473924 hasBestOaLocation W43834739241 @default.
- W4383473924 hasConcept C119857082 @default.
- W4383473924 hasConcept C13280743 @default.
- W4383473924 hasConcept C133162039 @default.
- W4383473924 hasConcept C154945302 @default.
- W4383473924 hasConcept C169590947 @default.
- W4383473924 hasConcept C170858558 @default.
- W4383473924 hasConcept C177264268 @default.
- W4383473924 hasConcept C185798385 @default.
- W4383473924 hasConcept C199360897 @default.
- W4383473924 hasConcept C199519371 @default.
- W4383473924 hasConcept C205649164 @default.
- W4383473924 hasConcept C26517878 @default.
- W4383473924 hasConcept C2776760102 @default.
- W4383473924 hasConcept C2777904410 @default.
- W4383473924 hasConcept C38652104 @default.
- W4383473924 hasConcept C41008148 @default.
- W4383473924 hasConcept C70437156 @default.
- W4383473924 hasConceptScore W4383473924C119857082 @default.
- W4383473924 hasConceptScore W4383473924C13280743 @default.
- W4383473924 hasConceptScore W4383473924C133162039 @default.
- W4383473924 hasConceptScore W4383473924C154945302 @default.
- W4383473924 hasConceptScore W4383473924C169590947 @default.
- W4383473924 hasConceptScore W4383473924C170858558 @default.
- W4383473924 hasConceptScore W4383473924C177264268 @default.
- W4383473924 hasConceptScore W4383473924C185798385 @default.
- W4383473924 hasConceptScore W4383473924C199360897 @default.
- W4383473924 hasConceptScore W4383473924C199519371 @default.
- W4383473924 hasConceptScore W4383473924C205649164 @default.
- W4383473924 hasConceptScore W4383473924C26517878 @default.
- W4383473924 hasConceptScore W4383473924C2776760102 @default.
- W4383473924 hasConceptScore W4383473924C2777904410 @default.
- W4383473924 hasConceptScore W4383473924C38652104 @default.
- W4383473924 hasConceptScore W4383473924C41008148 @default.
- W4383473924 hasConceptScore W4383473924C70437156 @default.
- W4383473924 hasLocation W43834739241 @default.
- W4383473924 hasOpenAccess W4383473924 @default.
- W4383473924 hasPrimaryLocation W43834739241 @default.
- W4383473924 hasRelatedWork W1963955771 @default.
- W4383473924 hasRelatedWork W2091716239 @default.
- W4383473924 hasRelatedWork W2374073571 @default.
- W4383473924 hasRelatedWork W283806354 @default.
- W4383473924 hasRelatedWork W3131442838 @default.
- W4383473924 hasRelatedWork W4220706374 @default.
- W4383473924 hasRelatedWork W4236262624 @default.
- W4383473924 hasRelatedWork W4244179825 @default.
- W4383473924 hasRelatedWork W4285428896 @default.
- W4383473924 hasRelatedWork W4308641647 @default.
- W4383473924 isParatext "false" @default.
- W4383473924 isRetracted "false" @default.
- W4383473924 workType "article" @default.