Matches in SemOpenAlex for { <https://semopenalex.org/work/W2891815651> ?p ?o ?g. }
Showing items 1 to 89 of
89
with 100 items per page.
- W2891815651 abstract "Music relies heavily on self-reference to build structure and meaning. Weexplore the Transformer architecture (Vaswani et al., 2017) as a generativemodel for music, as self-attention has shown compelling results on tasks thatrequire long-term structure such as Wikipedia summary generation (Liu et al,2018). However, timing information is critical for polyphonic music, andTransformer does not explicitly model absolute or relative timing in itsstructure. To address this challenge, Shaw et al. (2018) introduced relativeposition representations to self-attention to improve machine translation.However, the formulation was not scalable to longer sequences. We propose animproved formulation which reduces the memory requirements of the relativeposition computation from $O(l^2d)$ to $O(ld)$, making it possible to trainmuch longer sequences and achieve faster convergence. In experiments onsymbolic music we find that relative self-attention substantially improvessample quality for unconditioned generation and is able to generate sequencesof lengths longer than those from the training set. When primed with an initialsequence, the model generates continuations that develop the prime coherentlyand exhibit long-term structure. Relative self-attention can be instrumental incapturing richer relationships within a musical piece." @default.
- W2891815651 created "2018-09-27" @default.
- W2891815651 creator A5007555810 @default.
- W2891815651 creator A5021878400 @default.
- W2891815651 creator A5022416424 @default.
- W2891815651 creator A5029197877 @default.
- W2891815651 creator A5050626617 @default.
- W2891815651 creator A5055592482 @default.
- W2891815651 creator A5069219684 @default.
- W2891815651 creator A5083209727 @default.
- W2891815651 date "2018-09-12" @default.
- W2891815651 modified "2023-09-27" @default.
- W2891815651 title "An Improved Relative Self-Attention Mechanism for Transformer with Application to Music Generation" @default.
- W2891815651 cites W147294073 @default.
- W2891815651 cites W1498436455 @default.
- W2891815651 cites W1813659000 @default.
- W2891815651 cites W1819710477 @default.
- W2891815651 cites W2135181320 @default.
- W2891815651 cites W2137619888 @default.
- W2891815651 cites W2170111110 @default.
- W2891815651 cites W2753868141 @default.
- W2891815651 cites W2754229890 @default.
- W2891815651 cites W2963403868 @default.
- W2891815651 hasPublicationYear "2018" @default.
- W2891815651 type Work @default.
- W2891815651 sameAs 2891815651 @default.
- W2891815651 citedByCount "26" @default.
- W2891815651 countsByYear W28918156512018 @default.
- W2891815651 countsByYear W28918156512019 @default.
- W2891815651 countsByYear W28918156512020 @default.
- W2891815651 countsByYear W28918156512021 @default.
- W2891815651 crossrefType "posted-content" @default.
- W2891815651 hasAuthorship W2891815651A5007555810 @default.
- W2891815651 hasAuthorship W2891815651A5021878400 @default.
- W2891815651 hasAuthorship W2891815651A5022416424 @default.
- W2891815651 hasAuthorship W2891815651A5029197877 @default.
- W2891815651 hasAuthorship W2891815651A5050626617 @default.
- W2891815651 hasAuthorship W2891815651A5055592482 @default.
- W2891815651 hasAuthorship W2891815651A5069219684 @default.
- W2891815651 hasAuthorship W2891815651A5083209727 @default.
- W2891815651 hasConcept C11413529 @default.
- W2891815651 hasConcept C121332964 @default.
- W2891815651 hasConcept C128979739 @default.
- W2891815651 hasConcept C154945302 @default.
- W2891815651 hasConcept C15744967 @default.
- W2891815651 hasConcept C165801399 @default.
- W2891815651 hasConcept C19417346 @default.
- W2891815651 hasConcept C39890363 @default.
- W2891815651 hasConcept C41008148 @default.
- W2891815651 hasConcept C62520636 @default.
- W2891815651 hasConcept C66322947 @default.
- W2891815651 hasConceptScore W2891815651C11413529 @default.
- W2891815651 hasConceptScore W2891815651C121332964 @default.
- W2891815651 hasConceptScore W2891815651C128979739 @default.
- W2891815651 hasConceptScore W2891815651C154945302 @default.
- W2891815651 hasConceptScore W2891815651C15744967 @default.
- W2891815651 hasConceptScore W2891815651C165801399 @default.
- W2891815651 hasConceptScore W2891815651C19417346 @default.
- W2891815651 hasConceptScore W2891815651C39890363 @default.
- W2891815651 hasConceptScore W2891815651C41008148 @default.
- W2891815651 hasConceptScore W2891815651C62520636 @default.
- W2891815651 hasConceptScore W2891815651C66322947 @default.
- W2891815651 hasLocation W28918156511 @default.
- W2891815651 hasOpenAccess W2891815651 @default.
- W2891815651 hasPrimaryLocation W28918156511 @default.
- W2891815651 hasRelatedWork W1819710477 @default.
- W2891815651 hasRelatedWork W2064675550 @default.
- W2891815651 hasRelatedWork W2099471712 @default.
- W2891815651 hasRelatedWork W2194775991 @default.
- W2891815651 hasRelatedWork W2413794162 @default.
- W2891815651 hasRelatedWork W2752134738 @default.
- W2891815651 hasRelatedWork W2773734397 @default.
- W2891815651 hasRelatedWork W2789541106 @default.
- W2891815651 hasRelatedWork W2940744433 @default.
- W2891815651 hasRelatedWork W2949382160 @default.
- W2891815651 hasRelatedWork W2963341956 @default.
- W2891815651 hasRelatedWork W2963403868 @default.
- W2891815651 hasRelatedWork W2963575853 @default.
- W2891815651 hasRelatedWork W2963681776 @default.
- W2891815651 hasRelatedWork W2964089206 @default.
- W2891815651 hasRelatedWork W2964110616 @default.
- W2891815651 hasRelatedWork W2964308564 @default.
- W2891815651 hasRelatedWork W2989571009 @default.
- W2891815651 hasRelatedWork W3038344852 @default.
- W2891815651 hasRelatedWork W3090327238 @default.
- W2891815651 isParatext "false" @default.
- W2891815651 isRetracted "false" @default.
- W2891815651 magId "2891815651" @default.
- W2891815651 workType "article" @default.