Matches in SemOpenAlex for { <https://semopenalex.org/work/W4285207429> ?p ?o ?g. }
Showing items 1 to 78 of
78
with 100 items per page.
- W4285207429 endingPage "15" @default.
- W4285207429 startingPage "2" @default.
- W4285207429 abstract "Over the past few years, attention is shining in the field of deep learning, especially in the domain of natural language processing (NLP). Its impressive effectiveness, along with ubiquitous implementations, have aroused our interest in efficiently scheduling the data-flow of corresponding computations onto architectures with many computing units to realize parallel computing. In this paper, based on manually analyzing the optimum scheduling solutions for small instances, which are obtained by a satisfiability checking (SAT) solver, we propose a general scheduling solution to parallelize the processing of attention layers that are widely adopted in recent deep learning models. According to the solution, for the proposed hardware system with m processing elements (PEs) connected in a unidirectional ring, a m-time speed up is achievable. For two specific application schemes of attention, we respectively recognize that almost 25% and 50% of the original computations have become redundant under those certain circumstances. To avoid unnecessary computing with corresponding gains in processing latency, we have come up with strategies of optimization accordingly, which further lead to another two scheduling solutions. By avoiding the redundancy, the adoptions of the optimized scheduling solutions are able to additionally bring near 25% and 50% reduction in execution cycles, respectively for the two application schemes. To prove the correctness of these solutions, we have mathematically revealed their validity, as well as utilized SAT solver to conduct the verification by adopting the solutions themselves as additional constraints for the formulated SAT problems." @default.
- W4285207429 created "2022-07-14" @default.
- W4285207429 creator A5027837299 @default.
- W4285207429 creator A5045896058 @default.
- W4285207429 creator A5079108909 @default.
- W4285207429 date "2022-01-01" @default.
- W4285207429 modified "2023-10-05" @default.
- W4285207429 title "Parallel Scheduling Attention Mechanism: Generalization and Optimization" @default.
- W4285207429 cites W2100609826 @default.
- W4285207429 cites W2152839228 @default.
- W4285207429 cites W2246401527 @default.
- W4285207429 cites W2442974303 @default.
- W4285207429 cites W2585720638 @default.
- W4285207429 cites W2771654260 @default.
- W4285207429 cites W2802023636 @default.
- W4285207429 cites W2809276277 @default.
- W4285207429 cites W2898390739 @default.
- W4285207429 cites W2940882113 @default.
- W4285207429 cites W2963367478 @default.
- W4285207429 cites W2980186997 @default.
- W4285207429 cites W2981413347 @default.
- W4285207429 cites W3016542674 @default.
- W4285207429 cites W3041347357 @default.
- W4285207429 cites W3112630515 @default.
- W4285207429 cites W3128354418 @default.
- W4285207429 doi "https://doi.org/10.2197/ipsjtsldm.15.2" @default.
- W4285207429 hasPublicationYear "2022" @default.
- W4285207429 type Work @default.
- W4285207429 citedByCount "0" @default.
- W4285207429 crossrefType "journal-article" @default.
- W4285207429 hasAuthorship W4285207429A5027837299 @default.
- W4285207429 hasAuthorship W4285207429A5045896058 @default.
- W4285207429 hasAuthorship W4285207429A5079108909 @default.
- W4285207429 hasBestOaLocation W42852074291 @default.
- W4285207429 hasConcept C11413529 @default.
- W4285207429 hasConcept C120314980 @default.
- W4285207429 hasConcept C126255220 @default.
- W4285207429 hasConcept C173608175 @default.
- W4285207429 hasConcept C199360897 @default.
- W4285207429 hasConcept C206729178 @default.
- W4285207429 hasConcept C26713055 @default.
- W4285207429 hasConcept C2778770139 @default.
- W4285207429 hasConcept C33923547 @default.
- W4285207429 hasConcept C41008148 @default.
- W4285207429 hasConcept C45374587 @default.
- W4285207429 hasConcept C55439883 @default.
- W4285207429 hasConceptScore W4285207429C11413529 @default.
- W4285207429 hasConceptScore W4285207429C120314980 @default.
- W4285207429 hasConceptScore W4285207429C126255220 @default.
- W4285207429 hasConceptScore W4285207429C173608175 @default.
- W4285207429 hasConceptScore W4285207429C199360897 @default.
- W4285207429 hasConceptScore W4285207429C206729178 @default.
- W4285207429 hasConceptScore W4285207429C26713055 @default.
- W4285207429 hasConceptScore W4285207429C2778770139 @default.
- W4285207429 hasConceptScore W4285207429C33923547 @default.
- W4285207429 hasConceptScore W4285207429C41008148 @default.
- W4285207429 hasConceptScore W4285207429C45374587 @default.
- W4285207429 hasConceptScore W4285207429C55439883 @default.
- W4285207429 hasIssue "0" @default.
- W4285207429 hasLocation W42852074291 @default.
- W4285207429 hasOpenAccess W4285207429 @default.
- W4285207429 hasPrimaryLocation W42852074291 @default.
- W4285207429 hasRelatedWork W1493274197 @default.
- W4285207429 hasRelatedWork W1594283178 @default.
- W4285207429 hasRelatedWork W1882733036 @default.
- W4285207429 hasRelatedWork W1964876237 @default.
- W4285207429 hasRelatedWork W1992741870 @default.
- W4285207429 hasRelatedWork W2055471337 @default.
- W4285207429 hasRelatedWork W2080076943 @default.
- W4285207429 hasRelatedWork W2160425906 @default.
- W4285207429 hasRelatedWork W2315378174 @default.
- W4285207429 hasRelatedWork W2546696010 @default.
- W4285207429 hasVolume "15" @default.
- W4285207429 isParatext "false" @default.
- W4285207429 isRetracted "false" @default.
- W4285207429 workType "article" @default.