Matches in SemOpenAlex for { <https://semopenalex.org/work/W4200633178> ?p ?o ?g. }
Showing items 1 to 57 of
57
with 100 items per page.
- W4200633178 abstract "ByteScheduler partitions and rearranges tensor transmissions to improve the communication efficiency of distributed Deep Neural Network (DNN) training. The configuration of hyper-parameters (i.e., the partition size and the credit size) is critical to the effectiveness of partitioning and rearrangement. Currently, ByteScheduler adopts Bayesian Optimization (BO) to find the optimal configuration for the hyper-parameters beforehand. In practice, however, various runtime factors (e.g., worker node status and network conditions) change over time, making the statically-determined one-shot configuration result suboptimal for real-world DNN training. To address this problem, we present a real-time configuration method (called AutoByte) that automatically and timely searches the optimal hyper-parameters as the training systems dynamically change. AutoByte extends the ByteScheduler framework with a meta-network, which takes the system's runtime statistics as its input and outputs predictions for speedups under specific configurations. Evaluation results on various DNN models show that AutoByte can dynamically tune the hyper-parameters with low resource usage, and deliver up to 33.2% higher performance than the best static configuration in ByteScheduler." @default.
- W4200633178 created "2021-12-31" @default.
- W4200633178 creator A5016523647 @default.
- W4200633178 creator A5033581609 @default.
- W4200633178 creator A5048817987 @default.
- W4200633178 creator A5088713944 @default.
- W4200633178 date "2021-12-26" @default.
- W4200633178 modified "2023-10-16" @default.
- W4200633178 title "Automatic Configuration for Optimal Communication Scheduling in DNN Training" @default.
- W4200633178 doi "https://doi.org/10.48550/arxiv.2112.13509" @default.
- W4200633178 hasPublicationYear "2021" @default.
- W4200633178 type Work @default.
- W4200633178 citedByCount "0" @default.
- W4200633178 crossrefType "posted-content" @default.
- W4200633178 hasAuthorship W4200633178A5016523647 @default.
- W4200633178 hasAuthorship W4200633178A5033581609 @default.
- W4200633178 hasAuthorship W4200633178A5048817987 @default.
- W4200633178 hasAuthorship W4200633178A5088713944 @default.
- W4200633178 hasBestOaLocation W42006331781 @default.
- W4200633178 hasConcept C114614502 @default.
- W4200633178 hasConcept C119857082 @default.
- W4200633178 hasConcept C120314980 @default.
- W4200633178 hasConcept C126255220 @default.
- W4200633178 hasConcept C154945302 @default.
- W4200633178 hasConcept C206729178 @default.
- W4200633178 hasConcept C33923547 @default.
- W4200633178 hasConcept C41008148 @default.
- W4200633178 hasConcept C42812 @default.
- W4200633178 hasConcept C50644808 @default.
- W4200633178 hasConcept C79403827 @default.
- W4200633178 hasConceptScore W4200633178C114614502 @default.
- W4200633178 hasConceptScore W4200633178C119857082 @default.
- W4200633178 hasConceptScore W4200633178C120314980 @default.
- W4200633178 hasConceptScore W4200633178C126255220 @default.
- W4200633178 hasConceptScore W4200633178C154945302 @default.
- W4200633178 hasConceptScore W4200633178C206729178 @default.
- W4200633178 hasConceptScore W4200633178C33923547 @default.
- W4200633178 hasConceptScore W4200633178C41008148 @default.
- W4200633178 hasConceptScore W4200633178C42812 @default.
- W4200633178 hasConceptScore W4200633178C50644808 @default.
- W4200633178 hasConceptScore W4200633178C79403827 @default.
- W4200633178 hasLocation W42006331781 @default.
- W4200633178 hasOpenAccess W4200633178 @default.
- W4200633178 hasPrimaryLocation W42006331781 @default.
- W4200633178 hasRelatedWork W1504980819 @default.
- W4200633178 hasRelatedWork W1882733036 @default.
- W4200633178 hasRelatedWork W1992741870 @default.
- W4200633178 hasRelatedWork W2160425906 @default.
- W4200633178 hasRelatedWork W2356889451 @default.
- W4200633178 hasRelatedWork W2546696010 @default.
- W4200633178 hasRelatedWork W2961085424 @default.
- W4200633178 hasRelatedWork W4286629047 @default.
- W4200633178 hasRelatedWork W4306674287 @default.
- W4200633178 hasRelatedWork W4224009465 @default.
- W4200633178 isParatext "false" @default.
- W4200633178 isRetracted "false" @default.
- W4200633178 workType "article" @default.