Matches in SemOpenAlex for { <https://semopenalex.org/work/W3046490292> ?p ?o ?g. }
Showing items 1 to 89 of
89
with 100 items per page.
- W3046490292 abstract "Deep Learning (DL) models for semantic image segmentation are an emerging trend in response to the explosion of multi-class, high resolution image and video data. However, segmentation models are highly compute-intensive, and even the fastest Volta GPUs cannot train them in a reasonable time frame. In our experiments, we observed just 6.7 images/second on a single Volta GPU for training DeepLab-v3+ (DLv3+), a state-of-the-art Encoder-Decoder model for semantic image segmentation. For comparison, a Volta GPU can process 300 images/second for training ResNet-50, a state-of-the-art model for image classification. In this context, we see a clear opportunity to utilize supercomputers to speed up training of segmentation models. However, most published studies on the performance of novel DL models such as DLv3+ require the user to significantly change Horovod, MPI, and the DL model to improve performance. Our work proposes an alternative tuning method that achieves near-linear scaling without significant changes to Horovod, MPI, or the DL model. In this paper, we select DLv3+ as the candidate TensorFlow model and implement Horovod-based distributed training for DLv3+. We observed poor default scaling performance of DLv3+ on the Summit system at Oak Ridge National Laboratory. To address this, we conducted an in-depth performance tuning of various Horovod/MPI knobs to achieve better performance over the default parameters. We present a comprehensive scaling comparison for Horovod with MVAPICH2-GDR up to 132 GPUs on Summit. Our optimization approach achieves near-linear (92%) scaling with MVAPICH2-GDR. We achieved a “mIOU” accuracy of 80.8% for distributed training, which is on par with published accuracy for this model. Further, we demonstrate an improvement in scaling efficiency by 23.9% over default Horovod training, which translates to a 1.3× speedup in training performance." @default.
- W3046490292 created "2020-08-07" @default.
- W3046490292 creator A5004330728 @default.
- W3046490292 creator A5015153122 @default.
- W3046490292 creator A5024879682 @default.
- W3046490292 creator A5034293705 @default.
- W3046490292 creator A5057769693 @default.
- W3046490292 date "2020-05-01" @default.
- W3046490292 modified "2023-09-23" @default.
- W3046490292 title "Efficient Training of Semantic Image Segmentation on Summit using Horovod and MVAPICH2-GDR" @default.
- W3046490292 cites W1903029394 @default.
- W3046490292 cites W2412782625 @default.
- W3046490292 cites W2580688187 @default.
- W3046490292 cites W2604721644 @default.
- W3046490292 cites W2783231089 @default.
- W3046490292 cites W2886189612 @default.
- W3046490292 cites W2898319404 @default.
- W3046490292 cites W2921208939 @default.
- W3046490292 cites W2952046647 @default.
- W3046490292 cites W2987776948 @default.
- W3046490292 cites W3017432630 @default.
- W3046490292 doi "https://doi.org/10.1109/ipdpsw50202.2020.00171" @default.
- W3046490292 hasPublicationYear "2020" @default.
- W3046490292 type Work @default.
- W3046490292 sameAs 3046490292 @default.
- W3046490292 citedByCount "4" @default.
- W3046490292 countsByYear W30464902922021 @default.
- W3046490292 countsByYear W30464902922023 @default.
- W3046490292 crossrefType "proceedings-article" @default.
- W3046490292 hasAuthorship W3046490292A5004330728 @default.
- W3046490292 hasAuthorship W3046490292A5015153122 @default.
- W3046490292 hasAuthorship W3046490292A5024879682 @default.
- W3046490292 hasAuthorship W3046490292A5034293705 @default.
- W3046490292 hasAuthorship W3046490292A5057769693 @default.
- W3046490292 hasConcept C100970517 @default.
- W3046490292 hasConcept C108583219 @default.
- W3046490292 hasConcept C111919701 @default.
- W3046490292 hasConcept C118505674 @default.
- W3046490292 hasConcept C119857082 @default.
- W3046490292 hasConcept C124504099 @default.
- W3046490292 hasConcept C151730666 @default.
- W3046490292 hasConcept C154945302 @default.
- W3046490292 hasConcept C205649164 @default.
- W3046490292 hasConcept C2524010 @default.
- W3046490292 hasConcept C2778848561 @default.
- W3046490292 hasConcept C2779343474 @default.
- W3046490292 hasConcept C3261483 @default.
- W3046490292 hasConcept C33923547 @default.
- W3046490292 hasConcept C41008148 @default.
- W3046490292 hasConcept C5274069 @default.
- W3046490292 hasConcept C86803240 @default.
- W3046490292 hasConcept C89600930 @default.
- W3046490292 hasConcept C99844830 @default.
- W3046490292 hasConceptScore W3046490292C100970517 @default.
- W3046490292 hasConceptScore W3046490292C108583219 @default.
- W3046490292 hasConceptScore W3046490292C111919701 @default.
- W3046490292 hasConceptScore W3046490292C118505674 @default.
- W3046490292 hasConceptScore W3046490292C119857082 @default.
- W3046490292 hasConceptScore W3046490292C124504099 @default.
- W3046490292 hasConceptScore W3046490292C151730666 @default.
- W3046490292 hasConceptScore W3046490292C154945302 @default.
- W3046490292 hasConceptScore W3046490292C205649164 @default.
- W3046490292 hasConceptScore W3046490292C2524010 @default.
- W3046490292 hasConceptScore W3046490292C2778848561 @default.
- W3046490292 hasConceptScore W3046490292C2779343474 @default.
- W3046490292 hasConceptScore W3046490292C3261483 @default.
- W3046490292 hasConceptScore W3046490292C33923547 @default.
- W3046490292 hasConceptScore W3046490292C41008148 @default.
- W3046490292 hasConceptScore W3046490292C5274069 @default.
- W3046490292 hasConceptScore W3046490292C86803240 @default.
- W3046490292 hasConceptScore W3046490292C89600930 @default.
- W3046490292 hasConceptScore W3046490292C99844830 @default.
- W3046490292 hasLocation W30464902921 @default.
- W3046490292 hasOpenAccess W3046490292 @default.
- W3046490292 hasPrimaryLocation W30464902921 @default.
- W3046490292 hasRelatedWork W2790662084 @default.
- W3046490292 hasRelatedWork W2948658236 @default.
- W3046490292 hasRelatedWork W2960184797 @default.
- W3046490292 hasRelatedWork W2972093541 @default.
- W3046490292 hasRelatedWork W3115553566 @default.
- W3046490292 hasRelatedWork W3135174555 @default.
- W3046490292 hasRelatedWork W3209779739 @default.
- W3046490292 hasRelatedWork W4223943233 @default.
- W3046490292 hasRelatedWork W4285827401 @default.
- W3046490292 hasRelatedWork W4293211451 @default.
- W3046490292 isParatext "false" @default.
- W3046490292 isRetracted "false" @default.
- W3046490292 magId "3046490292" @default.
- W3046490292 workType "article" @default.