FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W4377231507> ?p ?o ?g. }

• W4377231507 endingPage "2950" @default.
• W4377231507 startingPage "2938" @default.
• W4377231507 abstract "Training Deep Neural Networks (DNN) concurrently is becoming increasingly important for deep learning practitioners, e.g., <italic xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>hyperparameter optimization (HPO)</i> and <italic xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>neural architecture search (NAS)</i> . The GPU memory capacity is the impediment that prohibits multiple DNNs from being trained on the same GPU due to the large memory usage during training. In this paper, we propose <italic xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>Waterwave</i> , a GPU memory flow engine for concurrent deep learning training. First, to address the memory explosion brought by the long time lag between memory allocation and deallocation time, we develop an allocator tailored for multi-streams. By making the allocator aware of the stream information, a <italic xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>prioritized allocation</i> is conducted based on the chunk's <italic xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>synchronization</i> attributes, allowing us to provide useable memory after scheduling rather than waiting it to be really released after GPU computation. Second, <italic xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>Waterwave</i> partitions the compute graph to a set of continuous <italic xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>node groups</i> and then performs finer-grained scheduling: <italic xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>NodeGroup pipeline execution</i> , to guarantee a proper memory requests order. <italic xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>Waterwave</i> can accomplish up to 96.8% of the maximum batch size of solo training. Additionally, in scenarios with high memory demand, <italic xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>Waterwave</i> can outperform existing spatial sharing and temporal sharing by up to 12x and 1.49x, respectively." @default.
• W4377231507 created "2023-05-23" @default.
• W4377231507 creator A5016486553 @default.
• W4377231507 creator A5022262922 @default.
• W4377231507 creator A5027296110 @default.
• W4377231507 creator A5057744965 @default.
• W4377231507 creator A5064596191 @default.
• W4377231507 date "2023-10-01" @default.
• W4377231507 modified "2023-10-05" @default.
• W4377231507 title "Waterwave: A GPU Memory Flow Engine for Concurrent DNN Training" @default.
• W4377231507 cites W2133796939 @default.
• W4377231507 cites W2183341477 @default.
• W4377231507 cites W2194775991 @default.
• W4377231507 cites W2489529491 @default.
• W4377231507 cites W2606722458 @default.
• W4377231507 cites W2772678505 @default.
• W4377231507 cites W2883283076 @default.
• W4377231507 cites W2893114493 @default.
• W4377231507 cites W2962821792 @default.
• W4377231507 cites W2964243274 @default.
• W4377231507 cites W2969388332 @default.
• W4377231507 cites W3012479151 @default.
• W4377231507 cites W3012514909 @default.
• W4377231507 cites W4200282803 @default.
• W4377231507 cites W4247353671 @default.
• W4377231507 doi "https://doi.org/10.1109/tc.2023.3278530" @default.
• W4377231507 hasPublicationYear "2023" @default.
• W4377231507 type Work @default.
• W4377231507 citedByCount "0" @default.
• W4377231507 crossrefType "journal-article" @default.
• W4377231507 hasAuthorship W4377231507A5016486553 @default.
• W4377231507 hasAuthorship W4377231507A5022262922 @default.
• W4377231507 hasAuthorship W4377231507A5027296110 @default.
• W4377231507 hasAuthorship W4377231507A5057744965 @default.
• W4377231507 hasAuthorship W4377231507A5064596191 @default.
• W4377231507 hasBestOaLocation W43772315072 @default.
• W4377231507 hasConcept C154945302 @default.
• W4377231507 hasConcept C162262903 @default.
• W4377231507 hasConcept C173608175 @default.
• W4377231507 hasConcept C41008148 @default.
• W4377231507 hasConceptScore W4377231507C154945302 @default.
• W4377231507 hasConceptScore W4377231507C162262903 @default.
• W4377231507 hasConceptScore W4377231507C173608175 @default.
• W4377231507 hasConceptScore W4377231507C41008148 @default.
• W4377231507 hasIssue "10" @default.
• W4377231507 hasLocation W43772315071 @default.
• W4377231507 hasLocation W43772315072 @default.
• W4377231507 hasOpenAccess W4377231507 @default.
• W4377231507 hasPrimaryLocation W43772315071 @default.
• W4377231507 hasRelatedWork W2016931788 @default.
• W4377231507 hasRelatedWork W2043640140 @default.
• W4377231507 hasRelatedWork W2122690624 @default.
• W4377231507 hasRelatedWork W2748952813 @default.
• W4377231507 hasRelatedWork W2899084033 @default.
• W4377231507 hasRelatedWork W4250204977 @default.
• W4377231507 hasRelatedWork W4252124899 @default.
• W4377231507 hasRelatedWork W4312676584 @default.
• W4377231507 hasRelatedWork W4361194010 @default.
• W4377231507 hasRelatedWork W4376862083 @default.
• W4377231507 hasVolume "72" @default.
• W4377231507 isParatext "false" @default.
• W4377231507 isRetracted "false" @default.
• W4377231507 workType "article" @default.