Matches in SemOpenAlex for { <https://semopenalex.org/work/W3211700711> ?p ?o ?g. }
Showing items 1 to 86 of
86
with 100 items per page.
- W3211700711 abstract "The demand for large-scale deep learning is increasing, and distributed training is the current mainstream solution. Ring AllReduce is widely used as a data parallel decentralized algorithm. However, in a heterogeneous environment, each worker calculates the same amount of data, so that there is a lot of waiting time loss among different workers, which makes the algorithm unable to adapt well to heterogeneous clusters. Resources are not used as they should be. In this paper, we design an implementation of static allocation algorithm. The dataset is artificially allocated to each worker, and samples are drawn proportionally for training, thereby speeding up the training speed of the network in a heterogeneous environment. We verify the convergence and influence on training speed of the network model under this algorithm on one machine with multi-card and multi-machine with multi-card. On this basis of feasibility, we propose a self-adaptive allocation algorithm that allows each machine to find the data it needs to adapt to the current environment. The self-adaptive allocation algorithm can reduce the training time by nearly one-third to half compared to the same proportional allocation.In order to better show the applicability of the algorithm in heterogeneous clusters, We replace a poorly performing worker with a good performing worker or add a poorly performing worker to the heterogeneous cluster. Experimental results show that training time will decrease as the overall performance improves. Therefore, it means that resources are fully used. Further, this algorithm is not only suitable for straggler problems, but also for most heterogeneous situations. It can be used as a plug-in for AllReduce and its variant algorithms." @default.
- W3211700711 created "2021-11-22" @default.
- W3211700711 creator A5000104685 @default.
- W3211700711 creator A5016458854 @default.
- W3211700711 creator A5068205989 @default.
- W3211700711 date "2021-11-16" @default.
- W3211700711 modified "2023-09-23" @default.
- W3211700711 title "Task allocation for decentralized training in heterogeneous environment" @default.
- W3211700711 cites W1686810756 @default.
- W3211700711 cites W2007339694 @default.
- W3211700711 cites W2039708501 @default.
- W3211700711 cites W2060393849 @default.
- W3211700711 cites W2117689653 @default.
- W3211700711 cites W2133693888 @default.
- W3211700711 cites W2182420680 @default.
- W3211700711 cites W2274287116 @default.
- W3211700711 cites W2336650964 @default.
- W3211700711 cites W2523435939 @default.
- W3211700711 cites W2612026221 @default.
- W3211700711 cites W2622263826 @default.
- W3211700711 cites W2741736887 @default.
- W3211700711 cites W2765170276 @default.
- W3211700711 cites W2787998955 @default.
- W3211700711 cites W2936343019 @default.
- W3211700711 cites W2972087877 @default.
- W3211700711 cites W2973606585 @default.
- W3211700711 cites W3118608800 @default.
- W3211700711 doi "https://doi.org/10.48550/arxiv.2111.08272" @default.
- W3211700711 hasPublicationYear "2021" @default.
- W3211700711 type Work @default.
- W3211700711 sameAs 3211700711 @default.
- W3211700711 citedByCount "0" @default.
- W3211700711 crossrefType "posted-content" @default.
- W3211700711 hasAuthorship W3211700711A5000104685 @default.
- W3211700711 hasAuthorship W3211700711A5016458854 @default.
- W3211700711 hasAuthorship W3211700711A5068205989 @default.
- W3211700711 hasBestOaLocation W32117007111 @default.
- W3211700711 hasConcept C108037233 @default.
- W3211700711 hasConcept C120314980 @default.
- W3211700711 hasConcept C121332964 @default.
- W3211700711 hasConcept C127413603 @default.
- W3211700711 hasConcept C153294291 @default.
- W3211700711 hasConcept C154945302 @default.
- W3211700711 hasConcept C158207573 @default.
- W3211700711 hasConcept C162324750 @default.
- W3211700711 hasConcept C201995342 @default.
- W3211700711 hasConcept C2777211547 @default.
- W3211700711 hasConcept C2777303404 @default.
- W3211700711 hasConcept C2780451532 @default.
- W3211700711 hasConcept C41008148 @default.
- W3211700711 hasConcept C50522688 @default.
- W3211700711 hasConcept C555944384 @default.
- W3211700711 hasConcept C76155785 @default.
- W3211700711 hasConceptScore W3211700711C108037233 @default.
- W3211700711 hasConceptScore W3211700711C120314980 @default.
- W3211700711 hasConceptScore W3211700711C121332964 @default.
- W3211700711 hasConceptScore W3211700711C127413603 @default.
- W3211700711 hasConceptScore W3211700711C153294291 @default.
- W3211700711 hasConceptScore W3211700711C154945302 @default.
- W3211700711 hasConceptScore W3211700711C158207573 @default.
- W3211700711 hasConceptScore W3211700711C162324750 @default.
- W3211700711 hasConceptScore W3211700711C201995342 @default.
- W3211700711 hasConceptScore W3211700711C2777211547 @default.
- W3211700711 hasConceptScore W3211700711C2777303404 @default.
- W3211700711 hasConceptScore W3211700711C2780451532 @default.
- W3211700711 hasConceptScore W3211700711C41008148 @default.
- W3211700711 hasConceptScore W3211700711C50522688 @default.
- W3211700711 hasConceptScore W3211700711C555944384 @default.
- W3211700711 hasConceptScore W3211700711C76155785 @default.
- W3211700711 hasLocation W32117007111 @default.
- W3211700711 hasOpenAccess W3211700711 @default.
- W3211700711 hasPrimaryLocation W32117007111 @default.
- W3211700711 hasRelatedWork W1485627940 @default.
- W3211700711 hasRelatedWork W1587227328 @default.
- W3211700711 hasRelatedWork W1596201972 @default.
- W3211700711 hasRelatedWork W1596615495 @default.
- W3211700711 hasRelatedWork W1764817458 @default.
- W3211700711 hasRelatedWork W1967954938 @default.
- W3211700711 hasRelatedWork W2063279839 @default.
- W3211700711 hasRelatedWork W2081647779 @default.
- W3211700711 hasRelatedWork W2160425906 @default.
- W3211700711 hasRelatedWork W2998813341 @default.
- W3211700711 isParatext "false" @default.
- W3211700711 isRetracted "false" @default.
- W3211700711 magId "3211700711" @default.
- W3211700711 workType "article" @default.