FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2978889483> ?p ?o ?g. }

• W2978889483 abstract "Distributed machine learning (ML) can bring more computational resources to bear than single-machine learning, thus enabling reductions in training time. Distributed learning partitions models and data over many machines, allowing model and dataset sizes beyond the available compute power and memory of a single machine. In practice though, distributed ML is challenging when distribution is mandatory, rather than chosen by the practitioner. In such scenarios, data could unavoidably be separated among workers due to limited memory capacity per worker or even because of data privacy issues. There, existing distributed methods will utterly fail due to dominant transfer costs across workers, or do not even apply. We propose a new approach to distributed fully connected neural network learning, called independent subnet training (IST), to handle these cases. In IST, the original network is decomposed into a set of narrow subnetworks with the same depth. These subnetworks are then trained locally before parameters are exchanged to produce new subnets and the training cycle repeats. Such a naturally model parallel approach limits memory usage by storing only a portion of network parameters on each device. Additionally, no requirements exist for sharing data between workers (i.e., subnet training is local and independent) and communication volume and frequency are reduced by decomposing the original network into independent subnets. These properties of IST can cope with issues due to distributed data, slow interconnects, or limited device memory, making IST a suitable approach for cases of mandatory distribution. We show experimentally that IST results in training times that are much lower than common distributed learning approaches." @default.
• W2978889483 created "2019-10-10" @default.
• W2978889483 creator A5002684888 @default.
• W2978889483 creator A5024280658 @default.
• W2978889483 creator A5027798417 @default.
• W2978889483 date "2019-10-04" @default.
• W2978889483 modified "2023-10-16" @default.
• W2978889483 title "Distributed Learning of Deep Neural Networks using Independent Subnet Training" @default.
• W2978889483 cites W1442374986 @default.
• W2978889483 cites W1522301498 @default.
• W2978889483 cites W1598866093 @default.
• W2978889483 cites W1686810756 @default.
• W2978889483 cites W1788418780 @default.
• W2978889483 cites W1836465849 @default.
• W2978889483 cites W2042465463 @default.
• W2978889483 cites W2060279602 @default.
• W2978889483 cites W2083842231 @default.
• W2978889483 cites W2087402357 @default.
• W2978889483 cites W2095705004 @default.
• W2978889483 cites W2103869314 @default.
• W2978889483 cites W2112972442 @default.
• W2978889483 cites W2117499659 @default.
• W2978889483 cites W2120432001 @default.
• W2978889483 cites W2138243089 @default.
• W2978889483 cites W2142354294 @default.
• W2978889483 cites W2144839971 @default.
• W2978889483 cites W2146502635 @default.
• W2978889483 cites W2163605009 @default.
• W2978889483 cites W2166706236 @default.
• W2978889483 cites W2168231600 @default.
• W2978889483 cites W2177838837 @default.
• W2978889483 cites W2194775991 @default.
• W2978889483 cites W2336650964 @default.
• W2978889483 cites W2402144811 @default.
• W2978889483 cites W2407022425 @default.
• W2978889483 cites W2436522418 @default.
• W2978889483 cites W2460815216 @default.
• W2978889483 cites W2514742857 @default.
• W2978889483 cites W2523246573 @default.
• W2978889483 cites W2524428287 @default.
• W2978889483 cites W2530417694 @default.
• W2978889483 cites W2605372163 @default.
• W2978889483 cites W2615262195 @default.
• W2978889483 cites W2622263826 @default.
• W2978889483 cites W2747280560 @default.
• W2978889483 cites W2749988060 @default.
• W2978889483 cites W2757910899 @default.
• W2978889483 cites W2766164908 @default.
• W2978889483 cites W2769644379 @default.
• W2978889483 cites W2770102447 @default.
• W2978889483 cites W2787998955 @default.
• W2978889483 cites W2788463493 @default.
• W2978889483 cites W2793117408 @default.
• W2978889483 cites W2797583228 @default.
• W2978889483 cites W2888206291 @default.
• W2978889483 cites W2899771611 @default.
• W2978889483 cites W2902280036 @default.
• W2978889483 cites W2903771406 @default.
• W2978889483 cites W2913310393 @default.
• W2978889483 cites W2913545852 @default.
• W2978889483 cites W2921416272 @default.
• W2978889483 cites W2926101387 @default.
• W2978889483 cites W2930786691 @default.
• W2978889483 cites W2943305848 @default.
• W2978889483 cites W2945785363 @default.
• W2978889483 cites W2951781666 @default.
• W2978889483 cites W2952388062 @default.
• W2978889483 cites W2963114950 @default.
• W2978889483 cites W2963248893 @default.
• W2978889483 cites W2963307318 @default.
• W2978889483 cites W2963374099 @default.
• W2978889483 cites W2963433607 @default.
• W2978889483 cites W2963959597 @default.
• W2978889483 cites W2964004663 @default.
• W2978889483 cites W2964125128 @default.
• W2978889483 cites W2970289928 @default.
• W2978889483 cites W2970421227 @default.
• W2978889483 cites W2972453513 @default.
• W2978889483 cites W3029645440 @default.
• W2978889483 cites W3101036738 @default.
• W2978889483 cites W6908809 @default.
• W2978889483 cites W3142400242 @default.
• W2978889483 doi "https://doi.org/10.48550/arxiv.1910.02120" @default.
• W2978889483 hasPublicationYear "2019" @default.
• W2978889483 type Work @default.
• W2978889483 sameAs 2978889483 @default.
• W2978889483 citedByCount "5" @default.
• W2978889483 countsByYear W29788894832019 @default.
• W2978889483 countsByYear W29788894832020 @default.
• W2978889483 countsByYear W29788894832021 @default.
• W2978889483 crossrefType "posted-content" @default.
• W2978889483 hasAuthorship W2978889483A5002684888 @default.
• W2978889483 hasAuthorship W2978889483A5024280658 @default.
• W2978889483 hasAuthorship W2978889483A5027798417 @default.
• W2978889483 hasBestOaLocation W29788894831 @default.
• W2978889483 hasConcept C119857082 @default.
• W2978889483 hasConcept C120314980 @default.
• W2978889483 hasConcept C121332964 @default.
• W2978889483 hasConcept C150899416 @default.
• W2978889483 hasConcept C154945302 @default.

• next