Matches in SemOpenAlex for { <https://semopenalex.org/work/W4285200511> ?p ?o ?g. }
Showing items 1 to 81 of
81
with 100 items per page.
- W4285200511 endingPage "1314" @default.
- W4285200511 startingPage "1302" @default.
- W4285200511 abstract "Fused Multiply-Add (FMA) functional units constitute a fundamental hardware component to train Deep Neural Networks (DNNs). Its silicon area grows quadratically with the mantissa bit count of the computer number format, which has motivated the adoption of the BrainFloat16 format (BF16). BF16 features 1 sign, 8 exponent and 7 explicit mantissa bits. Some approaches to train DNNs achieve significant performance benefits by using the BF16 format. However, these approaches must combine BF16 with the standard IEEE 754 Floating-Point 32-bit (FP32) format to achieve state-of-the-art training accuracy, which limits the impact of adopting BF16. This article proposes the first approach able to train complex DNNs entirely using the BF16 format. We propose a new class of FMA operators, <inline-formula><tex-math notation=LaTeX>$mathrm{FMA}^{mathrm {bf}16}_{mathrm{n}_mathrm{m}}$</tex-math></inline-formula> , that entirely rely on BF16 FMA hardware instructions and deliver the same accuracy as FP32. <inline-formula><tex-math notation=LaTeX>$mathrm{FMA}^{mathrm {bf}16}_{mathrm{n}_mathrm{m}}$</tex-math></inline-formula> operators achieve performance improvements within the 1.28-1.35× range on ResNet101 with respect to FP32. <inline-formula><tex-math notation=LaTeX>$mathrm{FMA}^{mathrm {bf}16}_{mathrm{n}_mathrm{m}}$</tex-math></inline-formula> enables training complex DNNs on simple low-end hardware devices without requiring expensive FP32 FMA functional units." @default.
- W4285200511 created "2022-07-14" @default.
- W4285200511 creator A5019142601 @default.
- W4285200511 creator A5027015251 @default.
- W4285200511 creator A5049697257 @default.
- W4285200511 creator A5054163015 @default.
- W4285200511 creator A5077760205 @default.
- W4285200511 date "2022-07-01" @default.
- W4285200511 modified "2023-10-16" @default.
- W4285200511 title "A BF16 FMA is All You Need for DNN Training" @default.
- W4285200511 cites W2013278460 @default.
- W4285200511 cites W2134633067 @default.
- W4285200511 cites W2159211021 @default.
- W4285200511 cites W2194775991 @default.
- W4285200511 cites W2963163009 @default.
- W4285200511 cites W2963909453 @default.
- W4285200511 cites W2967088612 @default.
- W4285200511 cites W2981980078 @default.
- W4285200511 cites W2999599167 @default.
- W4285200511 cites W3100278010 @default.
- W4285200511 cites W3202633081 @default.
- W4285200511 doi "https://doi.org/10.1109/tetc.2022.3187770" @default.
- W4285200511 hasPublicationYear "2022" @default.
- W4285200511 type Work @default.
- W4285200511 citedByCount "4" @default.
- W4285200511 countsByYear W42852005112022 @default.
- W4285200511 countsByYear W42852005112023 @default.
- W4285200511 crossrefType "journal-article" @default.
- W4285200511 hasAuthorship W4285200511A5019142601 @default.
- W4285200511 hasAuthorship W4285200511A5027015251 @default.
- W4285200511 hasAuthorship W4285200511A5049697257 @default.
- W4285200511 hasAuthorship W4285200511A5054163015 @default.
- W4285200511 hasAuthorship W4285200511A5077760205 @default.
- W4285200511 hasBestOaLocation W42852005112 @default.
- W4285200511 hasConcept C118615104 @default.
- W4285200511 hasConcept C134306372 @default.
- W4285200511 hasConcept C138885662 @default.
- W4285200511 hasConcept C139676723 @default.
- W4285200511 hasConcept C159985019 @default.
- W4285200511 hasConcept C192562407 @default.
- W4285200511 hasConcept C204323151 @default.
- W4285200511 hasConcept C2780388253 @default.
- W4285200511 hasConcept C33923547 @default.
- W4285200511 hasConcept C41008148 @default.
- W4285200511 hasConcept C41895202 @default.
- W4285200511 hasConcept C45357846 @default.
- W4285200511 hasConcept C94375191 @default.
- W4285200511 hasConceptScore W4285200511C118615104 @default.
- W4285200511 hasConceptScore W4285200511C134306372 @default.
- W4285200511 hasConceptScore W4285200511C138885662 @default.
- W4285200511 hasConceptScore W4285200511C139676723 @default.
- W4285200511 hasConceptScore W4285200511C159985019 @default.
- W4285200511 hasConceptScore W4285200511C192562407 @default.
- W4285200511 hasConceptScore W4285200511C204323151 @default.
- W4285200511 hasConceptScore W4285200511C2780388253 @default.
- W4285200511 hasConceptScore W4285200511C33923547 @default.
- W4285200511 hasConceptScore W4285200511C41008148 @default.
- W4285200511 hasConceptScore W4285200511C41895202 @default.
- W4285200511 hasConceptScore W4285200511C45357846 @default.
- W4285200511 hasConceptScore W4285200511C94375191 @default.
- W4285200511 hasIssue "3" @default.
- W4285200511 hasLocation W42852005111 @default.
- W4285200511 hasLocation W42852005112 @default.
- W4285200511 hasOpenAccess W4285200511 @default.
- W4285200511 hasPrimaryLocation W42852005111 @default.
- W4285200511 hasRelatedWork W1534043652 @default.
- W4285200511 hasRelatedWork W2052747888 @default.
- W4285200511 hasRelatedWork W2066401278 @default.
- W4285200511 hasRelatedWork W2074838321 @default.
- W4285200511 hasRelatedWork W2085861978 @default.
- W4285200511 hasRelatedWork W2096400301 @default.
- W4285200511 hasRelatedWork W2338700700 @default.
- W4285200511 hasRelatedWork W2900525721 @default.
- W4285200511 hasRelatedWork W3176160629 @default.
- W4285200511 hasRelatedWork W2464545161 @default.
- W4285200511 hasVolume "10" @default.
- W4285200511 isParatext "false" @default.
- W4285200511 isRetracted "false" @default.
- W4285200511 workType "article" @default.