Matches in SemOpenAlex for { <https://semopenalex.org/work/W4328137372> ?p ?o ?g. }
Showing items 1 to 64 of
64
with 100 items per page.
- W4328137372 abstract "Feedforward deep neural networks (DNNs) commonly involve layer-wise linear operations and subsequent nonlinear operations, which are repeated through all layers. The nonlinear operations by nonlinear activations in each layer remarkably enhance the expressiveness of DNNs, resulting in the great success in a variety of application domains. Although the necessity of layer-wise nonlinear operations is agreed, the optimal nonlinearity for each layer in a given DNN is not clear. In this regard, we propose an easy-to-use method to layer-wise measure the optimal nonlinearity for a given DNN using its replica termed a linear-nonlinear network (LiNLNet). The key to the LiNLNet is the use of linear-nonlinear units (LiNLUs) whose degree of nonlinearity is parameterized by a trainable parameter p. The parameter p is shared among all LiNLUs in a given layer, thus indicating the layer-wise optimal nonlinearity. This method allows layer-level pruning such that the layers that do not require nonlinearity are merged into the subsequent layers, reducing computational complexity. For proofs of concept, we applied the proposed method to a MLP, AlexNet, VGG16, and ResNet18 on CIFAR-10 and ImageNet. The results commonly indicate the last hidden layer as a linear layer that may be merged into the output layer, reducing memory usage by 27% while maintaining the accuracy for LiNL-AlexNet on ImageNet." @default.
- W4328137372 created "2023-03-22" @default.
- W4328137372 creator A5036862125 @default.
- W4328137372 creator A5082971950 @default.
- W4328137372 date "2023-03-01" @default.
- W4328137372 modified "2023-10-05" @default.
- W4328137372 title "LiNLNet: Gauging required nonlinearity in deep neural networks" @default.
- W4328137372 cites W2117539524 @default.
- W4328137372 doi "https://doi.org/10.1063/5.0134713" @default.
- W4328137372 hasPublicationYear "2023" @default.
- W4328137372 type Work @default.
- W4328137372 citedByCount "0" @default.
- W4328137372 crossrefType "journal-article" @default.
- W4328137372 hasAuthorship W4328137372A5036862125 @default.
- W4328137372 hasAuthorship W4328137372A5082971950 @default.
- W4328137372 hasBestOaLocation W43281373721 @default.
- W4328137372 hasConcept C108010975 @default.
- W4328137372 hasConcept C11413529 @default.
- W4328137372 hasConcept C121332964 @default.
- W4328137372 hasConcept C154945302 @default.
- W4328137372 hasConcept C158622935 @default.
- W4328137372 hasConcept C159985019 @default.
- W4328137372 hasConcept C165464430 @default.
- W4328137372 hasConcept C192562407 @default.
- W4328137372 hasConcept C2779227376 @default.
- W4328137372 hasConcept C41008148 @default.
- W4328137372 hasConcept C50644808 @default.
- W4328137372 hasConcept C62520636 @default.
- W4328137372 hasConcept C6557445 @default.
- W4328137372 hasConcept C86803240 @default.
- W4328137372 hasConceptScore W4328137372C108010975 @default.
- W4328137372 hasConceptScore W4328137372C11413529 @default.
- W4328137372 hasConceptScore W4328137372C121332964 @default.
- W4328137372 hasConceptScore W4328137372C154945302 @default.
- W4328137372 hasConceptScore W4328137372C158622935 @default.
- W4328137372 hasConceptScore W4328137372C159985019 @default.
- W4328137372 hasConceptScore W4328137372C165464430 @default.
- W4328137372 hasConceptScore W4328137372C192562407 @default.
- W4328137372 hasConceptScore W4328137372C2779227376 @default.
- W4328137372 hasConceptScore W4328137372C41008148 @default.
- W4328137372 hasConceptScore W4328137372C50644808 @default.
- W4328137372 hasConceptScore W4328137372C62520636 @default.
- W4328137372 hasConceptScore W4328137372C6557445 @default.
- W4328137372 hasConceptScore W4328137372C86803240 @default.
- W4328137372 hasFunder F4320322120 @default.
- W4328137372 hasFunder F4320338298 @default.
- W4328137372 hasIssue "1" @default.
- W4328137372 hasLocation W43281373721 @default.
- W4328137372 hasOpenAccess W4328137372 @default.
- W4328137372 hasPrimaryLocation W43281373721 @default.
- W4328137372 hasRelatedWork W1973679146 @default.
- W4328137372 hasRelatedWork W2081650522 @default.
- W4328137372 hasRelatedWork W2146380059 @default.
- W4328137372 hasRelatedWork W2326122716 @default.
- W4328137372 hasRelatedWork W2357085366 @default.
- W4328137372 hasRelatedWork W2386387936 @default.
- W4328137372 hasRelatedWork W2514162834 @default.
- W4328137372 hasRelatedWork W2899217644 @default.
- W4328137372 hasRelatedWork W3047144510 @default.
- W4328137372 hasRelatedWork W3124308485 @default.
- W4328137372 hasVolume "1" @default.
- W4328137372 isParatext "false" @default.
- W4328137372 isRetracted "false" @default.
- W4328137372 workType "article" @default.