FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W4281569193> ?p ?o ?g. }

• W4281569193 endingPage "109058" @default.
• W4281569193 startingPage "109058" @default.
• W4281569193 abstract "The learning rate is the most crucial hyper-parameter of a neural network that has a significant impact on its performance. In this article, a novel learning rate setting idea termed randomness distribution learning rate (RDLR) is presented to regulate the learning rate value. The proposed RDLR shifts the learning rate from deterministic to random and sets the value based on the state of the network. The RDLR uses the distance between neurons rather than the covariance matrix to get the redundancy of the network, as well as the Monte Carlo method, and to simplify the neuron to a point to reduce calculation costs. The proposed algorithms do not regulate the learning rate value of each epoch but rather the mathematical expectation and distribution of the learning rate during the training process. The neural network can jump out of the local minimum or unstable area using our algorithms and obtain the minimum point of the area in gradient space. The RDLR algorithms reduce the impact of tiny changes in learning rate value and streamline the tuning process of neural networks. The RDLR saves calculation costs and can work independently or cooperate with the traditional algorithms. In conjunction with traditional learning rate algorithms, the RDLR can set the same learning rate strategy for all layers in a neural network or keep the same mathematical expectation of the learning rate of each layer while adjusting their impulse. The experiments show that the RDLR can improve the performance of a neural network while keeping other hyper-parameters not changed. It is a novel method for adjusting the training process by dynamically changing the random distribution of the learning rate. Our algorithm can monitor the state of the neural network and keep injecting randomness into the neural network training based on the redundancy of the neurons. Furthermore, our algorithm does not require any additional hyper-parameters. The experiments show that our RDLR can improve the performance of multiple structure neural networks in various tasks when applied to a variety of loss functions and data augment methods. • A novel learning rate algorithm based on the random distribution. It adjusts the mathematical expectation and learning rate distribution, separately. • A novel idea to adjust the learning rate based on the current state of the neurons. If the gradient descent increases the redundancy of the neurons too much, the RDLR will try to jump out of the current state. • A calculation cost saved algorithm. Our algorithm uses the Monte Carlo method and dimensional reduction algorithm to save the calculation." @default.
• W4281569193 created "2022-05-27" @default.
• W4281569193 creator A5019160495 @default.
• W4281569193 creator A5043378483 @default.
• W4281569193 creator A5067200719 @default.
• W4281569193 date "2022-07-01" @default.
• W4281569193 modified "2023-10-06" @default.
• W4281569193 title "Dynamic random distribution learning rate for neural networks training" @default.
• W4281569193 cites W2064675550 @default.
• W4281569193 cites W2074008662 @default.
• W4281569193 cites W2194775991 @default.
• W4281569193 cites W2265846598 @default.
• W4281569193 cites W2744616361 @default.
• W4281569193 cites W2963446712 @default.
• W4281569193 cites W2963694768 @default.
• W4281569193 cites W3036643472 @default.
• W4281569193 cites W3105286923 @default.
• W4281569193 cites W3109196171 @default.
• W4281569193 cites W3110713756 @default.
• W4281569193 cites W3158302821 @default.
• W4281569193 cites W3198908542 @default.
• W4281569193 cites W3213480973 @default.
• W4281569193 cites W3215594189 @default.
• W4281569193 cites W4210355273 @default.
• W4281569193 doi "https://doi.org/10.1016/j.asoc.2022.109058" @default.
• W4281569193 hasPublicationYear "2022" @default.
• W4281569193 type Work @default.
• W4281569193 citedByCount "3" @default.
• W4281569193 countsByYear W42815691932023 @default.
• W4281569193 crossrefType "journal-article" @default.
• W4281569193 hasAuthorship W4281569193A5019160495 @default.
• W4281569193 hasAuthorship W4281569193A5043378483 @default.
• W4281569193 hasAuthorship W4281569193A5067200719 @default.
• W4281569193 hasConcept C110121322 @default.
• W4281569193 hasConcept C119857082 @default.
• W4281569193 hasConcept C121332964 @default.
• W4281569193 hasConcept C134306372 @default.
• W4281569193 hasConcept C153294291 @default.
• W4281569193 hasConcept C154945302 @default.
• W4281569193 hasConcept C2777211547 @default.
• W4281569193 hasConcept C33923547 @default.
• W4281569193 hasConcept C41008148 @default.
• W4281569193 hasConcept C50644808 @default.
• W4281569193 hasConceptScore W4281569193C110121322 @default.
• W4281569193 hasConceptScore W4281569193C119857082 @default.
• W4281569193 hasConceptScore W4281569193C121332964 @default.
• W4281569193 hasConceptScore W4281569193C134306372 @default.
• W4281569193 hasConceptScore W4281569193C153294291 @default.
• W4281569193 hasConceptScore W4281569193C154945302 @default.
• W4281569193 hasConceptScore W4281569193C2777211547 @default.
• W4281569193 hasConceptScore W4281569193C33923547 @default.
• W4281569193 hasConceptScore W4281569193C41008148 @default.
• W4281569193 hasConceptScore W4281569193C50644808 @default.
• W4281569193 hasLocation W42815691931 @default.
• W4281569193 hasOpenAccess W4281569193 @default.
• W4281569193 hasPrimaryLocation W42815691931 @default.
• W4281569193 hasRelatedWork W2386387936 @default.
• W4281569193 hasRelatedWork W2961085424 @default.
• W4281569193 hasRelatedWork W3046775127 @default.
• W4281569193 hasRelatedWork W3170094116 @default.
• W4281569193 hasRelatedWork W4205958290 @default.
• W4281569193 hasRelatedWork W4285260836 @default.
• W4281569193 hasRelatedWork W4286629047 @default.
• W4281569193 hasRelatedWork W4306321456 @default.
• W4281569193 hasRelatedWork W4306674287 @default.
• W4281569193 hasRelatedWork W4224009465 @default.
• W4281569193 hasVolume "124" @default.
• W4281569193 isParatext "false" @default.
• W4281569193 isRetracted "false" @default.
• W4281569193 workType "article" @default.