Matches in SemOpenAlex for { <https://semopenalex.org/work/W4287183748> ?p ?o ?g. }
Showing items 1 to 71 of
71
with 100 items per page.
- W4287183748 abstract "Democratization of machine learning requires architectures that automatically adapt to new problems. Neural Differential Equations (NDEs) have emerged as a popular modeling framework by removing the need for ML practitioners to choose the number of layers in a recurrent model. While we can control the computational cost by choosing the number of layers in standard architectures, in NDEs the number of neural network evaluations for a forward pass can depend on the number of steps of the adaptive ODE solver. But, can we force the NDE to learn the version with the least steps while not increasing the training cost? Current strategies to overcome slow prediction require high order automatic differentiation, leading to significantly higher training time. We describe a novel regularization method that uses the internal cost heuristics of adaptive differential equation solvers combined with discrete adjoint sensitivities to guide the training process towards learning NDEs that are easier to solve. This approach opens up the blackbox numerical analysis behind the differential equation solver's algorithm and directly uses its local error estimates and stiffness heuristics as cheap and accurate cost estimates. We incorporate our method without any change in the underlying NDE framework and show that our method extends beyond Ordinary Differential Equations to accommodate Neural Stochastic Differential Equations. We demonstrate how our approach can halve the prediction time and, unlike other methods which can increase the training time by an order of magnitude, we demonstrate similar reduction in training times. Together this showcases how the knowledge embedded within state-of-the-art equation solvers can be used to enhance machine learning." @default.
- W4287183748 created "2022-07-25" @default.
- W4287183748 creator A5023814642 @default.
- W4287183748 creator A5043195923 @default.
- W4287183748 creator A5072818182 @default.
- W4287183748 creator A5084500492 @default.
- W4287183748 date "2021-05-09" @default.
- W4287183748 modified "2023-10-18" @default.
- W4287183748 title "Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristics" @default.
- W4287183748 doi "https://doi.org/10.48550/arxiv.2105.03918" @default.
- W4287183748 hasPublicationYear "2021" @default.
- W4287183748 type Work @default.
- W4287183748 citedByCount "0" @default.
- W4287183748 crossrefType "posted-content" @default.
- W4287183748 hasAuthorship W4287183748A5023814642 @default.
- W4287183748 hasAuthorship W4287183748A5043195923 @default.
- W4287183748 hasAuthorship W4287183748A5072818182 @default.
- W4287183748 hasAuthorship W4287183748A5084500492 @default.
- W4287183748 hasBestOaLocation W42871837481 @default.
- W4287183748 hasConcept C111919701 @default.
- W4287183748 hasConcept C11413529 @default.
- W4287183748 hasConcept C126255220 @default.
- W4287183748 hasConcept C127705205 @default.
- W4287183748 hasConcept C134306372 @default.
- W4287183748 hasConcept C154945302 @default.
- W4287183748 hasConcept C186219872 @default.
- W4287183748 hasConcept C199360897 @default.
- W4287183748 hasConcept C2776135515 @default.
- W4287183748 hasConcept C2778770139 @default.
- W4287183748 hasConcept C28826006 @default.
- W4287183748 hasConcept C33923547 @default.
- W4287183748 hasConcept C34862557 @default.
- W4287183748 hasConcept C41008148 @default.
- W4287183748 hasConcept C50644808 @default.
- W4287183748 hasConcept C51544822 @default.
- W4287183748 hasConcept C51955184 @default.
- W4287183748 hasConcept C78045399 @default.
- W4287183748 hasConceptScore W4287183748C111919701 @default.
- W4287183748 hasConceptScore W4287183748C11413529 @default.
- W4287183748 hasConceptScore W4287183748C126255220 @default.
- W4287183748 hasConceptScore W4287183748C127705205 @default.
- W4287183748 hasConceptScore W4287183748C134306372 @default.
- W4287183748 hasConceptScore W4287183748C154945302 @default.
- W4287183748 hasConceptScore W4287183748C186219872 @default.
- W4287183748 hasConceptScore W4287183748C199360897 @default.
- W4287183748 hasConceptScore W4287183748C2776135515 @default.
- W4287183748 hasConceptScore W4287183748C2778770139 @default.
- W4287183748 hasConceptScore W4287183748C28826006 @default.
- W4287183748 hasConceptScore W4287183748C33923547 @default.
- W4287183748 hasConceptScore W4287183748C34862557 @default.
- W4287183748 hasConceptScore W4287183748C41008148 @default.
- W4287183748 hasConceptScore W4287183748C50644808 @default.
- W4287183748 hasConceptScore W4287183748C51544822 @default.
- W4287183748 hasConceptScore W4287183748C51955184 @default.
- W4287183748 hasConceptScore W4287183748C78045399 @default.
- W4287183748 hasLocation W42871837481 @default.
- W4287183748 hasOpenAccess W4287183748 @default.
- W4287183748 hasPrimaryLocation W42871837481 @default.
- W4287183748 hasRelatedWork W11033548 @default.
- W4287183748 hasRelatedWork W12166850 @default.
- W4287183748 hasRelatedWork W12170026 @default.
- W4287183748 hasRelatedWork W1407330 @default.
- W4287183748 hasRelatedWork W2203340 @default.
- W4287183748 hasRelatedWork W4919037 @default.
- W4287183748 hasRelatedWork W6083205 @default.
- W4287183748 hasRelatedWork W6182124 @default.
- W4287183748 hasRelatedWork W8049136 @default.
- W4287183748 hasRelatedWork W8920422 @default.
- W4287183748 isParatext "false" @default.
- W4287183748 isRetracted "false" @default.
- W4287183748 workType "article" @default.