Matches in SemOpenAlex for { <https://semopenalex.org/work/W3046173836> ?p ?o ?g. }
- W3046173836 abstract "Physics-informed neural networks (PINNs) have lately received great attention thanks to their flexibility in tackling a wide range of forward and inverse problems involving partial differential equations. However, despite their noticeable empirical success, little is known about how such constrained neural networks behave during their training via gradient descent. More importantly, even less is known about why such models sometimes fail to train at all. In this work, we aim to investigate these questions through the lens of the Neural Tangent Kernel (NTK); a kernel that captures the behavior of fully-connected neural networks in the infinite width limit during training via gradient descent. Specifically, we derive the NTK of PINNs and prove that, under appropriate conditions, it converges to a deterministic kernel that stays constant during training in the infinite-width limit. This allows us to analyze the training dynamics of PINNs through the lens of their limiting NTK and find a remarkable discrepancy in the convergence rate of the different loss components contributing to the total training error. To address this fundamental pathology, we propose a novel gradient descent algorithm that utilizes the eigenvalues of the NTK to adaptively calibrate the convergence rate of the total training error. Finally, we perform a series of numerical experiments to verify the correctness of our theory and the practical effectiveness of the proposed algorithms. The data and code accompanying this manuscript are publicly available at url{https://github.com/PredictiveIntelligenceLab/PINNsNTK}." @default.
- W3046173836 created "2020-08-03" @default.
- W3046173836 creator A5002562845 @default.
- W3046173836 creator A5011622029 @default.
- W3046173836 creator A5042155480 @default.
- W3046173836 date "2020-07-28" @default.
- W3046173836 modified "2023-09-26" @default.
- W3046173836 title "When and why PINNs fail to train: A neural tangent kernel perspective" @default.
- W3046173836 cites W1522301498 @default.
- W3046173836 cites W1533861849 @default.
- W3046173836 cites W1985583842 @default.
- W3046173836 cites W2749028154 @default.
- W3046173836 cites W2766678531 @default.
- W3046173836 cites W2785626633 @default.
- W3046173836 cites W2803629276 @default.
- W3046173836 cites W2809090039 @default.
- W3046173836 cites W2885696722 @default.
- W3046173836 cites W2897097528 @default.
- W3046173836 cites W2899283552 @default.
- W3046173836 cites W2900369848 @default.
- W3046173836 cites W2903926864 @default.
- W3046173836 cites W2908541468 @default.
- W3046173836 cites W2913473169 @default.
- W3046173836 cites W2914483840 @default.
- W3046173836 cites W2947859416 @default.
- W3046173836 cites W2948230027 @default.
- W3046173836 cites W2957969499 @default.
- W3046173836 cites W2963112935 @default.
- W3046173836 cites W2969381807 @default.
- W3046173836 cites W2971043187 @default.
- W3046173836 cites W2971095480 @default.
- W3046173836 cites W2973886134 @default.
- W3046173836 cites W2990386917 @default.
- W3046173836 cites W2993063996 @default.
- W3046173836 cites W2998004632 @default.
- W3046173836 cites W2999467285 @default.
- W3046173836 cites W3000676377 @default.
- W3046173836 cites W3008118574 @default.
- W3046173836 cites W3010839048 @default.
- W3046173836 cites W3010849941 @default.
- W3046173836 cites W3011147100 @default.
- W3046173836 cites W3011817850 @default.
- W3046173836 cites W3014468003 @default.
- W3046173836 cites W3014618972 @default.
- W3046173836 cites W3016239339 @default.
- W3046173836 cites W3035493266 @default.
- W3046173836 cites W3036843665 @default.
- W3046173836 cites W3047245470 @default.
- W3046173836 cites W3099057226 @default.
- W3046173836 cites W3191617243 @default.
- W3046173836 cites W14377099 @default.
- W3046173836 doi "https://doi.org/10.48550/arxiv.2007.14527" @default.
- W3046173836 hasPublicationYear "2020" @default.
- W3046173836 type Work @default.
- W3046173836 sameAs 3046173836 @default.
- W3046173836 citedByCount "33" @default.
- W3046173836 countsByYear W30461738362020 @default.
- W3046173836 countsByYear W30461738362021 @default.
- W3046173836 countsByYear W30461738362023 @default.
- W3046173836 crossrefType "posted-content" @default.
- W3046173836 hasAuthorship W3046173836A5002562845 @default.
- W3046173836 hasAuthorship W3046173836A5011622029 @default.
- W3046173836 hasAuthorship W3046173836A5042155480 @default.
- W3046173836 hasBestOaLocation W30461738361 @default.
- W3046173836 hasConcept C11413529 @default.
- W3046173836 hasConcept C118615104 @default.
- W3046173836 hasConcept C126255220 @default.
- W3046173836 hasConcept C134306372 @default.
- W3046173836 hasConcept C138187205 @default.
- W3046173836 hasConcept C151201525 @default.
- W3046173836 hasConcept C153258448 @default.
- W3046173836 hasConcept C154945302 @default.
- W3046173836 hasConcept C159985019 @default.
- W3046173836 hasConcept C162324750 @default.
- W3046173836 hasConcept C192562407 @default.
- W3046173836 hasConcept C204323151 @default.
- W3046173836 hasConcept C207467116 @default.
- W3046173836 hasConcept C2524010 @default.
- W3046173836 hasConcept C2777303404 @default.
- W3046173836 hasConcept C28826006 @default.
- W3046173836 hasConcept C33923547 @default.
- W3046173836 hasConcept C41008148 @default.
- W3046173836 hasConcept C50522688 @default.
- W3046173836 hasConcept C50644808 @default.
- W3046173836 hasConcept C74193536 @default.
- W3046173836 hasConceptScore W3046173836C11413529 @default.
- W3046173836 hasConceptScore W3046173836C118615104 @default.
- W3046173836 hasConceptScore W3046173836C126255220 @default.
- W3046173836 hasConceptScore W3046173836C134306372 @default.
- W3046173836 hasConceptScore W3046173836C138187205 @default.
- W3046173836 hasConceptScore W3046173836C151201525 @default.
- W3046173836 hasConceptScore W3046173836C153258448 @default.
- W3046173836 hasConceptScore W3046173836C154945302 @default.
- W3046173836 hasConceptScore W3046173836C159985019 @default.
- W3046173836 hasConceptScore W3046173836C162324750 @default.
- W3046173836 hasConceptScore W3046173836C192562407 @default.
- W3046173836 hasConceptScore W3046173836C204323151 @default.
- W3046173836 hasConceptScore W3046173836C207467116 @default.
- W3046173836 hasConceptScore W3046173836C2524010 @default.
- W3046173836 hasConceptScore W3046173836C2777303404 @default.