FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2962376003> ?p ?o ?g. }

• W2962376003 abstract "Recent results in the literature indicate that a residual network (ResNet) composed of a single residual block outperforms linear predictors, in the sense that all local minima in its optimization landscape are at least as good as the best linear predictor. However, these results are limited to a single residual block (i.e., shallow ResNets), instead of the deep ResNets composed of multiple residual blocks. We take a step towards extending this result to deep ResNets. We start by two motivating examples. First, we show that there exist datasets for which all local minima of a fully-connected ReLU network are no better than the best linear predictor, whereas a ResNet has strictly better local minima. Second, we show that even at the global minimum, the representation obtained from the residual block outputs of a 2-block ResNet do not necessarily improve monotonically over subsequent blocks, which highlights a fundamental difficulty in analyzing deep ResNets. Our main theorem on deep ResNets shows under simple geometric conditions that, any critical point in the optimization landscape is either (i) at least as good as the best linear predictor; or (ii) the Hessian at this critical point has a strictly negative eigenvalue. Notably, our theorem shows that a chain of multiple skip-connections can improve the optimization landscape, whereas existing results study direct skip-connections to the last hidden layer or output layer. Finally, we complement our results by showing benign properties of the near-identity regions of deep ResNets, showing depth-independent upper bounds for the risk attained at critical points as well as the Rademacher complexity." @default.
• W2962376003 created "2019-07-23" @default.
• W2962376003 creator A5045562332 @default.
• W2962376003 creator A5058767558 @default.
• W2962376003 creator A5078288116 @default.
• W2962376003 date "2019-07-09" @default.
• W2962376003 modified "2023-09-27" @default.
• W2962376003 title "Are deep ResNets provably better than linear predictors" @default.
• W2962376003 cites W2078626246 @default.
• W2962376003 cites W2131703182 @default.
• W2962376003 cites W2194775991 @default.
• W2962376003 cites W2302255633 @default.
• W2962376003 cites W2399994860 @default.
• W2962376003 cites W2596692027 @default.
• W2962376003 cites W2709553318 @default.
• W2962376003 cites W2765428107 @default.
• W2962376003 cites W2777138330 @default.
• W2962376003 cites W2777256551 @default.
• W2962376003 cites W2780951787 @default.
• W2962376003 cites W2786983307 @default.
• W2962376003 cites W2797262859 @default.
• W2962376003 cites W2797993462 @default.
• W2962376003 cites W2803955134 @default.
• W2962376003 cites W2810898455 @default.
• W2962376003 cites W2891942459 @default.
• W2962376003 cites W2892675615 @default.
• W2962376003 cites W2892723008 @default.
• W2962376003 cites W2949978219 @default.
• W2962376003 cites W2952574409 @default.
• W2962376003 cites W2962761235 @default.
• W2962376003 cites W2962810483 @default.
• W2962376003 cites W2962857907 @default.
• W2962376003 cites W2962933129 @default.
• W2962376003 cites W2963336603 @default.
• W2962376003 cites W2963427613 @default.
• W2962376003 cites W2963446085 @default.
• W2962376003 cites W2963651774 @default.
• W2962376003 cites W2963739978 @default.
• W2962376003 cites W2964072429 @default.
• W2962376003 cites W2964232029 @default.
• W2962376003 hasPublicationYear "2019" @default.
• W2962376003 type Work @default.
• W2962376003 sameAs 2962376003 @default.
• W2962376003 citedByCount "1" @default.
• W2962376003 countsByYear W29623760032019 @default.
• W2962376003 crossrefType "posted-content" @default.
• W2962376003 hasAuthorship W2962376003A5045562332 @default.
• W2962376003 hasAuthorship W2962376003A5058767558 @default.
• W2962376003 hasAuthorship W2962376003A5078288116 @default.
• W2962376003 hasConcept C108583219 @default.
• W2962376003 hasConcept C111472728 @default.
• W2962376003 hasConcept C11413529 @default.
• W2962376003 hasConcept C114614502 @default.
• W2962376003 hasConcept C134306372 @default.
• W2962376003 hasConcept C138885662 @default.
• W2962376003 hasConcept C154945302 @default.
• W2962376003 hasConcept C155512373 @default.
• W2962376003 hasConcept C17744445 @default.
• W2962376003 hasConcept C186633575 @default.
• W2962376003 hasConcept C199539241 @default.
• W2962376003 hasConcept C203616005 @default.
• W2962376003 hasConcept C2776359362 @default.
• W2962376003 hasConcept C2777210771 @default.
• W2962376003 hasConcept C2780586882 @default.
• W2962376003 hasConcept C28826006 @default.
• W2962376003 hasConcept C33923547 @default.
• W2962376003 hasConcept C41008148 @default.
• W2962376003 hasConcept C94625758 @default.
• W2962376003 hasConceptScore W2962376003C108583219 @default.
• W2962376003 hasConceptScore W2962376003C111472728 @default.
• W2962376003 hasConceptScore W2962376003C11413529 @default.
• W2962376003 hasConceptScore W2962376003C114614502 @default.
• W2962376003 hasConceptScore W2962376003C134306372 @default.
• W2962376003 hasConceptScore W2962376003C138885662 @default.
• W2962376003 hasConceptScore W2962376003C154945302 @default.
• W2962376003 hasConceptScore W2962376003C155512373 @default.
• W2962376003 hasConceptScore W2962376003C17744445 @default.
• W2962376003 hasConceptScore W2962376003C186633575 @default.
• W2962376003 hasConceptScore W2962376003C199539241 @default.
• W2962376003 hasConceptScore W2962376003C203616005 @default.
• W2962376003 hasConceptScore W2962376003C2776359362 @default.
• W2962376003 hasConceptScore W2962376003C2777210771 @default.
• W2962376003 hasConceptScore W2962376003C2780586882 @default.
• W2962376003 hasConceptScore W2962376003C28826006 @default.
• W2962376003 hasConceptScore W2962376003C33923547 @default.
• W2962376003 hasConceptScore W2962376003C41008148 @default.
• W2962376003 hasConceptScore W2962376003C94625758 @default.
• W2962376003 hasLocation W29623760031 @default.
• W2962376003 hasOpenAccess W2962376003 @default.
• W2962376003 hasPrimaryLocation W29623760031 @default.
• W2962376003 hasRelatedWork W2766965791 @default.
• W2962376003 hasRelatedWork W2787132992 @default.
• W2962376003 hasRelatedWork W2810862998 @default.
• W2962376003 hasRelatedWork W2889756710 @default.
• W2962376003 hasRelatedWork W2910114055 @default.
• W2962376003 hasRelatedWork W2922277331 @default.
• W2962376003 hasRelatedWork W2925259031 @default.
• W2962376003 hasRelatedWork W2947319091 @default.
• W2962376003 hasRelatedWork W2950051890 @default.
• W2962376003 hasRelatedWork W2951610569 @default.

• next