Matches in SemOpenAlex for { <https://semopenalex.org/work/W4284664534> ?p ?o ?g. }
Showing items 1 to 67 of
67
with 100 items per page.
- W4284664534 abstract "Inserting an SVD meta-layer into neural networks is prone to make the covariance ill-conditioned, which could harm the model in the training stability and generalization abilities. In this paper, we systematically study how to improve the covariance conditioning by enforcing orthogonality to the Pre-SVD layer. Existing orthogonal treatments on the weights are first investigated. However, these techniques can improve the conditioning but would hurt the performance. To avoid such a side effect, we propose the Nearest Orthogonal Gradient (NOG) and Optimal Learning Rate (OLR). The effectiveness of our methods is validated in two applications: decorrelated Batch Normalization (BN) and Global Covariance Pooling (GCP). Extensive experiments on visual recognition demonstrate that our methods can simultaneously improve the covariance conditioning and generalization. Moreover, the combinations with orthogonal weight can further boost the performances." @default.
- W4284664534 created "2022-07-08" @default.
- W4284664534 creator A5027171279 @default.
- W4284664534 creator A5046597133 @default.
- W4284664534 creator A5088917149 @default.
- W4284664534 date "2022-07-05" @default.
- W4284664534 modified "2023-09-28" @default.
- W4284664534 title "Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality" @default.
- W4284664534 doi "https://doi.org/10.48550/arxiv.2207.02119" @default.
- W4284664534 hasPublicationYear "2022" @default.
- W4284664534 type Work @default.
- W4284664534 citedByCount "0" @default.
- W4284664534 crossrefType "posted-content" @default.
- W4284664534 hasAuthorship W4284664534A5027171279 @default.
- W4284664534 hasAuthorship W4284664534A5046597133 @default.
- W4284664534 hasAuthorship W4284664534A5088917149 @default.
- W4284664534 hasBestOaLocation W42846645341 @default.
- W4284664534 hasConcept C105795698 @default.
- W4284664534 hasConcept C11413529 @default.
- W4284664534 hasConcept C134306372 @default.
- W4284664534 hasConcept C136886441 @default.
- W4284664534 hasConcept C144024400 @default.
- W4284664534 hasConcept C153180895 @default.
- W4284664534 hasConcept C154945302 @default.
- W4284664534 hasConcept C17137986 @default.
- W4284664534 hasConcept C177148314 @default.
- W4284664534 hasConcept C178650346 @default.
- W4284664534 hasConcept C19165224 @default.
- W4284664534 hasConcept C22789450 @default.
- W4284664534 hasConcept C2524010 @default.
- W4284664534 hasConcept C33923547 @default.
- W4284664534 hasConcept C41008148 @default.
- W4284664534 hasConcept C45262634 @default.
- W4284664534 hasConcept C70437156 @default.
- W4284664534 hasConceptScore W4284664534C105795698 @default.
- W4284664534 hasConceptScore W4284664534C11413529 @default.
- W4284664534 hasConceptScore W4284664534C134306372 @default.
- W4284664534 hasConceptScore W4284664534C136886441 @default.
- W4284664534 hasConceptScore W4284664534C144024400 @default.
- W4284664534 hasConceptScore W4284664534C153180895 @default.
- W4284664534 hasConceptScore W4284664534C154945302 @default.
- W4284664534 hasConceptScore W4284664534C17137986 @default.
- W4284664534 hasConceptScore W4284664534C177148314 @default.
- W4284664534 hasConceptScore W4284664534C178650346 @default.
- W4284664534 hasConceptScore W4284664534C19165224 @default.
- W4284664534 hasConceptScore W4284664534C22789450 @default.
- W4284664534 hasConceptScore W4284664534C2524010 @default.
- W4284664534 hasConceptScore W4284664534C33923547 @default.
- W4284664534 hasConceptScore W4284664534C41008148 @default.
- W4284664534 hasConceptScore W4284664534C45262634 @default.
- W4284664534 hasConceptScore W4284664534C70437156 @default.
- W4284664534 hasLocation W42846645341 @default.
- W4284664534 hasOpenAccess W4284664534 @default.
- W4284664534 hasPrimaryLocation W42846645341 @default.
- W4284664534 hasRelatedWork W1991269640 @default.
- W4284664534 hasRelatedWork W2016839265 @default.
- W4284664534 hasRelatedWork W2018445155 @default.
- W4284664534 hasRelatedWork W2375904069 @default.
- W4284664534 hasRelatedWork W2769034141 @default.
- W4284664534 hasRelatedWork W2921836287 @default.
- W4284664534 hasRelatedWork W2944724518 @default.
- W4284664534 hasRelatedWork W2951391129 @default.
- W4284664534 hasRelatedWork W4284664534 @default.
- W4284664534 hasRelatedWork W2508457823 @default.
- W4284664534 isParatext "false" @default.
- W4284664534 isRetracted "false" @default.
- W4284664534 workType "article" @default.