Matches in SemOpenAlex for { <https://semopenalex.org/work/W3092664076> ?p ?o ?g. }
Showing items 1 to 94 of
94
with 100 items per page.
- W3092664076 abstract "It was conjectured that any neural network of any structure and arbitrary differentiable transfer functions at the nodes cannot learn the following problem sample efficiently when trained with gradient descent: The instances are the rows of a $d$-dimensional Hadamard matrix and the target is one of the features, i.e. very sparse. We essentially prove this conjecture: We show that after receiving a random training set of size $k < d$, the expected square loss is still $1-frac{k}{(d-1)}$. The only requirement needed is that the input layer is fully connected and the initial weight vectors of the input nodes are chosen from a rotation invariant distribution. Surprisingly the same type of problem can be solved drastically more efficient by a simple 2-layer linear neural network in which the $d$ inputs are connected to the output node by chains of length 2 (Now the input layer has only one edge per input). When such a network is trained by gradient descent, then it has been shown that its expected square loss is $frac{log d}{k}$. Our lower bounds essentially show that a sparse input layer is needed to sample efficiently learn sparse targets with gradient descent when the number of examples is less than the number of input features." @default.
- W3092664076 created "2020-10-22" @default.
- W3092664076 creator A5001603308 @default.
- W3092664076 creator A5056776503 @default.
- W3092664076 creator A5079068357 @default.
- W3092664076 date "2020-10-16" @default.
- W3092664076 modified "2023-09-27" @default.
- W3092664076 title "A case where a spindly two-layer linear network whips any neural network with a fully connected input layer." @default.
- W3092664076 cites W1577111082 @default.
- W3092664076 cites W2031979166 @default.
- W3092664076 cites W2069317438 @default.
- W3092664076 cites W2129658066 @default.
- W3092664076 cites W2595713799 @default.
- W3092664076 cites W2899476926 @default.
- W3092664076 cites W2906838315 @default.
- W3092664076 cites W3046601824 @default.
- W3092664076 cites W3046612445 @default.
- W3092664076 cites W3101388328 @default.
- W3092664076 cites W602904462 @default.
- W3092664076 hasPublicationYear "2020" @default.
- W3092664076 type Work @default.
- W3092664076 sameAs 3092664076 @default.
- W3092664076 citedByCount "0" @default.
- W3092664076 crossrefType "posted-content" @default.
- W3092664076 hasAuthorship W3092664076A5001603308 @default.
- W3092664076 hasAuthorship W3092664076A5056776503 @default.
- W3092664076 hasAuthorship W3092664076A5079068357 @default.
- W3092664076 hasConcept C106487976 @default.
- W3092664076 hasConcept C114614502 @default.
- W3092664076 hasConcept C118615104 @default.
- W3092664076 hasConcept C119857082 @default.
- W3092664076 hasConcept C121332964 @default.
- W3092664076 hasConcept C153258448 @default.
- W3092664076 hasConcept C159985019 @default.
- W3092664076 hasConcept C178790620 @default.
- W3092664076 hasConcept C184720557 @default.
- W3092664076 hasConcept C185592680 @default.
- W3092664076 hasConcept C192562407 @default.
- W3092664076 hasConcept C202444582 @default.
- W3092664076 hasConcept C202615002 @default.
- W3092664076 hasConcept C2779227376 @default.
- W3092664076 hasConcept C2780990831 @default.
- W3092664076 hasConcept C33923547 @default.
- W3092664076 hasConcept C41008148 @default.
- W3092664076 hasConcept C50644808 @default.
- W3092664076 hasConcept C62520636 @default.
- W3092664076 hasConcept C62611344 @default.
- W3092664076 hasConceptScore W3092664076C106487976 @default.
- W3092664076 hasConceptScore W3092664076C114614502 @default.
- W3092664076 hasConceptScore W3092664076C118615104 @default.
- W3092664076 hasConceptScore W3092664076C119857082 @default.
- W3092664076 hasConceptScore W3092664076C121332964 @default.
- W3092664076 hasConceptScore W3092664076C153258448 @default.
- W3092664076 hasConceptScore W3092664076C159985019 @default.
- W3092664076 hasConceptScore W3092664076C178790620 @default.
- W3092664076 hasConceptScore W3092664076C184720557 @default.
- W3092664076 hasConceptScore W3092664076C185592680 @default.
- W3092664076 hasConceptScore W3092664076C192562407 @default.
- W3092664076 hasConceptScore W3092664076C202444582 @default.
- W3092664076 hasConceptScore W3092664076C202615002 @default.
- W3092664076 hasConceptScore W3092664076C2779227376 @default.
- W3092664076 hasConceptScore W3092664076C2780990831 @default.
- W3092664076 hasConceptScore W3092664076C33923547 @default.
- W3092664076 hasConceptScore W3092664076C41008148 @default.
- W3092664076 hasConceptScore W3092664076C50644808 @default.
- W3092664076 hasConceptScore W3092664076C62520636 @default.
- W3092664076 hasConceptScore W3092664076C62611344 @default.
- W3092664076 hasLocation W30926640761 @default.
- W3092664076 hasOpenAccess W3092664076 @default.
- W3092664076 hasPrimaryLocation W30926640761 @default.
- W3092664076 hasRelatedWork W1705588077 @default.
- W3092664076 hasRelatedWork W1988019790 @default.
- W3092664076 hasRelatedWork W2064381290 @default.
- W3092664076 hasRelatedWork W2118833474 @default.
- W3092664076 hasRelatedWork W2613481513 @default.
- W3092664076 hasRelatedWork W2625063094 @default.
- W3092664076 hasRelatedWork W2751060898 @default.
- W3092664076 hasRelatedWork W2791562219 @default.
- W3092664076 hasRelatedWork W2901741727 @default.
- W3092664076 hasRelatedWork W2925259031 @default.
- W3092664076 hasRelatedWork W2946840143 @default.
- W3092664076 hasRelatedWork W2947319091 @default.
- W3092664076 hasRelatedWork W2970890756 @default.
- W3092664076 hasRelatedWork W2989472764 @default.
- W3092664076 hasRelatedWork W3003477032 @default.
- W3092664076 hasRelatedWork W3035519553 @default.
- W3092664076 hasRelatedWork W3078407515 @default.
- W3092664076 hasRelatedWork W3088153346 @default.
- W3092664076 hasRelatedWork W3177662876 @default.
- W3092664076 hasRelatedWork W3201856707 @default.
- W3092664076 isParatext "false" @default.
- W3092664076 isRetracted "false" @default.
- W3092664076 magId "3092664076" @default.
- W3092664076 workType "article" @default.