Matches in SemOpenAlex for { <https://semopenalex.org/work/W4367060891> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W4367060891 abstract "Can we recover the hidden parameters of an Artificial Neural Network (ANN) by probing its input-output mapping? We propose a systematic method, called `Expand-and-Cluster' that needs only the number of hidden layers and the activation function of the probed ANN to identify all network parameters. In the expansion phase, we train a series of networks of increasing size using the probed data of the ANN as a teacher. Expansion stops when a minimal loss is consistently reached in networks of a given size. In the clustering phase, weight vectors of the expanded students are clustered, which allows structured pruning of superfluous neurons in a principled way. We find that an overparameterization of a factor four is sufficient to reliably identify the minimal number of neurons and to retrieve the original network parameters in $80%$ of tasks across a family of 150 toy problems of variable difficulty. Furthermore, shallow and deep teacher networks trained on MNIST data can be identified with less than $5%$ overhead in the neuron number. Thus, while direct training of a student network with a size identical to that of the teacher is practically impossible because of the highly non-convex loss function, training with mild overparameterization followed by clustering and structured pruning correctly identifies the target network." @default.
- W4367060891 created "2023-04-27" @default.
- W4367060891 creator A5003480477 @default.
- W4367060891 creator A5006129821 @default.
- W4367060891 creator A5061199946 @default.
- W4367060891 creator A5089319205 @default.
- W4367060891 date "2023-04-25" @default.
- W4367060891 modified "2023-09-26" @default.
- W4367060891 title "Expand-and-Cluster: Exact Parameter Recovery of Neural Networks" @default.
- W4367060891 doi "https://doi.org/10.48550/arxiv.2304.12794" @default.
- W4367060891 hasPublicationYear "2023" @default.
- W4367060891 type Work @default.
- W4367060891 citedByCount "0" @default.
- W4367060891 crossrefType "posted-content" @default.
- W4367060891 hasAuthorship W4367060891A5003480477 @default.
- W4367060891 hasAuthorship W4367060891A5006129821 @default.
- W4367060891 hasAuthorship W4367060891A5061199946 @default.
- W4367060891 hasAuthorship W4367060891A5089319205 @default.
- W4367060891 hasBestOaLocation W43670608911 @default.
- W4367060891 hasConcept C108010975 @default.
- W4367060891 hasConcept C111919701 @default.
- W4367060891 hasConcept C119857082 @default.
- W4367060891 hasConcept C134306372 @default.
- W4367060891 hasConcept C14036430 @default.
- W4367060891 hasConcept C153180895 @default.
- W4367060891 hasConcept C154945302 @default.
- W4367060891 hasConcept C164866538 @default.
- W4367060891 hasConcept C182365436 @default.
- W4367060891 hasConcept C190502265 @default.
- W4367060891 hasConcept C2779960059 @default.
- W4367060891 hasConcept C31258907 @default.
- W4367060891 hasConcept C33923547 @default.
- W4367060891 hasConcept C38365724 @default.
- W4367060891 hasConcept C41008148 @default.
- W4367060891 hasConcept C50644808 @default.
- W4367060891 hasConcept C6557445 @default.
- W4367060891 hasConcept C73555534 @default.
- W4367060891 hasConcept C78458016 @default.
- W4367060891 hasConcept C86803240 @default.
- W4367060891 hasConceptScore W4367060891C108010975 @default.
- W4367060891 hasConceptScore W4367060891C111919701 @default.
- W4367060891 hasConceptScore W4367060891C119857082 @default.
- W4367060891 hasConceptScore W4367060891C134306372 @default.
- W4367060891 hasConceptScore W4367060891C14036430 @default.
- W4367060891 hasConceptScore W4367060891C153180895 @default.
- W4367060891 hasConceptScore W4367060891C154945302 @default.
- W4367060891 hasConceptScore W4367060891C164866538 @default.
- W4367060891 hasConceptScore W4367060891C182365436 @default.
- W4367060891 hasConceptScore W4367060891C190502265 @default.
- W4367060891 hasConceptScore W4367060891C2779960059 @default.
- W4367060891 hasConceptScore W4367060891C31258907 @default.
- W4367060891 hasConceptScore W4367060891C33923547 @default.
- W4367060891 hasConceptScore W4367060891C38365724 @default.
- W4367060891 hasConceptScore W4367060891C41008148 @default.
- W4367060891 hasConceptScore W4367060891C50644808 @default.
- W4367060891 hasConceptScore W4367060891C6557445 @default.
- W4367060891 hasConceptScore W4367060891C73555534 @default.
- W4367060891 hasConceptScore W4367060891C78458016 @default.
- W4367060891 hasConceptScore W4367060891C86803240 @default.
- W4367060891 hasLocation W43670608911 @default.
- W4367060891 hasOpenAccess W4367060891 @default.
- W4367060891 hasPrimaryLocation W43670608911 @default.
- W4367060891 hasRelatedWork W2150503081 @default.
- W4367060891 hasRelatedWork W2810865670 @default.
- W4367060891 hasRelatedWork W2886643713 @default.
- W4367060891 hasRelatedWork W2936783136 @default.
- W4367060891 hasRelatedWork W2994740422 @default.
- W4367060891 hasRelatedWork W3156786002 @default.
- W4367060891 hasRelatedWork W4225679450 @default.
- W4367060891 hasRelatedWork W4293191432 @default.
- W4367060891 hasRelatedWork W4293869292 @default.
- W4367060891 hasRelatedWork W1629725936 @default.
- W4367060891 isParatext "false" @default.
- W4367060891 isRetracted "false" @default.
- W4367060891 workType "article" @default.