FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2588048611> ?p ?o ?g. }

• W2588048611 endingPage "1164.e7" @default.
• W2588048611 startingPage "1153" @default.
• W2588048611 abstract "•Sparse synaptic wiring can optimize a neural representation for associative learning•Maximizing dimension predicts the degree of connectivity for cerebellum-like circuits•Supervised plasticity of input connections is needed to exploit dense wiring•Performance of a Hebbian readout neuron is formally related to dimension Synaptic connectivity varies widely across neuronal types. Cerebellar granule cells receive five orders of magnitude fewer inputs than the Purkinje cells they innervate, and cerebellum-like circuits, including the insect mushroom body, also exhibit large divergences in connectivity. In contrast, the number of inputs per neuron in cerebral cortex is more uniform and large. We investigate how the dimension of a representation formed by a population of neurons depends on how many inputs each neuron receives and what this implies for learning associations. Our theory predicts that the dimensions of the cerebellar granule-cell and Drosophila Kenyon-cell representations are maximized at degrees of synaptic connectivity that match those observed anatomically, showing that sparse connectivity is sometimes superior to dense connectivity. When input synapses are subject to supervised plasticity, however, dense wiring becomes advantageous, suggesting that the type of plasticity exhibited by a set of synapses is a major determinant of connection density. Synaptic connectivity varies widely across neuronal types. Cerebellar granule cells receive five orders of magnitude fewer inputs than the Purkinje cells they innervate, and cerebellum-like circuits, including the insect mushroom body, also exhibit large divergences in connectivity. In contrast, the number of inputs per neuron in cerebral cortex is more uniform and large. We investigate how the dimension of a representation formed by a population of neurons depends on how many inputs each neuron receives and what this implies for learning associations. Our theory predicts that the dimensions of the cerebellar granule-cell and Drosophila Kenyon-cell representations are maximized at degrees of synaptic connectivity that match those observed anatomically, showing that sparse connectivity is sometimes superior to dense connectivity. When input synapses are subject to supervised plasticity, however, dense wiring becomes advantageous, suggesting that the type of plasticity exhibited by a set of synapses is a major determinant of connection density. Extensive synaptic connectivity is often cited as a key element of neural computation, a prime example being the cerebral cortex, where neurons each receive thousands of inputs. However, the majority of neurons in the human brain, the 50 billion neurons that form the cerebellar granule-cell layer, each receive input from only about four of the mossy fibers innervating the cerebellum (Eccles et al., 1966Eccles J.C. Llinás R. Sasaki K. The mossy fibre-granule cell relay of the cerebellum and its inhibitory control by Golgi cells.Exp. Brain Res. 1966; 1: 82-101Crossref PubMed Scopus (131) Google Scholar, Llinás et al., 2003Llinás R.R. Walton K.D. Lang E.J. Cerebellum.in: Shepherd G.M. The Synaptic Organization of the Brain. 5th edition. Oxford University Press, 2003: 271-309Google Scholar). Sparse input is a feature shared by neurons that play roles analogous to granule cells in other neural circuits with cerebellum-like structures, such as the dorsal cochlear nucleus, the electrosensory lobe of electric fish, and the insect mushroom body (Mugnaini et al., 1980Mugnaini E. Osen K.K. Dahl A.L. Friedrich Jr., V.L. Korte G. Fine structure of granule cells and related interneurons (termed Golgi cells) in the cochlear nuclear complex of cat, rat and mouse.J. Neurocytol. 1980; 9: 537-570Crossref PubMed Scopus (199) Google Scholar, Bell et al., 2008Bell C.C. Han V. Sawtell N.B. Cerebellum-like structures and their implications for cerebellar function.Annu. Rev. Neurosci. 2008; 31: 1-24Crossref PubMed Scopus (199) Google Scholar, Keene and Waddell, 2007Keene A.C. Waddell S. Drosophila olfactory memory: single genes to complex neural circuits.Nat. Rev. Neurosci. 2007; 8: 341-354Crossref PubMed Scopus (297) Google Scholar). What is the functional role of diversity in synaptic connectivity, and what determines the appropriate number of input connections to a given neuronal type? In this study, we address these questions by investigating the ability of populations of neurons with different degrees of connectivity to support associative learning. Both cerebellar and cerebrocortical regions are involved in a variety of experience-dependent adaptive behaviors (Raymond et al., 1996Raymond J.L. Lisberger S.G. Mauk M.D. The cerebellum: a neuronal learning machine?.Science. 1996; 272: 1126-1131Crossref PubMed Scopus (476) Google Scholar, Buonomano and Merzenich, 1998Buonomano D.V. Merzenich M.M. Cortical plasticity: from synapses to maps.Annu. Rev. Neurosci. 1998; 21: 149-186Crossref PubMed Scopus (1600) Google Scholar). In cerebellar cortex and other cerebellum-like circuits, synaptic modifications associated with learning occur among the elaborate dendrites of densely connected output neurons— for example, cerebellar Purkinje cells (Ito et al., 1982Ito M. Sakurai M. Tongroach P. Climbing fibre induced depression of both mossy fibre responsiveness and glutamate sensitivity of cerebellar Purkinje cells.J. Physiol. 1982; 324: 113-134Crossref PubMed Scopus (668) Google Scholar) and the output neurons of the Drosophila mushroom body (Hige et al., 2015Hige T. Aso Y. Modi M.N. Rubin G.M. Turner G.C. Heterosynaptic plasticity underlies aversive olfactory learning in Drosophila.Neuron. 2015; 88: 985-998Abstract Full Text Full Text PDF PubMed Scopus (187) Google Scholar). Classic Marr-Albus theories of associative learning propose that the abundance of granule cells supports a high-dimensional representation of the information conveyed to the cerebellum by mossy fibers and that the large number of synapses received by Purkinje cells allows them access to this representation to form associations (Marr, 1969Marr D. A theory of cerebellar cortex.J. Physiol. 1969; 202: 437-470Crossref PubMed Scopus (2187) Google Scholar, Albus, 1971Albus J.S. A theory of cerebellar function.Math. Biosci. 1971; 10: 25-61Crossref Scopus (1695) Google Scholar). These theories assume that the inputs to granule cells are random and are not modified during learning. Anatomical and physiological studies suggest that the handful of inputs received by granule cells in the electrosensory lobe of electric fish (Kennedy et al., 2014Kennedy A. Wayne G. Kaifosh P. Alviña K. Abbott L.F. Sawtell N.B. A temporal basis for predicting the sensory consequences of motor commands in an electric fish.Nat. Neurosci. 2014; 17: 416-422Crossref PubMed Scopus (97) Google Scholar) and by Kenyon cells, the granule-cell analogs of the mushroom body, are a random subset of the afferents to these structures (Murthy et al., 2008Murthy M. Fiete I. Laurent G. Testing odor response stereotypy in the Drosophila mushroom body.Neuron. 2008; 59: 1009-1023Abstract Full Text Full Text PDF PubMed Scopus (116) Google Scholar, Caron et al., 2013Caron S.J.C. Ruta V. Abbott L.F. Axel R. Random convergence of olfactory inputs in the Drosophila mushroom body.Nature. 2013; 497: 113-117Crossref PubMed Scopus (240) Google Scholar, Gruntman and Turner, 2013Gruntman E. Turner G.C. Integration of the olfactory code across dendritic claws of single mushroom body neurons.Nat. Neurosci. 2013; 16: 1821-1829Crossref PubMed Scopus (100) Google Scholar). In many regions of cerebellar cortex, granule cells receive diverse (Huang et al., 2013Huang C.C. Sugino K. Shima Y. Guo C. Bai S. Mensh B.D. Nelson S.B. Hantman A.W. Convergence of pontine and proprioceptive streams onto multimodal cerebellar granule cells.eLife. 2013; 2: e00400Google Scholar, Chabrol et al., 2015Chabrol F.P. Arenz A. Wiechert M.T. Margrie T.W. DiGregorio D.A. Synaptic diversity enables temporal coding of coincident multisensory inputs in single neurons.Nat. Neurosci. 2015; 18: 718-727Crossref PubMed Scopus (118) Google Scholar, Ishikawa et al., 2015Ishikawa T. Shimuta M. Häusser M. Multimodal sensory integration in single cerebellar granule cells in vivo.eLife. 2015; 4: e12916Crossref PubMed Scopus (85) Google Scholar; but see Jörntell and Ekerot, 2006Jörntell H. Ekerot C.F. Properties of somatosensory synaptic integration in cerebellar granule cells in vivo.J. Neurosci. 2006; 26: 11786-11797Crossref PubMed Scopus (236) Google Scholar, Bengtsson and Jörntell, 2009Bengtsson F. Jörntell H. Sensory transmission in cerebellar granule cells relies on similarly coded mossy fiber inputs.Proc. Natl. Acad. Sci. USA. 2009; 106: 2389-2394Crossref PubMed Scopus (80) Google Scholar; and Discussion), though not completely random (Billings et al., 2014Billings G. Piasini E. Lőrincz A. Nusser Z. Silver R.A. Network structure within the cerebellar input layer enables lossless sparse encoding.Neuron. 2014; 83: 960-974Abstract Full Text Full Text PDF PubMed Scopus (85) Google Scholar), mossy-fiber input. In Marr-Albus theories, learning in cerebellar cortex relies exclusively on climbing-fiber-dependent modifications of the connections between parallel fibers and Purkinje cells, but unsupervised forms of plasticity have been reported for synapses from mossy fibers onto granule cells (Hansel et al., 2001Hansel C. Linden D.J. D’Angelo E. Beyond parallel fiber LTD: the diversity of synaptic and non-synaptic plasticity in the cerebellum.Nat. Neurosci. 2001; 4: 467-475Crossref PubMed Scopus (488) Google Scholar, Schweighofer et al., 2001Schweighofer N. Doya K. Lay F. Unsupervised learning of granule cell sparse codes enhances cerebellar adaptive control.Neuroscience. 2001; 103: 35-50Crossref PubMed Scopus (94) Google Scholar, Gao et al., 2012Gao Z. van Beugen B.J. De Zeeuw C.I. Distributed synergistic plasticity and cerebellar learning.Nat. Rev. Neurosci. 2012; 13: 619-635Crossref PubMed Scopus (344) Google Scholar, Gao et al., 2016Gao Z. Proietti-Onori M. Lin Z. Ten Brinke M.M. Boele H.J. Potters J.W. Ruigrok T.J. Hoebeek F.E. De Zeeuw C.I. Excitatory cerebellar nucleocortical circuit provides internal amplification during associative conditioning.Neuron. 2016; 89: 645-657Abstract Full Text Full Text PDF PubMed Scopus (96) Google Scholar, D’Angelo, 2014D’Angelo E. The organization of plasticity in the cerebellar cortex: from synapses to control.Prog. Brain Res. 2014; 210: 31-58Crossref PubMed Scopus (83) Google Scholar; but see Rylkova et al., 2015Rylkova D. Crank A.R. Linden D.J. Chronic in vivo imaging of ponto-cerebellar mossy fibers reveals morphological stability during whisker sensory manipulation in the adult rat.eNeuro. 2015; 2 (ENEURO.0075–15.2015)Crossref PubMed Scopus (6) Google Scholar). The logic of experience-dependent circuit modifications is less clear in cerebral cortex, where densely connected neurons exhibit diverse forms of synaptic plasticity (Abbott and Nelson, 2000Abbott L.F. Nelson S.B. Synaptic plasticity: taming the beast.Nat. Neurosci. 2000; 3: 1178-1183Crossref PubMed Scopus (1461) Google Scholar). Recent theoretical studies have proposed that populations of randomly connected cerebrocortical neurons support high-dimensional representations that enhance the ability of readout neurons to learn associations (Hansel and van Vreeswijk, 2012Hansel D. van Vreeswijk C. The mechanism of orientation selectivity in primary visual cortex without a functional map.J. Neurosci. 2012; 32: 4049-4064Crossref PubMed Scopus (86) Google Scholar, Rigotti et al., 2013Rigotti M. Barak O. Warden M.R. Wang X.J. Daw N.D. Miller E.K. Fusi S. The importance of mixed selectivity in complex cognitive tasks.Nature. 2013; 497: 585-590Crossref PubMed Scopus (739) Google Scholar, Barak et al., 2013Barak O. Rigotti M. Fusi S. The sparseness of mixed selectivity neurons controls the generalization-discrimination trade-off.J. Neurosci. 2013; 33: 3844-3856Crossref PubMed Scopus (102) Google Scholar, Babadi and Sompolinsky, 2014Babadi B. Sompolinsky H. Sparseness and expansion in sensory representations.Neuron. 2014; 83: 1213-1226Abstract Full Text Full Text PDF PubMed Scopus (109) Google Scholar), much as in theories of cerebellar cortex. Although these studies support the idea of random, high-dimensional representations as substrates for associative learning, they do not explain why the degree of synaptic connectivity in granule-cell and cerebrocortical layers is so different. To address this issue, we explore the effects of degree of connectivity, balance of excitation and inhibition, and synaptic weight distribution on the ability of a large neural representation to support associative learning. We also investigate whether synaptic plasticity of input connections (e.g., mossy fiber to granule cell), augmenting plasticity of output connections (e.g., granule cell to Purkinje cell, as in Marr-Albus theories), improves performance. In these analyses, we distinguish between unsupervised synaptic plasticity that normalizes or otherwise modulates the gain of synaptic input without the aid of a feedback (“error”) signal and supervised synaptic plasticity that exploits feedback signals to reshape the neural representation based on prior experience. Using a combination of analytic calculation and computer simulation, we find that the number of connections per neuron required to produce a high-dimensional representation increases slowly with the number of neurons. For a wide range of conditions (but not all), dimension and learning performance are maximized if the number of inputs is small. These results apply when the input synapses onto the granule or granule-like cells are selected randomly or approximately randomly and are not modified by a supervised learning procedure. In contrast, if we permit supervised modification of these synapses during the learning of a task, dense connectivity becomes advantageous. Our theory predicts a degree of connectivity that maximizes dimension for cerebellum-like structures and quantitatively matches what is observed for cerebellar granule cells and Kenyon cells of the Drosophila mushroom body. It also predicts that supervised synaptic plasticity at multiple stages of processing is necessary to exploit the dense connectivity of cerebral cortex. The “granule-cell” layer we consider consists of M neurons that each receive K excitatory connections from a random subset of N input channels and feedforward inhibition via a globally connected inhibitory neuron (Figure 1A ). We assume that, as in the cerebellum and its analogs, M>N. We refer to K as the synaptic degree. Because the neurons in the second layer are selective to combinations of their inputs, we refer to the two layers as the input (e.g., mossy-fiber) and mixed (e.g., granule-cell) layers, respectively. The ratio of the number of mixed-layer neurons to inputs, M/N, which we call the expansion ratio, is a critical parameter for our study. Our analysis assumes that the input channels are independent, so multiple channels with redundant activity are classified as a single input. This means that, for the Drosophila mushroom body, the number of distinct inputs is N = 50, an estimate of the number of antennal lobe glomeruli, while the estimated number of Kenyon cells is M = 2,000 (Keene and Waddell, 2007Keene A.C. Waddell S. Drosophila olfactory memory: single genes to complex neural circuits.Nat. Rev. Neurosci. 2007; 8: 341-354Crossref PubMed Scopus (297) Google Scholar), yielding an expansion ratio of 40 (Figure 1B). For the cerebellum, the number of mossy fibers and granule cells presynaptic to a single Purkinje cell in the rat are estimated to be N = 7,000 and M = 209,000 (Marr, 1969Marr D. A theory of cerebellar cortex.J. Physiol. 1969; 202: 437-470Crossref PubMed Scopus (2187) Google Scholar, Harvey and Napper, 1991Harvey R.J. Napper R.M. Quantitative studies on the mammalian cerebellum.Prog. Neurobiol. 1991; 36: 437-463Crossref PubMed Scopus (153) Google Scholar, Tyrrell and Willshaw, 1992Tyrrell T. Willshaw D. Cerebellar cortex: its simulation and the relevance of Marr’s theory.Philos. Trans. R. Soc. Lond. B Biol. Sci. 1992; 336: 239-257Crossref PubMed Scopus (98) Google Scholar; see STAR Methods), meaning the expansion ratio in this case is 30 if the mossy fibers are assumed to be independent (Figure 1C). A necessary requirement to produce a high-dimensional representation in the mixed layer is that its neurons should respond to different ensembles of inputs. To understand what is needed to produce this response heterogeneity, we first ask the simple question: what values of K ensure that, with high probability, each mixed-layer neuron receives a distinct subset of inputs? This question is a variant of the “birthday problem,” which concerns the likelihood of M people being born on unique days of the year. More generally, one can ask: what is the probability p of M draws (with replacement) from R equally likely possibilities all being different? In our case, the number of possibilities is R=N choose K, the number of ways a mixed-layer neuron can choose K presynaptic partners from N inputs. The maximum value of this probability is always at K=N/2, but this maximum typically lies in the middle of a large range of K for which p is extremely close to 1. We therefore denote by K∗ the smallest value of K for which p attains at least 95% of its maximum. The level of 95% is, of course, arbitrary, but the value of K∗ typically changes very little, if at all, if p is varied over a reasonable range near 100% (see below). Using numbers appropriate for the mushroom body, N = 50 and M = 2,000, leads to K∗ = 7, equal to the average observed number of projection-neuron inputs received by Kenyon cells (Caron et al., 2013Caron S.J.C. Ruta V. Abbott L.F. Axel R. Random convergence of olfactory inputs in the Drosophila mushroom body.Nature. 2013; 497: 113-117Crossref PubMed Scopus (240) Google Scholar; p takes the values 0.88, 0.98, and 0.996 for K = 6, 7, and 8). Using the same criterion for the cerebellum (N = 7,000 and M = 209,000) leads to K∗=4, equal to the typical number observed anatomically (Eccles et al., 1966Eccles J.C. Llinás R. Sasaki K. The mossy fibre-granule cell relay of the cerebellum and its inhibitory control by Golgi cells.Exp. Brain Res. 1966; 1: 82-101Crossref PubMed Scopus (131) Google Scholar; p takes values of 0.69, 0.9998, and greater than 0.9999 for K = 3, 4, and 5). That a small number of connections is sufficient to ensure each mixed-layer neuron receives a unique set of inputs owes to the rapid growth of N choose K with K. In other words, the combinatorial explosion in the number of possible wirings with synaptic degree permits even very sparsely connected systems to eliminate duplication in the mixed layer with high probability. The combinatorial calculation in the previous section treated two mixed-layer neurons as distinct even if only one of their inputs differed, but partially overlapping inputs may introduce correlations between neural responses that reduce the quality of the mixed-layer representation. To provide a more nuanced analysis, we define and calculate a quantitative measure that characterizes the mixed-layer representation: its dimension. We define the dimension of a system with M degrees of freedom, x=(x1,x2,…xM), asdim(x)=(∑i=1Mλi)2∑i=1Mλi2,(Equation 1) where λi are the eigenvalues of the covariance matrix of x computed by averaging over the distribution of inputs to the system (Abbott et al., 2011Abbott L.F. Rajan K. Sompolinsky H. Interactions between intrinsic and stimulus-evoked activity in recurrent neural networks.in: Ding M. Glanzman D.L. The Dynamic Brain: An Exploration of Neuronal Variability and Its Functional Significance. Oxford University Press, 2011: 65-82Crossref Scopus (41) Google Scholar). If the components of x are independent and have the same variance, all the eigenvalues are equal and dim(x)=M. Conversely, if the components are correlated so that the data points are distributed equally in each dimension of an m-dimensional subspace of the full M-dimensional space, only m eigenvalues will be nonzero and dim(x)=m (Figure 2A ). Later, we will show that this measure is closely related to the classification performance of a readout of mixed-layer responses. Increased dimension of the mixed-layer representation requires, in addition to mixing of the inputs received by each neuron, a nonlinearity provided by their input-output response function. We begin by analyzing the effect of input mixing alone and then address the effect of nonlinearity for the case of purely excitatory input (later we will add inhibition). In studying the effect of mixing alone, the components of the vector we consider, which we call h, are the total synaptic currents received by each of the mixed-layer neurons. These currents are given by h=Js, where J is the M×N matrix of synaptic weights describing the strengths of the connections from the input layer to the mixed layer and s is the vector of activities for the input layer. We consider the case of uncorrelated inputs with a uniform variance across them. If the excitatory connections onto the mixed-layer neurons have homogeneous weights,dim(h)≈N1+N/M+(K−1)2/N,(Equation 2) when M and N are large. Two features of this result are noteworthy. First, dim(h)≤N even when M≫N (Figure 2B, light curve). This is because h is a linear function of the activity of the input layer s, so it cannot have dimension higher than N. Second, increasing the synaptic degree K reduces the dimension (Figure 2C, light curve). This is due to the final term in the denominator, which arises from correlations between the entries of h. On average, two mixed-layer neurons share K2/N of their K inputs, leading to an average correlation of K/N. Indeed, as K approaches N, all the currents received by mixed-layer neurons become identical, and dim(h) approaches 1. Thus, unlike the combinatorial calculation of the previous section, this analysis indicates that increasing K beyond 1 is detrimental because it reduces the dimension due to increased correlations even when the identities of the inputs to each neuron are different. However, our real interest is the dimension of the nonlinear output of the mixed layer. We consider mixed-layer neurons with binary outputs given by m=Θ(h−θ), where Θ is a step function (applied element-wise) and θ is a vector of activity thresholds, one for each neuron. The thresholds are chosen so that each neuron is active with probability f, averaged across all the input patterns which, for now, we take to be standard Gaussians (we refer to the input-layer response to a particular stimulus as an input pattern). We call f the coding level of the mixed layer. Under these assumptions, the mixed-layer dimension is given bydim(m)≈11M+〈ρij〉2+Var(ρij),(Equation 3) where ρij is the correlation coefficient of the activities mi and mj of neurons i and j averaged over the distribution of input patterns (a more general expression holds when the coding levels of the neurons are not identical; see STAR Methods). Thus, the maximum mixed-layer dimension is limited by correlations among its neurons and saturates to a limiting value as the expansion ratio grows (Equation 3; Figure 2B, dark curve; also see Babadi and Sompolinsky, 2014Babadi B. Sompolinsky H. Sparseness and expansion in sensory representations.Neuron. 2014; 83: 1213-1226Abstract Full Text Full Text PDF PubMed Scopus (109) Google Scholar). The limiting value scales linearly with N (since Var(ρij)∼1/N; Figure S1). For a coding level of f=0.1, the saturation suggests that expansion ratios beyond 10–50 do not increase the mixed-layer dimension. This maximum expansion ratio and the maximum dimension both increase as the coding level is reduced (Figure S1), but as we will see, representations with extremely low coding levels do not necessarily lead to improved discrimination (Barak et al., 2013Barak O. Rigotti M. Fusi S. The sparseness of mixed selectivity neurons controls the generalization-discrimination trade-off.J. Neurosci. 2013; 33: 3844-3856Crossref PubMed Scopus (102) Google Scholar, Babadi and Sompolinsky, 2014Babadi B. Sompolinsky H. Sparseness and expansion in sensory representations.Neuron. 2014; 83: 1213-1226Abstract Full Text Full Text PDF PubMed Scopus (109) Google Scholar). We next investigate the dependence of mixed-layer dimension on K. When M is small, the mixed-layer dimension is similar to the input-current dimension and nearly constant over a wide range of K values (Figure 2C, top). However, for larger M, dimension initially grows with K, as increased K results in each mixed-layer neuron being selective to different combinations of inputs (Figure 2C, middle and bottom). The dimension is maximized for an intermediate value of K that depends on N, f, and the distribution of synaptic weights. For the case of homogeneous excitatory synaptic weights, this value is K=9 for N=1,000 and f=0.1. Above this value, dimension decreases because of positive average correlations among the mixed-layer neurons (Figure 2C; Figure S1). Thus, the detrimental effect of even small average correlations (〈ρij〉 in the denominator of Equation 3) on dimension leads to dimension being maximized at small K. Many studies have shown that inhibition can decorrelate neural activity (Ecker et al., 2010Ecker A.S. Berens P. Keliris G.A. Bethge M. Logothetis N.K. Tolias A.S. Decorrelated neuronal firing in cortical microcircuits.Science. 2010; 327: 584-587Crossref PubMed Scopus (446) Google Scholar, Renart et al., 2010Renart A. de la Rocha J. Bartho P. Hollender L. Parga N. Reyes A. Harris K.D. The asynchronous state in cortical circuits.Science. 2010; 327: 587-590Crossref PubMed Scopus (683) Google Scholar, Wiechert et al., 2010Wiechert M.T. Judkewitz B. Riecke H. Friedrich R.W. Mechanisms of pattern decorrelation by recurrent neuronal circuits.Nat. Neurosci. 2010; 13: 1003-1010Crossref PubMed Scopus (77) Google Scholar), so we next investigate whether inhibition can increase dimension by reducing correlations among mixed-layer neurons. In the Drosophila mushroom body, a single GABAergic interneuron on each side of the brain inhibits the Kenyon-cell population globally (Liu and Davis, 2009Liu X. Davis R.L. The GABAergic anterior paired lateral neuron suppresses and is suppressed by olfactory learning.Nat. Neurosci. 2009; 12: 53-59Crossref PubMed Scopus (154) Google Scholar; Figure 1B). In the cerebellum, Golgi cells are vastly outnumbered by the granule cells they inhibit (Eccles et al., 1966Eccles J.C. Llinás R. Sasaki K. The mossy fibre-granule cell relay of the cerebellum and its inhibitory control by Golgi cells.Exp. Brain Res. 1966; 1: 82-101Crossref PubMed Scopus (131) Google Scholar; Figure 1C). We therefore begin by introducing a single neuron that inhibits all mixed-layer neurons in proportion to its input, which is equal to the summed input-layer activity (Figure 1A). When inhibition is tuned to balance excitation on average (see STAR Methods), the distribution of the correlation between inputs received by pairs of mixed-layer neurons has zero mean, although the variance remains finite (Figure S1). Consequently, dim(h) does not decrease with K, and dim(m) does not exhibit a peak at small K (Figure 2D). However, even in this case, dimension quickly approaches its maximum (which occurs at K=N/2), and when N=1,000 and the expansion ratio is large, it attains 95% of its maximum value at K=29, or approximately 3% connectivity. Furthermore, the increase in dimension due to inhibition is only substantial for sufficiently large K. Thus, nonlinear mixed-layer neurons with small synaptic degree are sufficient to produce high-dimensional representations. This observation is consistent with the combinatorial argument of the first section, which showed that the explosion in possible wirings with synaptic degree leads to few redundant mixed-layer neurons, even when the degree is small. The present analysis also shows that positive average correlations limit dimension when mixed-layer neurons receive purely excitatory input, and that when K is large global inhibition can increase dimension through decorrelation. Although our analytic calculations are most easily performed for systems with feedforward inhibition, we verified with simulations that our qualitative results also hold for sufficiently strong recurrent inhibition (Figure S2). The observations of the previous section suggest that a representation formed by many neurons with small synaptic degree may be higher dimensional than one formed by fewer neurons with large synaptic degree. Therefore, when constructing a randomly wired neural system with limited resources, the former strategy may be preferable. To formalize this intuition, we ask: what combination of mixed-layer neuron number M and synaptic degree K maximizes the dimension of the mixed-layer representation when the total number of connections S=MK is limited to some maximum value? This is equivalent to fixing the number of presynaptic sites to which mixed-layer neurons can connect. For f=0.1 and S = 14,000, consistent with the Drosophila mushroom body (Keene and Waddell, 2007Keene A.C. Waddell S. Drosophila olfactory memory: single genes to complex neural circuits.Nat. Rev. Neurosci. 2007; 8: 341-354Crossref PubMed Scopus (297) Google Scholar, Caron et al., 2013Caron S.J.C. Ruta V. Abbott L.F. Axel R. Random convergence of olfactory inputs in the Drosophila mushroom body.Nature. 2013; 497: 113-117Crossref PubMed Scopus (240) Google Scholar), the optimum occurs at K=4 when inhibition is absent or K=8 when it is present (Figure 3A ), close to the observed value of 7. For f=0.01 and S=8.4×105, parameters consistent with the cerebellar granule-cell representation (Eccles et al., 1966Eccles J.C. Llinás" @default.
• W2588048611 created "2017-02-24" @default.
• W2588048611 creator A5020551914 @default.
• W2588048611 creator A5023563090 @default.
• W2588048611 creator A5037127074 @default.
• W2588048611 creator A5060223489 @default.
• W2588048611 creator A5073266403 @default.
• W2588048611 date "2017-03-01" @default.
• W2588048611 modified "2023-10-16" @default.
• W2588048611 title "Optimal Degrees of Synaptic Connectivity" @default.
• W2588048611 cites W1489333352 @default.
• W2588048611 cites W1502963283 @default.
• W2588048611 cites W1629577552 @default.
• W2588048611 cites W1722081514 @default.
• W2588048611 cites W1965413214 @default.
• W2588048611 cites W1967886764 @default.
• W2588048611 cites W1972329131 @default.
• W2588048611 cites W1972466164 @default.
• W2588048611 cites W1976738367 @default.
• W2588048611 cites W1985392528 @default.
• W2588048611 cites W1988013646 @default.
• W2588048611 cites W1991915036 @default.
• W2588048611 cites W1993533483 @default.
• W2588048611 cites W1998233818 @default.
• W2588048611 cites W2004437322 @default.
• W2588048611 cites W2016326504 @default.
• W2588048611 cites W2022062647 @default.
• W2588048611 cites W2027565428 @default.
• W2588048611 cites W2030450972 @default.
• W2588048611 cites W2030933387 @default.
• W2588048611 cites W2033708181 @default.
• W2588048611 cites W2037593288 @default.
• W2588048611 cites W2047527379 @default.
• W2588048611 cites W2049171959 @default.
• W2588048611 cites W2065575763 @default.
• W2588048611 cites W2072365940 @default.
• W2588048611 cites W2072802414 @default.
• W2588048611 cites W2076069761 @default.
• W2588048611 cites W2078932002 @default.
• W2588048611 cites W2082265738 @default.
• W2588048611 cites W2090765114 @default.
• W2588048611 cites W2090852124 @default.
• W2588048611 cites W2092938751 @default.
• W2588048611 cites W2093356195 @default.
• W2588048611 cites W2103019454 @default.
• W2588048611 cites W2106566258 @default.
• W2588048611 cites W2110456678 @default.
• W2588048611 cites W2123953466 @default.
• W2588048611 cites W2128986855 @default.
• W2588048611 cites W2139560011 @default.
• W2588048611 cites W2145264576 @default.
• W2588048611 cites W2147981220 @default.
• W2588048611 cites W2153473965 @default.
• W2588048611 cites W2155051950 @default.
• W2588048611 cites W2155741587 @default.
• W2588048611 cites W2158935809 @default.
• W2588048611 cites W2163618989 @default.
• W2588048611 cites W2164906588 @default.
• W2588048611 cites W2165095254 @default.
• W2588048611 cites W2187086501 @default.
• W2588048611 cites W2270672111 @default.
• W2588048611 cites W2270852892 @default.
• W2588048611 cites W2327209428 @default.
• W2588048611 cites W2341943149 @default.
• W2588048611 cites W2403384342 @default.
• W2588048611 cites W278926048 @default.
• W2588048611 cites W2919115771 @default.
• W2588048611 doi "https://doi.org/10.1016/j.neuron.2017.01.030" @default.
• W2588048611 hasPubMedCentralId "https://www.ncbi.nlm.nih.gov/pmc/articles/5379477" @default.
• W2588048611 hasPubMedId "https://pubmed.ncbi.nlm.nih.gov/28215558" @default.
• W2588048611 hasPublicationYear "2017" @default.
• W2588048611 type Work @default.
• W2588048611 sameAs 2588048611 @default.
• W2588048611 citedByCount "251" @default.
• W2588048611 countsByYear W25880486112017 @default.
• W2588048611 countsByYear W25880486112018 @default.
• W2588048611 countsByYear W25880486112019 @default.
• W2588048611 countsByYear W25880486112020 @default.
• W2588048611 countsByYear W25880486112021 @default.
• W2588048611 countsByYear W25880486112022 @default.
• W2588048611 countsByYear W25880486112023 @default.
• W2588048611 crossrefType "journal-article" @default.
• W2588048611 hasAuthorship W2588048611A5020551914 @default.
• W2588048611 hasAuthorship W2588048611A5023563090 @default.
• W2588048611 hasAuthorship W2588048611A5037127074 @default.
• W2588048611 hasAuthorship W2588048611A5060223489 @default.
• W2588048611 hasAuthorship W2588048611A5073266403 @default.
• W2588048611 hasBestOaLocation W25880486111 @default.
• W2588048611 hasConcept C15744967 @default.
• W2588048611 hasConcept C169760540 @default.
• W2588048611 hasConceptScore W2588048611C15744967 @default.
• W2588048611 hasConceptScore W2588048611C169760540 @default.
• W2588048611 hasFunder F4320306082 @default.
• W2588048611 hasFunder F4320306399 @default.
• W2588048611 hasFunder F4320306420 @default.
• W2588048611 hasFunder F4320332161 @default.
• W2588048611 hasIssue "5" @default.
• W2588048611 hasLocation W25880486111 @default.

• next