FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2346887341> ?p ?o ?g. }

• W2346887341 abstract "Information maximization is a common framework of unsupervised learning, which may be used for extracting informative representations y of the observed patterns x. The key idea there is to maximize mutual information (MI), which is a formal measure of coding efficiency. Unfortunately, exact maximization of MI is computationally tractable only in a few special cases; more generally, approximations need to be considered. Here we describe a family of variational lower bounds on mutual information which gives rise to a formal and theoretically rigorous approach to information maximization in large-scale stochastic channels. We hope that the results presented in this work are potentially interesting for maximizing mutual information from several perspectives. First of all, our method optimizes a proper lower bound, rather than a surrogate objective criterion or an approximation of MI (which may only be accurate under specific asymptotic assumptions, and weak or even undefined when the assumptions are violated). Secondly, the flexibility of the choice of the variational distribution makes it possible to generalize and improve simple bounds on MI. For example, we may introduce tractable auxiliary variational bounds on MI, which may be used to improve on any simple generic approach without altering properties of the original channel. Thirdly, the suggested variational framework is typically simpler than standard variational approaches to maximizing the conditional likelihood in stochastic autoencoder models, while it leads to the same fixed points in its simplest formulation; this gives rise to more efficient optimization procedures. Finally, in some cases the variational framework results in optimization procedures which only require local computations, which may be particularly attractive from the neuro-biological perspective. Possibly the most important contribution of this work is a rigorous and general framework for maximizing the mutual information in intrinsically intractable channels. We show that it gives rise to simple, stable, and easily generalizable optimization procedures, which outperform and supersede many of the common approximate information-maximizing techniques. We demonstrate our results by considering clustering, dimensionality reduction, and binary stochastic coding problems, and discuss a link to approximate statistical inference." @default.
• W2346887341 created "2016-06-24" @default.
• W2346887341 creator A5015937614 @default.
• W2346887341 date "2006-01-01" @default.
• W2346887341 modified "2023-09-24" @default.
• W2346887341 title "Variational Information Maximization in Stochastic Environments" @default.
• W2346887341 cites W111669422 @default.
• W2346887341 cites W115285041 @default.
• W2346887341 cites W1481229783 @default.
• W2346887341 cites W1489119587 @default.
• W2346887341 cites W1493132161 @default.
• W2346887341 cites W1497193185 @default.
• W2346887341 cites W1505265967 @default.
• W2346887341 cites W1509562192 @default.
• W2346887341 cites W1513861746 @default.
• W2346887341 cites W1514382660 @default.
• W2346887341 cites W1516111018 @default.
• W2346887341 cites W1516663666 @default.
• W2346887341 cites W1520426560 @default.
• W2346887341 cites W1520448186 @default.
• W2346887341 cites W1525921156 @default.
• W2346887341 cites W1526146785 @default.
• W2346887341 cites W1528056001 @default.
• W2346887341 cites W1534416612 @default.
• W2346887341 cites W1546961578 @default.
• W2346887341 cites W1554348720 @default.
• W2346887341 cites W1554663460 @default.
• W2346887341 cites W1555711139 @default.
• W2346887341 cites W1560512119 @default.
• W2346887341 cites W1561865685 @default.
• W2346887341 cites W1563088657 @default.
• W2346887341 cites W1564008636 @default.
• W2346887341 cites W1569914580 @default.
• W2346887341 cites W1578660878 @default.
• W2346887341 cites W1578739277 @default.
• W2346887341 cites W1579853615 @default.
• W2346887341 cites W1583912886 @default.
• W2346887341 cites W1589277047 @default.
• W2346887341 cites W1589527987 @default.
• W2346887341 cites W1592590442 @default.
• W2346887341 cites W1593793857 @default.
• W2346887341 cites W1594874645 @default.
• W2346887341 cites W1604803556 @default.
• W2346887341 cites W1607153847 @default.
• W2346887341 cites W1624804034 @default.
• W2346887341 cites W1630120088 @default.
• W2346887341 cites W1640070940 @default.
• W2346887341 cites W1651818244 @default.
• W2346887341 cites W1657213141 @default.
• W2346887341 cites W1662191912 @default.
• W2346887341 cites W1667165204 @default.
• W2346887341 cites W1669104078 @default.
• W2346887341 cites W1680579736 @default.
• W2346887341 cites W1746680969 @default.
• W2346887341 cites W1804017542 @default.
• W2346887341 cites W1839293085 @default.
• W2346887341 cites W1844433497 @default.
• W2346887341 cites W1891202952 @default.
• W2346887341 cites W1965555277 @default.
• W2346887341 cites W1969466749 @default.
• W2346887341 cites W1971074050 @default.
• W2346887341 cites W1977556410 @default.
• W2346887341 cites W1978183128 @default.
• W2346887341 cites W1981745143 @default.
• W2346887341 cites W1985926373 @default.
• W2346887341 cites W1986007546 @default.
• W2346887341 cites W1990645294 @default.
• W2346887341 cites W1995875735 @default.
• W2346887341 cites W2007995670 @default.
• W2346887341 cites W2011838500 @default.
• W2346887341 cites W2017257315 @default.
• W2346887341 cites W2019359195 @default.
• W2346887341 cites W2021123210 @default.
• W2346887341 cites W2023963201 @default.
• W2346887341 cites W2026205678 @default.
• W2346887341 cites W2026799324 @default.
• W2346887341 cites W2029236670 @default.
• W2346887341 cites W2037139490 @default.
• W2346887341 cites W2047408445 @default.
• W2346887341 cites W2048305092 @default.
• W2346887341 cites W2049387919 @default.
• W2346887341 cites W2049633694 @default.
• W2346887341 cites W2053565514 @default.
• W2346887341 cites W2054283707 @default.
• W2346887341 cites W2055339690 @default.
• W2346887341 cites W2060542838 @default.
• W2346887341 cites W2063971957 @default.
• W2346887341 cites W2066149233 @default.
• W2346887341 cites W2066873261 @default.
• W2346887341 cites W2068484625 @default.
• W2346887341 cites W2070586573 @default.
• W2346887341 cites W2078626246 @default.
• W2346887341 cites W2082195896 @default.
• W2346887341 cites W2082555627 @default.
• W2346887341 cites W2083380015 @default.
• W2346887341 cites W2084840427 @default.
• W2346887341 cites W2085927826 @default.
• W2346887341 cites W2087347434 @default.
• W2346887341 cites W2091891040 @default.
• W2346887341 cites W2096175520 @default.

• next