FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2171973575> ?p ?o ?g. }

• W2171973575 abstract "Performance at top k predictions, where instances are ranked by a (learned) scoring model, has been used as an evaluation metric in machine learning for various reasons such as where the entire corpus is unknown (e.g., the web) or where the results are to be used by a person with limited time or resources (e.g., ranking financial news stories where the investor only has ti me to look at relatively few stories per day). This evaluation metric is primarily us ed to report whether the performance of a given method is significantly better than other (baselin e) methods. It has not, however, been used to show whether the result is significant when compared to the simplest of baselines — the random model. If no models outperform the random model at a given confidence interval, then the results may not be worth reporting. This paper introduces a technique to perform an analysis of the expected performance of the top k predictions from the random model given k and a p-value on an evaluation dataset D. The technique is based on the realization that the distribu tion of the number of positives seen in the top k predictions follows a hypergeometric distribution, which has welldefined statistical density functions. As this distributio n is discrete, we show that using parametric estimations based on a binomial distribution are almost always in complete agreement with the discrete distribution and that, if they differ, an interpol ation of the discrete bounds gets very close to the parametric estimations. The technique is demonstrated on results from three prior published works, in which it clearly shows that even though performance is greatly increased (sometimes over 100%) with respect to the expected performance of the random model (at p = 0.5), these results, although qualitatively impressive, are not always as signi ficant ( p = 0.1) as might be suggested by the impressive qualitative improvements. The technique is used to show, given k, both how many positive instances are needed to achieve a specific sign ificance threshold is as well as how significant a given top k performance is. The technique when used in a more global setting is able to identify the crossover points, with respect to k, when a method becomes significant for a given p. Lastly, the technique is used to generate a complete confide nce curve, which shows a general trend over all k and visually shows where a method is significantly better tha n the random model over all values of k." @default.
• W2171973575 created "2016-06-24" @default.
• W2171973575 creator A5047798029 @default.
• W2171973575 date "2005-01-01" @default.
• W2171973575 modified "2023-09-28" @default.
• W2171973575 title "Significance Testing Against the Random Model for Scoring Models on Top k Predictions" @default.
• W2171973575 cites W109393027 @default.
• W2171973575 cites W1507029541 @default.
• W2171973575 cites W1550206324 @default.
• W2171973575 cites W1804928435 @default.
• W2171973575 cites W1999817920 @default.
• W2171973575 cites W202303397 @default.
• W2171973575 cites W2040424159 @default.
• W2171973575 cites W2081925477 @default.
• W2171973575 cites W2096152098 @default.
• W2171973575 cites W2096942889 @default.
• W2171973575 cites W2099708568 @default.
• W2171973575 cites W2118020653 @default.
• W2171973575 cites W2143017915 @default.
• W2171973575 cites W2147152072 @default.
• W2171973575 cites W2918131810 @default.
• W2171973575 cites W49486804 @default.
• W2171973575 hasPublicationYear "2005" @default.
• W2171973575 type Work @default.
• W2171973575 sameAs 2171973575 @default.
• W2171973575 citedByCount "0" @default.
• W2171973575 crossrefType "posted-content" @default.
• W2171973575 hasAuthorship W2171973575A5047798029 @default.
• W2171973575 hasConcept C100906024 @default.
• W2171973575 hasConcept C105795698 @default.
• W2171973575 hasConcept C110121322 @default.
• W2171973575 hasConcept C117251300 @default.
• W2171973575 hasConcept C134306372 @default.
• W2171973575 hasConcept C154945302 @default.
• W2171973575 hasConcept C162324750 @default.
• W2171973575 hasConcept C176217482 @default.
• W2171973575 hasConcept C176671685 @default.
• W2171973575 hasConcept C187736073 @default.
• W2171973575 hasConcept C189430467 @default.
• W2171973575 hasConcept C21547014 @default.
• W2171973575 hasConcept C2780898871 @default.
• W2171973575 hasConcept C2781089630 @default.
• W2171973575 hasConcept C2781315470 @default.
• W2171973575 hasConcept C33923547 @default.
• W2171973575 hasConcept C41008148 @default.
• W2171973575 hasConcept C41054675 @default.
• W2171973575 hasConcept C44249647 @default.
• W2171973575 hasConceptScore W2171973575C100906024 @default.
• W2171973575 hasConceptScore W2171973575C105795698 @default.
• W2171973575 hasConceptScore W2171973575C110121322 @default.
• W2171973575 hasConceptScore W2171973575C117251300 @default.
• W2171973575 hasConceptScore W2171973575C134306372 @default.
• W2171973575 hasConceptScore W2171973575C154945302 @default.
• W2171973575 hasConceptScore W2171973575C162324750 @default.
• W2171973575 hasConceptScore W2171973575C176217482 @default.
• W2171973575 hasConceptScore W2171973575C176671685 @default.
• W2171973575 hasConceptScore W2171973575C187736073 @default.
• W2171973575 hasConceptScore W2171973575C189430467 @default.
• W2171973575 hasConceptScore W2171973575C21547014 @default.
• W2171973575 hasConceptScore W2171973575C2780898871 @default.
• W2171973575 hasConceptScore W2171973575C2781089630 @default.
• W2171973575 hasConceptScore W2171973575C2781315470 @default.
• W2171973575 hasConceptScore W2171973575C33923547 @default.
• W2171973575 hasConceptScore W2171973575C41008148 @default.
• W2171973575 hasConceptScore W2171973575C41054675 @default.
• W2171973575 hasConceptScore W2171973575C44249647 @default.
• W2171973575 hasLocation W21719735751 @default.
• W2171973575 hasOpenAccess W2171973575 @default.
• W2171973575 hasPrimaryLocation W21719735751 @default.
• W2171973575 hasRelatedWork W1707694319 @default.
• W2171973575 hasRelatedWork W1874293905 @default.
• W2171973575 hasRelatedWork W1958502404 @default.
• W2171973575 hasRelatedWork W1973909163 @default.
• W2171973575 hasRelatedWork W2009546080 @default.
• W2171973575 hasRelatedWork W2029323071 @default.
• W2171973575 hasRelatedWork W2038084560 @default.
• W2171973575 hasRelatedWork W2050990691 @default.
• W2171973575 hasRelatedWork W2079776777 @default.
• W2171973575 hasRelatedWork W2117991484 @default.
• W2171973575 hasRelatedWork W2230486844 @default.
• W2171973575 hasRelatedWork W2340833078 @default.
• W2171973575 hasRelatedWork W2516440327 @default.
• W2171973575 hasRelatedWork W2568639833 @default.
• W2171973575 hasRelatedWork W2792812004 @default.
• W2171973575 hasRelatedWork W2793281865 @default.
• W2171973575 hasRelatedWork W3131682670 @default.
• W2171973575 hasRelatedWork W603862922 @default.
• W2171973575 hasRelatedWork W644441382 @default.
• W2171973575 hasRelatedWork W2585785862 @default.
• W2171973575 isParatext "false" @default.
• W2171973575 isRetracted "false" @default.
• W2171973575 magId "2171973575" @default.
• W2171973575 workType "article" @default.