Matches in SemOpenAlex for { <https://semopenalex.org/work/W2474835145> ?p ?o ?g. }
- W2474835145 endingPage "18" @default.
- W2474835145 startingPage "1" @default.
- W2474835145 abstract "Defect prediction models help software quality assurance teams to allocate their limited resources to the most defect-prone modules. Model validation techniques, such as <inline-formula><tex-math notation=LaTeX>$k$</tex-math> </inline-formula> -fold cross-validation, use historical data to estimate how well a model will perform in the future. However, little is known about how accurate the estimates of model validation techniques tend to be. In this paper, we investigate the bias and variance of model validation techniques in the domain of defect prediction. Analysis of 101 public defect datasets suggests that 77 percent of them are highly susceptible to producing unstable results– - selecting an appropriate model validation technique is a critical experimental design choice. Based on an analysis of 256 studies in the defect prediction literature, we select the 12 most commonly adopted model validation techniques for evaluation. Through a case study of 18 systems, we find that single-repetition holdout validation tends to produce estimates with 46-229 percent more bias and 53-863 percent more variance than the top-ranked model validation techniques. On the other hand, out-of-sample bootstrap validation yields the best balance between the bias and variance of estimates in the context of our study. Therefore, we recommend that future defect prediction studies avoid single-repetition holdout validation, and instead, use out-of-sample bootstrap validation." @default.
- W2474835145 created "2016-07-22" @default.
- W2474835145 creator A5011588138 @default.
- W2474835145 creator A5043053174 @default.
- W2474835145 creator A5081449581 @default.
- W2474835145 creator A5091586373 @default.
- W2474835145 date "2017-01-01" @default.
- W2474835145 modified "2023-10-10" @default.
- W2474835145 title "An Empirical Comparison of Model Validation Techniques for Defect Prediction Models" @default.
- W2474835145 cites W1831050183 @default.
- W2474835145 cites W1920585257 @default.
- W2474835145 cites W1971654961 @default.
- W2474835145 cites W1973009402 @default.
- W2474835145 cites W1975040830 @default.
- W2474835145 cites W1976526581 @default.
- W2474835145 cites W1981251392 @default.
- W2474835145 cites W1984425068 @default.
- W2474835145 cites W1999830936 @default.
- W2474835145 cites W2005504865 @default.
- W2474835145 cites W2007705030 @default.
- W2474835145 cites W2019338079 @default.
- W2474835145 cites W2020181407 @default.
- W2474835145 cites W2021688474 @default.
- W2474835145 cites W2025700486 @default.
- W2474835145 cites W2028896946 @default.
- W2474835145 cites W2037557484 @default.
- W2474835145 cites W2037668591 @default.
- W2474835145 cites W2039081830 @default.
- W2474835145 cites W2046830558 @default.
- W2474835145 cites W2057328179 @default.
- W2474835145 cites W2060374620 @default.
- W2474835145 cites W2063876764 @default.
- W2474835145 cites W2068430427 @default.
- W2474835145 cites W2068816944 @default.
- W2474835145 cites W2070567333 @default.
- W2474835145 cites W2072785870 @default.
- W2474835145 cites W2073241381 @default.
- W2474835145 cites W2073792037 @default.
- W2474835145 cites W2082183626 @default.
- W2474835145 cites W2084085041 @default.
- W2474835145 cites W2091498228 @default.
- W2474835145 cites W2094104004 @default.
- W2474835145 cites W2094764356 @default.
- W2474835145 cites W2097883090 @default.
- W2474835145 cites W2098673848 @default.
- W2474835145 cites W2100697007 @default.
- W2474835145 cites W2103296684 @default.
- W2474835145 cites W2105776892 @default.
- W2474835145 cites W2105981176 @default.
- W2474835145 cites W2109325327 @default.
- W2474835145 cites W2110298216 @default.
- W2474835145 cites W2111814036 @default.
- W2474835145 cites W2112658968 @default.
- W2474835145 cites W2115709314 @default.
- W2474835145 cites W2118978333 @default.
- W2474835145 cites W2119910794 @default.
- W2474835145 cites W2123279272 @default.
- W2474835145 cites W2124172959 @default.
- W2474835145 cites W2124545752 @default.
- W2474835145 cites W2126436234 @default.
- W2474835145 cites W2126957654 @default.
- W2474835145 cites W2129925362 @default.
- W2474835145 cites W2135198476 @default.
- W2474835145 cites W2137122305 @default.
- W2474835145 cites W2138428785 @default.
- W2474835145 cites W2138827327 @default.
- W2474835145 cites W2139749383 @default.
- W2474835145 cites W2139855137 @default.
- W2474835145 cites W2140785063 @default.
- W2474835145 cites W2142481192 @default.
- W2474835145 cites W2145236186 @default.
- W2474835145 cites W2145680370 @default.
- W2474835145 cites W2146338950 @default.
- W2474835145 cites W2146474553 @default.
- W2474835145 cites W2148371346 @default.
- W2474835145 cites W2149014999 @default.
- W2474835145 cites W2151666086 @default.
- W2474835145 cites W2154446978 @default.
- W2474835145 cites W2157825442 @default.
- W2474835145 cites W2159747233 @default.
- W2474835145 cites W2163018352 @default.
- W2474835145 cites W2163732854 @default.
- W2474835145 cites W2164627280 @default.
- W2474835145 cites W2166573308 @default.
- W2474835145 cites W2172232422 @default.
- W2474835145 cites W2218021565 @default.
- W2474835145 cites W2330210193 @default.
- W2474835145 cites W2367798545 @default.
- W2474835145 cites W2787894218 @default.
- W2474835145 cites W28412257 @default.
- W2474835145 cites W2911964244 @default.
- W2474835145 cites W3106889297 @default.
- W2474835145 cites W3141989311 @default.
- W2474835145 cites W353957520 @default.
- W2474835145 cites W4206600618 @default.
- W2474835145 cites W4236586490 @default.
- W2474835145 cites W4241607140 @default.
- W2474835145 cites W4243379839 @default.