FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3157391736> ?p ?o ?g. }

• W3157391736 endingPage "2990" @default.
• W3157391736 startingPage "2989" @default.
• W3157391736 abstract "Colleagues have joined me in this Reply – we are grateful for the chance to consider the letter by Raiteri (2021) which covered one important part of Gandevia's editorial (Gandevia, 2021). The piece included one figure showing two curves in an attempt to deal with the interpretation of P-values and the replicability of an experiment based on P-values. The intended messages from this component of the editorial were the following. First, the likelihood to obtain P < 0.05 in a replication increases markedly if very low P-values are obtained in the initial experiment (say below 0.001) (see Fig. 1 in Gandevia, 2021). Second, any initial P-value around 0.05 does not signify any trend at all because the probability of an exact replication to obtain P < 0.05 is approximately 50% (see below). This message is intended to deflate any excitement accompanying an initial P-value just below 0.05 as an indication of significance. Perhaps you could equally think of such a value as a trend for in-significance. This point is not contested. Interestingly, for an initial P-value around 0.01 the probability of obtaining P < 0.01 in a replication is also approximately 50%. Third, the prediction intervals for a replication are extraordinarily wide – almost unbelievably so. For an initial experiment which obtains P = 0.05, the 80% interval for replication P-values is huge: from 0.0002 to 0.65 (for a two-tailed test; see Fig. 1). (It was given for a one-tailed test in the original graph with a lower, but still huge, interval from 0.00008 to 0.44 (Gandevia, 2021)). The above values for the prediction intervals are derived from Cumming (2008; Appendix B). They assume exact replications that are identical with the initial experiment, except using a new sample. It is not required to assume, as Raiteri (2021) stated, that the initial experiment estimated the true effect size exactly. In summary, the prediction intervals for replication P-values, the grey bars in Fig. 1, arise from idealized exact replications but, even so, are alarmingly wide. Now to some clarification of the first and third messages. For the first message, the data in the original figure (Gandevia, 2021) were taken from Goodman (1992) for P = 0.001–0.10 and from Curran-Everett (2016) for P < 0.001. They relate to an attempted exact replication, notwithstanding the acknowledged difficulty in such an attempt. The original text neglected to say that the probabilities were for the preferred two-tailed replication. Goodman's data for P = 0.001–0.10 were chosen deliberately from the right column of his Table 1 because it used ‘the more realistic scenario that we do not know that μ [true effect size]… and our uncertainty is modelled as a prior probability of μ that is locally uniform, that is the distribution of μ prior to the second experiment is proportional to its likelihood function in the first experiment’. It deserves mention that the general shape of the relationship between the P-value from the original experiment and the probability of its replication (at say 0.05) is largely independent of sample size and also the statistic used (Boos & Stefanski, 2011). Further, the approximate shape can be predicted by a number of statistical approaches (Boos & Stefanski, 2011; see also Cumming, 2008). A practical corollary is that when a replication is attempted, a large P-value can become small, and a small P-value can become large (e.g. Amrhein et al. 2019b)! This is illustrated formally in Fig. 1 in which the wide prediction intervals are clear even for studies in which the P-value from the initial experiment is very low (say <0.001). Further, these prediction intervals are not influenced by the sample or effect size, provided only that the sample is not too small (Cumming, 2008). For the third message, arguments from Miller & Schwarz (2011) are cited by Raiteri (2021) and used to paint an even gloomier view about the difficulty of predicting replication P-values. They model the research ‘world’ as more complex than the simple one a physiologist may encounter in an attempted exact replication – not all parameters including factors related to the research field, and the individual experimenters, are known. Thus, we can expect that P-values will vary even more widely than in Fig. 1, and perhaps even sufficiently to justify Miller and Schwarz's conclusion that ‘accurate estimates of replication probability are generally unattainable’. Individual researchers will have to decide whether to consider the reproducibility of their results based on the massive range of replication intervals (Fig. 1) and the issues raised by Miller & Schwarz (2011). Alternatively, they can use different approaches: for example, the use of confidence intervals and meta-analysis from repeated experiments (Cumming & Calin-Jageman, 2017). They can reinterpret and re-express the observed P-values based on the concept of a false discovery rate (Colquhoun, 2014, 2017). Finally, they can take some reassurance in the (rare) event that should the initial study have a P-value below 0.05 and this probability is achieved in a replication, then the likelihood of a ‘true’ finding is substantially increased. Our discussion indicates that, unless P-values are extremely small, they give very little or essentially no information (Miller & Schwarz, 2011; Raiteri, 2021) about what P-value a replication is likely to obtain. More broadly, a P-value, as a single number, cannot make salient the degree of uncertainty in a result, for example because it mixes information on the size of the effect and how precisely it was measured (Amrhein et al. 2017). In contrast, the extent of a confidence interval (CI), which some prefer to label a compatibility interval (Amrhein et al. 2019a,b), makes that uncertainty salient: a short CI gives reassurance, a long CI gives a disappointing message that there is much uncertainty in knowledge of the effect size being investigated. On average, there is an 83% chance that a replication result (point estimate) falls within a 95% CI (Cumming, 2014). Thus, the observed 95% CI provides on average an 83% prediction interval for a replication result, although we should not forget the ‘dance of the confidence intervals’ (Cumming, 2014): this shows how a valid interval will change from sample to sample due to random variation (Amrhein et al. 2019a,b). But, in contrast to a P-value, a CI directly uncovers a range of values that replication estimates could take. Scrutinizing results through replication plus using other techniques and arguments to reach conclusions is crucial to the progress of physiology, indeed all science. It should not be obstructed by the misunderstanding and misuse of P-values and other statistical measures. No competing interests declared. All authors have read and approved the final version of this manuscript and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All persons designated as authors qualify for authorship, and all those who qualify for authorship are listed. None." @default.
• W3157391736 created "2021-05-10" @default.
• W3157391736 creator A5000112938 @default.
• W3157391736 creator A5040419891 @default.
• W3157391736 creator A5054085861 @default.
• W3157391736 creator A5075835913 @default.
• W3157391736 date "2021-05-08" @default.
• W3157391736 modified "2023-09-24" @default.
• W3157391736 title "Replication: Do not trust your p‐value, be it small or large" @default.
• W3157391736 cites W1986038840 @default.
• W3157391736 cites W2040032582 @default.
• W3157391736 cites W2042125492 @default.
• W3157391736 cites W2073157712 @default.
• W3157391736 cites W2127875809 @default.
• W3157391736 cites W2136634361 @default.
• W3157391736 cites W2397078876 @default.
• W3157391736 cites W2610297457 @default.
• W3157391736 cites W2919176916 @default.
• W3157391736 cites W2924950986 @default.
• W3157391736 cites W2952752960 @default.
• W3157391736 cites W3124125535 @default.
• W3157391736 cites W3159015164 @default.
• W3157391736 doi "https://doi.org/10.1113/jp281614" @default.
• W3157391736 hasPubMedId "https://pubmed.ncbi.nlm.nih.gov/33963767" @default.
• W3157391736 hasPublicationYear "2021" @default.
• W3157391736 type Work @default.
• W3157391736 sameAs 3157391736 @default.
• W3157391736 citedByCount "7" @default.
• W3157391736 countsByYear W31573917362021 @default.
• W3157391736 countsByYear W31573917362022 @default.
• W3157391736 countsByYear W31573917362023 @default.
• W3157391736 crossrefType "journal-article" @default.
• W3157391736 hasAuthorship W3157391736A5000112938 @default.
• W3157391736 hasAuthorship W3157391736A5040419891 @default.
• W3157391736 hasAuthorship W3157391736A5054085861 @default.
• W3157391736 hasAuthorship W3157391736A5075835913 @default.
• W3157391736 hasBestOaLocation W31573917361 @default.
• W3157391736 hasConcept C105795698 @default.
• W3157391736 hasConcept C12590798 @default.
• W3157391736 hasConcept C2776291640 @default.
• W3157391736 hasConcept C33923547 @default.
• W3157391736 hasConceptScore W3157391736C105795698 @default.
• W3157391736 hasConceptScore W3157391736C12590798 @default.
• W3157391736 hasConceptScore W3157391736C2776291640 @default.
• W3157391736 hasConceptScore W3157391736C33923547 @default.
• W3157391736 hasIssue "11" @default.
• W3157391736 hasLocation W31573917361 @default.
• W3157391736 hasLocation W31573917362 @default.
• W3157391736 hasOpenAccess W3157391736 @default.
• W3157391736 hasPrimaryLocation W31573917361 @default.
• W3157391736 hasRelatedWork W2031325922 @default.
• W3157391736 hasRelatedWork W2067733742 @default.
• W3157391736 hasRelatedWork W2901033488 @default.
• W3157391736 hasRelatedWork W2956287921 @default.
• W3157391736 hasRelatedWork W3016766501 @default.
• W3157391736 hasRelatedWork W3086458689 @default.
• W3157391736 hasRelatedWork W4205713785 @default.
• W3157391736 hasRelatedWork W4241315537 @default.
• W3157391736 hasRelatedWork W4282545449 @default.
• W3157391736 hasRelatedWork W4287666211 @default.
• W3157391736 hasVolume "599" @default.
• W3157391736 isParatext "false" @default.
• W3157391736 isRetracted "false" @default.
• W3157391736 magId "3157391736" @default.
• W3157391736 workType "article" @default.