Matches in SemOpenAlex for { <https://semopenalex.org/work/W3110368543> ?p ?o ?g. }
- W3110368543 endingPage "29" @default.
- W3110368543 startingPage "1" @default.
- W3110368543 abstract "Flaky tests are tests that can non-deterministically pass or fail for the same code version. These tests undermine regression testing efficiency, because developers cannot easily identify whether a test fails due to their recent changes or due to flakiness. Ideally, one would detect flaky tests right when flakiness is introduced, so that developers can then immediately remove the flakiness. Some software organizations, e.g., Mozilla and Netflix, run some tools—detectors—to detect flaky tests as soon as possible. However, detecting flaky tests is costly due to their inherent non-determinism, so even state-of-the-art detectors are often impractical to be used on all tests for each project change. To combat the high cost of applying detectors, these organizations typically run a detector solely on newly added or directly modified tests, i.e., not on unmodified tests or when other changes occur (including changes to the test suite, the code under test, and library dependencies). However, it is unclear how many flaky tests can be detected or missed by applying detectors in only these limited circumstances. To better understand this problem, we conduct a large-scale longitudinal study of flaky tests to determine when flaky tests become flaky and what changes cause them to become flaky. We apply two state-of-theart detectors to 55 Java projects, identifying a total of 245 flaky tests that can be compiled and run in the code version where each test was added. We find that 75% of flaky tests (184 out of 245) are flaky when added, indicating substantial potential value for developers to run detectors specifically on newly added tests. However, running detectors solely on newly added tests would still miss detecting 25% of flaky tests. The percentage of flaky tests that can be detected does increase to 85% when detectors are run on newly added or directly modified tests. The remaining 15% of flaky tests become flaky due to other changes and can be detected only when detectors are always applied to all tests. Our study is the first to empirically evaluate when tests become flaky and to recommend guidelines for applying detectors in the future." @default.
- W3110368543 created "2020-12-07" @default.
- W3110368543 creator A5000577598 @default.
- W3110368543 creator A5017132290 @default.
- W3110368543 creator A5036039220 @default.
- W3110368543 creator A5048118068 @default.
- W3110368543 creator A5053080959 @default.
- W3110368543 creator A5081289583 @default.
- W3110368543 date "2020-11-13" @default.
- W3110368543 modified "2023-10-11" @default.
- W3110368543 title "A large-scale longitudinal study of flaky tests" @default.
- W3110368543 cites W1976954202 @default.
- W3110368543 cites W2014515160 @default.
- W3110368543 cites W2050127001 @default.
- W3110368543 cites W2073655054 @default.
- W3110368543 cites W2079811826 @default.
- W3110368543 cites W2096713153 @default.
- W3110368543 cites W2107500604 @default.
- W3110368543 cites W2131445143 @default.
- W3110368543 cites W2293797831 @default.
- W3110368543 cites W2589663975 @default.
- W3110368543 cites W2591823109 @default.
- W3110368543 cites W2594511738 @default.
- W3110368543 cites W2786619380 @default.
- W3110368543 cites W2901387825 @default.
- W3110368543 cites W2951170114 @default.
- W3110368543 cites W2955940099 @default.
- W3110368543 cites W2957454295 @default.
- W3110368543 cites W3040168066 @default.
- W3110368543 cites W3090144865 @default.
- W3110368543 cites W3091127659 @default.
- W3110368543 cites W3091299963 @default.
- W3110368543 cites W3102799430 @default.
- W3110368543 cites W3124196018 @default.
- W3110368543 cites W4232512250 @default.
- W3110368543 cites W4245116308 @default.
- W3110368543 cites W4252613971 @default.
- W3110368543 cites W4254906220 @default.
- W3110368543 cites W4291213652 @default.
- W3110368543 doi "https://doi.org/10.1145/3428270" @default.
- W3110368543 hasPublicationYear "2020" @default.
- W3110368543 type Work @default.
- W3110368543 sameAs 3110368543 @default.
- W3110368543 citedByCount "30" @default.
- W3110368543 countsByYear W31103685432021 @default.
- W3110368543 countsByYear W31103685432022 @default.
- W3110368543 countsByYear W31103685432023 @default.
- W3110368543 crossrefType "journal-article" @default.
- W3110368543 hasAuthorship W3110368543A5000577598 @default.
- W3110368543 hasAuthorship W3110368543A5017132290 @default.
- W3110368543 hasAuthorship W3110368543A5036039220 @default.
- W3110368543 hasAuthorship W3110368543A5048118068 @default.
- W3110368543 hasAuthorship W3110368543A5053080959 @default.
- W3110368543 hasAuthorship W3110368543A5081289583 @default.
- W3110368543 hasBestOaLocation W31103685431 @default.
- W3110368543 hasConcept C111919701 @default.
- W3110368543 hasConcept C119857082 @default.
- W3110368543 hasConcept C127413603 @default.
- W3110368543 hasConcept C128942645 @default.
- W3110368543 hasConcept C151552104 @default.
- W3110368543 hasConcept C152877465 @default.
- W3110368543 hasConcept C161821725 @default.
- W3110368543 hasConcept C166957645 @default.
- W3110368543 hasConcept C177264268 @default.
- W3110368543 hasConcept C186846655 @default.
- W3110368543 hasConcept C199360897 @default.
- W3110368543 hasConcept C200601418 @default.
- W3110368543 hasConcept C2776760102 @default.
- W3110368543 hasConcept C2777904410 @default.
- W3110368543 hasConcept C41008148 @default.
- W3110368543 hasConcept C529173508 @default.
- W3110368543 hasConcept C548217200 @default.
- W3110368543 hasConcept C76155785 @default.
- W3110368543 hasConcept C79581498 @default.
- W3110368543 hasConcept C94915269 @default.
- W3110368543 hasConcept C95457728 @default.
- W3110368543 hasConceptScore W3110368543C111919701 @default.
- W3110368543 hasConceptScore W3110368543C119857082 @default.
- W3110368543 hasConceptScore W3110368543C127413603 @default.
- W3110368543 hasConceptScore W3110368543C128942645 @default.
- W3110368543 hasConceptScore W3110368543C151552104 @default.
- W3110368543 hasConceptScore W3110368543C152877465 @default.
- W3110368543 hasConceptScore W3110368543C161821725 @default.
- W3110368543 hasConceptScore W3110368543C166957645 @default.
- W3110368543 hasConceptScore W3110368543C177264268 @default.
- W3110368543 hasConceptScore W3110368543C186846655 @default.
- W3110368543 hasConceptScore W3110368543C199360897 @default.
- W3110368543 hasConceptScore W3110368543C200601418 @default.
- W3110368543 hasConceptScore W3110368543C2776760102 @default.
- W3110368543 hasConceptScore W3110368543C2777904410 @default.
- W3110368543 hasConceptScore W3110368543C41008148 @default.
- W3110368543 hasConceptScore W3110368543C529173508 @default.
- W3110368543 hasConceptScore W3110368543C548217200 @default.
- W3110368543 hasConceptScore W3110368543C76155785 @default.
- W3110368543 hasConceptScore W3110368543C79581498 @default.
- W3110368543 hasConceptScore W3110368543C94915269 @default.
- W3110368543 hasConceptScore W3110368543C95457728 @default.
- W3110368543 hasFunder F4320306076 @default.