tempus-fugit

Java micro-library for writing & testing concurrent code

Intermittent Tests

As much as possible, you aim to have a completely deterministic tests but despite your best efforts, the odd flickering test can still get through. Occasionally, you might want to run such a test repeatedly to get an idea of its indeterminacy. The Intermittent annotation can be combined with the IntermittentTestRunner to provide this behaviour along side junit.

You simply mark a junit test method (or class) as potentially intermittent using the Intermittent annotation as follows.

1
2
3
4
5

@Test
@Intermittent
public void flickering() {
   // ...
}

You can then use the @RunWith annotation to run the test using the IntermittentTestRunner. Any @Before or @After methods will be run once for each test repetition. The example below also shows that the repetition count can be overridden on the method annotation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

@RunWith(IntermittentTestRunner.class)
public class IntermittentTestRunnerTest {

    private static int testCounter = 0;
    private static int afterCounter = 0;
    private static int afterClassCounter = 0;

    @Test
    @Intermittent(repetition = 99)
    public void annotatedTest() {
        testCounter++;
    }

    @After
    public void assertAfterIsCalledRepeatedlyForAnnotatedTests() {
        assertThat(testCounter, is(equalTo(++afterCounter)));
    }

    @AfterClass
    public static void assertAfterClassIsCalledOnce() {
        assertThat(++afterClassCounter, is(equalTo(1)));
    }

    @AfterClass
    public static void assertAnnotatedTestRunsMultipleTimes() {
        assertThat(testCounter, is(equalTo(99)));
    }
}

If you annotate the class rather than individual test methods, every test method of the class will be treated as if it were marked as @Intermittent.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

@RunWith(IntermittentTestRunner.class)
@Intermittent(repetition = 10)
public class IntermittentTestRunnerTest {

    private static int testCounter = 0;

    @Test
    public void annotatedTest() {
        testCounter++;
    }

    @Test
    public void anotherAnnotatedTest() {
        testCounter++;
    }

    @AfterClass
    public static void assertAnnotatedTestRunsMultipleTimes() {
        assertThat(testCounter, is(equalTo(20)));
    }

}

Next, JUnit Integration: Running Tests in Parallel »