Using the WaitFor
class along with Web Driver allows you to do the former with something like this.
1 2 3 4 5 6 |
|
This creates an implicit assertion. If the Condition
returns true
after any number of attempts, the test code continues without incident. If, however, it doesn’t find the value within the timeout, a TimeoutException
will be thrown failing your test.
In tempus-fugit 1.2, there’s a new method on WaitFor
to convert the TimeoutException
above into a JUnit failure.
1 2 3 |
|
Combine it with Conditions.assertion
to periodically query the UI and compare it against a Hamcrest Matcher
. For example,
1
|
|
where, getResultFrom(ui)
returns an instance of ProbeFor<T>
(a self describing Callable
used to query the UI for a T
).
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
Now rather than fail with a TimeoutException
, if the assertion hasn’t matched before the timeout expires, a regular JUnit failure (AssertionError
) will be thrown. In the IDE, it’ll look something like this.
java.lang.AssertionError: 'number of results' field in the UI
Expected: is "100 matches"
but: <was "">, <was "">, <was "">, <was "">, <was "200 matches">
The describeTo
method describes the probe and the Matcher
describes what was expected. Because the UI was polled several times, each value was displayed in the ‘but was’ section. On the fifth attempt, a value was found but it didn’t match the expectation.
For CPU bound tasks, Goetz (2002, 2006.) recommends
threads = number of CPUs + 1
Which is intuitive as if a CPU is being kept busy, we can’t do more work than the number of CPUs. Goetz purports that the additional CPU has been shown as an improvement over omitting it (2006.), but others don’t agree and suggest the number of CPUs is optimal.
Working out the optimal number for IO bound tasks is less obvious. During an IO bound task, a CPU will be left idle (waiting or blocking). This idle time can be better used in initiating another IO bound request.
Subramaniam (2011, p.31) describes the optimal number of threads in terms of the following formula.
threads = number of cores / (1 – blocking coefficient)
And Goetz (2002) describes the optimal number of threads in terms of the following.
threads = number of cores * (1 + wait time / service time)
Where we can think of wait time / service
time as a measure of how contended the task is.
Just out of interest, we can show that both IO bound formulas are equivalent. Starting with Goetz’s formula, we assert that w+s=1
and remove the service time (s
) giving the following
We can continue by multiplying both sides by 1-w
reducing the right hand side to c
before reversing the operation and revealing Subramaniam’s expression.
Thanks to Jazzatola for mathematical input.
InterruptedException
s?
1 2 3 4 5 |
|
You could rethrow if it’s appropriate but often its not. If that’s the case, you should set the interrupt status flag associated with the current thread. For example,
1 2 3 4 5 |
|
Typically, if a Java method throws InterruptedException
, it will have reset the interrupt status before doing so. In the example above, you’re just restoring it and preserving a flag to indicate the thread had once been interrupted.
The reason for preserving this status flag is in case code other than your own depends on it. For example, a framework (which you may not appreciate is actually being used), may be doing something like the following to (correctly) support interruption.
1 2 3 |
|
You should generally never just catch the exception and ignore it; it may cripple code depending on the status flag for correct behaviour. In the same way, it’s rarely a good idea to catch and exception and just log it, especially in the case of InterruptedException
.
Mockery
warns if the mockery is being used by multiple threads. The SingleThreadedPolicy
will output the following.
2012-05-31 07:35:35 ERROR Finalizer [Console$Logger] - the Mockery is not thread-safe: use a Synchroniser to ensure thread safety
If you really need multi-threaded access to the mockery, it’s a straight forward fix to swap the policy out. As in the log line above though, sometimes the JVM’s finaliser thread sticks it’s oar in and confuses the SingleThreadedPolicy
.
To get rid of this, you can set a custom threading policy that performs the same check as the default, just not when the finaliser thread is involved.
1 2 3 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
|
See the bug report JMOCK-256 for more details.
]]>Synchroniser
serialises access to the mock object’s “context”, it means all invocations of mocked methods call will be synchronized
on the same monitor, effectively forcing them to run in sequence without thread safety concerns. As it uses synchronized
though, you can (with some effort) get into trouble with tests that never finish.
If you’re seeing this kind of thing, apart from using the @Test(timeout=1000)
annotation, you might consider an alternative ThreadingPolicy
implementation using Lock
s that can timeout and maintain liveliness.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 |
|
Mockery
from multiple threads. Happily, since JMock 2.6.0, you can set a threading policy per mockery.
1 2 3 |
|
The Synchroniser
forces serialisation of each mocked method call using synchronized
. Use it when you’re running multi-threaded style tests using JMock. The default behaviour will warn you if a mockery is being used like this.
the Mockery is not thread-safe: use a Synchroniser to ensure thread safety
]]>DeadlockDetector
class allows you to programmatically detect basic deadlocks in your Java code. You can output deadlocks using the following code (note that printing a thread dump using the ThreadDump
class will automatically attempt to find any deadlocks).
1
|
|
The DeadlockDecector
class will spot Java monitor cyclic locking problems as well as Lock
based cyclic problems. It’s implementation is basically the same as that used by jconsole
and jstack
. The types of deadlock it can detect can be illustrated in the example below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
|
Here, the Kidnapper is unwilling to release poor Nibbles the Cat until he has the Cash but our Negotiator is unwilling to part with the Cash until he has poor Nibbles. The deadlock detector displays this situation as follows.
Deadlock detected
=================
"Negotiator-Thread-1":
waiting to lock Monitor of ...DeadlockDetectorTest$Cat@ce4a8a
which is held by "Kidnapper-Thread-0"
"Kidnapper-Thread-0":
waiting to lock Monitor of ...DeadlockDetectorTest$Cash@7fc8b2
which is held by "Negotiator-Thread-1"
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
|
Where keep
, take
and release
methods are pedagogically named methods wrapping the Lock.lock
and Lock.unlock
methods.
1 2 3 4 5 6 7 8 9 10 11 |
|
Same scenario as before, a deadlock ensues which is shown as.
Deadlock detected
=================
"Negotiator-Thread-3":
waiting to lock Monitor of java.util.concurrent.locks.ReentrantLock$NonfairSync@266bade9
which is held by "Kidnapper-Thread-2"
"Kidnapper-Thread-2":
waiting to lock Monitor of java.util.concurrent.locks.ReentrantLock$NonfairSync@6766afb3
which is held by "Negotiator-Thread-3"
]]>GOOS amongst other people recommend separating the concurrency policy from the parts of the system that are doing the work. So, for example, if you have some form of “executor” which is responsible for farming work out concurrently and behaviour defined separately, you can test each independently and just verify that they collaborate to achieve “worker” behaviour concurrently. Make sense?
As an example, this test from tempus-fugit demonstrates the idea. The Scheduler
’s behaviour (which is essentially to “schedule” tasks) is independent from how it actually achieves this. In this case, it delegates to an Executor
and so this doesn’t need to be tested with any threads. It’s a simple collaborator style test.
Having said that, there may be times you actually want to run your class ‘in context’ in a multi-threaded way. The trick here is to keep the test deterministic. Well, I say that, there’s a couple of choices…
If you can setup your test to progress in a deterministic way, waiting at key points for conditions to be met before moving forward, you can try to simulate a specific process interleaving to test. This means understanding exactly what you want to test (for example, forcing the code into a deadlock) and stepping through deterministically (for example, using abstractions like CountdownLatch
to synchronise the moving parts).
When you attempt to make some multi-threaded test syncrhonise its moving parts, you can use whatever concurrency abstraction is available to you but it’s difficult because its concurrent; things could happen in an unexpected order. Often people try to mitigate this in tests by introducing sleep
calls. We generally don’t like to sleep in a test because it can introduce non-determinism. Just because the right sleep amount on one machine usually causes the affect you’re looking for, it doesn’t mean it’ll be the same on the next machine. It’ll also make the test run slower and when you’ve got thousands of tests to run, every ms counts. If you try and lower the sleep period, more non-determinism comes in. It’s not pretty.
Some examples of forcing specific interleaving include
CountdownLatch
Another gotcha is where the main test thread will finish before any newly spawned threads under test complete. This is an easy trap to fall into with UI testing. Waiting for a specific condition rather than allowing the test thread to finish often helps. For example using WaitFor. See the article Be Explicit with the UI Thread for more details around this for UI testing.
Another choice is to bombard your classes in an attempt to overload them and force them to betray some subtle concurrency issue. Here, just as in the other style, you’ll need to setup up specific assertions so that you can tell if and when the classes betray themselves. Of course there is no guarantee that you’ll simulate a problem, you might never see the unlike timing needed.
The tempus-fugit library offers a declarative way to setup tests to run repeatedly and in parallel, see Load / Soak Tests.
]]>