I’ve been using jMock for more than a year now, but recently I came across mockito. In this post, I’ll show the difference between jMock and mockito – without judging which one is better – using the basic features I’ve been using the most often from jMock. I admit that there are more features than presented in this post, but for starters, I find these enough.
The versions under comparison:
The Test Object
For this demonstration, I’m using a very fictional, but simple code example which fits the purpose of comparison.
The skeleton of the main class which is going to be tested:
public class PosTerminal { private static final String SHOP_ACCOUNT_NUMBER = "00000000-00000000"; private final ReceiptPrinter receiptPrinter; private final BankConnection bankConnection; public PosTerminal(BankConnection bankConnection, ReceiptPrinter receiptPrinter) { this.bankConnection = bankConnection; this.receiptPrinter = receiptPrinter; } public boolean buyWithCard(int amountOfMoney, String cardNumber) { } }
The skeleton of the first injected class:
public class BankConnection { public Account getAccountByCardNumber(String cardNumber) { return null; } public Account getAccountByAccountNumber(String accountNumber) { return null; } }
The skeleton of the second injected class:
public class ReceiptPrinter { public void print(String string) { } }
And the interface for accounts:
public interface Account { boolean withdraw(int amountOfMoney, String destinationAccountNumber); boolean enter(int amountOfMoney, String sourceAccountNumber); void rollback(); void commit(); }
Creating Mocks
jMock
import org.jmock.Mockery; import org.junit.Test; public class PosTerminalTest { @Test public void shouldPrintSuccessfulWithdrawal() { Mockery context = new Mockery(); BankConnection bankConnection = context.mock(BankConnection.class); ReceiptPrinter receiptPrinter = context.mock(ReceiptPrinter.class); // the test case continues... } }
mockito
import static org.mockito.Mockito.mock; import org.junit.Test; public class PosTerminalTest { @Test public void shouldPrintSuccessfulWithdrawal() { BankConnection bankConnection = mock(BankConnection.class); ReceiptPrinter receiptPrinter = mock(ReceiptPrinter.class); // the test case continues... } }
There is no significant difference here. jMock uses a context for handling mocks, while mockito solves it with statically imported methods.
Mocking Classes
If you compile and execute the examples above, the following exception is raised when the jMock test case is executed:
java.lang.IllegalArgumentException: com.zsoltfabok.dojo.mocks.BankConnection is not an interface
jMock is designed to mock interfaces, not classes. It forces the developer to use interfaces, which is a good programming practice, but sometimes it is an overhead. Imagine that you are working on a legacy code. There is no way to extract interfaces for every single class just to satisfy a mocking framework. Fortunately, you can set jMock so that it can mock classes:
// the same as before ... Mockery context = new Mockery() {{ setImposteriser(ClassImposteriser.INSTANCE); }}; // the same as before ...
There is no need for such setting in the mockito based test case.
Two Of The Same Kind of Mocks
jMock
// the same as before ... Account customerAccount = context.mock(Account.class); Account shopAccount = context.mock(Account.class); // the same as before ...
mockito
// the same as before ... Account customerAccount = mock(Account.class); Account shopAccount = mock(Account.class); // the same as before ...
No difference here at first sight. After executing the jMock test case, the following exception is raised:
java.lang.IllegalArgumentException: a mock with name account already exists
Internally, jMock uses name references for handling mocks, and if no name is specified, the name will be the type of the mock. In the example above, we have two Accounts with the same name. The problem can be solved easily:
// the same as before ... Account customerAccount = context.mock(Account.class, "customer account"); Account shopAccount = context.mock(Account.class, "shop account"); // the same as before ...
Annotation
Based on the mockito and jMock documentation it is possible to setup mocks with annotations, which is excellent, because with this approach, I can make the test case much simpler.
jMock
Although the current stable jMock documentation describes the @Mock annotation, I was unable to find it in the framework.
mockito
import org.mockito.Mock; // Other import statements ... @RunWith(MockitoJUnitRunner.class) public class PosTerminalTest { @Mock private BankConnection bankConnection; @Mock private ReceiptPrinter receiptPrinter; @Mock private Account customerAccount; @Mock private Account shopAccount; @Test public void shouldPrintSuccessfulWithdrawal() { } }
Note that in order to make it work, you need a @RunWith annotation.
Call Verification and Return Value Handling
This is the good part; how our class communicates with the injected classes. The verification is quite simple: check whether or not a method of the mock is called. The other thing is how our class can handle the return value of a method of a mock.
jMock
@Test
public void shouldPrintSuccessfulWithdrawal() {
PosTerminal posTerminal = new PosTerminal(bankConnection, receiptPrinter);
context.checking(new Expectations() {{
oneOf(bankConnection).getAccountByCardNumber("1000-0000-0001-0003");
will(returnValue(customerAccount));
// Other oneOf() will() statements ...
oneOf(receiptPrinter).print("Successful withdrawal");
}});
posTerminal.buyWithCard(100, "1000-0000-0001-0003");
}mockito
@Test
public void shouldPrintSuccessfulWithdrawal() {
PosTerminal posTerminal = new PosTerminal(bankConnection, receiptPrinter);
when(bankConnection.getAccountByCardNumber("1000-0000-0001-0003"))
.thenReturn(customerAccount);// Other when(...).thenReturn(...) calls ... posTerminal.buyWithCard(100, "1000-0000-0001-0003"); verify(receiptPrinter).print("Successful withdrawal"); }
The order in which the methods are called is important. In case of jMock, the Expectations shall come before the method call, and in case of mockito the verify() always afterwards. There is no significant difference between jMock and mockito in this case, besides readability. The verify() method call clearly states that the we are just checking if a method is called. This clear difference makes the code more readable for me. Another readability thing is that for me, the inner class Expectations is less readable. If the mocks are not fields, then, because of the inner class, I have to make the mocks final.
Argument Matchers
Sometimes one or more arguments are not relevant when a method of a mock object is called.
jMock
context.checking(new Expectations() {{ // Other oneOf() will() statements ... oneOf(customerAccount).withdraw(with(any(Integer.class)), with(any(String.class))); will(returnValue(true)); oneOf(shopAccount).enter(with(100), with(any(String.class))); will(returnValue(true)); // Other oneOf() will() statements ... }});
mockito
import static org.mockito.Matchers.anyInt; import static org.mockito.Matchers.anyString; import static org.mockito.Matchers.eq; // Other parts of the code ... @Test public void shouldPrintSuccessfulWithdrawal() { // Other when(...).thenReturn(...) calls ... when(customerAccount.withdraw(anyInt(), anyString())).thenReturn(true); when(shopAccount.enter(eq(100), anyString())).thenReturn(true); // Other when(...).thenReturn(...) and verify(...) calls ... }
The are no differences here, but I find the mockito version more readable. One additional thing is that neither of the frameworks support mixing the matchers with exact matches, for example:
jMock
Mind the missing with() call:
context.checking(new Expectations() {{ // Other oneOf() will() statements ... oneOf(shopAccount).enter(100, with(any(String.class))); will(returnValue(true)); // Other oneOf() will() statements ... }});
The following exception will be raised during execution:
java.lang.IllegalArgumentException: not all parameters were given explicit matchers: either all parameters must be specified by matchers or all must be specified by values, you cannot mix matchers and values
mockito
Mind the missing eq() call:
// Other when(...).thenReturn(...) calls ... when(shopAccount.enter(100, anyString())).thenReturn(true); // Other when(...).thenReturn(...) and verify(...) calls ...
The following exception will be raised during execution:
org.mockito.exceptions.misusing.InvalidUseOfMatchersException:
Invalid use of argument matchers!
2 matchers expected, 1 recorded.
This exception may occur if matchers are combined with raw values:
//incorrect:
someMethod(anyObject(), “raw String”);
When using matchers, all arguments have to be provided by matchers.
For example:
//correct:
someMethod(anyObject(), eq(“String by matcher”));
org.mockito.exceptions.misusing.InvalidUseOfMatchersException: Invalid use of argument matchers!2 matchers expected, 1 recorded.This exception may occur if matchers are combined with raw values: //incorrect: someMethod(anyObject(), “raw String”);When using matchers, all arguments have to be provided by matchers.For example: //correct: someMethod(anyObject(), eq(“String by matcher”));
The output of mockito provides a bit more information.
Number of Invocations
Sometimes the number of invocations of a certain method is important. The approaches of jMock and mockito are quite the same, except that the number of invocations is used with the verify() method in mockito.
jMock
context.checking(new Expectations() {{ // Other oneOf() will() statements ... exactly(1).of(receiptPrinter).print(with(any(String.class))); }});
or
context.checking(new Expectations() {{ // Other oneOf() will() statements ... atLeast(1).of(receiptPrinter).print(with(any(String.class))); }});
or
context.checking(new Expectations() {{ // Other oneOf() will() statements ... never(receiptPrinter).print(with(any(String.class))); }});
mockito
// Other when(...).thenReturn(...) calls ... verify(receiptPrinter, times(1)).print(anyString()); // Other verify() calls ...
or
// Other when(...).thenReturn(...) calls ... verify(receiptPrinter, atLeast(1)).print(anyString()); // Other verify() calls ...
or
// Other when(...).thenReturn(...) calls ... verify(receiptPrinter, never()).print(anyString()); // Other verify() calls ...
There are plenty of other possibilities. jMock has allowing() and ignoring() options, which are not available in mockito. On the other hand, mockito has an only() method which makes the code more readable. In jMock, if an unexpected method call is made, a RuntimeException is thrown. In mockito this feature is not available. There is an issue for it with number 162.
Consecutive Calls
jMock
context.checking(new Expectations() {{ // Other oneOf() will() statements ... oneOf(shopAccount).enter(with(100), with(any(String.class))); will(onConsecutiveCalls(returnValue(true), returnValue(false), returnValue(false))); }});
mockito
// Other when(...).thenReturn(...) calls ... when(shopAccount.enter(eq(100), anyString())).thenReturn(true, false, false);
No difference, but I find mockito more readable.
Verify Call Order
jMock
@Test
public void shouldPrintSuccessfulWithdrawal() {
PosTerminal posTerminal = new PosTerminal(bankConnection, receiptPrinter);
final Sequence sequence = context.sequence("account order");
context.checking(new Expectations() {{
// Other oneOf() will() statements ...
oneOf(customerAccount).withdraw(with(any(Integer.class)),
with(any(String.class)));
will(returnValue(true));
inSequence(sequence);
atLeast(1).of(shopAccount).enter(with(100), with(any(String.class)));
will(returnValue(true));
inSequence(sequence);
}});
posTerminal.buyWithCard(100, "1000-0000-0001-0003");
}mockito
@Test
public void shouldPrintSuccessfulWithdrawal() {
PosTerminal posTerminal = new PosTerminal(bankConnection, receiptPrinter);
posTerminal.buyWithCard(100, "1000-0000-0001-0003");
InOrder inOrder = Mockito.inOrder(customerAccount, shopAccount);
inOrder.verify(customerAccount).withdraw(anyInt(), anyString());
inOrder.verify(shopAccount).enter(anyInt(), anyString());
verify(receiptPrinter).print(anyString());
}One significant difference here: in case of mockito the involved mocks need to be injected when creating the InOrder (Sequence equivalent) object.
Exception Handling
jMock
context.checking(new Expectations() {{ oneOf(bankConnection).getAccountByCardNumber("1000-0000-0001-0003"); will(throwException(new RuntimeException())); }});
mockito
when(bankConnection.getAccountByCardNumber("1000-0000-0001-0003")) .thenThrow(new RuntimeException());
No difference, other than readability.
Setup
In order to make the examples compile and run I had to add the following external libraries.
jMock
- jmock-2.5.1.jar
- jmock-junit4-2.5.1.jar
- hamcrest-core-1.1.jar
- hamcrest-library-1.1.jar
- jmock-legacy-2.5.1.jar
- cglib-2.1_3-src.jar
- cglib-nodep-2.1_3.jar
- objenesis-1.0.jar
mockito
- mockito-all-1.8.5.jar
Some of the jMock libraries were added after I tried to run my test cases and it took me quite some time to find out the right build path. With mockito, I just had to add one simple jar file.
Conclusion
jMock and mockito are both very well documented with examples and good practical hints. However, based on the examples above I find the test cases more readable with mockito. In my next project I’m definitely going to work with mockito. One good advice: if you make the same decision, do not mix jMock and mockito in your test code. It will make your code unreadable, and additionally it might not even work: it would be hard to execute a mockito mock related code in a jMock TestRunner (mind the @RunWith annotations).
Isn’t the writer biased a little bit towards Mockito?
I personally like the concept of jMock, namely that the method call and the answer -return value- provided to it are separated into two statements… I get far more easier what the test is all about this way… Probably I’m biased against Mockito.
Yes, I’m bit biased towards mockito, and only because of readability. I’m always looking for readable code and this stands for test code as well.
What I don’t like about it is that the number of invocations go somewhere inside the statement. In the other case it is right at the beginning, so one can catch up more easily with the execution flow… But anyway, annotations based mocking sounds promising. I’ve got to try it.
java.lang.IllegalArgumentException: a mock with name account already existsThis post solved my issue. Thank you.
Your welcome, I’m glad that my post was helpful
The article starts with …”without judging which one is better”…and ends with …”In my next project I’m definitely going to work with mockito“. I smell an inconsistency here? Anyway, really good article especially while studying for interviews.
Tamas, those seem like very minor gripes in the scheme of things, but as you mentioned, we all have our biases. I have found that syntax is almost always cleaner in Mockito. Zsolt has already mentioned many of the other positives. I, like Zsolt, had been using JMock in a large enterprise project exclusively for quite a while. Also like Zsolt, we are now switching to Mockito because of two main differentiators. 1) Mockito separates stubbing from verification. In JMock, they are all considered “expectations”. This often makes testing code cluttered and unclear. 2) Mockito allows me to stub and verify only the methods I find significant. JMock forces me to write expectations for all methods that are called on a mocked object. My unit tests are now much more concise and less verbose.
I disagress with Brian on 2nd statement. I am using Jmock 2.5.1. You dont need to mock the methods , if thats not being invoked. It does not mean that I am Jmock fan.
I have a question as well.
Does mockito supports to all the versions of Junit?
How the versioning related with Junit and mockito?
Hi Kasi, I don’t think that the mockito versioning has any connection to the Junit versioning. There are completely different products. I didn’t find any evidence why mockito cannot work with different Junit version.
Interesting Article, Thanks. One question:
It looks like the following syntax is permissable in JMock:
oneOf(shopAccount).enter(with(100), with(any(String.class)));
will(onConsecutiveCalls(returnValue(true), returnValue(false),
throwException(new RuntimeException()));
Is this the case? If so, how can Mockito handle that with its thenReturn() and thenThrow() syntax?
Hi Andrew,
One way of doing this, however it won’t throw an exception after the second call:
when(bankConnection.getAccountByCardNumber(“1000-0000-0001-0003″)).thenReturn(customerAccount).thenThrow(new RuntimeException());
This worked, though:
when(bankConnection.getAccountByCardNumber(“1000-0000-0001-0003″)).thenReturn(customerAccount);
when(bankConnection.getAccountByCardNumber(“1000-0000-0001-0003″)).thenThrow(new RuntimeException());
It is hard to say why mockito behaves like this.
Are you sure that you want a test case like this? For me it looks like it tests at least two things.