Unit testing best practices

[nims0c]

What is the best practice for testing instance methods: 

(i) create a new object within each test method, or

(ii) create a single object, assign it to a field in the test class, and reference it in various test methods?

public class MyClass {
  public MyClass() {
  }
  
  public String method1() {
    //...
  }
  
  public int method2 () {
    //...
  }
}

Option (i)

@Test
public class MyClassTest {
  @Test
  public void testMethod1() {
    MyClass obj1 = new MyClass();
    String s = obj1.method1;
    //...
  }
  
  @Test
  public void testMethod2() {
    MyClass obj2 = new MyClass();
    int i = obj2.method2;
    //...
  }
}

Option (ii)

@Test
public class MyClassTest {
  MyClass myClass = new MyClass();

  @Test
  public void testMethod1() {
    String s = myClass.method1;
    //...
  }
  
  @Test
  public void testMethod2() {
    int i = myClass.method2;
    //...
  }
}

Furthermore, I use TestNG and was wondering when it's necessary to use @BeforeClass? I know it makes the annotated method run before the first test method. Is this necessary in the Option (ii) example so that the object is created before any tests are executed?

[clegger]

I know you are asking about best practices, but I'll spin the question a little and ask if you are expecting any side effects?

In this case a side effect would be previous calls to method1 or method2 causing subsequent calls to these methods to return different results.  Ideally your code would not have such side effects, but if they did then you should be testing a cross-section of calls (using one instance of the object) to ensure the side effects behaved as expected.

 

When my team writes test classes we try to group them into the concept of a scenario.  One scenario = one test class.  Usually one instance of an object for the class under test is used for a scenario.  We may have multiple scenarios exercising one class so there may be multiple test classes for each class.

 

The scenarios help us create meaningful names for our test classes and test methods, but sometimes they get really wordy.

[madknight3]

While I'd probably go with option 1 regardless, it's probably fine to do option two if things are immutable and don't have any side effects. I'm of the opinion that you shouldn't have separate test methods depending on each other as it can lead to problems. So if you need to test some mutable state/side effects do so in a single test method, not across multiple test methods.

 

The few unit test frameworks I've seen/used have Setup/Teardown methods. You can take advantage of those if they are available, or you can manually do it in each test. This way you make sure the state from one test won't effect another. This same idea can be used when dealing with other things that need cleanup (databases, file systems, etc) that you may have to write automated tests for.

[Nuluvius]

It depends upon the type of testing really.

 

Generally when considering Test Driven Development (TDD); where you write your failing test, you implement to make it pass and then you refactor, it's best to do as little driving to set things up as possible. In other words you want to keep your state unpolluted and everything as decoupled as possible. This lends itself better to a very granular setup and teardown approach. Which is the @BeforeMethod and @AfterMethod in the context of your particular testing framework. If you find that you are having to drive things out to get to the point that you want to test then you are not doing it right and should step back and think about your design.

 

It's also best to follow the Ports and Adapters Pattern/Hexagonal Architecture. As a very brief overview of this, consider that you have a simple system with some public API and some back end database. Now consider that you are wiring tests for every single thing that you are implementing. You'll have hundreds of tests (and mocks) in hardly no time at all. What happens when you change something? You'll end up with a massive overhead of having to chase down lots of tests/mocks that are now broken, some of which probably won't even mean anything in the new context, and spend time understanding and fixing them. This is developer testing or 'driving in a low gear', in other words you have coupled your tests to your implementation details and simply put coupling kills! Moreover it's bloody slow.

 

The better approach is to say something like: 'Right I have my system but what are the interfaces - the Ports?' Well those are the public API and the interface that sits between it and the database. The database and whatever uses the public facing API are therefore the Adapters - we test/mock here instead - end to end and thus we are 'driving in high gear'. It's much faster but we do end up with less tests and mocks however they are now completely decoupled from our implementation details altogether. Our tests are more meaningful, robust but more importantly they serve to validate the behaviour irrespective of how we shift about the internals - and if you really think about it... Why would we even need to test our internals in any event? They weren't ever meant to be on show to the public anyway!

 

Any time you have something that interacts with the outside it is usually identified as a Port - whether that's writing to a file, a database or reading user input. Any of the Adaptors on the Ports can be changed for tests/mocks - our database, file or the user who is banging away at their keyboard. Of course there's more to think about as is touched on in that link - further layers to separate our system into.

 

If you are integration testing then you'd likely be using the other variations of configuration since you'd be specifically after testing how the system performs when it's assembled (full or partial coupling) and functioning (acting under different behaviours) - here's the better context for going after the side effects.

 

Ultimately I think that you need to have a think about drawing a distinction between the types of testing that you are interested in carrying out.