BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News Categorizing Tests

Categorizing Tests

Leia em Português

This item in japanese

In a recent discussion on the test driven development Yahoo! group Carlos Ble shared his understanding of test categories after doing his research:

Eventually the picture I've got in my mind is:
Developer tests:
Unit tests : Isolated, atomic, innocuous: exercised with xUnit
Integration tests:
Isolated tests that might change the state of the system, i.e: saving into database, writing file... An integration test does not represent a functional requirement as is. Can be written for xUnit. They check the integration of our code with a third party tool or with the different layers of our own code, i.e: the business logic layer requires the data access layer
Functional tests: (also known as System tests)
A test that excercises part of the system as a whole, some functional requirement. It might change the status of the system.
Product Owner tests:
Acceptance tests: Functional tests which input and output can be validated by a non-technical person, the product owner.

 

John Donaldson shared a multi-dimensional model of tests that look at test roles and test types:

I like the test view you give. But I think it's an instance of a bigger model, where you have (at least) actor-roles and test-types.

Actor-roles: developer, tester, QA, user, sponsor, etc.

Test-types: unit, integration, functional, system, acceptance, soak, smoke, etc.

In any given situation you probably which role gets to do which test - but in the next project it may be different.

Dale Emery suggested that not knowing what kind of test you are writing is a code smell and indicates lack of clarity. At the same time, a test may be fall under several categories and what is important is your current point of view:

The challenge, I think, is that any given test could be reasonably classified in numerous ways, depending on what dimensions you're focusing on. And there are a lot of dimensions that people use to classify tests. I've identified some here: http://cwd.dhemery.com/2004/04/dimensions/

So I'm less interested in knowing "what kind" of test it is, and more interested in knowing where a given test falls along the dimensions that are most important to me, depending on my purposes at any given moment. The ones that I think about most often are:

  • What "unit" does this test define and test? (What system, subsystem, object, collaboration...) - What feature does the test define and test?
  • Who are this test's primary customers? Who most cares about the results of executing this test?
  • What decisions will be made based on the results of executing this test?

Charlie Poole gives a detailed analysis of Carlos' categorization and suggests:

In my opinion, the most important distinction is between tests of developer intent and tests of customer intent.

This discussion highlights the fact that test categorization can be very confusing - especially to the newcomer.  The majority opinion is that a dimensional approach to test categorization is the way to go and that the type of categorization that is relevant is dependent on the user's current intentions and context.

Rate this Article

Adoption
Style

BT