QS Part 1 - A New Approach to Unit and Integration Testing

Introduction

Unit test frameworks have been available for the last decade and provide a fairly uniform way of representing tests and have a large number of features in common. This can hardly surprise as these frameworks have been built to support the same concept of unit testing. Enter Quality Gate One Studio (QS) which has been built with integration testing in mind, including performance testing and usage model testing and aims to solve a very different problem.

Background

Wikipedia gives the following definitions of unit and integration testing:

Unit testing is a method by which individual units of source code are tested to determine if they are fit for use. A unit is the smallest testable part of an application. In procedural programming a unit may be an individual function or procedure [...] Ideally, each test case is independent from the others: substitutes like method stubs, mock objects, fakes and test harnesses can be used to assist testing a module in isolation.

The purpose of integration testing is to verify functional, performance, and reliability requirements placed on major design items. These "design items", i.e. assemblages (or groups of units), are exercised through their interfaces using Black box testing, success and error cases being simulated via appropriate parameter and data inputs.

While unit testing is about testing individual methods and classes and using stubs and mocks to isolate test cases from the surroundings and from each other (and frequently used in a white box approach), integration testing is a black box approach with a focus on input data. The Wikipedia article has a structural perspective on integration test but I think it is important to add the perspective of state as well and how this requires tests to be careful about the order of actions against the system under test.

The QS Approach

While QS uses the same notions as other test frameworks such as test classes, test methods, setup/teardown and asserts in the code, it differs fundamentally from unit testing frameworks in other areas:

Test methods may take any number of parameters, but unlike data driven extensions to unit frameworks, it doesn’t use data sources to feed parameter values.
Test methods can emit test data back to QS.
Test data must classified by one or more classification properties (classification properties have discrete types, in practice either Booleans or enums).
A test method is no longer the fundamental unit of a test, but must be coupled with an expression over its parameters. This expression defines one or more test conditions, which roughly amount to the preconditions that must be satisfied by the parameters of the test method. This makes the generated “test conditions” the fundamental units of the test.
QS takes an active part in managing test data (parameters and output) both by storing output and figuring out with which test conditions it can be used, but also, if a parameter value is not available for a test condition that must be run, by working its way backwards to figure out which test conditions it must execute to get the necessary test data.
The management and flow of test data is very powerful but I will leave that for the next article and move on with a simple example illustrating classifications, expressions and conditions and conclude it with a performance measurement.

Testing the Modulus Operator

I decided to start out with something simple and settled on the modulus operator ‘%’ applied to the built in types in .NET. For that we need some numbers while QS insists on classifications, so we need to provide some (equivalence) partitioning of the range of numbers available for the different types.

Read more: Codeproject

Posted via email from .NET Info