Model-Based Testing and Validation with Reactis®

2  Reactis Tester

As shown in Figure 2, Reactis Tester offers automatic test generation from C programs. The generated test suites provide comprehensive coverage of different test coverage metrics - including the Modified Condition/Decision Coverage (MC/DC) test coverage measure mandated by the US Federal Aviation Administration (FAA) in its DO-178/B guidelines - while at the same time minimizing redundancy in tests. Each test case in a test suite consists of a sequence of inputs fed into the model as well as the responses to those inputs generated by the model.

The automatically-generated test data may then be used for a variety of purposes, including the following:

Finding runtime errors.

The tests help uncover runtime errors in models, including memory errors, overflows and divide-by-zero errors.

Regression testing.

When a program is modified, tests generated from the older version may be run on the newer version to understand the impact of the changes on program behavior.

Reactis Tester enables engineers to maximize the effectiveness of testing while reducing the time actually spent on testing.

---

Figure 2: Reactis Tester automatically generates comprehensive yet compact test suites.

---

The embedded software applications that Reactis for C initially targets typically operate as follows:

1. Read one or more input values from environment.
2. Perform internal calculations.
3. Update outputs to control the environment in some way.
4. Repeat.

To support this framework, Reactis for C lets the user specify an entry function. An entry function is a C function which acts as interface between the application and the test environment. Each argument of the entry function acts as either an input or an output. To test a program Reactis for C repeatedly does the following:

1. Calculate a set of inputs for the entry function.
2. Invoke the entry function with the selected inputs as arguments.
3. Record the inputs fed into and the outputs produced by the entry function.
4. Repeat.

The structure of a Tester-generated test is shown in Figure 3. A test may be viewed as a matrix in which each row corresponds to either an input or output to the entry function and each column represents a simulation step. As shown in Figure 4, a test suite consists of a set of tests. When running a test suite, the model is reset to its initial state after one test completes and before the next test begins.

---

Figure 3: Structure of a Reactis-generated test.

---

---

Figure 4: Structure of a Reactis-generated test suite.

---

Test suites are constructed by simulating a program and recording the input and output values at each step. The program computes the outputs at each step, but several approaches are possible for selecting the input values to drive simulation. The input data could be captured during field testing or constructed manually by an engineer, but these are expensive tasks. Alternatively, the inputs could be generated randomly; however, this approach yields tests with poor coverage.

Reactis Tester employs a novel, patented approach called guided simulation to generate quality input data automatically. The idea behind this approach is to use algorithms and heuristics to automatically generate inputs that cause coverage targets (i.e. program elements that the user wants to ensure are executed at least once) that have not yet been covered to be executed. Reactis currently allows users to track several different classes of coverage targets (also called coverage criteria or coverage metrics).

Reactis tracks C Statements, Decisions, Conditions, and MC/DC targets. Decisions are boolean-valued expressions used to determine which execution path to follow. Conditions are atomic predicates from which decisions are constructed. Modified Condition/Decision Coverage (MC/DC) targets track whether each condition in a decision independently effects the outcome of the decision.