Model-Based Testing and Validation with Reactis®

Reactis Tester automatically generates test suites from models. The test suites provide comprehensive coverage of different test-quality metrics---including the Modified Condition/Decision Coverage (MC/DC) test coverage measure mandated by the US Federal Aviation Administration (FAA) in its DO-178/B guidelines---while at the same time minimizing redundancy in tests. Each test case in a test suite consists of a sequence of inputs fed into the model as well as the responses to those inputs generated by the model. These tests may then be used for a variety of purposes, including the following.

Implementation conformance.

The tests may be applied to source-code implementations of models to ensure conformance with model behavior.

Model testing and debugging.

The tests may be run on the models themselves to study and revise model behavior.

Reverse engineering of models from source.

Tests may be generated from models derived from legacy code in order to check conformance between model and code.

Test generation with Reactis Tester enables engineers to maximize the effectiveness of testing while reducing the time actually spent on testing. Tester takes a Simulink model as input and generates a test suite.

The structure of Tester-generated test suites is shown below. A test may be viewed as a matrix in which each row corresponds to either an inport or outport and each column represents a simulation step.

Test suites are constructed by simulating a model and recording the input and output values at each step. The model computes the outputs at each step, but several approaches are possible for selecting the input values to drive simulation. The input data could be captured during field testing or constructed manually by an engineer, but these are expensive tasks. Alternatively, the inputs could be generated randomly; however, this approach yields tests with poor coverage.

The Reactis Tester test generation engine employs a novel approach called guided simulation to generate quality input data automatically. The idea behind this approach is to use algorithms and heuristics to automatically generate inputs that cause coverage targets (i.e. model elements that the user wants to ensure are executed at least once) that have not yet been covered to be executed. Reactis currently allows users to track several different classes of targets, or coverage criteria. Two of the criteria supported by Reactis involve only Simulink, three are specific to Stateflow, and the remaining are generic in the sense that they include targets within both the Simulink and the Stateflow portions of a model.

Simulink-specific:

Conditional subsystems. Branches of the following blocks: Dead Zone, Logical Operator, MinMax, Multiport Switch, Relational Operator, Saturation, Switch. Lookup Table blocks.

Stateflow-specific

States. Condition actions. Transition actions.

Generic

Decisions from logic blocks in Simulink or transition segments in Stateflow. Conditions (the atomic predicates that are the building blocks of decisions). Modified Condition/Decision Coverage (MC/DC) targets. Boundary values.