2 Reactis Tester
Reactis Tester dramatically reduces the time and effort required
to generate comprehensive test suites.
As shown in Figure 2, Reactis Tester automatically
generates tests for C programs. The test suites generated by Tester
provide comprehensive coverage of a variety of test coverage metrics,
including the
Modified Condition/Decision Coverage (MC/DC) test coverage
mandated by the US Federal Aviation Administration (FAA) in its DO-178/B
guidelines.
In addition to maximizing coverage, Tester-generated
test suites are automatically minimized to eliminate redundancy,
saving testing time.
Each test case
in a test suite consists of a sequence of program inputs
and a sequence of program outputs generated in response to those inputs.
| Figure 2: Reactis Tester automatically generates
comprehensive yet compact test suites. |
The automatically-generated test data may then be used for a variety of
purposes, including the following:
- Finding runtime errors.
- The tests help uncover runtime errors
in C programs, including memory errors, overflows, and divide-by-zero errors.
- Regression testing.
- When a program is modified, tests generated
from the older version may be run on the newer version to understand
the impact of the changes on program behavior.
Reactis Tester enables engineers to maximize the effectiveness of testing while
reducing the time actually spent on testing.
| Figure 3: A reactive control system. |
The embedded software applications that Reactis for C initially targets
are reactive control systems,
which typically operate as depicted in Figure 3.
Such systems use a control loop to interact with their external environment,
which consists of the following steps:
-
Read one or more input values from environment.
- Perform internal calculations based on the input values and the internal system state.
- Update outputs to control the environment.
- Repeat.
To support this framework, Reactis for C lets you specify a test
harness consisting of an entry function, a set of inputs, and
a set of outputs.
The entry function is a C function which acts as interface between the
application and the test environment.
Each argument of the entry function acts as either an input or an output.
In addition, global variables can be selected for use as inputs or outputs.
When testing a program, Reactis for C repeatedly performs the following steps:
-
Calculate a set of inputs for the harness. Record the values of all inputs.
- Call the entry function with the selected inputs.
- When the entry function returns, record the values of all outputs.
| Figure 4: Structure of a Reactis for C-generated test. |
Figure 4 shows the structure of a Tester-generated test.
Tests are comprised of simulation steps.
Each step is a single call to the program entry function, including the
inputs (arguments passed to the entry function and global variables)
and outputs (value returned by the entry function, final values of
any arguments passed by reference and global variables).
A test can be represented by a matrix in which each row contains
the sequence of values for a specific input or output and
each column contains all the input and output values for a single step.
A test suite consists of a set of tests, as shown in Figure 5.
When a test suite is executed the program is reset to its initial state
before each test begins.
| Figure 5: Structure of a Reactis for C-generated test suite. |
Test suites are constructed by executing a program while recording the input and
output values at each step.
Outputs are computed by the program at each step, but
there are several ways in which the input values that drive the testing
can be chosen.
Input data could be manually constructed or recorded during field testing,
but these are expensive tasks. Alternatively,
input data can randomly generated.
Unfortunately the latter approach produces tests with poor coverage. Reactis Tester tracks several different classes of coverage targets.
A coverage target is a program element that should be exercised during testing.
A coverage criterion or coverage metric defines how to calculate
a set of coverage targets from the syntax of a program. For example,
the Statement Coverage metric identifies all statements in a
program as coverage targets, i.e. every statement should be executed
to satisfy Statement Coverage. The coverage criteria tracked by Tester include
Statement, Decision, Condition, and MC/DC coverage.
Decisions are boolean expressions which determine the path of program execution.
Conditions are atomic predicates contained within decisions.
Modified Condition/Decision Coverage (MC/DC) targets
are used to ensure that each condition independently effects the
outcome of the enclosing decision. Reactis Tester employs a novel, patented approach called guided
simulation to generate quality input data automatically. Guided
simulation uses algorithms and heuristics to automatically select
inputs which exercise coverage targets that have not yet been covered
during each simulation step.
| 
