Test Selection

• Failure: Observable difference in behavior, such as a crash or wrong output.

• Fault: Error in the code.

• Error: Incorrect state of the program.

• Unit tests: Test correct behavior in one module.

• Integration tests: Test subsystem and interaction between modules.

• System tests: Test the whole system.

• Acceptance tests: Check whether the software is usable.

• Big Bangs: Test all modules in isolation, then merge them together. Often leads to failure.

• Incremental: Test one module, then add another one, and continue adding modules incrementally. Modules not added yet may be simulated with mocks.

- C0 Statement Coverage: Each statement in the code must be executed at least once. (nodes)

- C1 Decision Coverage: Each decision (boolean expression) must evaluate to both true and false at least once. (result)

- C2 Condition Coverage: Each condition in a decision must evaluate to both true and false at least once. (each var)

- C3 Condition Decision Coverage: Each condition in a decision (C2) and the overall decision result must evaluate to both true and false at least once.

- C4 Modified Condition Decision Coverage (MCDC): All combinations of conditions and decision results must be tested.

- C5 Multiple Condition Coverage: All combinations of multiple conditions must be tested, but this can be infeasible for large numbers of conditions.

- C6 Path Coverage: All possible paths through the code must be executed, but this may not be practical due to potentially infinite paths.

- Selection Criteria: tests to include in test suite

- Quality Indicator: can indicate quality/thoroughness of test suite

- Stopping Criteria: determining when testing can be considered complete

- Focus: Data flow testing is concerned with where variables are assigned values and where those values are used in the code.

- Objective: The goal is to test different data flow paths to uncover potential data-related issues or bugs.

Testing combinations that cover all interactions between pairs of input parameters.

-> testing all combinations infeasable -> approximation with combinatorial testing

Purpose: Check requirements and find failures;

Issues:

• “Testing can never show the absence of errors, only their presence” – E.W. Dijkstra

• Test oracle problem (oracle not always clear)

• Test selection and stopping criteria hard

walkthroughs, reviews, inspections

can only be used after parts of testing have been implemented

bevor that: walkthroughs, reviews, …

➢ Path coverage ≻ boundary-interior coverage ≻ branch coverage ≻ statement coverage

➢ Multiple condition coverage ≻ MC/DC ≻ C/D coverage ≻ condition coverage

group possible input data into blocks of a partition, or “equivalence classes”

-> Classification Tree

Today: based on requirements -> no automation in general

• Failure: Observable difference in behavior between actual and intended behaviour

• Error: Deviation of system’s actual and indended state (incorrect internal state)

• Fault: Actual/Hypothesized reason for deviation