Black Box Strategy Testing

Black box testing or behavioral testing is a strategy (method) for testing the functional behavior of an object (program, system) from the point of view of the outside world, which does not use knowledge about the internal structure of the tested object. Strategy refers to systematic methods for selecting and creating tests for a test suite. The behavioral test strategy is based on technical requirements and their specifications ^[1] .

Content

1 The concept of "black" box
2 Study of the behavior of the "black" box
3 Principles of Black Box Testing
- 3.1 Properties of a correctly selected test
- 3.2 Black Box Testing Techniques
  - 3.2.1 Equivalent split
  - 3.2.2 Boundary value analysis
  - 3.2.3 Analysis of causal relationships
  - 3.2.4 Assumption of an error
4 notes
5 Literature

The concept of a black box

The "black box" refers to the object of study, the internal structure of which is unknown. The concept of a “black box” was proposed by U. R. Ashby . In cybernetics, it allows you to study the behavior of systems, that is, their reactions to a variety of external influences and at the same time abstract from their internal structure.

Manipulating only with inputs and outputs, you can conduct certain studies. In practice, the question always arises of how the black box homomorphism reflects the adequacy of its studied model, that is, how fully the basic properties of the original are reflected in the model.

The description of any control system in time is characterized by a picture of the sequence of its states in the process of moving toward its goal. The transformation in the control system can be either one-to-one and then it is called isomorphic, or only unambiguous, in one direction. In this case, the transformation is called homomorphic.

The “black” box is a complex homomorphic model of a cybernetic system in which diversity is respected. It is only then a satisfactory system model when it contains such an amount of information that reflects the diversity of the system. It can be assumed that the greater the number of perturbations acting on the inputs of the system model, the greater the variety the regulator should have.

Currently, two types of "black" boxes are known. The first type includes any “black” box, which can be considered as an automaton , called finite or infinite. The behavior of such "black" boxes is known. The second type includes such "black" boxes, whose behavior can be observed only in the experiment. In this case, a hypothesis is expressed explicitly or implicitly about the predictability of the behavior of the black box in a probabilistic sense. Without a preliminary hypothesis, any generalization is impossible, or, as they say, it is impossible to draw an inductive conclusion based on experiments with the black box. To designate the model of the “black” box, N. Wiener proposed the concept of a “white” box. The “white” box consists of known components, that is, known X, Y, δ, λ. Its contents are specially selected to implement the same dependence of the output on the input as the corresponding "black" box. In the process of research and generalizations, hypotheses and establishing patterns, it becomes necessary to adjust the organization of the “white” box and change models. In this regard, when modeling, the researcher must necessarily repeatedly refer to the scheme of relations “black” - “white” box.

The study of the behavior of the black box

Let us examine how the behavior of the “black” box of the second type is studied and investigated. Suppose that a certain control system is given whose internal structure is unknown. The control system has inputs $X(x_{1},x_{2},x_{3},...,x_{n})$ ${\ displaystyle X (x_ {1}, x_ {2}, x_ {3}, ..., x_ {n})}$ and exits $Y(y_{1},y_{2},y_{3},...,y_{m})$ ${\ displaystyle Y (y_ {1}, y_ {2}, y_ {3}, ..., y_ {m})}$ .

A way to study the behavior of this “black” box is to conduct an experiment, the results of which can be presented in the form of a table. one.

This method of researching the “black” box is called protocol. Values of input quantities at time instants $(t_{1},t_{2},...,t_{k})$ ${\ displaystyle (t_ {1}, t_ {2}, ..., t_ {k})}$ can be chosen arbitrarily.

Table 1

**The method of researching the "black" box**
Input Status	Output Status	Time
$x_{1}(t_{1}),x_{2}(t_{1}),...,x_{n}(t_{1})$ ${\ displaystyle x_ {1} (t_ {1}), x_ {2} (t_ {1}), ..., x_ {n} (t_ {1})}$	$y_{1}(t_{1}),y_{2}(t_{1}),...,y_{m}(t_{1})$ ${\ displaystyle y_ {1} (t_ {1}), y_ {2} (t_ {1}), ..., y_ {m} (t_ {1})}$	$t_{1}$ ${\ displaystyle t_ {1}}$
$x_{1}(t_{2}),x_{2}(t_{2}),...,x_{n}(t_{2})$ ${\ displaystyle x_ {1} (t_ {2}), x_ {2} (t_ {2}), ..., x_ {n} (t_ {2})}$	$y_{1}(t_{2}),y_{2}(t_{2}),...,y_{m}(t_{2})$ ${\ displaystyle y_ {1} (t_ {2}), y_ {2} (t_ {2}), ..., y_ {m} (t_ {2})}$	$t_{2}$ ${\ displaystyle t_ {2}}$
............	............	....
............	............	....
$x_{1}(t_{k}),x_{2}(t_{k}),...,x_{n}(t_{k})$ ${\ displaystyle x_ {1} (t_ {k}), x_ {2} (t_ {k}), ..., x_ {n} (t_ {k})}$	$y_{1}(t_{k}),y_{2}(t_{k}),...,y_{m}(t_{k})$ ${\ displaystyle y_ {1} (t_ {k}), y_ {2} (t_ {k}), ..., y_ {m} (t_ {k})}$	$t_{k}$ ${\ displaystyle t_ {k}}$

Another research method is to apply some standard sequences to the inputs. This method is especially attractive because it allows you to compare the behavior of several "black" boxes with the condition of choosing those that will meet the requirements.

The study of control systems is associated with the concepts of " probabilistic automaton ", "probabilistic system", which requires the study of their probabilistic properties. For these purposes, it is possible to construct a probability matrix (Table 2), in which for each input $x_{i}$ ${\ displaystyle x_ {i}}$ and each exit $y_{i}$ ${\ displaystyle y_ {i}}$ conditional probability is indicated $p_{i}$ ${\ displaystyle p_ {i}}$ , what $y_{i}$ ${\ displaystyle y_ {i}}$ arises in response to $x_{i}$ ${\ displaystyle x_ {i}}$ [7] given in table. 2.

The development of methods for constructing mathematical models of the "black" box is one of the important cybernetic problems. If a mathematical model of the “black” box is available, it becomes possible to attribute it to any one class, all systems of which are isomorphic in behavior.

Creating a mathematical description of a black box is a kind of art. In some cases, it is possible to form an algorithm in accordance with which the “black” box responds to an arbitrary input signal. For most cases, attempts are made to establish differential equations that connect the reaction of the black box with its inputs or, as they say, with its input stimuli.

For science, the black box method is very important. With its help in science, many outstanding discoveries have been made. For example, the scientist Harvey in the XVII century predicted the structure of the heart. He simulated the work of the heart by a pump, borrowing ideas from a completely different area of his modern knowledge - hydraulics. The practical value of the “black” box method consists, firstly, in the possibility of studying very complex dynamic systems, and, secondly, in the possibility of replacing one “box” with another. The surrounding reality and biology give a lot of examples of revealing the structure of systems using the black box method.

Black Box Testing Principles

In this method, the program is considered as a black box . The purpose of testing is to clarify the circumstances in which the behavior of the program does not meet the specification . To detect all errors in the program, it is necessary to perform exhaustive testing , that is, testing on all kinds of data sets. For most programs, this is not possible, therefore, reasonable testing is used , in which testing the program is limited to a small subset of all kinds of data sets. In this case, it is necessary to choose the most suitable subsets, the subsets with the highest probability of detecting errors.

Properties of a correctly selected test

Reduces by more than one the number of other tests that should be designed for reasonable testing.
It covers a significant part of other possible tests, which to some extent indicates the presence or absence of an error before and after a limited set of tests.

Black Box Testing Techniques

Equivalent split.
Boundary value analysis.
Analysis of cause and effect relationships.
Assumption of an error.

Let us consider each of these methods in more detail:

Equivalent Split

The method is based on two points:

The source data must be divided into a finite number of equivalence classes. One equivalence class contains such tests that if one test from the equivalence class detects some error, then any other test from this equivalence class should also detect the same error.
Each test should include, as far as possible, the maximum number of equivalence classes in order to minimize the total number of tests.

The development of tests by this method is carried out in two stages : the allocation of equivalence classes and the construction of the test.

Equivalence classes are distinguished by choosing each input condition, which are taken using the technical specifications or specifications and are divided into two or more groups. To do this, use the following table:

Input condition	The correct equivalence classes	Invalid equivalence classes
''	''	''

The allocation of equivalence classes is a heuristic method, however, there are a number of rules:

If the input condition describes a range of values, for example, “An integer takes a value from 0 to 999”, then there is one valid equivalence class and two incorrect ones.
If the input condition describes the number of values, for example, “The number of lines in the input file lies in the interval (1..6),” then there is also one valid class and two incorrect ones.
If the input condition describes the set of input values, then the number of correct classes is determined equal to the number of elements in the set of input values. If the input condition describes the “should be” situation, for example, “The first character should be uppercase”, then one class is correct and one is wrong.
If there is reason to believe that the elements within the same equivalence class can be interpreted differently by the program, it is necessary to subdivide this class. At this step, the tester based on the table should compile tests covering all the correct and incorrect equivalence classes. In this case, the compiler should minimize the total number of tests.

Test Definition:

Each equivalence class is assigned a unique number.
If you still have the correct classes not included in the tests, then tests are written that cover the maximum possible number of classes.
If the wrong classes not included in the tests remain, then they write tests that cover only one class.

Boundary Analysis

Boundary conditions are situations that arise at the upper and lower boundaries of input equivalence classes.

The analysis of boundary values differs from the equivalent partition in the following:

The choice of any element in the equivalence class as representative is carried out in such a way as to test each boundary of this class with a test.
When developing tests, not only input values (input space) are considered, but also output (output space).

The method requires a certain degree of creativity and specialization in the problem under consideration.

There are several rules:

Build tests with incorrect input data for a situation of a slight excess of the range of values. If the input values should be in the range [-1.0 .. +1.0], check −1.0, 1.0, −1.000001, 1.000001.
Be sure to write tests for the minimum and maximum range limits.
Use the first two rules for each of the input values (use point 2 for all output values).
If the input and output of the program is an ordered set, focus on the first and last elements of the list.

The analysis of the boundary values, if applied correctly, allows you to detect a large number of errors. However, defining these boundaries for each task can be a separate difficult task. Also, this method does not check combinations of input values.

Causation Analysis

Stages of building a test:

The specification is broken down into work areas.
The specification defines many causes and effects. The reason is understood as a separate input condition or equivalence class. A corollary is an output condition or system transformation. Here, each cause and effect is assigned a number.
Based on the analysis of the semantic (semantic) content of the specification, a truth table is constructed in which various combinations of causes are sequentially sorted and consequences for each combination of reasons are determined.

The table is accompanied by notes specifying limitations and describing combinations that are not possible. The disadvantage of this approach is a poor study of the boundary conditions.

Error Assumption

A tester with extensive experience seeks out errors without any methods, but at the same time, he unconsciously uses the method of assuming an error. This method is largely based on intuition. The main idea of the method is to make a list that lists possible errors and situations in which these errors could occur. Then, based on the list, tests are compiled.

Notes

↑ Beiser, 2004 , p. 33.

Literature

Ross Ashby W. Chapter 6. The Black Box // Introduction to Cybernetics = An Introduction to Cybernetics. - Publishing house of foreign literature, 1959. - S. 127-169. - 432 s.
Beizer B. Black Box Testing. Technology functional testing software and systems. - Peter, 2004 .-- 320 p. - ISBN 5-94723-698-2 .

[_48d682a6aa129611-1] Beiser, 2004 , p. 33.