02.10.2014 08:41 |
Rescue for developers
Figure 1: The harmonized medical device standard IEC 62304 calls for multiple test levels, corresponding to different levels of abstraction of development. (Image: Wood Mallet)
The software quality is of decisive importance for complex medical devices. If blood glucose meters crash at particularly high values or lose image archives when updating x-rays, then patients and users are quickly in danger. In order to protect itself as a manufacturer of medical devices from software malfunction, you have to work carefully in all stages of development. The harmonized standard IEC 62304 describes the minimum requirements for the process of development and testing. The standard requires several levels of testing (Figure 1).
- The software system test proves that the product meets all software requirements
- The software integration test verifies the interaction of the software components among themselves.
- The verification of the software units ensures adherence to the software design safely. . Here While a test is not explicitly required, in practice, however, made this a rule of unit tests use
In order to effectively carry out a test, a whole range of activities are necessary:
- test plan and specify:. should be specified as the test steps to be carried out to look, what results are expected during the execution of the test, and when a test is passed or failed
- Tests run: The test steps are carried out and recorded the results
- evaluate the tests. The results will be evaluated to determine which tests are passed
- All these activities..
Considering the cost of software testing, then jump two problems in the eye: First, the essential expenses incurred towards the end of a development project. On the other must run again in the correction of detected errors under certain circumstances, large parts of software testing. If a development project works iteratively, so planning from the outset with a series of intermediate versions, then the test phases are repeatedly go through. A manual test execution, with the people all the steps of the test specification to perform, record and evaluate, then hardly efficient. Here be worth automated tests.
Automated Software Test
The easiest way is still to achieve test automation in unit testing. He checks the smallest units of software, so for example functions or classes, so often it is also the test routines in the same programming language to implement. Help test frameworks such as cpp or unit testing tools such as Cantata or vector Cast. Since the unit test verifies individual software units, there are in the design of the test a lot of freedom: not to be tested software units can be replaced by stubs or mocks, which are under the control of the test tool. So even difficult error situations, such as memory bottlenecks, safe simulate and verify the response of the test object.
At a glance
software system test: This sounds like another category of unit and integration test, but actually refers to the complete software in a complete system. Here fail pure software frameworks, a HIL test stand is required. The can also build with commercial eval board itself and thus support modern development processes.
The software integration test can be automated with the same tools as the Unit test. The difference lies in the view when creating the test cases: The unit test is a white-box testing, in which (ie the source code) uses the knowledge of the internal structure of the test units. During integration testing, however only considering the (hopefully sufficiently documented) interface of the components and tests against this interface, without accessing the implementation details. But the test can still be done by programmed test routines.
In the software system test automation is no longer so simple. His aim must be to test the unmodified possible software system on the target hardware against its requirements. These tests is difficult to automate, because the software system in the form in which it is incorporated into a medical device, often has no more interfaces may be caused on the testing and error conditions. The tests therefore have access to the existing (usually electronic) interfaces of the target hardware. It is therefore also called a hardware-in-the-loop testing (HiL) test. The following sections describe an example of such a HiL test that checks a control device for a medical therapy unit completely automated.
Automated software system test point
therapy devices such as ventilators, dialysis machines and infusion devices often a typical system architecture, as it is sketched in Figure 2.
Figure 2: The system architecture of typical medical equipment works with a supervisor who monitors the software operations in the controller. (Image: Wood Mallet)
The interaction with the user is via a dedicated computer or processor For example, a touch screen controls. The actual control of the therapy is performed by a control unit, which is labeled in the figure as a “controller.” This controller has many interfaces to the hardware, ie to the sensors and actuators, with which the treatment is controlled. For reasons of security functional, it also are often an independent channel, here the “Supervisor” means. He makes sure that the patient is also not compromised by the device when a software or hardware fault in the controller or the user interface should be present. Typical task of the supervisor is to monitor the therapy parameter limits and recognize dangerous situations and, for example, respond to alarms. The supervisor also draws on sensors and actuators of the hardware, although under certain circumstances to other, redundant
dialysis machine as an example
In the example system -. A dialysis machine – is the bulk of the the software under test in the controller and in the supervisor, and these two processors are located on a common board. In order to automate both the test node, a test system is needed that simulates or monitor the electrical interfaces of the board. A more detailed analysis of the hardware in the sample project revealed that following these interfaces (about 150 signals in total) are to use:
- A whole range of digital I / O interfaces
- Multiple serial RS-232 interfaces
- I 2 C interfaces, especially for sensors
- PWM signals to control actuators
- A few analog input and output signals
A common approach to simulate these interfaces for a HiL test now would be to take up a catalog, according to many standard IO buy tickets and it set up a test system. In standard card then a standard software could be used to automate the testing.
Specially developed test stand
In the sample project this way was not coming through, and this for four main reasons :
- To achieve test run times of a few days, several test stations should be built. The cost for the standard IO cards would therefore like strong significant.
- Many of the signals must be treated only with a very low time resolution, therefore no powerful simulation hardware was needed.
- A few signals, however, must be operated with very high time resolution, so that standard hardware did not seem suitable.
- Many problems in the construction of test systems arise in the wiring, so be definitely developed a circuit board for the interconnection should.
But the main argument against a standard solution was the effort to develop appropriate test scripts. To simultaneously develop about 600 test scripts with the software product, the aim was to raise the semantics of the test script in the same abstraction level as the software requirements. This in turn reduces both the time for the development of the scripts as well as the time for the review of the requirements. And that just seemed a natural development of the test system in a short time possible.
Test System
In order to develop a test system quickly, and to keep costs at a reasonable level, it offered to build the test system using standard components. In the example here about finished Eval boards were used for FTDI 4232 chip, simulate the optional 32 digital IOs, four serial ports or I 2 C-ports. This is sufficient for simulating and measuring all signals, which are not relevant to extreme time resolution. For the remaining interfaces with special requirements an evaluation board came for a STM32 microprocessor used. All these components are connected via USB to a PC, the test scripts running (Figure 3).
Figure 3: The block diagram of the automation solution is to realize that it all input and output capabilities of the tested system controls and monitors in the best HiL tradition. (Image: Wood Mallet)
In this particular case, the board with the system under test was applied directly to the test stand mounted and direct 1: 1 connected wiring. This makes the problems with the cabling have significantly reduced and the system is very stable.
To automatically perform several test runs after the other, it was necessary to control the power supply of the system under test, so it automatically switched on and to turn off. It quickly became clear that problems occur when the power is interrupted to the system under test, the test stand, however, is live – in the case of parts of the circuit under test are powered and can be damaged. Therefore, all connections through the tri-state transceiver were isolated. The software of the microcontroller also ensures that the insulation will not be lifted until the power supply is active.
Test Software
The software with which the tests are carried out automatically, consisted of several parts:
- Embedded Software for the built in TestStand microcontroller
- PC application that performs the test scripts and the results of the tests and recorded
- The actual test scripts.
..
In order to achieve the aim of adapting the abstraction level of the test scripts in about the software requirements, a special, simple XML language is defined by analysis of the requirements . This comes with surprisingly few commands:
- “set”: Modifies a simulated size
- “wait”. Expected entry of an observed state
- “expect”. expected the arrival of a serial message
- “monitor”. Monitors the system state concurrent
- “repeat”:.. Repeats instructions for a number or a time
- “verify”: Displays a trace to a request
<. li> “send”: Sends a serial message
<. li> “delay”: Waits
Image 4: The test cases can be formulated with fairly simple XML files. (Image: Wood Mallet)
Despite the few commands thus be defined very powerful test scripts. The example in Figure 4 shows a test case for a very simple request, namely the self-test the alarm LED. The corresponding Requirement “REQ1234: Test the alarm LED” asks: “After the power-on self-test, the alarm LED flashes five times with 1 Hz and 50% duty cycle.” Figure 4 shows the associated test script
.
Long test runs
The procedure for an automated software system test differs significantly from unit-or integration tests, mainly because here the unchanged system software is tested. Therefore, the software believes to perform a normal therapy. In the example, this is a dialysis treatment, and the like takes another four hours. Just as much time therefore requires a complete run of a smooth test situation.
this easy passing, often referred to as a fair-weather or Greenlight scenario has an essential importance. First, he must demonstrate that a complete treatment can be carried out correctly. Secondly, it is the basis of almost all other tests, the bad weather or Redlight tests. For example, to check the response of the Supervisor software to the malfunction of a sensor leads to the GreenLight scenario up to the point where you want to simulate the misconduct. Then, the test stand will reproduce the error and check the response. Maybe the system is then in an error state, and for the next test you have to start over.
Two weeks on a weekend
In the specific example project there were at the end of two fair-weather tests (for different treatment options) with a duration of four hours and several hundred bad weather tests, with a total running time of about 14 days. By distributing on six test stands, this resulted in a run time for a complete software system test of 2.5 days – ideally over a weekend. Comparing this with the previously estimated for manual test execution times of three months at six testers, then it becomes obvious that the automation of software testing system not only saves costs. It is even more essential for agile software development with Sprint cycles of two weeks.
No comments:
Post a Comment