Project Topics
Get Complete Project Material File(s) Now! »
Value checking based Testing tools
Value checking based testing tools perform tests by checking a known correct output for a specific input. These tools can perform automatic tests but lack the capability of generating test data. The programmer defines the test data in tables of inputs and corresponding outputs. To be able to determine the correct output the programmer performing the test must have full knowledge of the program that is tested. In some cases this includes access to the source code to determine every possible path [11].
The automatic process in this type of tool is the verification part, once the tables have been written the program will verify all output values and typically terminates the program once an incorrect value has been detected. For a test to pass all inputs must match the correct output.
This type of tool can be time demanding and lack flexibility as it requires manually written test data before any automatic testing can be done. Also, the number of tests that can be performed is limited.
Specification based testing tools
Today testing of software for commercial use is often performed by a third part [14], because the developer is considered to know too much about the program to perform objective testing. This requires the use of specification based testing as the programmer performing the test does not have access to the source code. Many agile software devel-opment techniques such as extreme programming, which has become a commonly used technique in object oriented programming, is also based on specification based testing [15].
Because of this specification based testing has become a popular testing technique and has lead to the development of many different specification based testing tools. This chapter will explain some of these tools.
QuickCheck
QuickCheck [18] is an automatic specification based testing tool developed by Koen Claessen and John Hughes. The goal when developing this tool was to make it light-weight i.e. the tool is meant to support agile development technique [19]. Quickcheck is limited for Haskell programs and the final tool is only 300 lines of code.
A specification in QuickCheck is implemented as a predicate called property. The pro-grammer defines one or more properties for each program part that is to be be tested. The program is then tested for a large number of randomly generated data inputs and the property is used to verify the program. If every condition in the property holds the test has passed.
QuickCheck uses random data generation but to ensure complete and thorough testing the programmer can limit the amount and type of data generated, making the test more limited and accurate. QuickCheck encourages unit testing by defining a property for each part of the program but it can also test complete program as a separate unit. The results of the test is normally reported simply as passed or failed but it also has the possibility to collect data to produce a histogram. Listing 3.5 shows a property and also how the call to QuickCheck is done. The property describes a rule defining that if two vectors are concatenated and then reversed the result should be the same as if the second vector was reversed and then concatenated with the reverse of vector 1.
Programming in Streambits
This section describes the different parts and the structure of StreamBits from the pro-grammers point of view. StreamBits is designed for executing multiple tasks on multiple data streams concurrently, this is done by using the standard components pipelines and filters.
Components
There are three kinds of components from which a StreamBit program is built of: pipelines, filters and tapes(fig.3.1).
Filters is the component designed to perform the work on a given stream of input data. All filters are added to a pipeline which is the main part of a program. By adding filters in a different order the pipeline can be designed to perform many different tasks. Filters can be executed concurrently within a pipeline forming a multi threaded program.
Every connection inside the pipeline is done by tapes which are implemented as FIFO array blocking queues that stores the streams that comes out of each filter and passes on the resulting stream to the next filter. Each filter has two tapes, one config tape for the configuration and one data tape for the data stream. The tapes have two functions called pop to retrieve values from the input tape and push to add values to the output tape.
Init- Configure- Work
Each filter can work in three different modes: init, config and work. The init mode is used for initiation of the filter. The config is used for reconfiguration of the filter during runtime. The work mode is the part of the filter that process the data stream. This is done by reading the data stream from the in tape and perform different operations on the data before passing it on to the next filter. Fire is a command used to execute the pipeline. Once the fire command has been given it will keep all filters working by calling each filters Configure and Work mode continuously in that order. During one configure session the filter reads the config stream and sets different variables inside the filter which describes how the filter should behave in the next work session.
The fact that configure is called continuously means that the entire pipeline can be recon-figured during runtime. Listing 3.6 shows a filter which purpose is dependent on the config stream. The variable mode determines if the filter should add or subtract the incoming values, since Configure is called continuously and before Work, mode can toggle from one work session to another. This means that the current configuration will be applied to each value that is popped from the data stream during the following work mode.
Table of contents :
1 Introduction
1.1 Automatic testing
1.2 StreamBits
1.3 Project goals
2 Literature review
2.1 Specification based testing
2.1.1 Formal Specification languages
2.1.2 Design by contract
2.2 Value checking based testing
3 Background
3.1 Value checking based Testing tools
3.1.1 JUnit
3.2 Specification based testing tools
3.2.1 iContract
3.2.2 Korat
3.2.3 QuickCheck
3.3 Programming in Streambits
3.3.1 Components
3.3.2 Data types
3.3.3 Init- Configure- Work
3.3.4 Main function
3.3.5 Threaded Framework and streamrate
4 Implementation
4.1 The test tool
4.1.1 Autotest
4.1.2 Data generators
4.1.3 Config
4.1.4 Data collector
4.1.5 The log filter
4.1.6 Stream rate test
4.2 Test program from programmers point of view
4.2.1 Main
4.2.2 Property
4.2.3 Class Test
4.3 Execution of test tool
5 Results
5.1 Adder
5.1.1 Functionality
5.1.2 Test
5.1.3 Results
5.2 Matrix multiplication
5.2.1 Functionality
5.2.2 Test
5.2.3 Results
6 Conclusion
6.1 Future work
References
