Martha Dunia Delgado Dapena¹ , Yucely López Trujillo² e Indira Chávez Valiente³

¹ Computer Engineering. M.Sc., in Applied Computing. Ph.D., in Technical Sciences, Insitituto Superior Politécnico "José Antonio Echevarria" (CUJAE), La Habana, Cuba. Centro de estudios de ingeniería de sistemas (CEIS), School of Computer Engineering, Insitituto Superior Politécnico "José Antonio Echevarria" (CUJAE), La Habana, Cuba. marta@ceis.cujae.edu.cu

² Computer Engineering. M.Sc., in Applied Computing. Ph.D., in Technical Sciences, Insitituto Superior Politécnico "José Antonio Echevarria" (CUJAE), La Habana, Cuba. Centro de estudios de ingeniería de sistemas (CEIS), School of Computer Engineering, Insitituto Superior Politécnico "José Antonio Echevarria" (CUJAE), La Habana, Cuba. ylopez@ceis.cujae.edu.cu

³Computer Engineering. M.Sc., in Applied Computing. Ph.D., in Technical Sciences, Insitituto Superior Politécnico "José Antonio Echevarria" (CUJAE), La Habana, Cuba. Centro de estudios de ingeniería de sistemas (CEIS), School of Computer Engineering, Insitituto Superior Politécnico "José Antonio Echevarria" (CUJAE), La Habana, Cuba. ichavez@ceis.cujae.edu.cu

ABSTRACT

This paper presents a proposal for storage structure and retrieval mechanisms used for implementing case-based reasoning (CBR) in generating functional test procedures in software projects. This proposal was based on software project t functional requirements and sets out the proposed algorithms for considering the similarity between each pair of projects as well as those leading to adapting the solution found in the case base.

Keywords: case based reasoning, software engineering, software quality, functional testing, artificial intelligence.

Received: june 5th 2009

Accepted: november 15th 2010

Introduction

The issue of software quality receives a lot of attention nowadays. Although companies devote significant resources to adopting or defining quality standards, the results have not met expectations since productivity has been low, the actual amount of resources consumed has been almost unpredictable and has almost never resulted in the required quality and professionalism. The tests to be performed (Everett, 2007) (Pressman, 2005) can be planned from the earliest stages of software life-cycle, even before reaching the codification stage. This prevents errors being inserted into an application, which can become quite difficult to find in later stages.

Despite being good practice, few are benefiting from this idea and only begin to think about the tests after having the code. Most software development teams have limited time for creating detailed and well-planned testing, consequently leading to the execution of mainly functional tests without prior planning. In these circumstances, one can think of working on accumulating experience in the computer which has been obtained from each test. Doing so, a bank of cases can be created to support new projects' testing phase and, in particular, generating test cases and procedures from others which have already been used and have led to identifying flaws in previous projects.

Software development methodologies include testing; some place more emphasis on this aspect, as in the case of test-driven development (Augustine, 2005). Other methodologies consider testing as a developmental stage that should ensure final product quality (Jacobson, 2004). Nonetheless, all proposals agree that it is an issue of paramount importance and should be planned during early stages of software development. Some studies have referred to the importance of estimating the effort associated with testing to determine automation or performance in each case (Singh and Misra 2008) and also test automation in specific environments (Xie and Memon 2007) (Masood Bhatti Ghafoor and Mathur 2009)(KO and Myers 2010). One set of proposals directly addresses the reduction of test cases (Heimdahl, 2004) (Mogyorodi, 2008) (Polo,2005) (Black, 2004); they use algorithms in which it is vital to have sufficient time for the testing stage, this being sometimes impossible.

New processes and methodologies for designing tests are available (Myers, 2004) (Guvenc, 2006) (Meyer, 2006) (Gutiérrez, 2007) (Dias, 2008) (Naslavsky, 2008) but they do not usually allow reuse of cases or previously-generated procedures.

The proposal presented in this paper only takes into account testing software requirements (functional or black box) whose main objective is to validate whether observed software behavior complies with its specifications. This work forms part of a larger project by a group of researchers and teachers from the Systems Engineering Study Center (CEIS), forming part of the Computer Science Faculty at the José Antonio Echeverría Higher Polytechnic Institute (CUJAE). Its main objective was to make proposals for planning functional tests and reusing them.

The project consisted of three phases. The first consisted of translating software requirements written in non-formal language to a list of functional requirements reduced to a verb and noun characterising the functionality in question. Several processes were carried out in the second phase generating a set of test cases and procedures for a software project based on functional tests from previous projects. A third phase reduced the set of cases and test procedures proposed in accordance with priority criteria and will be considered as a first approach to functional test planning.

This report focus on the project's second phase related to algorithms and mechanisms generating functional test procedures in a software project.

Artificial intelligence (AI) techniques were used for achieving a tool for supporting this type of decision. Case-based reasoning (CBR)) (Althoff, 2001) was used, taking into consideration that potential experts find it very difficult to establish chains of generalized rules generating test cases and test procedures for use in any type of project, and it is easier to express knowledge in terms of cases.

CBR has had applications in software engineering (Shepperd, 2003) (Ganesan, 2000) (García, 2001). Several AI techniques have been used in software testing; some examples would be analysing results using a neural network and data-mining algorithms presented by Last (Last, 2006), prioritising test cases using genetic algorithms (Elbaum, 2004) (Fraser, 2007)

Case-based reasoning for reusing test procedures on software projects

Base case structure

Information regarding the projects and all defects detected were stored in a repository presented by Delgado (Delgado, 2006). This repository had been enriched with test cases, test procedures and recovery mechanisms. The latter allowed retrieval of test cases (TC) and test procedures (TP) associated with the projects stored in the case database which were similar to a new project for which one may want to generate a TC. It should be noted that this proposal consisted of generation during early stages where input informatio was required for a new project.

It has been necessary to associate project requirements and a description of different scenarios to be considered in the functional test (functional test description - FTD). The FTD contained the order of actions involved in developing functionality and included all possible scenarios to be considered in the requirement's FDT. This entity's internal structure has been taken from Delgado's definition (2006).

The FTD was linked to each TP designed for performing functional testing and these, in turn, were referenced by each TC where they were used.

The structure defined for storing the TP is shown in Table 2. TP were identified in the project database by combining the pair (<VERB>, <NOUN>) defining its name.

The NOUN identifying the TP referred to a frame within a hierarchy of frames expressing relationships between entities in a particular domain and were used in this proposal for recovery of TP amongst similar projects. The frame structure modelling a particular entity is shown in Table 1.

At its input section, TC also referred to the TP step where the project tester had to enter values for corresponding attributes and entities. Separate storage of TP and TC ensured that the same TP could be used in various TC. As can be seen, both structures were linked to the functional logic of the requirement to be tested; they combined the order in which actions were executed, as described in the TP, with TC input values. TP were determined by the verb and the TC were determined by nouns corresponding to the entities and attributes referenced in the TC. These entities were directly related to frame hierarchy.

A set of TP patterns (Pattern_TP) based on the pattern definitions described by Delgado (2006) had to be defined to reuse information about the order of executing actions included in a new project's TP.

Each pattern represented the set of steps to be followed by the tester during test execution. For example, for specific verb such as "to register", regardless of whether one wanted to test the functionality "register student" or "register teacher", the set of steps to be executed were the same. With this definition, each TP was linked to one or more Pattern_TP, allowing such Pattern_TP to be reused in generating TP in projects having similar characteristics to those stored in the project database.

Pattern_TP formed part of a semantic network in which there were three types of relationship: "is a", "includes to" and "extends from". These types of relationships were determined by the relationship between verbs identifying each Pattern_TP. Table 4 describes the semantic meaning of these relationships where A and B were type Pattern_TP and belonged to the semantic network.

A fourth kind of relationship named "is equivalent to",expressing equivalence, existed between the verbs. For example, the verbs "register" and "create" were equivalent. All equivalent verbs were associated with a single Pattern_TP in this kind of relationship.

The Pattern_TP contained slots that could be instantiated at the time of TP adaptation in a particular project. Table 4 shows the types of slots that could be presented in a Pattern_TP and where the

Determining similarity between projects

To run the CBR system recovery phase, i.e. project recovery (cases),it was first necessary to analyse which were each software project's distinguishing features. In this case, each project requirement was considered as a trait predictor, as presented in Delgado(2006), in software project revisions.

The function of similarity between projects

f(w_i,δ_i(x_i(O₀),x_i(O_B)))=∑ⁿ_i=lw_iδ_i(x_i(O₀),X_i(O_B))/∑ⁿ_i=l w_i   where ^wi was the level of importance or significance of each requirement within the new problem. This gave greater emphasis to the requirements analyst or inspector considering the most significant within the project to be reviewed.

δ_i(x_i(O₀),x_i(O_B))=f_r((w_k,δ_i(z_i(x_i(O₀)),z_i(x_i(O_B))))²_k=l ,          where

f_r((w_k,δ_ij(z_k(y_i),z_k(y_j)))²_k=l)=∑_k=l²w_k.δ_ij(z_k(y_i),z_k(y_j)) , was the funtion evaluating similarity between a pair of requirements. This function was used to combine the similarity values between nouns and verbs and obtain the value of similarity between two requirements. A semantic network in which there were two main types of relationships, hierarchical relationships ("is a") and equivalence relations ("is equivalent to") was used for determining the similarity between nouns (and verbs). The function used for determining the similarity between verbs δ_ij(z₁(y_i),z₁(y_j))  and between nouns δ_ij(z₂(y_i),z₂(y_j))  was presented in Delgado (2006).

The cases compared with the new problem were stored in a linear list sorted by descending function value f . This information was used in generating a new solution.

Generating and adapting a new solution

After performing the search, the project having the highest similarity value f was used as a potential solution; it was checked that it covered all the requirements of a new project. In case not all requirements were covered, the potential solution was completed with the requirements of the second most similar case, and so on, until all the new project's requirements were completed or all recovered cases were analysed.

When a requirement from the new project x_i(O_N)  was associated with another from a project stored in the case database x_j(O_N) , the set of TC and TP which have been recovered can be assigned to the new project requirement. However, the degree of similarity between the two requirements had to be assessed to decide whether they could take the TC and TP as they appeared in the case database or whether some adaptation were needed. If the value of f was "1" TC and TP could be taken as they were because it was exactly the same requirement, otherwise two cases would deserve special treatment:

1. Value of f was [μ, 1), where μ was a threshold value. If the value of f was greater than μ then the pair of requirements was similar. Algorithm 1 (Figure 1 y 3) shows how the adjustment was made in this case.

2. Value of f was "0". No requirement found in the case database similar to the requirement of a new project. Algorithm 2 (Figure 2)shows how the adjustment was made in this case.

Two thresholds values were used in developing these algorithms to decide when a pair of verbs and a few nouns could be considered similar. They were defined for verbs μ₁  and μ₂  for nouns.

The TP so generated were used for generating TC for using information stored in the network of entities and the set of patterns which had been generated. The TP and TC obtained were presented to the user. Doing so, adjustments could be made to the solution and new

considerations could be incorporated, according to a new project's t specific situation. The new solution was stored in the case database as a new case.

Conclusions

This paper proposes generating functional test procedures based on the use of case-based reasoning for reusing previously generated information. The proposal's fundamental contribution lies in designing algorithms leading to the recovery and adaptation of test procedures stored in the project database and allows past experience to be taken advantage of.

However, this research is not a complete proposal; further work is being carried out on developing the remaining phases of the project to provide a solution establishing the priority in which test cases must be run, considering time constraints. At the moment, efforts are directed towards implementing a tool for experimenting during teaching to generate a test bench for checking the results to be obtainned from the proposal.