An Analysis and Extension of Category Partition Testing in the Presence of Constraints

Category Partition (CP) is a Black Box testing technique that formalizes the specification of the input domain of the system under test A CP specification is driven by the tester’s expertise and comprises of parameters, categories (characteristics of parameters) and choices (acceptable values for categories) required for extensively testing the system.To ensure that the combinations of choices are feasible and account for valid sets of user requirements, constraints are introduced. A constraint can be a specification for permitted choice combination or a choice annotation as Error or Single. In a typical development environment where testing is driven by stringent deadlines a tester might have to decide how much constraints are enough to attain the maximum level of test completeness. The present work will assist a test engineer in making this decision. This thesis contributes by concluding, based on experimental evaluation of academic and industrial case studies, that in case of limited resources an equally effective test suite can be attained by meticulously defining Error and Single annotations in a CP specification.Combining choices while accounting for constraints and forming Each Choice adequate test set, for instance, is achievable, thanks to Constrained Covering Arrays from combinatorial testing. But, Base Choice criterion besides being an integral and domain specific criterion has not been defined to specifically account for constraints on choices. This thesis contributes by introducing two extensions to the Base Choice criterion: Constrained Base Choice and Extended Constrained Base Choice, to account to constraints resulting from complex input domain. Results show the performance of the new criteria equivalent to a 3-way selection criterion with a much smaller cost. Another contribution of this thesis is the survey and compilation of exhaustively searched 75 algorithms and tools for generating a combinatorial test suite. These algorithms and tools are categorized based on various features required for implementing the CP technique including their support for selection criteria, constraints, mixed covering arrays etc. Results can be of interest to test engineers who are looking for a Covering array tool befitting to their needs.