Tag Archives: test case

Using All-Pairs wisely- Independent and Dependent Variables

A Little Background on All-Pairs

All-pairs is a tool written by James Bach which is used to generate the optimized test cases when we are dealing with data-intrinsic scenarios.

The problem statement:

Let’s take a simple e.g. where we have 3 radio-options in a form and each radio-option has 3 choices

Radio Button Option 1:

  • Option A
  • Option Bohn B
  • Option C

Radio Button Option 2:

  • Option X
  • Option Y
  • Option Z

Radio Button Option 3:

  • Option M
  • Option N
  • Option O

The possible number of test cases for these 3 radio options will be 3*3*3 =27.

It is very time-consuming and tedious to test all 27 test cases. A tester in this condition will randomly select 4-5 combinations and test them. But there is randomness in this process which a tester can’t justify. Why a particular test case was given preference and others were left out.

The all-pair tool is handy in this situation.

All-pair’s philosophy:

The philosophy of All-pair is to “create tests that pair each value of each of the variables with each value of each other variable at least once”.

 Hence we can optimize the test scenarios and be sure that we have tested each value of the each variable at least once. For e.g. the optimized test scenarios of the above problem would look like this:

TestCase Radio-Option 1 Radio-Option 2 Radio-Option 3 Pairing
1 Option A Option X Option M 3
2 Option A Option Y Option N 3
3 Option A Option Z Option O 3
4 Option B Option X Option N 3
5 Option B Option Y Option M 3
6 Option B Option Z Option M 2
7 Option C Option X Option O 3
8 Option C Option Y Option M 2
9 Option C Option Z Option N 3
10 Option B Option Y Option O 2

So we have narrowed our test scenarios form 27 to 10.

The optimization is much appreciated when the number of variables and number of choices corresponding to each variable is high. For instance, to try all combinations of 10 variables with ten values each we would require 10,000,000,000 test cases. All-pairs only require 177 cases.

The Subway menu Example :

A typical subway menu looks is a 7 step process and each of them provide us with multiple options to choose from. for e.g.:

Size of Sub 6 inch Foot Long
Bread Italian herbs Honey Oat Wheat Hearty Italian
Filling Veggie Delight Chicken Teriyaki Turky Breast Ham
Toasted Yes No
Salad Yes No
Sauce Light mayo Honey Mustard BBQ
Fresh Value Meal Soda Crisps Cookies

If the QA have to test wether we can create a sub using these combinations then he will have to test 2*4*4*2*2*3*3 =1152 combinations.All-pairs reduces the number of test cases to 24.

The Twist in the problem: Adding dependencies between variables

Now, if I add dependencies to these scenarios, will All-pairs be effective in this case? For e.g. There are 3 variables, Variable A, B,C and each variable can have 3 possible values  Valid, Invalid, Null.

Variable A Variable B Variable C
Valid Valid Valid
Invalid Invalid Invalid
Null Null Null

There are additional dependent conditions.

  1. You can access Variable B only if variable A is valid.
  2. You can access variable C if variable A and Variable B are both valid.

If one uses All-pairs here to generate optimised test cases, we will have following output:

TestCase Variable A Variable B Variable C Pairing
1 Valid Valid Valid 3
2 Valid Invalid Invalid 3
3 Valid Null Null 3
4 Invalid Valid Invalid 3
5 Invalid Invalid Valid 3
6 Invalid Null Valid 2
7 Null Valid Null 3
8 Null Invalid Valid 2
9 Null Null Invalid 3
10 Invalid Invalid Null 2

So what’s missing?

  1. We have never tested the variable C for invalid and null scenarios because of the added dependencies.
  2. The test cases 5,6,10 are redundant as scenario 4 had already covered them.
  3. The test cases 8, 9 are also redundant and don’t add any value.

The Actual test scenarios should be:

Test Case Variable A Variable B Variable C
1 Valid Valid Valid
2 Valid Valid Invalid
3 Valid Valid Null
4 Valid Invalid Any Value
5 Valid Null Any Value
6 Invalid Any Value Any Value
7 Null Any Value Any Value

The additional dependencies have actually helped us to reduce our test scenarios.

Conclusion and learning:

All-pairs is designed for creating optimized test scenarios, if only and only if, the each variables are independent and options within each variables are mutually exclusive.

As a user of the tool, it is one’s responsibility to handle the dependencies externally and use All-pairs with independent variable set.

1 Comment

Posted by on January 11, 2012 in Testing


Tags: , , , ,