RSS

Using All-Pairs wisely- Independent and Dependent Variables

11 Jan

A Little Background on All-Pairs

All-pairs is a tool written by James Bach which is used to generate the optimized test cases when we are dealing with data-intrinsic scenarios.

The problem statement:

Let’s take a simple e.g. where we have 3 radio-options in a form and each radio-option has 3 choices

Radio Button Option 1:

  • Option A
  • Option Bohn B
  • Option C

Radio Button Option 2:

  • Option X
  • Option Y
  • Option Z

Radio Button Option 3:

  • Option M
  • Option N
  • Option O

The possible number of test cases for these 3 radio options will be 3*3*3 =27.

It is very time-consuming and tedious to test all 27 test cases. A tester in this condition will randomly select 4-5 combinations and test them. But there is randomness in this process which a tester can’t justify. Why a particular test case was given preference and others were left out.

The all-pair tool is handy in this situation.

All-pair’s philosophy:

The philosophy of All-pair is to “create tests that pair each value of each of the variables with each value of each other variable at least once”.

 Hence we can optimize the test scenarios and be sure that we have tested each value of the each variable at least once. For e.g. the optimized test scenarios of the above problem would look like this:

TestCase Radio-Option 1 Radio-Option 2 Radio-Option 3 Pairing
1 Option A Option X Option M 3
2 Option A Option Y Option N 3
3 Option A Option Z Option O 3
4 Option B Option X Option N 3
5 Option B Option Y Option M 3
6 Option B Option Z Option M 2
7 Option C Option X Option O 3
8 Option C Option Y Option M 2
9 Option C Option Z Option N 3
10 Option B Option Y Option O 2

So we have narrowed our test scenarios form 27 to 10.

The optimization is much appreciated when the number of variables and number of choices corresponding to each variable is high. For instance, to try all combinations of 10 variables with ten values each we would require 10,000,000,000 test cases. All-pairs only require 177 cases.

The Subway menu Example :

A typical subway menu looks is a 7 step process and each of them provide us with multiple options to choose from. for e.g.:

Size of Sub 6 inch Foot Long
Bread Italian herbs Honey Oat Wheat Hearty Italian
Filling Veggie Delight Chicken Teriyaki Turky Breast Ham
Toasted Yes No
Salad Yes No
Sauce Light mayo Honey Mustard BBQ
Fresh Value Meal Soda Crisps Cookies

If the QA have to test wether we can create a sub using these combinations then he will have to test 2*4*4*2*2*3*3 =1152 combinations.All-pairs reduces the number of test cases to 24.

The Twist in the problem: Adding dependencies between variables

Now, if I add dependencies to these scenarios, will All-pairs be effective in this case? For e.g. There are 3 variables, Variable A, B,C and each variable can have 3 possible values  Valid, Invalid, Null.

Variable A Variable B Variable C
Valid Valid Valid
Invalid Invalid Invalid
Null Null Null

There are additional dependent conditions.

  1. You can access Variable B only if variable A is valid.
  2. You can access variable C if variable A and Variable B are both valid.

If one uses All-pairs here to generate optimised test cases, we will have following output:

TestCase Variable A Variable B Variable C Pairing
1 Valid Valid Valid 3
2 Valid Invalid Invalid 3
3 Valid Null Null 3
4 Invalid Valid Invalid 3
5 Invalid Invalid Valid 3
6 Invalid Null Valid 2
7 Null Valid Null 3
8 Null Invalid Valid 2
9 Null Null Invalid 3
10 Invalid Invalid Null 2

So what’s missing?

  1. We have never tested the variable C for invalid and null scenarios because of the added dependencies.
  2. The test cases 5,6,10 are redundant as scenario 4 had already covered them.
  3. The test cases 8, 9 are also redundant and don’t add any value.

The Actual test scenarios should be:

Test Case Variable A Variable B Variable C
1 Valid Valid Valid
2 Valid Valid Invalid
3 Valid Valid Null
4 Valid Invalid Any Value
5 Valid Null Any Value
6 Invalid Any Value Any Value
7 Null Any Value Any Value

The additional dependencies have actually helped us to reduce our test scenarios.

Conclusion and learning:

All-pairs is designed for creating optimized test scenarios, if only and only if, the each variables are independent and options within each variables are mutually exclusive.

As a user of the tool, it is one’s responsibility to handle the dependencies externally and use All-pairs with independent variable set.

Advertisements
 
1 Comment

Posted by on January 11, 2012 in Testing

 

Tags: , , , ,

One response to “Using All-Pairs wisely- Independent and Dependent Variables

  1. Manish Kumar

    January 16, 2012 at 12:27 pm

    Good post,Rupesh.

    In my view, PICT is one such tool that helps you to create pairwise tests even when there are constraints or dependencies between values of different parameters.You can download PICT from here –
    http://download.microsoft.com/download/f/5/5/f55484df-8494-48fa-8dbd-8c6f76cc014b/pict33.msi

    You can read about PICT here –
    http://www.amibugshare.com/pict/help.html

    Regards,

    Manish

     

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: