A Little Background on All-Pairs
All-pairs is a tool written by James Bach which is used to generate the optimized test cases when we are dealing with data-intrinsic scenarios.
The problem statement:
Let’s take a simple e.g. where we have 3 radio-options in a form and each radio-option has 3 choices
Radio Button Option 1:
- Option A
- Option Bohn B
- Option C
Radio Button Option 2:
- Option X
- Option Y
- Option Z
Radio Button Option 3:
- Option M
- Option N
- Option O
The possible number of test cases for these 3 radio options will be 3*3*3 =27.
It is very time-consuming and tedious to test all 27 test cases. A tester in this condition will randomly select 4-5 combinations and test them. But there is randomness in this process which a tester can’t justify. Why a particular test case was given preference and others were left out.
The all-pair tool is handy in this situation.
The philosophy of All-pair is to “create tests that pair each value of each of the variables with each value of each other variable at least once”.
Hence we can optimize the test scenarios and be sure that we have tested each value of the each variable at least once. For e.g. the optimized test scenarios of the above problem would look like this:
|TestCase||Radio-Option 1||Radio-Option 2||Radio-Option 3||Pairing|
|1||Option A||Option X||Option M||3|
|2||Option A||Option Y||Option N||3|
|3||Option A||Option Z||Option O||3|
|4||Option B||Option X||Option N||3|
|5||Option B||Option Y||Option M||3|
|6||Option B||Option Z||Option M||2|
|7||Option C||Option X||Option O||3|
|8||Option C||Option Y||Option M||2|
|9||Option C||Option Z||Option N||3|
|10||Option B||Option Y||Option O||2|
So we have narrowed our test scenarios form 27 to 10.
The optimization is much appreciated when the number of variables and number of choices corresponding to each variable is high. For instance, to try all combinations of 10 variables with ten values each we would require 10,000,000,000 test cases. All-pairs only require 177 cases.
The Subway menu Example :
A typical subway menu looks is a 7 step process and each of them provide us with multiple options to choose from. for e.g.:
|Size of Sub||6 inch||Foot Long|
|Bread||Italian herbs||Honey Oat||Wheat||Hearty Italian|
|Filling||Veggie Delight||Chicken Teriyaki||Turky Breast||Ham|
|Sauce||Light mayo||Honey Mustard||BBQ|
|Fresh Value Meal||Soda||Crisps||Cookies|
If the QA have to test wether we can create a sub using these combinations then he will have to test 2*4*4*2*2*3*3 =1152 combinations.All-pairs reduces the number of test cases to 24.
The Twist in the problem: Adding dependencies between variables
Now, if I add dependencies to these scenarios, will All-pairs be effective in this case? For e.g. There are 3 variables, Variable A, B,C and each variable can have 3 possible values Valid, Invalid, Null.
|Variable A||Variable B||Variable C|
There are additional dependent conditions.
- You can access Variable B only if variable A is valid.
- You can access variable C if variable A and Variable B are both valid.
If one uses All-pairs here to generate optimised test cases, we will have following output:
|TestCase||Variable A||Variable B||Variable C||Pairing|
So what’s missing?
- We have never tested the variable C for invalid and null scenarios because of the added dependencies.
- The test cases 5,6,10 are redundant as scenario 4 had already covered them.
- The test cases 8, 9 are also redundant and don’t add any value.
The Actual test scenarios should be:
|Test Case||Variable A||Variable B||Variable C|
|6||Invalid||Any Value||Any Value|
|7||Null||Any Value||Any Value|
The additional dependencies have actually helped us to reduce our test scenarios.
Conclusion and learning:
All-pairs is designed for creating optimized test scenarios, if only and only if, the each variables are independent and options within each variables are mutually exclusive.
As a user of the tool, it is one’s responsibility to handle the dependencies externally and use All-pairs with independent variable set.