Why Parameterize?
Parameterizing tests means allowing some data used by the test to be provided from outside the test, rather than hard-coded in the test. This is critical to creating a flexible and maintainable test suite.
As a common example, consider the base URL for an application. Hard-coding the URL into each test results in a lot of manual re-work when that URL changes. Parameterizing the starting URL in each test allows it to be defined in one place and passed into each test. This has two benefits: (1) maintenance is easy when the inevitable change comes (a single place to change) and (2) the suite is now more flexible - changing the parameter source allows the test suite to be directed at multiple target systems. For example: a test system while the tester is building the test, a staging system for nightly builds and a production system for final verification tests.
Variable Initialization
Test parameters are provided to the test during the initialization of the execution context. The context then exposes those parameters as variables, which can be referenced with a Variable value source.
Note that these values are stored in the test scope, so they are accessible to any step in the test, regardless of how deeply the step is nested in function calls, etc.
Variable Sources
Muse currently supports these sources for test parameters:
- variable lists in project
- test defaults
- parameterized test suites
They are evaluated in the order listed. This means that the test defaults can overwrite values provided from variable lists. A parameterized test suite can then overwrite those values from it's own list.
Test Defaults
The simplest way to pass parameters to a test, the Test Defaults are simply a collection of named value sources that are configured along with the test. During initialization of the test execution context, they are evaluated and the result is stored, by name, as a variable in the execution context. The test defaults can be edited in the Test Editor.
In this example, three search terms are configured for the test (example #6 from the Muse Examples project):
Defining values as test defaults is a good way to document the data that the test needs.
Parameterized Test Suites
This is a test suite that runs the same test many times, with different data. For example, you could supply a variety of search terms, product quantities and total prices in a test of a shopping cart, to test a variety of user visits.
These can work nicely in combination with test defaults, since the test developer can use the defaults when running the test in isolation during development. Those values can then be overridden by the parameters in the test suite.
Variable Lists
Variable Lists are collections of named value sources, much like the test defaults. However, a single variable list can be applied to many tests, and that can be based on conditional logic — see the examples later on this page for some of the ways this can be used. Project Variable lists can be edited with the Variable List Editor.
In this example, the list defines a base URL, a browser and a browser provider. Note that the value sources specified here are not limited to simple constant values. In this example, the browser and browser-provider variables reference a Project Resource value source, which will be resolved when the test runs.
These variables can then be accessed within the test as shown in the two first steps of this test:
Following the above examples means that the tester can change which browser is used by all tests by simply changing the browser variable in the list. Additionally, you could run the tests with multiple browsers, by building a parameterized test suite that provided additional values for the browser variable.
Application of Variable Lists to Tests
In order to apply a variable list to a test, it must be added to a context-initializer configuration in the project. In this simple example, the list is applied to all tests in all environments, because the include condition will always evaluate to true.
The use of Variable Lists can be configured even farther to create a more flexible test suite. Variable Lists can be applied conditionally to a test based on the environment, the test, or the phase of the moon, if that's what you need (that last one would need some custom code…but not much).
This can be done with a combination of System Variables and Context Initializers.
The following are examples of what can be done:
Target different systems
Imagine you need to target a different test system with the same set of tests. Perhaps the tester wants to target the development server while building tests but then the staging server when the tests run on the continuous integration server. How do you handle this?
1. Create two variable lists, one for the development environment and another for the staging environment.
2. Create a Context Initializer to apply one of the two lists based a condition of the environment, such as an environment variable, username or hostname. This example uses the staging list when running on the cis machine. Otherwise, it uses the local list.
Supply different credentials
Imagine that some of your tests act as a customer, so they need a customer login. But a few tests need to use an administrator login. All your tests use a login function, that you've built to perform the login process. You could copy the login function and reference two different variables to supply customer or login credentials. Then you'd have duplicate logic to maintain. Instead, you can continue to use the single login function and:
1. Create two variable lists: one for customer credentials and one for admin credentials.
2. Add an admin tag to the tests that require administrator privileges.
3. Configure a Context Initializer that applies the correct variable list based on the presence of the admin tag on the test.
