- BEHAVIOR-DRIVEN DEVELOPMENT VS. TESTING by James Bach
- Testing like the TSA (here is the criticizm of TDD approach but there is one recomendation here which fits the topic. I mean the recomendation not to use Cucumber)
- Do not use it for white box unit testing - Main reason is that BDD makes an overhead in a form of extra resources (like feature files) and the code binding text instructions to the actual executable code. Another key factor here is that white box unit testing is done by developers for developers. It means that the ability to represent readable tests isn't advantage here as developers understand the code pretty well. All the above facts are multiplied by the fact that test development shouldn't take more time than development as the value the developer brings is reflected in the code under test rether then in tests covering that code. All those things make Cucumber not applicable for developers
- Make conventions for building test instructions - In many cases I encountered the situation when the same action was represented in many different forms with minimal differences. E.g. there were phrases like:
When I click on Submit button When I click on Submit When I click on the Submit button When I click on the "Submit" buttonAs you can see the meaning of the above strings is the same though for Cucumber they are different. Use common convention to follow in order to prevent such chaos. E.g. you can state that all control names must be quoted and preceeded with the article. Also you should identify what the control type is. In this case only last line is valid. Of course, initially it will require a lot of reviews and corrections but after some time it becomes a habit. It's actually the same as coding style. It good practice when all people write the code in the same fashion so any developer can read and understand the code of others. The same thing is here.
- Use proper level of detail - There are tests which verify some specific work flow checking all steps in detail. But also there are tests which just use that work flow but they are intereted in further checks affected by some other functionality. In this case we don't need to specify all details. Some general action description should be enough. E.g. we have test like:
Given I'm on home page When I click on the "Orders" link Then I should see the "Orders" page is open When I click on the "Create" link Then I should see the "Create Order" page is open When I enter "Test" value into the "Order Name" field And click on the "Submit" button Then I should see the "Success" message appearsand we have another test which requires several orders for further processing. The second test can use the following:
When I create the "Test" order And I create the "Test1" orderIn this example we put instructions from the first test and wrapped them with the single instruction. Actually if we do the same with the code we just make a layer of re-usable functions which represent some domain layer. So, that would be done anyway. We just have to group that functionality properly. But the main idea is that if we don't care about how we do some actions but we just need the result then we can use more high-level instructions.
- Use parameters - key flexibility of Cucumber and all other similar engines is in ability to vary some part of instructions. Actually the code is bound to some text in feature files using regular expressions with back-linked variables. E.g. if we have instructions like:
When I click on the "Create" button And I click on the "Submit" buttonwe don't need to bind separately both instructions. We can create method for clicking any button by their name and bind it to the following regular expression:
/I click on the "(.*)" button/In the cucumber the text in quotes will be passed as parameter. If we need to pass complex parameter (e.g. we need to fill some set of fields in some specific form) we can use tables and test instructions will look like:
When I fill in the "Order" form with the following options: | Order Name | Date | Client | Price | | Test1 | 2012-02-12 | John B. | 200$ |Of course, we should implement appropriate method performing those actions.
- Try to re-use text instructions as much as possible - BDD with it's extra layer of text instructions requires extra eforts to support them and their bindings to the actual code. In order to optimize that expences we should re-use existing text instructions. The more time each specific text instruction is in use the less effort is required to maintain the code in case we need to do some changes which should be reflected in all affected tests.
- Use examples when you check multiple input options for the same work flow - It's more about test design and it can be the biggest pain. Actually, here is an example of such situation. Let's take an example of poorly designed test:
Scenario 1: Account is in credit+ Given the account is in credit And the card is valid And the dispenser contains cash When the customer requests cash Then check that the account is debited And check that cash is dispensed And check that the card is returned And check that nothing happens that shouldn’t happen and everything else happens that should happen for all variations of this scenario and all possible states of the ATM and all possible states of the customer’s account and all possible states of the rest of the database and all possible states of the system as a whole, and anything happening in the cloud that should not matter but might matter.The problem here is that there's necessity of tests verifying different combinations of inputs (different situation) but it was designed in plain manner where we perform basic check and then specify in high-level that we should check various combinations. Actually, when we want to check the system behavior under different input but with the same work flow we should use data-driven approach when we define common workflow, identify varying parameters and bind that work flow to the table of input data and expected results. In this example the common work flow skeleton can be represented in the following way:
Scenario Outline: Account is in credit+ Given the account is <Account State> And the card is <Card State> And the dispenser <Dispenser State> When the customer requests cash Then check that the account <Account Result> And check that cash <Cash Result> And check that the card <Card Result>Note that the keyword Scenario was replaced with Scenario Outline and varying parameters were replaced with some keywords between < and > characters. The only thing left here is to add the data table with inputs and expected results. So, the above test should be appended with the following lines:
Examples: | Account State | Card State | Dispenser State | Account Result | Cash Result | Card Result | | in credit | valid | contains cash | is debited | is dispensed | is returned | | in credit | invalid | contains cash | isn't debited | isn't dispensed | is returned | | in credit | expired | contains cash | isn't debited | isn't dispensed | is returned | ....Here is example how we extend our test coverage varying Card State option. The more rows we add the more cases will be checked. The scenario outline will be executed as much as the number of rows and each run uses data from appropriate row.