Test Automation tools: trends

While making decision which tool is more suitable for test automation we should always take into account the popularity of each specific tool. Why? Because firstly it's about people. What's the point in taking tool which doesn't have users? Thus, we'll have to invest some money into their learning which is extra expences. Secondly, the more popular tool is the more materials are available either in internet or in some other materials. So, we should take a look at some trends to identify which tool is growing with it's popularity and which one is going to be a history.

BDD engines comparison (Cucumber, Freshen, JBehave,NBehave,SpecFlow, Behat)

Introduction

Cucumber is not the only engine supporting natural language instructions. It's just one implementation of natural language instructions interpreter. The actual language to write tests with is called Gherkin. And it has different implementations adopted to different programming language. Thus we have:

• Cucumber for Ruby
• JBehave for Java
• NBehave and SpecFlow for C#
• Freshen for Python
• Behat for PHP

This list isn't complete as there can be many other similar engines which are simply less popular. All of them have some common set of supported features but there're some restrictions and abilities specific to the actual engine. So, the aim of this post is to collect useful features for each listed above engine and present it in some comparable form. Key features to be mentioned are:

Cucumber: How to avoid major mistakes

Recently I came across several posts criticizing Cucumber and approach itself. Some of the links:

• BEHAVIOR-DRIVEN DEVELOPMENT VS. TESTING by James Bach
• Testing like the TSA (here is the criticizm of TDD approach but there is one recomendation here which fits the topic. I mean the recomendation not to use Cucumber)

and many others. Generally speaking the idea is that we should put off our pink sun glasses and look at the real world. Cucumber (and all it's analogs designed for other programming languages) is quite attractive solution providing the ability to write descriptive automated tests. But this attractiveness hides a myriad ways to "shoot your leg". And the more people use the approach of executable specifications the more people have bad experience. Why this happens?

Article: Is it sane to compare Selenium with QTP

Just read article here: Is it sane to compare Selenium with QTP?. Seems like all the article can be minimized to the following part:

Well, when did Selenium claim that it can tests window based applications. Selenium is browser automation framework. Comparing Selenium and QTP on this front is like comparing apples with oranges.

One paragraph says more than all others. Why such questions appear at all?

P.S.: wonder what's the goal of disabling text highlight operation on the page. If I want to copy\paste content I simply can view the page source.

Ruby: retrieve OLE object from HWND

I continue investigating the possibility to create custom auto-clicker. This time it's for my current project, so there'll be no blackjack. But anyway, the problems still appear and I have to resolve them. Just like this time when I've got the way to find Win32 windows by their specific attributes. Ruby contains magic gems like:

• win32-api
  
• windows-api
  

With their help I can easily get an access to Win32 API functions and call them from Ruby. But it works up to the moment when you encounter windows of Internet Explorer_Server class. Their children are HTML elements and they're not accessible directly via API. It's not a problem if we talk about browser. We can use Selenium or Watir to do all we need. But what if we talk about embeded Internet Explorer_Server windows which are placed on non-browser window? One of the ways is to get an access to corresponsing OLE object and hit it's properties and methods. Oh, good idea. But the main question is: how? All we know is just and HWND of this window. There's nothing impossible and this problem has the solution as well.

Selenium IDE: the way to test the locators

Selenium IDE is quite restricted solution and it's not recommended to use it as a solution for full scale test automation. But there're some areas where it helps a lot. Of course it can record sample tests and convert it into the code. That's the main Selenium IDE feature. But not the only one. Selenium IDE is very helpful when we want to test our locators and make sure they point to proper element. In this post I'll try to make step by step instructions how to do that.

Intro: Our own test automation tool

All the time I have to deal with some test automation tool. And all the time I have to live with the difficulties of each tool. If we have some huge vendor tool it's definitely something VBScript/JScript-based with all related restrictions. Or at least there're restrictions of programming language. It's critical because various wizards are not enough to make robust and reliable tests. Sometimes I have to make quite precise tuning to make my solution working the way I need. And even more, such wizards also consume memory and make overall work slower.

Selenium Camp. Impressions and thoughts

First Selenium conference in Europe took place this Saturday the February 26. It was a good chance to see how people use Selenium, some best practices, approaches. Of course, I had a chance to share my knowledge as well since I was one of the presentors.

Making Git working with Hudson

Once again I've started building CI infrastructure for another test automation project. Everything was clear and obvious. Actually, just another "toolset zoo" was involved. There were: PHPUnit, Selenium-RC, PHPDocumentor, Ant, Hudson and Git. Well, everything was obvious and clear. Ant makes all the stuff to prepare all resources and run single command. Hudson was supposed to download resources from Git and run the build.

Behavior-driven development introduction

What is the behavior-driven development approach. It's just another approach to build automated tests, like decomposition, keyword-driven, data-driven, object-based. Key feature of this approach is that each action is represented with some specific sentence describing actions to be done. And it's usually natural-language instruction. So, we actually describe some behaviour of AUT after some actions with it. So, at the highest level test is represented something like that:

Development skills for test automation

The more time I work with test automation the more times I'm assured that there's nothing to do in this area without development vision of the overall solution. Where it is essential?

SOA Testing: what should we know about it

It's obvoius that testing is not limited by clicks on application GUI. All we have as a part of the product should be tested. And sometimes we can interact on non-GUI level. In particular it key feature of services testing.

Test Automation infrastructure in Java

In general, it's good when you set up your automated testing using the same programming language as the application under test is written on. It provides the integration with unit tests, integration tests which are typically interact on code level.

Another good thing is that all the infrastructure and approaches generally used for development are applicable for test automation.

Choose proper test automation tool

Proper automation starts with proper approach selection and proper processes definition. But these are just first steps which require further correlations and updates as long as automation grows. And in most cases these updates/corellations are related to the specifics of used toolset. So, the more project specifics your toolset fits the less corrective actions you should make in your automation processes.

What should we do for proper test automation tools selection?