Test Automation infrastructure in Java

In general, it's good when you set up your automated testing using the same programming language as the application under test is written on. It provides the integration with unit tests, integration tests which are typically interact on code level.

Another good thing is that all the infrastructure and approaches generally used for development are applicable for test automation.

Let's take a look at typical test automation solution written on Java. I'm going to make high-level overview of what we have and what we can use. I'll describe each particular case in separate posts. Current post will just give an overview of what we can use for test automation solution written in Java.

What does it have inside? In most cases all tests are done using some specific test engine like JUnit, TestNG. At least, it's first thing we can take while starting automated tests development. So, basically it's some set of test classes + some common API.

What we're interesting first before running the tests? Of course, we should check whether our solution is compilable at all. It's done by standard java tooling, e.g. javac.

When we write some auxilary API we should take care of code reusability. If so, we have some functionality which is widely used within tests as well as some auxilary functionality which can be used in many other projects with similar technology set. As the result, it's nice to have this code properly documented. Though it's boring task it should take place. And javadoc can help us with this. We just have to write javadoc-specific comments for our methods, classes, variables etc.

A lot of attention should be paid to the code quality itself. It should follow some conventions, it shouldn't contain any risky code constructions, repetitive code should be minimized to maintain code easily. Typically it can be handled by code review. But in most cases such task is boring with a number of not essential checkpoints which can be verified without human interaction. Actually, developers already use tools which can make basic code analysis automatically. E.g. jlint tool can check whether code conforms naming and formatting conventions.

In addition to that there's specific code analyzer like PMD. It looks for dead code, suspicious try/catches, unsafe loops and code "generated" by copy/paste. All this information can be collected with minimal human interaction.

And the last thing left for automated testing is to run the tests and collect results. It's all about test engines used. Though, we can use common driver which:

1. Compiles the source code
   
2. Checks the code for correctness
   
3. Generates javadoc
   
4. Starts test execution and collect results
   

E.g. we can use ANT for this task. It already contains tasks for java code compilation and documentation generation in basic complectation. Code analyzers for Java usually have their own ANT-task implementations. TestNG and JUnit test engines have separate tasks for test execution and results collection in appropriate formats. And finally, we can setup e-mail notification about results by means ANT tasks. Is that all? I wouldn't be so quick in such decision. All that stuff can be executed by some schedule or even continuously. Ideally, it should be accompanied by unit-tests as well as application under test compilation and deployment/installation. But it's too time consuming to be done on regular basis. At the same time we can use another thing widely used for development. I'm talking about continuous integration solutions like Cruise Control, Hudson, Team City etc. When developer commits the code into version control system this event typically triggers code compilation task. If there're some problems with compilation the error notification is sent. If we make our test automation solution on the same programming language as application under test is written we can use continuous integration tools in the same way. Integrating all the above stuff we can set the requirements which can be controlled automatically:

1. All the code doesn't contain any warnings and errors (at least errors but warnings are undesirable too)
   
2. All the code is formatted properly and no suspicious code is in use
   
3. Each test run is performed either by single click or even started automatically
   
4. All things people should do with already written tests is to make them up to date and analyze the output
   

With properly selected toolset such goals can be easily achieved.

The only thing left is to make sure that people are ready for that as traditional test automation tools brought a lot of "bad habits". We have to be ready for such integration and control if we want to make really working and high-quality solution. All the stuff described above is widely used in development in order to provide high quality solution. So, test automation is good to go the same way. And we have to be ready for that.