Problem Solved: run TestNG test from JUnit using Maven

Background

Recently I've been developing some applicaiton component which was supposed to run with TestNG. Actually, it was another extension of TestNG test. So, in order to test that functionality I had to emulate TestNG suite run within the test body. Well, performing TestNG run programmatically wasn't a problem. It can be done using code like:

import org.testng.TestListenerAdapter;
import org.testng.TestNG;

.......
        TestListenerAdapter tla = new TestListenerAdapter();
        TestNG testng = new TestNG();
        testng.setTestClasses(new Class[] {SomeTestClass.class});
        testng.addListener(tla);
        testng.run();
.......

where SomeTestClass is some existing TestNG class. This can be even used with JUnit (which was my case as I mainly used JUnit for the test suite). So, technically TestNG can be executed from JUnit test and vice versa.

Problem

The problem appeared when I tried to run JUnit test performing TestNG run via Maven. Normally tests are picked up using surefire plugin which can be included into Maven pom.xml file with the entry like this:

 <plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.18.1</version>
 </plugin>

If you use JUnit only it picks up JUnit tests. In my case I also used JUnit for running test suites but one test required TestNG test class (actually the class with TestNG @Test annotation) as well as I had to add TestNG Maven dependency. In this case only this TestNG class was executed during test run. So, how to let Maven know that I want to run exactly JUnit tests but not TestNG ones while both of them are present within the same Maven project?

Aerial: Introduction to Test Scenarios Generation Based on Requirements

In one of the previous articles dedicated to moving to executable requirements I've described some high-level ideas about how such requirements should look like. Main idea is that document containing requirements/specifications can be the source for generating test cases. But in the previous article we just made brief overview on how it should work in theory. Now it's time to turn theory into practice. In this article I'll make brief introduction to new engine I've started working with. So, let me introduce the Aerial the magic box which makes our requirements executable.

In this article I'll describe main principles of this engine and provide some base examples to bring an idea of what it is and where it is used.

GitHub: integrating project into entire GitHub infrastructure

GitHub provides quite good set of toolsets which can be used to support full development cycle and prepare infrastructure to all necessary output artifacts. Recently, I've described samples how to provide integration with GitHub issue tracker and to wrap it with Maven plugin. So, actually, I have dedicated Maven plugin project which I can use as a separate project. In order to make it as much self-contained as it's needed we should:

1. Set up new project in the GitHub which includes:
   1. New repository setup
   2. Build script preparation
   3. Release settings
2. Prepare project documentation generation
3. Integration with CI

So, at the end, all we have to do is to work with our IDE on adding new code or making some changes to existing modules. Everything else is supported by GitHub infrastructure as soon as we push our changes into remote repository.

Not bad at the beginning. So, let's get started.

GitHub: test tracking via Maven plugin

Recently we've organized integration with GitHub by retrieving issues content into the files. All this was done in a form of command line utility. In this post I'll extend the functionality to use this solution as Maven plugin.

GitHub: organizing and automating test tracking with Java

GitHub provides some tracker where we can keep our tasks. The GitHub API provides the ability to access that resources programmatically. Thus, with the help of API I could integrate JBehave and GitHub so that I can run JBehave tests stored in GitHub tracker. But it's not the only case.

With the help of API I can set the correspondence between tests and features. This gives the ability to make the traceability of tests and features they're applied for. Also, what if we use something different from JBehave, e.g. Cucumber, Sprecflow which use flat files for input? In this article I'll describe how to:

• Organize tests with GitHub tracker
• Automatically generate traceability matrix
• Automatically generate feature files based on the content of the test issues

Sirius: Adding code style checks

We already set up the build process for Sirius as well as we reserved the placeholders for testing. So, further improvement in this area is to extend build with different tests and checkpoints which make more detail verification. This time we'll add more modules which perform static testing for our components and they should fail the build if static checks didn't pass (especially taking into account that it was initially planned).

What would we look for? Basically that's:

• Conformity to coding standards - all code standards are made to provide unified approach of writing code as well as reading and maintaining it.
• Various cases of improper code usage - a lot of actual application errors are results of improper code use or use of potentially dangerous code constructions. So, the earlier we find them the less number of silly mistakes we take during further testing stages

Actually, we've made first approximation for that while we checked the compilation ability. If system isn't compilable it's not working at all. So, we're just making more detailed checks.

Since Sirius modules use different programming languages the tool set will be varying based on that. We'll use the following tool set for that:

Language	Engine
Java	Check Style PMD
C#	StyleCop
Ruby	rubocop

OK. Let's see how it would be added.

Java: JBehave integration with JIRA

During BDD engines comparison I noticed that JBehave supports various data sources for the stories but not just text files. This is quite valuable feature as we can share the stories accross the teams. But what if we share that using existing infrastructure? I mean all the tests are typically stored under some tracking system where they can be easily queried from. Such tracking systems should be the place where the tests are written to and updated at. So, why shouldn't we integrate that with the engine so that tests are designed, stored and modified in the tracking system and the engine like JBehave picks them up from there? That would be the tight integration between tests and their automated implementation. In this post I'll show how to make such integration between JIRA and JBehave.

Sirius: Testing setup

Since Sirius is going to be used for testing purposes it's not polite to leave it without testing. Major specifics of this solution is that different parts of entire system are written using different programming languages. At the same time all those parts reflect the same functionality. It means that the tests would be the same for all clients. Well, they would be the same for server part as well (at least most of the core tests). It's not very nice to copy/paste them and then migrate to different languages. So, obviously there's going to be a lot of duplicating work. So, I have to do something to minimize that duplication as well as minimize the maintenance effort in case I want to change some tests. It's valid especially for client side as core part doesn't have any local specific logic. All immediate client methods are actually callers for the same server side methods.

So, there's going to be multiple languages but the same interface for tests. It's not trivial solution but I found that.

Sirius: Dev environment setup

Introduction

All right, I've decided to start development but it's not enough just to write the code. I also should be able to build the packages, test them somehow, provide some documentation and do many other activities. But firstly, I don't want to make a lot of routine work by myself. Let the machine do that for me. Also, I want to be sure that the way the system works for me locally is the same as for any other machine. And eventually, I don't want to keep the source code only locally. It's unsafe as my local machine can be damaged and all information would be lost in this case. So, we need some additional software which makes the development infrastructure and which should be configured in the way convenient to use.

This article is about to describe the infrastructure setup as well as how to configure all this software to point to each other.

Test Automation tools: trends

While making decision which tool is more suitable for test automation we should always take into account the popularity of each specific tool. Why? Because firstly it's about people. What's the point in taking tool which doesn't have users? Thus, we'll have to invest some money into their learning which is extra expences. Secondly, the more popular tool is the more materials are available either in internet or in some other materials. So, we should take a look at some trends to identify which tool is growing with it's popularity and which one is going to be a history.

Test Automation infrastructure in Java

In general, it's good when you set up your automated testing using the same programming language as the application under test is written on. It provides the integration with unit tests, integration tests which are typically interact on code level.

Another good thing is that all the infrastructure and approaches generally used for development are applicable for test automation.