Sirius C#: adding UI Automation library

Recently we've created code interacting with Win32 and Web. Now it's time to expand the coverage to the .NET area. For this purpose we have dedicated library called UI Automation. This library is provided with .NET framework and contains basic API interacting with window objects. At the same time this library can be used for interaction with the standard Win32 controls, however it can be used as an auxiliary modules to expand the coverage of existing library I created before for interacting with Win32 elements. In this article I will create sample control classes and make some demo showing how it works. Also, in this example I'll take the tab control and create simple tests interacting with it. At this point that would be just stand-alone library but in the future it will be integrated into entire Sirius project.

The following items will be covered:

1. Adding UI Automation library into the project
2. Finding window objects
   1. Using simple search criteria
   2. Using complex search criteria
3. Interacting with window objects
4. Creating class for common window object
5. Creating class for tab control
6. Sample test

Adding UI Automation library into the project

As it was mentioned before the UI Automation library is provided with .Net framework and can be added as reference. So, in order to add UI Automation library to the existing project we should do the following:

1. Create new C# project or open existing one in Visual Studio
2. Right-click on References node and select Add Reference popup menu item
3. In the references dialog we should switch to .NET tab and select the following items:
   • UIAutomationClient
   • UIAutomationTypes
4. Click OK

After that our project is ready for using UI Automation library

Finding window objects

The core class which provides interaction with UI elements is AutomationElement. It's main entry point to get an access to control internal properties and methods. Initially the only instance accessible is the root element which corresponds to the desktop window. It can be accessed as:

AutomationElement root = AutomationElement.RootElement;

All window objects are child windows of this element. There're 2 main methods which are responsible for search:

• FindFirst - finds first element that matches search criteria
• FindAll - gets the list of all elements matching the search criteria

Both methods have the following parameters:

Find<First|All>(TreeScope scope,Condition condition)

Where:

• scope - identifies the search scope of the element. It can be only child windows or including descendants with different combinations
• condition - the search criteria itself. It is based on the values of some specific properties

The scope parameter is just the constant. The condition parameter require more attention as it is complex structure. Conditions can be divided into simple (containing only one search criteria) and complex (containing the combination of simple criteria).

Using simple search criteria

As it was mentioned before, simple search criteria is based on one property only. To identify such simple condition we should use PropertyCondition class. That can be done in the following way:

Condition nameCondition = new PropertyCondition(AutomationElement.NameProperty, "Common Controls Examples");

That will match the window with the "Common Controls Examples" text. So, if we look for the top level window with such header we should use the code like:

AutomationElement root = AutomationElement.RootElement;
Condition nameCondition = new PropertyCondition(AutomationElement.NameProperty, "Common Controls Examples");
AutomationElement mainWin = root.FindFirst(TreeScope.Children, nameCondition);

After this code the mainWin variable will contain the reference to the automation element accessing the properties and methods of the top level window with "Common Controls Examples" text.

Using complex search criteria

In some cases it's not enough just to look for control using single property. E.g. there may be several elements with some specific text but they have different classes. Or the target control may have several possible property values. For this purpose we should use complex search criteria which includes several settings. The following code shows how to create complex condition based on specified class and text:

        public static Condition ByTypeAndName(ControlType type, String name) 
        {
            Condition[] locators = 
            { 
                new PropertyCondition(AutomationElement.NameProperty, name),
                new PropertyCondition(AutomationElement.ControlTypeProperty, type)
            };

            return new AndCondition(locators);
        }

Using any combination of above conditions we can define search criteria for any object and find that based on criteria specified.

Writing custom conditions object

As it's seen from previous examples the conditions definition is quite complex operation and takes too much code to write. So, it would be reasonable to wrap the most frequently used conditions with some methods which simplify condition definitions. For this purpose I've created the CustomConditions class with the following content:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows.Automation;

namespace Sirius.Win32.Lib
{
    public class CustomConditions
    {
        public static Condition ByHandle(int hwnd) 
        {
            return new PropertyCondition(AutomationElement.NativeWindowHandleProperty, hwnd);
        }

        public static Condition ByName(String name)
        {
            return new PropertyCondition(AutomationElement.NameProperty, name);
        }

        public static Condition ByTypeAndName(ControlType type, String name) 
        {
            Condition[] locators = 
            { 
                new PropertyCondition(AutomationElement.NameProperty, name),
                new PropertyCondition(AutomationElement.ControlTypeProperty, type)
            };

            return new AndCondition(locators);
        }
    }
}

So, in further examples I'll use this class to make instructions more compact.

Interacting with window objects

Once we located the object we should perform some actions on it. Initially the only interface we have is an instance of the AutomationElement class which contains only very generic set of properties and methods. But if we want to perform some control specific operation we should use something else. Each control specific set of actions is wrapped with specific bundles or patterns. They contain methods and properties specific to some control types. E.g. general window manipulation functionality is wrapped with WindowPattern class. If specific object supports that pattern the functionality will be accessible. Otherwise we'll get the exception.

So, let's try to make some sample. Let's say we should find some top level window with the "Common Controls Examples" text and then close it. It can be done in the following way:

AutomationElement root = AutomationElement.RootElement;
Condition nameCondition = new PropertyCondition(AutomationElement.NameProperty, "Common Controls Examples");
AutomationElement mainWin = root.FindFirst(TreeScope.Children, nameCondition);

WindowPattern winOps = tabElement.GetCurrentPattern(WindowPattern.Pattern) as WindowPattern;
winOps.Close();

The highlighted part shows the example of window object interaction. Actually, AutomationElement object contains only references while pattern classes are responsible for performing actions on objects.

Creating class for common window object

Before, I've described how to perform interaction with window objects using UI Automation library. But the library itself is very generic and contains only basic abstractions. If we want to make some library we should wrap that functionality with more usable interface. Firstly, we'll create class for common window operations. At the moment we should wrap only search functionality. So, we'll create the class skeleton:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows.Automation;

namespace Sirius.Win32.Lib
{
    public class Window
    {
    }
}

Next step is to wrap the reference to root element because AutomationElement.RootElement statement is quite long while it's quite frequently used. So, we'll update our class with the following code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows.Automation;

namespace Sirius.Win32.Lib
{
    public class Window
    {
        public AutomationElement Root 
        {
            get 
            {
                return AutomationElement.RootElement;
            }
        }
    }
}

Then we should add methods which search for windows. Currently I need the following searches:

• By specified window handle
• By name
• By specified name and parent window

The code implementing those methods looks like:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows.Automation;

namespace Sirius.Win32.Lib
{
    public class Window
    {
        public AutomationElement Root 
        {
            get 
            {
                return AutomationElement.RootElement;
            }
        }

        public AutomationElement Find(int hwnd) 
        {
            return Root.FindFirst(TreeScope.Subtree, CustomConditions.ByHandle(hwnd));
        } 

        public int Find(String className,String name,int index) 
        {
            AutomationElementCollection elements = Root.FindAll(TreeScope.Children, CustomConditions.ByName(name));
            if (elements.Count <= index) 
            {
                return 0;
            }

            AutomationElement element = elements[index];

            return element.Current.NativeWindowHandle;
        }

        public int Find(int parent, String className, String name, int index) 
        {
            AutomationElement baseElement = Find(parent);
            if (baseElement == null) 
            {
                return 0;
            }

            AutomationElementCollection elements = baseElement.FindAll(TreeScope.Subtree, CustomConditions.ByName(name));
            if (elements.Count <= index)
            {
                return 0;
            }

            AutomationElement element = elements[index];

            return element.Current.NativeWindowHandle;
        }
    }
}

That's it. Now we're done with common window class. Current implementation of it can be found here.

Creating class for tab control

For the tab control we should firstly define class containing functionality interacting with controls of that type. It looks like:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Sirius.Win32.Lib.Controls.Interfaces;
using System.Windows.Automation;

namespace Sirius.Win32.Lib.Controls
{
    public class Tab : Window
    {
    }
}

All other functionality to be added is about searching for element and getting/setting selection. For that purpose we should use SelectionItemPattern class.

Find child control

Since tab control is always child element we should make a search based on parent window. Also, there can be multiple elements of that type. It means that the unique identifier here is index. Based on these assumptions we should add the following method to the Tab class:

        public int Find(int parent, int index) 
        {
            AutomationElement baseElement = base.Find(parent);
            Condition locator = new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Tab);
            AutomationElementCollection elements = baseElement.FindAll(TreeScope.Subtree, locator);
            if (elements.Count <= index) 
            {
                return 0;
            }
            return elements[index].Current.NativeWindowHandle;
        }

The highlighted part contains the code initializing search criteria. We search for element by specified type defined by ControlType.Tab constant.

Get tab control values

Once we can find tab control element we can get the information from it. Currently what we need is:

• Total number of tabs
• Currently selected tab index
• Currently selected tab name
• Get all available item names

All those methods are done with the following code:

        public int GetItemsCount(int hwnd) 
        {
            int count = 0;
            AutomationElement element = Find(hwnd);
            AutomationElement tabElement = TreeWalker.RawViewWalker.GetFirstChild(element);

            if (tabElement != null) { count++; }

            while (tabElement != null)
            {
                count++;
                tabElement = TreeWalker.RawViewWalker.GetNextSibling(tabElement);
            }

            return count;
        }

        public int GetSelectedIndex(int hwnd) 
        {
            int index = -1;

            AutomationElement element = Find(hwnd);
            AutomationElement tabElement = TreeWalker.RawViewWalker.GetFirstChild(element);

            while (tabElement != null)
            {
                index++;
                SelectionItemPattern changeTab_aeTabPage = tabElement.GetCurrentPattern(SelectionItemPattern.Pattern) as SelectionItemPattern;

                if (changeTab_aeTabPage.Current.IsSelected)
                { 
                    return index;
                }
                tabElement = TreeWalker.RawViewWalker.GetNextSibling(tabElement);
            }
            
            return -1;
        }

        public String GetSelectedItem(int hwnd)
        {
            return GetItemNames(hwnd)[GetSelectedIndex(hwnd)];
        }

        public String[] GetItemNames(int hwnd) 
        {
            List elementNames = new List();

            AutomationElement element = Find(hwnd);
            AutomationElement tabElement = TreeWalker.RawViewWalker.GetFirstChild(element);

            while (tabElement != null)
            {
                elementNames.Add(tabElement.Current.Name);
                tabElement = TreeWalker.RawViewWalker.GetNextSibling(tabElement);
            }

            return elementNames.ToArray();
        }

Perform selection

Selection can be done by specifying either tab name of tab index. That's reflected in 2 methods performing the same action but with different input parameters. They are:

        public void Select(int hwnd,int index) 
        {
            int count = 0;

            AutomationElement element = Find(hwnd);
            AutomationElement tabElement = TreeWalker.RawViewWalker.GetFirstChild(element);

            while (tabElement != null)
            {
                if (count == index)
                {
                    SelectionItemPattern changeTab_aeTabPage =
                        tabElement.GetCurrentPattern(SelectionItemPattern.Pattern) as SelectionItemPattern;

                    changeTab_aeTabPage.Select();
                    return;
                }
                else 
                {
                    count++;
                }
                tabElement = TreeWalker.RawViewWalker.GetNextSibling(tabElement);
            }
        }

        public void Select(int hwnd,String item) 
        {
            AutomationElement element = Find(hwnd);
            AutomationElement tabElement = TreeWalker.RawViewWalker.GetFirstChild(element);

            while (tabElement != null)
            {
                if (tabElement.Current.Name.Equals(item))
                {
                    SelectionItemPattern changeTab_aeTabPage =
                        tabElement.GetCurrentPattern(SelectionItemPattern.Pattern) as SelectionItemPattern;
                    changeTab_aeTabPage.Select();
                    return;
                }

                tabElement = TreeWalker.RawViewWalker.GetNextSibling(tabElement);
            }
        }

These are all methods we need at the moment. Current implementation of Tab control class can be found here.

Sample test

Once we have all necessary code we can create some sample test which shows how to use that API. The scenario is pretty simple:

1. Open "Common Controls Examples" application (can be found here)
2. Get list of all tab names
3. For each tab name perform the following:
   1. Select tab by specific name
   2. Verify that the tab with specified name is selected
   3. Verify that the tab with the specified index is selected

The entire test code looks like:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using NUnit.Framework;
using System.Diagnostics;
using Sirius.Win32.Lib;
using Sirius.Win32.Lib.Controls;

namespace SiriusCSharp.Client.Tests.Tests.Win32Lib
{
    public class TabControlTests : ControlTestsCommon
    {
        protected Process controlsApp = null;
        protected Window win;
        protected Tab tab;
        protected int mainHwnd;

        [SetUp]
        public void Before()
        {
            controlsApp = Process.Start( @"D:\Work\SiriusDev\Sirius\TestApps\win32\Controls.exe");
            win = new Window();
            mainHwnd = win.Find("", "Common Controls Examples", 0);

            tab = new Tab();

        }

        [TearDown]
        public void After()
        {
            controlsApp.Kill();
        }

        [Test]
        [Category("TabControl")]
        public void TestTabNames() 
        {
            int htab = tab.Find(mainHwnd, 0);

            String[] names = tab.GetItemNames(htab);
            int index = 0;
            foreach(String name in names)
            {
                tab.Select(htab,name);
                Assert.AreEqual(index, tab.GetSelectedIndex(htab));
                Assert.AreEqual(name, tab.GetSelectedItem(htab));
                index++;
            }
        }
    }
}

Current implementation can be found here.

Summary

Current sample just shows the way to use UI Automation library. In the future the library created can be expanded to cover more different control types. Even more, that library can be provided as the standalone package.