Desktop Application Automation:

.NET Developers perspective

www.luxoft.com
Automation definition and goals
Microsoft UI Automation (UIA) is an application programming interface (API) that allows one
to access, identify, and manipulate the user interface (UI) elements of another application
[Wikipedia]

Application testing
Accessibility scenarios

www.luxoft.com
Availability
Available since Windows Vista / Windows Server 2008
Part of Windows 7 and above right after OS install
Automates Win32, WinForms, WPF, UWP and other applications
Ability to automate depends on UIA support that can be implemented for any UI in theory
Frameworks and technologies listed above have UIA support out of the box

www.luxoft.com
UIA terminology
Server application that is being automated
Client automation driver, e.g. automated UI test
AutomationElement basic building block; every piece of UI that is exposed to automation
is represented with AutomationElement. They are contained in tree structure with desktop
as the root
Theres no one-to-one relationship between automation elements and controls, but often it
is the case
Control Patterns support the methods, properties, events, and relationships needed to
define a discrete piece of functionality available in a control

www.luxoft.com
History and UIA versions
UIA is a successor to Microsoft Active Accessibility (MSAA)
MSAA functionality is preserved in UIA in form of LegacyIAccessible pattern
UIAv2
Initial implementation
Set of methods consumed with simple P/Invoke
UIAv3
Exists side-by-side with UIAv2 in the same dll
COM object implementation

www.luxoft.com
UIAv2 vs UIAv3
UIAv2
Better support for old applications, like WinForms or Win32
Some stability issues
Has managed .NET UIA wrapper
UIAv3
More stable, faster
Allows custom user-defined patterns
Due to different implementation, control tree for some standard controls differ from UIAv2
No Microsoft supported means to consume from .NET application

www.luxoft.com
Helper libraries
Since both P/Invoke and COM are inconvenient in .NET, several helper libraries emerged:
TestStack.White
Based on Managed UIA wrapper
Has separate branch using UIAv3, mostly preserving same API layout
FlaUI
Most recent, supports both UIAv2 and UIAv3 out of the box
More extensible

www.luxoft.com
Core features
UIA elements discovering and identification
Endpoint on UIA server to communicate with
Communication protocol
Getting elements properties:
What is the text on this button I have?
Invoking elements interactions:
Click on the button
Getting notifications on UIA events
Call delegate every time new window is opened

www.luxoft.com
Tools
UIAv2
UISpy
UIAv3
Inspect
VisualUIVerify
FlaInspect

www.luxoft.com
Architecture picture

www.luxoft.com
UIA client specifics
Initialize UIA
Get root AutomationElement
Descend via automationElement.Find*() family of methods to find necessary element
Get its pattern implementation you need with GetPattern method
Get properties and/or invoke pattern methods as necessary

www.luxoft.com
UIA server specifics
Entry point is WM_GETOBJECT windows message
Server needs to return pointer to raw element provider (IRawElementProviderSimple
implementation in most cases)
That implementation is called by core UIA for patterns, methods and properties
For example, to get value of some property its ID is provided by core UIA and value is
expected in return
To invoke pattern method ID of the method is provided along with object[] with invocation
arguments
Very tedious to implement directly
Good thing is, all standard MS controls implement this out of the box

www.luxoft.com
WPF/UWP specifics
Since WPF/UWP dont have HWND for each control they cant receive WM_GETOBJECT
directly at control
Instead, it provides WM_GETOBJECT functionality on the window-level, and everything else
is done inside WPF
This functionality relies on AutomationPeer class (or descendant) exposed from
UIElement.OnCreateAutomationPeer method, and visual tree to discover child elements
Thus, if you have custom control in WPF that does not have UIA implemented (and mostly it
is the case with third-party controls like Telerik or DevExpress) and you need to automate it
you have to provide AutomationPeer for it

www.luxoft.com
Resources
MSDN page on UIA
https://msdn.microsoft.com/en-us/library/ms747327(v=vs.110). aspx
TestStack.White
https://github.com/TestStack/White
FlaUI
https://github.com/Roemer/FlaUI
Custom UIA patterns usage in .NET
https://github.com/TestStack/uia-custom-pattern-managed
Series of blog posts on custom patterns in UIA by M. Bernstein
https://blogs.msdn.microsoft.com/winuiautomation/2010/12/07/uia-custom-patterns-part-1 /
https://blogs.msdn.microsoft.com/winuiautomation/2010/12/10/uia-custom-patterns-part-2 /
https://blogs.msdn.microsoft.com/winuiautomation/2010/12/27/uia-custom-patterns-part-3 /

www.luxoft.com
 THANK YOU

Q&A

www.luxoft.com