Test Automation Tool

Elements of a Test Automation Tool

I have someone show me or tell me about a new automation tool a couple of times a year. Sometimes it is something the person made themselves or a product they read about somewhere. Other times, it is a partner that is showing off their automation and tries to say that it covers everything.

The tool usually comes down to a single part of what I’m about to describe. The ability to write a script, capture a screenshot, push buttons or for the code-centric, a library, harness or development environment addition is usually a long ways away from having all the pieces required to do automation.

I’m sure I’ve forgotten some things. I’m also sure that some are unnecessary in certain environments. But, most of these are generic and are required in some form.

So, here is my list:

Provide a way to program (from least to most sophisticated)
–API’s for use with C
–vendor script (Like 4Test that is used with SilkTest)
–open source language with interface (Perl, Python, Ruby or PHP)
–action word (or even multi-level action word) system

UI Elements
–identify all UI elements native to the OS
–provide a way to identify custom elements, gadgets and other non-native elements
–associate each instance with a name (or multiple names, for robustness)
–give them names (provide a naming structure). Sometimes call declarations.
–provide a bitmap and coordinate system as a fallback

Framework hooks
–start a script
–start a testcase
–end a testcase
–end a script

Test case management system
–way to identify a test case
–way to identify a test suite
–way to feed data into a test case (making it a “data driven” test case)

Test Execution
–Ability to execute a single test case (from the command line)
–Ability to execute a test suite
–Ability to execute a group of test suites

Logging system
–Pass, Fail, Skip
–Capture error messages
–Ability to log warnings and other debugging text
–Ability to save log to a file or to a remote system/database

Validation function (did the test pass?)
–Bitmap compare
–String compare
–Window exists

Agent (or other method to accomplish the same thing)
–Run the least possible code on the system under test
–Have a way to communicate with the system under test (via TCP/IP, serial, USB, etc.)
–Have another way to communicate with the system under test (you may want to test the communication channel you usually use for control, so there needs to be a way to switch to a secondary channel).
–Have a way to automatically start and stop the agent code (at boot or at application launch)
–Have a way to install and uninstall the agent without user interaction (including it in the build is not good enough, unless you are willing and able to ship it as part of the product).
–Have source code to the agent. This is essential if you are working on a non-product OS (i.e. you are developing an OS or parts of an OS and not just an application).


No comments yet

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: