Recently I’ve been working with a client and the testing team use HP/Mercury Quality Center (QC) and related testing products. The product they use to do automated functional UI testing of windows and web applications is HP/Mercury Quick Test Professional.

One of my current goals for this client is to get a complete end-to-end CI build, test, deploy up and running with Team Build. One of the bright testers that we have came up with the idea of automatically running the QuickTest Pro (QTP) tests as part of the build process.

It turns out that QTP comes with a COM automation object that we can use to drive the unit tests with. This was a perfect candidate for a custom MSBuild task.

The <QuickTestPro> MSBuild task

This task was inspired by the VbScript samples in the QTP AutomationObjectModel.chm help file & Carel Lotz’s blog post on Part 7: Continuous Integration - The QtpBuild.

You can download the source code for the task from here: QuickTestPro.cs.txt

This custom task implements the Microsoft.Build.Utilities.Task base class and does the following things:

• Create an instance of QuickTest.Application COM object
• Launch the application
• Remove the read-only attribute for the test case directories
• Run the tests for each directory defined in the TestCases property of the task.
• Remove the existing object repositories, as these are stored in a binary format and usually map to a path on the test developer’s local machine
• Add the object repositories (.tsr files) specified by the ObjectRepositories property of the task.
• Run the tests and place the results in the $(ResultsLocation)test-script-name folder, which then get copied to the drop location.
• Log the results
• Close the QuickTest Pro application

Here’s the guts of the code:

public override bool Execute()
{
    bool isSuccess = true;

    QuickTest.Application qtApp = new QuickTest.ApplicationClass();

    qtApp.Visible = true;
    qtApp.Launch();

    qtApp.Options.Run.ImageCaptureForTestResults = “OnError”;
    qtApp.Options.Run.RunMode = “Fast”;
    qtApp.Options.Run.ViewResults = false;

    QuickTest.RunResultsOptions qtResultsOptions = new QuickTest.RunResultsOptions();

    // Run each of the test cases specified.
    foreach (ITaskItem testCaseItem in TestCases)
    {
        string testCase = testCaseItem.ItemSpec;

        if (!Directory.Exists(testCase))
        {
            Log.LogError(”Test case does not exist: {0}”, testCase);
            break;
        }

        // Remove read only attributes for all files in test case directory
        foreach (string file in Directory.GetFiles(testCase, “*”, SearchOption.AllDirectories))
            File.SetAttributes(file, FileAttributes.Normal);

        qtApp.Open(testCase, false, false);

        // Replace ObjectRepositories with our own one.
        if (File.Exists(ObjectRepository))
        {
            for (int i = 1; i <= qtApp.Test.Actions.Count; i++)
            {
                int count = qtApp.Test.Actions[i].ObjectRepositories.Count;
                if (count > 1)
                {
                    Log.LogError(”{0} ObjectRepositories specified. Only 1 supported”, count);
                    return false;
                }

                qtApp.Test.Actions[i].ObjectRepositories.Remove(1);
                qtApp.Test.Actions[i].ObjectRepositories.Add(ObjectRepository, 1);
            }
        }

        // Instruct QuickTest to perform next step when error occurs
        qtApp.Test.Settings.Run.OnError = “NextStep”;

        // Set results location for test name
        qtResultsOptions.ResultsLocation = Path.Combine(ResultsLocation, qtApp.Test.Name);

        // Run the test
        Log.LogMessage(MessageImportance.High, “Running QuickTest {0} from {1}”, qtApp.Test.Name, qtApp.Test.Location);
        qtApp.Test.Run(qtResultsOptions, true, null);

        // Log the result
        Log.LogMessage(MessageImportance.High, “QuickTest Result: {0}”, qtApp.Test.LastRunResults.Status);
        isSuccess = isSuccess && (qtApp.Test.LastRunResults.Status == “Passed” || qtApp.Test.LastRunResults.Status == “Warnings”);

        qtApp.Test.Close();
    }

    qtApp.Quit();
    qtResultsOptions = null;
    qtApp = null;

    return isSuccess;
}

You will need to compile this task and copy the assembly to your build server. C rogram FilesMSBuildQuickTestPro.MSBuildTasks.dll is what I’ll use.

One thing to note when using the QuickTest.Application COM object via a .NET Interop is that all the array indexes are base = 1. This means that you will have difficulties using foreach() and for (int i = 1; i < qtSomething.Count; i++) is the best option.

Setting up Team Build

For this task to work, it needs to interact with the desktop. Because Team Build normally runs as a service, it doesn’t have a desktop to interact with! Team Build 2008 supports running in interactive mode for this exact purpose. You should follow the steps in my previous blog post for Running an Interactive Team Build 2008 Agent.

Modifying TFSBuild.proj to run the tests

To actually get Team Build to run the QuickTest Pro tests, we need override the AfterTest target and call our new target.

In source control, checkout the TFSBuild.proj file for your build definition and insert the following snippet at the end, just before the </Project> tag:

<Target Name="AfterTest">    <CallTarget Targets="QuickTest" /> </Target> <UsingTask TaskName="QuickTestPro" AssemblyFile="$(MSBuildExtensionsPath)QuickTestPro.MSBuildTasks.dll" /> <Target Name="QuickTest">    <PropertyGroup>      <QuickTestScriptDir>$(SolutionRoot)UItestsquick-test1;$(SolutionRoot)UItestsquick-test2;</QuickTestScriptDir>    </PropertyGroup>    <QuickTestPro      TestCases="$(QuickTestScriptDir)"      ResultsLocation="$(TestResultsRoot)"      ObjectRepositories="$(SolutionRoot)UItestsrepository1.tsr;$(SolutionRoot)UItestsrepository2.tsr;"      ContinueOnError="true"    /> </Target>

The task uses the ITaskItem[] type for the TestCases and ObjectRepositories properties which means we can pass in a single value or an array of values separated by semi-colons.

Possible Extensions

• Transform the QTP results.xml files to a Visual Studio Test Results File (*.trx) and publish the results to TFS so that you get data warehouse metrics on UI tests.
• Dynamically run all found test scripts/object repositories by adding <CreateItem Include=”$(SolutionRoot)tests*”> to the QuickTest target.

More Information

For more information, the following resources will be helpful:

• AutomationObjectModel.chm from the C rogram FilesMercury InteractiveQuickTest Professionalhelp directory of a machine with QTP installed. (Documentation for the Application Class is a good place to start)
• MSDN Microsoft.Build.Utilities.Task base class
• MSDN How to: Write a Task