Test Synchronization

See Tasktop Editions table to determine if your edition contains Test Synchronization functionality.

Introduction

Tasktop Integration Hub offers integration solutions to flow test artifacts such as test results, test steps, and their associated tests, test runs, test instances, and folder structures.  Please review sections below to learn more about supported test scenarios in Tasktop

Test Step Synchronization

Test Step synchronization is currently supported for the following artifacts:

  • Design Steps on ALM Tests
  • Run Steps on ALM Test Runs 

To flow test steps, please follow the instructions below.

(lightbulb) Note: If you would like to flow test steps in Tricentis Tosca, please see details below.  


Repository Connection

First, connect to your ALM repository by following the instructions here

You can learn more about ALM-specific configuration in our Connector Docs.

Model

You can find general details on how to create a model here.

To flow test steps, when creating your Test model, select 'Model with Test Steps' as your model type.

This provides a model configuration pane with two panels:

  1. Test Fields: In the top section, add any fields you'd like to flow on the test (or test run) artifact that are not part of its associated test steps.
  2. Test Step Fields: In the bottom section, add any fields you'd like to flow that are a part of test steps.  You'll see that Tasktop provides some Smart Fields that are test step specific to help you get started, but you can add any other desired fields by leaving the Smart Field blank.

Test Step Smart Fields

Here is an example of a very simple Test Model with Test Steps:

Example Model with Test Steps

Converting Model Types

Standard Models can be converted to Test Models simply by choosing 'Model with Test Steps' in the Model Type drop down.

Models with Test Steps cannot be converted back to Standard Models, but this should not present any challenges to integration scenarios.  If Test Step Flow is not enabled in the related integrations, the test step fields in the model will simply be ignored.

Collection

You can find general details on how to create a collection here.

You will see a 'Map Test Fields' sash on your collection if:

  • Your model is a Model with Test Steps, and
  • Your artifact is either ALM Tests or ALM Test Runs

Test Steps Field Mapping Sash

The process to map test step fields is very similar to the process on the normal Field Mapping screen.  Note that both relationship(s) and other field types for test steps will be mapped on this one sash.

(lightbulb) Tests and Test Steps do not require a typical relationship field mapping to link them.  We've added behind-the-scenes smarts to couple them for you.

Integration

You can find general details on how to configure an integration here.

In order to see a Test Step Flow link on the integration configuration screen, the following conditions must be met:

  • Model is of type 'Model with Test Steps'
  • Artifacts in both collections have test steps
  • The relevant Tasktop connectors must support test steps (see Connector Documentation to confirm)

Test Step Flow Link

Clicking the link will bring you to the Test Step Flow screen, where you can click 'Select' to choose your desired Test Step Flow style:

Test Step Flow Screen

(info) Due to the sequential nature of test steps, test step flow has some unique properties:

  • Test steps can only flow in one direction
  • The target collection’s test steps will be kept in strict synchronization with the source collection: this means that all updates, deletions, and re-ordering of test steps will be mimicked in the target collection. Since the source collection is the system of record, any test steps that exist in the target collection but not the source collection will be removed.
  • If the test steps on the target artifact are changed by an end-user, they will be updated by Tasktop when one of the fields or ordering on the source artifact's test steps is changed.

Note: comments and attachments are not currently supported on test steps.

Test Result Synchronization

Many organizations have been using Micro Focus ALM (aka Quality Center) for quality management for years. But where once it was the only tool used for testing, today enterprises are augmenting ALM with additional tools to align with their agile and test automation efforts. That includes tools like Tricentis Tosca. Even as new tools are introduced, ALM remains popular and continues to play an important role in test management, especially when it comes to manual testing, defect management, and quality reporting.

The challenges for QA teams and leadership are how to restore visibility into coverage, quality, and cost, now that testing data is split across multiple tools.

Tasktop enables users to flow test results into Micro Focus ALM in order to take advantage of ALM's reporting capabilities while using other tools, such as Tricentis Tosca, for their test planning and execution.  

The method outlined below will enable you to flow test results into Micro Focus ALM from Tricentis Tosca or from another ALM instance.  Due to the architectural specificity of each external tool, the methods below cannot be used for other endpoints.

You can watch this demo video to learn more:

Test Architecture

Before you begin configuring your integration, it's important to understand how test artifacts relate to one another.

While the goal of this integration is to flow test results, the architecture required to do so is more complex than one might assume.  Test Results are a field on Test Runs.  To create a Test Run in ALM, a Test Instance must exist.  For a Test Instance to exist, both a Test and a Test Set must exist (the test is 'added' to the test set and that creates a test instance).  For a Test to exist, you need a Test Folder.  And for a Test Instance to exist, you need a Test Set and a Test Set Folder.  That's 6 different artifacts, just to flow a Test Run!

ALM Test Architecture

But don't worry - instead of six complex integrations, Tasktop cuts that configuration in half.  To set up this integration scenario, you will set up three integrations:

IntegrationContainer AContainer BWork Item
Test Design

Test Folder


--Test
Test Planning

Test Set Folder


Test SetTest Instance
Test Execution----Test Run

Test Synchronization Set-Up

Once configured, your integrations will look like the images below.

(lightbulb) To keep your integrations in order, we recommend appending a number to the beginning of each title, i.e. "1 - Test Design," "2 - Test Planning," "3 - Test Execution"

Test Integrations - Landscape View

Test Integrations - List View

Before You Begin

Before you get started, familiarize yourself with the following steps of integration configuration:

Connecting to your External Tools

Creating a Model

Creating a Collection

(lightbulb) Review the details in the sections below to ensure that any required fields are mapped in your collection

Configuring an Integration

Integration 1: Test Design 

The first integration you will configure is a Container + Work Item Synchronization flowing Test Folders/Test Case Folders (container) and Tests/Test Cases (work item). 

Test Design Integration

Containers Supported

  • Micro Focus ALM Test Folders 
    • Parent field must be mapped to preserve folder hierarchy
  • Tricentis Tosca Test Case Folders
    • Parent field must be mapped to preserve folder hierarchy

Artifacts Supported

  • Micro Focus ALM Tests
    • Subject field must be mapped (this points to the Test folder)
    • When flowing tests out of ALM, multiple test configurations are not supported.  Tests must have a single test configuration.
  • Tricentis Tosca Test Cases
    • Test Case Folder field must be mapped

This integration can be run independently, as the Test Folders and Tests do not require any other artifacts to exist before they can be created.   Artifact Creation Flow can be one-way or two-way.

Integration 2: Test Planning

The Test Planning integration is a Container + Work Item Synchronization that utilizes child containers, flowing Test Set Folders/Execution List Folders (container), Test Sets/Execution Lists (child container), and Test Instances/Execution Entries (Work Item).  

Test Planning Integration

To configure this integration, you will use the normal 'Container + Work Item Synchronization' template.  Tasktop has behind-the-scenes magic that will allow you to include a child-container integration once it sees the appropriate collections created:

Container Collections:

  • ALM Test Set Folders 
    • Parent field must be mapped to preserve folder hierarchy
  • Tosca Execution List Folders
    • Parent field must be mapped to preserve folder hierarchy

Child Container Collections:

  • ALM Test Sets
    • Parent field must be mapped to preserve folder hierarchy
  • Tosca Execution Lists
    • Parent field must be mapped to preserve folder hierarchy

Work Item Collections:

  • ALM Test Instances 
    • Test field must be mapped
    • Test Set field must be mapped
  • Tosca Execution Entries
    • Test Case field must be mapped
    • Execution List field must be mapped

Step 1: Test Set Folder/Execution List Folders

Once your collections have been created and configured, create a Container + Work Item Synchronization.  First, configure the top-level container integration:

  • ALM <=> ALM: Test Set Folder to Test Set Folder, or
  • Tosca <=> ALM: Execution List Folder to Test Set Folder

Container Creation Flow will most likely be one way into ALM, but this will depend on the use case.

Once this integration has been configured, you'll see an option to create a child container integration

(lightbulb) Note: The Test Instance (or Execution Entry) collection (i.e. the work item collection for this integration) must exist before the "New Child Container Integration" button will appear while configuring this integration.

Click 'New Child Container Integration

Step 2: Test Sets/Execution Lists

Your Child Container Integration will be

  • ALM <=> ALM: Test Set to Test Set, or
  • Tosca <=> ALM: Execution List to Test Set

This integration will likely not require container mirroring configuration, as it will inherit that from the parent container integration.

Step 3: Test Instances/Execution Entry

Finally, you will configure your Work Item integration.  It will either be:

  • ALM <=> ALM: Test Instance to Test Instance
  • Tosca <=> ALM: Execution Entry to Test Instance

Click the 'New Work Item Integration' button to add the integration.

Add New Work Item Integration

Here is what your fully configured integration will look like:

Fully Configured Test Planning Integration

Step 4: Run Integration

(lightbulb) Before you run this integration, you must have the Test Design integration configured and running. This is because in order to create a Test Instance, both a Test Set and a Test (created via the Test Design integration) must exist.  When a Test is added to a Test Set, a Test Instance is created.

To run the integration, click the green 'Run All' button.

Run Integration

Integration 3: Test Execution

The Test Execution integration is a Work Item Synchronization that flows Test results located on ALM Test Runs or Tosca Execution Test Case Logs

Test Execution Integration

Supported Artifacts:

  • ALM Test Run
    • Test Instance field must be mapped
  • Tosca Execution Test Case Log
    • Execution Entry field must be mapped

Artifact Routing and Filtering

Because Test Runs and Execution Test Case Logs live at the project level, Artifact Routing will only need to be configured at the Project level.

Since routing is at the project level, you may be asking, "How will I know that only the Test Runs that I care about are synchronizing?  I don't want every single Test Run in this project to flow!"  And you're in luck.  Test Execution Integrations behave a little differently from typical integrations: Tasktop will use built-in magic to only flow the Test Runs or Execution Test Case Logs that are associated with a Test Instance/Execution Entry that is also configured to flow. 

It's that simple!

For this reason, you will see an Issue on the Activity Screen of Tasktop if you attempt to run this integration without running an associated Test Planning integration (Remember: Test Runs require that an associated Test Instance exist first).

Race Conditions

Due to the interdependencies between the three integrations, the order that artifacts synchronize in matters.

In this Test Management integration scenario, Tasktop won't create an artifact if its parent container or other required artifact does not yet exist.

Examples of when Tasktop won't flow a work item:

  • Trying to create a Test Run without the correct Test Instance already existing in the target system
  • Trying to create a Test Instance without the correct Test already existing in the target system

Here's an example of what you can expect to see in a race condition:

  • Create a test instance and immediately run the test
  • If Tasktop picks up the Test Run first, it will not have the necessary Test Instance on the target to attach to and will error
  • Once Tasktop picks up the Test Instance & synchronizes it, then the Test Run will be able to flow across on retry and the error will clear

You are most likely to see this condition when first setting up your integrations. For this reason, we recommend setting up the integrations 'from top to bottom'. In other words, start with the Test Design integration. Then move on to the Test Planning integration. And finally, set up the Test Execution integration. If you have the integrations running in that order, you'll be more likely to flow any required artifacts before any dependant artifacts attempt to flow. 

(lightbulb) To keep your integrations in order in the Integration List View, we recommend appending a number to the beginning of each title, i.e. "1 - Test Design," "2 - Test Planning," "3 - Test Execution"

Another possible time when this race condition could occur is if you have vastly different change detection intervals on your integrations. For example, if you have a short interval on your Test Execution integration, but a much longer interval on your Test Planning integration. 

Flowing Test Step Results in Tricentis Tosca 

Because test steps are represented as a field on the Execution Test Case Log in Tosca, rather than as a unique child artifact (as they are in ALM), if you would like to flow test steps from Tosca ↔ ALM or from Tosca ↔ Tosca, it must be configured as part of a Test Execution integration.  

To configure test step flow for Tosca, follow the instructions below:

  1. Create a rich text field on the model used in the Test Execution integration.  For explanatory purposes, we'll call this the "Test Steps" field in the model.  Since test steps are represented as a field, rather than a separate artifact, your model type will be Standard Model.
  2. Within ALM, create a custom text field on the ALM Test Run artifact.  This field will accept the combined results of all the associated test steps from Tosca.  For explanatory purposes, let's call this the LogInfo field.
  3. In the ALM Test Run collection, map the Loginfo field to the Test Steps field in the model.
    Map LogInfo field to Test Steps model field
  4. In the Tosca Execution Test Case Log collection, map the AggregatedDescription field to the Test Steps field in the model.
    Map AggregatedDescription field to Test Steps model field
  5. Once the integration is run, you'll see that the results of each individual step in Tosca are now populated in the new custom LogInfo field on the ALM Test Run.
    Test Step Results flow to ALM Custom Field