ClearQuest Installation & User Guide

Print this topicEmail this topic

The Rally connector for ClearQuest allows customers to use Rally for Agile lifecycle management while still using ClearQuest for management of issues (or other types of objects). The connector can synchronize most ClearQuest record types with Rally stories, defects, or test cases. Most fields between the two systems can be synchronized.

The connector runs as a Ruby script on a Windows machine inside your network behind your firewall using ClearQuest's COM API and accesses Rally through the Rally Web Services API. The connector is configured through an XML file. A log file is created to track all changes made in Rally and ClearQuest by the connector. The connector requires a custom field in each system to store the unique ID of the linked objects in the other system. The connector copies fields from Rally or ClearQuest based on a field mapping you specify in your configuration file. You can map standard and custom fields between the two systems.

The connector provides five services to synchronize objects between Rally and ClearQuest:

  • Copy entities created in ClearQuest to Rally work items
  • Copy work items created in Rally to ClearQuest entities
  • Update Rally work items based on changes made to ClearQuest entities
  • Update ClearQuest entities based on changes made to Rally work items
  • Update Rally fields and ClearQuest fields

The configuration file specifies which services to run, and in what order.

p>An alternative update service, UPDATE_RALLYFIELDS_AND_OTHER, searches Rally first for updates and pushes only the recently changed fields to the other system. It then searches the other system for updates and pushes all mapped fields in to Rally.

This makes the chances of overwrites of data highly unlikely. The only scenario where an overwrite might occur is if Peter made changes in Rally and Al made changes in the other system for the same fields, the Rally information is copied over to the other system (possibly overwriting the data of the other system), then all data is copied over to the Rally system.

It is important to recognize that the data models of Rally and ClearQuest are not identical. The data models of the two connected systems are only partially compatible. We recommend that you take some time to identify the key data items that you want to track in both systems and what you want the policy to be as far as having a primary system of record. You'll see specific information later in this document discussing some of the things to be considered and some potential approaches.

This installation guide includes:

The Rally connector for ClearQuest is available for Unlimited Edition customers and Integration Hub partners.

Basic installation steps

  1. Install the connector (run the installer or extract the zip).
  2. Make configuration changes in Rally and in Clear Quest.
  3. Edit the cg_config.xml file and make changes to reflect your environment.
  4. Run rally2_cq_connector.exe to start the synchronization process.

System setup

The following people should be involved in getting fields set up in both Rally and ClearQuest:

  • A ClearQuest administrator who can create a custom field on the record type you want to synchronize with Rally
  • A Rally administrator who can create a custom field on the Rally artifact you want to synchronize with ClearQuest
  • Someone with access to the windows machine described above to install and configure the connector

Think about the process you want to setup between Rally and ClearQuest:

  • Where do objects start and end their lifecycle?
  • Which fields need updated?

You will also need to:

  • Identify a test instance of ClearQuest to use for testing the connector
  • Identify a test Rally Workspace/Project to use for testing the connector
  • Find a Windows machine to run the Rally Connector for ClearQuest
  • This machine must have the ClearQuest Windows client installed on it (in order to have the needed Windows DLLs installed for the connector to use)

Get the connector

To obtain the connector for ClearQuest, click Contact Support at the bottom of any page in Rally to open a support case and request the connector.

Software and hardware requirements

  • A Rally subscription
  • A Rally account with administrator privileges (for setup only, but not needed to run the connector)
  • The URL for the Rally server hosting your subscription, typically rally1.rallydev.com (excluding the HTTPS prefix as the connector adds https:// to what is specified)
  • The ClearQuest user specified to run the connector needs to have SQL editor privileges and needs to subscribe to the ClearQuest database (if not already subscribed)
    • ClearQuest designer privileges are needed to initially setup the connector but not to run the connector
  • Windows XP or Windows 7
  • If using a proxy, the server and port number of the server
  • ClearQuest version 7.x
  • ClearQuest for Windows client installed on the Windows machine where you plan to run the connector

Install the connector

(Optional) If your site uses a proxy server to access Rally, you will need to set the http_proxy environment variable. To do this on Windows 7:

  1. From Start, right-click on Computer, then select Properties.
  2. Click Advanced system settings.
  3. On System Properties, click Environment Variables.
  4. On Environment Variables, click New under System Variables.
  5. On New System Variables, enter http_proxy in the Variable name field.
  6. Enter your proxy's full HTTP URL in the Variable value field (such as http://www-cache:8000).
  7. Click OK (three times) to save and exit this process.

You may need to restart your system in order for the change to take effect.

Installation for Windows

Double-click the RallyConnectorforClearQuestInstall-<version>.exe to open the install wizard. Follow the prompts to complete the installation.


installcq

The Windows installer will install the following files (by default in C:\Program Files\RallyConnectorfor<connector_name>):

  • rally2_cq_connector.exe—Executable to run the connector
  • cq_config.xml—Sample configuration file
  • WinTail.exe*—Utility to tail the log file output for debugging purposes
  • startcq.bat—Batch file that automatically runs the connector and shows the log file; useful for testing

*WinTail is a free text file tail utility for Windows, generously made available by Bare Metal Software Pty Ltd. for redistribution without restriction. We have found it to be far superior to Notepad and other text editors for watching log files in real time as they are updated by the connector.

ClearQuest setup

Create an External ID field in ClearQuest

Contact your ClearQuest administrator to perform these updates.

  1. Log in to ClearQuest Designer and check out the appropriate schema.
  2. Go to Record Types → Defect.
  3. Right-click on an empty row and select Add Field.
  4. In the dialog, enter a Field Name of ExternalID, DB Column Name of ExternalID, and select Type SHORT_STRING.
  5. Commit the changes to the database schema from File → Check In.
  6. If the check in does not produce any errors, commit the changes to the defect repository from Database → Upgrade Database.

Make note of the display name of this field. This adds a string field to the ClearQuest form that contains the Rally ID. If you wish to view this field from your defect form, modify the database schema to include this field.

Rally setup

Create an External ID field in Rally

  1. Log in to Rally as a workspace or subscription administrator.
  2. Go to Setup → Workspaces & Projects.
  3. Select the workspace you wish to map to ClearQuest.
  4. On the workspace's detail page, click Work Products & Fields in the sidebar and ensure the Work Product Type selected is Defect.
  5. Click Actions → New Field.
  6. Enter a Name of ExternalID, Display Name of ExternalID (name and display name must match), and type of String.

Make note of the name of this field. Once you start using the connector, this will contain the external ID of the ClearQuest defect.

Connector configuration

Edit the configuration file

The configuration file is part of the delivered zip file and should be in the same directory where you extracted the Ruby gem. For ClearQuest, the default configuration file has a name of cq_config.xml. We recommend making a backup copy of the cq_config.xml in case you need to reference a valid configuration file later.

Edit cq_config.xml by entering the appropriate values between each begin and end tag.

<Config>
        <RallyConnection>
                <Url>rally1.rallydev.com</Url>
                <WorkspaceName>MyWorkspace</WorkspaceName>
                <Projects>
                        <Project>MyProject1</Project>
                        <Project>MyProject2</Project>
                </Projects>
                <User>user@company.com</User>
                <Password>MyPassword</Password>
                <ArtifactType>Defect</ArtifactType>
                <ExternalIDField>ExternalID</ExternalIDField>
                <CopySelectors>
                        <CopySelector>Foo > 1</CopySelector>
                        <CopySelector>Bar > 2</CopySelector>
                </CopySelectors>
                <UpdateSelectors>
                        <UpdateSelector>Field1 = abc</UpdateSelector>
                        <UpdateSelector>Field2 = xyz</UpdateSelector>
                </UpdateSelectors>
        </RallyConnection>

        <CQConnection>
                <Database>SAMPL</Database>
                <User>cqusername</User>
                <Password>cqpassword</Password>
                <ArtifactType>Defect</ArtifactType>
                <ExternalIDField>rally_object_id</ExternalIDField>
                <ExternalEndUserIDField>rally_formatted_id</ExternalEndUserIDField>
        </CQConnection>

        <Connector>
                <FieldMapping>
                        <Field><Rally>Name</Rally><Other>Headline</Other></Field>
                </FieldMapping>
        </Connector>

        <ConnectorRunner>
                <Preview>False</Preview>
                <Services>UPDATE_RALLY_TO_OTHER,  COPY_OTHER_TO_RALLY</Services>
        </ConnectorRunner>
</Config>

Each major section is surrounded by a tag to delineate that section. In the example above, here is a description of each section:

  • RallyConnection

    Defines the connection information for Rally, including Rally URL, user, password, and so on.

  • CQConnection

    Defines the connection information for ClearQuest, including the database name, artifact type, and so on.

  • Connector

    Defines the field mapping between the two systems. Generally, strings should be mapped to strings, integers to integers, and so on. The order of the fields in the mapping section determines the order in which those fields get updated in ClearQuest.

  • ConnectorRunner

    Specifies a run interval for how often the connector will query each system to trigger creates/updates and what services should run.

Field mapping

The field mapping section is located between the <Connector> tags in the configuration file and defines which fields map between the two systems.

For example, this definition sets up a mapping between the Rally field Name with the CQ field Headline. On a create or update, the connector will only update the Name field in Rally and the Headline field in CQ.

<Connector>
    <FieldMapping>
        <Field><Rally>Name</Rally><Other>Headline</Other></Field>
    </FieldMapping>
    ....
</Connector>

When you set up your mapping between the two systems, ensure the fields are compatible between the two systems (an integer field should map to an integer field in the other system, a rich text should map to a rich text in the other system, and so on).

Otherwise, you might experience situations where information is not created or updated between the two systems and you will see an error in the log file. For example, the connector will post an error for a particular work item if you try to post a string to a custom field of type integer in Rally.

You can add subsequent mappings by appending to this list. For example, this sets up a mapping to Rally Name → Headline, Rally Description → Description, Rally Priority → Priority:

<Connector>
    <FieldMapping>
        <Field><Rally>Name</Rally>       <Other>Headline</Other></Field>
        <Field><Rally>Description</Rally><Other>Description</Other></Field>
        <Field><Rally>Priority</Rally>   <Other>Priority</Other></Field>
    </FieldMapping>
    ....
</Connector>

If you are mapping a drop-down value in Rally to ClearQuest, this assumes that the drop-down values match. Otherwise, the connector will generate an error that the value was not found in the list.

If your drop-down values are different between the two systems, see Map Project Fields.

Field handlers

Map drop-down values

Map users

Map project fields

XML tags

Below is a more detailed explanation of each tag section in the configuration file.

XML tags in the <RallyConnection> section:

Tag Name Description Sample Values
     
<RallyConnection>
Required
Opening parent tag for this section.  
<Url>
Required
Server used to connect to Rally (excluding the HTTPS prefix, as the connector adds https:// to what is specified). sandbox.rallydev.com
rally1.rallydev.com
myRally.mydomain.com
192.168.23.24
<User>
Required
Login name for user to make the Web Services requests to create and update work items in Rally. user@company.com
<Password>
Required
Password associated with the above <User>. Note: The connector will encode (not encrypt) this password string and update it in the configuration file to avoid clear-text passwords being stored. mypassword
<WorkspaceName>
Required
Workspace in Rally you want to copy and update work items. My Workspace
MyWorkspace
<Projects>
Required
Contains a list of project tags. Each tag refers to one Rally project that will be used when finding new Rally work items to copy to the other system. For updating work items from Rally to the other system, all projects in <WorkspaceName> are considered. At least one Rally project must be specified in this tag. <Project>Rally1</Project>
<Project>Rally2</Project>
<ArtifactType>
Required
Type of artifact you want to create and update in Rally. (Defect, defect),
(Story, UserStory, HierarchicalRequirement), TestCase
<ExternalIDField>
Required
Rally custom string field (name and display name must be identical) that stores the unique ID for the other system. Refer to the Create an External ID Field in Rally section above. CQID
<SuppressDeprecationWarning> Changes the WARN message about Rally WSAPI version deprecation in the logfile to an INFO message.
<CopySelectors> Criteria to use when finding new Rally issues to copy to ClearQuest. Multiple criteria are ANDed together. An individual selector specification has the form:
<field><relation><value>
where:
   <field> is the name of a ClearQuest defect field
   <relation> is one of =, !=, gt, lt, gte, lte
   <value> is a valid value for the <field>
<CopySelector>Status = Open</CopySelector>
<CopySelector>Priority = Low</CopySelector>
<UpdateSelectors> Criteria to use when finding existing Rally issues to be updated in ClearQuest. Multiple criteria are ANDed together. An individual selector specification has the form:
<field> <relation> <value>
where:
   <field> is the name of a ClearQuest defect field
   <relation> is either = or !=
   <value> is a valid value for the <field>
<UpdateSelector>Release != alpha</UpdateSelector>
</RallyConnection>
Required
Closing parent tag for this section.  

Tip: Avoid special characters contained in a Rally workspace or project name that are markup-sensitive. For example:

& ampersand becomes &amp;
> greater than becomes &gt;
< less than becomes&lt;

Example: Research & Development → Research &amp; Development

XML tags in the <CQConnection> section:

Tag Name Description Sample Values
     
<CQConnection>
Required
Opening parent tag for this section.  
<User>
Required
The user that makes the API requests to create and update entities in ClearQuest. Needs SQL editor permissions. myuser
<Password>
Required
Password for user to make the API request to create and update entities in ClearQuest. Note: The first time the connector runs, it will encode the password so it is not saved in plain text. mypassword
<Database>
Required
Database schema name for ClearQuest. SAMPL
<DatabaseSet> Name of the ClearQuest connection that you have setup with the ClearQuest client for Windows. This name may differ based on the how the user named the connection. This tag is not needed if you have only one connection set up. 7.0.0 MyConnection
<ArtifactType>
Required
Database table name for entity you want to create and update in ClearQuest. Defect
<Entity> Use the Entity tag if the ClearQuest system is set up where the record type name and database name are different. Defect
<ExternalIDField>
Required
ClearQuest custom string database column name that stores the unique ID for the Rally work item. See Create an External ID Field in ClearQuest. rallyid rally_object_id
<ExternalEndUserIDField> ClearQuest custom string database column name that stores the formatted ID (DExx or USxx) for the Rally work item. See Create an External ID Field in ClearQuest. rallyid, rally_formatted_id
<TempTimezoneOffset> Offset from UTC (in hours) of the time zone that the server is running in. For example, for a server running in the Mountain time zone during daylight savings time (7 hours earlier than UTC), the entry would be <TempTimezoneOffset>-7</TempTimezoneOffset>. This optional tag is only needed if entries in the action_timestamp column in the ClearQuest History database are not in UTC. We eventually intend to remove this tag if it becomes possible to query the ClearQuest server directly for its time zone. -5
</CQConnection>
Required
Closing parent tag for this section.  

 

XML tags in the <Connector> section:

Tag Name Description Sample Values
     
<Connector>
Required
Opening parent tag for this section.  
<FieldMapping>
Required
Determine what fields map between the two systems. See Field Mapping.
<OtherFieldHandlers> <OtherEnumFieldHandler> Allows a user to map non-alike, drop-down values between the two systems. See Mapping Drop-Down Values section.
</Connector>
Required
Closing parent tag for this section.  

 

XML tags in the <ConnectorRunner> section:

Tag Name Description Sample Values
     
<ConnectorRunner>
Required
Opening parent tag for this section.  
<Preview> Allows the user to enable a preview mode for testing where NO objects are copied or updated in either system. false true
<Services>
Required

Use the copy services to initially reflect items between systems, then use the update services to keep those reflected items up to date.

The alternative update service searches Rally first for updates and pushes only the recently changed fields to the other system. It then searches the other system for updates and pushes all mapped fields into Rally. This allows changes done in both systems to not cause overwrites of old data.

Copy services:

COPY_JIRA_TO_RALLY
COPY_RALLY_TO_JIRA

Update services:

UPDATE_JIRA_TO_RALLY
UPDATE_RALLY_TO_JIRA

Alternative update service:

UPDATE_RALLYFIELDS_AND_OTHER (may not be used in conjunction with the other update services)

See FAQ for more information.

<LogLevel> Determines what type of messages are written to the log file. The highest level is Debug where all messages are displayed. The default log level is Info. Fatal, Error, Warn, Info, Debug
</ConnectorRunner>
Required
Closing parent tag for this section.  

Tip: Incrementally set up the connector! Start with a basic configuration file, test that you can connect to ClearQuest and Rally in a test environment. Once you validate this is set up correctly, then start customizing the field mapping and field handler sections.

Run the connector

Once the cq_config.xml file is setup, you can start running the connector. On Windows, you can simply open a DOS shell window and enter the command:

rally2_cq_connector.exe  cq_config.xml

Double-click on WinTail.exe to run WinTail. In WinTail, open the file rallylog.log to watch the log messages generated by the connector. This is an easy way to confirm that the connector is working properly, or to discover any errors.

Tip: We ship a startcq.bat file to make it easier to test. Double-click on the batch file to automatically run the connector and open the log file in WinTail.

To stop the service, use Control-C in the command shell.

Multiple configuration files

For users who want to map to more than one workspace in Rally, need to map multiple artifact types, or need to map to multiple containers (such as domains and projects in Quality Center or ClearQuest databases) in the other system, setting up multiple configuration files may make sense. Rally recommends naming the configuration files using descriptive names for easier troubleshooting.

To run multiple instances of the connector for different configuration files, pass in the list of config files as arguments to rally2_cq__connector.exe as follows:

rally2_qc_connector.exe  config_workspaceA.xml  config_workspaceB.xml

The connector will process the configuration files based on the command line argument order and processes one file at at time.

Once it has processed every configuration file, the connector will sleep based on the sleep value. The default is 15 minutes. To change the sleep value between runs, set this value (in minutes) as the last command line argument. We recommend a sleep value of 10 minutes or more and advise against anything less.

rally2_cq_connector.exe  config_workspaceA.xml  config_workspaceB.xml  10

 

Troubleshoot the connector

Once the connector is running, all errors are written to a log file in your working directory. Informational messages, warnings, and errors are written to the log file depending on the value of the <LogLevel> tag in the <ConnectorRunner> section of the configuration file.

To see the most recent log messages on Windows, rename the extension of the WinTail file from .dat to .exe and then double-click on the WinTail.exe and browse to the rallylog.log file.

Before the connector starts synchronizing objects between the two systems, it performs validation as follows:

  • Connecting to Rally is successful
  • Connecting to the other system is successful
  • Fields in the field mapping exist in Rally and the other system
  • Ensures that each field handler has a corresponding field mapping section in the configuration file

To confirm the validation is successful without moving objects between the two systems, ensure you set the <Preview> tag in the configuration file to true. This is useful if you want to experiment with some different configuration options to debug an issue.

If you still cannot determine the root cause of an issue, email the entire log file and config file to Rally Support.

Map Rally projects

If you are copying defects from ClearQuest to Rally using the RallyReferenceFieldHandler with Projects, you may see a Could not find Rally Project with name error in the log file if the connector user does not have editor permissions to that project in Rally.

ClearQuest state workflow

The connector supports a field mapping between Rally and ClearQuest states; however, if you have set up rules where a ClearQuest state can only be set if certain required fields are set (such as ClearQuest state of Closed requires the Resolution field set), updates may not occur when updating the state from Rally to ClearQuest. ClearQuest will update all other fields on the defect except the state and fail silently.

To work around this issue, you can set up a default value for the ClearQuest required fields in your configuration file. Contact Rally Support for more information on setting up this configuration. Due to the high customization of ClearQuest workflows, another solution we recommend is to use a custom field in ClearQuest that maps to the Rally state. Someone at your organization would then need to create a reconciliation report in ClearQuest that shows any defects that have different states for Rally and the custom ClearQuest field (DE13787).

Map releases and iterations with multiple projects

If you are using a <RallyReferenceHandler> to map releases and iterations with the same name and multiple projects, you may receive an error if the release or iteration you are updating on the defect references a different project than the defect is assigned. The connector only returns the first release or iteration with the matching name in Rally, so it could have a reference to a release or iteration in a different project.

New lines in ClearQuest

Rich text fields like Description do not transfer the new lines to Rally. Rally uses HTML whereas ClearQuest uses its own markup language. We plan to support transferring new lines in the future.

Revision history:

  • 4.3.0 - 16-Jan-2014
    • Upgrade from Ruby 1.9.2-p290 to Ruby 2.0.0-p247
    • Performance improvement gained by using rally_api rather than rally_rest_api
    • Added more build tests
    • Connector would validate only the first config file on command line (DE17492)
    • Added <SuppressDeprecationWarning> XML tag
  • 2.8.8-107 - 18-Dec-2012
    • Enhancements:
      • Minor logging improvements
    • Fixes:
      • DE15642: Email notification of errors and warnings, in the <Emailer> feature, would not send email if the <Level> was set to Error
    • Known issues:
      • (none)
  • 2.8.6-105 - 16-Oct-2012
    • Enhancements/Fixes:
      • Added ID in log message so user can see what artifact caused an error
      • Added more debug output in Rally connection section
      • Improved performance by removing refresh on workspace read in find-workspace loop
      • Added Rally URL and user name to connection message
      • Added check for nil return value on tags and keywords
    • Known issues:
      • (none)
  • 2.7.9-097 - 26-Jun-2012
    • Enhancements/Fixes:
      • Added more field handlers
  • 2.7.6-091 - 20-May-2012
    • Enhancements/Fixes:
      • Added <RallyToQCTestStepLinker>, <ValidationStep> Last </ValidationStep>, and <StepFields>
      • Added <RelatedObjectLinkers>, <QCToRallyTestStepLinker />
      • Added <RallyDateTimeFieldHandler>
      • Modifications for migrating items from the Test Lab to Rally TestCases/TestSets/TestFolders
  • 2.7.4-089 - 13-May-2012
    • Enhancements/Fixes:
      • Added ability for FormattedID in OTHER system to have lower case prefix (us123) or upper case prefix (US123)
      • Added subfield specifications on CopySelectors from Rally using the precopy feature.
      • Fixed handling of bad characters in a Requirement Name field
      • Expanded CopySelector/UpdateSelector criteria expression to include: field = (Foo or Bar)
      • Added facility to specify TestFolder or RequirementFolder ID instead of name (bypasses duplicate folder name issues)
      • Added capability to reflect Story relationships, TestCase relationships, Defect relationships
      • Added XML include capability so common text can be included as a separate file
English

Feedback

Please send us your feedback regarding our help site. For feedback regarding the Rally product, click here to open a support case or click here to submit a new feature request
English
By submitting this form, you accept the Mollom privacy policy.
© 2014 Rally Software Development Corp | Legal