Mercurial Installation & User Guide

The Mercurial to CA Agile Central connector comprises code that skims the Mercurial system for recent commits. It extracts information from the commit messages, and posts changeset/change information to a CA Agile Central SCMRepository, according to specifications in a configuration file. While it is possible to have the connector installed on only one computer for your company, you can also choose to install the connector on multiple computers depending on how your organization is structured and the number of repositories that need to be reflected in CA Agile Central.

You can put formatted IDs (such as US42) in commit messages. The connector processes all commits for configured repositories and will push information about those commits into changeset objects and change objects in CA Agile Central. If a valid Formatted ID is found in a commit message, the changeset created by the connector will be associated with that defect, story, or task in CA Agile Central.

A typical workflow is for one of your team members to be assigned DE42 in CA Agile Central. When they make a code change and commit, they should put a message in the commit like "Found an issue with line breaks - fixed DE42". The connector will then find that commit message and link the information about the commit to the defect in CA Agile Central. Please note this message is case-sensitive, "fixed" is not the same as "Fixed".

This user guide includes the following:

Supported platforms

The connector supports running on a variety of platforms. This software is written in Ruby and has been tested using Ruby 2.0.0-p247, but should also work with stable 2.x versions. If this software is for use on a Linux, Unix, or MacOSX system, it must be installed subsequent to installing the http client and rally_api gems.

You can run the connector from either of the following:

  • In Windows as a packaged executable (recommended if you want the connector to run on a Windows machine)
  • In any OS through Ruby as a packaged gem (on any OS with Ruby 2.0.0 or greater installed)

Prerequisites

  • Choose a machine that will run the connector
  • If the connector machine is a different machine than the Mercurial server, note the server address/name
  • Gather the paths to the repositories from which you will gather information
  • Choose the CA Agile Central workspace in which the information will be going

Mercurial server versus ssh

The primary use case is for this connector to be run on the platform that the Mercurial software and repository reside on. While it is possible to run this connector on a platform that is not the Mercurial platform, that use case assumes the use of ssh and the proper setup of public and private key information. Setting up ssh and key files is beyond the scope of this document; consult the internet for documents regarding ssh and PKI.

If you are running the connector on Windows, you will need to have an executable for ssh if you plan to connect to a remote box. Packages such as Cygwin and the Git installer for Windows include an ssh.exe. You will then need to add the location of that ssh.exe to your PATH environment variable to be able successfully connect to a remote repository or server.

If you are using the executable for Windows, there are no further prerequisites. If you intend to run the Ruby source, you will need to install Ruby version 2.0.0 or greater on the machine that will run the connector. We recommend getting the Ruby installer for Windows from rubyinstaller.org or using rvm from rvm.io for Linux and Unix users.

Download the connector

To download the connector, follow the steps here.

Install the connector

Installer versus operator filesystem permissions

As this connector is intended to run on an ongoing basis initiated by either cron or Windows Task Scheduler (WTS), it is important to consider the identity of the user who installs the connector versus the identity of the user when run through cron or WTS with regard to the permissioning and ACL of the directories and files in the connector installation. The runner of the connector must be able to read all files and must be able to write in the configs and logs subdirectories.

Distribution contents

$ unzip -l hg2rally-gemconn-1.3.2-cib1043.zip
Archive:   hg2rally-gemconn-1.3.2-cib1043.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
        0  08-01-15 10:40   hg2rally-1.3.2/
        0  08-01-15 10:40   hg2rally-1.3.2/configs/
     3026  08-01-15 10:40   hg2rally-1.3.2/configs/sample.yml
      132  08-01-15 10:40   hg2rally-1.3.2/configs/user_map.txt
        0  08-01-15 10:40   hg2rally-1.3.2/extension/
      315  08-01-15 10:40   hg2rally-1.3.2/extension/metpost.rb
    13859  08-01-15 10:40   hg2rally-1.3.2/extension/statex.rb
     1868  08-01-15 10:40   hg2rally-1.3.2/hg2rally.rb
        0  08-01-15 10:40   hg2rally-1.3.2/lib/
    14417  08-01-15 10:40   hg2rally-1.3.2/lib/hg_connection.rb
     1770  08-01-15 10:40   hg2rally-1.3.2/LICENSE
        0  08-01-15 10:40   hg2rally-1.3.2/logs/
        0  08-01-15 10:40   hg2rally-1.3.2/plugin/
     1660  08-01-15 10:40   hg2rally-1.3.2/plugin/blank.rb
     2283  08-01-15 10:40   hg2rally-1.3.2/plugin/chainlkup.rb
     3810  08-01-15 10:40   hg2rally-1.3.2/plugin/domainaug.rb
     6668  08-01-15 10:40   hg2rally-1.3.2/plugin/emailaddr.rb
    10330  08-01-15 10:40   hg2rally-1.3.2/plugin/filemap.rb
     2010  08-01-15 10:40   hg2rally-1.3.2/plugin/passthru.rb
     6826  08-01-15 10:40   hg2rally-1.3.2/plugin/ralkup.rb
     5277  08-01-15 10:40   hg2rally-1.3.2/plugin/viscount.rb
    18981  08-01-15 10:40   hg2rally-1.3.2/README
     4530  08-01-15 10:40   hg2rally-1.3.2/User-Mapping-Strategies
    40960  08-01-15 10:40   hg2rally-1.3.2/vcseif-1.3.2.gem
 --------                   -------
   138722                   24 files

Install on Windows with the pre-built executable

If you are using the Windows executable based connector, you need to run the installer exe to install the connector to your desired location on your computer.

Install using the Ruby gem distribution

Prerequisites

Ruby version 2.0.0 or greater (recommend using rvm or rbenv for this) http client version 2.6.0.1 (available at http://www.rubygems.org/gems/httpclient) rally_api version 1.2.1 or better (available at http://www.rubygems.org/gems/rally_api).

Install the http client and rally_api gems before installing the connector, using the gem install command. Additional information about Ruby gems and their installation can be obtained from http://www.ruby-lang.org/en/libraries and other websites related to the Ruby language and ecosystem.

Unpack and set up the source directory
  1. Unpack the source distribution into a directory you want to use as the operational home for this software.

    Tip: The distribution unpacks into a hg2rally-*-<version>-<build_ident> directory.

  2. Create a symlink with a shorter name to point to either under the same directory root or from somewhere else (/user/local or /opt/local).

    For example:

    ln -s /opt/sw/hg2rally-gemconn-1.3.2-cib1043 /opt/local/hgr

  3. Install the vcseif gem: # using the above example of having created a symlink to the installation root.
    $ cd /opt/local/hgr
    $ gem install --local vcseif-<version>.gem --no-ri --no-rdoc

Configure Mercurial

Consider how much Mercurial history you want to have reflected in CA Agile Central

It is possible to operate the connector to catch up with the complete history of commits in Mercurial. You should assemble a pro/con list of considerations, reasons, and benefits to help you arrive at a strategy that works for your organization with respect to whether you need a complete commit history reflected in CA Agile Central or just reflected from some particular point in time.

Configure CA Agile Central

Verify that your target WorkspaceConfiguration has BuildandChangsetEnabled set

Your CA Agile Central workspace administrator needs to enable this for your target workspace. If it is not enabled, the connector will work, but you will not be able to see any of the changeset/change information that is associated with your CA Agile Central work items (story, defect, task).

Configure the connector

Copy the sample.yml file in configs to a new name, for example, repo_one.yaml. Edit the copied file and adjust some of the sample values for servers, credentials workspaces, repository names, and paths to values appropriate for your environment. The file is in YAML format. The YAML format uses indentation to represent structure and colon character (:) and dash character (-) to denote name-value separation and list elements. Be aware of preserving those syntactical elements in your edited file.

Within the Rally section of your YAML configuration file, there is an entry where you can name the SCMRepository in the CA Agile Central system to which changeset/change items will be associated. This SCMRepository is a named container which need not exist before running the connector; if it does not exist, the connector will create the SCMRepository in the CA Agile Central user's target workspace.

There is an optional Lookback configuration item that can appear on either or both of the CA Agile Central and Mercurial sections of the configuration file. The value for this parameter is expressed in terms of minutes (default or with the use of the 'm' character) or hours using a 'h' character or days by using a 'd' character. Examples:

Lookback: 90
Lookback: 120 m
Lookback: 8 h
Lookback: 10 d

If you do not explicitly provide a Lookback value, the connector uses a default value of one hour. CA Agile Central recommends that if you specify a value for Lookback in either section that you also specify a Lookback in the counterpart section with the same value. If the Lookback values are not identical, there is the possibility under some circumstances that a changeset from Mercurial would not be reflected in CA Agile Central due to the nature of distributed version control systems recording the original commit time and not the time the commits are pushed to a master repository and the window of time consideration being too short for the connector to pick up such commits.

Assess how user names are alike or differ in your Mercurial system and in your CA Agile Central subscription. If the user names are universally identical in both systems, then you can either comment out all Author sub-items underneath the Transforms section or you can set the value for the Author field in the Transforms section to 'Passthru'. If on the other hand there is a deterministic mapping that can transform a Mercurial user value to the corresponding correct CA Agile Central username value, you'll need to adjust the transformation value for Author to the appropriate classname. Consult the User-Mapping-Strategies text document to determine which technique fits your particular situation.

Once you have identified a suitable Author transformation technique, you will need to edit the configuration to identify the Ruby class name that will implement that transformation. The Ruby class name must exist in a file of Ruby source code that is in a file in the plugin subdirectory.

Mapping SCMRepositories to VCS repositories

The VCS Connector was designed to operate where this is a one-to-one relationship between a CA Agile Central SCMRepository and a VCS repository (Git, GitHub, Subversion, Mercurial, and so on). When run, the VCS Connector creates a CA Agile Central SCMRepository item for the value specific in the connector config file if it does not already exist. This removes a small amount of administrative burden in that you do not have to create the CA Agile Central SCMRepository item before you run a configuration mentioning a new repository. A one-to-many mapping can create performance issues.

For each repository, there is a config file and a timefile. The timefile records the timestamp of the second-to-last commit for the repository. For example, you have an Apricot repository that had some flurry of commits last fall but nothing since and you also have a Banana repository that has had activity with the last couple of months. When commits for these two repositories get poured into a single CA Agile Central SCMRepository, it has the following effect. When processing the config for the Apricot repository, it is searching for recent Changeset records in CA Agile Central where recent is defined as the value in the timefile for Apricot. Since the last commit to that repository was last fall, there is going to be an excessive amount of information read-out of the single SCMRepository (because it is looking at all Changesets since last fall, not just the ones for Apricot).

Example repository configuration file:

---

VCSConnector:

Rally:
    Server              : us1.rallydev.com
    Protocol            : https
    Username            : jojo@muchly.com
    Password            : 22333four
#   APIKey              : _hgiotewhgeiwhgh325930503453490    # can be used in place of Username / Password entries
    Workspace           : VanillaBean
    RepositoryName      : Balloon
#   Proxy               : some_proxy.yoursite.com:9876  # or an IP address:port
#   ProxyUser           : outbound
#   ProxyPassword       : bvc-357%GNQ
    Lookback            : 90  # in minutes by default, use m/h/d suffix for minutes/hours/days
    UpdateArtifactState : False
    StateExtractorClass : BasicActionsAndArtifactsExtractor(message)
    Debug               : False

Mercurial:
#   Server              : HGsvr.company.com            # specified if repo is not local
    RepositoryBase      : /var/www/foo/bar
    Lookback            : 90
    MaxItems            : 100
#   RevURI              : http://yourserver.com:8008/hgifc/bigproject/rev/{revnumber}   # if running web access to Hg repo
#   FileURI             : http://yourserver.com:8008/hgifc/bigproject/file/{revnumber}/{filepath}  # if running web access
    Debug               : False

Services:
    Preview             : False
    LogLevel            : Debug

Transform:
    Author              : Passthru

...

Under the Mercurial section, the RevURI and FileURI are meant to be the root URI if you are using a web front end for your system. The URI will be used as a base for the link to the changesets and files. For example, if you have http://server:port/Mercurial/rev/ as your root, a link will be made in CA Agile Central for the changeset to http://server:port/Mercurial/rev/12345 for changset 12345. The strings {revnumber} and {filepath} will be substituted in when the connector builds the link stored in CA Agile Central.

Note that the Proxy* items are commented out by using a # in front of the item. Within a YAML file, any line beginning with a # character is ignore and any content following a # character sequence is ignored (including the # sequence).

Create a CA Agile Central API key

A CA Agile Central API Key can be created to use to access your subscription data without using your username and password.

To use the API Key in a connector, edit the Rally section in the config .yml file and add an APIKey line. When the APIKey configuration entry is specified, omit the Username and Password from the Rally config section. If an APIKey entry is present, the username and password are not used and a warning will appear in the log file if they are included in the config file. This is true even if the APIKey entry value is invalid, blank, or nil. If your subscription administrator has configured your subscription to be SSO only, you no longer need to add the user associated with the given APIKey value to the whitelist of authorized users.

The connector now uses rally_api version 1.2.1. Please note that the APIKey specified must have full access, a read-only api key will not allow the connector to write to CA Agile Central.

For help creating a full-access API Key, please visit http://help.rallydev.com/rally-application-manager.

Run the connector

Note: You should defer running the connector until you have at least one commit in your repository.

Within the Services section of your config file ("repo_one.yaml" for example), is an entry for Preview that is set to False. You may want to set this value to True in the initial setup phase to verify that the connector can successfully connect to CA Agile Central with the credentials and information you have provided, as well as proving out the use of the hg command. Your PATH environment variable must contain a filesystem path where the hg command can be found. See Linux / Unix / MacOSX / Windows documentation on how to set environment variables for use within a *nix cron job (or Windows Task Scheduler entry). Once you've determined that a connector run in Preview mode operates as expected, you can set the Preview value in your config to a False value.

On an ongoing basis, you can use cron / Windows Task Scheduler (or any other job-task scheduling software) to run the connector periodically. Initially, CA Agile Central recommends the connector to be run every 15 minutes during normal business hours and less frequently during non-business hours.

You can have numerous configuration files in the configs subdirectory and specify them all forinvocation either by name or by globbing (wild-card syntax). For example:

For Windows:
    hg2rally.exe  apricot  banana  cherry  dogwood
For Ruby:
    ruby  hg2rally.rb  apricot  banana  cherry  dogwood
The files apricot.yml, banana.yml, cherry.yml and dogwood.yml must exist in the configs subdirectory. The connector only looks for config files in the configs subdirectory under the installation base directory.

Whenever the connector is run, an entry is made in the logs/hg2rally.log file to note the invocation. For each config named at invocation, there will be an entry in that file noting the return code from running that config. When the connector run has finished with all configs, an entry is written to note the completion of the connector run.

Additionally, there is a file written in the logs subdirectory for each config named that will have more detail about the activity that occurs during the run. You can adjust the verbosity written to these log files by modifying the LogLevel value in the config file. Normally, the LogLevel would be set to Info. Should you encounter situations where the connector does not complete normally, you can adjust the LogLevel to Debug and run the connector to begin the troubleshooting process. These log files can be sent to CA Agile Central Support to expedite the investigation of a case.

The connector writes a file in the base installation directory corresponding to the config name with the date of the last commit processed. The file is named config_time.file and has a time entry in the form YYYY-mm-dd HH:MM:SS Z (for Zulu time). When first run, there won't be a time file for the config and the connector defaults to looking for commits that occurred after 2010-01-01 00:00:00 Z. You can override that behavior by creating and editing a time file for the config you are about to process. By providing an entry in the format mentioned above, you can control that point from which commits are processed.

Extend the Connector

Within the extension sub-directory of the installation, there is an exemplar Ruby class in the file named statex.rb. The class definition contained in that file is named BasicActionsAndArtifactsExtractor. This example class demonstrates the basic technique of examining a commit message and extracting Rally artifact identifiers and state transition target values. Using this class when the config file item UpdateArtifactState value is set to True results in the transition of the State (or ScheduleState in the case of UserStory) of the identified artifact to the state value mentioned in the commit message.

For example, if there is a CA Agile Central Defect (identified as DE1987) mentioned in the commit message with a new valid state value either immediately preceding or following, then the connector will change the State of the identified artifact in CA Agile Central to that state.

Example:
hg commit my_file.java -m "Fixed DE1987 by changing preamble paragraph 3"

If CA Agile Central Defect DE1987 had a state of Open prior to the commit and run of the connector, then subsequent to the operation of the connector processing this particular changeset, the state of CA Agile Central Defect DE1987 would display as Fixed. Note this message is case-sensitive, "fixed" is not the same as "Fixed".

The extension subdirectory allows you to provide your own message processing to extract CA Agile Central artifact identifiers and state changes if the example provided does not fit your conventions. Your extension must be a class written in Ruby and must provide an instance method called service which takes the commit message as an argument and must return a Ruby Hash instance with entries keyed by a State name (Fixed, Completed, and so on) or nil with a Ruby Array as the associated value populated by CA Agile Central artifact identifiers (FormattedID).

Support

Contact Support with any questions, issues, or suggestions by clicking the Contact Support link at the bottom of any CA Agile Central page.

Revision history

  • 1.4.3-master --- 24-Oct-2016
    • Enhancements:
      • Updated the logic to allow for processing after 1 commit.
  • 1.4.1-master --- 17-Oct-2016
    • Fixes:
      • Addressed uncaught exception when processing an empty repository.
  • 1.4.0 - master-1046 (Nov-2015)
    • Enhancements:
      • Brand redesign.
  • 1.3.2 - master-1043 (Aug-2015)
    • Enhancements:
      • Added support for use of APIKey entry and revised documentation to encourage use of this
        in preference to use of still supported ApiKey.
  • 1.3.1 - master-1040 (Jul-2015)
    • Enhancements:
      • Added support for downcasing any CA Agile Central username value, inverted sequence of retrieval from
        CA Agile Central and target VCS system, and upped dependency on rally_api to 1.2.1.
  • 1.3.0 - master-1040 (April-2015)
    • Fixes:
      • Handle commit messages with multi-byte characters whose length exceeds the CA Agile Central Changeset Message field length limit of 4000 bytes.
    • Enhancements:
      • Updated dependency on rally_api to 1.2.0
  • 1.2.9 - master-1039 (March-2015)
    • Fixes:
      • Refined handling of commit messages in excess of 4000 characters to account for JSON encoding expansion.
  • 1.2.8 - master-1038 (March-2015)
    • Enhancements:
      • Improved the ability to identify artifact identifiers in commit messages surrounded by bracketing characters.
  • 1.2.6 - master-1037 (January-2015)
    • Enhancements:
      • Updated rally_vcs_connection.rb configureExtensionEnvironment and statex.rb to accommodate custom artifact prefixes (such as 'BUG' for Defect).
      • Updated dependency for rally_api to version 1.1.2.
  • 1.2.5 - master-1035 (October-2014)
    • Enhancements:
      • Updated to use CA Agile Central Web Services API v2.0 and the updated rally_api gem.
  • 1.2.4 - master-1033 (16-Sep-2014)
    • Enhancements/Fixes:
      • Added support for using API Key in lieu of username and password in the CA Agile Central portion of the configuration file.
      • Updated to rally_api toolkit version 1.0.1.
  • 1.2.3 - master-1030 (19-Aug-2014)
    • Enhancements/Fixes:
      • Fixed DE21993 - to meet CA Agile Central's commit message field limitation of 4000 characters, if a commit message exceeds the character limit, characters over the limit are truncated. Upon truncation, the commit message is appended with an annotation of the truncation and a warning message displays in the log file.
  • Feedback

    Need more help? The CA Agile Central Community is your one-stop shop for self-service and support. To submit feedback or cases to CA Agile Central Support, find answers, and collaborate with others, please join us in the CA Agile Central Community.