$Date: 2004/11/13 16:33:56 $
$Revision: 1.9 $
Editors: C. Arnault, RD Schaffer
Contributors: S. Albrand, D. Rousseau
We present here an introduction to developing software in Atlas. A Configuration Management Tool ( CMT) is used to build (compile and link) and run our software. The software is grouped into separate packages, where some packages may be related or "depend" upon each other. This dependency means that a package "requires" information from other packages. For example, this "requirement" may be the location of some "include" files or it could be the need to link against the library produced in another package. Thus the dependency between packages expresses a certain direction of flow of information as well as the order in which the build process must take place.
CMT groups together all of the information needed to build or run the "build-products" of a package in a single requirements file in each package, which can be found in a cmt directory. The information is expressed in term of variables called macro's, and set's where the former are "internal" variables only seen by the requirements files or the makefiles, whereas the latter can be exported to ones session as environment variables. The dependency between packages is expressed by use statements where a package says which other packages it depends upon. (See [6.] for an explanation of the syntax.) Note that the macro and set variables in one package are visible from other packages that use it. In this way, we have grouped many commonly used definitions into an AtlasPolicy package which all packages in Atlas are supposed to depend upon.
CMT can be used in a wide variety of ways. In Atlas we have adopted a number of conventions which hopefully simplify software development. Firstly, users set (or change) their session environment variables with the use of a login requirements file that each user has in a directory their home area. This is explained in section [3.]. Secondly, for user development of the software, one may have one or more "development" areas where one can check out packages from CVS, modify the software and build the software against a particular release of the Atlas software (see section [3.] for defining/changing your development area and section [5.] for examples). Finally, there is a template package TestRelease which one normally checks out along with other packages. The TestRelease package is typically where one can use the cmt broadcast command, for example, to broadcast a gmake command to all of ones checked-out packages to build them. As well, the TestRelease package has a run directory where, for example, one conventionally runs athena with one's packages under development. Section [5.] provides examples for the use of the TestRelease package. Note that for specific applications, packages similar to TestRelease have been developed to run them.
- CMT is an open-source product developed at LAL-Orsay. All information about CMT (complete documentation, how to download and install it on local sites, mailing list, etc...) can be found on its web site.
- A CMT package is a directory structure like
<package>/<version>/cmt/requirementswhere <version> is the name of the CVS tag given to this package. The "contents" of a package are below the <version> directory. All packages have at least a cmt directory containing a requirements file.- A requirements file contains the information to build its components (e.g. compile/link files for libraries or applications) and use them at run time (e.g. specifying where a package's shared library is located). A package's requirements file also specifies which other packages it depends upon (with use statements). A requirements file in one package has access to the (public) information in the requirements files of these other packages (e.g. to access "include files", to know which compiler options to use, or to know which libraries to link against).
- A CMT constituent or component is an element that can be "built" with gmake. For example, a library may be built by compiling C++ files and then putting their executable object code into a shared or archive library. Some components are built by default, i.e. with just gmake, others are built explicitly, i.e. gmake <constituent-name>.
- A CMT sub-package is a directory structure like
<parent>/<package>/<version>/cmt/requirementswhere the sub-package <package> lies below below another CMT package <parent>.- A Leaf package is a CMT package with no sub-packages (and no sub-containers) and will be simply named a Package
- A Container package is a CMT package with sub-packages or sub-container packages. By convention in Atlas, a container package is used as a grouping mechanism for its sub-packages or sub-container packages and does not build its own components.
Figure 1.
Here is a typical structure of a container package A with two subpackages B and C. Each package has a cmt directory and requirements file. The package B has the typical structure of a package with C++ classes: header (.h) files are stored in the directory B and the source (.cxx) files are stored in the src directory. There are also directories for documentation (doc), tests (test) and Athena jobOption files (share). Finally, there is a directory, i686-rh73-gcc32-opt created when one runs gmake. This directory is where compiled files, libraries and executables are created. The name of this directory corresponds to the value of the CMTCONFIG environment variable. This name varies according to the platform (linux or sun), compiler (gcc or CC) and with the type of build (debug or optimized), which keeps separate the binary files for the different types of builds.- There are three different kinds of libraries that one can build: archive, shared, and component. All three physically contain binary (executable) code from compiled files. Rather the distinction comes from how they are used.
- When an application links against an archive library (or static library) library's binary code is physically copied into the application. So when the application runs, it executes code that it "contains".
- When an application links against a shared library the library's binary code is NOT copied but only "referenced". So when the application runs, it must be able to locate the library by name to run the binary code contained in the library. Locating a library is done, for example on linux, via the path-variable LD_LIBRARY_PATH which contains a list of directories where shared libraries may be found. Generally an application may refer to several shared libraries. In this case, only the application needs to specify this set. Shared libraries themselves need not specifiy the other shared libraries they refer to (i.e. their external symbols are not resolved)
- A component library is a a shared library which is built like an application. That is a component library is linked against all other shared libraries that it needs to "have its external symbols resolved". An application, then, may decide at run-time to dynamically load and run a "component" which is in a specific component library. For this one just needs to locate the library by name and have a standard mechanism to find the entry point in the component library. For example, Athena Algorithms and Services are components in component libraries which are dynamically loaded by specifying them in the jobOptions.
In practice, applications built with shared/component libraries are used for development or interactive sessions. Whereas applications linked against archive libraries are more suitable for production work.
- All scripts mentioned in this document exist for the 2 shell families:
- .sh = for sh, bash, zsh
(for bash, convert all "source xxx.sh" to ". xxx.sh")- .csh = for csh, tcsh
- Executable instructions are prefixed by either sh> or csh> when they are specific to the corresponding shell family or simply > when they are not specific. They are written in bold-blue.
- CMT statements are written in bold-green
This section should be read first and is mandatory for a proper understanding of any other operationThis section describes how to configure the session environment for working with a given release of the Atlas software. The first step, bootstrap procedure, creates a home requirements file and setup scripts for new users. The second step is the most frequently used step which sets or changes one's session environment. The last step is what needs to be done when moving to a new version of CMT.
- New users must follow a bootstrap procedure [3.1] to create their own login requirements file and setup scripts.
- Most often users will simply source [3.2] their home setup scripts.
- Occasionally, the version of CMT changes and it is required to re-configure [3.3] one's home setup scripts.
3.1 - Bootstrap procedure
The Bootstrap procedure is used to create a home requirements file and setup scripts.
There are three steps to this procedure:
- Setup up the correct version of CMT:
sh> . <location of CMT>/setup.sh csh> source <location of CMT>/setup.cshwhere <location of CMT> differs from site to site. At CERN it is:/afs/cern.ch/sw/contrib/CMT/<version>/mgr/We are currently working with patch versions of CMT v1r16, where the current version is v1r16p20040901
- please contact your local site manager about the specific installation
- refer to the CMT manual in case you need to install CMT on your laptop or site
- Create, or replace, your login requirements file [3.1.1]:
- Generate the setup files for your login requirements file:
This last step (i.e. cmt config) only needs to be done once. Further modifications of the login requirements file will be taken into account when one sources the setup files. One will only need to re-configure when the version of CMT changes.> cd ${HOME}/cmthome > cmt config3.1.1 - Creating and customizing the login requirments file
Several parameters, including some environment variables need to be set to use a release, for example:
- the version of CMT
- the release number (eg. 5.2.0)
- some general purpose environment variables
CMTCONFIG the current build directory, is a configuration tag CMTSITE the current site, is a configuration tag CMTPATH the list of package areas PATH access to executables LD_LIBRARY_PATH access to shared libraries We recommend a mechanism based on a login requirements file and corresponding setup scripts to initialize these settings. A login requirements file is no different than a requirements file in a package. It is only used to set environment variables, and not to compile code or build libraries.
3.1.1.1 - Creating the requirements file
- Select a location for maintaining your login requirements file and the associated setup scripts. This can be anywhere, e.g. in your HOME directory (${HOME}) but we recommend to use by convention a dedicated location ${HOME}/cmthome. This convention is used throughout this document.
- Create an empty requirements file in the selected location and fill it with a simple template of a login requirements file.
It should be noted here that within this template, the section referencing the AtlasSettings package is mandatory so as to obtain the full set of Atlas conventions:
# the AtlasSettings package centralizes the setting of basic # environment variables use AtlasSettings v* $(ATLAS_DIST_AREA)In more detail, AtlasSettings sets up:
- CVSROOT which points to the CVS repository works with your afs password
- CMTCVSOFFSET to be "offline" which allows to omit this dir when checking out packages with "cmt co XXX"
- CVSIGNORE - the list of files to ignore when checking in files
- CMTPATH to include the desired release area
- SITEROOT - the base of all atlas software at a particular site
- CMTCONFIG - where build products are kept in each package
- At CERN, adds in the right compiler to be used to PATH and LD_LIBRARY_PATH
3.1.1.2 - Customizing the requirements file
In a requirements file, one can set parameters that are internal to CMT, macro's, or ones that are exported to the environment, set's. One can also prepend/append/remove items to path-like environment variables (e.g. PATH or LD_LIBRARY_PATH). (See [6.] for an explanation of the requirements file syntax.)
One may customize the following:
- The site description, by setting the CMTSITE value (several typical values are suggested).
set CMTSITE CERNThis is normally set to the site where one is working, e.g. CERN, and to EXTSITE for portables.- The base location of all releases (this will generally differ from site to site) by setting the ATLAS_DIST_AREA value. For CERN, set:
macro ATLAS_DIST_AREA "/afs/cern.ch/atlas/software/dist"- The release number, by setting the ATLAS_RELEASE value (see Release status)
macro ATLAS_RELEASE "5.2.0"- The location of your preferred development or work area, where packages will be checked out, by setting the devarea value
macro devarea "${HOME}/MyTest" macro home_dir "${HOME}" path_remove CMTPATH "${home_dir}" path_prepend CMTPATH "$(devarea)"Where the last three lines first remove from $CMTPATH any devarea previously defined in your ${HOME} directory, and then prepends the new devarea.If you have just created your requirements file for the first time, then go back to item 3. of the Bootstrap procedure [3.1], to be sure you have generated the setup files.
Otherwise, you will only need to source [3.2] your setup scripts to update the environment (e.g. CMTPATH and CMTCONFIG) for these changes to your home requirements file.
3.2 - Setting up one's session environment
We assume here that the login requirements has been correctly prepared [3.1] in a dedicated location, (by convention we assume this location is ${HOME}/cmthome). Each session should start with the following action:sh> source ${HOME}/cmthome/setup.sh csh> source ${HOME}/cmthome/setup.cshThis will set environment variables with the default values given in the login requirements file. This will also set the default compilation mode to be debug.
To instead set the compilation mode to optimized, one does:> source ${HOME}/cmthome/setup.[c]sh -tag=optThis is an example of how to "tune" one's environment using tag specifications. In general, one can source a setup file as:
> source ${HOME}/cmthome/setup.[c]sh -tag=tag1,tag2where multiple tags are in a comma-separated list, no spaces.
One can switch back and forth between debug and optimized mode by alternately using:> source ${HOME}/cmthome/setup.[c]sh -tag=opt > source ${HOME}/cmthome/setup.[c]sh -tag=dbgPlease note that this operation can be performed multiple times from anywhere, i.e. not only from the directory where the login requirements file is located. (This was not true for CMT versions v1r10p20011126 and earlier.)This mechanism can also be extented by creating user defined tags, for selecting various alternate configurations such as (see [6.] for the requirements file syntax):
- Releases
macro ATLAS_RELEASE "5.2.0" \ 5.0.0 "5.0.0" \ 5.1.0 "5.1.0" \ 5.2.0 "5.2.0"> source ${HOME}/cmthome/setup.[c]sh > source ${HOME}/cmthome/setup.[c]sh -tag=5.0.0 > source ${HOME}/cmthome/setup.[c]sh -tag=5.1.0alternatively select the release 5.2.0 (default), 5.0.0 or 5.1.0 as the base release.- Nightly builds
macro ATLAS_RELEASE "5.2.0" \ 5.0.0 "5.0.0" \ 5.1.0 "5.1.0" \ 5.2.0 "5.2.0" \ rel0 "nightlies/rel/atlrel_0" \ rel1 "nightlies/rel/atlrel_1" \ rel2 "nightlies/rel/atlrel_2" \ rel3 "nightlies/rel/atlrel_3" \ rel4 "nightlies/rel/atlrel_4" \ rel5 "nightlies/rel/atlrel_5" \ rel6 "nightlies/rel/atlrel_6"> source ${HOME}/cmthome/setup.[c]sh -tag=rel0 [1] > source ${HOME}/cmthome/setup.[c]sh -tag=rel1,opt [2]
- work with the nightly #0 using debug code (debug is default)
- work with the nightly #1 using optimized code
- Work or development areas
path_remove CMTPATH /carnault/ path_prepend CMTPATH "" tools "/afs/cern.ch/user/c/carnault/Tools" path_prepend CMTPATH "" area1 "/afs/cern.ch/user/c/carnault/MyTest1" path_prepend CMTPATH "" area2 "/afs/cern.ch/user/c/carnault/MyTest2"> source ${HOME}/cmthome/setup.[c]sh -tag=5.1.0,area1,tools,opt [1] > source ${HOME}/cmthome/setup.[c]sh -tag=rel2,opt,area1 [2]
- Work with release 5.1.0, and two development areas: area1 and Tools. Use optimized code.
- Work with the nightly #2 and development area area1; use optimized code.
- One may also associate tags with one another. For example,
tag area1 5.1.0 [1] tag area2 rel2 [2]
- area1 is associated with release 5.1.0,
- area2 is associated with the nightly release rel2
In this way, one does not need to specify the release tag when sourcing the setup:
> source ${HOME}/cmthome/setup.[c]sh -tag=area1,tools,opt [1] > source ${HOME}/cmthome/setup.[c]sh -tag=opt,area2 [2]After one has setup the environment, one can query the environment to understand the various settings.
3.3 - Re-configuring for a CMT version change
This is in fact the same operation as the bootstrap procedure [3.1], where one leaves out the step of creating the login requirements file:
- Setup up the correct version of CMT
sh> . <location of CMT>/setup.sh csh> source <location of CMT>/setup.cshwhere <location of CMT> differs from site to site. At CERN it is:/afs/cern.ch/sw/contrib/CMT/<version>/mgr/The current version of CMT is v1r16p20040901, and the previous one was v1r14p20031120 (see Release status to know which version of CMT to use - older releases may need an older CMT version.)
- Generate the setup files for your login requirements file
> cd ${HOME}/cmthome > cmt config- Then you will re-source the setup files with the tags you want as described in [3.2]
sh> source ${HOME}/cmthome/setup.sh -tag=<your tags> csh> source ${HOME}/cmthome/setup.csh -tag=<your tags>- If you already have packages checked out in a work area, then you will need to run cmt config in each package so as to recreate the setup and cleanup scripts. This may be done from the TestRelease package with a cmt broadcast command:
> cd ${HOME}/MyTest/TestRelease/TestRelease-00-00-14/cmt > cmt broadcast cmt config
Often one wants to answer the question "what is my current context?". This is typically to understand whether one has set things up correctly or to report problems. As well, this is helpful for more advanced investigation to understand a package's configuration, e.g. to find out what are the compilation flags and which ones are set. This section presents some useful queries and explains the responses:
- What is the current version of CMT ?
> cmt version v1r16p20040901- A few important environment variables to test:
> echo $CMTROOT /afs/cern.ch/sw/contrib/CMT/v1r16p20040901 > echo $CMTCONFIG i686-rh73-gcc32-dbg > echo $CMTPATH /afs/cern.ch/user/c/carnault/MyTest1:/afs/cern.ch/atlas/software/dist/5.2.0: /afs/cern.ch/atlas/offline/external/Gaudi/0.11.4CMTROOT is the path to the CMT package. CMTCONFIG is where build products are kept in each package. And CMTPATH is a path-like variable containing a set work areas.
- Which release am I using ?
- Which packages areas do I see ?
> cmt show path # Add path /afs/cern.ch/user/c/carnault/MyTest1 from initialization # Add path /afs/cern.ch/atlas/software/dist/5.2.0 from initialization # Add path /afs/cern.ch/atlas/offline/external/Gaudi/0.11.4 from initialization # Add path /afs/cern.ch/sw/contrib from default path # /afs/cern.ch/user/c/carnault/MyTest1:/afs/cern.ch/atlas/software/dist/5.2.0: /afs/cern.ch/atlas/offline/external/Gaudi/0.11.4:/afs/cern.ch/sw/contribHere the path consists of C. Arnault's development area, release 5.2.0's area, the Gaudi area, and the area where CMT itself is located.
- What are the current configuration tags ?
> cmt show tags CMTv1 (from CMTVERSION) CMTr14 (from CMTVERSION) CMTp20030620 (from CMTVERSION) Linux (from uname) package AtlasPolicy implies [Unix] i686-rh73-gcc32-dbg (from CMTCONFIG) package ExternalPolicy implies [rh73_gcc32dbx Linux redhat73 gcc-3.2 debug] CERN (from CMTSITE) package ExternalPolicy implies [ATLAS] Default (from Default) Unix (from package CMT) debug (from package CMT) package AtlasPolicy implies [dbg] gcc-3.2 (from package GaudiPolicy) redhat73 (from package GaudiPolicy) rh73_gcc32 (from package GaudiPolicy) package GaudiPolicy implies [Linux redhat73 gcc-3.2] rh73_gcc32dbx (from package GaudiPolicy) package GaudiPolicy implies [rh73_gcc32 debug] ATLAS (from package ExternalPolicy) dbg (from package AtlasPolicy) package AtlasPolicy implies [debug]From this one can see that the user is working at CERN on RedHat 7.3 with gcc 3.2 in debug mode.
Generally it is wise to systematically provide the output of all of these commands when sending bug reports.
Finally various queries are often useful to understand what might go wrong:
> cmt show uses [1] > cmt show constituents [2] > cmt show patterns [3] > cmt show macros [4] > cmt show sets [5]
- displays the dependency graph of the current package
- displays the constituents (applications, libraries or generated documents) declared by this package (or implicitly declared through patterns)
- displays all patterns available to this package (acquired from packages in the use graph) and those effectively applied.
- displays all macros and their values
- displays all environment variables and their values
Further investigation for a particular macro or set parameter:
> cmt show macro <macro> [1] > cmt show macro_value <macro>[2] > cmt show set <set> [3] > cmt show set_value <set> [4]
- displays in which package <macro> is set and a non-expanded value
- displays the expanded value of <macro>
- displays in which package <set> is set and a non-expanded value
- displays the expanded value of <set>
5.1 - Scenario #1: Running Athena
This primary scenario explains how to define the simplest environment required to run athena which uses prebuilt components from the release. In this scenario, we do not intend to modify any code yet, but we will be able to modify job-options.The current state of the software requires to first copy several data files to the working directory (job options, data sets, scripts, etc...). A conventional Make target is used for this operation.
In order to connect to the release, and to acquire settings from packages of this release, a TestRelease empty package is created, with a requirements file specifying the list of requested packages.
A simple template of this package is available from the standard CVS repository, and can be checked out as follows:
> cd ${HOME}/MyTest [1] > cmt co TestRelease [2] # ================= working on package TestRelease version TestRelease-00-00-14 (At head) path # get top files # get branches cmt doc Creating setup scripts. [3] Creating cleanup scripts.
The next operation performs the copying of the runtime files to the local directory:
- One must be exactly in one's development area to properly checkout a package (see [3.1.1.2] for setting up one's development area)
- A checkout builds the proper CMT structure.
- The cmt config command is automatically applied after the checkout, and thus does not need to be applied by hand. This generates the proper setup scripts.
> cd TestRelease/TestRelease-00-00-14/cmt [1] > source setup.[c]sh [2] > cmt show constituent_names [3] runtime apps TestReleaseChk install_readme config TestReleasecwreport myctest > cmt show macro_value constituents [4] runtime apps config > gmake [5]
Now running athena is possible by selecting one of the job options files available in the ../run directory:
- In general, all configuration or build actions should be performed while being in the cmt directory
- this will set all environment variables required to work with this package in particular (eg LD_LIBRARY_PATH)
- (optional step) using cmt show constituent_names one can display all declared constituents
- (optional step) whereas the macro constituents specifies all constituents to be built by default
- without arguments the default target all is built, which in turn triggers the build of all elements in the macro $(constituents)
> cd ../run [1] > athena HelloWorldOptions.txt
- By convention, applications are executed from the run directory. This is currently required because there are some data files read from the same directory as the application is run. These are installed in ../run by the gmake step above.
The TestRelease package is already configured to see all packages from the release (try some query commands). This happens via the use statement:
use AtlasRelease AtlasRelease-*in the requirements file. Since all packages are seen, this may cause some commands to be slow, for example, cmt show macro ... because the macros of all packages consulted. It is possible to improve this by restricting the use statements to the sets of packages which are strictly needed (eg. to those with athena components that one wants to run).
5.2 - Scenario #2: Checking out a package
This section presents the main scenarios for checking out packages.
5.2.1 - checking out a single package
This section discusses how to properly select and checkout a package. One should:
- completely identify the package of interest
- select a work area
- perform the checkout
1. completely identify the package of interest
To identify a package you must know its name, and the name if its container(s) if it is a sub-package. As well, the package tag associated with a given release is needed to check out a specific version of the package.
Both CMT and the taginfo tool offer queries to extract this information, or one can simply look at a release or use the Tag Collector.
(Note: to use the taginfo command, you need to have setup your runtime environment as described in [5.1]. For this it is sufficient to have done the sourcing of the setup in the TestRelease package:> cd TestRelease/TestRelease-00-00-14/cmt > source setup.[c]sh) For example, if one wants to find out about the GeneratorModules package, one can alternately:> ls /afs/cern.ch/atlas/software/dist/5.2.0/Generators/GeneratorModules [1] GeneratorModules-01-02-35 ... > taginfo package -name=GeneratorModules [2] simplePackage;offline/Generators/GeneratorModules/;GeneratorModules;GeneratorModules-01-02-35 > taginfo package -name=GeneratorModules -release=5.0.0 [3] simplePackage;offline/Generators/GeneratorModules/;GeneratorModules;GeneratorModules-01-02-30 > taginfo package -name=GeneratorModules -release=under_construction [4] simplePackage;offline/Generators/GeneratorModules/;GeneratorModules;GeneratorModules-01-02-38 > cmt show packages | grep GeneratorModules [5] GeneratorModules GeneratorModules-01-02-35 /afs/cern.ch/atlas/software/dist/5.2.0/Generators/GeneratorModules > cmt cvstags Generators/GeneratorModules [6] GeneratorModules-01-02-38 GeneratorModules-01-02-37 GeneratorModules-01-02-36 ... > taginfo cvstags -name=GeneratorModules -all=1 | cut -d';' -f4 [7] GeneratorModules-01-02-19 GeneratorModules-01-02-18 GeneratorModules-01-02-17 ...
- This lists the version of the package in release 5.2.0 from the cern dist area
- This command shows the GeneratorModules package for the latest frozen release
- This command shows the GeneratorModules package for release 5.0.0
- This command shows the GeneratorModules package for the release currently being worked on
- This command shows all packages reachable from the current CMTPATH list
- This command shows all CVS tags currently applied to this package
- The same query but using taginfo instead
2. select a work area
A work area is a directory where one checks out one or more packages to work on. When there is more than one package in a work area, CMT needs to know where this work area is, so as to include it in the package search-path-list. This is simply done by installing the work area in the environment variable CMTPATH.
In the special case of working on only a single package, it is not required to add one's work area to CMTPATH. In this case you will be building in the cmt directory of this package and only the base release is needed and is already in CMTPATH. Thus for this simple scenario you may skip the rest of this subsection and checkout your package in any directory.
To add your work directory to CMTPATH, one may use the shell commands :
sh> export CMTPATH=${HOME}/MyTest:${CMTPATH} csh> setenv CMTPATH ${HOME}/MyTest:${CMTPATH}Or one may use the method via your in the login requirements file (as described in [3.1.1.2]):
macro home_dir "${HOME}" path_remove CMTPATH "${home_dir}" [1] path_prepend CMTPATH "${HOME}/MyTest" [2][3]After modifying your login requirements file, you must again source your home setup file:
- this will first remove from CMTPATH from any item containing the string "${HOME}" or your home directory path
- this adds the selected work area at top of the path list
- the order in CMTPATH is relevant, i.e. one does path_prepend as opposed to path_append because packages are searched for in the the directories of CMTPATH from left-to-right
> source ${HOME}/cmthome/setup.[c]shwith any tags you may want. See Setting up for sessions [3.2]. (For more details see how to select a work area)3. perform the checkout and rebuild the package
One can either check out the HEAD revision or a specific tagged revision of a package.
For the HEAD revision of a package, one does a cmt co without any options:> cd ${HOME}/MyTest [1] > cmt co Generators/GeneratorModules [2] # ================= working on package GeneratorModules version GeneratorModules-01-02-13 (At head) path Generators # get top files # get branches GeneratorModules cmt doc src share # Required version v17r10 of package CLHEP incompatible with selected version CLHEP-00-01-07 [3] Creating setup scripts. [4] Creating cleanup scripts. [4] > cd Generators/GeneratorModules/*/cmt [5] > gmake [6]To check out a specific tagged revision of a package, one uses the -r option of the cmt co command. Note that in this case, the checkout operation will produce what CVS calls a "sticky tag" which means that in principle no modification is expected to be checked back in for this package (i.e. one can only check back in changes for HEAD revisions of packages, unless the specified tag is a branch tag). This is typically useful when a previous version of this package is required.
- one must be in the work area directory
- without tag specification, the HEAD revision will be checked out
- this is a warning and can be ignored
- a cmt config action is automatically executed, so as to generate the setup scripts for the checked out package
- build operations must be performed from the cmt directory
- we first rebuild this package
> taginfo package -name=GeneratorModules -release=3.0.1 [1] simplePackage;offline/Generators/GeneratorModules/;GeneratorModules;GeneratorModules-01-02-03 > cmt co -r GeneratorModules-01-02-03 Generators/GeneratorModules [2] # ================= working on package GeneratorModules version GeneratorModules-01-02-03 path Generators # get top files # get branches GeneratorModules cmt doc src share # Required version v17r10 of package CLHEP incompatible with selected version CLHEP-00-01-07 Creating setup scripts. Creating cleanup scripts. Installing the baseline directory > cd Generators/GeneratorModules/*/cmt > cvs status -v requirements [3] =================================================================== File: requirements Status: Up-to-date Working revision: 1.26 Repository revision: 1.26 /atlascvs/offline/Generators/GeneratorModules/cmt/requirements,v Sticky Tag: GeneratorModules-01-02-03 (revision: 1.26) Sticky Date: (none) ...
- taginfo is used to query what was the tag of this package in the release 3.0.1
- the -r option is used to select a particular tag for this version
- one may verify that this package has got a sticky tag, meaning that no modification can be committed
5.2.2 - checking out all packages of a container
In this section, we explain how to obtain into the work area all packages belonging to a given container for a particular release. The example here uses release 4.0.0, and it assumes that you have set the to be your default release (see [3.2]). The containers available in release 4.0.0 may be shown by the following command:> taginfo release -name=4.0.0 | grep -i containerPackage topContainerPackage;offline/Applications/;Applications;Applications-01-04-12 topContainerPackage;offline/Calorimeter/;Calorimeter;Calorimeter-02-00-01 topContainerPackage;offline/Control/;Control;Control-01-03-04 containerPackage;offline/Control/AthenaExamples/;AthenaExamples;AthenaExamples-01-02-25 topContainerPackage;offline/Database/;Database;Database-00-00-55 topContainerPackage;offline/Applications/;Applications;Applications-01-04-09 ... containerPackage;offline/LArCalorimeter/LArCnv/;LArCnv;LArCnv-00-05-05 ...Then the checkout operation is performed in two steps:Once all packages are retreived, some global operations may be performed on them via the cmt broadcast command as follows:
retreiving the container itself using the correct tag for release 4.0.0
> cd ${HOME}/MyTest > cmt co -r LArCnv-00-05-05 LArCalorimeter/LArCnv- retreiving the packages directly contained in the container
> cmt co -requirements LArCalorimeter/LArCnv/LArCnv-00-05-05/cmt/requirementswhere the arugment to -requirements is the requirements file of LArCnv that you have checked out.> cd LArCalorimeter/LArCnv/LArCnv-00-05-05/cmt [1] > cmt broadcast [2] > cmt broadcast cmt config [3] > cmt broadcast gmake [4]The cmt broadcast command may also be issued from the cmt directory of the TestRelease package. Make sure you have a TestRelease package checked out into your working area:
- We move to the cmt directory of the container, so as to have a consistent vision over all packages of the container
- This minimal broadcast command (with no shell action specified) is only meant to display the list of packages that will be reached during any broadcast command. This dummy command is also useful to verify that the CMTPATH is correctly setup (since broadcast is applied onto all packages reached from the first item of CMTPATH).
- This is ONLY needed to be done ONCE. This creates setup and cleanup scripts in each package. Normally cmt config is done at checkout time, but this has not been implemented for the -requirements option.
- Then it's likely that you will want to rebuild them
> cd ${HOME}/MyTest > cmt co TestReleaseGo to the cmt directory of TestRelease:> cd TestRelease/TestRelease-00-00-14/cmtModify the uses in the requirements file. Changeuse AtlasRelease AtlasRelease-*to#use AtlasRelease AtlasRelease-* [1] use LArCnv LArCnv-* LArCalorimeter [2]Then simply broadcast the gmake command:
- Comment out the dependency on AtlasRelease which in turn depends on ALL packages in the release
- Add use for the package LArCnv, all versions, which is a sub-package of LArCalorimeter
> cmt broadcast gmakewhere we have assumed that a cmt config already has been broadcast ONCE.
5.3 - Scenario #3: Modifying an algorithm
We now combine the two previous scenarios ( [5.1] and [5.2]) to check out, build and run the AthExHelloWorld example package. For a single package, one can build and run athena from the package itself.> cd ${HOME}/MyTest [1] > cmt co Control/AthenaExamples/AthExHelloWorld [2] # ================= working on package AthExHelloWorld version AthExHelloWorld-00-00-12 (At head) path Control/AthenaExamples # get top files # get branches AthExHelloWorld cmt share src Creating setup scripts. Creating cleanup scripts. > cd Control/AthenaExamples/AthExHelloWorld/AthExHelloWorld-00-00-12/cmt > gmake [3] ... > source setup.[c]sh [4] > cd ../share [5] > athena HelloWorldOptions.txt ... > emacs ../src/... [6] ... > (cd ../cmt; gmake) [6] > athena HelloWorldOptions.txt [6]
- Again, we must start from the development area
- The path to the package has to be completely specified, including its container names. Note: "Creating xxx scripts" means that cmt config has been run
- The component can be immediately built
- To test the component one has to set the needed environment variables
- HelloWorldOptions.txt is in the share directory
- This is the main development cycle
Alternately, one may build and run from the TestRelease package. This is useful when more than one package is checked out in one's development area.
Modify the use statements in the requirements file of TestRelease:use Control Control-* [1] # use Event Event-* [1] use Generators Generators-* [1] # use Simulation Simulation-* [1] ... #use AtlasRelease AtlasRelease-* [2]Now repeat the steps above, this time from TestRelease:
- Needed for runtime environment - access to the needed component libraries
- Reduce the number of dependent packages
> cd ${HOME}/MyTest/TestRelease/TestRelease-00-00-14/cmt > cmt broadcast gmake [1] > source setup.[c]sh [2] > cd ../run [3] > athena HelloWorldOptions.txt > ... > emacs .../src/HelloWorld.cxx [4] > ... > cmt broadcast gmake [4] > athena HelloWorldOptions.txt [4]Finally, when one has several packages checked out, there are two useful options to the cmt broadcast which allow one to selectively build certain packages:
- A broadcast can run any command in all dependent packages in one's work area
- Setup runtime environment
- JobOption files have been copied in the gmake step into the run directory
- Again, the main development cycle
> cmt broadcast -select=AthExHello gmake [1] > cmt broadcast -begin=AthExHello gmake [2]
- The -select option will select only those packages whose full directory path contains the provided AthExHello string
- The -begin option will begin the gmake with the package that matches the provided string, thus continuing until TestRelease itself
5.4 - Checking in a package
We assume here that a package had first been checked out, and some modifications were done to it.
5.5 - Tagging a package
5.6 - Creating a new package
We explain briefly here the syntax of a requirements file. In a requirements file, one can set parameters that are internal to CMT, macro's, or ones that are exported to the environment, set's. One can also prepend/append/remove items to path-like environment variables (e.g. PATH or LD_LIBRARY_PATH).
These all follow the same syntax, e.g.:macro ATLAS_DIST_AREA "/afs/cern.ch/atlas/software/dist"where there is a keyword (macro) followed by the name of the parameter (ATLAS_DIST_AREA) and a default value (/afs/cern.ch/atlas/software/dist), which must be within " " if it includes spaces. This may be followed by other values and the tag which selects them.
For the macro ATLAS_RELEASE example of [3.2]:macro ATLAS_RELEASE "4.0.0" \ 3.0.2 "3.0.2" \ 3.2.1 "3.2.1" \ rel0 "nightlies/rel/atlrel_0" \ rel1 "nightlies/rel/atlrel_1" \ rel2 "nightlies/rel/atlrel_2" \ rel3 "nightlies/rel/atlrel_3" \ rel4 "nightlies/rel/atlrel_4" \ rel5 "nightlies/rel/atlrel_5" \ rel6 "nightlies/rel/atlrel_6"the default value is 4.0.0. There are other values with their corresponding tags, e.g. rel0 is the tag for nightlies/rel/atlrel_0. Note that the "\" character is a continuation character and that macro statements with tags are conventionally formatted as above to easily match the tag/value pairs.
6.1 - Conventional macros
(to be completed)
6.2 - Conventional patterns
(to be completed)