tdaq-09-01-00

Introduction

Doxygen Javadoc

The ATLAS TDAQ software version tdaq-09-01-00 has been released on 15th July 2020.

Please do not use tdaq-09-01-01, that will never go to Point 1 but is only for having a Python 2 based build until off-line has officially switched.

Availability and Installation

Outside of Point 1 the software should be used via CVMFS. It's official location is 

/cvmfs/atlas.cern.ch/repo/sw/tdaq/tdaq/tdaq-09-01-00/

At Point 1 the software is as usual available at

/sw/atlas/tdaq/tdaq-09-01-00/

The software can also be installed locally via ayum.

git clone https://gitlab.cern.ch/atlas-sit/ayum.git
source ayum/setup.sh

Modify the prefix entries in the yum repository files in ayum/etc/yum.repos.d/*.repo to point to the desired destination.

ayum install tdaq-09-01-00_x86_64-centos7-gcc8-opt

In case the LCG RPMs are not found, add this to etc/yum.repos.d/lcg.repo:

[lcg-repo-97]
name=LCG 97 Repository
baseurl=http://lcgpackages.web.cern.ch/lcgpackages/lcg/repo/7/LCG_97a/
enabled=1
prefix=[...your prefix...]

Configurations

The release is available for the following configurations:

  • x86_64-centos7-gcc8-opt
  • x86_64-centos7-gcc8-dbg

External Software

LCG_97apython3

The version of the external LCG software is LCG_97apython3.

TDAQ Specific External Software

Package Version
cmzq 3.0.2
zyre 1.1.0
libfabric 1.6.2
nlohmann/json 2.1.1
pugixml 1.9
ipbus-software 2.7.2
microhttpd 0.9.59
mailinglogger 5.0.0
netifaces 0.10.9
paramiko[gssapi] 2.7.1

CES

Package: CES
Jira: ATDAQCCCES

Live documentation about recoveries and procedure can be found here.

Changes in recoveries and/or procedures:

  • HLT recoveries: sending FORK_AFTER_CRASH only if RUNNING (ADHI-4780).

Internal changes:

  • The ESPER CEP engine has been updated to version 8.5;
  • Fixed memory leak in the Metaspace (ATDAQCCCES-154);
  • Added sTGC as a possible source of trigger on hold;
  • Avoid concurrent executions of the ESPER compiler.

Igui

Package: Igui

  • Commit & Reload functionality adapted to the new OKS GIT back-end (ADTCC-227):

    A new reload panel has been developed in order to allow proper configuration reload when the new OKS GIT back-end is used. In this new version of the panel, the user is asked to select the configuration version to reload on the basis of the last commits available in the GIT repository. The selection of the configuration to be reloaded is no more file-based, but the user is asked to select a GIT commit that, in principle, may include changes to several separate files. In such a way, there is not any risk no more to select a set of incompatible files. The list of modified files per commit is available un-hiding a specific column in the table (that can be done changing the table properties using the little button located on the right upper corner of the table). Of course, the selection of a specific commit implies to import all the changes applied in the previous commits.

Jers - Java ERS

tdaq-09-00-01

Use of TDAQ_ERS_TIMESTAMP_FORMAT with default value "yyyy-MMM-dd HH:mm:ss,SSS z" to allow defining timestamp format like in ERS, including milliseconds. The format used before was "DDD MMM dd HH:mm:ss z yyyy" e.g. Mon Mar 30 15:06:28 CEST 2020.

MonInfoGatherer

Add support for dynamically labelled axes (ADHI-4769).

ProcessManager

Package: ProcessManager

The ProcessManager twiki can be found here.

Internal changes:

  • pmg_start_app: fixed issue with parsing complex environment in command line options.

RCUtils

Package: RCUtils

  • Adapting to the new OKS GIT back-end (ADTCC-227):
    The proper OKS version to use is set by the setup_daq script;
    The RunParams IS server is started early by setup_daq in order to allow storing information about the OKS version to use;
  • Fixed several issues in setup_daq building the RootController environment in the case of complex environment variables;
  • Several fixes to the DAQ efficiency tools in the area of the Hold Trigger accounting.

RunControl

Package: RunControl
Jira: ATDAQCCRC

This is the link to the main RunControl twiki.

Internal changes:

  • Python library: fixed initialization of the python sub-system;
  • When OKS GIT is used, the RootController sets the currently used configuration version (ADTCC-227).

SFOng

  • replaced deprecated tbb::tbb_hasher with std::hash
  • replaced deprecated tbb::atomic with std::atomic
  • fixed: starting a new run without going to shutdown (stop/start) could produce Late events at the beginning of the new run for a transition period

TDAQExtJars

Package: TDAQExtJars

  • The following packages have been updated to a newer version:
    JFreeChart (from 1.0.16 to 1.5.0)
    jide-oss/common (from 3.5.8 to 3.7.7)
    com.jhlabs.filters (2.0.235-1)
    mysql-connector-java (from 3.0.16 to 8.0.19)
    oracle.ojdbc (from 11.2.0.3.0 to 19.3.0.0)
    jldap (from 2007.10.12 to 2009-10-07)
    esper (from 8.4.0 to 8.5.0)

Test Manager

TriggerCommander

Package: TriggerCommander

Clients

A backward compatible change (for clients) has been introduced in the TriggerCommander/MasterTrigger.h interface. The hold() method has been extended with a boolean flag extended_lvl1id which defaults to false.

The method will return the last ECR as it did in the past. When the new flag is set to true it will instead return the full Level 1 ID (from which the ECR can be extracted as the uppermost 8 bits).

Implementations of the MasterTrigger interface

An implementation of the MasterTrigger interface has to modify its overloaded method:

class X : public MasterTrigger {
  ...

  uint32_t hold(const std::string& dm, bool extended_lvl1id = false) override;
  ...
};

coldpie

  • Added few missing features that were not implemented before
  • Record indexing operator now returns field value instead of Field instance

config

tdaq-09-01-00

New way to access config versions

Jira: ADTCC-228

Using oks git repository one can access details of the available config versions. The details of version are described by methods of new c++ Version class:

const std::string& get_id() const noexcept;
const std::string& get_user() const noexcept;
const std::time_t get_timestamp() const noexcept;
const std::string& get_comment() const noexcept;
const std::vector<std::string>& get_files() const noexcept;

and similar java ones:

String get_id();
String get_user();
Instant get_timestamp();
String get_comment();
String[] get_files();

To obtain new versions created on remote origin after currently used version, or externally modified files, use new get_changes() method:

std::vector<daq::config::Version> get_changes(); // c++
Version[] get_changes(); // java

To access historical versions use new get_versions() method:

enum QueryType { query_by_date, query_by_id, query_by_tag };

std::vector<dVersion> get_versions(const std::string& since, const std::string& until, QueryType type, bool skip_irrelevant); // c++
Version[] get_versions(String since, String until, Version.QueryType type, boolean skip_irrelevant); // java

The config_dump application has new command line option to print information about archived versions:

-v | --versions  rint details of versions providing 4 parameters all|skip date|id|tag since until

For example:

$ config_dump -d oksconfig:daq/schema/core.schema.xml -v all date '2020-05-19' '2020-05-20'

create() database method

Jira: ADTCC-240

Remove the server_name argument in the Configuration create() method. In case of rdbconfig implementation the user has to connect with rdb writer before calling create() method.

dal

tdaq-09-01-00

SW repository generation utility was removed

Remove dal_create_sw_repository. The create_repo.py has to be used instead. See CMake TWiki for more information.

dbe

Package: dbe
Jira: ATLASDBE

  • Schema editor: the schema file to be loaded can be passed as a command line argument using the -f option;

  • Support for new archival mechanism based on the OKS-GIT back-end (ADTCC-241):

    Configurations archived in the GIT repository can be loaded using the -v command line option and specifying the GIT hash corresponding the desired configuration.

dcm

DVS (Diagnostics and Verification Framework)

DVS GUI (graphical UI for DVS)

See also related DVS and TestManager packages.

DVS Tests

dynlibs - Load shared libraries

This package is deprecated from tdaq-09-00-00 onwards.

Please use plain dlopen() or boost::dll instead. Note that unlike in this package, the boost::dll::shared_library object has to stay in scope as long as the shared library is used !

Example of possible replacement

#include <boost/dll.hpp>

double example()
{
   boost::dll::shared_library lib("libmyplugin.so",
                                  boost::dll::load_mode::type::search_system_folders |
                                  boost::dll::load_mode::type::rtld_now);
   // Get pointer to function with given signature
   auto f = lib.get<double(double, double)>("my_function")
   return f(10.2, 3.141);
}

emon

  • Connections between event samplers and monitors have been optimized. Existing configurations should be adjusted to benefit form that. Previously the NumberOfSamplers parameter has been used to define a number of samplers to be connected by all monitors of a group that uses the same selection criteria. In the new implementation this number defines a number of samplers that each individual monitor has to connect. That makes no difference for monitors that used to connect to a single sampler and do't form a group. For the monitors that share the same selection criteria, like for example the Global Monitoring tasks, this number should be changed to the old number divided to the number of the monitors in the group. For Athena monitoring the corresponding parameter of a JO file is called KeyCount.

HLTSV - HLT Supervisor

jeformat

Package: jeformat

Changes in public APIs:

  • Added sTGC to the detector mask.

ls

Package: ls

  • Fixed issue with sqlite file permissions (ADTCC-245):

    The name of the user running the logger application can now be set in the connection string with a place-holder. As an example, here is a possible set of command line options:

    -e mts -S app!=mts-sender* -t ${TDAQ_VERSION} -p ${TDAQ_PARTITION} -c ${LOGGER_CONNECT} -n RunParams -s 10000 -b sqlite_file:${TDAQ_LOGS_PATH}/Logger_BackupDb_$(USER).db

    The $(USER) placeholder will be then replaced with the real user name.

mda

Implemented support for future upgrades of MDA database schema. Original intent was to enable storing histograms in ROOT file in their original form without replacing colons with underscores but that did not work out (see ADHI-4177 for details). Schema version support ay still be useful for future updates so I keep it.

MTS

OKS

tdaq-09-01-00

Jira: ADTCC-214, ADTCC-226 and ADTCC-227

Replace cvs repository by git repository.

The oks repository is stored on a git server. To access oks configuration the git repository is cloned into a temporal area. When changes are committed, the git server validates them verifying consistency of all repository files and checking user permissions as defined by the Access Manager policy. If commit is successful, the git server updates latest version of oks files on the filesystem (git repository mapping). Contrary to cvs implementation, the repository mapping is not required to access oks data using git repository. It is only implemented for convenience of users (fast browse of files) or filesystem-based access.

This is the main difference between oks git and cvs repository implementations: * the oks git implementation uses git database to store and access oks files (like DBMS) and ignores the files on the filesystem; * the oks cvs implementation used filesystem to store and access oks files.

The ignorance of files on the filesystem by oks git is done on purpose to preserve used configurations, and to implement configuration archiving on top of git database.

If user needs to access configuration from files stored on filesystem, it is necessary to disable oks git as explained in next section.

Environment Variables

The TDAQ_DB_REPOSITORY is the only setting that enables or disables use of the oks git repository. If it is set, the oks git repository is used. If not set, the filesystem repositories are used.

There are 3 shell functions available after release setup on TBed and Point-1 to enable, disable and get status of the oks git repository: * oks-git-on - to enable use of the oks git repository * oks-git-off - to disable use of the oks git repository * oks-git-status - to give status of the oks git repository use

The variable TDAQ_DB_REPOSITORY contains one or several git URLs (it does not point to a filesystem anymore). If there are several URLs, the used one depends on the OKS_GIT_PROTOCOL variable. If it is not set, the first URL is used. To get URL use oks_git_repository utility, e.g. to clone repository:

$ git clone `oks_git_repository`

The TDAQ_DB_USER_REPOSITORY can point to user git area. If it is set, oks-based tools will not clone git repository, but will use that area instead. It is responsibility of user to create that area and remove when not needed.

The TDAQ_DB_VERSION can be used to access particular revision of oks git repository. It is used internally by setup_daq to preserve concrete revision to be used by data taking session. In case of configuration reload this variable is set into a new value and distributed to all processes accessing oks configuration. The variable can be used in two formats: * hash:$value - select revision by explicit hash * date:$value - select latest revision before given date or timestamp If the variable is not defined, the latest revision will be checked out.

The variable OKS_REPOSITORY_MAPPING_DIR points to the repository mapping.

As before, the TDAQ_DB_PATH can be used for a filesystem-based access and can contain several colon-separated filesystem repositories. With filesystem based access, the oks data from git repository can be included using the repository mapping variable, for example:

$ unset TDAQ_DB_REPOSITORY
$ export TDAQ_DB_PATH=$OKS_REPOSITORY_MAPPING_DIR:/det/a/b/c:/home/x/y/z

File paths

To access an oks file from git repository one has to use its repository filename, for example:

$ oks_data_editor daq/segments/setup.data.xml

One should not use any filesystem relative or absolute paths referencing oks git repository files. There are two exception.

Filepaths relative to repository mapping

is the filepaths relative to the OKS_REPOSITORY_MAPPING_DIR. That is done to help users with shell path path-completion (hitting TAB key), for example the following commands will correctly checkout the files from git repository and run appropriate tools:

$ export OKS_REPOSITORY_MAPPING_DIR=/tbed/git/oks/tdaq-99-00-00
$ oks_data_editor /tbed/git/oks/tdaq-99-00-00/daq/partitions/part_hlt.data.xml
$ setup_daq /tbed/git/oks/tdaq-99-00-00/daq/partitions/part_hlt.data.xml part_hlt

However, the preferable way is to use the repository filenames, e.g.:

$ oks_data_editor daq/partitions/part_hlt.data.xml
$ setup_daq daq/partitions/part_hlt.data.xml part_hlt

Filepaths relative to user repository

If the files are checked out by user, one should set TDAQ_DB_USER_REPOSITORY to access them.

The following example will result an error, since user provides path relative to current working directory:

$ cd `mktemp -d`
$ git clone `oks_git_repository` .
$ oks_dump -f oks_dump -f daq/segments/setup.data.xml

To make it working it is necessary to set the above variable:

$ cd `mktemp -d`
$ git clone `oks_git_repository` .
$ export TDAQ_DB_USER_REPOSITORY=`pwd`
$ oks_dump -f oks_dump -f daq/segments/setup.data.xml

OKS tools

The tools from previous implementation can be used with oks git repository.

oks_dump

As in the previous implementation, the oks_dump can be used for fast check of oks configuration. Now it shows details of git operations, if the git repository is used, e.g.:

$ oks_dump -f daq/segments/setup-initial.data.xml
2020-May-15 20:04:12.645 [OKS checkout] => oks-checkout.sh -u /tmp/oks.HJdfFK --hash "d3000e127c6b0fa9c5499ce995847acd96dc61f5"
git clone -q -n ssh://pc-tbed-git.cern.ch/srv/gitea/atlas-gitea/tdaq-99-00-00.git .
git checkout -q d3000e127c6b0fa9c5499ce995847acd96dc61f5
checkout oks version d3000e127c6b0fa9c5499ce995847acd96dc61f5
2020-May-15 20:04:13.708 [OKS checkout] => done in 94.142 ms
Reading data file "/tmp/oks.HJdfFK/daq/segments/setup-initial.data.xml" in normal format (24228 bytes)...
...

oks-import.sh

As before, this tool allows to import user files from previous repositories or release installation areas. Change current working directory to a repository root and import required files and directories.

For example, to import some files from TDAQ release:

bash$ cd ${TDAQ_INST_PATH}/share/data
bash$ oks-import.sh -m "import release files" daq/schema daq/sw/*.data.xml daq/segments/common-environment.data.xml

To import some sub-detector folders from previous release:

bash$ cd /atlas/oks/tdaq-09-00-00
bash$ oks-import.sh -m "commit message" tile/hw tile/sw

OKS data editor

New functionality: * commit updated files into git repository * show notification about new commits in the git repository and merge local changes with master branch * browse archived versions in the git repository and switch to an archived version

There is no need to create a temporal user repository, it is done automatically.

OKS archive

To clone git repository use oks_clone_repository utility. The user can specify output directory and particular version by commit hash, date or tag. For example checkout by tag configuration used for run 26713 and partition part_hlt:

bash$ oks_clone_repository --version tag:r26713@part_hlt

Direct use of GIT interface

The git interface is exposed to user. It is possible to clone git repository, copy, delete files, use any editors to update them and commit changes back to the git repository, e.g.:

$ cd `mktemp -d`
$ git clone `oks_git_repository` .
$ modify, copy, remove any files
$ git commit -m 'describe changes' x y z ...
$ git push origin

Archiving

The oks2coral archiving in Oracle is disabled. If files are stored in git repository, they are automatically archived into git database.

The run number database stores a reference to the used config version and the git repository is tagged by run number and partition name.

OKS

tdaq-09-01-00

  • replace boost::filesystem by std::filesystem
  • receiver application:
  • performance improvements in pre-compaction algorithm, see ADAMATLAS-383
  • monitoring the pre-compaction and the commit functions times
  • repository serevr monitoring:
  • publish information about disk used space

RDB

tdaq-09-01-00

Jira: ADTCC-227 Changes caused by replacement of OKS cvs server by git technology.

IDL Changes

Add methods to access oks git repository revisions:

RDBRepositoryVersionList get_changes() raises (CannotProceed);
RDBRepositoryVersionList get_versions(in boolean skip_irrelevant, in VersionsQuerytType query_type, in string since, in string until) raises (CannotProceed);

Modify reload method: * remove ReloadParam parameter because possibilities to reload partition and all rdb servers of IPC domain were never used in practice; the reload command reloads this server only; it this server is master RDB server, it pushes changes to mirror, pool and slave servers * change meaning of the RDBNameList names: * in case of oks repository used its vector must contain one and only one config version string; * otherwise, in case of filesystem repository used, as before, it contains lists of files to be reloaded.

Utility Changes

Following above changes of IDL, the rdb_admin modified command line option to reload databases and added options to display new and archived configuration versions.

rn (Run Number)

tdaq-09-01-00

Jira: ADTCC-242

Store start of run timestamp and run duration with nanoseconds resolution. If oks repository is used, store oks config version into run number database and tag oks repository by run number / partition name.

swrod

  • Two attributes of the SwRodInputLink class have been renamed:
  • DetectorID => DetectorResourceId

  • DetectorName => DetectorResourceName

  • HLT Request Handler implementation has been updated to improve performance. As a result an object of the SwRodHLTRequestHandler class has to be linked with each SwRodRob object via its Consumers relationship. A link between SwRodConfiguration and SwRodHLTRequestHandler objects must be removed.

  • By default ROBs and E-Links IDs in a SW ROD OKS configuration are shown in hexadecimal format.

  • Common SW ROD configuration objects, which don't normally require customization have been placed to the daq/sw/swrod-common.data.xml OKS configuration class that is installed to the installed/share/data area of the TDAQ release. Any SW ROD OKS configuration is advised to use these objects instead of creating custom ones unless any parameters modification is required.

  • GBTModeBuilder and FullModeBuilder algorithms now automatically detect if the respective SW ROD application uses TTC data handler for getting L1 Accept packets from FELIX. If that is the case the algorithms will run in TTC-aware mode, otherwise they will be data-driven. Contrary to the previous release there is no need to change the value of the Type parameter of the ROBFragmentBuilder class in the OKS configuration to change the fragment building mode. In the new implementation one should either link an instance of the SwRodL1AInputHandler with the SwRodConfiguration object via the L1AHandler relationship to use TTC-aware variant of the chosen algorithm or otherwise to leave this relationship empty to use the fragment building algorithm in data-driven mode.

  • Each detector custom plugin may now provide a function for data integrity validation following the example given below. If a plugin provides such a function the function name shall be set to the DataIntegrityChecker attribute of the SwRodCustomProcessingLib OKS configuration object that describes this plugin.

extern "C"
std::optional<bool> dataIntegrityChecker(const uint8_t * data) {
    if (contains_checksum(data)) {
        uint8_t checksum = get_checksum(data);
        return checksum == calculate_checksum(data) ? true : false;
    }
    return std::nullopt;
}
  • The package provides so called Felix Emulator that can be used to send input data to an arbitrary SW ROD Application via Netio protocol. OKS file data/FelixEmulatorSegment.data.xml shows how to configure Felix Emulator. This example defines a FelixEmulator that can provide input to the SW ROD Application defined in the data/SwRodSegment.data.xml file. The FelixEmulator uses internal data generators to create L1A and data packets and sends these packets to the SW ROD Application. The Emulator is implemented by the SW ROD framework using a special plugin that can publish generated data via Netio protocol. Generated data packets can be customized either by modifying the DataInput interface implementation provided by the test/core/InternalDataGenerator.h(cpp) files or by creating a new plugin that declares and implements a new generator class that inherits the swrod::test::InternalDataGenerator and overrides its generatePacket() virtual function.