Nightly TDAQ Release

These are the evolving release notes for the next major TDAQ release.

source /cvmfs/atlas.cern.ch/repo/sw/tdaq/tools/cmake_tdaq/bin/cm_setup.sh nightly
mkdir work
cd work
cp /cvmfs/atlas.cern.ch/repo/sw/tdaq/tools/cmake_tdaq/cmake/templates/CMakeLists.txt .
getpkg owl
mkdir build
cd build
cmake ..
make
make install

Use of Python 2 based system commands

With the move to Python 3 inside the TDAQ software there is now a fundamental incompability between the normal system setup and the TDAQ environment. If one wants to call a system command that is implemented in Python (2 for CentOS 7), like yum or auth-get-sso-cookie the environment has manipulated.

In most cases something like the following will be enough:

env -u PYTHONHOME -u PYTHONPATH auth-get-sso-cookie ...

If the command loads compiled libraries as well, it may be necessary to add a -u LD_LIBRARY_PATH as well.

BeamSpotUtils

Updates to support new HLT histogram naming convention for Run3.
Add support for new track-based method.
Improvements and refactoring to support easier testing.

CES

Package: CES
Jira: ATDAQCCCES

Live documentation about recoveries and procedure can be found here.

Changes in recoveries and/or procedures:

When the TDAQ_CES_FORCE_REMOVAL_AUTOMATIC environment variable is set to true, then no acknowledgment is asked by the operator for a stop-less removal action to be executed (regardless of the machine or beam mode);
Stop-less removal involving the SwRod: the reporting application now always receives the initial list of components back (and not only the valid components, as it was before);
After each clock switch, a new command is sent to AFP and ZDC;
The BeamSpotArchiver_PerBunchLiveMon application it not started at warm stop anymore;
The DCM is now notified when a SwROD dies or is restarted (in the same way as it happens with a ROS or a SFO).

Internal changes:

Following changes in the MasterTrigger interface;
Fixed bug not allowing the auto-pilot to be disabled in some conditions;
Fixed bug causing the ERS HoldingTriggerAction message to not be sent.

Igui

The Igui twiki can be found here.

The Igui settings can now be configured via a configuration file. The file must be formatted as a property file. The Igui settings can still be configured using command line (i.e., system) properties and environment variables as described here; at the same time priorities for configuration items have been defined: - Items defined in the configuration file have the highest priority; - Items not found in the configuration file are then searched in system properties; - If no properties are defined for a configuration item, then environment variables are used.

The configuration file can be defined via the igui.property.file property. If that property is not defined, then a configuration file is searched in <USER-HOME>/.igui/igui_<TDAQ-RELEASE>.properties.

Here is an example of a configuration file containing all the available options:

# [See https://twiki.cern.ch/twiki/bin/viewauth/Atlas/DaqHltIGUI#Settings for more details](https://gitlab.cern.ch/atlas-tdaq-software/Igui)

# [Environment variables can be used with the following format:](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [${ENV_VAR_NAME:-defaultValue}](https://gitlab.cern.ch/atlas-tdaq-software/Igui)

# [The default ERS subscription](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [That can also be defined via the TDAQ_IGUI_ERS_SUBSCRIPTION environment variable](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
igui.ers.subscription=sev = FATAL

# [The location of the OKS file holding the description of the default ERS filter](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [That can also be defined via the TDAQ_IGUI_ERS_FILTER_FILE environment variable](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
igui.ers.filter.file=/atlas/oks/${TDAQ_RELEASE:-tdaq-10-00-00}/combined/sw/Igui-ers.data.xml

# [The log-book URL](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [That can also be defined via the TDAQ_ELOG_SERVER_URL environment variable](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
igui.elog.url=http://pc-atlas-www.cern.ch/elisa/api/

# [The OKS GIT web view URL](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [That can also be defined via the TDAQ_IGUI_DB_GIT_BASE environment variable](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
igui.git.url=http://pc-tdq-git.cern.ch/gitea/oks/${TDAQ_RELEASE:-tdaq-10-00-00}/commit/

# [The browser to be used to open web links](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [That can also be defined via the TDAQ_IGUI_USE_BROWSER environment variable](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
igui.use.browser=firefox

# [Whether to show the warning at CONNET or not](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [That can also be defined via the TDAQ_IGUI_NO_WARNING_AT_CONNECT environment variable](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
igui.hide.connect.warning=true

# [Whether to ask for the control resource when the Igui is started](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [That can also be defined via the TDAQ_IGUI_USE_RM environment variable](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
igui.useRM=true

# [The timeout (in minutes) after which an Igui in status display mode will show a dialog informing ](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [the operator that it is going to be terminated](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [That can also be defined via the TDAQ_IGUI_FORCEDSTOP_TIMEOUT environment variable](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
tdaq.igui.forcedstop.timeout=60

# [Message to be shown by a confirmation dialog before the STOP command is sent](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [That can also be defined via the TDAQ_IGUI_STOP_WARNING environment variable](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
igui.stop.warning=Do not stop the partition@${TDAQ_PARTITION:-ATLAS}

# [Number of rows to be shown by default by the ERS panel](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [That can also be defined via the TDAQ_IGUI_ERS_ROWS environment variable](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
igui.ers.rows=1000

# [Panels to forcibly load when the Igui is started](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [That can also be defined via the TDAQ_IGUI_PANEL_FORCE_LOAD environment variable](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [Panels are identified by their class names; multiple panels can be separated by ":"](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
igui.panel.force.load=PmgGui.PmgISPanel:IguiPanels.DFPanel.DFPanel

# [Name of the main RDB server](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [WARNING: do not define this property unless you really know what you are doing](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
igui.rdb.name=RDB

# [Name of the read/write RDB server](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
# [WARNING: do not define this property unless you really know what you are doing](https://gitlab.cern.ch/atlas-tdaq-software/Igui)
igui.rdb_rw.name=RDB_RW

Jers - Java ERS

tdaq-11-04-00

Use of unmaintained SPI (https://github.com/rspilker/spi) for annotating classes providing ers.Stream interface for runtime registration in ERS StreamFactory is replaced with org.reflections (https://github.com/ronmamo/reflections). Now stream classes are declared (annotated) like

@ErsStreamName(name="null")
public class NullStream extends AbstractOutputStream { ...

The list of packages for search of stream providers is ers and mts.

MonInfoGatherer

Alternative implementation for merging non-histogram data (ADHI-4842). This should resolve most of the timing issues we have seen with DCM data in the past. It is enabled by default but can be disabled with ISDynAnyNG configuration parameter (see ConfigParameters.md).

PartitionMaker

The underlying implementation of the pm_farm.py tool has been changed from ParallelSSH to fabric. The reason is that the former seems to be mostly unmaintained in the last couple of years.

ProcessManager

The ProcessManager twiki can be found here.

RunControl

This is the link to the main RunControl twiki.

SFOng

added: periodic update of free buffer counter (IS) even when no data are received was: "0" free buffers was published when no data received giving the wrong impression that SFOng was about to assert backpressure.

TDAQExtJars

tdaq-12-00-00

The following packages have been updated to a newer version: io.jsonwebtoken (from 0.11.2 to 0.12.6)
The following packages have been added: bouncycastle (version 1.79)

TriggerCommander

Package: TriggerCommander

Implementations of the MasterTrigger interface

An implementation of the MasterTrigger interface has to implement a new method:

class X : public MasterTrigger {
  ...

  void setPrescalesAndBunchgroup(uint32_t l1p, uint32_t hltp, uint32_t bg) override;
  ...
};

beauty

nightly

Remove automated setting of pBeast server

The initial Beauty implementation provided code automatically setting the pBeast server based on the values of environmental variables. This logic proved to be weak and not easily maintainable: * especially in testbed, the pBeast server name has changed many times * overloading the pBeast environmental variable for the authentication method for setting the server name is strictly speaking incorrect.

The new implementation removes completely this detection code. It is instead responsibility of the user to always provide a server name to the Beauty constructor.

Old code relying on the previous implicit mechanism will fail:

>>> import beauty
>>> beauty.Beauty()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() missing 1 required positional argument: 'server'

Control pBeast authentication method

Different pBeast servers require different authentication methods. Currently two methods are support: * no authentication required * authentication via an existing Kerberos token

The authentication method to be used can be controlled via a dedicated environmental variable (PBEAST_SERVER_SSO_SETUP_TYPE) or via the library API. The latter method is now exposed in the Beauty interface. The Beauty.__init__ method accepts a keyword argument cookie_setup. Valid values are: * None → the default behaviour of the pBeast library will be used. The environment variable will be respected, if set * Beauty.NOCOOKIE → no authentication required * Beauty.AUTOUPDATEKERBEROS → authentication via Kerberos token

>>> import beauty
>>> b = beauty.Beauty('http://someserver', cookie_setup=beauty.Beauty.NOCOOKIE)

CMake for TDAQ

tdaq-common-12-00-00

The -fwrapv option has been added to the default compiler options. This makes sure that signed integer overflow has defined behaviour rather than UB, which led to some surprising changes in optimized gcc 14 builds.

Add new `EXTRA_EXTERNALS` keyword to `tdaq_project()`

Packages referenced in the EXTRA_EXTERNALS argument are added as an RPM dependency to the project. This means they are automatically installed at Point 1 and other private installations. This is used for package that are not used by TDAQ but required by some downstream detector project.

Add each externals path to `CMAKE_PREFIX_PATH`

Each LCG package referenced in the EXTERNALS argument to tdaq_project() is automatically added to the cmake prefix path. This ensures that it can be found even if the LCG package name differs from the actual upstream package name (Example: nlohmann_json vs. jsonmcpp).

Set CMake Policy CMP0144 to `NEW`

Given a package foo CMake will automatically add the content of the ${FOO_ROOT} variable to the search path. That means that all LCG packages that export CMake config file will be found (as long as the LCG package names is the same as upstream), as variables of that form have always been defined by cmake_tdaq.

It might also pick-up unwanted stuff. In this case one can either add a cmake_policy(SET CMP0144 OLD) on the appropriate level, or just unset(FOO_ROOT CACHE) where needed.

An example is git which is part of LCG. However, that version does not work properly without some environment set to the right paths (as git contains hard-coded paths determined at build time). Functions like ExternalProject_Add() may fail in that case.

Use Boost native cmake config files instead of FindBoost.cmake

This should be mostly compatible as long as the Boost targets are used.

Use ROOT native cmake config files instead of FindROOT.cmake

Again this should be compatible as long as ROOT targets are used.

Compressed debug sections

Debug sections are compressed with zlib. The *.debug files are put into a .debug subdirectory of bin or lib rsp. They are found transparently by gdb and other tools which need them.

Initial support for C++20 modules

This only works (for some definition of "works") with gcc >= 14 and Ninja instead of Make. Note that gcc 13 is most likely the default compiler for 2025, so this should be restricted to private tests. Ninja is included in the PATH if it found at cm_setup time.

cm_setup nightly gcc14
cmake -B build -G Ninja
cmake --build build -j

Enable monolithic RPM generation, make src install optional

Downstream projects should be able to call cpack after a build and get a usable tar file or monolithic RPM.

cmake_minimum_required(VERSION 3.29.0)

# [We make both detector and TDAQ version variables so the](https://gitlab.cern.ch/atlas-tdaq-software/cmake_tdaq)
# [can be overridden from the command line for test builds](https://gitlab.cern.ch/atlas-tdaq-software/cmake_tdaq)
# [cmake -D TDAQ_VERSION=99.0.0 ...](https://gitlab.cern.ch/atlas-tdaq-software/cmake_tdaq)
set(DET_VERSION 1.0.0 CACHE STRING "DET Release version")
set(TDAQ_VERSION 11.2.0 CACHE STRING "TDAQ Release version")

project(DET VERSION ${DET_VERSION} LANGUAGES C CXX)

# [Optional, leave out if you want source code installed into project](https://gitlab.cern.ch/atlas-tdaq-software/cmake_tdaq)
# [this has to come *before* find_package(TDAQ)](https://gitlab.cern.ch/atlas-tdaq-software/cmake_tdaq)
option(TDAQ_INSTALL_SOURCE "Disable full source code installation")

find_package(TDAQ)
include(CTest)

# [This enables a monolithic RPM or tar file](https://gitlab.cern.ch/atlas-tdaq-software/cmake_tdaq)
set(CPACK_MONOLITHIC_INSTALL TRUE)

# [optional, similar for other variables](https://gitlab.cern.ch/atlas-tdaq-software/cmake_tdaq)
set(CPACK_RPM_PACKAGE_DESCRIPTION "My DET project")

# [Note: For RPM installation, tdaq, zeromq and protobuf are all dependencies](https://gitlab.cern.ch/atlas-tdaq-software/cmake_tdaq)
# [and are automatically installed.](https://gitlab.cern.ch/atlas-tdaq-software/cmake_tdaq)
tdaq_project(DET ${DET_VERSION}
             USES tdaq ${TDAQ_VERSION}
             EXTERNALS zeromq
             EXTRA_EXTERNALS protobuf)

coca

Tag coca-03-16-00
Add support for location overrides via metadata config.
Add coca-cli script which will be main CLI for coca in the future.
Tag coca-03-15-12
Fix stringification of MD5 checksum.
Tag coca-03-15-11
Fix the location of py.typed in CMakeLists.txt
Tag coca-03-15-10
Replace deprecated filesystem path method.
Tag coca-03-15-09
Add type annotations for Python modules.

coldpie

tag coldpie-00-06-00
dropped Python2 support
tag coldpie-00-05-01
add coral.pyi and cool.pyi type-annotated interface specifications

DAQ Tokens

DPoP (Demonstrating Proof of Possession) Support

DAQ tokens will contain an associated DPoP proof which will be checked on the receiver side. This allows to associate an access token with a certain operation, making it impossible to replay a stolen/lost token against a different server for a different functionality.

From the user's point of view this appears as an additional optional string parameter to acquire() and verify(). The argument should be a URI like string characterizing the desired operation. It can be as simple as a generic service indicator like pmg: in which case it provides domain separation (e.g. such a token cannot be used for changing trigger keys, or run control commands). Or, it can contain the called method and arguments themselves, in which case it can only be replayed with the exact same parameters, e.g. trg://ATLAS/setPrescales/3423/2532

Furthermore the verify() function takes an optional parameter max_age which specifies the maximum allowed age of the DPoP proof. The default is 15 seconds (i.e. the token is useless after 15 seconds). Since a DPoP proof is generated by the client for each request its lifetime can be as short or long as needed (defined by the server), up to the lifetime of the access token itself

This concerns only the implementations of various CORBA services that use the access manager for authorization decisions and is transparent for everyone else.

Helper functions for credentials

#include "daq_tokens/credentials.h"

Use daq::tokens::get_credentials() to either acquire a token or just get the current user name.

Use daq::tokens::verify_credentials(creds) to check the received credentials and return the associated user name.

Both functions take an optional operation string parameter for use with DPoP proofs. The verify function takes another optional integer parameter specifying the maximum age of the DPoP proof in seconds (default is 15 seconds).

Similarly the daq.tokens.Credentials package in Java provides static get() and verify() functions with multiple overloads to create or verify credentials.

tdaq-11-04-00

Support `oidc-agent`

Add a new method (agent) to acquire a token from a running oidc-agent. To make use of that the oidc-agent-desktop package has to be installed on the system (e.g. lxplus). You have to start the agent manually if your system is not configured to do it automatically.

eval $(oicd-agent-service use)

Note: This does not work on lxplus on the moment, use

eval $(oidc-agent)

Create an entry with the name atlas-tdaq-token:

oidc-gen --flow=device --client-id atlas-tdaq-token -m --pub atlas-tdaq-token
Select CERN as issuer.
Type return for scopes
Specify a password (not your CERN password...!)

Every time you want make use of the agent, make sure it is started and then run

oidc-add atlas-tdaq-token
[enter your password]

Set the TDAQ_TOKEN_ACQUIRE environment variable to agent. Use any interactive TDAQ commands as usual.

Support local token cache

Add a new method to acquire token from a refresh token stored in a cache file: 'cache'. This is intended for interactive use or for long running jobs where a refresh token is acquired out of band and provisioned for the specific use case.

Example:

export TDAQ_TOKEN_ACQUIRE="cache browser"

If set in an interactive shell, the first time a token is acquired by any application the cache is empty. The browser will open a window to let the user authenticate as usual. The refresh token returned from this exchange is stored in the cache.

When another application requests a token, it will find the refresh token in the cache and use it to acquire an access token, transparent for the user. When the refresh token expires (at the end of the SSO session), the user will be prompted again via the browser for authentication.

Instead of browser the device method can be given if the user session is e.g. via ssh and not able to start a browser.

The refresh token can also be acquired out of band e.g. by using the sso-helper functions. In particular one can request a token with offline_access that will only expire if it hasn't been used for three months.

source ssh-helper.sh
token_from_browser atlas-tdaq-token | jq -r .refresh_token > $HOME/.cache/tdaq-sso-cache
chmod og-rwx $HOME/.cache/tdaq-sso-cache

The default cache location is $HOME/.cache/tdaq-sso-cache with a fallback to /tmp/tdaq-sso-cache-$USER if the home directory does not exist. The file must be only readable by the owner or it will be ignored. The location can also be set explicitly by the defining the TDAQ_TOKEN_CACHE variable.

All this together can be used for a long running job that needs TDAQ authentication like this:

source ssh-helper.sh
token_from_browser atlas-tdaq-token offline_access | jq -r .refresh_token > $HOME/.config/my-sso-cache
chmod og-rwx $HOME/.config/my-sso-cache

and then use it:

cm_setup
export TDAQ_TOKEN_CACHE=$HOME/.config/my-sso-cache
start-my-job

tdaq-11-03-00

Add a new method to acquire a token if the user has a kerberos ticket: 'http'. This connects to a URL (default: ${TDAQ_TOKEN_HTTP_URL:-https://vm-atdaq-token.cern.ch/token} ) which is protected by GSSAPI and will return a token if the authentication succeeds. This is intended to replace the gssapi mechanism going forward.

A PEM file can contain more than one public key. All will be processed by verify(). This allows to rotate in a new key while the old one is still valid without changing TDAQ_TOKEN_PUBLIC_KEY_URL manually - which is anyway impossible for already running processes.

dqmf

The new DQMF configuration supports run type dependent values of DQ parameter thresholds and reference objects.

Run type specific references

The DQReference class has a new attribute called ValidFor, which can be used to define a type of runs this reference object can be used. Available run types are:

PP - proton-proton physics runs
HI - heavy ion physics runs
Cosmic - comics runs
Default - any run type for which no reference is explicitly defined

One can associate multiple objects of DQReference class with the same DQ parameter, in which case DQMF will choose an appropriate one depending on the current run type. If no suitable DQReference object is defined for a valid run type, then dqm_core::BadConfig exception will be thrown and the corresponding DQ parameter will be ignored. It's therefore required to always have at least one instance of DQReference class with Default value of the ValidFor attribute associated with an DQ parameter that needs a reference. In addition one can also add specific references for PP, HI or Cosmic run types.

Run type specific references

The same run types can be used to customize values of DQ parameter thresholds using a syntax described below. A common way of defining a threshold is to provide a string that has the following format:

<threshold> ::= <threshold name> = <threshold value>
<threshold name> ::= <string>
<threshold value> :: = <floating point number>

The new DQMF still supports this syntax and will use the given value as default. In addition one can also define different values for specific run types using more general syntax supported by the new version of DQM framework.

<threshold value> ::= <run type specific value> [; <run type specific value>]
<run type specific value> ::= <run type> : <floating point number> | <floating point number>
<run type> ::= "pp" | "hi" | "cosmic" | "PP" | "HI" | "COSMIC"

More generally a threshold value may contain multiple tokens for different run types separated by semicolon. Each token either contains a run type and a floating point number separated by colon or just a floating point number, which in this case will be used as default value for all run types that don't have a specific number defined. For example:

MyThreshold=pp:0.123; hi:0.456; 0.789

This definition will ask DQMF to use the value 0.123 for proton-proton runs, the value 0.456 for heavy ion runs and the value 0.789 for any other run type. DQMF expects to get a well defined value of any threshold for all possible run types, which in practice means that a default value must always be present in every threshold definition. If this is not the case DQMF will throw dqm_core::BadConfig exception and will ignore the corresponding DQ parameter. This approach is fully backward compatible as all existing threshold definitions will be treated as providing default values that are suitable for any run type.

DVS GUI (graphical UI for DVS)

See also related DVS and TestManager packages.

dynlibs - Load shared libraries

This package is deprecated from tdaq-09-00-00 onwards.

Please use plain dlopen() or boost::dll instead. Note that unlike in this package, the boost::dll::shared_library object has to stay in scope as long as the shared library is used !

Example of possible replacement

#include <boost/dll.hpp>

double example()
{
   boost::dll::shared_library lib("libmyplugin.so",
                                  boost::dll::load_mode::type::search_system_folders |
                                  boost::dll::load_mode::type::rtld_now);
   // Get pointer to function with given signature
   auto f = lib.get<double(double, double)>("my_function")
   return f(10.2, 3.141);
}

emon

Connections between event samplers and monitors have been optimized. Existing configurations should be adjusted to benefit form that. Previously the NumberOfSamplers parameter has been used to define a number of samplers to be connected by all monitors of a group that uses the same selection criteria. In the new implementation this number defines a number of samplers that each individual monitor has to connect. That makes no difference for monitors that used to connect to a single sampler and do't form a group. For the monitors that share the same selection criteria, like for example the Global Monitoring tasks, this number should be changed to the old number divided to the number of the monitors in the group. For Athena monitoring the corresponding parameter of a JO file is called KeyCount.

IPC

TLS Support

IPC based servers and clients can use TLS to protect their communication.

The TDAQ_IPC_ENABLE_TLS environment variable can be used to turn this capability on (if value = "1") or off (if value = "0"). This is typically a global setting as part of the common environment. If enabled clients will be able to talk to servers which require TLS.

For a C++ based server to enforce TLS two environment variables should be set in its environment (in addition to the variable above):

ORBendPoint=giop:ssl::
ORBendPointPublish=giop:ssl::

An OKS VariableSet called TDAQ_IPC_TLSSERVER_Environment is available that can be linked to an Application or Binary object.

For testing the variables can be also set manually in an interactive environment.

For Java based servers the following property has to be set:

jacorb.security.ssl.server.required_options=20

There is an overhead involved in setting up an TLS connection, as well as for the encryption during the data transfer phase. Therefore it should be used only for servers that receive confidential data like authorization tokens, and it should be avoided for anything where potentially large amounts of data are exchanged, like IS servers.

In practice currently only the following applications require this:

pmgserver
run controllers
master trigger implementations
RDB writer

Note that the way TLS is used in the IPC package does not provide authentication, only confidentiality and integrity. Authentication has to be handled on the application level.

mda

tdaq-12-00-00

tag mda-07-20-00
Add mda-cli script replacing mda_annotations and mda_schema.
tag mda-07-19-01
Add type annotations for Python modules.

MTS

tdaq-11-04-00

Subscription syntax

Extended the subscription syntax allowing - ':' symbol in tokens - wildcard in the beginning of a token - spaces as part of context values, e.g. in function or file names

Current subscription syntax (quasy EBNF notation):

key = 'app' | 'msg' | 'qual'
sev = 'sev'
par = '(param|par)(+[a-zA-Z_-])'
context_item = 'app' | 'host' | 'user' | 'package' | 'file' | 'function' | 'line'
context = 'context(context_item)'
sevlev = 'fatal' | 'error' | 'warning' | 'info' | 'information'
token_wildcard = -'*'+[a-zA-Z_.:-]-'*'
token = +[a-zA-Z_.:-]
char = // any character exclusing quotes
quoted_string = '"'+char'"' | "'"+char"'"
item = (key ('=' | '!=') token_wildcard) | (sev ('=' | '!=') sevlev) | (par ('=' | '!=') token)
    | (context ('=' | '!=') quoted_string)
factor = item | ( expression ) | ('not' factor)
expression = '*' | factor *(('and' expression) | ('or' expression))

An example of subscription is

sev=ERROR or (msg=mtssender::* and (msg=*Died or msg=*::longmessageid) and param(pid)!=666 and context(line)!=1 and context(function) != 'const int daq::function')

Use new way of declaring jERS stream implementations

Follow the changes in jERS, using ErsStreamName annotation to declare a class implementing certain ERS stream:

@ErsStreamName(name="mts")
public class MTSInStream extends ers.AbstractInputStream { ...

ohp

OHP configuration has been extended to support run type dependent reference files. This modification is fully backward compatible.
Behavior of OHP has been changed if a wrong reference file is defined in the configuration. Previously such an problem was ignored. In the current release OHP prints error message and exits.

Run type specific references

A new attribute called run-type has been added to the source XML tag, which can be used to define a type of runs a particular reference file should be used for. Valid values of this attribute are:

PP - proton-proton physics runs
HI - heavy ion physics runs
Cosmic - comics runs
Default - any run type for which no reference file is explicitly defined

Note that comparison is case-insensitive, so any combination of lower and upper case letters, like for example pp or pP would be equally valid. For backward compatibility if run-type attribute is not present then Default value will be used.

One can declare the source XML tag multiple times with different values of run-type attribute, in which case OHP will choose an appropriate one depending on the current run type. If no suitable reference file is defined for the current run type, then no reference histograms will be displayed. It's therefore strongly recommended to define at least one source XML tag with Default value (or without the run-type attribute). In addition one can also declare specific reference files for proton-proton, heavy ion and Cosmic run types.

Note that it's also possible to define multiple reference files for the same run type, in which case the files will be searched for a given reference object consecutively and the first one that is found will be used.

Here is a possible example of a run type dependent reference configuration:

<general>
  <variable name="REF_PATH" value="/atlas/moncfg/"/>
</general>
<reference>
  <source type="file:" value="${REF_PATH}/default_references.root"/>
  <source type="file:" run-type="HI" value="${REF_PATH}/hi_references.root"/>
  <source type="file:" run-type="PP" value="${REF_PATH}/pp_references.root"/>
  <source type="file:" run-type="Cosmic" value="${REF_PATH}/cosmic_references.root"/>
</reference>

swrod

Memory Management

Memory management of the SW ROD fragment builders can now be configured via the MemoryPool OKS class, which in the previous releases used to be ignored. Each SwRodApplication object has been already linked with an instance of the MemoryPool, but in the new release it will be used to define the default configuration for all ROBs handled by the given SW ROD application. This configuration can be overridden for a particular ROB by linking another instance of the MemoryPool with the corresponding SwRodRob object. The meaning of the two MemoryPool attributes is the following:

PageSize - default size of individual memory pages allocated by this memory pool. Note that this value can be overridden by fragment builder implementation, which is explained by the algorithm's description in the updated User's Guide.
NumberOfPages - the number of pages that will be pre-allocated before a new run is started. The maximum number of pages that can be allocated by the memory pool is unlimited.

If one memory page is not large enough to hold the data of a particular ROB fragment the fragment builders will allocate extra pages. More information is given in the updated User's Guide.

Note that the MaxMessageSize parameter of the SwRodFragmentBuilder class has now slightly different meaning with respect to the previous SW ROD versions. Now it truly means what its name implies, i.e. it defines the maximum size of a single data packet that will be accepted by the algorithm. Packets with the sizes exceeding this limit will be discarded. Packets of a smaller size are guaranteed to be added to the ROB fragment payload without truncation.

Support of netio and netio-next

Starting from this release SW ROD doesn't support any more the legacy netio protocol and therefore cannot be used to receive data from the old felix-core systems. For receiving data from felix-star via the new netio-next protocol one must use the FelixClient interface, which can be configured via the SwRodFelixInput OKS class. The direct use of the netio-next API is also no longer supported. Because of this both the SwRodNetioInput and the SwRodNetioNextInput classes have been removed from the SW ROD OKS schema file.

transport

Clients using classes from the transport package should look into more modern network libraries like boost::asio until the C++ standard contains an official network library.

webdaq

Support an alternative API where each function has an additional CURL * argument. This allows the user complete control

webis_server

Added the C++ based webproxy executable. This implements a subset of the full webis_server executable, namely:

Read and write IS objects in the 'compact' format
Publish ROOT histograms in binary format

This is a Phase II prototype for initial measurements when used with HLT and global monitoring jobs.

tdaq-11-03-00

The JSON based API no longer HTML escapes the metadata for the object. This is unnecessary since the JSON invalid characters are different from HTML and the JSON encode does its own escaping anyway.

The one mostly affected are histograms requested as raw IS value in JSON - however, this should be a basically non-existant use case. The type names e.g. change from HistogramData<int> to HistogramData<int>.

Keys	Action
`?`	Open this help
`n`	Next page
`p`	Previous page
`s`	Search

Nightly TDAQ Release

Use of Python 2 based system commands

tdaq-11-04-00

tdaq-12-00-00

nightly

Remove automated setting of pBeast server

Control pBeast authentication method

tdaq-common-12-00-00

Add new EXTRA_EXTERNALS keyword to tdaq_project()

Add each externals path to CMAKE_PREFIX_PATH

Set CMake Policy CMP0144 to NEW

Use Boost native cmake config files instead of FindBoost.cmake

Use ROOT native cmake config files instead of FindROOT.cmake

Compressed debug sections

Initial support for C++20 modules

Enable monolithic RPM generation, make src install optional

DPoP (Demonstrating Proof of Possession) Support

Helper functions for credentials

tdaq-11-04-00

Support oidc-agent

Support local token cache

tdaq-11-03-00

Run type specific references

Run type specific references

Example of possible replacement

TLS Support

tdaq-12-00-00

tdaq-11-04-00

Subscription syntax

Use new way of declaring jERS stream implementations

Run type specific references

Memory Management

Support of netio and netio-next

tdaq-11-03-00

Add new `EXTRA_EXTERNALS` keyword to `tdaq_project()`

Add each externals path to `CMAKE_PREFIX_PATH`

Set CMake Policy CMP0144 to `NEW`

Support `oidc-agent`