Modified the bundle script to handle non-standard pytest output directories...

[bieryAtFnal]

…, both ones that are specified in the environment and ones that are specified in command line arguments.

Description

This PR is correlated with DUNE-DAQ/integrationtest#140.

The changes in this PR include ones that should have been included in the bundle script previously and ones that add new functionality. In particular,

• We should have been alert to the use of a non-standard pytest environment previously. That is remedied in this PR.
• It will likely be useful to occasionally specify a non-standard pytest output dir, and this PR includes changes to support a user-defined output directory as a command-line option to the bundle script.

The logic that is used to determine if a non-standard pytest output directory has been specified in the Linux shell environment has the following features.

• It checks to see if any of several possible env vars have been set. If any of them have been set, the first one that we find is used. If none are found, then we default to "/tmp".
  • The env vars in this list were found in the AI response to a Google search for "how to tell pytest to use a different temporary directory". The precedence that is represented in the ordering of the list was determined via local testing.
• In all cases, a user-specified command-line value for the tmpdir over-rides values found in env vars.
• This functionality is implemented in a Python function in the integrationtest repo and a Python script in this repo.

Here is a screenshot of the Google search output:

To provide consistent behavior between the bundle script and the integtest that currently checks the amount of free space on the output disk (long_window_readout_test.py), the latter was modified to make use of the new get_pytest_tmpdir function from the integtest repo.

These changes can be tested by specifying a --tmpdir command-line option to the bundle script, or by setting some combination of the following environmental variables to an existing directory on the local computer: PYTEST_DEBUG_TEMPROOT, TMPDIR, TEMP, TMP.

The following instructions may also be useful:

DATE_PREFIX=`date '+%d%b'`
TIME_SUFFIX=`date '+%H%M'`

source /cvmfs/dunedaq.opensciencegrid.org/setup_dunedaq.sh
setup_dbt latest
dbt-create -n NFD_DEV_251230_A9 ${DATE_PREFIX}FDDevTest_${TIME_SUFFIX}
cd ${DATE_PREFIX}FDDevTest_${TIME_SUFFIX}/sourcecode

git clone https://github.com/DUNE-DAQ/daqsystemtest.git -b kbiery/bundlescript_nonstandard_pytest_outdir
cd ..

dbt-workarea-env

git clone https://github.com/DUNE-DAQ/integrationtest.git -b kbiery/get_pytest_tmpdir_function
cd integrationtest; pip install -U . ; cd ..

dbt-build -j 12
dbt-workarea-env

sed -i 's,"min_size_bytes": 7272,"min_size_bytes": 7273,' sourcecode/daqsystemtest/integtest/minimal_system_quick_test.py
sed -i 's,"max_size_bytes": 14472,"max_size_bytes": 14471,' sourcecode/daqsystemtest/integtest/minimal_system_quick_test.py

daqsystemtest_integtest_bundle.sh -l0

echo ""
echo -e "\U1F535 \U2705 Note that when the previous regression test failed, a copy of the pytest output directory was created in /tmp/pytest-of-${USER}. \U2705 \U1F535"
echo ""
echo ""

daqsystemtest_integtest_bundle.sh -l0 --tmpdir=/tmp/pytest-of-${USER}

echo ""
echo -e "\U1F535 \U2705 Note that when the previous regression test failed, a copy of the pytest output directory was created in /tmp/pytest-of-${USER}/pytest-of-${USER} (two levels down). \U2705 \U1F535"
echo ""
echo ""

daqsystemtest_integtest_bundle.sh -l0 --tmpdir=/tmp/pytest-of-dunedaq

echo ""
echo -e "\U1F535 \U2705 Note that the bundle script never really got started in this latest test because it couldn't write to the requested tmpdir. \U2705 \U1F535"
echo ""

Type of change

• New feature or enhancement (non-breaking change which adds functionality)
• Optimization (non-breaking change that improves code/performance)

Testing checklist

• Unit tests pass (e.g. dbt-build --unittest)
• Minimal system quicktest passes (pytest -s minimal_system_quick_test.py)
• Full set of integration tests pass (daqsystemtest_integtest_bundle.sh)

Further checks

• Code is commented where needed, particularly in hard-to-understand areas