diff --git a/docs/datamodel/README.md b/docs/datamodel/README.md index 0bb45889..af7430b2 100644 --- a/docs/datamodel/README.md +++ b/docs/datamodel/README.md @@ -1,5 +1,5 @@ --- -sort: 9 +sort: 10 title: Appendix: data model reference --- diff --git a/docs/gettingstarted/gitbasics.md b/docs/gettingstarted/gitbasics.md index 74377da6..84243d10 100644 --- a/docs/gettingstarted/gitbasics.md +++ b/docs/gettingstarted/gitbasics.md @@ -48,11 +48,7 @@ Selected sections: - [git - the simple guide](https://rogerdudler.github.io/git-guide/) - [Oh Shit, Git!?!](https://ohshitgit.com/) -## Tips - -### Tools - -[Visual Studio Code (VS Code)](https://code.visualstudio.com/) – Integrated development environment +## Tools [Meld](https://meldmerge.org/) – Visual diff and merge tool that can be used with Git commands `git difftool` and `git mergetool` diff --git a/docs/gettingstarted/installing.md b/docs/gettingstarted/installing.md index 29fdfcb5..2da80189 100644 --- a/docs/gettingstarted/installing.md +++ b/docs/gettingstarted/installing.md @@ -25,20 +25,20 @@ ALICE uses aliBuild to build software. aliBuild: According to your operating system, please follow the prerequisites below. You will find a list of packages to install and configurations to perform. -##### Primary supported platform: +##### Primary supported platform * [CentOS 7](#prereq-for-centos7) * [CentOS/AlmaLinux 8](#prereq-for-centos8) * [AlmaLinux 9](#prereq-for-alma9) -##### Platforms supported on a best-effort basis: +##### Platforms supported on a best-effort basis * [macOS Sonoma and Sequoia (14.0, 15.0)](#prereq-for-macos) * [Ubuntu (20.04 LTS, 22.04 LTS, 24.04 LTS)](#prereq-for-ubuntu) * [Fedora](#prereq-for-fedora) * Linux Mint - * Follow the instructions for the Ubuntu version your Linux Mint version is based on. - * Specify the corresponding Ubuntu architecture when running the `aliBuild` command using the `-a` option (e.g. `-a ubuntu2004_x86-64` for Ubuntu 20.04). Use the `-a` option also with the `alienv` command. + * Follow the instructions for the Ubuntu version your Linux Mint version is based on. + * Specify the corresponding Ubuntu architecture when running the `aliBuild` command using the `-a` option (e.g. `-a ubuntu2004_x86-64` for Ubuntu 20.04). Use the `-a` option also with the `alienv` command. If your operating system is *not* in any list, it does not mean our software won't work on it; it will be just more difficult for you to get support for it. @@ -47,6 +47,7 @@ Only in case you cannot install aliBuild in the way described above, you can ins
aliBuild prerequisites for CentOS7
With root permission, i.e. `sudo` or as `root`install the prerequisits using: + ```bash cat << EOF > /etc/yum.repos.d/alice-system-deps.repo [alice-system-deps] @@ -64,6 +65,7 @@ yum install -y alibuild
aliBuild prerequisites for CentOS8
With root permission, i.e. `sudo` or as `root` install the prerequisits using: + ```bash yum install -y epel-release yum install -y dnf-plugins-core @@ -86,6 +88,7 @@ yum install -y alice-o2-full-deps alibuild
aliBuild prerequisites for AlmaLinux9
With root permission, i.e. `sudo` or as `root`install the prerequisits using: + ```bash dnf install -y epel-release dnf-plugins-core dnf update -y @@ -120,7 +123,6 @@ Xcode bundles the necessary tools to build software in the apple ecosystem inclu * Install Homebrew using the [instructions on their webpage](https://brew.sh/). * Once installed detect any problems regarding Homebrew and your system using: `brew doctor` - **Uninstall ROOT**
If you have an existing ROOT installation on your system, this will interfere with the aliBuild installation, and will cause difficult-to-debug compilation errors. @@ -138,7 +140,6 @@ Note that Homebrew does not run as root. Do not prepend sudo to any of the follo ```echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> ~/.zprofile``` * Close Terminal and reopen it to apply changes. -
aliBuild prerequisites for Ubuntu
ALICE software on Ubuntu is supported on a best effort basis. There is no guarantee that software builds or runs correctly. Support requests might have low priority. We were able to successfully build on: @@ -150,13 +151,17 @@ ALICE software on Ubuntu is supported on a best effort basis. There is no guaran **Install required system packages**
With root permissions, i.e. sudo, update your package sources: + ```bash sudo apt update -y ``` + With root permissions, i.e. `sudo`, install the following packages: + ```bash sudo apt install -y curl libcurl4-gnutls-dev build-essential gfortran libmysqlclient-dev xorg-dev libglu1-mesa-dev libfftw3-dev libxml2-dev git unzip autoconf automake autopoint texinfo gettext libtool libtool-bin pkg-config bison flex libperl-dev libbz2-dev swig liblzma-dev libnanomsg-dev rsync lsb-release environment-modules libglfw3-dev libtbb-dev python3-dev python3-venv python3-pip graphviz libncurses-dev software-properties-common gtk-doc-tools ``` + **Installing aliBuild**
AliBuild, our build tool, is installed as a standard ubuntu package, provided you enable the alisw PPA repository. This is done with: @@ -170,6 +175,7 @@ sudo apt install python3-alibuild
aliBuild prerequisites for Fedora
With root permissions, i.e. `sudo` or as `root` install the prerequisites using: + ```bash yum install -y epel-release yum install -y dnf-plugins-core @@ -192,10 +198,12 @@ yum install -y alice-o2-full-deps alibuild ### Configure aliBuild After you are done installing alibuild you need to configure it by adding the two following lines to your `~/.bashrc`, `~/.bash_profile`, `~/.zshrc` or `~/.zprofile` (depending on your operating system and configuration): + ```bash export ALIBUILD_WORK_DIR="$HOME/alice/sw" eval "$(alienv shell-helper)" ``` + The first line tells what directory is used as "build cache", the second line installs a "shell helper" that makes easier to run certain aliBuild-related commands. You need to close and reopen your terminal for the change to be effective. The directory `~/alice/sw` will be created the first time you run aliBuild. @@ -204,6 +212,8 @@ Note that this directory tends to grow in size over time, and it is the one you When `aliBuild`is installed on your computer and your prerequisits are statisfied, you can move to the next step. +See also [shell rc utilities](../tools/README.md#shell-rc-utilities). + ## Prepare your source code We assume your work area is `~/alice`. @@ -318,6 +328,8 @@ For example: `ninja PWGCF/Tasks/install` ninja install > ../log 2>&1 && echo "Good" || echo "Bad" ``` +See also [shell rc utilities](../tools/README.md#shell-rc-utilities). + A specific executable can be built in the staging directory `stage/bin` with ```bash diff --git a/docs/tools/README.md b/docs/tools/README.md new file mode 100644 index 00000000..08810b62 --- /dev/null +++ b/docs/tools/README.md @@ -0,0 +1,137 @@ +--- +sort: 8 +title: Tools +--- + +# Tools + +Tools which are not part of the O2(Physics) analysis framework and can make your work with it much more effective. + +## Setup diagnostic tool + +This simple Bash script prints a summary of some basic information about your system and your O2Physics installation setup. + +It can be useful to provide this summary when [requesting support](../troubleshooting/README.md#reporting-problems) concerning a problem with your analysis framework. + +Download the [`summarise_o2p_setup.sh`](summarise_o2p_setup.sh) script and run it with `bash` in your `alice` directory. + +## Dependency finder + +Tool to explore input/output dependencies between O2Physics workflows and tables. + +See the dedicated [Dependency finder](dependencyFinder.md) page. + +## O2 linter + +Tool to detect O2-specific (and some common C++) issues in the code. + +See the dedicated [O2 linter](o2linter.md) page. + +## Pre-commit hooks + +[Git hooks](https://git-scm.com/book/ms/v2/Customizing-Git-Git-Hooks) are scripts hooked to certain Git commands. +Pre-commit hooks run when you execute `git commit` and are used to validate the staged changes that are about to be committed. +If any hook fails, the commit is not made. + +In the O2Physics repository, pre-commit hooks are [configured](https://github.com/AliceO2Group/O2Physics/blob/master/.pre-commit-config.yaml) to format C++ code with [clang-format](https://clang.llvm.org/docs/ClangFormat.html) and lint it (check for issues) with [cpplint](https://github.com/cpplint/cpplint). +This way you can make sure that your changes will pass the C++ formatting check and the cpplint check in [MegaLinter](https://megalinter.io/) when you make your pull request. + +### How to use pre-commit hooks + +1. [Install pre-commit](https://pre-commit.com/#installation) (typically `pip install pre-commit`). +1. Enter the `O2Physics` directory. +1. [Install the Git hook scripts](https://pre-commit.com/#3-install-the-git-hook-scripts): `pre-commit install`. + +Next time you execute `git commit`, the hooks will run automatically. + +- If the clang-format hook fails, it means your staged files have been formatted. +- If the cpplint hook fails, the linter has found issues that need to be fixed. +- Updated files need to be staged with `git add` before attempting `git commit` again. + +## [clang-tidy](https://clang.llvm.org/extra/clang-tidy/) + +`clang-tidy` is a clang-based C++ linter tool for diagnosing and fixing typical programming errors, like style violations or bugs. +(See the [list of checks](https://clang.llvm.org/extra/clang-tidy/checks/list.html).) + +### Prerequisites for using clang-tidy + +- You need to have O2Physics successfully compiled. +- Verify that there is a valid symbolic link `compile_commands.json` in the O2Physics directory pointing to the `alice/sw/BUILD/.../O2Physics` directory. + +### Tips + +#### Cleaning `#include`s + +The [`misc-include-cleaner`](https://clang.llvm.org/extra/clang-tidy/checks/misc/include-cleaner.html) check can fix missing and unused `#include`s. +This helps to apply the [Include What You Use](https://github.com/AliceO2Group/O2Physics/issues/8357) principle which allows to maintain header dependencies clean. + +#### Sorting `#include`s + +The [`llvm-include-order`](https://clang.llvm.org/extra/clang-tidy/checks/llvm/include-order.html) check sorts `#include`s from most specific to least specific to ensure that the headers does not have any hidden dependencies. + +Unfortunately, the [LLVM include style](https://llvm.org/docs/CodingStandards.html#include-style) is incompatible with the [Google include style](https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes) applied by `cpplint`. + +#### Testing (and fixing) many files at once + +Here is an example of how to run the `misc-include-cleaner` check in parallel on all `.h` and `.cxx` files in the current directory. + +```bash +parallel "clang-tidy --fix -checks=-*,misc-include-cleaner {}; echo \"{} \$?\"" ::: $(find -name "*.h" -o -name "*.cxx") > "clang-tidy.log" +``` + +The [`parallel`](https://www.gnu.org/software/parallel/) command is used to parallelise the execution of the `clang-tidy` command for all files. + +For each file, `clang-tidy` will first try to compile it and then run the enabled check(s) and fix found problems (the `--fix` option). +The messages are redirected into `clang-tidy.log`. +The file name and the exit code are printed below the output of `clang-tidy` so that you can get the list of files for which `clang-tidy` failed with `grep " 1$" "clang-tidy.log"`. + +## [Visual Studio Code (VS Code)](https://code.visualstudio.com/) + +Integrated development environment + +See [how to run O2 linter from VS Code](o2linter.md#in-visual-studio-code) + +### Useful extensions + +- [clangd](https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd) - C/C++ completion, navigation, and insights +- [json](https://marketplace.visualstudio.com/items?itemName=ZainChen.json) - Json for Visual Studio Code +- [Markdown All in One](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one) - All you need to write Markdown (keyboard shortcuts, table of contents, auto preview and more) +- [Python](https://marketplace.visualstudio.com/items?itemName=ms-python.python) - Python language support with extension access points for IntelliSense (Pylance), Debugging (Python Debugger), linting, formatting, refactoring, unit tests, and more. +- [Ruff](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff) - A Visual Studio Code extension with support for the Ruff linter and formatter. +- [ShellCheck](https://marketplace.visualstudio.com/items?itemName=timonwong.shellcheck) - Integrates ShellCheck into VS Code, a linter for Shell scripts. +- [YAML](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) - YAML Language Support by Red Hat, with built-in Kubernetes syntax support +- [Remote - SSH](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh) - Open any folder on a remote machine using SSH and take advantage of VS Code's full feature set. +- [PDF Viewer](https://marketplace.visualstudio.com/items?itemName=mathematic.vscode-pdf) - Portable document format (PDF) viewer for Visual Studio Code. + +## Shell rc utilities + +You can use the [`bashrc-alice.sh`](bashrc-alice.sh) file to add ALICE-related utilities in your Bash environment. + +The file provides: + +- variables needed to [configure aliBuild](../gettingstarted/installing.md#configure-alibuild) +- utility functions for [recompiling with `ninja`](../gettingstarted/installing.md#building-partially-for-development-using-ninja) +- utility functions for [debugging compilation and runtime failures](../troubleshooting/README.md#finding-problems) + +Add the [`bashrc-alice.sh`](bashrc-alice.sh) file in your home directory (`~`) and source it in your Bash environment by adding the following line in the `~/.bashrc` file. + +```bash +source bashrc-alice.sh +``` + +## [Run 3 validation framework](https://github.com/AliceO2Group/Run3AnalysisValidation) + +The Run 3 validation framework is a tool for an easy execution, testing and validation of any O2Physics analysis code on large local data samples. + +It is extensively configurable and provides similar features as AliHyperloop, such as: + +- Dataset management +- Support of linked-derived-data input +- Automatic JSON configuration based on input properties +- Automatic workflow topology generation based on "wagon" dependencies +- Job parallelisation +- Table saving +- Output merging +- Diagnostics + +Among several other utilities, it includes a [maintenance script](https://github.com/AliceO2Group/Run3AnalysisValidation?tab=readme-ov-file#keep-your-repositories-and-installations-up-to-date-and-clean) for automated maintenance of Git repositories and aliBuild packages. diff --git a/docs/tools/bashrc-alice.sh b/docs/tools/bashrc-alice.sh new file mode 100644 index 00000000..1d33e6b1 --- /dev/null +++ b/docs/tools/bashrc-alice.sh @@ -0,0 +1,86 @@ +#!/bin/bash + +# alice + +# aliBuild +export ALIBUILD_WORK_DIR="$HOME/alice/sw" +eval "$(alienv shell-helper)" + +# aliases +alias loadali='alienv enter AliPhysics/latest' +alias loado2='alienv enter O2/latest' +alias loado2p='alienv enter O2Physics/latest' +alias root='root -l' + +# Recompile an aliBuild development package with ninja. +# Usage: recompile +# Arguments and are optional. +# Command "recompile O2Physics" will invoke "ninja install" for the "master" branch of O2Physics. +# Command "recompile O2Physics my-branch" will invoke "ninja install" for the "my-branch" branch of O2Physics. +# Command "recompile O2Physics my-branch Common" will invoke "ninja Common/install" for the "my-branch" branch of O2Physics. +recompile() { + # set -o xtrace # to print out each command + [ "$1" ] || { echo "Provide a package name"; return 1; } + package="$1" + branch="master" + [ "$2" ] && branch="$2" + target="" + target_name="all" + [ "$3" ] && { target="$3/"; target_name="$3"; } + dir_pwd=$(pwd) + dir_build="$ALIBUILD_WORK_DIR/BUILD/${package}-latest-${branch}/${package}" + log="$(dirname "$dir_build")/log" + log_err="$(dirname "$dir_build")/log_err" + cd "$dir_build" || { echo "Could not enter $dir_build"; return 1; } + direnv allow || { echo "Failed to allow direnv"; return 1; } + eval "$(direnv export "$SHELL")" + echo "Recompiling ${package}_${branch}_${target_name}..." + start=$(date +%s) + ninja "${target}install" > "$log" 2>&1 + ec=$? + end=$(date +%s) + echo "Compilation exited with: $ec" + echo "See the log at: $log" + if [ "$ec" != "0" ]; then + grep -e "FAILED:" -e "error:" "$log" > "$log_err" + echo "See the errors at: $log_err" + fi + echo "Took $((end - start)) seconds." + cd "$dir_pwd" || return 1 + # set +o xtrace + return $ec +} + +# Recompile O2 with ninja. +recompile-o2() { recompile "O2" "$1" "$2"; } +# Recompile O2Physics with ninja. +recompile-o2p() { recompile "O2Physics" "$1" "$2"; } + +# Find the workflow that produces a given table. +# Limited functionality. Use find_dependencies.py for a full search. +find-o2-table-producer() { + # Check that we are inside the O2 or the O2Physics directory. + [[ "$PWD/" != *"/O2"*"/"* ]] && { echo "You must be inside the O2 or the O2Physics directory."; return 1; } + [ ! "$1" ] && { echo "Provide a table name."; return 1; } + # Find files that produce the table. + table="$1" + echo "Table $table is produced in:" + files=$(grep -r -i --include="*.cxx" "" | grep -E 'Produces|Spawns' | cut -d: -f1 | sort -u) + for f in $files; do + # Extract the workflow name from the CMakeLists.txt in the same directory. + wf=$(grep -B 1 "$(basename "$f")" "$(dirname "$f")/CMakeLists.txt" | head -n 1 | cut -d\( -f2) + echo "$wf in $f" + done +} + +# Find compilation error messages in a compilation log. +debug-o2-compile() { + [ "$1" ] || { echo "Provide a log file"; return 1; } + grep -n -e "FAILED:" -e "error:" -e "warning:" "$1" +} + +# Find runtime error messages in an execution log. +debug-o2-run() { + [ "$1" ] || { echo "Provide a log file"; return 1; } + grep -n -e "\\[ERROR\\]" -e "\\[FATAL\\]" -e "segmentation" -e "Segmentation" -e "SEGMENTATION" -e "command not found" -e "Program crashed" -e "Error:" -e "Error in " -e "\\[WARN\\]" -e "Warning in " "$1" +} diff --git a/docs/tools/dependencyFinder.md b/docs/tools/dependencyFinder.md new file mode 100644 index 00000000..650cefbe --- /dev/null +++ b/docs/tools/dependencyFinder.md @@ -0,0 +1,189 @@ +--- +sort: 1 +title: Dependency finder +--- + +# Dependency finder + +Dependency finder is a tool to explore input/output dependencies between O2Physics workflows and tables. + +## Usage overview + +The [`find_dependencies.py`](https://github.com/AliceO2Group/O2Physics/blob/master/Scripts/find_dependencies.py) script is available in the O2Physics repository and must be executed inside the O2Physics environment: + +```bash +$O2PHYSICS_ROOT/share/scripts/find_dependencies.py +``` + +The full list of options is displayed when the option `-h` is provided: + +```text +usage: find_dependencies.py [-h] [-t TABLE [TABLE ...]] [-w WORKFLOW [WORKFLOW ...]] [-T TABLE_REV [TABLE_REV ...]] [-W WORKFLOW_REV [WORKFLOW_REV ...]] [-c] [-g {pdf,svg,png}] + [-x EXCLUDE [EXCLUDE ...]] [-l LEVELS] + +Find dependencies required to produce a given table or to run a given workflow. + +optional arguments: + -h, --help show this help message and exit + -t TABLE [TABLE ...] table(s) for normal (backward) search (i.e. find producers) + -w WORKFLOW [WORKFLOW ...] + workflow(s) for normal (backward) search (i.e. find inputs) + -T TABLE_REV [TABLE_REV ...] + table(s) for reverse (forward) search (i.e. find consumers) + -W WORKFLOW_REV [WORKFLOW_REV ...] + workflow(s) for reverse (forward) search (i.e. find outputs) + -c be case-sensitive with table names + -g {pdf,svg,png} make a topology graph in a given format + -x EXCLUDE [EXCLUDE ...] + tables and workflows to exclude + -l LEVELS maximum number of workflow tree levels (default = 0, include all if < 0) +``` + +## Modes + +Modes define the direction of search through the dependency tree. + +Supported options are `-t`, `-w`, `-T`, `-W`. +Options can be used together and each option takes an arbitrary number of arguments. + +### Backward mode + +The backward mode searches for **parents** of the given object in the dependency tree. + +#### Table producers (`-t`) + +Gives a list of workflows that produce a given table. + +Example: `-t BC_001` + +```text +Table: BC_001 + +BC_001 <- ['o2-analysis-bc-converter'] +``` + +Examples of use: + +- Find a helper task (e.g. a converter) that produces a missing input table. +- Find a producer workflow to inspect how a table is filled. + +#### Workflow inputs (subscriptions) (`-w`) + +Gives a list of tables consumed by a given workflow. + +Example: `-w o2-analysis-bc-converter` + +```text +Workflow: o2-analysis-bc-converter + +o2-analysis-bc-converter <- ['BC'] +``` + +Examples of use: + +- Find dependencies of a Hyperloop wagon (i.e. the list of directly required workflows) (with the `-l 1` option). +- Resolve unclear table name aliases in subscriptions. + +### Forward mode + +The forward mode searches for **children** of the given object in the dependency tree. + +#### Table consumers (`-T`) + +Gives a list of workflows that consume a given table. + +Example: `-T BC` + +```text +Table: BC + +BC -> ['o2-analysis-bc-converter'] +``` + +Examples of use: + +- Find workflows affected by the modification of a table. +- Find a workflow which consumes a table that is not supposed to be required. + +#### Workflow outputs (`-W`) + +Gives a list of tables produced by a given workflow. + +Example: `-W o2-analysis-bc-converter` + +```text +Workflow: o2-analysis-bc-converter + +o2-analysis-bc-converter -> ['BC_001'] +``` + +Examples of use: + +- List all output tables of a workflow that contains multiple structs. +- See the table descriptions of tables whose description names are different from their type names. +- See the table descriptions of tables table whose description names are generated in macros. + +## Dependency levels (`-l`) + +If provided with an integer number, dependencies are searched recursively up to the provided number of workflow levels. +The levels are indicated with indentation in the output. + +If the provided number is negative, all levels are considered. + +If not provided, only direct dependencies (`-l 0`) are considered. + +Example: `-t timestamps -l 1` + +```text +Table: timestamps + +timestamps <- ['o2-analysis-timestamp'] + +Workflow dependency tree: + +o2-analysis-timestamp <- ['BC_001'] + BC_001 <- ['o2-analysis-bc-converter'] + o2-analysis-bc-converter <- ['BC'] +``` + +## Exclude (`-x`) + +Workflows and tables can be excluded from the search using the `-x` option. + +## Graphical output (`-g`) + +The dependency tree can be visualised in a graph. (Requires [Graphviz](https://graphviz.org/) installed.) + +Example: `-w o2-analysis-event-selection -l 1 -x o2-analysis-onthefly-tracker o2-analysis-track-propagation-tester -g png` + +```text +Workflow: o2-analysis-event-selection + +o2-analysis-event-selection <- ['BC_001', 'FDD_001', 'FT0', 'FV0A', 'FV0C', 'RUN2BCINFO_001', 'TIMESTAMPS', 'ZDC_001', 'BCSEL', 'COLLISION_001', 'TRACK', 'TRACKEXTRA_002', 'TRACK_IU'] + BC_001 <- ['o2-analysis-bc-converter'] + o2-analysis-bc-converter <- ['BC'] + FDD_001 <- ['o2-analysis-fdd-converter'] + o2-analysis-fdd-converter <- ['FDD'] + RUN2BCINFO_001 <- ['o2-analysis-run2bcinfos-converter'] + o2-analysis-run2bcinfos-converter <- ['RUN2BCINFO'] + TIMESTAMPS <- ['o2-analysis-timestamp'] + o2-analysis-timestamp <- ['BC_001'] + ZDC_001 <- ['o2-analysis-zdc-converter'] + o2-analysis-zdc-converter <- ['BC_001', 'ZDC'] + BCSEL <- ['o2-analysis-event-selection'] + COLLISION_001 <- ['o2-analysis-collision-converter'] + o2-analysis-collision-converter <- ['COLLISION'] + TRACK <- ['o2-analysis-track-propagation'] + o2-analysis-track-propagation <- ['BC_001', 'COLLISION_001', 'MCPARTICLE_001', 'MCTRACKLABEL', 'TIMESTAMPS', 'TRACKCOV_IU', 'TRACKEXTRA_002', 'TRACK_IU'] + TRACKEXTRA_002 <- ['o2-analysis-tracks-extra-v002-converter'] + o2-analysis-tracks-extra-v002-converter <- ['TRACKEXTRA', 'TRACKEXTRA_001', 'TRACKEXTRA_002'] + +Making dot file in: o2-analysis-event-selection.gv +Making graph in: o2-analysis-event-selection.png +``` + +The output graph `o2-analysis-event-selection.png` is shown below: + +
+o2-analysis-event-selection +
diff --git a/docs/tools/o2-analysis-event-selection.png b/docs/tools/o2-analysis-event-selection.png new file mode 100644 index 00000000..b0840f4b Binary files /dev/null and b/docs/tools/o2-analysis-event-selection.png differ diff --git a/docs/tools/o2linter.md b/docs/tools/o2linter.md new file mode 100644 index 00000000..ef9f93e9 --- /dev/null +++ b/docs/tools/o2linter.md @@ -0,0 +1,92 @@ +--- +sort: 2 +title: O2 linter +--- + +# O2 linter + +O2 linter is a static-analysis tool that detects O2-specific (and some common C++) issues in the code. + +Categories of tests: + +- Coding practice +- Mandatory documentation +- Naming conventions +- PWG-specific conventions + +O2 linter runs tests on provided files and reports files and lines which failed, together with an error message for a given test and the test name in square brackets. + +A summary breakdown of the numbers of found issues per test is presented at the end in a table. +For each test, there is a brief rationale and a list of references to clarify the motivation behind the rule. + +## How to run O2 linter + +The [`o2_linter.py`](https://github.com/AliceO2Group/O2Physics/blob/master/Scripts/) script is avaiable in the O2Physics repository and can be used in several ways. + +### On the command line + +You can execute the script locally from the O2Physics directory with: + +```bash +python3 Scripts/o2_linter.py +``` + +Example: `python3 Scripts/o2_linter.py $(find . -path ./.git -prune -o -type f)` to lint all O2Physics files. + +### In Visual Studio Code + +The linter can be executed from Visual Studio Code editor as [the default test task](https://code.visualstudio.com/docs/debugtest/testing#_task-integration) to lint the currently opened file. +The linter output can be parsed by the `problemMatcher` to highlight issues directly in the code editor and list them in the problem panel. + +You can use [`tasks.json`](tasks.json) to add the configuration in `O2Physics/.vscode/tasks.json`. + +It may be convenient to [define a keyboard shortcut](https://code.visualstudio.com/docs/configure/keybindings) for running the default test task. + +### On GitHub push events + +You can let GitHub run the O2 linter automatically in your fork O2Physics repository whenever you push commits. + +In your fork O2Physics repository go to: "Settings -> Actions -> General -> Actions permissions" and enable the "Allow all actions and reusable workflows" option. + +When you push to your remote branch, O2 linter will lint the files modified compared to the remote master branch in your fork. +(If you want to lint only your commits, make sure your remote master branch is up to date.) + +The output can be found in the log of the GitHub action. + +### In GitHub pull requests + +O2 linter runs automatically on pull requests and lints files modified by the PR (compared to the target branch). + +The total numbers of errors, warnings, and disabled issues are reported for convenience as an auto-updating comment in the PR, while the detailed breakdown per test remains available in the log of the GitHub action. + +The first ten errors also appear as annotations directly in the source code. + +## Customisation + +Tests can be disabled per line and categories of issues can be tolerated per directory. + +### Disable a test for a line + +Disabling tests per line should be reserved only for false positives and special exceptions. + +One can disable a test for a line by adding a comment with `o2-linter: disable=` followed by the name of the test and parentheses with a reason for the exception. + +### Tolerate a category of issues in a directory + +Found issues can be treated as errors (critical issues) or warnings (tolerated issues). Both levels are reported in the log but only errors make the linter fail and show up in the PR as GitHub annotations. + +To tolerate issues for a given test in a directory, create a file `o2linter_config` in the directory and add a line with the name of the test in it. + +Please keep in mind that non-critical issues are still genuine issues and fixing them improves the code for everybody. + +## References + +- [C++ Core Guidelines](https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines) +- [LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html) +- [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html) +- [The Zen of Python](https://peps.python.org/pep-0020/) +- [Style Guide for Python Code](https://peps.python.org/pep-0008/) +- [ALICE O2 Coding Guidelines](https://github.com/AliceO2Group/CodingGuidelines) +- [PWG-HF guidelines](../advanced-specifics/pwghf.md#contribute) +- [Proposal of the O2 linter](https://indico.cern.ch/event/1482467/#29-development-of-the-o2-linte) +- [Update of the O2 linter](https://indico.cern.ch/event/1513748/#29-o2-linter-development) diff --git a/docs/troubleshooting/summarise_o2p_setup.sh b/docs/tools/summarise_o2p_setup.sh similarity index 100% rename from docs/troubleshooting/summarise_o2p_setup.sh rename to docs/tools/summarise_o2p_setup.sh diff --git a/docs/tools/tasks.json b/docs/tools/tasks.json new file mode 100644 index 00000000..d2e0b0d7 --- /dev/null +++ b/docs/tools/tasks.json @@ -0,0 +1,36 @@ +{ + "version": "2.0.0", + "tasks": [ + { + "label": "Run O2 Linter", + "type": "process", + "command": "python3", + "args": [ + "${workspaceFolder}/Scripts/o2_linter.py", + "${file}" + ], + "group": { + "kind": "test", + "isDefault": true + }, + "problemMatcher": { + "owner": "python", + "source": "O2 linter", + "fileLocation": "absolute", + "pattern": { + "regexp": "^(.*):(\\d+):\\s+(info|warning|error):\\s+(.*)\\s+\\[(.*)\\]$", + "file": 1, + "line": 2, + "severity": 3, + "message": 4, + "code": 5 + } + }, + "presentation": { + "revealProblems": "onProblem", + "panel": "dedicated" + }, + "detail": "Runs O2 linter on the current file." + } + ] +} diff --git a/docs/troubleshooting/README.md b/docs/troubleshooting/README.md index a762c0de..ff57e8c2 100644 --- a/docs/troubleshooting/README.md +++ b/docs/troubleshooting/README.md @@ -1,5 +1,5 @@ --- -sort: 8 +sort: 9 title: Troubleshooting --- @@ -21,6 +21,8 @@ grep -e "FAILED:" -e "error:" -e "warning:" ../log | cut -d : -f -3 If you want to see the full messages, remove the `| cut...` part. +See also [shell rc utilities](../tools/README.md#shell-rc-utilities). + ### Runtime problems If your O2 code is crashing and you have difficulties finding the reason, try the following tips. @@ -37,6 +39,8 @@ Search for keywords related to problems in the log file using the `grep` command grep -e "\\[ERROR\\]" -e "\\[FATAL\\]" -e "segmentation" -e "Segmentation" -e "SEGMENTATION" -e "command not found" -e "Program crashed" -e "Error:" -e "Error in " -e "\\[WARN\\]" -e "Warning in " stdout.log ``` +See also [shell rc utilities](../tools/README.md#shell-rc-utilities). + Try to understand and resolve the first reported problem. ```note @@ -48,7 +52,7 @@ You can inspect a failing [AliHyperloop test](../hyperloop/userdocumentation.md# See the [Support](../gettingstarted/support.md) section for the list of Mattermost channels where you can ask for help. When asking for help with compilation or with running a local analysis, it may be useful to provide basic information about your O2Physics setup. -You can generate it by running the [`summarise_o2p_setup.sh`](summarise_o2p_setup.sh) bash script in your `alice` directory. +You can generate it using the [Setup diagnostic tool](../tools/README.md#setup-diagnostic-tool). When posting on Mattermost: