PHPC-2261: Revamp Evergreen config (#1473)

* Remove unused functions and tasks

* Fix indentation

* POC: Use task group to test local deployments

* Add make target to test without building

This is necessary if we don't want to change all `.dep` files to the new path.

* Cache compiled build for later use

* Rename tasks for better filtering

* Remove working dir upload as it's already uploaded along with the build

* Update tags in test tasks

* Create build variants for building driver

* Fetch correct build file

* Extract configuration to separate files

* Add task group for builds

* Add builds for different libmongoc versions

* Test MongoDB 7.0 on Debian 11

* Move Atlas Connectivity and crypt_shared tests to new format

* Move storage engine and libmongoc variants to new format

* Run smoketest after build

* Extract generatable tasks

* Remove obsolete file

* Remove obsolete variant

* Move requireApiVersion tests to new format

* Move loadbalanced tests to new format

* Move OCSP tests to new format

* Generate test and build tasks from templates

* Remove obsolete files

* Refactor generation script

* Remove version identifier from libmongoc task names

* Add variant tags to facilitate patch filters

* Define pull request aliases in evergreen config

* Add architecture documentation

* Use regular expression for PR aliases

* Mark generated files as such

* Remove unnecessary functions

- Artifactory uploads are not used, so we can save on disk space
- Evergreen isn't used to test Windows builds, so we don't need to make Windows fixes
- Files should already be executable

* Update CI documentation

* Remove unused YAML tags

* Document references of libmongoc build tasks

* Document tags used in tasks

* Make versions in config generator more readable

* Split templates and generated configs

* Add blank lines to task files for readability

* Add note about libmongoc build tasks in contributing guide

* Use real server version in architecture docs

* Treat loadbalanced as its own topology in tests

* Kill tasks after 10 minutes

Author: alcaeus

.evergreen/architecture.md

@@ -0,0 +1,64 @@
+# Evergreen Build System Architecture
+
+To avoid unnecessary tasks from being started, this project uses two separate build steps. The first step compiles the
+PHP extension on the requested operating system and PHP version, caching the result. The second step runs tests on
+various platforms, re-using the cached build artifacts from the first step.
+
+## Build Step 1: Compile PHP Extension
+
+The extension is built for all supported PHP versions on all supported platforms. This build step is run for every
+commit pushed to the development branch, as well as for stable branches and pull requests to these branches. The build
+step includes a smoke test that attempts to load the compiled extension to catch errors that occur during extension
+initialisation. If a build step fails, test tasks are skipped for that platform and PHP version.
+
+### Build tasks
+
+Build tasks are generated automatically and included in the main config file. The `build-variants.yml` file contains
+the list of supported platforms that the extension is built for. `build-task-groups.yml` defines the task groups that
+contain the build tasks. Note there is a separate task that skips PHP 7.4 and 8.0, as these versions do not support
+OpenSSL 3 (currently used on RHEL 9+ and Ubuntu 22.04+).
+
+## Build Step 2: Run Tests
+
+Tests are not offered for all platforms, and only select tasks are run for commits and pull requests. First, all
+combinations of MongoDB topologies (standalone, replica set, sharded cluster, and load balanced) are run for all
+supported MongoDB versions. Then, all PHP versions are tested against the latest stable version of MongoDB. Finally,
+a last build variant runs tests on PHP 8.2 against the latest stable version of MongoDB using different libmongoc
+versions.
+
+## Modifying generated tasks
+
+Most build and test tasks are generated using the `generate-config.php` script and referenced using tags. To modify the
+configuration, change the appropriate `_template-<build>.yml` file in the `build` or `test` directory, then run the
+generator script to update the files. The generator script will output a list of include statements that can be copied
+to the main `config.yml` file to include all generated tasks.
+
+## Adding new tasks
+
+Adding new tasks for a new MongoDB version or PHP version is done by modifying `generate-config.php` and including the
+version in the corresponding arrays, then regenerating configuration. Note that obsolete files will not be deleted
+automatically, but they won't be in the list of include files printed by the generator script.
+
+## Patch Aliases
+
+Aliases are configured in the main Evergreen configuration file. `github_pr_aliases` defines aliases for tasks to be run
+for every pull request.
+
+## Task Tags
+
+Tasks are tagged to allow for a better selection in build variants. The following tags are used for build tasks:
+- `build`: All build tasks are tagged with `build`.
+- `build-libmongoc`: These build tasks build with different libmongoc versions.
+- `php<version>`: These tags allow selection based on PHP version, e.g. `php8.2`.
+
+Test tasks use the following tags:
+- `local`: All tasks that run a local MongoDB cluster for testing.
+- `<version>`: These tags allow selection based on MongoDB version, e.g. `7.0`.
+- `standalone`, `replicaset`, `sharded`: These tags allow selection based on the MongoDB topology.
+- `loadbalanced`: Allows for selecting tests using a load balancer
+- `ocsp`: Used for all OCSP tasks
+- `versioned_api`: Used for tests that use the stable API
+- `skip_crypt_shared`: These tasks skip installing the shared library to test with libmongocrypt
+- `nodb`: These tasks do not rely on a local database
+- `atlas`: These tasks work on a MongoDB Atlas cluster
+- `storage-engines`: Tag used for tasks that test the `inmemory` and `mmapv1` storage engines

.evergreen/compile-unix.sh

@@ -1,6 +1,24 @@
-#!/bin/sh
+#!/bin/bash
 set -o errexit  # Exit the script with error if any of the commands fail
 
+# Find PHP binary path for the requested version
+if [ -z "$PHP_PATH" ]; then
+  if [ -d "/opt/php/${PHP_VERSION}-64bit/bin" ]; then
+     PHP_PATH="/opt/php/${PHP_VERSION}-64bit/bin"
+  else
+     # Try to find the newest version matching our constant
+     PHP_PATH=`find /opt/php/ -maxdepth 1 -type d -name "${PHP_VERSION}*-64bit" -print | sort -V -r | head -n 1`/bin
+  fi
+fi
+
+if [ ! -x "$PHP_PATH/php" ]; then
+   echo "Could not find PHP binaries for version ${PHP_VERSION}. Listing available versions..."
+   ls -1 /opt/php
+   exit 1
+fi
+
+PATH="$PHP_PATH:$PATH"
+
 # Supported/used environment variables:
 #   MARCH             Machine Architecture. Defaults to lowercase uname -m
 #   LIBMONGOC_VERSION Optional libmongoc version (regenerate version file if set)
@@ -45,7 +63,7 @@ case "$OS" in
 esac
 
 # Report the current PHP version
-echo "PHP: `php --version | head -1`"
+echo "PHP: `php --version | head -n 1`"
 
 phpize
 ./configure --enable-mongodb-developer-flags
@@ -55,4 +73,4 @@ if [ -n "$LIBMONGOC_VERSION" ]; then
    make libmongoc-version-current
 fi
 
-make
+make test TESTS="tests/smoketest.phpt"