CI: enable parallel testing in CI builds

The test-ci target now uses 2 processes by default, but the amount of
parallelism is tuned for each CI service and build environment based on
results of a number of test runs.  Some CI services use super-
oversubscribed build machines that can barely run the curl tests
already with no parallelism without frequently failing with
timing-induced failures. These continue to be run without parallelism.
Other services provide two fast, unloaded cores and these run with 14
processes, which is a good default for this kind of environment.

Here's a summary of the number of test processes by CI service:

  Appveyor - 2 (Windows MSVC), 1 (others)
  Azure - 2
  Circle CI - 14
  Cirrus - 28 (macOS), 14 (Linux), 7 (FreeBSD), 5 (macOS torture), 2 (Windows)
  GitHub Actions - 3 (macOS), 2 (Linux)

Some of these are a bit conservative to keep timing-induced flakiness down.

The net result is that the first test results should arrive only
3 minutes after a commit submission.

Changes merged via separate commits:
- 2a7c8b27fd #14171
- 72341068a2
- efce544418 #14244
- c6cf411bac

Ref: #10818
Closes #11510
This commit is contained in:
Dan Fandrich 2023-06-22 10:03:16 -07:00 committed by Viktor Szakats
parent fadb2ee6ec
commit 0324d557e4
No known key found for this signature in database
GPG Key ID: B5ABD165E2AEF201
3 changed files with 3 additions and 2 deletions

View File

@ -137,6 +137,7 @@ if [[ "${TFLAGS}" != 'skipall' ]] && \
elif [ -x "$(cygpath 'C:/msys64/usr/bin/curl.exe')" ]; then
TFLAGS+=" -ac $(cygpath 'C:/msys64/usr/bin/curl.exe')"
fi
TFLAGS+=' -j0'
if [ "${BUILD_SYSTEM}" = 'CMake' ]; then
cmake --build _bld --config "${PRJ_CFG}" --target test-ci
else

View File

@ -88,6 +88,6 @@ add_runtests(test-am "-a -am")
add_runtests(test-full "-a -p -r")
# ~flaky means that it'll ignore results of tests using the flaky keyword
add_runtests(test-nonflaky "-a -p ~flaky ~timing-dependent")
add_runtests(test-ci "-a -p ~flaky ~timing-dependent -r -rm")
add_runtests(test-ci "-a -p ~flaky ~timing-dependent -r -rm -j2")
add_runtests(test-torture "-a -t")
add_runtests(test-event "-a -e")

View File

@ -133,7 +133,7 @@ TEST_E = -a -e
TEST_NF = -a -p ~flaky ~timing-dependent
# special CI target derived from nonflaky with CI-specific flags
TEST_CI = $(TEST_NF) -r -rm
TEST_CI = $(TEST_NF) -r -rm -j2
endif
CD2NROFF = $(top_srcdir)/scripts/cd2nroff $< >$@