These ensure a more consistent style and can find some errors statically
that would otherwise only be seen at run-time. Also, bump the Ubuntu
version of some other checks to get newer versions of some linters.
Closes#15067
As a workaround for the latest Debian image missing the necessary keys:
```
No signatures for debian:testing-20240926-slim
Administrative keys for debian:testing-20240926-slim
Repository Key: 5717dcd81d9fb5b73aa15f2d887a6a0de543829ab9b2d411acce9219c2f8ba3a
Root Key: 575d013f89e3cbbb19e0fb06aa33566c22718318e0c9ffb1ab5cc4291e07bf84
No valid trust data for testing-20240926-slim
```
https://github.com/curl/curl/actions/runs/11067644862/job/30752034560?pr=15065
Ref: 0ad1ad511b
This needs the version in the GHA context, so move those back
from external file `VERSIONS`.
Also move back `VERSIONS` content in it previous place to
`.circleci/config.yml`. Update renovate config.
Renovate bot should keep updating the wolfSSL version, but from now on
not in one, but two files.
Follow-up to 820afa2b7c#15030
Follow-up to 73a3602120Closes#15130
- add `macos-15` Sequoia to combinations jobs.
- add `llvm@18` to combinations jobs for `macos-15`.
- fix cmake, autotools configurations for llvm 18.
It requires `CMAKE_OSX_SYSROOT` for cmake, and `--sysroot` within `CC`
for both cmake and autotools.
(Also add `--target` to `CC` for all jobs.)
- stop `brew update` on `macos-12`.
`macos-12` is Monterey. Homebrew stopped providing binaries for it
after the latest macOS was out. It means newer versions of packages
have to be built from source, which is slow and undesired. Try to
avoid it using this trick. If it doesn't work, `macos-12` jobs will
have to be dropped.
- update Xcode support matrix.
Ref: https://github.com/actions/runner-images/blob/main/images/macos/macos-15-arm64-Readme.mdCloses#15108
Cygwin jobs took 30 minutes and was the reason for moving them to their
own workflow. After recent changes, they finish in 8 minutes, which is
on par or lower than the longest Windows jobs. Thus, Cygwin can now be
moved back to the Windows workflow without causing extra wait for all
the workflow jobs to complete (and allowing a manual retry for the
remaining flaky jobs).
Notice that Cygwin still uses the cygwin/cygwin-install-action action to
install Cygwin itself. This action uses no caching and the time it takes
fluctuates widely depending on mirror speed, congestion and other
things. If Cygwin tends to hold up jobs again for this or other reasons,
this patch may be reverted.
Follow-up to 80a8e24956#15083
Follow-up to 2d8464c4cb#14366Closes#15086
Move test runs from autotools jobs to cmake ones for Cygwin and MSYS.
This makes producing test results and finishing the workflows faster,
because the cmake build steps are much faster than autotools in these
envs.
Also:
- drop building examples with Cygwin autotools. It takes almost
4 minutes. Keep building them with cmake, taking 20 seconds.
- drop building examples in MSYS autotools jobs that run tests.
Keep building them in autotools jobs without tests. It makes
the longest running job 2 minutes shorter.
After this patch Cygwin job times are on par with or lower than Windows
ones. It means Cygwin doesn't cause extra delay to finish the whole
workflow, allowing to re-merge these jobs into the Windows workflow.
Closes#15083
- http3-linux: add newlines for readability.
- http3-linux: use `make pytest` to run pytest.
- checksrc: use `|` multiline to sync with other uses.
- checksrc: prefer `$()`.
- prefer `>-` over `>` for folded option lists.
- cygwin: drop a `grep` no longer necessary.
Closes#15081
It seems like the Ubuntu apt mirror list or other preloaded data on the
Linux runner can go stale (?) throughout a single week causing failures
like this, in some of the jobs:
```
Ign:16 http://security.ubuntu.com/ubuntu jammy-updates/main amd64 libapr1-dev amd64 1.7.0-8ubuntu0.22.04.1
Err:16 mirror+file:/etc/apt/apt-mirrors.txt jammy-updates/main amd64 libapr1-dev amd64 1.7.0-8ubuntu0.22.04.1
404 Not Found [IP: 40.81.13.82 80]
Get:24 http://azure.archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnghttp2-dev amd64 1.43.0-1ubuntu0.2 [117 kB]
Get:25 http://azure.archive.ubuntu.com/ubuntu jammy-updates/main amd64 vsftpd amd64 3.0.5-0ubuntu1.1 [123 kB]
Fetched 4029 kB in 2s (1682 kB/s)
E: Failed to fetch mirror+file:/etc/apt/apt-mirrors.txt/pool/main/a/apr/libapr1-dev_1.7.0-8ubuntu0.22.04.1_amd64.deb 404 Not Found [IP: 40.81.13.82 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Error: Process completed with exit code 100.
```
https://github.com/curl/curl/actions/runs/11074211380/job/30772496037#step:32:63
Restore `apt update` to fix it.
Follow-up to 1b8449674a#14992Closes#15082
`singleuse.pl` requires non-unity builds. With unity builds, it remains
silent.
- make some jobs non-unity to allow singleuse to do its job.
- since all jobs used the `--unit` singleuse checker option, make this
option permanent.
Closes#15065
- reduce job timeouts for msys2 and old-mingw-w64 jobs.
- fold long lines and reorder options in MSVC jobs.
- make some options default in MSVC jobs.
Closes#15017
- CI default: raise parallelism to 20
- CI valgind: set parallelism to 6
- CI non-native: adapt parallelism for OS builds
- CI Windows: no longer ignore FTP, TFTP, MQTT and SMTP
- CI Windows: restrict test timeout to 10 minutes
- CI Windows: do not run tests for msh3 build
- tests, various: restrict curl invocation to ipv4 to avoid talking to
any ipv6 test server running in parallel
- tests: requiring http/2 server no longer needs to mention http server
- test 190: use a fixed timeout of 10 seconds instead of %FTPTIME2, as
that value rises under parallel load beyond what the server is waiting
and then produces different results
- test 1540: add debug logging
- testrunner: add verify checks for http/2 and http/3 server
Closes#15040
It enables running pytests in cmake jobs, regardless of underlying build
tool choice (= makes it work with ninja.)
Also:
- drop pytest logic launching `make` and exiting in case of failure.
Maybe there is a better way and keep this functionality somehow, bind
it to a command-line option? make it fail softly?
- GHA/linux: invoke pytest via the build, not directly.
- autotools: add missing dummy runtests targets when cross-compiling.
Closes#15034
Valgrind jobs are slow, drop it from jobs where its use is redundant
and/or has limited impact:
- BearSSL: deprecated.
- LibreSSL heimdal with autotools.
Keep valgrind for the same job with cmake.
- msh3.
- IntelC no-SSL.
Keep valgrind for IntelC OpenSSL.
- OpenSSL 3.
All OpenSSL jobs are v3 now, keep valgrind for the `-O3`, and
libssh2 + sync-resolver variants.
Closes#15020
Add `CURL_CA_SEARCH_SAFE` build-time option to enable CA bundle search
in the `curl` tool directory. The lookup method was already used to find
`.curlrc` and `_curlrc` (on Windows). On Windows it overrides the unsafe
default `SearchPath()` method.
Enable with:
- cmake: `-DCURL_CA_SEARCH_SAFE=ON`
- autotools: `--enable-ca-search-safe`
- raw: `CPPFLAGS=-DCURL_CA_SEARCH_SAFE`
On Windows, before this patch the whole `PATH` was searched for
a CA bundle. `PATH` may contain unwanted or world-writable locations,
including the current directory. Searching them all is convenient to
pick up any CA bundle, but not secure.
The Muldersoft curl distro implements such CA search via a custom
patch for Windows:
cd652d4792/patch/curl_tool_doswin.diff (L50)
MSYS2/mingw-w64 distro has also been rolling a patch solving this:
https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-curl/0001-Make-cURL-relocatable.patchhttps://github.com/msys2/MINGW-packages/blob/master/mingw-w64-curl/pathtools.c
Also add option to fully disable Windows CA search:
- cmake: `-DCURL_DISABLE_CA_SEARCH=ON`
- autotools: `--disable-ca-search`
- raw: `CPPFLAGS=-DCURL_DISABLE_CA_SEARCH`.
Both options are considered EXPERIMENTAL, with possible incompatible
changes or even (partial) removal in the future, depending on feedback.
An alternative, secure option is to embed the CA bundle into the binary.
Safe search can be extended to other platforms if necessary or useful,
by using `_NSGetExecutablePath()` (macOS),
`/proc/self/exe` (Linux/Cygwin), or `argv[0]`.
Closes#14582
- linux: bump up test parallelism for valgrind tests to `-j4`
(from `-j2`). (EXPERIMENTAL)
- linux: drop `apt-get update` for the default architecture on the GHA
native runner. It makes prereq install steps complete faster.
The runner image gets weekly updates, and that should be enough to
guarantee fresh packages in most cases:
https://github.com/actions/runner-images/commits/main/images/ubuntu/Ubuntu2204-Readme.md
- aws-lc: use ninja with cmake.
- aws-lc: build examples with cmake.
- aws-lc: drop `apt update`.
- aws-lc, wolfssl, linux32, http3-linux: move building examples to
the last step.
Follow-up to 45202cbba4#14906
- windows: formatting.
Closes#14992
They are still slow in these jobs/combinations.
- non-native/FreeBSD/arm64 autotools +36s
- non-native/FreeBSD/arm64 cmake +1m
- windows/linux-cross-mingw-w64 autotools +33s
These ones remain:
- linux/aws-lc cmake +6s
- windows/linux-cross-mingw-w64 cmake +12s
Follow-up to 71cf0d1fca#14772
Add CI checker to compare `libcurl.pc` and `curl-config` files
generated by autotools and cmake builds.
Fix differences and apply tiny cleanups:
- curl-config: use single-quotes for literals.
- curl-config: quote all variables.
- curl-config: replace double with single quotes in a substituted value
that's always literal (`@prefix@`).
- libcurl.pc: spelling in `Description:`.
- libcurl.pc: avoid substitution in a comment.
- cmake: fill `libdir` with `${exec_prefix}` instead of a literal.
To sync with './configure'.
- configure: fix `CURL_CA_BUNDLE` value to not generate nested quotes
in `curl-config`.
- configure: add missing `LDFLAGS` to `Libs.private` in `libcurl.pc`.
To sync with CMake.
- cmake: skip adding `CMAKE_C_IMPLICIT_LINK_LIBRARIES` for MINGW and
UNIX. They added these values as seen in CI:
MINGW: `-lmingw32 -lgcc -lmoldname -lmingwex -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 -lgcc -lmoldname -lmingwex`
Linux: `-lgcc -lgcc_s -lc -lgcc -lgcc_s`
- cmake: delete FIXME about enabling libssh2 by default.
`./configure` has special defaults for these packages (called: "off"):
brotli, zstd, libpsl, libssh2, libssh, wolfssl, librtmp
It looks for them, but only at system locations, which makes them
never detected e.g. on macOS. CMake doesn't offer such default mode
for now.
- GHA/macos: drop now redundant `-DCURL_DISABLE_LDAPS=ON`.
- cmake: use `CMAKE_INSTALL_INCLUDEDIR` and `CMAKE_INSTALL_LIBDIR`
instead of hardcoded `include`/`lib` when generating `libcurl.pc`.
Updates to the GHA workflow:
- move autotools out-of-tree and rename cmake out-of-tree directory
to `bld_cm` to tell it's cmake.
- disable static libcurl for `./configure` to match cmake.
- enable `pkg-config` debug output with `./configure`.
- dump list of Homebrew packages on macOS.
- dump `./configure` detailed logs.
- disable zstd and brotli for Linux, to match cmake.
There remain differences, mostly due to detection order and method. Also
some values are inherently different when using CMake and autotools,
such as `--cc`, `--configure`. autotools also generates duplicates for
`-lssl` and `-lcrypto`. macOS LDAP wants to link `-lber` while autotools
doesn't. Some build defaults are also different in autotools and cmake.
These differences are smoothened out for now by the checker script, or
via build options. Notice that lib order (a dupes) _can_ be significant
in some cases. E.g. the binutils linker is infamous for that on Windows.
Closes#14681
