This needs the version in the GHA context, so move those back
from external file `VERSIONS`.
Also move back `VERSIONS` content in it previous place to
`.circleci/config.yml`. Update renovate config.
Renovate bot should keep updating the wolfSSL version, but from now on
not in one, but two files.
Follow-up to 820afa2b7c#15030
Follow-up to 73a3602120Closes#15130
- http3-linux: add newlines for readability.
- http3-linux: use `make pytest` to run pytest.
- checksrc: use `|` multiline to sync with other uses.
- checksrc: prefer `$()`.
- prefer `>-` over `>` for folded option lists.
- cygwin: drop a `grep` no longer necessary.
Closes#15081
It seems like the Ubuntu apt mirror list or other preloaded data on the
Linux runner can go stale (?) throughout a single week causing failures
like this, in some of the jobs:
```
Ign:16 http://security.ubuntu.com/ubuntu jammy-updates/main amd64 libapr1-dev amd64 1.7.0-8ubuntu0.22.04.1
Err:16 mirror+file:/etc/apt/apt-mirrors.txt jammy-updates/main amd64 libapr1-dev amd64 1.7.0-8ubuntu0.22.04.1
404 Not Found [IP: 40.81.13.82 80]
Get:24 http://azure.archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnghttp2-dev amd64 1.43.0-1ubuntu0.2 [117 kB]
Get:25 http://azure.archive.ubuntu.com/ubuntu jammy-updates/main amd64 vsftpd amd64 3.0.5-0ubuntu1.1 [123 kB]
Fetched 4029 kB in 2s (1682 kB/s)
E: Failed to fetch mirror+file:/etc/apt/apt-mirrors.txt/pool/main/a/apr/libapr1-dev_1.7.0-8ubuntu0.22.04.1_amd64.deb 404 Not Found [IP: 40.81.13.82 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Error: Process completed with exit code 100.
```
https://github.com/curl/curl/actions/runs/11074211380/job/30772496037#step:32:63
Restore `apt update` to fix it.
Follow-up to 1b8449674a#14992Closes#15082
`singleuse.pl` requires non-unity builds. With unity builds, it remains
silent.
- make some jobs non-unity to allow singleuse to do its job.
- since all jobs used the `--unit` singleuse checker option, make this
option permanent.
Closes#15065
- CI default: raise parallelism to 20
- CI valgind: set parallelism to 6
- CI non-native: adapt parallelism for OS builds
- CI Windows: no longer ignore FTP, TFTP, MQTT and SMTP
- CI Windows: restrict test timeout to 10 minutes
- CI Windows: do not run tests for msh3 build
- tests, various: restrict curl invocation to ipv4 to avoid talking to
any ipv6 test server running in parallel
- tests: requiring http/2 server no longer needs to mention http server
- test 190: use a fixed timeout of 10 seconds instead of %FTPTIME2, as
that value rises under parallel load beyond what the server is waiting
and then produces different results
- test 1540: add debug logging
- testrunner: add verify checks for http/2 and http/3 server
Closes#15040
It enables running pytests in cmake jobs, regardless of underlying build
tool choice (= makes it work with ninja.)
Also:
- drop pytest logic launching `make` and exiting in case of failure.
Maybe there is a better way and keep this functionality somehow, bind
it to a command-line option? make it fail softly?
- GHA/linux: invoke pytest via the build, not directly.
- autotools: add missing dummy runtests targets when cross-compiling.
Closes#15034
Valgrind jobs are slow, drop it from jobs where its use is redundant
and/or has limited impact:
- BearSSL: deprecated.
- LibreSSL heimdal with autotools.
Keep valgrind for the same job with cmake.
- msh3.
- IntelC no-SSL.
Keep valgrind for IntelC OpenSSL.
- OpenSSL 3.
All OpenSSL jobs are v3 now, keep valgrind for the `-O3`, and
libssh2 + sync-resolver variants.
Closes#15020
- linux: bump up test parallelism for valgrind tests to `-j4`
(from `-j2`). (EXPERIMENTAL)
- linux: drop `apt-get update` for the default architecture on the GHA
native runner. It makes prereq install steps complete faster.
The runner image gets weekly updates, and that should be enough to
guarantee fresh packages in most cases:
https://github.com/actions/runner-images/commits/main/images/ubuntu/Ubuntu2204-Readme.md
- aws-lc: use ninja with cmake.
- aws-lc: build examples with cmake.
- aws-lc: drop `apt update`.
- aws-lc, wolfssl, linux32, http3-linux: move building examples to
the last step.
Follow-up to 45202cbba4#14906
- windows: formatting.
Closes#14992
- Move `docs/examples` builds under a separate target.
- Make `BUILD_EXAMPLES` default to `ON`. It means to generate the rules
for `docs/examples` by default, but not build them. To build them,
an explicit `make curl-examples` (or ninja, etc) command is necessary.
This syncs behaviour with autotools, and also how both cmake and
autotools are building tests.
- GHA: update cmake jobs to use the new way of building examples.
- GHA: move examples build step at the end of the job, after building
and running tests. This allows to have build and test run results
faster, and leave the seldom-changing examples build to the end.
Building examples is the slowest build step with no practical way to
make them fast.
- appveyor: enable building examples in two old-MSVC jobs.
- examples: fix examples to build cleanly with old MSVC versions.
- GHA/non-native: move example build log under a GHA foldable section.
- GHA/windows: move building examples into separate step for Linux cross
jobs.
Follow-up to dfdd978f7c#13491Closes#14906
Ubuntu has the `pkgconf` package installed by default that implements
the `pkg-config` command. Switch CI `apt` commands over to `pkgconf`
to avoid replacing it in every job run.
Avoids:
```
The following packages will be REMOVED:
pkgconf r-base-dev
The following NEW packages will be installed:
[...] pkg-config [...]
```
https://github.com/curl/curl/actions/runs/10949915766/job/30404126342?pr=14972#step:2:20Closes#14974
CI:
- GHA/windows: enable OpenSSH server, SysInternals `handle`, `impacket`.
Skip `impacket` on MSYS2 due to install failure.
Skip OpenSSH server for old/standalone mingw-w64 (building curl
without SSH there.)
- GHA/windows: make test tool installs a separate step.
- GHA/cygwin: enable OpenSSH server.
Skip `impacket`: it's compiling for 7 minutes then breaks.
Skip `stunnel` due to sluggish test run performance.
(This update is unrelated to Azure jobs.)
- GHA/linux: migrate Linux jobs from Azure CI.
- GHA/linux: migrate scanbuild job from Azure CI.
- GHA/linux: enable libssh2 in a job. Also enable valgrind.
- CI/windows: enable SSPI in two jobs.
- CI/windows: disable zlib in one more job.
- CI/windows: improve `if` condition checking GnuTLS.
- CI/windows: ignore SFTP/SCP tests as necessary.
- universally ignore SCP tests, they fail everywhere.
- ignore test 612.
- ignore test 613 616 618 with MSYS2 mingw-w64.
- ignore test 614 with libssh.
- ignore all SFTP with MSYS2 native.
- ignore all SFTP with vcpkg with `libssh2[core,zlib]`.
- ignore a couple of SFTP tests with MSYS2 mingw-w64.
(This matches settings on Azure CI.)
- GHA/windows: ignore failing 1451 'Basic SMB request' test for
old mingw-w64 7.3.0 (but not for 9.5.0!):
```
2024-09-11 21:45:59,738 ERROR smbComNegotiate: b'NT LM 0.12\x00' is not in list
[...]
curl: (7) Could not connect to server
[...]
FAIL 1451: 'Basic SMB request' SMB
```
Ref: https://github.com/curl/curl/actions/runs/10816280746/job/30007130770#step:13:3546
- CI/Azure: delete, now moved to GHA.
sshserver:
- sshserver: fix permissions for SSH host key on Windows, allowing sshd
to launch.
- sshserver: fix initializing config paths with Cygwin/MSYS2 sshd.
- sshserver: fix initializing config paths in parallel builds.
- sshserver: delete redundant `DenyUsers`. This also opens the way
to allow multiple usernames.
- sshserver: fix `AllowUsers` for GHA/windows, by allowing the
domainless username again.
Follow-up to 3ee7c676ec#5721
- sshserver: fix `Deprecated option` warnings in `sshd_config`:
```
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 7: Deprecated option AuthorizedKeysFile2
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 25: Deprecated option KeyRegenerationInterval
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 35: Deprecated option RhostsRSAAuthentication
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 36: Deprecated option RSAAuthentication
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 37: Deprecated option ServerKeyBits
D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 41: Deprecated option UseLogin
```
- sshserver: fix `Deprecated option` warnings in `ssh_config` with
Cygwin/MSYS2.
- sshserver: fix dumping config files due to the filenames missing their
full paths.
- sshserver: add workaround to make `logmsg` messages visible.
Before this patch they only went to a file and never shown.
runtests:
- runtests: log details when these Windows commands are called:
`handle`, `taskkill`, `tasklist`.
- runtests: add documentation links to Windows tools:
`handle`, `taskkill`, `tasklist`, `icacls`.
- runtests: add `-t` (kill whole tree) option to `taskkill` in
`servers.pm`, syncing it with the other `taskkill` call.
Follow-up to bc72a78a11#14488
- runtests: show warning if Sysinternals `handle` tool is missing.
- runtests: drop Windows XP Home compatibility `tskill` call.
The call was made on all Windows versions. It's possibly overkill to
do this, because XP Home is probably rarely used for running curl
tests these days. In case it's needed, it'd be better to put it under
an explicit option.
- runtests: show Perl version and path.
Fix/silence fallouts:
- unit2603: fix building with disabled HTTP support.
- unit2604: silence `-Woverlength-strings` warnings in C89 mode.
- test437, test1614: fix to pass with no-IPv6 builds.
Closes#14859
There is a 4-5x difference in test run times. Make the reason more
obvious by adding valgrind to the job names.
Also:
- tidy up job names.
Closes#14853
* Documentation is updated to describe new required version, and to link
to the upstream README about cryptography providers.
* GitHub workflow is updated to fetch 0.14.0.
* Breaking changes in`lib/vtls/rustls.c` are addressed:
* The `rustls_client_config_builder_build()` function now uses an out
parameter for the built config instead of returning it directly.
This allows the building process to fail if the default crypto
provider state isn't appropriate, or another error condition occurs.
* Default ciphersuites are collected using renamed functions named to
make it clear the ciphersuites are associated with the default
crypto provider.
* Customization of ciphersuites is now done via
a `rustls_crypto_provider_builder` used to instantiate a
`rustls_crypto_provider`. The customized provider can then can be
used with `rustls_client_config_builder_new_custom` in place of
providing ciphersuites directly.
* `rustls_connection_get_negotiated_ciphersuite()` now returns the
ciphersuite ID directly.
Closes#14889
The SSL_Session object is mutated during connection inside openssl,
and it might not be thread-safe. Besides, according to documentation
of openssl:
```
SSL_SESSION objects keep internal link information about the session
cache list, when being inserted into one SSL_CTX object's session
cache. One SSL_SESSION object, regardless of its reference count,
must therefore only be used with one SSL_CTX object (and the SSL
objects created from this SSL_CTX object).
```
If I understand correctly, it is not safe to share it even in a
single thread.
Instead, serialize the SSL_SESSION before adding it to the cache,
and deserialize it after retrieving it from the cache, so that no
concurrent write to the same object is infeasible.
Also
- add a ci test for thread sanitizer
- add a test for sharing ssl sessions concurrently
- avoid redefining memory functions when not building libcurl, but
including the soruce in libtest
- increase the concurrent connections limit in sws
Notice that there are fix for a global data race for openssl which
is not yet release. The fix is cherry pick for the ci test with
thread sanitizer.
d8def79838Closes#14751
Running the http pytest had to be done from tests directory or above,
because the repeat argument fixture was defined in tests/conftest.py.
However, the repeat argument is not needed because its functionality
can be provided by pytest-repeat as documented in the test's
README.md. So, removed the pytest_addoption function for the repeat
argument and the pytest_report_header function is moved to
tests/http/conftest.py.
TODO: Remove repeat argument from all tests. As a stopgap, a
one-element list is defined for it for now.
Closes#14611
Before this patch `pkg-config`-based detection was ignored, and used
solely as a path hint for native detection.
- fix `pkg_search_module()` result prefix to match what code expects:
`_GSS` (was: `_GSS_PKG`). Update variable that were in sync with old
prefix.
- update the pkg-config codepath to use `_GSS_MODULE_NAME` to detect
GSS flavour. This requires CMake 3.16.
Otherwise fall back to the old method. (The old method doesn't seem to
work anymore (?) as of CMake 3.30.1. Documented
`<prefix>_<modulename>_VERSION` variable is defined, but empty.)
- update the pkg-config codepath to use `_GSS_VERSION` set by CMake.
Resort to the old code when this variable is empty. (The old code
doesn't seem to work anymore (?) as of CMake 3.30.1)
- fix pkg-config codepath to set the documented result variables.
- align native detection variable names with those generated by
`pkg_search_module()` in the pkg-config codepath.
- GHA/macos: enable GSS Heimdal in a cmake job.
Uses the native detection.
- GHA/linux: enable GSS Heimdal in cmake and autotools jobs.
CMake uses `pkg-config`-based detection.
- suppress test 2077 and 2078 results on Linux + Heimdal.
```
FAIL-IGNORED 2077: 'curl --fail --negotiate to unauthenticated service fails' HTTP, HTTP GET, GSS-API
FAIL-IGNORED 2078: 'curl --negotiate should not send empty POST request only' HTTP, HTTP GET, GSS-API
```
Failing with valgrind errors in both autotools and cmake builds:
https://github.com/curl/curl/actions/runs/10282222581/job/28453472068?pr=14430#step:38:3638https://github.com/curl/curl/actions/runs/10282222581/job/28453473398?pr=14430#step:38:7831Closes#14430
- extend existing Linux workflow with CMake support.
Including running pytest the first time with CMake.
- cmake: generate `tests/config` and `tests/http/config.ini`.
Required for pytest tests.
Uses basic detection logic. Feel free to take it from here.
Also dump config files in a CI step for debugging purposes.
- cmake: build `tests/http/clients` programs.
- fix portability issues with `tests/http/clients` programs.
Some of them use `getopt()`, which is not supported by MSVC.
Fix the rest to compile in CI (old-mingw-w64, MSVC, Windows).
- GHA/linux: add CMake job matching an existing autotools one.
- GHA/linux: test `-DCURL_LIBCURL_VERSIONED_SYMBOLS=ON`
in the new CMake job.
- reorder testdeps to build server, client tests first and then
libtests and units, to catch errors in the more complex/unique
sources earlier.
- sort list in `tests/http/clients/Makefile.inc`.
Closes#14382
- run tests via `make test-ci` instead of `make check` with autotools.
- add `x86_64` job for FreeBSD, with tests.
It matches the existing Cirrus CI job, with these differences:
- finishes 3x faster (thanks to parallel tests enabled).
- librtmp is not enabled because it's slated for removal by FreeBSD.
(already past the removal deadline, thought the package still
installs.)
- DICT and TELNET servers fail to start. Couldn't figure out why.
It means skipping test 1450 and 1452.
- it runs more tests, e.g. websockets and ip6-localhost.
- no `pkg update -f`.
- it misses the `CRYPTOGRAPHY_DONT_BUILD_RUST=1`, `pkg delete curl`,
`chmod 777`, `sudo -u nobody` and `sysctl net.inet.tcp.blackhole`
tricks. The latter is the default in these runners, the others did
not affect results.
- set `-j0` for tests in the NetBSD job. Flaky otherwise.
Closes#14244
Also mention in HTTP3.md
OpenSSL has a bug that messes the config `--libdir=path` to become the
wrong path in its pkgconfig files. If we just pass `--libdir=lib` it
should avoid this.
Ref: #14099
See also: https://github.com/openssl/openssl/issues/23569Closes#14102
- add special sauce to disable unwanted peer verification by mbedtls
when negotiating TLS v1.3
- add special sauce for MBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET
return code on *writing* TLS data. We assume the data had not been
written and EAGAIN.
- return correct Curl error code when peer verification failed.
- disable test_08_05 with 50 HTTP/1.1 connections, as mbedtls reports a
memory allocation failed during handshake.
- bump CI mbedtls version to 3.6.0
Fixes#13653Closes#13838
- bump rest of the workflows (windows, macos, distrocheck).
- non-native virtualized envs have 2 CPUs, bump down accordingly.
(for `vmactions/omnios-vm` it's just a guess.)
- bump all to nproc + 1.
Follow-up to e838b341a0#12927Closes#13807
For better build performance. Dependency tracking causes a build
overhead while compiling to help a subsequent build, but in CI there is
never one and the extra work is discarded.
Closes#13794
Fixes:
- in uds tests, abort also silently on os errors
- be conservative on the h3 goaway duration
- detect curl debug build and use in checks
- fix caddy version check for slight difference under linux
- set caddy default path fitting for linux
- fix deprecation warnings in valid time checks
FTP tests:
- add '--with-test-vsftpd=path' to configure
- use vsftpd default path suitable for linux
- add test_30 with plain FTP tests
- add test_31 with --ssl-reqd FTP tests
- add vsftpd to linux GHA for pytest workflows
Closes#13661
