Commit Graph

600 Commits

Author SHA1 Message Date
Viktor Szakats
2638570241
GHA/linux: fix pip3 install impacket breakage
An upstream update `impacket` pip package started requiring `blinker`.
An older version is shipping with Ubuntu, causing this on install:
```
  Attempting uninstall: blinker
    Found existing installation: blinker 1.7.0
ERROR: Cannot uninstall blinker 1.7.0, RECORD file not found. Hint: The package was installed by debian.
```

Fix it by switching to venv and install everything separate from the
system.

The overhead is the same as using `pip --ignore-installed`, which also
installs everything from scratch.

The 3rd option is to uninstall the system `python3-blinker` package, but
it was the slowest.

Closes #15578
2024-11-14 02:14:02 +01:00
Viktor Szakats
3c6bf10863
GHA/macos: follow Homebrew and switch to pkgconf
Homebrew switched to `pkgconf`, and now pkg-config installs an extra
package. Update package list to avoid that.

The side-effect of `pkgconf` is that this former log message:
```
--   Package 'libcrypto', required by 'libssh2', not found
```
https://github.com/curl/curl/actions/runs/11779568834/job/32808325442#step:7:84

is replaced by this, and repeated 10 times:
```
Package libcrypto was not found in the pkg-config search path.
Perhaps you should add the directory containing `libcrypto.pc'
to the PKG_CONFIG_PATH environment variable
Package 'libcrypto', required by 'libssh2', not found
```
https://github.com/curl/curl/actions/runs/11792711391/job/32846858320#step:7:85

Closes #15575
2024-11-13 18:19:38 +01:00
Daniel Engberg
acc73edce8
GHA/non-native: streamline installed packages on FreeBSD
Instead of installing the cmake package which is a meta-port (includes
documentation and manpages etc) install cmake-core which is cmake itself
to save a few cpu cycles.

Also drop nghttp2 in favour of the slimmer libnghttp2.

Closes #15540
2024-11-12 13:02:44 +01:00
renovate[bot]
b723f6a445
GHA: update four dependencies
- gnutls/gnutls to v3.8.8
- rojopolis/spellcheck-github-actions digest to 403efe0
- awslabs/aws-lc to v1.38.0
- github/codeql-action digest to 4f3212b

Closes #15487
Closes #15490
Closes #15516
Closes #15528
2024-11-09 14:30:53 +01:00
Viktor Szakats
9416e49ceb
GHA/non-native: enable nghttp2 in OmniOS job
Closes #15530
2024-11-09 01:15:03 +01:00
renovate[bot]
f6dd3a66b2
ci: Update vmactions/omnios-vm digest to 16b5996
Closes #15529
2024-11-09 00:52:35 +01:00
Viktor Szakats
1cd745a581
ECH: enable support for the AWS-LC backend
Extend existing ECH support for BoringSSL to its AWS-LC fork.

Also enable ECH in AWS-LC CI jobs.

```
curl 8.11.0-DEV (x86_64-pc-linux-gnu) libcurl/8.11.0-DEV AWS-LC/1.37.0 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libpsl/0.21.2
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli ECH HSTS HTTPS-proxy IPv6 Largefile libz NTLM PSL SSL threadsafe UnixSockets zstd
```

Closes #15499
2024-11-08 13:59:39 +01:00
Daniel Stenberg
cbafcec50b
curl: --test-duphandle in debug builds runs "duphandled"
Using this option (only available in debug builds) makes curl always
call curl_easy_duphandle() on the handle before using it.

To help us catch curl_easy_duphandle() mistakes better.

Add a CI job using this.

Bonus: the previous runtests option -e is now also supported as
--test-event

Closes #15504
2024-11-08 13:22:47 +01:00
Viktor Szakats
4d8ab1f1b8
GHA/macos: let gcc dictate the configured Apple SDK
As discovered earlier, Homebrew gcc is built against a specific Apple
SDK version and doesn't work when matched up with a different version,
e.g. the one advertised as default by the macos runner image.

Before this patch this was resolved with brute force by zapping the
hack-layer gcc component to avoid the bad interference. This worked
for us, but it's fragile, accidental and doesn't translate to
real-world build environments. Thus, impractical.

Avoid this by explicitly selecting the SDK version gcc was built for and
meant to be used with, as shown by `gcc --print-sysroot`.

It assumes that the gcc binaries preinstalled on the runner images
always ship with the SDK version they reference. It also assumes
this works with and without `brew update`.

Also:
- add 4 quick build-only jobs to test all gcc/macos combos.
- list SDKs offered via CommandLineTools.

Suggested-by: Bo Anderson
Ref: https://github.com/Homebrew/homebrew-core/issues/194778#issuecomment-2462764619

Follow-up to c349bd668c #14097
Closes #15518
2024-11-08 10:48:12 +01:00
Viktor Szakats
cb092c0a7e
GHA: add apt update where missing
To mitigate this kind of (repeat) errors:
```
Err:14 http://azure.archive.ubuntu.com/ubuntu noble-updates/main amd64 python3-werkzeug all 3.0.1-3ubuntu0.1
  404  Not Found [IP: 40.81.13.82 80]
E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/p/python-werkzeug/python3-werkzeug_3.0.1-3ubuntu0.1_all.deb
Fetched 10.4 MB in 4s (2593 kB/s)
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
```
https://github.com/curl/curl/actions/runs/11732257460/job/32684111508#step:3:74

Follow-up to 842f88434f #15082
Closes #15519
2024-11-08 10:48:11 +01:00
Stefan Eissing
bcf8a84881
mbedtls: call psa_crypt_init() in global init
Run mbedtls' psa_crypt_init() in the general global init, optionally
protected by mbedtls locks when available.

CI: when building mbedtls, enabled thread safety

Reported-by: wxiaoguang on github
Fixes #15500
Closes #15505
2024-11-07 13:14:41 +01:00
Viktor Szakats
380790b244
GHA/non-native: fix installing OpenLDAP on OpenBSD
Also:
- drop failing manual install of perl. It's there by default now.
- add link to OpenBSD package management FAQ page.

Closes #15491
2024-11-06 02:42:09 +01:00
Viktor Szakats
087f77d855
GHA/macos: drop WebSockets from job names
It's enabled in all jobs by default now.

Follow-up to cd63629733 #15076
2024-11-06 01:43:55 +01:00
Viktor Szakats
2f22fc10e5
GHA/linux-old: adjust configure job name 2024-10-31 13:26:27 +01:00
Viktor Szakats
9acecc923d
tidy-up: whitespace, fix CI spacecheck for docs
Also: fixup CI spacecheck to apply to docs again.

Closes #15423
2024-10-30 23:15:28 +01:00
renovate[bot]
85ee614020
ci: update rojopolis/spellcheck-github-actions digest to 74c2a14
Closes #15412
2024-10-29 15:52:30 +01:00
Viktor Szakats
1a2d38c47c
GHA/windows: avoid curl.exe libtool wrapper
Avoid the `curl.exe` wrapper binary created by libtool, and run the real
`curl.exe` directly for tests and version information.

This solution was used in Azure jobs. I missed it when migrating jobs
to GHA.

Applies to tests run in the `mingw, AM x86_64 c-ares U` job, which has
seen unexplained flakiness.

Ref: 354afc891d #6049
Follow-up to e53523fef0 #14859

Closes #15437
2024-10-29 09:20:37 +01:00
Daniel Stenberg
770702fa3d
CI: bump wolfSSH and wolfSSL
- wolfSSH 1.4.18
- wolfSSL 5.7.4

Closes #15427
2024-10-28 09:38:37 +01:00
Jay Satiro
ac7ae08f02
schannel: reclassify extra-verbose schannel_recv messages
- Create a new macro SCH_DEV() to manage verbose debug messages that are
  only useful for debugging Schannel recv decryption.

schannel_recv contains a lot of useful debug messages to help debug the
function, however in practice they are not otherwise useful and showing
them in debug builds adds a lot of noise.

To show these messages curl must now be built with
CURL_SCHANNEL_DEV_DEBUG defined.

Prior to this change many, but not all, extra-verbose messages were
wrapped in DEBUGF() so they were only shown in debug builds.

Ref: https://github.com/curl/curl/issues/14807

Closes #14826
2024-10-24 10:34:20 +02:00
renovate[bot]
38c57bdf0d
GHA: update five dependencies
- rojopolis/spellcheck-github-actions digest to ab8ac45
- nghttp2/nghttp2 to v1.64.0
- actions/cache digest to 6849a64
- github/codeql-action digest to 6624720
- Update actions/checkout digest to 11bd719

Closes #15341
Closes #15346
Closes #15365
Closes #15366
Closes #15387
2024-10-24 10:03:31 +02:00
Viktor Szakats
6b440704d8
GHA: drop "3" from openssl names and keys
Also:
- drop patch suffix from cache key for thread-sanitizer local build
  Follow-up to 73d2779196 #15379

Closes #15383
2024-10-23 19:36:03 +02:00
Viktor Szakats
5f9411f953
GHA/windows: work around Git for Windows perf regression
Fix the significant perf regression for vcpkg jobs by switching to the
MSYS2 shell environment from Git for Windows. This env is already used
for old-mingw-w64 job that remained unaffected by this issue.

The issue began with the windows-runner update 20241015.1.0. It bumped
Git for Windows from Git 2.46.2.windows.1 to Git 2.47.0.windows.1. GfW
bumped its MSYS2 components, including `msys-2.0.dll`. That's Cygwin
code, which may have contributed to this. Pipes were involved and
`runtests.pl` relies on pipes heavily in parallel mode. (The issue was
not seen with parallel tests disabled, in retrospect.)

This is useful as a permanent solution too. It drop GfW as a dependency
and makes Windows jobs use one less shell/env flavour.

Long term it might help to use native Windows Perl to avoid the MSYS
layer completely, if there is a way to make that work.

Assortment of possibly related links:
https://cygwin.com/pipermail/cygwin/2024-August/256398.html
f78009cb1c
7f3c225325

https://github.com/actions/runner-images/issues/10843
https://github.com/git-for-windows/git/issues/5199
https://github.com/git-for-windows/msys2-runtime/pull/75
7913a41703
555afcb2f3
1c5f4dcdc5

Follow-up to c33174d42f #15364
Follow-up to 1e0305973c #15356

Closes #15380
2024-10-23 13:00:41 +02:00
Viktor Szakats
73d2779196
GHA/linux: drop patch from openssl3 thread sanitizer
The patch is now part of the 3.4.0 stable release.
(Turns out it was part of 3.3.2 already.)

Also:
- rename this local build to match the scheme used with wolfssl.
- drop '3' from local openssl build name.
- sync job name with others.
- quote step names where missing.

Follow-up to a2bcec0ee0 #14751
Closes #15379
2024-10-23 12:02:17 +02:00
renovate[bot]
e1099726a5
CI: update dependency openssl/openssl to v3.4.0
Closes #15377
2024-10-23 11:24:21 +02:00
Max Dymond
e43d37c541
ci: fix renovate's matching for OpenSSL and quictls
Renovate only matches on the raw version numbers of a package, but
OpenSSL includes `openssl-` as a prefix in the version number. This
change means that the match string now expects the `openssl-` prefix
and will just update the version portion.

This also updates quictls so that renovate can detect and update the
version correctly.

Closes #15359
2024-10-23 10:40:43 +02:00
Viktor Szakats
b327a53f09
GHA: use --no-install-suggests --no-install-recommends where missing
It prevents `apt-get install` commands installing unnecessary packages.

Makes the 8 HTTP/3 jobs around 30 seconds faster each.

before: https://github.com/curl/curl/actions/runs/11466168597
after:  https://github.com/curl/curl/actions/runs/11469013245?pr=15373

Closes #15373
2024-10-23 01:29:52 +02:00
Viktor Szakats
605bc2d2c6
GHA/linux: merge 32-bit Linux workflow
Also:
- tidy up a step condition.

Closes #15370
2024-10-22 23:38:13 +02:00
Viktor Szakats
b6219cd932
GHA/linux: merge torture jobs into the main workflow
They complete in 4 and 7 minutes, and do not hold back the main Linux
workflow.

Also:
- bump default parallelism for `test-torture` target to `-j20`
  (was: `-j2`).
- drop redundant package install from `rustls` jobs.

Closes #15360
2024-10-22 20:20:42 +02:00
Viktor Szakats
52851d3256
GHA/macos: use test-torture target for torture tests
They used `test-ci` before this patch.

Closes #15369
2024-10-22 20:20:42 +02:00
Viktor Szakats
c33174d42f
GHA/windows: increase timeout for vcpkg jobs due to slowness
The openssl job no longer fits into 10 minutes since the 20241015.1.0
GHA windows-latest image update. This caused all runs to fail.

The `run tests` step takes ~10 minutes now, up from ~4. This is
6 minutes more than before these updates. It's seen with other vcpkg
jobs too, tests run slower than half speed since.

Bump the timeout to make it, though the headroom is now less than it
was.

Before:
https://github.com/curl/curl/actions/runs/11386748199/job/31679733295
https://github.com/curl/curl/actions/runs/11347976608/job/31560690219

After:
https://github.com/curl/curl/actions/runs/11462332743/job/31893491625?pr=15364

Ref: fcc4cdb1d0
Ref: https://github.com/curl/curl/pull/15335#issuecomment-2423759953
Follow-up to 1e0305973c #15356
Closes #15364
2024-10-22 19:13:08 +02:00
Daniel Stenberg
dcb27fdd40
GHA: fix the msh3 renovate thing
Follow-up to 943df95ae7

Closes #15363
2024-10-22 15:48:27 +02:00
Stefan Eissing
943df95ae7
CI: run with standard mod_http2
We used to include a special mod_h2 in our CI that supports the
directive H2MaxDataFrameLen for test_02_20. Since then, ubuntu-lastest
includes a more recent apache httpd. Let's see if we can live without
the special.

Closes #15353
2024-10-22 15:31:46 +02:00
Tal Regev
c2e2636773
GHA/windows: add http3 to libressl vcpkg job
Also drop a libressl workaround no longer necessary.

Closes #15338
2024-10-22 14:23:12 +02:00
Viktor Szakats
1e0305973c
GHA/windows: ignore results for test 987
987 is `SMTPS with redundant explicit SSL request`.

Root cause undiscovered.

Started failing after GHA bumping the windows image to `20241015.1.0`:
fcc4cdb1d0
https://github.com/actions/runner-images/pull/10803

vcpkg packages also got bumped as a result. They seem unrelated:
c-ares  1.33.1 -> 1.34.1
nghttp2 1.62.1 -> 1.63.0
nghttp3 1.5.0  -> 1.6.0
(there may be more)

Ref: https://github.com/curl/curl/pull/15335#issuecomment-2423759953

Closes #15356
2024-10-22 14:17:58 +02:00
Viktor Szakats
0978afd7aa
GHA/linux: tidy up and performance
- replace openssl3 default local build with packaged one.
- drop valgrind from IntelC job.
- drop IntelC no-ssl job.
- bump local openssl to 3.3.2.
- disable tests in the cmake variant of a job.
- add comment to the remaining local openssl3 build.
  We can drop the patch after upgrading to upcoming 3.4.0.
- drop gcc-11 from jobs. packaged gcc is now newer at 13.2.0.
  (saves more than 1m install time for each of the 5 jobs.)
  Follow-up to 9cc9a6472c #9454

Closes #15349
2024-10-22 13:15:37 +02:00
Viktor Szakats
2816cba2d7
GHA/linux: bump to quictls 3.3.0
Closes #15334
2024-10-20 01:16:28 +02:00
renovate[bot]
e29629a402
GHA: update ngtcp2/ngtcp2 and awslabs/aws-lc
- update ngtcp2/ngtcp2 to v1.8.1
- update awslabs/aws-lc to v1.37.0

Closes #15318
Closes #15329
2024-10-18 16:43:52 +02:00
Daniel Stenberg
b9877b74c5
GHA: drop the hyper job
Hyper support is being removed in 2025. No one works on it. Getting
flaky test runs with this job adds nothing to the project.

Closes #15326
2024-10-17 22:56:45 +02:00
Daniel Stenberg
38bfe1c2aa
GHA: switch off proselint
Because we cannot disable the individual warnings we do not care about,
making this tool almost unusable for our purposes. See
https://github.com/amperser/proselint/issues/1367

Instead, make 'very' a banned word (as recently that has been what
proselint most commonly points out for us).

Closes #15314
2024-10-17 13:36:42 +02:00
Viktor Szakats
1e01e2b549
GHA/macos: merge autotools and cmake jobs
To match other workflows and to avoid repetition in rules.

Also:
- fix build example step for cmake. update a job to use it.
- use `cmake` to invoke the builds (instead of ninja directly).
- extend test 2100 exclusion to more jobs.
  It fails with all `!debug gcc-12` jobs with autotools.
  With cmake this only happened for gcc-12 Secure Transport jobs
  for some reason.

Closes #15312
2024-10-16 23:25:18 +02:00
Dan Fandrich
a2f913ef6a CI: explicitly specify the OS version when necessary
Commit 8ea120f6 added --break-system-packages which works in Ubuntu
24.04 but not 22.04, so explicitly specify that version in the runner
instead of relying on ubuntu-latest to provide it.  Some runners have
regressed back to 22.04 for ubuntu-latest, resulting in build failures.
2024-10-16 09:56:14 -07:00
renovate[bot]
1cf187a4f6
CI: update GHA dependencies
- update Mbed-TLS/mbedtls to v3.6.2
- update libressl-portable/portable to v4
- update github/codeql-action digest to f779452

Closes #15301
Closes #15299
Closes #15188
2024-10-16 08:34:19 +02:00
Viktor Szakats
3040971d12
GHA: silence proselint warnings and an error
Fix new issues found by `proselint`.

Also:
- silence this technical warning:
  ```
  :0: DeprecationWarning: /home/runner/.proselintrc was found instead of a JSON file. Rename to /home/runner/.proselintrc.json.
  ```
- fix an input filename.
  `proselints` fails now if an input file is missing.

Reported-by: Jay Satiro
Bug: https://github.com/curl/curl/pull/15291#issuecomment-2410505100
Closes #15293
2024-10-15 16:44:17 +02:00
Viktor Szakats
e20b139a1d
GHA/linux: add cmake job for system mbedTLS with pkg-config
Add a build-only cmake job with system mbedTLS package and `pkg-config`
enabled. Ubuntu 24.04 comes with mbedTLS 2.28.8 which supports
`pkg-config`.

Follow-up to 7bab201abe #15193
Closes #15286
2024-10-15 01:40:20 +02:00
Viktor Szakats
8ea120f615
GHA/linux: fixup pip for Ubuntu 24.04
`ubuntu-latest` became `ubuntu-24.04` today.

Closes #15287
2024-10-14 03:17:36 +02:00
Viktor Szakats
e8a007de0d
GHA: optimize test prereq steps
- Linux: move test and pytest prereqs right before test run.
  - returns build phase results faster.
  - allows skipping steps for jobs that don't need them.
  - makes dependencies more transparent.
- sync prereq install step names.
- use `tests/requirements.txt` more.

Closes #15275
2024-10-13 11:34:59 +02:00
Tal Regev
66cc015756
GHA/windows: drop vcpkg workaround
Reverts 841f42150a #15133
Closes #15217
2024-10-12 22:47:31 +02:00
Viktor Szakats
79809ffe1e
GHA/linux: mbedTLS 3.6.1
Closes #15274
2024-10-12 08:58:19 +02:00
renovate[bot]
ba68eb02fa
CI: update rojopolis/spellcheck, actions/checkout, actions/upload-artifact
- update rojopolis/spellcheck-github-actions digest to 7ff888c
- update actions/checkout digest to eef6144
- update actions/upload-artifact digest to b4b15b8

Closes #15207
Closes #15184
Closes #15187
2024-10-11 23:53:42 +02:00
dependabot[bot]
7d53a59292
CI: bump github/codeql-action, vmactions/omnios-vm and actions/cache
- bump github/codeql-action from 3.26.10 to 3.26.11
- bump vmactions/omnios-vm from 1.0.7 to 1.0.8
- bump actions/cache from 4.0.2 to 4.1.1

Closes #15178
Closes #15179
Closes #15244
2024-10-11 23:44:19 +02:00