Commit Graph

20 Commits

Author SHA1 Message Date
Dan Fandrich
fb35a5fe2c CI: enable RTMP and WebSockets in old Linux build
Try to enable as much as possible to check for compatibility with old
dependencies.
2024-09-21 00:27:20 -07:00
Viktor Szakats
60c3d04465
autotools: add support for 'unity' builds, enable in CI
Implement the "unity" builds as known from CMake, but for autotools.
It's limited to `lib` and `src` (CMake also supports it in `tests`).

Enable with: `--enable-unity` (disabled by default)

Unity builds speed up builds significantly. Cygwin and Windows builds in
particular, but the effect is noticeable on most systems. It also allows
discovering unity issues with autotools, benefitting also CMake when
building the same combination. In CI it makes turnaround times quicker.

This closes build performance with CMake. autotools still lags behind
because it builds shared and static libcurl in two, separate passes.
CMake does it in one. Manpage compilation isn't batched, it is in CMake.
After unity and test bundle support the slowest parts of the build are
the configuration phase (which is effectively a tedious, non-parallel,
compilation and/or linking of 300+ tiny programs. The next bottleneck
is compiling individual examples and finally test servers (only slow
with autotools).

The autotools implementation is slightly less efficient than CMake,
because 3 sources are permanently excluded while in CMake this isn't
necessary and solved more efficiently while building libtests. There is
also no 'unity' support for tests, making them a less efficient also.

Enable it in CI for most `configure` jobs. Except in GHA/dist (though
it works fine there too), to use the default config there. Also skip for
the Linux AWC-LC job where it made builds time a few seconds longer
(reason undiscovered.)

Autotools test suite builds compared between master -> `--enable-unity`:
- GHA/Linux: 32s -> 12s
  https://github.com/curl/curl/actions/runs/10705668823/job/29681617374
  https://github.com/curl/curl/actions/runs/10742978889/job/29796766297
- GHA/macOS: 37s -> 10s
  https://github.com/curl/curl/actions/runs/10705668813/job/29681632885
  https://github.com/curl/curl/actions/runs/10742978699/job/29796768875
- GHA/FreeBSD: 15m25 -> 10m58 (full workflow time, ~qemu)
  https://github.com/curl/curl/actions/runs/10705668811/job/29681607915
  https://github.com/curl/curl/actions/runs/10742978937/job/29796766115
- GHA/Cygwin: 3m32 -> 1m21
  https://github.com/curl/curl/actions/runs/10705668809/job/29681609965
  https://github.com/curl/curl/actions/runs/10742978645/job/29796756933
- GHA/MSYS2: 2m42 -> 50s
  https://github.com/curl/curl/actions/runs/10705668808/job/29681621166
  https://github.com/curl/curl/actions/runs/10742978662/job/29799739289
- GHA/mingw-w64: 5m32 -> 1m23
  https://github.com/curl/curl/actions/runs/10705668808/job/29681628787
  https://github.com/curl/curl/actions/runs/10742978662/job/29799741568

Closes #14815
2024-09-20 23:53:33 +02:00
Viktor Szakats
e53523fef0
CI: move Azure jobs to GHA, fix fallouts, sshserver, runtests tweaks
CI:

- GHA/windows: enable OpenSSH server, SysInternals `handle`, `impacket`.
  Skip `impacket` on MSYS2 due to install failure.
  Skip OpenSSH server for old/standalone mingw-w64 (building curl
  without SSH there.)
- GHA/windows: make test tool installs a separate step.
- GHA/cygwin: enable OpenSSH server.
  Skip `impacket`: it's compiling for 7 minutes then breaks.
  Skip `stunnel` due to sluggish test run performance.
  (This update is unrelated to Azure jobs.)
- GHA/linux: migrate Linux jobs from Azure CI.
- GHA/linux: migrate scanbuild job from Azure CI.
- GHA/linux: enable libssh2 in a job. Also enable valgrind.
- CI/windows: enable SSPI in two jobs.
- CI/windows: disable zlib in one more job.
- CI/windows: improve `if` condition checking GnuTLS.
- CI/windows: ignore SFTP/SCP tests as necessary.
  - universally ignore SCP tests, they fail everywhere.
  - ignore test 612.
  - ignore test 613 616 618 with MSYS2 mingw-w64.
  - ignore test 614 with libssh.
  - ignore all SFTP with MSYS2 native.
  - ignore all SFTP with vcpkg with `libssh2[core,zlib]`.
  - ignore a couple of SFTP tests with MSYS2 mingw-w64.
  (This matches settings on Azure CI.)
- GHA/windows: ignore failing 1451 'Basic SMB request' test for
  old mingw-w64 7.3.0 (but not for 9.5.0!):
  ```
  2024-09-11 21:45:59,738 ERROR smbComNegotiate: b'NT LM 0.12\x00' is not in list
  [...]
  curl: (7) Could not connect to server
  [...]
  FAIL 1451: 'Basic SMB request' SMB
  ```
  Ref: https://github.com/curl/curl/actions/runs/10816280746/job/30007130770#step:13:3546
- CI/Azure: delete, now moved to GHA.

sshserver:

- sshserver: fix permissions for SSH host key on Windows, allowing sshd
  to launch.
- sshserver: fix initializing config paths with Cygwin/MSYS2 sshd.
- sshserver: fix initializing config paths in parallel builds.
- sshserver: delete redundant `DenyUsers`. This also opens the way
  to allow multiple usernames.
- sshserver: fix `AllowUsers` for GHA/windows, by allowing the
  domainless username again.
  Follow-up to 3ee7c676ec #5721
- sshserver: fix `Deprecated option` warnings in `sshd_config`:
  ```
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 7: Deprecated option AuthorizedKeysFile2
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 25: Deprecated option KeyRegenerationInterval
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 35: Deprecated option RhostsRSAAuthentication
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 36: Deprecated option RSAAuthentication
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 37: Deprecated option ServerKeyBits
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 41: Deprecated option UseLogin
  ```
- sshserver: fix `Deprecated option` warnings in `ssh_config` with
  Cygwin/MSYS2.
- sshserver: fix dumping config files due to the filenames missing their
  full paths.
- sshserver: add workaround to make `logmsg` messages visible.
  Before this patch they only went to a file and never shown.

runtests:

- runtests: log details when these Windows commands are called:
  `handle`, `taskkill`, `tasklist`.
- runtests: add documentation links to Windows tools:
  `handle`, `taskkill`, `tasklist`, `icacls`.
- runtests: add `-t` (kill whole tree) option to `taskkill` in
  `servers.pm`, syncing it with the other `taskkill` call.
  Follow-up to bc72a78a11 #14488
- runtests: show warning if Sysinternals `handle` tool is missing.
- runtests: drop Windows XP Home compatibility `tskill` call.
  The call was made on all Windows versions. It's possibly overkill to
  do this, because XP Home is probably rarely used for running curl
  tests these days. In case it's needed, it'd be better to put it under
  an explicit option.
- runtests: show Perl version and path.

Fix/silence fallouts:

- unit2603: fix building with disabled HTTP support.
- unit2604: silence `-Woverlength-strings` warnings in C89 mode.
- test437, test1614: fix to pass with no-IPv6 builds.

Closes #14859
2024-09-16 10:00:17 +02:00
Dan Fandrich
b61d6e088e GHA/linux-old: add an autoconf/automake build
This verifies that using older autotools still works. The results of the
autoconf build should end up being identical to the second of the two
cmake builds (the difference in the platform string notwithstanding).

Closes #14816
2024-09-09 10:19:11 -07:00
Viktor Szakats
6aa5f25c64
GHA/linux-old: split test step into build and run
To see how much time each takes.

Closes #14791
2024-09-05 12:26:03 +02:00
Viktor Szakats
f73f6bf9f8
GHA: add yamlcheck
Uses `yamllint`.

Also:
- fix warning for existing YAML files:
  ```
  [truthy] truthy value should be one of [false, true]
  ```

Closes #14650
2024-08-23 18:42:55 +02:00
Viktor Szakats
7b14449790
cmake: add support for versioned symbols option
Implement the `--enable-versioned-symbols` feature available in
`./configure` for CMake.

Enable with `-DCURL_LIBCURL_VERSIONED_SYMBOLS=ON`. Customize the version
prefix with `-DCURL_LIBCURL_VERSIONED_SYMBOLS_PREFIX=MYPREFIX_`. By
default the prefix matches what autotools uses.

When enabled, the feature is detected and a warning shown if
unavailable. (E.g. on Apple and Windows, it isn't.)

Included `HIDDEN {};` to match autotools, though I don't know if it's
necessary, useful or making any difference.

Differences from the autotools implementation:
- soversion is dynamic instead of hard-coded.
- omits referencing non-curl symbols.
- allows prefix/flavour override.
- more universal feature detection.
- doesn't rely on the in-repo `lib/libcurl.vers.in` file.

Also:
- add mbedTLS and BearSSL versioned symbol prefix support to autotools.
- enable this option in an old-linux job.

Follow-up to 7cc2e8b349
Fixes #14349
Closes #14378
2024-08-05 01:44:57 +02:00
Viktor Szakats
e8acb2e55a
cmake: detect libssh via pkg-config
Also:
- fix broken libssh `pkg-config` behaviour on old Linux.
  (when found, `LIBSSH_LINK_LIBRARIES` remains undefined.)

- delete manual libssh config from Old Linux CI job,
  it's no longer necessary.

Closes #14199
2024-07-29 21:08:17 +02:00
Viktor Szakats
efce544418
GHA/non-native: improve, migrate x86_64 FreeBSD with tests from Cirrus CI
- run tests via `make test-ci` instead of `make check` with autotools.
- add `x86_64` job for FreeBSD, with tests.
  It matches the existing Cirrus CI job, with these differences:
  - finishes 3x faster (thanks to parallel tests enabled).
  - librtmp is not enabled because it's slated for removal by FreeBSD.
    (already past the removal deadline, thought the package still
    installs.)
  - DICT and TELNET servers fail to start. Couldn't figure out why.
    It means skipping test 1450 and 1452.
  - it runs more tests, e.g. websockets and ip6-localhost.
  - no `pkg update -f`.
  - it misses the `CRYPTOGRAPHY_DONT_BUILD_RUST=1`, `pkg delete curl`,
    `chmod 777`, `sudo -u nobody` and `sysctl net.inet.tcp.blackhole`
    tricks. The latter is the default in these runners, the others did
    not affect results.
- set `-j0` for tests in the NetBSD job. Flaky otherwise.

Closes #14244
2024-07-29 20:46:40 +02:00
Dan Fandrich
b9fea261e6 CI: bump the libc6 on the linux-old build
This contains some security fixes for nscd.
2024-07-05 20:22:26 -07:00
Dan Fandrich
ebd61fa34e CI: Fix typo in comment 2024-07-05 14:45:41 -07:00
renovate[bot]
8b368fa3c9
GHA: update pinned actions
- github/codeql-action digest to 23acc5c
- actions/checkout digest to 692973e
- rojopolis/spellcheck-github-actions digest to d354a4d

Closes #13935
Closes #13945
Closes #13946
2024-06-14 09:36:22 +02:00
Viktor Szakats
464282ddfb
GHA: bump all build jobs to nproc+1
- bump rest of the workflows (windows, macos, distrocheck).

- non-native virtualized envs have 2 CPUs, bump down accordingly.
  (for `vmactions/omnios-vm` it's just a guess.)

- bump all to nproc + 1.

Follow-up to e838b341a0 #12927
Closes #13807
2024-06-02 21:39:49 +02:00
dependabot[bot]
ac68a2dd85
GHA: bump actions/checkout from 4.1.4 to 4.1.6
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.4 to 4.1.6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](0ad4b8fada...a5ac7e51b4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Closes #13720
2024-05-22 09:14:10 +02:00
renovate[bot]
7398037abe
GHA: pin dependencies
Closes #13628
2024-05-15 13:35:45 +02:00
Stefan Eissing
e1cf21b5c2
GHA: repair the linux-old job
package libc6_2.28-10+deb10u2_amd64.deb changed to
libc6_2.28-10+deb10u3_amd64.deb

Closes #13564
2024-05-08 23:40:33 +02:00
Viktor Szakats
0469c68f49
ci: add CMake build variation, fixup libssh detection in linux-old
To test without c-ares and hit `easy_lock.h` on an old system. Use this
new build step to introduce small variations, and also test libssh2.

Also add workaround to existing job to enable libssh. (CMake's generic
auto-detection doesn't seem to work here.):
```
CMake Warning at CMakeLists.txt:908 (find_package):
  Could not find a package configuration file provided by "libssh" with any
  of the following names:

    libsshConfig.cmake
    libssh-config.cmake
```
Ref: https://github.com/curl/curl/actions/runs/8661316091/job/23750974358#step:5:69

Closes #13361
2024-04-14 02:30:40 +00:00
Viktor Szakats
f43545e91b
cmake: fix CURL_WERROR=ON for old CMake and use it in GHA/linux-old
- cmake: fix `-pedantic-errors` for old CMake with `CURL_WERROR=ON` set.

  `-pedantic-errors` option throws a warning with GCC (all versions) and
  makes `check_symbol_exists()` fail in CMake versions older than
  v3.23.0 (2022-03-29), when CMake introduced a workaround:

  https://gitlab.kitware.com/cmake/cmake/-/issues/13208
  eeb4540116
  1ab7c3cd28

  Follow-up to 3829759bd0 #12489

- set `CURL_WERROR=ON` for the `linux-old` job in CI.

Closes #13282
2024-04-05 13:05:19 +00:00
Viktor Szakats
b6f2e33118
GHA: enable unity mode for cmake jobs + tidy-ups
Unity mode is not supported by CMake v3.7.2 used in linux-old, but
enable it anyway for consistency and to kick in automatically once
migrating to a newer old Linux in the future.

Also:
- replace `CMAKE_COMPILE_WARNING_AS_ERROR` with `CURL_WERROR`.
- delete default build option `PICKY_COMPILER=ON`.

Closes #13277
2024-04-04 22:58:50 +00:00
Dan Fandrich
376cd67035 CI: Add CI build on Debian stretch to test old support
This version still has ELTS support and contains some old versions of
key components like cmake to help prevent us from breaking that support.

Closes #13029
2024-04-04 09:23:15 -07:00