Commit Graph

150 Commits

Author SHA1 Message Date
Viktor Szakats
59f4727480
appveyor: run VS2008 job with old CMake 3.12.2
Switch VS2008 job the oldest runner machine. It adds the oldest CMake to
the Windows mix, from 2018-11-30. Not a beauty, missing support for `-B`
and Unity, but it's a version curl supports. It's newer than Old Linux.
The previous oldest was 3.16.2. It remains used with VS2010-VS2017.

Also:
- fix VS2008 job to actually build examples.
- switch VS2019 job to OpenSSL 1.1.0 that wasn't tested before.
  Migrate OpenSSL 1.0.2 to the VS2008 job.
- measure run time of individual build steps.

Follow-up to 01c25e3b00 #16458
Closes #16505
2025-02-27 12:11:08 +01:00
Viktor Szakats
01c25e3b00
CI: misc improvements, restore VS2008 job
- appveyor: restore VS2008 job, after fixing its issues.
  Enable OpenSSL in it. It takes 1 minute.
  Follow-up to 9b0467b169 #16453
  Follow-up to edfa537100 #16456
- appveyor: make a copy of OpenSSL DLLs to have them picked up as an
  artifact (disabled by default) to aid local tests.
- appveyor: dump CMake configuration logs on failure.
- appveyor: tidy up job parameter defaults.
- GHA/windows: add pre-fill check option for dl-mingw jobs.
- GHA/windows: fix pre-fill check option for MSYS jobs by installing
  `diffutils`.
  Follow-up to e7adf3e837 #15841
- GHA/windows: de-duplicate to `PATH` commands for Cygwin.
- GHA/windows: drop `$SYSTEMROOT/System32` from `PATH` for Cygwin
  configure. It's not needed.
  Follow-up to 36fd2dd6ee #13599
- list `.pdb` files in curl version step for MSVC.
  Ref: #16439

Cherry-picked from #16394
Closes #16458
2025-02-24 22:47:37 +01:00
Viktor Szakats
0c933081fb
GHA/appveyor: add VS2019 job, merge two others, fix build fallout
- add VS2019 job, with Schannel + OpenSSL 1.0.2.
  First MultiSSL job here and add the last missing modern VS version.

- fix builds with mixed ALPN capabilities in MultiSSL unity builds.
  Caused by reusing `HAS_ALPN` between TLS modules without
  resetting it. Fix it by using unique names for each backend.

- merge a VS2010 job into a VS2012. With MultiSSL and x86 OpenSSL.

- make a job static.

- fix `Shared`/`Static` in a job name.

- add `Shared` to job names.

Closes #16231
2025-02-07 10:52:56 +01:00
Viktor Szakats
5fd7bd4379
appveyor: bump to OpenSSL 3.4
Closes #16064
2025-01-21 12:42:33 +01:00
Viktor Szakats
55f3ba447f
CI: add/extend curl binary info, VS2010 32-bit, misc improvements
- fix `find` commands to not miss items.

- call `file` on the built files in `curl -V` steps.
  To give more feedback on what was built.

- add `curl info` step for cross-jobs that can't do a `curl -V`.
  It lists the files built and calls `file` on them.

- appveyor: make a VS2010 32-bit to match the VS2008 job it replaced.
  Follow-up to d34aeecb08 #15934

- GHA/windows: drop the word "old" from standalone mingw-w64 jobs to not
  conflate it with "old mingw" we no longer support (while also keeping
  it short).

Cherry-picked from #15975
Closes #16001
2025-01-14 17:46:49 +01:00
Viktor Szakats
d34aeecb08
appveyor: bump VS2008 jobs to VS2010
VS2008 has been partly broken for a while with its shared-debug builds
crashing on startup. Its compiler output (UTF-16 HTML) was also barely
readable even after conversion. It's also the only platform in CI
missing `stdint.h`.

This patch migrates a VS2008 job to VS2010 and drops another that
already had a VS2010 equivalent.

We recommend switching to VS2010 or newer when using MSVC to build curl.

Ref: #15907
Closes #15934
2025-01-08 12:33:31 +01:00
Viktor Szakats
38ff7ba501
appveyor: always use cmake -A option to select x64
The `Win64` generator suffix alternative was required by old CMake
versions (<3.1) only:
https://cmake.org/cmake/help/v3.22/generator/Visual%20Studio%2010%202010.html

Closes #15935
2025-01-08 00:33:02 +01:00
Viktor Szakats
cba0d3abcb
appveyor: reorder builds to get useful results earlier
Also to align with existing VS2010. Keeping the VS2008 job first to give
a quick sniff test for MSVC builds.

Follow-up to 08ff33e483 #15923
Follow-up to 50f6a6b1d4 #15926

Closes #15932
2025-01-07 20:14:03 +01:00
Viktor Szakats
554d4aee4d
appveyor: fixup job name
Follow-up to 50f6a6b1d4 #15926
2025-01-07 15:02:22 +01:00
Viktor Szakats
50f6a6b1d4
msvc: add VS2012 CI job, fix compiler warnings in LDAP and tests
Follow-up to 8cfc936f5c #10674
Follow-up to d620ec6791 #14700

Closes #15926
2025-01-07 14:51:48 +01:00
Viktor Szakats
08ff33e483
msvc: fix building with HAVE_INET_NTOP and MSVC <=1900
MSVC 1900 and older is missing a `const` specifier in the `inet_ntop()`
declaration for the second argument. A workaround was in place for it
in cmake, but it didn't cover all necessary versions.

Replace the workaround with a different one, move it to `lib/inet_ntop.c`
and extend to all necessary MSVC versions.

Also add CI jobs for the older MSVC versions: 2013, 2015, 2017.

Closes #15923
2025-01-07 14:07:19 +01:00
Viktor Szakats
fb711b5098
build: fix clang-cl builds, add CI job
- appveyor: add build-only job for clang-cl.

- cmake: `-pedantic-errors` enables `-Werror,-Wlanguage-extension-token`
  automatically, which makes `__int64` detection fail.
  Explictly disable this compiler warning for clang-cl to make the
  feature detection work and to accept `__int64` in the source code.

- cmake: disable `-Wlanguage-extension-token` warning for clang-cl
  to fix these when encountering `__int64`:
  ```
  lib/formdata.c(797,29): error : extension used [-Werror,-Wlanguage-extension-token]
  lib/warnless.c(117,33): error : extension used [-Werror,-Wlanguage-extension-token]
  lib/warnless.c(60,28): message : expanded from macro 'CURL_MASK_SCOFFT'
  lib/warnless.c(59,38): message : expanded from macro 'CURL_MASK_UCOFFT'
  include\curl/system.h(352,40): message : expanded from macro 'CURL_TYPEOF_CURL_OFF_T'
  ```

- make `__GNUC__` warning suppressions apply to `__clang__` too.
  Necessary for clang-cl, which defines the latter, but not the former.
  (Regular clang defines both.)

- examples: fix clang-cl compiler warning in `http2-upload.c`.
  ```
  docs\examples\http2-upload.c(56,5): error : no previous prototype for function 'my_gettimeofday' [-Werror,-Wmissing-prototypes]
  docs\examples\http2-upload.c(56,1): message : declare 'static' if the function is not intended to be used outside of this translation unit
  ```

- unit2604: add missing `#pragma GCC diagnostic pop`.
  Follow-up to e53523fef0 #14859

- unit1652: limit compiler warning suppression to GCC.
  They do not affect clang builds.
  Follow-up to 71cf0d1fca #14772

Closes #15449
2024-10-30 23:15:32 +01:00
Viktor Szakats
398025a41f
appveyor: fix job names, tidy-up
- add 'Shared' to job names where missing.
- dedupe setting the default `HTTP_ONLY` env.
- fix typo in job name.

Cherry-picked from #15414
Closes #15422
2024-10-27 14:26:45 +01:00
Viktor Szakats
7bc653a1d6
appveyor: bump to OpenSSL 3.3
Closes #15085
2024-09-30 10:54:46 +02:00
Viktor Szakats
069a9654ca
appveyor: delete unused WebSockets option remains [ci skip] 2024-09-30 10:37:00 +02:00
Viktor Szakats
71cf0d1fca
tests: speed up builds with single-binary test bundles
Add support for single-block binaries that contain all libtests and
unit tests respectively.

Enable with:
- autotools: `--enable-test-bundles`
- cmake: `-DCURL_TEST_BUNDLES=ON`

(They are compatible with `--enable-unity` and `-DCMAKE_UNITY_BUILD=ON`
options, for further speed-up.)

Makes libtests and unit tests build _fast_, needing little disk space
even in static mode. Similar to CMake unity mode, but with a custom
script, also supporting autotools builds.

The price is having to deal with symbols/macros colliding between
`lib*.c` and `unit*.c` sources. Maybe with naming conventions or other
solutions this can be improved gradually and reduce the need for manual
intervention by `mk-bundle.mk`. I've included a script that does the bulk
of detecting name collisions.

Also:
- CI: enable test bundles.
- CI: build tests in more jobs.
- lib2305: fix FILE handle leak.
- unit1661: fix memleak found by torture test by releasing the `bufref`
  structure in `unit_stop()` that was allocated in `unit_setup()`.
  ```
  test 1661...[bufref unit tests]
  Leak detected: memory still allocated: 13 bytes
   allocated by /home/runner/work/curl/curl/tests/unit/unit1661.c:70
   1661: torture FAILED: function number 1 in test.
  ```
  Ref: https://github.com/curl/curl/actions/runs/10967279334/job/30456745290?pr=14772#step:8:41

Similar test suite builds with autotools default and cmake+bundle+unity:
- GHA/Linux: 33s vs 7s
  https://github.com/curl/curl/actions/runs/10705668823/job/29681617374
- GHA/macOS 34s vs 2s
  https://github.com/curl/curl/actions/runs/10705668813/job/29681632885
- GHA/FreeBSD: 15m25 vs 6m21 (full workflow time, ~qemu)
  https://github.com/curl/curl/actions/runs/10705668811/job/29681607915
- GHA/Cygwin: 9m52 vs 32s
  https://github.com/curl/curl/actions/runs/10705668809/job/29681609965
- GHA/MSYS2: 3m52 vs 14s
  https://github.com/curl/curl/actions/runs/10705668808/job/29681624295
- GHA/mingw-w64: 5m45 vs 30s
  https://github.com/curl/curl/actions/runs/10705668808/job/29681628787

Autotools test suite builds compared between master -> `--enable-test-bundles`:
- GHA/Linux: 33s -> 9s (run tests: 22m23 -> 20m44)
  https://github.com/curl/curl/actions/runs/10710030193/job/29695932185
  https://github.com/curl/curl/actions/runs/10967831456/job/30458220344
- GHA/macOS: 25s -> 4s (run tests: 2m58 -> 2m24)
  https://github.com/curl/curl/actions/runs/10710030195/job/29695938444
  https://github.com/curl/curl/actions/runs/10967831452/job/30458225762
- GHA/non-native (FreeBSD): 4m8 -> 3m12 (full workflow time, ~qemu)
  https://github.com/curl/curl/actions/runs/10710030198/job/29695928401
  https://github.com/curl/curl/actions/runs/10967831458/job/30458212692
- GHA/Cygwin: 9m25 -> 1m9 (run tests: 9m19 -> 3m28)
  https://github.com/curl/curl/actions/runs/10710030212/job/29695928213
  https://github.com/curl/curl/actions/runs/10967831453/job/30458213268
- GHA/MSYS2: 3m54 -> 32s (run tests: 6m3 -> 3m59)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704850591
  https://github.com/curl/curl/actions/runs/10967831449/job/30459280005
- GHA/mingw-w64: 5m42 -> 1m5 (run tests: 7m41 -> 5m36)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704852058
  https://github.com/curl/curl/actions/runs/10967831449/job/30459280862
- Azure MSYS2 mingw64 openssl: 38m55 -> 11m58
  https://dev.azure.com/daniel0244/curl/_build/results?buildId=25546&view=logs&j=b58b8c59-0f61-52e9-0f9e-fad562a1e77f&t=0f9230a7-3b10-53ca-9938-700ece377c5e
  https://dev.azure.com/daniel0244/curl/_build/results?buildId=25547&view=logs&jobId=39473db1-3945-55d5-deb5-c218fad88dce&j=b58b8c59-0f61-52e9-0f9e-fad562a1e77f&t=0f9230a7-3b10-53ca-9938-700ece377c5e
- Azure Ubuntu default: 2m15 -> 55s (all build)
  https://dev.azure.com/daniel0244/curl/_build/results?buildId=25546&view=logs&j=9d58b9ac-e1e6-53b6-f83a-1f9f1d912522&t=a6b38d83-e7cf-5a9b-c762-a178412717b7
  https://dev.azure.com/daniel0244/curl/_build/results?buildId=25547&view=logs&jobId=39473db1-3945-55d5-deb5-c218fad88dce&j=9d58b9ac-e1e6-53b6-f83a-1f9f1d912522&t=a6b38d83-e7cf-5a9b-c762-a178412717b7

Cmake test suite builds compared between master -> `-DCURL_TEST_BUNDLES=ON` + unity:
- GHA/Linux: 29s -> 7s (run tests: 4m50 -> 4m57, 20m43 -> 20m45)
  https://github.com/curl/curl/actions/runs/10710030193/job/29695941814
  https://github.com/curl/curl/actions/runs/10705668823/job/29681622201
- GHA/Linux old: 44s -> 13s (bundle+no unity) (run tests: 5m5 -> 5m6)
  https://github.com/curl/curl/actions/runs/10718264094/job/29719794727
  https://github.com/curl/curl/actions/runs/10718653175/job/29721009613
- GHA/macOS: 32s -> 2s (run tests: 2m43 -> 2m40)
  https://github.com/curl/curl/actions/runs/10710030195/job/29695931956
  https://github.com/curl/curl/actions/runs/10705668813/job/29681638937
- GHA/non-native (*BSD): inconclusive (full workflow time, ~qemu)
  https://github.com/curl/curl/actions/runs/10710030198
  https://github.com/curl/curl/actions/runs/10705668811
- GHA/Cygwin: 3m9 -> 32s
  https://github.com/curl/curl/actions/runs/10710030212/job/29695929075
  https://github.com/curl/curl/actions/runs/10705668809/job/29681609965
- GHA/MSYS2: 2m24 -> 14s
  https://github.com/curl/curl/actions/runs/10710030190/job/29704850996
  https://github.com/curl/curl/actions/runs/10705668808/job/29681624295
- GHA/mingw-w64: 3m56 -> 30s (run tests: 4m2 -> 3m52)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704852219
  https://github.com/curl/curl/actions/runs/10705668808/job/29681631393
- GHA/mingw-w64-old: 7m19 -> 1m44 (run tests: 3m30 -> 2m53)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704849763
  https://github.com/curl/curl/actions/runs/10705668808/job/29681622329
- GHA/MSVC: 3m22 -> 13s (run tests: 9m43 -> 4m22)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704850411
  https://github.com/curl/curl/actions/runs/10705668808/job/29681623313
- AppVeyor CI MSVC 2008: 4m3 -> 45s (full build)
- AppVeyor CI MSVC 2010: 2m56 -> 1m8 (full build)
- AppVeyor CI MSVC 2022: 10m19 -> 2m23 (full build)
  https://ci.appveyor.com/project/curlorg/curl/builds/50538455
  https://ci.appveyor.com/project/curlorg/curl/builds/50536558
- AppVeyor CI total build time: 10m30 (master) -> 6m48 (unity) -> 4m5 (bundle) -> 3m24 (bundle+unity) -> 5m7 (bundle+unity+all jobs building tests)

Closes #14772
2024-09-22 09:51:15 +02:00
Viktor Szakats
45202cbba4
cmake: separate target for examples, optimize CI, fix fallouts
- Move `docs/examples` builds under a separate target.

- Make `BUILD_EXAMPLES` default to `ON`. It means to generate the rules
  for `docs/examples` by default, but not build them. To build them,
  an explicit `make curl-examples` (or ninja, etc) command is necessary.
  This syncs behaviour with autotools, and also how both cmake and
  autotools are building tests.

- GHA: update cmake jobs to use the new way of building examples.

- GHA: move examples build step at the end of the job, after building
  and running tests. This allows to have build and test run results
  faster, and leave the seldom-changing examples build to the end.
  Building examples is the slowest build step with no practical way to
  make them fast.

- appveyor: enable building examples in two old-MSVC jobs.

- examples: fix examples to build cleanly with old MSVC versions.

- GHA/non-native: move example build log under a GHA foldable section.

- GHA/windows: move building examples into separate step for Linux cross
  jobs.

Follow-up to dfdd978f7c #13491
Closes #14906
2024-09-20 23:53:33 +02:00
Viktor Szakats
e53523fef0
CI: move Azure jobs to GHA, fix fallouts, sshserver, runtests tweaks
CI:

- GHA/windows: enable OpenSSH server, SysInternals `handle`, `impacket`.
  Skip `impacket` on MSYS2 due to install failure.
  Skip OpenSSH server for old/standalone mingw-w64 (building curl
  without SSH there.)
- GHA/windows: make test tool installs a separate step.
- GHA/cygwin: enable OpenSSH server.
  Skip `impacket`: it's compiling for 7 minutes then breaks.
  Skip `stunnel` due to sluggish test run performance.
  (This update is unrelated to Azure jobs.)
- GHA/linux: migrate Linux jobs from Azure CI.
- GHA/linux: migrate scanbuild job from Azure CI.
- GHA/linux: enable libssh2 in a job. Also enable valgrind.
- CI/windows: enable SSPI in two jobs.
- CI/windows: disable zlib in one more job.
- CI/windows: improve `if` condition checking GnuTLS.
- CI/windows: ignore SFTP/SCP tests as necessary.
  - universally ignore SCP tests, they fail everywhere.
  - ignore test 612.
  - ignore test 613 616 618 with MSYS2 mingw-w64.
  - ignore test 614 with libssh.
  - ignore all SFTP with MSYS2 native.
  - ignore all SFTP with vcpkg with `libssh2[core,zlib]`.
  - ignore a couple of SFTP tests with MSYS2 mingw-w64.
  (This matches settings on Azure CI.)
- GHA/windows: ignore failing 1451 'Basic SMB request' test for
  old mingw-w64 7.3.0 (but not for 9.5.0!):
  ```
  2024-09-11 21:45:59,738 ERROR smbComNegotiate: b'NT LM 0.12\x00' is not in list
  [...]
  curl: (7) Could not connect to server
  [...]
  FAIL 1451: 'Basic SMB request' SMB
  ```
  Ref: https://github.com/curl/curl/actions/runs/10816280746/job/30007130770#step:13:3546
- CI/Azure: delete, now moved to GHA.

sshserver:

- sshserver: fix permissions for SSH host key on Windows, allowing sshd
  to launch.
- sshserver: fix initializing config paths with Cygwin/MSYS2 sshd.
- sshserver: fix initializing config paths in parallel builds.
- sshserver: delete redundant `DenyUsers`. This also opens the way
  to allow multiple usernames.
- sshserver: fix `AllowUsers` for GHA/windows, by allowing the
  domainless username again.
  Follow-up to 3ee7c676ec #5721
- sshserver: fix `Deprecated option` warnings in `sshd_config`:
  ```
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 7: Deprecated option AuthorizedKeysFile2
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 25: Deprecated option KeyRegenerationInterval
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 35: Deprecated option RhostsRSAAuthentication
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 36: Deprecated option RSAAuthentication
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 37: Deprecated option ServerKeyBits
  D:/a/curl/curl/bld/tests/log/2/server/curl_sshd_config line 41: Deprecated option UseLogin
  ```
- sshserver: fix `Deprecated option` warnings in `ssh_config` with
  Cygwin/MSYS2.
- sshserver: fix dumping config files due to the filenames missing their
  full paths.
- sshserver: add workaround to make `logmsg` messages visible.
  Before this patch they only went to a file and never shown.

runtests:

- runtests: log details when these Windows commands are called:
  `handle`, `taskkill`, `tasklist`.
- runtests: add documentation links to Windows tools:
  `handle`, `taskkill`, `tasklist`, `icacls`.
- runtests: add `-t` (kill whole tree) option to `taskkill` in
  `servers.pm`, syncing it with the other `taskkill` call.
  Follow-up to bc72a78a11 #14488
- runtests: show warning if Sysinternals `handle` tool is missing.
- runtests: drop Windows XP Home compatibility `tskill` call.
  The call was made on all Windows versions. It's possibly overkill to
  do this, because XP Home is probably rarely used for running curl
  tests these days. In case it's needed, it'd be better to put it under
  an explicit option.
- runtests: show Perl version and path.

Fix/silence fallouts:

- unit2603: fix building with disabled HTTP support.
- unit2604: silence `-Woverlength-strings` warnings in C89 mode.
- test437, test1614: fix to pass with no-IPv6 builds.

Closes #14859
2024-09-16 10:00:17 +02:00
Viktor Szakats
b0a1c9bdca
CI: update names of jobs that are now building tests [ci skip]
Follow-up to d620ec6791 #14700
2024-09-06 17:14:29 +02:00
Viktor Szakats
d620ec6791
CI: add test timeouts, more cmake build tests, fix VS2010 C warning
- GHA/macos: set timeout for test runs.
  Double the value for autotools to fit torture tests.
- GHA/cygwin: reduce test run timeout for autotools.
- GHA/cygwin: enable building tests with cmake.
- GHA/windows: enable building tests with MSYS cmake.
- GHA/windows: enable building tests with MSVC UWP.
- appveyor: enable building tests with VS2008 x86 and VS2010 x64.
- tests: add workaround compiler warnings when building with VS2010:
  ```
  tests\server\util.c(482): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(486): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(490): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(720): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(726): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(732): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(781): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(785): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(789): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  ```
  Ref: https://ci.appveyor.com/project/curlorg/curl/builds/50485633/job/4ujlwxod3cexmn2q#L1535

With this, tests are built in all GHA cygwin/windows jobs.

Timeouts avoid long runs with runaway issues, example:
https://github.com/curl/curl/actions/runs/10575522173

Closes #14700
2024-08-27 23:40:48 +02:00
Viktor Szakats
c57d3aeb55
appveyor: drop uploading artifacts
Uploading artifacts sometimes results in this error:
```
Uploading artifacts...
[1/1] _bld\src\curl.exe (2,022,912 bytes)...100%
Error uploading artifact to the storage: Remote server returned 503: Service Temporarily Unavailable
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/50424126/job/e4envval6xkicv1i#L123

The artifacts are also probably not useful to upload for every run. Also
note that they were missing external DLL dependencies.

Leave the logic there commented, to make it easy to enable as needed for
debugging or testing artifacts locally.

Closes #14581
2024-08-18 12:22:53 +02:00
Viktor Szakats
efce544418
GHA/non-native: improve, migrate x86_64 FreeBSD with tests from Cirrus CI
- run tests via `make test-ci` instead of `make check` with autotools.
- add `x86_64` job for FreeBSD, with tests.
  It matches the existing Cirrus CI job, with these differences:
  - finishes 3x faster (thanks to parallel tests enabled).
  - librtmp is not enabled because it's slated for removal by FreeBSD.
    (already past the removal deadline, thought the package still
    installs.)
  - DICT and TELNET servers fail to start. Couldn't figure out why.
    It means skipping test 1450 and 1452.
  - it runs more tests, e.g. websockets and ip6-localhost.
  - no `pkg update -f`.
  - it misses the `CRYPTOGRAPHY_DONT_BUILD_RUST=1`, `pkg delete curl`,
    `chmod 777`, `sudo -u nobody` and `sysctl net.inet.tcp.blackhole`
    tricks. The latter is the default in these runners, the others did
    not affect results.
- set `-j0` for tests in the NetBSD job. Flaky otherwise.

Closes #14244
2024-07-29 20:46:40 +02:00
Viktor Szakats
59dc9f7e69
build: untangle CURLDEBUG and DEBUGBUILD macros
`CURLDEBUG` is meant to enable memory tracking, but in a bunch of cases,
it was protecting debug features that were supposed to be guarded with
`DEBUGBUILD`.

Replace these uses with `DEBUGBUILD`.

This leaves `CURLDEBUG` uses solely for its intended  purpose: to enable
the memory tracking debug feature.

Also:
- autotools: rely on `DEBUGBUILD` to enable `checksrc`.
  Instead of `CURLDEBUG`, which worked in most cases because debug
  builds enable `CURLDEBUG` by default, but it's not accurate.
- include `lib/easyif.h` instead of keeping a copy of a declaration.
- add CI test jobs for the build issues discovered.

Ref: https://github.com/curl/curl/pull/13694#issuecomment-2120311894
Closes #13718
2024-05-28 08:12:00 +02:00
Viktor Szakats
f709a45c57
appveyor: fixup job name [ci skip]
Follow-up to fc8e0dee30 #13694
2024-05-27 23:22:42 +02:00
Viktor Szakats
fc8e0dee30
build: untangle UNITTESTS and DEBUGBUILD macros
- fix `DEBUGBUILD` guards that should be `UNITTESTS`, in libcurl code
  used by unit tests.
- fix guards for libcurl functions used in unit tests only.
- sync `UNITTEST` attribute between declarations and definitions.
- drop `DEBUGBUILD` guard from test `unit2600`.
- fix guards for libcurl HSTS code used by both a unit test (`unit1660`)
  and `test0446`.
- update an existing AppVeyor CI job to test the issues fixed.

This fixes building tests with `CURLDEBUG` enabled but `DEBUGBUILD`
disabled. This can happen when building tests with CMake with
`ENABLE_DEBUG=ON` in Release config, or with `ENABLE_CURLDEBUG=ON`
and _without_ `ENABLE_DEBUG=ON`. Possibly also with autotools
when using `--enable-curldebug` without `--enable-debug`.

Test results:
- before:
  https://ci.appveyor.com/project/curlorg/curl/builds/49835609
  https://ci.appveyor.com/project/curlorg/curl/builds/49898529/job/k8qpbs8idby70smw
  https://github.com/curl/curl/actions/runs/9259078835/job/25470318167?pr=13798#step:13:821
- after: https://ci.appveyor.com/project/curlorg/curl/builds/49839255
  (the two failures are unrelated, subject to PR #13705)

Ref: #13592 (issue discovery)
Ref: #13689 (CI testing this PR with `DEBUGBUILD`/`CURLDEBUG` combinations)
Closes #13694
2024-05-27 21:15:50 +02:00
Viktor Szakats
c29a20d465
CI: tidy up skipping tests build/run in Windows jobs
Simplify controlling whether to build and/run tests in a CI job.

Apply the TFLAGS='skipall' (do not build nor run tests) or
'skiprun' (build, but do not run) method already used with old-mingw-w64
and msvc jobs to existing Windows jobs in GHA and AppVeyor.

Also:
- add Cygwin/cmake test build and run steps while here.
- replace `DISABLED_TESTS` with `TFLAGS` in AppVeyor.

Closes #13796
2024-05-27 19:26:51 +02:00
Viktor Szakats
ea98445680
cmake: ENABLE_DEBUG=ON to always set -DDEBUGBUILD
Before this patch `ENABLE_DEBUG=ON` always enabled the TrackMemory
(aka `ENABLE_CURLDEBUG=ON`) feature, but required the `Debug` CMake
configration to actually enable curl debug features
(aka `-DDEBUGBUILD`).

Curl debug features do not require compiling with C debug options. This
also made enabling debug features unintuitive and complicated to use.
Due to other issues (subject to PR #13694) it also caused an error in
default (and `Release`/`MinSizeRel`/`RelWithDebInfo`) configs, when
building the `testdeps` target:
```
ld: CMakeFiles/unit1395.dir/unit1395.c.o: in function `test':
unit1395.c:(.text+0x1a0): undefined reference to `dedotdotify'
```
Ref: https://github.com/curl/curl/actions/runs/9037287098/job/24835990826#step:3:2483

Fix it by always defining `DEBUGBUILD` when setting `ENABLE_DEBUG=ON`.
Decoupling this option from the selected CMake configuration.

Note that after this patch `ENABLE_DEBUG=ON` unconditionally enables
curl debug features. These features are insecure and unsuited for
production. Make sure to omit this option when building for production
in default, `Release` (and other not-`Debug`) modes.

Also delete a workaround no longer necessary in GHA CI jobs.

Ref: 1a62b6e68c (2015-03-03)
Ref: #13583
Closes #13592
2024-05-27 17:57:52 +02:00
Viktor Szakats
c8f61dbba6
GHA: add three MSVC jobs
Continuing the theme, add 3 MSVC jobs with tests, matching
configurations used on AppVeyor. MSVC versions are identical:
19.39.33523.0 + Windows SDK 10.0.22621.0.

Also enable websockets, and build examples. Tests are run in parallel
(`-j14`), with improved performance.

Job performance:
```
                                                       AppVeyor  GHA
                                                                 w/examples
                                                       --------  ----------
CMake, VS2022, Debug, x64, Schannel, Static, Unicode   38m 4s    11m57s
CMake, VS2022, Debug, x64, no SSL, Static              35m15s    12m 6s
CMake, VS2022, Debug, x64, no SSL, Static, HTTP only   25m25s    10m36s
```
Based on these runs:
https://ci.appveyor.com/project/curlorg/curl/builds/49884748
https://github.com/curl/curl/actions/runs/9229448468

This is the first time examples are built in CI with MSVC: Fix all
warnings and errors that came up via
d4b8589055 #13771.

Closes #13766
2024-05-27 13:32:28 +02:00
Viktor Szakats
0914d8aadd
GHA: add three old (gcc 6, 7, 9) mingw-w64 jobs
Re-implement old mingw-w64 jobs in GHA. This allows to use the latest
Windows runners, replacing Windows Server 2012 R2 (gcc 6) and Windows
Server 2016 (gcc 7, 9) with Windows Server 2022.

GHA runners are also significantly faster, and allow running tests in
parallel (`-j14`). It also offloads 3 more long-running jobs from
AppVeyor CI.

These jobs download (then cache) the mingw-w64 packages from their
original location, which allows flexibility in choosing which versions
and flavours (win32/POSIX, SEH/DWARF, 64/32-bit) we want to test in CI.
The new jobs use these distros:
- https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/ (for gcc 7, same as on AppVeyor)
- https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/ (for gcc 6, same as on AppVeyor)
- https://winlibs.com/ (for gcc 9)

I matched existing AppVeyor job configs, with these differences:
- gcc 6.4.0 instead of 6.3.0.
  (same distro as on AppVeyor, but the latest bugfix release)
- gcc 9.5.0 instead of 9.1.0 and a different (but compatible) binary distro.
  (in AppVeyor this relies on an old MSYS2 pre-installed on the runner)
- using win32 builds instead of posix for gcc 6.4.0 and 7.3.0.
- websockets enabled.
- always build examples.
- always build tests (this wasn't done for 6.4.0 with AppVeyor CI).

I did not replicate existing test exclusions, and oddly enough the few
failures (so far) were different from MSYS2 jobs and also from their
AppVeyor CI counterparts.

Also:
- delete redundant (default) `-u` option from `cygpath` calls.
- allow matrix options to override default ones in CMake.
- detect and use Windows-supplied curl for `TFLAGS` `-ac` option.
  (it's available in modern runners.)
- delete the 3 AppVeyor CI jobs now replicated in GHA.
- appveyor: prefer `SYSTEMROOT` over `WINDIR`.
- tidy-up quotes.

Job performance:
```
                                                                 AppVeyor  GHA
                                                                           w/examples
                                                                           w/tests
                                                                 --------  ----------
CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static, no-unity   1m25s     8m50s
CMake, mingw-w64, gcc 7, Debug, x64, Schannel, Static, Unicode   31m45s     9m39s
CMake, mingw-w64, gcc 9, Debug, x64, Schannel, Static            28m25s    13m38s
```
Based on these runs:
https://ci.appveyor.com/project/curlorg/curl/builds/49880799
https://github.com/curl/curl/actions/runs/9218292508

Notice that building examples and tests is time consuming.

We can tweak any build parameter as necessary to make them more useful
and/or without clogging the job queue or introducing flakiness.

Closes #13759
2024-05-27 12:15:41 +02:00
Viktor Szakats
36fd2dd6ee
GHA: add MSYS, mingw-w64, Cygwin jobs
- re-implement autotools MSYS and Cygwin AppVeyor jobs in GHA.
  Now build with SSL and PSL to improve test coverage.
- re-implement MSYS2 mingw-w64 gcc 13 AppVeyor job in GHA.
  `CMake, mingw-w64, gcc 13, Debug, x64, Schannel, Static, Unicode`
- add new cmake Cygwin job (build-only).
- enable `-j14` parallelism when running tests.
- delete the 5 migrated jobs from AppVeyor CI.
- add 2 build-only mingw-w64 builds, gcc Release and clang OpenSSL.
- also enable brotli, libssh2, nghttp2 for more test coverage.

These jobs offer better performance, more flexibility and
parallelization compared to the AppVeyor ones they replace. It also
offloads AppVeyor, allowing to iterate faster. They also appear more
reliable than e.g. Azure Windows jobs, where runners are prone to fail
[1].

Closes #13599

[1]:
`Exit code 143 returned from process: file name 'C:\Windows\system32\docker.EXE',
arguments 'exec -i   6b13a669c6dfe7fb9f59414369872fd64d61c7182f880c3d39c135cb4c115c8f
C:\__a\externals\node\bin\node.exe C:\__w\_temp\containerHandlerInvoker.js'.`
2024-05-18 00:30:36 +02:00
Viktor Szakats
ec498695fc
appveyor: guard against crash-build with VS2008
The combination of `-DDEBUGBUILD`, a shared `curl.exe`, and the VS2008
compiler creates a `curl.exe` segfaulting on startup:

```
+ _bld/src/curl.exe --version
./appveyor.sh: line 122:   793 Segmentation fault      "${curl}" --version
Command exited with code 139
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49817266/job/651iy6qn1e238pqj#L191

Add job that triggers the issue and add the necessary logic to skip
running the affected `curl.exe`.

Ref: #13592
Closes #13654
2024-05-15 14:15:24 +02:00
Viktor Szakats
dad03dc593
appveyor: tidy-ups
- delete a duplicate line.
- simplify a `make` call.
- merge two `if` branches.
- reorder autotools options for clarity.
- add `--enable-warnings` where missing (it's also the default.)
- add empty lines to YAML for readability.
- use lowercase install prefix/directory.

Closes #13598
2024-05-12 18:53:04 +02:00
Viktor Szakats
2725488bea
CI: ignore test 286 on Appveyor gcc 7 build
Disabled earlier for gcc 9 builds. gcc 7 uses the same runner and
prone to similar intermittent failures.

Follow-up to f1e05a6e6e #12106 #12040
Closes #13575
2024-05-10 16:49:09 +02:00
Viktor Szakats
217878bade
appveyor: make gcc 6 mingw64 job build-only
This job has proven to be the flakiest of all, and it's also the oldest
Windows runner we had tests running on: 'Visual Studio 2015', that is
running on Windows Server 2012 R2:
  https://www.appveyor.com/docs/windows-images-software/

Turn off tests on this job to help stabilizing CI runs.

This was also one of the slowest running job amongst the AppVeyor CI ones.

Flakiness data:
  https://testclutch.curl.se/static/reports/summary.html
Entries:
  Appveyor / CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static, no-unity (curl) [current]
  Appveyor / CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static (curl) [former]

Closes #13566
2024-05-08 23:39:45 +02:00
Viktor Szakats
b445818586
appveyor: make VS2010 job build-only, enable Schannel, fix compiler warnings
Tests were consistently flaky for a while.

Also fix compiler warnings in `CertOpenStore()` calls for old MSVC compilers:
```
C:/projects/curl/lib/vtls/schannel.c(688):
  warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater size
C:/projects/curl/lib/vtls/schannel_verify.c(642):
  warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater size
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49580310/job/ywu2y44kymgc0nif#L106

Closes #13330
2024-04-10 07:32:38 +00:00
Daniel Stenberg
6389ba87b8
projects: drop MSVC project files for recent versions
We encourage users to generate visual studio project files using CMake.

We keep project files in git for ancient visual studio versions that
cmake cannot generate files for, but we no longer ship the project files
in the tarballs.

appveyor: switch VisualStudioSolution job to VC12 (Visual Studio 2013)

Co-Authored-by: Viktor Szakats
Co-Authored-by: Jay Satiro

Closes #13311
2024-04-10 07:55:24 +02:00
Viktor Szakats
feb1a3527e
appveyor: re-enable OpenSSL 3, bump to 3.2.1
Ref: b62454a875 #13266
Closes #13329
2024-04-09 21:16:53 +00:00
Viktor Szakats
29bfde9fea
appveyor: enable cmake unity mode by default
Leave one non-unity cmake job. This makes the jobs finish slightly
quicker, while giving more coverage for unity issues.

Before:
https://ci.appveyor.com/project/curlorg/curl/builds/49496977
https://ci.appveyor.com/project/curlorg/curl/builds/49500372
After:
https://ci.appveyor.com/project/curlorg/curl/builds/49500338

Also fixup unrelated whitespace.

Reviewed-by: Daniel Stenberg
Closes #13217
2024-04-04 09:31:11 +00:00
Viktor Szakats
5023ffad2c
cmake: generate misc manpages and install mk-ca-bundle.pl
- install `mk-ca-bundle.pl` like autotools does.

- generate and install `mk-ca-bundle.1` and `curl-config.1` like
  autotools. This fixes tests 1140 and 1173.

  Reported-by: Dan Fandrich
  Fixes #13194

- add option `BUILD_MISC_DOCS` to control building the above two
  manpages. Enabled by default.

- appveyor: stop disabling tests 1140 and 1173.

Reviewed-by: Daniel Stenberg
Closes #13197
2024-04-04 09:20:04 +00:00
Viktor Szakats
b62454a875
appveyor: OpenSSL 3 no longer found by CMake, revert to 1.1.1
OpenSSL moved directories, and bumped versions in AppVeyor CI.

Downgrading is not an ideal solution, but however trivial the solution
may be, I failed to come with anything that made CMake recognize either
OpenSSL 3.1 or 3.2.

Possibly caused by:
702e8cdca0
https://github.com/appveyor/build-images/pull/149

Closes #13266
2024-04-03 14:30:55 +00:00
Dan Fandrich
dff74ae8bb appveyor: Properly skip if only CircleCI is changed 2024-03-01 01:16:24 -08:00
Daniel Stenberg
eefcc1bda4
docs: introduce "curldown" for libcurl man page format
curldown is this new file format for libcurl man pages. It is markdown
inspired with differences:

- Each file has a set of leading headers with meta-data
- Supports a small subset of markdown
- Uses .md file extensions for editors/IDE/GitHub to treat them nicely
- Generates man pages very similar to the previous ones
- Generates man pages that still convert nicely to HTML on the website
- Detects and highlights mentions of curl symbols automatically (when
  their man page section is specified)

tools:

- cd2nroff: converts from curldown to nroff man page
- nroff2cd: convert an (old) nroff man page to curldown
- cdall: convert many nroff pages to curldown versions
- cd2cd: verifies and updates a curldown to latest curldown

This setup generates .3 versions of all the curldown versions at build time.

CI:

Since the documentation is now technically markdown in the eyes of many
things, the CI runs many more tests and checks on this documentation,
including proselint, link checkers and tests that make sure we capitalize the
first letter after a period...

Closes #12730
2024-01-23 00:29:02 +01:00
Daniel Stenberg
e2fbe56610
CI: spellcheck/appveyor: invoke configure --without-libpsl
Follow-up to 2998874bb6
2024-01-09 16:00:29 +01:00
Viktor Szakats
2d4d0c1fd3
appveyor: replace PowerShell with bash + parallel autotools
PowerShell works (after a steep development curve), but one property of
it stuck and kept causing unresolvable usability issues: With
`$ErrorActionPreference=Stop`, it does abort on failures, but shows only
the first line of the error message. In `Continue` mode, it shows the
full error message, but doesn't stop on all errors. Another issue is
PowerShell considering any stderr output as if the command failed (this
has been improved in 7.2 (2021-Nov), but fixed versions aren't running
in CI and will not be for a long time in all test images.)

Thus, we're going with bash.

Also:
- use `-j2` with autotools tests, making them finish 5-15 minutes per
  job faster.
- omit `POSIX_PATH_PREFIX`.
- use `WINDIR`.
- prefer forward slashes.

Follow-up to: 75078a415d #11999
Ref: #12444

Fixes #12560
Closes #12572
2023-12-21 22:54:07 +00:00
Viktor Szakats
a91746b4c7
appveyor: switch to out-of-tree builds
With cmake and autotools.

Closes #12550
2023-12-19 12:45:14 +00:00
Viktor Szakats
026122efd3
appveyor: make VS2008-built curl tool runnable
By linking the CRT statically. This avoids the error about missing
runtime DLL `MSVCR90.dll` when running the freshly built `curl.exe`.

Closes #12263
2023-11-03 22:53:45 +00:00
Viktor Szakats
4f591db44c
appveyor: bump one job to OpenSSL 3.1 (was 1.1.1)
Use 3.1 with the modern runner image.

We still use 1.1.1 in 8 jobs.

1.1.1 is EOL since 2023-09-11:
https://www.openssl.org/blog/blog/2023/03/28/1.1.1-EOL/

Also:
- add missing SSL-backend to job descriptions.
- tidy up CPU in job descriptions.

Closes #12226
2023-10-30 10:46:40 +00:00
Viktor Szakats
2100d9fde2
cmake: pre-fill rest of detection values for Windows
The goal of this patch is to avoid unnecessary feature detection work
when doing Windows builds with CMake. Do this by pre-filling well-known
detection results for Windows and specifically for mingw-w64 and MSVC
compilers. Also limit feature checks to platforms where the results are
actually used. Drop a few redundant ones. And some tidying up.

- pre-fill remaining detection values in Windows CMake builds.

  Based on actual detection results observed in CI runs, preceding
  similar work over libssh2 and matching up values with
  `lib/config-win32.h`.

  This brings down CMake configuration time from 58 to 14 seconds on the
  same local machine.

  On AppVeyor CI this translates to:
  - 128 seconds -> 50 seconds VS2022 MSVC with OpenSSL (per CMake job):
    https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/4gw66ecrjpy7necb#L296
    https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/8m4fwrr2fe249uo8#L186
  - 62 seconds -> 16 seconds VS2017 MINGW (per CMake job):
    https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/s1y8q5ivlcs7ub29?fullLog=true#L290
    https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/pchpxyjsyc9kl13a?fullLog=true#L194

  The formula is about 1-3 seconds delay for each detection. Almost all
  of these trigger a full compile-link cycle behind the scenes, slow
  even today, both cross and native, mingw-w64 and apparently MSVC too.
  Enabling .map files or other custom build features slows it down
  further. (Similar is expected for autotools configure.)

- stop detecting `idn2.h` if idn2 was deselected.
  autotools does this.

- stop detecting `idn2.h` if idn2 was not found.
  This deviates from autotools. Source code requires both header and
  lib, so this is still correct, but faster.

- limit `ADDRESS_FAMILY` detection to Windows.

- normalize `HAVE_WIN32_WINNT` value to lowercase `0x0a12` format.

- pre-fill `HAVE_WIN32_WINNT`-dependent detection results.
  Saving 4 (slow) feature-detections in most builds: `getaddrinfo`,
  `freeaddrinfo`, `inet_ntop`, `inet_pton`

- fix pre-filled `HAVE_SYS_TIME_H`, `HAVE_SYS_PARAM_H`,
  `HAVE_GETTIMEOFDAY` for mingw-w64.
  Luckily this do not change build results, as `WIN32` took
  priority over `HAVE_GETTIMEOFDAY` with the current source
  code.

- limit `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` and
  `HAVE_CLOCK_GETTIME_MONOTONIC` detections to non-Windows.
  We're not using these in the source code for Windows.

- reduce compiler warning noise in CMake internal logs:
  - fix to include `winsock2.h` before `windows.h`.
    Apply it to autotools test snippets too.
  - delete previous `-D_WINSOCKAPI_=` hack that aimed to fix the above.
  - cleanup `CMake/CurlTests.c` to emit less warnings.

- delete redundant `HAVE_MACRO_SIGSETJMP` feature check.
  It was the same check as `HAVE_SIGSETJMP`.

- delete 'experimental' marking from `CURL_USE_OPENSSL`.

- show CMake version via `CMakeLists.txt`.
  Credit to the `zlib-ng` project for the idea:
  61e181c8ae/CMakeLists.txt (L7)

- make `CMake/CurlTests.c` pass `checksrc`.

- `CMake/WindowsCache.cmake` tidy-ups.

- replace `WIN32` guard with `_WIN32` in `CMake/CurlTests.c`.

Closes #12044
2023-10-24 21:06:36 +00:00
Dan Fandrich
f1e05a6e6e CI: ignore test 286 on Appveyor gcc 9 build
This test fails sometimes with a super fast retry loop due to what may
just be a compiler bug. The test results are ignored on the one CI job
where it occurs because there seems to be nothing we can do to fix it.

Fixes #12040
Closes #12106
2023-10-13 09:01:16 -07:00
Viktor Szakats
500f28f414
appveyor: bump mingw-w64 job to gcc 13 (was: 8)
This sets gcc 6, 7, 9, 13 in our test mix (was: 6, 7, 8, 9).
Adding a modern gcc version to the tests.

(The gcc 8 job used to take around 50 minutes. The new image with gcc 13
finished in 32, 35, 34 minutes in the 3 test runs so far.)

It also adds a modern CMake version and OS env to our mingw-w64 builds.

Closes #12051
2023-10-07 14:46:37 +00:00