Commit Graph

1154 Commits

Author SHA1 Message Date
Daniel Stenberg
e27abfe2f1
version: rename c-ares-rr to asyn-rr
Works better if we later introduce another way to do the asynch RR
resolves (together with the threaded resolver) that does not use c-ares.

Closes #16090
2025-01-27 08:11:44 +01:00
Daniel Stenberg
0d4fdbf15d
asyn-thread: use c-ares to resolve HTTPS RR
Allow building with c-ares and yet use threaded resolver for the main
host A/AAAA resolving:

  `--with-ares` provides the c-ares install path and defaults to use
  c-ares for name resolving

  `--with-threaded-resolver` still uses c-ares in the build (for HTTPS)
  but uses the threaded resolver for "normal" resolves.

It works similarly for cmake: ENABLE_ARES enables ares, and if
ENABLE_THREADED_RESOLVER also is set, c-ares is used for HTTPS RR and
the threaded resolver for "normal" resolves.

HTTPSRR and c-ares-rr are new features return by curl_version_info() and
thus shown by curl -V.

The c-ares-rr feature bit is there to make it possible to distinguish
between builds using c-ares for all name resolves and builds that use
the threaded resolves for the regular name resolves and c-ares for
HTTPSRR only. "c-ares-rr" means it does not use c-ares for "plain" name
resolves.

HTTPSRR support is EXPERIMENTAL only.

Closes #16054
2025-01-25 23:46:14 +01:00
Viktor Szakats
2c4bfefe91
build: stop detecting sched_yield() on Windows
On Windows a successful `sched_yield()` detection requires mingw-w64
built with POSIX threads (not Win32 threads) and GCC (not llvm/clang).
(linking to `winpthread` via custom options may also work.)

In CMake builds, it was pre-cached as unavailable before this patch.

When detected (via autotools), it got only used for Windows XP or older
targets combined with a non-GCC, non-clang compiler that doesn't support
`__builtin_ia32_pause()`, or with the Intel C compiler. According to
`lib/easy_lock.h`.

mingw-w64 only supports GCC and clang, leaving a very narrow chance when
`shed_yield()` gets called on Windows. Even then, `sched_yield()` is
implemented in `winpthread` as `Sleep(0)`, which may or not be a useful.
It's also trivial to implement locally if it is, and such rare build
combination is also deemed useful.

Thus, this patch marks `sched_yields()` permanently unavailable on the
Windows platform also with autotools, and instead of pre-caching, skip
this feature check with CMake.

This syncs `HAVE_SCHED_YIELDS` between builds methods on Windows.

Follow-up to 9b517c8b69 #11973
Follow-up to 23af112f55 #8680

Closes #16037
2025-01-17 23:31:58 +01:00
Viktor Szakats
140790d74d
configure: UWP and Android follow-up fixes
- limit UWP detection to native Windows.
- add missing double-quotes to variable.
- drop interlock and sync order in buildinfo flags.

Follow-up to 56a74fac47 #16014
Follow-up to f7bb6c1f64 #16020

Closes #16027
2025-01-17 01:50:27 +01:00
Viktor Szakats
f7bb6c1f64
autotools: add support for mingw UWP builds
To match cmake builds.

- GHA/windows: allow autotools UWP builds.
- detect UWP and add to `buildinfo.txt`.
  Consider it enabled if `CPPFLAGS` contains `-DWINSTORECOMPAT`.
- disable telnet with UWP.
- enable Unicode with UWP.
- do not use `wldap32` with UWP.
- do not enable `USE_WIN32_CRYPTO` with UWP.
- make sure to link to `ws2_32` in UWP builds.
  To fix `undefined reference to `in6addr_any'` when linking
  `tests/server` programs. More in the comment.

Closes #16020
2025-01-16 19:42:13 +01:00
Stefan Eissing
fa5d5ac1c9
pytest: use httpd/apache2 directly, no apachectl
Since the script 'apachectl' from the httpd project is severly mutilated
on several distros, use the executable httpd/apache2 directly in pytest
runs.

Remove detection of apachectl form autoconf and cmake.

Closes #16000
2025-01-14 11:33:49 +01:00
sftcd
1ae47b91a3
ECH: update APIs to those agreed with OpenSSL maintainers
Closes #15945
2025-01-10 08:36:25 +01:00
Stefan Eissing
515a21f350
vtls: feature ssls-export for SSL session im-/export
Adds the experimental feature `ssls-export` to libcurl and curl for
importing and exporting SSL sessions from/to a file.

* add functions to libcurl API
* add command line option `--ssl-sessions <filename>` to curl
* add documenation
* add support in configure
* add support in cmake
+ add pytest case

Closes #15924
2025-01-08 23:32:07 +01:00
Viktor Szakats
0f1b23e960
build: replace configure check with PP condition (Android <21)
To make it build again with CMake + Android 20 and earlier.

8e34505776 synced `getpwuid_r()` detection
in cmake with autotools. It means cmake started detecting it with
Android <21 just like autotools, and thus cmake builds also need to
tackle the missing declaration with old Android SDK versions. Use a PP
solution, allowing to drop the autotools-specific on used before this
patch.

Follow-up to 8e34505776 #15164
Follow-up to 9c33813d83 #2609
Ref: #2058
Closes #15871
2025-01-01 04:55:54 +01:00
Viktor Szakats
10fe952da0
build: fix unsigned time_t detection for cmake, MS-DOS, AmigaOS
- cmake: add auto-detection. Sync this with autotools.
- enable for MS-DOS and AmigaOS builds.
  (auto-detection doesn't work for cross-builds.)
- tidy up detection snippet.
- fix comment.

Closes #15868
2024-12-31 11:36:56 +01:00
Viktor Szakats
f60f872bcd
configure: drop unused detections and macros
- drop `HAVE_IOCTL` macro, drop exception.
- drop unused `setjmp.h` detection, drop exception.
  It's a C89 header and result also not used in detections.
- use C89 `stdlib.h` without detection.
  (It's still being detected by autotools anyway.)

Closes #15867
2024-12-31 11:36:55 +01:00
Viktor Szakats
fabfa8e402
clang-tidy: add to CI, add cmake support, fix fallouts
build:
- autotools: fix to build generated sources for the `tidy` target.
- autotools: allow passing custom clang-tidy options via
  `CURL_CLANG_TIDYFLAGS` env.
- cmake: add `CURL_CLANG_TIDY` option to configure for `clang-tidy`.
  Also add:
  - `CLANG_TIDY` variable to customize the `clang-tidy` tool.
  - `CURL_CLANG_TIDYFLAGS` to pass custom options to `clang-tidy`.
- apply `--enable-werror` and `-DCURL_WERROR=ON` to `clang-tidy`.

CI/GHA:
- add clang-tidy job for Linux, using autotools and clang-tidy v18.
  This one needs to disable `clang-analyzer-valist.Uninitialized`
  to avoid false positives:
  https://github.com/llvm/llvm-project/issues/40656
  Duration: 5.5 minutes
- add clang-tidy job for macOS, using cmake and clang-tidy v19.
  This one also covers tests and examples, and doesn't hit the false
  positives seen with llvm v18 and earlier.
  Duration: 4.5 minutes
- Linux/macOS: skip installing test dependencies when not building or
  running tests.

fix fallouts reported by `clang-tidy`:
- lib:
  - cf-h2-proxy: unused assignment in non-debug builds.
  - cf-socket: silence warning.
    FIXME: https://github.com/curl/curl/pull/15825#issuecomment-2561867769
  - ftp: NULL passed to `strncmp()`.
  - http2: NULL-ptr deref.
  - mprintf: silence warning.
- src/tool_writeout: NULL passed to `fputs()`.
- examples:
  - invalid file pointers.
  - missing `fclose()`.
- tests:
  - http/clients/hx-download: memory leaks on error.
  - http/clients/hx-download: memory leak on repeat `-r` option.
  - server: double `fclose()`.
    https://www.man7.org/linux/man-pages/man3/fclose.3.html
  - server: invalid file pointer/handle.
  - server/getpart: unused assignments.
  - server/mqttd: leak on failed `realloc()`.
  - server/tftpd: NULL passed to `strcmp()`.

Closes #15825
2024-12-27 13:42:32 +01:00
Viktor Szakats
6dacd2f208
src: add CURL_STRICMP() macro, use _stricmp() on Windows
Add `CURL_STRICMP()` macro that works on all platforms depending on
which lib C function is available.

Make sure to always use `_stricmp()` on Windows, which is the
non-deprecated, official API for this on this platform. Before this
patch it used a MinGW-specific call, or a deprecated compatibility
wrapper with MSVC.

Drop `stricmp` variant detections on Windows with autotools.

https://learn.microsoft.com/cpp/c-runtime-library/reference/stricmp-wcsicmp-mbsicmp-stricmp-l-wcsicmp-l-mbsicmp-l

Ref: #15652
Closes #15788
2024-12-23 22:06:07 +01:00
Daniel Stenberg
fc3e1cbc50
hyper: drop support
lib : remove all hyper code
configure: stop detecting hyper
docs: no more mention of hyper
tests: mo more special-handling of hyper builds
CI: no jobs using hyper

Closes #15120
2024-12-21 11:33:05 +01:00
Viktor Szakats
a3585c9576
Makefile.mk: drop in favour of autotools and cmake (MS-DOS, AmigaOS3)
`Makefile.mk` supported MS-DOS and Amiga, but `./configure` also
supported them in a better tested and more flexible way.

This patch also adds CMake support for MS-DOS/DJGPP and Amiga OS 3.

`Makefile.mk` was not maintained. Delete it in favour of first-tier
build methods.

Also include some non-MS-DOS/AmigaOS-specific tidy-up, see details at
the end of this message.

Details:

- fix/silence all MS-DOS/DJGPP build warnings and issues.
- add MS-DOS support to cmake.
  - default to `ENABLE_THREADED_RESOLVER=OFF` for MS-DOS.
  - add support for `WATT_ROOT`.
  - use static libcurl with MS-DOS.
  - fixup default CMake suffixes/prefixes for DJGPP.
  - disable hidden symbols for MS-DOS. Not supported on MS-DOS.
  - opt-in MS-DOS into `USE_UNIX_SOCKETS`.
- improve MS-DOS support in autotools.
  - default to `--disable-threaded-resolver` for MS-DOS.
- make sure to use `close_s()` (from Watt-32) with autotools and cmake.
  `Makefile.mk` used it before this patch.
- GHA: add DJGPP cmake (~30s) and autotools (~60s) build jobs.
  Also build tests and examples with cmake.
- improve AmigaOS support in autotools:
  - configure: detect `CloseSocket()` when it's a macro.
  - configure: fix `IoctlSocket` detection on AmigaOS.
  - curl-amissl.m4: pass AmiSSL libs to tests/servers.
- add AmigaOS3 support to cmake:
  - cmake: fix `HAVE_IOCTLSOCKET_CAMEL` and
    `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` detections.
  - set necessary system libs.
  - add AmiSSL support.
  - inet_ntop, inet_pton: fix using it for AmigaOS. cmake detects them,
    and they did not compile with AmigaOS.
  - cmake: better sync `gethostname` detection with autotools.
    Fixes detection for AmigaOS, where `gethostname` is a macro.
  - cmake: fix `sys/utime.h` detection on AmigaOS.
  - cmake: force-disable `getaddrinfo` for AmigaOS.
  - cmake: tweak threading and static/shared default for AmigaOS.
  - cmake: rely on manual variable `AMIGA` to enable the platform.
- GHA: add AmigaOS cmake and autotools (~45s) jobs.
  Also build tests and examples with cmake.
- INSTALL: update MS-DOS and AmigaOS build instructions.
- amigaos: fix `-Wpointer-sign` and
  `zero or negative size array '_args'` in `Printf()`.
- amigaos: fix `-Wpointer-sign`
- amigaos: fix `-Wredundant-decls` `errno` and `h_errno`.
- amigaos: brute-force silence `lseek()` size warnings.
- amigaos: server/resolve: silence `-Wdiscarded-qualifiers`.
- amigaos: server/resolve: fix `-Wpointer-sign`.
- amigaos: fix `CURL_SA_FAMILY_T` type.
- nonblock: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
  `ioctl` is also detected, but fails when used. Make the above override
  it for a successful build.
  Authored-by: Darren Banfi
  Fixes #15537
  Closes #15603
- tftpd: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
- tftpd: tidy-up conditional code.
- curl: set stack size to 16384 for AmigaOS3/4
  Overriding the default 4096.
  Suggested-by: Darren Banfi
  Ref: https://github.com/curl/curl/pull/15543#issuecomment-2498783123
  Ref: https://wiki.amigaos.net/wiki/Controlling_Application_Stack
- functypes.h: fix `SEND_QUAL_ARG2` for AmigaOS.
- tftp: add missing cast in sendto() call for AmigaOS.
- getinfo: fix warning with AmigaOS.
- tool_operate: silence warning with AmigaOS
- amigaos: fix building libtests due to missing `RLIMIT_NOFILE`.
- curl_gethostname: silence warning for AmigaOS.
- ftp: silence `-Wtype-limits` for AmigaOS.
- libtest: fix timeval initialization for AmigaOS.
- examples: fix `timeval` initialization for AmigaOS.
- examples: silence warning for AmigaOS.
- configure: fix IPv6 detection for cross-builds.
- netrc: fix to build with AmigaOS cleanly.
- buildinfo: detect and add `DOS` tag for MS-DOS builds.
- buildinfo: add `AMIGA` to buildinfo.txt in auttools.
- build: move `USE_WATT32` macro definition to cmake/configure.

Non-MS-DOS/AmigeOS-specific tidy-ups:

- configure: sync `sa_family_t` detection with cmake.
- configure: sync `ADDRESS_FAMILY` detection signals with cmake.
- doh: use `CURL_SA_FAMILY_T`.
- lib: drop mingw-specific `CURL_SA_FAMILY_T` workaround.
- cmake: extend instead of override check-specific
  configurations/requirements.
  This allows to honor global requirements added earlier.
  Necessary for AmigaOS for example.
- cmake: omit warning on disabled IPv6 for MS-DOS and AmigaOS.
  No IPv6 support on these platforms. Also sync with autotools.
- lib1960: use libcurl `inet_pton()` wrapper.
- cmake: detect LibreSSL (to match autotools).
- cmake: say the specific OpenSSL flavour detected.
- hostip: add missing `HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID` guard.
- lib: simplify classic mac feature guards.

Follow-up to a8861b6ccd #9764

Closes #15543
2024-12-16 23:20:55 +01:00
Viktor Szakats
dc874d4369
build: drop unused feature macros, update exception list
- cmp-config.pl: add remaining exceptions. Sort list.

- drop unused `HAVE_SYS_WAIT_H`.
  Follow-up to 50def7c881 #13249

- drop unused `HAVE_FCHMOD`.
  Follow-up to 03cb1ff4d6 #12395

- autotools: stop promoting variables to macros:
  `USE_NGTCP2_CRYPTO_*`, `USE_NGTCP2_H3`, `USE_OPENSSL_H3`,
  `HAVE_LIBRESSL`.
  They are not used in the source.

- cmake: drop unused `HAVE_O_NONBLOCK`,
  `HAVE_DISABLED_NONBLOCKING`.

- lib: drop `NEED_MALLOC_H`.
  It was used in manual-build cases for Amiga/MS-DOS/Windows/WinCE, but
  never by autotools/cmake, thus apparently unnecessary.

- lib: drop unused `NEED_MEMORY_H`.

- lib: simplify classic mac feature guards,
  drop `HAVE_EXTRA_STRICMP_H` and `HAVE_EXTRA_STRDUP_H`.

- autotools: drop unused `HAVE_GETHOSTBYNAME` detection.

- autotools: drop unused OpenSSL feature tests:
  `HAVE_ERR_H`, `HAVE_PEM_H`, `HAVE_RSA_H`

- autotools: drop unused OpenSSL feature tests:
  `HAVE_X509_H`, `HAVE_CRYPTO_H`, `HAVE_SSL_H`.

  They performed a fallback check when the primary check missed
  `openssl/x509.h`. Though if any other prefixed headers were found,
  OpenSSL is already assumed detected.

  The fallback check was looking for 3 unprefixed OpenSSL headers, and
  if all found, marked OpenSSL found internally, but did not promote
  it to `curl_config.h` via `USE_OPENSSL`. Meaning it either didn't do
  anything or may have continued with an inconsistent state.
  Added in d99c20f628 (2008)

  At the time, there was an extra `AC_DEFINE(USE_SSLEAY, 1 ...` logic
  after this code, which kicked in in the fallback case, but that code
  was deleted in 709cf76f6b (2015)

  Follow-up to 709cf76f6b

- autotools: drop `AC_SUBST()` where the value is explicitly set anyway
  and the macro is unused.

- autotools: replace `AC_SUBST(VAR, 1)` with local variable assigments,
  where the `@VAR@` macro is unused. Also dedupe the local variable if
  there was a parallel one used for the same purpose.

- autotools: drop local feature variables that were never used.

- autotools: drop unused `CURL_CHECK_OPTION_NTLM_WB`,
  `CURL_CHECK_NTLM_WB`.
  Also stop setting unused `NTLM_WB_ENABLED` macro for VMS.
  Follow-up to 50def7c881 #13249

- autotools: drop unused `PKGADD_*`.
  Follow-up to bae0d473f5 #3331

- autotools: drop unused `CURL_NETWORK_LIBS`.
  Follow-up to 3af75e18d6 #14697

Closes #15577
2024-12-16 18:05:24 +01:00
Viktor Szakats
2d4852af83
build: fix tests when documentation/manual is disabled
- build: fix to exclude 'documentation' tests when building
  the documentation is explicitly disabled. Both for cmake
  and `./configure`.

  Reported-by: Daniel Engberg
  Fixes #15703

- test481, test482: fix for builds with no manual.

  Reported-by: Daniel Engberg
  Fixes #15703

- configure: fix to always detect Perl. Running tests require
  it when run. Before this patch Perl wasn't detected when
  documentation/manual/embedded-CA were all disabled,
  making tests fail to start.

- test1177: add keyword `documentation`. It depends no
  `curl_version_info.3`.

- GHA/linux: test cmake and `./configure` with docs and
  manual disabled and tests run.

Closes #15704
2024-12-09 12:45:29 +01:00
Viktor Szakats
2f03242316
build: fix ECH to always enable HTTPS RR
The ECH feature cannot be built without HTTPS RR.

ECH automatically implied HTTPS RR in `./configure` but not in CMake,
winbuild, documentation.

Also update documentation and CI configs.

Follow-up to a362962b72 #11922
Closes #15648
2024-11-28 11:55:05 +01:00
Viktor Szakats
cb7558af39
configure: replace $# shell syntax
With a more portable alternative.

Fixes (seen on macOS):
```
../configure: line 47131: 1: command not found
```
Ref: https://github.com/curl/curl/actions/runs/11846071276/job/33012894013#step:7:635

Follow-up to e244d50064 #15550
Closes #15584
2024-11-15 11:50:45 +01:00
Viktor Szakats
c66964f280
configure: add FIXMEs for disabled pkg-config references
Follow-up to d511ec8b0a #15573
2024-11-14 22:44:52 +01:00
Viktor Szakats
d511ec8b0a
build: omit certain deps from libcurl.pc unless found via pkg-config
The idea of linking dependencies found to `libcurl.pc` turns out not
to work in practice in some cases.

Specifically: gss, ldap, mbedtls, libmsh3, rustls

A `.pc` may not work or be missing for a couple of reasons:
- not all build methods generate it: mbedTLS, Rustls
- generated file is broken: msh3
  Ref: https://github.com/nibanks/msh3/pull/225
- installed package flavour isn't shipping with one:
  FreeBSD GSS, OmniOS LDAP, macOS LDAP

The effect of such issues shall be subtle in theory, because
`libcurl.pc` normally lists these dependencies in the `Requires.private`
section meant for static linking. But, e.g. `pkg-config --exists`
requires these to be present, and builds sometimes use this check
regardless of build type. This bug is not present in `pkgconf`; it only
checks for them when `--static` is also passed.

Fix these by adding affected `.pc` references to `libcurl.pc` only when
we detected the dependency via `pkg-config`.

There are a few side-effects of this solution:
- references are never added for dependencies where curl doesn't
  implement `pkg-config` detection. These are:
  - autotools: ldap, mbedtls, msh3
  - cmake: ldap (pending #15273)
- generated `libcurl.pc` depends on the build-time environment.
- generated `libcurl.pc` depends on curl build tool (cmake, autotools).
- generated `libcurl.pc` depends on curl build implementation details.

Make an exception for GNU GSS, where I blindly guess that `gss.pc` is
always available, as no issues were reported.

Other, not mentioned, dependencies continue to be added regardless
of the detection method.

Reported-by: Harmen Stoppels, Thomas, Daniel Engberg, Andy Fiddaman
Fixes #15469
Fixes #15507
Fixes #15535
Fixes https://github.com/curl/curl/pull/15163#issuecomment-2473358444
Closes #15573
2024-11-14 22:38:47 +01:00
Viktor Szakats
e244d50064
configure: do not echo most inherited LDFLAGS to config files
`libcurl.pc` `Libs.private` (since 8.11.0, and in `Libs` before 7.20.0)
and `curl-config` `--static-libs` (since 7.17.1, and in `Libs` between
7.7.2-7.25.0). This included all flags inherited from the environment,
in addition to those coming from dependency detections.

To avoid spilling all linker flags inherited from the environment to
the libcurl config files, this patch omits them all, except `-L`, `-F`,
`--library-path=` and `-framework` options, which are still passed.
The rationale for the exceptions is that `LIBS` is passed as-is, and
`LDFLAGS`, `LIBS` are the canonical way to pass custom libs options
to a build. `LIBS` may not work without a matching custom libpath.

This brings autotools behaviour closer to cmake, and `curl-config`
closer to `libcurl.pc`.

Follow-up to 9f56bb608e #14681
Follow-up to 4c8adc8fee
Reported-by: Peter Marko
Fixes #15533
Closes #15550
2024-11-14 09:55:45 +01:00
Viktor Szakats
8b76a8aeb2
build: use _fseeki64() on Windows, drop detections
A recent update caused CMake builds to mis-detect this symbol on iOS.
Auto-detection also seems redundant given that it's a Windows-only
function and most Windows builds were already opted-in.

Drop detections and use it in all Windows builds with large file support
enabled.

Feature history:
- pririotizing for Windows: aaacd02466 #14678
- Windows opt-in cmake: 8e74c0729d #11950
- Windows opt-in: aa6c94c5bf #11944
- use in libcurl: 9c7165e96a #11918
- use in example: 817d1c0106

Regression from 8e34505776 #15164

Reported-by: Maarten Billemont
Fixes #15525
Closes #15526
2024-11-11 14:25:04 +01:00
Stefan Eissing
3a35901a11
wolfssl: coexist with openssl, further work
Build wolfSSL master with

./configure --prefix=/path --enable-ip-alt-name --enable-quic
--enable-earlydata --enable-psk --enable-opensslcoexist

and configure curl with openssl + wolfssl. Normal tests run.

pytest session resumption fails, as wolfssl does not handle the
new_session callback without opensslextra right now.

Closes #15481
2024-11-04 14:48:30 +01:00
Jonas 'Sortie' Termansen
78c3172921
curl_addrinfo: support operating systems with only getaddrinfo(3)
The gethostbyname(3) family was removed in POSIX-1.2008 in favor of
getaddrinfo(3) introduced in POSIX-1.2001. Modern POSIX systems such as
Sortix does not have gethostbyname nor the related definitions and
structures.

curl already only uses getaddrinfo(3) if available and thread safe,
although there is mild breakage if the related gethostbyname definitions
are missing.

This change attempts to fix that breakage:

Remove an unnecessary configure error if gethostbyname is missing since
getaddrinfo is enough as a fallback.

Rewrite Curl_ip2addr to not use struct hostent as it no longer is
standardized and create the struct Curl_addrinfo directly.

Only define the Curl_he2ai function on non-getaddrinfo systems where it
is going to be used with struct hoestent.

Revoke the fallback logic for when it's unknown whether getaddrinfo is
thread safe. It doesn't appear to make any sense since h_errno is
unrelated to getaddrinfo. The logic prevents new POSIX.1-2024 systems
from passing the thread safety test since h_errno does not exist anymore
and POSIX already requires getaddrinfo to be thread safe. There's
already a denylist in place for operating systems with known buggy
implementations.

Closes #15475
2024-11-02 22:28:59 +01:00
Daniel Stenberg
0cececef0f
config: rename the OS define to CURL_OS to reduce collision risk
Reported-by: Jon Rumsey
Fixes #15455
Closes #15457
2024-10-30 14:59:54 +01:00
Viktor Szakats
8e34505776
cmake: untangle feature detection interdependencies
- reduce `check_include_file_concat()` use to those headers that either
  depend on a previously detected header, or another header or symbol
  detection depend on it.

- replace `check_symbol_exists()` with `check_function_exists()` for
  functions that are detected with `AC_CHECK_FUNCS()` in `./configure`.
  This makes `setmode()` no longer be detected with MSYS, syncing
  this with `./configure`. Instead `_setmode()` is used now also in
  CMake MSYS builds. This is consistent with Cygwin builds also.

- add comment about which header/symbol detection depends on what
  header. Based on `./configure` mainly.

- form `CURL_TEST_DEFINES` manually, and include only those macros which
  are actually used in `CMake/CurlTests.c`.

- change `curl_internal_test()` to use `CMAKE_REQUIRED_DEFINITIONS`,
  instead of `CMAKE_REQUIRED_FLAGS` to simplify the logic, and to allow
  dropping the latter macro completely.

- drop `windows.h` from header and symbol checks.

- `./configure`: add comment about whether `netinet/in6.h`, `sys/un.h`
  are indeed meant to be included for all detections. There is a chance
  they were added there by accident.

Detection resuls were cross-checked between
436bbbe7ab (master) and
48ff4694e608ccfdedf7ce5bab2b96d6b2c23cda (this PR), for CI GHA Linux,
Linux HTTP/3, non-native, macOS and Windows jobs.

Closes #15164
2024-10-11 17:44:45 +02:00
Viktor Szakats
3e7a6fbb8e
configure: add GSS to libcurl.pc Depends:
GSS was the last (known) missing dependency missing from `libcurl.pc`.

Closes #15177
2024-10-08 11:49:18 +02:00
Viktor Szakats
86d5c2651d
configure: drop unused bare socket.h detection
Added in 37eba37019 (2009-06-17) to help
detecting socket functions.

But, this `socket.h` isn't used in the source code since
90dd1fc664 #8288 (2022-01-16).

Closes #15173
2024-10-07 12:31:37 +02:00
Viktor Szakats
2d1959dd0c
configure: drop duplicate feature checks for poll(), if_nametoindex()
Before this patch they were detected via manual methods, then with
`AC_CHECK_FUNCS()`.

Delete the manual checks and keep the latter.

Also delete `CURL_INCLUDES_POLL()` which is no longer used after
the above.

Closes #15170
2024-10-07 12:31:21 +02:00
Viktor Szakats
5e70566094
build: detect and use _setmode() with Cygwin/MSYS, also use on Windows
Before this patch `setmode()` was not detected with Cygwin/MSYS, because
it's a macro, not a function, and detection is looking for a function.

Switching to symbol detection doesn't work because it mis-detects it on
BSD systems which features a function with the same name but different
functionality and arguments.

Fix it by looking for a `_setmode()` function on Cygwin/MSYS, and use it
if available.

`_setmode()` is recommended over `setmode()` by Windows documentation so
use that on Windows too. It seems to be available on all supported
compilers, so omit detection.

https://learn.microsoft.com/cpp/c-runtime-library/reference/posix-setmode
https://learn.microsoft.com/cpp/c-runtime-library/reference/setmode

Officially Windows requires argument `_O_BINARY` with an underscore.
`O_BINARY` is also supported but bound to conditions. Continue to use it
for simplicity. Cygwin supports `O_BINARY` (no underscore).

Closes #15169
2024-10-07 12:31:21 +02:00
Daniel Stenberg
c72cefea0f
select: use poll() if existing, avoid poll() with no sockets
poll() on macOS 10.12 was deemed broken in 2016 when we discovered that
it misbehaves when provided with no sockets to wait for. The
HAVE_POLL_FINE is used to mark a poll() implementation that behaves
correctly: it *should* still wait the timeout time.

curl has therefore opted to use select() on Apple operating systems ever
since. To avoid the risk that this or other breakage cause problems.

However, using select() internally is also bad because it suffers from
problems when using file descriptors beyond 1024.

This change makes poll() used if it is present, but if there is no
sockets to wait for it avoids using poll() and instead falls back to
select() - but without any sockets to wait for there is no 1024 problem.

This removes all previous special-handling involving HAVE_POLL_FINE.

ref: https://daniel.haxx.se/blog/2016/10/11/poll-on-mac-10-12-is-broken/

Closes #15096
2024-10-01 15:11:50 +02:00
Stefan Eissing
97c0f89bd0
quic: use send/recvmmsg when available
add checks for sendmmsg in configure and CmakeLists.txt for enabling use
of these functions in ngtcp2/quiche quic.

Closes #14880
2024-09-30 08:51:16 +02:00
Daniel Stenberg
d78e129d50
WebSockets: make support official (non-experimental)
Inverts the configure/cmake options to instead provide options that
disable WebSockets and have them (ws + wss) enabled by default.

Closes #14936
2024-09-27 13:20:25 +02:00
Viktor Szakats
b4cf21b45d
build: clarify CA embed is for curl tool, mark default, improve summary
- say that CA embed is for the curl tool.
- show "no" in summary when there is no CA embed.
- cmake: sync wording.

Closes #15035
2024-09-25 09:10:45 +02:00
Viktor Szakats
68a224c291
tidy-up: indentation in autotools sources
Indentation in `configure.ac` and `.m4` files.

Closes #14692
2024-09-25 09:10:45 +02:00
Viktor Szakats
f6036dead5
build: show if CA bundle to embed was found
Follow-up to 8a3740bc8e #14059
Closes #15027
2024-09-24 13:36:14 +02:00
Viktor Szakats
22652a5a4c
curl: add options for safe/no CA bundle search (Windows)
Add `CURL_CA_SEARCH_SAFE` build-time option to enable CA bundle search
in the `curl` tool directory. The lookup method was already used to find
`.curlrc` and `_curlrc` (on Windows). On Windows it overrides the unsafe
default `SearchPath()` method.

Enable with:
- cmake: `-DCURL_CA_SEARCH_SAFE=ON`
- autotools: `--enable-ca-search-safe`
- raw: `CPPFLAGS=-DCURL_CA_SEARCH_SAFE`

On Windows, before this patch the whole `PATH` was searched for
a CA bundle. `PATH` may contain unwanted or world-writable locations,
including the current directory. Searching them all is convenient to
pick up any CA bundle, but not secure.

The Muldersoft curl distro implements such CA search via a custom
patch for Windows:
cd652d4792/patch/curl_tool_doswin.diff (L50)

MSYS2/mingw-w64 distro has also been rolling a patch solving this:
https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-curl/0001-Make-cURL-relocatable.patch
https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-curl/pathtools.c

Also add option to fully disable Windows CA search:
- cmake: `-DCURL_DISABLE_CA_SEARCH=ON`
- autotools: `--disable-ca-search`
- raw: `CPPFLAGS=-DCURL_DISABLE_CA_SEARCH`.

Both options are considered EXPERIMENTAL, with possible incompatible
changes or even (partial) removal in the future, depending on feedback.

An alternative, secure option is to embed the CA bundle into the binary.

Safe search can be extended to other platforms if necessary or useful,
by using `_NSGetExecutablePath()` (macOS),
`/proc/self/exe` (Linux/Cygwin), or `argv[0]`.

Closes #14582
2024-09-22 18:17:25 +02:00
Viktor Szakats
668584a94f
build: drop exclamation marks, use "msh3/msquic" where incomplete
Closes #15002
2024-09-22 15:49:40 +02:00
Viktor Szakats
71cf0d1fca
tests: speed up builds with single-binary test bundles
Add support for single-block binaries that contain all libtests and
unit tests respectively.

Enable with:
- autotools: `--enable-test-bundles`
- cmake: `-DCURL_TEST_BUNDLES=ON`

(They are compatible with `--enable-unity` and `-DCMAKE_UNITY_BUILD=ON`
options, for further speed-up.)

Makes libtests and unit tests build _fast_, needing little disk space
even in static mode. Similar to CMake unity mode, but with a custom
script, also supporting autotools builds.

The price is having to deal with symbols/macros colliding between
`lib*.c` and `unit*.c` sources. Maybe with naming conventions or other
solutions this can be improved gradually and reduce the need for manual
intervention by `mk-bundle.mk`. I've included a script that does the bulk
of detecting name collisions.

Also:
- CI: enable test bundles.
- CI: build tests in more jobs.
- lib2305: fix FILE handle leak.
- unit1661: fix memleak found by torture test by releasing the `bufref`
  structure in `unit_stop()` that was allocated in `unit_setup()`.
  ```
  test 1661...[bufref unit tests]
  Leak detected: memory still allocated: 13 bytes
   allocated by /home/runner/work/curl/curl/tests/unit/unit1661.c:70
   1661: torture FAILED: function number 1 in test.
  ```
  Ref: https://github.com/curl/curl/actions/runs/10967279334/job/30456745290?pr=14772#step:8:41

Similar test suite builds with autotools default and cmake+bundle+unity:
- GHA/Linux: 33s vs 7s
  https://github.com/curl/curl/actions/runs/10705668823/job/29681617374
- GHA/macOS 34s vs 2s
  https://github.com/curl/curl/actions/runs/10705668813/job/29681632885
- GHA/FreeBSD: 15m25 vs 6m21 (full workflow time, ~qemu)
  https://github.com/curl/curl/actions/runs/10705668811/job/29681607915
- GHA/Cygwin: 9m52 vs 32s
  https://github.com/curl/curl/actions/runs/10705668809/job/29681609965
- GHA/MSYS2: 3m52 vs 14s
  https://github.com/curl/curl/actions/runs/10705668808/job/29681624295
- GHA/mingw-w64: 5m45 vs 30s
  https://github.com/curl/curl/actions/runs/10705668808/job/29681628787

Autotools test suite builds compared between master -> `--enable-test-bundles`:
- GHA/Linux: 33s -> 9s (run tests: 22m23 -> 20m44)
  https://github.com/curl/curl/actions/runs/10710030193/job/29695932185
  https://github.com/curl/curl/actions/runs/10967831456/job/30458220344
- GHA/macOS: 25s -> 4s (run tests: 2m58 -> 2m24)
  https://github.com/curl/curl/actions/runs/10710030195/job/29695938444
  https://github.com/curl/curl/actions/runs/10967831452/job/30458225762
- GHA/non-native (FreeBSD): 4m8 -> 3m12 (full workflow time, ~qemu)
  https://github.com/curl/curl/actions/runs/10710030198/job/29695928401
  https://github.com/curl/curl/actions/runs/10967831458/job/30458212692
- GHA/Cygwin: 9m25 -> 1m9 (run tests: 9m19 -> 3m28)
  https://github.com/curl/curl/actions/runs/10710030212/job/29695928213
  https://github.com/curl/curl/actions/runs/10967831453/job/30458213268
- GHA/MSYS2: 3m54 -> 32s (run tests: 6m3 -> 3m59)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704850591
  https://github.com/curl/curl/actions/runs/10967831449/job/30459280005
- GHA/mingw-w64: 5m42 -> 1m5 (run tests: 7m41 -> 5m36)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704852058
  https://github.com/curl/curl/actions/runs/10967831449/job/30459280862
- Azure MSYS2 mingw64 openssl: 38m55 -> 11m58
  https://dev.azure.com/daniel0244/curl/_build/results?buildId=25546&view=logs&j=b58b8c59-0f61-52e9-0f9e-fad562a1e77f&t=0f9230a7-3b10-53ca-9938-700ece377c5e
  https://dev.azure.com/daniel0244/curl/_build/results?buildId=25547&view=logs&jobId=39473db1-3945-55d5-deb5-c218fad88dce&j=b58b8c59-0f61-52e9-0f9e-fad562a1e77f&t=0f9230a7-3b10-53ca-9938-700ece377c5e
- Azure Ubuntu default: 2m15 -> 55s (all build)
  https://dev.azure.com/daniel0244/curl/_build/results?buildId=25546&view=logs&j=9d58b9ac-e1e6-53b6-f83a-1f9f1d912522&t=a6b38d83-e7cf-5a9b-c762-a178412717b7
  https://dev.azure.com/daniel0244/curl/_build/results?buildId=25547&view=logs&jobId=39473db1-3945-55d5-deb5-c218fad88dce&j=9d58b9ac-e1e6-53b6-f83a-1f9f1d912522&t=a6b38d83-e7cf-5a9b-c762-a178412717b7

Cmake test suite builds compared between master -> `-DCURL_TEST_BUNDLES=ON` + unity:
- GHA/Linux: 29s -> 7s (run tests: 4m50 -> 4m57, 20m43 -> 20m45)
  https://github.com/curl/curl/actions/runs/10710030193/job/29695941814
  https://github.com/curl/curl/actions/runs/10705668823/job/29681622201
- GHA/Linux old: 44s -> 13s (bundle+no unity) (run tests: 5m5 -> 5m6)
  https://github.com/curl/curl/actions/runs/10718264094/job/29719794727
  https://github.com/curl/curl/actions/runs/10718653175/job/29721009613
- GHA/macOS: 32s -> 2s (run tests: 2m43 -> 2m40)
  https://github.com/curl/curl/actions/runs/10710030195/job/29695931956
  https://github.com/curl/curl/actions/runs/10705668813/job/29681638937
- GHA/non-native (*BSD): inconclusive (full workflow time, ~qemu)
  https://github.com/curl/curl/actions/runs/10710030198
  https://github.com/curl/curl/actions/runs/10705668811
- GHA/Cygwin: 3m9 -> 32s
  https://github.com/curl/curl/actions/runs/10710030212/job/29695929075
  https://github.com/curl/curl/actions/runs/10705668809/job/29681609965
- GHA/MSYS2: 2m24 -> 14s
  https://github.com/curl/curl/actions/runs/10710030190/job/29704850996
  https://github.com/curl/curl/actions/runs/10705668808/job/29681624295
- GHA/mingw-w64: 3m56 -> 30s (run tests: 4m2 -> 3m52)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704852219
  https://github.com/curl/curl/actions/runs/10705668808/job/29681631393
- GHA/mingw-w64-old: 7m19 -> 1m44 (run tests: 3m30 -> 2m53)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704849763
  https://github.com/curl/curl/actions/runs/10705668808/job/29681622329
- GHA/MSVC: 3m22 -> 13s (run tests: 9m43 -> 4m22)
  https://github.com/curl/curl/actions/runs/10710030190/job/29704850411
  https://github.com/curl/curl/actions/runs/10705668808/job/29681623313
- AppVeyor CI MSVC 2008: 4m3 -> 45s (full build)
- AppVeyor CI MSVC 2010: 2m56 -> 1m8 (full build)
- AppVeyor CI MSVC 2022: 10m19 -> 2m23 (full build)
  https://ci.appveyor.com/project/curlorg/curl/builds/50538455
  https://ci.appveyor.com/project/curlorg/curl/builds/50536558
- AppVeyor CI total build time: 10m30 (master) -> 6m48 (unity) -> 4m5 (bundle) -> 3m24 (bundle+unity) -> 5m7 (bundle+unity+all jobs building tests)

Closes #14772
2024-09-22 09:51:15 +02:00
Viktor Szakats
0aece8f66d
tidy-up: indent, whitespace, #error in make files
Replace invalid C with `#error`.

Cherry-picked from #14692
Closes #14997
2024-09-22 09:51:15 +02:00
Viktor Szakats
867c187fdf
build: use configurehelp.pm.in with autotools and cmake
Before this patch, each build tool generated `tests/configurehelp.pm`
manually.

Ref: https://github.com/curl/curl/pull/14802#issuecomment-2332734326
Closes #14819
2024-09-21 12:21:14 +02:00
Viktor Szakats
7100c5bc9b
build: tidy up and improve versioned-symbols options
- autotools: add support for custom prefix.
  Usage: `--enable-versioned-symbols=MYPREFIX_`
  Catching up with cmake.

- add default versioned prefix for Rustls builds.

- delete `HIDDEN` entry from version script `lib/libcurl.vers.in`.
  It referred to symbol names that never existed in libcurl.

- cmake: use `lib/libcurl.vers.in` as a template instead of generating
  it from scratch. Adapt `./configure` accordingly, and add comments
  to keep hard-wired soname in sync with `lib/Makefile.soname`.

- autotools: delete Schannel and Secure Transport version prefixes.
  Windows and macOS don't support the versioned symbols option.

Follow-up to 7b14449790 #14378
Closes #14818
2024-09-21 12:21:14 +02:00
Viktor Szakats
30ab1133c4
configure: catch Apple in more target triplets
Before this patch, only these triplets were considered Apple:
`<cpu>-apple-darwin`

After this patch, these are also considered Apple:
`<cpu>-apple-(ios*|tvos*|visionos*|watchos*|<ETC>)`

`$host_os` (the last third of the triplet) still has a valid use
to differentiate between OS flavours, though for now this isn't
used, aligning with CMake.

Closes #14728
2024-09-21 12:21:13 +02:00
Viktor Szakats
ce7d0d4137
ipfs: add options to disable
- CPPFLAGS: `-DCURL_DISABLE_IPFS`
- configure: `--disable-ipfs`
- cmake: `-DCURL_DISABLE_IPFS=ON`

Fixes #14824
Closes #14827
2024-09-21 12:08:32 +02:00
Viktor Szakats
60c3d04465
autotools: add support for 'unity' builds, enable in CI
Implement the "unity" builds as known from CMake, but for autotools.
It's limited to `lib` and `src` (CMake also supports it in `tests`).

Enable with: `--enable-unity` (disabled by default)

Unity builds speed up builds significantly. Cygwin and Windows builds in
particular, but the effect is noticeable on most systems. It also allows
discovering unity issues with autotools, benefitting also CMake when
building the same combination. In CI it makes turnaround times quicker.

This closes build performance with CMake. autotools still lags behind
because it builds shared and static libcurl in two, separate passes.
CMake does it in one. Manpage compilation isn't batched, it is in CMake.
After unity and test bundle support the slowest parts of the build are
the configuration phase (which is effectively a tedious, non-parallel,
compilation and/or linking of 300+ tiny programs. The next bottleneck
is compiling individual examples and finally test servers (only slow
with autotools).

The autotools implementation is slightly less efficient than CMake,
because 3 sources are permanently excluded while in CMake this isn't
necessary and solved more efficiently while building libtests. There is
also no 'unity' support for tests, making them a less efficient also.

Enable it in CI for most `configure` jobs. Except in GHA/dist (though
it works fine there too), to use the default config there. Also skip for
the Linux AWC-LC job where it made builds time a few seconds longer
(reason undiscovered.)

Autotools test suite builds compared between master -> `--enable-unity`:
- GHA/Linux: 32s -> 12s
  https://github.com/curl/curl/actions/runs/10705668823/job/29681617374
  https://github.com/curl/curl/actions/runs/10742978889/job/29796766297
- GHA/macOS: 37s -> 10s
  https://github.com/curl/curl/actions/runs/10705668813/job/29681632885
  https://github.com/curl/curl/actions/runs/10742978699/job/29796768875
- GHA/FreeBSD: 15m25 -> 10m58 (full workflow time, ~qemu)
  https://github.com/curl/curl/actions/runs/10705668811/job/29681607915
  https://github.com/curl/curl/actions/runs/10742978937/job/29796766115
- GHA/Cygwin: 3m32 -> 1m21
  https://github.com/curl/curl/actions/runs/10705668809/job/29681609965
  https://github.com/curl/curl/actions/runs/10742978645/job/29796756933
- GHA/MSYS2: 2m42 -> 50s
  https://github.com/curl/curl/actions/runs/10705668808/job/29681621166
  https://github.com/curl/curl/actions/runs/10742978662/job/29799739289
- GHA/mingw-w64: 5m32 -> 1m23
  https://github.com/curl/curl/actions/runs/10705668808/job/29681628787
  https://github.com/curl/curl/actions/runs/10742978662/job/29799741568

Closes #14815
2024-09-20 23:53:33 +02:00
Viktor Szakats
50e2cb5894
build: buildinfo.txt improvements
- cmake: drop `configure.os`.
  This also includes OS version, but thus far it's not important enough
  to include it.
- autotools: drop redundant, autotools-only `{target|host}.vendor`.
  (it's part of the triplet in `{target|host}`.)
- swap order to `*.cpu` -> `*.os` to match triplet-order.
- cmake: drop redundant `target`.
  It's manually filled and only in a (so far) few CI jobs. Let's revisit
  when this becomes useful.
- move `buildinfo.txt` to build root.
- dist: add `buildinfo.txt` to `DISTCLEANFILES`.
- autotools: detect human readable compiler version.
- autotools: replace `XXYY` `compiler.version` with "X.Y"-style.
  (also to match cmake.)
- autotools: use distinct `compiler_id` for Apple clang: `APPLECLANG`.
  To match cmake and also because the the "X.Y"-style version number
  is the Apple version, while `XXYY` was a value roughly translated to
  mainline llvm/clang version.
- show buildinfo at the end of the configure stage, when run in CI, or
  when `CURL_BUILDINFO` or `CURL_CI` env is set.

Follow-up to 1fdea16846 #14802
Assisted-by: Dan Fandrich
Ref: https://github.com/curl/curl/pull/14802#issuecomment-2334942991
Closes #14822
2024-09-19 15:56:30 +02:00
Viktor Szakats
d1bf447992
build: limit arc4random detection to no-SSL configs
`arc4random()` is no longer used if any TLS backend is active.
Limit feature detection to builds with no TLS backend.

Closes #14909
2024-09-19 15:44:59 +02:00
Viktor Szakats
9783c4540f
autotools: fix MS-DOS builds
- disable threaded resolver by default.
- fix forming WATT path. Requires `WATT_PATH` env.

Closes #14814
2024-09-06 21:47:35 +02:00
Viktor Szakats
1fdea16846
build: generate buildinfo.txt for test logs
Also:
- read `buildinfo.txt` from `runtests.pl` and dump it to the log.
- cmake: show `CROSS` target flag for cross-builds.
- cmake: add logic to detect arguments passed via the command-line.

It is meant to help filling out missing datapoints in the testclutch
matrix.

Closes #14802
2024-09-06 21:46:04 +02:00