Commit Graph

770 Commits

Author SHA1 Message Date
Daniel Stenberg
c445b7426a
checksrc: introduce 'banfunc' to ban specific functions
Use 'banfunc' and 'allowfunc' in .checksrc to specify which functions to
ban or allow to be used. This saves us from having to edit the script
going forward when we want to ban or allow specific functions.

This replaces a set of previous rules and all banned functions are now
checked with the BANNEDFUNC rule.

There is a set of default banned functions, shown by invoking
./checksrc.

Also, -a and -b options are added to specify allowed or banned functions
on the command line.

Closes #15835
2024-12-30 08:25:56 +01:00
Viktor Szakats
fabfa8e402
clang-tidy: add to CI, add cmake support, fix fallouts
build:
- autotools: fix to build generated sources for the `tidy` target.
- autotools: allow passing custom clang-tidy options via
  `CURL_CLANG_TIDYFLAGS` env.
- cmake: add `CURL_CLANG_TIDY` option to configure for `clang-tidy`.
  Also add:
  - `CLANG_TIDY` variable to customize the `clang-tidy` tool.
  - `CURL_CLANG_TIDYFLAGS` to pass custom options to `clang-tidy`.
- apply `--enable-werror` and `-DCURL_WERROR=ON` to `clang-tidy`.

CI/GHA:
- add clang-tidy job for Linux, using autotools and clang-tidy v18.
  This one needs to disable `clang-analyzer-valist.Uninitialized`
  to avoid false positives:
  https://github.com/llvm/llvm-project/issues/40656
  Duration: 5.5 minutes
- add clang-tidy job for macOS, using cmake and clang-tidy v19.
  This one also covers tests and examples, and doesn't hit the false
  positives seen with llvm v18 and earlier.
  Duration: 4.5 minutes
- Linux/macOS: skip installing test dependencies when not building or
  running tests.

fix fallouts reported by `clang-tidy`:
- lib:
  - cf-h2-proxy: unused assignment in non-debug builds.
  - cf-socket: silence warning.
    FIXME: https://github.com/curl/curl/pull/15825#issuecomment-2561867769
  - ftp: NULL passed to `strncmp()`.
  - http2: NULL-ptr deref.
  - mprintf: silence warning.
- src/tool_writeout: NULL passed to `fputs()`.
- examples:
  - invalid file pointers.
  - missing `fclose()`.
- tests:
  - http/clients/hx-download: memory leaks on error.
  - http/clients/hx-download: memory leak on repeat `-r` option.
  - server: double `fclose()`.
    https://www.man7.org/linux/man-pages/man3/fclose.3.html
  - server: invalid file pointer/handle.
  - server/getpart: unused assignments.
  - server/mqttd: leak on failed `realloc()`.
  - server/tftpd: NULL passed to `strcmp()`.

Closes #15825
2024-12-27 13:42:32 +01:00
Viktor Szakats
250d613763
binmode: convert to macro and use it from tests
And use it from src and tests.

Syncing this functionality between platforms and build targets.

Also: Stop redefining `O_BINARY` in src, and use a local macro with
the same effect. `O_BINARY` is used in `CURL_SET_BINMODE()` to decide
if this functionality is supported, and redefining it makes this check
pass always in unity builds. The check is required for Apple OS, because
it offers a `setmode()` function, successfully detected by both CMake
and autotools, but that function has a different functionality and
signature than that expected by `CURL_SET_BINMODE()`.

Also:
- drop MetaWare High C (MS-DOS) support for set binmode.
- tests/libtest/Makefile.inc: dedupe comments.
- lib/curl_setup_once.h: tidy up feature guards for `io.h`, `fcntl.h`.

Ref: #15652
Closes #15787
2024-12-21 13:29:24 +01:00
Viktor Szakats
a3585c9576
Makefile.mk: drop in favour of autotools and cmake (MS-DOS, AmigaOS3)
`Makefile.mk` supported MS-DOS and Amiga, but `./configure` also
supported them in a better tested and more flexible way.

This patch also adds CMake support for MS-DOS/DJGPP and Amiga OS 3.

`Makefile.mk` was not maintained. Delete it in favour of first-tier
build methods.

Also include some non-MS-DOS/AmigaOS-specific tidy-up, see details at
the end of this message.

Details:

- fix/silence all MS-DOS/DJGPP build warnings and issues.
- add MS-DOS support to cmake.
  - default to `ENABLE_THREADED_RESOLVER=OFF` for MS-DOS.
  - add support for `WATT_ROOT`.
  - use static libcurl with MS-DOS.
  - fixup default CMake suffixes/prefixes for DJGPP.
  - disable hidden symbols for MS-DOS. Not supported on MS-DOS.
  - opt-in MS-DOS into `USE_UNIX_SOCKETS`.
- improve MS-DOS support in autotools.
  - default to `--disable-threaded-resolver` for MS-DOS.
- make sure to use `close_s()` (from Watt-32) with autotools and cmake.
  `Makefile.mk` used it before this patch.
- GHA: add DJGPP cmake (~30s) and autotools (~60s) build jobs.
  Also build tests and examples with cmake.
- improve AmigaOS support in autotools:
  - configure: detect `CloseSocket()` when it's a macro.
  - configure: fix `IoctlSocket` detection on AmigaOS.
  - curl-amissl.m4: pass AmiSSL libs to tests/servers.
- add AmigaOS3 support to cmake:
  - cmake: fix `HAVE_IOCTLSOCKET_CAMEL` and
    `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` detections.
  - set necessary system libs.
  - add AmiSSL support.
  - inet_ntop, inet_pton: fix using it for AmigaOS. cmake detects them,
    and they did not compile with AmigaOS.
  - cmake: better sync `gethostname` detection with autotools.
    Fixes detection for AmigaOS, where `gethostname` is a macro.
  - cmake: fix `sys/utime.h` detection on AmigaOS.
  - cmake: force-disable `getaddrinfo` for AmigaOS.
  - cmake: tweak threading and static/shared default for AmigaOS.
  - cmake: rely on manual variable `AMIGA` to enable the platform.
- GHA: add AmigaOS cmake and autotools (~45s) jobs.
  Also build tests and examples with cmake.
- INSTALL: update MS-DOS and AmigaOS build instructions.
- amigaos: fix `-Wpointer-sign` and
  `zero or negative size array '_args'` in `Printf()`.
- amigaos: fix `-Wpointer-sign`
- amigaos: fix `-Wredundant-decls` `errno` and `h_errno`.
- amigaos: brute-force silence `lseek()` size warnings.
- amigaos: server/resolve: silence `-Wdiscarded-qualifiers`.
- amigaos: server/resolve: fix `-Wpointer-sign`.
- amigaos: fix `CURL_SA_FAMILY_T` type.
- nonblock: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
  `ioctl` is also detected, but fails when used. Make the above override
  it for a successful build.
  Authored-by: Darren Banfi
  Fixes #15537
  Closes #15603
- tftpd: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
- tftpd: tidy-up conditional code.
- curl: set stack size to 16384 for AmigaOS3/4
  Overriding the default 4096.
  Suggested-by: Darren Banfi
  Ref: https://github.com/curl/curl/pull/15543#issuecomment-2498783123
  Ref: https://wiki.amigaos.net/wiki/Controlling_Application_Stack
- functypes.h: fix `SEND_QUAL_ARG2` for AmigaOS.
- tftp: add missing cast in sendto() call for AmigaOS.
- getinfo: fix warning with AmigaOS.
- tool_operate: silence warning with AmigaOS
- amigaos: fix building libtests due to missing `RLIMIT_NOFILE`.
- curl_gethostname: silence warning for AmigaOS.
- ftp: silence `-Wtype-limits` for AmigaOS.
- libtest: fix timeval initialization for AmigaOS.
- examples: fix `timeval` initialization for AmigaOS.
- examples: silence warning for AmigaOS.
- configure: fix IPv6 detection for cross-builds.
- netrc: fix to build with AmigaOS cleanly.
- buildinfo: detect and add `DOS` tag for MS-DOS builds.
- buildinfo: add `AMIGA` to buildinfo.txt in auttools.
- build: move `USE_WATT32` macro definition to cmake/configure.

Non-MS-DOS/AmigeOS-specific tidy-ups:

- configure: sync `sa_family_t` detection with cmake.
- configure: sync `ADDRESS_FAMILY` detection signals with cmake.
- doh: use `CURL_SA_FAMILY_T`.
- lib: drop mingw-specific `CURL_SA_FAMILY_T` workaround.
- cmake: extend instead of override check-specific
  configurations/requirements.
  This allows to honor global requirements added earlier.
  Necessary for AmigaOS for example.
- cmake: omit warning on disabled IPv6 for MS-DOS and AmigaOS.
  No IPv6 support on these platforms. Also sync with autotools.
- lib1960: use libcurl `inet_pton()` wrapper.
- cmake: detect LibreSSL (to match autotools).
- cmake: say the specific OpenSSL flavour detected.
- hostip: add missing `HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID` guard.
- lib: simplify classic mac feature guards.

Follow-up to a8861b6ccd #9764

Closes #15543
2024-12-16 23:20:55 +01:00
Viktor Szakats
b9895b9d3f
cmake: namespace functions and macros
Prefix (or suffix) curl-defined macro and function names with `curl`.
To avoid collisions with non-curl CMake and to make them recognizable
as curl-specific calls.

Closes #15498
2024-12-16 21:55:00 +01:00
Daniel Stenberg
c2ac9ea1ee
checksrc: ban use of sscanf()
Using sscanf() is not a (security) problem in itself, but we strongly
discorage using it for parsing input since it is hard to use right, easy
to mess up and often makes for sloppy error checking.

Allow it in examples and tests

Closes #15687
2024-12-13 09:43:05 +01:00
Daniel Stenberg
d5c738c608
strparse: string parsing helper functions
Designed to aid converting off from sscanf parsers. sscanf is hard to
use right, easy to mess up and often makes for sloppy error checking.

The new parsers allow more exact and pedandic parsing.

This new set of functions should be possible to use (and extend) and
switch over other libcurl parser code to use going forward.

Adapts the following to use the new functions:

- altsvc.c
- hsts.c
- http_aws_sigv4.c

Bonus: fewer memory copies, fewer stack buffers.

Test: Unit test1664

Docs: docs/internals/STRPARSE.md

Closes #15692
2024-12-12 16:00:52 +01:00
Viktor Szakats
a72b479dec
build: fix MSVC UWP builds
The MSVC UWP job in CI did not actually enable UWP. Fix this and
the fallouts discovered after enabling it.

- GHA/windows: make sure to enable UWP in MSVC vcpkg UWP job.
  Use the CMake options and C flags already used for mingw-w64, but use
  `WINAPI_FAMILY_PC_APP` instead of the deprecated `WINAPI_FAMILY_APP`.
  (The former is not supported by mingw-w64, so leave it there as-is.)
  Follow-up to cb22cfca69 #14077

- GHA/windows: by default the MSVC UWP job became 2x-3x slower than
  others after actually enabling UWP. Most of it is caused by
  CMake/MSBuild automatically building full APPX containers for each
  `.exe` target. This includes 21 CMake feature detections. Each
  detection app is built into a 15MB APPX project, with code signing,
  logos, etc. Example:
    https://github.com/curl/curl/actions/runs/12056968170/job/33620610958
  Disable this overhead for curl build targets via custom
  `CMAKE_VS_GLOBALS` options. I've found no way to apply them to feature
  detection targets, so those remain slow.

- cmake: automatically enable Unicode for UWP builds. It's required.
  Also stop enabling it manually in the existing CI job.

- tests: fix `getpid()` use for Windows UWP:
  ```
  tests\server\util.c(281,21): warning C4013: 'getpid' undefined; assuming extern returning int
  ```
  Ref: https://github.com/curl/curl/actions/runs/12061215311/job/33632904249#step:11:38

- src/tool_doswin: disable `GetLoadedModulePaths()` for UWP.
  mingw-w64 UWP was okay with this, but MS SDK headers are not.
  This makes `--dump-module-paths` return empty for UWP builds.
  ```
  src\tool_doswin.c(620,3): error C2065: 'MODULEENTRY32': undeclared identifier
  src\tool_doswin.c(626,11): warning C4013: 'CreateToolhelp32Snapshot' undefined; assuming extern returning int
  src\tool_doswin.c(626,36): error C2065: 'TH32CS_SNAPMODULE': undeclared identifier
  src\tool_doswin.c(632,7): warning C4013: 'Module32First' undefined; assuming extern returning int
  ```
  Ref: https://github.com/curl/curl/actions/runs/12055081933/job/33614629930#step:9:35

- examples: fix `websocket.c` to include `winsock2.h` before `windows.h`
  to make it build with MSVC UWP:
  ```
  include\curl\curl.h(143,16): error C2061: syntax error: identifier 'curl_socket_t'
  include\curl\curl.h(143,16): error C2059: syntax error: ';'
  include\curl\curl.h(417,52): error C2146: syntax error: missing ')' before identifier 'curlfd'
  include\curl\curl.h(417,38): error C2081: 'curl_socket_t': name in formal parameter list illegal
  ```
  Ref: https://github.com/curl/curl/actions/runs/12055317910/job/33615644427#step:14:126

- GHA/windows: silence linker warning with MSVC UWP builds:
  ```
  LINK : warning LNK4075: ignoring '/INCREMENTAL' due to '/OPT:ICF' specification
  ```
  Ref: https://github.com/curl/curl/actions/runs/12055696808/job/33616629610#step:11:38

- GHA/windows: set `/INCREMENTAL:NO` for all MSVC jobs to improve
  performance a little.

- cmake: show `UWP` platform flag.

Ref: #15652
Closes #15657
2024-11-28 14:24:03 +01:00
Viktor Szakats
9126eb5a8a
cmake: replace CURL_*_DIR with {PROJECT,CMAKE_CURRENT}_*_DIR
It reduces the number of synonym variables in the code.
Makes it easier to grok and grep.

- replace `CURL_SOURCE_DIR`
  with `PROJECT_SOURCE_DIR`.

- replace `CURL_BINARY_DIR`
  with `PROJECT_BINARY_DIR` or `CMAKE_CURRENT_BINARY_DIR`.

- replace a single use of `CMAKE_BINARY_DIR`
  with `PROJECT_BINARY_DIR`.

- replace `CMAKE_CURRENT_*_DIR`
  with `PROJECT_*_DIR` where it makes the code more uniform.

- quote an argument (formatting).

Closes #15331
2024-10-22 19:13:08 +02:00
Daniel Stenberg
e33cf006e9
server/mqttd: fix two memory leaks
Closes #15290
2024-10-14 12:47:21 +02:00
Daniel Stenberg
45b388fdc7
tests/server/util.c: remove use of strncpy
... and ban the function in code in this directory.

Closes #15213
2024-10-10 07:47:01 +02:00
Viktor Szakats
6cfb615e92
sws: fix unused static function with TCP_NODELAY undefined
Closes #15171
2024-10-07 12:31:21 +02:00
Daniel Stenberg
e9eda865df
warnless: remove curlx_sktosi and curlx_sitosk
They were only used by test server code. Use plain old typecasts there.

Closes #15153
2024-10-05 00:28:06 +02:00
Daniel Stenberg
c72cefea0f
select: use poll() if existing, avoid poll() with no sockets
poll() on macOS 10.12 was deemed broken in 2016 when we discovered that
it misbehaves when provided with no sockets to wait for. The
HAVE_POLL_FINE is used to mark a poll() implementation that behaves
correctly: it *should* still wait the timeout time.

curl has therefore opted to use select() on Apple operating systems ever
since. To avoid the risk that this or other breakage cause problems.

However, using select() internally is also bad because it suffers from
problems when using file descriptors beyond 1024.

This change makes poll() used if it is present, but if there is no
sockets to wait for it avoids using poll() and instead falls back to
select() - but without any sockets to wait for there is no 1024 problem.

This removes all previous special-handling involving HAVE_POLL_FINE.

ref: https://daniel.haxx.se/blog/2016/10/11/poll-on-mac-10-12-is-broken/

Closes #15096
2024-10-01 15:11:50 +02:00
Stefan Eissing
7f3d598276
CI: improvements in test reliability and performance
- CI default: raise parallelism to 20
- CI valgind: set parallelism to 6
- CI non-native: adapt parallelism for OS builds
- CI Windows: no longer ignore FTP, TFTP, MQTT and SMTP
- CI Windows: restrict test timeout to 10 minutes
- CI Windows: do not run tests for msh3 build
- tests, various: restrict curl invocation to ipv4 to avoid talking to
  any ipv6 test server running in parallel
- tests: requiring http/2 server no longer needs to mention http server
- test 190: use a fixed timeout of 10 seconds instead of %FTPTIME2, as
  that value rises under parallel load beyond what the server is waiting
  and then produces different results
- test 1540: add debug logging
- testrunner: add verify checks for http/2 and http/3 server

Closes #15040
2024-09-27 16:47:37 +02:00
Gabriel Marin
b70e8f4b9b
cleanup: added space around ternary expressions
Closes #14990
2024-09-25 08:02:01 +02:00
Viktor Szakats
22652a5a4c
curl: add options for safe/no CA bundle search (Windows)
Add `CURL_CA_SEARCH_SAFE` build-time option to enable CA bundle search
in the `curl` tool directory. The lookup method was already used to find
`.curlrc` and `_curlrc` (on Windows). On Windows it overrides the unsafe
default `SearchPath()` method.

Enable with:
- cmake: `-DCURL_CA_SEARCH_SAFE=ON`
- autotools: `--enable-ca-search-safe`
- raw: `CPPFLAGS=-DCURL_CA_SEARCH_SAFE`

On Windows, before this patch the whole `PATH` was searched for
a CA bundle. `PATH` may contain unwanted or world-writable locations,
including the current directory. Searching them all is convenient to
pick up any CA bundle, but not secure.

The Muldersoft curl distro implements such CA search via a custom
patch for Windows:
cd652d4792/patch/curl_tool_doswin.diff (L50)

MSYS2/mingw-w64 distro has also been rolling a patch solving this:
https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-curl/0001-Make-cURL-relocatable.patch
https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-curl/pathtools.c

Also add option to fully disable Windows CA search:
- cmake: `-DCURL_DISABLE_CA_SEARCH=ON`
- autotools: `--disable-ca-search`
- raw: `CPPFLAGS=-DCURL_DISABLE_CA_SEARCH`.

Both options are considered EXPERIMENTAL, with possible incompatible
changes or even (partial) removal in the future, depending on feedback.

An alternative, secure option is to embed the CA bundle into the binary.

Safe search can be extended to other platforms if necessary or useful,
by using `_NSGetExecutablePath()` (macOS),
`/proc/self/exe` (Linux/Cygwin), or `argv[0]`.

Closes #14582
2024-09-22 18:17:25 +02:00
Viktor Szakats
3efba94f77
cmake: allow building tests in unity mode
Makes building tests noticeably faster.

Apply changes/fixes/workarounds to make Unity work:
- rename test variables to avoid collisions or shadowing each other when
  combined into single units.
- add workaround to avoid applying `lib/memdebug.h` overrides to system
  headers declaring/defining `getaddrinfo()`/`freeaddrinfo()` for
  `tests/server/resolve.c`. This replaces a previous workaround that
  worked for that specific source.
- rename test macro `CTRL` clashing with Cygwin `sys/ioctl.h`.
- add include guard to `test.h`.

Also:
- exclude `tests/http/clients` which are all single-source. (like
  `docs/examples`.)

Build time improvements for tests:
- AppVeyor CI:
  - MSVC 2008, 2010: 1 minute faster (4m8s -> 2m56s, 3m19s -> 2m24s)
  - MSVC 2022 arm64: 3.5 minutes faster (10m18s -> 6m48s)
  before: https://ci.appveyor.com/project/curlorg/curl/builds/50522785
  after: https://ci.appveyor.com/project/curlorg/curl/builds/50522942
- GHA:
  - Cygwin: 1.5 minutes faster (3m13s -> 1m43s)
    before: https://github.com/curl/curl/actions/runs/10681535327/job/29605384398
    after: https://github.com/curl/curl/actions/runs/10680818726/job/29603130637
  - Windows:
    before: https://github.com/curl/curl/actions/runs/10680818713
    after: https://github.com/curl/curl/actions/runs/10683850187
    - MSYS2, mingw-w64: 1 minute faster
    - MSVC: 30 seconds faster (3m17s -> 2m48s)
  - macOS: double speed (39s -> 18s)
    before: https://github.com/curl/curl/actions/runs/10680818753/job/29603133447
    after: https://github.com/curl/curl/actions/runs/10683850174/job/29612914515
  - Linux: almost double speed (30/31s -> 18s)
    before: https://github.com/curl/curl/actions/runs/10681535311/job/29605387156
    after: https://github.com/curl/curl/actions/runs/10680818721/job/29603133976
  - non-native: no obvious effect.
    before: https://github.com/curl/curl/actions/runs/10680818722
    after: https://github.com/curl/curl/actions/runs/10683850187
  - Old Linux: Unity mode not supported by old CMake, no effect.

Closes #14765
2024-09-19 21:32:58 +02:00
Viktor Szakats
bc2f72b9ae
tidy-up: rename CURL_WINDOWS_APP to CURL_WINDOWS_UWP
Rename internal macro to make its purpose more obvious.

After this patch `grep -i uwp` shows all the code related to UWP.

Ref: https://curl.se/mail/lib-2024-09/0014.html
Closes #14881
2024-09-19 19:24:12 +02:00
Daniel Stenberg
fbf5d507ce
lib/src: white space edits to comply better with code style
... as checksrc now finds and complains about these.

Closes #14921
2024-09-19 14:59:12 +02:00
Viktor Szakats
888662b74a
tests: delete duplicate macro check
Follow-up to e9a7d4a1c8 #12376

Closes #14963
2024-09-19 12:43:54 +02:00
Stefan Eissing
8ad3597d2d
tests: testrunner reliability improvements
- perform torture tests with '-j2' for shorter runtime
- when waiting on test results overly long, log the tests
  waited for and eventually log the test log directories
  for easier analysis what is wrong in CI jobs.
- sockfilt.c: treat the windows errno 109 (ERROR_BROKEN_PIPE)
  as a socket closed by the client and do not exit.
- when verifying https server, do not in addition check
  the http server behind it also
- when tearing down the stunnel of a non-responsive https
  server, tear down the http server with it

Closes #14960
2024-09-18 23:09:16 +02:00
Gabriel Marin
5a263710f6
lib, src, tests: added space around ternary expressions
Closes #14912
2024-09-18 15:27:26 +02:00
Stefan Eissing
e70c22b622
tests: tweak lock file handling and timers
- when server are killed by a test case, do not wait for the server lock
  file to go away. These tests are mostly about client timeouts and the
  server will hang until killed.
- when killing a server successfully, check for a remaining lock file,
  log its existence and remove it.
- lower the delay timings on SLOWDOWN by half
- add SLOWDOWNDATA server command to only slow down the FTP data bytes,
  not the control ones.
- lower some timeout values

Closes #14835
2024-09-13 09:46:25 +02:00
Viktor Szakats
805bbf7c55
NTLM_WB: delete remains in tests, docs updates
Also update more docs to reflect the removal of NTLM_WB.

Follow-up to 50def7c881 #13249
Closes #14832
2024-09-10 14:13:36 +02:00
Viktor Szakats
a744b7bb45
server/getpart: delete unused code
Follow-up to 2610142139 #8378
Closes #14829
2024-09-09 16:57:36 +02:00
Daniel Stenberg
4ff04615a0
lib: use FMT_ as prefix instead of CURL_FORMAT_
For printf format defines used internally. Makes the code slighly
easier to read.

Closes #14764
2024-09-03 08:45:45 +02:00
Aki
a2bcec0ee0
openssl: fix the data race when sharing an SSL session between threads
The SSL_Session object is mutated during connection inside openssl,
and it might not be thread-safe. Besides, according to documentation
of openssl:

```
SSL_SESSION objects keep internal link information about the session
cache list, when being inserted into one SSL_CTX object's session
cache. One SSL_SESSION object, regardless of its reference count,
must therefore only be used with one SSL_CTX object (and the SSL
objects created from this SSL_CTX object).
```
If I understand correctly, it is not safe to share it even in a
single thread.

Instead, serialize the SSL_SESSION before adding it to the cache,
and deserialize it after retrieving it from the cache, so that no
concurrent write to the same object is infeasible.

Also
 - add a ci test for thread sanitizer
 - add a test for sharing ssl sessions concurrently
 - avoid redefining memory functions when not building libcurl, but
   including the soruce in libtest
 - increase the concurrent connections limit in sws

Notice that there are fix for a global data race for openssl which
is not yet release. The fix is cherry pick for the ci test with
thread sanitizer.
d8def79838

Closes #14751
2024-09-02 23:35:44 +02:00
Viktor Szakats
d620ec6791
CI: add test timeouts, more cmake build tests, fix VS2010 C warning
- GHA/macos: set timeout for test runs.
  Double the value for autotools to fit torture tests.
- GHA/cygwin: reduce test run timeout for autotools.
- GHA/cygwin: enable building tests with cmake.
- GHA/windows: enable building tests with MSYS cmake.
- GHA/windows: enable building tests with MSVC UWP.
- appveyor: enable building tests with VS2008 x86 and VS2010 x64.
- tests: add workaround compiler warnings when building with VS2010:
  ```
  tests\server\util.c(482): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(486): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(490): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(720): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(726): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(732): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(781): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(785): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  tests\server\util.c(789): warning C4306: 'type cast' : conversion from 'int' to 'void (__cdecl *)(int)' of greater size
  ```
  Ref: https://ci.appveyor.com/project/curlorg/curl/builds/50485633/job/4ujlwxod3cexmn2q#L1535

With this, tests are built in all GHA cygwin/windows jobs.

Timeouts avoid long runs with runaway issues, example:
https://github.com/curl/curl/actions/runs/10575522173

Closes #14700
2024-08-27 23:40:48 +02:00
Viktor Szakats
8b09138083
cmake: minor tidy-ups
- show `OpenSSL v3+` when detected (as in `./configure`).
  (this string also makes its way to `curl-config`.)
- prefer `unset(VAR)` over `set(VAR)`.
  Same effect, but `unset()` tells the intent unambiguously.
  https://cmake.org/cmake/help/latest/command/set.html
- drop "implementation" from an `option()` description.
- FindGSS: replace legacy keyword alias with modern alternative.
  https://cmake.org/cmake/help/latest/command/get_filename_component.html
- move `CURL_STATIC_CRT` logic next to its `option()`.
- improve order of `libcurl.pc`/`curl-config` variable init lines.
- tests: drop/shorten custom target names.
  They inflated generated make files by 550KB.
  Keep target name logic for sync between code snippets.
  Follow-up to a2ef5d36b3 #14660
- clear a variable after use.
- restore `STATUS` for `Features:`/`Protocols:` `message()`s:
  Without it the output goes to stderr, and appears in red in CMake GUI.
  It doesn't seem possible to show a line on stdout without leading
  underscores to match `curl -V` and `./configure` output.
  Partial revert of acbc6b703f #14197
- WindowsCache: move `HAVE_LINUX_TCP_H` into the header group.
- move strings to the same line as their `STRING` keyword.
- formatting in generated code.
- delete bogus comment.
- unfold lines for readability.
- fix a too long line. (for cmakelint)
- missing quotes, whitespace, comments.

Closes #14610
2024-08-26 11:00:08 +02:00
Viktor Szakats
a2ef5d36b3
cmake: sync code between test/example targets
- reuse local variable names.
- sync `PROJECT_LABEL`, add where missing.
- namespace all target names.
- bind header directories to each target.
- tests/server: limit `CURL_STATICLIB` to Windows (as in autotools.)
- drop functions with a single caller.

Closes #14660
2024-08-23 18:42:56 +02:00
Viktor Szakats
c04504885d
src: fix potential macro confusion in cmake unity builds
Sources used `lib/curlx.h` with both `ENABLE_CURLX_PRINTF` set and unset
before including it.

In a cmake "unity" batch where the first included source had it unset,
the next sources did not get the macros requested with
`ENABLE_CURLX_PRINTF` because `lib/curl.x` had already been included
without them.

Fix it by by making the macros enabled permanently and globally for
internal sources, and dropping `ENABLE_CURLX_PRINTF`.

This came up while testing unity builds with smaller batches. The full,
default unity build where all `src` is bundled up in a single unit, was
not affected.

Fixes:
```
$ cmake -B build -DCMAKE_UNITY_BUILD=ON -DCMAKE_UNITY_BUILD_BATCH_SIZE=15
$ make -C build
...
curl/src/tool_getparam.c: In function ‘getparameter’:
curl/src/tool_getparam.c:2409:11: error: implicit declaration of function ‘msnprintf’; did you mean ‘vsnprintf’? [-Wimplicit-function-declaration]
 2409 |           msnprintf(buffer, sizeof(buffer), "%" CURL_FORMAT_CURL_OFF_T "-",
      |           ^~~~~~~~~
      |           vsnprintf
curl/src/tool_getparam.c:2409:11: warning: nested extern declaration of ‘msnprintf’ [-Wnested-externs]
[...]
```

Reported-by: Daniel Stenberg
Bug: https://github.com/curl/curl/pull/14626#issuecomment-2301663491

Closes #14632
2024-08-22 10:45:04 +02:00
Viktor Szakats
b042d5297d
tidy-up: misc spelling (bit, ASCII)
Closes #14559
2024-08-15 15:30:09 +02:00
Viktor Szakats
9f23c8f201
cmake: fix and tidy up c-ares builds, enable in more CI jobs
- set `CARES_NO_DEPRECATED` to avoid c-ares deprecation warnings.
  Like autotools does.

- drop unused c-ares header directory when building libtests and test
  servers.

- disable TrackMemory (aka `CURLDEBUG`) feature for MSVC builds.
  It fails on test 558 and 1330, saying that TrackMemory isn't working.
  Left a FIXME about it.

- GHA/macos: enable c-ares in a cmake job.

- GHA/windows: enable c-ares in MSVC job.
  Fixes #14202

- GHA/windows: add c-ares mingw autotools job with tests.
  (move `--with-windows-unicode` option from 'default' job to
  this one to keep the former "default".)
  Put these tests on ignore for now:
  ```
  FAIL 472: 'aws-sigv4 with query' HTTP, aws-sigv4
  FAIL 1299: 'Send "OPTIONS *" with --request-target' HTTP, --request-target
  FAIL 1613: 'Send "OPTIONS *" with --request-target to a proxy' HTTP, HTTP proxy, --request-target
  ```
  Ref: https://github.com/curl/curl/actions/runs/10388126947/job/28765761441#step:15:5902

  472: fails with mingw despite Unicode enabled.
  1299, 1613: seem like a case of expanding '*' to a filename.

- GHA/windows: enable c-ares in mingw cmake job.

Closes #14541
2024-08-15 09:44:52 +02:00
Viktor Szakats
db39c668a8
cmake: sync up result variable names in Find modules
- bearssl, c-ares, gss, libpsl, libssh2, mbedtls:
  Before this patch these Find modules returned results via
  `<NAME>_INCLUDE_DIR` and `<NAME>_LIBRARY`.

  This patch makes them return `<NAME>_INCLUDE_DIRS` (note the `S`)
  and `<NAME>_LIBRARIES` like other modules already did.

- bearssl, mbedtls:
  Before this patch these Find modules allowed custom configuration
  via `<NAME>_INCLUDE_DIRS` (note the `S`).

  This patch makes them accept `<NAME>_INCLUDE_DIR`, like the rest of
  the modules did.

  Deprecate the old variables, but keep accepting them for
  compatibility.

- bearssl: add missing `mark_as_advanced()` call.

Closes #14542
2024-08-14 14:58:58 +02:00
Viktor Szakats
457427e03f
build: silence C4232 MSVC warnings in vcpkg ngtcp2 builds
Silence bogus MSVC warning C4232. Use the method already used
for similar cases earlier.

Also fixup existing suppressions to use pragma push/pop.

```
lib\vquic\curl_ngtcp2.c(709,40): error C2220: the following warning is treated as an error
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'client_initial': address of dllimport 'ngtcp2_crypto_client_initial_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'recv_crypto_data': address of dllimport 'ngtcp2_crypto_recv_crypto_data_cb' is not static, identity not guaran
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'encrypt': address of dllimport 'ngtcp2_crypto_encrypt_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'decrypt': address of dllimport 'ngtcp2_crypto_decrypt_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'hp_mask': address of dllimport 'ngtcp2_crypto_hp_mask_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'recv_retry': address of dllimport 'ngtcp2_crypto_recv_retry_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'update_key': address of dllimport 'ngtcp2_crypto_update_key_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'delete_crypto_aead_ctx': address of dllimport 'ngtcp2_crypto_delete_crypto_aead_ctx_cb' is not static, identit
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'delete_crypto_cipher_ctx': address of dllimport 'ngtcp2_crypto_delete_crypto_cipher_ctx_cb' is not static, ide
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'get_path_challenge_data': address of dllimport 'ngtcp2_crypto_get_path_challenge_data_cb' is not static, ident
```
Ref: https://github.com/curl/curl/actions/runs/10343459009/job/28627621355#step:10:30

Cherry-picked from #14495
Co-authored-by: Tal Regev
Ref: #14383
Closes #14510
2024-08-13 09:28:27 +02:00
Viktor Szakats
f3b14e1b0c
tidy-up: delete Makefile.inc from EXTRA_DIST
autotools is adding them automatically. Delete the few ones that were
also added manually.

Closes #14496
2024-08-12 14:54:47 +02:00
Viktor Szakats
c2889a7b41
cmake: more syntax tidy-up
- quote string literals.
  In the hope it improves syntax-highlighting and readability.

- use lowercase, underscore-prefixed local var names.
  As a hint for scope, to help readability.

- prefer `pkg_search_module` (over `pkg_check_modules`).
  They are the same, but `pkg_search_module` stops searching
  at the first hit.

- more `IN LISTS` in `foreach()`.

- OtherTests.cmake: clear `CMAKE_EXTRA_INCLUDE_FILES` after use.

- add `PROJECT_LABEL` for http/client and unit test targets.

- sync `Find*` module comments and formatting.

- drop a few local variables.

- drop bogus `CARES_LIBRARIES` from comment.

- unquote numeric literal.

Follow-up to acbc6b703f #14197
Closes #14388
2024-08-07 23:41:27 +02:00
Viktor Szakats
493c6d79eb
cmake: delete MSVC warning suppression for tests/server
Server code no longer produces this warning.

Closes #14428
2024-08-07 11:35:16 +02:00
Viktor Szakats
c6fb9895b6
cmake: cleanup header paths
- sync build-dir/source-dir header path order with autotools, by
  including build-dir first, then source-dir.
  This prevents out-of-tree builds breaking due to leftover generated
  headers in the source tree.

- tests/unit: move `src` ahead of `libtest` in header path, syncing with
  autotools.

- stop adding non-existing generated `include` dir to header path.
  There are no generated `include` headers and this directory is either
  missing in out-of-tree builds or the same as the one already added
  globally via the root `CMakeLists.txt`.

- lib: stop adding a duplicate source include directory to the header
  path.
  It's already added globally via the root `CMakeLists.txt`.

- lib: stop adding the project root to the header path.

- docs/examples: drop internal header paths.
  Examples do not and should not use internal headers.

- replace `curl_setup_once.h` in comments with `curl_setup.h`,
  the header actually used, and also referred to in autotools comments.

- add comment why we need `src` in include path for `tests/server`.

- add quotes around header directories.

Closes #14416
2024-08-06 22:33:05 +02:00
Viktor Szakats
f81f351b9a
tidy-up: OS names
Use these words and casing more consistently across text, comments and
one curl tool output:
AIX, ALPN, ANSI, BSD, Cygwin, Darwin, FreeBSD, GitHub, HP-UX, Linux,
macOS, MS-DOS, MSYS, MinGW, NTLM, POSIX, Solaris, UNIX, Unix, Unicode,
WINE, WebDAV, Win32, winbind, WinIDN, Windows, Windows CE, Winsock.

Mostly OS names and a few more.

Also a couple of other minor text fixups.

Closes #14360
2024-08-04 19:17:45 +02:00
Viktor Szakats
acbc6b703f
cmake: tidy-ups
- tidy-up comments.
- use lowercase, underscore prefixed names for internal variables.
- use `IN LISTS` and `IN ITEMS` in `foreach()` loops.
- rename variable name `OUTPUT` to a more distinctive one.
- tidy-up `STREQUAL` syntax.
- delete commented code.
- indent/whitespace.

Closes #14197
2024-08-03 20:49:15 +02:00
Viktor Szakats
f518c73a87
cmake: drop unused internal variable
Closes #14361
2024-08-03 09:22:26 +02:00
Viktor Szakats
998b17ea7f
windows: fix UWP builds, add GHA job
Add new job to test building for UWP (aka `CURL_WINDOWS_APP`).

Fix fallouts when building for UWP:
- rand: do not use `BCryptGenRandom()`.
- cmake: disable using win32 LDAP.
- cmake: disable telnet.
- version_win32: fix code before declaration.
- schannel: disable `HAS_MANUAL_VERIFY_API`.
- schannel: disable `SSLSUPP_PINNEDPUBKEY`
  and make `schannel_checksum()` a stub.
  Ref: e178fbd40a #1429
- schannel: make `cert_get_name_string()` a failing stub.
- system_win32: make `Curl_win32_impersonating()` a failing stub.
- system_win32: try to fix `Curl_win32_init()` (untested).
- threads: fix to use `CreateThread()`.
- src: disable searching `PATH` for the CA bundle.
- src: disable bold text support and capability detection.
- src: disable `getfiletime()`/`setfiletime()`.
- tests: make `win32_load_system_library()` a failing stub.
- tests/server/util: make it compile.
- tests/server/sockfilt: make it compile.
- tests/lib3026: fix to use `CreateThread()`.

See individual commits for build error details.

Some of these fixes may have better solutions, and some may not work
as expected. The goal of this patch is to make curl build for UWP.

Closes #13870
2024-06-05 00:52:24 +02:00
Viktor Szakats
75f182ffb6
tests: delete redundant !MSDOS guard
This fix was supposed to be committed earlier, but ended up missing from
the final commit.

Follow-up to e9a7d4a1c8 #12376
Closes #13878
2024-06-04 18:47:18 +02:00
Dan Fandrich
4a7bb40dea tests: don't log buffer length in throwing away message
It's not available at that point, and it will be written in the
non-error case right afterward.
2024-06-03 22:21:09 -07:00
Dan Fandrich
bc21c505e4 tests: log "Throwing away" messages before throwing away
In case the read that follows hangs we'll get a clue as to what it was
doing.
2024-06-03 17:00:42 -07:00
Viktor Szakats
59dc9f7e69
build: untangle CURLDEBUG and DEBUGBUILD macros
`CURLDEBUG` is meant to enable memory tracking, but in a bunch of cases,
it was protecting debug features that were supposed to be guarded with
`DEBUGBUILD`.

Replace these uses with `DEBUGBUILD`.

This leaves `CURLDEBUG` uses solely for its intended  purpose: to enable
the memory tracking debug feature.

Also:
- autotools: rely on `DEBUGBUILD` to enable `checksrc`.
  Instead of `CURLDEBUG`, which worked in most cases because debug
  builds enable `CURLDEBUG` by default, but it's not accurate.
- include `lib/easyif.h` instead of keeping a copy of a declaration.
- add CI test jobs for the build issues discovered.

Ref: https://github.com/curl/curl/pull/13694#issuecomment-2120311894
Closes #13718
2024-05-28 08:12:00 +02:00
Viktor Szakats
0e176cabe4
cmake: whitespace, formatting/tidy-up in comments
Also correct casing in a few option descriptions.

Closes #13711
2024-05-27 18:07:10 +02:00
Viktor Szakats
0c49ea4ff2
tests: tidy up types in server code
Cherry-picked from #13489
Closes #13610
2024-05-13 11:16:26 +02:00