- Make curl_multi_waitfds consistent with the documentation.
Issue Addressed:
- The documentation of curl_multi_waitfds indicates that users should
be able to call curl_multi_waitfds with a NULL ufds. However, before
this change, the function would return CURLM_BAD_FUNCTION_ARGUMENT.
- Additionally, the documentation suggests that users can use this
function to determine the number of file descriptors (fds) needed.
However, the function would stop counting fds if the supplied fds
were exhausted.
Changes Made:
- NULL ufds Handling: curl_multi_waitfds can now accept a NULL ufds if
size is also zero.
- Counting File Descriptors: If curl_multi_waitfds is passed a NULL
ufds, or the size of ufds is insufficient, the output parameter
fd_count will return the number of fds needed. This value may be
higher than actually needed but never lower.
Testing:
- Test 2405 has been updated to cover the usage scenarios described
above.
Fixes https://github.com/curl/curl/issues/15146
Closes https://github.com/curl/curl/pull/15155
- Parse the input string without modifying it.
Prior to this change a segfault could occur if the input string was
const because the tokenizer modified the input string. For example if
the user set CURLOPT_COOKIELIST to a const string then libcurl would
likely cause a crash when modifying that string. Even if the string was
not const or a crash did not occur there was still the incorrect and
unexpected modification of the user's input string.
This issue was caused by 30da1f59 (precedes 8.11.0) which refactored
some options parsing and eliminated the copy of the input string. Also,
an earlier commit f88cc654 incorrectly cast the input pointer when
passing it to strtok.
Co-authored-by: Daniel Stenberg
Closes https://github.com/curl/curl/pull/15826
build:
- autotools: fix to build generated sources for the `tidy` target.
- autotools: allow passing custom clang-tidy options via
`CURL_CLANG_TIDYFLAGS` env.
- cmake: add `CURL_CLANG_TIDY` option to configure for `clang-tidy`.
Also add:
- `CLANG_TIDY` variable to customize the `clang-tidy` tool.
- `CURL_CLANG_TIDYFLAGS` to pass custom options to `clang-tidy`.
- apply `--enable-werror` and `-DCURL_WERROR=ON` to `clang-tidy`.
CI/GHA:
- add clang-tidy job for Linux, using autotools and clang-tidy v18.
This one needs to disable `clang-analyzer-valist.Uninitialized`
to avoid false positives:
https://github.com/llvm/llvm-project/issues/40656
Duration: 5.5 minutes
- add clang-tidy job for macOS, using cmake and clang-tidy v19.
This one also covers tests and examples, and doesn't hit the false
positives seen with llvm v18 and earlier.
Duration: 4.5 minutes
- Linux/macOS: skip installing test dependencies when not building or
running tests.
fix fallouts reported by `clang-tidy`:
- lib:
- cf-h2-proxy: unused assignment in non-debug builds.
- cf-socket: silence warning.
FIXME: https://github.com/curl/curl/pull/15825#issuecomment-2561867769
- ftp: NULL passed to `strncmp()`.
- http2: NULL-ptr deref.
- mprintf: silence warning.
- src/tool_writeout: NULL passed to `fputs()`.
- examples:
- invalid file pointers.
- missing `fclose()`.
- tests:
- http/clients/hx-download: memory leaks on error.
- http/clients/hx-download: memory leak on repeat `-r` option.
- server: double `fclose()`.
https://www.man7.org/linux/man-pages/man3/fclose.3.html
- server: invalid file pointer/handle.
- server/getpart: unused assignments.
- server/mqttd: leak on failed `realloc()`.
- server/tftpd: NULL passed to `strcmp()`.
Closes#15825
When a QUIC TLS session announced early data support and
'CURLSSLOPT_EARLYDATA' is set for the transfer, send initial request and
body (up to the 128k we buffer) as 0RTT when curl is built with
ngtcp2+gnutls.
QUIC 0RTT needs not only the TLS session but the QUIC transport
paramters as well. Store those and the earlydata max value together with
the session in the cache.
Add test case for h3 use of this. Enable quic early data in nghttpx for
testing.
Closes#15667
We started using codeql for static code analysis in 7183f5acc3,
June 2020.
Since then, not a single commit has been merged into the source code
repository citing codeql as source or reason. Yet, it keeps getting
updated and we get constant reminders to upgrade the pinning it to the
latest hash.
During 4.5 years with intense development and significant code churn.
While Coverity, scan-build and CodeSonar have belped us point out many
mistakes, codeql has remained silent (or had false positives).
For this little gain, I think we spend a disproportionate amount of work
on codeql maintanance.
We can try again in a future if we think it improves.
Assisted-by: Viktor Szakats
Closes#15798
And use it from src and tests.
Syncing this functionality between platforms and build targets.
Also: Stop redefining `O_BINARY` in src, and use a local macro with
the same effect. `O_BINARY` is used in `CURL_SET_BINMODE()` to decide
if this functionality is supported, and redefining it makes this check
pass always in unity builds. The check is required for Apple OS, because
it offers a `setmode()` function, successfully detected by both CMake
and autotools, but that function has a different functionality and
signature than that expected by `CURL_SET_BINMODE()`.
Also:
- drop MetaWare High C (MS-DOS) support for set binmode.
- tests/libtest/Makefile.inc: dedupe comments.
- lib/curl_setup_once.h: tidy up feature guards for `io.h`, `fcntl.h`.
Ref: #15652Closes#15787
Let CURLINFO_QUEUE_TIME_T count only the time a transfer spends queued,
including possible redirect requests.
Add var 'time_queue' for reporting the time in write outs.
Add test for verifying correct reporting.
Closes#15512
Allowing --variable read a portion of provided files, makes curl work on
partial files for any options that accepts strings. Like --data and others.
The byte offset is provided within brackets, with a semicolon separator
like: --variable name@file;[100-200]"
Inspired by #14479
Assisted-by: Manuel Einfalt
Test 784 - 789. Documentation update provided.
Closes#15739
lib : remove all hyper code
configure: stop detecting hyper
docs: no more mention of hyper
tests: mo more special-handling of hyper builds
CI: no jobs using hyper
Closes#15120
For MinGW this is `-DNO_OLDNAMES`, with MSVC it is
`-D_CRT_DECLARE_NONSTDC_NAMES=0`.
There have been some support for this before this patch.
After this patch this is extended to all examples.
(And also the standalone http/client programs, if here.)
Cherry-picked from #15652Closes#15789
Described in detail in internal doc TLS-SESSIONS.md
Main points:
- use a new `ssl_peer_key` for cache lookups by connection filters
- recognize differences between TLSv1.3 and other tickets
* TLSv1.3 tickets are single-use, cache can hold several of them for a peer
* TLSv1.2 are reused, keep only a single one per peer
- differentiate between ticket BLOB to store (that could be persisted) and object instances
- use put/take/return pattern for cache access
- remember TLS version, ALPN protocol, time received and lifetime of ticket
- auto-expire tickets after their lifetime
Closes#15774
When a specific hostname matched, and only a password is set before
another machine is specified in the netrc file, the parser would not be
happy and stop there and return the password-only state. It instead
continued and did not return a match.
Add test 2005 to verify this case
Regression from e9b9bba, shipped in 8.11.1.
Reported-by: Ben Zanin
Fixes#15767Closes#15768
Adjusted test 186 to verify.
Regression in 9664d5a547, shipped in 8.11.1
Reported-by: IcedCoffeee on github
Assisted-by: Jay Satiro
Fixes#15761Closes#15762
It would previously wrongly also catch function calls to function names
ending with 'return'
Amended test1185.
Reported-by: Stefan Eissing
Closes#15764
`Makefile.mk` supported MS-DOS and Amiga, but `./configure` also
supported them in a better tested and more flexible way.
This patch also adds CMake support for MS-DOS/DJGPP and Amiga OS 3.
`Makefile.mk` was not maintained. Delete it in favour of first-tier
build methods.
Also include some non-MS-DOS/AmigaOS-specific tidy-up, see details at
the end of this message.
Details:
- fix/silence all MS-DOS/DJGPP build warnings and issues.
- add MS-DOS support to cmake.
- default to `ENABLE_THREADED_RESOLVER=OFF` for MS-DOS.
- add support for `WATT_ROOT`.
- use static libcurl with MS-DOS.
- fixup default CMake suffixes/prefixes for DJGPP.
- disable hidden symbols for MS-DOS. Not supported on MS-DOS.
- opt-in MS-DOS into `USE_UNIX_SOCKETS`.
- improve MS-DOS support in autotools.
- default to `--disable-threaded-resolver` for MS-DOS.
- make sure to use `close_s()` (from Watt-32) with autotools and cmake.
`Makefile.mk` used it before this patch.
- GHA: add DJGPP cmake (~30s) and autotools (~60s) build jobs.
Also build tests and examples with cmake.
- improve AmigaOS support in autotools:
- configure: detect `CloseSocket()` when it's a macro.
- configure: fix `IoctlSocket` detection on AmigaOS.
- curl-amissl.m4: pass AmiSSL libs to tests/servers.
- add AmigaOS3 support to cmake:
- cmake: fix `HAVE_IOCTLSOCKET_CAMEL` and
`HAVE_IOCTLSOCKET_CAMEL_FIONBIO` detections.
- set necessary system libs.
- add AmiSSL support.
- inet_ntop, inet_pton: fix using it for AmigaOS. cmake detects them,
and they did not compile with AmigaOS.
- cmake: better sync `gethostname` detection with autotools.
Fixes detection for AmigaOS, where `gethostname` is a macro.
- cmake: fix `sys/utime.h` detection on AmigaOS.
- cmake: force-disable `getaddrinfo` for AmigaOS.
- cmake: tweak threading and static/shared default for AmigaOS.
- cmake: rely on manual variable `AMIGA` to enable the platform.
- GHA: add AmigaOS cmake and autotools (~45s) jobs.
Also build tests and examples with cmake.
- INSTALL: update MS-DOS and AmigaOS build instructions.
- amigaos: fix `-Wpointer-sign` and
`zero or negative size array '_args'` in `Printf()`.
- amigaos: fix `-Wpointer-sign`
- amigaos: fix `-Wredundant-decls` `errno` and `h_errno`.
- amigaos: brute-force silence `lseek()` size warnings.
- amigaos: server/resolve: silence `-Wdiscarded-qualifiers`.
- amigaos: server/resolve: fix `-Wpointer-sign`.
- amigaos: fix `CURL_SA_FAMILY_T` type.
- nonblock: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
`ioctl` is also detected, but fails when used. Make the above override
it for a successful build.
Authored-by: Darren Banfi
Fixes#15537Closes#15603
- tftpd: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
- tftpd: tidy-up conditional code.
- curl: set stack size to 16384 for AmigaOS3/4
Overriding the default 4096.
Suggested-by: Darren Banfi
Ref: https://github.com/curl/curl/pull/15543#issuecomment-2498783123
Ref: https://wiki.amigaos.net/wiki/Controlling_Application_Stack
- functypes.h: fix `SEND_QUAL_ARG2` for AmigaOS.
- tftp: add missing cast in sendto() call for AmigaOS.
- getinfo: fix warning with AmigaOS.
- tool_operate: silence warning with AmigaOS
- amigaos: fix building libtests due to missing `RLIMIT_NOFILE`.
- curl_gethostname: silence warning for AmigaOS.
- ftp: silence `-Wtype-limits` for AmigaOS.
- libtest: fix timeval initialization for AmigaOS.
- examples: fix `timeval` initialization for AmigaOS.
- examples: silence warning for AmigaOS.
- configure: fix IPv6 detection for cross-builds.
- netrc: fix to build with AmigaOS cleanly.
- buildinfo: detect and add `DOS` tag for MS-DOS builds.
- buildinfo: add `AMIGA` to buildinfo.txt in auttools.
- build: move `USE_WATT32` macro definition to cmake/configure.
Non-MS-DOS/AmigeOS-specific tidy-ups:
- configure: sync `sa_family_t` detection with cmake.
- configure: sync `ADDRESS_FAMILY` detection signals with cmake.
- doh: use `CURL_SA_FAMILY_T`.
- lib: drop mingw-specific `CURL_SA_FAMILY_T` workaround.
- cmake: extend instead of override check-specific
configurations/requirements.
This allows to honor global requirements added earlier.
Necessary for AmigaOS for example.
- cmake: omit warning on disabled IPv6 for MS-DOS and AmigaOS.
No IPv6 support on these platforms. Also sync with autotools.
- lib1960: use libcurl `inet_pton()` wrapper.
- cmake: detect LibreSSL (to match autotools).
- cmake: say the specific OpenSSL flavour detected.
- hostip: add missing `HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID` guard.
- lib: simplify classic mac feature guards.
Follow-up to a8861b6ccd#9764Closes#15543
Prefix (or suffix) curl-defined macro and function names with `curl`.
To avoid collisions with non-curl CMake and to make them recognizable
as curl-specific calls.
Closes#15498
Using sscanf() is not a (security) problem in itself, but we strongly
discorage using it for parsing input since it is hard to use right, easy
to mess up and often makes for sloppy error checking.
Allow it in examples and tests
Closes#15687
Designed to aid converting off from sscanf parsers. sscanf is hard to
use right, easy to mess up and often makes for sloppy error checking.
The new parsers allow more exact and pedandic parsing.
This new set of functions should be possible to use (and extend) and
switch over other libcurl parser code to use going forward.
Adapts the following to use the new functions:
- altsvc.c
- hsts.c
- http_aws_sigv4.c
Bonus: fewer memory copies, fewer stack buffers.
Test: Unit test1664
Docs: docs/internals/STRPARSE.md
Closes#15692
This fix impacts ws-data.c and ws-pingpong.c. sleep() replaced with
system PROCESS_DELAY_() having the same resolution as usleep().
Fixes#15711Closes#15712
Signed-off-by: Randall S. Becker <randall.becker@nexbridge.ca>
The date parser function is very forgiving and skips most "irrelevant"
characters in its hunt for a date to figure out. Therefore it is
important to make sure the date string is properly null terminated so
that it does not accidentally parse a piece of whatever text follows
after the date.
Add test483: test (overly) long expire dates in cookies
Closes#15709
- build: fix to exclude 'documentation' tests when building
the documentation is explicitly disabled. Both for cmake
and `./configure`.
Reported-by: Daniel Engberg
Fixes#15703
- test481, test482: fix for builds with no manual.
Reported-by: Daniel Engberg
Fixes#15703
- configure: fix to always detect Perl. Running tests require
it when run. Before this patch Perl wasn't detected when
documentation/manual/embedded-CA were all disabled,
making tests fail to start.
- test1177: add keyword `documentation`. It depends no
`curl_version_info.3`.
- GHA/linux: test cmake and `./configure` with docs and
manual disabled and tests run.
Closes#15704
The base64 mime encoder stalls when it cannot encode a full 3 byte input
set into the read buffer. The workaround for this limitation was
incomplete and could lead to stalled transfers when the last chunk to
upload was smaller than 4 bytes.
Use a tmp buffer on small reads to allow mime encoders more space to put
their things.
Add test case reproducing the issue and fix.
Reported-by: Alexis Savin
Fixes#15688Closes#15691
Fix regression that no longer printed the error messages about expired
certificates in openssl. Add test case for openssl/gnutls/wolfssl.
Fixes#15612
Reported-by: hiimmat on github
Closes#15613
Add test_02_33 to run with various values for the multi option
CURLMOPT_MAX_HOST_CONNECTIONS and CURLOPT_FRESH_CONNECT to trigger
connection pool limit handling code.
Closes#15494
The MSVC UWP job in CI did not actually enable UWP. Fix this and
the fallouts discovered after enabling it.
- GHA/windows: make sure to enable UWP in MSVC vcpkg UWP job.
Use the CMake options and C flags already used for mingw-w64, but use
`WINAPI_FAMILY_PC_APP` instead of the deprecated `WINAPI_FAMILY_APP`.
(The former is not supported by mingw-w64, so leave it there as-is.)
Follow-up to cb22cfca69#14077
- GHA/windows: by default the MSVC UWP job became 2x-3x slower than
others after actually enabling UWP. Most of it is caused by
CMake/MSBuild automatically building full APPX containers for each
`.exe` target. This includes 21 CMake feature detections. Each
detection app is built into a 15MB APPX project, with code signing,
logos, etc. Example:
https://github.com/curl/curl/actions/runs/12056968170/job/33620610958
Disable this overhead for curl build targets via custom
`CMAKE_VS_GLOBALS` options. I've found no way to apply them to feature
detection targets, so those remain slow.
- cmake: automatically enable Unicode for UWP builds. It's required.
Also stop enabling it manually in the existing CI job.
- tests: fix `getpid()` use for Windows UWP:
```
tests\server\util.c(281,21): warning C4013: 'getpid' undefined; assuming extern returning int
```
Ref: https://github.com/curl/curl/actions/runs/12061215311/job/33632904249#step:11:38
- src/tool_doswin: disable `GetLoadedModulePaths()` for UWP.
mingw-w64 UWP was okay with this, but MS SDK headers are not.
This makes `--dump-module-paths` return empty for UWP builds.
```
src\tool_doswin.c(620,3): error C2065: 'MODULEENTRY32': undeclared identifier
src\tool_doswin.c(626,11): warning C4013: 'CreateToolhelp32Snapshot' undefined; assuming extern returning int
src\tool_doswin.c(626,36): error C2065: 'TH32CS_SNAPMODULE': undeclared identifier
src\tool_doswin.c(632,7): warning C4013: 'Module32First' undefined; assuming extern returning int
```
Ref: https://github.com/curl/curl/actions/runs/12055081933/job/33614629930#step:9:35
- examples: fix `websocket.c` to include `winsock2.h` before `windows.h`
to make it build with MSVC UWP:
```
include\curl\curl.h(143,16): error C2061: syntax error: identifier 'curl_socket_t'
include\curl\curl.h(143,16): error C2059: syntax error: ';'
include\curl\curl.h(417,52): error C2146: syntax error: missing ')' before identifier 'curlfd'
include\curl\curl.h(417,38): error C2081: 'curl_socket_t': name in formal parameter list illegal
```
Ref: https://github.com/curl/curl/actions/runs/12055317910/job/33615644427#step:14:126
- GHA/windows: silence linker warning with MSVC UWP builds:
```
LINK : warning LNK4075: ignoring '/INCREMENTAL' due to '/OPT:ICF' specification
```
Ref: https://github.com/curl/curl/actions/runs/12055696808/job/33616629610#step:11:38
- GHA/windows: set `/INCREMENTAL:NO` for all MSVC jobs to improve
performance a little.
- cmake: show `UWP` platform flag.
Ref: #15652Closes#15657
- make sure that a match that returns a username also returns a
password, that should be blank if no password is found
- fix handling of multiple logins for same host where the password/login
order might be reversed.
- reject credentials provided in the .netrc if they contain ASCII control
codes - if the used protocol does not support such (like HTTP and WS do)
Reported-by: Harry Sintonen
Add test 478, 479 and 480 to verify. Updated unit 1304.
Closes#15586
`libcurl.pc` `Libs.private` (since 8.11.0, and in `Libs` before 7.20.0)
and `curl-config` `--static-libs` (since 7.17.1, and in `Libs` between
7.7.2-7.25.0). This included all flags inherited from the environment,
in addition to those coming from dependency detections.
To avoid spilling all linker flags inherited from the environment to
the libcurl config files, this patch omits them all, except `-L`, `-F`,
`--library-path=` and `-framework` options, which are still passed.
The rationale for the exceptions is that `LIBS` is passed as-is, and
`LDFLAGS`, `LIBS` are the canonical way to pass custom libs options
to a build. `LIBS` may not work without a matching custom libpath.
This brings autotools behaviour closer to cmake, and `curl-config`
closer to `libcurl.pc`.
Follow-up to 9f56bb608e#14681
Follow-up to 4c8adc8fee
Reported-by: Peter Marko
Fixes#15533Closes#15550
Using this option (only available in debug builds) makes curl always
call curl_easy_duphandle() on the handle before using it.
To help us catch curl_easy_duphandle() mistakes better.
Add a CI job using this.
Bonus: the previous runtests option -e is now also supported as
--test-event
Closes#15504
The netrc init was only done in the Curl_open, meaning that a duplicated
handle would not get inited properly.
Added test 2309 to verify. It does netrc auth with a duplicated handle.
Regression from 3b43a05e00
Reported-by: tranzystorekk on github
Fixes#15496Closes#15503
Build wolfSSL master with
./configure --prefix=/path --enable-ip-alt-name --enable-quic
--enable-earlydata --enable-psk --enable-opensslcoexist
and configure curl with openssl + wolfssl. Normal tests run.
pytest session resumption fails, as wolfssl does not handle the
new_session callback without opensslextra right now.
Closes#15481
Use `__APPLE__` macro to detect Apple OS instead of relying on
the string in `CURL_OS`.
This also fixes detection with default CMake builds where `CURL_OS` is
`Darwin`. The code before this patch was expecting this substring in
lowercase.
Closes#15461
- appveyor: add build-only job for clang-cl.
- cmake: `-pedantic-errors` enables `-Werror,-Wlanguage-extension-token`
automatically, which makes `__int64` detection fail.
Explictly disable this compiler warning for clang-cl to make the
feature detection work and to accept `__int64` in the source code.
- cmake: disable `-Wlanguage-extension-token` warning for clang-cl
to fix these when encountering `__int64`:
```
lib/formdata.c(797,29): error : extension used [-Werror,-Wlanguage-extension-token]
lib/warnless.c(117,33): error : extension used [-Werror,-Wlanguage-extension-token]
lib/warnless.c(60,28): message : expanded from macro 'CURL_MASK_SCOFFT'
lib/warnless.c(59,38): message : expanded from macro 'CURL_MASK_UCOFFT'
include\curl/system.h(352,40): message : expanded from macro 'CURL_TYPEOF_CURL_OFF_T'
```
- make `__GNUC__` warning suppressions apply to `__clang__` too.
Necessary for clang-cl, which defines the latter, but not the former.
(Regular clang defines both.)
- examples: fix clang-cl compiler warning in `http2-upload.c`.
```
docs\examples\http2-upload.c(56,5): error : no previous prototype for function 'my_gettimeofday' [-Werror,-Wmissing-prototypes]
docs\examples\http2-upload.c(56,1): message : declare 'static' if the function is not intended to be used outside of this translation unit
```
- unit2604: add missing `#pragma GCC diagnostic pop`.
Follow-up to e53523fef0#14859
- unit1652: limit compiler warning suppression to GCC.
They do not affect clang builds.
Follow-up to 71cf0d1fca#14772Closes#15449
