Described in detail in internal doc TLS-SESSIONS.md
Main points:
- use a new `ssl_peer_key` for cache lookups by connection filters
- recognize differences between TLSv1.3 and other tickets
* TLSv1.3 tickets are single-use, cache can hold several of them for a peer
* TLSv1.2 are reused, keep only a single one per peer
- differentiate between ticket BLOB to store (that could be persisted) and object instances
- use put/take/return pattern for cache access
- remember TLS version, ALPN protocol, time received and lifetime of ticket
- auto-expire tickets after their lifetime
Closes#15774
It has been a synonym for `USE_OPENSSL` since
709cf76f6b (2015).
The few uses of this on GitHub also set `USE_OPENSSL` and
should be fine. Those which don't, please replace
`-DMACOS_SSL_SUPPORT` with `-DUSE_OPENSSL`.
Closes#15777
The compare_func() can violate the antisymmetric property required by
qsort. Specifically, when both aa->len == 0 and bb->len == 0, the
function returns conflicting results (-1 for compare_func(a, b) and -1
for compare_func(b, a)).
This violates the rules of qsort and may lead to undefined behavior,
including incorrect sorting or memory corruption in glibc [1].
Add a check to return 0 when both lengths are zero, ensuring proper
behavior and preventing undefined behavior in the sorting process.
Ref: https://www.qualys.com/2024/01/30/qsort.txt [1]
Closes#15778
For TLS backends that don't need these functions, they now use plain
NULL pointers instead of setting a function that does nothing.
Helps making it clearer that a specific TLS handler does not provide
anything specific for that action.
Closes#15772
When a specific hostname matched, and only a password is set before
another machine is specified in the netrc file, the parser would not be
happy and stop there and return the password-only state. It instead
continued and did not return a match.
Add test 2005 to verify this case
Regression from e9b9bba, shipped in 8.11.1.
Reported-by: Ben Zanin
Fixes#15767Closes#15768
On apple builds, the gssapi/ldap/securetransport headers deprecate
almost everything which leads to a wall of compiler warnings on use in
code.
Suppress those warning that may hide other warnings/errors.
Closes#15763
- make colliding vtls static function names unique.
- wolfssl: stop including an unused compatibility header.
- cmake: adapt detection logic for openssl+wolfssl coexist.
- wolfssl: fix to use native wolfSSL API in ECH codepath.
- openssl+wolfssl: fix ECH code to coexist.
Requires a post wolfSSL v5.7.4, recent master for `OPENSSL_COEXIST`
feature, and `CPPFLAGS=-DOPENSSL_COEXIST`.
Ref: https://github.com/wolfSSL/wolfssl/issues/8194Closes#15596
`Makefile.mk` supported MS-DOS and Amiga, but `./configure` also
supported them in a better tested and more flexible way.
This patch also adds CMake support for MS-DOS/DJGPP and Amiga OS 3.
`Makefile.mk` was not maintained. Delete it in favour of first-tier
build methods.
Also include some non-MS-DOS/AmigaOS-specific tidy-up, see details at
the end of this message.
Details:
- fix/silence all MS-DOS/DJGPP build warnings and issues.
- add MS-DOS support to cmake.
- default to `ENABLE_THREADED_RESOLVER=OFF` for MS-DOS.
- add support for `WATT_ROOT`.
- use static libcurl with MS-DOS.
- fixup default CMake suffixes/prefixes for DJGPP.
- disable hidden symbols for MS-DOS. Not supported on MS-DOS.
- opt-in MS-DOS into `USE_UNIX_SOCKETS`.
- improve MS-DOS support in autotools.
- default to `--disable-threaded-resolver` for MS-DOS.
- make sure to use `close_s()` (from Watt-32) with autotools and cmake.
`Makefile.mk` used it before this patch.
- GHA: add DJGPP cmake (~30s) and autotools (~60s) build jobs.
Also build tests and examples with cmake.
- improve AmigaOS support in autotools:
- configure: detect `CloseSocket()` when it's a macro.
- configure: fix `IoctlSocket` detection on AmigaOS.
- curl-amissl.m4: pass AmiSSL libs to tests/servers.
- add AmigaOS3 support to cmake:
- cmake: fix `HAVE_IOCTLSOCKET_CAMEL` and
`HAVE_IOCTLSOCKET_CAMEL_FIONBIO` detections.
- set necessary system libs.
- add AmiSSL support.
- inet_ntop, inet_pton: fix using it for AmigaOS. cmake detects them,
and they did not compile with AmigaOS.
- cmake: better sync `gethostname` detection with autotools.
Fixes detection for AmigaOS, where `gethostname` is a macro.
- cmake: fix `sys/utime.h` detection on AmigaOS.
- cmake: force-disable `getaddrinfo` for AmigaOS.
- cmake: tweak threading and static/shared default for AmigaOS.
- cmake: rely on manual variable `AMIGA` to enable the platform.
- GHA: add AmigaOS cmake and autotools (~45s) jobs.
Also build tests and examples with cmake.
- INSTALL: update MS-DOS and AmigaOS build instructions.
- amigaos: fix `-Wpointer-sign` and
`zero or negative size array '_args'` in `Printf()`.
- amigaos: fix `-Wpointer-sign`
- amigaos: fix `-Wredundant-decls` `errno` and `h_errno`.
- amigaos: brute-force silence `lseek()` size warnings.
- amigaos: server/resolve: silence `-Wdiscarded-qualifiers`.
- amigaos: server/resolve: fix `-Wpointer-sign`.
- amigaos: fix `CURL_SA_FAMILY_T` type.
- nonblock: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
`ioctl` is also detected, but fails when used. Make the above override
it for a successful build.
Authored-by: Darren Banfi
Fixes#15537Closes#15603
- tftpd: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
- tftpd: tidy-up conditional code.
- curl: set stack size to 16384 for AmigaOS3/4
Overriding the default 4096.
Suggested-by: Darren Banfi
Ref: https://github.com/curl/curl/pull/15543#issuecomment-2498783123
Ref: https://wiki.amigaos.net/wiki/Controlling_Application_Stack
- functypes.h: fix `SEND_QUAL_ARG2` for AmigaOS.
- tftp: add missing cast in sendto() call for AmigaOS.
- getinfo: fix warning with AmigaOS.
- tool_operate: silence warning with AmigaOS
- amigaos: fix building libtests due to missing `RLIMIT_NOFILE`.
- curl_gethostname: silence warning for AmigaOS.
- ftp: silence `-Wtype-limits` for AmigaOS.
- libtest: fix timeval initialization for AmigaOS.
- examples: fix `timeval` initialization for AmigaOS.
- examples: silence warning for AmigaOS.
- configure: fix IPv6 detection for cross-builds.
- netrc: fix to build with AmigaOS cleanly.
- buildinfo: detect and add `DOS` tag for MS-DOS builds.
- buildinfo: add `AMIGA` to buildinfo.txt in auttools.
- build: move `USE_WATT32` macro definition to cmake/configure.
Non-MS-DOS/AmigeOS-specific tidy-ups:
- configure: sync `sa_family_t` detection with cmake.
- configure: sync `ADDRESS_FAMILY` detection signals with cmake.
- doh: use `CURL_SA_FAMILY_T`.
- lib: drop mingw-specific `CURL_SA_FAMILY_T` workaround.
- cmake: extend instead of override check-specific
configurations/requirements.
This allows to honor global requirements added earlier.
Necessary for AmigaOS for example.
- cmake: omit warning on disabled IPv6 for MS-DOS and AmigaOS.
No IPv6 support on these platforms. Also sync with autotools.
- lib1960: use libcurl `inet_pton()` wrapper.
- cmake: detect LibreSSL (to match autotools).
- cmake: say the specific OpenSSL flavour detected.
- hostip: add missing `HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID` guard.
- lib: simplify classic mac feature guards.
Follow-up to a8861b6ccd#9764Closes#15543
Prefix (or suffix) curl-defined macro and function names with `curl`.
To avoid collisions with non-curl CMake and to make them recognizable
as curl-specific calls.
Closes#15498
Instead of redefining the `inline` keyword, introduce curl's own
`CURL_INLINE` macro and set it depending on the compiler's capabilities,
or use its value set via custom C flags.
Also keep honoring a custom `inline` macro, if set.
Closes#15523
- cmp-config.pl: add remaining exceptions. Sort list.
- drop unused `HAVE_SYS_WAIT_H`.
Follow-up to 50def7c881#13249
- drop unused `HAVE_FCHMOD`.
Follow-up to 03cb1ff4d6#12395
- autotools: stop promoting variables to macros:
`USE_NGTCP2_CRYPTO_*`, `USE_NGTCP2_H3`, `USE_OPENSSL_H3`,
`HAVE_LIBRESSL`.
They are not used in the source.
- cmake: drop unused `HAVE_O_NONBLOCK`,
`HAVE_DISABLED_NONBLOCKING`.
- lib: drop `NEED_MALLOC_H`.
It was used in manual-build cases for Amiga/MS-DOS/Windows/WinCE, but
never by autotools/cmake, thus apparently unnecessary.
- lib: drop unused `NEED_MEMORY_H`.
- lib: simplify classic mac feature guards,
drop `HAVE_EXTRA_STRICMP_H` and `HAVE_EXTRA_STRDUP_H`.
- autotools: drop unused `HAVE_GETHOSTBYNAME` detection.
- autotools: drop unused OpenSSL feature tests:
`HAVE_ERR_H`, `HAVE_PEM_H`, `HAVE_RSA_H`
- autotools: drop unused OpenSSL feature tests:
`HAVE_X509_H`, `HAVE_CRYPTO_H`, `HAVE_SSL_H`.
They performed a fallback check when the primary check missed
`openssl/x509.h`. Though if any other prefixed headers were found,
OpenSSL is already assumed detected.
The fallback check was looking for 3 unprefixed OpenSSL headers, and
if all found, marked OpenSSL found internally, but did not promote
it to `curl_config.h` via `USE_OPENSSL`. Meaning it either didn't do
anything or may have continued with an inconsistent state.
Added in d99c20f628 (2008)
At the time, there was an extra `AC_DEFINE(USE_SSLEAY, 1 ...` logic
after this code, which kicked in in the fallback case, but that code
was deleted in 709cf76f6b (2015)
Follow-up to 709cf76f6b
- autotools: drop `AC_SUBST()` where the value is explicitly set anyway
and the macro is unused.
- autotools: replace `AC_SUBST(VAR, 1)` with local variable assigments,
where the `@VAR@` macro is unused. Also dedupe the local variable if
there was a parallel one used for the same purpose.
- autotools: drop local feature variables that were never used.
- autotools: drop unused `CURL_CHECK_OPTION_NTLM_WB`,
`CURL_CHECK_NTLM_WB`.
Also stop setting unused `NTLM_WB_ENABLED` macro for VMS.
Follow-up to 50def7c881#13249
- autotools: drop unused `PKGADD_*`.
Follow-up to bae0d473f5#3331
- autotools: drop unused `CURL_NETWORK_LIBS`.
Follow-up to 3af75e18d6#14697Closes#15577
Designed to aid converting off from sscanf parsers. sscanf is hard to
use right, easy to mess up and often makes for sloppy error checking.
The new parsers allow more exact and pedandic parsing.
This new set of functions should be possible to use (and extend) and
switch over other libcurl parser code to use going forward.
Adapts the following to use the new functions:
- altsvc.c
- hsts.c
- http_aws_sigv4.c
Bonus: fewer memory copies, fewer stack buffers.
Test: Unit test1664
Docs: docs/internals/STRPARSE.md
Closes#15692
When employing eventfd for socketpair, there is only one file
descriptor. Closing that fd twice might result in fd corruption.
Thus, we should avoid closing the eventfd twice, following the
pattern in lib/multi.c.
Fixes#15725Closes#15727
Reported-by: Christian Heusel
The date parser function is very forgiving and skips most "irrelevant"
characters in its hunt for a date to figure out. Therefore it is
important to make sure the date string is properly null terminated so
that it does not accidentally parse a piece of whatever text follows
after the date.
Add test483: test (overly) long expire dates in cookies
Closes#15709
When a floating point precision or string width are provided as a
base-10 number, the code could miss to detect integer overflows if the
provided value was exactly 2147483648 or 2147483649 (2147483647 being
the maxium value a signed integer can hold).
The chance that such values would actually ever be used is slim.
This change fixes the detection to also cover those edge cases.
Closes#15699
The channel binding data dynbuf was not set correctly making it fail with
CURLE_TOO_LARGE too easily.
Reported-by: galen11 on github
Fixes#15685Closes#15694
When compiled with BUILD_STATIC_LIBS=ON and SHARE_LIB_OBJECT=OFF compile
definition CURL_STATICLIB was not set for static library. It seems to be
copy-paste error in the lib/CMakeLists.txt.
This pull request fixes it.
Closes#15695
The base64 mime encoder stalls when it cannot encode a full 3 byte input
set into the read buffer. The workaround for this limitation was
incomplete and could lead to stalled transfers when the last chunk to
upload was smaller than 4 bytes.
Use a tmp buffer on small reads to allow mime encoders more space to put
their things.
Add test case reproducing the issue and fix.
Reported-by: Alexis Savin
Fixes#15688Closes#15691
Other programs (Postman, Chrome, Python request) use a 16 byte cnonce
and there are instances of server-side implementations that don't
support the larger lengths curl used previously.
Fixes#15653
Reported-by: Florian Eckert
Closes#15670
Fix regression that no longer printed the error messages about expired
certificates in openssl. Add test case for openssl/gnutls/wolfssl.
Fixes#15612
Reported-by: hiimmat on github
Closes#15613
- Restore some necessary options for builds without HTTP and MQTT.
The logic to turn off a segment of options in builds without HTTP and
MQTT was too expansive. Those builds (such as FTP-only builds) could not
use options such as CURLOPT_URL or CURLOPT_USERNAME etc.
Prior to this change 30da1f59 (precedes 8.11.0) refactored the options
processing and caused this issue.
Reported-by: Yoshimasa Ohno
Fixes https://github.com/curl/curl/issues/15634
Closes https://github.com/curl/curl/pull/15640
The MSVC UWP job in CI did not actually enable UWP. Fix this and
the fallouts discovered after enabling it.
- GHA/windows: make sure to enable UWP in MSVC vcpkg UWP job.
Use the CMake options and C flags already used for mingw-w64, but use
`WINAPI_FAMILY_PC_APP` instead of the deprecated `WINAPI_FAMILY_APP`.
(The former is not supported by mingw-w64, so leave it there as-is.)
Follow-up to cb22cfca69#14077
- GHA/windows: by default the MSVC UWP job became 2x-3x slower than
others after actually enabling UWP. Most of it is caused by
CMake/MSBuild automatically building full APPX containers for each
`.exe` target. This includes 21 CMake feature detections. Each
detection app is built into a 15MB APPX project, with code signing,
logos, etc. Example:
https://github.com/curl/curl/actions/runs/12056968170/job/33620610958
Disable this overhead for curl build targets via custom
`CMAKE_VS_GLOBALS` options. I've found no way to apply them to feature
detection targets, so those remain slow.
- cmake: automatically enable Unicode for UWP builds. It's required.
Also stop enabling it manually in the existing CI job.
- tests: fix `getpid()` use for Windows UWP:
```
tests\server\util.c(281,21): warning C4013: 'getpid' undefined; assuming extern returning int
```
Ref: https://github.com/curl/curl/actions/runs/12061215311/job/33632904249#step:11:38
- src/tool_doswin: disable `GetLoadedModulePaths()` for UWP.
mingw-w64 UWP was okay with this, but MS SDK headers are not.
This makes `--dump-module-paths` return empty for UWP builds.
```
src\tool_doswin.c(620,3): error C2065: 'MODULEENTRY32': undeclared identifier
src\tool_doswin.c(626,11): warning C4013: 'CreateToolhelp32Snapshot' undefined; assuming extern returning int
src\tool_doswin.c(626,36): error C2065: 'TH32CS_SNAPMODULE': undeclared identifier
src\tool_doswin.c(632,7): warning C4013: 'Module32First' undefined; assuming extern returning int
```
Ref: https://github.com/curl/curl/actions/runs/12055081933/job/33614629930#step:9:35
- examples: fix `websocket.c` to include `winsock2.h` before `windows.h`
to make it build with MSVC UWP:
```
include\curl\curl.h(143,16): error C2061: syntax error: identifier 'curl_socket_t'
include\curl\curl.h(143,16): error C2059: syntax error: ';'
include\curl\curl.h(417,52): error C2146: syntax error: missing ')' before identifier 'curlfd'
include\curl\curl.h(417,38): error C2081: 'curl_socket_t': name in formal parameter list illegal
```
Ref: https://github.com/curl/curl/actions/runs/12055317910/job/33615644427#step:14:126
- GHA/windows: silence linker warning with MSVC UWP builds:
```
LINK : warning LNK4075: ignoring '/INCREMENTAL' due to '/OPT:ICF' specification
```
Ref: https://github.com/curl/curl/actions/runs/12055696808/job/33616629610#step:11:38
- GHA/windows: set `/INCREMENTAL:NO` for all MSVC jobs to improve
performance a little.
- cmake: show `UWP` platform flag.
Ref: #15652Closes#15657
The ECH feature cannot be built without HTTPS RR.
ECH automatically implied HTTPS RR in `./configure` but not in CMake,
winbuild, documentation.
Also update documentation and CI configs.
Follow-up to a362962b72#11922Closes#15648
- GHA/windows: switch mingw-w64 UWP CI job to use UCRT.
`msvcr120_app` was missing `getch()` for example.
Follow-up to f988842d85#15637
This job tests compiling for UWP correctly, but the the resulting
`curl.exe` still doesn't look like a correct UWP app, now exiting
on startup with: `curl: error initializing curl library`.
- tool_getpass: restore `getch()` for UWP builds.
Follow-up to f988842d85#15637
- schannel: silence `-Werror=null-dereference` warning in mingw-w64 UWP:
```
lib/vtls/schannel_verify.c: In function 'Curl_verify_host':
lib/vtls/schannel_verify.c:558:33: error: null pointer dereference [-Werror=null-dereference]
558 | for(i = 0; i < alt_name_info->cAltEntry; ++i) {
| ~~~~~~~~~~~~~^~~~~~~~~~~
lib/vtls/schannel_verify.c:559:50: error: null pointer dereference [-Werror=null-dereference]
559 | PCERT_ALT_NAME_ENTRY entry = &alt_name_info->rgAltEntry[i];
| ~~~~~~~~~~~~~^~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/12022656065/job/33515255397?pr=15638#step:19:27
Follow-up to 9640a8ef6f#15421
- GHA/windows: fix `find` command in MSVC job step.
Follow-up to 5f9411f953#15380
- GHA/windows: drop unnecessary `windowsappcompat` lib from mingw-w64
UWP job. Also drop related MSYS2 package.
- GHA/windows: cmake 3.31.0 still invokes `windres` with wrong options
with mingw-w64 UPW. Update curl version in comment accordingly.
- GHA/windows: tidy up mingw-w64 UWP spec logic, limit it to gcc.
- GHA/windows: update comments on `curl.exe` UWP startup errors.
Closes#15638
Issue is reproducible for me if I have made request with multi handle,
then I make request that will take very long and then I make request
that should be fast again, however what happens it is that it seems
to think that timeout was not changed and it makes it not call initial
`CURLMOPT_TIMERFUNCTION`.
Closes#15627
It was missing while detecting `wolfSSL_DES_ecb_encrypt`,
`wolfSSL_BIO_new` and `wolfSSL_BIO_set_shutdown`.
We have not seen it causing issues in stable wolfSSL releases as of
v5.7.4, until a recent commit in wolfSSL master, which broke detections:
```
curl/CMakeFiles/CMakeScratch//CheckSymbolExists.c:8:19: error: ‘wolfSSL_BIO_new’ undeclared (first use in this function); did you mean ‘wolfSSL_CTX_new’?
curl/CMakeFiles/CMakeScratch//CheckSymbolExists.c:8:19: error: ‘wolfSSL_BIO_set_shutdown’ undeclared (first use in this function); did you mean ‘wolfSSL_set_shutdown’?
```
This in turn disabled `HTTPS-proxy` and failed related pytests:
https://github.com/curl/curl/actions/runs/11953800545/job/33324250039?pr=15620
wolfSSL source diff causing the regression:
be70bea687..c06f65a8ac
The wolfSSL build says:
```
Note: Make sure your application includes "wolfssl/options.h" before any other wolfSSL headers.
You can define "WOLFSSL_USE_OPTIONS_H" in your application to include this automatically.
```
This patch makes sure to follow this rule across the curl codebase.
Also:
- include `wolfssl/options.h` first in `lib/vtls/wolfssl.c`.
It was preceded by `wolfssl/version.h`, which did not cause issues.
Background for the pre-existing include order:
Ref: deb9462ff2#3903
Ref: https://curl.se/mail/lib-2015-04/0069.html
Bug: https://github.com/curl/curl/pull/15620#issuecomment-2491872463
Follow-up to d68a121266#14064Closes#15623
- make sure that a match that returns a username also returns a
password, that should be blank if no password is found
- fix handling of multiple logins for same host where the password/login
order might be reversed.
- reject credentials provided in the .netrc if they contain ASCII control
codes - if the used protocol does not support such (like HTTP and WS do)
Reported-by: Harry Sintonen
Add test 478, 479 and 480 to verify. Updated unit 1304.
Closes#15586
- fix socket/sockindex confusion on writes:
The callstack used to end with `Curl_write_plain()` accepting a socket
till 7.87.0. This call got swapped for `Curl_conn_send()`, expecting
a sockindex. `socket_write()` was updated accordingly. Its callers
missed it and continued operating on sockets: `do_sec_send()`,
`sec_write()`, passing it down the stack and `Curl_conn_send()`
resolving it as if it were a sockindex.
It affected FTP Kerberos authentication.
Discovered through MSVC warnings:
```
curl\lib\krb5.c(652,28): warning C4244: 'function': conversion from 'curl_socket_t' to 'int', possible loss of data
curl\lib\krb5.c(654,28): warning C4244: 'function': conversion from 'curl_socket_t' to 'int', possible loss of data
curl\lib\krb5.c(656,26): warning C4244: 'function': conversion from 'curl_socket_t' to 'int', possible loss of data
curl\lib\krb5.c(657,26): warning C4244: 'function': conversion from 'curl_socket_t' to 'int', possible loss of data
curl\lib\krb5.c(665,24): warning C4244: 'function': conversion from 'curl_socket_t' to 'int', possible loss of data
curl\lib\krb5.c(666,24): warning C4244: 'function': conversion from 'curl_socket_t' to 'int', possible loss of data
```
Ref: https://github.com/curl/curl/actions/runs/11846599621/job/33014592805#step:9:32
Follow-up to 5651a36d1a#10280
Bug: https://github.com/curl/curl/pull/15549#issuecomment-2474154067Fixes#15582
- fix uninitialized buffer:
```
curl\lib\krb5.c(288,1): warning C4701: potentially uninitialized local variable '_gssresp' used
```
Ref: https://github.com/curl/curl/actions/runs/11848626645/job/33020501026?pr=15585#step:9:31
- silence unreachable code compiler warning:
```
curl\lib\krb5.c(370,1): warning C4702: unreachable code
```
Ref: https://github.com/curl/curl/actions/runs/11848626645/job/33020501026?pr=15585#step:9:30Closes#15585
krb5 defines `strtok_r` for Windows unconditionally in its public
header:
dc5554394e/src/include/win-mac.h (L214-L215)
resulting in this warning:
```
lib\strtok.h(31,9): warning C4005: 'strtok_r': macro redefinition
C:\vcpkg\installed\x64-windows\include\win-mac.h(215,9):
see previous definition of 'strtok_r'
```
The krb5 macro collides with curl's internal definition, in case
the `strtok_r` function is undetected and falling back to a local
replacement.
Reported-by: Tal Regev
Bug: https://github.com/curl/curl/pull/15549#issuecomment-2468251761Closes#15564
A recent update caused CMake builds to mis-detect this symbol on iOS.
Auto-detection also seems redundant given that it's a Windows-only
function and most Windows builds were already opted-in.
Drop detections and use it in all Windows builds with large file support
enabled.
Feature history:
- pririotizing for Windows: aaacd02466#14678
- Windows opt-in cmake: 8e74c0729d#11950
- Windows opt-in: aa6c94c5bf#11944
- use in libcurl: 9c7165e96a#11918
- use in example: 817d1c0106
Regression from 8e34505776#15164
Reported-by: Maarten Billemont
Fixes#15525Closes#15526
