A newly introduced use of getsockname() in the cli tool makes it require
the ascii wrapper module, which is not available outside of the library:
as the tool only uses the address family field (binary), disable
wrappers outside of libcurl.
Fix setsockopt() parameter type mismatch using a (void *) cast.
Sync ILE/RPG binding.
Closes#13930
Instead of bolting on the extra CRLF to the final header - as that makes
the behavior inconsistent and not as documented. The final CRLF is now
also made unconditional, just like it is for HTTP.
Reported-by: dogma
Bug: https://curl.se/mail/lib-2024-06/0033.htmlCloses#13925
- clarify Curl_xfer_setup() with RECV/SEND flags and different calls for
which socket they operate on. Add a shutdown flag for secondary
sockets
- change Curl_xfer_setup() calls to new functions
- implement non-blocking connection shutdown at the end of receiving or
sending a transfer
Closes#13913
- new struct curl_pollfds and struct curl_waitfds
- add structs and methods to init/add/cleanup an array of pollfd and
struct curl_waitfd. Use in multi_wait() and multi_waitfds() to
populate the sets for polling.
- place USE_WINSOCK WSAEventSelect() setting into a separate loop over
all collected pfds
Closes#13900
This adds connection shutdown infrastructure and first use for FTP. FTP
data connections, when not encountering an error, are now shut down in a
blocking way with a 2sec timeout.
- add cfilter `Curl_cft_shutdown` callback
- keep a shutdown start timestamp and timeout at connectdata
- provide shutdown timeout default and member in
`data->set.shutdowntimeout`.
- provide methods for starting, interrogating and clearing
shutdown timers
- provide `Curl_conn_shutdown_blocking()` to shutdown the
`sockindex` filter chain in a blocking way. Use that in FTP.
- add `Curl_conn_cf_poll()` to wait for socket events during
shutdown of a connection filter chain.
This gets the monitoring sockets and events via the filters
"adjust_pollset()" methods. This gives correct behaviour when
shutting down a TLS connection through a HTTP/2 proxy.
- Implement shutdown for all socket filters
- for HTTP/2 and h2 proxying to send GOAWAY
- for TLS backends to the best of their capabilities
- for tcp socket filter to make a final, nonblocking
receive to avoid unwanted RST states
- add shutdown forwarding to happy eyeballers and
https connect ballers when applicable.
Closes#13904
- decouple need to recv/send from negotiation state, we need
this later in shutdown handling as well
- move ssl enums from urldata.h to vtls_int.h
- implement use of `connssl->io_need` in vtls.c. and all backends
Closes#13879
When reaching the set maximum limit of allowed connections, allow a new
connection anyway if the transfer is created for the (internal) purpose
of doing a DoH name resolve. Otherwise, unrelated "normal" transfers can
starve out new DoH requests making it impossible to name resolve for new
transfers.
Bug: https://curl.se/mail/lib-2024-06/0001.html
Reported-by: kartatz
Closes#13880
Add new job to test building for UWP (aka `CURL_WINDOWS_APP`).
Fix fallouts when building for UWP:
- rand: do not use `BCryptGenRandom()`.
- cmake: disable using win32 LDAP.
- cmake: disable telnet.
- version_win32: fix code before declaration.
- schannel: disable `HAS_MANUAL_VERIFY_API`.
- schannel: disable `SSLSUPP_PINNEDPUBKEY`
and make `schannel_checksum()` a stub.
Ref: e178fbd40a#1429
- schannel: make `cert_get_name_string()` a failing stub.
- system_win32: make `Curl_win32_impersonating()` a failing stub.
- system_win32: try to fix `Curl_win32_init()` (untested).
- threads: fix to use `CreateThread()`.
- src: disable searching `PATH` for the CA bundle.
- src: disable bold text support and capability detection.
- src: disable `getfiletime()`/`setfiletime()`.
- tests: make `win32_load_system_library()` a failing stub.
- tests/server/util: make it compile.
- tests/server/sockfilt: make it compile.
- tests/lib3026: fix to use `CreateThread()`.
See individual commits for build error details.
Some of these fixes may have better solutions, and some may not work
as expected. The goal of this patch is to make curl build for UWP.
Closes#13870
Introduce new notation for CURLOPT_INTERFACE / --interface:
ifhost!<interface>!<host>
Binding to an interface doesn't set the address, and an interface can
have multiple addresses.
When binding to an address (without interface), the kernel is free to
choose the route, and it can route through any device that can access
the target address, not necessarily the one with the chosen address.
Moreover, it is possible for different interfaces to have the same IP
address, on which case we need to provide a way to be more specific.
Factor out the parsing part of interface option, and add unit tests:
1663.
Closes#13719
Currently, we use `pipe` for `wakeup_create`, which requires ***two***
file descriptors. Furthermore, given its complexity inside, `pipe` is a
bit heavyweight for just a simple event wait/notify mechanism.
`eventfd` would be a more suitable solution for this kind of scenario,
kernel also advocates for developers to use `eventfd` instead of `pipe`
in some simple use cases:
Applications can use an eventfd file descriptor instead of a pipe
(see pipe(2) in all cases where a pipe is used simply to signal
events. The kernel overhead of an eventfd file descriptor is much
lower than that of a pipe, and only one file descriptor is required
(versus the two required for a pipe).
This change adds the new backend of `eventfd` for `wakeup_create` and
uses it where available, eliminating the overhead of `pipe`. Also, it
optimizes the `wakeup_create` to eliminate the system calls that make
file descriptors non-blocking by moving the logic of setting
non-blocking flags on file descriptors to `socketpair.c` and using
`SOCK_NONBLOCK` for `socketpair(2)`, `EFD_NONBLOCK` for `eventfd(2)`.
Ref:
https://man7.org/linux/man-pages/man7/pipe.7.htmlhttps://man7.org/linux/man-pages/man2/eventfd.2.htmlhttps://man7.org/linux/man-pages/man2/socketpair.2.htmlhttps://www.gnu.org/software/gnulib/manual/html_node/eventfd.htmlCloses#13874
- add special sauce to disable unwanted peer verification by mbedtls
when negotiating TLS v1.3
- add special sauce for MBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET
return code on *writing* TLS data. We assume the data had not been
written and EAGAIN.
- return correct Curl error code when peer verification failed.
- disable test_08_05 with 50 HTTP/1.1 connections, as mbedtls reports a
memory allocation failed during handshake.
- bump CI mbedtls version to 3.6.0
Fixes#13653Closes#13838
- similar to openssl, use a shared 'credentials' instance
among TLS connections with a plain configuration.
- different to openssl, a connection with a client certificate
is not eligible to sharing.
- document CURLOPT_CA_CACHE_TIMEOUT in man page
Closes#13795
Used for extracting:
- when used asking for a scheme, it will return CURLUE_NO_SCHEME if the
stored information was a guess
- when used asking for a URL, the URL is returned without a scheme, like
when previously given to the URL parser when it was asked to guess
- as soon as the scheme is set explicitly, it is no longer internally
marked as guessed
The idea being:
1. allow a user to figure out if a URL's scheme was set as a result of
guessing
2. extract the URL without a guessed scheme
3. this makes it work similar to how we already deal with port numbers
Extend test 1560 to verify.
Closes#13616
Previously a large table of ciphers was used to determine the default
ciphers and to lookup manually selected ciphers names.
With the lookup of the manually selected cipher names moved to
Curl_cipher_suite_walk_str() the large table is no longer needed for
that purpose.
The list of manually selected cipher can now be intersected with the
ciphers supported by Secure Transport (SSLGetSupportedCiphers()),
instead of using the fixed table for that.
The other use of the table was to filter the list of all supported
ciphers offered by Secure Transport to create a list of ciphers to
use by default, excluding ciphers in the table marked as weak.
Instead of using a complement based approach (exclude weak), switch
to using an intersection with a smaller list of ciphers deemed
appropriate.
Closes#13823
- Rename: Curl_sndbufset => Curl_sndbuf_init
- Rename: win_update_buffer_size => win_update_sndbuf_size
- Save the last set SO_SNDBUF size to compare against so that we can
avoid setsockopt calls every second.
This is a follow-up to 0b520e12 which moved the SO_SNDBUF update check
into cf-socket. This change improves it further by making the function
names easier to understand and reducing the amount of setsockopt calls.
Closes https://github.com/curl/curl/pull/13827
- determine the actual poll timeout *after* all sockets
have been collected. Protocols and connection filters may
install new timeouts during collection.
- add debug logging to test1533 where the mistake was noticed
Reported-by: Matt Jolly
Fixes#13782Closes#13825
- Move the code that updates the SO_SNDBUF size for Windows to
cf_socket_send.
Prior to this change the code was in readwrite_upload but the socket
filter is the more appropriate place because it applies to all sends.
Background:
For Windows users SO_SNDBUF (the total per-socket buffer size reserved
by Winsock for sends) is updated dynamically by libcurl during the
transfer. This is because Windows does not do it automatically for
non-blocking sockets and without it the performance of large transfers
may suffer.
Closes https://github.com/curl/curl/pull/13763
Refactors canon_query, so it could use the encoding part of the function
to use it in the path.
As the path doesn't encode '/', but encode '=', I had to add some
conditions to know If I was doing the query or path encoding.
Also, instead of adding a `bool in_path` variable, I use `bool
*found_equals` to know if the function was called for the query or path,
as found_equals is used only in query_encoding.
Test 472 verifies.
Reported-by: Alexander Shtuchkin
Fixes#13754Closes#13814
Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
The versions check wrongly complained and return error if the *minimum*
version was set to something less than 1.3. QUIC is always TLS 1.3, but
that means minimum 1.2 is still fine to ask for.
This also renames the local variable to make the mistake harder to make
in the future.
Regression shipped in 8.8.0
Follow-up to 3210101088
Reported-by: fds242 on github
Fixes#13799Closes#13802
local ftp upload tests sometimes failed with an invalid TLS record being
reported by gnutls. vsftp did log that the shutdown was not regarded as
clean, failing the control connection thereafter.
These changes make test_31_05 work reliable locally.
- on closing the SSL filter, shutdown READ *and* WRITE
- on closing, try a receive after shutdown is sent
- convert to DEBUGF to CURL_TRC_CF
Closes#13790
`CURLDEBUG` is meant to enable memory tracking, but in a bunch of cases,
it was protecting debug features that were supposed to be guarded with
`DEBUGBUILD`.
Replace these uses with `DEBUGBUILD`.
This leaves `CURLDEBUG` uses solely for its intended purpose: to enable
the memory tracking debug feature.
Also:
- autotools: rely on `DEBUGBUILD` to enable `checksrc`.
Instead of `CURLDEBUG`, which worked in most cases because debug
builds enable `CURLDEBUG` by default, but it's not accurate.
- include `lib/easyif.h` instead of keeping a copy of a declaration.
- add CI test jobs for the build issues discovered.
Ref: https://github.com/curl/curl/pull/13694#issuecomment-2120311894Closes#13718
It affected cmake-unity shared-curltool curldebug mingw-w64 gcc builds
when building the `testdeps` target.
Apply the solution already used in `lib/base64.c` and `lib/dynbuf.c`
to fix it.
Also update an existing GHA CI job to test the issue fixed.
```
In file included from curl/lib/version_win32.c:35,
from curl/_bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:145:
curl/lib/memdebug.h:52:14: error: redundant redeclaration of 'curl_dbg_logfile' [-Werror=redundant-decls]
52 | extern FILE *curl_dbg_logfile;
| ^~~~~~~~~~~~~~~~
In file included from curl/src/slist_wc.c:32,
from curl/_bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:4:
curl/lib/memdebug.h:52:14: note: previous declaration of 'curl_dbg_logfile' with type 'FILE *' {aka 'struct _iobuf *'}
52 | extern FILE *curl_dbg_logfile;
| ^~~~~~~~~~~~~~~~
curl/lib/memdebug.h:55:44: error: redundant redeclaration of 'curl_dbg_malloc' [-Werror=redundant-decls]
55 | CURL_EXTERN ALLOC_FUNC ALLOC_SIZE(1) void *curl_dbg_malloc(size_t size,
| ^~~~~~~~~~~~~~~
curl/lib/memdebug.h:55:44: note: previous declaration of 'curl_dbg_malloc' with type 'void *(size_t, int, const char *)' {aka 'void *(long long unsigned int, int, const char *)'}
55 | CURL_EXTERN ALLOC_FUNC ALLOC_SIZE(1) void *curl_dbg_malloc(size_t size,
| ^~~~~~~~~~~~~~~
[...]
curl/lib/memdebug.h:110:17: error: redundant redeclaration of 'curl_dbg_fclose' [-Werror=redundant-decls]
110 | CURL_EXTERN int curl_dbg_fclose(FILE *file, int line, const char *source);
| ^~~~~~~~~~~~~~~
curl/lib/memdebug.h:110:17: note: previous declaration of 'curl_dbg_fclose' with type 'int(FILE *, int, const char *)' {aka 'int(struct _iobuf *, int, const char *)'}
110 | CURL_EXTERN int curl_dbg_fclose(FILE *file, int line, const char *source);
| ^~~~~~~~~~~~~~~
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49840554/job/a4aoet17e9qnqx1a#L362
After: https://ci.appveyor.com/project/curlorg/curl/builds/49843735/job/hbo2uah2vj0ns523
Ref: #13689 (CI testing this PR with `DEBUGBUILD`/`CURLDEBUG`/shared-static combinations)
Depends-on: #13694
Depends-on: #13800Closes#13705
