Fix FTP protocol to flush the pingpong's send buffer before receiving a
response from the server, as it may never come otherwise.
Fixes FTP/FTPS tests with `CURL_DBG_SOCK_WBLOCK=90` set.
Closes#14452
- tidy up two `MATCHES` expression by avoiding macros expansion and
adding quotes. Then convert then to `STREQUAL` to match other places
in the code doing the same checks.
- fix setting `_ALL_SOURCE` for AIX to match what autotools does.
- delete stray `_ALL_SOURCE` reference from `lib/config_riscos.h`
- simplify/fix two `STREQUAL ""` checks.
The one in the `openssl_check_symbol_exists()` macro succeeded
regardless of the value. The other could return TRUE when
`CMAKE_OSX_SYSROOT` was undefined.
- delete code for CMake versions (<3.7) we no longer support.
- prefer `LIST(APPEND ...)` to extend `CURL_LIBS`.
- use `CURL_LIBS` to add the `network` lib for Haiku.
Before this patch it was done via raw C flags. I could not test this.
- move `_WIN32_WINNT`-related code next to each other.
It also moves detection to the top, allowing more code to use
the result.
- merge two `WIN32` blocks.
- rename internal variables to underscore + lowercase.
- unwrap a line, indent, whitespace.
Closes#14450
- quote string literals.
In the hope it improves syntax-highlighting and readability.
- use lowercase, underscore-prefixed local var names.
As a hint for scope, to help readability.
- prefer `pkg_search_module` (over `pkg_check_modules`).
They are the same, but `pkg_search_module` stops searching
at the first hit.
- more `IN LISTS` in `foreach()`.
- OtherTests.cmake: clear `CMAKE_EXTRA_INCLUDE_FILES` after use.
- add `PROJECT_LABEL` for http/client and unit test targets.
- sync `Find*` module comments and formatting.
- drop a few local variables.
- drop bogus `CARES_LIBRARIES` from comment.
- unquote numeric literal.
Follow-up to acbc6b703f#14197Closes#14388
- rely on the new flush to handle blocked sends. No longer
do simulated EAGAIN on (partially) blocked sends with their
need to handle repeats.
- fix some debug handling CURL_SMALLREQSEND env var
- add some assertings in request.c for affirming we do it right
- enhance assertion output in test_16 for easier analysis
Closes#14435
```
lib/krb5.c:343:39: warning: cast from 'void **' to 'unsigned char **' increases required alignment from 2 to 8 [-Wcast-align]
(unsigned char **)&_gssresp.value,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
Seen on macOS Intel with Apple clang and brew heimdal 7.8.0_1.
Closes#14433
- make mentioning `-v` on the curl command line increase the
verbosity of the trace output
- related discussion https://github.com/curl/curl/discussions/13810
- make a single -v revert all previous -v+ changes
- make --no-verbose also reset all trace configs
Closes#13977
- sync build-dir/source-dir header path order with autotools, by
including build-dir first, then source-dir.
This prevents out-of-tree builds breaking due to leftover generated
headers in the source tree.
- tests/unit: move `src` ahead of `libtest` in header path, syncing with
autotools.
- stop adding non-existing generated `include` dir to header path.
There are no generated `include` headers and this directory is either
missing in out-of-tree builds or the same as the one already added
globally via the root `CMakeLists.txt`.
- lib: stop adding a duplicate source include directory to the header
path.
It's already added globally via the root `CMakeLists.txt`.
- lib: stop adding the project root to the header path.
- docs/examples: drop internal header paths.
Examples do not and should not use internal headers.
- replace `curl_setup_once.h` in comments with `curl_setup.h`,
the header actually used, and also referred to in autotools comments.
- add comment why we need `src` in include path for `tests/server`.
- add quotes around header directories.
Closes#14416
Replicate the method used by autotools and cmake to enable libidn2.
This way `lib/curl_setup.h` sets `USE_LIBIDN2` automatically.
Before this patch, `USE_LIBIDN2` was enabled directly, shortcutting
internal logic prioritizing IDN backends.
(This is academic now because `Makefile.mk` no longer supports
other IDN backends. But still useful for clarity.)
Closes#14421
Also use an `#undef` hack for CMake Unity builds to avoid the previously
included `memdebug.h` header messing up the declarations pulled in by
`uv.h`:
```
In file included from ~/curl/bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:88:
In file included from ~/curl/src/tool_operate.c:54:
In file included from /usr/local/Cellar/libuv/1.48.0/include/uv.h:71:
In file included from /usr/local/Cellar/libuv/1.48.0/include/uv/unix.h:34:
/Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk/usr/include/netdb.h:269:7: error: expected parameter declarator
void freeaddrinfo(struct addrinfo *);
^
~/curl/lib/memdebug.h:167:31: note: expanded from macro 'freeaddrinfo'
curl_dbg_freeaddrinfo(data, __LINE__, __FILE__)
^
```
Follow-up to 38d334e3e1#14298Closes#14399
If a request containing two headers that have equivalent prefixes (ex.
"x-amz-meta-test:test" and "x-amz-meta-test-two:test2") AWS expects the
header with the shorter name to come first. The previous implementation
used `strcmp` on the full header. Using the example, this would result
in a comparison between the ':' and '-' chars and sort
"x-amz-meta-test-two" before "x-amz-meta-test", which produces a
different "StringToSign" than the one calculated by AWS.
Test 1976 verifies
Closes#14370
Bring setting ciphers with WolfSSL in line with other SSL backends,
to make the curl interface more consistent across the backends.
Now the tls1.3 ciphers are set with the --tls13-ciphers option, when
not set the default tls1.3 ciphers are used. The tls1.2 (1.1, 1.0)
ciphers are set with the --ciphers option, when not set the default
tls1.2 ciphers are used. The ciphers available for the connection
are now a union of the tls1.3 and tls1.2 ciphers.
This changes the behaviour for WolfSSL when --ciphers is set, but
--tls13-ciphers is not set. Now the ciphers set with --ciphers
are combined with the default tls1.3 ciphers, whereas before solely
the ciphers of --ciphers were used.
Thus before when no tls1.3 ciphers were specified in --ciphers,
tls1.3 was completely disabled. This might not be what the user
expected, especially as this does not happen with OpenSSL.
Closes#14385
Bring setting ciphers with mbedTLS in line with other SSL backends,
to make the curl interface more consistent across the backends.
Now the tls1.3 ciphers are set with the --tls13-ciphers option, when
not set the default tls1.3 ciphers are used. The tls1.2 (1.1, 1.0)
ciphers are set with the --ciphers option, when not set the default
tls1.2 ciphers are used. The ciphers available for the connection
are now a union of the tls1.3 and tls1.2 ciphers.
This changes the behaviour for mbedTLS when --ciphers is set, but
--tls13-ciphers is not set. Now the ciphers set with --ciphers
are combined with the default tls1.3 ciphers, whereas before solely
the ciphers of --ciphers were used.
Thus before when no tls1.3 ciphers were specified in --ciphers,
tls1.3 was completely disabled. This might not be what the user
expected, especially as this does not happen with OpenSSL.
Closes#14384
- replace the counting of upload lengths with the new eos send flag
- improve frequency of stream draining to happen less on events where it
is not needed
- this PR is based on #14220
http2, cf-h2-proxy: fix EAGAINed out buffer
- in adjust pollset and shutdown handling, a non-empty `ctx->outbufq`
must trigger send polling, irregardless of http/2 flow control
- in http2, fix retry handling of blocked GOAWAY frame
test case improvement:
- let client 'upload-pausing' handle http versions
Closes#14253
Improve handling of shutdown when sending gets blocked.
Add workaround for <https://github.com/wolfSSL/wolfssl/issues/7784>
where wolfSSL keeps on adding close notify messages to its outgoing
buffer on ever attempt.
Closes#14376
Make sure that `io_need` is cleared and set at the filter operations.
Add some more tracing for shutdown situations.
Improve shutdown handling for blocked sends. OpenSSL is a bit tricksy
here that it only reports WANT_WRITE on SSL_shutdown(), but never on
SSL_read() on blocked sends. So we need to use both.
At last, set SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER when available since we
are not always retrying sends from the very same address, as testing
showed.
Closes#14375
Initializes 'no_signal' to TRUE, so that a call to sigpipe_apply() after
init ignores the signal (unless CURLOPT_NOSIGNAL) is set.
I have read the existing code multiple times now and I think it gets the
initial state reversed this missing to ignore.
Regression from 17e6f06ea3
Reported-by: Rasmus Thomsen
Fixes#14344Closes#14390
wolfSSL supports setting certificates/private keys from memory blobs
which allow us to implement both CURLOPT_SSLCERT_BLOB and
CURLOPT_SSLKEY_BLOB options.
Closes#14018
Implement the `--enable-versioned-symbols` feature available in
`./configure` for CMake.
Enable with `-DCURL_LIBCURL_VERSIONED_SYMBOLS=ON`. Customize the version
prefix with `-DCURL_LIBCURL_VERSIONED_SYMBOLS_PREFIX=MYPREFIX_`. By
default the prefix matches what autotools uses.
When enabled, the feature is detected and a warning shown if
unavailable. (E.g. on Apple and Windows, it isn't.)
Included `HIDDEN {};` to match autotools, though I don't know if it's
necessary, useful or making any difference.
Differences from the autotools implementation:
- soversion is dynamic instead of hard-coded.
- omits referencing non-curl symbols.
- allows prefix/flavour override.
- more universal feature detection.
- doesn't rely on the in-repo `lib/libcurl.vers.in` file.
Also:
- add mbedTLS and BearSSL versioned symbol prefix support to autotools.
- enable this option in an old-linux job.
Follow-up to 7cc2e8b349Fixes#14349Closes#14378
- Determine which connection cache is or will be used by the easy handle
and perform connection upkeep on that cache.
This change allows curl_easy_upkeep to be effective on easy handles that
are using a shared connection cache, either from a user created shared
connection cache or a user created multi which has its own shared
connection cache.
Prior to this change curl_easy_upkeep would upkeep the connection cache
for the easy handle only if that cache was from the multi owned by the
easy handle (ie curl_easy_perform was previously called and there's a
connection cache exclusive to the easy handle in
data->multi_easy->conn_cache).
Ref: https://curl.se/mail/lib-2024-01/0016.html
Closes https://github.com/curl/curl/pull/12677
- For HTTPS if http2-prior-knowledge is set then only offer h2 (HTTP/2)
alpn to the server for protocol negotiation.
Prior to this change both HTTP/2 ("h2") and HTTP/1.1 ("http/1.1") were
offered for ALPN when http2-prior-knowledge was set.
CURL_HTTP_VERSION_2_PRIOR_KNOWLEDGE (tool: --http2-prior-knowledge) is
meant to send non-TLS HTTP requests HTTP/2 when it is known the server
supports them. However when HTTPS is used then it attempts to first
negotiate the connection with ALPN. In that case the user likely does
not want to offer http/1.1 to the server as an acceptable protocol.
Reported-by: kit-ty-kate@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/9963
Closes https://github.com/curl/curl/pull/14266
Use these words and casing more consistently across text, comments and
one curl tool output:
AIX, ALPN, ANSI, BSD, Cygwin, Darwin, FreeBSD, GitHub, HP-UX, Linux,
macOS, MS-DOS, MSYS, MinGW, NTLM, POSIX, Solaris, UNIX, Unix, Unicode,
WINE, WebDAV, Win32, winbind, WinIDN, Windows, Windows CE, Winsock.
Mostly OS names and a few more.
Also a couple of other minor text fixups.
Closes#14360
- tidy-up comments.
- use lowercase, underscore prefixed names for internal variables.
- use `IN LISTS` and `IN ITEMS` in `foreach()` loops.
- rename variable name `OUTPUT` to a more distinctive one.
- tidy-up `STREQUAL` syntax.
- delete commented code.
- indent/whitespace.
Closes#14197
Since data can be held in connection filter buffers when sending gives
EAGAIN, add methods to query this and perform flushing of those buffers.
The transfer loop will continue sending until all upload data is
processed and the connection is flushed.
- add `CF_QUERY_SEND_PENDING` to query filters
- add `CF_CTRL_DATA_SEND_FLUSH` to flush filters
- change `Curl_req_want_send()` to query the connection
if it needs flushing
- use `Curl_req_want_send()` to determine the POLLOUT
in the PERFORMING multi state
- implement flush handling in the HTTP/2 connection filter
Closes#14271
Adds a `bool eos` flag to send methods to indicate that the data
is the last chunk the invovled transfer wants to send to the server.
This will help protocol filters like HTTP/2 and 3 to forward the
stream's EOF flag and also allow to EAGAIN such calls when buffers
are not yet fully flushed.
Closes#14220
- check that `struct ssl_peer` is only initialized once
- fix vtls peer init to run only once
- check in peer init that hostname is not empty, fail otherwise
Closes#14152
Replace Curl_resolv_unlock() with Curl_resolv_unlink():
-replace inuse member with refcount in Curl_dns_entry
- pass Curl_dns_entry ** to unlink, so it gets always cleared
- solve potential (but unlikley) UAF in FTP's handling of looked up
Curl_dns_entry. Esp. do not use addr information after unlinking an entry.
In reality, the unlink will not free memory, as the dns entry is still
referenced by the hostcache. But this is not safe and relying on no other
code pruning the cache in the meantime.
- pass permanent flag when adding a dns entry instead of fixing timestamp
afterwards.
url.c: fold several static *resolve_* functions into one.
Closes#14195
- in DEBUGBUILD, all specifying if true random numbers
are desired or simulated ones via CURL_ENTROPY
- allows to use randoms in other DEBUG checks to not
interfere with the CURL_ENTROPY
- without this change, any Curl_rand() use will alter
results of some AUTHENTICATION methods like DIGEST
Closes#14264
Fixes:
```
curl\lib\vtls\cipher_suite.c(193,3): error C2220: the following warning is treated as an error
curl\lib\vtls\cipher_suite.c(193,3): warning C4310: cast truncates constant value
```
Closes#14341
Raise the limit for certification information from 10 thousand to 100
thousand bytes. Certificates can be larger than 10k.
Change the infof() debug output to add '...' at the end when the max
limit it can handle is exceeded.
Reported-by: Sergio Durigan Junior
Fixes#14352Closes#14354
- multi.c: when ratelimiting a transfer stops (MSTATE_RATELIMITING ->
MSTATE_PERFORMING), run the MSTATE_PERFORMING state right away
- urldata.h: factor out upload and download progress counters into a
struct, use that for passing these to progress update functions
- progress.c/getinfo.c: change names of moved progress counters
- progress.c: use new structs and a helper struct to factor repeated
calculation into static helpers
Closes#14335
Set the initial stream window size to 64KB and increase that to the 10MB
we used to start with on the first server reply, unless a rate limit is
in effect.
Continously monitory changes to the transfers rate limit and adjust the
stream window size accordingly. `max_recv_speed` is a transfer propert
that can be changed during processing by a callback.
Closes#14326
The MSVC compiler cannot have forward declaration with const and static
variable, causing this error:
```
curl\lib\vtls\vtls.c(417,44): warning C4132: 'Curl_ssl_multi': const object should be initialized
```
Ref: #14276Closes#14305
Turns out that MAC != OSX, despite what these names otherwise mean and
what's suggested by source code comments. "MAC" in fact means Darwin
(aka Apple), not macOS. "OSX" means macOS.
GitHub bumped the macos-14 runner default to Xcode 15.4, hitting the
llvm@15 incompatibility bug by default. Meaning the previous workaround
for the SDK bug is necessary.
This patch extend the workaround to not apply to mobile OS variants.
Follow-up to ff784af461#14159
Reported-by: Sergey
Confirmed-by: Marcel Raad
Fixes#14269Closes#14275
