Commit Graph

14227 Commits

Author SHA1 Message Date
Stefan Eissing
433d73033e
url: connection reuse on h3 connections
- When searching for existing connections, interpret the
  default CURL_HTTP_VERSION_2TLS as "anything goes". This
  will allow us to reuse HTTP/3 connections better
- add 'http/1.1' as allowed protocol identifier in Alt-Svc
  files
- add test_02_0[345] for testing protocol selection on
  provided alt-svc files

Fixes #14890
Reported-by: MacKenzie
Closes #14966
2024-09-20 23:34:40 +02:00
Daniel Stenberg
876047d1c3
libssh2: use the Curl_* memory functions to avoid memdebug
This prevents our torture tests from detecting and getting trapped by
memory leaks in libssh2.

Closes #14984
2024-09-20 22:47:18 +02:00
Stefan Eissing
5895b71b0e
libssh.c: handle EGAINS during proto-connect correctly
The implementation did not check the poll flags when a ssh_connect()
EAGAINed. Also the poll check did not allow for both directions
to be signalled by libssh, which it does.

This appeared during CI testing with higher paralellism.

Closes #14982
2024-09-20 17:03:56 +02:00
Stefan Eissing
b20ac93f41
multi.c: make stronger check for paused transfer before asserting
With higher parallelism in CI, the ASSERT triggered on pause tests.
Strengthen the check. We might want to think about removing
KEEP_RECV_PAUSE|KEEP_SEND_PAUSE altogether.

Closes #14981
2024-09-20 17:01:34 +02:00
Viktor Szakats
3efba94f77
cmake: allow building tests in unity mode
Makes building tests noticeably faster.

Apply changes/fixes/workarounds to make Unity work:
- rename test variables to avoid collisions or shadowing each other when
  combined into single units.
- add workaround to avoid applying `lib/memdebug.h` overrides to system
  headers declaring/defining `getaddrinfo()`/`freeaddrinfo()` for
  `tests/server/resolve.c`. This replaces a previous workaround that
  worked for that specific source.
- rename test macro `CTRL` clashing with Cygwin `sys/ioctl.h`.
- add include guard to `test.h`.

Also:
- exclude `tests/http/clients` which are all single-source. (like
  `docs/examples`.)

Build time improvements for tests:
- AppVeyor CI:
  - MSVC 2008, 2010: 1 minute faster (4m8s -> 2m56s, 3m19s -> 2m24s)
  - MSVC 2022 arm64: 3.5 minutes faster (10m18s -> 6m48s)
  before: https://ci.appveyor.com/project/curlorg/curl/builds/50522785
  after: https://ci.appveyor.com/project/curlorg/curl/builds/50522942
- GHA:
  - Cygwin: 1.5 minutes faster (3m13s -> 1m43s)
    before: https://github.com/curl/curl/actions/runs/10681535327/job/29605384398
    after: https://github.com/curl/curl/actions/runs/10680818726/job/29603130637
  - Windows:
    before: https://github.com/curl/curl/actions/runs/10680818713
    after: https://github.com/curl/curl/actions/runs/10683850187
    - MSYS2, mingw-w64: 1 minute faster
    - MSVC: 30 seconds faster (3m17s -> 2m48s)
  - macOS: double speed (39s -> 18s)
    before: https://github.com/curl/curl/actions/runs/10680818753/job/29603133447
    after: https://github.com/curl/curl/actions/runs/10683850174/job/29612914515
  - Linux: almost double speed (30/31s -> 18s)
    before: https://github.com/curl/curl/actions/runs/10681535311/job/29605387156
    after: https://github.com/curl/curl/actions/runs/10680818721/job/29603133976
  - non-native: no obvious effect.
    before: https://github.com/curl/curl/actions/runs/10680818722
    after: https://github.com/curl/curl/actions/runs/10683850187
  - Old Linux: Unity mode not supported by old CMake, no effect.

Closes #14765
2024-09-19 21:32:58 +02:00
Viktor Szakats
aa1a153910
lib: fix unity builds with BearSSL, MSH3, Quiche, OmniOS
- fix MSH3 static symbol clash.
- fix Quiche static symbol clash.
- fix local macro clash with BearSSL header.
- fix local macro clash with OmniOS system header.
  ```
  In file included from ../../lib/urldata.h:197,
                     from ../../lib/altsvc.c:32,
                     from libcurlall.c:2:
    ../../lib/cf-socket.h:55:25: error: expected ':', ',', ';', '}' or '__attribute__' before '.' token
       55 | #define sa_addr _sa_ex_u.addr
          |                         ^
    In file included from ../../lib/urldata.h:197,
                     from ../../lib/altsvc.c:32,
                     from libcurlall.c:2:
    ../../lib/cf-socket.h:55:25: error: expected ':', ',', ';', '}' or '__attribute__' before '.' token
       55 | #define sa_addr _sa_ex_u.addr
          |                         ^
  ```
  Ref: https://github.com/curl/curl/actions/runs/10738314933/job/29781644299?pr=14772#step:3:6115

Discovered while adding support for "unity" builds for autotools.

Required-by: #14922
Cherry-picked from #14815
Closes #14932
2024-09-19 20:27:01 +02:00
Viktor Szakats
bc2f72b9ae
tidy-up: rename CURL_WINDOWS_APP to CURL_WINDOWS_UWP
Rename internal macro to make its purpose more obvious.

After this patch `grep -i uwp` shows all the code related to UWP.

Ref: https://curl.se/mail/lib-2024-09/0014.html
Closes #14881
2024-09-19 19:24:12 +02:00
Viktor Szakats
445fb81237
cmake, Makefile.mk: use -isystem for dep headers, silence BearSSL issues
Patch started out for working around compiler warnings in BearSSL latest
tarball release v0.6 (2018-08-14) and Apple clang 14 with CMake.

Then turned into patching CMake and `Makefile.mk` builds to use
`-isystem` instead `-I` when adding header directories for
dependencies. This avoids compiler warnings in dependency headers,
syncing behaviour with autotools.

Also:
- `Makefile.mk`: add support for BearSSL.
- delete warning suppression for mbedTLS headers. No longer necessary
  after this patch.
  Follow-up to 434db995a7 #12720

Silenced BearSSL warnings:
```
In file included from curl/lib/vtls/bearssl.c:28:
In file included from bearssl/inc/bearssl.h:127:
bearssl/inc/bearssl_hash.h:727:5: warning: 'BR_DOXYGEN_IGNORE' is not defined, evaluates to 0 [-Wundef]
    ^
bearssl/inc/bearssl_hash.h:745:5: warning: 'BR_DOXYGEN_IGNORE' is not defined, evaluates to 0 [-Wundef]
    ^
In file included from curl/lib/vtls/bearssl.c:28:
In file included from bearssl/inc/bearssl.h:136:
bearssl/inc/bearssl_ssl.h:1253:20: warning: implicit conversion loses integer precision: 'unsigned int' to 'uint16_t' (aka 'unsigned short') [-Wimplicit-int-conversion]
        cc->version_min = version_min;
                        ~ ^~~~~~~~~~~
bearssl/inc/bearssl_ssl.h:1254:20: warning: implicit conversion loses integer precision: 'unsigned int' to 'uint16_t' (aka 'unsigned short') [-Wimplicit-int-conversion]
        cc->version_max = version_max;
                        ~ ^~~~~~~~~~~
bearssl/inc/bearssl_ssl.h:1327:28: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'uint16_t' (aka 'unsigned short') [-Wimplicit-int-conversion]
        ctx->protocol_names_num = num;
                                ~ ^~~
5 warnings generated.
```

(These warnings were fixed in BearSSL Git master in 2019 via
2893441f2efd4603ddd6d7f49011bdda096a4a87 and
ecdf89770ee82dfea6186fb4369cff3d06cd852e.)

Also these two cases, which are caused by an unidentified component
(outside curl) cranking up MSVC warnings in external headers to `/W4`
when ZLIB is deselected:
https://github.com/curl/curl/pull/14859#issuecomment-2351809153

mbedTLS 3.6.1:
```
C:\vcpkg\installed\x64-windows\include\psa\crypto_struct.h(254,13): error C2220: the following warning is treated as an error [D:\a\curl\curl\bld\lib\libcurl_object.vcxproj]
  (compiling source file 'CMakeFiles/libcurl_object.dir/Unity/unity_0_c.c')
C:\vcpkg\installed\x64-windows\include\psa\crypto_struct.h(254,13): warning C4200: nonstandard extension used: zero-sized array in struct/union [D:\a\curl\curl\bld\lib\libcurl_object.vcxproj]
  (compiling source file 'CMakeFiles/libcurl_object.dir/Unity/unity_0_c.c')
```
Ref: https://github.com/curl/curl/actions/runs/10842694205/job/30107466989?pr=14859#step:10:29

nghttp3 1.5.0:
```
C:\vcpkg\installed\x64-windows\include\nghttp3\nghttp3.h(2678,1): error C2220: the following warning is treated as an error [D:\a\curl\curl\bld\lib\libcurl_object.vcxproj]
  (compiling source file 'CMakeFiles/libcurl_object.dir/Unity/unity_0_c.c')
  C:\vcpkg\installed\x64-windows\include\nghttp3\nghttp3.h(2678,1): warning C4324: 'nghttp3_pri': structure was padded due to alignment specifier [D:\a\curl\curl\bld\lib\libcurl_object.vcxproj]
  (compiling source file 'CMakeFiles/libcurl_object.dir/Unity/unity_0_c.c')
```
Ref: https://github.com/curl/curl/actions/runs/10871875297/job/30166233862?pr=14859#step:10:28

Closes #14763
2024-09-19 19:24:01 +02:00
Viktor Szakats
8d32e878a0
cmake: delete unused NEED_LBER_H, HAVE_LDAP_H
Both are used by `./configure` internally, not by curl C code.
CMake adds `NEED_LBER_H` to `CURL_TEST_DEFINES`, which is used by
`curl_internal_test()`, but there is no LDAP-related test made with that
call.

Thus, stop detecting and publishing these from CMake.

`NEED_LBER_H` added in 7320e53d9e.
`HAVE_LDAP_H` added in 4c5307b456 (initial CMake commit).

Closes #14690
2024-09-19 17:48:57 +02:00
Daniel Stenberg
fbf5d507ce
lib/src: white space edits to comply better with code style
... as checksrc now finds and complains about these.

Closes #14921
2024-09-19 14:59:12 +02:00
Viktor Szakats
56b0442e4c
urlapi: drop unused header
Closes #14867
2024-09-19 12:56:21 +02:00
Gabriel Marin
5a263710f6
lib, src, tests: added space around ternary expressions
Closes #14912
2024-09-18 15:27:26 +02:00
Jon Rumsey
1ec5336b66
negotiate: conditional check around GSS & SSL specific code
Fixes #14938
Reported-by: lomberd2 on github
Fixes #14952
Closes #14954
2024-09-18 15:04:06 +02:00
Daniel Stenberg
6d0a48e586
sendf: add condition to max-filesize check
Since the max filesize check should not be performed while the body is
ignored.

Follow-up to aef384a7df
Closes #14958
2024-09-18 15:00:29 +02:00
Viktor Szakats
0ba70dd13c
singleuse: drop Curl_memrchr() for no-HTTP builds
Make single-use function check CI test pass by dropping the global
`Curl_memrchr()` function from the build when it has no caller.

Fixes:
```
Curl_memrchr in curl_memrchr
```
https://github.com/curl/curl/actions/runs/10816280747/job/30007145114

Seen while migrating affected job from Azure to GHA.
Ref: #14859

Closes #14919
2024-09-15 16:33:24 +02:00
Daniel Stenberg
aef384a7df
http: make max-filesize check not count ignored bodies
Add test 477 to verify

Reported-by: MasterInQuestion on github
Fixes #14899
Closes #14900
2024-09-14 22:51:57 +02:00
Daniel Stenberg
7eda757d99
FTP: partly revert eeb7c12807
Since ASCII transfers on FTP means sending CRLF line endings, we should
still keep converting them to LF-only on platforms where text files
typically do not use CRLF.

This also DOES NOT convert existing CRLF line endings on ASCII uploads
but only does stand-alone LF => CRLF.

Regression from eeb7c12807 shipped in 8.10.0

Reported-by: finkjsc on github
Fixes #14873
Closes #14875
2024-09-14 17:23:04 +02:00
Daniel Stenberg
2b652b8634
transfer: remove redundant variable use: select_bits
Follow-up to a33bcc9b53

Pointed out by CodeSonar

Closes #14907
2024-09-14 17:20:27 +02:00
Stefan Eissing
50166c0de8
connect: store connection info when really done
Output the 'Connected to...' info message when the connection has been
fully established and all information is available.

Due to our happy eyeballing, we should not emit info messages in
filters, because they may be part of an eyeballing attempt and may be
discarded later for another chain.

Closes #14897
2024-09-13 22:27:11 +02:00
Stefan Eissing
a33bcc9b53
transfer: fix sendrecv() without interim poll
There was a "clever" optimization that skipped sendrecv() handling when
the transfer's pollset was empty. This happens for paused transfers, for
example.

Unforunately, if the libcurl application never calls curl_multi_poll(),
the pollset is and will aways remain empty, prevent the transfer from
progressing.

Remove this "optimization" and always try send/receive where applicable.

Fixes #14898
Reported-by: Victor Kislov
Closes #14901
2024-09-13 22:21:33 +02:00
Daniel McCarney
8972845123
vtls/rustls: support strong CSRNG data
Now that the curl rustls vtls backend is using rustls 0.14 we can
address the weak random situation by using
`rustls_default_crypto_provider_random()` to provide a `Curl_ssl`
`random` callback that fills the provided buffer with cryptographically
secure random data.

The mentions in `docs/` about weak RNG when using rustls are removed as
they are no longer applicable.

Closes #14889
2024-09-13 14:11:56 +02:00
Daniel McCarney
6d9b40d6a4
vtls/rustls: simplify ciphersuite skipping
Now that the rustls vtls backend is using rustls 0.14 we can take
advantage of `rustls_supported_ciphersuite_protocol_version()` to skip
TLS 1.3 and TLS 1.2 ciphersuites as required without needing to
interrogate the ciphersuite names as `rustls_str`s.

Closes #14889
2024-09-13 14:11:52 +02:00
Daniel McCarney
f09adc3ad1
vtls/rustls: rustls-ffi 0.14.0 update
* Documentation is updated to describe new required version, and to link
  to the upstream README about cryptography providers.
* GitHub workflow is updated to fetch 0.14.0.
* Breaking changes in`lib/vtls/rustls.c` are addressed:
  * The `rustls_client_config_builder_build()` function now uses an out
    parameter for the built config instead of returning it directly.
    This allows the building process to fail if the default crypto
    provider state isn't appropriate, or another error condition occurs.
  * Default ciphersuites are collected using renamed functions named to
    make it clear the ciphersuites are associated with the default
    crypto provider.
  * Customization of ciphersuites is now done via
    a `rustls_crypto_provider_builder` used to instantiate a
    `rustls_crypto_provider`. The customized provider can then can be
    used with `rustls_client_config_builder_new_custom` in place of
    providing ciphersuites directly.
  * `rustls_connection_get_negotiated_ciphersuite()` now returns the
    ciphersuite ID directly.

Closes #14889
2024-09-13 14:11:49 +02:00
Daniel McCarney
65b8d89463
vtls/rustls: differentiate error messages
It's easier to diagnose a problem when there is one place where the
error message can be emitted. For that reason this commit updates two
errors that were shared between other fallible operations to use unique
messages.

Closes #14889
2024-09-13 14:11:42 +02:00
Daniel McCarney
d38458d823
vtls/rustls: simplify builder cleanup
Don't build `config_builder` just to free the resulting config, free the
builder directly.

When `cr_init_backend` encounters an error condition setting up the
Rustls client configuration it must do something with the
`config_builder` that was constructed earlier to avoid a memory leak.

The previous implementation preferred to use a pattern of building the
builder (thus consuming it) and then freeing the built config (to avoid
a memory leak). However, the purpose/intent is clearer when we just free
the builder directly instead of building it and freeing the result.

Closes #14889
2024-09-13 14:11:26 +02:00
Stefan Eissing
bef0acaf21
request: correctly reset the eos_sent flag
Add test cases

Bug: https://marc.info/?l=git&m=172620452502747&w=2
Reported-by: Patrick Steinhardt
Closes #14895
2024-09-13 13:28:10 +02:00
Gabriel Marin
79f0007c25
setopt: remove superfluous use of ternary expressions
Closes #14884
2024-09-12 19:12:06 +02:00
Daniel McCarney
61e48b4df1
vtls: fix Curl_ssl_conn_config_match doc param
The configuration data compared to `candidate` comes from `data`, not
`conn`. This commit fixes the docstring to reflect this.

Closes #14887
2024-09-12 19:06:34 +02:00
Stefan Eissing
70d3a9b6aa
http2: when uploading data from stdin, fix eos forwarding
When uploading data from stdin ('-T -'), and the EOS was only detected
on a 0-length read, the EOS was not forwarded to the filters. This led
HTTP/2 to hang on not forwarding this to the server.

Added test_07_14 to reproduce and verify.

Fixes #14870
Reported-by: nekopsykose on github
Closes #14877
2024-09-12 13:46:38 +02:00
Stefan Eissing
283af039c8
QUIC: on connect, keep on trying on draining server
Do not give up connect on servers that are in draining state. This might
indicate the QUIC server restarting and the UDP packet routing still
hitting the instance shutting down.

Instead keep on connecting until the overall TIMEOUT fires.

Closes #14863
2024-09-12 08:24:10 +02:00
Jan Venekamp
0ca15307a3
rustls: fixed minor logic bug in default cipher selection
Follow-up to 1e03d4b

Closes #14840
2024-09-11 23:17:31 +02:00
Harry Sintonen
6a9f3764f1
lib: fix AF_INET6 use outside of USE_IPV6
Closes #14858
2024-09-11 23:12:22 +02:00
Daniel Stenberg
48f61e781a
multi: check that the multi handle is valid in curl_multi_assign
By requiring that the multi handle is fine, it can detect bad usage
better and by that avoid crashes. Like in the #14860 case, which is an
application calling curl_multi_assign() with a NULL pointer multi
handle.

Reported-by: Carlo Cabrera
Fixes #14860
Closes #14862
2024-09-11 23:04:11 +02:00
Daniel Stenberg
0d6c8b7534
lib: enable strerror and strncpy checksrc warnings in subdirs
Closes #14834
2024-09-09 16:51:21 +02:00
Daniel Stenberg
63ebc48b69
content_encoding: avoid getting all encodings unless necessary
The error_do_write() function may very well return witout needing the
listing of all encoding types so postpone that call until it is needed.

Closes #14831
2024-09-09 16:50:22 +02:00
Daniel Stenberg
80df6a5c12
checksrc: add STRNCPY as an opt-in rule to detect and error on strncpy
make "lib/.checksrc" enable it

Closes #14830
2024-09-09 13:47:50 +02:00
Daniel Stenberg
344a177aac
lib: remove the final strncpy() calls
wolfssl: use strcpy() as the target buffer is > 40 bytes

gethostname: return failure if the buffer is too small instead

Closes #14830
2024-09-09 13:47:38 +02:00
Jay Satiro
eb8ad66f6c asyn-thread: stop using GetAddrInfoExW on Windows
- For the threaded resolver backend on Windows, revert back to
  exclusively use the threaded resolver with libcurl-owned threading
  instead of GetAddrInfoExW with Windows-owned threading.

Winsock (the Windows sockets library) has a bug where it does not wait
for all of the name resolver threads it is managing to terminate before
returning from WSACleanup. The threads continue to run and may cause a
crash.

This commit is effectively a revert of several commits that encompass
all GetAddrInfoExW code in libcurl. A manual review of merge conflicts
was used to resolve minor changes that had modified the code for
aesthetic or build reasons in other commits.

Prior to this change if libcurl was built with the threaded resolver
backend for Windows, and Windows 8 or later was the operating system at
runtime, and the caller was not impersonating another user, then libcurl
would use GetAddrInfoExW to handle asynchronous name lookups.

GetAddrInfoExW support was added in a6bbc87f, which preceded 8.6.0, and
prior to that the threaded resolver backend used libcurl-owned threading
exclusively on Windows.

Reported-by: Ionuț-Francisc Oancea
Reported-by: Razvan Pricope

Ref: https://developercommunity.visualstudio.com/t/ASAN:-heap-use-after-free-in-NdrFullPoin/10654169

Fixes https://github.com/curl/curl/issues/13509#issuecomment-2225338110
Closes https://github.com/curl/curl/pull/14794

---

Revert "asyn-thread: avoid using GetAddrInfoExW with impersonation"

This reverts commit 0caadc1f24.

Conflicts:
	lib/system_win32.c

--

Revert "asyn-thread: fix curl_global_cleanup crash in Windows"

This reverts commit 428579f5d1.

--

Revert "system_win32: fix a function pointer assignment warning"

This reverts commit 26f002e02e.

--

Revert "asyn-thread: use GetAddrInfoExW on >= Windows 8"

This reverts commit a6bbc87f9e.

Conflicts:
	lib/asyn-thread.c
	lib/system_win32.c

--
2024-09-08 11:39:30 -04:00
Daniel Stenberg
24606191f8
doh: remove redundant checks
Pointed out by CodeSonar

Closes #14823
2024-09-07 22:30:21 +02:00
Viktor Szakats
f6955e4215
Makefile.mk: update to use Markdown sources for manual
Closes #14813
2024-09-06 21:57:54 +02:00
Stefan Eissing
b1f0b8f601
pop3: fix multi-line with LIST arg
The POP3 LIST command is not multi-line when having an argument. Fix the
definition to correct the behaviour.

Reported-by: ralfjunker on github
Fixes #14801
Closes #14808
2024-09-06 10:42:14 +02:00
Stefan Eissing
435dd8aa6e
doh: cleanups
Mostly cleanup on identifiers of DoH code.
Always use 'Curl_doh_cleanup()' for releasing resources.

More concise and telling names (ymmv):

* prefix all static functions with 'doh_' for unity builds
* doh_encode -> doh_req_encode
* doh_decode -> doh_resp_decode
* struct dohdata -> struct doh_probes
* probe's 'serverdoh' -> 'resp_body'
* probe's 'dohbuffer' -> 'req_body'
* probe's 'headers' -> 'req_hds'
* 'dohprobe()' -> doh_run_probe()'
* 'DOH_PROBE_SLOTS' -> 'DOH_SLOT_COUNT'
* 'DOH_PROBE_SLOT_IPADDR_V4' -> 'DOH_SLOT_IPV4'
* 'DOH_PROBE_SLOT_IPADDR_V6' -> 'DOH_SLOT_IPV6'
* 'DOH_PROBE_SLOT_HTTPS' -> 'DOH_SLOT_HTTPS_RR'

Closes #14783
2024-09-06 08:58:52 +02:00
Daniel Stenberg
0d1504b205
libcurl.def: move from / into lib
It is meant for the library, it belongs in lib/

Closes #14796
2024-09-05 17:05:40 +02:00
Jay Satiro
519be2b9d8 system_win32: fix typo 2024-09-05 10:10:20 -04:00
Daniel Stenberg
f905769fe2
llist: only provide Curl_llist_tail in unit test builds
... since it is not used otherwsie. Also fix Curl_node_prev to have the
same status.

Closes #14790
2024-09-05 13:09:13 +02:00
Stefan Eissing
db5eae1127
cf-socket: fix listen pollset for FTP active mode
Follow-up to a07ba37b5e which did not
solve the issue of corrent polling for FTP active data connections.

Added test cases for active up-/download.

Closes #14786
2024-09-05 11:00:55 +02:00
Daniel Stenberg
464d466aea
smb: convert superflous assign into assert
Since the value is already supposed to be correct, verify that with and
assert instead of doing an assign that is not needed.

Bonus: remove unnecessary clearing of data

Closes #14784
2024-09-05 09:25:08 +02:00
Daniel Stenberg
3e7ddf94a2
schannel: avoid malloc for CAinfo_blob_digest
...it can just as well be part of the struct always.

Closes #14777
2024-09-05 09:23:48 +02:00
Daniel Stenberg
6588a7f039
openssl: certinfo errors now fail correctly
If there is a (memory) error when creating the certinfo data, the code
would previously continue which could lead to a partial/broken response.
Now, the first error aborts and cleans up the entire thing.

A certinfo "collection" error is however still not considered an error
big enough to stop the handshake.

Bonus 1: made two functions static (and removed the Curl_ prefix) that
were not used outside of openssl.c

Bonus 2: removed the unused function Curl_ossl_set_client_cert

Closes #14780
2024-09-04 23:41:44 +02:00
Daniel Stenberg
bca9c7719a
lib: make SSPI global symbols use Curl_ prefix
Do not pollute the namespace.

Closes #14776
2024-09-04 14:55:10 +02:00