Enable TLS Early Data for wolfSSL:
- merge WOLFSSL_CTX and WOLFSSL setup from ngtcp2 with the general
implemenation in wolfssl.c
- enable for QUIC via ngtcp2
- give Curl_vquic_tls_init() a `struct alpn_spec` like used for the TCP
case. Adapt gnutls and other users.
- enable pytest test cases for early data with wolfSSL
and while this messes up wolfssl.c anyway, do
- rename all struct/functions with prefix 'wolfssl_' to 'wssl_' to not
pollute that name prefix
- rename `ctx/handle` to `ssl_ctx/ssl`, as used in openssl case
Closes#16167
Remove `blocking` argument from cfilter's connect method.
Implement blocking behaviour in Curl_conn_connect() instead for all
filter chains.
Update filters implementations. Several of which did never use the
paramter (QUIC for example). Simplifies connect handling in TLS filters
that no longer need to loop
Fixed a blocking connect call in FTP when waiting on a socket accept()
which only worked because the filter did not implement it.
Closes#16397
The variable `conn->httpversion` was used for several purposes and it
was unclear at which time the value represents what.
- rename `conn->httpversion` to `conn->httpversion_seen`
This makes clear that the variable only records the last
HTTP version seen on the connection - if any. And that it
no longer is an indication of what version to use.
- Change Alt-Svc handling to no longer modify `conn->httpversion`
but set `data->state.httpwant` for influencing the HTTP version
to use on a transfer.
- Add `data->req.httpversion_sent` to have a record of what
HTTP version was sent in a request
- Add connection filter type CF_TYPE_HTTP
- Add filter query `CF_QUERY_HTTP_VERSION` to ask what HTTP
filter version is in place
- Lookup filters HTTP version instead of using `conn->httpversion`
Test test_12_05 now switches to HTTP/1.1 correctly and the
expectations have been fixed.
Removed the connection fitler "is_httpN()" checks and using
the version query instead.
Closes#16073
Make transfer attach/detach to/from connections chepaer.
- the "attach" event was no longer implemented by any filter
- the "detach" did the same as the "done" event for the filters
who still implemented it. It should be superfluous as the "done"
must always happen.
Closes#16067
Adds the experimental feature `ssls-export` to libcurl and curl for
importing and exporting SSL sessions from/to a file.
* add functions to libcurl API
* add command line option `--ssl-sessions <filename>` to curl
* add documenation
* add support in configure
* add support in cmake
+ add pytest case
Closes#15924
When a QUIC TLS session announced early data support and
'CURLSSLOPT_EARLYDATA' is set for the transfer, send initial request and
body (up to the 128k we buffer) as 0RTT when curl is built with
ngtcp2+gnutls.
QUIC 0RTT needs not only the TLS session but the QUIC transport
paramters as well. Store those and the earlydata max value together with
the session in the cache.
Add test case for h3 use of this. Enable quic early data in nghttpx for
testing.
Closes#15667
Described in detail in internal doc TLS-SESSIONS.md
Main points:
- use a new `ssl_peer_key` for cache lookups by connection filters
- recognize differences between TLSv1.3 and other tickets
* TLSv1.3 tickets are single-use, cache can hold several of them for a peer
* TLSv1.2 are reused, keep only a single one per peer
- differentiate between ticket BLOB to store (that could be persisted) and object instances
- use put/take/return pattern for cache access
- remember TLS version, ALPN protocol, time received and lifetime of ticket
- auto-expire tickets after their lifetime
Closes#15774
The vquic_recv_packets() function already loops when not all requested
packets can be received (until EAGAIN) and there is not need to do that
again in ngtcp2.
Closes#15466
Use session cache for QUIC when built with quictls or wolfSSL.
Add test_017_10 for verifying QUIC TLS session reuse when built with
quictls, gnutls or wolfssl.
Closes#15358
Add session reuse for QUIC transfers using GnuTLS. This does not include
support for TLS early data, yet.
Fix check of early data support in common GnuTLS init code to not access
the filter context, as the struct varies between TCP and QUIC
connections.
Closes#15265
- fix MSH3 static symbol clash.
- fix Quiche static symbol clash.
- fix local macro clash with BearSSL header.
- fix local macro clash with OmniOS system header.
```
In file included from ../../lib/urldata.h:197,
from ../../lib/altsvc.c:32,
from libcurlall.c:2:
../../lib/cf-socket.h:55:25: error: expected ':', ',', ';', '}' or '__attribute__' before '.' token
55 | #define sa_addr _sa_ex_u.addr
| ^
In file included from ../../lib/urldata.h:197,
from ../../lib/altsvc.c:32,
from libcurlall.c:2:
../../lib/cf-socket.h:55:25: error: expected ':', ',', ';', '}' or '__attribute__' before '.' token
55 | #define sa_addr _sa_ex_u.addr
| ^
```
Ref: https://github.com/curl/curl/actions/runs/10738314933/job/29781644299?pr=14772#step:3:6115
Discovered while adding support for "unity" builds for autotools.
Required-by: #14922
Cherry-picked from #14815Closes#14932
Do not give up connect on servers that are in draining state. This might
indicate the QUIC server restarting and the UDP packet routing still
hitting the instance shutting down.
Instead keep on connecting until the overall TIMEOUT fires.
Closes#14863
This is a better match for what they do and the general "cpool"
var/function prefix works well.
The pool now handles very long hostnames correctly.
The following changes have been made:
* 'struct connectdata', e.g. connections, keep new members
named `destination` and ' destination_len' that fully specifies
interface+port+hostname of where the connection is going to.
This is used in the pool for "bundling" of connections with
the same destination. There is no limit on the length any more.
* Locking: all locks are done inside conncache.c when calling
into the pool and released on return. This eliminates hazards
of the callers keeping track.
* 'struct connectbundle' is now internal to the pool. It is no
longer referenced by a connection.
* 'bundle->multiuse' no longer exists. HTTP/2 and 3 and TLS filters
no longer need to set it. Instead, the multi checks on leaving
MSTATE_CONNECT or MSTATE_CONNECTING if the connection is now
multiplexed and new, e.g. not conn->bits.reuse. In that case
the processing of pending handles is triggered.
* The pool's init is provided with a callback to invoke on all
connections being discarded. This allows the cleanups in
`Curl_disconnect` to run, wherever it is decided to retire
a connection.
* Several pool operations can now be fully done with one call.
Pruning dead connections, upkeep and checks on pool limits
can now directly discard connections and need no longer return
those to the caller for doing that (as we have now the callback
described above).
* Finding a connection for reuse is now done via `Curl_cpool_find()`
and the caller provides callbacks to evaluate the connection
candidates.
* The 'Curl_cpool_check_limits()' now directly uses the max values
that may be set in the transfer's multi. No need to pass them
around. Curl_multi_max_host_connections() and
Curl_multi_max_total_connections() are gone.
* Add method 'Curl_node_llist()' to get the llist a node is in.
Used in cpool to verify connection are indeed in the list (or
not in any list) as they need to.
I left the conncache.[ch] as is for now and also did not touch the
documentation. If we update that outside the feature window, we can
do this in a separate PR.
Multi-thread safety is not achieved by this PR, but since more details
on how pools operate are now "internal" it is a better starting
point to go for this in the future.
Closes#14662
`data->id` is unique in *most* situations, but not in all. If a libcurl
application uses more than one connection cache, they will overlap. This
is a rare situations, but libcurl apps do crazy things. However, for
informative things, like tracing, `data->id` is superior, since it
assigns new ids in curl's serial curl_easy_perform() use.
Introduce `data->mid` which is a unique identifer inside one multi
instance, assigned on multi_add_handle() and cleared on
multi_remove_handle().
Use the `mid` in DoH operations and also in h2/h3 stream hashes.
Reported-by: 罗朝辉
Fixes#14414Closes#14499
Silence bogus MSVC warning C4232. Use the method already used
for similar cases earlier.
Also fixup existing suppressions to use pragma push/pop.
```
lib\vquic\curl_ngtcp2.c(709,40): error C2220: the following warning is treated as an error
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'client_initial': address of dllimport 'ngtcp2_crypto_client_initial_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'recv_crypto_data': address of dllimport 'ngtcp2_crypto_recv_crypto_data_cb' is not static, identity not guaran
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'encrypt': address of dllimport 'ngtcp2_crypto_encrypt_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'decrypt': address of dllimport 'ngtcp2_crypto_decrypt_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'hp_mask': address of dllimport 'ngtcp2_crypto_hp_mask_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'recv_retry': address of dllimport 'ngtcp2_crypto_recv_retry_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'update_key': address of dllimport 'ngtcp2_crypto_update_key_cb' is not static, identity not guaranteed
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'delete_crypto_aead_ctx': address of dllimport 'ngtcp2_crypto_delete_crypto_aead_ctx_cb' is not static, identit
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'delete_crypto_cipher_ctx': address of dllimport 'ngtcp2_crypto_delete_crypto_cipher_ctx_cb' is not static, ide
lib\vquic\curl_ngtcp2.c(709,40): warning C4232: nonstandard extension used: 'get_path_challenge_data': address of dllimport 'ngtcp2_crypto_get_path_challenge_data_cb' is not static, ident
```
Ref: https://github.com/curl/curl/actions/runs/10343459009/job/28627621355#step:10:30
Cherry-picked from #14495
Co-authored-by: Tal Regev
Ref: #14383Closes#14510
Members of the filter context, like stream hash and buffers, need to be
initialized early and protected by a flag to also avoid double cleanup.
This allow the context to be used safely before a connect() is started
and the other parts of the context are set up.
Closes#14505
- Turned them all into functions to also do asserts etc.
- The llist related structs got all their fields renamed in order to make
sure no existing code remains using direct access.
- Each list node struct now points back to the list it "lives in", so
Curl_node_remove() no longer needs the list pointer.
- Rename the node struct and some of the access functions.
- Added lots of ASSERTs to verify API being used correctly
- Fix some cases of API misuse
Add docs/LLIST.md documenting the internal linked list API.
Closes#14485
Instead of having an especially "unique" linked list handler for the
main list of easy handles within the multi handle, this now uses a
regular Curl_llist for this as well.
With this change, it is also clearer that every easy handle added to a
multi handle belongs to one and only one out of three different lists:
process - the general one for normal transfer processing
pending - queued up waiting to get a connection (MSTATE_PENDING)
msgsent - transfer completed (MSTATE_MSGSENT)
An easy handle must therefore be removed from the current list before it
gets added to another.
Closes#14474
Adds a `bool eos` flag to send methods to indicate that the data
is the last chunk the invovled transfer wants to send to the server.
This will help protocol filters like HTTP/2 and 3 to forward the
stream's EOF flag and also allow to EAGAIN such calls when buffers
are not yet fully flushed.
Closes#14220
Adds a `bool eos` flag to send methods to indicate that the data is the
last chunk the invovled transfer wants to send to the server.
This will help protocol filters like HTTP/2 and 3 to forward the
stream's EOF flag and also allow to EAGAIN such calls when buffers are
not yet fully flushed.
Closes#14220
Currently we're waiting for sendbuf_len_in_flight to hit zero before
resuming upload which means we're blocking and waiting for _all_ acks to
arrive before sending more data. This causes significant delays especially
when ack delay is used on the server side.
The fix addresses several issues in h3 over ngtcp2:
- On ack we now call nghttp3_conn_resume_stream() when we have more
data to send.
- upload_left was incorrectly computed on CF_CTRL_DATA_DONE_SEND as
we need to subtract the ammount of data we have in flight.
- Remove upload_blocked_len as we Curl_bufq_write call will do the
right thing when called from cf_ngtcp2_send.
Fixes#14198Closes#14209
Based on the standards and guidelines we use for our documentation.
- expand contractions (they're => they are etc)
- host name = > hostname
- file name => filename
- user name = username
- man page => manpage
- run-time => runtime
- set-up => setup
- back-end => backend
- a HTTP => an HTTP
- Two spaces after a period => one space after period
Closes#14073
This adds connection shutdown infrastructure and first use for FTP. FTP
data connections, when not encountering an error, are now shut down in a
blocking way with a 2sec timeout.
- add cfilter `Curl_cft_shutdown` callback
- keep a shutdown start timestamp and timeout at connectdata
- provide shutdown timeout default and member in
`data->set.shutdowntimeout`.
- provide methods for starting, interrogating and clearing
shutdown timers
- provide `Curl_conn_shutdown_blocking()` to shutdown the
`sockindex` filter chain in a blocking way. Use that in FTP.
- add `Curl_conn_cf_poll()` to wait for socket events during
shutdown of a connection filter chain.
This gets the monitoring sockets and events via the filters
"adjust_pollset()" methods. This gives correct behaviour when
shutting down a TLS connection through a HTTP/2 proxy.
- Implement shutdown for all socket filters
- for HTTP/2 and h2 proxying to send GOAWAY
- for TLS backends to the best of their capabilities
- for tcp socket filter to make a final, nonblocking
receive to avoid unwanted RST states
- add shutdown forwarding to happy eyeballers and
https connect ballers when applicable.
Closes#13904
- identify ngtcp2 and nghttp3 error codes that are fatal
- close quic connection on fatal errors
- refuse further filter operations once connection is closed
- confusion about the nghttp3 API. We should close the QUIC stream on
cancel and not use the nghttp3 calls intended to be invoked when the
QUIC stream was closed by the peer.
Closes#13562
- quiche: error transfers that try to receive on a closed
or draining connection
- ngtcp2: use callback for extending max bidi streams. This
allows more precise calculation of MAX_CONCURRENT as we
only can start a new stream when the server acknowledges
the close - not when we locally have closed it.
- remove a fprintf() from h2-download client to avoid excess
log files on tests timing out.
Closes#13475
- add session with destructor callback
- remove vtls `session_free` method
- let `Curl_ssl_addsessionid()` take ownership
of session object, freeing it also on failures
- change tls backend use
- test_17, add tests for SSL session resumption
Closes#13386
- errors returned by Curl_xfer_write_resp() and the header variant are
not errors in the protocol. The result needs to be returned on the
next recv() from the protocol filter.
- make xfer write errors for response data cause the stream to be
cancelled
- added pytest test_02_14 and test_02_15 to verify that also for
parallel processing
Reported-by: Laramie Leavitt
Fixes#13411Closes#13424
- add `Curl_hash_offt` as hashmap between a `curl_off_t` and
an object. Use this in h2+h3 connection filters to associate
`data->id` with the internal stream state.
- changed implementations of all affected connection filters
- removed `h2_ctx*` and `h3_ctx*` from `struct HTTP` and thus
the easy handle
- solves the problem of attaching "foreign protocol" easy handles
during connection shutdown
Test 1616 verifies the new hash functions.
Closes#13204
- fix flow handling in ngtcp2 to ACK data on streams
we abort ourself.
- extend test_02_23* cases to also run for h3
- skip test_02_23* for OpenSSL QUIC as it gets stalled
on progressing the connection
Closes#13374
- add curl_int64_t signed 64-bit type for lib use
- define CURL_PRId64, CURL_PRIu64 format ids
- use curl_int64_t in vquic
curl_int64_t signed complements the existing curl_uint64_t unsigned.
Note that `curl_int64_t` and `int64_t` are assignable from each other
but not identical. Some platforms with 64 long type defint int64_t as
"long long" (staring at macOS) which messes up things like pointers and
format identifiers.
Closes https://github.com/curl/curl/pull/13293
