New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
the equivalent CURLOPT_ done in the library)
- starts a QUIC/HTTP/3 connect right away. Should that not
succeed after 100ms (subject to change), a parallel attempt
is started for HTTP/2 and HTTP/1.1 via TCP
- both attempts are subject to IPv6/IPv4 eyeballing, same
as happens for other connections
- tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT
- use a `soft` timeout at half the value. When the soft timeout
expires, the HTTPS-CONNECT filter checks if the QUIC filter
has received any data from the server. If not, it will start
the HTTP/2 attempt.
HTTP/3(ngtcp2) improvements.
- setting call_data in all cfilter calls similar to http/2 and vtls filters
for use in callback where no stream data is available.
- returning CURLE_PARTIAL_FILE for prematurely terminated transfers
- enabling pytest test_05 for h3
- shifting functionality to "connect" UDP sockets from ngtcp2
implementation into the udp socket cfilter. Because unconnected
UDP sockets are weird. For example they error when adding to a
pollset.
HTTP/3(quiche) improvements.
- fixed upload bug in quiche implementation, now passes 251 and pytest
- error codes on stream RESET
- improved debug logs
- handling of DRAIN during connect
- limiting pending event queue
HTTP/2 cfilter improvements.
- use LOG_CF macros for dynamic logging in debug build
- fix CURLcode on RST streams to be CURLE_PARTIAL_FILE
- enable pytest test_05 for h2
- fix upload pytests and improve parallel transfer performance.
GOAWAY handling for ngtcp2/quiche
- during connect, when the remote server refuses to accept new connections
and closes immediately (so the local conn goes into DRAIN phase), the
connection is torn down and a another attempt is made after a short grace
period.
This is the behaviour observed with nghttpx when we tell it to shut
down gracefully. Tested in pytest test_03_02.
TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).
- new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation.
Invoke:
python3 tests/tests-httpd/scorecard.py --help
for usage.
Improvements on gathering connect statistics and socket access.
- new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters
report connection statistics. This is triggered when the connection
has completely connected.
- new void Curl_pgrsTimeWas(..) method to report a timer update with
a timestamp of when it happend. This allows for updating timers
"later", e.g. a connect statistic after full connectivity has been
reached.
- in case of HTTP eyeballing, the previous changes will update
statistics only from the filter chain that "won" the eyeballing.
- new cfilter query CF_QUERY_SOCKET for retrieving the socket used
by a filter chain.
Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket()
for convenient use of this query.
- Change VTLS backend to query their sub-filters for the socket when
checks during the handshake are made.
HTTP/3 documentation on how https eyeballing works.
TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).
Scorecard with Caddy.
- configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing
- tests/tests-httpd/scorecard.py now measures download speeds with caddy
pytest improvements
- adding Makfile to clean gen dir
- adding nghttpx rundir creation on start
- checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old.
- catch exception when checking for caddy existance on system.
Closes#10349
The documentation of `%{header_json}` missed to mention since which
version this variable for `--write-out` is present.
Based on commit https://github.com/curl/curl/commit/4133a69f2daa476bb
we can determine from the tags were this commit is present that the
first version to include it was `7.83.0`.
This could be also checked with:
`git tag --contains 4133a69f2daa476bb6d902687f1dd6660ea9c3c5`
Closes#10395
- Clarify that in Windows batch files the % must be escaped as %%, and
at the command prompt it cannot be escaped which could lead to
incorrect expansion.
Prior to this change the doc implied % must be escaped as %% in win32
always.
---
Examples showing how a write-out argument is received by curl:
If curl --write-out "%{http_code}" is executed in a batch file:
{http_code}
If curl --write-out "%%{http_code}" is executed in a batch file:
%{http_code}
If curl --write-out "%{http_code}" is executed from the command prompt:
%{http_code}
If curl --write-out "%%{http_code}" is executed from the command prompt:
%%{http_code}
At the command prompt something like "%{speed_download}%{http_code}"
would first be parsed by the command interpreter as %{speed_download}%
and would be expanded as environment variable {speed_download} if it
existed, though that's highly unlikely since Windows environment names
don't use braces.
---
Reported-by: Muhammad Hussein Ammari
Ref: https://github.com/bagder/everything-curl/pull/279
Fixes https://github.com/curl/curl/issues/10323
Closes https://github.com/curl/curl/pull/10337
Bump the limit from 512K. There might be reasons for applications using
h3 to set larger buffers and there is no strong reason for curl to have
a very small maximum.
Ref: https://curl.se/mail/lib-2023-01/0026.htmlCloses#10256
Because only the libssh2 backend not supports it and thus this should
return error if this option is used other backends.
Reported-by: Harry Sintonen
Closes#10255
- Warn that in Windows if libcurl is running from a DLL and if
CURLOPT_HEADERDATA is set then CURLOPT_WRITEFUNCTION or
CURLOPT_HEADERFUNCTION must be set as well, otherwise the user may
experience crashes.
We already have a similar warning in CURLOPT_WRITEDATA. Basically, in
Windows libcurl could crash writing a FILE pointer that was created by
a different C runtime. In Windows each DLL that is part of a program may
or may not have its own C runtime.
Ref: https://github.com/curl/curl/issues/10231
Closes https://github.com/curl/curl/pull/10233
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING
checksrc is updated to allow non-year using copyright statements
Closes#10205
Ref: #10163
- This is a niche TLS library, only running on some IBM systems
- no regular curl contributors use this backend
- no CI builds use or verify this backend
- gskit, or the curl adaption for it, lacks many modern TLS features
making it an inferior solution
- build breakages in this code take weeks or more to get detected
- fixing gskit code is mostly done "flying blind"
Closes#10201
- it does not add a lot of value
- we do not test-build it to verify because of its dependencies
- unclear for what GTK versions it works or not
Reported-by: odek86 on github
Fixes#10197Closes#10198
- curl_ws_send returns CURLE_SEND_ERROR if data->conn is gone
- curl_ws_recv returns CURLE_GOT_NOTHING on connection close
- curl_ws_recv.3: mention new return code for connection close + example
embryo
Closes#10084
- CURL_GLOBAL_SSL
This option was changed in libcurl 7.57.0 and clearly it has not caused
too many issues and a lot of time has passed.
- Store TLS context per transfer instead of per connection
This is a possible future optimization. One that is much less important
and interesting since the added support for CA caching.
- Microsoft telnet server
This bug was filed in May 2007 against curl 7.16.1 and we have not
received further reports.
- active FTP over a SOCKS
Actually, proxies in general is not working with active FTP mode. This
is now added in proxy documentation.
- DICT responses show the underlying protocol
curl still does this, but since this is now an established behavior
since forever we cannot change it easily and adding an option for it
seems crazy as this protocol is not so little its not worth it. Let's
just live with it.
- Secure Transport disabling hostname validation also disables SNI
This is an already documented restriction in Secure Transport.
- CURLOPT_SEEKFUNCTION not called with CURLFORM_STREAM
The curl_formadd() function is marked and documented as deprecated. No
point in collecting bugs for it. It should not be used further.
- STARTTRANSFER time is wrong for HTTP POSTs
After close source code inspection I cannot see how this is true or that
there is any special treatment for different HTTP methods. We also have
not received many further reports on this, making me strongly suspect
that this is no (longer an) issue.
- multipart formposts file name encoding
The once proposed RFC 5987-encoding is since RFC 7578 documented as MUST
NOT be used. The since then implemented MIME API allows the user to set
the name on their own and can thus provide it encoded as it wants.
- DoH is not used for all name resolves when enabled
It is questionable if users actually want to use DoH for interface and
FTP port name resolving. This restriction is now documented and we
advice users against using name resolving at all for these functions.
Closes#10043
