Rename `close` and `connect` in `struct Curl_cftype` for
consistency and to avoid clashes with macros of the same name
(the standard AmigaOS networking connect() function is implemented
via a macro).
Closes#11491
- fix HTTP/2 check to not declare a connection dead when
the read attempt results in EAGAIN
- add H2-PROXY alive check as for HTTP/2 that was missing
and is needed
- add attach/detach around Curl_conn_is_alive() and remove
these in filter methods
- add checks for number of connections used in some test_10
proxy tunneling tests
Closes#11368
- add an `id` long to Curl_easy, -1 on init
- once added to a multi (or its own multi), it gets
a non-negative number assigned by the connection cache
- `id` is unique among all transfers using the same
cache until reaching LONG_MAX where it will wrap
around. So, not unique eternally.
- CURLINFO_CONN_ID returns the connection id attached to
data or, if none present, data->state.lastconnect_id
- variables and type declared in tool for write out
Closes#11185
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.
You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.
It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.
Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.
Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
`statename`.
- rename `ssl_backend_data` structure to unique names for each
TLS-backend, along with the `ssl_connect_data` struct member
referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
builds when compiling `lib/ldap.c`. To support "unity" builds, we had
to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
`schannel.h` option) _globally_. This caused an indirect inclusion of
Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
as well. This requires `[P]UNICODE_STRING` types, which is apperantly
not defined automatically (as seen with both MSVS and mingw-w64).
This patch includes `<subauth.h>` to fix it.
Ref: https://github.com/curl/curl/runs/13987772013
Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
is built in static mode.
KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250
Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_buildCloses#11095
Out of 415 labels throughout the code base, 86 of those labels were
not at the start of the line. Which means labels always at the start of
the line is the favoured style overall with 329 instances.
Out of the 86 labels not at the start of the line:
* 75 were indented with the same indentation level of the following line
* 8 were indented with exactly one space
* 2 were indented with one fewer indentation level then the following
line
* 1 was indented with the indentation level of the following line minus
three space (probably unintentional)
Co-Authored-By: Viktor Szakats
Closes#11134
If the previous line starts with if/while/for AND ends with a closed
parenthesis and there's an equal number of open and closed parentheses
on that line, verify that this line is indented $indent more steps, if
not a cpp line.
Also adjust the fall-out from this fix.
Closes#11054
The open paren check wants to warn for spaces before open parenthesis
for if/while/for but also for any function call. In order to avoid
catching function pointer declarations, the logic allows a space if the
first character after the open parenthesis is an asterisk.
I also spotted what we did not include "switch" in the check but we should.
This check is a little lame, but we reduce this problem by not allowing
that space for if/while/for/switch.
Reported-by: Emanuele Torre
Closes#11044
The leftmost "label" of the host name can now only match against single
'*'. Like the browsers have worked for a long time.
- extended unit test 1397 for this
- move some SOURCE variables from unit/Makefile.am to unit/Makefile.inc
Reported-by: Hiroki Kurosawa
Closes#11018
- with `--proxy-http2` allow h2 ALPN negotiation to
forward proxies
- applies to http: requests against a https: proxy only,
as https: requests will auto-tunnel
- adding a HTTP/1 request parser in http1.c
- removed h2h3.c
- using new request parser in nghttp2 and all h3 backends
- adding test 2603 for request parser
- adding h2 proxy test cases to test_10_*
scorecard.py: request scoring accidentally always run curl
with '-v'. Removed that, expect double numbers.
labeller: added http1.* and h2-proxy sources to detection
Closes#10967
- currently only on debug build and when env variable
CURL_PROXY_TUNNEL_H2 is present.
- will ALPN negotiate with the proxy server and switch
tunnel filter based on the protocol negotiated.
- http/1.1 tunnel code moved into cf-h1-proxy.[ch]
- http/2 tunnel code implemented in cf-h2-proxy.[ch]
- tunnel start and ALPN set remains in http_proxy.c
- moving all haproxy related code into cf-haproxy.[ch]
VTLS changes
- SSL filters rely solely on the "alpn" specification they
are created with and no longer check conn->bits.tls_enable_alpn.
- checks on which ALPN specification to use (or none at all) are
done in vtls.c when creating the filter.
Testing
- added a nghttpx forward proxy to the pytest setup that
speaks HTTP/2 and forwards all requests to the Apache httpd
forward proxy server.
- extending test coverage in test_10 cases
- adding proxy tests for direct/tunnel h1/h2 use of basic auth.
- adding test for http/1.1 and h2 proxy tunneling to pytest
Closes#10780
- eliminate receive loop in vtls to fill buffer. This may
lead to partial reads of data which is counter productive
- let http2 instead loop smarter to process pending network
data without transfer switches
scorecard improvements
- do not start caddy when only httpd is requested
- allow curl -v to stderr file on --curl-verbose
Closes#10891
- when rustls is told to recieve more TLS data and its internal
plaintext buffers are full, it returns an IOERROR
- avoid receiving TLS data while plaintext is not read empty
pytest:
- increase curl run timeout when invoking pytest with higher verbosity
Closes#10876
* Configure changes to detect AWS-LC
* CMakeLists.txt changes to detect AWS-LC
* Compile-time branches needed to support AWS-LC
* Correctly set OSSL_VERSION and report AWS-LC release number
* GitHub Actions script to build with autoconf and cmake against AWS-LC
AWS-LC is a BoringSSL/OpenSSL derivative
For more information see https://github.com/awslabs/aws-lc/Closes#10320
SSL backends like OpenSSL/wolfSSL and other return the content of one
TLS record on read, but usually there are more available.
Change the vtls cfilter recv() function to fill the given buffer until a
read would block.
Closes#10736
Various compile failures in gskit.c;
- pipe_ssloverssl() needs Curl_easy data parameter for
Curl_conn_cf_get_socket(cf, data)
- key_passwd is in ssl_config, not conn_config
- close_on() has 2 parameters, not 4
- getsockopt() needs to call Curl_conn_cf_get_socket(), not
cxn->sock[FIRSTSOCKET]
Fixes#10799Closes#10800
In pytest'ing the situation occored that wolfSSL reported an
IO error when the underlying BIO operation was returning an
CURLE_AGAIN condition.
Readding the `io_result` filter context member to detect such
situations.
Also, making sure that the returned CURLcode is initialized
on all recv operations outcome.
Closes#10716
- add parameter to `conn_is_alive()` cfilter method that returns
if there is input data waiting on the connection
- refrain from re-using connnection from the cache that have
input pending
- adapt http/2 and http/3 alive checks to digest pending input
to check the connection state
- remove check_cxn method from openssl as that was just doing
what the socket filter now does.
- add tests for connection reuse with special server configs
Closes#10690
- time_connect was not updated when the overall connection failed,
e.g. when SSL verification was unsuccessful, refs #10670
- rework gather those values to interrogate involved filters,
also from all eyeballing attempts, to report the maximum of
those values.
- added 3 test cases in test_06 to check reported values on
successful, partially failed and totally failed connections.
Reported-by: Master Inspire
Fixes#10670Closes#10671
- refs #10646 where reuse was attempted on closed connections in the
cache, leading to an exhaustion of retries on a transfer
- the mistake was that poll events like POLLHUP, POLLERR, etc
were regarded as "not dead".
- change cf-socket filter check to regard such events as inidication
of corpsiness.
- vtls filter checks: fixed interpretation of backend check result
when inconclusive to interrogate status further down the filter
chain.
Reported-by: SendSonS on github
Fixes#10646Closes#10652
Since cbf57176 the Cirrus CI 'macOS arm64 SecureTransport http2' has
been failing due to c89 warnings mixed code/declaration. That commit is
not the cause so I assume something has changed in the CI outside of our
configuration. Anyway, we don't mix code/declaration so this is the fix
for that.
Closes https://github.com/curl/curl/pull/10574
Turns out "/usr/include/et/com_err.h" typedefs this type (without proper
variable scoping).
comerr is the "common error description library" that apparently might be used
by krb5 code, which then makes this header get used in a curl build.
Reported-by: Bruno Henrique Batista Cruz da Silva
Fixed#10502Closes#10500
- adding pytest test_10 cases for proxy httpd setup tests
- fixing openssl bug in https: proxy hostname verification that
used the hostname of the request and not the proxy name.
Closes#10498
Changes numeric error codes into verbose error codes in two places.
Adds a prefix indicating that the error came from rustls, and in some
places which function it came from.
Adds special handling for RUSTLS_RESULT_UNEXPECTED_EOF, since the
default message of "Unexpected EOF" is insufficiently explanatory.
Closes#10463
Follow-up to e8b00fcd6a
Due to the new 'if(!nonblocking)' check on the block a level above,
there is no need to check for it again within the same conditional.
Detected by Coverity
Closes#10473
move Curl_ssl_setup_x509_store() call after the first send (ClientHello)
this gives time to parse CA anchors while waiting on the server reply
Ref: #10389Closes#10432
I believe the code was secure before this, but limiting the accepted
name length to what is used in the structures should help Coverity's
analysis.
Closes#10431
New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
the equivalent CURLOPT_ done in the library)
- starts a QUIC/HTTP/3 connect right away. Should that not
succeed after 100ms (subject to change), a parallel attempt
is started for HTTP/2 and HTTP/1.1 via TCP
- both attempts are subject to IPv6/IPv4 eyeballing, same
as happens for other connections
- tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT
- use a `soft` timeout at half the value. When the soft timeout
expires, the HTTPS-CONNECT filter checks if the QUIC filter
has received any data from the server. If not, it will start
the HTTP/2 attempt.
HTTP/3(ngtcp2) improvements.
- setting call_data in all cfilter calls similar to http/2 and vtls filters
for use in callback where no stream data is available.
- returning CURLE_PARTIAL_FILE for prematurely terminated transfers
- enabling pytest test_05 for h3
- shifting functionality to "connect" UDP sockets from ngtcp2
implementation into the udp socket cfilter. Because unconnected
UDP sockets are weird. For example they error when adding to a
pollset.
HTTP/3(quiche) improvements.
- fixed upload bug in quiche implementation, now passes 251 and pytest
- error codes on stream RESET
- improved debug logs
- handling of DRAIN during connect
- limiting pending event queue
HTTP/2 cfilter improvements.
- use LOG_CF macros for dynamic logging in debug build
- fix CURLcode on RST streams to be CURLE_PARTIAL_FILE
- enable pytest test_05 for h2
- fix upload pytests and improve parallel transfer performance.
GOAWAY handling for ngtcp2/quiche
- during connect, when the remote server refuses to accept new connections
and closes immediately (so the local conn goes into DRAIN phase), the
connection is torn down and a another attempt is made after a short grace
period.
This is the behaviour observed with nghttpx when we tell it to shut
down gracefully. Tested in pytest test_03_02.
TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).
- new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation.
Invoke:
python3 tests/tests-httpd/scorecard.py --help
for usage.
Improvements on gathering connect statistics and socket access.
- new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters
report connection statistics. This is triggered when the connection
has completely connected.
- new void Curl_pgrsTimeWas(..) method to report a timer update with
a timestamp of when it happend. This allows for updating timers
"later", e.g. a connect statistic after full connectivity has been
reached.
- in case of HTTP eyeballing, the previous changes will update
statistics only from the filter chain that "won" the eyeballing.
- new cfilter query CF_QUERY_SOCKET for retrieving the socket used
by a filter chain.
Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket()
for convenient use of this query.
- Change VTLS backend to query their sub-filters for the socket when
checks during the handshake are made.
HTTP/3 documentation on how https eyeballing works.
TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).
Scorecard with Caddy.
- configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing
- tests/tests-httpd/scorecard.py now measures download speeds with caddy
pytest improvements
- adding Makfile to clean gen dir
- adding nghttpx rundir creation on start
- checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old.
- catch exception when checking for caddy existance on system.
Closes#10349
- Curl_write_plain/Curl_read_plain have been eliminated. Last code use
now uses Curl_conn_send/recv so that requests use conn->send/revc
callbacks which defaults to cfilters use.
- Curl_recv_plain/Curl_send_plain have been internalized in cf-socket.c.
- USE_RECV_BEFORE_SEND_WORKAROUND (active on Windows) has been moved
into cf-socket.c. The pre_recv buffer is held at the socket filter
context. `postponed_data` structures have been removed from
`connectdata`.
- the hanger in HTTP/2 request handling was a result of read buffering
on all sends and the multi handling is not prepared for this. The
following happens:
- multi preforms on a HTTP/2 easy handle
- h2 reads and processes data
- this leads to a send of h2 data
- which receives and buffers before the send
- h2 returns
- multi selects on the socket, but no data arrives (its in the buffer already)
the workaround now receives data in a loop as long as there is something in
the buffer. The real fix would be for multi to change, so that `data_pending`
is evaluated before deciding to wait on the socket.
io_buffer, optional, in cf-socket.c, http/2 sets state.drain if lower
filter have pending data.
This io_buffer is only available/used when the
-DUSE_RECV_BEFORE_SEND_WORKAROUND is active, e.g. on Windows
configurations. It also maintains the original checks on protocol
handler being HTTP and conn->send/recv not being replaced.
The HTTP/2 (nghttp2) cfilter now sets data->state.drain when it finds
out that the "lower" filter chain has still pending data at the end of
its IO operation. This prevents the processing from becoming stalled.
Closes#10280
- Skip content type SSL3_RT_HEADER in verbose TLS output.
This commit prevents bogus and misleading verbose TLS header messages as
discussed in #10299.
Assisted-by: Peter Wu
Closes https://github.com/curl/curl/pull/10299
This avoids UAF issues when `curl_global_cleanup()` is called before all
transfers have been completely handled. Unfortunately this seems to be a
more common pattern than we like.
Closes#10285
- new functions and macros for cfilter debugging
- set CURL_DEBUG with names of cfilters where debug logging should be
enabled
- use GNUC __attribute__ to enable printf format checks during compile
Closes#10271
Using common method for SSL_CTX initialization of verfiy peer and CA
settings. This also provides X509_STORE sharing to become available for
ngtcp2+openssl HTTP/3.
Reported-by: violetlige on github
Fixes#10222Closes#10239
NSS currently uses the default Curl_none_data_pending() method which
always returns false, causing TLS buffered input data to be missed.
The current commit implements the nss_data_pending() method that properly
monitors the presence of available TLS data.
Ref:#10077
Closes#10225
SecureTransport expects result code errSSLWouldBlock when the requested
length could not be sent/recieved in full. The previous code returned
noErr, which let SecureTransport to believe that the IO had terminated
prematurely.
Fixes#10227Closes#10235
- by using BIT() instead of bool
- imap: shrink struct
- ftp: make state 'unsigned char'
- ftp: sort ftp_conn struct entries on size
- urldata: use smaller fields for SSL version info storage
- pop3: reduce the pop3_conn struct size
- smtp: reduce the size of the smtp structs
Closes#10186
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING
checksrc is updated to allow non-year using copyright statements
Closes#10205
The only TLS auth type libcurl ever supported is SRP and that is the
default type. Since nobody ever sets any other type, there is no point
in wasting space to store the set type and code to check the type.
If TLS auth is used, SRP is now implied.
Closes#10181
Refactoring of connection setup and happy eyeballing. Move
nghttp2. ngtcp2, quiche and msh3 into connection filters.
- eyeballing cfilter that uses sub-filters for performing parallel connects
- socket cfilter for all transport types, including QUIC
- QUIC implementations in cfilter, can now participate in eyeballing
- connection setup is more dynamic in order to adapt to what filter did
really connect. Relevant to see if a SSL filter needs to be added or
if SSL has already been provided
- HTTP/3 test cases similar to HTTP/2
- multiuse of parallel transfers for HTTP/3, tested for ngtcp2 and quiche
- Fix for data attach/detach in VTLS filters that could lead to crashes
during parallel transfers.
- Eliminating setup() methods in cfilters, no longer needed.
- Improving Curl_conn_is_alive() to replace Curl_connalive() and
integrated ssl alive checks into cfilter.
- Adding CF_CNTRL_CONN_INFO_UPDATE to tell filters to update
connection into and persist it at the easy handle.
- Several more cfilter related cleanups and moves:
- stream_weigth and dependency info is now wrapped in struct
Curl_data_priority
- Curl_data_priority members depend is available in HTTP2|HTTP3
- Curl_data_priority members depend on NGHTTP2 support
- handling init/reset/cleanup of priority part of url.c
- data->state.priority same struct, but shallow copy for compares only
- PROTOPT_STREAM has been removed
- Curl_conn_is_mulitplex() now available to check on capability
- Adding query method to connection filters.
- ngtcp2+quiche: implementing query for max concurrent transfers.
- Adding is_alive and keep_alive cfilter methods. Adding DATA_SETUP event.
- setting keepalive timestamp on connect
- DATA_SETUP is called after the connection has been completely
setup (but may not connected yet) to allow filters to initialize
data members they use.
- there is no socket to be had with msh3, it is unclear how select
shall work
- manual test via "curl --http3 https://curl.se" fail with "empty
reply from server".
- Various socket/conn related cleanups:
- Curl_socket is now Curl_socket_open and in cf-socket.c
- Curl_closesocket is now Curl_socket_close and in cf-socket.c
- Curl_ssl_use has been replaced with Cur_conn_is_ssl
- Curl_conn_tcp_accepted_set has been split into
Curl_conn_tcp_listen_set and Curl_conn_tcp_accepted_set
with a clearer purpose
Closes#10141
- keeping the "current" easy handle registered at SSL* is no longer
necessary, since the "calling" data object is already stored in the
cfilter's context (and used by other SSL backends from there).
- The "detach" of an easy handle that goes out of scope is then avoided.
- using SSL_set0_wbio for clear reference counting where available.
Closes#10151
When utf8asn1str fails there is no allocation returned, so freeing
the return pointer in **to is at best a no-op and at worst a double-
free bug waiting to happen. The current coding isn't hiding any such
bugs but to future proof, avoid freeing the return value pointer iff
the function failed.
Closes: #10087
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- fixes `Curl_ssl_cf_get_ssl()` to detect also the first filter instance
as ssl (refs #10053)
- replaces `Curl_ssl_use()` with the correct `Curl_conn_is_ssl()`
Closes#10054Fixes#10053
Reported-by: Patrick Monnerat
BIO_read and BIO_write return negative numbers on error, including
retryable ones. A regression from 55807e6. Both branches should be
returning -1.
The APIs are patterned after POSIX read and write which, similarly,
return -1 on errors, not zero, with EAGAIN treated as an error.
Bug: https://github.com/curl/curl/issues/10013#issuecomment-1335308146
Reported-by: David Benjamin
Closes#10021
- `Curl_ssl_get_config()` now returns the first config if no SSL proxy
filter is active
- socket filter starts connection only on first invocation of its
connect method
Fixes#9982Closes#9983
In non-IPv6 builds the conn parameter is unused, and compilers which
run with "-Werror=unused-parameter" (or similar) warnings turned on
fails to build. Below is an excerpt from a CI job:
vtls/openssl.c: In function ‘Curl_ossl_verifyhost’:
vtls/openssl.c:2016:75: error: unused parameter ‘conn’ [-Werror=unused-parameter]
2016 | CURLcode Curl_ossl_verifyhost(struct Curl_easy *data, struct connectdata *conn,
| ~~~~~~~~~~~~~~~~~~~~^~~~
Closes: #9970
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- almost all backend calls pass the Curl_cfilter intance instead of
connectdata+sockindex
- ssl_connect_data is remove from struct connectdata and made internal
to vtls
- ssl_connect_data is allocated in the added filter, kept at cf->ctx
- added function to let a ssl filter access its ssl_primary_config and
ssl_config_data this selects the propert subfields in conn and data,
for filters added as plain or proxy
- adjusted all backends to use the changed api
- adjusted all backends to access config data via the exposed
functions, no longer using conn or data directly
cfilter renames for clear purpose:
- methods `Curl_conn_*(data, conn, sockindex)` work on the complete
filter chain at `sockindex` and connection `conn`.
- methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter
instance.
- methods `Curl_conn_cf()` work on/with filter instances at a
connection.
- rebased and resolved some naming conflicts
- hostname validation (und session lookup) on SECONDARY use the same
name as on FIRST (again).
new debug macros and removing connectdata from function signatures where not
needed.
adapting schannel for new Curl_read_plain paramter.
Closes#9919
- Adding Curl_conn_is_ip_connected() to check if network connectivity
has been reached
- having ftp wait for network connectivity before proceeding with
transfers.
Fixes test failures 1631 and 1632 with hyper.
Closes#9952
Prior to this change Curl_read_plain would attempt to read the
socket directly. On Windows that's a problem because recv data may be
cached by libcurl and that data is only drained using Curl_recv_plain.
Rather than rewrite Curl_read_plain to handle cached recv data, I
changed it to wrap Curl_recv_plain, in much the same way that
Curl_write_plain already wraps Curl_send_plain.
Curl_read_plain -> Curl_recv_plain
Curl_write_plain -> Curl_send_plain
This fixes a bug in the schannel backend where decryption of arbitrary
TLS records fails because cached recv data is never drained. We send
data (TLS records formed by Schannel) using Curl_write_plain, which
calls Curl_send_plain, and that may do a recv-before-send
("pre-receive") to cache received data. The code calls Curl_read_plain
to read data (TLS records from the server), which prior to this change
did not call Curl_recv_plain and therefore cached recv data wasn't
retrieved, resulting in malformed TLS records and decryption failure
(SEC_E_DECRYPT_FAILURE).
The bug has only been observed during Schannel TLS 1.3 handshakes. Refer
to the issue and PR for more information.
--
This is take 2 of the original fix. It preserves the original behavior
of Curl_read_plain to write 0 to the bytes read parameter on error,
since apparently some callers expect that (SOCKS tests were hanging).
The original fix which landed in 12e1def5 and was later reverted in
18383fbf failed to work properly because it did not do that.
Also, it changes Curl_write_plain the same way to complement
Curl_read_plain, and it changes Curl_send_plain to return -1 instead of
0 on CURLE_AGAIN to complement Curl_recv_plain.
Behavior on error with these changes:
Curl_recv_plain returns -1 and *code receives error code.
Curl_send_plain returns -1 and *code receives error code.
Curl_read_plain returns error code and *n (bytes read) receives 0.
Curl_write_plain returns error code and *written receives 0.
--
Ref: https://github.com/curl/curl/issues/9431#issuecomment-1312420361
Assisted-by: Joel Depooter
Reported-by: Egor Pugin
Fixes https://github.com/curl/curl/issues/9431
Closes https://github.com/curl/curl/pull/9949
Prior to this change Curl_read_plain would attempt to read the
socket directly. On Windows that's a problem because recv data may be
cached by libcurl and that data is only drained using Curl_recv_plain.
Rather than rewrite Curl_read_plain to handle cached recv data, I
changed it to wrap Curl_recv_plain, in much the same way that
Curl_write_plain already wraps Curl_send_plain.
Curl_read_plain -> Curl_recv_plain
Curl_write_plain -> Curl_send_plain
This fixes a bug in the schannel backend where decryption of arbitrary
TLS records fails because cached recv data is never drained. We send
data (TLS records formed by Schannel) using Curl_write_plain, which
calls Curl_send_plain, and that may do a recv-before-send
("pre-receive") to cache received data. The code calls Curl_read_plain
to read data (TLS records from the server), which prior to this change
did not call Curl_recv_plain and therefore cached recv data wasn't
retrieved, resulting in malformed TLS records and decryption failure
(SEC_E_DECRYPT_FAILURE).
The bug has only been observed during Schannel TLS 1.3 handshakes. Refer
to the issue and PR for more information.
Ref: https://github.com/curl/curl/issues/9431#issuecomment-1312420361
Assisted-by: Joel Depooter
Reported-by: Egor Pugin
Fixes https://github.com/curl/curl/issues/9431
Closes https://github.com/curl/curl/pull/9904
- general construct/destroy in connectdata
- default implementations of callback functions
- connect: cfilters for connect and accept
- socks: cfilter for socks proxying
- http_proxy: cfilter for http proxy tunneling
- vtls: cfilters for primary and proxy ssl
- change in general handling of data/conn
- Curl_cfilter_setup() sets up filter chain based on data settings,
if none are installed by the protocol handler setup
- Curl_cfilter_connect() boot straps filters into `connected` status,
used by handlers and multi to reach further stages
- Curl_cfilter_is_connected() to check if a conn is connected,
e.g. all filters have done their work
- Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE
indicators for multi select to work
- Curl_cfilter_data_pending() asks filters if the have incoming
data pending for recv
- Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks
installed in conn->recv/conn->send for io handling
- Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters
and addition/removal of a `data` from their connection
- adding vtl functions to prevent use of Curl_ssl globals directly
in other parts of the code.
Reviewed-by: Daniel Stenberg
Closes#9855
Adds a new option to control the maximum time that a cached
certificate store may be retained for.
Currently only the OpenSSL backend implements support for
caching certificate stores.
Closes#9620
This patch aims to cleanup the use of `process.h` header and the macro
`HAVE_PROCESS_H` associated with it.
- `process.h` is always available on Windows. In curl, it is required
only for `_beginthreadex()` in `lib/curl_threads.c`.
- `process.h` is also available in MS-DOS. In curl, its only use was in
`lib/smb.c` for `getpid()`. But `getpid()` is in fact declared by
`unistd.h`, which is always enabled via `lib/config-dos.h`. So the
header is not necessary.
- `HAVE_PROCESS_H` was detected by CMake, forced to 1 on Windows and
left to real detection for other platforms.
It was also set to always-on in `lib/config-win32.h` and
`lib/config-dos.h`.
In autotools builds, there was no detection and the macro was never
set.
Based on these observations, in this patch we:
- Rework Windows `getpid` logic in `lib/smb.c` to always use the
equivalent direct Win32 API function `GetCurrentProcessId()`, as we
already did for Windows UWP apps. This makes `process.h` unnecessary
here on Windows.
- Stop #including `process.h` into files where it was not necessary.
This is everywhere, except `lib/curl_threads.c`.
> Strangely enough, `lib/curl_threads.c` compiled fine with autotools
> because `process.h` is also indirecty included via `unistd.h`. This
> might have been broken in autotools MSVC builds, where the latter
> header is missing.
- Delete all remaining `HAVE_PROCESS_H` feature guards, for they were
unnecessary.
- Delete `HAVE_PROCESS_H` detection from CMake and predefined values
from `lib/config-*.h` headers.
Reviewed-by: Jay Satiro
Closes#9703
No point in having two entry points for the same functions.
Also merged the *safe* function treatment into these so that they can
also be used when one or both pointers are NULL.
Closes#9837
