Commit Graph

162 Commits

Author SHA1 Message Date
Stefan Eissing
db91dbbf2c
curl_log: for failf/infof and debug logging implementations
- new functions and macros for cfilter debugging
 - set CURL_DEBUG with names of cfilters where debug logging should be
   enabled
 - use GNUC __attribute__ to enable printf format checks during compile

Closes #10271
2023-01-12 10:14:17 +01:00
Daniel Stenberg
1485e89213
misc: reduce struct and struct field sizes
- by using BIT() instead of bool
- imap: shrink struct
- ftp: make state 'unsigned char'
- ftp: sort ftp_conn struct entries on size
- urldata: use smaller fields for SSL version info storage
- pop3: reduce the pop3_conn struct size
- smtp: reduce the size of the smtp structs

Closes #10186
2023-01-04 15:36:36 +01:00
Daniel Stenberg
2bc1d775f5
copyright: update all copyright lines and remove year ranges
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205
2023-01-03 09:19:21 +01:00
Daniel Stenberg
becfe2ec78
urldata: cease storing TLS auth type
The only TLS auth type libcurl ever supported is SRP and that is the
default type. Since nobody ever sets any other type, there is no point
in wasting space to store the set type and code to check the type.

If TLS auth is used, SRP is now implied.

Closes #10181
2022-12-31 16:56:21 +01:00
Stefan Eissing
71b7e01610
lib: connect/h2/h3 refactor
Refactoring of connection setup and happy eyeballing. Move
nghttp2. ngtcp2, quiche and msh3 into connection filters.

 - eyeballing cfilter that uses sub-filters for performing parallel connects
 - socket cfilter for all transport types, including QUIC
 - QUIC implementations in cfilter, can now participate in eyeballing
 - connection setup is more dynamic in order to adapt to what filter did
   really connect.  Relevant to see if a SSL filter needs to be added or
   if SSL has already been provided
 - HTTP/3 test cases similar to HTTP/2
 - multiuse of parallel transfers for HTTP/3, tested for ngtcp2 and quiche

 - Fix for data attach/detach in VTLS filters that could lead to crashes
   during parallel transfers.
 - Eliminating setup() methods in cfilters, no longer needed.
 - Improving Curl_conn_is_alive() to replace Curl_connalive() and
   integrated ssl alive checks into cfilter.
 - Adding CF_CNTRL_CONN_INFO_UPDATE to tell filters to update
   connection into and persist it at the easy handle.

 - Several more cfilter related cleanups and moves:
   - stream_weigth and dependency info is now wrapped in struct
     Curl_data_priority
   - Curl_data_priority members depend is available in HTTP2|HTTP3
   - Curl_data_priority members depend on NGHTTP2 support
   - handling init/reset/cleanup of priority part of url.c
   - data->state.priority same struct, but shallow copy for compares only

 - PROTOPT_STREAM has been removed
   - Curl_conn_is_mulitplex() now available to check on capability

 - Adding query method to connection filters.
   - ngtcp2+quiche: implementing query for max concurrent transfers.

 - Adding is_alive and keep_alive cfilter methods. Adding DATA_SETUP event.
   - setting keepalive timestamp on connect
   - DATA_SETUP is called after the connection has been completely
     setup (but may not connected yet) to allow filters to initialize
     data members they use.

 - there is no socket to be had with msh3, it is unclear how select
   shall work

 - manual test via "curl --http3 https://curl.se" fail with "empty
   reply from server".

 - Various socket/conn related cleanups:
   - Curl_socket is now Curl_socket_open and in cf-socket.c
   - Curl_closesocket is now Curl_socket_close and in cf-socket.c
   - Curl_ssl_use has been replaced with Cur_conn_is_ssl
   - Curl_conn_tcp_accepted_set has been split into
     Curl_conn_tcp_listen_set and Curl_conn_tcp_accepted_set
     with a clearer purpose

Closes #10141
2022-12-30 16:43:19 +01:00
Stefan Eissing
b42156b825
cfilter: improve SSL connection checks
- fixes `Curl_ssl_cf_get_ssl()` to detect also the first filter instance
  as ssl (refs #10053)

- replaces `Curl_ssl_use()` with the correct `Curl_conn_is_ssl()`

Closes #10054
Fixes #10053

Reported-by: Patrick Monnerat
2022-12-08 14:27:06 +01:00
Daniel Stenberg
07203b0bb8
openssl/mbedtls: use %d for outputing port with failf (int)
Coverity CID 1517100

Also, remove some int typecasts in vtls.c for the port number

Closes #10001
2022-11-29 23:16:47 +01:00
Stefan Eissing
55807e6c05
tls: backends use connection filters for IO, enabling HTTPS-proxy
- OpenSSL (and compatible)
 - BearSSL
 - gnutls
 - mbedtls
 - rustls
 - schannel
 - secure-transport
 - wolfSSL (v5.0.0 and newer)

 This leaves only the following without HTTPS-proxy support:
 - gskit
 - nss
 - wolfSSL (versions earlier than v5.0.0)

Closes #9962
2022-11-28 13:56:23 +01:00
Stefan Eissing
f18956d0ca
cfilter: re-add conn as parameter to cfilter setup methods
- `Curl_ssl_get_config()` now returns the first config if no SSL proxy
  filter is active

- socket filter starts connection only on first invocation of its
  connect method

Fixes #9982
Closes #9983
2022-11-26 16:56:46 +01:00
Daniel Stenberg
9ecebb5d00
vtls: repair build with disabled proxy
Closes #9974
2022-11-24 09:04:14 +01:00
Stefan Eissing
af22c2a546
vtls: localization of state data in filters
- almost all backend calls pass the Curl_cfilter intance instead of
   connectdata+sockindex
 - ssl_connect_data is remove from struct connectdata and made internal
   to vtls
 - ssl_connect_data is allocated in the added filter, kept at cf->ctx

 - added function to let a ssl filter access its ssl_primary_config and
   ssl_config_data this selects the propert subfields in conn and data,
   for filters added as plain or proxy
 - adjusted all backends to use the changed api
 - adjusted all backends to access config data via the exposed
   functions, no longer using conn or data directly

cfilter renames for clear purpose:

 - methods `Curl_conn_*(data, conn, sockindex)` work on the complete
   filter chain at `sockindex` and connection `conn`.
 - methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter
   instance.
 - methods `Curl_conn_cf()` work on/with filter instances at a
   connection.
 - rebased and resolved some naming conflicts
 - hostname validation (und session lookup) on SECONDARY use the same
   name as on FIRST (again).

new debug macros and removing connectdata from function signatures where not
needed.

adapting schannel for new Curl_read_plain paramter.

Closes #9919
2022-11-22 14:25:50 +01:00
Stefan Eissing
4a8b4a1b66
cfiler: filter types have flags indicating what they do
- Adding Curl_conn_is_ip_connected() to check if network connectivity
  has been reached

- having ftp wait for network connectivity before proceeding with
  transfers.

Fixes test failures 1631 and 1632 with hyper.

Closes #9952
2022-11-22 08:20:38 +01:00
Daniel Stenberg
aecc549699
vtls: fix build without proxy support
Follow-up to dafdb20a26

Closes #9895
2022-11-12 00:14:24 +01:00
Stefan Eissing
dafdb20a26
lib: connection filters (cfilter) addition to curl:
- general construct/destroy in connectdata
 - default implementations of callback functions
 - connect: cfilters for connect and accept
 - socks: cfilter for socks proxying
 - http_proxy: cfilter for http proxy tunneling
 - vtls: cfilters for primary and proxy ssl
 - change in general handling of data/conn
 - Curl_cfilter_setup() sets up filter chain based on data settings,
   if none are installed by the protocol handler setup
 - Curl_cfilter_connect() boot straps filters into `connected` status,
   used by handlers and multi to reach further stages
 - Curl_cfilter_is_connected() to check if a conn is connected,
   e.g. all filters have done their work
 - Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE
   indicators for multi select to work
 - Curl_cfilter_data_pending() asks filters if the have incoming
   data pending for recv
 - Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks
   installed in conn->recv/conn->send for io handling
 - Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters
   and addition/removal of a `data` from their connection
 - adding vtl functions to prevent use of Curl_ssl globals directly
   in other parts of the code.

Reviewed-by: Daniel Stenberg
Closes #9855
2022-11-11 15:17:51 +01:00
Michael Drake
3c16697ebd
openssl: reduce CA certificate bundle reparsing by caching
Closes #9620
2022-11-08 10:06:12 +01:00
Daniel Stenberg
3f039dfd6f
strcase: use curl_str(n)equal for case insensitive matches
No point in having two entry points for the same functions.

Also merged the *safe* function treatment into these so that they can
also be used when one or both pointers are NULL.

Closes #9837
2022-11-01 17:01:26 +01:00
Daniel Stenberg
ed5095ed94
strcase: add and use Curl_timestrcmp
This is a strcmp() alternative function for comparing "secrets",
designed to take the same time no matter the content to not leak
match/non-match info to observers based on how fast it is.

The time this function takes is only a function of the shortest input
string.

Reported-by: Trail of Bits

Closes #9658
2022-10-08 11:50:47 +02:00
Daniel Stenberg
336f7cd5af
vtls: make Curl_ssl_backend() return the enum type curl_sslbackend
To fix the icc warning enumerated type mixed with another type

Closes #9179
2022-07-23 13:38:33 +02:00
Daniel Stenberg
3fa343a35c
urldata: make 'buffer_size' an unsigned int
It is already capped at READBUFFER_MAX which fits easily in 32 bits.

Closes #9098
2022-07-04 23:03:44 +02:00
Jay Satiro
a8a4abb2ae vtls: make curl_global_sslset thread-safe
.. and update some docs to explain curl_global_* is now thread-safe.

Follow-up to 23af112 which made curl_global_init/cleanup thread-safe.

Closes https://github.com/curl/curl/pull/9016
2022-06-16 03:18:40 -04:00
max.mehl
ad9bc5976d
copyright: make repository REUSE compliant
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869
2022-06-13 09:13:00 +02:00
Daniel Stenberg
8d86718f3f
opts: deprecate RANDOM_FILE and EGDSOCKET
These two options were only ever used for the OpenSSL backend for
versions before 1.1.0. They were never used for other backends and they
are not used with recent OpenSSL versions. They were never used much by
applications.

The defines RANDOM_FILE and EGD_SOCKET can still be set at build-time
for ancient EOL OpenSSL versions.

Closes #8670
2022-05-17 11:05:54 +02:00
Daniel Stenberg
f18af4f874
tls: check more TLS details for connection reuse
CVE-2022-27782

Reported-by: Harry Sintonen
Bug: https://curl.se/docs/CVE-2022-27782.html
Closes #8825
2022-05-09 23:13:53 +02:00
Patrick Monnerat
852aa5ad35
url: check sasl additional parameters for connection reuse.
Also move static function safecmp() as non-static Curl_safecmp() since
its purpose is needed at several places.

Bug: https://curl.se/docs/CVE-2022-22576.html

CVE-2022-22576

Closes #8746
2022-04-25 11:44:05 +02:00
MAntoniak
e9e79622d0
vtls: fix socket check conditions
fix condition to check the second socket during associate and
disassociate connection

Closes #8493
2022-02-22 13:39:54 +01:00
MAntoniak
ccc2752ce8
ssl: reduce allocated space for ssl backend when FTP is disabled
Add assert() for the backend pointer in many places

Closes #8471
2022-02-21 08:39:55 +01:00
Daniel Stenberg
2610142139
lib: remove support for CURL_DOES_CONVERSIONS
TPF was the only user and support for that was dropped.

Closes #8378
2022-02-04 08:05:35 +01:00
Daniel Stenberg
2218c3a57e
vtls: pass on the right SNI name
The TLS backends convert the host name to SNI name and need to use that.
This involves cutting off any trailing dot and lowercasing.

Co-authored-by: Jay Satiro
Closes #8320
2022-01-27 08:36:10 +01:00
Daniel Stenberg
223f26c28a
mesalink: remove support
Mesalink has ceased development. We can no longer encourage use of it.
It seems to be continued under the name TabbySSL, but no attempts have
(yet) been to make curl support it.

Fixes #8188
Closes #8191
2022-01-10 11:27:59 +01:00
Michael Kaufmann
60738f398c vtls: Fix a memory leak if an SSL session cannot be added to the cache
On connection shutdown, a new TLS session ticket may arrive after the
SSL session cache has already been destructed. In this case, the new
SSL session cannot be added to the SSL session cache.

The callers of Curl_ssl_addsessionid() need to know whether the SSL
session has been added to the cache. If it has not been added, the
reference counter of the SSL session must not be incremented, or memory
used by the SSL session must be freed. This is now possible with the new
output parameter "added" of Curl_ssl_addsessionid().

Fixes #7683
Closes #7752
2021-09-24 13:23:42 +02:00
Daniel Stenberg
5ea3145850
vtls: fix connection reuse checks for issuer cert and case sensitivity
CVE-2021-22924

Reported-by: Harry Sintonen
Bug: https://curl.se/docs/CVE-2021-22924.html
2021-07-20 09:36:06 +02:00
Daniel Stenberg
e7416cfd2b
infof: remove newline from format strings, always append it
- the data needs to be "line-based" anyway since it's also passed to the
  debug callback/application

- it makes infof() work like failf() and consistency is good

- there's an assert that triggers on newlines in the format string

- Also removes a few instances of "..."

- Removes the code that would append "..." to the end of the data *iff*
  it was truncated in infof()

Closes #7357
2021-07-07 22:54:01 +02:00
Daniel Stenberg
63c7668182
version: turn version number functions into returning void
... as we never use the return codes from them.

Reviewed-by: Daniel Gustafsson
Closes #7319
2021-06-30 23:23:34 +02:00
Daniel Stenberg
97fa671a73
vtls: use free() not curl_free()
curl_free() is provided for users of the API to free returned data,
there's no need to use it internally.

Closes #7318
2021-06-30 13:50:18 +02:00
Jay Satiro
b31d9ccfc2 vtls: fix warning due to function prototype mismatch
b09c8ee changed the function prototype. Caught by Visual Studio.
2021-06-21 01:59:05 -04:00
Daniel Stenberg
a6da296867
vtls: only store TIMER_APPCONNECT for non-proxy connect
Introducing a 'isproxy' argument to the connect function so that it
knows wether to store the time stamp or not.

Reported-by: Yongkang Huang
Fixes #7274
Closes #7274
2021-06-19 23:02:16 +02:00
Daniel Stenberg
a5adf8cb14
openssl: don't remove session id entry in disassociate
When a connection is disassociated from a transfer, the Session ID entry
should remain.

Regression since 7f4a9a9 (shipped in libcurl 7.77.0)
Reported-by: Gergely Nagy
Reported-by: Paul Groke

Fixes #7222
Closes #7230
2021-06-11 12:52:55 +02:00
Daniel Stenberg
c7b93a76ea
vtls: exit addsessionid if no cache is inited
Follow-up to b249592d29

Avoids NULL pointer derefs.

Closes #7165
2021-06-01 13:40:30 +02:00
Daniel Stenberg
894c74738f
Curl_ssl_getsessionid: fail if no session cache exists
This function might get called for an easy handle for which the session
cache hasn't been setup. It now just returns a "miss" in that case.

Reported-by: Christoph M. Becker
Fixes #7148
Closes #7153
2021-05-31 20:41:40 +02:00
Harry Sintonen
7f4a9a9b2a
openssl: associate/detach the transfer from connection
CVE-2021-22901

Bug: https://curl.se/docs/CVE-2021-22901.html
2021-05-24 13:15:10 +02:00
Gilles Vollant
77fc3859b2 SSL: support in-memory CA certs for some backends
- New options CURLOPT_CAINFO_BLOB and CURLOPT_PROXY_CAINFO_BLOB to
  specify in-memory PEM certificates for OpenSSL, Schannel (Windows)
  and Secure Transport (Apple) SSL backends.

Prior to this change PEM certificates could only be imported from a file
and not from memory.

Co-authored-by: moparisthebest@users.noreply.github.com

Ref: https://github.com/curl/curl/pull/4679
Ref: https://github.com/curl/curl/pull/5677
Ref: https://github.com/curl/curl/pull/6109

Closes https://github.com/curl/curl/pull/6662
2021-05-05 02:29:16 -04:00
Patrick Monnerat
a4554b2c5e
vtls: reset ssl use flag upon negotiation failure
Fixes the segfault in ldaps disconnect.

Reported-by: Illarion Taev
Fixes #6934
Closes #6937
2021-04-23 14:41:39 +02:00
Daniel Stenberg
2c26eeef12
vtls: fix addsessionid for non-proxy builds
Follow-up to b09c8ee157
Fixes #6812
Closes #6811
2021-03-29 14:20:13 +02:00
Daniel Stenberg
b09c8ee157
vtls: add 'isproxy' argument to Curl_ssl_get/addsessionid()
To make sure we set and extract the correct session.

Reported-by: Mingtao Yang
Bug: https://curl.se/docs/CVE-2021-22890.html

CVE-2021-22890
2021-03-28 23:19:56 +02:00
Jacob Hoffman-Andrews
246399a874
vtls: initial implementation of rustls backend
This adds a new TLS backend, rustls. It uses the C-to-rustls bindings
from https://github.com/abetterinternet/crustls.

Rustls is at https://github.com/ctz/rustls/.

There is still a fair bit to be done, like sending CloseNotify on
connection shutdown, respecting CAPATH, and properly indicating features
like "supports TLS 1.3 ciphersuites." But it works well enough to make
requests and receive responses.

Blog post for context:
https://www.abetterinternet.org/post/memory-safe-curl/

Closes #6350
2021-02-09 11:06:18 +01:00
Jacob Hoffman-Andrews
c30bf22f1c
vtls: factor out Curl_ssl_getsock to field of Curl_ssl
Closes #6558
2021-02-08 14:10:55 +01:00
Daniel Stenberg
2c4b338f7e
vtls: remove md5sum
As it is not used anymore.

Reported-by: Jacob Hoffman-Andrews
Bug: https://curl.se/mail/lib-2021-02/0000.html

Closes #6557
2021-02-02 08:13:09 +01:00
Patrick Monnerat
bbe3aa9f88
vtls: reduce conn->data use
Closes #6474
2021-01-19 09:14:40 +01:00
Daniel Stenberg
c2c9e44a4d
vtls: remove Curl_ prefix from static functions 2021-01-14 08:09:09 +01:00
Daniel Stenberg
8ab78f720a
misc: fix "warning: empty expression statement has no effect"
Turned several macros into do-while(0) style to allow their use to work
find with semicolon.

Bug: 08e8455ddd (commitcomment-45433279)
Follow-up to 08e8455ddd
Reported-by: Gisle Vanem
Closes #6376
2020-12-26 23:44:17 +01:00