Commit Graph

240 Commits

Author SHA1 Message Date
Viktor Szakats
3f8fc25720
cmake: add support for "unity" builds
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.

It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.

Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.

Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
  These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
  `statename`.
- rename `ssl_backend_data` structure to unique names for each
  TLS-backend, along with the `ssl_connect_data` struct member
  referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
  builds when compiling `lib/ldap.c`. To support "unity" builds, we had
  to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
  `schannel.h` option) _globally_. This caused an indirect inclusion of
  Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
  as well. This requires `[P]UNICODE_STRING` types, which is apperantly
  not defined automatically (as seen with both MSVS and mingw-w64).
  This patch includes `<subauth.h>` to fix it.
  Ref: https://github.com/curl/curl/runs/13987772013
  Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
  trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
  is built in static mode.

KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
  https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250

Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes #11095
2023-06-07 13:06:08 +00:00
Stefan Eissing
4ae2d9f24d
proxy: http2 proxy tunnel implementation
- currently only on debug build and when env variable
  CURL_PROXY_TUNNEL_H2 is present.
- will ALPN negotiate with the proxy server and switch
  tunnel filter based on the protocol negotiated.
- http/1.1 tunnel code moved into cf-h1-proxy.[ch]
- http/2 tunnel code implemented in cf-h2-proxy.[ch]
- tunnel start and ALPN set remains in http_proxy.c
- moving all haproxy related code into cf-haproxy.[ch]

VTLS changes
- SSL filters rely solely on the "alpn" specification they
  are created with and no longer check conn->bits.tls_enable_alpn.
- checks on which ALPN specification to use (or none at all) are
  done in vtls.c when creating the filter.

Testing
- added a nghttpx forward proxy to the pytest setup that
  speaks HTTP/2 and forwards all requests to the Apache httpd
  forward proxy server.
- extending test coverage in test_10 cases
- adding proxy tests for direct/tunnel h1/h2 use of basic auth.
- adding test for http/1.1 and h2 proxy tunneling to pytest

Closes #10780
2023-04-06 13:04:46 +02:00
Stefan Eissing
671158242d
connections: introduce http/3 happy eyeballs
New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
  the equivalent CURLOPT_ done in the library)
- starts a QUIC/HTTP/3 connect right away. Should that not
  succeed after 100ms (subject to change), a parallel attempt
  is started for HTTP/2 and HTTP/1.1 via TCP
- both attempts are subject to IPv6/IPv4 eyeballing, same
  as happens for other connections
- tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT
- use a `soft` timeout at half the value. When the soft timeout
  expires, the HTTPS-CONNECT filter checks if the QUIC filter
  has received any data from the server. If not, it will start
  the HTTP/2 attempt.

HTTP/3(ngtcp2) improvements.
- setting call_data in all cfilter calls similar to http/2 and vtls filters
  for use in callback where no stream data is available.
- returning CURLE_PARTIAL_FILE for prematurely terminated transfers
- enabling pytest test_05 for h3
- shifting functionality to "connect" UDP sockets from ngtcp2
  implementation into the udp socket cfilter. Because unconnected
  UDP sockets are weird. For example they error when adding to a
  pollset.

HTTP/3(quiche) improvements.
- fixed upload bug in quiche implementation, now passes 251 and pytest
- error codes on stream RESET
- improved debug logs
- handling of DRAIN during connect
- limiting pending event queue

HTTP/2 cfilter improvements.
- use LOG_CF macros for dynamic logging in debug build
- fix CURLcode on RST streams to be CURLE_PARTIAL_FILE
- enable pytest test_05 for h2
- fix upload pytests and improve parallel transfer performance.

GOAWAY handling for ngtcp2/quiche
- during connect, when the remote server refuses to accept new connections
  and closes immediately (so the local conn goes into DRAIN phase), the
  connection is torn down and a another attempt is made after a short grace
  period.
  This is the behaviour observed with nghttpx when we tell it to  shut
  down gracefully. Tested in pytest test_03_02.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
  copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
  and connection related things based on the negotiated protocol (or lack thereof).

- new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation.
  Invoke:
    python3 tests/tests-httpd/scorecard.py --help
  for usage.

Improvements on gathering connect statistics and socket access.
- new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters
  report connection statistics. This is triggered when the connection
  has completely connected.
- new void Curl_pgrsTimeWas(..) method to report a timer update with
  a timestamp of when it happend. This allows for updating timers
  "later", e.g. a connect statistic after full connectivity has been
  reached.
- in case of HTTP eyeballing, the previous changes will update
  statistics only from the filter chain that "won" the eyeballing.
- new cfilter query CF_QUERY_SOCKET for retrieving the socket used
  by a filter chain.
  Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket()
  for convenient use of this query.
- Change VTLS backend to query their sub-filters for the socket when
  checks during the handshake are made.

HTTP/3 documentation on how https eyeballing works.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
  copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
  and connection related things based on the negotiated protocol (or lack thereof).

Scorecard with Caddy.
- configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing
- tests/tests-httpd/scorecard.py now measures download speeds with caddy

pytest improvements
- adding Makfile to clean gen dir
- adding nghttpx rundir creation on start
- checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old.
- catch exception when checking for caddy existance on system.

Closes #10349
2023-02-02 09:57:34 +01:00
Stefan Eissing
9e93bd47c2 vtls: Manage current easy handle in nested cfilter calls
The previous implementation cleared `data` so the outer invocation lost
its data, which could lead to a crash.

Bug: https://github.com/curl/curl/issues/10336
Reported-by: Fujii Hironori

Closes https://github.com/curl/curl/pull/10340
2023-01-26 03:05:01 -05:00
Daniel Stenberg
2bc1d775f5
copyright: update all copyright lines and remove year ranges
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205
2023-01-03 09:19:21 +01:00
Daniel Stenberg
becfe2ec78
urldata: cease storing TLS auth type
The only TLS auth type libcurl ever supported is SRP and that is the
default type. Since nobody ever sets any other type, there is no point
in wasting space to store the set type and code to check the type.

If TLS auth is used, SRP is now implied.

Closes #10181
2022-12-31 16:56:21 +01:00
Daniel Stenberg
df856cb5c9
vtls: use ALPN HTTP/1.0 when HTTP/1.0 is used
Previously libcurl would use the HTTP/1.1 ALPN id even when the
application specified HTTP/1.0.

Reported-by: William Tang
Ref: #10183
2022-12-31 16:53:21 +01:00
Stefan Eissing
27ec767ebd
gnutls: use common gnutls init and verify code for ngtcp2
Closes #10007
2022-12-01 10:57:36 +01:00
Stefan Eissing
55807e6c05
tls: backends use connection filters for IO, enabling HTTPS-proxy
- OpenSSL (and compatible)
 - BearSSL
 - gnutls
 - mbedtls
 - rustls
 - schannel
 - secure-transport
 - wolfSSL (v5.0.0 and newer)

 This leaves only the following without HTTPS-proxy support:
 - gskit
 - nss
 - wolfSSL (versions earlier than v5.0.0)

Closes #9962
2022-11-28 13:56:23 +01:00
Stefan Eissing
af22c2a546
vtls: localization of state data in filters
- almost all backend calls pass the Curl_cfilter intance instead of
   connectdata+sockindex
 - ssl_connect_data is remove from struct connectdata and made internal
   to vtls
 - ssl_connect_data is allocated in the added filter, kept at cf->ctx

 - added function to let a ssl filter access its ssl_primary_config and
   ssl_config_data this selects the propert subfields in conn and data,
   for filters added as plain or proxy
 - adjusted all backends to use the changed api
 - adjusted all backends to access config data via the exposed
   functions, no longer using conn or data directly

cfilter renames for clear purpose:

 - methods `Curl_conn_*(data, conn, sockindex)` work on the complete
   filter chain at `sockindex` and connection `conn`.
 - methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter
   instance.
 - methods `Curl_conn_cf()` work on/with filter instances at a
   connection.
 - rebased and resolved some naming conflicts
 - hostname validation (und session lookup) on SECONDARY use the same
   name as on FIRST (again).

new debug macros and removing connectdata from function signatures where not
needed.

adapting schannel for new Curl_read_plain paramter.

Closes #9919
2022-11-22 14:25:50 +01:00
Stefan Eissing
dafdb20a26
lib: connection filters (cfilter) addition to curl:
- general construct/destroy in connectdata
 - default implementations of callback functions
 - connect: cfilters for connect and accept
 - socks: cfilter for socks proxying
 - http_proxy: cfilter for http proxy tunneling
 - vtls: cfilters for primary and proxy ssl
 - change in general handling of data/conn
 - Curl_cfilter_setup() sets up filter chain based on data settings,
   if none are installed by the protocol handler setup
 - Curl_cfilter_connect() boot straps filters into `connected` status,
   used by handlers and multi to reach further stages
 - Curl_cfilter_is_connected() to check if a conn is connected,
   e.g. all filters have done their work
 - Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE
   indicators for multi select to work
 - Curl_cfilter_data_pending() asks filters if the have incoming
   data pending for recv
 - Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks
   installed in conn->recv/conn->send for io handling
 - Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters
   and addition/removal of a `data` from their connection
 - adding vtl functions to prevent use of Curl_ssl globals directly
   in other parts of the code.

Reviewed-by: Daniel Stenberg
Closes #9855
2022-11-11 15:17:51 +01:00
Michael Drake
3c16697ebd
openssl: reduce CA certificate bundle reparsing by caching
Closes #9620
2022-11-08 10:06:12 +01:00
Patrick Monnerat
72652c0613
http, vauth: always provide Curl_allow_auth_to_host() functionality
This function is currently located in the lib/http.c module and is
therefore disabled by the CURL_DISABLE_HTTP conditional token.

As it may be called by TLS backends, disabling HTTP results in an
undefined reference error at link time.

Move this function to vauth/vauth.c to always provide it and rename it
as Curl_auth_allowed_to_host() to respect the vauth module naming
convention.

Closes #9600
2022-09-27 14:05:37 +02:00
Daniel Stenberg
472f1cbe7e
NPN: remove support for and use of
Next Protocol Negotiation is a TLS extension that was created and used
for agreeing to use the SPDY protocol (the precursor to HTTP/2) for
HTTPS. In the early days of HTTP/2, before the spec was finalized and
shipped, the protocol could be enabled using this extension with some
servers.

curl supports the NPN extension with some TLS backends since then, with
a command line option `--npn` and in libcurl with
`CURLOPT_SSL_ENABLE_NPN`.

HTTP/2 proper is made to use the ALPN (Application-Layer Protocol
Negotiation) extension and the NPN extension has no purposes
anymore. The HTTP/2 spec was published in May 2015.

Today, use of NPN in the wild should be extremely rare and most likely
totally extinct. Chrome removed NPN support in Chrome 51, shipped in
June 2016. Removed in Firefox 53, April 2017.

Closes #9307
2022-09-05 07:39:02 +02:00
max.mehl
ad9bc5976d
copyright: make repository REUSE compliant
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869
2022-06-13 09:13:00 +02:00
Daniel Stenberg
f18af4f874
tls: check more TLS details for connection reuse
CVE-2022-27782

Reported-by: Harry Sintonen
Bug: https://curl.se/docs/CVE-2022-27782.html
Closes #8825
2022-05-09 23:13:53 +02:00
Daniel Stenberg
0935315562
gnutls: don't leak the SRP credentials in redirects
Follow-up to 620ea21410 and 139a54ed0a

Reported-by: Harry Sintonen
Closes #8752
2022-04-26 07:46:20 +02:00
Daniel Stenberg
6968fb9d54
lib: remove exclamation marks
... from infof() and failf() calls. Make them less attention seeking.

Closes #8713
2022-04-16 11:55:05 +02:00
Daniel Stenberg
34ebf3f90d
vtls: use a generic "ALPN, server accepted" message
Closes #8657
2022-03-31 14:21:01 +02:00
Daniel Stenberg
55043b40c1
vtls: use a backend standard message for "ALPN: offers %s"
I call it VTLS_INFOF_ALPN_OFFER_1STR, the '1str' meaning that the
infof() call also needs a string argument: the ALPN ID.

Closes #8657
2022-03-31 14:20:56 +02:00
Daniel Stenberg
3bc5b32db5
vtls: provide a unified APLN-disagree string for all backends
Also rephrase to make it sound less dangerous:

 "ALPN: server did not agree on a protocol. Uses default."

Reported-by: Nick Coghlan
Fixes #8643
Closes #8651
2022-03-31 09:20:16 +02:00
Jay Satiro
8b1cae63b7 gtls: fix build for disabled TLS-SRP
Prior to this change if, at build time, the GnuTLS backend was found to
have TLS-SRP support (HAVE_GNUTLS_SRP) but TLS-SRP was disabled in curl
via --disable-tls-srp (!USE_TLS_SRP) then a build error would occur.

Bug: https://curl.se/mail/lib-2022-03/0046.html
Reported-by: Robert Brose

Closes https://github.com/curl/curl/pull/8604
2022-03-22 14:24:08 -04:00
MAntoniak
ccc2752ce8
ssl: reduce allocated space for ssl backend when FTP is disabled
Add assert() for the backend pointer in many places

Closes #8471
2022-02-21 08:39:55 +01:00
Daniel Stenberg
2218c3a57e
vtls: pass on the right SNI name
The TLS backends convert the host name to SNI name and need to use that.
This involves cutting off any trailing dot and lowercasing.

Co-authored-by: Jay Satiro
Closes #8320
2022-01-27 08:36:10 +01:00
Daniel Stenberg
8fbd6feddf
ngtcp2: verify the server certificate for the gnutls case
Closes #8178
2021-12-28 23:34:30 +01:00
Daniel Stenberg
21248e052d
checksrc: detect more kinds of NULL comparisons we avoid
Co-authored-by: Jay Satiro
Closes #8180
2021-12-27 23:39:26 +01:00
Daniel Stenberg
4a2ab69ab5
gtls: check return code for gnutls_alpn_set_protocols
Closes #8181
2021-12-26 11:28:23 +01:00
Daniel Gustafsson
12246eddc5 lib: avoid fallthrough cases in switch statements
Commit b5a434f7f0 inhibits the warning
on implicit fallthrough cases, since the current coding of indicating
fallthrough with comments is falling out of fashion with new compilers.
This attempts to make the issue smaller by rewriting fallthroughs to no
longer fallthrough, via either breaking the cases or turning switch
statements into if statements.

  lib/content_encoding.c: the fallthrough codepath is simply copied
    into the case as it's a single line.
  lib/http_ntlm.c: the fallthrough case skips a state in the state-
    machine and fast-forwards to NTLMSTATE_LAST. Do this before the
    switch statement instead to set up the states that we actually
    want.
  lib/http_proxy.c: the fallthrough is just falling into exiting the
    switch statement which can be done easily enough in the case.
  lib/mime.c: switch statement rewritten as if statement.
  lib/pop3.c: the fallthrough case skips to the next state in the
    statemachine, do this explicitly instead.
  lib/urlapi.c: switch statement rewritten as if statement.
  lib/vssh/wolfssh.c: the fallthrough cases fast-forwards the state
    machine, do this by running another iteration of the switch
    statement instead.
  lib/vtls/gtls.c: switch statement rewritten as if statement.
  lib/vtls/nss.c: the fallthrough codepath is simply copied into the
    case as it's a single line. Also twiddle a comment to not be
    inside a non-brace if statement.

Closes: #7322
See-also: #7295
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2021-09-29 10:00:52 +02:00
Michael Kaufmann
60738f398c vtls: Fix a memory leak if an SSL session cannot be added to the cache
On connection shutdown, a new TLS session ticket may arrive after the
SSL session cache has already been destructed. In this case, the new
SSL session cannot be added to the SSL session cache.

The callers of Curl_ssl_addsessionid() need to know whether the SSL
session has been added to the cache. If it has not been added, the
reference counter of the SSL session must not be incremented, or memory
used by the SSL session must be freed. This is now possible with the new
output parameter "added" of Curl_ssl_addsessionid().

Fixes #7683
Closes #7752
2021-09-24 13:23:42 +02:00
Daniel Stenberg
5ea3145850
vtls: fix connection reuse checks for issuer cert and case sensitivity
CVE-2021-22924

Reported-by: Harry Sintonen
Bug: https://curl.se/docs/CVE-2021-22924.html
2021-07-20 09:36:06 +02:00
Daniel Stenberg
e7416cfd2b
infof: remove newline from format strings, always append it
- the data needs to be "line-based" anyway since it's also passed to the
  debug callback/application

- it makes infof() work like failf() and consistency is good

- there's an assert that triggers on newlines in the format string

- Also removes a few instances of "..."

- Removes the code that would append "..." to the end of the data *iff*
  it was truncated in infof()

Closes #7357
2021-07-07 22:54:01 +02:00
Daniel Stenberg
bfa0309175
gnutls: set the preferred TLS versions in correct order
Regression since 781864bedb (curl 7.77.0)

Reported-by: civodul on github
Assisted-by: Nikos Mavrogiannopoulos
Fixes #7277
Closes #7278
2021-06-18 23:22:21 +02:00
Michael Kaufmann
b249592d29 ssl: read pending close notify alert before closing the connection
This avoids a TCP reset (RST) if the server initiates a connection
shutdown by sending an SSL close notify alert and then closes the TCP
connection.

For SSL connections, usually the server announces that it will close the
connection with an SSL close notify alert. curl should read this alert.
If curl does not read this alert and just closes the connection, some
operating systems close the TCP connection with an RST flag.

See RFC 1122, section 4.2.2.13

If curl reads the close notify alert, the TCP connection is closed
normally with a FIN flag.

The new code is similar to existing code in the "SSL shutdown" function:
try to read an alert (non-blocking), and ignore any read errors.

Closes #7095
2021-06-01 09:40:40 +02:00
Harry Sintonen
7f4a9a9b2a
openssl: associate/detach the transfer from connection
CVE-2021-22901

Bug: https://curl.se/docs/CVE-2021-22901.html
2021-05-24 13:15:10 +02:00
Daniel Stenberg
577f19397c
GnuTLS: don't allow TLS 1.3 for versions that don't support it
Follow-up to 781864bedb

... as they don't understand it and will return error at us!

Closes #7014
2021-05-06 16:04:30 +02:00
Daniel Stenberg
781864bedb
gnutls: make setting only the MAX TLS allowed version work
Previously, settting only the max allowed TLS version, leaving the
minimum one at default, didn't actually set it and left it to default
(TLS 1.3) too!

As a bonus, this change also removes the dead code handling of SSLv3
since that version can't be set anymore (since eff614fb02).

Reported-by: Daniel Carpenter
Fixes #6998
Closes #7000
2021-05-05 12:51:56 +02:00
Jacob Hoffman-Andrews
a3268eca79
tls: add USE_HTTP2 define
This abstracts across the two HTTP/2 backends: nghttp2 and Hyper.

Add our own define for the "h2" ALPN protocol, so TLS backends can use
it without depending on a specific HTTP backend.

Closes #6959
2021-04-29 15:04:39 +02:00
Michał Antoniak
e4ba999646
vtls: deduplicate some DISABLE_PROXY ifdefs
continue from #5735

- using SSL_HOST_NAME, SSL_HOST_DISPNAME, SSL_PINNED_PUB_KEY for other
  tls backend

- create SSL_HOST_PORT

Closes #6660
2021-04-22 22:50:56 +02:00
Daniel Stenberg
063d3f3b96
tidy-up: make conditional checks more consistent
... remove '== NULL' and '!= 0'

Closes #6912
2021-04-22 09:10:17 +02:00
Daniel Stenberg
f573998c22
TLS: fix HTTP/2 selection
for GnuTLS, BearSSL, mbedTLS, NSS, SChannnel, Secure Transport and
wolfSSL...

Regression since 88dd1a8a11 (shipped in 7.76.0)
Reported-by: Kenneth Davidson
Reported-by: romamik om github
Fixes #6825
Closes #6827
2021-04-02 22:53:17 +02:00
Daniel Stenberg
b09c8ee157
vtls: add 'isproxy' argument to Curl_ssl_get/addsessionid()
To make sure we set and extract the correct session.

Reported-by: Mingtao Yang
Bug: https://curl.se/docs/CVE-2021-22890.html

CVE-2021-22890
2021-03-28 23:19:56 +02:00
Daniel Stenberg
70472a44de
urldata: remove the _ORIG suffix from string names
It doesn't provide any useful info but only makes the names longer.

Closes #6624
2021-02-19 08:07:19 +01:00
Daniel Stenberg
e06fa7462a
gnutls: assume nettle crypto support
nettle has been the default crypto library with GnuTLS since 2010. By
dropping support for the previous libcrypto, we simplify code.

Closes #6625
2021-02-18 23:06:47 +01:00
Daniel Stenberg
88dd1a8a11
urldata: don't touch data->set.httpversion at run-time
Rename it to 'httpwant' and make a cloned field in the state struct as
well for run-time updates.

Also: refuse non-supported HTTP versions. Verified with test 129.

Closes #6585
2021-02-12 08:13:37 +01:00
Jacob Hoffman-Andrews
c30bf22f1c
vtls: factor out Curl_ssl_getsock to field of Curl_ssl
Closes #6558
2021-02-08 14:10:55 +01:00
Daniel Stenberg
2c4b338f7e
vtls: remove md5sum
As it is not used anymore.

Reported-by: Jacob Hoffman-Andrews
Bug: https://curl.se/mail/lib-2021-02/0000.html

Closes #6557
2021-02-02 08:13:09 +01:00
Patrick Monnerat
bbe3aa9f88
vtls: reduce conn->data use
Closes #6474
2021-01-19 09:14:40 +01:00
Daniel Stenberg
215db086e0
lib: pass in 'struct Curl_easy *' to most functions
... in most cases instead of 'struct connectdata *' but in some cases in
addition to.

- We mostly operate on transfers and not connections.

- We need the transfer handle to log, store data and more. Everything in
  libcurl is driven by a transfer (the CURL * in the public API).

- This work clarifies and separates the transfers from the connections
  better.

- We should avoid "conn->data". Since individual connections can be used
  by many transfers when multiplexing, making sure that conn->data
  points to the current and correct transfer at all times is difficult
  and has been notoriously error-prone over the years. The goal is to
  ultimately remove the conn->data pointer for this reason.

Closes #6425
2021-01-17 23:56:09 +01:00
Daniel Stenberg
a3e1e6b3b4
gnutls: remove Curl_ prefix from static functions 2021-01-14 08:09:08 +01:00
Daniel Stenberg
4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Harry Sintonen
8bcb23593e
gnutls: fix memory leaks (certfields memory wasn't released)
Closes #6153
2020-11-02 09:59:48 +01:00
Kamil Dudka
7920be9473
vtls: deduplicate some DISABLE_PROXY ifdefs
... in the code of gtls, nss, and openssl

Closes #5735
2020-10-01 16:36:24 +02:00
Gergely Nagy
182ff2d63c
vtls: deduplicate client certificates in ssl_config_data
Closes #5629
2020-09-14 12:56:47 +02:00
Daniel Stenberg
17fcdf6a31
lib: fix -Wassign-enum warnings
configure --enable-debug now enables -Wassign-enum with clang,
identifying several enum "abuses" also fixed.

Reported-by: Gisle Vanem
Bug: 879007f811 (commitcomment-42087553)

Closes #5929
2020-09-08 13:53:02 +02:00
Daniel Stenberg
68a5132474
TLS: fix SRP detection by using the proper #ifdefs
USE_TLS_SRP will be true if *any* selected TLS backend can use SRP

HAVE_OPENSSL_SRP is defined when OpenSSL can use it

HAVE_GNUTLS_SRP is defined when GnuTLS can use it

Clarify in the curl_verison_info docs that CURL_VERSION_TLSAUTH_SRP is
set if at least one of the supported backends offers SRP.

Reported-by: Stefan Strogin
Fixes #5865
Closes #5870
2020-08-28 14:13:05 +02:00
Daniel Stenberg
a0c461434c
gtls: survive not being able to get name/issuer
Closes #5778
2020-08-05 14:44:48 +02:00
Alex Kiernan
0fda8db95c
gnutls: repair the build with CURL_DISABLE_PROXY
`http_proxy`/`proxy_ssl`/`tunnel_proxy` will not be available in `conn`
if `CURL_DISABLE_PROXY` is enabled.  Repair the build with that
configuration.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Closes #5645
2020-07-04 00:18:43 +02:00
Alex Kiernan
691df98d51 gnutls: Fetch backend when using proxy
Fixes: 89865c149 ("gnutls: remove the BACKEND define kludge")
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2020-07-03 14:11:37 +01:00
Daniel Stenberg
842f73de58
timeouts: change millisecond timeouts to timediff_t from time_t
For millisecond timers we like timediff_t better. Also, time_t can be
unsigned so returning a negative value doesn't work then.

Closes #5479
2020-05-30 23:10:57 +02:00
Viktor Szakats
308c243db5
all: fix codespell errors
Reviewed-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Closes https://github.com/curl/curl/pull/5452
2020-05-25 19:44:04 +00:00
Emil Engler
42d8d9a7e8
GnuTLS: Backend support for CURLINFO_SSL_VERIFYRESULT
Closes #5287
2020-04-30 14:40:54 +02:00
Emil Engler
2d137dedb3
GnuTLS: Don't skip really long certificate fields
Closes #5271
2020-04-21 08:10:55 +02:00
Daniel Stenberg
0891fe7ade
gnutls: bump lowest supported version to 3.1.10
GnuTLS 3.1.10 added new functions we want to use. That version was
released on Mar 22, 2013. Removing support for older versions also
greatly simplifies the code.

Ref: #5271
Closes #5276
2020-04-21 08:09:41 +02:00
Dirkjan Bussink
d590908318
gnutls: ensure TLS 1.3 when SRP isn't requested
When SRP is requested in the priority string, GnuTLS will disable
support for TLS 1.3. Before this change, curl would always add +SRP to
the priority list, effectively always disabling TLS 1.3 support.

With this change, +SRP is only added to the priority list when SRP
authentication is also requested. This also allows updating the error
handling here to not have to retry without SRP. This is because SRP is
only added when requested and in that case a retry is not needed.

Closes #5223
2020-04-13 00:06:02 +02:00
Daniel Stenberg
89865c149d
gnutls: remove the BACKEND define kludge 2020-03-19 13:22:34 +01:00
Steve Holme
9d5893105d
gtls: Fixed compilation when using GnuTLS < 3.5.0
Reverts the functionality from 41fcb4f when compiling with GnuTLS older
than 3.5.0.

Reviewed-by: Daniel Stenberg
Closes #4984
2020-02-27 15:17:57 +00:00
Daniel Stenberg
3a34b930aa
gtls: fix the copyright year
Follow-up from 41fcb4f609
2020-02-21 15:43:45 +01:00
jethrogb
41fcb4f609
GnuTLS: Always send client cert
TLS servers may request a certificate from the client. This request
includes a list of 0 or more acceptable issuer DNs. The client may use
this list to determine which certificate to send. GnuTLS's default
behavior is to not send a client certificate if there is no
match. However, OpenSSL's default behavior is to send the configured
certificate. The `GNUTLS_FORCE_CLIENT_CERT` flag mimics OpenSSL
behavior.

Authored-by: jethrogb on github
Fixes #1411
Closes #4958
2020-02-21 15:33:36 +01:00
Daniel Stenberg
c2b01cce5c
gtls: make gnutls_bye() not wait for response on shutdown
... as it can make it wait there for a long time for no good purpose.

Patched-by: Jay Satiro
Reported-by: Bylon2 on github
Adviced-by: Nikos Mavrogiannopoulos

Fixes #4487
Closes #4541
2019-10-31 09:10:50 +01:00
Marcel Raad
527461285f
vtls: fix narrowing conversion warnings
Curl_timeleft returns `timediff_t`, which is 64 bits wide also on
32-bit systems since commit b1616dad8f.

Closes https://github.com/curl/curl/pull/4398
2019-09-23 09:44:35 +02:00
Daniel Gustafsson
8986d3c6f4 ftp: move ftp_ccc in under featureflag
Commit e91e481612 moved ftp_ccc in under
the FTP featureflag in the UserDefined struct, but vtls callsites were
still using it unprotected.

Closes #3912
Fixes: https://curl.haxx.se/dev/log.cgi?id=20190520044705-29865
Reviewed-by: Daniel Stenberg, Marcel Raad
2019-05-21 09:38:11 +02:00
Marcel Raad
10db3ef21e
lib: reduce variable scopes
Fixes Codacy/CppCheck warnings.

Closes https://github.com/curl/curl/pull/3872
2019-05-20 08:51:11 +02:00
Daniel Stenberg
8ece8177f1
cleanup: remove FIXME and TODO comments
They serve very little purpose and mostly just add noise. Most of them
have been around for a very long time. I read them all before removing
or rephrasing them.

Ref: #3876
Closes #3883
2019-05-16 09:16:56 +02:00
Daniel Stenberg
be6e281cf2
multi: provide Curl_multiuse_state to update information
As soon as a TLS backend gets ALPN conformation about the specific HTTP
version it can now set the multiplex situation for the "bundle" and
trigger moving potentially queued up transfers to the CONNECT state.
2019-05-01 22:51:23 +02:00
Daniel Stenberg
15cbf8dec6
gnutls: remove call to deprecated gnutls_compression_get_name
It has been deprecated by GnuTLS since a year ago and now causes build
warnings.

Ref: https://gitlab.com/gnutls/gnutls/commit/b0041897d2846737f5fb0f
Docs: https://www.gnutls.org/manual/html_node/Compatibility-API.html

Closes #3636
2019-03-02 11:13:57 +01:00
Daniel Stenberg
dcd6f81025
snprintf: renamed and we now only use msnprintf()
The function does not return the same value as snprintf() normally does,
so readers may be mislead into thinking the code works differently than
it actually does. A different function name makes this easier to detect.

Reported-by: Tomas Hoger
Assisted-by: Daniel Gustafsson
Fixes #3296
Closes #3297
2018-11-23 08:26:51 +01:00
Han Han
78ff4e0de3
ssl: replace all internal uses of CURLE_SSL_CACERT
Closes #3291
2018-11-20 14:57:00 +01:00
Daniel Stenberg
ebfe02f73c
gtls: Values stored to but never read
Detected by clang-tidy

Closes #3176
2018-10-26 13:51:07 +02:00
Loganaden Velvindron
9bdadbbdee
GnutTLS: TLS 1.3 support
Closes #2971
2018-09-21 09:13:33 +02:00
Jay Satiro
2e5651a5ce vtls: fix ssl version "or later" behavior change for many backends
- Treat CURL_SSLVERSION_MAX_NONE the same as
  CURL_SSLVERSION_MAX_DEFAULT. Prior to this change NONE would mean use
  the minimum version also as the maximum.

This is a follow-up to 6015cef which changed the behavior of setting
the SSL version so that the requested version would only be the minimum
and not the maximum. It appears it was (mostly) implemented in OpenSSL
but not other backends. In other words CURL_SSLVERSION_TLSv1_0 used to
mean use just TLS v1.0 and now it means use TLS v1.0 *or later*.

- Fix CURL_SSLVERSION_MAX_DEFAULT for OpenSSL.

Prior to this change CURL_SSLVERSION_MAX_DEFAULT with OpenSSL was
erroneously treated as always TLS 1.3, and would cause an error if
OpenSSL was built without TLS 1.3 support.

Co-authored-by: Daniel Gustafsson

Fixes https://github.com/curl/curl/issues/2969
Closes https://github.com/curl/curl/pull/3012
2018-09-20 14:12:25 -04:00
Daniel Stenberg
e66cca046c
vtls: use unified "supports" bitfield member in backends
... instead of previous separate struct fields, to make it easier to
extend and change individual backends without having to modify them all.

closes #2547
2018-05-04 22:31:19 +02:00
Daniel Stenberg
dc1b6c5a00
build: cleanup to fix clang warnings/errors
unit1309 and vtls/gtls: error: arithmetic on a null pointer treated as a
cast from integer to pointer is a GNU extension

Reported-by: Rikard Falkeborn

Fixes #2466
Closes #2468
2018-04-08 13:59:30 +02:00
Wyatt O'Day
336b6a32c0
tls: fix mbedTLS 2.7.0 build + handle sha256 failures
(mbedtls 2.70 compiled with MBEDTLS_DEPRECATED_REMOVED)

Closes #2453
2018-04-06 14:21:50 +02:00
Marcel Raad
24753bcd48
unit1309: fix warning on Windows x64
When targeting x64, MinGW-w64 complains about conversions between
32-bit long and 64-bit pointers. Fix this by reusing the
GNUTLS_POINTER_TO_SOCKET_CAST / GNUTLS_SOCKET_TO_POINTER_CAST logic
from gtls.c, moving it to warnless.h as CURLX_POINTER_TO_INTEGER_CAST /
CURLX_INTEGER_TO_POINTER_CAST.

Closes https://github.com/curl/curl/pull/2341
2018-02-28 20:04:48 +01:00
Patrick Monnerat
8aee8a6a2d vtls: change struct Curl_ssl close' field name to close_one'.
On OS/400, `close' is an ASCII system macro that corrupts the code if
not used in a context not targetting the close() system API.
2017-10-19 19:55:17 +01:00
Daniel Stenberg
6b84438d9a
code style: use spaces around equals signs 2017-09-11 09:29:50 +02:00
Johannes Schindelin
b59288f881
vtls: refactor out essential information about the SSL backends
There is information about the compiled-in SSL backends that is really
no concern of any code other than the SSL backend itself, such as which
function (if any) implements SHA-256 summing.

And there is information that is really interesting to the user, such as
the name, or the curl_sslbackend value.

Let's factor out the latter into a publicly visible struct. This
information will be used in the upcoming API to set the SSL backend
globally.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:58 +02:00
Johannes Schindelin
b0989cd3ab
vtls: allow selecting which SSL backend to use at runtime
When building software for the masses, it is sometimes not possible to
decide for all users which SSL backend is appropriate.

Git for Windows, for example,  uses cURL to perform clones, fetches and
pushes via HTTPS, and some users strongly prefer OpenSSL, while other
users really need to use Secure Channel because it offers
enterprise-ready tools to manage credentials via Windows' Credential
Store.

The current Git for Windows versions use the ugly work-around of
building libcurl once with OpenSSL support and once with Secure Channel
support, and switching out the binaries in the installer depending on
the user's choice.

Needless to say, this is a super ugly workaround that actually only
works in some cases: Git for Windows also comes in a portable form, and
in a form intended for third-party applications requiring Git
functionality, in which cases this "swap out libcurl-4.dll" simply is
not an option.

Therefore, the Git for Windows project has a vested interest in teaching
cURL to make the SSL backend a *runtime* option.

This patch makes that possible.

By running ./configure with multiple --with-<backend> options, cURL will
be built with multiple backends.

For the moment, the backend can be configured using the environment
variable CURL_SSL_BACKEND (valid values are e.g. "openssl" and
"schannel").

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:58 +02:00
Johannes Schindelin
a53bda35e9
vtls: fold the backend ID into the Curl_ssl structure
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:58 +02:00
Johannes Schindelin
70f1db321a
vtls: encapsulate SSL backend-specific data
So far, all of the SSL backends' private data has been declared as
part of the ssl_connect_data struct, in one big #if .. #elif .. #endif
block.

This can only work as long as the SSL backend is a compile-time option,
something we want to change in the next commits.

Therefore, let's encapsulate the exact data needed by each SSL backend
into a private struct, and let's avoid bleeding any SSL backend-specific
information into urldata.h. This is also necessary to allow multiple SSL
backends to be compiled in at the same time, as e.g. OpenSSL's and
CyaSSL's headers cannot be included in the same .c file.

To avoid too many malloc() calls, we simply append the private structs
to the connectdata struct in allocate_conn().

This requires us to take extra care of alignment issues: struct fields
often need to be aligned on certain boundaries e.g. 32-bit values need to
be stored at addresses that divide evenly by 4 (= 32 bit / 8
bit-per-byte).

We do that by assuming that no SSL backend's private data contains any
fields that need to be aligned on boundaries larger than `long long`
(typically 64-bit) would need. Under this assumption, we simply add a
dummy field of type `long long` to the `struct connectdata` struct. This
field will never be accessed but acts as a placeholder for the four
instances of ssl_backend_data instead. the size of each ssl_backend_data
struct is stored in the SSL backend-specific metadata, to allow
allocate_conn() to know how much extra space to allocate, and how to
initialize the ssl[sockindex]->backend and proxy_ssl[sockindex]->backend
pointers.

This would appear to be a little complicated at first, but is really
necessary to encapsulate the private data of each SSL backend correctly.
And we need to encapsulate thusly if we ever want to allow selecting
CyaSSL and OpenSSL at runtime, as their headers cannot be included within
the same .c file (there are just too many conflicting definitions and
declarations for that).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:58 +02:00
Johannes Schindelin
d65e6cc4fc
vtls: prepare the SSL backends for encapsulated private data
At the moment, cURL's SSL backend needs to be configured at build time.
As such, it is totally okay for them to hard-code their backend-specific
data in the ssl_connect_data struct.

In preparation for making the SSL backend a runtime option, let's make
the access of said private data a bit more abstract so that it can be
adjusted later in an easy manner.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:58 +02:00
Johannes Schindelin
69039fd1fa
getinfo: access SSL internals via Curl_ssl
In the ongoing endeavor to abstract out all SSL backend-specific
functionality, this is the next step: Instead of hard-coding how the
different SSL backends access their internal data in getinfo.c, let's
implement backend-specific functions to do that task.

This will also allow for switching SSL backends as a runtime option.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:57 +02:00
Johannes Schindelin
6f1eec14e0
vtls: remove obsolete declarations of SSL backend functionality
These functions are all available via the Curl_ssl struct now, no need
to declare them separately anymore.

As the global declarations are removed, the corresponding function
definitions are marked as file-local. The only two exceptions here are
Curl_mbedtls_shutdown() and Curl_polarssl_shutdown(): only the
declarations were removed, there are no function definitions to mark
file-local.

Please note that Curl_nss_force_init() is *still* declared globally, as
the only SSL backend-specific function, because it was introduced
specifically for the use case where cURL was compiled with
`--without-ssl --with-nss`. For details, see f3b77e561 (http_ntlm: add
support for NSS, 2010-06-27).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:57 +02:00
Johannes Schindelin
f0b4db1ab0
vtls: move the SUPPORT_HTTPS_PROXY flag into the Curl_ssl struct
That will allow us to choose the SSL backend at runtime.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:56 +02:00
Johannes Schindelin
937899a3b8
vtls: convert the have_curlssl_* constants to runtime flags
The entire idea of introducing the Curl_ssl struct to describe SSL
backends is to prepare for choosing the SSL backend at runtime.

To that end, convert all the #ifdef have_curlssl_* style conditionals
to use bit flags instead.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:56 +02:00
Johannes Schindelin
0a083a66bc
vtls: move sha256sum into the Curl_ssl struct
The SHA-256 checksumming is also an SSL backend-specific function.
Let's include it in the struct declaring the functionality of SSL
backends.

In contrast to MD5, there is no fall-back code. To indicate this, the
respective entries are NULL for those backends that offer no support for
SHA-256 checksumming.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:56 +02:00
Johannes Schindelin
e35205a0c4
vtls: move md5sum into the Curl_ssl struct
The MD5 summing is also an SSL backend-specific function. So let's
include it, offering the previous fall-back code as a separate function
now: Curl_none_md5sum(). To allow for that, the signature had to be
changed so that an error could be returned from the implementation
(Curl_none_md5sum() can run out of memory).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:56 +02:00
Johannes Schindelin
52e8237bfc
vtls: use the Curl_ssl struct to access all SSL backends' functionality
This is the first step to unify the SSL backend handling. Now all the
SSL backend-specific functionality is accessed via a global instance of
the Curl_ssl struct.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:56 +02:00
Johannes Schindelin
e09bb63ed8
vtls: declare Curl_ssl structs for every SSL backend
The idea of introducing the Curl_ssl struct was to unify how the SSL
backends are declared and called. To this end, we now provide an
instance of the Curl_ssl struct for each and every SSL backend.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:56 +02:00
Johannes Schindelin
b31d1dcddb
vtls: make sure all _cleanup() functions return void
This patch makes the signature of the _cleanup() functions consistent
among the SSL backends, in preparation for unifying the way all SSL
backends are accessed.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:55 +02:00