Commit Graph

115 Commits

Author SHA1 Message Date
Viktor Szakats
71d3ab5813
vtls: fix static function name collisions between TLS backends
When using CMake Unity build.

- use unique name for `set_ssl_version_min_max()`
  Fixes collision between GnuTLS, mbedTLS and SecureTransport.
  ```
  lib\vtls\mbedtls.c(317,1): error C2084: function 'CURLcode set_ssl_version_min_max(Curl_easy *,ssl_peer *,ssl_primary_config *,const char **,const char *)' already has a body
  lib\vtls\mbedtls.c(837,49): warning C4133: 'function': incompatible types - from 'Curl_cfilter *' to 'Curl_easy *'
  lib\vtls\mbedtls.c(837,53): warning C4133: 'function': incompatible types - from 'Curl_easy *' to 'ssl_peer *'
  lib\vtls\mbedtls.c(837,25): error C2198: 'set_ssl_version_min_max': too few arguments for call
  ```

- use unique name for `do_file_type()`
  Fixes collision between GnuTLS, OpenSSL and wolfSSL.
  ```
  lib\vtls\openssl.c(1053,12): error C2084: function 'gnutls_x509_crt_fmt_t do_file_type(const char *)' already has a body
  ```

Ref: https://github.com/curl/curl/actions/runs/10341162641/job/28622681573?pr=14484#step:10:31
Cherry-picked from #14495
Closes #14516
2024-08-13 09:28:27 +02:00
Max Faxälv
0a5ea09a91
spnego_gssapi: implement TLS channel bindings for openssl
Channel Bindings are used to tie the session context to a specific TLS
channel. This is to provide additional proof of valid identity,
mitigating authentication relay attacks.

Major web servers have the ability to require (None/Accept/Require)
GSSAPI channel binding, rendering Curl unable to connect to such
websites unless support for channel bindings is implemented.

IIS calls this feature Extended Protection (EPA), which is used in
Enterprise environments using Kerberos for authentication.

This change require krb5 >= 1.19, otherwise channel bindings won't be
forwarded through SPNEGO.

Co-Authored-By: Steffen Kieß <947515+steffen-kiess@users.noreply.github.com>
Closes #13098
2024-08-12 19:16:54 +02:00
Jan Venekamp
5c2ab55abe
vtls: add SSLSUPP_CIPHER_LIST
Added SSLSUPP_CIPHER_LIST so be able to differniate SSL Backends
that support CURLOPT_SSL_CIPHER_LIST.

Closes #14406
2024-08-07 08:50:42 +02:00
Viktor Szakats
f81f351b9a
tidy-up: OS names
Use these words and casing more consistently across text, comments and
one curl tool output:
AIX, ALPN, ANSI, BSD, Cygwin, Darwin, FreeBSD, GitHub, HP-UX, Linux,
macOS, MS-DOS, MSYS, MinGW, NTLM, POSIX, Solaris, UNIX, Unix, Unicode,
WINE, WebDAV, Win32, winbind, WinIDN, Windows, Windows CE, Winsock.

Mostly OS names and a few more.

Also a couple of other minor text fixups.

Closes #14360
2024-08-04 19:17:45 +02:00
Stefan Eissing
911c3166b6
lib: add eos flag to send methods
Adds a `bool eos` flag to send methods to indicate that the data
is the last chunk the invovled transfer wants to send to the server.

This will help protocol filters like HTTP/2 and 3 to forward the
stream's EOF flag and also allow to EAGAIN such calls when buffers
are not yet fully flushed.

Closes #14220
2024-08-03 19:53:54 +02:00
Daniel Stenberg
25321de30e
Revert "lib: send eos flag"
This reverts commit be93299f10.
2024-07-19 01:38:05 +02:00
Stefan Eissing
be93299f10
lib: send eos flag
Adds a `bool eos` flag to send methods to indicate that the data is the
last chunk the invovled transfer wants to send to the server.

This will help protocol filters like HTTP/2 and 3 to forward the
stream's EOF flag and also allow to EAGAIN such calls when buffers are
not yet fully flushed.

Closes #14220
2024-07-18 23:27:35 +02:00
Viktor Szakats
5fa534b0da
sectransp: fix clang compiler warnings, stop silencing them
Fix `-Wpointer-bool-conversion` warnings with the method suggested by
both Apple clang and mainline llvm. This was already tried and dropped
in #1705 (in year 2017), but the issue reported there no longer
replicates.

Verified with Apple clang 14, llvm 15, llvm 18 and gcc 11, 14 that the
generated objects are bit by bit identical before and after this patch.

Also:
- stop silencing `-Wtautological-pointer-compare`. This warning don't
  seem to be appearing anymore (with or without this patch), at least
  with the tested compilers and SDKs (clang 13.1.6-16.0.0beta, llvm 15,
  18, gcc 11, 14) and minimum macOS target of 10.8. Older targets fail
  to build curl with SecureTransport.

- silence `-Wunreachable-code` for clang only. Previously I applied it
  also to GCC, by mistake.
  Ref: 8d7172d20a

Apple clang `-Wpointer-bool-conversion`:
```
curl/lib/vtls/sectransp.c:1103:6: error: address of function 'SSLCreateContext' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
  if(SSLCreateContext) {  /* use the newer API if available */
  ~~ ^~~~~~~~~~~~~~~~
curl/lib/vtls/sectransp.c:1103:6: note: prefix with the address-of operator to silence this warning
  if(SSLCreateContext) {  /* use the newer API if available */
     ^
     &
```
Ref: https://github.com/curl/curl/actions/runs/9819538439/job/27113201384#step:8:382

llvm `-Wpointer-bool-conversion`:
```
curl/lib/vtls/sectransp.c:2663:8: error: address of function 'SSLCreateContext' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
    if(SSLCreateContext)
    ~~ ^~~~~~~~~~~~~~~~
curl/lib/vtls/sectransp.c:2663:8: note: prefix with the address-of operator to silence this warning
    if(SSLCreateContext)
       ^
       &
```
Ref: https://github.com/curl/curl/actions/runs/9819538439/job/27113200291#step:8:417

gcc still needs `-Waddress` suppressed to avoid these:
```
curl/lib/vtls/n/sectransp.c: In function 'getsubject':
curl/lib/vtls/n/sectransp.c:379:6: warning: the address of 'SecCertificateCopyLongDescription' will always evaluate as 'true' [-Waddress]
  379 |   if(&SecCertificateCopyLongDescription)
      |      ^
[...]
```

Follow-up to 59cadacfcc #14128
Follow-up to af271ce9b9 #1722
Follow-up to 2b7ce3f56d #1706
Cherry-picked from #14097
Closes #14162
2024-07-13 12:00:13 +02:00
Viktor Szakats
59cadacfcc
build: sync warning options between autotools, cmake & compilers
- cmake: enable Apple-specific `-Werror=partial-availability` to match
  autotools.

- autotools: enable `-pedantic-errors` with llvm/clang to match gcc and
  CMake.

- autotools: enable `-Werror-implicit-function-declaration` for
  llvm/clang to match gcc.

- cmake: enable `-Werror-implicit-function-declaration` to match
  autotools.

- move `-Wpointer-bool-conversion` from autotools to the local file
  (`sectransp.c`) it was meant to apply. This way it applies to all
  build methods.

- autotoos: show `CURL_CFLAG_EXTRAS` in the `./configure` summary.
  (it may contain `-Werror` and/or `-pedentic-errors`.)

Cherry-picked from #14097
Closes #14128
2024-07-10 11:30:40 +02:00
Stefan Eissing
46a26f122a
vtls: replace addsessionid with set_sessionid
- deduplicate the code in many tls backends that check
  for an existing id and delete it before adding the new one
- rename ssl_primary_config's `sessionid` bool to `cache_session`

Closes #14121
2024-07-09 23:14:58 +02:00
Viktor Szakats
b05dc7eb35
sectransp: fix HAVE_BUILTIN_AVAILABLE checks to not emit warnings
`HAVE_BUILTIN_AVAILABLE` is a curl macro set via autotools and cmake.
Like other `HAVE_`s it signals availability if defined.

SecureTransport code was specifically looking for the value 1, which
triggered compiler warnings when the feature was not present.

Replace the existing workaround of locally suppressing the compiler
warning with using `defined()`.

autotools:
```
767 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~
../../lib/vtls/sectransp.c: In function 'sectransp_connect_step1':
../../lib/vtls/sectransp.c:1140:52: error: "HAVE_BUILTIN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef]
 1140 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~
../../lib/vtls/sectransp.c:1240:52: error: "HAVE_BUILTIN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef]
 1240 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~
../../lib/vtls/sectransp.c: In function 'sectransp_connect_step2':
```
Ref: https://github.com/curl/curl/actions/runs/9815428701/job/27104448045#step:6:499

cmake gcc:
```
 1140 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~
/Users/runner/work/curl/curl/lib/vtls/sectransp.c:1240:52: error: "HAVE_BUILTIN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef]
 1240 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~
/Users/runner/work/curl/curl/lib/vtls/sectransp.c: In function 'sectransp_connect_step2':
/Users/runner/work/curl/curl/lib/vtls/sectransp.c:2231:51: error: "HAVE_BUILTIN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef]
 2231 | #if(CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILABLE == 1
      |                                                   ^~~~~~~~~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9815428701/job/27104445425#step:8:355

Cherry-picked from #14097
Closes #14122
2024-07-08 17:22:21 +02:00
Daniel Stenberg
c074ba64a8
code: language cleanup in comments
Based on the standards and guidelines we use for our documentation.

 - expand contractions (they're => they are etc)
 - host name = > hostname
 - file name => filename
 - user name = username
 - man page => manpage
 - run-time => runtime
 - set-up => setup
 - back-end => backend
 - a HTTP => an HTTP
 - Two spaces after a period => one space after period

Closes #14073
2024-07-01 22:58:55 +02:00
Stefan Eissing
c9b95c0bb3
lib: graceful connection shutdown
When libcurl discards a connection there are two phases this may go
through: "shutdown" and "closing". If a connection is aborted, the
shutdown phase is skipped and it is closed right away.

The connection filters attached to the connection implement the phases
in their `do_shutdown()` and `do_close()` callbacks. Filters carry now a
`shutdown` flags next to `connected` to keep track of the shutdown
operation.

Filters are shut down from top to bottom. If a filter is not connected,
its shutdown is skipped. Notable filters that *do* something during
shutdown are HTTP/2 and TLS. HTTP/2 sends the GOAWAY frame. TLS sends
its close notify and expects to receive a close notify from the server.

As sends and receives may EAGAIN on the network, a shutdown is often not
successful right away and needs to poll the connection's socket(s). To
facilitate this, such connections are placed on a new shutdown list
inside the connection cache.

Since managing this list requires the cooperation of a multi handle,
only the connection cache belonging to a multi handle is used. If a
connection was in another cache when being discarded, it is removed
there and added to the multi's cache. If no multi handle is available at
that time, the connection is shutdown and closed in a one-time,
best-effort attempt.

When a multi handle is destroyed, all connection still on the shutdown
list are discarded with a final shutdown attempt and close. In curl
debug builds, the environment variable `CURL_GRACEFUL_SHUTDOWN` can be
set to make this graceful with a timeout in milliseconds given by the
variable.

The shutdown list is limited to the max number of connections configured
for a multi cache. Set via CURLMOPT_MAX_TOTAL_CONNECTIONS. When the
limit is reached, the oldest connection on the shutdown list is
discarded.

- In multi_wait() and multi_waitfds(), collect all connection caches
  involved (each transfer might carry its own) into a temporary list.
  Let each connection cache on the list contribute sockets and
  POLLIN/OUT events it's connections are waiting for.

- in multi_perform() collect the connection caches the same way and let
  them peform their maintenance. This will make another non-blocking
  attempt to shutdown all connections on its shutdown list.

- for event based multis (multi->socket_cb set), add the sockets and
  their poll events via the callback. When `multi_socket()` is invoked
  for a socket not known by an active transfer, forward this to the
  multi's cache for processing. On closing a connection, remove its
  socket(s) via the callback.

TLS connection filters MUST NOT send close nofity messages in their
`do_close()` implementation. The reason is that a TLS close notify
signals a success. When a connection is aborted and skips its shutdown
phase, the server needs to see a missing close notify to detect
something has gone wrong.

A graceful shutdown of FTP's data connection is performed implicitly
before regarding the upload/download as complete and continuing on the
control connection. For FTP without TLS, there is just the socket close
happening. But with TLS, the sent/received close notify signals that the
transfer is complete and healthy. Servers like `vsftpd` verify that and
reject uploads without a TLS close notify.

- added test_19_* for shutdown related tests
- test_19_01 and test_19_02 test for TCP RST packets
  which happen without a graceful shutdown and should
  no longer appear otherwise.
- add test_19_03 for handling shutdowns by the server
- add test_19_04 for handling shutdowns by curl
- add test_19_05 for event based shutdowny by server
- add test_30_06/07 and test_31_06/07 for shutdown checks
  on FTP up- and downloads.

Closes #13976
2024-06-26 08:33:17 +02:00
Stefan Eissing
c31041b17e
connection: shutdown TLS (for FTP) better
This adds connection shutdown infrastructure and first use for FTP. FTP
data connections, when not encountering an error, are now shut down in a
blocking way with a 2sec timeout.

    - add cfilter `Curl_cft_shutdown` callback
    - keep a shutdown start timestamp and timeout at connectdata
    - provide shutdown timeout default and member in
      `data->set.shutdowntimeout`.
    - provide methods for starting, interrogating and clearing
      shutdown timers
    - provide `Curl_conn_shutdown_blocking()` to shutdown the
      `sockindex` filter chain in a blocking way. Use that in FTP.
    - add `Curl_conn_cf_poll()` to wait for socket events during
      shutdown of a connection filter chain.
      This gets the monitoring sockets and events via the filters
      "adjust_pollset()" methods. This gives correct behaviour when
      shutting down a TLS connection through a HTTP/2 proxy.
    - Implement shutdown for all socket filters
      - for HTTP/2 and h2 proxying to send GOAWAY
      - for TLS backends to the best of their capabilities
      - for tcp socket filter to make a final, nonblocking
        receive to avoid unwanted RST states
    - add shutdown forwarding to happy eyeballers and
      https connect ballers when applicable.

Closes #13904
2024-06-10 13:08:12 +02:00
Stefan Eissing
937ba94ed5
vtls: new io_need flags for poll handling
- decouple need to recv/send from negotiation state, we need
  this later in shutdown handling as well
- move ssl enums from urldata.h to vtls_int.h
- implement use of `connssl->io_need` in vtls.c. and all backends

Closes #13879
2024-06-05 09:03:38 +02:00
Viktor Szakats
0887297100
lib/v*: tidy up types and casts
Also add a couple of negative checks.

Cherry-picked from #13489
Closes #13622
2024-06-02 19:27:17 +02:00
Jan Venekamp
4e2c45110c
sectransp: remove large cipher table
Previously a large table of ciphers was used to determine the default
ciphers and to lookup manually selected ciphers names.

With the lookup of the manually selected cipher names moved to
Curl_cipher_suite_walk_str() the large table is no longer needed for
that purpose.

The list of manually selected cipher can now be intersected with the
ciphers supported by Secure Transport (SSLGetSupportedCiphers()),
instead of using the fixed table for that.

The other use of the table was to filter the list of all supported
ciphers offered by Secure Transport to create a list of ciphers to
use by default, excluding ciphers in the table marked as weak.

Instead of using a complement based approach (exclude weak), switch
to using an intersection with a smaller list of ciphers deemed
appropriate.

Closes #13823
2024-06-01 11:00:06 +02:00
Jan Venekamp
2b52fe4115
sectransp: use common code for cipher suite lookup
Take advantage of the Curl_cipher_suite_walk_str() and
Curl_cipher_suite_get_str() functions introduced in commit fba9afe.

Closes #13521
2024-05-29 13:08:14 +02:00
Stefan Eissing
e101a7a8b0
multi: add multi->proto_hash, a key-value store for protocol data
- add `Curl_hash_add2()` that passes a destructor function for
  the element added. Call element destructor instead of hash
  destructor if present.
- multi: add `proto_hash` for protocol related information,
  remove `struct multi_ssl_backend_data`.
- openssl: use multi->proto_hash to keep x509 shared store
- schannel: use multi->proto_hash to keep x509 shared store
- vtls: remove Curl_free_multi_ssl_backend_data() and its
  equivalents in the TLS backends

Closes #13345
2024-05-26 00:15:01 +02:00
Stefan Eissing
fb22459dc1
vtls: TLS session storage overhaul
- add session with destructor callback
- remove vtls `session_free` method
- let `Curl_ssl_addsessionid()` take ownership
  of session object, freeing it also on failures
- change tls backend use
- test_17, add tests for SSL session resumption

Closes #13386
2024-04-26 13:58:36 +02:00
Viktor Szakats
f33ee2e73b
tls: fix SecureTransport + BearSSL cmake unity builds
Avoid clashing static function names by namespacing them.

Pointed-out-by: Jan Venekamp
Ref: https://github.com/curl/curl/pull/13442#discussion_r1576350700
Closes #13450
2024-04-24 10:08:24 +02:00
Stefan Eissing
3210101088
tls: use shared init code for TCP+QUIC
Closes #13172
2024-04-09 09:08:05 +02:00
Stefan Eissing
e87751d69a vtls: fix tls proxy peer verification
- When verifying a proxy certificate for an ip address, use the correct
  ip family.

Prior to this change the "connection" ip family was used, which was not
necessarily the same.

Reported-by: HsiehYuho@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/12831
Closes https://github.com/curl/curl/pull/12931
2024-02-16 18:00:21 -05:00
Daniel Stenberg
dd0f680fc0
sectransp: do verify_cert without memdup for blobs
Since the information is then already stored in memory, this can avoid
an extra set of malloc + free calls.

Closes #12679
2024-01-10 23:22:52 +01:00
Daniel Stenberg
7c992dd9f8
lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding
Since the copy does not stop at a null byte, let's not call it anything
that makes you think it works like the common strndup() function.

Based on feedback from Jay Satiro, Stefan Eissing and Patrick Monnerat

Closes #12490
2023-12-08 17:22:33 +01:00
Daniel Stenberg
7309b9cbbf
lib: strndup/memdup instead of malloc, memcpy and null-terminate
- bufref: use strndup
 - cookie: use strndup
 - formdata: use strndup
 - ftp: use strndup
 - gtls: use aprintf instead of malloc + strcpy * 2
 - http: use strndup
 - mbedtls: use strndup
 - md4: use memdup
 - ntlm: use memdup
 - ntlm_sspi: use strndup
 - pingpong: use memdup
 - rtsp: use strndup instead of malloc, memcpy and null-terminate
 - sectransp: use strndup
 - socks_gssapi.c: use memdup
 - vtls: use dynbuf instead of malloc, snprintf and memcpy
 - vtls: use strdup instead of malloc + memcpy
 - wolfssh: use strndup

Closes #12453
2023-12-07 08:47:44 +01:00
Stefan Eissing
8b99e6f23f
sectransp_ make TLSCipherNameForNumber() available in non-verbose config
Reported-by: Cajus Pollmeier
Closes #12476
Fixes #12474
2023-12-06 23:03:35 +01:00
Viktor Szakats
84338c4de2
build: add more picky warnings and fix them
Enable more picky compiler warnings. I've found these options in the
nghttp3 project when implementing the CMake quick picky warning
functionality for it [1].

`-Wunused-macros` was too noisy to keep around, but fixed a few issues
it revealed while testing.

- autotools: reflect the more precisely-versioned clang warnings.
  Follow-up to 033f8e2a08 #12324
- autotools: sync between clang and gcc the way we set `no-multichar`.
- autotools: avoid setting `-Wstrict-aliasing=3` twice.
- autotools: disable `-Wmissing-noreturn` for MSYS gcc targets [2].
  It triggers in libtool-generated stub code.

- lib/timeval: delete a redundant `!MSDOS` guard from a `WIN32` branch.

- lib/curl_setup.h: delete duplicate declaration for `fileno`.
  Added in initial commit ae1912cb0d
  (1999-12-29). This suggests this may not be needed anymore, but if
  it does, we may restore this for those specific (non-Windows) systems.
- lib: delete unused macro `FTP_BUFFER_ALLOCSIZE` since
  c1d6fe2aaa.
- lib: delete unused macro `isxdigit_ascii` since
  f65f750742.
- lib/mqtt: delete unused macro `MQTT_HEADER_LEN`.
- lib/multi: delete unused macro `SH_READ`/`SH_WRITE`.
- lib/hostip: add `noreturn` function attribute via new `CURL_NORETURN`
  macro.
- lib/mprintf: delete duplicate declaration for `Curl_dyn_vprintf`.
- lib/rand: fix `-Wunreachable-code` and related fallouts [3].
- lib/setopt: fix `-Wunreachable-code-break`.
- lib/system_win32 and lib/timeval: fix double declarations for
  `Curl_freq` and `Curl_isVistaOrGreater` in CMake UNITY mode [4].
- lib/warnless: fix double declarations in CMake UNITY mode [5].
  This was due to force-disabling the header guard of `warnless.h` to
  to reapply it to source code coming after `warnless.c` in UNITY
  builds. This reapplied declarations too, causing the warnings.
  Solved by adding a header guard for the lines that actually need
  to be reapplied.
- lib/vauth/digest: fix `-Wunreachable-code-break` [6].
- lib/vssh/libssh2: fix `-Wunreachable-code-break` and delete redundant
  block.
- lib/vtls/sectransp: fix `-Wunreachable-code-break` [7].
- lib/vtls/sectransp: suppress `-Wunreachable-code`.
  Detected in `else` branches of dynamic feature checks, with results
  known at compile-time, e.g.
  ```c
  if(SecCertificateCopySubjectSummary)  /* -> true */
  ```
  Likely fixable as a separate micro-project, but given SecureTransport
  is deprecated anyway, let's just silence these locally.
- src/tool_help: delete duplicate declaration for `helptext`.
- src/tool_xattr: fix `-Wunreachable-code`.
- tests: delete duplicate declaration for `unitfail` [8].
- tests: delete duplicate declaration for `strncasecompare`.
- tests/libtest: delete duplicate declaration for `gethostname`.
  Originally added in 687df5c8c3
  (2010-08-02).
  Got complicated later: c49e9683b8
  If there are still systems around with warnings, we may restore the
  prototype, but limited for those systems.
- tests/lib2305: delete duplicate declaration for
  `libtest_debug_config`.
- tests/h2-download: fix `-Wunreachable-code-break`.

[1] a70edb08e9/cmake/PickyWarningsC.cmake
[2] https://ci.appveyor.com/project/curlorg/curl/builds/48553586/job/3qkgjauiqla5fj45?fullLog=true#L1675
[3] https://github.com/curl/curl/actions/runs/6880886309/job/18716044703?pr=12331#step:7:72
    https://github.com/curl/curl/actions/runs/6883016087/job/18722707368?pr=12331#step:7:109
[4] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrriklpf1ut#L204
[5] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrriklpf1ut#L218
[6] https://github.com/curl/curl/actions/runs/6880886309/job/18716042927?pr=12331#step:7:290
[7] https://github.com/curl/curl/actions/runs/6891484996/job/18746659406?pr=12331#step:9:1193
[8] https://github.com/curl/curl/actions/runs/6882803986/job/18722082562?pr=12331#step:33:1870

Closes #12331
2023-11-21 16:35:42 +00:00
Stefan Eissing
fa714830e9
vtls/vquic, keep peer name information together
- add `struct ssl_peer` to keep hostname, dispname and sni
  for a filter
- allocate `sni` for use in VTLS backend
- eliminate `Curl_ssl_snihost()` and its use of the download buffer
- use ssl_peer in SSL and QUIC filters

Closes #12349
2023-11-19 13:55:22 +01:00
Viktor Szakats
a9fd0d0083
build: always revert #pragma GCC diagnostic after use
Before this patch some source files were overriding gcc warning options,
but without restoring them at the end of the file. In CMake UNITY builds
these options spilled over to the remainder of the source code,
effecitvely disabling them for a larger portion of the codebase than
intended.

`#pragma clang diagnostic` didn't have such issue in the codebase.

Reviewed-by: Marcel Raad
Closes #12352
2023-11-18 11:16:32 +00:00
Sam James
bc8509a748
misc: fix -Walloc-size warnings
GCC 14 introduces a new -Walloc-size included in -Wextra which gives:

```
src/tool_operate.c: In function ‘add_per_transfer’:
src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ for type ‘struct per_transfer’ with size ‘480’ [-Walloc-size]
  213 |   p = calloc(sizeof(struct per_transfer), 1);
      |     ^
src/var.c: In function ‘addvariable’:
src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type ‘struct var’ with size ‘32’ [-Walloc-size]
  361 |   p = calloc(sizeof(struct var), 1);
      |     ^
```

The calloc prototype is:
```
void *calloc(size_t nmemb, size_t size);
    ```

So, just swap the number of members and size arguments to match the
prototype, as we're initialising 1 struct of size `sizeof(struct
...)`. GCC then sees we're not doing anything wrong.

Closes #12292
2023-11-11 23:35:47 +01:00
Stefan Eissing
47f5b1a37f
lib: introduce struct easy_poll_set for poll information
Connection filter had a `get_select_socks()` method, inspired by the
various `getsocks` functions involved during the lifetime of a
transfer. These, depending on transfer state (CONNECT/DO/DONE/ etc.),
return sockets to monitor and flag if this shall be done for POLLIN
and/or POLLOUT.

Due to this design, sockets and flags could only be added, not
removed. This led to problems in filters like HTTP/2 where flow control
prohibits the sending of data until the peer increases the flow
window. The general transfer loop wants to write, adds POLLOUT, the
socket is writeable but no data can be written.

This leads to cpu busy loops. To prevent that, HTTP/2 did set the
`SEND_HOLD` flag of such a blocked transfer, so the transfer loop cedes
further attempts. This works if only one such filter is involved. If a
HTTP/2 transfer goes through a HTTP/2 proxy, two filters are
setting/clearing this flag and may step on each other's toes.

Connection filters `get_select_socks()` is replaced by
`adjust_pollset()`. They get passed a `struct easy_pollset` that keeps
up to `MAX_SOCKSPEREASYHANDLE` sockets and their `POLLIN|POLLOUT`
flags. This struct is initialized in `multi_getsock()` by calling the
various `getsocks()` implementations based on transfer state, as before.

After protocol handlers/transfer loop have set the sockets and flags
they want, the `easy_pollset` is *always* passed to the filters. Filters
"higher" in the chain are called first, starting at the first
not-yet-connection one. Each filter may add sockets and/or change
flags. When all flags are removed, the socket itself is removed from the
pollset.

Example:

 * transfer wants to send, adds POLLOUT
 * http/2 filter has a flow control block, removes POLLOUT and adds
   POLLIN (it is waiting on a WINDOW_UPDATE from the server)
 * TLS filter is connected and changes nothing
 * h2-proxy filter also has a flow control block on its tunnel stream,
   removes POLLOUT and adds POLLIN also.
 * socket filter is connected and changes nothing
 * The resulting pollset is then mixed together with all other transfers
   and their pollsets, just as before.

Use of `SEND_HOLD` is no longer necessary in the filters.

All filters are adapted for the changed method. The handling in
`multi.c` has been adjusted, but its state handling the the protocol
handlers' `getsocks` method are untouched.

The most affected filters are http/2, ngtcp2, quiche and h2-proxy. TLS
filters needed to be adjusted for the connecting handshake read/write
handling.

No noticeable difference in performance was detected in local scorecard
runs.

Closes #11833
2023-10-25 09:34:32 +02:00
Viktor Szakats
fd328fcaf1
cmake: fix unity builds for more build combinations
By using unique static function/variable names in source files
implementing these interfaces.

- OpenLDAP combined with any SSH backend.

- MultiSSL with mbedTLS, OpenSSL, wolfSSL, SecureTransport.

Closes #12027
2023-10-04 15:36:06 +00:00
Viktor Szakats
4727d32e2e
sectransp: fix compiler warnings
https://github.com/curl/curl-for-win/actions/runs/6037489221/job/16381860220#step:3:11046
```
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:2435:14: warning: unused variable 'success' [-Wunused-variable]
    OSStatus success;
             ^
/Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:3300:44: warning: unused parameter 'sha256len' [-Wunused-parameter]
                                    size_t sha256len)
                                           ^
```

Closes #11773
2023-08-31 23:02:43 +00:00
Daniel Stenberg
5e2beb3395
spelling: use 'reuse' not 're-use' in code and elsewhere
Unify the spelling as both versions were previously used intermittently

Closes #11717
2023-08-23 23:22:36 +02:00
Stefan Eissing
e12b39e133
trace: make tracing available in non-debug builds
Add --trace-config to curl

Add curl_global_trace() to libcurl

Closes #11421
2023-08-03 17:32:25 +02:00
Daniel Stenberg
0b947e8ca2
sectransp: prevent CFRelease() of NULL
When SecCertificateCopyCommonName() returns NULL, the common_name
pointer remains set to NULL which apparently when calling CFRelease() on
(sometimes?) crashes.

Reported-by: Guillaume Algis
Fixes #9194
Closes #11554
2023-08-01 08:17:26 +02:00
Daniel Stenberg
7c8bae0d9c
nss: remove support for this TLS library
Closes #11459
2023-07-29 23:44:28 +02:00
Ondřej Koláček
9630fb9f2a
sectransp: fix EOF handling
Regression since the large refactor from 2022

Closes #11427
2023-07-13 14:35:21 +02:00
Viktor Szakats
3f8fc25720
cmake: add support for "unity" builds
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.

It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.

Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.

Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
  These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
  `statename`.
- rename `ssl_backend_data` structure to unique names for each
  TLS-backend, along with the `ssl_connect_data` struct member
  referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
  builds when compiling `lib/ldap.c`. To support "unity" builds, we had
  to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
  `schannel.h` option) _globally_. This caused an indirect inclusion of
  Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
  as well. This requires `[P]UNICODE_STRING` types, which is apperantly
  not defined automatically (as seen with both MSVS and mingw-w64).
  This patch includes `<subauth.h>` to fix it.
  Ref: https://github.com/curl/curl/runs/13987772013
  Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
  trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
  is built in static mode.

KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
  https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250

Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes #11095
2023-06-07 13:06:08 +00:00
Daniel Stenberg
127eb0d83a
misc: fix spelling mistakes
Reported-by: musvaage on github
Fixes #11171
Closes #11172
2023-05-23 10:42:09 +02:00
Daniel Stenberg
954c7dfb91
sectransp.c: make the code c89 compatible
Follow-up to dd2bb48552

Reported-by: FeignClaims on github
Fixes #11155
Closes #11159
2023-05-21 14:02:11 +02:00
Emanuele Torre
f198d33e8d
checksrc: disallow spaces before labels
Out of 415 labels throughout the code base, 86 of those labels were
not at the start of the line. Which means labels always at the start of
the line is the favoured style overall with 329 instances.

Out of the 86 labels not at the start of the line:
* 75 were indented with the same indentation level of the following line
* 8 were indented with exactly one space
* 2 were indented with one fewer indentation level then the following
  line
* 1 was indented with the indentation level of the following line minus
  three space (probably unintentional)

Co-Authored-By: Viktor Szakats

Closes #11134
2023-05-18 20:45:04 +02:00
Daniel Stenberg
9ce7eee070
checksrc: find bad indentation in conditions without open brace
If the previous line starts with if/while/for AND ends with a closed
parenthesis and there's an equal number of open and closed parentheses
on that line, verify that this line is indented $indent more steps, if
not a cpp line.

Also adjust the fall-out from this fix.

Closes #11054
2023-04-28 23:11:00 +02:00
Daniel Stenberg
d567cca1de
checksrc: fix SPACEBEFOREPAREN for conditions starting with "*"
The open paren check wants to warn for spaces before open parenthesis
for if/while/for but also for any function call. In order to avoid
catching function pointer declarations, the logic allows a space if the
first character after the open parenthesis is an asterisk.

I also spotted what we did not include "switch" in the check but we should.

This check is a little lame, but we reduce this problem by not allowing
that space for if/while/for/switch.

Reported-by: Emanuele Torre
Closes #11044
2023-04-27 17:24:47 +02:00
Stefan Eissing
4ae2d9f24d
proxy: http2 proxy tunnel implementation
- currently only on debug build and when env variable
  CURL_PROXY_TUNNEL_H2 is present.
- will ALPN negotiate with the proxy server and switch
  tunnel filter based on the protocol negotiated.
- http/1.1 tunnel code moved into cf-h1-proxy.[ch]
- http/2 tunnel code implemented in cf-h2-proxy.[ch]
- tunnel start and ALPN set remains in http_proxy.c
- moving all haproxy related code into cf-haproxy.[ch]

VTLS changes
- SSL filters rely solely on the "alpn" specification they
  are created with and no longer check conn->bits.tls_enable_alpn.
- checks on which ALPN specification to use (or none at all) are
  done in vtls.c when creating the filter.

Testing
- added a nghttpx forward proxy to the pytest setup that
  speaks HTTP/2 and forwards all requests to the Apache httpd
  forward proxy server.
- extending test coverage in test_10 cases
- adding proxy tests for direct/tunnel h1/h2 use of basic auth.
- adding test for http/1.1 and h2 proxy tunneling to pytest

Closes #10780
2023-04-06 13:04:46 +02:00
Stefan Eissing
31889210b9
secure-transport: fix recv return code handling
Return code handling of recv calls were not always correct when an error
occured or the connection was closed.

Closes #10717
2023-03-14 09:59:27 +01:00
Stefan Eissing
7c5637b8b4
url: fix logic in connection reuse to deny reuse on "unclean" connections
- add parameter to `conn_is_alive()` cfilter method that returns
  if there is input data waiting on the connection
- refrain from re-using connnection from the cache that have
  input pending
- adapt http/2 and http/3 alive checks to digest pending input
  to check the connection state
- remove check_cxn method from openssl as that was just doing
  what the socket filter now does.
- add tests for connection reuse with special server configs

Closes #10690
2023-03-07 12:50:31 +01:00
Daniel Stenberg
c50a6eee04
sectransp: make read_cert() use a dynbuf when loading
Closes #10632
2023-03-01 09:38:43 +01:00
Jay Satiro
47129b2b4b sectransp: fix compiler warning c89 mixed code/declaration
Since cbf57176 the Cirrus CI 'macOS arm64 SecureTransport http2' has
been failing due to c89 warnings mixed code/declaration. That commit is
not the cause so I assume something has changed in the CI outside of our
configuration. Anyway, we don't mix code/declaration so this is the fix
for that.

Closes https://github.com/curl/curl/pull/10574
2023-02-23 03:47:31 -05:00