Syncing this up with CMake.
Source code uses the built-in `OPENSSL_IS_AWSLC` and
`OPENSSL_IS_BORINSSL` macros to detect BoringSSL and AWS-LC. No help is
necessary from the build tools.
The one use of `HAVE_BORINGSSL` in the source turned out to be no longer
necessary for warning-free BoringSSL + Schannel builds. Ref: #1610#2634
autotools detects this anyway for display purposes.
CMake detects this to decide whether to use the BoringSSL-specific
crypto lib with ngtcp2. It detects AWS-LC, but doesn't use the detection
result just yet (planned in #12066).
Ref: #11964
Reviewed-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Closes#12065
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.
You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.
It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.
Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.
Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
`statename`.
- rename `ssl_backend_data` structure to unique names for each
TLS-backend, along with the `ssl_connect_data` struct member
referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
builds when compiling `lib/ldap.c`. To support "unity" builds, we had
to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
`schannel.h` option) _globally_. This caused an indirect inclusion of
Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
as well. This requires `[P]UNICODE_STRING` types, which is apperantly
not defined automatically (as seen with both MSVS and mingw-w64).
This patch includes `<subauth.h>` to fix it.
Ref: https://github.com/curl/curl/runs/13987772013
Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
is built in static mode.
KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250
Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_buildCloses#11095
- Curl_write_plain/Curl_read_plain have been eliminated. Last code use
now uses Curl_conn_send/recv so that requests use conn->send/revc
callbacks which defaults to cfilters use.
- Curl_recv_plain/Curl_send_plain have been internalized in cf-socket.c.
- USE_RECV_BEFORE_SEND_WORKAROUND (active on Windows) has been moved
into cf-socket.c. The pre_recv buffer is held at the socket filter
context. `postponed_data` structures have been removed from
`connectdata`.
- the hanger in HTTP/2 request handling was a result of read buffering
on all sends and the multi handling is not prepared for this. The
following happens:
- multi preforms on a HTTP/2 easy handle
- h2 reads and processes data
- this leads to a send of h2 data
- which receives and buffers before the send
- h2 returns
- multi selects on the socket, but no data arrives (its in the buffer already)
the workaround now receives data in a loop as long as there is something in
the buffer. The real fix would be for multi to change, so that `data_pending`
is evaluated before deciding to wait on the socket.
io_buffer, optional, in cf-socket.c, http/2 sets state.drain if lower
filter have pending data.
This io_buffer is only available/used when the
-DUSE_RECV_BEFORE_SEND_WORKAROUND is active, e.g. on Windows
configurations. It also maintains the original checks on protocol
handler being HTTP and conn->send/recv not being replaced.
The HTTP/2 (nghttp2) cfilter now sets data->state.drain when it finds
out that the "lower" filter chain has still pending data at the end of
its IO operation. This prevents the processing from becoming stalled.
Closes#10280
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING
checksrc is updated to allow non-year using copyright statements
Closes#10205
- almost all backend calls pass the Curl_cfilter intance instead of
connectdata+sockindex
- ssl_connect_data is remove from struct connectdata and made internal
to vtls
- ssl_connect_data is allocated in the added filter, kept at cf->ctx
- added function to let a ssl filter access its ssl_primary_config and
ssl_config_data this selects the propert subfields in conn and data,
for filters added as plain or proxy
- adjusted all backends to use the changed api
- adjusted all backends to access config data via the exposed
functions, no longer using conn or data directly
cfilter renames for clear purpose:
- methods `Curl_conn_*(data, conn, sockindex)` work on the complete
filter chain at `sockindex` and connection `conn`.
- methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter
instance.
- methods `Curl_conn_cf()` work on/with filter instances at a
connection.
- rebased and resolved some naming conflicts
- hostname validation (und session lookup) on SECONDARY use the same
name as on FIRST (again).
new debug macros and removing connectdata from function signatures where not
needed.
adapting schannel for new Curl_read_plain paramter.
Closes#9919
By default, the PFXImportCertStore API persists the key in the user's
key store (as though the certificate was being imported for permanent,
ongoing use.)
The documentation specifies that keys that are not to be persisted
should be imported with the flag PKCS12_NO_PERSIST_KEY.
NOTE: this flag is only supported on versions of Windows newer than XP
and Server 2003.
--
This is take 2 of the original fix. It extends the lifetime of the
client certificate store to that of the credential handle. The original
fix which landed in 70d010d and was later reverted in aec8d30 failed to
work properly because it did not do that.
Minor changes were made to the schannel credential context to support
closing the client certificate store handle at the end of an SSL session.
--
Reported-by: ShadowZzj@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/9300
Supersedes https://github.com/curl/curl/pull/9363
Closes https://github.com/curl/curl/pull/9460
- Support TLS 1.3 as the default max TLS version for Windows Server 2022
and Windows 11.
- Support specifying TLS 1.3 ciphers via existing option
CURLOPT_TLS13_CIPHERS (tool: --tls13-ciphers).
Closes https://github.com/curl/curl/pull/8419
Same issue as here [1], but this time when building curl with BoringSSL
for Windows with LDAP(S) or Schannel support enabled.
Apply the same fix [2] for these source files as well.
This can also be fixed by moving `#include "urldata.h"` _before_
including `winldap.h` and `schnlsp.h` respectively. This seems like
a cleaner fix, though I'm not sure why it works and if it has any
downside.
[1] https://github.com/curl/curl/issues/5669
[2] fbe07c6829
Co-authored-by: Jay Satiro
Closes#9110
Prior to this change _mingw.h needed to be included in each unit before
evaluating __MINGW{32,64}_xxx_VERSION macros since it defines them. It
is included only in some mingw headers (eg stdio.h) and not others
(eg windows.h) so it's better to explicitly include it once.
Closes https://github.com/curl/curl/pull/9036
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.
This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.
Closes#8869
This array is only used by the SCHANNEL_CRED struct in the
schannel_acquire_credential_handle function. It can therefore be kept as
a local variable. This is a minor update to
bbb71507b7.
This change also updates the NUM_CIPHERS value to accurately count the
number of ciphers options listed in schannel.c, which is 47 instead of
45. It is unlikely that anyone tries to set all 47 values, but if they
had tried, the last two would not have been set.
Closes#8469
The TLS backends convert the host name to SNI name and need to use that.
This involves cutting off any trailing dot and lowercasing.
Co-authored-by: Jay Satiro
Closes#8320
Curl_verify_certificate() must use the Curl_ prefix since it is globally
available in the lib and otherwise steps outside of our namespace!
Closes#3201
- Move verify_certificate functionality in schannel.c into a new
file called schannel_verify.c. Additionally, some structure defintions
from schannel.c have been moved to schannel.h to allow them to be
used in schannel_verify.c.
- Make verify_certificate functionality for Schannel available on
all versions of Windows instead of just Windows CE. verify_certificate
will be invoked on Windows CE or when the user specifies
CURLOPT_CAINFO and CURLOPT_SSL_VERIFYPEER.
- In verify_certificate, create a custom certificate chain engine that
exclusively trusts the certificate store backed by the CURLOPT_CAINFO
file.
- doc updates of --cacert/CAINFO support for schannel
- Use CERT_NAME_SEARCH_ALL_NAMES_FLAG when invoking CertGetNameString
when available. This implements a TODO in schannel.c to improve
handling of multiple SANs in a certificate. In particular, all SANs
will now be searched instead of just the first name.
- Update tool_operate.c to not search for the curl-ca-bundle.crt file
when using Schannel to maintain backward compatibility. Previously,
any curl-ca-bundle.crt file found in that search would have been
ignored by Schannel. But, with CAINFO support, the file found by
that search would have been used as the certificate store and
could cause issues for any users that have curl-ca-bundle.crt in
the search path.
- Update url.c to not set the build time CURL_CA_BUNDLE if the selected
SSL backend is Schannel. We allow setting CA location for schannel
only when explicitly specified by the user via CURLOPT_CAINFO /
--cacert.
- Add new test cases 3000 and 3001. These test cases check that the first
and last SAN, respectively, matches the connection hostname. New test
certificates have been added for these cases. For 3000, the certificate
prefix is Server-localhost-firstSAN and for 3001, the certificate
prefix is Server-localhost-secondSAN.
- Remove TODO 15.2 (Add support for custom server certificate
validation), this commit addresses it.
Closes https://github.com/curl/curl/pull/1325
These functions are all available via the Curl_ssl struct now, no need
to declare them separately anymore.
As the global declarations are removed, the corresponding function
definitions are marked as file-local. The only two exceptions here are
Curl_mbedtls_shutdown() and Curl_polarssl_shutdown(): only the
declarations were removed, there are no function definitions to mark
file-local.
Please note that Curl_nss_force_init() is *still* declared globally, as
the only SSL backend-specific function, because it was introduced
specifically for the use case where cURL was compiled with
`--without-ssl --with-nss`. For details, see f3b77e561 (http_ntlm: add
support for NSS, 2010-06-27).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The entire idea of introducing the Curl_ssl struct to describe SSL
backends is to prepare for choosing the SSL backend at runtime.
To that end, convert all the #ifdef have_curlssl_* style conditionals
to use bit flags instead.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This is the first step to unify the SSL backend handling. Now all the
SSL backend-specific functionality is accessed via a global instance of
the Curl_ssl struct.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The idea of introducing the Curl_ssl struct was to unify how the SSL
backends are declared and called. To this end, we now provide an
instance of the Curl_ssl struct for each and every SSL backend.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This commit is several drafts squashed together. The changes from each
draft are noted below. If any changes are similar and possibly
contradictory the change in the latest draft takes precedence.
Bug: https://github.com/bagder/curl/issues/244
Reported-by: Chris Araman
%%
%% Draft 1
%%
- return 0 if len == 0. that will have to be documented.
- continue on and process the caches regardless of raw recv
- if decrypted data will be returned then set the error code to CURLE_OK
and return its count
- if decrypted data will not be returned and the connection has closed
(eg nread == 0) then return 0 and CURLE_OK
- if decrypted data will not be returned and the connection *hasn't*
closed then set the error code to CURLE_AGAIN --only if an error code
isn't already set-- and return -1
- narrow the Win2k workaround to only Win2k
%%
%% Draft 2
%%
- Trying out a change in flow to handle corner cases.
%%
%% Draft 3
%%
- Back out the lazier decryption change made in draft2.
%%
%% Draft 4
%%
- Some formatting and branching changes
- Decrypt all encrypted cached data when len == 0
- Save connection closed state
- Change special Win2k check to use connection closed state
%%
%% Draft 5
%%
- Default to CURLE_AGAIN in cleanup if an error code wasn't set and the
connection isn't closed.
%%
%% Draft 6
%%
- Save the last error only if it is an unrecoverable error.
Prior to this I saved the last error state in all cases; unfortunately
the logic to cover that in all cases would lead to some muddle and I'm
concerned that could then lead to a bug in the future so I've replaced
it by only recording an unrecoverable error and that state will persist.
- Do not recurse on renegotiation.
Instead we'll continue on to process any trailing encrypted data
received during the renegotiation only.
- Move the err checks in cleanup after the check for decrypted data.
In either case decrypted data is always returned but I think it's easier
to understand when those err checks come after the decrypted data check.
%%
%% Draft 7
%%
- Regardless of len value go directly to cleanup if there is an
unrecoverable error or a close_notify was already received. Prior to
this change we only acknowledged those two states if len != 0.
- Fix a bug in connection closed behavior: Set the error state in the
cleanup, because we don't know for sure it's an error until that time.
- (Related to above) In the case the connection is closed go "greedy"
with the decryption to make sure all remaining encrypted data has been
decrypted even if it is not needed at that time by the caller. This is
necessary because we can only tell if the connection closed gracefully
(close_notify) once all encrypted data has been decrypted.
- Do not renegotiate when an unrecoverable error is pending.
%%
%% Draft 8
%%
- Don't show 'server closed the connection' info message twice.
- Show an info message if server closed abruptly (missing close_notify).
