Commit Graph

13380 Commits

Author SHA1 Message Date
David Benjamin
9eb774304e
openssl: use X509_ALGOR_get0 instead of reaching into X509_ALGOR
While the struct is still public in OpenSSL, there is a (somewhat
inconvenient) accessor. Use it to remain compatible if it becomes opaque
in the future.

Closes #12038
2023-10-06 14:51:20 +02:00
Daniel Stenberg
6dd6654f75
multi: set CURLM_CALL_MULTI_PERFORM after switch to DOING_MORE
Since there is nothing to wait for there. Avoids the test 1233 hang
reported in #12033.

Reported-by: Dan Fandrich
Closes #12042
2023-10-06 08:49:58 +02:00
Jay Satiro
021d04f291 idn: fix WinIDN null ptr deref on bad host
- Return CURLE_URL_MALFORMAT if IDN hostname cannot be converted from
  UTF-8 to UTF-16.

Prior to this change a failed conversion erroneously returned CURLE_OK
which meant 'decoded' pointer (what would normally point to the
punycode) would not be written to, remain NULL and be dereferenced
causing an access violation.

Closes https://github.com/curl/curl/pull/11983
2023-10-05 03:11:41 -04:00
Daniel Stenberg
3ef3eaa27e
base64: also build for curl
Since the tool itself now uses the base64 code using the curlx way, it
needs to build also when the tool needs it. Starting now, the tool build
defines BULDING_CURL to allow lib-side code to use it.

Follow-up to 2e160c9c65

Closes #12010
2023-10-04 23:22:16 +02:00
Viktor Szakats
fd328fcaf1
cmake: fix unity builds for more build combinations
By using unique static function/variable names in source files
implementing these interfaces.

- OpenLDAP combined with any SSH backend.

- MultiSSL with mbedTLS, OpenSSL, wolfSSL, SecureTransport.

Closes #12027
2023-10-04 15:36:06 +00:00
Viktor Szakats
3b6d18bbf6
spelling: fix codespell 2.2.6 typos
Closes #12019
2023-10-03 21:37:56 +00:00
Viktor Szakats
83ec54e1b9
build: alpha-sort source files for lib and src
Closes #12014
2023-10-03 12:58:17 +00:00
Viktor Szakats
f42a279ee3
cmake: fix unity with Windows Unicode + TrackMemory
Found the root cause of the startup crash in unity builds with Unicode
and TrackMemory enabled at the same time.

We must make sure that the `memdebug.h` header doesn't apply to
`lib/curl_multibyte.c` (as even noted in a comment there.) In unity
builds all headers apply to all sources, including `curl_multibyte.c`.
This probably resulted in an infinite loop on startup.

Exclude this source from unity compilation with TrackMemory enabled,
in both libcurl and curl tool. Enable unity mode for a debug Unicode
CI job to keep it tested. Also delete the earlier workaround that
fully disabled unity for affected builds.

Follow-up to d82b080f63 #12005
Follow-up to 3f8fc25720 #11095

Closes #11928
2023-10-03 09:43:46 +00:00
Stefan Eissing
6b9a591bf7
h2: testcase and fix for pausing h2 streams
- refs #11982 where it was noted that paused transfers may
  close successfully without delivering the complete data
- made sample poc into tests/http/client/h2-pausing.c and
  added test_02_27 to reproduce

Closes #11989
Fixes #11982
Reported-by: Harry Sintonen
2023-09-30 23:53:33 +02:00
Viktor Szakats
f85dcaa6d2
cmake: detect sys/wait.h and netinet/udp.h
Ref: #11964 (effort to sync cmake detections with autotools)

Closes #11996
2023-09-30 15:17:21 +00:00
Daniel Stenberg
91878ebeca
lib: provide and use Curl_hexencode
Generates a lower case ASCII hex output from a binary input.

Closes #11990
2023-09-30 11:45:39 +02:00
Daniel Stenberg
463528b0f8
wolfssl: ignore errors in CA path
The default wolfSSL_CTX_load_verify_locations() function is quite picky
with the certificates it loads and will for example return error if just
one of the certs has expired.

With the *_ex() function and its WOLFSSL_LOAD_FLAG_IGNORE_ERR flag, it
behaves more similar to what OpenSSL does by default.

Even the set of default certs on my Debian unstable has several expired
ones.

Assisted-by: Juliusz Sosinowicz
Assisted-by: Michael Osipov

Closes #11987
2023-09-30 11:19:38 +02:00
Viktor Szakats
da5dcb7007
cmake: detect HAVE_CLOCK_GETTIME_MONOTONIC_RAW
Based on existing autotools logic.

Ref: #11964 (effort to sync cmake detections with autotools)

Closes #11981
2023-09-29 18:30:34 +00:00
Viktor Szakats
ca7daadd9b
cmake: fix HAVE_WRITABLE_ARGV detection
Move detection before the creation of detection results in
`curl_config.h`.

Ref: #11964 (effort to sync cmake detections with autotools)

Closes #11978
2023-09-29 18:29:55 +00:00
Viktor Szakats
1bc69df7b4
tidy-up: use more example domains
Also make use of the example TLD:
https://en.wikipedia.org/wiki/.example

Reviewed-by: Daniel Stenberg
Closes #11992
2023-09-29 18:25:56 +00:00
Daniel Stenberg
9b517c8b69
cmake: add missing checks
- check for arc4random. To make rand.c use it accordingly.
- check for fcntl
- fix fseek detection
- add SIZEOF_CURL_SOCKET_T
- fix USE_UNIX_SOCKETS
- define HAVE_SNPRINTF to 1
- check for fnmatch
- check for sched_yield
- remove HAVE_GETPPID duplicate from curl_config.h
- add HAVE_SENDMSG

Ref: #11964

Co-authored-by: Viktor Szakats
Closes #11973
2023-09-28 23:00:43 +02:00
Daniel Stenberg
db07376a3e
lib: remove TIME_WITH_SYS_TIME
It is not used in any code anywhere.

Ref: #11964
Closes #11975
2023-09-28 22:58:36 +02:00
Daniel Stenberg
290622cea6
cmake: add check for suseconds_t
And fix the HAVE_LONGLONG define

Ref: #11964
Closes #11977
2023-09-28 22:56:24 +02:00
Viktor Szakats
a8c773845f
tidy-up: whitespace fixes
Closes #11972
2023-09-28 12:03:28 +00:00
Viktor Szakats
781242ffa4
cmake: detect TLS-SRP in OpenSSL/wolfSSL/GnuTLS
With new option `CURL_DISABLE_SRP=ON` to force-disable it.
To match existing option and detection logic in autotools.

Also:
- fix detecting GnuTLS.
  We assume `nettle` as a GnuTLS dependency.
- add CMake GnuTLS CI job.
- bump AppVeyor CMake OpenSSL MSVC job to OpenSSL 1.1.1 (from 1.0.2)
  TLS-SRP fails to detect with 1.0.2 due to an OpenSSL header bug.
- fix compiler warning when building with GnuTLS and disabled TLS-SRP.
- fix comment typos, whitespace.

Ref: #11964

Closes #11967
2023-09-28 10:50:56 +00:00
Loïc Yhuel
b5bb84cbef
connect: only start the happy eyeballs timer when needed
The timeout is only used when there is a second address family, for the
delayed eyeballer.

Closes #11939
2023-09-28 10:21:32 +02:00
Stefan Eissing
0bd9e137e3
lib: move handling of data->req.writer_stack into Curl_client_write()
- move definitions from content_encoding.h to sendf.h
- move create/cleanup/add code into sendf.c
- installed content_encoding writers will always be called
  on Curl_client_write(CLIENTWRITE_BODY)
- Curl_client_cleanup() frees writers and tempbuffers from
  paused transfers, irregardless of protocol

Closes #11908
2023-09-28 10:00:13 +02:00
Loïc Yhuel
d39863d27a
multi: round the timeout up to prevent early wakeups
Curl_timediff rounds down to the millisecond, so curl_multi_perform can
be called too early, then we get a timeout of 0 and call it again.

The code already handled the case of timeouts which expired less than
1ms in the future.  By rounding up, we make sure we will never ask the
platform to wake up too early.

Closes #11938
2023-09-28 09:52:20 +02:00
Daniel Stenberg
1f92db87e0
inet_ntop: add typecast to silence Coverity
CID 1024653:  Integer handling issues  (SIGN_EXTENSION)

Suspicious implicit sign extension: "src[i]" with type "unsigned char
const" (8 bits, unsigned) is promoted in "src[i] << (1 - i % 2 << 3)" to
type "int" (32 bits, signed), then sign-extended to type "unsigned long"
(64 bits, unsigned).  If "src[i] << (1 - i % 2 << 3)" is greater than
0x7FFFFFFF, the upper bits of the result will all be 1.

111         words[i/2] |= (src[i] << ((1 - (i % 2)) << 3));

The value will not be greater than 0x7FFFFFFF so this still cannot
happen.

Also, switch to ints here instead of longs. The values stored are 16 bit
so at least no need to use 64 bit variables. Also, longs are 32 bit on
some platforms so this logic still needs to work with 32 bits.

Closes #11960
2023-09-27 13:18:22 +02:00
Daniel Stenberg
01d8473b25
connect: expire the timeout when trying next
... so that it gets called again immediately and can continue trying
addresses to connect to. Otherwise it might unnecessarily wait for a
while there.

Fixes #11920
Reported-by: Loïc Yhuel
Closes #11935
2023-09-27 10:35:36 +02:00
Daniel Stenberg
afb4b15614
http: remove wrong comment for http_should_fail
Reported-by: Christian Schmitz
Ref: #11936
Closes #11941
2023-09-27 09:15:03 +02:00
Viktor Szakats
1411c5eb33
cmake: add feature checks for memrchr and getifaddrs
- `HAVE_MEMRCHR` for `memrchr`.
- `HAVE_GETIFADDRS` for `getifaddrs`.
  This was present in `lib/curl_config.h.cmake` but missed the detection
  logic.

To match existing autotools feature checks.

Closes #11954
2023-09-26 22:10:28 +00:00
Viktor Szakats
96c29900bc
build: delete checks for C89 standard headers
Delete checks and guards for standard C89 headers and assume these are
available: `stdio.h`, `string.h`, `time.h`, `setjmp.h`, `stdlib.h`,
`stddef.h`, `signal.h`.

Some of these we already used unconditionally, some others we only used
for feature checks.

Follow-up to 9c7165e96a #11918 (for `stdio.h` in CMake)

Closes #11940
2023-09-26 14:25:10 +00:00
Stefan Eissing
d15ffe18d9 multiif.h: remove Curl_multi_dump declaration
Follow-up to d850eea2 which removed the Curl_multi_dump definition.

Closes https://github.com/curl/curl/pull/11946
2023-09-26 03:51:13 -04:00
Jay Satiro
aa6c94c5bf config-win32: define HAVE__FSEEKI64
Follow-up to 9c7165e9 which added an fseeko wrapper to the lib that
calls _fseeki64 if it is available.

Closes https://github.com/curl/curl/pull/11944
2023-09-26 03:45:20 -04:00
Stefan Eissing
34cdcb9b96 openssl: improve ssl shutdown handling
- If SSL shutdown is not finished then make an additional call to
  SSL_read to gather additional tracing.

- Fix http2 and h2-proxy filters to forward do_close() calls to the next
  filter.

For example h2 and SSL shutdown before and after this change:

Before:

Curl_conn_close -> cf_hc_close -> Curl_conn_cf_discard_chain ->
ssl_cf_destroy

After:

Curl_conn_close -> cf_hc_close -> cf_h2_close -> cf_setup_close ->
ssl_cf_close

Note that currently the tracing does not show output on the connection
closure handle. Refer to discussion in #11878.

Ref: https://github.com/curl/curl/discussions/11878

Closes https://github.com/curl/curl/pull/11858
2023-09-26 03:40:20 -04:00
Loïc Yhuel
579f09343d
multi: fix small timeouts
Since Curl_timediff rounds down to the millisecond, timeouts which
expire in less than 1ms are considered as outdated and removed from the
list. We can use Curl_timediff_us instead, big timeouts could saturate
but this is not an issue.

Closes #11937
2023-09-26 00:58:47 +02:00
Viktor Szakats
a8ebde99f7
cmake: lib CURL_STATICLIB fixes (Windows)
- always define `CURL_STATICLIB` when building libcurl for Windows.

  This disables `__declspec(dllexport)` for exported libcurl symbols.
  In normal mode (hide symbols) these exported symbols are specified
  via `libcurl.def`. When not hiding symbols, all symbols are exported
  by default.

  Regression from 1199308dbc

  Fixes #11844

- fix to omit `libcurl.def` when not hiding private symbols.

  Regression from 2ebc74c36a

- fix `ENABLED_DEBUG=ON` + shared curl tool Windows builds by also
  omitting `libcurl.def` in this case, and exporting all symbols
  instead. This ensures that a shared curl tool can access all debug
  functions which are not normally exported from libcurl DLL.

- delete `INTERFACE_COMPILE_DEFINITIONS "CURL_STATICLIB"` for "objects"
  target.

  Follow-up to 2ebc74c36a

- delete duplicate `BUILDING_LIBCURL` definitions.

- fix `HIDES_CURL_PRIVATE_SYMBOLS` to not overwrite earlier build settings.

  Follow-up to 1199308dbc

Closes #11914
2023-09-25 22:05:29 +00:00
Daniel Stenberg
9ffd411735
curl_multi_get_handles: get easy handles from a multi handle
Closes #11750
2023-09-25 20:16:58 +02:00
Stefan Eissing
bb4032a152
http: h1/h2 proxy unification
- use shared code for setting up the CONNECT request
  when tunneling, used in HTTP/1.x and HTTP/2 proxying
- eliminate use of Curl_buffer_send() and other manipulations
  of `data->req` or `data->state.ulbuf`

Closes #11808
2023-09-25 20:12:18 +02:00
Natanael Copa
9c7165e96a
lib: use wrapper for curl_mime_data fseek callback
fseek uses long offset which does not match with curl_off_t. This leads
to undefined behavior when calling the callback and caused failure on
arm 32 bit.

Use a wrapper to solve this and use fseeko which uses off_t instead of
long.

Thanks to the nice people at Libera IRC #musl for helping finding this
out.

Fixes #11882
Fixes #11900
Closes #11918
2023-09-25 20:03:09 +02:00
Daniel Stenberg
358f7e7577
warnless: remove unused functions
Previously put there for use with the intel compiler

Closes #11932
2023-09-25 17:06:33 +02:00
Daniel Stenberg
d850eea2d0
multi: remove Curl_multi_dump
A debug-only function that is basically never used. Removed to ease the
use of the singleuse script to detect non-static functions not used
outside the file where it is defined.

Closes #11931
2023-09-25 12:35:40 +02:00
Jay Satiro
ab18c04218 url: fix netrc info message
- Fix netrc info message to use the generic ".netrc" filename if the
  user did not specify a netrc location.

- Update --netrc doc to add that recent versions of curl on Windows
  prefer .netrc over _netrc.

Before:
* Couldn't find host google.com in the (nil) file; using defaults

After:
* Couldn't find host google.com in the .netrc file; using defaults

Closes https://github.com/curl/curl/pull/11904
2023-09-24 03:37:13 -04:00
Dan Fandrich
739a9e8e12 wolfssh: do cleanup in Curl_ssh_cleanup
Closes: #11921
2023-09-23 23:15:11 -07:00
Viktor Szakats
6a85659e7d
Makefile.mk: always set CURL_STATICLIB for lib (Windows)
Also fix to export all symbols in Windows debug builds, making
`-debug-dyn` builds work with `-DCURL_STATICLIB` set.

Ref: https://github.com/curl/curl/pull/11914 (same for CMake)

Closes #11924
2023-09-23 20:42:47 +00:00
Daniel Stenberg
aa9a6a1770
quic: set ciphers/curves the same way regular TLS does
for OpenSSL/BoringSSL

Fixes #11796
Reported-by: Karthikdasari0423 on github
Assisted-by: Jay Satiro
Closes #11836
2023-09-23 11:36:05 +02:00
Daniel Stenberg
914e49b9b7
lib: let the max filesize option stop too big transfers too
Previously it would only stop them from getting started if the size is
known to be too big then.

Update the libcurl and curl docs accordingly.

Fixes #11810
Reported-by: Elliot Killick
Assisted-by: Jay Satiro
Closes #11820
2023-09-23 11:20:20 +02:00
Viktor Szakats
38029101e2
mingw: delete support for legacy mingw.org toolchain
Drop support for "old" / "legacy" / "classic" / "v1" / "mingw32" MinGW:
  https://en.wikipedia.org/wiki/MinGW, https://osdn.net/projects/mingw/
Its homepage used to be http://mingw.org/ [no HTTPS], and broken now.
It supported the x86 CPU only and used a old Windows API header and
implib set, often causing issues. It also misses most modern Windows
features, offering old versions of both binutils and gcc (no llvm/clang
support). It was last updated 2 years ago.

curl now relies on toolchains based on the mingw-w64 project:
https://www.mingw-w64.org/  https://sourceforge.net/projects/mingw-w64/
https://www.msys2.org/  https://github.com/msys2/msys2
https://github.com/mstorsjo/llvm-mingw
(Also available via Linux and macOS package managers.)

Closes #11625
2023-09-23 09:12:57 +00:00
Daniel Stenberg
5595e33617
bufq: remove Curl_bufq_skip_and_shift (unused)
Closes #11915
2023-09-23 10:50:10 +02:00
Viktor Szakats
48ba5d9007
cmake: fix unity symbol collisions in h2 builds
Regression from 331b89a319

Reviewed-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Closes #11912
2023-09-22 09:19:10 +00:00
Jakub Jelen
35eb2614d8
libssh: cap SFTP packet size sent
Due to libssh limitations

Signed-off-by: Jakub Jelen <jjelen@redhat.com>

Closes #11804
2023-09-21 16:53:30 +02:00
Jay Satiro
80fc040e45 libssh2: fix error message on failed pubkey-from-file
- If libssh2_userauth_publickey_fromfile_ex returns -1 then show error
  message "SSH public key authentication failed: Reason unknown (-1)".

When libssh2_userauth_publickey_fromfile_ex returns -1 it does so as a
generic error and therefore doesn't set an error message. AFAICT that is
not documented behavior.

Prior to this change libcurl retrieved the last set error message which
would be from a previous function failing. That resulted in misleading
auth failed error messages in verbose mode.

Bug: https://github.com/curl/curl/issues/11837#issue-1891827355
Reported-by: consulion@users.noreply.github.com

Closes https://github.com/curl/curl/pull/11881
2023-09-21 02:59:21 -04:00
Stefan Eissing
8898257446
lib: disambiguate Curl_client_write flag semantics
- use CLIENTWRITE_BODY *only* when data is actually body data
- add CLIENTWRITE_INFO for meta data that is *not* a HEADER
- debug assertions that BODY/INFO/HEADER is not used mixed
- move `data->set.include_header` check into Curl_client_write
  so protocol handlers no longer have to care
- add special in FTP for `data->set.include_header` for historic,
  backward compatible reasons
- move unpausing of client writes from easy.c to sendf.c, so that
  code is in one place and can forward flags correctly

Closes #11885
2023-09-21 08:56:50 +02:00
Patrick Monnerat
bbac7c19e5
tftpd: always use curl's own tftp.h
Using the system's provided arpa/tftp.h and optimizing, GCC 12 detects
and reports a stringop-overread warning:

tftpd.c: In function ‘write_behind.isra’:
tftpd.c:485:12: warning: ‘write’ reading between 1 and 2147483647 bytes from a region of size 0 [-Wstringop-overread]
  485 |     return write(test->ofile, writebuf, count);
      |            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from tftpd.c:71:
/usr/include/arpa/tftp.h:58:30: note: source object ‘tu_data’ of size 0
   58 |                         char tu_data[0];        /* data or error string */
      |                              ^~~~~~~

This occurs because writebuf points to this field and the latter
cannot be considered as being of dynamic length because it is not
the last field in the structure. Thus it is bound to its declared
size.

This commit always uses curl's own version of tftp.h where the
target field is last in its structure, effectively avoiding the
warning.

As HAVE_ARPA_TFTP_H is not used anymore, cmake/configure checks for
arpa/tftp.h are removed.

Closes #11897
2023-09-21 08:47:07 +02:00