Commit Graph

12751 Commits

Author SHA1 Message Date
Prithvi MK
b80dae232e
c-hyper: fix multi-request mechanism
It makes test 565 run fine.

Fixes #8896
Closes #10080
Assisted-by: Daniel Stenberg
2022-12-11 14:15:53 +01:00
Ryan Schmidt
ac4554870d
system.h: fix socklen_t, curl_off_t, long long for Classic Mac OS
Change "__MWERKS__" to "macintosh". When this block was originally added
in 3ac6929 it was probably intended to handle classic Mac OS since the
previous classic Mac OS build procedure for curl (which was removed in
bf327a9) used Metrowerks CodeWarrior.

But there are other classic Mac OS compilers, such as the MPW compilers,
that were not handled by this case. For classic Mac OS,
CURL_TYPEOF_CURL_SOCKLEN_T needs to match what's provided by the
third-party GUSI library, which does not vary by compiler.

Meanwhile CodeWarrior works on platforms other than classic Mac OS, and
they may need different definitions. Separate blocks could be added
later for any of those platforms that curl doesn't already support.

Closes #10049
2022-12-09 08:21:15 +01:00
Daniel Gustafsson
60453483b5 noproxy: guard against empty hostnames in noproxy check
When checking for a noproxy setting we need to ensure that we get
a hostname passed in. If there is no hostname then there cannot be
a matching noproxy rule for it by definition.

Closes: #10057
Reported-by: Geeknik Labs
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2022-12-08 23:37:32 +01:00
Daniel Stenberg
c8d24d4e77
c-hyper: CONNECT respones are not server responses
Together with d31915a8db it makes test 265 run fine.

Fixes #8853
Assisted-by: Prithvi MK
Assisted-by: Sean McArthur
Closes #10060
2022-12-08 23:36:23 +01:00
Stefan Eissing
b42156b825
cfilter: improve SSL connection checks
- fixes `Curl_ssl_cf_get_ssl()` to detect also the first filter instance
  as ssl (refs #10053)

- replaces `Curl_ssl_use()` with the correct `Curl_conn_is_ssl()`

Closes #10054
Fixes #10053

Reported-by: Patrick Monnerat
2022-12-08 14:27:06 +01:00
Ryan Schmidt
33f1f4f795 config-mac: define HAVE_SYS_IOCTL_H
This is needed to compile nonblock.c on classic Mac OS with Grand
Unified Socket Interface (GUSI) because nonblock.c uses FIONBIO which is
defined in <sys/filio.h> which is included by <sys/ioctl.h>.

Ref: https://sourceforge.net/projects/gusi/

Closes https://github.com/curl/curl/pull/10042
2022-12-07 15:56:43 -05:00
Jay Satiro
fe3463eba7 build: assume errno.h is always available
- Remove errno.h detection from all build configurations.

errno.h is a standard header according to C89.

Closes https://github.com/curl/curl/pull/9986
2022-12-07 14:41:09 -05:00
Jay Satiro
57d3477e77 build: assume assert.h is always available
- Remove assert.h detection from all build configurations.

assert.h is a standard header according to C89.

I had proposed this several years ago as part of a larger change that
was abandoned.

Ref: https://github.com/curl/curl/issues/1237#issuecomment-277500720

Closes https://github.com/curl/curl/pull/9985
2022-12-07 14:30:32 -05:00
andy5995
68ffb4063d misc: Fix incorrect spelling
Fix various uses of connnect by replacing them with connect.

Closes: #10045
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
2022-12-06 23:18:23 +01:00
Stefan Eissing
2d3e20a2fc
wolfssl: remove special BIO return code handling
- rely solely on the retry flag in BIO, similar to OpenSSL vtls
  implementation.

Ref: https://github.com/curl/curl/pull/10021#issuecomment-1336147053

Closes #10033
2022-12-06 11:08:12 +01:00
Daniel Stenberg
5ec759247a
openssl: return -1 on error in the BIO callbacks
BIO_read and BIO_write return negative numbers on error, including
retryable ones. A regression from 55807e6. Both branches should be
returning -1.

The APIs are patterned after POSIX read and write which, similarly,
return -1 on errors, not zero, with EAGAIN treated as an error.

Bug: https://github.com/curl/curl/issues/10013#issuecomment-1335308146
Reported-by: David Benjamin
Closes #10021
2022-12-06 09:50:29 +01:00
Ryan Schmidt
0ed8ff2d37
config-mac: remove HAVE_SYS_SELECT_H
When compiling for classic Mac OS with GUSI, there is no sys/select.h.
GUSI provides the "select" function prototype in sys/time.h.

Closes #10039
2022-12-06 09:43:03 +01:00
Ryan Schmidt
82b3481309
setup: do not require __MRC__ defined for Mac OS 9 builds
Partially reverts "somewhat protect Mac OS X users from using Mac OS 9
config file", commit 62519bfe05.

Do things that are specific to classic Mac OS (i.e. include config-mac.h
in curl_setup.h and rename "main" to "curl_main" in tool_setup.h) when
only "macintosh" is defined. Remove the additional condition that
"__MRC__" should be defined since that would only be true with the MPW
MrC compiler which prevents the use of other reasonable compilers like
the MPW SC compiler and especially the Metrowerks CodeWarrior compilers.
"macintosh" is only defined by classic Mac OS compilers so this change
should not affect users of Mac OS X / OS X / macOS / any other OS.

Closes #10037
2022-12-06 09:35:47 +01:00
Daniel Stenberg
c6f602c93f
base64: faster base64 decoding
- by using a lookup table instead of strchr()
- by doing full quantums first, then padding

Closes #10032
2022-12-06 08:57:04 +01:00
mickae1
8fe33a9661
libssh2: return error when ssh_hostkeyfunc returns error
return CURLE_PEER_FAILED_VERIFICATION if verification with the callback
return a result different than CURLKHMATCH_OK

Closes #10034
2022-12-06 00:15:18 +01:00
Viktor Szakats
2b584fffce
Makefile.mk: improve a GNU Make hack [ci skip]
Replace the hack of using `$() ` to represent a single space. The new
method silences the `--warn-undefined-variables` debug warning and it's
also a better-known form of solving this problem.

Reviewed-by: Jay Satiro
Closes #10031
2022-12-05 14:31:32 +00:00
Daniel Stenberg
fc02a78081
base64: encode without using snprintf
For speed. In some tests, this approch is 29 times faster!

Closes #10026
2022-12-05 11:33:34 +01:00
Daniel Stenberg
fa467a2fa9
base64: better alloc size
The previous algorithm allocated more bytes than necessary.

Suggested-by: xtonik on github
Fixes #10024
Closes #10025
2022-12-05 08:11:24 +01:00
Ryan Schmidt
29bb9ba431
config-mac: fix typo: size_T -> size_t
Both MPW and CodeWarrior compilers complained about this.

Closes #10029
2022-12-05 08:10:04 +01:00
Viktor Szakats
8fc2423338
Makefile.mk: address minor issues
- Fix `NROFF` auto-detection with certain shell/make-build combinations:

  When a non-MSYS2 GNU Make runs inside an MSYS2 shell, Make executes
  the detection command as-is via `CreateProcess()`. It fails because
  `command` is an `sh` built-in. Ensure to explicitly invoke the shell.

- Initialize user-customizable variables:

  Silences a list of warnings when running GNU Make with the option
  `--warn-undefined-variables`. Another benefit is that it's now easy
  to look up all user-customizable `Makefile.mk` variables by grepping
  for ` ?=` in the curl source tree.

  Suggested-by: Gisle Vanem
  Ref: https://github.com/curl/curl/pull/9764#issuecomment-1330674433

- Fix `MKDIR` invocation:

  Avoid a warning and potential issue in envs without forward-slash
  support.

Closes #10000
2022-12-02 15:20:41 +00:00
Rob de Wit
73c4f9696a
curl_get_line: allow last line without newline char
improve backwards compatibility

Test 3200 verifies

Closes #9973
2022-12-02 11:28:43 +01:00
Daniel Stenberg
73d6f41489
cookie: open cookie jar as a binary file
On Windows there is a difference and for text files, ^Z means end of
file which is not desirable.

Ref: #9973
Closes #10017
2022-12-02 10:07:41 +01:00
Stefan Eissing
5c27f6c452
openssl: fix for BoringSSL BIO result interpretation mixups
Reported-by: Robin Marx
Fixes #10013
Closes #10015
2022-12-01 18:27:27 +01:00
Stefan Eissing
27ec767ebd
gnutls: use common gnutls init and verify code for ngtcp2
Closes #10007
2022-12-01 10:57:36 +01:00
Baitinq on github
b8ffb02e84
aws_sigv4: fix typos in aws_sigv4.c
Closes #10008
2022-12-01 08:34:37 +01:00
Daniel Stenberg
39bbd1cafb
openssl: prefix errors with '[lib]/[version]: '
To help users understand where this (cryptic) error message comes from.

Suggested-by: Philip Sanetra
Ref: #10002
Closes #10004
2022-11-30 14:05:57 +01:00
Daniel Stenberg
07203b0bb8
openssl/mbedtls: use %d for outputing port with failf (int)
Coverity CID 1517100

Also, remove some int typecasts in vtls.c for the port number

Closes #10001
2022-11-29 23:16:47 +01:00
Daniel Stenberg
ea557cfa27
lib/optiontable.pl: adapt to CURLOPTDEPRECATED()
Follow-up from 6967571bf2

Reported-by: Gisle Vanem

Fixes #9992
Closes #9993
2022-11-29 14:43:42 +01:00
Ikko Ashimine
ae99ac6ffb vtls: fix typo in vtls_int.h
paramter -> parameter

Closes: #9996
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
2022-11-28 19:51:22 +01:00
Stefan Eissing
55807e6c05
tls: backends use connection filters for IO, enabling HTTPS-proxy
- OpenSSL (and compatible)
 - BearSSL
 - gnutls
 - mbedtls
 - rustls
 - schannel
 - secure-transport
 - wolfSSL (v5.0.0 and newer)

 This leaves only the following without HTTPS-proxy support:
 - gskit
 - nss
 - wolfSSL (versions earlier than v5.0.0)

Closes #9962
2022-11-28 13:56:23 +01:00
Patrick Monnerat
eb559c8056
mime: relax easy/mime structures binding
Deprecation and removal of codeset conversion support from the library
have released the strict need for an early binding of mime structures to
an easy handle (https://github.com/curl/curl/commit/2610142).

This constraint currently forces to create the handle before the mime
structure and the latter cannot be attached to another handle once
created (see https://curl.se/mail/lib-2022-08/0027.html).

This commit removes the handle pointers from the mime structures
allowing more flexibility on their use.

When an easy handle is duplicated, bound mime structures must however
still be duplicated too as their components hold send-time dynamic
information.

Closes #9927
2022-11-28 09:09:54 +01:00
fractal-access
76b3f5f2cf
ftp: support growing files with CURLOPT_IGNORE_CONTENT_LENGTH
When using the option CURLOPT_IGNORE_CONTENT_LENGTH (set.ignorecl in
code) to support growing files in FTP, the code should ignore the
initial size it gets from the server as this will not be the final size
of the file. This is done in ftp_state_quote() to prevent a size request
being issued in the initial sequence. However, in a later call to
ftp_state_get_resp() the code attempts to get the size of the content
again if it doesn't already have it, by parsing the response from the
RETR request. This fix prevents this parsing of the response to get the
size when the set.ignorecl option is set. This should maintain the size
value as -1, unknown, in this situation.

Closes #9772
2022-11-26 17:02:54 +01:00
Stefan Eissing
f18956d0ca
cfilter: re-add conn as parameter to cfilter setup methods
- `Curl_ssl_get_config()` now returns the first config if no SSL proxy
  filter is active

- socket filter starts connection only on first invocation of its
  connect method

Fixes #9982
Closes #9983
2022-11-26 16:56:46 +01:00
Casey Bodley
7f8e6da6dc
aws_sigv4: consult x-%s-content-sha256 for payload hash
`Curl_output_aws_sigv4()` doesn't always have the whole payload in
memory to generate a real payload hash. this commit allows the user to
pass in a header like `x-amz-content-sha256` to provide their desired
payload hash

some services like s3 require this header, and may support other values
like s3's `UNSIGNED-PAYLOAD` and `STREAMING-AWS4-HMAC-SHA256-PAYLOAD`
with special semantics. servers use this header's value as the payload
hash during signature validation, so it must match what the client uses
to generate the signature

CURLOPT_AWS_SIGV4.3 now describes the content-sha256 interaction

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Closes #9804
2022-11-25 09:24:07 +01:00
Daniel Stenberg
1b3973180b
lib: rewind BEFORE request instead of AFTER previous
This makes a big difference for cases when the rewind is not actually
necessary to perofm (for example HTTP response code 301 converts to GET)
and therefore the rewind can be avoided. In particular for situations
when that rewind fails, for example when reading from a pipe or similar.

Reported-by: Ali Utku Selen

Fixes #9735
Closes #9958
2022-11-25 08:30:26 +01:00
Daniel Stenberg
9ecebb5d00
vtls: repair build with disabled proxy
Closes #9974
2022-11-24 09:04:14 +01:00
Daniel Gustafsson
3ae63a121d openssl: silence compiler warning when not using IPv6
In non-IPv6 builds the conn parameter is unused, and compilers which
run with "-Werror=unused-parameter" (or similar) warnings turned on
fails to build. Below is an excerpt from a CI job:

  vtls/openssl.c: In function ‘Curl_ossl_verifyhost’:
  vtls/openssl.c:2016:75: error: unused parameter ‘conn’ [-Werror=unused-parameter]
   2016 | CURLcode Curl_ossl_verifyhost(struct Curl_easy *data, struct connectdata *conn,
        |                                                       ~~~~~~~~~~~~~~~~~~~~^~~~

Closes: #9970
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2022-11-23 12:10:30 +01:00
Daniel Gustafsson
2d9fee4504 netware: remove leftover traces
Commit 3b16575ae9 removed support for
building on Novell Netware, but a few leftover traces remained. This
removes the last bits.

Closes: #9966
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2022-11-23 11:56:52 +01:00
Ryan Schmidt
e007ace30d
curl_endian: remove Curl_write64_le from header
The actual function was already removed in 4331c6dc.

See #7280
Closes #9968
2022-11-23 09:07:12 +01:00
Stefan Eissing
af22c2a546
vtls: localization of state data in filters
- almost all backend calls pass the Curl_cfilter intance instead of
   connectdata+sockindex
 - ssl_connect_data is remove from struct connectdata and made internal
   to vtls
 - ssl_connect_data is allocated in the added filter, kept at cf->ctx

 - added function to let a ssl filter access its ssl_primary_config and
   ssl_config_data this selects the propert subfields in conn and data,
   for filters added as plain or proxy
 - adjusted all backends to use the changed api
 - adjusted all backends to access config data via the exposed
   functions, no longer using conn or data directly

cfilter renames for clear purpose:

 - methods `Curl_conn_*(data, conn, sockindex)` work on the complete
   filter chain at `sockindex` and connection `conn`.
 - methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter
   instance.
 - methods `Curl_conn_cf()` work on/with filter instances at a
   connection.
 - rebased and resolved some naming conflicts
 - hostname validation (und session lookup) on SECONDARY use the same
   name as on FIRST (again).

new debug macros and removing connectdata from function signatures where not
needed.

adapting schannel for new Curl_read_plain paramter.

Closes #9919
2022-11-22 14:25:50 +01:00
Viktor Szakats
a8861b6ccd
Makefile.mk: portable Makefile.m32
Update bare GNU Make `Makefile.m32` to:

- Move objects into a subdirectory.
- Add support for MS-DOS. Tested with DJGPP.
- Add support for Watt-32 (on MS-DOS).
- Add support for AmigaOS.
- Rename `Makefile.m32` to `Makefile.mk`
- Replace `ARCH` with `TRIPLET`.
- Build `tool_hugehelp.c` proper (when tools are available).
- Drop MS-DOS compatibility macro `USE_ZLIB` (replaced by `HAVE_LIBZ`)
- Add support for `ZLIB_LIBS` to override `-lz`.
- Omit object files when building examples.
- Default `CC` to `gcc` once again, for convenience. (Caveat: compiler
  name `cc` cannot be set now.)
- Set `-DCURL_NO_OLDIES` for examples, like autotools does.
- Delete `makefile.dj` files. Notice the configuration details and
  defaults are not retained with the new method.
- Delete `makefile.amiga` files. A successful build needs a few custom
  options. We're also not retaining all build details from the existing
  Amiga make files.
- Rename `Makefile.m32` to `Makefile.mk` to reflect that they are not
  Windows/MinGW32-specific anymore.
- Add support for new `CFG` options: `-map`, `-debug`, `-trackmem`
- Set `-DNDEBUG` by default.
- Allow using `-DOS=...` in all `lib/config-*.h` headers, syncing this
  with `config-win32.h`.
- Look for zlib parts in `ZLIB_PATH/include` and `ZLIB_PATH/lib`
  instead of bare `ZLIB_PATH`.

Note that existing build configurations for MS-DOS and AmigaOS likely
become incompatible with this change.

Example AmigaOS configuration:
```
export CROSSPREFIX=/opt/amiga/bin/m68k-amigaos-
export CC=gcc
export CPPFLAGS='-DHAVE_PROTO_BSDSOCKET_H'
export CFLAGS='-mcrt=clib2'
export LDFLAGS="${CFLAGS}"
export LIBS='-lnet -lm'
make -C lib -f Makefile.mk
make -C src -f Makefile.mk
```

Example MS-DOS configuration:
```
export CROSSPREFIX=/opt/djgpp/bin/i586-pc-msdosdjgpp-
export WATT_PATH=/opt/djgpp/net/watt
export ZLIB_PATH=/opt/djgpp
export OPENSSL_PATH=/opt/djgpp
export OPENSSL_LIBS='-lssl -lcrypt'
export CFG=-zlib-ssl
make -C lib -f Makefile.mk
make -C src -f Makefile.mk
```

Closes #9764
2022-11-22 08:28:41 +00:00
Stefan Eissing
4a8b4a1b66
cfiler: filter types have flags indicating what they do
- Adding Curl_conn_is_ip_connected() to check if network connectivity
  has been reached

- having ftp wait for network connectivity before proceeding with
  transfers.

Fixes test failures 1631 and 1632 with hyper.

Closes #9952
2022-11-22 08:20:38 +01:00
Jay Satiro
4f42150d04 sendf: change Curl_read_plain to wrap Curl_recv_plain (take 2)
Prior to this change Curl_read_plain would attempt to read the
socket directly. On Windows that's a problem because recv data may be
cached by libcurl and that data is only drained using Curl_recv_plain.

Rather than rewrite Curl_read_plain to handle cached recv data, I
changed it to wrap Curl_recv_plain, in much the same way that
Curl_write_plain already wraps Curl_send_plain.

Curl_read_plain -> Curl_recv_plain
Curl_write_plain -> Curl_send_plain

This fixes a bug in the schannel backend where decryption of arbitrary
TLS records fails because cached recv data is never drained. We send
data (TLS records formed by Schannel) using Curl_write_plain, which
calls Curl_send_plain, and that may do a recv-before-send
("pre-receive") to cache received data. The code calls Curl_read_plain
to read data (TLS records from the server), which prior to this change
did not call Curl_recv_plain and therefore cached recv data wasn't
retrieved, resulting in malformed TLS records and decryption failure
(SEC_E_DECRYPT_FAILURE).

The bug has only been observed during Schannel TLS 1.3 handshakes. Refer
to the issue and PR for more information.

--

This is take 2 of the original fix. It preserves the original behavior
of Curl_read_plain to write 0 to the bytes read parameter on error,
since apparently some callers expect that (SOCKS tests were hanging).
The original fix which landed in 12e1def5 and was later reverted in
18383fbf failed to work properly because it did not do that.

Also, it changes Curl_write_plain the same way to complement
Curl_read_plain, and it changes Curl_send_plain to return -1 instead of
0 on CURLE_AGAIN to complement Curl_recv_plain.

Behavior on error with these changes:

Curl_recv_plain returns -1 and *code receives error code.
Curl_send_plain returns -1 and *code receives error code.
Curl_read_plain returns error code and *n (bytes read) receives 0.
Curl_write_plain returns error code and *written receives 0.

--

Ref: https://github.com/curl/curl/issues/9431#issuecomment-1312420361

Assisted-by: Joel Depooter
Reported-by: Egor Pugin

Fixes https://github.com/curl/curl/issues/9431
Closes https://github.com/curl/curl/pull/9949
2022-11-20 03:54:36 -05:00
Sean McArthur
8c859cdb69
hyper: classify headers as CONNECT and 1XX
Closes #9947
2022-11-19 13:01:12 +01:00
Stefan Eissing
8ed97ad2bd
ftp: fix "AUTH TLS" on primary conn and for SSL in PASV second conn
Follow-up to dafdb20a26

Reported-by: Anthony Hu
Closes #9948
2022-11-19 12:30:29 +01:00
Stefan Eissing
ff8fc809c5
http: restore h3 to working condition after connection filter introduction
Follow-up to dafdb20a26

HTTP/3 needs a special filter chain, since it does the TLS handling
itself. This PR adds special setup handling in the HTTP protocol handler
that takes are of it.

When a handler, in its setup method, installs filters, the default
behaviour for managing the filter chain is overridden.

Reported-by: Karthikdasari0423 on github

Fixes #9931
Closes #9945
2022-11-19 00:00:27 +01:00
Daniel Stenberg
c7cd781a63
urldata: change port num storage to int and unsigned short
Instead of long.

Closes #9946
2022-11-18 23:54:38 +01:00
Daniel Stenberg
18383fbf72
Revert "sendf: change Curl_read_plain to wrap Curl_recv_plain"
This reverts commit 12e1def51a.

It introduced SOCKS proxy fails, like test 700 never ending.

Reopens #9431
2022-11-18 11:00:29 +01:00
Jay Satiro
12e1def51a sendf: change Curl_read_plain to wrap Curl_recv_plain
Prior to this change Curl_read_plain would attempt to read the
socket directly. On Windows that's a problem because recv data may be
cached by libcurl and that data is only drained using Curl_recv_plain.

Rather than rewrite Curl_read_plain to handle cached recv data, I
changed it to wrap Curl_recv_plain, in much the same way that
Curl_write_plain already wraps Curl_send_plain.

Curl_read_plain -> Curl_recv_plain
Curl_write_plain -> Curl_send_plain

This fixes a bug in the schannel backend where decryption of arbitrary
TLS records fails because cached recv data is never drained. We send
data (TLS records formed by Schannel) using Curl_write_plain, which
calls Curl_send_plain, and that may do a recv-before-send
("pre-receive") to cache received data. The code calls Curl_read_plain
to read data (TLS records from the server), which prior to this change
did not call Curl_recv_plain and therefore cached recv data wasn't
retrieved, resulting in malformed TLS records and decryption failure
(SEC_E_DECRYPT_FAILURE).

The bug has only been observed during Schannel TLS 1.3 handshakes. Refer
to the issue and PR for more information.

Ref: https://github.com/curl/curl/issues/9431#issuecomment-1312420361

Assisted-by: Joel Depooter
Reported-by: Egor Pugin

Fixes https://github.com/curl/curl/issues/9431
Closes https://github.com/curl/curl/pull/9904
2022-11-18 03:04:13 -05:00
Daniel Stenberg
37ca6f0f9a
url: move back the IDN conversion of proxy names
Regression: in commit 53bcf55 we moved the IDN conversion calls to
happen before the HSTS checks. But the HSTS checks are only done on the
server host name, not the proxy names. By moving the proxy name IDN
conversions, we accidentally broke the verbose output showing the proxy
name.

This change moves back the IDN conversions for the proxy names to the
place in the code path they were before 53bcf55.

Reported-by: Andy Stamp
Fixes #9937
Closes #9939
2022-11-18 08:22:05 +01:00