When there are filters addded for both socket and SSL, the code
previously checked the SSL sockets during connect when it *should* first
check the socket layer until that has connected.
Fixes#10157Fixes#10146Closes#10160
Reviewed-by: Stefan Eissing
As is supported by recent libssh2, but should just be ignored by older
versions.
Reported-by: norbertmm on github
Assisted-by: norbertmm on github
Fixes#10143Closes#10145
The message "Mark bundle as not supporting multiuse" was added at commit
29364d93 when an http/2-related bug was fixed, and it appears to be a
leftover trace message.
This message should be removed because:
* it conveys no information to the user
* it is enabled in the default build (--enable-verbose)
* it reads like a warning/unexpected condition
* it is equivalent to "Detected http proto < 2", which is
not a useful message.
* it is a time-wasting red-herring for anyone who encounters
it for the first time while investigating some other, real
problem.
This commit removes the trace message "Mark bundle as not
supporting multiuse"
Closes#10159
- source_quote, source_prequote and source_postquote have not been used since
5e0d9aea3; September 2006
- make several fields conditional on proxy support
- make three quote struct fields conditional on FTP || SSH
- make 'mime_options' depend on MIME
- make trailer_* fields depend on HTTP
- change 'gssapi_delegation' from long to unsigned char
- make 'localportrange' unsigned short instead of int
- conn->trailer now depends on HTTP
Closes#10147
The cookiefile entries are set into the handle and should remain set for
the lifetime of the handle so that duplicating it also duplicates the
list. Therefore, the struct field is moved from 'state' to 'set'.
Fixes#10133Closes#10134
When checking if there is a "secure context", which it is if the
connection is to localhost even if the protocol is HTTP, the comparison
for ::1 was done incorrectly and included brackets.
Reported-by: BratSinot on github
Fixes#10120Closes#10121
Otherwise it stores the info HSTS into the persistent cache for the IDN
name which will not match when the HSTS status is later checked for
using the decoded name.
Reported-by: Hiroki Kurosawa
Closes#10111
Follow-up to ac612dfeee
strtoul() accepts a leading minus so better make sure there is none
Extended test 356 somewhat to use a huge negative 64 bit number that
otherwise becomes a low positive number.
Closes#10095
When utf8asn1str fails there is no allocation returned, so freeing
the return pointer in **to is at best a no-op and at worst a double-
free bug waiting to happen. The current coding isn't hiding any such
bugs but to future proof, avoid freeing the return value pointer iff
the function failed.
Closes: #10087
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- curl_ws_send returns CURLE_SEND_ERROR if data->conn is gone
- curl_ws_recv returns CURLE_GOT_NOTHING on connection close
- curl_ws_recv.3: mention new return code for connection close + example
embryo
Closes#10084
Change "__MWERKS__" to "macintosh". When this block was originally added
in 3ac6929 it was probably intended to handle classic Mac OS since the
previous classic Mac OS build procedure for curl (which was removed in
bf327a9) used Metrowerks CodeWarrior.
But there are other classic Mac OS compilers, such as the MPW compilers,
that were not handled by this case. For classic Mac OS,
CURL_TYPEOF_CURL_SOCKLEN_T needs to match what's provided by the
third-party GUSI library, which does not vary by compiler.
Meanwhile CodeWarrior works on platforms other than classic Mac OS, and
they may need different definitions. Separate blocks could be added
later for any of those platforms that curl doesn't already support.
Closes#10049
When checking for a noproxy setting we need to ensure that we get
a hostname passed in. If there is no hostname then there cannot be
a matching noproxy rule for it by definition.
Closes: #10057
Reported-by: Geeknik Labs
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- fixes `Curl_ssl_cf_get_ssl()` to detect also the first filter instance
as ssl (refs #10053)
- replaces `Curl_ssl_use()` with the correct `Curl_conn_is_ssl()`
Closes#10054Fixes#10053
Reported-by: Patrick Monnerat
Fix various uses of connnect by replacing them with connect.
Closes: #10045
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
BIO_read and BIO_write return negative numbers on error, including
retryable ones. A regression from 55807e6. Both branches should be
returning -1.
The APIs are patterned after POSIX read and write which, similarly,
return -1 on errors, not zero, with EAGAIN treated as an error.
Bug: https://github.com/curl/curl/issues/10013#issuecomment-1335308146
Reported-by: David Benjamin
Closes#10021
Partially reverts "somewhat protect Mac OS X users from using Mac OS 9
config file", commit 62519bfe05.
Do things that are specific to classic Mac OS (i.e. include config-mac.h
in curl_setup.h and rename "main" to "curl_main" in tool_setup.h) when
only "macintosh" is defined. Remove the additional condition that
"__MRC__" should be defined since that would only be true with the MPW
MrC compiler which prevents the use of other reasonable compilers like
the MPW SC compiler and especially the Metrowerks CodeWarrior compilers.
"macintosh" is only defined by classic Mac OS compilers so this change
should not affect users of Mac OS X / OS X / macOS / any other OS.
Closes#10037
Replace the hack of using `$() ` to represent a single space. The new
method silences the `--warn-undefined-variables` debug warning and it's
also a better-known form of solving this problem.
Reviewed-by: Jay Satiro
Closes#10031
- Fix `NROFF` auto-detection with certain shell/make-build combinations:
When a non-MSYS2 GNU Make runs inside an MSYS2 shell, Make executes
the detection command as-is via `CreateProcess()`. It fails because
`command` is an `sh` built-in. Ensure to explicitly invoke the shell.
- Initialize user-customizable variables:
Silences a list of warnings when running GNU Make with the option
`--warn-undefined-variables`. Another benefit is that it's now easy
to look up all user-customizable `Makefile.mk` variables by grepping
for ` ?=` in the curl source tree.
Suggested-by: Gisle Vanem
Ref: https://github.com/curl/curl/pull/9764#issuecomment-1330674433
- Fix `MKDIR` invocation:
Avoid a warning and potential issue in envs without forward-slash
support.
Closes#10000
Deprecation and removal of codeset conversion support from the library
have released the strict need for an early binding of mime structures to
an easy handle (https://github.com/curl/curl/commit/2610142).
This constraint currently forces to create the handle before the mime
structure and the latter cannot be attached to another handle once
created (see https://curl.se/mail/lib-2022-08/0027.html).
This commit removes the handle pointers from the mime structures
allowing more flexibility on their use.
When an easy handle is duplicated, bound mime structures must however
still be duplicated too as their components hold send-time dynamic
information.
Closes#9927
When using the option CURLOPT_IGNORE_CONTENT_LENGTH (set.ignorecl in
code) to support growing files in FTP, the code should ignore the
initial size it gets from the server as this will not be the final size
of the file. This is done in ftp_state_quote() to prevent a size request
being issued in the initial sequence. However, in a later call to
ftp_state_get_resp() the code attempts to get the size of the content
again if it doesn't already have it, by parsing the response from the
RETR request. This fix prevents this parsing of the response to get the
size when the set.ignorecl option is set. This should maintain the size
value as -1, unknown, in this situation.
Closes#9772
- `Curl_ssl_get_config()` now returns the first config if no SSL proxy
filter is active
- socket filter starts connection only on first invocation of its
connect method
Fixes#9982Closes#9983
`Curl_output_aws_sigv4()` doesn't always have the whole payload in
memory to generate a real payload hash. this commit allows the user to
pass in a header like `x-amz-content-sha256` to provide their desired
payload hash
some services like s3 require this header, and may support other values
like s3's `UNSIGNED-PAYLOAD` and `STREAMING-AWS4-HMAC-SHA256-PAYLOAD`
with special semantics. servers use this header's value as the payload
hash during signature validation, so it must match what the client uses
to generate the signature
CURLOPT_AWS_SIGV4.3 now describes the content-sha256 interaction
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Closes#9804
This makes a big difference for cases when the rewind is not actually
necessary to perofm (for example HTTP response code 301 converts to GET)
and therefore the rewind can be avoided. In particular for situations
when that rewind fails, for example when reading from a pipe or similar.
Reported-by: Ali Utku Selen
Fixes#9735Closes#9958
In non-IPv6 builds the conn parameter is unused, and compilers which
run with "-Werror=unused-parameter" (or similar) warnings turned on
fails to build. Below is an excerpt from a CI job:
vtls/openssl.c: In function ‘Curl_ossl_verifyhost’:
vtls/openssl.c:2016:75: error: unused parameter ‘conn’ [-Werror=unused-parameter]
2016 | CURLcode Curl_ossl_verifyhost(struct Curl_easy *data, struct connectdata *conn,
| ~~~~~~~~~~~~~~~~~~~~^~~~
Closes: #9970
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Commit 3b16575ae9 removed support for
building on Novell Netware, but a few leftover traces remained. This
removes the last bits.
Closes: #9966
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- almost all backend calls pass the Curl_cfilter intance instead of
connectdata+sockindex
- ssl_connect_data is remove from struct connectdata and made internal
to vtls
- ssl_connect_data is allocated in the added filter, kept at cf->ctx
- added function to let a ssl filter access its ssl_primary_config and
ssl_config_data this selects the propert subfields in conn and data,
for filters added as plain or proxy
- adjusted all backends to use the changed api
- adjusted all backends to access config data via the exposed
functions, no longer using conn or data directly
cfilter renames for clear purpose:
- methods `Curl_conn_*(data, conn, sockindex)` work on the complete
filter chain at `sockindex` and connection `conn`.
- methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter
instance.
- methods `Curl_conn_cf()` work on/with filter instances at a
connection.
- rebased and resolved some naming conflicts
- hostname validation (und session lookup) on SECONDARY use the same
name as on FIRST (again).
new debug macros and removing connectdata from function signatures where not
needed.
adapting schannel for new Curl_read_plain paramter.
Closes#9919
Update bare GNU Make `Makefile.m32` to:
- Move objects into a subdirectory.
- Add support for MS-DOS. Tested with DJGPP.
- Add support for Watt-32 (on MS-DOS).
- Add support for AmigaOS.
- Rename `Makefile.m32` to `Makefile.mk`
- Replace `ARCH` with `TRIPLET`.
- Build `tool_hugehelp.c` proper (when tools are available).
- Drop MS-DOS compatibility macro `USE_ZLIB` (replaced by `HAVE_LIBZ`)
- Add support for `ZLIB_LIBS` to override `-lz`.
- Omit object files when building examples.
- Default `CC` to `gcc` once again, for convenience. (Caveat: compiler
name `cc` cannot be set now.)
- Set `-DCURL_NO_OLDIES` for examples, like autotools does.
- Delete `makefile.dj` files. Notice the configuration details and
defaults are not retained with the new method.
- Delete `makefile.amiga` files. A successful build needs a few custom
options. We're also not retaining all build details from the existing
Amiga make files.
- Rename `Makefile.m32` to `Makefile.mk` to reflect that they are not
Windows/MinGW32-specific anymore.
- Add support for new `CFG` options: `-map`, `-debug`, `-trackmem`
- Set `-DNDEBUG` by default.
- Allow using `-DOS=...` in all `lib/config-*.h` headers, syncing this
with `config-win32.h`.
- Look for zlib parts in `ZLIB_PATH/include` and `ZLIB_PATH/lib`
instead of bare `ZLIB_PATH`.
Note that existing build configurations for MS-DOS and AmigaOS likely
become incompatible with this change.
Example AmigaOS configuration:
```
export CROSSPREFIX=/opt/amiga/bin/m68k-amigaos-
export CC=gcc
export CPPFLAGS='-DHAVE_PROTO_BSDSOCKET_H'
export CFLAGS='-mcrt=clib2'
export LDFLAGS="${CFLAGS}"
export LIBS='-lnet -lm'
make -C lib -f Makefile.mk
make -C src -f Makefile.mk
```
Example MS-DOS configuration:
```
export CROSSPREFIX=/opt/djgpp/bin/i586-pc-msdosdjgpp-
export WATT_PATH=/opt/djgpp/net/watt
export ZLIB_PATH=/opt/djgpp
export OPENSSL_PATH=/opt/djgpp
export OPENSSL_LIBS='-lssl -lcrypt'
export CFG=-zlib-ssl
make -C lib -f Makefile.mk
make -C src -f Makefile.mk
```
Closes#9764
- Adding Curl_conn_is_ip_connected() to check if network connectivity
has been reached
- having ftp wait for network connectivity before proceeding with
transfers.
Fixes test failures 1631 and 1632 with hyper.
Closes#9952
Prior to this change Curl_read_plain would attempt to read the
socket directly. On Windows that's a problem because recv data may be
cached by libcurl and that data is only drained using Curl_recv_plain.
Rather than rewrite Curl_read_plain to handle cached recv data, I
changed it to wrap Curl_recv_plain, in much the same way that
Curl_write_plain already wraps Curl_send_plain.
Curl_read_plain -> Curl_recv_plain
Curl_write_plain -> Curl_send_plain
This fixes a bug in the schannel backend where decryption of arbitrary
TLS records fails because cached recv data is never drained. We send
data (TLS records formed by Schannel) using Curl_write_plain, which
calls Curl_send_plain, and that may do a recv-before-send
("pre-receive") to cache received data. The code calls Curl_read_plain
to read data (TLS records from the server), which prior to this change
did not call Curl_recv_plain and therefore cached recv data wasn't
retrieved, resulting in malformed TLS records and decryption failure
(SEC_E_DECRYPT_FAILURE).
The bug has only been observed during Schannel TLS 1.3 handshakes. Refer
to the issue and PR for more information.
--
This is take 2 of the original fix. It preserves the original behavior
of Curl_read_plain to write 0 to the bytes read parameter on error,
since apparently some callers expect that (SOCKS tests were hanging).
The original fix which landed in 12e1def5 and was later reverted in
18383fbf failed to work properly because it did not do that.
Also, it changes Curl_write_plain the same way to complement
Curl_read_plain, and it changes Curl_send_plain to return -1 instead of
0 on CURLE_AGAIN to complement Curl_recv_plain.
Behavior on error with these changes:
Curl_recv_plain returns -1 and *code receives error code.
Curl_send_plain returns -1 and *code receives error code.
Curl_read_plain returns error code and *n (bytes read) receives 0.
Curl_write_plain returns error code and *written receives 0.
--
Ref: https://github.com/curl/curl/issues/9431#issuecomment-1312420361
Assisted-by: Joel Depooter
Reported-by: Egor Pugin
Fixes https://github.com/curl/curl/issues/9431
Closes https://github.com/curl/curl/pull/9949
Follow-up to dafdb20a26
HTTP/3 needs a special filter chain, since it does the TLS handling
itself. This PR adds special setup handling in the HTTP protocol handler
that takes are of it.
When a handler, in its setup method, installs filters, the default
behaviour for managing the filter chain is overridden.
Reported-by: Karthikdasari0423 on github
Fixes#9931Closes#9945
Prior to this change Curl_read_plain would attempt to read the
socket directly. On Windows that's a problem because recv data may be
cached by libcurl and that data is only drained using Curl_recv_plain.
Rather than rewrite Curl_read_plain to handle cached recv data, I
changed it to wrap Curl_recv_plain, in much the same way that
Curl_write_plain already wraps Curl_send_plain.
Curl_read_plain -> Curl_recv_plain
Curl_write_plain -> Curl_send_plain
This fixes a bug in the schannel backend where decryption of arbitrary
TLS records fails because cached recv data is never drained. We send
data (TLS records formed by Schannel) using Curl_write_plain, which
calls Curl_send_plain, and that may do a recv-before-send
("pre-receive") to cache received data. The code calls Curl_read_plain
to read data (TLS records from the server), which prior to this change
did not call Curl_recv_plain and therefore cached recv data wasn't
retrieved, resulting in malformed TLS records and decryption failure
(SEC_E_DECRYPT_FAILURE).
The bug has only been observed during Schannel TLS 1.3 handshakes. Refer
to the issue and PR for more information.
Ref: https://github.com/curl/curl/issues/9431#issuecomment-1312420361
Assisted-by: Joel Depooter
Reported-by: Egor Pugin
Fixes https://github.com/curl/curl/issues/9431
Closes https://github.com/curl/curl/pull/9904
Regression: in commit 53bcf55 we moved the IDN conversion calls to
happen before the HSTS checks. But the HSTS checks are only done on the
server host name, not the proxy names. By moving the proxy name IDN
conversions, we accidentally broke the verbose output showing the proxy
name.
This change moves back the IDN conversions for the proxy names to the
place in the code path they were before 53bcf55.
Reported-by: Andy Stamp
Fixes#9937Closes#9939
Field feature_names contains a null-terminated sorted array of feature
names. Bitmask field features is deprecated.
Documentation is updated. Test 1177 and tests/version-scan.pl updated to
match new documentation format and extended to check feature names too.
Closes#9583
- buffers updated correctly when handling partial frames
- callbacks no longer invoked for incomplete payload data of 0 length
- curl_ws_recv no longer returns with 0 length partial payload
Closes#9890
The previously set default value of 8 (64-bit) is only correct for
mingw-w64 and only when we set `_FILE_OFFSET_BITS` to 64 (the default
when building curl). For MSVC, old MinGW and other Windows compilers,
the correct value is 4 (32-bit). Adjust condition accordingly. Also
drop the manual override option.
Regression in 7.86.0 (from 68fa9bf3f5)
Bug: https://github.com/curl/curl/pull/9712#issuecomment-1307330551
Reported-by: Peter Piekarski
Reviewed-by: Jay Satiro
Closes#9872
This struct field MUST remain what the application set it to, so that
handle reuse and handle duplication work.
Instead, the request state bit 'no_body' is introduced for code flows
that need to change this in run-time.
Closes#9888
- general construct/destroy in connectdata
- default implementations of callback functions
- connect: cfilters for connect and accept
- socks: cfilter for socks proxying
- http_proxy: cfilter for http proxy tunneling
- vtls: cfilters for primary and proxy ssl
- change in general handling of data/conn
- Curl_cfilter_setup() sets up filter chain based on data settings,
if none are installed by the protocol handler setup
- Curl_cfilter_connect() boot straps filters into `connected` status,
used by handlers and multi to reach further stages
- Curl_cfilter_is_connected() to check if a conn is connected,
e.g. all filters have done their work
- Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE
indicators for multi select to work
- Curl_cfilter_data_pending() asks filters if the have incoming
data pending for recv
- Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks
installed in conn->recv/conn->send for io handling
- Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters
and addition/removal of a `data` from their connection
- adding vtl functions to prevent use of Curl_ssl globals directly
in other parts of the code.
Reviewed-by: Daniel Stenberg
Closes#9855
Unlike `CONNECT`, currently we don't keep track whether `PROXY` is
already sent or not. This causes `PROXY` header to be sent twice during
`MSTATE_TUNNELING` and `MSTATE_PROTOCONNECT`.
Closes#9878Fixes#9442
Adds a new option to control the maximum time that a cached
certificate store may be retained for.
Currently only the OpenSSL backend implements support for
caching certificate stores.
Closes#9620
