Commit Graph

1826 Commits

Author SHA1 Message Date
Daniel Stenberg
8e85764b7b
lib: remove unused functions, make single-use static
Closes #11174
2023-05-23 11:08:51 +02:00
Emanuele Torre
f198d33e8d
checksrc: disallow spaces before labels
Out of 415 labels throughout the code base, 86 of those labels were
not at the start of the line. Which means labels always at the start of
the line is the favoured style overall with 329 instances.

Out of the 86 labels not at the start of the line:
* 75 were indented with the same indentation level of the following line
* 8 were indented with exactly one space
* 2 were indented with one fewer indentation level then the following
  line
* 1 was indented with the indentation level of the following line minus
  three space (probably unintentional)

Co-Authored-By: Viktor Szakats

Closes #11134
2023-05-18 20:45:04 +02:00
Daniel Stenberg
f1e5d8a731
url: provide better error message when URLs fail to parse
By providing the URL API error message into the error message.

Ref: #11129
Closes #11137
2023-05-18 20:42:17 +02:00
Shohei Maeda
ac5ad52142
url: fix null dispname for --connect-to option
Closes #11106
2023-05-12 17:52:07 +02:00
Daniel Stenberg
21e7e44fb9
url: fix PVS nits
- expression 'hostptr' is always true
- a part of conditional expression is always true: proxypasswd
- expression 'proxyuser' is always true
- avoid multiple Curl_now() calls in allocate_conn

Ref: #10929
Closes #10959
2023-04-14 09:29:33 +02:00
Daniel Stenberg
4cfa5bcc9a
urlapi: cleanups
- move host checks together
- simplify the scheme parser loop and the end of host name parser
- avoid itermediate buffer storing in multiple places
- reduce scope for several variables
- skip the Curl_dyn_tail() call for speed
- detect IPv6 earlier and skip extra checks for such hosts
- normalize directly in dynbuf instead of itermediate buffer
- split out the IPv6 parser into its own funciton
- call the IPv6 parser directly for ipv6 addresses
- remove (unused) special treatment of % in host names
- junkscan() once in the beginning instead of scattered
- make junkscan return error code
- remove unused query management from dedotdotify()
- make Curl_parse_login_details use memchr
- more use of memchr() instead of strchr() and less strlen() calls
- make junkscan check and return the URL length

An optimized build runs one of my benchmark URL parsing programs ~41%
faster using this branch. (compared against the shipped 7.88.1 library
in Debian)

Closes #10935
2023-04-13 08:41:40 +02:00
Daniel Stenberg
712e5f1e7f
CURLPROXY_HTTPS2: for HTTPS proxy that may speak HTTP/2
Setting this proxy type allows curl to negotiate and use HTTP/2 with
HTTPS proxies.

Closes #10900
2023-04-08 00:27:53 +02:00
Daniel Stenberg
5fec927374
url: remove call to Curl_llist_destroy in Curl_close
A list that is created with a NULL "destructor" does not need to be
destroyed. Not calling it is faster than calling it.

Closes #10846
2023-03-27 23:36:16 +02:00
Daniel Stenberg
843b3baa3e
multi: remove PENDING + MSGSENT handles from the main linked list
As they are not driving transfers or any socket activity, the main loop
does not need to iterate over these handles. A performance improvement.

They are instead only held in their own separate lists.

'data->multi' is kept a pointer to the multi handle as long as the easy
handle is actually part of it even when the handle is moved to the
pending/msgsent lists. It needs to know which multi handle it belongs
to, if for example curl_easy_cleanup() is called before the handle is
removed from the multi handle.

Alll 'data->multi' pointers of handles still part of the multi handle
gets cleared by curl_multi_cleanup() which "orphans" all previously
attached easy handles.

This is take 2. The first version was reverted for the 8.0.1 release.

Assisted-by: Stefan Eissing
Closes #10801
2023-03-26 17:43:28 +02:00
Daniel Stenberg
e5e8815532
lib: skip Curl_llist_destroy calls
Linked lists themselves do not carry any allocations, so for the lists
that do not have have a set destructor we can just skip the
Curl_llist_destroy() call and save CPU time.

Closes #10764
2023-03-15 00:10:21 +01:00
Daniel Stenberg
8f4608468b
ftp: add more conditions for connection reuse
Reported-by: Harry Sintonen
Closes #10730
2023-03-13 09:07:23 +01:00
Daniel Stenberg
cb49e67303
url: only reuse connections with same GSS delegation
Reported-by: Harry Sintonen
Closes #10731
2023-03-10 23:30:34 +01:00
Daniel Stenberg
f384d40718
url: remove dummy protocol handler
Just two added checks were needed saves a whole handler struct.

Closes #10727
2023-03-10 23:19:20 +01:00
Daniel Stenberg
af369db4d3
url: fix the SSH connection reuse check
Reported-by: Harry Sintonen
Closes #10735
2023-03-10 17:54:32 +01:00
Jay Satiro
b559ef6f36 url: fix cookielist memleak when curl_easy_reset
- Free set.cookelist in Curl_freeset instead of Curl_close.

Prior to this change the cookielist linked list wasn't freed by
curl_easy_reset which calls Curl_freeset to free all set.

Bug: https://github.com/curl/curl/issues/10694#issuecomment-1458619157
Reported-by: Sergey Ryabinin

Closes https://github.com/curl/curl/pull/10709
2023-03-10 03:49:16 -05:00
Stefan Eissing
7c5637b8b4
url: fix logic in connection reuse to deny reuse on "unclean" connections
- add parameter to `conn_is_alive()` cfilter method that returns
  if there is input data waiting on the connection
- refrain from re-using connnection from the cache that have
  input pending
- adapt http/2 and http/3 alive checks to digest pending input
  to check the connection state
- remove check_cxn method from openssl as that was just doing
  what the socket filter now does.
- add tests for connection reuse with special server configs

Closes #10690
2023-03-07 12:50:31 +01:00
Stefan Eissing
821f6e2a89
CURLOPT_PIPEWAIT: allow waited reuse also for subsequent connections
As tested in test_02_07, when firing off 200 urls with --parallel, 199
wait for the first connection to be established. if that is multiuse,
urls are added up to its capacity.

The first url over capacity opens another connection. But subsequent
urls found the same situation and open a connection too. They should
have waited for the second connection to actually connect and make its
capacity known.

This change fixes that by

- setting `connkeep()` early in the HTTP setup handler. as otherwise
  a new connection is marked as closeit by default and not considered
  for multiuse at all
- checking the "connected" status for a candidate always and continuing
  to PIPEWAIT if no alternative is found.

pytest:
- removed "skip" from test_02_07
- added test_02_07b to check that http/1.1 continues to work as before

Closes #10456
2023-02-21 11:12:48 +01:00
Stefan Eissing
e187d69e48
urldata: remove now from struct SingleRequest - not needed
Closes #10549
2023-02-17 23:23:33 +01:00
Pronyushkin Petr
690c43b3ee
url: fix part of conditional expression is always true
Closes #10407
2023-02-06 08:51:31 +01:00
Stefan Eissing
5651a36d1a
cf-socket: improvements in socket I/O handling
- Curl_write_plain/Curl_read_plain have been eliminated. Last code use
  now uses Curl_conn_send/recv so that requests use conn->send/revc
  callbacks which defaults to cfilters use.
- Curl_recv_plain/Curl_send_plain have been internalized in cf-socket.c.
- USE_RECV_BEFORE_SEND_WORKAROUND (active on Windows) has been moved
  into cf-socket.c. The pre_recv buffer is held at the socket filter
  context.  `postponed_data` structures have been removed from
  `connectdata`.
- the hanger in HTTP/2 request handling was a result of read buffering
  on all sends and the multi handling is not prepared for this. The
  following happens:

   - multi preforms on a HTTP/2 easy handle
   - h2 reads and processes data
   - this leads to a send of h2 data
   - which receives and buffers before the send
   - h2 returns
   - multi selects on the socket, but no data arrives (its in the buffer already)
   the workaround now receives data in a loop as long as there is something in
   the buffer. The real fix would be for multi to change, so that `data_pending`
   is evaluated before deciding to wait on the socket.

io_buffer, optional, in cf-socket.c, http/2 sets state.drain if lower
filter have pending data.

This io_buffer is only available/used when the
-DUSE_RECV_BEFORE_SEND_WORKAROUND is active, e.g. on Windows
configurations. It also maintains the original checks on protocol
handler being HTTP and conn->send/recv not being replaced.

The HTTP/2 (nghttp2) cfilter now sets data->state.drain when it finds
out that the "lower" filter chain has still pending data at the end of
its IO operation. This prevents the processing from becoming stalled.

Closes #10280
2023-01-31 11:23:59 +01:00
Stefan Eissing
f8da4f2f2d vtls: fix hostname handling in filters
- Copy the hostname and dispname to ssl_connect_data.

Use a copy instead of referencing the `connectdata` instance since this
may get free'ed on connection reuse.

Reported-by: Stefan Talpalaru
Reported-by: sergio-nsk@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/10273
Fixes https://github.com/curl/curl/issues/10309

Closes https://github.com/curl/curl/pull/10310
2023-01-20 00:40:18 -05:00
Daniel Stenberg
7ad8a7ba9e
noproxy: support for space-separated names is deprecated
To be removed in July 2024.

Assisted-by: Michael Osipov
Fixes #10209
Closes #10215
2023-01-04 09:15:37 +01:00
Andrei Rybak
bb393e521f
lib: fix typos in comments which repeat a word
Remove erroneously duplicated words in code comments of files
`lib.connect.c` and `lib/url.c`.

Closes #10220
2023-01-04 09:12:58 +01:00
Stefan Eissing
6a8d7ef981
cf-socket: keep sockaddr local in the socket filters
- copy `struct Curl_addrinfo` on filter setup into context
- remove `struct Curl_addrinfoi *` with `struct Curl_sockaddr_ex *` in
  connectdata that is set and NULLed by the socket filter
- this means we have no reference to the resolver info in connectdata or
  its filters
- trigger the CF_CTRL_CONN_INFO_UPDATE event when the complete filter
  chain reaches connected status
- update easy handle connection information on CF_CTRL_DATA_SETUP event.

Closes #10213
2023-01-03 17:33:29 +01:00
Daniel Stenberg
2bc1d775f5
copyright: update all copyright lines and remove year ranges
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205
2023-01-03 09:19:21 +01:00
Daniel Stenberg
becfe2ec78
urldata: cease storing TLS auth type
The only TLS auth type libcurl ever supported is SRP and that is the
default type. Since nobody ever sets any other type, there is no point
in wasting space to store the set type and code to check the type.

If TLS auth is used, SRP is now implied.

Closes #10181
2022-12-31 16:56:21 +01:00
Stefan Eissing
71b7e01610
lib: connect/h2/h3 refactor
Refactoring of connection setup and happy eyeballing. Move
nghttp2. ngtcp2, quiche and msh3 into connection filters.

 - eyeballing cfilter that uses sub-filters for performing parallel connects
 - socket cfilter for all transport types, including QUIC
 - QUIC implementations in cfilter, can now participate in eyeballing
 - connection setup is more dynamic in order to adapt to what filter did
   really connect.  Relevant to see if a SSL filter needs to be added or
   if SSL has already been provided
 - HTTP/3 test cases similar to HTTP/2
 - multiuse of parallel transfers for HTTP/3, tested for ngtcp2 and quiche

 - Fix for data attach/detach in VTLS filters that could lead to crashes
   during parallel transfers.
 - Eliminating setup() methods in cfilters, no longer needed.
 - Improving Curl_conn_is_alive() to replace Curl_connalive() and
   integrated ssl alive checks into cfilter.
 - Adding CF_CNTRL_CONN_INFO_UPDATE to tell filters to update
   connection into and persist it at the easy handle.

 - Several more cfilter related cleanups and moves:
   - stream_weigth and dependency info is now wrapped in struct
     Curl_data_priority
   - Curl_data_priority members depend is available in HTTP2|HTTP3
   - Curl_data_priority members depend on NGHTTP2 support
   - handling init/reset/cleanup of priority part of url.c
   - data->state.priority same struct, but shallow copy for compares only

 - PROTOPT_STREAM has been removed
   - Curl_conn_is_mulitplex() now available to check on capability

 - Adding query method to connection filters.
   - ngtcp2+quiche: implementing query for max concurrent transfers.

 - Adding is_alive and keep_alive cfilter methods. Adding DATA_SETUP event.
   - setting keepalive timestamp on connect
   - DATA_SETUP is called after the connection has been completely
     setup (but may not connected yet) to allow filters to initialize
     data members they use.

 - there is no socket to be had with msh3, it is unclear how select
   shall work

 - manual test via "curl --http3 https://curl.se" fail with "empty
   reply from server".

 - Various socket/conn related cleanups:
   - Curl_socket is now Curl_socket_open and in cf-socket.c
   - Curl_closesocket is now Curl_socket_close and in cf-socket.c
   - Curl_ssl_use has been replaced with Cur_conn_is_ssl
   - Curl_conn_tcp_accepted_set has been split into
     Curl_conn_tcp_listen_set and Curl_conn_tcp_accepted_set
     with a clearer purpose

Closes #10141
2022-12-30 16:43:19 +01:00
Daniel Stenberg
076a2f6291
share: add sharing of HSTS cache among handles
Closes #10138
2022-12-27 15:22:01 +01:00
Hannah Schierling
a5cf4a87cf
url: fix build with --disable-cookies
Struct `UserDefined` has no member `cookielist` if
`CURL_DISABLE_COOKIES` is defined.

Follow-up to af5999a

Closes #10158
2022-12-26 09:27:39 +01:00
Daniel Stenberg
db07301fbb
urldata: remove unused struct fields, made more conditional
- source_quote, source_prequote and source_postquote have not been used since
  5e0d9aea3; September 2006

- make several fields conditional on proxy support

- make three quote struct fields conditional on FTP || SSH

- make 'mime_options' depend on MIME

- make trailer_* fields depend on HTTP

- change 'gssapi_delegation' from long to unsigned char

- make 'localportrange' unsigned short instead of int

- conn->trailer now depends on HTTP

Closes #10147
2022-12-23 08:45:05 +01:00
Daniel Stenberg
f4883ba966
urldata: make set.http200aliases conditional on HTTP being present
And make a few SSH-only fields depend on SSH

Closes #10140
2022-12-22 23:07:46 +01:00
Daniel Stenberg
af5999a674
urldata: move the cookefilelist to the 'set' struct
The cookiefile entries are set into the handle and should remain set for
the lifetime of the handle so that duplicating it also duplicates the
list. Therefore, the struct field is moved from 'state' to 'set'.

Fixes #10133
Closes #10134
2022-12-22 16:06:37 +01:00
Daniel Stenberg
cc0aaf6e82
idn: rename the files to idn.[ch] and hold all IDN functions
Closes #10094
2022-12-15 22:56:58 +01:00
Daniel Stenberg
301812bec2
idn: remove Curl_win32_ascii_to_idn
It was not used. Introduce a new IDN header for the prototype(s).

Closes #10094
2022-12-15 22:56:52 +01:00
Patrick Monnerat
eb559c8056
mime: relax easy/mime structures binding
Deprecation and removal of codeset conversion support from the library
have released the strict need for an early binding of mime structures to
an easy handle (https://github.com/curl/curl/commit/2610142).

This constraint currently forces to create the handle before the mime
structure and the latter cannot be attached to another handle once
created (see https://curl.se/mail/lib-2022-08/0027.html).

This commit removes the handle pointers from the mime structures
allowing more flexibility on their use.

When an easy handle is duplicated, bound mime structures must however
still be duplicated too as their components hold send-time dynamic
information.

Closes #9927
2022-11-28 09:09:54 +01:00
Stefan Eissing
f18956d0ca
cfilter: re-add conn as parameter to cfilter setup methods
- `Curl_ssl_get_config()` now returns the first config if no SSL proxy
  filter is active

- socket filter starts connection only on first invocation of its
  connect method

Fixes #9982
Closes #9983
2022-11-26 16:56:46 +01:00
Stefan Eissing
af22c2a546
vtls: localization of state data in filters
- almost all backend calls pass the Curl_cfilter intance instead of
   connectdata+sockindex
 - ssl_connect_data is remove from struct connectdata and made internal
   to vtls
 - ssl_connect_data is allocated in the added filter, kept at cf->ctx

 - added function to let a ssl filter access its ssl_primary_config and
   ssl_config_data this selects the propert subfields in conn and data,
   for filters added as plain or proxy
 - adjusted all backends to use the changed api
 - adjusted all backends to access config data via the exposed
   functions, no longer using conn or data directly

cfilter renames for clear purpose:

 - methods `Curl_conn_*(data, conn, sockindex)` work on the complete
   filter chain at `sockindex` and connection `conn`.
 - methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter
   instance.
 - methods `Curl_conn_cf()` work on/with filter instances at a
   connection.
 - rebased and resolved some naming conflicts
 - hostname validation (und session lookup) on SECONDARY use the same
   name as on FIRST (again).

new debug macros and removing connectdata from function signatures where not
needed.

adapting schannel for new Curl_read_plain paramter.

Closes #9919
2022-11-22 14:25:50 +01:00
Daniel Stenberg
37ca6f0f9a
url: move back the IDN conversion of proxy names
Regression: in commit 53bcf55 we moved the IDN conversion calls to
happen before the HSTS checks. But the HSTS checks are only done on the
server host name, not the proxy names. By moving the proxy name IDN
conversions, we accidentally broke the verbose output showing the proxy
name.

This change moves back the IDN conversions for the proxy names to the
place in the code path they were before 53bcf55.

Reported-by: Andy Stamp
Fixes #9937
Closes #9939
2022-11-18 08:22:05 +01:00
Alexandre Ferrieux
49798cac83
CURLOPT_QUICK_EXIT: don't wait for DNS thread on exit
Fixes #2975
Closes #9147
2022-11-18 00:17:27 +01:00
Daniel Stenberg
b7e4c41aa3
url: only acknowledge fresh_reuse for non-followed transfers
... to make sure NTLM auth sticks to the connection it needs, as
verified by 2032.

Follow-up to fa0b922761

Assisted-by: Stefan Eissing
Closes #9905
2022-11-14 14:53:15 +01:00
Daniel Stenberg
bf12c2bed6
lib: remove bad set.opt_no_body assignments
This struct field MUST remain what the application set it to, so that
handle reuse and handle duplication work.

Instead, the request state bit 'no_body' is introduced for code flows
that need to change this in run-time.

Closes #9888
2022-11-11 15:56:05 +01:00
Stefan Eissing
dafdb20a26
lib: connection filters (cfilter) addition to curl:
- general construct/destroy in connectdata
 - default implementations of callback functions
 - connect: cfilters for connect and accept
 - socks: cfilter for socks proxying
 - http_proxy: cfilter for http proxy tunneling
 - vtls: cfilters for primary and proxy ssl
 - change in general handling of data/conn
 - Curl_cfilter_setup() sets up filter chain based on data settings,
   if none are installed by the protocol handler setup
 - Curl_cfilter_connect() boot straps filters into `connected` status,
   used by handlers and multi to reach further stages
 - Curl_cfilter_is_connected() to check if a conn is connected,
   e.g. all filters have done their work
 - Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE
   indicators for multi select to work
 - Curl_cfilter_data_pending() asks filters if the have incoming
   data pending for recv
 - Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks
   installed in conn->recv/conn->send for io handling
 - Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters
   and addition/removal of a `data` from their connection
 - adding vtl functions to prevent use of Curl_ssl globals directly
   in other parts of the code.

Reviewed-by: Daniel Stenberg
Closes #9855
2022-11-11 15:17:51 +01:00
Michael Drake
1fdca35ddd
curl.h: add CURLOPT_CA_CACHE_TIMEOUT option
Adds a new option to control the maximum time that a cached
certificate store may be retained for.

Currently only the OpenSSL backend implements support for
caching certificate stores.

Closes #9620
2022-11-08 10:06:12 +01:00
Daniel Stenberg
3f039dfd6f
strcase: use curl_str(n)equal for case insensitive matches
No point in having two entry points for the same functions.

Also merged the *safe* function treatment into these so that they can
also be used when one or both pointers are NULL.

Closes #9837
2022-11-01 17:01:26 +01:00
Ayesh Karunaratne
4484270afc
misc: typo and grammar fixes
- Replace `Github` with `GitHub`.
- Replace `windows` with `Windows`
- Replace `advice` with `advise` where a verb is used.
- A few fixes on removing repeated words.
- Replace `a HTTP` with `an HTTP`

Closes #9802
2022-10-27 10:01:30 +02:00
Viktor Szakats
811c799f2d
cmake: really enable warnings with clang
Even though `PICKY_COMPILER=ON` is the default, warnings were not
enabled when using llvm/clang, because `CMAKE_COMPILER_IS_CLANG` was
always false (in my tests at least).

This is the single use of this variable in curl, and in a different
place we already use `CMAKE_C_COMPILER_ID MATCHES "Clang"`, which works
as expected, so change the condition to use that instead.

Also fix the warnings uncovered by the above:

- lib: add casts to silence clang warnings

- schannel: add casts to silence clang warnings in ALPN code

  Assuming the code is correct, solve the warnings with a cast.
  This particular build case isn't CI tested.

  There is a chance the warning is relevant for some platforms, perhaps
  Windows 32-bit ARM7.

Closes #9783
2022-10-26 09:56:52 +00:00
Daniel Stenberg
55e1875729
http_proxy: restore the protocol pointer on error
Reported-by: Trail of Bits

Closes #9790
2022-10-24 15:30:25 +02:00
Daniel Stenberg
53bcf55b45
url: use IDN decoded names for HSTS checks
Reported-by: Hiroki Kurosawa

Closes #9791
2022-10-24 12:00:54 +02:00
Daniel Stenberg
1e9a538e05
noproxy: support proxies specified using cidr notation
For both IPv4 and IPv6 addresses. Now also checks IPv6 addresses "correctly"
and not with string comparisons.

Split out the noproxy checks and functionality into noproxy.c

Added unit test 1614 to verify checking functions.

Reported-by: Mathieu Carbonneaux

Fixes #9773
Fixes #5745
Closes #9775
2022-10-21 13:39:20 +02:00
Daniel Stenberg
57e2bb52aa
netrc: remove the two 'changed' arguments
As no user of these functions used the returned content.
2022-10-13 23:42:07 +02:00
Daniel Stenberg
a2aa980886
netrc: use the URL-decoded user
When the user name is provided in the URL it is URL encoded there, but
when used for authentication the encoded version should be used.

Regression introduced after 7.83.0

Reported-by: Jonas Haag
Fixes #9709
Closes #9715
2022-10-13 23:42:07 +02:00
Daniel Stenberg
ed5095ed94
strcase: add and use Curl_timestrcmp
This is a strcmp() alternative function for comparing "secrets",
designed to take the same time no matter the content to not leak
match/non-match info to observers based on how fast it is.

The time this function takes is only a function of the shortest input
string.

Reported-by: Trail of Bits

Closes #9658
2022-10-08 11:50:47 +02:00
Daniel Stenberg
99d3682303
url: rename function due to name-clash in Watt-32
Follow-up to 2481dbe5f4 and applies the change the way it was
intended.
2022-09-27 09:34:22 +02:00
Viktor Szakats
92b9624a68
windows: adjust name of two internal public functions
According to `docs/INTERNALS.md`, internal function names spanning source
files start with uppercase `Curl_`. Bring these two functions in
alignment with this.

This also stops exporting them from `libcurl.dll` in autotools builds.

Reviewed-by: Daniel Stenberg

Closes #9598
2022-09-26 22:32:23 +00:00
Daniel Stenberg
1a87a1efba
url: a zero-length userinfo part in the URL is still a (blank) user
Adjusted test 1560 to verify

Reported-by: Jay Satiro

Fixes #9088
Closes #9590
2022-09-26 07:45:53 +02:00
Patrick Monnerat
91e06e687a
lib: prepare the incoming of additional protocols
Move the curl_prot_t to its own conditional block. Introduce symbol
PROTO_TYPE_SMALL to control it.

Fix a cast in a curl_prot_t assignment.
Remove an outdated comment.

Follow-up to cd5ca80.

Closes #9534
2022-09-23 08:39:44 +02:00
Patrick Monnerat
9d51329047
setopt: use the handler table for protocol name to number conversions
This also returns error CURLE_UNSUPPORTED_PROTOCOL rather than
CURLE_BAD_FUNCTION_ARGUMENT when a listed protocol name is not found.

A new schemelen parameter is added to Curl_builtin_scheme() to support
this extended use.

Note that disabled protocols are not recognized anymore.

Tests adapted accordingly.

Closes #9472
2022-09-16 23:29:01 +02:00
Daniel Stenberg
846678541b
urlapi: detect scheme better when not guessing
When the parser is not allowed to guess scheme, it should consider the
word ending at the first colon to be the scheme, independently of number
of slashes.

The parser now checks that the scheme is known before it counts slashes,
to improve the error messge for URLs with unknown schemes and maybe no
slashes.

When following redirects, no scheme guessing is allowed and therefore
this change effectively prevents redirects to unknown schemes such as
"data".

Fixes #9503
2022-09-15 09:31:40 +02:00
Daniel Stenberg
cd5ca80f00
urldata: use a curl_prot_t type for storing protocol bits
This internal-use-only storage type can be bumped to a curl_off_t once
we need to use bit 32 as the previous 'unsigned int' can no longer hold
them all then.

The websocket protocols take bit 30 and 31 so they are the last ones
that fit within 32 bits - but cannot properly be exported through APIs
since those use *signed* 32 bit types (long) in places.

Closes #9481
2022-09-13 11:26:49 +02:00
Daniel Stenberg
664249d095
ws: initial websockets support
Closes #8995
2022-09-09 15:11:14 +02:00
Daniel Stenberg
f703cf971c
urlapi: leaner with fewer allocs
Slightly faster with more robust code. Uses fewer and smaller mallocs.

- remove two fields from the URL handle struct
- reduce copies and allocs
- use dynbuf buffers more instead of custom malloc + copies
- uses dynbuf to build the host name in reduces serial alloc+free within
  the same function.
- move dedotdotify into urlapi.c and make it static, not strdup the input
  and optimize it by checking for . and / before using strncmp
- remove a few strlen() calls
- add Curl_dyn_setlen() that can "trim" an existing dynbuf

Closes #9408
2022-09-07 10:21:45 +02:00
Daniel Stenberg
8dd95da35b
ctype: remove all use of <ctype.h>, use our own versions
Except in the test servers.

Closes #9433
2022-09-06 08:32:36 +02:00
Daniel Stenberg
472f1cbe7e
NPN: remove support for and use of
Next Protocol Negotiation is a TLS extension that was created and used
for agreeing to use the SPDY protocol (the precursor to HTTP/2) for
HTTPS. In the early days of HTTP/2, before the spec was finalized and
shipped, the protocol could be enabled using this extension with some
servers.

curl supports the NPN extension with some TLS backends since then, with
a command line option `--npn` and in libcurl with
`CURLOPT_SSL_ENABLE_NPN`.

HTTP/2 proper is made to use the ALPN (Application-Layer Protocol
Negotiation) extension and the NPN extension has no purposes
anymore. The HTTP/2 spec was published in May 2015.

Today, use of NPN in the wild should be extremely rare and most likely
totally extinct. Chrome removed NPN support in Chrome 51, shipped in
June 2016. Removed in Firefox 53, April 2017.

Closes #9307
2022-09-05 07:39:02 +02:00
Viktor Szakats
c9061f242b
misc: spelling fixes
Found using codespell 2.2.1.

Also delete the redundant protocol designator from an archive.org URL.

Reviewed-by: Daniel Stenberg
Closes #9403
2022-08-31 14:31:01 +00:00
Orgad Shaneh
c40ec3178f
url: really use the user provided in the url when netrc entry exists
If the user is specified as part of the URL, and the same user exists
in .netrc, Authorization header was not sent at all.

The user and password fields were assigned in conn->user and password
but the user was not assigned to data->state.aptr, which is the field
that is used in output_auth_headers and friends.

Fix by assigning the user also to aptr.

Amends commit d1237ac906.

Fixes #9243
2022-08-29 17:25:29 +02:00
Jay Satiro
8bd03516d6
url: treat missing usernames in netrc as empty
- If, after parsing netrc, there is a password with no username then
  set a blank username.

This used to be the case prior to 7d600ad (precedes 7.82). Note
parseurlandfillconn already does the same thing for URLs.

Reported-by: Raivis <standsed@users.noreply.github.com>
Testing-by: Domen Kožar

Fixes https://github.com/curl/curl/issues/8653
Closes #9334
Closes #9066
2022-08-29 17:24:22 +02:00
Emil Engler
2c86f1b345
url: output the maximum when rejecting a url
This commit changes the failf message to output the maximum length, when
curl refuses to process a URL because it is too long.

See: #9317
Closes: #9327
2022-08-22 23:36:02 +02:00
Daniel Stenberg
37dbbbb6c1
Curl_close: call Curl_resolver_cancel to avoid memory-leak
There might be a pending (c-ares) resolve that isn't free'd up yet.

Closes #9310
2022-08-16 08:54:30 +02:00
Daniel Stenberg
657101ec0c
url: reject URLs with hostnames longer than 65535 bytes
It *probably* causes other problems too since DNS can't resolve such
long names, but the SNI field in TLS is limited to 16 bits length.

Closes #9317
2022-08-15 10:52:26 +02:00
Emil Engler
764c958c52
refactor: split resolve_server() into functions
This commit splits the branch-heavy resolve_server() function into
various sub-functions, in order to reduce the amount of nested
if/else-statements.

Beside this, it also removes many else-sequences, by returning in the
previous if-statement.

Closes #9283
2022-08-12 16:27:36 +02:00
Daniel Stenberg
0f23341953
connect: set socktype/protocol correctly
So that an address used from the DNS cache that was previously used for
QUIC can be reused for TCP and vice versa.

To make this possible, set conn->transport to "unix" for unix domain
connections ... and store the transport struct field in an unsigned char
to use less space.

Reported-by: ウさん
Fixes #9274
Closes #9276
2022-08-08 13:26:11 +02:00
Daniel Stenberg
3be0c67379
url: enumerated type mixed with another type
Follow-up to 1c58e7ae99ce2030213f28b

Closes #9179
2022-07-23 13:39:06 +02:00
Daniel Stenberg
fe93bb4d1f
doh: move doh related struct definitions to doh.h
and make 'dnstype' in 'struct dnsprobe' use the DNStype to fix the icc compiler warning:

  doh.c(924): error #188: enumerated type mixed with another type

Reported-by: Matthew Thompson
Ref #9156
Closes #9174
2022-07-18 11:55:26 +02:00
Daniel Stenberg
1d85d2e4f9
urldata: make 'use_port' an usigned short
... instead of a long. It is already enforced to not attempt to set any
value outside of 16 bits unsigned.

Closes #9099
2022-07-04 19:42:40 +02:00
max.mehl
ad9bc5976d
copyright: make repository REUSE compliant
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869
2022-06-13 09:13:00 +02:00
Daniel Stenberg
90a7017acb
url: URL encode the path when extracted, if spaces were set 2022-06-08 16:32:46 +02:00
Daniel Stenberg
45de940ceb
lib: make more protocol specific struct fields #ifdefed
... so that they don't take up space if the protocols are disabled in
the build.

Closes #8944
2022-06-01 14:31:49 +02:00
Daniel Stenberg
4651945822
urldata: reduce size of a few struct fields
When the values are never larger than 32 bit, ints are better than longs.

Closes #8940
2022-06-01 08:12:09 +02:00
Daniel Stenberg
b1c1df0b6b
urldata: remove three unused booleans from struct UserDefined
- is_fwrite_set
- free_referer
- strip_path_slash

Closes #8940
2022-06-01 08:12:09 +02:00
Daniel Stenberg
eeaae10c0f
netrc: support quoted strings
The .netrc parser now accepts strings within double-quotes in order to
deal with for example passwords containing white space - which
previously was not possible.

A password that starts with a double-quote also ends with one, and
double-quotes themselves are escaped with backslashes, like \". It also
supports \n, \r and \t for newline, carriage return and tabs
respectively.

If the password does not start with a double quote, it will end at first
white space and no escaping is performed.

WARNING: this change is not entirely backwards compatible. If anyone
previously used a double-quote as the first letter of their password,
the parser will now get it differently compared to before. This is
highly unfortunate but hard to avoid.

Reported-by: ImpatientHippo on GitHub
Fixes #8908
Closes #8937
2022-05-31 09:04:56 +02:00
Balakrishnan Balasubramanian
dfa84a0450
socks: support unix sockets for socks proxy
Usage:
  curl -x "socks5h://localhost/run/tor/socks" "https://example.com"

Updated runtests.pl to run a socksd server listening on unix socket

Added tests test1467 test1468

Added documentation for proxy command line option and socks proxy
options

Closes #8668
2022-05-19 15:35:03 +02:00
Daniel Stenberg
8d86718f3f
opts: deprecate RANDOM_FILE and EGDSOCKET
These two options were only ever used for the OpenSSL backend for
versions before 1.1.0. They were never used for other backends and they
are not used with recent OpenSSL versions. They were never used much by
applications.

The defines RANDOM_FILE and EGD_SOCKET can still be set at build-time
for ancient EOL OpenSSL versions.

Closes #8670
2022-05-17 11:05:54 +02:00
Daniel Stenberg
06d12105c7
url: free old conn better on reuse
Make use of conn_free() better and avoid duplicate code.

Reported-by: Andrea Pappacoda
Fixes #8841
Closes #8842
2022-05-14 18:05:20 +02:00
Kamil Dudka
9494cdc3d2 url: remove redundant #ifdefs in allocate_conn()
No change in behavior intended by this commit.
2022-05-11 15:51:29 +02:00
Daniel Stenberg
1645e9b445
url: check SSH config match on connection reuse
CVE-2022-27782

Reported-by: Harry Sintonen
Bug: https://curl.se/docs/CVE-2022-27782.html
Closes #8825
2022-05-09 23:13:53 +02:00
Daniel Stenberg
f18af4f874
tls: check more TLS details for connection reuse
CVE-2022-27782

Reported-by: Harry Sintonen
Bug: https://curl.se/docs/CVE-2022-27782.html
Closes #8825
2022-05-09 23:13:53 +02:00
Adam Rosenfield
e07a9b668a
conn: fix typo 'connnection' -> 'connection' in two function names
Closes #8759
2022-04-27 11:50:13 +02:00
Daniel Stenberg
030adbceeb
conncache: remove name arg from Curl_conncache_find_bundle
To simplify, and also since the returned name is not the full actual
name used for the check. The port number and zone id is also involved,
so just showing the name is misleading.

Closes #8750
2022-04-25 16:25:42 +02:00
Patrick Monnerat
852aa5ad35
url: check sasl additional parameters for connection reuse.
Also move static function safecmp() as non-static Curl_safecmp() since
its purpose is needed at several places.

Bug: https://curl.se/docs/CVE-2022-22576.html

CVE-2022-22576

Closes #8746
2022-04-25 11:44:05 +02:00
Daniel Stenberg
6968fb9d54
lib: remove exclamation marks
... from infof() and failf() calls. Make them less attention seeking.

Closes #8713
2022-04-16 11:55:05 +02:00
Daniel Stenberg
b716b5aa7e
lib: #ifdef on USE_HTTP2 better
... as nghttp2 might not be the library that provides HTTP/2 support.

Closes #8661
2022-04-01 13:45:52 +02:00
Daniel Stenberg
d1e4a67734
header api: add curl_easy_header and curl_easy_nextheader
Add test 1940 to 1946 to verify.

Closes #8593
2022-03-22 08:24:22 +01:00
MAntoniak
ccc2752ce8
ssl: reduce allocated space for ssl backend when FTP is disabled
Add assert() for the backend pointer in many places

Closes #8471
2022-02-21 08:39:55 +01:00
Daniel Stenberg
7d600ad1c3
urldata: remove conn->bits.user_passwd
The authentication status should be told by the transfer and not the
connection.

Reported-by: John H. Ayad
Fixes #8449
Closes #8451
2022-02-16 10:28:31 +01:00
MAntoniak
06eb208126
misc: remove unused data when IPv6 is not supported
Closes #8430
2022-02-13 12:08:50 +01:00
Daniel Stenberg
c3331a029c
url: exclude zonefrom_url when no ipv6 is available
Closes #8439
2022-02-11 23:27:47 +01:00
Daniel Stenberg
2610142139
lib: remove support for CURL_DOES_CONVERSIONS
TPF was the only user and support for that was dropped.

Closes #8378
2022-02-04 08:05:35 +01:00
Daniel Stenberg
91e3996aa7
urldata: CONN_IS_PROXIED replaces bits.close when proxy can be disabled
To remove run-time checks for such builds.

Closes #8350
2022-01-30 12:01:25 +01:00
Daniel Stenberg
b27ad8e1d3
url: revert the removal of trailing dot from host name
Reverts 5de8d84098 (May 2014, shipped in 7.37.0) and the
follow-up changes done afterward.

Keep the dot in names for everything except the SNI to make curl behave
more similar to current browsers. This means 'name' and 'name.' send the
same SNI for different 'Host:' headers.

Updated test 1322 accordingly

Fixes #8290
Reported-by: Charles Cazabon
Closes #8320
2022-01-27 08:35:18 +01:00
lucas
519b62ccd0
docs: update IETF links to use datatracker
The tools.ietf.org domain has been deprecated a while now, with the
links being redirected to datatracker.ietf.org.

Rather than make people eat that redirect time, this change switches the
URL to a more canonical source.

Closes #8317
2022-01-21 23:06:36 +01:00
Daniel Stenberg
e74a6b7b72
url: make Curl_disconnect return void
1. The function would only ever return CURLE_OK anyway
 2. Only one caller actually used the return code
 3. Most callers did (void)Curl_disconnect()

Closes #8303
2022-01-20 09:07:30 +01:00
Daniel Stenberg
d1237ac906
url: given a user in the URL, find pwd for that user in netrc
Add test 380 and 381 to verify, edited test 133

Reported-by: Manfred Schwarb
Fixes #8241
Closes #8243
2022-01-09 14:16:41 +01:00
MAntoniak
1f31bdb10b
url: reduce ssl backend count for CURL_DISABLE_PROXY builds
Closes #8212
2022-01-02 22:53:47 +01:00
Daniel Stenberg
21248e052d
checksrc: detect more kinds of NULL comparisons we avoid
Co-authored-by: Jay Satiro
Closes #8180
2021-12-27 23:39:26 +01:00
MAntoniak
39406280bb
misc: remove unused doh flags when CURL_DISABLE_DOH is defined
Closes #8148
2021-12-15 08:01:21 +01:00
MrDwZ
1cdfdcde3b check ssl_config when re-use proxy connection 2021-12-14 14:46:10 +01:00
Daniel Stenberg
a5f5687368
urlapi: make Curl_is_absolute_url always use MAX_SCHEME_LEN
Instad of having all callers pass in the maximum length, always use
it. The passed in length is instead used only as the length of the
target buffer for to storing the scheme name in, if used.

Added the scheme max length restriction to the curl_url_set.3 man page.

Follow-up to 45bcb2eaa7

Closes #8047
2021-11-25 08:33:48 +01:00
Daniel Stenberg
0942017162
url.c: fix the SIGPIPE comment for Curl_close
Closes #7984
2021-11-10 14:50:28 +01:00
x2018
d55b8b6a45
url: check the return value of curl_url()
Closes #7917
2021-10-28 15:22:19 +02:00
Eddie Lumpkin
8e701cc978
lib: fixing comment spelling typos in lib files
Closes #7894
Signed-off-by: ewlumpkin <ewlumpkin@gmail.com>
2021-10-22 22:53:44 +02:00
Daniel Stenberg
bffe90cf2e
url: set "k->size" -1 at start of request
The size of the transfer is unknown at that point.

Fixes #7871
Closes #7872
2021-10-18 16:27:02 +02:00
Jeffrey Tolar
5f563495f1
CURLOPT_MAXLIFETIME_CONN: maximum allowed lifetime for conn reuse
... and close connections that are too old instead of reusing them.

By default, this behavior is disabled.

Bug: https://curl.se/mail/lib-2021-09/0058.html
Closes #7751
2021-10-06 14:38:59 +02:00
i-ky
3363eeb262
urlapi: add curl_url_strerror()
Add curl_url_strerror() to convert CURLUcode into readable string and
facilitate easier troubleshooting in programs using URL API.
Extend CURLUcode with CURLU_LAST for iteration in unit tests.
Update man pages with a mention of new function.
Update example code and tests with new functionality where it fits.

Closes #7605
2021-09-27 08:28:46 +02:00
Daniel Stenberg
e41e1b2a4d
url: fix compiler warning in no-verbose builds
Follow-up from 2f0bb864c1

Closes #7700
2021-09-10 19:10:14 +02:00
Daniel Stenberg
2f0bb864c1
lib: don't use strerror()
We have and provide Curl_strerror() internally for a reason: strerror()
is not necessarily thread-safe so we should always try to avoid it.

Extended checksrc to warn for this, but feature the check disabled by
default and only enable it in lib/

Closes #7685
2021-09-09 07:53:42 +02:00
Daniel Stenberg
5ea3145850
vtls: fix connection reuse checks for issuer cert and case sensitivity
CVE-2021-22924

Reported-by: Harry Sintonen
Bug: https://curl.se/docs/CVE-2021-22924.html
2021-07-20 09:36:06 +02:00
MAntoniak
9a47d77126
lib: fix compiler warnings with CURL_DISABLE_NETRC
warning C4189: 'netrc_user_changed': local variable is initialized but
not referenced

warning C4189: 'netrc_passwd_changed': local variable is initialized but
not referenced

Closes #7423
2021-07-18 18:37:05 +02:00
Daniel Stenberg
e7416cfd2b
infof: remove newline from format strings, always append it
- the data needs to be "line-based" anyway since it's also passed to the
  debug callback/application

- it makes infof() work like failf() and consistency is good

- there's an assert that triggers on newlines in the format string

- Also removes a few instances of "..."

- Removes the code that would append "..." to the end of the data *iff*
  it was truncated in infof()

Closes #7357
2021-07-07 22:54:01 +02:00
Daniel Stenberg
d696ee00ee
lib: use %u instead of %ld for port number printf
Follow-up to 764c6bd3bf which changed the type of some port number
fields. Detected by Coverity (CID 1486624) etc.

Closes #7325
2021-06-30 23:25:35 +02:00
Daniel Stenberg
81cc2e48da
url: (void)-prefix a curl_url_get() call
Coverity (CID 1486645) pointed out a use of curl_url_get() in the
parse_proxy function where the return code wasn't checked. A
(void)-prefix makes the intention obvious.

Closes #7320
2021-06-30 14:58:47 +02:00
Andrei Rybak
278b46751e misc: fix typos in comments which repeat a word
Fix typos in code comments which repeat various words.  In trivial
cases, just delete the repeated word.  Reword the affected sentence in
"lib/url.c" for it to make sense.

Closes #7303
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
2021-06-28 12:41:56 +02:00
Richard Whitehouse
0842175fa4
multi: alter transfer timeout ordering
- Check whether a connection has succeded before checking whether it's
  timed out.

  This means if we've connected quickly, but subsequently been
  descheduled, we allow the connection to succeed. Note, if we timeout,
  but between checking the timeout, and connecting to the server the
  connection succeeds, we will allow it to go ahead. This is viewed as
  an acceptable trade off.

- Add additional failf logging around failed connection attempts to
  propogate the cause up to the caller.

Co-Authored-by: Martin Howarth
Closes #7178
2021-06-24 15:51:39 +02:00
Viktor Szakats
2026124691
idn: fix libidn2 with windows unicode builds
Unicode Windows builds use UTF-8 strings internally in libcurl,
so make sure to call the UTF-8 flavour of the libidn2 API. Also
document that Windows builds with libidn2 and UNICODE do expect
CURLOPT_URL as an UTF-8 string.

Reported-by: dEajL3kA on github
Assisted-by: Jay Satiro
Reviewed-by: Marcel Raad
Closes #7246
Fixes #7228
2021-06-15 12:10:48 +00:00
Daniel Stenberg
73f52ad763
url.c: remove two variable assigns that are never read
Pointed out by scan-build

Closes #7248
2021-06-13 23:42:46 +02:00
Daniel Stenberg
14a2ca85ec
conn_shutdown: if closed during CONNECT cleanup properly
Reported-by: Alex Xu
Reported-by: Phil E. Taylor

Fixes #7236
Closes #7237
2021-06-11 12:43:53 +02:00
Alexis Vachette
d8dcb399b8
url: bad CURLOPT_CONNECT_TO syntax now returns error
Added test 3020 to verify

Closes #7183
2021-06-04 14:08:30 +02:00
Daniel Stenberg
1c1d9f1aff
hsts: ignore numberical IP address hosts
Also, use a single function library-wide for detecting if a given hostname is
a numerical IP address.

Reported-by: Harry Sintonen
Fixes #7146
Closes #7149
2021-05-30 19:49:40 +02:00
Lucas Clemente Vella
84d2839740
CURLOPT_IPRESOLVE: preventing wrong IP version from being used
In some situations, it was possible that a transfer was setup to
use an specific IP version, but due do DNS caching or connection
reuse, it ended up using a different IP version from requested.

This commit changes the effect of CURLOPT_IPRESOLVE from simply
restricting address resolution to preventing the wrong connection
type being used, when choosing a connection from the pool, and
to restricting what addresses could be used when establishing
a new connection.

It is important that all addresses versions are resolved, even if
not used in that transfer in particular, because the result is
cached, and could be useful for a different transfer with a
different CURLOPT_IPRESOLVE setting.

Closes #6853
2021-05-20 16:58:31 +02:00
Daniel Stenberg
0c55fbab45
conn: add 'attach' to protocol handler, make libssh2 use it
The libssh2 backend has SSH session associated with the connection but
the callback context is the easy handle, so when a connection gets
attached to a transfer, the protocol handler now allows for a custom
function to get used to set things up correctly.

Reported-by: Michael O'Farrell
Fixes #6898
Closes #7078
2021-05-17 17:57:50 +02:00
Gilles Vollant
77fc3859b2 SSL: support in-memory CA certs for some backends
- New options CURLOPT_CAINFO_BLOB and CURLOPT_PROXY_CAINFO_BLOB to
  specify in-memory PEM certificates for OpenSSL, Schannel (Windows)
  and Secure Transport (Apple) SSL backends.

Prior to this change PEM certificates could only be imported from a file
and not from memory.

Co-authored-by: moparisthebest@users.noreply.github.com

Ref: https://github.com/curl/curl/pull/4679
Ref: https://github.com/curl/curl/pull/5677
Ref: https://github.com/curl/curl/pull/6109

Closes https://github.com/curl/curl/pull/6662
2021-05-05 02:29:16 -04:00
Daniel Stenberg
063d3f3b96
tidy-up: make conditional checks more consistent
... remove '== NULL' and '!= 0'

Closes #6912
2021-04-22 09:10:17 +02:00
Daniel Stenberg
9cb48457c6
ConnectionExists: respect requests for h1 connections better
... for situations when multiplexing isn't enabled on the h2 connection
and h1 is explicitly requested for the transfer.

Assisted-by: Gergely Nagy
2021-04-21 08:20:24 +02:00
Daniel Stenberg
d71ff2b9db
hsts: enable by default
No longer considered experimental.

Closes #6700
2021-04-19 08:22:16 +02:00
Daniel Stenberg
95cbcec8f9
urldata: merge "struct DynamicStatic" into "struct UrlState"
Both were used for the same purposes and there was no logical separation
between them. Combined, this also saves 16 bytes in less holes in my
test build.

Closes #6798
2021-03-26 23:19:20 +01:00
Daniel Stenberg
78f642ffab
config: remove CURL_SIZEOF_CURL_OFF_T use only SIZEOF_CURL_OFF_T
Make the code consistently use a single name for the size of the
"curl_off_t" type.

Closes #6702
2021-03-11 10:11:56 +01:00
Daniel Gustafsson
313faa1bcb url: fix typo in comment
Correct a small typo which snuck in with a304051620.
2021-02-28 21:13:26 +01:00
Daniel Stenberg
70472a44de
urldata: remove the _ORIG suffix from string names
It doesn't provide any useful info but only makes the names longer.

Closes #6624
2021-02-19 08:07:19 +01:00
Daniel Stenberg
d25c479e84
url: fix memory leak if OOM in the HSTS handling
Reported-by: Viktor Szakats
Bug: https://github.com/curl/curl/pull/6627#issuecomment-781626205

Closes #6628
2021-02-19 08:05:40 +01:00
Daniel Stenberg
6174fc993a
url.c: use consistent error message for failed resolve 2021-02-18 11:10:36 +01:00
Daniel Stenberg
f248a13065
parse_proxy: fix a memory leak in the OOM path
Reported-by: Jay Satiro
Reviewed-by: Jay Satiro
Reviewed-by: Emil Engler

Closes #6614
Bug: https://github.com/curl/curl/pull/6591#issuecomment-780396541
2021-02-17 22:54:42 +01:00
Jay Satiro
568190f493 url: fix possible use-after-free in default protocol
Prior to this change if the user specified a default protocol and a
separately allocated non-absolute URL was used then it was freed
prematurely, before it was then used to make the replacement URL.

Bug: https://github.com/curl/curl/issues/6604#issuecomment-780138219
Reported-by: arvids-kokins-bidstack@users.noreply.github.com

Closes https://github.com/curl/curl/pull/6613
2021-02-17 16:15:11 -05:00
Daniel Stenberg
cfff12a0b3
lib: remove 'conn->data' completely
The Curl_easy pointer struct entry in connectdata is now gone. Just
before commit 215db086e0 landed on January 8, 2021 there were 919
references to conn->data.

Closes #6608
2021-02-16 10:48:57 +01:00
Jay Satiro
53022e1893 doh: add options to disable ssl verification
- New libcurl options CURLOPT_DOH_SSL_VERIFYHOST,
  CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the
  same as their respective counterparts.

- New curl tool options --doh-insecure and --doh-cert-status do the same
  as their respective counterparts.

Prior to this change DOH SSL certificate verification settings for
verifyhost and verifypeer were supposed to be inherited respectively
from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug
were not. As a result DOH verification remained at the default, ie
enabled, and it was not possible to disable. This commit changes
behavior so that the DOH verification settings are independent and not
inherited.

Ref: https://github.com/curl/curl/pull/4579#issuecomment-554723676

Fixes https://github.com/curl/curl/issues/4578
Closes https://github.com/curl/curl/pull/6597
2021-02-14 18:20:48 -05:00
Daniel Stenberg
46620b9743
http: use credentials from transfer, not connection
HTTP auth "accidentally" worked before this cleanup since the code would
always overwrite the connection credentials with the credentials from
the most recent transfer and since HTTP auth is typically done first
thing, this has not been an issue. It was still wrong and subject to
possible race conditions or future breakage if the sequence of functions
would change.

The data.set.str[] strings MUST remain unmodified exactly as set by the
user, and the credentials to use internally are instead set/updated in
state.aptr.*

Added test 675 to verify different credentials used in two requests done
over a reused HTTP connection, which previously behaved wrongly.

Fixes #6542
Closes #6545
2021-02-13 22:36:15 +01:00
Daniel Stenberg
88dd1a8a11
urldata: don't touch data->set.httpversion at run-time
Rename it to 'httpwant' and make a cloned field in the state struct as
well for run-time updates.

Also: refuse non-supported HTTP versions. Verified with test 129.

Closes #6585
2021-02-12 08:13:37 +01:00
Jay Satiro
cb2dc1ba89 lib: use int type for more port variables
This is a follow-up to 764c6bd. Prior to that change port variables
were usually type long.

Closes https://github.com/curl/curl/pull/6553
2021-02-09 02:53:31 -05:00
Daniel Stenberg
1c3def53c7
urldata: remove duplicate 'upkeep_interval_ms' from connectdata
... and rely only on the value already set in Curl_easy.

Closes #6534
2021-01-27 09:19:08 +01:00
Daniel Stenberg
d6a37c23a3
urldata: remove 'local_ip' from the connectdata struct
As the info is already stored in the transfer handle anyway, there's no
need to carry around a duplicate buffer for the life-time of the handle.

Closes #6534
2021-01-27 09:19:05 +01:00
Daniel Stenberg
764c6bd3bf
urldata: remove duplicate port number storage
... and use 'int' for ports. We don't use 'unsigned short' since -1 is
still often used internally to signify "unknown value" and 0 - 65535 are
all valid port numbers.

Closes #6534
2021-01-27 09:19:01 +01:00
Daniel Stenberg
642d78026f
urldata: remove the duplicate 'ip_addr_str' field
... as the numerical IP address is already stored and kept in 'primary_ip'.

Closes #6534
2021-01-27 09:18:51 +01:00