Commit Graph

1070 Commits

Author SHA1 Message Date
Stefan Eissing
af22c2a546
vtls: localization of state data in filters
- almost all backend calls pass the Curl_cfilter intance instead of
   connectdata+sockindex
 - ssl_connect_data is remove from struct connectdata and made internal
   to vtls
 - ssl_connect_data is allocated in the added filter, kept at cf->ctx

 - added function to let a ssl filter access its ssl_primary_config and
   ssl_config_data this selects the propert subfields in conn and data,
   for filters added as plain or proxy
 - adjusted all backends to use the changed api
 - adjusted all backends to access config data via the exposed
   functions, no longer using conn or data directly

cfilter renames for clear purpose:

 - methods `Curl_conn_*(data, conn, sockindex)` work on the complete
   filter chain at `sockindex` and connection `conn`.
 - methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter
   instance.
 - methods `Curl_conn_cf()` work on/with filter instances at a
   connection.
 - rebased and resolved some naming conflicts
 - hostname validation (und session lookup) on SECONDARY use the same
   name as on FIRST (again).

new debug macros and removing connectdata from function signatures where not
needed.

adapting schannel for new Curl_read_plain paramter.

Closes #9919
2022-11-22 14:25:50 +01:00
Stefan Eissing
ff8fc809c5
http: restore h3 to working condition after connection filter introduction
Follow-up to dafdb20a26

HTTP/3 needs a special filter chain, since it does the TLS handling
itself. This PR adds special setup handling in the HTTP protocol handler
that takes are of it.

When a handler, in its setup method, installs filters, the default
behaviour for managing the filter chain is overridden.

Reported-by: Karthikdasari0423 on github

Fixes #9931
Closes #9945
2022-11-19 00:00:27 +01:00
Stefan Eissing
a8e6351e12
proxy: refactor haproxy protocol handling as connection filter
Closes #9893
2022-11-15 15:56:25 +01:00
Daniel Stenberg
fa0b922761
http: mark it 'this_is_a_follow' in the Location: logic
To make regular auth "reloads" to not count as redirects.

Verified by test 3101

Fixes #9885
Closes #9887
2022-11-12 00:12:46 +01:00
Daniel Stenberg
bf12c2bed6
lib: remove bad set.opt_no_body assignments
This struct field MUST remain what the application set it to, so that
handle reuse and handle duplication work.

Instead, the request state bit 'no_body' is introduced for code flows
that need to change this in run-time.

Closes #9888
2022-11-11 15:56:05 +01:00
Stefan Eissing
dafdb20a26
lib: connection filters (cfilter) addition to curl:
- general construct/destroy in connectdata
 - default implementations of callback functions
 - connect: cfilters for connect and accept
 - socks: cfilter for socks proxying
 - http_proxy: cfilter for http proxy tunneling
 - vtls: cfilters for primary and proxy ssl
 - change in general handling of data/conn
 - Curl_cfilter_setup() sets up filter chain based on data settings,
   if none are installed by the protocol handler setup
 - Curl_cfilter_connect() boot straps filters into `connected` status,
   used by handlers and multi to reach further stages
 - Curl_cfilter_is_connected() to check if a conn is connected,
   e.g. all filters have done their work
 - Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE
   indicators for multi select to work
 - Curl_cfilter_data_pending() asks filters if the have incoming
   data pending for recv
 - Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks
   installed in conn->recv/conn->send for io handling
 - Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters
   and addition/removal of a `data` from their connection
 - adding vtl functions to prevent use of Curl_ssl globals directly
   in other parts of the code.

Reviewed-by: Daniel Stenberg
Closes #9855
2022-11-11 15:17:51 +01:00
Fata Nugraha
11ad25fff4
http: do not send PROXY more than once
Unlike `CONNECT`, currently we don't keep track whether `PROXY` is
already sent or not. This causes `PROXY` header to be sent twice during
`MSTATE_TUNNELING` and `MSTATE_PROTOCONNECT`.

Closes #9878
Fixes #9442
2022-11-10 12:24:48 +01:00
Daniel Stenberg
2bc04d4980
rtsp: fix RTSP auth
Verified with test 3100

Fixes #4750
Closes #9870
2022-11-09 09:40:00 +01:00
Rose
f151ec6c10 lib: fix some type mismatches and remove unneeded typecasts
Many of these castings are unneeded if we change the variables to work
better with each other.

Ref: https://github.com/curl/curl/pull/9823

Closes https://github.com/curl/curl/pull/9835
2022-11-08 03:11:01 -05:00
Daniel Stenberg
52cc4a85fd
style: use space after comment start and before comment end
/* like this */

/*not this*/

checksrc is updated accordingly

Closes #9828
2022-10-30 22:31:29 +01:00
Ayesh Karunaratne
4484270afc
misc: typo and grammar fixes
- Replace `Github` with `GitHub`.
- Replace `windows` with `Windows`
- Replace `advice` with `advise` where a verb is used.
- A few fixes on removing repeated words.
- Replace `a HTTP` with `an HTTP`

Closes #9802
2022-10-27 10:01:30 +02:00
Shaun Mirani
6efb6b1e77
url: allow non-HTTPS HSTS-matching for debug builds
Closes #9728
2022-10-13 23:28:43 +02:00
Daniel Stenberg
b46136f9b1
http: try parsing Retry-After: as a number first
Since the date parser allows YYYYMMDD as a date format (due to it being
a bit too generic for parsing this particular header), a large integer
number could wrongly match that pattern and cause the parser to generate
a wrong value.

No date format accepted for this header starts with a decimal number, so
by reversing the check and trying a number first we can deduct that if
that works, it was not a date.

Reported-by Trail of Bits

Closes #9718
2022-10-13 17:27:08 +02:00
Patrick Monnerat
72652c0613
http, vauth: always provide Curl_allow_auth_to_host() functionality
This function is currently located in the lib/http.c module and is
therefore disabled by the CURL_DISABLE_HTTP conditional token.

As it may be called by TLS backends, disabling HTTP results in an
undefined reference error at link time.

Move this function to vauth/vauth.c to always provide it and rename it
as Curl_auth_allowed_to_host() to respect the vauth module naming
convention.

Closes #9600
2022-09-27 14:05:37 +02:00
Daniel Stenberg
660cf3d4ed
lib: the number four in a sequence is the "fourth"
Spelling is hard

Closes #9535
2022-09-19 08:48:49 +02:00
Daniel Stenberg
664249d095
ws: initial websockets support
Closes #8995
2022-09-09 15:11:14 +02:00
Daniel Stenberg
472f1cbe7e
NPN: remove support for and use of
Next Protocol Negotiation is a TLS extension that was created and used
for agreeing to use the SPDY protocol (the precursor to HTTP/2) for
HTTPS. In the early days of HTTP/2, before the spec was finalized and
shipped, the protocol could be enabled using this extension with some
servers.

curl supports the NPN extension with some TLS backends since then, with
a command line option `--npn` and in libcurl with
`CURLOPT_SSL_ENABLE_NPN`.

HTTP/2 proper is made to use the ALPN (Application-Layer Protocol
Negotiation) extension and the NPN extension has no purposes
anymore. The HTTP/2 spec was published in May 2015.

Today, use of NPN in the wild should be extremely rare and most likely
totally extinct. Chrome removed NPN support in Chrome 51, shipped in
June 2016. Removed in Firefox 53, April 2017.

Closes #9307
2022-09-05 07:39:02 +02:00
Daniel Stenberg
8d1da2e172
http: typecast the httpreq assignment to avoid icc compiler warning
error #188: enumerated type mixed with another type

Closes #9179
2022-07-23 13:39:00 +02:00
Daniel Stenberg
48d7064a49
cookie: apply limits
- Send no more than 150 cookies per request
- Cap the max length used for a cookie: header to 8K
- Cap the max number of received Set-Cookie: headers to 50

Bug: https://curl.se/docs/CVE-2022-32205.html
CVE-2022-32205
Reported-by: Harry Sintonen
Closes #9048
2022-06-26 11:00:48 +02:00
max.mehl
ad9bc5976d
copyright: make repository REUSE compliant
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869
2022-06-13 09:13:00 +02:00
Daniel Stenberg
c9b60f0053
http: restore header folding behavior
Folded header lines will now get passed through like before. The headers
API is adapted and will provide the content unfolded.

Added test 1274 and extended test 1940 to verify.

Reported-by: Petr Pisar
Fixes #8844
Closes #8899
2022-05-24 23:33:47 +02:00
Daniel Stenberg
d7b970e46b
http: move Curl_allow_auth_to_host()
It was mistakenly put within the CURL_DISABLE_HTTP_AUTH #ifdef

Reported-by: Michael Olbrich
Fixes #8772
Closes #8775
2022-04-29 22:56:47 +02:00
Daniel Stenberg
139a54ed0a
openssl: don't leak the SRP credentials in redirects either
Follow-up to 620ea21410

Reported-by: Harry Sintonen
Closes #8751
2022-04-25 22:34:40 +02:00
Daniel Stenberg
6e65999395
http: avoid auth/cookie on redirects same host diff port
CVE-2022-27776

Reported-by: Harry Sintonen
Bug: https://curl.se/docs/CVE-2022-27776.html
Closes #8749
2022-04-25 13:05:40 +02:00
Daniel Stenberg
6968fb9d54
lib: remove exclamation marks
... from infof() and failf() calls. Make them less attention seeking.

Closes #8713
2022-04-16 11:55:05 +02:00
Daniel Stenberg
854ec76551
http: streamclose "already downloaded"
Instead of connclose()ing, since when HTTP/2 is used it doesn't need to
close the connection as stopping the current transfer is enough.

Reported-by: Evangelos Foutras
Closes #8665
2022-04-01 23:30:43 +02:00
Daniel Stenberg
218cc70028
http: correct the header error message to say colon
Not semicolon

Reported-by: Gisle Vanem
Ref: #8666
Closes #8667
2022-04-01 17:00:18 +02:00
Daniel Stenberg
b716b5aa7e
lib: #ifdef on USE_HTTP2 better
... as nghttp2 might not be the library that provides HTTP/2 support.

Closes #8661
2022-04-01 13:45:52 +02:00
Daniel Stenberg
3fa634a337
http: close the stream (not connection) on time condition abort
Closes #8664
2022-04-01 13:22:59 +02:00
Daniel Stenberg
d1e4a67734
header api: add curl_easy_header and curl_easy_nextheader
Add test 1940 to 1946 to verify.

Closes #8593
2022-03-22 08:24:22 +01:00
Daniel Stenberg
dcdf2e8a36
http: return error on colon-less HTTP headers
It's a protocol violation and accepting them leads to no good.

Add test case 398 to verify

Closes #8610
2022-03-21 08:37:24 +01:00
Daniel Stenberg
526e0ef4cb
http: reject header contents with nul bytes
They are not allowed by the protocol and allowing them risk that curl
misbehaves somewhere where C functions are used but won't work on the
full contents. Further, they are not supported by hyper and they cause
problems for the new coming headers API work.

Updated test 262 to verify and enabled it for hyper as well

Closes #8601
2022-03-17 10:17:31 +01:00
Daniel Stenberg
8984a42ae4
http: fix "unused parameter ‘conn’" warning
Follow-up from 7d600ad1c3

Spotted on appveyor

Closes #8465
2022-02-17 07:55:17 +01:00
Daniel Stenberg
7d600ad1c3
urldata: remove conn->bits.user_passwd
The authentication status should be told by the transfer and not the
connection.

Reported-by: John H. Ayad
Fixes #8449
Closes #8451
2022-02-16 10:28:31 +01:00
HenrikHolst
9bc3cebc92
misc: remove strlen for Curl_checkheaders + Curl_checkProxyheaders
Closes #8409
2022-02-10 08:51:06 +01:00
HenrikHolst
b807219292
misc: reduce strlen() calls with Curl_dyn_add()
Use STRCONST() to switch from Curl_dyn_add() to Curl_dyn_addn() for
string literals.

Closes #8398
2022-02-09 13:52:47 +01:00
HenrikHolst
4028892f14
http: make Curl_compareheader() take string length arguments too
Also add STRCONST, a macro that returns a string literal and it's length
for functions that take "string,len"

Removes unnecesary calls to strlen().

Closes #8391
2022-02-07 14:37:58 +01:00
Daniel Stenberg
c85178a94c
http2: allow CURLOPT_HTTPHEADER change ":scheme"
The only h2 psuedo header that wasn't previously possible to change by a
user. This change also makes it impossible to send a HTTP/1 header that
starts with a colon, which I don't think anyone does anyway.

The other pseudo headers are possible to change indirectly by doing the
rightly crafted request.

Reported-by: siddharthchhabrap on github
Fixes #8381
Closes #8393
2022-02-07 13:55:57 +01:00
Daniel Stenberg
2610142139
lib: remove support for CURL_DOES_CONVERSIONS
TPF was the only user and support for that was dropped.

Closes #8378
2022-02-04 08:05:35 +01:00
lucas
519b62ccd0
docs: update IETF links to use datatracker
The tools.ietf.org domain has been deprecated a while now, with the
links being redirected to datatracker.ietf.org.

Rather than make people eat that redirect time, this change switches the
URL to a more canonical source.

Closes #8317
2022-01-21 23:06:36 +01:00
Daniel Stenberg
21248e052d
checksrc: detect more kinds of NULL comparisons we avoid
Co-authored-by: Jay Satiro
Closes #8180
2021-12-27 23:39:26 +01:00
Vladimir Panteleev
1b9f1f6055
http: Fix CURLOPT_HTTP200ALIASES
The httpcode < 100 check was also triggered when none of the fields were
parsed, thus making the if(!nc) block unreachable.

Closes #8171
2021-12-21 08:44:55 +01:00
Daniel Stenberg
6e061ae63a
http: enable haproxy support for hyper backend
This is done by having native code do the haproxy header output before
hyper issues its request. The little downside with this approach is that
we need the entire Curl_buffer_send() function built, which is otherwise
not used for hyper builds.

If hyper ends up getting native support for the haproxy protocols we can
backpedal on this.

Enables test 1455 and 1456

Closes #8034
2021-11-19 14:06:42 +01:00
Daniel Stenberg
c67a32fc56
http: reject HTTP response codes < 100
... which then also includes negative ones as test 1430 uses.

This makes native + hyper backend act identically on this and therefore
test 1430 can now be enabled when building with hyper. Adjust test 1431
as well.

Closes #7909
2021-10-27 10:26:15 +02:00
Jay Satiro
b1d08d295f http: set content length earlier
- Make content length (ie download size) accessible to the user in the
  header callback, but only after all headers have been processed (ie
  only in the final call to the header callback).

Background:

For a long time the content length could be retrieved in the header
callback via CURLINFO_CONTENT_LENGTH_DOWNLOAD_T as soon as it was parsed
by curl.

Changes were made in 8a16e54 (precedes 7.79.0) to ignore content length
if any transfer encoding is used. A side effect of that was that
content length was not set by libcurl until after the header callback
was called the final time, because until all headers are processed it
cannot be determined if content length is valid.

This change keeps the same intention --all headers must be processed--
but now the content length is available before the final call to the
header function that indicates all headers have been processed (ie
a blank header).

Bug: https://github.com/curl/curl/commit/8a16e54#r57374914
Reported-by: sergio-nsk@users.noreply.github.com

Co-authored-by: Daniel Stenberg

Fixes https://github.com/curl/curl/issues/7804
Closes https://github.com/curl/curl/pull/7803
2021-10-15 03:40:16 -04:00
Daniel Stenberg
d36fded576
http: remove assert that breaks hyper
Reported-by: Jay Satiro
Fixes #7852
Closes #7855
2021-10-14 11:16:37 +02:00
Daniel Stenberg
87907d56f0
http: fix Basic auth with empty name field in URL
Add test 367 to verify.

Reported-by: Rick Lane
Fixes #7819
Closes #7820
2021-10-06 22:45:52 +02:00
Daniel Stenberg
beb8990d93
http: fix the broken >3 digit response code detection
When the "reason phrase" in the HTTP status line starts with a digit,
that was treated as the forth response code digit and curl would claim
the response to be non-compliant.

Added test 1466 to verify this case.

Regression brought by 5dc594e44f
Reported-by: Glenn de boer
Fixes #7738
Closes #7739
2021-09-17 17:52:00 +02:00
Daniel Stenberg
8a16e54c0c
http: ignore content-length if any transfer-encoding is used
Fixes #7643
Closes #7649
2021-08-31 10:04:03 +02:00
Daniel Stenberg
5dc594e44f
http: disallow >3-digit response codes
Make the built-in HTTP parser behave similar to hyper and reject any
HTTP response using more than 3 digits for the response code.

Updated test 1432 accordingly.
Enabled test 1432 in the hyper builds.

Closes #7641
2021-08-26 22:43:38 +02:00
Daniel Stenberg
e5d77dc2ca
c-hyper: handle HTTP/1.1 => HTTP/1.0 downgrade on reused connection
Enable test 1074

Closes #7617
2021-08-23 16:12:33 +02:00
Daniel Stenberg
c495dcd02e
http: consider cookies over localhost to be secure
Updated test31.
Added test 392 to verify secure cookies used for http://localhost

Reviewed-by: Daniel Gustafsson
Fixes #6733
Closes #7263
2021-08-10 11:20:21 +02:00
MAntoniak
9a47d77126
lib: fix compiler warnings with CURL_DISABLE_NETRC
warning C4189: 'netrc_user_changed': local variable is initialized but
not referenced

warning C4189: 'netrc_passwd_changed': local variable is initialized but
not referenced

Closes #7423
2021-07-18 18:37:05 +02:00
Daniel Stenberg
e7416cfd2b
infof: remove newline from format strings, always append it
- the data needs to be "line-based" anyway since it's also passed to the
  debug callback/application

- it makes infof() work like failf() and consistency is good

- there's an assert that triggers on newlines in the format string

- Also removes a few instances of "..."

- Removes the code that would append "..." to the end of the data *iff*
  it was truncated in infof()

Closes #7357
2021-07-07 22:54:01 +02:00
Daniel Stenberg
0b0269341b
c-hyper: add support for transfer-encoding in the request
Closes #7348
2021-07-05 22:46:01 +02:00
Jay Satiro
ca8893468f http: fix crash in rate-limited upload
- Don't set the size of the piece of data to send to the rate limit if
  that limit is larger than the buffer size that will hold the piece.

Prior to this change if CURLOPT_MAX_SEND_SPEED_LARGE
(curl tool: --limit-rate) was set then it was possible that a temporary
buffer used for uploading could be written to out of bounds. A likely
scenario for this would be a non-trivial amount of post data combined
with a rate limit larger than CURLOPT_UPLOAD_BUFFERSIZE (default 64k).

The bug was introduced in 24e469f which is in releases since 7.76.0.

perl -e "print '0' x 200000" > tmp
curl --limit-rate 128k -d @tmp httpbin.org/post

Reported-by: Richard Marion

Fixes https://github.com/curl/curl/issues/7308
Closes https://github.com/curl/curl/pull/7315
2021-06-29 15:18:08 -04:00
Daniel Stenberg
a629506d42
http: make the haproxy support work with unix domain sockets
... it should then pass on "PROXY UNKNOWN" since it doesn't know the
involved IP addresses.

Reported-by: Valentín Gutiérrez
Fixes #7290
Closes #7291
2021-06-24 09:01:49 +02:00
Daniel Stenberg
a6da296867
vtls: only store TIMER_APPCONNECT for non-proxy connect
Introducing a 'isproxy' argument to the connect function so that it
knows wether to store the time stamp or not.

Reported-by: Yongkang Huang
Fixes #7274
Closes #7274
2021-06-19 23:02:16 +02:00
Daniel Stenberg
0c55fbab45
conn: add 'attach' to protocol handler, make libssh2 use it
The libssh2 backend has SSH session associated with the connection but
the callback context is the easy handle, so when a connection gets
attached to a transfer, the protocol handler now allows for a custom
function to get used to set things up correctly.

Reported-by: Michael O'Farrell
Fixes #6898
Closes #7078
2021-05-17 17:57:50 +02:00
Peng-Yu Chen
dbb88523ab
http: use calculated offsets inst of integer literals for header parsing
Assumed to be a minor coding style improvement with no behavior change.

A modern compiler is expected to have the calculation optimized during
compilation. It may be deemed okay even if that's not the case, since
the added overhead is considered very low.

Closes #7032
2021-05-08 17:00:32 +02:00
Daniel Stenberg
1763aceb0c
http: limit the initial send amount to used upload buffer size
Previously this logic would cap the send to CURL_MAX_WRITE_SIZE bytes,
but for the situations where a larger upload buffer has been set, this
function can benefit from sending more bytes. With default size used,
this does the same as before.

Also changed the storage of the size to an 'unsigned int' as it is not
allowed to be set larger than 2M.

Also added cautions to the man pages about changing buffer sizes in
run-time.

Closes #7022
2021-05-07 08:51:39 +02:00
Daniel Stenberg
04cc27460e
http: reset the header buffer when sending the request
A reused transfer handle could otherwise reuse the previous leftover
buffer and havoc would ensue.

Reported-by: sergio-nsk on github
Fixes #7018
Closes #7021
2021-05-06 23:28:45 +02:00
Harry Sintonen
a60294cf43
Curl_http_header: check for colon when matching Persistent-Auth
Closes #6993
2021-05-03 16:35:22 +02:00
Harry Sintonen
b75620b9a0
Curl_http_input_auth: require valid separator after negotiation type
Closes #6993
2021-05-03 16:35:14 +02:00
Daniel Stenberg
8b9de77cd2
http: fix the check for 'Authorization' with Bearer
The code would wrongly check for it using an additional colon.

Reported-by: Blake Burkhart
Closes #6988
2021-05-02 12:08:20 +02:00
Daniel Stenberg
d71ff2b9db
hsts: enable by default
No longer considered experimental.

Closes #6700
2021-04-19 08:22:16 +02:00
Daniel Stenberg
b5726e5549
send_speed: simplify the checks for if a speed limit is set
... as we know the value cannot be set to negative: enforced by
setopt()
2021-03-27 12:38:28 +01:00
Daniel Stenberg
24e469f6d6
http: cap body data amount during send speed limiting
By making sure never to send off more than the allowed number of bytes
per second the speed limit logic is given more room to actually work.

Reported-by: Fabian Keil
Bug: https://curl.se/mail/lib-2021-03/0042.html
Closes #6797
2021-03-27 12:38:15 +01:00
Daniel Stenberg
95cbcec8f9
urldata: merge "struct DynamicStatic" into "struct UrlState"
Both were used for the same purposes and there was no logical separation
between them. Combined, this also saves 16 bytes in less holes in my
test build.

Closes #6798
2021-03-26 23:19:20 +01:00
Daniel Stenberg
3bbf62b5a4
http: strip default port from URL sent to proxy
To make sure the Host: header and the URL provide the same authority
portion when sent to the proxy, strip the default port number from the
URL if one was provided.

Reported-by: Michael Brown
Fixes #6769
Closes #6778
2021-03-23 13:33:49 +01:00
Daniel Stenberg
6d176bee59
http: make 416 not fail with resume + CURLOPT_FAILONERRROR
When asked to resume a download, libcurl will convert that to HTTP logic
and if then the entire file is already transferred it will result in a
416 response from the HTTP server. With CURLOPT_FAILONERRROR set in that
scenario, it should *not* lead to an error return.

Updated test 1156, added test 1273

Reported-by: Jonathan Watt
Fixes #6740
Closes #6753
2021-03-17 08:26:46 +01:00
Daniel Stenberg
4c0206f90b
http: remove superfluous NULL assign
Closes #6727
2021-03-11 22:45:23 +01:00
Daniel Stenberg
723c0e75fa
http2: remove conn->data use
... but instead use a private alternative that points to the "driving
transfer" from the connection. We set the "user data" associated with
the connection to be the connectdata struct, but when we drive transfers
the code still needs to know the pointer to the transfer. We can change
the user data to become the Curl_easy handle, but with older nghttp2
version we cannot dynamically update that pointer properly when
different transfers are used over the same connection.

Closes #6520
2021-02-15 16:33:53 +01:00
Daniel Stenberg
46620b9743
http: use credentials from transfer, not connection
HTTP auth "accidentally" worked before this cleanup since the code would
always overwrite the connection credentials with the credentials from
the most recent transfer and since HTTP auth is typically done first
thing, this has not been an issue. It was still wrong and subject to
possible race conditions or future breakage if the sequence of functions
would change.

The data.set.str[] strings MUST remain unmodified exactly as set by the
user, and the credentials to use internally are instead set/updated in
state.aptr.*

Added test 675 to verify different credentials used in two requests done
over a reused HTTP connection, which previously behaved wrongly.

Fixes #6542
Closes #6545
2021-02-13 22:36:15 +01:00
Daniel Stenberg
88dd1a8a11
urldata: don't touch data->set.httpversion at run-time
Rename it to 'httpwant' and make a cloned field in the state struct as
well for run-time updates.

Also: refuse non-supported HTTP versions. Verified with test 129.

Closes #6585
2021-02-12 08:13:37 +01:00
Daniel Stenberg
115c9e27f5
ftp: add 'prefer_ascii' to the transfer state struct
... and make sure the code never updates 'set.prefer_ascii' as it breaks
handle reuse which should use the setting as the user specified it.

Added test 1569 to verify: it first makes an FTP transfer with ';type=A'
and then another without type on the same handle and the second should
then use binary. Previously, curl failed this.

Closes #6578
2021-02-09 14:06:28 +01:00
Jacob Hoffman-Andrews
246399a874
vtls: initial implementation of rustls backend
This adds a new TLS backend, rustls. It uses the C-to-rustls bindings
from https://github.com/abetterinternet/crustls.

Rustls is at https://github.com/ctz/rustls/.

There is still a fair bit to be done, like sending CloseNotify on
connection shutdown, respecting CAPATH, and properly indicating features
like "supports TLS 1.3 ciphersuites." But it works well enough to make
requests and receive responses.

Blog post for context:
https://www.abetterinternet.org/post/memory-safe-curl/

Closes #6350
2021-02-09 11:06:18 +01:00
Jay Satiro
cb2dc1ba89 lib: use int type for more port variables
This is a follow-up to 764c6bd. Prior to that change port variables
were usually type long.

Closes https://github.com/curl/curl/pull/6553
2021-02-09 02:53:31 -05:00
Daniel Stenberg
d836d362ec
urldata: remove the 'rtspversion' field
from struct connectdata and the corresponding code in http.c that set
it. It was never used for anything!

Closes #6581
2021-02-08 22:39:34 +01:00
Jay Satiro
98577e4029 build: fix --disable-http-auth
Broken since 215db08 (precedes 7.75.0).

Reported-by: Benbuck Nason

Fixes https://github.com/curl/curl/issues/6567
2021-02-04 14:49:56 -05:00
Jay Satiro
145848e2d1 build: fix --disable-dateparse
Broken since 215db08 (precedes 7.75.0).

Bug: https://curl.se/mail/lib-2021-02/0008.html
Reported-by: Firefox OS
2021-02-04 14:13:00 -05:00
Daniel Stenberg
1dc8aa870e
hostip/proxy: remove conn->data use
Closes #6513
2021-01-26 10:04:47 +01:00
Patrick Monnerat
ecb13416e3
lib: remove conn->data uses
Closes #6499
2021-01-24 18:15:03 +01:00
Daniel Stenberg
e8710633ed
http: empty reply connection are not left intact
... so mark the connection as closed in this condition to prevent that
verbose message to wrongly appear.

Reported-by: Matt Holt
Bug: https://twitter.com/mholt6/status/1352130240265375744
Closes #6503
2021-01-21 16:39:35 +01:00
Daniel Stenberg
c977a6d0dc
chunk/encoding: remove conn->data references
... by anchoring more functions on Curl_easy instead of connectdata

Closes #6498
2021-01-21 13:19:58 +01:00
Fabian Keil
0015e06151
http: get CURLOPT_REQUEST_TARGET working with a HTTP proxy
Added test 1613 to verify.

Closes #6490
2021-01-20 15:21:13 +01:00
Daniel Stenberg
219d9f8620
transfer: remove conn->data use
Closes #6486
2021-01-19 17:07:02 +01:00
Daniel Stenberg
ea6ce0afad
http: make providing Proxy-Connection header not cause duplicated headers
Fixes test 1180

Bug: https://curl.se/mail/lib-2021-01/0095.html
Reported-by: Fabian Keil
Closes #6472
2021-01-19 11:40:15 +01:00
Daniel Stenberg
a304051620
lib: more conn->data cleanups
Closes #6479
2021-01-19 09:14:51 +01:00
Patrick Monnerat
bbe3aa9f88
vtls: reduce conn->data use
Closes #6474
2021-01-19 09:14:40 +01:00
Daniel Stenberg
215db086e0
lib: pass in 'struct Curl_easy *' to most functions
... in most cases instead of 'struct connectdata *' but in some cases in
addition to.

- We mostly operate on transfers and not connections.

- We need the transfer handle to log, store data and more. Everything in
  libcurl is driven by a transfer (the CURL * in the public API).

- This work clarifies and separates the transfers from the connections
  better.

- We should avoid "conn->data". Since individual connections can be used
  by many transfers when multiplexing, making sure that conn->data
  points to the current and correct transfer at all times is difficult
  and has been notoriously error-prone over the years. The goal is to
  ultimately remove the conn->data pointer for this reason.

Closes #6425
2021-01-17 23:56:09 +01:00
Fabian Keil
4f61fd87b2
misc: fix typos
Bug: https://curl.se/mail/lib-2021-01/0063.html
Closes #6434
2021-01-11 15:12:25 +01:00
Daniel Stenberg
8b2dec6ab7
http: make 'authneg' also work for Hyper
When doing a request with a request body expecting a 401/407 back, that
initial request is sent with a zero content-length. Test 177 and more.

Closes #6424
2021-01-10 14:51:33 +01:00
Daniel Stenberg
ab525c059e
http: have CURLOPT_FAILONERROR fail after all headers
... so that Retry-After and other meta-content can still be used.

Added 1634 to verify. Adjusted test 194 and 281 since --fail now also
includes the header-terminating CRLF in the output before it exits.

Fixes #6408
Closes #6409
2021-01-06 15:14:40 +01:00
Daniel Stenberg
fce912139a
c-hyper: add timecondition to the request
Test 77-78

Closes #6391
2021-01-01 00:49:37 +01:00
Daniel Stenberg
aba01da639
failf: remove newline from formatting strings
... as failf adds one itself.

Also: add an assert() to failf() that triggers on a newline in the
format string!

Closes #6365
2020-12-25 23:42:16 +01:00
Matthias Gatto
eb69797766
http: Make the call to v4 signature
This patch allow to call the v4 signature introduce in previous commit

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
2020-12-21 16:28:03 +01:00
Daniel Stenberg
58974d25d8
lib: introduce c-hyper for using Hyper
... as an alternative HTTP backend within libcurl.
2020-12-18 09:58:04 +01:00
Daniel Stenberg
a5bc272223
http: show the request as headers even when split-sending
When the initial request isn't possible to send in its entirety, the
remainder of request would be delivered to the debug callback as data
and would wrongly be counted internally as body-bytes sent.

Extended test 1295 to verify.

Closes #6328
2020-12-17 14:00:27 +01:00
Rikard Falkeborn
920f49a20b
infof/failf calls: fix format specifiers
Update a few format specifiers to match what is being printed.

Closes #6241
2020-11-24 13:18:41 +01:00
Daniel Stenberg
a95a6ce6b8
urldata: remove 'void *protop' and create the union 'p'
... to avoid the use of 'void *' for the protocol specific structs done
per transfer.

Closes #6238
2020-11-23 16:16:16 +01:00
Daniel Stenberg
4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Daniel Stenberg
7385610d0c
hsts: add support for Strict-Transport-Security
- enable in the build (configure)
- header parsing
- host name lookup
- unit tests for the above
- CI build
- CURL_VERSION_HSTS bit
- curl_version_info support
- curl -V output
- curl-config --features
- CURLOPT_HSTS_CTRL
- man page for CURLOPT_HSTS_CTRL
- curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl)
- man page for --hsts
- save cache to disk
- load cache from disk
- CURLOPT_HSTS
- man page for CURLOPT_HSTS
- added docs/HSTS.md
- fixed --version docs
- adjusted curl_easy_duphandle

Closes #5896
2020-11-03 16:08:42 +01:00
Daniel Stenberg
d70a5b5a0f
sendf: move the verbose-check into Curl_debug
Saves us from having the same check done everywhere.

Closes #6159
2020-11-02 22:49:36 +01:00
Daniel Stenberg
584ccb5ef2
http: pass correct header size to debug callback for chunked post
... when the chunked framing was added, the size of the "body part" of
the data was calculated wrongly so the debug callback would get told a
header chunk a few bytes too big that would also contain the first few
bytes of the request body.

Reported-by: Dirk Wetter
Ref: #6144
Closes #6147
2020-10-31 23:46:27 +01:00
Daniel Stenberg
96450a1a33
alt-svc: enable by default
Remove CURLALTSVC_IMMEDIATELY, which was never implemented/supported.

alt-svc support in curl is no longer considered experimental

Closes #5868
2020-10-25 23:08:54 +01:00
Daniel Gustafsson
2aac895fb6 src: Consistently spell whitespace without whitespace
Whitespace is spelled without a space between white and space, so
make sure to consistently spell it that way across the codebase.

Closes #6023
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Emil Engler <me@emilengler.com>
2020-09-30 21:10:14 +02:00
Daniel Stenberg
6434a73984
Curl_handler: add 'family' to each protocol
Makes get_protocol_family() faster and it moves the knowledge about the
"families" to each protocol handler, where it belongs.

Closes #5986
2020-09-21 23:30:16 +02:00
Daniel Stenberg
d75e3ab74c
CURLINFO_EFFECTIVE_METHOD: added
Provide the HTTP method that was used on the latest request, which might
be relevant for users when there was one or more redirects involved.

Closes #5511
2020-07-14 17:53:45 +02:00
Daniel Stenberg
032e838b73
terminology: call them null-terminated strings
Updated terminology in docs, comments and phrases to refer to C strings
as "null-terminated". Done to unify with how most other C oriented docs
refer of them and what users in general seem to prefer (based on a
single highly unscientific poll on twitter).

Reported-by: coinhubs on github
Fixes #5598
Closes #5608
2020-06-28 00:31:24 +02:00
Daniel Stenberg
ff43fb6dec
http: fix proxy auth with blank password
Regression in 7.71.0

Added test case 346 to verify.

Reported-by: Kristoffer Gleditsch
Fixes #5613
Closes #5616
2020-06-28 00:29:59 +02:00
Daniel Stenberg
d5ed571948
url: allow user + password to contain "control codes" for HTTP(S)
Reported-by: Jon Johnson Jr
Fixes #5582
Closes #5592
2020-06-25 09:57:19 +02:00
Daniel Stenberg
e15e51384a
http: move header storage to Curl_easy from connectdata
Since the connection can be used by many independent requests (using
HTTP/2 or HTTP/3), things like user-agent and other transfer-specific
data MUST NOT be kept connection oriented as it could lead to requests
getting the wrong string for their requests. This struct data was
lingering like this due to old HTTP1 legacy thinking where it didn't
mattered..

Fixes #5566
Closes #5567
2020-06-15 22:56:25 +02:00
Daniel Stenberg
eab2f95c0d
wording: avoid blacklist/whitelist stereotypes
Instead of discussing if there's value or meaning (implied or not) in
the colors, let's use words without the same possibly negative
associations.

Closes #5546
2020-06-10 08:49:17 +02:00
Daniel Stenberg
9c845be279
urldata: let the HTTP method be in the set.* struct
When the method is updated inside libcurl we must still not change the
method as set by the user as then repeated transfers with that same
handle might not execute the same operation anymore!

This fixes the libcurl part of #5462

Test 1633 added to verify.

Closes #5499
2020-06-02 16:30:36 +02:00
Daniel Stenberg
f3d501dc67
build: disable more code/data when built without proxy support
Added build to travis to verify

Closes #5466
2020-05-30 23:18:16 +02:00
Daniel Stenberg
7414fb25a2
urldata: connect related booleans live in struct ConnectBits
And remove a few unused booleans!

Closes #5461
2020-05-27 08:02:47 +02:00
Daniel Stenberg
ed35d6590e
dynbuf: introduce internal generic dynamic buffer functions
A common set of functions instead of many separate implementations for
creating buffers that can grow when appending data to them. Existing
functionality has been ported over.

In my early basic testing, the total number of allocations seem at
roughly the same amount as before, possibly a few less.

See docs/DYNBUF.md for a description of the API.

Closes #5300
2020-05-04 10:40:39 +02:00
Daniel Stenberg
df28ed6116
http: free memory when Alt-Used header creation fails due to OOM
Reported-by: James Fuller
Fixes #5268
Closes #5269
2020-04-20 14:24:56 +02:00
Daniel Stenberg
0ef54abf52
http: don't consider upload done if the request isn't completely sent off
Fixes #4919
Closes #5197
2020-04-08 08:14:23 +02:00
Daniel Stenberg
3e376059bb
http: allow Curl_add_buffer_send() to do a short first send by force
In a debug build, settting the environment variable "CURL_SMALLREQSEND"
will make the first HTTP request send not send more bytes than the set
amount, thus ending up verifying that the logic for handling a split
HTTP request send works correctly.
2020-04-08 08:12:59 +02:00
Daniel Stenberg
0caae3c4be
http: mark POSTs with no body as "upload done" from the start
As we have logic that checks if we get a >= 400 reponse code back before
the upload is done, which then got confused since it wasn't "done" but
yet there was no data to send!

Reported-by: IvanoG on github
Fixes #4996
Closes #5002
2020-03-02 15:43:04 +01:00
Daniel Stenberg
6375b205a9
http: added 417 response treatment
When doing a request with a body + Expect: 100-continue and the server
responds with a 417, the same request will be retried immediately
without the Expect: header.

Added test 357 to verify.

Also added a control instruction to tell the sws test server to not read
the request body if Expect: is present, which the new test 357 uses.

Reported-by: bramus on github
Fixes #4949
Closes #4964
2020-02-26 22:48:14 +01:00
Pedro Monreal
4b6fd29f1a cleanup: fix typos and wording in docs and comments
Closes #4869
Reviewed-by: Emil Engler and Daniel Gustafsson
2020-02-02 18:43:01 +01:00
nao
dea17b519d
http: move "oauth_bearer" from connectdata to Curl_easy
Fixes the bug where oauth_bearer gets deallocated when we re-use a
connection.

Closes #4824
2020-01-21 10:32:43 +01:00
Marcel Raad
291ed52122
lib: fix compiler warnings with CURL_DISABLE_VERBOSE_STRINGS
Closes https://github.com/curl/curl/pull/4775
2020-01-03 17:02:30 +01:00
Daniel Stenberg
ee263de7a3
conncache: fix multi-thread use of shared connection cache
It could accidentally let the connection get used by more than one
thread, leading to double-free and more.

Reported-by: Christopher Reid
Fixes #4544
Closes #4557
2019-12-09 15:30:09 +01:00
Daniel Stenberg
0044443a02
parsedate: offer a getdate_capped() alternative
... and use internally. This function will return TIME_T_MAX instead of
failure if the parsed data is found to be larger than what can be
represented. TIME_T_MAX being the largest value curl can represent.

Reviewed-by: Daniel Gustafsson
Reported-by: JanB on github
Fixes #4152
Closes #4651
2019-11-29 11:01:24 +01:00
Javier Blazquez
e0363a47de
ngtcp2: use overflow buffer for extra HTTP/3 data
Fixes #4525
Closes #4603
2019-11-18 16:40:08 +01:00
Daniel Stenberg
d0a7ee3f61
cookies: using a share with cookies shouldn't enable the cookie engine
The 'share object' only sets the storage area for cookies. The "cookie
engine" still needs to be enabled or activated using the normal cookie
options.

This caused the curl command line tool to accidentally use cookies
without having been told to, since curl switched to using shared cookies
in 7.66.0.

Test 1166 verifies

Updated test 506

Fixes #4429
Closes #4434
2019-09-28 18:10:43 +02:00
Daniel Stenberg
fe514ad9ae
http: fix warning on conversion from int to bit
Follow-up from 03ebe66d70
2019-09-21 23:09:59 +02:00
Daniel Stenberg
69ea985d4c
http: fix Expression 'http->postdata' is always false
Fixes warning detected by PVS-Studio
Fixes #4374
Reported-by: Valerii Zapodovnikov
2019-09-20 08:06:47 +02:00
Michael Kaufmann
2a2404153b http: merge two "case" statements 2019-09-18 13:45:41 +02:00
Daniel Stenberg
c4c9e070f3
Curl_fillreadbuffer: avoid double-free trailer buf on error
Reviewed-by: Jay Satiro
Reported-by: Thomas Vegas

Closes #4307
2019-09-09 09:36:27 +02:00
Daniel Stenberg
99775438d6
http: remove chunked-encoding and expect header use for HTTP/3 2019-08-20 16:53:40 +02:00
Daniel Stenberg
68fab35c73
http: fix use of credentials from URL when using HTTP proxy
When a username and password are provided in the URL, they were wrongly
removed from the stored URL so that subsequent uses of the same URL
wouldn't find the crendentials. This made doing HTTP auth with multiple
connections (like Digest) mishave.

Regression from 46e164069d (7.62.0)

Test case 335 added to verify.

Reported-by: Mike Crowe

Fixes #4228
Closes #4229
2019-08-16 10:11:22 +02:00
Daniel Stenberg
f2cc26456b
ngtcp2: do QUIC connections happy-eyeballs friendly 2019-08-14 23:14:12 +02:00
Daniel Stenberg
a93b43cde8
http3: fix the HTTP/3 in the request, make alt-svc set right versions
Closes #4200
2019-08-08 08:28:43 +02:00
Daniel Stenberg
98c3f14871
alt-svc: send Alt-Used: in redirected requests
RFC 7838 section 5:

   When using an alternative service, clients SHOULD include an Alt-Used
   header field in all requests.

Removed CURLALTSVC_ALTUSED again (feature is still EXPERIMENTAL thus
this is deemed ok).

You can disable sending this header just like you disable any other HTTP
header in libcurl.

Closes #4199
2019-08-08 08:24:37 +02:00
Daniel Stenberg
026840e35c
CURLOPT_HTTP_VERSION: seting this to 3 forces HTTP/3 use directly
Even though it cannot fall-back to a lower HTTP version automatically. The
safer way to upgrade remains via CURLOPT_ALTSVC.

CURLOPT_H3 no longer has any bits that do anything and might be removed
before we remove the experimental label.

Updated the curl tool accordingly to use "--http3".

Closes #4197
2019-08-08 08:17:24 +02:00
Daniel Stenberg
f933449d3b
CURLINFO_RETRY_AFTER: parse the Retry-After header value
This is only the libcurl part that provides the information. There's no
user of the parsed value. This change includes three new tests for the
parser.

Ref: #3794
2019-08-07 14:45:39 +02:00
Daniel Stenberg
dc35631ef7
quiche: first working HTTP/3 request
- enable debug log
 - fix use of quiche API
 - use download buffer
 - separate header/body

Closes #4193
2019-08-06 07:39:21 +02:00
Daniel Stenberg
a55faf33d4
cleanup: remove the 'numsocks' argument used in many places
It was used (intended) to pass in the size of the 'socks' array that is
also passed to these functions, but was rarely actually checked/used and
the array is defined to a fixed size of MAX_SOCKSPEREASYHANDLE entries
that should be used instead.

Closes #4169
2019-07-30 23:16:44 +02:00
Balazs Kovacsics
802aa5ae27
HTTP: use chunked Transfer-Encoding for HTTP_POST if size unknown
If using the read callback for HTTP_POST, and POSTFIELDSIZE is not set,
automatically add a Transfer-Encoding: chunked header, same as it is
already done for HTTP_PUT, HTTP_POST_FORM and HTTP_POST_MIME.  Update
test 1514 according to the new behaviour.

Closes #4138
2019-07-29 13:18:53 +02:00
Daniel Stenberg
3af0e76d1e
HTTP3: initial (experimental) support
USe configure --with-ngtcp2 or --with-quiche

Using either option will enable a HTTP3 build.
Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me>

Closes #3500
2019-07-21 23:49:03 +02:00
Peter Simonyi
855887af79
http: allow overriding timecond with custom header
With CURLOPT_TIMECONDITION set, a header is automatically added (e.g.
If-Modified-Since).  Allow this to be replaced or suppressed with
CURLOPT_HTTPHEADER.

Fixes #4103
Closes #4109
2019-07-14 16:17:15 +02:00
Daniel Gustafsson
11d7fe118d headers: Remove no longer exported functions
There were a leftover few prototypes of Curl_ functions that we used to
export but no longer do, this removes those prototypes and cleans up any
comments still referring to them.

Curl_write32_le(), Curl_strcpy_url(), Curl_strlen_url(), Curl_up_free()
Curl_concat_url(), Curl_detach_connnection(), Curl_http_setup_conn()
were made static in 05b100aee2.
Curl_http_perhapsrewind() made static in 574aecee20.

For the remainder, I didn't trawl the Git logs hard enough to capture
their exact time of deletion, but they were all gone: Curl_splayprint(),
Curl_http2_send_request(), Curl_global_host_cache_dtor(),
Curl_scan_cache_used(), Curl_hostcache_destroy(), Curl_second_connect(),
Curl_http_auth_stage() and Curl_close_connections().

Closes #4096
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2019-07-10 19:26:40 +02:00
Daniel Gustafsson
b96282010e http: clarify header buffer size calculation
The header buffer size calculation can from static analysis seem to
overlow as it performs an addition between two size_t variables and
stores the result in a size_t variable. Overflow is however guarded
against elsewhere since the input to the addition is regulated by
the maximum read buffer size. Clarify this with a comment since the
question was asked.

Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2019-06-24 23:35:06 +02:00
Daniel Stenberg
265f7f42f6
http2: call done_sending on end of upload
To make sure a HTTP/2 stream registers the end of stream.

Bug #4043 made me find this problem but this fix doesn't correct the
reported issue.

Closes #4068
2019-06-24 23:11:37 +02:00
Michael Kaufmann
2e5ceb3934
http: don't parse body-related headers bodyless responses
Responses with status codes 1xx, 204 or 304 don't have a response body. For
these, don't parse these headers:

- Content-Encoding
- Content-Length
- Content-Range
- Last-Modified
- Transfer-Encoding

This change ensures that HTTP/2 upgrades work even if a
"Content-Length: 0" or a "Transfer-Encoding: chunked" header is present.

Co-authored-by: Daniel Stenberg
Closes #3702
Fixes #3968
Closes #3977
2019-06-02 22:58:04 +02:00
Daniel Stenberg
5908009e31
http: fix "error: equality comparison with extraneous parentheses" 2019-05-29 08:17:08 +02:00
Daniel Stenberg
8ece8177f1
cleanup: remove FIXME and TODO comments
They serve very little purpose and mostly just add noise. Most of them
have been around for a very long time. I read them all before removing
or rephrasing them.

Ref: #3876
Closes #3883
2019-05-16 09:16:56 +02:00
Steve Holme
fe20826b58
http_ntlm_wb: Move the type-2 message processing into a dedicated function
This brings the code inline with the other HTTP authentication mechanisms.

Closes #3890
2019-05-16 00:03:30 +01:00
Steve Holme
e832d1ef74
http_negotiate: Move the Negotiate state out of the negotiatedata structure
Given that this member variable is not used by the SASL based protocols
there is no need to have it here.

Closes #3882
2019-05-15 00:32:42 +01:00
Steve Holme
85bef18ca1
http_ntlm: Move the NTLM state out of the ntlmdata structure
Given that this member variable is not used by the SASL based protocols
there is no need to have it here.
2019-05-15 00:31:45 +01:00
Daniel Stenberg
6d32b72822
proxy: acknowledge DISABLE_PROXY more 2019-05-13 08:17:10 +02:00
Daniel Stenberg
b084616210
parsedate: CURL_DISABLE_PARSEDATE 2019-05-13 08:17:10 +02:00
Daniel Stenberg
b95456f4e2
mime: acknowledge CURL_DISABLE_MIME 2019-05-13 08:17:09 +02:00
Daniel Stenberg
7c70a3b1ad
http: CURL_DISABLE_HTTP_AUTH 2019-05-13 08:17:09 +02:00
Jay Satiro
b5901cab57 http: Ignore HTTP/2 prior knowledge setting for HTTP proxies
- Do not switch to HTTP/2 for an HTTP proxy that is not tunnelling to
  the destination host.

We already do something similar for HTTPS proxies by not sending h2. [1]

Prior to this change setting CURL_HTTP_VERSION_2_PRIOR_KNOWLEDGE would
incorrectly use HTTP/2 to talk to the proxy, which is not something we
support (yet?). Also it's debatable whether or not that setting should
apply to HTTP/2 proxies.

[1]: https://github.com/curl/curl/commit/17c5d05

Bug: https://github.com/curl/curl/issues/3570
Bug: https://github.com/curl/curl/issues/3832

Closes https://github.com/curl/curl/pull/3853
2019-05-08 22:50:41 -04:00
Daniel Stenberg
c32248f17d
http: mark bundle as not for multiuse on < HTTP/2 response
Fixes #3813
Closes #3815
2019-05-01 22:51:23 +02:00
Daniel Stenberg
2f44e94efb
pipelining: removed
As previously planned and documented in DEPRECATE.md, all pipelining
code is removed.

Closes #3651
2019-04-06 22:49:50 +02:00
Dominik Hölzl
6c60355323
Negotiate: fix for HTTP POST with Negotiate
* Adjusted unit tests 2056, 2057
* do not generally close connections with CURLAUTH_NEGOTIATE after every request
* moved negotiatedata from UrlState to connectdata
* Added stream rewind logic for CURLAUTH_NEGOTIATE
* introduced negotiatedata::GSS_AUTHDONE and negotiatedata::GSS_AUTHSUCC
* Consider authproblem state for CURLAUTH_NEGOTIATE
* Consider reuse_forbid for CURLAUTH_NEGOTIATE
* moved and adjusted negotiate authentication state handling from
  output_auth_headers into Curl_output_negotiate
* Curl_output_negotiate: ensure auth done is always set
* Curl_output_negotiate: Set auth done also if result code is
  GSS_S_CONTINUE_NEEDED/SEC_I_CONTINUE_NEEDED as this result code may
  also indicate the last challenge request (only works with disabled
  Expect: 100-continue and CURLOPT_KEEP_SENDING_ON_ERROR -> 1)
* Consider "Persistent-Auth" header, detect if not present;
  Reset/Cleanup negotiate after authentication if no persistent
  authentication
* apply changes introduced with #2546 for negotiate rewind logic

Fixes #1261
Closes #1975
2019-03-14 09:26:03 +01:00
Marc Schlatter
dd8a19f8a0
http: send payload when (proxy) authentication is done
The check that prevents payload from sending in case of authentication
doesn't check properly if the authentication is done or not.

They're cases where the proxy respond "200 OK" before sending
authentication challenge. This change takes care of that.

Fixes #2431
Closes #3669
2019-03-13 10:58:52 +01:00
Daniel Stenberg
5345b04a43
alt-svc: add test 355 and 356 to verify with command line curl 2019-03-03 11:17:52 +01:00
Daniel Stenberg
e1be825453
alt-svc: the libcurl bits 2019-03-03 11:17:52 +01:00
Daniel Stenberg
65eb65fde6
urldata: simplify bytecounters
- no need to have them protocol specific

- no need to set pointers to them with the Curl_setup_transfer() call

- make Curl_setup_transfer() operate on a transfer pointer, not
  connection

- switch some counters from long to the more proper curl_off_t type

Closes #3627
2019-03-01 17:30:34 +01:00
Daniel Stenberg
f1d915ea49
http: set state.infilesize when sending formposts
Without it set, we would unwillingly triger the "HTTP error before end
of send, stop sending" condition even if the entire POST body had been
sent (since it wouldn't know the expected size) which would
unnecessarily log that message and close the connection when it didn't
have to.

Reported-by: Matt McClure
Bug: https://curl.haxx.se/mail/archive-2019-02/0023.html
Closes #3624
2019-02-28 15:24:03 +01:00
Daniel Stenberg
62a2534e4c
urldata: convert bools to bitfields and move to end
This allows the compiler to pack and align the structs better in
memory. For a rather feature-complete build on x86_64 Linux, gcc 8.1.2
makes the Curl_easy struct 4.9% smaller. From 6312 bytes to 6000.

Removed an unused struct field.

No functionality changes.

Closes #3610
2019-02-27 08:37:56 +01:00
Daniel Stenberg
942eb09e8a
http: make adding a blank header thread-safe
Previously the function would edit the provided header in-place when a
semicolon is used to signify an empty header. This made it impossible to
use the same set of custom headers in multiple threads simultaneously.

This approach now makes a local copy when it needs to edit the string.

Reported-by: d912e3 on github
Fixes #3578
Closes #3579
2019-02-19 10:18:47 +01:00
Daniel Stenberg
05b100aee2
cleanup: make local functions static
urlapi: turn three local-only functions into statics

conncache: make conncache_find_first_connection static

multi: make detach_connnection static

connect: make getaddressinfo static

curl_ntlm_core: make hmac_md5 static

http2: make two functions static

http: make http_setup_conn static

connect: make tcpnodelay static

tests: make UNITTEST a thing to mark functions with, so they can be static for
normal builds and non-static for unit test builds

... and mark Curl_shuffle_addr accordingly.

url: make up_free static

setopt: make vsetopt static

curl_endian: make write32_le static

rtsp: make rtsp_connisdead static

warnless: remove unused functions

memdebug: remove one unused function, made another static
2019-02-10 18:38:57 +01:00
Katsuhiko YOSHIDA
1f30dc886d
cookies: skip custom cookies when redirecting cross-site
Closes #3417
2019-01-09 15:18:08 +01:00
Daniel Stenberg
ebe658c1e5
Revert "http_negotiate: do not close connection until negotiation is completed"
This reverts commit 07ebaf8378.

This also reopens PR #3275 which brought the change now reverted.

Fixes #3384
Closes #3439
2019-01-07 09:36:36 +01:00
Daniel Stenberg
006ff62d8c
http: added options for allowing HTTP/0.9 responses
Added CURLOPT_HTTP09_ALLOWED and --http0.9 for this purpose.

For now, both the tool and library allow HTTP/0.9 by default.
docs/DEPRECATE.md lays out the plan for when to reverse that default: 6
months after the 7.64.0 release. The options are added already now so
that applications/scripts can start using them already now.

Fixes #2873
Closes #3383
2018-12-21 10:49:30 +01:00
Ayoub Boudhar
f464535bfd
http: Implement trailing headers for chunked transfers
This adds the CURLOPT_TRAILERDATA and CURLOPT_TRAILERFUNCTION
options that allow a callback based approach to sending trailing headers
with chunked transfers.

The test server (sws) was updated to take into account the detection of the
end of transfer in the case of trailing headers presence.

Test 1591 checks that trailing headers can be sent using libcurl.

Closes #3350
2018-12-14 10:10:48 +01:00
Daniel Gustafsson
7a09b52c98 cookies: leave secure cookies alone
Only allow secure origins to be able to write cookies with the
'secure' flag set. This reduces the risk of non-secure origins
to influence the state of secure origins. This implements IETF
Internet-Draft draft-ietf-httpbis-cookie-alone-01 which updates
RFC6265.

Closes #2956
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2018-12-13 09:57:58 +01:00
Jay Satiro
552f0205e6
http: fix HTTP auth to include query in URI
- Include query in the path passed to generate HTTP auth.

Recent changes to use the URL API internally (46e1640, 7.62.0)
inadvertently broke authentication URIs by omitting the query.

Fixes https://github.com/curl/curl/issues/3353
Closes #3356
2018-12-11 13:28:20 +01:00
Michael Kaufmann
c8bf8cc1e4
http: don't set CURLINFO_CONDITION_UNMET for http status code 204
The http status code 204 (No Content) should not change the "condition
unmet" flag. Only the http status code 304 (Not Modified) should do
this.

Closes #359
2018-12-11 13:22:42 +01:00
Johannes Schindelin
cbea2fd2c7
NTLM: force the connection to HTTP/1.1
Since v7.62.0, cURL tries to use HTTP/2 whenever the server announces
the capability. However, NTLM authentication only works with HTTP/1.1,
and will likely remain in that boat (for details, see
https://docs.microsoft.com/en-us/iis/get-started/whats-new-in-iis-10/http2-on-iis#when-is-http2-not-supported).

When we just found out that we want to use NTLM, and when the current
connection runs in HTTP/2 mode, let's force the connection to be closed
and to be re-opened using HTTP/1.1.

Fixes https://github.com/curl/curl/issues/3341.
Closes #3345

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-12-07 13:03:21 +01:00
Daniel Stenberg
dcd6f81025
snprintf: renamed and we now only use msnprintf()
The function does not return the same value as snprintf() normally does,
so readers may be mislead into thinking the code works differently than
it actually does. A different function name makes this easier to detect.

Reported-by: Tomas Hoger
Assisted-by: Daniel Gustafsson
Fixes #3296
Closes #3297
2018-11-23 08:26:51 +01:00
Elia Tufarolo
07ebaf8378
http_negotiate: do not close connection until negotiation is completed
Fix HTTP POST using CURLAUTH_NEGOTIATE.

Closes #3275
2018-11-16 23:38:47 +01:00
Viktor Szakats
ff9d7f4447 spelling fixes [ci skip]
as detected by codespell 1.14.0

Closes https://github.com/curl/curl/pull/3114
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
2018-10-08 19:37:40 +00:00
Daniel Stenberg
4058cf2a7f
http: fix memleak in rewind error path
If the rewind would fail, a strdup() would not get freed.

Detected by OSS-Fuzz

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10665
Closes #3044
2018-09-25 10:30:08 +02:00
Daniel Stenberg
e407e79c29
http: add missing return code check
Detected by Coverity. CID 1439610.

Follow-up from 46e164069d

Closes #3034
2018-09-23 23:08:39 +02:00
Daniel Stenberg
46e164069d
url: use the URL API internally as well
... to make it a truly unified URL parser.

Closes #3017
2018-09-22 11:58:10 +02:00
Daniel Stenberg
55dbcb061d
http: made Curl_add_buffer functions take a pointer-pointer
... so that they can clear the original pointer on failure, which makes
the error-paths and their cleanups easier.

Closes #2992
2018-09-16 23:22:37 +02:00
Daniel Stenberg
e6e9b006f7
upload: allocate upload buffer on-demand
Saves 16KB on the easy handle for operations that don't need that
buffer.

Part 1 of #2888
2018-08-18 15:48:08 +02:00
Daniel Stenberg
8440616f53
http: fix for tiny "HTTP/0.9" response
Deal with tiny "HTTP/0.9" (header-less) responses by checking the
status-line early, even before a full "HTTP/" is received to allow
detecting 0.9 properly.

Test 1266 and 1267 added to verify.

Fixes #2420
Closes #2872
2018-08-13 23:16:01 +02:00
Daniel Stenberg
2a278fd735
CURLINFO_SIZE_UPLOAD: fix missing counter update
Adds test 1522 for verification.

Reported-by: cjmsoregan
Fixes #2847
Closes #2864
2018-08-11 14:00:59 +02:00
Johannes Schindelin
6f5ef24f06
auth: pick Bearer authentication whenever a token is available
So far, the code tries to pick an authentication method only if
user/password credentials are available, which is not the case for
Bearer authentictation...

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes #2754
2018-07-24 10:11:49 +02:00
Johannes Schindelin
df57b439f4
auth: only ever pick CURLAUTH_BEARER if we *have* a Bearer token
The Bearer authentication was added to cURL 7.61.0, but there is a
problem: if CURLAUTH_ANY is selected, and the server supports multiple
authentication methods including the Bearer method, we strongly prefer
that latter method (only CURLAUTH_NEGOTIATE beats it), and if the Bearer
authentication fails, we will never even try to attempt any other
method.

This is particularly unfortunate when we already know that we do not
have any Bearer token to work with.

Such a scenario happens e.g. when using Git to push to Visual Studio
Team Services (which supports Basic and Bearer authentication among
other methods) and specifying the Personal Access Token directly in the
URL (this aproach is frequently taken by automated builds).

Let's make sure that we have a Bearer token to work with before we
select the Bearer authentication among the available authentication
methods.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes #2754
2018-07-24 10:11:25 +02:00
Daniel Stenberg
7b9bc96c77
http2: several cleanups
- separate easy handle from connections better
- added asserts on a number of places
- added sanity check of pipelines for debug builds

Closes #2751
2018-07-20 22:58:42 +02:00
Daniel Stenberg
5ae22272d1
Curl_debug: remove dead printhost code
The struct field is never set (since 5e0d9aea3) so remove the use of it
and remove the connectdata pointer from the prototype.

Reported-by: Tejas
Bug: https://curl.haxx.se/mail/lib-2018-06/0054.html
Closes #2647
2018-06-12 16:00:27 +02:00
Marian Klymov
c45360d463
cppcheck: fix warnings
- Get rid of variable that was generating false positive warning
(unitialized)

- Fix issues in tests

- Reduce scope of several variables all over

etc

Closes #2631
2018-06-11 11:14:48 +02:00
Rikard Falkeborn
8ea5d41fe0
strictness: correct {infof, failf} format specifiers
Closes #2623
2018-05-31 11:34:32 +02:00
Linus Lewandowski
239a7061f8
httpauth: add support for Bearer tokens
Closes #2102
2018-05-24 20:39:49 +02:00
Patrick Monnerat
09d16af49a http resume: skip body if http code 416 (range error) is ignored.
This avoids appending error data to already existing good data.

Test 92 is updated to match this change.
New test 1156 checks all combinations of --range/--resume, --fail,
Content-Range header and http status code 200/416.

Fixes #1163
Reported-By: Ithubg on github
Closes #2578
2018-05-22 13:23:02 +02:00
Daniel Stenberg
cb529b713f
checksrc: make sure sizeof() is used *with* parentheses
... and unify the source code to adhere.

Closes #2563
2018-05-21 23:21:47 +02:00
Rikard Falkeborn
eb49683e55
lib: Fix format specifiers 2018-05-14 09:42:27 +02:00
Daniel Stenberg
8c7b3737d2
http: restore buffer pointer when bad response-line is parsed
... leaving the k->str could lead to buffer over-reads later on.

CVE: CVE-2018-1000301
Assisted-by: Max Dymond

Detected by OSS-Fuzz.
Bug: https://curl.haxx.se/docs/adv_2018-b138.html
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7105
2018-05-14 07:40:31 +02:00