Next Protocol Negotiation is a TLS extension that was created and used
for agreeing to use the SPDY protocol (the precursor to HTTP/2) for
HTTPS. In the early days of HTTP/2, before the spec was finalized and
shipped, the protocol could be enabled using this extension with some
servers.
curl supports the NPN extension with some TLS backends since then, with
a command line option `--npn` and in libcurl with
`CURLOPT_SSL_ENABLE_NPN`.
HTTP/2 proper is made to use the ALPN (Application-Layer Protocol
Negotiation) extension and the NPN extension has no purposes
anymore. The HTTP/2 spec was published in May 2015.
Today, use of NPN in the wild should be extremely rare and most likely
totally extinct. Chrome removed NPN support in Chrome 51, shipped in
June 2016. Removed in Firefox 53, April 2017.
Closes#9307
As "(aq" and "(dq" to prevent them from implying a meaning in the nroff
output. This removes the need for using \& escapes in the .d files'
description parts.
Closes#9352
The example was missing a "--form" argument
I also replaced "--form" with "-F" to shorten the line a bit since it
was already very long.
And I also moved --form-escape from the "post" category to the "upload"
category (this is what I originally wanted to fix, before also noticing
the mistake in the example).
Closes#9298
This commit adds the dns category to the --resolve command line option,
because it can be interpreted as both: a low-level connection option and
an option related to the resolving of a hostname.
It is also not common for dns options to belong to the connection
category and vice versa. --ipv4 and --ipv6 are both good examples.
Closes#9229
On some systems, the gen.pl script takes nearly two minutes for the
generation of the main-page, which is a completely unacceptable time.
The slow performance has two causes:
1. Use of a regex locale operator
2. Useless invokations of loops
The commit addresses the first issue by replacing the "\W" wiht
[^a-zA-Z0-9_], which is, according to regex101.com, functionally
equivalent to the previous operation, except that it is obviously
limited to ASCII only, which is fine, as the curl project is
English-only anyway.
The second issue is being addressed by only running the loop if the line
contains a "--" in it. The loop may be completeley removed in the
future.
Co-authored-by: Emanuele Torre <torreemanuele6@gmail.com>
See #8299Fixes#9230Closes#9232
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.
This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.
Closes#8869
Prior to this change the doc said --max-time set the maximum time of the
'whole operation' which is not accurate. The option maps to
CURLOPT_TIMEOUT_MS which sets maximum transfer time.
For example, the maximum time on a transfer is reset if the transfer is
retried (--retry).
Reported-by: Nuru@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/8877Closes#8879
Usage:
curl -x "socks5h://localhost/run/tor/socks" "https://example.com"
Updated runtests.pl to run a socksd server listening on unix socket
Added tests test1467 test1468
Added documentation for proxy command line option and socks proxy
options
Closes#8668
Instead of saying "This option overrides NNN", now say "This option is
mutually exclusive to NNN" in the generated man page ouput, as the
option does not in all cases actually override the others but they are
always mutually exclusive.
Ref: #8704Closes#8716
Mostly based on recent language decisions from "everything curl":
- remove contractions (isn't => is not)
- *an* HTTP (consistency)
- runtime (no hyphen)
- backend (no hyphen)
- URL is uppercase
Closes#8646
If a transfer returns an error, using this option makes curl remove the
leftover downloded (partial) local file before exiting.
Added test 376 to verify
Closes#8503
Recent changes added support for filename .curlrc on Windows, and
when it's not found curl falls back on the original Windows filename
_curlrc. _curlrc was removed from the doc, however it is still valid.
Closes https://github.com/curl/curl/pull/8382
Adds these test cases:
383 - simple single command line option
384 - reading it from stdin
385 - getting two --json options on command line
386 - --next works after --json
Closes#8314
Previously it could mistakenly match partial names when there are
options that start with the same prefix, leading to the wrong references
used.
Closes#8299
As credentials can be quite different depending on the mechanism used,
there are no default mechanisms for LDAP and simple bind with a DN is
then used.
The caller has to provide mechanism(s) using CURLOPT_LOGIN_OPTIONS to
enable SASL authentication and disable simple bind.
Closes#8152
Mesalink has ceased development. We can no longer encourage use of it.
It seems to be continued under the name TabbySSL, but no attempts have
(yet) been to make curl support it.
Fixes#8188Closes#8191
Remove the lines saying "protocols: all". It makes the output in the
manpage look funny, and the expectation is probably by default that if
not anything is mentioned about protocols the option apply to them all.
Closes#8021
gen.pl now generates a warning if the "See Also" field is not filled in for a
command line option
All command line options now provide one or more related options. 167
"See alsos" added!
Closes#8019
Treat consecutive lines that start with a space to be "examples". They
are output enclosed by .nf and .fi
Updated form.d to use this new fanciness
Closes#8016
Until now, form field and file names where escaped using the
backslash-escaping algorithm defined for multipart mails. This commit
replaces this with the percent-escaping method for URLs.
As this may introduce incompatibilities with server-side applications, a
new libcurl option CURLOPT_MIME_OPTIONS with bitmask
CURLMIMEOPT_FORMESCAPE is introduced to revert to legacy use of
backslash-escaping. This is controlled by new cli tool option
--form-escape.
New tests and documentation are provided for this feature.
Reported by: Ryan Sleevi
Fixes#7789Closes#7805
... and allow single quotes to be used "normally" in the .d files.
Makes the output curl.1 use better nroff.
Reported-by: Sergio Durigan Junior
Ref: #7928Closes#7933
tool_listhelp.c is now a separate file with only the command line --help
output, exactly as generated by gen.pl. This makes it easier to generate
updates according to what's in the docs/cmdline-opts docs.
cd $srcroot/docs/cmdline-opts
./gen.pl listhelp *.d > $srcroot/src/tool_listhelp.c
With a configure build, this also works:
make -C src listhelp
Closes#7787
Follow-up to 15910dfd14
The previous strftime format used didn't work correctly on Windows, so
change to %B %d %Y which today looks like "September 29 2021".
Reported-by: Gisle Vanem
Bug: #7782Closes#7793
Since "too old" versions are no longer included in the generated man
page, this field is now mandatory so that it won't be forgotten and then
not included in the documentation.
Closes#7786
To make the man page more readable, this change removes all references
to changes in support/versions etc that happened before 7.30.0 from the
curl.1 output file. 7.30.0 was released on Apr 12 2013. This particular
limit is a bit arbitrary but was fairly easy to grep for.
It is handled like this: the 'Added' keyword is only used in output if
it refers to 7.30.0 or later. All occurances of "(Added in $VERSION)" in
description will be stripped out if the mentioned $VERSION is from
before 7.30.0. It is therefore important that the "Added in..."
references are always written exactly like that - and on a single line,
not split over two.
This change removes about 80 version number references from curl.1, down
to 138 from 218.
Closes#7786
The file format for each option now features a "Example:" header that
can provide one or more examples that get rendered appropriately in the
output. All options MUST have at least one example or gen.pl complains
at build-time.
This fix also does a few other minor format and consistency cleanups.
Closes#7654
Since this option is also used for FTP, it needs to work to set for
applications even if hyper doesn't support it for HTTP. Verified by test
1137.
Updated docs to specify that the option doesn't work for HTTP when using
the hyper backend.
Closes#7614
Mention options that are "global". A global command line option is one
that doesn't get reset at --next uses and therefore don't need to be
used again.
Reported-by: Josh Soref
Fixes#7457Closes#7510
- Add protocols field to max-filesize.d.
- Revert wording on unknown file size caveat and do not discuss specific
protocols in that section.
Partial revert of ecf0225. All max-filesize options now have the list of
protocols and it's clearer just to have that list without discussing
specific protocols in the caveat.
Reported-by: Josh Soref
Ref: https://github.com/curl/curl/issues/7453#issuecomment-884128762
Also make it clearer that the caveat 'if the file size is unknown it
the option will have no effect' may apply to protocols other than FTP
and HTTP.
Reported-by: Josh Soref
Fixes https://github.com/curl/curl/issues/7453
Since --cookie-jar is the preferred way to store cookies, no longer
suggest using --dump-header to do so.
Co-authored-by: Daniel Stenberg
Closes https://github.com/curl/curl/issues/7414
Write out directories rather than using the dirs abbrevation. Also
use plural form consistently, even if the code in the end might just
create a single directory.
Closes#7406
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
When using curl with the option `--etag-save` I expected it to save the
ETag without its surrounding quotes, as stated by the documentation in
the repository and by the generated man pages.
My first endeavour was to fix the program, but while investigating the
history of the relevant parts, I discovered that curl once saved the
ETag without the quotes. This was undone by Daniel Stenberg in commit
`98c94596f5928840177b6bd3c7b0f0dd03a431af`, therefore I decided that in
this case the documentation should be adjusted to match the behaviour of
curl.
The changed save behaviour also made parts of the `--etag-compare`
documentation wrong or superfluous, so I adjusted those accordingly.
Closes#7429
Warning: this will make existing curl command lines that use metalink to
stop working.
Reasons for removal:
1. We've found several security problems and issues involving the
metalink support in curl. The issues are not detailed here. When
working on those, it become apparent to the team that several of the
problems are due to the system design, metalink library API and what
the metalink RFC says. They are very hard to fix on the curl side
only.
2. The metalink usage with curl was only very briefly documented and was
not following the "normal" curl usage pattern in several ways, making
it surprising and non-intuitive which could lead to further security
issues.
3. The metalink library was last updated 6 years ago and wasn't so
active the years before that either. An unmaintained library means
there's a security problem waiting to happen. This is probably reason
enough.
4. Metalink requires an XML parsing library, which is complex code (even
the smaller alternatives) and to this day often gets security
updates.
5. Metalink is not a widely used curl feature. In the 2020 curl user
survey, only 1.4% of the responders said that they'd are using it. In
2021 that number was 1.2%. Searching the web also show very few
traces of it being used, even with other tools.
6. The torrent format and associated technology clearly won for
downloading large files from multiple sources in parallel.
Cloes #7176
... or the cookies won't get sent. Push users to using the "Netscape"
format instead, which curl uses when saving a cookie "jar".
Reported-by: Martin Dorey
Reviewed-by: Daniel Gustafsson
Fixes#6723Closes#7077
- Disable auto credentials by default. This is a breaking change
for clients that are using it, wittingly or not.
- New libcurl ssl option value CURLSSLOPT_AUTO_CLIENT_CERT tells libcurl
to automatically locate and use a client certificate for
authentication, when requested by the server.
- New curl tool options --ssl-auto-client-cert and
--proxy-ssl-auto-client-cert map to CURLSSLOPT_AUTO_CLIENT_CERT.
This option is only supported for Schannel (the native Windows SSL
library). Prior to this change Schannel would, with no notification to
the client, attempt to locate a client certificate and send it to the
server, when requested by the server. Since the server can request any
certificate that supports client authentication in the OS certificate
store it could be a privacy violation and unexpected.
Fixes https://github.com/curl/curl/issues/2262
Reported-by: Jeroen Ooms
Assisted-by: Wes Hinsley
Assisted-by: Rich FitzJohn
Ref: https://curl.se/mail/lib-2021-02/0066.html
Reported-by: Morten Minde Neergaard
Closes https://github.com/curl/curl/pull/6673
Instead output a warning about it and continue with the defaults.
These SSL versions are typically not supported by the TLS libraries since a
long time back already since they are inherently insecure and broken. Asking
for them to be used will just cause an error to be returned slightly later.
In the unlikely event that a user's TLS library actually still supports these
protocol versions, this change might make the request a little less insecure.
Closes#6772
- Document in DOH that some SSL settings are inherited but DOH hostname
and peer verification are not and are controlled separately.
- Document that CURLOPT_SSL_CTX_FUNCTION is inherited by DOH handles but
we're considering changing behavior to no longer inherit it. Request
feedback.
Closes https://github.com/curl/curl/pull/6688
Prior to this change the required argument was not shown.
curl.1 before: --stderr
curl.1 after: --stderr <file>
curl --help before:
--stderr Where to redirect stderr
curl --help after:
--stderr <file> Where to redirect stderr
Closes https://github.com/curl/curl/pull/6692
Previously only a single -b cookie parameter was supported with the last
one winning. This adds support for supplying multiple -b params to have
them serialized semicolon separated. Both cookiefiles and cookies can be
entered multiple times.
Closes#6649
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- add CURLINFO_REFERER libcurl option
- add --write-out '%{referer}' command-line option
- extend --xattr command-line option to fill user.xdg.referrer.url extended
attribute with the referrer (if there was any)
Closes#6591
- New libcurl options CURLOPT_DOH_SSL_VERIFYHOST,
CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the
same as their respective counterparts.
- New curl tool options --doh-insecure and --doh-cert-status do the same
as their respective counterparts.
Prior to this change DOH SSL certificate verification settings for
verifyhost and verifypeer were supposed to be inherited respectively
from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug
were not. As a result DOH verification remained at the default, ie
enabled, and it was not possible to disable. This commit changes
behavior so that the DOH verification settings are independent and not
inherited.
Ref: https://github.com/curl/curl/pull/4579#issuecomment-554723676
Fixes https://github.com/curl/curl/issues/4578
Closes https://github.com/curl/curl/pull/6597
- one source file per line
- convert tabs to spaces
- do not align line-continuation backslashes
- sort source files alphabetically
Reviewed-by: Daniel Stenberg
Closes#6593
- Deduplicate the logic used by write-out and write-out json.
Rather than have separate writeLong, writeString, etc, logic for
each of write-out and write-out json instead have respective shared
functions that can output either format and a 'use_json' parameter to
indicate whether it is json that is output.
This will make it easier to maintain. Rather than have to go through
two sets of logic now we only have to go through one.
- Support write-out %{errormsg} and %{exitcode} in json.
- Clarify in the doc that %{exitcode} is the exit code of the transfer.
Prior to this change it just said "The numerical exitcode" which
implies it's the exit code of the tool, and it's not necessarily that.
Closes https://github.com/curl/curl/pull/6544
In a url after <scheme>:// follows the possibly empty authority part
till the next /, so that url missed a /.
Closes#6573
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
- Add support services without region and service prefixes in
the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc)
by providing region and service parameters via aws-sigv4 option.
- Add [:region[:service]] suffix to aws-sigv4 option;
- Fix memory allocation errors.
- Refactor memory management.
- Use Curl_http_method instead() STRING_CUSTOMREQUEST.
- Refactor canonical headers generating.
- Remove repeated sha256_to_hex() usage.
- Add some docs fixes.
- Add some codestyle fixes.
- Add overloaded strndup() for debug - curl_dbg_strndup().
- Update tests.
Closes#6524
The script warns if the length of $opt and $desc is > 78. However, these
two variables are on totally separate lines so the check makes no sense.
Also the $bitmask field is totally forgotten. Currently this leads to
two warnings within `--resolve` and `--aws-sigv4`.
Closes#6438
We currently use both spellings the british "behaviour" and the american
"behavior". However "behavior" is more used in the project so I think
it's worth dropping the british name.
Closes#6395
