With this option, the entire download is skipped if the selected target
filename already exists when the opertion is about to begin.
Test 994, 995 and 996 verify.
Ref: #11012Closes#13993
Use these words and casing more consistently across text, comments and
one curl tool output:
AIX, ALPN, ANSI, BSD, Cygwin, Darwin, FreeBSD, GitHub, HP-UX, Linux,
macOS, MS-DOS, MSYS, MinGW, NTLM, POSIX, Solaris, UNIX, Unix, Unicode,
WINE, WebDAV, Win32, winbind, WinIDN, Windows, Windows CE, Winsock.
Mostly OS names and a few more.
Also a couple of other minor text fixups.
Closes#14360
Since the documentation text blob might be gzipped, it needs to search
for what to output in a streaming manner. It then first searches for
"\nALL OPTIONS".
Then, it looks for the start to display at "\n -[option]" and stops
again at "\n -". Except for the last option in the man page, which
ends at "\nFILES" - the subtitle for the section following all options
in the manpage.
Test 1707 to 1710 verify
Closes#13997
... or pick the last directory part from the path if available.
Instead of returning error.
Add test 690 and 691 to verify. Test 76 and 2036 no longer apply.
Closes#13988
- tidy-up comments.
- use lowercase, underscore prefixed names for internal variables.
- use `IN LISTS` and `IN ITEMS` in `foreach()` loops.
- rename variable name `OUTPUT` to a more distinctive one.
- tidy-up `STREQUAL` syntax.
- delete commented code.
- indent/whitespace.
Closes#14197
Do no more than 5 transfers per 15 seconds with "5/15s" or limit it to 3
transfers per 4 hours with "3/4h" etc.
Previously it would always only work with a single time unit.
Ref: #14242Closes#14245
Add the ability to embed a CA bundle into the curl binary. It is used
when no other runtime or build-time option set one.
This helps curl-for-win macOS and Linux builds to run standalone, and
also helps Windows builds to avoid picking up the CA bundle from an
arbitrary (possibly world-writable) location (though this behaviour is
not currently disablable).
Usage:
- cmake: `-DCURL_CA_EMBED=/path/to/curl-ca-bundle.crt`
- autotools: `--with-ca-embed=/path/to/curl-ca-bundle.crt`
- Makefile.mk: `CURL_CA_EMBED=/path/to/curl-ca-bundle.crt`
Also add new command-line option `--dump-ca-embed` to dump the embedded
CA bundle to standard output.
Closes#14059
- shows how to pass on local variables (better)
- start the transfers nicer (with curl_multi_socket_action)
- consistent and helpful function naming - to better show what functions
and callbacks that are used for what
- build warning-free with gcc -W -Wall -pedantic
Closes#14287
Because it is no longer needed to be done by a person as the dmaketgz
script does it by itself.
Removed two past release dates, added two new future ones
Closes#14267
- make DEFAULT sections less repetitive
- make historic mentions use HISTORY
- generate the protocols section on `# %PROTOCOLS%` instead of guessing
where to put it
- generate the availability section on `# %AVAILABILITY%` instead of
guessing where to put it
- make the protocols section more verbose
Closes#14227
- generate AVAILABILITY manpage sections automatically - for consistent
wording
- allows us to double-check against other documumentation (symbols-in-versions
etc)
- enables proper automation/scripting based on this data
- lots of them were wrong or missing in the manpages
- several of them repeated (sometimes mismatching) backend support info
Add test 1488 to verify "added-in" version numbers against
symbols-in-versions.
Closes#14217
The man pages for curl_easy_getinfo, curl_easy_setopt and
curl_multi_setopt now feature the lists of options alphabetically
sorted. Test 1139 verify that they are.
The curl_multi_setopt page also got brief explanations of the listed
options.
Closes#14156
Simplify making clean builds by silencing deprecation warnings inside
the example code where these may occur.
Drop related build tweaks/comments from GHA jobs.
Example warning:
```
curl/docs/examples/postit2-formadd.c:65:16: error: 'CURLFORM_COPYNAME' is deprecated: since 7.56.0. Use curl_mime_name() [-Werror=deprecated-declarations]
65 | CURLFORM_COPYNAME, "sendfile",
| ^~~~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9841099503/job/27166970904#step:10:829Closes#14123
Option cleanups:
--get is not upload
--form* are post
- added several options into ldap, smtp, imap and pop3
- shortened the category descriptions in the list
category curl fixes:
--create-dirs removed from 'curl'
--ftp-create-dirs removed from 'curl'
--netrc moved to 'auth' from 'curl'
--netrc-file moved to 'auth' from 'curl'
--netrc-optional moved to 'auth' from 'curl'
--no-buffer moved to 'output' from 'curl'
--no-clobber removed from 'curl'
--output removed from 'curl'
--output-dir removed from 'curl'
--remove-on-error removed from 'curl'
Add a "global" category:
- Made all "global" options set this category
Add a "deprecated" category:
- Moved the deprecated options to it (maybe they should not be in any
category long term)
Add a 'timeout' category
- Put a number of appropriate options in it
Add an 'ldap' category
- Put the LDAP related option in there
Remove categories "ECH" and "ipfs"
- They should not be categories. Had only one single option each.
Remove category "misc"
- It should not be a category as it is impossible to know when to browse
it.
--use-ascii moved to ftp and output
--xattr moved to output
--service-name moved to auth
Managen fixes:
- errors if an option is given a category name that is not already setup
for in code
- verifies that options set `scope: global` also is put in category
`global´
Closes#14101
Also mention in HTTP3.md
OpenSSL has a bug that messes the config `--libdir=path` to become the
wrong path in its pkgconfig files. If we just pass `--libdir=lib` it
should avoid this.
Ref: #14099
See also: https://github.com/openssl/openssl/issues/23569Closes#14102
- Add --output, --remove-on-error, --output-dir and --created-dirs to
the output help category
- Add --hostpubmd5, --hostpubsha256, --insecure (-k), and --pubkey to
the ssh help category
Closes#14076
GnuTLS todo item about using an equivalent of `SSL_peak()`, which
nicely escaped the word checks, is no longer relevant.
We do not use `SSL_peek()` anymore since connection filters were
introduced.
Closes#14091
Based on the standards and guidelines we use for our documentation.
- expand contractions (they're => they are etc)
- host name = > hostname
- file name => filename
- user name = username
- man page => manpage
- run-time => runtime
- set-up => setup
- back-end => backend
- a HTTP => an HTTP
- Two spaces after a period => one space after period
Closes#14073
Previously this was one single manpage for two functions but as they are
two separate ones since a while back, they should each clearly document
their single specific functions.
Follow-up to eefcc1bda4Closes#14068
- use imperative form
- use lowercase
- no period
- unify some phrases
- fix curl_multi_socket and curl_multi_socket_all to keep their own
descriptions
Closes#14062
These have lingered in the issue tracker for a long time without action.
We don't expect any fixes in the near term either. Move them to the
KNOWN_BUGS document.
Closes#12177Closes#12171Closes#13350Closes#14042
Fix issues detected.
Also:
- One of the `.vc` files used LF EOLs, while the other didn't.
Make that one also use LF EOLs, as this is apparently supported by
`nmake`.
- Drop `.dsw` and `.btn` types from `.gitattributes`.
The repository doesn't use them.
- Sync section order with the rest of files in
`tests/certs/EdelCurlRoot-ca.prm`.
- Indent/align `.prm` and `.pem` files.
- Delete dummy `[something]` section from `.prm` and `.pem` files.
Mental note:
MSVC `.sln` files seem to accept spaces for indentation and also support
LF line-endings. I cannot test this and I don't know what's more
convenient when updating them, so left them as-is, with specific
exclusions.
Closes#14031
When libcurl discards a connection there are two phases this may go
through: "shutdown" and "closing". If a connection is aborted, the
shutdown phase is skipped and it is closed right away.
The connection filters attached to the connection implement the phases
in their `do_shutdown()` and `do_close()` callbacks. Filters carry now a
`shutdown` flags next to `connected` to keep track of the shutdown
operation.
Filters are shut down from top to bottom. If a filter is not connected,
its shutdown is skipped. Notable filters that *do* something during
shutdown are HTTP/2 and TLS. HTTP/2 sends the GOAWAY frame. TLS sends
its close notify and expects to receive a close notify from the server.
As sends and receives may EAGAIN on the network, a shutdown is often not
successful right away and needs to poll the connection's socket(s). To
facilitate this, such connections are placed on a new shutdown list
inside the connection cache.
Since managing this list requires the cooperation of a multi handle,
only the connection cache belonging to a multi handle is used. If a
connection was in another cache when being discarded, it is removed
there and added to the multi's cache. If no multi handle is available at
that time, the connection is shutdown and closed in a one-time,
best-effort attempt.
When a multi handle is destroyed, all connection still on the shutdown
list are discarded with a final shutdown attempt and close. In curl
debug builds, the environment variable `CURL_GRACEFUL_SHUTDOWN` can be
set to make this graceful with a timeout in milliseconds given by the
variable.
The shutdown list is limited to the max number of connections configured
for a multi cache. Set via CURLMOPT_MAX_TOTAL_CONNECTIONS. When the
limit is reached, the oldest connection on the shutdown list is
discarded.
- In multi_wait() and multi_waitfds(), collect all connection caches
involved (each transfer might carry its own) into a temporary list.
Let each connection cache on the list contribute sockets and
POLLIN/OUT events it's connections are waiting for.
- in multi_perform() collect the connection caches the same way and let
them peform their maintenance. This will make another non-blocking
attempt to shutdown all connections on its shutdown list.
- for event based multis (multi->socket_cb set), add the sockets and
their poll events via the callback. When `multi_socket()` is invoked
for a socket not known by an active transfer, forward this to the
multi's cache for processing. On closing a connection, remove its
socket(s) via the callback.
TLS connection filters MUST NOT send close nofity messages in their
`do_close()` implementation. The reason is that a TLS close notify
signals a success. When a connection is aborted and skips its shutdown
phase, the server needs to see a missing close notify to detect
something has gone wrong.
A graceful shutdown of FTP's data connection is performed implicitly
before regarding the upload/download as complete and continuing on the
control connection. For FTP without TLS, there is just the socket close
happening. But with TLS, the sent/received close notify signals that the
transfer is complete and healthy. Servers like `vsftpd` verify that and
reject uploads without a TLS close notify.
- added test_19_* for shutdown related tests
- test_19_01 and test_19_02 test for TCP RST packets
which happen without a graceful shutdown and should
no longer appear otherwise.
- add test_19_03 for handling shutdowns by the server
- add test_19_04 for handling shutdowns by curl
- add test_19_05 for event based shutdowny by server
- add test_30_06/07 and test_31_06/07 for shutdown checks
on FTP up- and downloads.
Closes#13976
- up the limit: remove all mentions of 7.60 or earlier from manpage
7.60 is 6 years old now.
- warn on "broken" added in lines, as they avoid detection
- fixup added in markup in a few curldown files
Closes#14002
Support "curl -h --insecure" etc to output the manpage section for the
--insecure command line option in the terminal. Should be possible to
work with either long or short versions of command line options.
Closes#13990
If a malicious server can trigger a NULL dereference in curl or
otherwise cause curl to crash (and nothing worse), chances are big that
we do not consider that a security problem.
Closes#13974
There are a places where man pages reference deprecated CURLOPT options,
where it doesn't make sense, replace them with the reccomended
replacement option.
also remove reference to the removed mesalink TLS backend
Closes#13951
- cmake: populate for dependencies.
- autotools: populate for dependencies.
(including mbedtls, though the script does not detect
mbedtls through pkgconfig. mbedtls 3.6.0 now supports it.)
Skip dealing with gssapi in this patch.
Fixes#864Closes#13911
This adds connection shutdown infrastructure and first use for FTP. FTP
data connections, when not encountering an error, are now shut down in a
blocking way with a 2sec timeout.
- add cfilter `Curl_cft_shutdown` callback
- keep a shutdown start timestamp and timeout at connectdata
- provide shutdown timeout default and member in
`data->set.shutdowntimeout`.
- provide methods for starting, interrogating and clearing
shutdown timers
- provide `Curl_conn_shutdown_blocking()` to shutdown the
`sockindex` filter chain in a blocking way. Use that in FTP.
- add `Curl_conn_cf_poll()` to wait for socket events during
shutdown of a connection filter chain.
This gets the monitoring sockets and events via the filters
"adjust_pollset()" methods. This gives correct behaviour when
shutting down a TLS connection through a HTTP/2 proxy.
- Implement shutdown for all socket filters
- for HTTP/2 and h2 proxying to send GOAWAY
- for TLS backends to the best of their capabilities
- for tcp socket filter to make a final, nonblocking
receive to avoid unwanted RST states
- add shutdown forwarding to happy eyeballers and
https connect ballers when applicable.
Closes#13904
Multipath TCP (MPTCP), standardized in RFC8684 [1], is a TCP extension
that enables a TCP connection to use different paths.
Multipath TCP has been used for several use cases. On smartphones, MPTCP
enables seamless handovers between cellular and Wi-Fi networks while
preserving established connections. This use-case is what pushed Apple
to use MPTCP since 2013 in multiple applications [2]. On dual-stack
hosts, Multipath TCP enables the TCP connection to automatically use the
best performing path, either IPv4 or IPv6. If one path fails, MPTCP
automatically uses the other path.
To benefit from MPTCP, both the client and the server have to support
it. Multipath TCP is a backward-compatible TCP extension that is enabled
by default on recent Linux distributions (Debian, Ubuntu, Redhat, ...).
Multipath TCP is included in the Linux kernel since version 5.6 [3]. To
use it on Linux, an application must explicitly enable it when creating
the socket. No need to change anything else in the application.
This attached patch adds an --mptcp option which allows the creation of
an MPTCP socket instead of TCP on Linux. If Multipath TCP is not
supported on the system, an error will be reported. It is important to
note that if the end server doesn't support MPTCP, the connection will
continue after a seamless fallback to TCP.
Link: https://www.rfc-editor.org/rfc/rfc8684.html [1]
Link: https://www.tessares.net/apples-mptcp-story-so-far/ [2]
Link: https://www.mptcp.dev [3]
Co-developed-by: Dorian Craps (@CrapsDorian) <doriancraps@gmail.com>
Co-developed-by: Olivier Bonaventure (@obonaventure) <Olivier.Bonaventure@uclouvain.be>
Co-developed-by: Matthieu Baerts (@matttbe) <matttbe@kernel.org>
Signed-off-by: Dorian Craps <dorian.craps@student.vinci.be>
Closes#13278
Introduce new notation for CURLOPT_INTERFACE / --interface:
ifhost!<interface>!<host>
Binding to an interface doesn't set the address, and an interface can
have multiple addresses.
When binding to an address (without interface), the kernel is free to
choose the route, and it can route through any device that can access
the target address, not necessarily the one with the chosen address.
Moreover, it is possible for different interfaces to have the same IP
address, on which case we need to provide a way to be more specific.
Factor out the parsing part of interface option, and add unit tests:
1663.
Closes#13719
- Remove the locking callback code that demonstrates how to meet
requirements of threading libraries (mainly OpenSSL).
Locking callback code has not been needed for many years. According to
the documentation for OpenSSL and GnuTLS they are thread-safe by design,
assuming support for the underlying OS threading API is built-in.
Ref: https://github.com/curl/curl/pull/13850#issuecomment-2143538458
Closes https://github.com/curl/curl/pull/13851
- similar to openssl, use a shared 'credentials' instance
among TLS connections with a plain configuration.
- different to openssl, a connection with a client certificate
is not eligible to sharing.
- document CURLOPT_CA_CACHE_TIMEOUT in man page
Closes#13795
Used for extracting:
- when used asking for a scheme, it will return CURLUE_NO_SCHEME if the
stored information was a guess
- when used asking for a URL, the URL is returned without a scheme, like
when previously given to the URL parser when it was asked to guess
- as soon as the scheme is set explicitly, it is no longer internally
marked as guessed
The idea being:
1. allow a user to figure out if a URL's scheme was set as a result of
guessing
2. extract the URL without a guessed scheme
3. this makes it work similar to how we already deal with port numbers
Extend test 1560 to verify.
Closes#13616
Like when we list a series of options and then want to add "normal" text
again afterwards.
Without this, the indentation level wrongly continues even after the
final "##" header, making following text wrongly appear to belong to the
header above.
Adjusted several curldown files to use this.
Fixes#13803
Reported-by: Jay Satiro
Closes#13806
Delete a bunch of unnecessary-looking headers from some examples. This
is known to be tricky on AIX (perhaps also in other less-tested envs).
Let me know if any of this looks incorrect or outright fails on some
systems.
Follow-up to d4b8589055#13771Closes#13785
- `websocket.c`: use `Sleep()` on Windows.
`sleep()` and `unistd.h` are not available in MSVC.
- `http2-upload.c`: use local `gettimeofday()` implementation when
compiled with MSVC.
(Alternate solution is to disable the trace function for MSVC.)
Public domain code copied and adapted from libssh2:
e973493f99/src/misc.c (L719-L743)
- silence compiler warning for deprecated `inet_addr()`.
Also drop duplicate winsock2 include.
```
curl\docs\examples\externalsocket.c(125,32): error C2220: the following warning is treated as an error [curl\bld\docs\examples\curl-example-externalsocket.vcxproj]
curl\docs\examples\externalsocket.c(125,32): warning C4996: 'inet_addr': Use inet_pton() or InetPton() instead or define _WINSOCK_DEPRECATED_NO_WARNINGS to disable deprecated API warnings [curl\bld\docs\examples\curl-example-e
```
Ref: https://github.com/curl/curl/actions/runs/9227337318/job/25389073450#step:4:95
- silence an MSVC compiler warning. This is in conflict with `checksrc`
rules, so silence the rule in favour of the warning-free C syntax.
```
curl\docs\examples\multi-legacy.c(152,1): error C2220: the following warning is treated as an error [curl\bld\docs\examples\curl-example-multi-legacy.vcxproj]
curl\docs\examples\multi-legacy.c(152,1): warning C4706: assignment within conditional expression [curl\bld\docs\examples\curl-example-multi-legacy.vcxproj]
```
Ref: https://github.com/curl/curl/actions/runs/9227337318/job/25389073450#step:4:226
- do not use `sys/time.h` and `unistd.h` in Windows builds.
Some of these includes look unnecessary. Subject to another PR.
Cherry-picked from #13766Closes#13771
- Revert to the legacy TLS 1.2 key logging code for LibreSSL.
- Document SSLKEYLOGFILE for LibreSSL is TLS 1.2 max.
Prior to this change if the user specified a filename in the
SSLKEYLOGFILE environment variable and was using LibreSSL 3.5.0+ then
an empty file would be created and no keys would be logged.
This is effectively a revert of e43474b4 which changed openssl.c to use
SSL_CTX_set_keylog_callback for LibreSSL 3.5.0+. Unfortunately LibreSSL
added that function only as a stub that doesn't actually do anything.
Reported-by: Gonçalo Carvalho
Fixes https://github.com/curl/curl/issues/13672
Closes https://github.com/curl/curl/pull/13682
curl drops support for TLS libraries without TLS 1.3 capability after
May 2025.
It requires that a curl build using the library should be able to
negotiate and use TLS 1.3, or else it is not good enough. We support a
vast amount of other TLS libraries that are likely to satisfy users
better.
Closes#13544
- Clear data->state.os_errno before transfer.
- Explain the change in behavior in the CURLINFO_OS_ERRNO doc.
- Add to the CURLINFO_OS_ERRNO doc the list of libcurl network-related
errors that may cause the errno to be saved.
data->state.os_errno is saved before libcurl returns a network-related
failure such as connection failure. It is accessible to the user via
CURLINFO_OS_ERRNO so they can get more information about the failure.
Prior to this change it wasn't cleared before transfer, so if a user
retrieved the saved errno it could be from a previous transfer. That is
because an errno is not always saved for network-related errors.
Closes https://github.com/curl/curl/pull/13574
CURLOPT_EGDSOCKET and CURLOPT_RANDOM_FILE are both completely dead
so remove their example sections since the code there is useless.
There is still a way to inject a random file for OpenSSL older than
1.1.0 but it's not what the example showed (and it's not even done
with this option) so we refrain from documenting it here.
Closes: #13540
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
