The 550 is overused as a return code for multiple error case, e.g.
file not found and/or insufficient permissions to access the file.
So we cannot fail hard in this case.
Adjust test 511 since we now fail later.
Add new test 3027 which check that when MDTM failed, but the file could
actually be retrieved, that in this case no filetime is provided.
Reported-by: Michael Heimpold
Fixes#9357Closes#9387
connect_init() (lib/http_proxy.c) swaps out the protocol state while
working on the proxy connection, this is then restored by
Curl_connect_done() after the connection completes.
ftp_do_more() extracted the protocol state pointer to a local variable
at the start of the function then calls Curl_proxy_connect(). If the proxy
connection completes, Curl_proxy_connect() will call Curl_connect_done()
(via Curl_proxyCONNECT()), which restores data->req.p to point to the ftp
protocol state instead of the http proxy protocol state, but the local
variable in ftp_do_more still pointed to the old value.
Ultimately this meant that the state worked on by ftp_do_more() was the
http proxy state not the ftp state initialised by ftp_connect(), but
subsequent calls to any ftp_ function would use the original state.
For my use-case, the visible consequence was that ftp->downloadsize was
never set and so downloaded data was never returned to the application.
This commit updates the ftp protocol state pointer in ftp_do_more() after
Curl_proxy_connect() returns, ensuring that the correct state pointer is
used.
Fixes#8737Closes#9043
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.
This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.
Closes#8869
Follow-up to 21248e052d
Disabling "assignment within conditional expression" for MSVC needs to
be done before the function starts, for it to take effect.
Closes#8218
If a server pipelines future responses within the STARTTLS response, the
former are preserved in the pingpong cache across TLS negotiation and
used as responses to the encrypted commands.
This fix detects pipelined STARTTLS responses and rejects them with an
error.
CVE-2021-22947
Bug: https://curl.se/docs/CVE-2021-22947.html
In imap and pop3, check if TLS is required even when capabilities
request has failed.
In ftp, ignore preauthentication (230 status of server greeting) if TLS
is required.
Bug: https://curl.se/docs/CVE-2021-22946.html
CVE-2021-22946
The reverted commit introduced a logic error in code that was
correct.
The client using libcurl would notice the error since FTP file
uploads in active transfer mode would somtimes complete with
success despite no transfer having been performed and the
"uploaded" file thus not being on the remote server afterwards.
The FTP server would notice the error because it receives a
RST on the data connection it has established with the client
before any data was transferred at all.
The logic error happens if the STOR response from the server have
arrived by the time ftp_multi_statemach() in the affected code path
is called, but the incoming data connection have not arrived yet.
In that case, the processing of the STOR response will cause
'ftpc->wait_data_conn' to be set to TRUE, contradicting the comment
in the code. Since 'complete' will also be set, later logic would
believe the transfer was done.
In most cases, the STOR response will not have arrived yet when
the affected code path is executed, or the incoming connection will
also have arrived, and thus the error would not express itself.
But if the speed difference of the device using libcurl and the
FTP server is exactly right, the error may happen as often as in
one out of hundred file transfers.
This reverts commit 49f3117a23.
Bug: https://curl.se/mail/lib-2021-07/0025.htmlCloses#7362
- the data needs to be "line-based" anyway since it's also passed to the
debug callback/application
- it makes infof() work like failf() and consistency is good
- there's an assert that triggers on newlines in the format string
- Also removes a few instances of "..."
- Removes the code that would append "..." to the end of the data *iff*
it was truncated in infof()
Closes#7357
The libssh2 backend has SSH session associated with the connection but
the callback context is the easy handle, so when a connection gets
attached to a transfer, the protocol handler now allows for a custom
function to get used to set things up correctly.
Reported-by: Michael O'Farrell
Fixes#6898Closes#7078
If after a transfer is complete Curl_GetFTPResponse() returns an error,
curl would not free the ftp->pathalloc block.
Found by torture-testing test 576
Closes#6737
and rename it from 'ftp_list_only' since it is also used for SSH and
POP3. The state is updated internally for 'type=D' FTP URLs.
Added test case 1570 to verify.
Closes#6578
... and make sure the code never updates 'set.prefer_ascii' as it breaks
handle reuse which should use the setting as the user specified it.
Added test 1569 to verify: it first makes an FTP transfer with ';type=A'
and then another without type on the same handle and the second should
then use binary. Previously, curl failed this.
Closes#6578
Since the set value then risks getting used like that when the easy
handle is reused by the application.
Also: renamed the struct field from 'ftp_append' to 'remote_append'
since it is also used for SSH protocols.
Closes#6579
... as we ignore it anyway because servers don't report the correct size
and proftpd even blatantly returns a 550.
Updates a set of tests accordingly.
Reported-by: awesomenode on github
Fixes#6564Closes#6565
... in most cases instead of 'struct connectdata *' but in some cases in
addition to.
- We mostly operate on transfers and not connections.
- We need the transfer handle to log, store data and more. Everything in
libcurl is driven by a transfer (the CURL * in the public API).
- This work clarifies and separates the transfers from the connections
better.
- We should avoid "conn->data". Since individual connections can be used
by many transfers when multiplexing, making sure that conn->data
points to the current and correct transfer at all times is difficult
and has been notoriously error-prone over the years. The goal is to
ultimately remove the conn->data pointer for this reason.
Closes#6425
In the case of TFO, the remote host name is not resolved at the
connetion time.
For FTP that has lead to missing hostname for the secondary connection.
Therefore the name resolution is done at the time, when FTP requires it.
Fixes#6252Closes#6265Closes#6282
Added test 348 to verify. Added a 'STOR' command to the test FTP
server to enable test 348. Documented the command in FILEFORMAT.md
Reported-by: Duncan Wilcox
Fixes#6016Closes#6017
When using HTTPS proxy, SSL is used but not in the view of the FTP
protocol handler itself so separate the connection's use of SSL from the
FTP control connection's sue.
Reported-by: Mingtao Yang
Fixes#5523Closes#6006
... reuses the same dynamic buffer instead of doing repeated malloc/free
cycles.
Test case 100 (FTP dir list PASV) does 7 fewer memory allocation calls
after this change in my test setup (132 => 125), curl 7.72.0 needed 140
calls for this.
Test case 103 makes 9 less allocations now (130). Down from 149 in
7.72.0.
Closes#6004
These two files were always tightly connected and it was hard to
understand what went into which. This also allows us to make the
ftpsend() function static (moved from ftp.c).
Removed security.c
Renamed curl_sec.h to krb5.h
Closes#5987
If the received PASV response doesn't match the expected pattern, we
could end up reading uninitialized integers for IP address and port
number.
Issue pointed out by muse.dev
Closes#5972
This is primarily interesting for cases where CURLOPT_NOBODY is set as
previously curl would not return an error for this case.
MDTM getting 550 now also returns this error (it returned
CURLE_FTP_COULDNT_RETR_FILE before) in order to unify return codes for
missing files across protocols and specific FTP commands.
libcurl already returns error on a 550 as a MDTM response (when
CURLOPT_FILETIME is set). If CURLOPT_NOBODY is not set, an error would
happen subsequently anyway since the RETR command would fail.
Add test 1913 and 1914 to verify. Updated several tests accordingly due
to the updated SIZE behavior.
Reported-by: Tomas Berger
Fixes#5953Closes#5957
The shutdown function is for downgrading a connection from TLS to plain,
and this is not requested here.
Have ssl_close reset the TLS connection state.
This partially reverts commit f002c850d9
Reported-by: Rasmus Melchior Jacobsen
Reported-by: Denis Goleshchikhin
Fixes#5797
Updated terminology in docs, comments and phrases to refer to C strings
as "null-terminated". Done to unify with how most other C oriented docs
refer of them and what users in general seem to prefer (based on a
single highly unscientific poll on twitter).
Reported-by: coinhubs on github
Fixes#5598Closes#5608
- Stick to a single unified way to use structs
- Make checksrc complain on 'typedef struct {'
- Allow them in tests, public headers and examples
- Let MD4_CTX, MD5_CTX, and SHA256_CTX typedefs remain as they actually
typedef different types/structs depending on build conditions.
Closes#5338
Previously, after PASV and immediately after the data connection has
connected, the function would only return the control socket to wait for
which then made the data connection simply timeout and not get polled
correctly. This become obvious when running test 1631 and 1632 event-
based.
... as this is already done much earlier in the URL parser.
Also add test case 894 that verifies that pop3 with an encodedd CR in
the user name is rejected.
Closes#4887
... and use internally. This function will return TIME_T_MAX instead of
failure if the parsed data is found to be larger than what can be
represented. TIME_T_MAX being the largest value curl can represent.
Reviewed-by: Daniel Gustafsson
Reported-by: JanB on github
Fixes#4152Closes#4651
This is a protocol violation but apparently there are legacy proprietary
servers doing this.
Added test 336 and 337 to verify.
Reported-by: Philippe Marguinaud
Closes#4339
It was used (intended) to pass in the size of the 'socks' array that is
also passed to these functions, but was rarely actually checked/used and
the array is defined to a fixed size of MAX_SOCKSPEREASYHANDLE entries
that should be used instead.
Closes#4169
They serve very little purpose and mostly just add noise. Most of them
have been around for a very long time. I read them all before removing
or rephrasing them.
Ref: #3876Closes#3883
Due to limitations in Curl_resolver_wait_resolv(), it doesn't work for
DOH resolves. This fix disables DOH for those.
Limitation added to KNOWN_BUGS.
Fixes#3850Closes#3857
- no need to have them protocol specific
- no need to set pointers to them with the Curl_setup_transfer() call
- make Curl_setup_transfer() operate on a transfer pointer, not
connection
- switch some counters from long to the more proper curl_off_t type
Closes#3627
This allows the compiler to pack and align the structs better in
memory. For a rather feature-complete build on x86_64 Linux, gcc 8.1.2
makes the Curl_easy struct 4.9% smaller. From 6312 bytes to 6000.
Removed an unused struct field.
No functionality changes.
Closes#3610
Instead of using a fixed 256 byte buffer in the connectdata struct.
In my build, this reduces the size of the connectdata struct by 11.8%,
from 2160 to 1904 bytes with no functionality or performance loss.
This also fixes a bug in schannel's Curl_verify_certificate where it
called Curl_sspi_strerror when it should have called Curl_strerror for
string from GetLastError. the only effect would have been no text or the
wrong text being shown for the error.
Co-authored-by: Jay Satiro
Closes#3612
The timeout set with CURLOPT_TIMEOUT is no longer used when
disconnecting from one of the pingpong protocols (FTP, IMAP, SMTP,
POP3).
Reported-by: jasal82 on github
Fixes#3264Closes#3374
The function does not return the same value as snprintf() normally does,
so readers may be mislead into thinking the code works differently than
it actually does. A different function name makes this easier to detect.
Reported-by: Tomas Hoger
Assisted-by: Daniel Gustafsson
Fixes#3296Closes#3297
The result of a memory allocation should always be checked, as we may
run under memory pressure where even a small allocation can fail. This
adds checking and error handling to a few cases where the allocation
wasn't checked for success. In the ftp case, the freeing of the path
variable is moved ahead of the allocation since there is little point
in keeping it around across the strdup, and the separation makes for
more readable code. In nwlib, the lock is aslo freed in the error path.
Also bumps the copyright years on affected files.
Closes#3084
Reviewed-by: Jay Satiro <raysatiro@yahoo.com>
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Commit 8238ba9c5f inadvertently removed
the actual command to be sent from the send buffer in a refactoring.
Add back copying the command into the buffer. Also add more guards
against malformed input while at it.
Closes#2985
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- Get rid of variable that was generating false positive warning
(unitialized)
- Fix issues in tests
- Reduce scope of several variables all over
etc
Closes#2631
In the situation of a client connecting to an FTP server using an IPv6
tunnel proxy, the connection info will indicate that the connection is
IPv6. However, because the server behing the proxy is IPv4, it is
permissable to attempt PSV mode. In the case of the FTP server being
IPv4 only, EPSV will always fail, and with the current logic curl will
be unable to connect to the server, as the IPv6 fwdproxy causes curl to
think that EPSV is impossible.
Closes#2432
Refuse to operate when given path components featuring byte values lower
than 32.
Previously, inserting a %00 sequence early in the directory part when
using the 'singlecwd' ftp method could make curl write a zero byte
outside of the allocated buffer.
Test case 340 verifies.
CVE-2018-1000120
Reported-by: Duy Phan Thanh
Bug: https://curl.haxx.se/docs/adv_2018-9cd6.html
returning 'time_t' is problematic when that type is unsigned and we
return values less than zero to signal "already expired", used in
several places in the code.
Closes#2021
... since the 'tv' stood for timeval and this function does not return a
timeval struct anymore.
Also, cleaned up the Curl_timediff*() functions to avoid typecasts and
clean up the descriptive comments.
Closes#2011
... to cater for systems with unsigned time_t variables.
- Renamed the functions to curlx_timediff and Curl_timediff_us.
- Added overflow protection for both of them in either direction for
both 32 bit and 64 bit time_ts
- Reprefixed the curlx_time functions to use Curl_*
Reported-by: Peter Piekarski
Fixes#2004Closes#2005
... a single double quote could leave the entry path buffer without a zero
terminating byte. CVE-2017-1000254
Test 1152 added to verify.
Reported-by: Max Dymond
Bug: https://curl.haxx.se/docs/adv_20171004.html
... to make all libcurl internals able to use the same data types for
the struct members. The timeval struct differs subtly on several
platforms so it makes it cumbersome to use everywhere.
Ref: #1652Closes#1693
Add a new type of callback to Curl_handler which performs checks on
the connection. Alter RTSP so that it uses this callback to do its
own check on connection health.
... all other non-HTTP protocol schemes are now defaulting to "tunnel
trough" mode if a HTTP proxy is specified. In reality there are no HTTP
proxies out there that allow those other schemes.
Assisted-by: Ray Satiro, Michael Kaufmann
Closes#1505
A) reduces the timeout lists drastically
B) prevents a lot of superfluous loops for timers that expires "in vain"
when it has actually already been extended to fire later on
Mark intended fallthroughs with /* FALLTHROUGH */ so that gcc will know
it's expected and won't warn on [-Wimplicit-fallthrough=].
Closes https://github.com/curl/curl/pull/1297
Replace use of fixed macro BUFSIZE to define the size of the receive
buffer. Reappropriate CURLOPT_BUFFERSIZE to include enlarging receive
buffer size. Upon setting, resize buffer if larger than the current
default size up to a MAX_BUFSIZE (512KB). This can benefit protocols
like SFTP.
Closes#1222
* HTTPS proxies:
An HTTPS proxy receives all transactions over an SSL/TLS connection.
Once a secure connection with the proxy is established, the user agent
uses the proxy as usual, including sending CONNECT requests to instruct
the proxy to establish a [usually secure] TCP tunnel with an origin
server. HTTPS proxies protect nearly all aspects of user-proxy
communications as opposed to HTTP proxies that receive all requests
(including CONNECT requests) in vulnerable clear text.
With HTTPS proxies, it is possible to have two concurrent _nested_
SSL/TLS sessions: the "outer" one between the user agent and the proxy
and the "inner" one between the user agent and the origin server
(through the proxy). This change adds supports for such nested sessions
as well.
A secure connection with a proxy requires its own set of the usual SSL
options (their actual descriptions differ and need polishing, see TODO):
--proxy-cacert FILE CA certificate to verify peer against
--proxy-capath DIR CA directory to verify peer against
--proxy-cert CERT[:PASSWD] Client certificate file and password
--proxy-cert-type TYPE Certificate file type (DER/PEM/ENG)
--proxy-ciphers LIST SSL ciphers to use
--proxy-crlfile FILE Get a CRL list in PEM format from the file
--proxy-insecure Allow connections to proxies with bad certs
--proxy-key KEY Private key file name
--proxy-key-type TYPE Private key file type (DER/PEM/ENG)
--proxy-pass PASS Pass phrase for the private key
--proxy-ssl-allow-beast Allow security flaw to improve interop
--proxy-sslv2 Use SSLv2
--proxy-sslv3 Use SSLv3
--proxy-tlsv1 Use TLSv1
--proxy-tlsuser USER TLS username
--proxy-tlspassword STRING TLS password
--proxy-tlsauthtype STRING TLS authentication type (default SRP)
All --proxy-foo options are independent from their --foo counterparts,
except --proxy-crlfile which defaults to --crlfile and --proxy-capath
which defaults to --capath.
Curl now also supports %{proxy_ssl_verify_result} --write-out variable,
similar to the existing %{ssl_verify_result} variable.
Supported backends: OpenSSL, GnuTLS, and NSS.
* A SOCKS proxy + HTTP/HTTPS proxy combination:
If both --socks* and --proxy options are given, Curl first connects to
the SOCKS proxy and then connects (through SOCKS) to the HTTP or HTTPS
proxy.
TODO: Update documentation for the new APIs and --proxy-* options.
Look for "Added in 7.XXX" marks.
Visual C++ now complains about implicitly casting time_t (64-bit) to
long (32-bit). Fix this by changing some variables from long to time_t,
or explicitly casting to long where the public interface would be
affected.
Closes#1131
