Commit Graph

5127 Commits

Author SHA1 Message Date
Daniel Stenberg
34f090a5cc
test1526: enable for hyper
Closes #8128
2021-12-09 23:32:33 +01:00
Daniel Stenberg
8d0d5859e0
test1525: slightly tweaked for hyper
Closes #8128
2021-12-09 23:32:25 +01:00
Daniel Stenberg
6f73e68d18
test1156: enable for hyper
Minor reorg of the lib1156 code and it works fine for hyper.

Closes #8127
2021-12-09 23:30:34 +01:00
Daniel Stenberg
fde1f1bfe9
test661: enable for hyper
Closes #8126
2021-12-09 23:29:04 +01:00
Daniel Stenberg
f443834c7a
lib530: abort on curl_multi errors
This makes torture tests run more proper.

Also add an assert to trap situations where it would end up with no
sockets to wait for.

Closes #8121
2021-12-09 10:16:24 +01:00
Daniel Stenberg
45c6448e68
test1556: adjust for hyper
Closes #8105
2021-12-07 17:11:51 +01:00
Daniel Stenberg
1bea122114
test1554: adjust for hyper
Closes #8104
2021-12-07 17:09:14 +01:00
Daniel Stenberg
2b3dd01b77
multi: handle errors returned from socket/timer callbacks
The callbacks were partially documented to support this. Now the
behavior is documented and returning error from either of these
callbacks will effectively kill all currently ongoing transfers.

Added test 530 to verify

Reported-by: Marcelo Juchem
Fixes #8083
Closes #8089
2021-12-06 14:55:52 +01:00
Daniel Stenberg
92d1aee8b1
urlapi: accept port number zero
This is a regression since 7.62.0 (fb30ac5a2d).

Updated test 1560 accordingly

Reported-by: Brad Fitzpatrick
Fixes #8090
Closes #8091
2021-12-03 22:58:41 +01:00
Dan Fandrich
48cbd8f713 tests: Add some missing keywords to tests
These are needed to skip some tests when configure options have disabled
certain features.
2021-11-30 18:32:53 -08:00
Glenn Strauss
aae235b6ba
digest: compute user:realm:pass digest w/o userhash
https://datatracker.ietf.org/doc/html/rfc7616#section-3.4.4
  ... the client MUST calculate a hash of the username after
      any other hash calculation ...

Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>
Closes #8066
2021-11-30 13:57:46 +01:00
Daniel Stenberg
230bb3e278
docs: consistent manpage SYNOPSIS
Make all libcurl related options use .nf (no fill) for the SYNOPSIS
section - for consistent look. roffit then renders that section using
<pre> (monospace font) in html for the website.

Extended manpage-syntax (test 1173) with a basic check for it.

Closes #8062
2021-11-26 16:51:23 +01:00
Daniel Stenberg
4183b8fe9a
urlapi: provide more detailed return codes
Previously, the return code CURLUE_MALFORMED_INPUT was used for almost
30 different URL format violations. This made it hard for users to
understand why a particular URL was not acceptable. Since the API cannot
point out a specific position within the URL for the problem, this now
instead introduces a number of additional and more fine-grained error
codes to allow the API to return more exactly in what "part" or section
of the URL a problem was detected.

Also bug-fixes curl_url_get() with CURLUPART_ZONEID, which previously
returned CURLUE_OK even if no zoneid existed.

Test cases in 1560 have been adjusted and extended. Tests 1538 and 1559
have been updated.

Updated libcurl-errors.3 and curl_url_strerror() accordingly.

Closes #8049
2021-11-25 08:36:04 +01:00
Kevin Burke
cc2870e275
tests/CI.md: add more information on CI environments
Fixes #8012
Closes #8022
2021-11-23 09:38:44 +01:00
Daniel Stenberg
3e6eb18fce
urlapi: reject short file URLs
file URLs that are 6 bytes or shorter are not complete. Return
CURLUE_MALFORMED_INPUT for those. Extended test 1560 to verify.

Triggered by #8041
Closes #8042
2021-11-23 08:45:21 +01:00
Daniel Stenberg
6e061ae63a
http: enable haproxy support for hyper backend
This is done by having native code do the haproxy header output before
hyper issues its request. The little downside with this approach is that
we need the entire Curl_buffer_send() function built, which is otherwise
not used for hyper builds.

If hyper ends up getting native support for the haproxy protocols we can
backpedal on this.

Enables test 1455 and 1456

Closes #8034
2021-11-19 14:06:42 +01:00
Patrick Monnerat
b20b364764
mime: use percent-escaping for multipart form field and file names
Until now, form field and file names where escaped using the
backslash-escaping algorithm defined for multipart mails. This commit
replaces this with the percent-escaping method for URLs.

As this may introduce incompatibilities with server-side applications, a
new libcurl option CURLOPT_MIME_OPTIONS with bitmask
CURLMIMEOPT_FORMESCAPE is introduced to revert to legacy use of
backslash-escaping. This is controlled by new cli tool option
--form-escape.

New tests and documentation are provided for this feature.

Reported by: Ryan Sleevi
Fixes #7789
Closes #7805
2021-11-15 10:40:03 +01:00
Daniel Stenberg
b7e1443a1d
test1939: require proxy support to run
Follow-up to f0b7099a10

Closes #8011
2021-11-14 16:14:52 +01:00
Daniel Stenberg
4825d0cd3a
test302[12]: run only with the libssh2 backend
... as the others don't support --hostpubsha256

Reported-by: Paul Howarth
Fixes #8009
Closes #8010
2021-11-14 16:13:50 +01:00
Daniel Stenberg
f4cafa4f0d
runtests: make the SSH library a testable feature
libssh2, libssh and wolfssh
2021-11-14 16:13:44 +01:00
Daniel Stenberg
26247a0d7e
lib1939: make it endure torture tests
Follow-up to f0b7099a10

Closes #8007
2021-11-13 22:44:34 +01:00
Daniel Stenberg
f0b7099a10
multi: shut down CONNECT in Curl_detach_connnection
... to prevent a lingering pointer that would lead to a double-free.

Added test 1939 to verify.

Reported-by: Stephen M. Coakley
Fixes #7982
Closes #7986
2021-11-11 08:42:31 +01:00
Marc Hoersken
b3e1ed3160
tests: add Schannel-specific tests and disable unsupported ones
Adds Schannel variants of SSLpinning tests that include the option
--ssl-revoke-best-effort to ignore certificate revocation check
failures which is required due to our custom test CA certificate.

Disable the original variants if the Schannel backend is enabled.

Also skip all IDN tests which are broken while using an msys shell.

This is a step to simplify test exclusions for Windows and MinGW.

Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg
Closes #7968
2021-11-09 06:34:33 +01:00
Daniel Stenberg
eccaa83cee
test1173: make manpage-syntax.pl spot \n errors in examples 2021-11-05 08:28:30 +01:00
Daniel Stenberg
cc71d35265
tool_operate: a failed etag save now only fails that transfer
When failing to create the output file for saving an etag, only fail
that particular single transfer and allow others to follow.

In a serial transfer setup, if no transfer at all is done due to them
all being skipped because of this error, curl will output an error
message and return exit code 26.

Added test 369 and 370 to verify.

Reported-by: Earnestly on github
Ref: #7942
Closes #7945
2021-11-04 13:56:15 +01:00
Marc Hoersken
d52316e460
tests/smbserver.py: fix compatibility with impacket 0.9.23+
impacket now performs sanity checks if the requested and to
be served file path actually is inside the real share path.

Ref: https://github.com/SecureAuthCorp/impacket/pull/1066

Fixes #7924
Closes #7935
2021-11-01 20:50:35 +01:00
Marc Hoersken
33dc399474
tests: kill some test servers afterwards to avoid locked logfiles
Reviewed-by: Daniel Stenberg
Closes #7925
2021-11-01 12:11:17 +01:00
Marc Hoersken
9121032fe3
CI/runtests.pl: restore -u flag, but remove it from CI runs
This makes it possible to use -u again for local testing,
but removes the flag from CI config files and make targets.

Reviewed-by: Daniel Stenberg

Partially reverts #7841
Closes #7921
2021-10-31 11:28:25 +01:00
Daniel Stenberg
6204aab43a
test1160: edited to work with hyper
Closes #7912
2021-10-27 22:58:21 +02:00
Daniel Stenberg
3b24b91019
data/DISABLED: enable tests that now work with hyper
Closes #7911
2021-10-27 22:57:29 +02:00
Daniel Stenberg
ba9477345e
test559: add 'HTTP' in keywords
Makes it run fine with hyper

Closes #7911
2021-10-27 22:57:26 +02:00
Daniel Stenberg
df84a79711
test552: updated to work with hyper
Closes #7911
2021-10-27 22:57:17 +02:00
Daniel Stenberg
c67a32fc56
http: reject HTTP response codes < 100
... which then also includes negative ones as test 1430 uses.

This makes native + hyper backend act identically on this and therefore
test 1430 can now be enabled when building with hyper. Adjust test 1431
as well.

Closes #7909
2021-10-27 10:26:15 +02:00
Daniel Stenberg
8d9af7062b
hyper: disable test 1294 since hyper doesn't allow such crazy headers
Closes #7905
2021-10-25 23:32:45 +02:00
Daniel Stenberg
bc8674d87d
c-hyper: make CURLOPT_SUPPRESS_CONNECT_HEADERS work
Verified by the enabled test 1288

Closes #7905
2021-10-25 23:32:45 +02:00
Daniel Stenberg
f9910e85fa
test1287: make work on hyper
Closes #7905
2021-10-25 23:32:45 +02:00
Daniel Stenberg
5c26764cd5
test1266/1267: disabled on hyper: no HTTP/0.9 support
Closes #7905
2021-10-25 23:32:44 +02:00
Daniel Stenberg
24155569d8
man pages: require all to use the same section header order
This is the same order we already enforce among the options' man pages:
consistency is good. Add lots of previously missing examples.

Adjust the manpage-syntax script for this purpose, used in test 1173.

Closes #7904
2021-10-25 12:57:01 +02:00
Daniel Stenberg
7c7b32db2b
tests: enable more tests with hyper
Adjusted 1144, 1164 and 1176.

Closes #7900
2021-10-25 10:47:00 +02:00
Samuel Henrique
352077b4d3
tests: use python3 in test 1451
This is a continuation of commit ec91b5a690 in which
changing this test was missed.  There are no other python2 leftovers
now.

Based on a Debian patch originally written by Alessandro Ghedini
<ghedo@debian.org>

Closes #7899
2021-10-24 21:19:40 +02:00
Daniel Stenberg
1732502cb0
hyper: does not support disabling CURLOPT_HTTP_TRANSFER_DECODING
Simply because hyper doesn't have this ability. Mentioned in docs now.

Skip test 326 then

Closes #7889
2021-10-22 12:54:08 +02:00
Daniel Stenberg
f852f908a9
test262: don't attempt with hyper
This test verifies that curl works with binary zeroes in HTTP response
headers and hyper refuses such. They're not kosher http.

Closes #7889
2021-10-22 12:54:04 +02:00
Daniel Stenberg
cf088201a9
c-hyper: make test 217 run
Closes #7889
2021-10-22 12:54:00 +02:00
Daniel Stenberg
a76f3c592b
DISABLED: enable test 209+213 for hyper
Follow-up to 823d3ab855

Closes #7889
2021-10-22 12:53:57 +02:00
Daniel Stenberg
7b4d44e045
test207: accept a different error code for hyper
It returns HYPERE_UNEXPECTED_EOF for this case which we convert to the
somewhat generic CURLE_RECV_ERROR.

Closes #7889
2021-10-22 12:53:46 +02:00
Daniel Stenberg
823d3ab855
http_proxy: multiple CONNECT with hyper done better
Enabled test 206

Closes #7888
2021-10-21 17:28:27 +02:00
Daniel Stenberg
c75f14cb9d
test1185: verify checksrc
Closes #7866
2021-10-18 23:52:30 +02:00
Daniel Stenberg
53418dbc84
checksrc: ignore preprocessor lines
In order to check the actual code better, checksrc now ignores
everything that look like preprocessor instructions. It also means
that code in macros are now longer checked.

Note that some rules then still don't need to be followed when code is
exactly below a cpp instruction.

Removes two checksrc exceptions we needed previously because of
preprocessor lines being checked.

Reported-by: Marcel Raad
Fixes #7863
Closes #7864
2021-10-17 11:19:49 +02:00
Daniel Stenberg
efffa66f65
urlapi: skip a strlen(), pass in zero
... to let curl_easy_escape() itself do the strlen. This avoids a (false
positive) Coverity warning and it avoids us having to store the strlen()
return value in an int variable.

Reviewed-by: Daniel Gustafsson
Closes #7862
2021-10-15 23:22:14 +02:00
Daniel Stenberg
ccb466fd36
misc: update copyright years 2021-10-15 17:07:19 +02:00
Abhinav Singh
8c6f126279
aws-sigv4: make signature work when post data is binary
User sets the post fields size for binary data.  Hence, we should not be
using strlen on it.

Added test 1937 and 1938 to verify.

Closes #7844
2021-10-15 08:46:23 +02:00
Jay Satiro
afd489885e sws: fix memory leak on exit
- Free the allocated http request struct on cleanup.

Prior to this change if sws was built with leak sanitizer it would
report a memory leak error during testing.

Closes https://github.com/curl/curl/pull/7849
2021-10-14 03:24:18 -04:00
Борис Верховский
3aead057a1
curl: correct grammar in generated libcurl code
Closes #7802
2021-10-13 00:33:20 +02:00
Daniel Stenberg
3723ba62a5
tests: disable test 2043
It uses revoked.badssl.com which now is expired and therefor this now
permafails. We should not use external sites for tests, this test should
be converted to use our own infra.

Closes #7845
2021-10-13 00:02:30 +02:00
Daniel Stenberg
8a534f8838
runtests: split out ignored tests
Report ignore tests separately from the actual fails.

Don't exit non-zero if test servers couldn't get killed.

Assisted-by: Jay Satiro

Fixes #7818
Closes #7841
2021-10-12 17:00:50 +02:00
Daniel Stenberg
e38d2e5a50
test368: verify dash is appended for "-r [num]"
Follow-up to 8758a26f88
2021-10-11 22:52:30 +02:00
Daniel Stenberg
9a8564a920
urlapi: URL decode percent-encoded host names
The host name is stored decoded and can be encoded when used to extract
the full URL. By default when extracting the URL, the host name will not
be URL encoded to work as similar as possible as before. When not URL
encoding the host name, the '%' character will however still be encoded.

Getting the URL with the CURLU_URLENCODE flag set will percent encode
the host name part.

As a bonus, setting the host name part with curl_url_set() no longer
accepts a name that contains space, CR or LF.

Test 1560 has been extended to verify percent encodings.

Reported-by: Noam Moshe
Reported-by: Sharon Brizinov
Reported-by: Raul Onitza-Klugman
Reported-by: Kirill Efimov
Fixes #7830
Closes #7834
2021-10-11 17:04:14 +02:00
Marc Hoersken
ab1671cafe
CI/makefiles: introduce dedicated test target
Make it easy to use the same set of test flags
throughout all current and future CI builds.

Reviewed-by: Jay Satiro

Follow up to #7690
Closes #7785
2021-10-08 20:16:15 +02:00
Daniel Stenberg
87907d56f0
http: fix Basic auth with empty name field in URL
Add test 367 to verify.

Reported-by: Rick Lane
Fixes #7819
Closes #7820
2021-10-06 22:45:52 +02:00
Jeffrey Tolar
5f563495f1
CURLOPT_MAXLIFETIME_CONN: maximum allowed lifetime for conn reuse
... and close connections that are too old instead of reusing them.

By default, this behavior is disabled.

Bug: https://curl.se/mail/lib-2021-09/0058.html
Closes #7751
2021-10-06 14:38:59 +02:00
Daniel Stenberg
24ffdc8826
print_category: print help descriptions aligned
Adjust the description position to make an aligned column when doing
help listings, which is more pleasing to the eye.

Suggested-by: Gisle Vanem
Closes #7792
2021-10-01 08:35:59 +02:00
Daniel Stenberg
ffb634d4ef
tool_listhelp: easier to generate with gen.pl
tool_listhelp.c is now a separate file with only the command line --help
output, exactly as generated by gen.pl. This makes it easier to generate
updates according to what's in the docs/cmdline-opts docs.

  cd $srcroot/docs/cmdline-opts
  ./gen.pl listhelp *.d > $srcroot/src/tool_listhelp.c

With a configure build, this also works:

  make -C src listhelp

Closes #7787
2021-09-30 17:50:48 +02:00
Daniel Stenberg
acfe94c08f
scripts/copyright: .muse is .lift now
And update 5 files with old copyright year range
2021-09-28 16:42:18 +02:00
Max Dymond
a517378de5
CURLOPT_PREREQFUNCTION: add new callback
Triggered before a request is made but after a connection is set up

Changes:

- callback: Update docs and callback for pre-request callback
- Add documentation for CURLOPT_PREREQDATA and CURLOPT_PREREQFUNCTION,
- Add redirect test and callback failure test
- Note that the function may be called multiple times on a redirection
- Disable new 2086 test due to Windows weirdness

Closes #7477
2021-09-27 17:16:43 +02:00
Sergey Markelov
4b997626b1
urlapi: support UNC paths in file: URLs on Windows
- file://host.name/path/file.txt is a valid UNC path
  \\host.name\path\files.txt to a non-local file transformed into URI
  (RFC 8089 Appendix E.3)

- UNC paths on other OSs must be smb: URLs

Closes #7366
2021-09-27 08:32:41 +02:00
i-ky
3363eeb262
urlapi: add curl_url_strerror()
Add curl_url_strerror() to convert CURLUcode into readable string and
facilitate easier troubleshooting in programs using URL API.
Extend CURLUcode with CURLU_LAST for iteration in unit tests.
Update man pages with a mention of new function.
Update example code and tests with new functionality where it fits.

Closes #7605
2021-09-27 08:28:46 +02:00
Mats Lindestam
d1e7d9197b
libssh2: add SHA256 fingerprint support
Added support for SHA256 fingerprint in command line curl and in
libcurl.

Closes #7646
2021-09-26 23:20:53 +02:00
Rikard Falkeborn
e75be2c4b2
cleanup: constify unmodified static structs
Constify a number of static structs that are never modified. Make them
const to show this.

Closes #7759
2021-09-23 12:54:35 +02:00
Daniel Stenberg
883cf374d8
test897: verify delivery of IMAP post-body header content
The "content" is delivered as "body" by curl, but the envelope continues
after the body and the rest of it should be delivered as header.

The IMAP server can now get 'POSTFETCH' set to include more data to
include after the body and test 897 is done to verify that such "extra"
header data is in fact delivered by curl as header.

Ref: #7284 but fails to reproduce the issue

Closes #7748
2021-09-21 08:18:42 +02:00
Daniel Stenberg
beb8990d93
http: fix the broken >3 digit response code detection
When the "reason phrase" in the HTTP status line starts with a digit,
that was treated as the forth response code digit and curl would claim
the response to be non-compliant.

Added test 1466 to verify this case.

Regression brought by 5dc594e44f
Reported-by: Glenn de boer
Fixes #7738
Closes #7739
2021-09-17 17:52:00 +02:00
Daniel Stenberg
da2598f823
hsts: CURLSTS_FAIL from hsts read callback should fail transfer
... and have CURLE_ABORTED_BY_CALLBACK returned.

Extended test 1915 to verify.

Reported-by: Jonathan Cardoso
Fixes #7726
Closes #7729
2021-09-16 12:36:12 +02:00
Daniel Stenberg
5b7aaefc07
test1184: disable
The test should be fine and it works for me repeated when run manually,
but clearly it causes CI failures and it needs more research.

Reported-by: RiderALT on github
Fixes #7725
Closes #7732
2021-09-16 12:16:48 +02:00
Kamil Dudka
ab78d2c679 tests/sshserver.pl: make it work with openssh-8.7p1
... by not using options with no argument where an argument is required:

=== Start of file tests/log/ssh_server.log
curl_sshd_config line 6: no argument after keyword "DenyGroups"
curl_sshd_config line 7: no argument after keyword "AllowGroups"
curl_sshd_config line 10: Deprecated option AuthorizedKeysFile2
curl_sshd_config line 29: Deprecated option KeyRegenerationInterval
curl_sshd_config line 39: Deprecated option RhostsRSAAuthentication
curl_sshd_config line 40: Deprecated option RSAAuthentication
curl_sshd_config line 41: Deprecated option ServerKeyBits
curl_sshd_config line 45: Deprecated option UseLogin
curl_sshd_config line 56: no argument after keyword "AcceptEnv"
curl_sshd_config: terminating, 3 bad configuration options
=== End of file tests/log/ssh_server.log

=== Start of file log/sftp_server.log
curl_sftp_config line 33: Unsupported option "rhostsrsaauthentication"
curl_sftp_config line 34: Unsupported option "rsaauthentication"
curl_sftp_config line 52: no argument after keyword "sendenv"
curl_sftp_config: terminating, 1 bad configuration options
Connection closed.
Connection closed
=== End of file log/sftp_server.log

Closes #7724
2021-09-16 08:37:25 +02:00
Daniel Stenberg
54f6d836d8
hsts: handle unlimited expiry
When setting a blank expire string, meaning unlimited, curl would pass
TIME_T_MAX to getime_r() when creating the output, while on 64 bit
systems such a large value cannot be convetered to a tm struct making
curl to exit the loop with an error instead. It can't be converted
because the year it would represent doesn't fit in the 'int tm_year'
field!

Starting now, unlimited expiry is instead handled differently by using a
human readable expiry date spelled out as "unlimited" instead of trying
to use a distant actual date.

Test 1660 and 1915 have been updated to help verify this change.

Reported-by: Jonathan Cardoso
Fixes #7720
Closes #7721
2021-09-15 14:37:36 +02:00
Patrick Monnerat
8ef147c436
ftp,imap,pop3,smtp: reject STARTTLS server response pipelining
If a server pipelines future responses within the STARTTLS response, the
former are preserved in the pingpong cache across TLS negotiation and
used as responses to the encrypted commands.

This fix detects pipelined STARTTLS responses and rejects them with an
error.

CVE-2021-22947

Bug: https://curl.se/docs/CVE-2021-22947.html
2021-09-13 16:51:31 +02:00
Patrick Monnerat
364f174724
ftp,imap,pop3: do not ignore --ssl-reqd
In imap and pop3, check if TLS is required even when capabilities
request has failed.

In ftp, ignore preauthentication (230 status of server greeting) if TLS
is required.

Bug: https://curl.se/docs/CVE-2021-22946.html

CVE-2021-22946
2021-09-13 16:51:31 +02:00
Daniel Stenberg
4a8fa48946
http_proxy: fix the User-Agent inclusion in CONNECT
It should not refer to the uagent string that is allocated and created
for the end server http request, as that pointer may be cleared on
subsequent CONNECT requests.

Added test case 1184 to verify.

Reported-by: T200proX7 on github
Fixes #7705
Closes #7707
2021-09-13 12:29:59 +02:00
Marc Hoersken
60efeb1e0d
runtests: add option -u to error on server unexpectedly alive
Let's try to actually handle the server unexpectedly alive
case by first making them visible on CI builds as failures.

This is needed to detect issues with killing of the test
servers completely including nested process chains with
multiple PIDs per test server (including bash and perl).

On Windows/cygwin platforms this is especially helpful with
debugging PID mixups due to cygwin using its own PID space.

Reviewed-by: Daniel Stenberg
Closes #7180
2021-09-09 21:28:26 +02:00
Daniel Stenberg
d9b2d1d90f
copyrights: update copyright year ranges 2021-09-08 09:36:49 +02:00
Daniel Stenberg
eb2a5171da
curl: stop retry if Retry-After: is longer than allowed
If Retry-After: specifies a period that is longer than what fits within
--retry-max-time, then stop retrying immediately.

Added test 366 to verify.

Reported-by: Kari Pahula
Fixes #7675
Closes #7676
2021-09-06 14:46:45 +02:00
Daniel Stenberg
8fbdf8331e
test1173: check references to libcurl options
... that they refer to actual existing libcurl options.

Reviewed-by: Daniel Gustafsson
Closes #7656
2021-09-04 11:27:56 +02:00
Daniel Stenberg
12a2ed970f
opt-docs: verify man page sections + order
In every libcurl option man page there are now 8 mandatory sections that
must use the right name in the correct order and test 1173 verifies
this. Only 14 man pages needed adjustments.

The sections and the order is as follows:

 - NAME
 - SYNOPSIS
 - DESCRIPTION
 - PROTOCOLS
 - EXAMPLE
 - AVAILABILITY
 - RETURN VALUE
 - SEE ALSO

Reviewed-by: Daniel Gustafsson
Closes #7656
2021-09-04 11:27:46 +02:00
Daniel Stenberg
1731a77989
opt-docs: make sure all man pages have examples
Extended manpage-syntax.pl (run by test 1173) to check that every man
page for a libcurl option has an EXAMPLE section that is more than two
lines. Then fixed all errors it found and added examples.

Reviewed-by: Daniel Gustafsson
Closes #7656
2021-09-04 11:27:04 +02:00
Daniel Stenberg
424051d78b
connect: get local port + ip also when reusing connections
Regression. In d6a37c23a3 (7.75.0) we removed the duplicated storage
(connection + easy handle), so this info needs be extracted again even
for re-used connections.

Add test 435 to verify

Reported-by: Max Dymond
Fixes #7660
Closes #7662
2021-09-02 23:42:18 +02:00
Daniel Stenberg
3f4b1b4ee5
tests: adjust the tftpd output to work with hyper mode
By making them look less like http headers, the hyper mode "tweak"
doesn't interfere.

Enable test 2002 and 2003 in hyper builds (and 1280 which is unrelated
but should be enabled).

Closes #7658
2021-09-01 23:40:28 +02:00
Daniel Stenberg
e8c8775eaa
test365: verify response with chunked AND Content-Length headers 2021-08-31 10:04:03 +02:00
Daniel Stenberg
8a16e54c0c
http: ignore content-length if any transfer-encoding is used
Fixes #7643
Closes #7649
2021-08-31 10:04:03 +02:00
Daniel Stenberg
5dc594e44f
http: disallow >3-digit response codes
Make the built-in HTTP parser behave similar to hyper and reject any
HTTP response using more than 3 digits for the response code.

Updated test 1432 accordingly.
Enabled test 1432 in the hyper builds.

Closes #7641
2021-08-26 22:43:38 +02:00
Daniel Stenberg
9df52bf17a
test1280: CRLFify the response to please hyper
Closes #7639
2021-08-26 16:17:10 +02:00
Daniel Stenberg
9a30ac0957
tests: enable test 1129 for hyper builds
Closes #7638
2021-08-26 16:15:48 +02:00
Daniel Stenberg
e5d77dc2ca
c-hyper: handle HTTP/1.1 => HTTP/1.0 downgrade on reused connection
Enable test 1074

Closes #7617
2021-08-23 16:12:33 +02:00
Daniel Stenberg
38941ad6f3
c-hyper: deal with Expect: 100-continue combined with POSTFIELDS
Enable test 1130 and 1131

Closes #7616
2021-08-23 16:10:15 +02:00
a1346054
ec91b5a690
tests: be explicit about using 'python3' instead of 'python'
This fixes running tests in virtualenvs (or on distros) that no longer
have a symlink from python to python2 or python3.

Closes #7602
2021-08-23 15:59:43 +02:00
Daniel Stenberg
473b893c7e
DISABLED: enable 11 more tests for hyper builds
Closes #7612
2021-08-23 11:24:25 +02:00
Daniel Stenberg
46d4373ea7
setopt: enable CURLOPT_IGNORE_CONTENT_LENGTH for hyper
Since this option is also used for FTP, it needs to work to set for
applications even if hyper doesn't support it for HTTP. Verified by test
1137.

Updated docs to specify that the option doesn't work for HTTP when using
the hyper backend.

Closes #7614
2021-08-23 11:23:17 +02:00
Daniel Stenberg
0854781c18
test1138: remove trailing space to make work with hyper
Closes #7613
2021-08-23 11:21:38 +02:00
Daniel Stenberg
4e53b9430c
man pages: remove trailing whitespaces
Extended test 1173 (via the manpage-syntax.pl script) to detect and warn
for them.

Ref: #7602
Reported-by: a1346054 on github
Closes #7604
2021-08-21 16:44:03 +02:00
Daniel Stenberg
059379d20c
http_proxy: fix user-agent and custom headers for CONNECT with hyper
Enable test 287

Closes #7598
2021-08-20 13:51:06 +02:00
Daniel Stenberg
f46b83fc94
c-hyper: initial support for "dumping" 1xx HTTP responses
With the use hyper_request_on_informational()

Enable test 155 and 158

Closes #7597
2021-08-20 13:37:43 +02:00
Marc Hoersken
5b1c2dd1db
tests/*server.pl: flush output before executing subprocess
Also avoid shell processes staying around by using exec.
This is necessary to avoid output data being buffering
inside the process chain of Perl, Bash/Shell and our
test server binaries. On non-Windows systems the exec
will also make the subprocess replace the intermediate
shell, but on Windows it will at least bind the processes
together since there is no real fork or exec available.

See: https://cygwin.com/cygwin-ug-net/highlights.html
and: https://docs.microsoft.com/cpp/c-runtime-library/exec-wexec-functions
Ref: https://github.com/curl/curl/pull/7530#issuecomment-900949010

Reviewed-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Closes #7530
2021-08-18 20:59:36 +02:00
Marc Hoersken
b5abbc3d57
tests/server/*.c: align handling of portfile argument and file
1. Call the internal variable portname (like pidname) everywhere.
2. Have a variable wroteportfile (like wrotepidfile) everywhere.
3. Make sure the file is cleaned up on exit (like pidfile).
4. Add parameter --portfile to usage outputs everywhere.

Reviewed-by: Daniel Stenberg

Replaces #7523
Closes #7574
2021-08-17 21:43:31 +02:00
Daniel Stenberg
8c9722bb1c
misc: update incorrect copyright year ranges
Closes #7577
2021-08-16 11:21:47 +02:00
Daniel Stenberg
32f6812b5a
c-hyper: initial step for 100-continue support
Enabled test 154

Closes #7568
2021-08-13 23:18:21 +02:00
Daniel Stenberg
c495dcd02e
http: consider cookies over localhost to be secure
Updated test31.
Added test 392 to verify secure cookies used for http://localhost

Reviewed-by: Daniel Gustafsson
Fixes #6733
Closes #7263
2021-08-10 11:20:21 +02:00
Jay Satiro
26f93cf038 test1565: fix windows build errors
- Use our wait_ms() instead of sleep() since Windows doesn't have the
  latter.

- Use a separate variable to keep track of whether the pthread_t thread
  id is valid.

On Windows pthread_t is not an integer type. pthread offers no macro for
invalid pthread_t thread id, so validity is kept track of separately.

Closes https://github.com/curl/curl/pull/7527
2021-08-10 03:29:40 -04:00
Bin Lan
5b9fedb461
tool/tests: fix potential year 2038 issues
The length of 'long' in a 32-bit system is 32 bits, which cannot be used
to save timestamps after 2038. Most operating systems have extended
time_t to 64 bits.

Remove the castings to long.

Closes #7466
2021-07-30 16:06:37 +02:00
Bernhard M. Wiedemann
8d0c1137c1
tests: make three tests pass until 2037
after 2038 something in test1915 fails on 32-bit OSes

Closes #7512
2021-07-30 12:39:34 +02:00
Marc Hoersken
c34bd93f60
tests/*server.py: remove pidfile on server termination
Avoid pidfile leaking/laying around after server already exited.

Reviewed-by: Daniel Stenberg
Closes #7506
2021-07-29 08:08:07 +02:00
Marc Hoersken
4a7bf79fcc
tests/servers: remove obsolete pid variable
Variable is not used since pidfile handling moved to util.[ch]

Reviewed-by: Jay Satiro
Closes #7482
2021-07-25 20:00:18 +02:00
Marc Hoersken
d9d26a6b2c
tests/servers: use our platform-aware pid for server verification
The pid used for server verification is later stored as pid2 in
the hash of running test servers and therefore used for shutdown.

The pid used for shutdown must be the platform-aware (Win32) pid
to avoid leaking test servers while running them using Cygwin/msys.

Reviewed-by: Jay Satiro
Closes #7481
2021-07-25 19:59:58 +02:00
Marc Hoersken
893f39c596
tests/runtests.pl: cleanup copy&paste mistakes and unused code
Reviewed-by: Jay Satiro
Part of #7481
2021-07-25 19:59:14 +02:00
Josh Soref
de1004eb0f
cleanup: spell DoH with a lowercase o
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

Closes #7413
2021-07-16 23:53:13 +02:00
Daniel Stenberg
9053dbbf62
msnprintf: return number of printed characters excluding null byte
... even when the output is "capped" by the maximum length argument.

Clarified in the docs.

Closes #7361
2021-07-08 10:05:39 +02:00
Daniel Stenberg
e7416cfd2b
infof: remove newline from format strings, always append it
- the data needs to be "line-based" anyway since it's also passed to the
  debug callback/application

- it makes infof() work like failf() and consistency is good

- there's an assert that triggers on newlines in the format string

- Also removes a few instances of "..."

- Removes the code that would append "..." to the end of the data *iff*
  it was truncated in infof()

Closes #7357
2021-07-07 22:54:01 +02:00
Daniel Stenberg
c3d612662d
test1147: hyper doesn't allow "crazy" request headers like built-in
... so strip that from the test.

Closes #7349
2021-07-05 22:51:13 +02:00
Daniel Stenberg
a62ece3f57
c-hyper: bail on too long response headers
To match with built-in behaviors. Makes test 1154 work.

Closes #7350
2021-07-05 22:49:12 +02:00
Daniel Stenberg
0965348cd8
test1151: added missing CRLF to work with hyper
Closes #7350
2021-07-05 22:49:02 +02:00
Daniel Stenberg
e41feede34
test1116: hyper doesn't pass through "surprise-trailers"
Closes #7344
2021-07-05 22:39:42 +02:00
Daniel Stenberg
07fa74d34a
test1519: adjusted to work with hyper
Closes #7333
2021-07-01 14:26:06 +02:00
Daniel Stenberg
a80f8ea182
test1518: adjusted to work with hyper
... by making sure the stdout output doesn't look like HTTP headers.

Closes #7333
2021-07-01 14:25:57 +02:00
Daniel Stenberg
7f11282d5f
test1514: add a CRLF to the response to make it correct
Makes hyper accept it fine instead returning HYPERE_UNEXPECTED_EOF on
us.

Closes #7334
2021-07-01 14:19:55 +02:00
Daniel Stenberg
ca60a4398f
lib677: make it survive torture testing
Follow-up to a5ab72d5ed

Closes #7300
2021-06-27 23:20:18 +02:00
Daniel Stenberg
a5ab72d5ed
test677: IMAP CONNECT_ONLY, custom command and then exit
Adjusted ftpserver.pl to add support for the IMAP IDLE command

Adjusted test 660 to sync with the fix
2021-06-24 09:07:40 +02:00
Li Xinwei
30e491e5c9
cmake: fix support for UnixSockets feature on Win32
Move the definition of sockaddr_un struct from config-win32.h to
curl_setup.h, so that it could be shared by all build systems.

Add ADDRESS_FAMILY typedef for old mingw, now old mingw can also use
unix sockets.

Also fix the build of tests/server/sws.c on Win32 when USE_UNIX_SOCKETS
is defined.

Closes #7034
2021-06-21 14:52:27 +02:00
Daniel Stenberg
80e1054fe5
hyper: propagate errors back up from read callbacks
Makes test 513 work with hyper

Closes #7266
2021-06-17 08:19:31 +02:00
Daniel Stenberg
b67d3ba73e
curl_url_set: reject spaces in URLs w/o CURLU_ALLOW_SPACE
They were never officially allowed and slipped in only due to sloppy
parsing. Spaces (ascii 32) should be correctly encoded (to %20) before
being part of a URL.

The new flag bit CURLU_ALLOW_SPACE when a full URL is set, makes libcurl
allow spaces.

Updated test 1560 to verify.

Closes #7073
2021-06-15 10:49:49 +02:00
Daniel Stenberg
bbbc5de93f
netrc: skip 'macdef' definitions
Add test 494 to verify

Reported-by: Harry Sintonen
Fixes #7238
Closes #7244
2021-06-13 23:44:41 +02:00
Gealber Morales
791937b881
mqtt: add support for username and password
Minor-edits-by: Daniel Stenberg
Added test 2200 to 2205

Closes #7243
2021-06-12 23:50:13 +02:00
Daniel Stenberg
f60f51721c
runtests: init $VERSION to avoid warnings when using -l 2021-06-11 14:26:32 +02:00
Daniel Stenberg
a15d146f47
tests: update README about servers and port numbers
Closes #7242
2021-06-11 12:48:56 +02:00
Christian Weisgerber
02dfe71937
sws: malloc request struct instead of using stack
... 2MB requests is otherwise just too big for some systems.

(The allocations are not freed properly.)

Bug: https://curl.se/mail/lib-2021-06/0018.html

Closes #7235
2021-06-11 11:13:24 +02:00
yb999
b9cdcc7a17
tests: update README.md with a missing single quote
Closes #7231
2021-06-10 16:11:03 +02:00
Daniel Stenberg
7cfd8f037a
tests/data/DISABLED: add tests not working with hyper
The goal is to remove them all from here over time.

Closes #7209
2021-06-10 08:42:55 +02:00
Daniel Stenberg
dd47b0cce9
runtests: also find the last test in Makefile.inc
Closes #7209
2021-06-10 08:42:52 +02:00
Daniel Stenberg
77ae8f26ad
test3010: work with hyper mode
Closes #7209
2021-06-10 08:42:50 +02:00
Daniel Stenberg
75391045cf
test1594/1595/1596: fix to work in hyper mode
Closes #7209
2021-06-10 08:42:44 +02:00
Daniel Stenberg
30e50de53e
test1438/1457: add HTTP keyword to make hyper mode work
Closes #7209
2021-06-10 08:42:41 +02:00
Daniel Stenberg
5f70cb6461
test1340/1341: adjusted for hyper mode
Closes #7209
2021-06-10 08:42:38 +02:00
Daniel Stenberg
769d9176e3
test1218: adjusted for hyper mode
Closes #7209
2021-06-10 08:42:35 +02:00
Daniel Stenberg
6101ddc24e
test1216: adjusted for hyper mode
Closes #7209
2021-06-10 08:42:33 +02:00
Daniel Stenberg
340d6b4589
test1230: adjust to work in hyper mode
Closes #7209
2021-06-10 08:42:30 +02:00
Daniel Stenberg
c214a6a17b
c-hyper: abort CONNECT response reading early on non 2xx responses
Fixes test 493

Closes #7209
2021-06-10 08:42:27 +02:00
Daniel Stenberg
aeb064db03
test434: add HTTP keyword
Closes #7209
2021-06-10 08:42:23 +02:00
Daniel Stenberg
e7ae6f013d
test599: adjusted to work in hyper mode
Closes #7209
2021-06-10 08:42:19 +02:00
Daniel Stenberg
cd5ca9bb38
test566: adjust to work with hyper mode
Closes #7209
2021-06-10 08:42:13 +02:00
Daniel Stenberg
76035e612a
runtests: skip disabled tests unless -f is used
To make it easier to write ranges like '115 to 229' without that
explicitly enabling tests that are listed in DISABLED, this makes
runtests always skip disabled tests unless the -f command line option is
used.

Previously the code attempted to not run such tests, but didn't do it
correctly.

Closes #7212
2021-06-08 23:35:37 +02:00
Daniel Stenberg
3ac9b80525
test644: remove as duplicate of test 587
Closes #7208
2021-06-08 16:07:56 +02:00
Daniel Stenberg
48b0cca65e
test500: adjust to work with hyper mode 2021-06-07 23:54:54 +02:00
Daniel Stenberg
57e46dce58
test433: adjust for hyper mode
Closes #7205
2021-06-07 23:14:09 +02:00
Daniel Stenberg
528f22736d
test395: hyper cannot work around > 64 bit content-lengths like built-in
Closes #7205
2021-06-07 23:14:06 +02:00
Daniel Stenberg
77590673e9
test394: hyper returns a different error
Closes #7205
2021-06-07 23:14:03 +02:00
Daniel Stenberg
1449713dd3
test393: make Content-Length fit within 64 bit for hyper
Closes #7205
2021-06-07 23:14:00 +02:00
Daniel Stenberg
ae37f46e80
test347: CRLFify to work in hyper mode
Closes #7205
2021-06-07 23:13:55 +02:00
Daniel Stenberg
71da11a49d
test339: CRLFify better to work in hyper mode
Closes #7205
2021-06-07 23:13:47 +02:00
Daniel Stenberg
b26f1dc14f
test328: avoid a header-looking body to make hyper mode work
The test still works the same, just modified two bytes in the content.

Closes #7203
2021-06-07 13:26:27 +02:00
Daniel Stenberg
265b14d6b3
metalink: remove
Warning: this will make existing curl command lines that use metalink to
stop working.

Reasons for removal:

1. We've found several security problems and issues involving the
   metalink support in curl. The issues are not detailed here. When
   working on those, it become apparent to the team that several of the
   problems are due to the system design, metalink library API and what
   the metalink RFC says. They are very hard to fix on the curl side
   only.

2. The metalink usage with curl was only very briefly documented and was
   not following the "normal" curl usage pattern in several ways, making
   it surprising and non-intuitive which could lead to further security
   issues.

3. The metalink library was last updated 6 years ago and wasn't so
   active the years before that either. An unmaintained library means
   there's a security problem waiting to happen. This is probably reason
   enough.

4. Metalink requires an XML parsing library, which is complex code (even
   the smaller alternatives) and to this day often gets security
   updates.

5. Metalink is not a widely used curl feature. In the 2020 curl user
   survey, only 1.4% of the responders said that they'd are using it. In
   2021 that number was 1.2%. Searching the web also show very few
   traces of it being used, even with other tools.

6. The torrent format and associated technology clearly won for
   downloading large files from multiple sources in parallel.

Cloes #7176
2021-06-07 08:14:25 +02:00
Daniel Stenberg
510e6e9a19
tests/data/Makefile.inc: error: trailing backslash on last line
Follow-up to d8dcb399b8
2021-06-04 16:52:34 +02:00
Alexis Vachette
d8dcb399b8
url: bad CURLOPT_CONNECT_TO syntax now returns error
Added test 3020 to verify

Closes #7183
2021-06-04 14:08:30 +02:00
Daniel Stenberg
628ebd82b9
test269: disable for hyper
--ignore-content-length / CURLOPT_IGNORE_CONTENT_LENGTH doesn't work
with hyper.

Closes #7184
2021-06-03 23:11:11 +02:00
Daniel Stenberg
b193fd3bbd
runtests: enable 'hyper mode' only for HTTP tests
The 'hyper mode' makes line-ending checks work in the test suite for
when hyper is used. Now it also requires that HTTP or HTTPS are
mentioned as keywords to be enabled so that it doesn't wrongly adjusts
tests for other protocols.

This makes test 271 (TFTP) work again in hyper enabled builds.

Closes #7185
2021-06-03 23:08:47 +02:00
Alexis Vachette
cb39b4b7c4
hostip: bad CURLOPT_RESOLVE syntax now returns error
Added test 3019
Fixes #7170
Closes #7174
2021-06-03 13:53:18 +02:00
Daniel Stenberg
3c0f4622cd
runtests: parse data/Makefile.inc instead of using make
The warning about missing entries in that file then doesn't require that
the Makefile has been regenerated which was confusing.

The scan for the test num is a little more error prone than before
(since now it doesn't actually verify that it is legitimate Makefile
syntax), but I think it is good enough.

Closes #7177
2021-06-02 23:21:18 +02:00
Harry Sintonen
e4db6fb02c
filecheck: quietly remove test-place/*~
Closes #7179
2021-06-02 23:17:44 +02:00
Daniel Stenberg
6793332eba
CURLE_SETOPT_OPTION_SYNTAX: new error name for wrong setopt syntax
For options that pass in lists or strings that are subsequently parsed
and must be correct. This broadens the scope for the option previously
known as CURLE_TELNET_OPTION_SYNTAX but the old name is of course still
provided as a #define for existing applications.

Closes #7175
2021-06-02 23:03:55 +02:00
Daniel Stenberg
9dc0baf5c6
tests: fix Accept-Encoding strips to work with Hyper builds
The previous strip also removed the CR which turned problematic.

valgrind.supp: add zstd suppression using hyper

Reported-and-analyzed-by: Kevin Burke
Fixes #7169
Closes #7171
2021-06-02 22:59:33 +02:00
Harry Sintonen
8ccf75532b
mqtt: detect illegal and too large file size
Add test 3017 and 3018 to verify.
Closes #7166
2021-06-02 13:34:17 +02:00
Daniel Stenberg
9097843e8f
test178: adjust for hyper
Hyper returns the same error for wrong HTTP version as for negative
content-length. Test 178 verifies that negative content-length is
rejected but the hyper backend will return a different error for it (and
without any helpful message telling why the message was bad). It will
also not return any headers at all for the response, not even the ones
that arrived before the error.

Closes #7147
2021-05-29 23:16:04 +02:00
Viktor Szakats
3976dd67a2
docs: use --max-redirs instead of --max-redir
For consistency.

Closes #7130
2021-05-26 14:49:52 +02:00
Daniel Stenberg
82d334001f
copyright: update copyright year ranges to 2021 2021-05-26 08:18:11 +02:00
Jacob Hoffman-Andrews
a62e6435f4
rustls: switch read_tls and write_tls to callbacks
And update to 0.6.0, including a rename from session to connection for
many fields.

Closes #7071
2021-05-24 16:40:59 +02:00
Daniel Stenberg
b03b82a85f
docs/tests: remove freenode references 2021-05-24 00:21:00 +02:00
Lucas Clemente Vella
84d2839740
CURLOPT_IPRESOLVE: preventing wrong IP version from being used
In some situations, it was possible that a transfer was setup to
use an specific IP version, but due do DNS caching or connection
reuse, it ended up using a different IP version from requested.

This commit changes the effect of CURLOPT_IPRESOLVE from simply
restricting address resolution to preventing the wrong connection
type being used, when choosing a connection from the pool, and
to restricting what addresses could be used when establishing
a new connection.

It is important that all addresses versions are resolved, even if
not used in that transfer in particular, because the result is
cached, and could be useful for a different transfer with a
different CURLOPT_IPRESOLVE setting.

Closes #6853
2021-05-20 16:58:31 +02:00
Daniel Stenberg
d9eb3e316f
test2100: make it run with and require IPv6
Closes #7083
2021-05-20 08:32:43 +02:00
Daniel Stenberg
1f90d08371
tests/getpart: generate output URL encoded for better diffs
Closes #7083
2021-05-20 08:32:38 +02:00
Daniel Stenberg
030d539164
sws: allow HTTP requests up to 2MB in size
To allow tests with slightly larger payloads. Like #7071 ...

Closes #7075
2021-05-16 12:56:34 +02:00
Daniel Stenberg
e35d7c0492
mem-include-scan.pl: require a non-word letter before memory funcs
... so that ldap_memfree() for example doesn't match the scan for free.

Closes #7061
2021-05-15 17:47:37 +02:00
Dan Fandrich
56cf2de5ac tests: Fix some tag matching issues in a number of tests 2021-05-12 01:17:16 -07:00
Daniel Stenberg
51c0ebcff2
http: deal with partial CONNECT sends
Also added 'CURL_SMALLSENDS' to make Curl_write() send short packets,
which helped verifying this even more.

Add test 363 to verify.

Reported-by: ustcqidi on github
Fixes #6950
Closes #7024
2021-05-08 10:49:16 +02:00
Marc Hoersken
d698c70421
sockfilt: fix invalid increment of handles index variable nfd
Only increment the array index if we actually stored a handle.

Follow up to e917492048
Closes #6992
2021-05-05 22:30:48 +02:00
Marc Hoersken
b9239e0f9c
sockfilt: avoid getting stuck waiting for writable socket
Reset FD_WRITE event using the same approach as in multi.c

Follow up to b36442b243
Closes #6992
2021-05-05 22:30:12 +02:00
Jay Satiro
5a1ec19fda test678: Fix for Windows multibyte builds
Follow-up to 77fc385 from yesterday.

Bug: https://github.com/curl/curl/pull/6662#issuecomment-832966557
Reported-by: Marc Hörsken
2021-05-05 16:25:45 -04:00
Gilles Vollant
77fc3859b2 SSL: support in-memory CA certs for some backends
- New options CURLOPT_CAINFO_BLOB and CURLOPT_PROXY_CAINFO_BLOB to
  specify in-memory PEM certificates for OpenSSL, Schannel (Windows)
  and Secure Transport (Apple) SSL backends.

Prior to this change PEM certificates could only be imported from a file
and not from memory.

Co-authored-by: moparisthebest@users.noreply.github.com

Ref: https://github.com/curl/curl/pull/4679
Ref: https://github.com/curl/curl/pull/5677
Ref: https://github.com/curl/curl/pull/6109

Closes https://github.com/curl/curl/pull/6662
2021-05-05 02:29:16 -04:00
David Cook
70cf50fb4a
tests: ignore case of chunked hex numbers in tests
When hyper is used, it emits uppercase hexadecimal numbers for chunked
encoding lengths. Without hyper, lowercase hexadecimal numbers are used.
This change adds preprocessor statements to tests where this is an
issue, and adapts the fixtures to match.

Closes #6987
2021-05-04 15:57:57 +02:00
Daniel Stenberg
91d9fead66
libtest: remove lib530.c
Follow up from e50a877df when test 530 was removed. Since then this
source file has not been used/needed.

Closes #6999
2021-05-04 14:45:52 +02:00
Daniel Stenberg
23e9b96a64
FILEFORMAT: mention sectransp as a feature
Been supported since at least 40259ca65

Closes #7001
2021-05-04 14:45:01 +02:00
ejanchivdorj
94241a9e78
CURLcode: add CURLE_SSL_CLIENTCERT
When a TLS server requests a client certificate during handshake and
none can be provided, libcurl now returns this new error code
CURLE_SSL_CLIENTCERT

Only supported by Secure Transport and OpenSSL for TLS 1.3 so far.

Closes #6721
2021-05-03 17:11:01 +02:00
Daniel Stenberg
4708f29f1c
tests/disable-scan.pl: also scan all m4 files
Fixes test 1165 when functions are moved from configure.ac to files in
m4/
2021-04-22 23:19:47 +02:00
Daniel Stenberg
063d3f3b96
tidy-up: make conditional checks more consistent
... remove '== NULL' and '!= 0'

Closes #6912
2021-04-22 09:10:17 +02:00
Patrick Monnerat
34cf40321c
bufref: buffer reference support
A struct bufref holds a buffer pointer, a data size and a destructor.
When freed or its contents are changed, the previous buffer is implicitly
released by the associated destructor. The data size, although not used
internally, allows binary data support.

A unit test checks its handling methods: test 1661

Closes #6654
2021-04-22 09:05:53 +02:00
Patrick Monnerat
3fb6e5a010
test server: take care of siginterrupt() deprecation
Closes #6529
2021-04-22 08:32:43 +02:00
Marc Hoersken
2df64390b3
lib1564.c: enable last wakeup test part on Windows
Suggested-by: Gergely Nagy
Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad

Closes #6245
2021-04-21 20:32:33 +02:00
Daniel Stenberg
3057c6cd73
Revert "cmake: make libcurl library output name configurable"
This reverts commit 1cba36d216.

CMake provides properties that can be set on a target to rename the
output artifact without changing the name of a target.

Ref: #6899
2021-04-21 15:03:26 +02:00
Ralph Langendam
1cba36d216
cmake: make libcurl library output name configurable
Closes #6899
2021-04-21 10:03:07 +02:00
Daniel Stenberg
415d6fa7da
sws: #ifdef S_IFSOCK use
SCO OpenServer 5.0.7 does not define S_IFSOCK.

Reported-by: Kevin R. Bulgrien
Bug: https://curl.se/mail/lib-2021-04/0074.html
Closes #6926
2021-04-21 09:35:31 +02:00
Daniel Stenberg
04488851e2
urlapi: make sure no +/- signs are accepted in IPv4 numericals
Follow-up to 56a037cc0a. Extends test 1560 to verify.

Reported-by: Tuomas Siipola
Fixes #6916
Closes #6917
2021-04-21 09:17:55 +02:00
Daniel Stenberg
6b97f1311a
test1660: check the created HSTS file as text mode
Closes #6922
2021-04-21 08:18:52 +02:00
Daniel Stenberg
7569518e92
test 493: require https in curl to run
Closes #6927
2021-04-20 10:01:19 +02:00
Daniel Stenberg
56a037cc0a
urlapi: "normalize" numerical IPv4 host names
When the host name in a URL is given as an IPv4 numerical address, the
address can be specified with dotted numericals in four different ways:
a32, a.b24, a.b.c16 or a.b.c.d and each part can be specified in
decimal, octal (0-prefixed) or hexadecimal (0x-prefixed).

Instead of passing on the name as-is and leaving the handling to the
underlying name functions, which made them not work with c-ares but work
with getaddrinfo, this change now makes the curl URL API itself detect
and "normalize" host names specified as IPv4 numericals.

The WHATWG URL Spec says this is an okay way to specify a host name in a
URL. RFC 3896 does not allow them, but curl didn't prevent them before
and it seems other RFC 3896-using tools have not either. Host names used
like this are widely supported by other tools as well due to the
handling being done by getaddrinfo and friends.

I decided to add the functionality into the URL API itself so that all
users of these functions get the benefits, when for example wanting to
compare two URLs. Also, it makes curl built to use c-ares now support
them as well and make curl builds more consistent.

The normalization makes HTTPS and virtual hosted HTTP work fine even
when curl gets the address specified using one of the "obscure" formats.

Test 1560 is extended to verify.

Fixes #6863
Closes #6871
2021-04-19 08:34:55 +02:00
Daniel Stenberg
d71ff2b9db
hsts: enable by default
No longer considered experimental.

Closes #6700
2021-04-19 08:22:16 +02:00
Daniel Stenberg
6fc805d0c1
test972: verify the json output with jsonlint
Make sure one of the azure jobs has jsonlint installed so that the test
runs there.

Ref: #6905
2021-04-17 11:03:40 +02:00
Jay Satiro
2f78be51eb
tool_writeout: fix the HTTP_CODE json output
Update test 970 accordingly.

Reported-by: Michal Rus
Fixes #6905
Closes #6906
2021-04-17 11:03:30 +02:00
Victor Vieux
0d7c55bd57
tool_getparam: replace (in-place) '%20' by '+' according to RFC1866
Signed-off-by: Victor Vieux <victorvieux@gmail.com>

Closes #6895
2021-04-15 13:45:12 +02:00
Daniel Stenberg
7bdec2a08b
configure: provide --with-openssl, deprecate --with-ssl
Makes the option more explicit.

Closes #6887
2021-04-15 09:08:34 +02:00
Daniel Stenberg
520bd5225c
cookie: CURLOPT_COOKIEFILE set to NULL switches off cookies
Add test 676 to verify that setting CURLOPT_COOKIEFILE to NULL again clears
the cookiejar from memory.

Reported-by: Stefan Karpinski
Fixes #6889
Closes #6891
2021-04-14 23:09:36 +02:00
Patrick Monnerat
9c1e1a6105
ntlm: fix negotiated flags usage
According to Microsoft document MS-NLMP, current flags usage is not
accurate: flag NTLMFLAG_NEGOTIATE_NTLM2_KEY controls the use of
extended security in an NTLM authentication message and NTLM version 2
cannot be negotiated within the protocol.

The solution implemented here is: if the extended security flag is set,
prefer using NTLM version 2 (as a server featuring extended security
should also support version 2). If version 2 has been disabled at
compile time, use extended security.

Tests involving NTLM are adjusted to this new behavior.

Fixes #6813
Closes #6849
2021-04-09 09:40:56 +02:00
Michael Forney
812fce9dcf
configure: remove use of RETSIGTYPE
This was previously defined by the obsolete AC_TYPE_SIGNAL macro,
which was removed in 2682e5f5. The deprecation text says

> Your code may safely assume C89 semantics that RETSIGTYPE is void.

So, remove it and just use void instead.

Closes #6861
2021-04-07 16:01:05 +02:00
Daniel Stenberg
605aa03ac1
http_proxy: only loop on 407 + close if we have credentials
... to fix the retry-loop.

Add test 718 to verify.

Reported-by: Daniel Kurečka
Fixes #6828
Closes #6850
2021-04-05 22:49:07 +02:00
Luke Granger-Brown
6d930d7306
file: support GETing directories again
After 957bc1881e, we no longer compute an
expected_size for directories. This has the upshot that when we compare
even an empty Range with the available size, we fail.

This brings back the previous behaviour, which was to succeed, but with
empty content. This also removes the "Accept-ranges: bytes" header,
which is nonsensical on directories.

Adds test 3016
Fixes #6845
Closes #6846
2021-04-05 12:22:07 +02:00
Daniel Stenberg
33ba0ecf48
define: remove CURL_DISABLE_NTLM ifdefs
It was never defined anywhere. Fixed disable-scan (test 1165) to also
scan headers, which found this issue.

Closes #6809
2021-03-29 14:43:54 +02:00
Viktor Szakats
7214288898
transfer: strip credentials from the auto-referer header field
Added test 2081 to verify.

CVE-2021-22876

Bug: https://curl.se/docs/CVE-2021-22876.html
2021-03-28 23:19:55 +02:00
Daniel Stenberg
85e6975643
copyright: update copyright year ranges to 2021
Reviewed-by: Emil Engler
Closes #6802
2021-03-27 23:00:14 +01:00
Daniel Stenberg
d003b0213a
tests/README.md: mentioned that en_US.UTF-8 is required
Reported-by: Oumph on github
Fixes #6768
2021-03-26 17:54:12 +01:00
Daniel Stenberg
3bbf62b5a4
http: strip default port from URL sent to proxy
To make sure the Host: header and the URL provide the same authority
portion when sent to the proxy, strip the default port number from the
URL if one was provided.

Reported-by: Michael Brown
Fixes #6769
Closes #6778
2021-03-23 13:33:49 +01:00
Daniel Stenberg
bd738bef12
ftpserver: only load the preprocessed test file
We always preprocess and tests are no longer sensible to load "raw"

Closes #6738
2021-03-19 15:57:27 +01:00
Daniel Stenberg
e2b4df7b5e
tests: use %TESTNUMBER instead of fixed number
This makes the tests easier to copy and relocate to other test numbers
without having to update content.

Closes #6738
2021-03-19 15:57:21 +01:00
Daniel Stenberg
6d176bee59
http: make 416 not fail with resume + CURLOPT_FAILONERRROR
When asked to resume a download, libcurl will convert that to HTTP logic
and if then the entire file is already transferred it will result in a
416 response from the HTTP server. With CURLOPT_FAILONERRROR set in that
scenario, it should *not* lead to an error return.

Updated test 1156, added test 1273

Reported-by: Jonathan Watt
Fixes #6740
Closes #6753
2021-03-17 08:26:46 +01:00
Daniel Stenberg
f83d4ea56e
gen.pl: quote "bare" minuses in the nroff curl.1
Reported-by: Alejandro Colomar
Fixes #6698
Closes #6722
2021-03-14 23:44:43 +01:00
Daniel Stenberg
f073a427b4
test220/314: adjust to run with Hyper 2021-03-11 22:45:27 +01:00
Daniel Stenberg
25837e32b0
test306: make it not run with Hyper
... as it tests HTTP/0.9 which Hyper doesn't support.
2021-03-11 22:43:09 +01:00
Daniel Stenberg
366c0a4aee
test304: header CRLF cleanup to work with Hyper 2021-03-11 22:43:09 +01:00
Daniel Stenberg
7b2f0676c2
FTP: allow SIZE to fail when doing (resumed) upload
Added test 362 to verify.

Reported-by: Jordan Brown
Regression since 7ea2e1d0c5 (7.73.0)
Fixes #6715
Closes #6725
2021-03-11 16:13:11 +01:00
Daniel Stenberg
dcccd8dd25
configure: provide Largefile feature for curl-config
... as cmake now does it correctly, and make test1014 check for it

Closes #6702
2021-03-11 10:12:30 +01:00
kokke
4088b25b33
lib1536: check ptr against NULL before dereferencing it
Closes #6710
2021-03-10 14:03:57 +01:00
kokke
b2d906754c
lib1537: check ptr against NULL before dereferencing it
Fixes #6707
Closes #6708
2021-03-10 14:01:55 +01:00
Marc Hoersken
311c31ec8e
runtests.pl: kill processes locking test log files
Introduce a new runtests.pl command option: -rm

For now only required and implemented for Windows.
Ignore stunnel logs due to long running processes.

Requires Sysinternals handle[64].exe to be on PATH.

Reviewed-by: Jay Satiro

Ref: #6058
Closes #6179
2021-03-01 20:19:01 +01:00
Marc Hoersken
b2b3c91ed7
pathhelp.pm: fix use of pwd -L in Msys environment
While Msys2 has a pwd binary which supports -L,
Msys1 only has a shell built-in with that feature.

Reviewed-by: Jay Satiro

Part of #6179
2021-03-01 20:18:04 +01:00
Jay Satiro
5743d66b91 tool_help: Increase space between option and description
- Increase the minimum number of spaces between the option and the
  description from 1 to 2.

Before:
~~~
 -u, --user <user:password> Server user and password
 -A, --user-agent <name> Send User-Agent <name> to server
 -v, --verbose       Make the operation more talkative
 -V, --version       Show version number and quit
 -w, --write-out <format> Use output FORMAT after completion
     --xattr         Store metadata in extended file attributes
~~~

After:
~~~
 -u, --user <user:password>  Server user and password
 -A, --user-agent <name>  Send User-Agent <name> to server
 -v, --verbose       Make the operation more talkative
 -V, --version       Show version number and quit
 -w, --write-out <format>  Use output FORMAT after completion
     --xattr         Store metadata in extended file attributes
~~~

Closes https://github.com/curl/curl/pull/6674
2021-02-28 13:41:15 -05:00
Daniel Gustafsson
82c583dcf0 cookies: Support multiple -b parameters
Previously only a single -b cookie parameter was supported with the last
one winning.  This adds support for supplying multiple -b params to have
them serialized semicolon separated.  Both cookiefiles and cookies can be
entered multiple times.

Closes #6649
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2021-02-25 18:12:28 +01:00
Jay Satiro
3d954e4e26 test1188: Check for --fail HTTP status
- Change the test to check for curl error on HTTP 404 Not Found.

test1188 tests "--write-out with %{onerror} and %{urlnum} to stderr".
Prior to this change it did that by specifying a non-existent host which
would cause an error. ISPs may hijack DNS and resolve non-existent hosts
so the test would not work if that was the case.

Ref: https://en.wikipedia.org/wiki/DNS_hijacking#Manipulation_by_ISPs
Ref: https://github.com/curl/curl/issues/6621
Ref: https://github.com/curl/curl/pull/6623

Closes https://github.com/curl/curl/pull/6637
2021-02-20 14:41:13 -05:00
Viktor Szakats
44872aefc2
http: add support to read and store the referrer header
- add CURLINFO_REFERER libcurl option
- add --write-out '%{referer}' command-line option
- extend --xattr command-line option to fill user.xdg.referrer.url extended
  attribute with the referrer (if there was any)

Closes #6591
2021-02-19 13:57:19 +00:00
Daniel Stenberg
35ca04ce1b
test1188: change error from connect to resolve error
Using the %NOLISTENPORT to trigger a connection failure is somewhat
"risky" (since it isn't guaranteed to not be listened to) and caused
occasional CI problems. This fix changes the infused error to be a more
reliable one but still verifies the --write-out functionality properly -
which is the purpose of this test.

Reported-by: Jay Satiro
Fixes #6621
Closes #6623
2021-02-18 11:10:47 +01:00
Daniel Stenberg
46620b9743
http: use credentials from transfer, not connection
HTTP auth "accidentally" worked before this cleanup since the code would
always overwrite the connection credentials with the credentials from
the most recent transfer and since HTTP auth is typically done first
thing, this has not been an issue. It was still wrong and subject to
possible race conditions or future breakage if the sequence of functions
would change.

The data.set.str[] strings MUST remain unmodified exactly as set by the
user, and the credentials to use internally are instead set/updated in
state.aptr.*

Added test 675 to verify different credentials used in two requests done
over a reused HTTP connection, which previously behaved wrongly.

Fixes #6542
Closes #6545
2021-02-13 22:36:15 +01:00
Daniel Stenberg
e992770e8d
test433: clear some home dir env variables
Follow-up to bd6b54ba1f

... so that XDG_CONFIG_HOME is the only home dir variable set and thus
used correctly in the test!

Fixes #6599
Closes #6600
2021-02-12 14:02:07 +01:00
Daniel Stenberg
88dd1a8a11
urldata: don't touch data->set.httpversion at run-time
Rename it to 'httpwant' and make a cloned field in the state struct as
well for run-time updates.

Also: refuse non-supported HTTP versions. Verified with test 129.

Closes #6585
2021-02-12 08:13:37 +01:00
Viktor Szakats
bd6b54ba1f
tests: disable .curlrc in more environments
by also setting CURL_HOME and XDG_CONFIG_HOME envvars to the local
directory.

Reviewed-by: Daniel Stenberg
Fixes #6595
Closes #6596
2021-02-11 22:06:02 +00:00
Daniel Stenberg
8a964cb217
curl: add --fail-with-body
Prevent both --fail and --fail-with-body on the same command line.

Verify with test 349, 360 and 361.

Closes #6449
2021-02-11 08:35:27 +01:00
Jay Satiro
89e572af82 tests: Fix tests failing due to change in curl --help
Follow-up to parent 3183217 which added add missing <mode> argument to
--create-file-mode <mode>.

Ref: https://github.com/curl/curl/issues/6590
2021-02-10 15:57:26 -05:00
Marcel Raad
1636076e70
server: remove redundant condition
`end` is always non-null here.

Closes https://github.com/curl/curl/pull/6576
2021-02-10 16:47:34 +01:00
Marcel Raad
cc9c4e0f4e
mqttd: remove unused variable
Closes https://github.com/curl/curl/pull/6576
2021-02-10 16:47:21 +01:00
Marcel Raad
4c02d003ff
tests: reduce variable scopes
Closes https://github.com/curl/curl/pull/6576
2021-02-10 16:47:09 +01:00
Daniel Stenberg
528f71c2ec
ftp: add 'list_only' to the transfer state struct
and rename it from 'ftp_list_only' since it is also used for SSH and
POP3. The state is updated internally for 'type=D' FTP URLs.

Added test case 1570 to verify.

Closes #6578
2021-02-09 14:06:28 +01:00
Daniel Stenberg
115c9e27f5
ftp: add 'prefer_ascii' to the transfer state struct
... and make sure the code never updates 'set.prefer_ascii' as it breaks
handle reuse which should use the setting as the user specified it.

Added test 1569 to verify: it first makes an FTP transfer with ';type=A'
and then another without type on the same handle and the second should
then use binary. Previously, curl failed this.

Closes #6578
2021-02-09 14:06:28 +01:00
Jay Satiro
cb2dc1ba89 lib: use int type for more port variables
This is a follow-up to 764c6bd. Prior to that change port variables
were usually type long.

Closes https://github.com/curl/curl/pull/6553
2021-02-09 02:53:31 -05:00
Jay Satiro
65ca229461 tool_writeout: refactor write-out and write-out json
- Deduplicate the logic used by write-out and write-out json.

Rather than have separate writeLong, writeString, etc, logic for
each of write-out and write-out json instead have respective shared
functions that can output either format and a 'use_json' parameter to
indicate whether it is json that is output.

This will make it easier to maintain. Rather than have to go through
two sets of logic now we only have to go through one.

- Support write-out %{errormsg} and %{exitcode} in json.

- Clarify in the doc that %{exitcode} is the exit code of the transfer.

Prior to this change it just said "The numerical exitcode" which
implies it's the exit code of the tool, and it's not necessarily that.

Closes https://github.com/curl/curl/pull/6544
2021-02-09 02:48:27 -05:00
Stephan Szabo
1269c80af1 file: Support unicode urls on windows
Closes https://github.com/curl/curl/pull/6501
2021-02-09 02:45:51 -05:00
Fabian Keil
461e3ef680
tests/server: Bump MAX_TAG_LEN to 200
This is useful for tests containing HTML inside of <data> sections.
For <img> tags it's not uncommon to be longer than the previous
limit of 79 bytes.

An example of a previously problem-causing tag is:
<img src="http://config.privoxy.org/send-banner?type=auto" border="0" title="Killed-http://www.privoxy.org/images/privoxy.png-by-size" width="88" height="31">
which is needed for a Privoxy test for the banners-by-size filter.

Previously it caused server failures like:
12:29:05.786961 ====> Client connect
12:29:05.787116 accept_connection 3 returned 4
12:29:05.787194 accept_connection 3 returned 0
12:29:05.787285 Read 119 bytes
12:29:05.787345 Process 119 bytes request
12:29:05.787407 Got request: GET /banners-by-size/9 HTTP/1.1
12:29:05.787464 Requested test number 9 part 0
12:29:05.787686 getpart() failed with error: -2
12:29:05.787744 - request found to be complete (9)
12:29:05.787912 getpart() failed with error: -2
12:29:05.788048 Wrote request (119 bytes) input to log/server.input
12:29:05.788157 Send response test9 section <data>
12:29:05.788443 getpart() failed with error: -2
12:29:05.788498 instructed to close connection after server-reply
12:29:05.788550 ====> Client disconnect 0
12:29:05.871448 exit_signal_handler: 15
12:29:05.871714 signalled to die
12:29:05.872040 ========> IPv4 sws (port 21108 pid: 51758) exits with signal (15)
2021-02-09 08:16:40 +01:00
Fabian Keil
e11e5b2d46
tests/badsymbols.pl: when opening '$incdir' fails include it in the error message 2021-02-09 08:16:40 +01:00
Fabian Keil
e6f4ab3477
runtests.1: document -o, -P, -L, and -E 2021-02-09 08:16:40 +01:00
Fabian Keil
203dc00718
runtests.pl: add %TESTNUMBER variable to make copying tests more convenient 2021-02-09 08:16:40 +01:00
Fabian Keil
8ad31a8c6e
runtests.pl: add an -o option to change internal variables
runtests.pl has lots of internal variables one might want to
change in certain situations, but adding a dedicated option
for every single one of them isn't practical.

Usage:
./runtests.pl -o TESTDIR=$privoxy_curl_test_dir -o HOSTIP=10.0.0.1 ...
2021-02-09 00:28:29 +01:00