Commit Graph

5557 Commits

Author SHA1 Message Date
Daniel Stenberg
dd4d1a2695
transfer: clear credentials when redirecting to absolute URL
Make sure the user and password for the second request is taken from the
redirected-to URL.

Add test case 899 to verify.

Reported-by: James Lucas
Fixes #11410
Closes #11412
2023-07-09 11:12:13 +02:00
Stefan Eissing
def99e011e
hyper: fix EOF handling on input
We ran out of disc space due to an infinite loop with debug logging

Fixes #11377
Closes #11385
Reported-by: Dan Fandrich
2023-07-08 23:01:24 +02:00
Stefan Eissing
29f33b3400 http2: fix crash in handling stream weights
- Delay the priority handling until the stream has been opened.

- Add test2404 to reproduce and verify.

Weights may change "on the run", which is why there are checks in
general egress handling. These must not trigger when the stream has not
been opened yet.

Reported-by: jbgoog@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/11379
Closes https://github.com/curl/curl/pull/11384
2023-06-28 16:32:16 -04:00
Stefan Eissing
cae12480fc tests/http: Add mod_h2 directive H2ProxyRequests
master of mod_h2 now requires H2ProxyRequests directives for forward
proxying with HTTP/2 to work.

Ref: https://github.com/icing/mod_h2/commit/3897a7086

Closes https://github.com/curl/curl/pull/11392
2023-06-28 15:37:10 -04:00
Stefan Eissing
873b958d0b
http2: h2 and h2-PROXY connection alive check fixes
- fix HTTP/2 check to not declare a connection dead when
  the read attempt results in EAGAIN
- add H2-PROXY alive check as for HTTP/2 that was missing
  and is needed
- add attach/detach around Curl_conn_is_alive() and remove
  these in filter methods
- add checks for number of connections used in some test_10
  proxy tunneling tests

Closes #11368
2023-06-22 17:32:15 +02:00
Stefan Eissing
d435bf1baf
http2: error stream resets with code CURLE_HTTP2_STREAM
- refs #11357, where it was reported that HTTP/1.1 downgrades
  no longer works
- fixed with suggested change
- added test_05_03 and a new handler in the curltest module
  to reproduce that downgrades work

Fixes #11357
Closes #11362
Reported-by: Jay Satiro
2023-06-22 17:07:27 +02:00
Dan Fandrich
0526525c33 runtests: work around a perl without SIGUSR1
At least msys2 perl v5.32.1 doesn't seem to define this signal. Since
this signal is only used for debugging, just ignore if setting it fails.

Reported-by: Marcel Raad
Fixes #11350
Closes #11366
2023-06-21 23:22:07 -07:00
Dan Fandrich
3cfd9f2378 runtests: include missing valgrind package
use valgrind was missing which caused torture tests with valgrind
enabled to fail.

Reported-by: Daniel Stenberg
Fixes #11364
Closes #11365
2023-06-21 15:33:01 -07:00
Dan Fandrich
62a9731533 runtests: use more consistent failure lines
After a test failure log a consistent log message to make it easier to
parse the log file.  Also, log a consistent message with "ignored" for
failures that cause the test to be not considered at all. These should
perhaps be counted in the skipped category, but this commit does not
change that behaviour.
2023-06-21 14:42:37 -07:00
Dan Fandrich
69e3485e0d runtests: consistently write the test check summary block
The memory check character was erroneously omitted if the memory
checking file was not available for some reason, making the block of
characters an inconsistent length.
2023-06-21 12:39:51 -07:00
Dan Fandrich
47c04d7136 test2600: fix the description
It looks like it was cut-and-pasted.

Closes #11354
2023-06-21 11:46:01 -07:00
Stefan Eissing
81e6793ec9
hyper: unslow
- refs #11203 where hyper was reported as being slow
- fixes hyper_executor_poll to loop until it is out of
  tasks as advised by @seanmonstar in https://github.com/hyperium/hyper/issues/3237
- added a fix in hyper io handling for detecting EAGAIN
- added some debug logs to see IO results
- pytest http/1.1 test cases pass
- pytest h2 test cases fail on connection reuse. HTTP/2
  connection reuse does not seem to work. Hyper submits
  a request on a reused connection, curl's IO works and
  thereafter hyper declares `Hyper: [1] operation was canceled: connection closed`
  on stderr without any error being logged before.

Fixes #11203
Reported-by: Gisle Vanem
Advised-by: Sean McArthur
Closes #11344
2023-06-20 09:11:41 +02:00
Dan Fandrich
2c5c7abb15 libtest: display the times after a test timeout error
This is to help with test failure debugging.

Ref: #11328
Closes #11329
2023-06-19 17:14:27 -07:00
Dan Fandrich
39c234c429 test2600: bump a test timeout
Case 1 failed at least once on GHA by going 30 msec too long.

Ref: #11328
2023-06-19 17:14:27 -07:00
Dan Fandrich
089a84660d runtests: better detect and handle pipe errors in the controller
Errors reading and writing to the pipes are now better detected and
propagated up to the main test loop so it can be cleanly shut down. Such
errors are usually due to a runner dying so it doesn't make much sense
to try to continue the test run.
2023-06-19 17:14:27 -07:00
Dan Fandrich
7d01ec8b74 runtests: cleanly abort the runner if the controller dies
If the controller dies unexpectedly, have the runner stop its servers
and exit cleanly. Otherwise, the orphaned servers will stay running in
the background.
2023-06-19 17:14:27 -07:00
Dan Fandrich
2fe52412b1 runtests: improve error logging
Give more information about test harness error conditions to help figure
out what might be wrong. Print some internal test state when SIGUSR1 is
sent to runtests.pl.

Ref: #11328
2023-06-19 17:14:27 -07:00
Dan Fandrich
847e42d868 runtests: better handle ^C during slow tests
Since the SIGINT handler now just sets a flag that must be checked in the
main controller loop, make sure that runs periodically.  Rather than
blocking on a response from a test runner near the end of the test run,
add a short timeout to allow it.
2023-06-19 17:14:27 -07:00
Dan Fandrich
1177c741b9 runtests: rename server command file
The name ftpserver.cmd was historical and has been used for more than
ftp for many years now. Rename it to plain server.cmd to reduce
confusion.
2023-06-19 17:14:27 -07:00
Dan Fandrich
72c5bb14e7 tests: improve reliability of TFTP tests
Stop checking the timeout used by the client under test (for most
tests). The timeout will change if the TFTP test server is slow (such as
happens on an overprovisioned CI server) because the client will retry
and reduce its timeout, and the actual value is not important for most
tests.

test285 is changed a different way, by increasing the connect timeout.
This improves test coverage by allowing the changed timeout value to be
checked, but improves reliability with a carefully-chosen timeout that
not only allows twice the time to respond as before, but also allows
several retries before the client will change its timeout value.

Ref: #11328
2023-06-19 17:10:24 -07:00
Daniel Stenberg
355f4144df
cf-socket: skip getpeername()/getsockname for TFTP
Since the socket is not connected then the call fails. When the call
fails, failf() is called to write an error message that is then
surviving and is returned when the *real* error occurs later. The
earlier, incorrect, error therefore hides the actual error message.

This could be seen in stderr for test 1007

Test 1007 has now been extended to verify the stderr message.

Closes #11332
2023-06-19 09:39:49 +02:00
Marcel Raad
1eca27f100
test427: add cookies feature and keyword
This test doesn't work with `--disable-cookies`.

Closes https://github.com/curl/curl/pull/11320
2023-06-16 10:37:07 +02:00
Chris Talbot
64aefea3d9 imap: Provide method to disable SASL if it is advertised
- Implement AUTH=+LOGIN for CURLOPT_LOGIN_OPTIONS to prefer plaintext
  LOGIN over SASL auth.

Prior to this change there was no method to be able to fall back to
LOGIN if an IMAP server advertises SASL capabilities. However, this may
be desirable for e.g. a misconfigured server.

Per: https://www.ietf.org/rfc/rfc5092.html#section-3.2

";AUTH=<enc-auth-type>" looks to be the correct way to specify what
authenication method to use, regardless of SASL or not.

Closes https://github.com/curl/curl/pull/10041
2023-06-15 21:31:02 -04:00
Stefan Eissing
e024d5665d
lib: add CURLINFO_CONN_ID and CURLINFO_XFER_ID
- add an `id` long to Curl_easy, -1 on init
- once added to a multi (or its own multi), it gets
  a non-negative number assigned by the connection cache
- `id` is unique among all transfers using the same
  cache until reaching LONG_MAX where it will wrap
  around. So, not unique eternally.
- CURLINFO_CONN_ID returns the connection id attached to
  data or, if none present, data->state.lastconnect_id
- variables and type declared in tool for write out

Closes #11185
2023-06-12 23:53:00 +02:00
Daniel Stenberg
6c25cd4508
test427: verify sending more cookies than fit in a 8190 bytes line
curl will then only populate the header with cookies that fit, dropping
ones that otherwise would have been sent

Ref: https://curl.se/mail/lib-2023-06/0020.html

Closes #11303
2023-06-12 14:15:37 +02:00
Daniel Stenberg
1899899bb7
testutil: allow multiple %-operators on the same line
Closes #11303
2023-06-12 14:15:34 +02:00
Dan Fandrich
b65086a83a tests: fix error messages & handling around sockets
The wrong error code was checked on Windows on UNIX socket failures,
which could have caused all UNIX sockets to be reported as having
errored and the tests therefore skipped. Also, a useless error message
was displayed on socket errors in many test servers on Windows because
strerror() doesn't work on WinSock error codes; perror() is overridden
there to work on all errors and is used instead.

Ref #11258
Closes #11265
2023-06-09 15:05:19 -07:00
Daniel Stenberg
dacd25888f
curl_url_set: enforce the max string length check for all parts
Update the docs and test 1559 accordingly

Closes #11273
2023-06-08 23:40:08 +02:00
Daniel Stenberg
3c9256c8a0
urlapi: have *set(PATH) prepend a slash if one is missing
Previously the code would just do that for the path when extracting the
full URL, which made a subsequent curl_url_get() of the path to
(unexpectedly) still return it without the leading path.

Amend lib1560 to verify this. Clarify the curl_url_set() docs about it.

Bug: https://curl.se/mail/lib-2023-06/0015.html
Closes #11272
Reported-by: Pedro Henrique
2023-06-08 16:08:45 +02:00
Dan Fandrich
7d62f0d9b8 runtests; give each server a unique log lock file
Logs are written by several servers and all of them must be finished
writing before the test results can be determined. This means each
server must have its own lock file rather than sharing a single one,
which is how it was done up to now. Previously, the first server to
complete a test would clear the lock before the other server was done,
which caused flaky tests.

Lock files are now all found in their own directory, so counting locks
equals counting the files in that directory.  The result is that the
proxy logs are now reliably written which actually changes the expected
output for two tests.

Fixes #11231
Closes #11259
2023-06-07 16:02:58 -07:00
Dan Fandrich
7af151ded0 runtests: make test file directories in log/N
Test files in subdirectories were not created after parallel test log
directories were moved down a level due to a now-bad comparison.

Follow-up to 92d7dd39

Ref #11264
Closes #11267
2023-06-07 15:09:35 -07:00
Daniel Stenberg
78886afb50
ws: make the curl_ws_meta() return pointer a const
The returned info is read-only for the user.

Closes #11261
2023-06-07 23:37:21 +02:00
Daniel Stenberg
92d7dd3955
runtests: move parallel log dirs from logN to log/N
Having several hundreds of them in there gets annoying.

Closes #11264
2023-06-07 23:32:41 +02:00
Dan Fandrich
c6d97bcea6 test447: move the test file into %LOGDIR 2023-06-07 13:53:08 -07:00
Viktor Szakats
3f8fc25720
cmake: add support for "unity" builds
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.

It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.

Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.

Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
  These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
  `statename`.
- rename `ssl_backend_data` structure to unique names for each
  TLS-backend, along with the `ssl_connect_data` struct member
  referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
  builds when compiling `lib/ldap.c`. To support "unity" builds, we had
  to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
  `schannel.h` option) _globally_. This caused an indirect inclusion of
  Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
  as well. This requires `[P]UNICODE_STRING` types, which is apperantly
  not defined automatically (as seen with both MSVS and mingw-w64).
  This patch includes `<subauth.h>` to fix it.
  Ref: https://github.com/curl/curl/runs/13987772013
  Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
  trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
  is built in static mode.

KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
  https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250

Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes #11095
2023-06-07 13:06:08 +00:00
Daniel Stenberg
cd18e5c464
test/.gitignore: ignore log* 2023-06-07 14:42:48 +02:00
Dan Fandrich
6e4fedeef7 runtests: document the -j parallel testing option
Reported-by: Daniel Stenberg
Ref: #10818
Closes #11255
2023-06-05 13:39:44 -07:00
Dan Fandrich
f24b4b980d runtests: create multiple test runners when requested
Parallel testing is enabled by using a nonzero value for the -j option
to runtests.pl. Performant values seem to be about 7*num CPU cores, or
1.3*num CPU cores if Valgrind is in use.

Flaky tests due to improper log locking (bug #11231) are exacerbated
while parallel testing, so it is not enabled by default yet.

Fixes #10818
Closes #11246
2023-06-05 11:16:06 -07:00
Dan Fandrich
d454af4737 runtests: handle repeating tests in multiprocess mode
Such as what happens with the --repeat option.  Some functions are
changed to pass the runner ID instead of relying on the non-unique test
number.

Ref: #10818
2023-06-05 11:16:06 -07:00
Dan Fandrich
51c22af890 runtests: buffer logmsg while running singletest()
This allows all messages relating to a single test case to be displayed
together at the end of the test.

Ref: #10818
2023-06-05 11:16:06 -07:00
Dan Fandrich
296baf45a0 runtests: call initserverconfig() in the runner
This must be done so variables pick up the runner's unique $LOGDIR.

Ref: #10818
2023-06-05 11:16:06 -07:00
Dan Fandrich
78d8bc4c63 runtests: use a per-runner random seed
Each runner needs a unique random seed to reduce the chance of port
number collisions. The new scheme uses a consistent per-runner source of
randomness which results in deterministic behaviour, as it did before.

Ref: #10818
2023-06-05 11:16:06 -07:00
Dan Fandrich
4317c5549b runtests: complete main test loop refactor for multiple runners
The main test loop is now able to handle multiple runners, or no
additional runner processes at all. At most one process is still
created, however.

Ref: #10818
2023-06-05 11:16:06 -07:00
Dan Fandrich
acc0a92897 runtests: prepare main test loop for multiple runners
Some variables are expanded to arrays and hashes so that multiple
runners can be used for running tests.

Ref: #10818
2023-06-05 11:16:06 -07:00
Raito Bezarius
0a75964d0d
haproxy: add --haproxy-clientip flag to spoof client IPs
CURLOPT_HAPROXY_CLIENT_IP in the library

Closes #10779
2023-06-05 20:08:37 +02:00
Daniel Stenberg
ba669d072d
urlapi: scheme starts with alpha
Add multiple tests to lib1560 to verify

Fixes #11249
Reported-by: ad0p on github
Closes #11250
2023-06-05 16:28:27 +02:00
Daniel Stenberg
8cf4189529
tests/servers: generate temp names in /tmp for unix domain sockets
... instead of putting them in the regular pid directories because
systems generally have strict length requirements for the path name to
be shorter than 107 bytes and we easily hit that boundary otherwise.

The new concept generates two random names: one for the socks daemon and
one for http.

Reported-by: Andy Fiddaman
Fixes #11152
Closes #11166
2023-06-02 23:10:46 +02:00
Stefan Eissing
f4b5c88ab7
http2: better support for --limit-rate
- leave transfer loop when --limit-rate is in effect and has
  been received
- adjust stream window size to --limit-rate plus some slack
  to make the server observe the pacing we want
- add test case to confirm behaviour

Closes #11115
2023-06-02 23:06:12 +02:00
Daniel Stenberg
bfa7006424
test447: test PUTting a file that grows
... and have curl trim the end when it reaches the expected total amount
of bytes instead of over-sending.

Reported-by: JustAnotherArchivist on github
Closes #11223
2023-06-01 13:43:31 +02:00
Daniel Stenberg
4efa0b5749
tests/servers.pm: pick unused port number with a server socket
This change replaces the previous method of picking a port number at
random to try to start servers on, then retrying up to ten times with
new random numbers each time, with a function that creates a server
socket on port zero, thereby getting a suitable random port set by the
kernel. That server socket is then closed and that port number is used
to setup the actual test server on.

There is a risk that *another* server can be started on the machine in
the time gap, but the server verification feature will detect that.

Closes #11220
2023-05-31 14:00:47 +02:00