Commit Graph

5493 Commits

Author SHA1 Message Date
Stefan Eissing
1886eef7fa
http2: increase stream window size to 10 MB
Reported-by: pandada8 on github

Fixes #11162
Closes #11167
2023-05-20 23:05:07 +02:00
Daniel Stenberg
92772e6d39
urlapi: allow numerical parts in the host name
It can only be an IPv4 address if all parts are all digits and no more than
four parts, otherwise it is a host name. Even slightly wrong IPv4 will now be
passed through as a host name.

Regression from 17a15d8846 shipped in 8.1.0

Extended test 1560 accordingly.

Reported-by: Pavel Kalyugin
Fixes #11129
Closes #11131
2023-05-19 16:01:26 +02:00
Stefan Eissing
408eb87bb3
cf-socket: completely remove the disabled USE_RECV_BEFORE_SEND_WORKAROUND
Closes #11118
2023-05-18 20:55:16 +02:00
Emanuele Torre
df6c2f7b54
urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirects
curl_url_set(uh, CURLUPART_URL, redirurl, flags)  was not respecing
CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY in the host part of redirurl
when redirecting to an absolute URL.

Closes #11136
2023-05-18 20:52:59 +02:00
Emanuele Torre
f198d33e8d
checksrc: disallow spaces before labels
Out of 415 labels throughout the code base, 86 of those labels were
not at the start of the line. Which means labels always at the start of
the line is the favoured style overall with 329 instances.

Out of the 86 labels not at the start of the line:
* 75 were indented with the same indentation level of the following line
* 8 were indented with exactly one space
* 2 were indented with one fewer indentation level then the following
  line
* 1 was indented with the indentation level of the following line minus
  three space (probably unintentional)

Co-Authored-By: Viktor Szakats

Closes #11134
2023-05-18 20:45:04 +02:00
Marcel Raad
e587598534
md4: only build when used
Its only usage in curl_ntlm_core.c is guarded by `USE_CURL_NTLM_CORE`,
so let's use this here too.

Ref: https://github.com/curl/curl/issues/11098
Closes https://github.com/curl/curl/pull/11102
2023-05-13 09:57:35 +02:00
Daniel Stenberg
efa6c644aa
test2306: verify getting a second response with folded headers
Reproduces the isue #11101 and verifies the fix.

Verifies a17b2a503f
2023-05-12 17:50:33 +02:00
Daniel Stenberg
39a33fcac0
tool_operate: refuse (--data or --form) and --continue-at combo
libcurl assumes that a --continue-at resumption is done to continue an
upload using the read callback and neither --data nor --form use
that and thus won't do what the user wants. Whatever the user wants
with this strange combination.

Add test 426 to verify.

Reported-by: Smackd0wn on github
Fixes #11081
Closes #11083
2023-05-08 14:10:44 +02:00
Dan Fandrich
3d75029859 runtests: fix -c option when run with valgrind
The curl binary argument wasn't being quoted properly. This seems to
have broken at some point after quoting was added in commit 606b29fe.

Reported-by: Daniel Stenberg
Ref: #11073
Fixes #11074
Closes #11076
2023-05-05 09:52:38 -07:00
Dan Fandrich
faebcee349 runtests: support creating more than one runner process
The controller currently only creates and uses one, but more are now
possible.

Ref: #10818
2023-05-05 00:45:43 -07:00
Dan Fandrich
38465f9a55 runtests: spawn a new process for the test runner
When the -j option is given, a new process is spawned in which the test
programs are run and from which test servers are started. Only one
process can be started at once, but this is sufficient to test that the
infrastructure can isolate those functions in a new task. There should
be no visible difference between the two modes at the moment.

Ref: #10818
Closes #11064
2023-05-05 00:45:43 -07:00
Dan Fandrich
d4a1b5b60c runtests: turn singletest() into a state machine
This allows it to run in a non-blocking manner.

Ref: #10818
2023-05-05 00:45:43 -07:00
Dan Fandrich
a98277fcc7 runtests: change runner interface to be asynchronous
Program arguments are marshalled and then written to the end of a pipe
which is later read from and the arguments unmarshalled before the
desired function is called normally.  The function return values are
then marshalled and written into another pipe when is later read from
and unmarshalled before being returned to the caller.

The implementation is currently blocking but can be made non-blocking
without any changes to the API.  This allows calling multiple runners
without blocking in the future.

Ref: #10818
2023-05-05 00:45:43 -07:00
Dan Fandrich
0754de758a runtests: call citest_finishtest in singletest
This is where citest_starttest is called.

Ref: #10818
2023-05-05 00:45:43 -07:00
Dan Fandrich
d4d22adf40 runtests: add a runner initialization function
This sets up the runner environment to start running tests.

Ref: #10818
2023-05-05 00:45:43 -07:00
Dan Fandrich
82fa5ca836 runtests: remove directory from server filename variables
There will soon be multiple log directories so the paths will no longer
be static in runtests.pl. Also, get rid of $SERVER2IN which was not
used.

Ref: #10818
2023-05-05 00:45:43 -07:00
Dan Fandrich
b71a0c3ec9 runtests: reduce package exports after refactoring
Some recent refactoring made these export no longer necessary. Also,
stop displaying the Unix socket paths at startup since there will soon
be many of them and they're not that interesting.

Ref: #10818
2023-05-05 00:45:43 -07:00
Dan Fandrich
9fdad9dea1 runtests: use a function to obtain $LOGDIR for a test
This will no longer be static soon.

Ref: #10818
2023-05-05 00:45:43 -07:00
Daniel Stenberg
9ce7eee070
checksrc: find bad indentation in conditions without open brace
If the previous line starts with if/while/for AND ends with a closed
parenthesis and there's an equal number of open and closed parentheses
on that line, verify that this line is indented $indent more steps, if
not a cpp line.

Also adjust the fall-out from this fix.

Closes #11054
2023-04-28 23:11:00 +02:00
Stefan Eissing
a9b7f72bc9
http2: do flow window accounting for cancelled streams
- nghttp2 does not free connection level window flow for
  aborted streams
- when closing transfers, make sure that any buffered
  response data is "given back" to the flow control window
- add tests test_02_22 and test_02_23 to reproduce

Closes #11052
2023-04-28 13:55:39 +02:00
Marcel Raad
da2470de96
tests/http: make curl_setup.h the first include
This is required for the macros there to take effect for system
libraries. Specifically, including the system libraries first led to
warnings about `_FILE_OFFSET_BITS` being redefined in curl_config.h on
the Solaris autobuilds for ws-data.c and ws-pingpong.c.
Also make the curl includes come first for the other source files here
for consistency.

Closes https://github.com/curl/curl/pull/11046
2023-04-28 11:29:19 +02:00
Dan Fandrich
2572e1333e tests/http: fix out-of-tree builds
Add both lib/ directories (src & build) to the search path so
curl_setup.h and its dependencies can be found.

Followup-to acd82c8b

Ref: #11006
Closes #11036
2023-04-27 00:15:41 -07:00
Stefan Eissing
acd82c8bfd
tests/http: more tests with specific clients
- Makefile support for building test specific clients in tests/http/clients
- auto-make of clients when invoking pytest
- added test_09_02 for server PUSH_PROMISEs using clients/h2-serverpush
- added test_02_21 for lib based downloads and pausing/unpausing transfers

curl url parser:
- added internal method `curl_url_set_authority()` for setting the
  authority part of a url (used for PUSH_PROMISE)

http2:
- made logging of PUSH_PROMISE handling nicer

Placing python test requirements in requirements.txt files
- separate files to base test suite and http tests since use
  and module lists differ
- using the files in the gh workflows

websocket test cases, fixes for we and bufq
- bufq: account for spare chunks in space calculation
- bufq: reset chunks that are skipped empty
- ws: correctly encode frames with 126 bytes payload
- ws: update frame meta information on first call of collect
  callback that fills user buffer
- test client ws-data: some test/reporting improvements

Closes #11006
2023-04-26 23:24:46 +02:00
Daniel Stenberg
f98344c4ae
man pages: simplify the .TH sections
- remove the version numbers
- simplify the texts

The date and version number will be put there for releases when maketgz
runs the updatemanpages.pl script.

Closes #11029
2023-04-26 10:17:13 +02:00
Daniel Stenberg
199f2d440d
hostcheck: fix host name wildcard checking
The leftmost "label" of the host name can now only match against single
'*'. Like the browsers have worked for a long time.

- extended unit test 1397 for this
- move some SOURCE variables from unit/Makefile.am to unit/Makefile.inc

Reported-by: Hiroki Kurosawa
Closes #11018
2023-04-26 09:07:27 +02:00
Dan Fandrich
a4aebd73e2 smbserver: remove temporary files before exit
Each execution of test 1451 would leave a file in /tmp before. Since
Windows can't delete a file while it's open, all the temporary file
names are stored and deleted on exit.

Closes #10990
2023-04-25 16:12:20 -07:00
Stefan Eissing
930c00c259
Websocket en-/decoding
- state is fully kept at connection, since curl_ws_send() and
  curl_ws_rec() have lifetime beyond usual transfers
- no more limit on frame sizes

Reported-by: simplerobot on github
Fixes #10962
Closes #10999
2023-04-25 23:16:51 +02:00
Stefan Eissing
cab2d56ea5
h2/h3: replace state.drain counter with state.dselect_bits
- `drain` was used by http/2 and http/3 implementations to indicate
  that the transfer requires send/recv independant from its socket
  poll state. Intended as a counter, it was used as bool flag only.
- a similar mechanism exists on `connectdata->cselect_bits` where
  specific protocols can indicate something similar, only for the
  whole connection.
- `cselect_bits` are cleard in transfer.c on use and, importantly,
  also set when the transfer loop expended its `maxloops` tries.
  `drain` was not cleared by transfer and the http2/3 implementations
  had to take care of that.
- `dselect_bits` is cleared *and* set by the transfer loop. http2/3
  does no longer clear it, only set when new events happen.

This change unifies the handling of socket poll overrides, extending
`cselect_bits` by a easy handle specific value and a common treatment in
transfers.

Closes #11005
2023-04-25 17:49:28 +02:00
Dan Fandrich
189f9e23b6 runtests: support buffering log messages in runner & servers
Log messages generated with logmsg can now be buffered and returned from
the runner as a return value.  This will be needed with parallel testing
to allow all messages for one test to be displayed together instead of
interspersed with messages of multiple tests. Buffering can be disabled
by setting a logging callback function with setlogfunc, which is
currently being done to preserve existing logging behaviour for now.

Some additional output is generated in verbose and debugprotocol modes,
which don't always use logmsg. These modes also impact some servers
which generate extra messages. No attempt is made to buffer everything
if these modes are enabled.

Ref: #10818
Closes #11016
2023-04-24 16:28:23 -07:00
Dan Fandrich
c3453dcb9d runtests: more consistently use logmsg in server control code
Also, display an error when sshversioninfo returns one.

Ref: #10818
2023-04-24 16:04:24 -07:00
Dan Fandrich
b88ea272c1 runtests: create runner functions for clearlocks and stopservers
runtests.pl now uses runner for all server actions beyond the initial
variable configuration.

Ref: #10818
2023-04-24 16:04:24 -07:00
Dan Fandrich
640f4c2267 runtests: tightened servers package exports
The defaults are intended for runtests.pl, whereas runner.pm needs to
explicitly specify them.
2023-04-24 16:04:24 -07:00
Dan Fandrich
8da49c9e38 runtests: display logs on server failure in singletest()
This is closer to the place where logs are displayed on test failure.
Also, only display these logs if -p is given, which is the same flag
that controls display of test failure logs. Some server log files
need to be deleted later so that they stay around long enough to be
displayed on failure.

Ref: #10818
2023-04-24 16:03:44 -07:00
Dan Fandrich
90158f0bab runtests: turn a print into a logmsg
Also enable another couple of useful messages in verbose mode.

Ref: #10818
2023-04-24 12:13:17 -07:00
Dan Fandrich
a8706fd8d0 runtests: move showdiff into runtests.pl
It's not used anywhere else.
2023-04-22 13:07:35 -07:00
Dan Fandrich
20fa5b74a5 devtest: add a new script for testing the test harness
This is currently useful for starting a test server on its own without
an associated test, which can be used for interactive curl testing or
for validating parts of the test harness itself. More commands can be
added to perform additional functions in the future.

Ref: #10818
Closes #11008
2023-04-22 13:07:35 -07:00
Dan Fandrich
a549e046b1 runtests: refactor the main test loop into two
The test loop now has an initial loop that first runs through all
possible tests to build a set of those to attempt on this run based on
features and keywords and only then goes through that new list to run
them.  This actually makes it three loops through all tests cases, as
there is an existing loop that gathers possible test numbers from the
test files on disk.

This has two minor effects on the output: all the tests that will be
skipped are displayed at the start (instead of being interspersed with
other tests) and the -l option no longer shows a count of tests at the
end or a (misleading) statement that tests have run successfully. The
skipped tests are also omitted from the test results sent to AppVeyor
and Azure in CI builds.

Another effect is a reduction in the amount of work considered part of
the "Test definition reading and preparation time" reported with -r
making those figures slightly lower than before.

Ref: #10818
2023-04-22 13:07:35 -07:00
Dan Fandrich
020cf1c117 runtests: track only the current test timings in runner.pm
This avoids passing these data through through global variables, which
soon won't be possible.

Ref: #10818
2023-04-22 12:58:04 -07:00
Dan Fandrich
c6e7f6c61f runtests: skip test preprocessing when doing -l
This speeds up the output tremendously by avoiding unnecessary work.
2023-04-22 12:58:03 -07:00
Dan Fandrich
6210bc0111 runtests: simplify value returned regarding use of valgrind
As a side effect this will now also show in verbose mode that valgrind
is being skipped on tests that explicitly disable it, such as 600.

Ref: #10818
2023-04-22 12:58:03 -07:00
Dan Fandrich
4a41745e21 runtests: fix quoting in Appveyor and Azure test integration
Test 1442's name was not quoted correctly so wasn't registered in
Appveyor and it had the wrong name in Azure. The JSON string quotes were
also invalid, even though both servers happened to accept it regardless.

Closes #11010
2023-04-22 11:50:03 -07:00
Dan Fandrich
47f2e556aa runtests: spread out the port numbers used by servers
The server ports are chosen randomly for each server, but the random
ranges chosen were inconsistently-sized and overlapping. Now, they are
spread out more so at least the first random port chosen for each server
is guaranteed to not also be chosen by another server. The starting port
numbers are also raised to put them in the Ephemeral Port range—not the
range defined by RFC 6335 but the one used by Linux, which starts lower
and gives us more room to work with.

Reported-by: Daniel Stenberg
2023-04-18 13:18:17 -07:00
Dan Fandrich
04113319db runtests: fix problems on <killserver> failure
The verify time must be set in this case, like all cases. An error
message needs to be displayed as well.
2023-04-18 13:18:17 -07:00
Dan Fandrich
1f6a9f48aa runtests: fix perl warning when <tool> is wrong 2023-04-18 13:18:17 -07:00
Dan Fandrich
f9e8c5f578 runtests: don't try to stop stunnel before trying again
Calling stopserver() before retrying stunnel due to an error would stop
the dependent server (such as HTTP) meaning stunnel would have nothing
to talk to when it came up. Don't try to force a stop when it didn't
actually start.  Also, don't mark the server as bad for future use when
it starts up on a retry.

Reported-by: eaglegai at github
Tested-by: eaglegai at github
Fixes #10976
2023-04-18 13:18:17 -07:00
Dan Fandrich
2e0b70b8bc runtests: don't accidentally randomly choose the same port
If a server couldn't be started on a port, a new one is randomly chosen
and the server is tried again. Avoid accidentally using a
randomly-chosen 0 port offset by adding 1 to the random number.

Found-by: Daniel Stenberg
2023-04-18 13:18:17 -07:00
Dan Fandrich
b11840807b runtests: don't attempt to use a port we know is in use
This reduces the startup time when there is a known conflict on the
random port chosen for a server.  This was already done for stunnel, but
now it's done for all servers.
2023-04-18 13:18:17 -07:00
Dan Fandrich
200c409021 http-server: fix server name in a log message
This changed when the file was renamed in commit cbf57176
2023-04-18 13:18:17 -07:00
Dan Fandrich
707f74c04a runtests: refactor into more packages
testutil.pm now contains a few miscellaneous functions that are used in
several places but have no better place to live.  subvariables moves to
servers.pm since most variables that it substitutes relate to servers,
so this is the most appropriate place. Rename a few functions for better
naming consistency.

Ref: #10818
Closes #10995
2023-04-18 13:18:17 -07:00
Dan Fandrich
ba51b39732 runtests: call timestampskippedevents() in singletest
..rather than by the runner
2023-04-18 13:18:17 -07:00