Commit Graph

5578 Commits

Author SHA1 Message Date
Daniel Stenberg
03eba3c8b2
tests/tftpd+mqttd: make variables static to silence picky warnings
Closes #11594
2023-08-06 17:21:25 +02:00
Stefan Eissing
e12b39e133
trace: make tracing available in non-debug builds
Add --trace-config to curl

Add curl_global_trace() to libcurl

Closes #11421
2023-08-03 17:32:25 +02:00
Daniel Stenberg
260f51ae8b
test497: verify rejecting too large incoming headers 2023-08-03 17:08:56 +02:00
Stefan Eissing
944e219f10
http2: upgrade tests and add fix for non-existing stream
- check in h2 filter recv that stream actually exists
  and return error if not
- add test for parallel, extreme h2 upgrades that fail if
  connections get reused before fully switched
- add h2 upgrade upload test just for completeness

Closes #11563
2023-08-03 17:05:13 +02:00
Viktor Szakats
db70846e2e
tests: ensure libcurl.def contains all exports
Add `test1279` to verify that `libcurl.def` lists all exported API
functions found in libcurl headers.

Also:

- extend test suite XML `stdout` tag with the `loadfile` attribute.

- fix `tests/extern-scan.pl` and `test1135` to include websocket API.

- use all headers (sorted) in `test1135` instead of a manual list.

- add options `--sort`, `--heading=` to `tests/extern-scan.pl`.

- add `libcurl.def` to the auto-labeler GHA task.

Follow-up to 2ebc74c36a

Closes #11570
2023-08-03 11:07:47 +00:00
Stefan Eissing
bb07240263
tests: fix h3 server check and parallel instances
- fix check for availability of nghttpx server
- add `tcp` frontend config for same port as quic, as
  without this, port 3000 is bound which clashes for parallel
  testing

Closes #11553
2023-08-01 16:11:48 +02:00
Alexander Jaeger
959e613e7c
misc: fix various typos
Closes #11561
2023-08-01 14:09:32 +02:00
Daniel Stenberg
49e2443186
urlapi: make sure zoneid is also duplicated in curl_url_dup
Add several curl_url_dup() tests to the general lib1560 test.

Reported-by: Rutger Broekhoff
Bug: https://curl.se/mail/lib-2023-07/0047.html
Closes #11549
2023-08-01 08:00:28 +02:00
Daniel Stenberg
1032f56efa
curl: make %output{} in -w specify a file to write to
It can be used multiple times. Use %output{>>name} to append.

Add docs. Test 990 and 991 verify.

Idea: #11400
Suggested-by: ed0d2b2ce19451f2
Closes #11416
2023-07-31 23:10:59 +02:00
Daniel Stenberg
2e160c9c65
tool: add "variable" support
Add support for command line variables. Set variables with --variable
name=content or --variable name@file (where "file" can be stdin if set
to a single dash (-)).

Variable content is expanded in option parameters using "{{name}}"
(without the quotes) if the option name is prefixed with
"--expand-". This gets the contents of the variable "name" inserted, or
a blank if the name does not exist as a variable. Insert "{{" verbatim
in the string by prefixing it with a backslash, like "\\{{".

Import an environment variable with --variable %name. It makes curl exit
with an error if the environment variable is not set. It can also rather
get a default value if the variable does not exist, using =content or
@file like shown above.

Example: get the USER environment variable into the URL:

 --variable %USER
 --expand-url = "https://example.com/api/{{USER}}/method"

When expanding variables, curl supports a set of functions that can make
the variable contents more convenient to use. It can trim leading and
trailing white space with "trim", output the contents as a JSON quoted
string with "json", URL encode it with "url" and base 64 encode it with
"b64". To apply functions to a variable expansion, add them colon
separated to the right side of the variable. They are then performed in
a left to right order.

Example: get the contents of a file called $HOME/.secret into a variable
called "fix". Make sure that the content is trimmed and percent-encoded
sent as POST data:

  --variable %HOME=/home/default
  --expand-variable fix@{{HOME}}/.secret
  --expand-data "{{fix:trim:url}}"
  https://example.com/

Documented. Many new test cases.

Co-brainstormed-by: Emanuele Torre
Assisted-by: Jat Satiro
Closes #11346
2023-07-31 11:51:34 +02:00
Patrick Monnerat
038c46f61f
configure, cmake, lib: more form api deprecation
Introduce a --enable-form-api configure option to control its inclusion
in builds. The condition name defined for it is CURL_DISABLE_FORM_API.

Form api code is dependent of MIME: configure and CMake handle this
dependency automatically: CMake by making it a dependent option
explicitly, configure by inheriting the MIME value by default and
rejecting explicit incompatible values.

"form-api" is now a new hidden test feature.

Update libcurl modules to respect this option and adjust tests
accordingly.

Closes #9621
2023-07-31 08:31:38 +02:00
Viktor Szakats
86eff0b0d5
nss: delete more NSS references
Fix the distcheck CI failure and delete more NSS references.

Follow-up to 7c8bae0d9c

Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg
Closes #11548
2023-07-30 21:55:29 +00:00
Daniel Stenberg
7c8bae0d9c
nss: remove support for this TLS library
Closes #11459
2023-07-29 23:44:28 +02:00
Viktor Szakats
1199308dbc
cmake: support building static and shared libcurl in one go
This patch adds the ability to build a static and shared libcurl library
in a single build session. It also adds an option to select which one to
use when building the curl executable.

New build options:
- `BUILD_STATIC_LIBS`. Default: `OFF`.
  Enabled automatically if `BUILD_SHARED_LIBS` is `OFF`.
- `BUILD_STATIC_CURL`. Default: `OFF`.
  Requires `BUILD_STATIC_LIBS` enabled.
  Enabled automatically if building static libcurl only.
- `STATIC_LIB_SUFFIX`. Default: empty.
- `IMPORT_LIB_SUFFIX`. Default: `_imp` if implib filename would collide
  with static lib name (typically with MSVC) in Windows builds.
  Otherwise empty.

Also:

- Stop setting the `CURL_STATICLIB` macro via `curl_config.h`, and pass
  it directly to the compiler. This also allows to delete a condition
  from `tests/server/CMakeLists.txt`.

- Complete a TODO by following the logic used in autotools (also for
  `LIBCURL_NO_SHARED`), and set `-DCURL_STATICLIB` in `Cflags:` of
  `libcurl.pc` for _static-only_ curl builds.

- Convert an existing CI test to build both shared and static libcurl.

Closes #11505
2023-07-29 00:40:01 +00:00
Stefan Eissing
3f518ce4b1
CI ngtcp2+quictls: use nghttpx cache as in quiche build 2023-07-27 16:59:37 +02:00
Stefan Eissing
bea86dad6e
GHA: adding quiche workflow
- adding separate quiche workflow to also build nghttpx server for testing

Closes #11517
2023-07-26 18:19:30 +02:00
Stefan Eissing
c76df46a19
http: VLH, very large header test and fixes
- adding tests using very large passwords in auth
- fixes general http sending to treat h3 like h2, and
  not like http1.1
- eliminate H2_HEADER max definitions and use the commmon
  DYN_HTTP_REQUEST everywhere, different limits do not help
- fix http2 handling of requests denied by nghttp2 on send
  to immediately report the refused stream

Closes #11509
2023-07-24 19:25:20 +02:00
Alois Klink
849b3fed04
unittest/makefile: remove unneeded unit1621_LDADD
The `unit1621_LDADD` variable has the exact same value as the `LDADD`
flag in `Makefile.am`, except without `@LDFLAGS@ @LIBCURL_LIBS@`.

This was originally added by [98e6629][], but I can't see any reason
why it exists, so we should remove it to clean things up.

[98e6629]: 98e6629154

Closes #11494
2023-07-21 12:08:12 +02:00
Alois Klink
50c819e84d
unittest/makefile: remove unneeded unit1394_LDADD
These custom `unit1394_LDADD` and similar automake overrides are no
longer neded. They were originally added by added by [8dac7be][] for
metalink support, but are no longer after [265b14d][] removed metalink.

[8dac7be]: 8dac7be438
[265b14d]: 265b14d6b3

Closes #11494
2023-07-21 12:08:02 +02:00
Alois Klink
39e7c22bb4
cmake: add libcurlu/libcurltool for unit tests
Add a `libcurlu`/`libcurltool` static library that is compiled only for
unit tests. We use `EXCLUDE_FROM_ALL` to make sure that they're not
built by default, they're only built if unit tests are built.

These libraries allow us to compile every unit test with CMake.

Closes #11446
2023-07-21 12:04:45 +02:00
Daniel Stenberg
c42c6eb245
test979: test -u with redirect to (the same) absolute host
Verifies #11492
2023-07-21 11:59:25 +02:00
Daniel Stenberg
dd4d1a2695
transfer: clear credentials when redirecting to absolute URL
Make sure the user and password for the second request is taken from the
redirected-to URL.

Add test case 899 to verify.

Reported-by: James Lucas
Fixes #11410
Closes #11412
2023-07-09 11:12:13 +02:00
Stefan Eissing
def99e011e
hyper: fix EOF handling on input
We ran out of disc space due to an infinite loop with debug logging

Fixes #11377
Closes #11385
Reported-by: Dan Fandrich
2023-07-08 23:01:24 +02:00
Stefan Eissing
29f33b3400 http2: fix crash in handling stream weights
- Delay the priority handling until the stream has been opened.

- Add test2404 to reproduce and verify.

Weights may change "on the run", which is why there are checks in
general egress handling. These must not trigger when the stream has not
been opened yet.

Reported-by: jbgoog@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/11379
Closes https://github.com/curl/curl/pull/11384
2023-06-28 16:32:16 -04:00
Stefan Eissing
cae12480fc tests/http: Add mod_h2 directive H2ProxyRequests
master of mod_h2 now requires H2ProxyRequests directives for forward
proxying with HTTP/2 to work.

Ref: https://github.com/icing/mod_h2/commit/3897a7086

Closes https://github.com/curl/curl/pull/11392
2023-06-28 15:37:10 -04:00
Stefan Eissing
873b958d0b
http2: h2 and h2-PROXY connection alive check fixes
- fix HTTP/2 check to not declare a connection dead when
  the read attempt results in EAGAIN
- add H2-PROXY alive check as for HTTP/2 that was missing
  and is needed
- add attach/detach around Curl_conn_is_alive() and remove
  these in filter methods
- add checks for number of connections used in some test_10
  proxy tunneling tests

Closes #11368
2023-06-22 17:32:15 +02:00
Stefan Eissing
d435bf1baf
http2: error stream resets with code CURLE_HTTP2_STREAM
- refs #11357, where it was reported that HTTP/1.1 downgrades
  no longer works
- fixed with suggested change
- added test_05_03 and a new handler in the curltest module
  to reproduce that downgrades work

Fixes #11357
Closes #11362
Reported-by: Jay Satiro
2023-06-22 17:07:27 +02:00
Dan Fandrich
0526525c33 runtests: work around a perl without SIGUSR1
At least msys2 perl v5.32.1 doesn't seem to define this signal. Since
this signal is only used for debugging, just ignore if setting it fails.

Reported-by: Marcel Raad
Fixes #11350
Closes #11366
2023-06-21 23:22:07 -07:00
Dan Fandrich
3cfd9f2378 runtests: include missing valgrind package
use valgrind was missing which caused torture tests with valgrind
enabled to fail.

Reported-by: Daniel Stenberg
Fixes #11364
Closes #11365
2023-06-21 15:33:01 -07:00
Dan Fandrich
62a9731533 runtests: use more consistent failure lines
After a test failure log a consistent log message to make it easier to
parse the log file.  Also, log a consistent message with "ignored" for
failures that cause the test to be not considered at all. These should
perhaps be counted in the skipped category, but this commit does not
change that behaviour.
2023-06-21 14:42:37 -07:00
Dan Fandrich
69e3485e0d runtests: consistently write the test check summary block
The memory check character was erroneously omitted if the memory
checking file was not available for some reason, making the block of
characters an inconsistent length.
2023-06-21 12:39:51 -07:00
Dan Fandrich
47c04d7136 test2600: fix the description
It looks like it was cut-and-pasted.

Closes #11354
2023-06-21 11:46:01 -07:00
Stefan Eissing
81e6793ec9
hyper: unslow
- refs #11203 where hyper was reported as being slow
- fixes hyper_executor_poll to loop until it is out of
  tasks as advised by @seanmonstar in https://github.com/hyperium/hyper/issues/3237
- added a fix in hyper io handling for detecting EAGAIN
- added some debug logs to see IO results
- pytest http/1.1 test cases pass
- pytest h2 test cases fail on connection reuse. HTTP/2
  connection reuse does not seem to work. Hyper submits
  a request on a reused connection, curl's IO works and
  thereafter hyper declares `Hyper: [1] operation was canceled: connection closed`
  on stderr without any error being logged before.

Fixes #11203
Reported-by: Gisle Vanem
Advised-by: Sean McArthur
Closes #11344
2023-06-20 09:11:41 +02:00
Dan Fandrich
2c5c7abb15 libtest: display the times after a test timeout error
This is to help with test failure debugging.

Ref: #11328
Closes #11329
2023-06-19 17:14:27 -07:00
Dan Fandrich
39c234c429 test2600: bump a test timeout
Case 1 failed at least once on GHA by going 30 msec too long.

Ref: #11328
2023-06-19 17:14:27 -07:00
Dan Fandrich
089a84660d runtests: better detect and handle pipe errors in the controller
Errors reading and writing to the pipes are now better detected and
propagated up to the main test loop so it can be cleanly shut down. Such
errors are usually due to a runner dying so it doesn't make much sense
to try to continue the test run.
2023-06-19 17:14:27 -07:00
Dan Fandrich
7d01ec8b74 runtests: cleanly abort the runner if the controller dies
If the controller dies unexpectedly, have the runner stop its servers
and exit cleanly. Otherwise, the orphaned servers will stay running in
the background.
2023-06-19 17:14:27 -07:00
Dan Fandrich
2fe52412b1 runtests: improve error logging
Give more information about test harness error conditions to help figure
out what might be wrong. Print some internal test state when SIGUSR1 is
sent to runtests.pl.

Ref: #11328
2023-06-19 17:14:27 -07:00
Dan Fandrich
847e42d868 runtests: better handle ^C during slow tests
Since the SIGINT handler now just sets a flag that must be checked in the
main controller loop, make sure that runs periodically.  Rather than
blocking on a response from a test runner near the end of the test run,
add a short timeout to allow it.
2023-06-19 17:14:27 -07:00
Dan Fandrich
1177c741b9 runtests: rename server command file
The name ftpserver.cmd was historical and has been used for more than
ftp for many years now. Rename it to plain server.cmd to reduce
confusion.
2023-06-19 17:14:27 -07:00
Dan Fandrich
72c5bb14e7 tests: improve reliability of TFTP tests
Stop checking the timeout used by the client under test (for most
tests). The timeout will change if the TFTP test server is slow (such as
happens on an overprovisioned CI server) because the client will retry
and reduce its timeout, and the actual value is not important for most
tests.

test285 is changed a different way, by increasing the connect timeout.
This improves test coverage by allowing the changed timeout value to be
checked, but improves reliability with a carefully-chosen timeout that
not only allows twice the time to respond as before, but also allows
several retries before the client will change its timeout value.

Ref: #11328
2023-06-19 17:10:24 -07:00
Daniel Stenberg
355f4144df
cf-socket: skip getpeername()/getsockname for TFTP
Since the socket is not connected then the call fails. When the call
fails, failf() is called to write an error message that is then
surviving and is returned when the *real* error occurs later. The
earlier, incorrect, error therefore hides the actual error message.

This could be seen in stderr for test 1007

Test 1007 has now been extended to verify the stderr message.

Closes #11332
2023-06-19 09:39:49 +02:00
Marcel Raad
1eca27f100
test427: add cookies feature and keyword
This test doesn't work with `--disable-cookies`.

Closes https://github.com/curl/curl/pull/11320
2023-06-16 10:37:07 +02:00
Chris Talbot
64aefea3d9 imap: Provide method to disable SASL if it is advertised
- Implement AUTH=+LOGIN for CURLOPT_LOGIN_OPTIONS to prefer plaintext
  LOGIN over SASL auth.

Prior to this change there was no method to be able to fall back to
LOGIN if an IMAP server advertises SASL capabilities. However, this may
be desirable for e.g. a misconfigured server.

Per: https://www.ietf.org/rfc/rfc5092.html#section-3.2

";AUTH=<enc-auth-type>" looks to be the correct way to specify what
authenication method to use, regardless of SASL or not.

Closes https://github.com/curl/curl/pull/10041
2023-06-15 21:31:02 -04:00
Stefan Eissing
e024d5665d
lib: add CURLINFO_CONN_ID and CURLINFO_XFER_ID
- add an `id` long to Curl_easy, -1 on init
- once added to a multi (or its own multi), it gets
  a non-negative number assigned by the connection cache
- `id` is unique among all transfers using the same
  cache until reaching LONG_MAX where it will wrap
  around. So, not unique eternally.
- CURLINFO_CONN_ID returns the connection id attached to
  data or, if none present, data->state.lastconnect_id
- variables and type declared in tool for write out

Closes #11185
2023-06-12 23:53:00 +02:00
Daniel Stenberg
6c25cd4508
test427: verify sending more cookies than fit in a 8190 bytes line
curl will then only populate the header with cookies that fit, dropping
ones that otherwise would have been sent

Ref: https://curl.se/mail/lib-2023-06/0020.html

Closes #11303
2023-06-12 14:15:37 +02:00
Daniel Stenberg
1899899bb7
testutil: allow multiple %-operators on the same line
Closes #11303
2023-06-12 14:15:34 +02:00
Dan Fandrich
b65086a83a tests: fix error messages & handling around sockets
The wrong error code was checked on Windows on UNIX socket failures,
which could have caused all UNIX sockets to be reported as having
errored and the tests therefore skipped. Also, a useless error message
was displayed on socket errors in many test servers on Windows because
strerror() doesn't work on WinSock error codes; perror() is overridden
there to work on all errors and is used instead.

Ref #11258
Closes #11265
2023-06-09 15:05:19 -07:00
Daniel Stenberg
dacd25888f
curl_url_set: enforce the max string length check for all parts
Update the docs and test 1559 accordingly

Closes #11273
2023-06-08 23:40:08 +02:00
Daniel Stenberg
3c9256c8a0
urlapi: have *set(PATH) prepend a slash if one is missing
Previously the code would just do that for the path when extracting the
full URL, which made a subsequent curl_url_get() of the path to
(unexpectedly) still return it without the leading path.

Amend lib1560 to verify this. Clarify the curl_url_set() docs about it.

Bug: https://curl.se/mail/lib-2023-06/0015.html
Closes #11272
Reported-by: Pedro Henrique
2023-06-08 16:08:45 +02:00