Commit Graph

6324 Commits

Author SHA1 Message Date
Viktor Szakats
7a70b46481
tidy-up: misc
- lib1531: fix to use `curl_off_t`.
- src/tool_doswin: merge `_WIN32` guards.
- src/tool_doswin: limit `S_ISCHR()` to MS-DOS.
- src: dedupe `CURL_O_BINARY` macro into `tool_setup.h`.
- CMake/CurlTests.c: cleanup/formatting `HAVE_FILE_OFFSET_BITS` test.
- examples/httpput-postfields: delete unused include.
- spelling: MultiSSL.
- whitespace.

Ref: #15652
Closes #15799
2024-12-24 01:27:26 +01:00
Daniel Stenberg
9d5ecc9613
getinfo: provide info which auth was used for HTTP and proxy
CURLINFO_HTTPAUTH_USED and CURLINFO_PROXYAUTH_USED

Tested in 590 and 694

Ref: #12668
Idea-by: Ganesh Viswanathan
Closes #15450
2024-12-23 23:03:54 +01:00
Stefan Eissing
68bd759c2b
QUIC: 0RTT for gnutls via CURLSSLOPT_EARLYDATA
When a QUIC TLS session announced early data support and
'CURLSSLOPT_EARLYDATA' is set for the transfer, send initial request and
body (up to the 128k we buffer) as 0RTT when curl is built with
ngtcp2+gnutls.

QUIC 0RTT needs not only the TLS session but the QUIC transport
paramters as well. Store those and the earlydata max value together with
the session in the cache.

Add test case for h3 use of this. Enable quic early data in nghttpx for
testing.

Closes #15667
2024-12-23 17:07:15 +01:00
Daniel Stenberg
173805b2e7
GHA: drop codeql
We started using codeql for static code analysis in 7183f5acc3,
June 2020.

Since then, not a single commit has been merged into the source code
repository citing codeql as source or reason. Yet, it keeps getting
updated and we get constant reminders to upgrade the pinning it to the
latest hash.

During 4.5 years with intense development and significant code churn.
While Coverity, scan-build and CodeSonar have belped us point out many
mistakes, codeql has remained silent (or had false positives).

For this little gain, I think we spend a disproportionate amount of work
on codeql maintanance.

We can try again in a future if we think it improves.

Assisted-by: Viktor Szakats
Closes #15798
2024-12-23 08:51:21 +01:00
Daniel Stenberg
31c6b50cb2
curl: make --etag-save acknowledge --create-dirs
Add test 693 to verify

Fixes #15730
Suggested-by: Tamir Duberstein
Closes #15732
2024-12-21 14:12:16 +01:00
Viktor Szakats
250d613763
binmode: convert to macro and use it from tests
And use it from src and tests.

Syncing this functionality between platforms and build targets.

Also: Stop redefining `O_BINARY` in src, and use a local macro with
the same effect. `O_BINARY` is used in `CURL_SET_BINMODE()` to decide
if this functionality is supported, and redefining it makes this check
pass always in unity builds. The check is required for Apple OS, because
it offers a `setmode()` function, successfully detected by both CMake
and autotools, but that function has a different functionality and
signature than that expected by `CURL_SET_BINMODE()`.

Also:
- drop MetaWare High C (MS-DOS) support for set binmode.
- tests/libtest/Makefile.inc: dedupe comments.
- lib/curl_setup_once.h: tidy up feature guards for `io.h`, `fcntl.h`.

Ref: #15652
Closes #15787
2024-12-21 13:29:24 +01:00
Stefan Eissing
b06b35154f
getinfo: fix CURLINFO_QUEUE_TIME_T and add 'time_queue' var
Let CURLINFO_QUEUE_TIME_T count only the time a transfer spends queued,
including possible redirect requests.

Add var 'time_queue' for reporting the time in write outs.

Add test for verifying correct reporting.

Closes #15512
2024-12-21 12:04:48 +01:00
Daniel Stenberg
40c264db61
curl: add byte range support to --variable reading from file
Allowing --variable read a portion of provided files, makes curl work on
partial files for any options that accepts strings. Like --data and others.

The byte offset is provided within brackets, with a semicolon separator
like: --variable name@file;[100-200]"

Inspired by #14479
Assisted-by: Manuel Einfalt

Test 784 - 789. Documentation update provided.

Closes #15739
2024-12-21 11:46:27 +01:00
Daniel Stenberg
fc3e1cbc50
hyper: drop support
lib : remove all hyper code
configure: stop detecting hyper
docs: no more mention of hyper
tests: mo more special-handling of hyper builds
CI: no jobs using hyper

Closes #15120
2024-12-21 11:33:05 +01:00
Viktor Szakats
46093d9e0e
examples: make them compile with compatibility functions disabled (Windows)
For MinGW this is `-DNO_OLDNAMES`, with MSVC it is
`-D_CRT_DECLARE_NONSTDC_NAMES=0`.

There have been some support for this before this patch.
After this patch this is extended to all examples.

(And also the standalone http/client programs, if here.)

Cherry-picked from #15652
Closes #15789
2024-12-20 17:56:40 +01:00
Stefan Eissing
fa0ccd9f1f
lib: TLS session ticket caching reworked
Described in detail in internal doc TLS-SESSIONS.md

Main points:
- use a new `ssl_peer_key` for cache lookups by connection filters
- recognize differences between TLSv1.3 and other tickets
  * TLSv1.3 tickets are single-use, cache can hold several of them for a peer
  * TLSv1.2 are reused, keep only a single one per peer
- differentiate between ticket BLOB to store (that could be persisted) and object instances
- use put/take/return pattern for cache access
- remember TLS version, ALPN protocol, time received and lifetime of ticket
- auto-expire tickets after their lifetime

Closes #15774
2024-12-20 14:59:23 +01:00
Daniel Stenberg
9fce2c55d4
netrc: fix password-only entries
When a specific hostname matched, and only a password is set before
another machine is specified in the netrc file, the parser would not be
happy and stop there and return the password-only state. It instead
continued and did not return a match.

Add test 2005 to verify this case

Regression from e9b9bba, shipped in 8.11.1.

Reported-by: Ben Zanin
Fixes #15767
Closes #15768
2024-12-18 08:16:30 +01:00
Daniel Stenberg
f7e065f314
tool_formparse: accept digits in --form type= strings
Adjusted test 186 to verify.

Regression in 9664d5a547, shipped in 8.11.1

Reported-by: IcedCoffeee on github
Assisted-by: Jay Satiro
Fixes #15761
Closes #15762
2024-12-17 13:03:34 +01:00
Daniel Stenberg
54c5cb8b7f
checksrc: fix the return() checker
It would previously wrongly also catch function calls to function names
ending with 'return'

Amended test1185.

Reported-by: Stefan Eissing
Closes #15764
2024-12-17 11:27:55 +01:00
Viktor Szakats
a3585c9576
Makefile.mk: drop in favour of autotools and cmake (MS-DOS, AmigaOS3)
`Makefile.mk` supported MS-DOS and Amiga, but `./configure` also
supported them in a better tested and more flexible way.

This patch also adds CMake support for MS-DOS/DJGPP and Amiga OS 3.

`Makefile.mk` was not maintained. Delete it in favour of first-tier
build methods.

Also include some non-MS-DOS/AmigaOS-specific tidy-up, see details at
the end of this message.

Details:

- fix/silence all MS-DOS/DJGPP build warnings and issues.
- add MS-DOS support to cmake.
  - default to `ENABLE_THREADED_RESOLVER=OFF` for MS-DOS.
  - add support for `WATT_ROOT`.
  - use static libcurl with MS-DOS.
  - fixup default CMake suffixes/prefixes for DJGPP.
  - disable hidden symbols for MS-DOS. Not supported on MS-DOS.
  - opt-in MS-DOS into `USE_UNIX_SOCKETS`.
- improve MS-DOS support in autotools.
  - default to `--disable-threaded-resolver` for MS-DOS.
- make sure to use `close_s()` (from Watt-32) with autotools and cmake.
  `Makefile.mk` used it before this patch.
- GHA: add DJGPP cmake (~30s) and autotools (~60s) build jobs.
  Also build tests and examples with cmake.
- improve AmigaOS support in autotools:
  - configure: detect `CloseSocket()` when it's a macro.
  - configure: fix `IoctlSocket` detection on AmigaOS.
  - curl-amissl.m4: pass AmiSSL libs to tests/servers.
- add AmigaOS3 support to cmake:
  - cmake: fix `HAVE_IOCTLSOCKET_CAMEL` and
    `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` detections.
  - set necessary system libs.
  - add AmiSSL support.
  - inet_ntop, inet_pton: fix using it for AmigaOS. cmake detects them,
    and they did not compile with AmigaOS.
  - cmake: better sync `gethostname` detection with autotools.
    Fixes detection for AmigaOS, where `gethostname` is a macro.
  - cmake: fix `sys/utime.h` detection on AmigaOS.
  - cmake: force-disable `getaddrinfo` for AmigaOS.
  - cmake: tweak threading and static/shared default for AmigaOS.
  - cmake: rely on manual variable `AMIGA` to enable the platform.
- GHA: add AmigaOS cmake and autotools (~45s) jobs.
  Also build tests and examples with cmake.
- INSTALL: update MS-DOS and AmigaOS build instructions.
- amigaos: fix `-Wpointer-sign` and
  `zero or negative size array '_args'` in `Printf()`.
- amigaos: fix `-Wpointer-sign`
- amigaos: fix `-Wredundant-decls` `errno` and `h_errno`.
- amigaos: brute-force silence `lseek()` size warnings.
- amigaos: server/resolve: silence `-Wdiscarded-qualifiers`.
- amigaos: server/resolve: fix `-Wpointer-sign`.
- amigaos: fix `CURL_SA_FAMILY_T` type.
- nonblock: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
  `ioctl` is also detected, but fails when used. Make the above override
  it for a successful build.
  Authored-by: Darren Banfi
  Fixes #15537
  Closes #15603
- tftpd: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
- tftpd: tidy-up conditional code.
- curl: set stack size to 16384 for AmigaOS3/4
  Overriding the default 4096.
  Suggested-by: Darren Banfi
  Ref: https://github.com/curl/curl/pull/15543#issuecomment-2498783123
  Ref: https://wiki.amigaos.net/wiki/Controlling_Application_Stack
- functypes.h: fix `SEND_QUAL_ARG2` for AmigaOS.
- tftp: add missing cast in sendto() call for AmigaOS.
- getinfo: fix warning with AmigaOS.
- tool_operate: silence warning with AmigaOS
- amigaos: fix building libtests due to missing `RLIMIT_NOFILE`.
- curl_gethostname: silence warning for AmigaOS.
- ftp: silence `-Wtype-limits` for AmigaOS.
- libtest: fix timeval initialization for AmigaOS.
- examples: fix `timeval` initialization for AmigaOS.
- examples: silence warning for AmigaOS.
- configure: fix IPv6 detection for cross-builds.
- netrc: fix to build with AmigaOS cleanly.
- buildinfo: detect and add `DOS` tag for MS-DOS builds.
- buildinfo: add `AMIGA` to buildinfo.txt in auttools.
- build: move `USE_WATT32` macro definition to cmake/configure.

Non-MS-DOS/AmigeOS-specific tidy-ups:

- configure: sync `sa_family_t` detection with cmake.
- configure: sync `ADDRESS_FAMILY` detection signals with cmake.
- doh: use `CURL_SA_FAMILY_T`.
- lib: drop mingw-specific `CURL_SA_FAMILY_T` workaround.
- cmake: extend instead of override check-specific
  configurations/requirements.
  This allows to honor global requirements added earlier.
  Necessary for AmigaOS for example.
- cmake: omit warning on disabled IPv6 for MS-DOS and AmigaOS.
  No IPv6 support on these platforms. Also sync with autotools.
- lib1960: use libcurl `inet_pton()` wrapper.
- cmake: detect LibreSSL (to match autotools).
- cmake: say the specific OpenSSL flavour detected.
- hostip: add missing `HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID` guard.
- lib: simplify classic mac feature guards.

Follow-up to a8861b6ccd #9764

Closes #15543
2024-12-16 23:20:55 +01:00
Viktor Szakats
b9895b9d3f
cmake: namespace functions and macros
Prefix (or suffix) curl-defined macro and function names with `curl`.
To avoid collisions with non-curl CMake and to make them recognizable
as curl-specific calls.

Closes #15498
2024-12-16 21:55:00 +01:00
Viktor Szakats
b4aa93114c
cmake: replace unset(VAR) with set(VAR "") for init
Also add cleanup `unset()`s where missing.

Reported-by: Kai Pastor
Bug: https://github.com/curl/curl/pull/15255#issuecomment-2458659626
Follow-up to 8b09138083 #14610

Closes #15497
2024-12-16 21:12:51 +01:00
Daniel Stenberg
7347ddc9e2
tool_operate: make --etag-compare always accept a non-existing file
Consider it a blank etag. It allows for more use cases when the file
just might not have been created yet.

Closes #15737
2024-12-13 15:48:27 +01:00
Daniel Stenberg
a300879b63
curl: return error if etag options are used with multiple URLs
And document it.

Add tests 484 and 485

Fixes #15729
Reported-by: Tamir Duberstein
Closes #15731
2024-12-13 10:11:26 +01:00
Daniel Stenberg
c2ac9ea1ee
checksrc: ban use of sscanf()
Using sscanf() is not a (security) problem in itself, but we strongly
discorage using it for parsing input since it is hard to use right, easy
to mess up and often makes for sloppy error checking.

Allow it in examples and tests

Closes #15687
2024-12-13 09:43:05 +01:00
Daniel Stenberg
d5c738c608
strparse: string parsing helper functions
Designed to aid converting off from sscanf parsers. sscanf is hard to
use right, easy to mess up and often makes for sloppy error checking.

The new parsers allow more exact and pedandic parsing.

This new set of functions should be possible to use (and extend) and
switch over other libcurl parser code to use going forward.

Adapts the following to use the new functions:

- altsvc.c
- hsts.c
- http_aws_sigv4.c

Bonus: fewer memory copies, fewer stack buffers.

Test: Unit test1664

Docs: docs/internals/STRPARSE.md

Closes #15692
2024-12-12 16:00:52 +01:00
Randall S. Becker
aed732acb1
tests/http/clients: use proper sleep() call on NonStop
This fix impacts ws-data.c and ws-pingpong.c. sleep() replaced with
system PROCESS_DELAY_() having the same resolution as usleep().

Fixes #15711
Closes #15712

Signed-off-by: Randall S. Becker <randall.becker@nexbridge.ca>
2024-12-12 14:30:28 +01:00
Daniel Stenberg
a8c852b9a5
cookie: parse only the exact expire date
The date parser function is very forgiving and skips most "irrelevant"
characters in its hunt for a date to figure out. Therefore it is
important to make sure the date string is properly null terminated so
that it does not accidentally parse a piece of whatever text follows
after the date.

Add test483: test (overly) long expire dates in cookies

Closes #15709
2024-12-11 09:31:53 +01:00
Daniel Stenberg
75f0835513
lib517: extend the getdate test with quotes and leading "junk"
Closes #15708
2024-12-11 09:19:50 +01:00
Viktor Szakats
2d4852af83
build: fix tests when documentation/manual is disabled
- build: fix to exclude 'documentation' tests when building
  the documentation is explicitly disabled. Both for cmake
  and `./configure`.

  Reported-by: Daniel Engberg
  Fixes #15703

- test481, test482: fix for builds with no manual.

  Reported-by: Daniel Engberg
  Fixes #15703

- configure: fix to always detect Perl. Running tests require
  it when run. Before this patch Perl wasn't detected when
  documentation/manual/embedded-CA were all disabled,
  making tests fail to start.

- test1177: add keyword `documentation`. It depends no
  `curl_version_info.3`.

- GHA/linux: test cmake and `./configure` with docs and
  manual disabled and tests run.

Closes #15704
2024-12-09 12:45:29 +01:00
Marcel Raad
ebce0e7e4a
test2086: disable MSYS2's POSIX path conversion
Older MSYS2 versions treat the URL as paths list and convert them from
UNIX to Windows format. There's no path here that needs to be
converted, so disable path conversion for this test as done for others.

Fixes https://github.com/curl/curl/pull/15644#issuecomment-2511313206
Closes https://github.com/curl/curl/pull/15677
2024-12-08 23:03:23 +01:00
Stefan Eissing
ce949ba1dc
mime: fix reader stall on small read lengths
The base64 mime encoder stalls when it cannot encode a full 3 byte input
set into the read buffer. The workaround for this limitation was
incomplete and could lead to stalled transfers when the last chunk to
upload was smaller than 4 bytes.

Use a tmp buffer on small reads to allow mime encoders more space to put
their things.

Add test case reproducing the issue and fix.

Reported-by: Alexis Savin
Fixes #15688
Closes #15691
2024-12-05 15:44:51 +01:00
Daniel Stenberg
0169b80e5c
curl: --continue-at is mutually exclusive with --remove-on-error
Test 482 verifies

Fixes #15645
Reported-by: Harry Sintonen
Closes #15668
2024-12-02 21:22:57 +01:00
Daniel Stenberg
ffbcde0022
curl: --continue-at is mutually exclusive with --no-clobber
Test 481 verifies

Fixes #15645
Reported-by: Harry Sintonen
Closes #15668
2024-12-02 21:22:47 +01:00
Stefan Eissing
fd4528a8d8
OpenSSL: improvde error message on expired certificate
Fix regression that no longer printed the error messages about expired
certificates in openssl. Add test case for openssl/gnutls/wolfssl.

Fixes #15612
Reported-by: hiimmat on github
Closes #15613
2024-12-02 21:17:59 +01:00
Stefan Eissing
55968fd14b
pytest: add test for use of CURLMOPT_MAX_HOST_CONNECTIONS
Add test_02_33 to run with various values for the multi option
CURLMOPT_MAX_HOST_CONNECTIONS and CURLOPT_FRESH_CONNECT to trigger
connection pool limit handling code.

Closes #15494
2024-12-02 21:16:10 +01:00
Dan Fandrich
e9de867410 tests: add the ending time stamp in testcurl.pl
This provides better information on the length of the job and when it
completed.
2024-11-29 00:56:52 -08:00
Viktor Szakats
a72b479dec
build: fix MSVC UWP builds
The MSVC UWP job in CI did not actually enable UWP. Fix this and
the fallouts discovered after enabling it.

- GHA/windows: make sure to enable UWP in MSVC vcpkg UWP job.
  Use the CMake options and C flags already used for mingw-w64, but use
  `WINAPI_FAMILY_PC_APP` instead of the deprecated `WINAPI_FAMILY_APP`.
  (The former is not supported by mingw-w64, so leave it there as-is.)
  Follow-up to cb22cfca69 #14077

- GHA/windows: by default the MSVC UWP job became 2x-3x slower than
  others after actually enabling UWP. Most of it is caused by
  CMake/MSBuild automatically building full APPX containers for each
  `.exe` target. This includes 21 CMake feature detections. Each
  detection app is built into a 15MB APPX project, with code signing,
  logos, etc. Example:
    https://github.com/curl/curl/actions/runs/12056968170/job/33620610958
  Disable this overhead for curl build targets via custom
  `CMAKE_VS_GLOBALS` options. I've found no way to apply them to feature
  detection targets, so those remain slow.

- cmake: automatically enable Unicode for UWP builds. It's required.
  Also stop enabling it manually in the existing CI job.

- tests: fix `getpid()` use for Windows UWP:
  ```
  tests\server\util.c(281,21): warning C4013: 'getpid' undefined; assuming extern returning int
  ```
  Ref: https://github.com/curl/curl/actions/runs/12061215311/job/33632904249#step:11:38

- src/tool_doswin: disable `GetLoadedModulePaths()` for UWP.
  mingw-w64 UWP was okay with this, but MS SDK headers are not.
  This makes `--dump-module-paths` return empty for UWP builds.
  ```
  src\tool_doswin.c(620,3): error C2065: 'MODULEENTRY32': undeclared identifier
  src\tool_doswin.c(626,11): warning C4013: 'CreateToolhelp32Snapshot' undefined; assuming extern returning int
  src\tool_doswin.c(626,36): error C2065: 'TH32CS_SNAPMODULE': undeclared identifier
  src\tool_doswin.c(632,7): warning C4013: 'Module32First' undefined; assuming extern returning int
  ```
  Ref: https://github.com/curl/curl/actions/runs/12055081933/job/33614629930#step:9:35

- examples: fix `websocket.c` to include `winsock2.h` before `windows.h`
  to make it build with MSVC UWP:
  ```
  include\curl\curl.h(143,16): error C2061: syntax error: identifier 'curl_socket_t'
  include\curl\curl.h(143,16): error C2059: syntax error: ';'
  include\curl\curl.h(417,52): error C2146: syntax error: missing ')' before identifier 'curlfd'
  include\curl\curl.h(417,38): error C2081: 'curl_socket_t': name in formal parameter list illegal
  ```
  Ref: https://github.com/curl/curl/actions/runs/12055317910/job/33615644427#step:14:126

- GHA/windows: silence linker warning with MSVC UWP builds:
  ```
  LINK : warning LNK4075: ignoring '/INCREMENTAL' due to '/OPT:ICF' specification
  ```
  Ref: https://github.com/curl/curl/actions/runs/12055696808/job/33616629610#step:11:38

- GHA/windows: set `/INCREMENTAL:NO` for all MSVC jobs to improve
  performance a little.

- cmake: show `UWP` platform flag.

Ref: #15652
Closes #15657
2024-11-28 14:24:03 +01:00
Viktor Szakats
96f7547d9a
tests: re-enable 2086, and 472, 1299, 1613 for Windows
- GHA/windows: un-ignore tests 472 1299 1613.
  They were ignored for the mingw-w64 c-ares U job.
  They do run fine now:
  https://github.com/curl/curl/actions/runs/12032875421/job/33547724780?pr=15644

- globally re-enable test 2086.
  Comment says it only affected Windows.
  Seems to be running fine now. Example:
  Windows:
  https://github.com/curl/curl/actions/runs/12032875421/job/33547718309?pr=15644#step:13:3856
  Linux:
  https://github.com/curl/curl/actions/runs/12032875397/job/33545739712#step:41:3650

- update comments for disabled tests 1184, 1801.
  They affect all operating systems, likely all CIs.

  FAIL 1801: 'HTTP/2 upgrade with lying server' HTTP, HTTP/2
  https://github.com/curl/curl/actions/runs/12032362497/job/33544053001#step:14:4265
  FAIL 1184: 'HTTP 1.1 CONNECT with redirect and set -H user-agent' HTTP, HTTP CONNECT, HTTP proxy, proxytunnel
  https://github.com/curl/curl/actions/runs/12032362497/job/33544051415#step:14:5252

Closes #15644
2024-11-27 11:19:17 +01:00
Daniel Stenberg
e9b9bbac22
netrc: address several netrc parser flaws
- make sure that a match that returns a username also returns a
  password, that should be blank if no password is found

- fix handling of multiple logins for same host where the password/login
  order might be reversed.

- reject credentials provided in the .netrc if they contain ASCII control
  codes - if the used protocol does not support such (like HTTP and WS do)

Reported-by: Harry Sintonen

Add test 478, 479 and 480 to verify. Updated unit 1304.

Closes #15586
2024-11-17 11:33:56 +01:00
Daniel Stenberg
bc34ebc87b
curl.h: mark two error codes as obsolete
Not used by libcurl since many years

Closes #15538
2024-11-16 23:39:04 +01:00
Viktor Szakats
e244d50064
configure: do not echo most inherited LDFLAGS to config files
`libcurl.pc` `Libs.private` (since 8.11.0, and in `Libs` before 7.20.0)
and `curl-config` `--static-libs` (since 7.17.1, and in `Libs` between
7.7.2-7.25.0). This included all flags inherited from the environment,
in addition to those coming from dependency detections.

To avoid spilling all linker flags inherited from the environment to
the libcurl config files, this patch omits them all, except `-L`, `-F`,
`--library-path=` and `-framework` options, which are still passed.
The rationale for the exceptions is that `LIBS` is passed as-is, and
`LDFLAGS`, `LIBS` are the canonical way to pass custom libs options
to a build. `LIBS` may not work without a matching custom libpath.

This brings autotools behaviour closer to cmake, and `curl-config`
closer to `libcurl.pc`.

Follow-up to 9f56bb608e #14681
Follow-up to 4c8adc8fee
Reported-by: Peter Marko
Fixes #15533
Closes #15550
2024-11-14 09:55:45 +01:00
Dan Fandrich
e3aa2a07f9 tests: use the standard format of an IGNORED line
This is more consistent and allows Test Clutch to parse it properly.
Also, add --test-duphandle and --test-event to the runtests.pl help
text.
2024-11-12 22:23:57 -08:00
Daniel Stenberg
cbafcec50b
curl: --test-duphandle in debug builds runs "duphandled"
Using this option (only available in debug builds) makes curl always
call curl_easy_duphandle() on the handle before using it.

To help us catch curl_easy_duphandle() mistakes better.

Add a CI job using this.

Bonus: the previous runtests option -e is now also supported as
--test-event

Closes #15504
2024-11-08 13:22:47 +01:00
Daniel Stenberg
f5c616930b
duphandle: also init netrc
The netrc init was only done in the Curl_open, meaning that a duplicated
handle would not get inited properly.

Added test 2309 to verify. It does netrc auth with a duplicated handle.

Regression from 3b43a05e00

Reported-by: tranzystorekk on github
Fixes #15496
Closes #15503
2024-11-07 09:47:53 +01:00
Daniel Stenberg
9919149aef
cookie: treat cookie name case sensitively
Extend test 31 to verify

Reported-by: delogicsreal on github
Fixes #15492
Closes #15493
2024-11-06 13:09:20 +01:00
Stefan Eissing
3a35901a11
wolfssl: coexist with openssl, further work
Build wolfSSL master with

./configure --prefix=/path --enable-ip-alt-name --enable-quic
--enable-earlydata --enable-psk --enable-opensslcoexist

and configure curl with openssl + wolfssl. Normal tests run.

pytest session resumption fails, as wolfssl does not handle the
new_session callback without opensslextra right now.

Closes #15481
2024-11-04 14:48:30 +01:00
Viktor Szakats
25025419c9
pytest: include curl version string and python platform in log
For the Test Clutch matrix.

https://testclutch.curl.se/static/reports/feature-matrix.html

Assisted-by: Dan Fandrich
Closes #15470
2024-11-02 01:56:55 +01:00
Viktor Szakats
9d32724c12
certs: add missing -CAcreateserial option for LibreSSL
Also:
- display openssl path and version.
- quote a string.

Follow-up to 9b0c0d6ade #15129

Closes #15471
2024-11-01 15:36:48 +01:00
Dan Fandrich
d88db0b6b2 tests: use a set for several of the curl_props
These are used for member-of testing, so a set is more appropriate and
efficient. Also, fix a couple cases of bad exceptions.

Closes #15462
2024-10-31 20:14:44 -07:00
Viktor Szakats
329a8e9daa
unit1307: tidy up Apple OS detection
Use `__APPLE__` macro to detect Apple OS instead of relying on
the string in `CURL_OS`.

This also fixes detection with default CMake builds where `CURL_OS` is
`Darwin`. The code before this patch was expecting this substring in
lowercase.

Closes #15461
2024-10-31 09:09:01 +01:00
Viktor Szakats
fb711b5098
build: fix clang-cl builds, add CI job
- appveyor: add build-only job for clang-cl.

- cmake: `-pedantic-errors` enables `-Werror,-Wlanguage-extension-token`
  automatically, which makes `__int64` detection fail.
  Explictly disable this compiler warning for clang-cl to make the
  feature detection work and to accept `__int64` in the source code.

- cmake: disable `-Wlanguage-extension-token` warning for clang-cl
  to fix these when encountering `__int64`:
  ```
  lib/formdata.c(797,29): error : extension used [-Werror,-Wlanguage-extension-token]
  lib/warnless.c(117,33): error : extension used [-Werror,-Wlanguage-extension-token]
  lib/warnless.c(60,28): message : expanded from macro 'CURL_MASK_SCOFFT'
  lib/warnless.c(59,38): message : expanded from macro 'CURL_MASK_UCOFFT'
  include\curl/system.h(352,40): message : expanded from macro 'CURL_TYPEOF_CURL_OFF_T'
  ```

- make `__GNUC__` warning suppressions apply to `__clang__` too.
  Necessary for clang-cl, which defines the latter, but not the former.
  (Regular clang defines both.)

- examples: fix clang-cl compiler warning in `http2-upload.c`.
  ```
  docs\examples\http2-upload.c(56,5): error : no previous prototype for function 'my_gettimeofday' [-Werror,-Wmissing-prototypes]
  docs\examples\http2-upload.c(56,1): message : declare 'static' if the function is not intended to be used outside of this translation unit
  ```

- unit2604: add missing `#pragma GCC diagnostic pop`.
  Follow-up to e53523fef0 #14859

- unit1652: limit compiler warning suppression to GCC.
  They do not affect clang builds.
  Follow-up to 71cf0d1fca #14772

Closes #15449
2024-10-30 23:15:32 +01:00
Daniel Stenberg
0cececef0f
config: rename the OS define to CURL_OS to reduce collision risk
Reported-by: Jon Rumsey
Fixes #15455
Closes #15457
2024-10-30 14:59:54 +01:00
Viktor Szakats
bc6212004a
pytest: show curl features and protocols
For the Test Clutch matrix.

https://testclutch.curl.se/static/reports/feature-matrix.html

Closes #15452
2024-10-30 09:32:29 +01:00
Stefan Eissing
0d475da1c4
pytest: fix run against multissl curl
Changes to make a curl built with OpenSSL + GnuTLS to run successfully
in our pytests. Run

CURL_SSL_BACKEND=openssl pytest

to test a TLS backend other than the default.

Closes #15443
2024-10-29 11:13:56 +01:00
Viktor Szakats
74c7b672d9
runtests: add comment for handle64 pathsep requirement [ci skip]
Cherry-picked from #14949
2024-10-29 09:27:42 +01:00
Viktor Szakats
ef7399b8b5
runtests: pass single backslashes with Windows Perl
handle/handle64 requires a literal match with the filenames it's
listing.

Also:
- make handle64 log messages more unique to help text searches.
- update a comment with Windows Perl info.

Cherry-picked from #14949
Closes #15436
2024-10-29 09:20:37 +01:00
Stefan Eissing
701813b23f
tests/http: add --insecure tests
Add two test cases that connection using a hostname the server has no
certificate for. First, verify that the peer verification fail, as
expected. Second, provide '--insecure' to test that the connection
succeeded and returned some data.

Closes #15429
2024-10-28 13:04:28 +01:00
Stefan Eissing
0e0c8cdf89
tests/scorecard: allow remote server test
New args for scorecard.py:
* --remote ip:port to run tests against a remote server
* --start-only: to only start the servers, listing their ports

Start the server using "scorecard.py --start-only" on one machine and
then run the tests with "scorecard.py --remote ip:port" against that
machine.

Closes #15415
2024-10-28 11:41:19 +01:00
Viktor Szakats
f66af623cf
cmake: document -D and env build options
Extend `INSTALL-CMAKE` document with the list of available options,
a short description and default values.

The list may not be 100% complete.

There are no component boundaries in CMake, so the line is blurry
between curl options, CMake options, CMake Find modules options.
I included certain CMake options that seemed useful, and/or have
dedicated use withing curl's CMake source. But, all CMake built-in
options are usable, as documented upstream in CMake.

The naming of the options has a heritage and the inconsistencies with
it, including a lack of clear namespace. This may be subject to future
updates, also after figuring out which name has special meaning within
CMake and/or CMake projects out of unwritten convention or something
more tangible.

CMake allows to initialize any internal variable via `-D`. This may be
useful to pre-initialize/override feature check results. The list
doesn't contain these, and they remain officially undocumented.

Also:
- make adjustments to keep the spellchecker happy.
- retrofit description changes to the cmake sources.
- stop documenting deprecated `Find*` variables.

Reported-by: Daniel Stenberg
Fixes https://github.com/curl/curl/discussions/14885
Closes #15388
2024-10-24 23:06:40 +02:00
Stefan Eissing
8cb2d5f48a
quic: use the session cache with wolfSSL as well
Use session cache for QUIC when built with quictls or wolfSSL.

Add test_017_10 for verifying QUIC TLS session reuse when built with
quictls, gnutls or wolfssl.

Closes #15358
2024-10-24 14:55:23 +02:00
Stefan Eissing
358eae42a4
bearssl: improved session handling, test exceptions
Add length to session saves, making it clear that we are storing a byte
blob and allowing memcmp() on sameness check.

Remove some pytest skips for bearssl to see if they now work properly in
CI.

Closes #15395
2024-10-24 14:40:01 +02:00
Stefan Eissing
30f66c8ba4
mbedtls: handle session as blobs
Use mbedtls_ssl_session_load() and mbedtls_ssl_session_save() to convert
TLS sessions to byte blobs for the session cache.

Fix a skip message to better indicate why the test is skipped for
mbedtls.

Closes #15398
2024-10-24 14:37:59 +02:00
ArtSin
0325e1b9b2
mprintf: treat %o as unsigned, add tests for %o, %x, %X
`%x` and `%X` were already treated as unsigned, but `%o` was not, even
though it was used with unsigned numbers.

Closes #15348
2024-10-24 10:17:24 +02:00
Viktor Szakats
53fdc5faf3
runtests: use deterministic sort for TESTINFO lines
Sort TESTINFO lines by description within the number of skipped test.
It makes the list of skipped test groups easier to diff/compare between
jobs and runs.

Closes #15374
2024-10-23 10:42:30 +02:00
Daniel Stenberg
0e18bd3940
mk-lib1521: fix the long return code check
It worked mostly by accident since it checked the variable from the
previous setopt invoke.

Closes #15372
2024-10-23 00:41:05 +02:00
Yedaya Katsman
acd134cfe1 tests: Fix FILEFORMAT <file name=""> directive
Follow-up to dcc52095

Closes #15371
2024-10-22 13:41:02 -07:00
Viktor Szakats
b6219cd932
GHA/linux: merge torture jobs into the main workflow
They complete in 4 and 7 minutes, and do not hold back the main Linux
workflow.

Also:
- bump default parallelism for `test-torture` target to `-j20`
  (was: `-j2`).
- drop redundant package install from `rustls` jobs.

Closes #15360
2024-10-22 20:20:42 +02:00
Viktor Szakats
9126eb5a8a
cmake: replace CURL_*_DIR with {PROJECT,CMAKE_CURRENT}_*_DIR
It reduces the number of synonym variables in the code.
Makes it easier to grok and grep.

- replace `CURL_SOURCE_DIR`
  with `PROJECT_SOURCE_DIR`.

- replace `CURL_BINARY_DIR`
  with `PROJECT_BINARY_DIR` or `CMAKE_CURRENT_BINARY_DIR`.

- replace a single use of `CMAKE_BINARY_DIR`
  with `PROJECT_BINARY_DIR`.

- replace `CMAKE_CURRENT_*_DIR`
  with `PROJECT_*_DIR` where it makes the code more uniform.

- quote an argument (formatting).

Closes #15331
2024-10-22 19:13:08 +02:00
Stefan Eissing
943df95ae7
CI: run with standard mod_http2
We used to include a special mod_h2 in our CI that supports the
directive H2MaxDataFrameLen for test_02_20. Since then, ubuntu-lastest
includes a more recent apache httpd. Let's see if we can live without
the special.

Closes #15353
2024-10-22 15:31:46 +02:00
Daniel Stenberg
5ea61a0b54
hsts: support "implied LWS" properly around max-age
Adjust test 780 to verify.

Reported-by: newfunction
Closes #15330
2024-10-18 11:42:42 +02:00
Stefan Eissing
fe2a72029e
http2: auto reset stream on server eos
When a server signals EOS from its side and the curl upload is
unfinished and the server has not given a positive HTTP status response,
auto RST the stream to signal that the upload is incomplete and that the
whole transfer can be stopped.

Fixes the case where the server responds with 413 on an upload but does
not RST the stream from its side, as httpd and others do.

Reported-by: jkamp-aws on github
Fixes #15316
Closes #15325
2024-10-17 23:03:41 +02:00
Daniel Stenberg
2ae8d9b579
libtests: generate the lib1521 atomically
By renaming from a temporary file name to the .c once completed. This
avoids the risk that the checksrc job tries to verify the file before it
is complete, in parallel build setups.

Reported-by: Dan Frandrich
Fixes #15258
Closes #15327
2024-10-17 23:00:24 +02:00
Stefan Eissing
b42eb27c1f
openssl: improve retries on shutdown
Once SSL_shutdown() has been called, OpenSSL does not really seem to
like it when it is called again and the other side has some finally data
to deliver.

Instead SSL_read() needs to be used solely, once the close notify has
been sent from curl's side.

Closes #15321
2024-10-17 16:11:06 +02:00
Daniel Stenberg
193f1b4841
DISABLED: disable test 1060 with hyper
... as it has started to fail and nobody wants to debug this.

Closes #15319
2024-10-17 13:26:00 +02:00
Stefan Eissing
c97cd82821
tests/http: fix ubuntu GnuTLS CI failures
Override the system default config in test_17_09, since we want to check
all TLS versions. Provide own, empty config file to gnutls, so that any
system wide file has no effect.

The latest ubunu image in GH CI disables TLS 1.0 and 1.1
system wide for GnuTLS. Good intentions.

Closes #15310
2024-10-17 13:25:15 +02:00
Daniel Stenberg
beeeb85a78
tests: update some HTTP/2 over HTTPS tests
- improve descriptions
- require http/2, not h2c, since they are done over HTTPS

Closes #15317
2024-10-17 12:55:05 +02:00
Dan Fandrich
41c980bb0d tests: capture stdin to get the vsftpd version number
vsftpd 3.0 at least writes its version number to stdin (!) instead of
stderr. This works due for backwards compatibility reasons in UNIX, so
we must check stdin for anything written there to reliably parse the
version string.

Closes #15278
2024-10-16 09:21:40 -07:00
Viktor Szakats
3040971d12
GHA: silence proselint warnings and an error
Fix new issues found by `proselint`.

Also:
- silence this technical warning:
  ```
  :0: DeprecationWarning: /home/runner/.proselintrc was found instead of a JSON file. Rename to /home/runner/.proselintrc.json.
  ```
- fix an input filename.
  `proselints` fails now if an input file is missing.

Reported-by: Jay Satiro
Bug: https://github.com/curl/curl/pull/15291#issuecomment-2410505100
Closes #15293
2024-10-15 16:44:17 +02:00
Daniel Stenberg
8403e5a701
tests: fix callback signatures to please UndefinedBehaviorSanitizer
Make test applications use the correct prototypes for callbacks.

Closes #15289
2024-10-15 14:33:47 +02:00
Daniel Stenberg
ad1c49bc0e
lib: remove function pointer typecasts for hmac/sha256/md5
Make sure we use functions with the correct prototype.

Closes #15289
2024-10-15 14:32:39 +02:00
Daniel Stenberg
e33cf006e9
server/mqttd: fix two memory leaks
Closes #15290
2024-10-14 12:47:21 +02:00
Daniel Stenberg
9bee39bfed
url: use same credentials on redirect
Previously it could lose the username and only use the password.

Added test 998 and 999 to verify.

Reported-by: Tobias Bora
Fixes #15262
Closes #15282
2024-10-13 23:18:42 +02:00
Daniel Stenberg
eb77297ccc
lib: move curl_path.[ch] into vssh/
As this contains code only used by SSH backends.

Closes #15284
2024-10-13 23:16:18 +02:00
Stefan Eissing
3455d360ce
mbedTLS: fix handling of TLSv1.3 sessions
For TLSv1.3, if supported, observer special return code to retrieve
newly arrived session from mbedTLS.

Adjust test expectations now that TLSv1.3 session resumption works in
mbedTLS >= 3.6.0.

Based on #14135 by @ad-chaos
Closes #15245
2024-10-13 23:13:04 +02:00
Stefan Eissing
513904c264
wolfSSL: fix handling of TLSv1.3 sessions
Register a callback to get notified of new SSL sessions by wolfSSL.
Remove the explicit session retrieval after handshake, since this does
not work for TLSv1.3.

Adjust test expectations now that TLSv1.3 session resumption works
in wolfSSL.

Closes #15243
2024-10-13 23:11:39 +02:00
Viktor Szakats
75dfb7b649
pytest: include buildinfo.txt in the output
If present.

It aims to provide TextClutch the same build information that
`runtests.pl` already is providing.

Ref: https://testclutch.curl.se/static/reports/feature-matrix.html
Ref: #15256
Follow-up to 1fdea16846 #14802
Closes #15279
2024-10-13 11:34:26 +02:00
Dan Fandrich
a3601cf571 tests: allow pytests to run in out-of-tree builds
Some of the files it needs are in the build directory but the code did
not make a distinction.

Closes #15257
2024-10-12 10:47:57 -07:00
Stefan Eissing
fe8399f066
gnutls: use session cache for QUIC
Add session reuse for QUIC transfers using GnuTLS. This does not include
support for TLS early data, yet.

Fix check of early data support in common GnuTLS init code to not access
the filter context, as the struct varies between TCP and QUIC
connections.

Closes #15265
2024-10-11 23:37:47 +02:00
Dan Fandrich
954177b9da tool_xattr: create the user.creator xattr attribute
This indicates that the file was created by curl which can help a user
determine the origin of a file. Like the other attributes, this is only
enabled with the --xattr option.

Closes #15237
2024-10-11 10:13:37 -07:00
Daniel Stenberg
3b43a05e00
netrc: cache the netrc file in memory
So that on redirects etc it does not reread the file but just parses it
again.

Reported-by: Pierre-Etienne Meunier
Fixes #15248
Closes #15259
2024-10-11 14:40:12 +02:00
Stefan Eissing
962097b8dd
TLS: TLSv1.3 earlydata support for curl
Based on #14135, implement TLSv1.3 earlydata support for the curl
command line, libcurl and its implementation in GnuTLS.

If a known TLS session announces early data support, and the feature is
enabled *and* it is not a "connect-only" transfer, delay the TLS
handshake until the first request is being sent.

- Add --tls-earldata as new boolean command line option for curl.
- Add CURLSSLOPT_EARLYDATA to libcurl to enable use of the feature.
- Add CURLINFO_EARLYDATA_SENT_T to libcurl, reporting the amount of
  bytes sent and accepted/rejected by the server.

Implementation details:
- store the ALPN protocol selected at the SSL session.
- When reusing the session and enabling earlydata, use exactly
  that ALPN protocol for negoptiation with the server. When the
  sessions ALPN does not match the connections ALPN, earlydata
  will not be enabled.
- Check that the server selected the correct ALPN protocol for
  an earlydata connect. If the server does not confirm or reports
  something different, the connect fails.
- HTTP/2: delay sending the initial SETTINGS frames during connect,
  if not connect-only.

Verification:
- add test_02_32 to verify earlydata GET with nghttpx.
- add test_07_70 to verify earlydata PUT with nghttpx.
- add support in 'hx-download', 'hx-upload' clients for the feature

Assisted-by: ad-chaos on github
Closes #15211
2024-10-11 12:28:22 +02:00
Daniel Stenberg
45b388fdc7
tests/server/util.c: remove use of strncpy
... and ban the function in code in this directory.

Closes #15213
2024-10-10 07:47:01 +02:00
Daniel Stenberg
5ee43bb82e
tests: 780 - 783, new HSTS tests
test780: verify updated HSTS data in response header

test781: HSTS update expiry, with parent includeSubDomains domain
present

test782: HSTS update expiry, with two includeSubDomains domains present

test783: HSTS update expiry, removing includesubdomains in update
2024-10-09 13:48:14 +02:00
Daniel Stenberg
a94973805d
hsts: improve subdomain handling
- on load, only replace existing HSTS entries if there is a full host
  match

- on matching, prefer a full host match and secondary the longest tail
  subdomain match

Closes #15210
2024-10-09 13:48:08 +02:00
Daniel Stenberg
80dac51af8
test1915: remove wrong comment 2024-10-08 09:35:56 +02:00
Viktor Szakats
6cfb615e92
sws: fix unused static function with TCP_NODELAY undefined
Closes #15171
2024-10-07 12:31:21 +02:00
Daniel Stenberg
e9eda865df
warnless: remove curlx_sktosi and curlx_sitosk
They were only used by test server code. Use plain old typecasts there.

Closes #15153
2024-10-05 00:28:06 +02:00
Dan Fandrich
57cc523378 tests: enable additional ruff Python lint options
These all seem reasonable to enable for this code.
2024-10-04 14:03:17 -07:00
Dan Fandrich
0f7ba5c5bf tests: change Python code style to pass ruff checks
Most of the changes consisted of removing unused imports and unnecessary
f-strings.
2024-10-04 14:02:05 -07:00
Dan Fandrich
2f3b7f20fb tests: fix some Python typing issues
These otherwise raise errors in pytype. A few problematic methods
weren't being used and are deleted.
2024-10-04 13:57:04 -07:00
Stefan Eissing
bc055d08ac
test1915: add tracing and connect timeout
Since we see Windows fails of 1915, add tracing and a connect timeout.
The test uses a port no one is supposed to listen on, but Windows has
this weird wait logic. So, set a short timeout.

Closes #15107
2024-10-03 16:06:08 +02:00
Daniel Stenberg
566a6d7b09
urlapi: normalize the IPv6 address
As the parsing and address "regeneration" are done anyway, we might as
well use the updated version in the result and thereby A) get a
normalized (and lower cased) version of the address and B) avoid a
strcpy().

Updated test 1560 to verify.

Closes #15143
2024-10-03 16:05:03 +02:00
Daniel Stenberg
87e19ea68f
tests/valgrind.supp: remove a travis suppression, add a Debian
We have not used Travis for years. The Debian one appears on my dev
machine since a while back.

Closes #15142
2024-10-03 16:03:43 +02:00
Stefan Eissing
6c1b15768c
pytest: improve pytest_07_42a reliability
Due to timings on paused response receive, the error code may vary due
to the location where it is detected that the server closed the transfer
prematurely.

Be more lenient in always allowing PARTIAL_FILE as ok.

Closes #15138
2024-10-03 14:42:52 +02:00