- Change the swsbounce keyword to override the part number on a
subsequent request to the previous part number + 1.
Note the previous part number in this case is the part number that
was returned as a response to the previous request and contained
the swsbounce keyword.
Prior to this change swsbounce incremented the part number of the
subsequent request instead of overriding it, and did so in a more
limited fashion that prevented chaining swsbounce in multiple responses.
For example, if the test makes a request that causes the sws server to
return `<data>` as a response and that response contains `swsbounce`
then for the next response the sws server returns `<data1>`. If
`<data1>` also contains `swsbounce` then for the next response the sws
server now returns `<data2>` instead of the requested part.
Fixes https://github.com/curl/curl/discussions/16074
Closes https://github.com/curl/curl/pull/16085
Allow building with c-ares and yet use threaded resolver for the main
host A/AAAA resolving:
`--with-ares` provides the c-ares install path and defaults to use
c-ares for name resolving
`--with-threaded-resolver` still uses c-ares in the build (for HTTPS)
but uses the threaded resolver for "normal" resolves.
It works similarly for cmake: ENABLE_ARES enables ares, and if
ENABLE_THREADED_RESOLVER also is set, c-ares is used for HTTPS RR and
the threaded resolver for "normal" resolves.
HTTPSRR and c-ares-rr are new features return by curl_version_info() and
thus shown by curl -V.
The c-ares-rr feature bit is there to make it possible to distinguish
between builds using c-ares for all name resolves and builds that use
the threaded resolves for the regular name resolves and c-ares for
HTTPSRR only. "c-ares-rr" means it does not use c-ares for "plain" name
resolves.
HTTPSRR support is EXPERIMENTAL only.
Closes#16054
The variable `conn->httpversion` was used for several purposes and it
was unclear at which time the value represents what.
- rename `conn->httpversion` to `conn->httpversion_seen`
This makes clear that the variable only records the last
HTTP version seen on the connection - if any. And that it
no longer is an indication of what version to use.
- Change Alt-Svc handling to no longer modify `conn->httpversion`
but set `data->state.httpwant` for influencing the HTTP version
to use on a transfer.
- Add `data->req.httpversion_sent` to have a record of what
HTTP version was sent in a request
- Add connection filter type CF_TYPE_HTTP
- Add filter query `CF_QUERY_HTTP_VERSION` to ask what HTTP
filter version is in place
- Lookup filters HTTP version instead of using `conn->httpversion`
Test test_12_05 now switches to HTTP/1.1 correctly and the
expectations have been fixed.
Removed the connection fitler "is_httpN()" checks and using
the version query instead.
Closes#16073
- silence false positive picky warnings.
- avoid "possible noreturn" warnings for standalone tests and examples.
- fix to compile without `#pragma GCC diagnostic push` support.
- fix "#pragma GCC diagnostic not allowed inside functions".
Prerequisite for #15975 that needs GCC 4.4 for the latest pre-built
CeGCC/mingw32ce toolchain for Windows CE.
Cherry-picked from #15975Closes#16062
- setup-win32: drop duplicate `WIN32_LEAN_AND_MEAN` and `NOGDI` macros
(set earlier in `curl_setup.h`)
- config-win32ce: detect ARM for `CURL_OS` macro.
Based on: https://cegcc.sourceforge.net/docs/details.html
It should also apply to MSVC. (untested)
- indentation.
Closes#16029
With tests. (Most SFTP tests fail with wolfSSH.)
Also:
- disable pseudo-terminal in wolfSSH builds (also in CircleCI).
Not needed for curl.
- test642: add "compressed" to the description.
To avoid being the same as test600.
Follow-up to b7b4dc0d49#1735Closes#16025
- Fix a bug in EAGAIN handling when sending frames that led to a
corrupted last byte of the frame sent.
- Restore sanity to curl_ws_send() behaviour:
- Partial writes are reported as OK with the actual number of
payload bytes sent.
- CURLE_AGAIN is only returned when none of the payload bytes
(or for 0-length frames, not all of the frame header bytes)
could be sent.
- curl_ws_send() now behaves like a common send() call.
- Change 'ws-data' test client to allow concurrent send/recv
operations and vary frame sizes and repeat count.
- Add DEBUG env var CURL_WS_CHUNK_EAGAIN to simulate blocking
after a chunk of an encoded websocket frame has been sent.
- Add tests.
Prior to this change data corruption may occur when sending websocket
messages due to two bugs:
1) 3e64569a (precedes 8.10.0) caused a data corruption bug in the last
byte of frame of large messages.
2) curl_ws_send had non-traditional send behavior and could return
CURLE_AGAIN with bytes sent and expect the caller to adjust buffer
and buflen in a subsequent call. That behavior was not documented.
Reported-by: na-trium-144@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/15865
Fixes https://github.com/curl/curl/issues/15865#issuecomment-2569870144
Closes https://github.com/curl/curl/pull/15901
When enabled in the build.
Update test2100: verify with HTTPS RR included
Adjust runtests and server/disabled.c to include "HTTPSRR" as a feature
in the test suite.
Also, decode the ALPN list in HTTPS records straight into IDs. There's
no point in storing everything in string format. Skip ALPNs we do not
support.
Closes#16007
This typically happens if CURL_CONNECT_ONLY is used and a second
curl_easy_perform() is attempted.
A connection "taken over" with CURL_CONNECT_ONLY cannot be ended any
other way than a curl_easy_cleanup() on the easy handle that holds it.
Add test 696 to verify.
Closes#16003
Since the script 'apachectl' from the httpd project is severly mutilated
on several distros, use the executable httpd/apache2 directly in pytest
runs.
Remove detection of apachectl form autoconf and cmake.
Closes#16000
- Change setopt and pretransfer to always reset URL related variables
for a CURLU handle set CURLOPT_CURLU.
This change is to ensure we are in compliance with the doc which says
CURLU handles must be able to override a URL set via CURLOPT_URL and
that if the contents of the CURLU handle changes between transfers then
the updated contents must be used.
Prior to this change, although subsequent transfers appear to be
performed correctly in those cases, the work URL `data->state.url` was
not updated. CURLINFO_EFFECTIVE_URL returns data->state.url to the user
so it would return the URL from the initial transfer which was the wrong
URL. It's likely there are other cases as well.
Ref: https://curl.se/libcurl/c/CURLOPT_CURLU.html
Reported-by: Nicolás San Martín
Fixes https://github.com/curl/curl/issues/15984
Closes https://github.com/curl/curl/pull/15985
In the double output function when an extremely large width and
precision is set that reaches the libcurl maximum (325), the handling of
the precision part would do wrong which could lead to bad output.
Also: work-around for single-byte buffer snprintf overflow with mingw.
Extend test 557 to verify.
Coverity CID 1638751.
Closes#15988
Dedupe and migrate MSVC-specific warning suppressions to `curl_setup.h`.
Make cmake set `_CRT_SECURE_NO_DEPRECATE` for examples and standalone
tests, and stop setting `_CRT_NONSTDC_NO_DEPRECATE` for them.
Details:
- drop version guards. On ancient MSVC version these macro are a no-op.
- move to `curl_setup.h` from `config-win32*.h`.
- sync macro values with CMake.
- cmake: stop setting them globally in favour of `curl_setup.h`.
- cmake: re-add these macros to `docs/examples` and `tests/http/clients`,
which do not use `curl_setup.h`.
- cmake: drop `_CRT_NONSTDC_NO_DEPRECATE` for examples and tests.
They build fine without.
- update comments.
Closes#15960
The pending cookie RFC update (currently known as 6265bis draft-19) says
Let cookie-age-limit be the maximum age of the cookie (which name of
Max-Age and an attribute-value of expiry-time. SHOULD be 400 days or
less.
This change makes received cookies over the wire get capped to 400 days.
It does not cap the expiry date of cookies loaded from file.
It does this by rounding the expire time to a even minute. This, to
allow the test suite to do the same and have a chance to get the same
number for stable testing without requiring a debug build.
The test script generates TWO numbers in the output file for each
%days[] used in the input test file, and the function that subsequently
compares and verifies output is fine with *either* of the two numbers.
This is done so that if the test case is generated the second
immediately before curl runs, that updated expiry number is also deemed
okay. It still checks for an exact match of either number.
Closes#15937
The TE request header field is invalid in HTTP/2. Since clients may not
know in advance if a connection negotiates HTTP/2, automatically strip
such a header when h2 is in play.
Add test_01_10 to verify.
Reported-by: Jiri Stary
Fixes#15941Closes#15943
Adds the experimental feature `ssls-export` to libcurl and curl for
importing and exporting SSL sessions from/to a file.
* add functions to libcurl API
* add command line option `--ssl-sessions <filename>` to curl
* add documenation
* add support in configure
* add support in cmake
+ add pytest case
Closes#15924
Count connections to a host against a possibly configured destination
limit. Trigger multi `connchange` when a connection has been shutdown,
so pending transfers can try to get a connection once again.
Reported-by: baranyaib90 on github
Fixes#15857Closes#15879
Subparts may have been previously used as a top-level mime structure and
thus not rewound.
New test 695 checks the proper functioning in these particular conditions.
Reported-by: Qriist on github
Fixes#15842Closes#15911
- Treat negative Retry-After date-based times as 0.
- Treat Retry-After times greater than 6 hours as 6 hours.
Prior to this change Retry-After did not have a limited range and the
server could have set a time greater than 6 hours or a date in the past
that would result in a negative time, either of which may be unexpected
by the user.
The 6 hour limit is purposely not documented so that it can be changed
in the future if necessary.
Closes https://github.com/curl/curl/pull/15833
Follow-up to 40c264db61 after discussions on IRC.
The new style is
name[0-99]=contents
and
name[0-99]@filename
A) This does not cause the same problems with old curl versions trying
the new syntax as this way will cause old curls just fail with syntax
error and not risk using the wrong file.
B) Adds the same byte range support for "normal" assigns, which the
previous syntax did not. Thus lets a user get a partial content of a
variable etc.
Added test 790 and 791 to verify non-file assigns with ranges.
Closes#15862
mbedtls is picky when a mbedtls_ssl_write) was previously blocked. It
requires to be called with the same amount of bytes again, or it will
lose bytes, e.g. reporting all was sent but they were not. Remember the
blocked length and use that when set.
Reported-by: Tamás Bálint Misius
Fixes#15801Closes#15846
Use 'banfunc' and 'allowfunc' in .checksrc to specify which functions to
ban or allow to be used. This saves us from having to edit the script
going forward when we want to ban or allow specific functions.
This replaces a set of previous rules and all banned functions are now
checked with the BANNEDFUNC rule.
There is a set of default banned functions, shown by invoking
./checksrc.
Also, -a and -b options are added to specify allowed or banned functions
on the command line.
Closes#15835
The redirect logic was broken when the redirect-to URL was a relative
URL only as a fragment or query (starting with '#' or '?').
Extended test 1560 to reproduce, then verify.
Reported-by: Jeroen Ooms
Fixes#15836Closes#15848
- Make curl_multi_waitfds consistent with the documentation.
Issue Addressed:
- The documentation of curl_multi_waitfds indicates that users should
be able to call curl_multi_waitfds with a NULL ufds. However, before
this change, the function would return CURLM_BAD_FUNCTION_ARGUMENT.
- Additionally, the documentation suggests that users can use this
function to determine the number of file descriptors (fds) needed.
However, the function would stop counting fds if the supplied fds
were exhausted.
Changes Made:
- NULL ufds Handling: curl_multi_waitfds can now accept a NULL ufds if
size is also zero.
- Counting File Descriptors: If curl_multi_waitfds is passed a NULL
ufds, or the size of ufds is insufficient, the output parameter
fd_count will return the number of fds needed. This value may be
higher than actually needed but never lower.
Testing:
- Test 2405 has been updated to cover the usage scenarios described
above.
Fixes https://github.com/curl/curl/issues/15146
Closes https://github.com/curl/curl/pull/15155
- Parse the input string without modifying it.
Prior to this change a segfault could occur if the input string was
const because the tokenizer modified the input string. For example if
the user set CURLOPT_COOKIELIST to a const string then libcurl would
likely cause a crash when modifying that string. Even if the string was
not const or a crash did not occur there was still the incorrect and
unexpected modification of the user's input string.
This issue was caused by 30da1f59 (precedes 8.11.0) which refactored
some options parsing and eliminated the copy of the input string. Also,
an earlier commit f88cc654 incorrectly cast the input pointer when
passing it to strtok.
Co-authored-by: Daniel Stenberg
Closes https://github.com/curl/curl/pull/15826
build:
- autotools: fix to build generated sources for the `tidy` target.
- autotools: allow passing custom clang-tidy options via
`CURL_CLANG_TIDYFLAGS` env.
- cmake: add `CURL_CLANG_TIDY` option to configure for `clang-tidy`.
Also add:
- `CLANG_TIDY` variable to customize the `clang-tidy` tool.
- `CURL_CLANG_TIDYFLAGS` to pass custom options to `clang-tidy`.
- apply `--enable-werror` and `-DCURL_WERROR=ON` to `clang-tidy`.
CI/GHA:
- add clang-tidy job for Linux, using autotools and clang-tidy v18.
This one needs to disable `clang-analyzer-valist.Uninitialized`
to avoid false positives:
https://github.com/llvm/llvm-project/issues/40656
Duration: 5.5 minutes
- add clang-tidy job for macOS, using cmake and clang-tidy v19.
This one also covers tests and examples, and doesn't hit the false
positives seen with llvm v18 and earlier.
Duration: 4.5 minutes
- Linux/macOS: skip installing test dependencies when not building or
running tests.
fix fallouts reported by `clang-tidy`:
- lib:
- cf-h2-proxy: unused assignment in non-debug builds.
- cf-socket: silence warning.
FIXME: https://github.com/curl/curl/pull/15825#issuecomment-2561867769
- ftp: NULL passed to `strncmp()`.
- http2: NULL-ptr deref.
- mprintf: silence warning.
- src/tool_writeout: NULL passed to `fputs()`.
- examples:
- invalid file pointers.
- missing `fclose()`.
- tests:
- http/clients/hx-download: memory leaks on error.
- http/clients/hx-download: memory leak on repeat `-r` option.
- server: double `fclose()`.
https://www.man7.org/linux/man-pages/man3/fclose.3.html
- server: invalid file pointer/handle.
- server/getpart: unused assignments.
- server/mqttd: leak on failed `realloc()`.
- server/tftpd: NULL passed to `strcmp()`.
Closes#15825
When a QUIC TLS session announced early data support and
'CURLSSLOPT_EARLYDATA' is set for the transfer, send initial request and
body (up to the 128k we buffer) as 0RTT when curl is built with
ngtcp2+gnutls.
QUIC 0RTT needs not only the TLS session but the QUIC transport
paramters as well. Store those and the earlydata max value together with
the session in the cache.
Add test case for h3 use of this. Enable quic early data in nghttpx for
testing.
Closes#15667
We started using codeql for static code analysis in 7183f5acc3,
June 2020.
Since then, not a single commit has been merged into the source code
repository citing codeql as source or reason. Yet, it keeps getting
updated and we get constant reminders to upgrade the pinning it to the
latest hash.
During 4.5 years with intense development and significant code churn.
While Coverity, scan-build and CodeSonar have belped us point out many
mistakes, codeql has remained silent (or had false positives).
For this little gain, I think we spend a disproportionate amount of work
on codeql maintanance.
We can try again in a future if we think it improves.
Assisted-by: Viktor Szakats
Closes#15798
And use it from src and tests.
Syncing this functionality between platforms and build targets.
Also: Stop redefining `O_BINARY` in src, and use a local macro with
the same effect. `O_BINARY` is used in `CURL_SET_BINMODE()` to decide
if this functionality is supported, and redefining it makes this check
pass always in unity builds. The check is required for Apple OS, because
it offers a `setmode()` function, successfully detected by both CMake
and autotools, but that function has a different functionality and
signature than that expected by `CURL_SET_BINMODE()`.
Also:
- drop MetaWare High C (MS-DOS) support for set binmode.
- tests/libtest/Makefile.inc: dedupe comments.
- lib/curl_setup_once.h: tidy up feature guards for `io.h`, `fcntl.h`.
Ref: #15652Closes#15787
Let CURLINFO_QUEUE_TIME_T count only the time a transfer spends queued,
including possible redirect requests.
Add var 'time_queue' for reporting the time in write outs.
Add test for verifying correct reporting.
Closes#15512
Allowing --variable read a portion of provided files, makes curl work on
partial files for any options that accepts strings. Like --data and others.
The byte offset is provided within brackets, with a semicolon separator
like: --variable name@file;[100-200]"
Inspired by #14479
Assisted-by: Manuel Einfalt
Test 784 - 789. Documentation update provided.
Closes#15739
lib : remove all hyper code
configure: stop detecting hyper
docs: no more mention of hyper
tests: mo more special-handling of hyper builds
CI: no jobs using hyper
Closes#15120
For MinGW this is `-DNO_OLDNAMES`, with MSVC it is
`-D_CRT_DECLARE_NONSTDC_NAMES=0`.
There have been some support for this before this patch.
After this patch this is extended to all examples.
(And also the standalone http/client programs, if here.)
Cherry-picked from #15652Closes#15789
Described in detail in internal doc TLS-SESSIONS.md
Main points:
- use a new `ssl_peer_key` for cache lookups by connection filters
- recognize differences between TLSv1.3 and other tickets
* TLSv1.3 tickets are single-use, cache can hold several of them for a peer
* TLSv1.2 are reused, keep only a single one per peer
- differentiate between ticket BLOB to store (that could be persisted) and object instances
- use put/take/return pattern for cache access
- remember TLS version, ALPN protocol, time received and lifetime of ticket
- auto-expire tickets after their lifetime
Closes#15774
When a specific hostname matched, and only a password is set before
another machine is specified in the netrc file, the parser would not be
happy and stop there and return the password-only state. It instead
continued and did not return a match.
Add test 2005 to verify this case
Regression from e9b9bba, shipped in 8.11.1.
Reported-by: Ben Zanin
Fixes#15767Closes#15768
Adjusted test 186 to verify.
Regression in 9664d5a547, shipped in 8.11.1
Reported-by: IcedCoffeee on github
Assisted-by: Jay Satiro
Fixes#15761Closes#15762
It would previously wrongly also catch function calls to function names
ending with 'return'
Amended test1185.
Reported-by: Stefan Eissing
Closes#15764
`Makefile.mk` supported MS-DOS and Amiga, but `./configure` also
supported them in a better tested and more flexible way.
This patch also adds CMake support for MS-DOS/DJGPP and Amiga OS 3.
`Makefile.mk` was not maintained. Delete it in favour of first-tier
build methods.
Also include some non-MS-DOS/AmigaOS-specific tidy-up, see details at
the end of this message.
Details:
- fix/silence all MS-DOS/DJGPP build warnings and issues.
- add MS-DOS support to cmake.
- default to `ENABLE_THREADED_RESOLVER=OFF` for MS-DOS.
- add support for `WATT_ROOT`.
- use static libcurl with MS-DOS.
- fixup default CMake suffixes/prefixes for DJGPP.
- disable hidden symbols for MS-DOS. Not supported on MS-DOS.
- opt-in MS-DOS into `USE_UNIX_SOCKETS`.
- improve MS-DOS support in autotools.
- default to `--disable-threaded-resolver` for MS-DOS.
- make sure to use `close_s()` (from Watt-32) with autotools and cmake.
`Makefile.mk` used it before this patch.
- GHA: add DJGPP cmake (~30s) and autotools (~60s) build jobs.
Also build tests and examples with cmake.
- improve AmigaOS support in autotools:
- configure: detect `CloseSocket()` when it's a macro.
- configure: fix `IoctlSocket` detection on AmigaOS.
- curl-amissl.m4: pass AmiSSL libs to tests/servers.
- add AmigaOS3 support to cmake:
- cmake: fix `HAVE_IOCTLSOCKET_CAMEL` and
`HAVE_IOCTLSOCKET_CAMEL_FIONBIO` detections.
- set necessary system libs.
- add AmiSSL support.
- inet_ntop, inet_pton: fix using it for AmigaOS. cmake detects them,
and they did not compile with AmigaOS.
- cmake: better sync `gethostname` detection with autotools.
Fixes detection for AmigaOS, where `gethostname` is a macro.
- cmake: fix `sys/utime.h` detection on AmigaOS.
- cmake: force-disable `getaddrinfo` for AmigaOS.
- cmake: tweak threading and static/shared default for AmigaOS.
- cmake: rely on manual variable `AMIGA` to enable the platform.
- GHA: add AmigaOS cmake and autotools (~45s) jobs.
Also build tests and examples with cmake.
- INSTALL: update MS-DOS and AmigaOS build instructions.
- amigaos: fix `-Wpointer-sign` and
`zero or negative size array '_args'` in `Printf()`.
- amigaos: fix `-Wpointer-sign`
- amigaos: fix `-Wredundant-decls` `errno` and `h_errno`.
- amigaos: brute-force silence `lseek()` size warnings.
- amigaos: server/resolve: silence `-Wdiscarded-qualifiers`.
- amigaos: server/resolve: fix `-Wpointer-sign`.
- amigaos: fix `CURL_SA_FAMILY_T` type.
- nonblock: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
`ioctl` is also detected, but fails when used. Make the above override
it for a successful build.
Authored-by: Darren Banfi
Fixes#15537Closes#15603
- tftpd: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
- tftpd: tidy-up conditional code.
- curl: set stack size to 16384 for AmigaOS3/4
Overriding the default 4096.
Suggested-by: Darren Banfi
Ref: https://github.com/curl/curl/pull/15543#issuecomment-2498783123
Ref: https://wiki.amigaos.net/wiki/Controlling_Application_Stack
- functypes.h: fix `SEND_QUAL_ARG2` for AmigaOS.
- tftp: add missing cast in sendto() call for AmigaOS.
- getinfo: fix warning with AmigaOS.
- tool_operate: silence warning with AmigaOS
- amigaos: fix building libtests due to missing `RLIMIT_NOFILE`.
- curl_gethostname: silence warning for AmigaOS.
- ftp: silence `-Wtype-limits` for AmigaOS.
- libtest: fix timeval initialization for AmigaOS.
- examples: fix `timeval` initialization for AmigaOS.
- examples: silence warning for AmigaOS.
- configure: fix IPv6 detection for cross-builds.
- netrc: fix to build with AmigaOS cleanly.
- buildinfo: detect and add `DOS` tag for MS-DOS builds.
- buildinfo: add `AMIGA` to buildinfo.txt in auttools.
- build: move `USE_WATT32` macro definition to cmake/configure.
Non-MS-DOS/AmigeOS-specific tidy-ups:
- configure: sync `sa_family_t` detection with cmake.
- configure: sync `ADDRESS_FAMILY` detection signals with cmake.
- doh: use `CURL_SA_FAMILY_T`.
- lib: drop mingw-specific `CURL_SA_FAMILY_T` workaround.
- cmake: extend instead of override check-specific
configurations/requirements.
This allows to honor global requirements added earlier.
Necessary for AmigaOS for example.
- cmake: omit warning on disabled IPv6 for MS-DOS and AmigaOS.
No IPv6 support on these platforms. Also sync with autotools.
- lib1960: use libcurl `inet_pton()` wrapper.
- cmake: detect LibreSSL (to match autotools).
- cmake: say the specific OpenSSL flavour detected.
- hostip: add missing `HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID` guard.
- lib: simplify classic mac feature guards.
Follow-up to a8861b6ccd#9764Closes#15543
