Using sscanf() is not a (security) problem in itself, but we strongly
discorage using it for parsing input since it is hard to use right, easy
to mess up and often makes for sloppy error checking.
Allow it in examples and tests
Closes#15687
Designed to aid converting off from sscanf parsers. sscanf is hard to
use right, easy to mess up and often makes for sloppy error checking.
The new parsers allow more exact and pedandic parsing.
This new set of functions should be possible to use (and extend) and
switch over other libcurl parser code to use going forward.
Adapts the following to use the new functions:
- altsvc.c
- hsts.c
- http_aws_sigv4.c
Bonus: fewer memory copies, fewer stack buffers.
Test: Unit test1664
Docs: docs/internals/STRPARSE.md
Closes#15692
This fix impacts ws-data.c and ws-pingpong.c. sleep() replaced with
system PROCESS_DELAY_() having the same resolution as usleep().
Fixes#15711Closes#15712
Signed-off-by: Randall S. Becker <randall.becker@nexbridge.ca>
The date parser function is very forgiving and skips most "irrelevant"
characters in its hunt for a date to figure out. Therefore it is
important to make sure the date string is properly null terminated so
that it does not accidentally parse a piece of whatever text follows
after the date.
Add test483: test (overly) long expire dates in cookies
Closes#15709
- build: fix to exclude 'documentation' tests when building
the documentation is explicitly disabled. Both for cmake
and `./configure`.
Reported-by: Daniel Engberg
Fixes#15703
- test481, test482: fix for builds with no manual.
Reported-by: Daniel Engberg
Fixes#15703
- configure: fix to always detect Perl. Running tests require
it when run. Before this patch Perl wasn't detected when
documentation/manual/embedded-CA were all disabled,
making tests fail to start.
- test1177: add keyword `documentation`. It depends no
`curl_version_info.3`.
- GHA/linux: test cmake and `./configure` with docs and
manual disabled and tests run.
Closes#15704
The base64 mime encoder stalls when it cannot encode a full 3 byte input
set into the read buffer. The workaround for this limitation was
incomplete and could lead to stalled transfers when the last chunk to
upload was smaller than 4 bytes.
Use a tmp buffer on small reads to allow mime encoders more space to put
their things.
Add test case reproducing the issue and fix.
Reported-by: Alexis Savin
Fixes#15688Closes#15691
Fix regression that no longer printed the error messages about expired
certificates in openssl. Add test case for openssl/gnutls/wolfssl.
Fixes#15612
Reported-by: hiimmat on github
Closes#15613
Add test_02_33 to run with various values for the multi option
CURLMOPT_MAX_HOST_CONNECTIONS and CURLOPT_FRESH_CONNECT to trigger
connection pool limit handling code.
Closes#15494
The MSVC UWP job in CI did not actually enable UWP. Fix this and
the fallouts discovered after enabling it.
- GHA/windows: make sure to enable UWP in MSVC vcpkg UWP job.
Use the CMake options and C flags already used for mingw-w64, but use
`WINAPI_FAMILY_PC_APP` instead of the deprecated `WINAPI_FAMILY_APP`.
(The former is not supported by mingw-w64, so leave it there as-is.)
Follow-up to cb22cfca69#14077
- GHA/windows: by default the MSVC UWP job became 2x-3x slower than
others after actually enabling UWP. Most of it is caused by
CMake/MSBuild automatically building full APPX containers for each
`.exe` target. This includes 21 CMake feature detections. Each
detection app is built into a 15MB APPX project, with code signing,
logos, etc. Example:
https://github.com/curl/curl/actions/runs/12056968170/job/33620610958
Disable this overhead for curl build targets via custom
`CMAKE_VS_GLOBALS` options. I've found no way to apply them to feature
detection targets, so those remain slow.
- cmake: automatically enable Unicode for UWP builds. It's required.
Also stop enabling it manually in the existing CI job.
- tests: fix `getpid()` use for Windows UWP:
```
tests\server\util.c(281,21): warning C4013: 'getpid' undefined; assuming extern returning int
```
Ref: https://github.com/curl/curl/actions/runs/12061215311/job/33632904249#step:11:38
- src/tool_doswin: disable `GetLoadedModulePaths()` for UWP.
mingw-w64 UWP was okay with this, but MS SDK headers are not.
This makes `--dump-module-paths` return empty for UWP builds.
```
src\tool_doswin.c(620,3): error C2065: 'MODULEENTRY32': undeclared identifier
src\tool_doswin.c(626,11): warning C4013: 'CreateToolhelp32Snapshot' undefined; assuming extern returning int
src\tool_doswin.c(626,36): error C2065: 'TH32CS_SNAPMODULE': undeclared identifier
src\tool_doswin.c(632,7): warning C4013: 'Module32First' undefined; assuming extern returning int
```
Ref: https://github.com/curl/curl/actions/runs/12055081933/job/33614629930#step:9:35
- examples: fix `websocket.c` to include `winsock2.h` before `windows.h`
to make it build with MSVC UWP:
```
include\curl\curl.h(143,16): error C2061: syntax error: identifier 'curl_socket_t'
include\curl\curl.h(143,16): error C2059: syntax error: ';'
include\curl\curl.h(417,52): error C2146: syntax error: missing ')' before identifier 'curlfd'
include\curl\curl.h(417,38): error C2081: 'curl_socket_t': name in formal parameter list illegal
```
Ref: https://github.com/curl/curl/actions/runs/12055317910/job/33615644427#step:14:126
- GHA/windows: silence linker warning with MSVC UWP builds:
```
LINK : warning LNK4075: ignoring '/INCREMENTAL' due to '/OPT:ICF' specification
```
Ref: https://github.com/curl/curl/actions/runs/12055696808/job/33616629610#step:11:38
- GHA/windows: set `/INCREMENTAL:NO` for all MSVC jobs to improve
performance a little.
- cmake: show `UWP` platform flag.
Ref: #15652Closes#15657
- make sure that a match that returns a username also returns a
password, that should be blank if no password is found
- fix handling of multiple logins for same host where the password/login
order might be reversed.
- reject credentials provided in the .netrc if they contain ASCII control
codes - if the used protocol does not support such (like HTTP and WS do)
Reported-by: Harry Sintonen
Add test 478, 479 and 480 to verify. Updated unit 1304.
Closes#15586
`libcurl.pc` `Libs.private` (since 8.11.0, and in `Libs` before 7.20.0)
and `curl-config` `--static-libs` (since 7.17.1, and in `Libs` between
7.7.2-7.25.0). This included all flags inherited from the environment,
in addition to those coming from dependency detections.
To avoid spilling all linker flags inherited from the environment to
the libcurl config files, this patch omits them all, except `-L`, `-F`,
`--library-path=` and `-framework` options, which are still passed.
The rationale for the exceptions is that `LIBS` is passed as-is, and
`LDFLAGS`, `LIBS` are the canonical way to pass custom libs options
to a build. `LIBS` may not work without a matching custom libpath.
This brings autotools behaviour closer to cmake, and `curl-config`
closer to `libcurl.pc`.
Follow-up to 9f56bb608e#14681
Follow-up to 4c8adc8fee
Reported-by: Peter Marko
Fixes#15533Closes#15550
Using this option (only available in debug builds) makes curl always
call curl_easy_duphandle() on the handle before using it.
To help us catch curl_easy_duphandle() mistakes better.
Add a CI job using this.
Bonus: the previous runtests option -e is now also supported as
--test-event
Closes#15504
The netrc init was only done in the Curl_open, meaning that a duplicated
handle would not get inited properly.
Added test 2309 to verify. It does netrc auth with a duplicated handle.
Regression from 3b43a05e00
Reported-by: tranzystorekk on github
Fixes#15496Closes#15503
Build wolfSSL master with
./configure --prefix=/path --enable-ip-alt-name --enable-quic
--enable-earlydata --enable-psk --enable-opensslcoexist
and configure curl with openssl + wolfssl. Normal tests run.
pytest session resumption fails, as wolfssl does not handle the
new_session callback without opensslextra right now.
Closes#15481
Use `__APPLE__` macro to detect Apple OS instead of relying on
the string in `CURL_OS`.
This also fixes detection with default CMake builds where `CURL_OS` is
`Darwin`. The code before this patch was expecting this substring in
lowercase.
Closes#15461
- appveyor: add build-only job for clang-cl.
- cmake: `-pedantic-errors` enables `-Werror,-Wlanguage-extension-token`
automatically, which makes `__int64` detection fail.
Explictly disable this compiler warning for clang-cl to make the
feature detection work and to accept `__int64` in the source code.
- cmake: disable `-Wlanguage-extension-token` warning for clang-cl
to fix these when encountering `__int64`:
```
lib/formdata.c(797,29): error : extension used [-Werror,-Wlanguage-extension-token]
lib/warnless.c(117,33): error : extension used [-Werror,-Wlanguage-extension-token]
lib/warnless.c(60,28): message : expanded from macro 'CURL_MASK_SCOFFT'
lib/warnless.c(59,38): message : expanded from macro 'CURL_MASK_UCOFFT'
include\curl/system.h(352,40): message : expanded from macro 'CURL_TYPEOF_CURL_OFF_T'
```
- make `__GNUC__` warning suppressions apply to `__clang__` too.
Necessary for clang-cl, which defines the latter, but not the former.
(Regular clang defines both.)
- examples: fix clang-cl compiler warning in `http2-upload.c`.
```
docs\examples\http2-upload.c(56,5): error : no previous prototype for function 'my_gettimeofday' [-Werror,-Wmissing-prototypes]
docs\examples\http2-upload.c(56,1): message : declare 'static' if the function is not intended to be used outside of this translation unit
```
- unit2604: add missing `#pragma GCC diagnostic pop`.
Follow-up to e53523fef0#14859
- unit1652: limit compiler warning suppression to GCC.
They do not affect clang builds.
Follow-up to 71cf0d1fca#14772Closes#15449
Changes to make a curl built with OpenSSL + GnuTLS to run successfully
in our pytests. Run
CURL_SSL_BACKEND=openssl pytest
to test a TLS backend other than the default.
Closes#15443
handle/handle64 requires a literal match with the filenames it's
listing.
Also:
- make handle64 log messages more unique to help text searches.
- update a comment with Windows Perl info.
Cherry-picked from #14949Closes#15436
Add two test cases that connection using a hostname the server has no
certificate for. First, verify that the peer verification fail, as
expected. Second, provide '--insecure' to test that the connection
succeeded and returned some data.
Closes#15429
New args for scorecard.py:
* --remote ip:port to run tests against a remote server
* --start-only: to only start the servers, listing their ports
Start the server using "scorecard.py --start-only" on one machine and
then run the tests with "scorecard.py --remote ip:port" against that
machine.
Closes#15415
Extend `INSTALL-CMAKE` document with the list of available options,
a short description and default values.
The list may not be 100% complete.
There are no component boundaries in CMake, so the line is blurry
between curl options, CMake options, CMake Find modules options.
I included certain CMake options that seemed useful, and/or have
dedicated use withing curl's CMake source. But, all CMake built-in
options are usable, as documented upstream in CMake.
The naming of the options has a heritage and the inconsistencies with
it, including a lack of clear namespace. This may be subject to future
updates, also after figuring out which name has special meaning within
CMake and/or CMake projects out of unwritten convention or something
more tangible.
CMake allows to initialize any internal variable via `-D`. This may be
useful to pre-initialize/override feature check results. The list
doesn't contain these, and they remain officially undocumented.
Also:
- make adjustments to keep the spellchecker happy.
- retrofit description changes to the cmake sources.
- stop documenting deprecated `Find*` variables.
Reported-by: Daniel Stenberg
Fixes https://github.com/curl/curl/discussions/14885Closes#15388
Use session cache for QUIC when built with quictls or wolfSSL.
Add test_017_10 for verifying QUIC TLS session reuse when built with
quictls, gnutls or wolfssl.
Closes#15358
Add length to session saves, making it clear that we are storing a byte
blob and allowing memcmp() on sameness check.
Remove some pytest skips for bearssl to see if they now work properly in
CI.
Closes#15395
Use mbedtls_ssl_session_load() and mbedtls_ssl_session_save() to convert
TLS sessions to byte blobs for the session cache.
Fix a skip message to better indicate why the test is skipped for
mbedtls.
Closes#15398
Sort TESTINFO lines by description within the number of skipped test.
It makes the list of skipped test groups easier to diff/compare between
jobs and runs.
Closes#15374
They complete in 4 and 7 minutes, and do not hold back the main Linux
workflow.
Also:
- bump default parallelism for `test-torture` target to `-j20`
(was: `-j2`).
- drop redundant package install from `rustls` jobs.
Closes#15360
It reduces the number of synonym variables in the code.
Makes it easier to grok and grep.
- replace `CURL_SOURCE_DIR`
with `PROJECT_SOURCE_DIR`.
- replace `CURL_BINARY_DIR`
with `PROJECT_BINARY_DIR` or `CMAKE_CURRENT_BINARY_DIR`.
- replace a single use of `CMAKE_BINARY_DIR`
with `PROJECT_BINARY_DIR`.
- replace `CMAKE_CURRENT_*_DIR`
with `PROJECT_*_DIR` where it makes the code more uniform.
- quote an argument (formatting).
Closes#15331
We used to include a special mod_h2 in our CI that supports the
directive H2MaxDataFrameLen for test_02_20. Since then, ubuntu-lastest
includes a more recent apache httpd. Let's see if we can live without
the special.
Closes#15353
When a server signals EOS from its side and the curl upload is
unfinished and the server has not given a positive HTTP status response,
auto RST the stream to signal that the upload is incomplete and that the
whole transfer can be stopped.
Fixes the case where the server responds with 413 on an upload but does
not RST the stream from its side, as httpd and others do.
Reported-by: jkamp-aws on github
Fixes#15316Closes#15325
