Commit Graph

14509 Commits

Author SHA1 Message Date
Daniel Stenberg
5426cd5948
content_encoding: put the decomp buffers into the writer structs
- no more malloc/free per chunk
- removes the extra malloc entirely
- make the buffer (much) smaller (10MB => 16KB!)
- rename 'decomp' to 'buffer' to clarify purpose

Closes #16079
2025-01-24 14:04:23 +01:00
Daniel Stenberg
76f83f0db2
content_encoding: drop support for zlib before 1.2.0.4
zlib 1.2.0.4 was released on 10 August 2003

Closes #16079
2025-01-24 14:04:23 +01:00
Stefan Eissing
1213c31272
lib: redirect handling by protocol handler
Adds a `follow()` callback to protocol handlers, so they may decide how
to act on a `newurl` after a request has been done. This is optional.

This moves the HTTP code for handling redirects from multi.c to http.c
where it should be. If we ever add a protocol with its own logic, it
would install its own follow function.

Closes #16075
2025-01-24 11:00:34 +01:00
Stefan Eissing
e83818cae1
lib: clarify 'conn->httpversion'
The variable `conn->httpversion` was used for several purposes and it
was unclear at which time the value represents what.

- rename `conn->httpversion` to `conn->httpversion_seen`
  This makes clear that the variable only records the last
  HTTP version seen on the connection - if any. And that it
  no longer is an indication of what version to use.
- Change Alt-Svc handling to no longer modify `conn->httpversion`
  but set `data->state.httpwant` for influencing the HTTP version
  to use on a transfer.
- Add `data->req.httpversion_sent` to have a record of what
  HTTP version was sent in a request
- Add connection filter type CF_TYPE_HTTP
- Add filter query `CF_QUERY_HTTP_VERSION` to ask what HTTP
  filter version is in place
- Lookup filters HTTP version instead of using `conn->httpversion`

Test test_12_05 now switches to HTTP/1.1 correctly and the
expectations have been fixed.

Removed the connection fitler "is_httpN()" checks and using
the version query instead.

Closes #16073
2025-01-24 10:59:02 +01:00
Stefan Eissing
cd9107e012
cfilters: kill connection filter events attach+detach
Make transfer attach/detach to/from connections chepaer.

- the "attach" event was no longer implemented by any filter
- the "detach" did the same as the "done" event for the filters
  who still implemented it. It should be superfluous as the "done"
  must always happen.

Closes #16067
2025-01-22 08:28:38 +01:00
Daniel Stenberg
7c2b325004
curl_multi_fdset: include the shutdown connections in the set
They were previously missing.

Follow-up from c9b95c0bb3

Fixes #15156
Reported-by: Christopher Dannemiller
Closes #16049
2025-01-22 07:55:31 +01:00
Neil Johari
c80715169c
content_encoding: support use of custom libzstd memory functions
If ZSTD_STATIC_LINKING_ONLY is defined.

This functionality was introduced in zstd v0.8.1 in 2016 here:
facebook/zstd@be6180c

Closes #16028
2025-01-21 23:37:03 +01:00
Jay Satiro
789c7f1b6c easy_lock: use Sleep(1) for thread yield on old Windows
- Prefer Sleep(1) over sched_yield() for pre-Vista thread yield.

On Windows sched_yield is often implemented as Sleep(0) which only
yields to threads of highest priority to current priority. However,
during libcurl initialization if there is thread contention then it's
possible that there is a wait for a different library or OS thread of
a lesser priority and then the yield is not effective during that time.
On the other hand Sleep(1) will wait the minimum time slice which is
usually like 15ms or more.

Prior to this change 2c4bfef removed sched_yield detection on Windows,
which effectively removed the yield in the spin lock, and therefore this
change restores the yield but in a different way.

For Windows Vista and later we use SRW locks and do not have this issue.

Ref: https://github.com/curl/curl/pull/16037#issuecomment-2600161764
Ref: https://devblogs.microsoft.com/oldnewthing/20051004-09/?p=33923

Closes https://github.com/curl/curl/pull/16048
2025-01-21 01:45:21 -05:00
Viktor Szakats
292d81ca54
windows: merge config-win32ce.h into config-win32.h
They were more or less the same, but each missed some things the other
had. Windows CE is a subset of Win32, make the headers reflect that and
avoid duplications.

Ref: #15975
Closes #16038
2025-01-20 22:32:06 +01:00
Viktor Szakats
08e2cceaf1
cmake: drop fseeko() pre-fill and check for Windows
To sync detection code with autotools.

Closes #16041
2025-01-19 15:03:16 +01:00
Daniel Stenberg
8368249907
asyn-ares: initial HTTPS resolve support
Gets the ALPN list the same way DoH does. Needs c-ares 1.28.0 or later.

Thanks-to: Brad House

Closes #16039
2025-01-19 11:25:09 +01:00
Daniel Stenberg
ea76380299
mbedtls: prefix our feature defines with HAS_
Consistent naming helps.

Closes #16046
2025-01-19 11:23:22 +01:00
Stefan Eissing
25b445e479
TLS: check connection for SSL use, not handler
Protocol handler option PROTOPT_SSL is used to setup a connection
filters. Once that is done, used `Curl_conn_is_ssl()` to check if
a connection uses SSL.

There may be other reasons to add SSL to a connection, e.g. starttls.

Closes #16034
2025-01-17 14:04:20 +01:00
Viktor Szakats
2e2c6b0b4e
windows: drop dupe macros, detect CURL_OS for WinCE ARM, indentation
- setup-win32: drop duplicate `WIN32_LEAN_AND_MEAN` and `NOGDI` macros
  (set earlier in `curl_setup.h`)

- config-win32ce: detect ARM for `CURL_OS` macro.
  Based on: https://cegcc.sourceforge.net/docs/details.html
  It should also apply to MSVC. (untested)

- indentation.

Closes #16029
2025-01-17 12:56:43 +01:00
Stefan Eissing
02edae54e8 websocket: fix message send corruption
- Fix a bug in EAGAIN handling when sending frames that led to a
  corrupted last byte of the frame sent.

- Restore sanity to curl_ws_send() behaviour:

  - Partial writes are reported as OK with the actual number of
    payload bytes sent.

  - CURLE_AGAIN is only returned when none of the payload bytes
    (or for 0-length frames, not all of the frame header bytes)
    could be sent.

  - curl_ws_send() now behaves like a common send() call.

- Change 'ws-data' test client to allow concurrent send/recv
  operations and vary frame sizes and repeat count.

- Add DEBUG env var CURL_WS_CHUNK_EAGAIN to simulate blocking
  after a chunk of an encoded websocket frame has been sent.

- Add tests.


Prior to this change data corruption may occur when sending websocket
messages due to two bugs:

1) 3e64569a (precedes 8.10.0) caused a data corruption bug in the last
   byte of frame of large messages.

2) curl_ws_send had non-traditional send behavior and could return
   CURLE_AGAIN with bytes sent and expect the caller to adjust buffer
   and buflen in a subsequent call. That behavior was not documented.


Reported-by: na-trium-144@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/15865
Fixes https://github.com/curl/curl/issues/15865#issuecomment-2569870144
Closes https://github.com/curl/curl/pull/15901
2025-01-16 16:19:07 -05:00
Daniel Stenberg
13afb1128f
asyn-ares: acknowledge CURLOPT_DNS_SERVERS set to NULL
Since c-ares has no function to restore this value to default, it
instead closes and re-inits the c-ares channel.

Reported-by: Deniz Sökmen
Fixes #16015
Closes #16016
2025-01-16 20:22:12 +01:00
Viktor Szakats
f07612cd9a
tidy-up: extend CURL_O_BINARY to lib and tests
Move `CURL_O_BINARY` definition from src to lib and use it from lib and
tests code.

Closes #16009
2025-01-16 12:36:07 +01:00
Stefan Eissing
a6eac83481
cf-https-connect: look into httpsrr alpns when available
Improved the filter implementation to be flexible in which order h3 and
h2/h1 are attempted. When HTTPSRR is enabled, look at the ALPNs it found
and use the order given for connecting in default setups.

Closes #16012
2025-01-16 08:23:02 +01:00
Daniel Stenberg
bb93536270
doh: send HTTPS RR requests for all HTTP(S) transfers
When enabled in the build.

Update test2100: verify with HTTPS RR included

Adjust runtests and server/disabled.c to include "HTTPSRR" as a feature
in the test suite.

Also, decode the ALPN list in HTTPS records straight into IDs. There's
no point in storing everything in string format. Skip ALPNs we do not
support.

Closes #16007
2025-01-15 12:55:15 +01:00
Daniel Stenberg
5d70a5c5a4
doh: cleanups and extended HTTPS RR code
In preparation for using HTTPS outside of ECH, the parser now also
extracts the port number.

Plus other minor cleanups.

Closes #16007
2025-01-15 12:55:04 +01:00
Daniel Stenberg
be82a3605a
easy: make curl_easy_perform() return error if connection still there
This typically happens if CURL_CONNECT_ONLY is used and a second
curl_easy_perform() is attempted.

A connection "taken over" with CURL_CONNECT_ONLY cannot be ended any
other way than a curl_easy_cleanup() on the easy handle that holds it.

Add test 696 to verify.

Closes #16003
2025-01-14 16:32:46 +01:00
Daniel Stenberg
b9c173ebad
sha256/hmac: use these with TLS builds now
Used by the SSL session cache code.

Bug: https://curl.se/mail/lib-2025-01/0041.html
Reported-by: Aleksander Mazur
Closes #15994
2025-01-14 12:13:59 +01:00
Viktor Szakats
5474d70c3e
tidy-up: drop parenthesis around return expression
Closes #15990
2025-01-14 12:11:42 +01:00
Jay Satiro
5ffc73c78e transfer: fix CURLOPT_CURLU override logic
- Change setopt and pretransfer to always reset URL related variables
  for a CURLU handle set CURLOPT_CURLU.

This change is to ensure we are in compliance with the doc which says
CURLU handles must be able to override a URL set via CURLOPT_URL and
that if the contents of the CURLU handle changes between transfers then
the updated contents must be used.

Prior to this change, although subsequent transfers appear to be
performed correctly in those cases, the work URL `data->state.url` was
not updated. CURLINFO_EFFECTIVE_URL returns data->state.url to the user
so it would return the URL from the initial transfer which was the wrong
URL. It's likely there are other cases as well.

Ref: https://curl.se/libcurl/c/CURLOPT_CURLU.html

Reported-by: Nicolás San Martín

Fixes https://github.com/curl/curl/issues/15984
Closes https://github.com/curl/curl/pull/15985
2025-01-14 04:36:13 -05:00
Jay Satiro
8ab468c8aa mprintf: terminate snprintf output on windows
- Null terminate the end of the snprintf output buffer on Windows.

Old versions of the Windows CRT (which are often found on later versions
of Windows) do not terminate the snprintf output buffer if the output
reaches the max size.

This is a follow-up to parent 7e32f656 which made the same change but
limited it to mingw, however it is a CRT version issue irrespective of
compiler.

Ref: https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/snprintf-snprintf-snprintf-l-snwprintf-snwprintf-l?view=msvc-170#remarks

Closes https://github.com/curl/curl/pull/15997
2025-01-14 04:33:51 -05:00
Daniel Stenberg
7e32f65687
mprintf: fix integer handling in float precision
In the double output function when an extremely large width and
precision is set that reaches the libcurl maximum (325), the handling of
the precision part would do wrong which could lead to bad output.

Also: work-around for single-byte buffer snprintf overflow with mingw.

Extend test 557 to verify.

Coverity CID 1638751.

Closes #15988
2025-01-13 23:41:38 +01:00
Daniel Stenberg
23742ce015
cmake: enable SSLS-EXPORT in the build
Follow-up to 515a21f350

Closes #15991
2025-01-13 19:58:29 +01:00
Daniel Stenberg
9e54db2707
lib/easy.c: bring back the vtls/vtls_scache.h include 2025-01-13 18:26:19 +01:00
Daniel Stenberg
02ba03938f
telnet: handle single-byte input option
Coverity CID 1638753 correctly identies this code misbehaved if the
passed in suboption is exactly one byte long by substracting two from
the unsigned size_t variable.

Closes #15987
2025-01-13 14:29:02 +01:00
Daniel Stenberg
81a25ba7a4
altsvc: return error on dot-only name
Because it is a trailing dot that otherwise leads to a zero length name.

Coverity CID 1638755.

Closes #15986
2025-01-13 14:27:44 +01:00
Daniel Stenberg
cd43c92685
multihandle: add an ssl_scache here
The TLS session cache is now held by the multi handle unless it is
shared, so that all easy handles within a multi handle get the benefit
of sharing the same, larger, cache.

The multi handle session cache size is set to 25, unless it is the
internal one used for the easy interface - which still uses only 3.

Closes #15982
2025-01-13 10:32:03 +01:00
Daniel Stenberg
854e055a70
checksrc: use 'banfunc' proper in more places 2025-01-13 09:10:58 +01:00
Viktor Szakats
97164e99de
tidy-up: drop parenthesis around return values
Closes #15979
2025-01-13 02:44:40 +01:00
Viktor Szakats
231f868a4d
config: drop unused code and variables
- cmake, config-*: drop unused `PACKAGE*`, `VERSION` variables.
- config-win32: indentation
- config-win32ce: drop mingw-specific code.
  This header is not used with MinGW.
- config-win32ce: `_WIN64` is never true for Windows CE, drop.

Closes #15978
2025-01-13 02:44:40 +01:00
Viktor Szakats
14e279404b
cmake: drop VS2010 "Dialog Hell" workaround added in 2013
Delete the workaround added via a94a68a3c1
(2013-02-04). The commit message has no details. The comment mentions
"Dialog Hell", and seems to fix CMake missing to regenerate `CURL.sln`
with VS2010. It also added a FIXME saying the workaround can be deleted
with future versions of CMake.

At the time CMake's latest version was v2.8.10.

curl now requires v3.7 (2018) minimum, and v3.24 (2022) was the
latest CMake natively supporting VS2010. Assume this has since been
fixed.

Also: format an MSVC version reference in comment.

Closes #15973
2025-01-13 02:44:40 +01:00
Viktor Szakats
0ad30f0f64
msvc: require VS2005 for large file support
Large file support requires `_fseeki64()`. This function is offered in
VS2005 and upper.

VS2003 has it in the static CRT only, with declaration missing from
headers, so it's not usable.

Ref: https://archive.org/details/X10-38445 (MS Visual Studio .NET 2003)
Ref: 8b76a8aeb2 #15526

Closes #15958
2025-01-12 01:40:42 +01:00
Viktor Szakats
81680a4070
curl_setup: fix missing ADDRESS_FAMILY type in rare build cases
Build failed when both `ADDRESS_FAMILY` and `sockaddr_un` stuct were
missing from the Windows SDK, with UnixSockets enabled.

Seen with GNU 4.4.0 in CeGCC 0.59.1:
```
lib/curl_setup.h:983: error: expected specifier-qualifier-list before 'ADDRESS_FAMILY'
lib/curl_setup.h:985: warning: struct has no members
```

Also reported with VS2003:
https://datagirl.xyz/posts/wolfssl_curl_w2k.html

Closes #15969
2025-01-12 00:34:21 +01:00
Viktor Szakats
7eb4ddb850
windows: drop redundant USE_WIN32_SMALL_FILES macro
In effect it meant `_WIN32 && !USE_WIN32_LARGE_FILES`.
Replace it with these macros.

Also:
- configure: delete tautological check for small file support.
- configure: delete stray `_MSC_VER` reference. autotools does not
  support MSVC.
- drop tautological checks for WinCE in `config-win32*.h` when setting
  `USE_WIN32_LARGE_FILES`.
- merge related PP logic.
- prefer `#ifdef`, fix whitespace.

Suggested-by: Marcel Raad
Report: https://github.com/curl/curl/pull/15952#issuecomment-2580092328

Closes #15968
2025-01-12 00:34:21 +01:00
Viktor Szakats
b9840814ad
tidy-up: curl_setup.h, curl_setup_once.h, config-win32ce.h
- drop unused `LIBIDN_REQUIRED_VERSION` macro.
  Unused since 9c91ec7781

- drop compatibility error for `CURL_WANTS_CA_BUNDLE_ENV`.
  This macro was once set by `Makefile.mk` and Watcom makefiles.
  They are no longer supported, making the compatibility message moot.

  Follow-up to 7d7346519d #1542 (2017)
  Follow-up to c2aeb1b3ba #1538 (2017)

- document last MSVC supporting the `!_MT` condition
  Ref: https://stackoverflow.com/questions/2278919/are-the-time-functions-of-msvc-thread-safe

- flatten an `#if` tree, prefer `#ifdef`.

- fix/adjust indentation, whitespace.

Closes #15967
2025-01-12 00:34:21 +01:00
Viktor Szakats
de3e662ce1
msvc: tidy up _CRT_*_NO_DEPRECATE definitions
Dedupe and migrate MSVC-specific warning suppressions to `curl_setup.h`.
Make cmake set `_CRT_SECURE_NO_DEPRECATE` for examples and standalone
tests, and stop setting `_CRT_NONSTDC_NO_DEPRECATE` for them.

Details:
- drop version guards. On ancient MSVC version these macro are a no-op.
- move to `curl_setup.h` from `config-win32*.h`.
- sync macro values with CMake.
- cmake: stop setting them globally in favour of `curl_setup.h`.
- cmake: re-add these macros to `docs/examples` and `tests/http/clients`,
  which do not use `curl_setup.h`.
- cmake: drop `_CRT_NONSTDC_NO_DEPRECATE` for examples and tests.
  They build fine without.
- update comments.

Closes #15960
2025-01-12 00:34:21 +01:00
Viktor Szakats
0494566ae8
msvc: assume _INTEGRAL_MAX_BITS >= 64
curl requires C compilers supporting 64-bit `curl_off_t` type since
835682661c #10597 (v8.0.0).

Assume the MSVC compiler offers the necessary support.

It makes curl require Visual Studio .NET 2003, v7.1 (`_MSC_VER = 1310`).
With the possibility that 1300 (Visual Studio .NET, v7.0, 2002), or 1200
(Visual C++, 32-bit, v6.0, 1998) may also work.

Follow-up to ca18198dd4 #15952
Closes #15955
2025-01-10 11:27:55 +01:00
Stefan Eissing
34cebd8735
ssl session cache: change cache dimensions
Use a larger one when shared.

Closes #15953
2025-01-10 10:51:26 +01:00
sftcd
1ae47b91a3
ECH: update APIs to those agreed with OpenSSL maintainers
Closes #15945
2025-01-10 08:36:25 +01:00
Daniel Stenberg
386f570df6
cookie: cap expire times to 400 days
The pending cookie RFC update (currently known as 6265bis draft-19) says

  Let cookie-age-limit be the maximum age of the cookie (which name of
  Max-Age and an attribute-value of expiry-time. SHOULD be 400 days or
  less.

This change makes received cookies over the wire get capped to 400 days.

It does not cap the expiry date of cookies loaded from file.

It does this by rounding the expire time to a even minute. This, to
allow the test suite to do the same and have a chance to get the same
number for stable testing without requiring a debug build.

The test script generates TWO numbers in the output file for each
%days[] used in the input test file, and the function that subsequently
compares and verifies output is fine with *either* of the two numbers.

This is done so that if the test case is generated the second
immediately before curl runs, that updated expiry number is also deemed
okay. It still checks for an exact match of either number.

Closes #15937
2025-01-10 08:20:03 +01:00
Stefan Eissing
533dc84e6e
GnuTLS: fix 'time_appconnect' for early data
When using early data with GnuTLS, the the timer `appconnect`
had the value from the "pretended" connect, not when the actual
TLS handshake was done.

Closes #15954
2025-01-09 17:27:10 +01:00
Daniel Stenberg
a19b759dea
vtls_scache: fix possible null ptr deref
Pointed out by CodeSonar. It is probably not reachable, but might as
well just add a precaution.

Closes #15950
2025-01-09 17:25:48 +01:00
Daniel Stenberg
d0607b27b0
vtls_scache: avoid a "Redundant Condition"
Pointed out by CodeSonar. "ssl_config" can in fact not be NULL here.
Made it an assert instead.

Closes #15948
2025-01-09 17:23:26 +01:00
Viktor Szakats
ca18198dd4
msvc: drop checks for _MSC_VER >= 900
It's Visual C++, 32-bit, version 2.0, released in 1993. Used to verify
if `_INTEGRAL_MAX_BITS` is available.

After this patch we assume `_INTEGRAL_MAX_BITS` is always available in
MSVC.

Closes #15952
2025-01-09 14:52:45 +01:00
Viktor Szakats
8363dd51bc
msvc: drop checks for ancient versions
- drop version guard for `__inline`.
  Supported since `_MSC_VER` 1000.
  Visual C++, 32-bit, version 4.0 (1996)

- drop version guard for `__declspec(noreturn)` and `__forceinline`.
  Supported since `_MSC_VER` 1200.
  Visual C++, 32-bit, version 6.0 (1998)

For ancient versions, it's possible to override the default behaviour
by setting these macros via `CPPFLAGS`: `CURL_NORETURN`, `CURL_INLINE`,
`CURL_FORCEINLINE`

Closes #15946
2025-01-09 11:58:14 +01:00
Daniel Stenberg
d1c1c96e0a
curl_trc: repair build with verbose strings disabled
- moved the dummy functions into the C file, made them non-static
- added a Curl_trc_ssls dummy

Closes #15951
2025-01-09 09:23:03 +01:00