Commit Graph

6719 Commits

Author SHA1 Message Date
Daniel Stenberg
448e71d1b7
header.md: mention Authorization: and Cookie: special treatment
Fixes #16160
Reported-by: deliciouslytyped on github
Closes #16164
2025-02-06 11:40:56 +01:00
Daniel Stenberg
553248f501
libssh2: raise lowest supported version to 1.2.8
Shipped on April 5 2011

Closes #16199
2025-02-06 10:02:29 +01:00
Jay Satiro
3631c24861 docs: better explain multi-part byte range behavior
- Better explain that if the requested range (--range or CURLOPT_RANGE)
  contains multiple ranges then the response contains meta information
  in addition to the requested bytes.

Prior to this change it was noted that a multiple part response was
returned as-is but not what that meant. In particular, meta information
is returned in addition to the requested bytes and that may have been
unexpected.

Reported-by: Ralf A. Timmermann

Fixes https://github.com/curl/curl/issues/16139
Closes https://github.com/curl/curl/pull/16150
2025-02-06 03:09:45 -05:00
Daniel Stenberg
153453eb78
SPONSORS.md: clarify that we don't promise goods or services
Due to past events

Closes #16196
2025-02-05 23:40:24 +01:00
Daniel Stenberg
34cf9d54a4
RELEASE-NOTES: synced
curl 8.12.0 release

Also THANKS
2025-02-05 07:42:54 +01:00
MacKenzie
255e3b64df
docs/HTTPSRR.md: Typo fix AAA -> AAAA
Closes #16173
2025-02-04 23:20:18 +01:00
Daniel Stenberg
1b4e635134
HTTPSRR.md: implement what seems to be deployed
Closes #16158
2025-02-04 10:02:29 +01:00
Edoardo Lolletti
e0225f261e
symbols-in-versions: update version for LIBCURL_VERSION and LIBCURL_VERSION_NUM
Those 2 symbols were available since the first 7.1.1 release

Closes #16141
2025-02-03 20:20:49 +01:00
Calvin Ruocco
dc3252bedd
ws-docs: extend WebSocket documentation
Closes #16118
2025-02-03 20:07:30 +01:00
Viktor Szakats
4f95f32709
INSTALL-CMAKE.md: fix punctuation 2025-01-30 15:51:48 +01:00
Viktor Szakats
9eae606dfd
gitignore: restore explicit lists and fix them
Range matches are not supported with `+`: https://git-scm.com/docs/gitignore

Also sync `.gitignore` file lists with source files and alphasort them.

Follow-up to 5c31c2e670 #16093
Reported-by: Stefan Eissing
Fixes #16112
Fixes #16115
Closes #16116
2025-01-28 14:44:35 +01:00
Martin Harrigan
f6da27052e
docs/cmdline-opts/location.md: fix typos for location flag
Closes #16110
2025-01-28 14:07:39 +01:00
Viktor Szakats
f3920c7924
INTERNALS.md: sync wolfSSL version requirement with source code
`lib/vtls/wolfssl.c` enforces it via a hard #error.

Cherry-picked from #16104
2025-01-28 12:06:41 +01:00
Jay Satiro
4f99efb192 easy: allow connect-only handle reuse with easy_perform
- Detach and disconnect an attached connection before performing.

Prior to this change it was not possible to safely reuse an easy handle
with an attached connection in a second call to curl_easy_perform. The
only known case of this is a connect-only type handle where the
connection was detached when curl_easy_perform returned, only to be
reattached by either curl_easy_send/recv.

This commit effectively reverts 2f8ecd5d and be82a360, the latter of
which treated the reuse as an error. Prior to that change undefined
behavior may occur in such a case.

Bug: https://curl.se/mail/lib-2025-01/0044.html
Reported-by: Aleksander Mazur

Closes https://github.com/curl/curl/pull/16008
2025-01-28 03:27:04 -05:00
Viktor Szakats
5c31c2e670
tidy-up: .gitignore lines mostly
- `.gitignore`: delete, dedupe and move rules upwards.
  Ref: 6389ba87b8 #13311
- `.gitignore`: fix generated test sources.
  Follow-up to 71cf0d1fca #14772
- `.gitignore`: replace exe listings with a wildcard.
- lib: move `setup-*.h` from `EXTRA_DIST` to `CURL_HFILES`.
- `makedebug.bat`: uppercase an argument to match docs.
- GHA/non-native: delete stray env.
  Follow-up to 12a6de2f66 #16043
- sort source lists.

Closes #16093
2025-01-27 20:59:46 +01:00
Daniel Stenberg
cb4cd36fe7
VULN-DISCLOSURE-POLICY: on legacy dependencies
Problems that only trigger using *legacy* dependencies are not
considered security problems.

Closes #16086
2025-01-27 15:48:13 +01:00
Viktor Szakats
b01f44ec09
INSTALL.md: add CMake examples for macOS and iOS
Closes #16095
2025-01-27 12:49:12 +01:00
Daniel Stenberg
e27abfe2f1
version: rename c-ares-rr to asyn-rr
Works better if we later introduce another way to do the asynch RR
resolves (together with the threaded resolver) that does not use c-ares.

Closes #16090
2025-01-27 08:11:44 +01:00
Viktor Szakats
0035ff45c5
build: drop tool_hugehelp.c.cvs, tidy up macros, drop buildconf.bat
Rework the way `tool_hugehelp.c` is included in builds.

After this patch, with `./configure` and CMake `tool_hugehelp.c` is only
compiled when building with manuals enabled. With manuals disabled this
source file is not used anymore. The method is similar to how
8a3740bc8e implemented `tool_ca_embed.c`.
`./configure` always generates it as before, otherwise the build fails.

- winbuild: rework to not need `buildconf.bat`, but automatically use
  `tool_hugehelp.c` if present (e.g. when building from an official
  source tarball) and enable `USE_MANUAL` accordingly.

- `buildconf.bat`: after dropping `tool_hugehelp.c` generation, the only
  logic left was `cp Makefile.dist Makefile`. This allowed to launch
  winbuild builds via GNU Make in a Git repo. Drop this option together
  with the batch file.

- build `libcurltool` without `USE_MANUAL` macro to exclude the manual
  and the dependence on the generator commands. Drop relying on
  `UNITTESTS` for this purpose.
  Follow-up to 96843f4ef7 #16068

- `src/mkhelp.pl`: include `tool_hugehelp.h` before using `USE_MANUAL`
  to have it set in `config-*.h` builds with source tarballs created
  with manual but without zlib.

Closes #16081
2025-01-26 14:22:49 +01:00
Viktor Szakats
516cb98733
docs/HTTPSRR.md: fix typo in option name
Follow-up to 7f4f192f04 #16052
2025-01-26 02:23:23 +01:00
Daniel Stenberg
7f4f192f04
docs/HTTPSRR.md: initial HTTPS RR documentation
Closes #16052
2025-01-25 23:51:01 +01:00
Daniel Stenberg
0d4fdbf15d
asyn-thread: use c-ares to resolve HTTPS RR
Allow building with c-ares and yet use threaded resolver for the main
host A/AAAA resolving:

  `--with-ares` provides the c-ares install path and defaults to use
  c-ares for name resolving

  `--with-threaded-resolver` still uses c-ares in the build (for HTTPS)
  but uses the threaded resolver for "normal" resolves.

It works similarly for cmake: ENABLE_ARES enables ares, and if
ENABLE_THREADED_RESOLVER also is set, c-ares is used for HTTPS RR and
the threaded resolver for "normal" resolves.

HTTPSRR and c-ares-rr are new features return by curl_version_info() and
thus shown by curl -V.

The c-ares-rr feature bit is there to make it possible to distinguish
between builds using c-ares for all name resolves and builds that use
the threaded resolves for the regular name resolves and c-ares for
HTTPSRR only. "c-ares-rr" means it does not use c-ares for "plain" name
resolves.

HTTPSRR support is EXPERIMENTAL only.

Closes #16054
2025-01-25 23:46:14 +01:00
Daniel Stenberg
76f83f0db2
content_encoding: drop support for zlib before 1.2.0.4
zlib 1.2.0.4 was released on 10 August 2003

Closes #16079
2025-01-24 14:04:23 +01:00
Viktor Szakats
7e814c8717
build: fix compiling with GCC 4.x versions
- silence false positive picky warnings.
- avoid "possible noreturn" warnings for standalone tests and examples.
- fix to compile without `#pragma GCC diagnostic push` support.
- fix "#pragma GCC diagnostic not allowed inside functions".

Prerequisite for #15975 that needs GCC 4.4 for the latest pre-built
CeGCC/mingw32ce toolchain for Windows CE.

Cherry-picked from #15975
Closes #16062
2025-01-22 11:26:15 +01:00
Daniel Stenberg
f5f8f2c892
RELEASE-NOTES: synced 2025-01-22 08:09:51 +01:00
Yedaya Katsman
38ad3902bf
docs/TLS-SESSIONS: fix typo, the->they
Closes #16057
2025-01-21 09:23:26 +01:00
Viktor Szakats
12a6de2f66
GHA: add iOS jobs with LibreSSL, enable dependencies for Android via vcpkg
iOS:

- add jobs with autotools, CMake, CMake Xcode generator.
  The Xcode generator is >10x slower than Unix Makefiles. Keep it
  because it's the one recommended by CMake and for having its own
  quirks we may want to know about.
- build, cache and use LibreSSL for these jobs.
  With workaround for an iOS build issue fixed in master.
- make Xcode generator work by explicitly disabling code signing.
- make tests and examples build with the Xcode generator by setting
  `-DMACOSX_BUNDLE_GUI_IDENTIFIER=se.curl`, to avoid
  "Bundle identifier is missing" errors.
- cmake: disable `CURL_USE_PKGCONFIG` by default for Apple device.
- cmake: add `stdc++` library for BoringSSL and AWS-LC, with
  `OPENSSL_USE_STATIC_LIBS=ON` set.
- cmake: add workaround for Xcode generator issue, where it cannot
  handle two targets depending on one custom command. A better fix may
  be dropping `tool_hugehelp.c` and `tool_ca_embed.c` from curltool
  library. For a future PR.

Android:

- add vcpkg to Android jobs, enable dependencies.
  Assisted-by: Tal Regev via #16045
- make vcpkg work with autotools.
- pass `--with-brotli` to autotools to detect the vcpkg-supplied brotli.
- enable BoringSSL for Android and add a job with it.
- silence 457 CMake configure warnings about the Android NDK CMake
  scripts targeting freshly deprecated CMake versions.

These were much more involved than imagined. Basically nothing works out
of the box, and when combined, everything becomes a unique edge case.
autotools builds were a much easier to make work than CMake ones.

Also:

- GHA/non-native: re-sync names to be shorter and more aligned with
  other workflows.
- GHA: add `persist-credentials: false` where missing.

Unresolved issues:

- `OPENSSL_ROOT_DIR` ignored/mis-used when pointing it to LibreSSL.
  CMake seems to prepend the sysroot to the passed absolute directory.
  Found no workaround.
- CMake when combined with Android, both the Google-recommended method
  and the built-in CMake method fail to provide a way to avoid
  `pkg-config` packages at system directories. Failed to find a knob
  that can remove `/usr/include` from the search path. The workaround is
  to disable zstd. (I enabled it by default in this release, maybe
  premature?: f2adb3b6d7 #15431)
  Disabling `pkg-config` doesn't work because vcpkg dependencies do not
  link without it.
- CMake's Xcode generator is slow because each `try_compile()` feature
  check springs a new CMake + Xcode project taking a long time to run,
  just to compile single-liner C files. A known issue, with no solution.
  `-DCMAKE_MACOSX_BUNDLE=OFF` did not help, limiting build types to
  a single one (e.g. `Debug`) also had no effect.
   make | Xcode | GHA run
  :---- | :---- | :--------------------------------------------------------------------
    16s | 2m57s | https://github.com/curl/curl/actions/runs/12866334102/job/35868712426
    23s | 4m13s | https://github.com/curl/curl/actions/runs/12868128013/job/35874212461
    16s | 3m39s | https://github.com/curl/curl/actions/runs/12859073531/job/35849041880
    14s | 2m23s | https://github.com/curl/curl/actions/runs/12858298423/job/35847201313
    15s | 2m36s | https://github.com/curl/curl/actions/runs/12858058492/job/35846669761
    19s | 3m19s | https://github.com/curl/curl/actions/runs/12868919430/job/35876601168

Closes #16043
2025-01-20 22:32:06 +01:00
9cel
c5a736f2c9
docs: document the behavior of -- in the curl command line
Closes #16053
2025-01-20 14:27:05 +01:00
Daniel Stenberg
0f54bfd803
libcurl/opts: do not save files in dirs where attackers have access
libcurl cannot fully protect against attacks where an attacker has write
access to the same directory where it is directed to save files. This is
particularly sensitive if you save files using elevated privileges.

Previously only mentioned in VULN-DISCLOSURE-POLICY.md.

Highlighted-by: Donguk Kim

Closes #16051
2025-01-20 10:34:37 +01:00
Daniel Stenberg
9ce0bed1ff
curl_multi_waitfds.md: tidy up the example
- remove typecast
- shorten comment

Closes #16050
2025-01-19 11:45:25 +01:00
Viktor Szakats
56a74fac47
android: add CI jobs, buildinfo, cmake docs, disable CURL_USE_PKGCONFIG by default
- GHA/non-native: add Android builds, both cmake and autotools,
  both NDK 21 (oldest available) and 35 (newest available)
  https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md
  It comes with a maintenance burden to bump the oldest/latest values
  with CI runner updates.

- cmake: disable `CURL_USE_PKGCONFIG` by default for Android.
  To avoid picking up system package by default.

- build: add `ANDROID-<NDK-LEVEL>` flag to `buildinfo.txt`.
  Also detect NDK level with the CMake built-in build method:
  https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html#cross-compiling-for-android

- INSTALL.md: add CMake build instructions for Android.

- INSTALL.md: make NDK levels consistent in `./configure` example.

Closes #16014
2025-01-17 00:44:11 +01:00
Stefan Eissing
02edae54e8 websocket: fix message send corruption
- Fix a bug in EAGAIN handling when sending frames that led to a
  corrupted last byte of the frame sent.

- Restore sanity to curl_ws_send() behaviour:

  - Partial writes are reported as OK with the actual number of
    payload bytes sent.

  - CURLE_AGAIN is only returned when none of the payload bytes
    (or for 0-length frames, not all of the frame header bytes)
    could be sent.

  - curl_ws_send() now behaves like a common send() call.

- Change 'ws-data' test client to allow concurrent send/recv
  operations and vary frame sizes and repeat count.

- Add DEBUG env var CURL_WS_CHUNK_EAGAIN to simulate blocking
  after a chunk of an encoded websocket frame has been sent.

- Add tests.


Prior to this change data corruption may occur when sending websocket
messages due to two bugs:

1) 3e64569a (precedes 8.10.0) caused a data corruption bug in the last
   byte of frame of large messages.

2) curl_ws_send had non-traditional send behavior and could return
   CURLE_AGAIN with bytes sent and expect the caller to adjust buffer
   and buflen in a subsequent call. That behavior was not documented.


Reported-by: na-trium-144@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/15865
Fixes https://github.com/curl/curl/issues/15865#issuecomment-2569870144
Closes https://github.com/curl/curl/pull/15901
2025-01-16 16:19:07 -05:00
Daniel Stenberg
86f5653721
TODO: write an SMB test server to replace impacket
Closes #15697
Closes #16024
2025-01-16 20:51:00 +01:00
Daniel Stenberg
fb881abd72
docs/EXPERIMENTAL.md: add a mention of HTTPSRR as experimental
Closes #16017
2025-01-16 19:41:42 +01:00
Viktor Szakats
f739a6867b
INSTALL.md: tweak to the MSVC version line [ci skip]
Follow-up to 308437ac53 #15992
2025-01-15 02:33:54 +01:00
Daniel Stenberg
2f8ecd5dbd
CURLOPT_CONNECT_ONLY.md: an easy handle with this option set cannot be reused
Closes #16002
2025-01-14 16:26:34 +01:00
Viktor Szakats
308437ac53
INSTALL.md: document VS2008 and mingw-w64
Ref: #15972
Closes #15992
2025-01-14 15:58:39 +01:00
Viktor Szakats
5474d70c3e
tidy-up: drop parenthesis around return expression
Closes #15990
2025-01-14 12:11:42 +01:00
Stefan Eissing
fa5d5ac1c9
pytest: use httpd/apache2 directly, no apachectl
Since the script 'apachectl' from the httpd project is severly mutilated
on several distros, use the executable httpd/apache2 directly in pytest
runs.

Remove detection of apachectl form autoconf and cmake.

Closes #16000
2025-01-14 11:33:49 +01:00
Daniel Stenberg
862244636e
cmdline-opts/version.md: describe multissl, mention SSLS-EXPORT
Closes #15996
2025-01-14 11:08:24 +01:00
Daniel Stenberg
260b7d54a6
curl_easy_ssls_export/import.md: made for TLS protocols
Which then makes the generated man page also include details about the
specific backends that support this feature.

Follow-up to 515a21f350

Closes #15993
2025-01-13 23:39:31 +01:00
Daniel Stenberg
eb652da1b2
examples: use return according to code style 2025-01-13 09:10:58 +01:00
Viktor Szakats
de3e662ce1
msvc: tidy up _CRT_*_NO_DEPRECATE definitions
Dedupe and migrate MSVC-specific warning suppressions to `curl_setup.h`.
Make cmake set `_CRT_SECURE_NO_DEPRECATE` for examples and standalone
tests, and stop setting `_CRT_NONSTDC_NO_DEPRECATE` for them.

Details:
- drop version guards. On ancient MSVC version these macro are a no-op.
- move to `curl_setup.h` from `config-win32*.h`.
- sync macro values with CMake.
- cmake: stop setting them globally in favour of `curl_setup.h`.
- cmake: re-add these macros to `docs/examples` and `tests/http/clients`,
  which do not use `curl_setup.h`.
- cmake: drop `_CRT_NONSTDC_NO_DEPRECATE` for examples and tests.
  They build fine without.
- update comments.

Closes #15960
2025-01-12 00:34:21 +01:00
Viktor Szakats
fdc588dc10
cmake: deprecate winbuild, add migration guide from legacy build methods
We recommend migrating to CMake from winbuild and Visual Studio project
files. winbuild is deprecated and will be dropped in September 2025.

CMake supports all the features and options, with new ones added
promptly. It supports out-of-tree, unity and documentation builds.

- deprecate winbuild method in favour of CMake by September 2025.
- add migration guide from winbuild to CMake.
- add migration guide from Visual Studio Project Files to CMake.
- add deprecation message to winbuild.
  Need to ack with `WINBUILD_ACKNOWLEDGE_DEPRECATED=yes`
  Authored-by: Jay Satiro
- mention `CMAKE_BUILD_TYPE` option in `INSTALL-CMAKE`.
- document missing `SSH_PATH` winbuild option.

Closes #15920
2025-01-10 18:20:52 +01:00
Viktor Szakats
a19b79ce87
tidy-up: delete stray references from autotools, docs
- autotools: delete stray `VC14_LIB*` references.
- autotools: delete (now) empty `CLEANFILES`.
- autotools: delete no longer used lib/src .inc includes in root makefile.
- autotools: delete stray `cygwinbin` target.
- autotools: delete stray `pkgadd` target (Solaris).
- lib, src: delete stray files from `.gitignore`.
- INSTALL.md: delete reference to non-existing `src/config-win32.h`.
- lib/config-win32ce.h: whitespace.
- lib/config-win32ce.h: sync comments with `config-win32.h`.

Closes #15944
2025-01-09 02:12:53 +01:00
Daniel Stenberg
115ea7633e
DEPRECATE: remove msh3 in six months
The msh3 backed for QUIC and HTTP/3 was introduced in April 2022 but has
never been made to work properly. It has seen no visible traction or
developer activity from the msh3 main author (or anyone else seemingly
interested) in two years. As a non-functional backend, it only adds
friction and "weight" to the development and maintenance.

Meanwhile, we have a fully working backend in the ngtcp2 one and we have
two fully working backends in OpenSSL-QUIC and quiche well on their way
of ending their experimental status in a future.

We remove msh3 support from the curl source tree in July 2025.

Closes #15931
2025-01-08 23:57:36 +01:00
Stefan Eissing
515a21f350
vtls: feature ssls-export for SSL session im-/export
Adds the experimental feature `ssls-export` to libcurl and curl for
importing and exporting SSL sessions from/to a file.

* add functions to libcurl API
* add command line option `--ssl-sessions <filename>` to curl
* add documenation
* add support in configure
* add support in cmake
+ add pytest case

Closes #15924
2025-01-08 23:32:07 +01:00
Yedaya Katsman
e602f7f119
docs/examples/checksrc: don't allow snprintf specifically
This isn't needed anymore after https://github.com/curl/curl/pull/15835,
since banned functions are just allowed in general in
`docs/examples/.checksrc`, and emits a warning when running make
checksrc:

`invalid warning specified in .checksrc: "SNPRINTF"`

Closes #15916
2025-01-06 13:49:21 +01:00
Daniel Stenberg
893e57e84c
INFRASTRUCTURE.md: project infra
Closes #15906
2025-01-05 11:25:06 +01:00
Daniel Stenberg
abf8062449
CURLOPT_SEEKFUNCTION.md: used for FTP, HTTP and SFTP (only)
The same goes for *SEEKDATA.

Closes #15903
2025-01-02 22:31:11 +01:00
Daniel Stenberg
3eb57d6ba7
docs: use lowercase curl and libcurl
Adjusted badwords to find them.

Plus: make badwords run on all markdown files in the repo and update
markdowns previously unchecked

Closes #15898
2025-01-02 17:15:54 +01:00
Daniel Stenberg
e694c8284a
docs/libcurl/opts: clarify the return values
Expand a little.

- mention the type name of the return code
- avoid stating which exact return codes that might be returned, as that
  varies over time, builds and conditions
- avoid stating some always return OK
- refer to the manpage documenting all the return codes

Closes #15900
2025-01-02 17:13:33 +01:00
Daniel Stenberg
e256d9df6d
docs/cmdline-opts/_ENVIRONMENT.md: minor language fix
Closes #15897
2025-01-02 17:02:26 +01:00
Daniel Stenberg
4501b7e28d
docs/libcurl: return value overhall
Unified, extended, clarified the return values for numerous functions

Closes #15899
2025-01-02 16:58:05 +01:00
Viktor Szakats
27b9e76706
cmake: make system libraries dl, m, pthread customizable
via `DL_LIBRARY`, `MATH_LIBRARY`, `PTHREAD_LIBRARY` variables.

They are used in Rustls, wolfSSL Find modules.

Also:
- always use `NAMES` keyword in `find_library()` calls.
- respect `find_library()` results for `dl`, `m`, `pthread`.
- formatting.

Closes #15892
2025-01-02 12:55:50 +01:00
Daniel Stenberg
91587522a1
docs/HTTP-COOKIES.md: link to more information
Closes #15891
2025-01-01 22:45:48 +01:00
Jakub Jelen
999cc818c5
openssl: add support to use keys and certificates from PKCS#11 provider
In OpenSSL < 3.0, the modularity was provided by mechanism called
"engines". This is supported in curl, but the engines got deprecated
with OpenSSL 3.0 in favor of more versatile providers.

This adds a support for OpenSSL Providers, to use PKCS#11 keys, namely
through the pkcs11 provider. This is done using similar approach as the
engines and this is automatically built in when the OpenSSL 3 and newer
is used.

Signed-off-by: Jakub Jelen <jjelen@redhat.com>

Closes #15587
2025-01-01 14:47:31 +01:00
Jay Satiro
6c70ec16c7 http: ignore invalid Retry-After times
- Treat negative Retry-After date-based times as 0.

- Treat Retry-After times greater than 6 hours as 6 hours.

Prior to this change Retry-After did not have a limited range and the
server could have set a time greater than 6 hours or a date in the past
that would result in a negative time, either of which may be unexpected
by the user.

The 6 hour limit is purposely not documented so that it can be changed
in the future if necessary.

Closes https://github.com/curl/curl/pull/15833
2024-12-31 03:19:22 -05:00
Viktor Szakats
1e14e1682c
examples/block-ip: drop redundant memory.h include
Not used elsewhere in the repo.
2024-12-31 00:35:57 +01:00
Daniel Stenberg
c2d37463b9
curl/var: change byte offset syntax for assignments
Follow-up to 40c264db61 after discussions on IRC.

The new style is

  name[0-99]=contents

and

  name[0-99]@filename

A) This does not cause the same problems with old curl versions trying
the new syntax as this way will cause old curls just fail with syntax
error and not risk using the wrong file.

B) Adds the same byte range support for "normal" assigns, which the
previous syntax did not. Thus lets a user get a partial content of a
variable etc.

Added test 790 and 791 to verify non-file assigns with ranges.

Closes #15862
2024-12-30 15:55:54 +01:00
Daniel Stenberg
53fcac438a
TheArtOfHttpScripting.md: rewrite double 'that'
Closes #15863
2024-12-30 14:02:37 +01:00
Stefan Eissing
a2622cdbd5
mbedtls: fix handling of blocked sends
mbedtls is picky when a mbedtls_ssl_write) was previously blocked. It
requires to be called with the same amount of bytes again, or it will
lose bytes, e.g. reporting all was sent but they were not. Remember the
blocked length and use that when set.

Reported-by: Tamás Bálint Misius
Fixes #15801
Closes #15846
2024-12-30 10:20:27 +01:00
Daniel Stenberg
c445b7426a
checksrc: introduce 'banfunc' to ban specific functions
Use 'banfunc' and 'allowfunc' in .checksrc to specify which functions to
ban or allow to be used. This saves us from having to edit the script
going forward when we want to ban or allow specific functions.

This replaces a set of previous rules and all banned functions are now
checked with the BANNEDFUNC rule.

There is a set of default banned functions, shown by invoking
./checksrc.

Also, -a and -b options are added to specify allowed or banned functions
on the command line.

Closes #15835
2024-12-30 08:25:56 +01:00
Daniel Stenberg
66e5351e0a
urlapi: fix redirect to a new fragment or query (only)
The redirect logic was broken when the redirect-to URL was a relative
URL only as a fragment or query (starting with '#' or '?').

Extended test 1560 to reproduce, then verify.

Reported-by: Jeroen Ooms
Fixes #15836
Closes #15848
2024-12-30 08:23:26 +01:00
Christopher Dannemiller
c78044c07e multi: fix curl_multi_waitfds reporting of fd_count
- Make curl_multi_waitfds consistent with the documentation.

Issue Addressed:

 - The documentation of curl_multi_waitfds indicates that users should
   be able to call curl_multi_waitfds with a NULL ufds. However, before
   this change, the function would return CURLM_BAD_FUNCTION_ARGUMENT.
 - Additionally, the documentation suggests that users can use this
   function to determine the number of file descriptors (fds) needed.
   However, the function would stop counting fds if the supplied fds
   were exhausted.

Changes Made:

 - NULL ufds Handling: curl_multi_waitfds can now accept a NULL ufds if
   size is also zero.
 - Counting File Descriptors: If curl_multi_waitfds is passed a NULL
   ufds, or the size of ufds is insufficient, the output parameter
   fd_count will return the number of fds needed. This value may be
   higher than actually needed but never lower.

Testing:

 - Test 2405 has been updated to cover the usage scenarios described
   above.

Fixes https://github.com/curl/curl/issues/15146
Closes https://github.com/curl/curl/pull/15155
2024-12-29 01:05:09 -05:00
Jay Satiro
39e21794a7 cookie: fix crash in netscape cookie parsing
- Parse the input string without modifying it.

Prior to this change a segfault could occur if the input string was
const because the tokenizer modified the input string. For example if
the user set CURLOPT_COOKIELIST to a const string then libcurl would
likely cause a crash when modifying that string. Even if the string was
not const or a crash did not occur there was still the incorrect and
unexpected modification of the user's input string.

This issue was caused by 30da1f59 (precedes 8.11.0) which refactored
some options parsing and eliminated the copy of the input string. Also,
an earlier commit f88cc654 incorrectly cast the input pointer when
passing it to strtok.

Co-authored-by: Daniel Stenberg

Closes https://github.com/curl/curl/pull/15826
2024-12-27 13:16:08 -05:00
Viktor Szakats
fabfa8e402
clang-tidy: add to CI, add cmake support, fix fallouts
build:
- autotools: fix to build generated sources for the `tidy` target.
- autotools: allow passing custom clang-tidy options via
  `CURL_CLANG_TIDYFLAGS` env.
- cmake: add `CURL_CLANG_TIDY` option to configure for `clang-tidy`.
  Also add:
  - `CLANG_TIDY` variable to customize the `clang-tidy` tool.
  - `CURL_CLANG_TIDYFLAGS` to pass custom options to `clang-tidy`.
- apply `--enable-werror` and `-DCURL_WERROR=ON` to `clang-tidy`.

CI/GHA:
- add clang-tidy job for Linux, using autotools and clang-tidy v18.
  This one needs to disable `clang-analyzer-valist.Uninitialized`
  to avoid false positives:
  https://github.com/llvm/llvm-project/issues/40656
  Duration: 5.5 minutes
- add clang-tidy job for macOS, using cmake and clang-tidy v19.
  This one also covers tests and examples, and doesn't hit the false
  positives seen with llvm v18 and earlier.
  Duration: 4.5 minutes
- Linux/macOS: skip installing test dependencies when not building or
  running tests.

fix fallouts reported by `clang-tidy`:
- lib:
  - cf-h2-proxy: unused assignment in non-debug builds.
  - cf-socket: silence warning.
    FIXME: https://github.com/curl/curl/pull/15825#issuecomment-2561867769
  - ftp: NULL passed to `strncmp()`.
  - http2: NULL-ptr deref.
  - mprintf: silence warning.
- src/tool_writeout: NULL passed to `fputs()`.
- examples:
  - invalid file pointers.
  - missing `fclose()`.
- tests:
  - http/clients/hx-download: memory leaks on error.
  - http/clients/hx-download: memory leak on repeat `-r` option.
  - server: double `fclose()`.
    https://www.man7.org/linux/man-pages/man3/fclose.3.html
  - server: invalid file pointer/handle.
  - server/getpart: unused assignments.
  - server/mqttd: leak on failed `realloc()`.
  - server/tftpd: NULL passed to `strcmp()`.

Closes #15825
2024-12-27 13:42:32 +01:00
Viktor Szakats
421e592db2
cmake: add librtmp Find module
The new detection method also allows to enable librtmp without using
OpenSSL as a curl TLS backend at the same time.

Also:
- implement manual version detection for librtmp.
  Version info is in hex. With CMake 3.13 and newer, extract it as a hex
  number. With earlier CMake version, just strip the leading zeroes.
  Doing more here seems overkill because librtmp has been standing
  at 2.3/2.4 for a decade now. Bumping into hex digits seems unlikely
  before deprecating CMake 3.13 support.
  librtmp advertises v2.4 via its `pkg-config` module, and v2.3 via
  its public header. The latter shows up in `curl -V` and either can
  be shown at configure-time depending on detection method.
  This isn't a curl bug.
- GHA/macos: enable rtmp in a job.
- apply the "half-detection" fix to the Find module.
  `librtmp` is also affected (in CI too), because it depends on libssl and
  libcrypto.

Closes #15832
2024-12-27 12:52:23 +01:00
Jay Satiro
5e1700d167 curl_ws_recv.md: fix typo 2024-12-27 00:37:53 -05:00
Viktor Szakats
7a70b46481
tidy-up: misc
- lib1531: fix to use `curl_off_t`.
- src/tool_doswin: merge `_WIN32` guards.
- src/tool_doswin: limit `S_ISCHR()` to MS-DOS.
- src: dedupe `CURL_O_BINARY` macro into `tool_setup.h`.
- CMake/CurlTests.c: cleanup/formatting `HAVE_FILE_OFFSET_BITS` test.
- examples/httpput-postfields: delete unused include.
- spelling: MultiSSL.
- whitespace.

Ref: #15652
Closes #15799
2024-12-24 01:27:26 +01:00
Daniel Stenberg
822971170c
curl_url_set.md: adjust the added-in to 7.62.0
Reported-by: Jeroen Ooms
Fixes #15822
Closes #15823
2024-12-24 00:42:38 +01:00
Daniel Stenberg
9d5ecc9613
getinfo: provide info which auth was used for HTTP and proxy
CURLINFO_HTTPAUTH_USED and CURLINFO_PROXYAUTH_USED

Tested in 590 and 694

Ref: #12668
Idea-by: Ganesh Viswanathan
Closes #15450
2024-12-23 23:03:54 +01:00
Daniel Stenberg
0ebd286aa3
CURLOPT_PROXY.md: clarify the crendential support in proxy URLs
Ref: #15802
Closes #15805
2024-12-23 11:52:42 +01:00
Jay Satiro
a9d881c725 examples/block-ip: show how to block IP addresses
- Show how CURLOPT_OPENSOCKETFUNCTION can be used to block IP addresses.

This is a new example that demonstrates IP blocking.

Ref: https://github.com/curl/curl/discussions/15710#discussioncomment-11534877

Closes https://github.com/curl/curl/pull/15748
2024-12-22 03:11:19 -05:00
Daniel Stenberg
31c6b50cb2
curl: make --etag-save acknowledge --create-dirs
Add test 693 to verify

Fixes #15730
Suggested-by: Tamir Duberstein
Closes #15732
2024-12-21 14:12:16 +01:00
Daniel Stenberg
33ea71ea3a
TODO: sending only part of --data is now possible
Since 40c264db61

Closes #15794
2024-12-21 12:30:39 +01:00
Stefan Eissing
b06b35154f
getinfo: fix CURLINFO_QUEUE_TIME_T and add 'time_queue' var
Let CURLINFO_QUEUE_TIME_T count only the time a transfer spends queued,
including possible redirect requests.

Add var 'time_queue' for reporting the time in write outs.

Add test for verifying correct reporting.

Closes #15512
2024-12-21 12:04:48 +01:00
Daniel Stenberg
40c264db61
curl: add byte range support to --variable reading from file
Allowing --variable read a portion of provided files, makes curl work on
partial files for any options that accepts strings. Like --data and others.

The byte offset is provided within brackets, with a semicolon separator
like: --variable name@file;[100-200]"

Inspired by #14479
Assisted-by: Manuel Einfalt

Test 784 - 789. Documentation update provided.

Closes #15739
2024-12-21 11:46:27 +01:00
Daniel Stenberg
fc3e1cbc50
hyper: drop support
lib : remove all hyper code
configure: stop detecting hyper
docs: no more mention of hyper
tests: mo more special-handling of hyper builds
CI: no jobs using hyper

Closes #15120
2024-12-21 11:33:05 +01:00
Viktor Szakats
46093d9e0e
examples: make them compile with compatibility functions disabled (Windows)
For MinGW this is `-DNO_OLDNAMES`, with MSVC it is
`-D_CRT_DECLARE_NONSTDC_NAMES=0`.

There have been some support for this before this patch.
After this patch this is extended to all examples.

(And also the standalone http/client programs, if here.)

Cherry-picked from #15652
Closes #15789
2024-12-20 17:56:40 +01:00
Viktor Szakats
37fb50a858
examples/complicated: fix warnings, bump deprecated callback, tidy up
Also: make them C89, add consts.

Closes #15785
2024-12-20 17:56:35 +01:00
Stefan Eissing
fa0ccd9f1f
lib: TLS session ticket caching reworked
Described in detail in internal doc TLS-SESSIONS.md

Main points:
- use a new `ssl_peer_key` for cache lookups by connection filters
- recognize differences between TLSv1.3 and other tickets
  * TLSv1.3 tickets are single-use, cache can hold several of them for a peer
  * TLSv1.2 are reused, keep only a single one per peer
- differentiate between ticket BLOB to store (that could be persisted) and object instances
- use put/take/return pattern for cache access
- remember TLS version, ALPN protocol, time received and lifetime of ticket
- auto-expire tickets after their lifetime

Closes #15774
2024-12-20 14:59:23 +01:00
Daniel Stenberg
ed732e3596
examples/synctime.c: remove references to dead URLs and functionality
This example can use the Date: header of any server so there is no point
in linking to ancient URLs describeing a setup at NIST that no longer
exists.

Closes #15786
2024-12-20 13:39:12 +01:00
Daniel Stenberg
cfb97e1fcf
VULN-DISCLOSURE-POLICY.md: mention the not setting CVSS
Closes #15779
2024-12-19 22:59:54 +01:00
dwickr
c5052a0187 CURLOPT_RESOLVE.md: fix wording
(`*`) It's an asterisk, not an ampersand.

Closes https://github.com/curl/curl/pull/15770
2024-12-17 23:14:12 -05:00
Viktor Szakats
f2adb3b6d7
cmake: extend zlib's AUTO option to brotli, zstd and enable if found
- make `curl_dependency_option()` more generic.

- extend `CURL_BROTLI` and `CURL_ZSTD` options to accept
  `AUTO` in addition to existing `ON` and `OFF`.

- change `CURL_BROTLI` and `CURL_ZSTD` option default
  to `AUTO`. Was: `OFF`.
  It brings cmake behavior closer to `./configure`.
  Still different:
  - `./configure` defaults to `off` which means to check default
    locations. cmake checks more locations by default.
    (Also tried `NO_CMAKE_PATH`, but then it checked less locations.)
  - cmake returns both `brotlicommon` and `brotlidec` libs,
    while `./configure` only returns the latter.

- ci: drop explicit cmake options, that are now unnecessary.

- GHA/configure-vs-cmake: make adjustments to make tests pass.

Closes #15431
2024-12-17 04:06:45 +01:00
Viktor Szakats
290607f9d4
Revert: cmake: enable CURL_USE_PKGCONFIG for cross-MINGW
This reverts commit 39c06f7883 #15005.

Combined with most Find modules now supporting `pkg-config`
(39c741b7b0 #15408) this change made
mingw-cross builds fragile by picking up OS-native components. Also
adding `/usr/include` to the header path, confusing feature detection.
2024-12-17 03:21:22 +01:00
Viktor Szakats
a3585c9576
Makefile.mk: drop in favour of autotools and cmake (MS-DOS, AmigaOS3)
`Makefile.mk` supported MS-DOS and Amiga, but `./configure` also
supported them in a better tested and more flexible way.

This patch also adds CMake support for MS-DOS/DJGPP and Amiga OS 3.

`Makefile.mk` was not maintained. Delete it in favour of first-tier
build methods.

Also include some non-MS-DOS/AmigaOS-specific tidy-up, see details at
the end of this message.

Details:

- fix/silence all MS-DOS/DJGPP build warnings and issues.
- add MS-DOS support to cmake.
  - default to `ENABLE_THREADED_RESOLVER=OFF` for MS-DOS.
  - add support for `WATT_ROOT`.
  - use static libcurl with MS-DOS.
  - fixup default CMake suffixes/prefixes for DJGPP.
  - disable hidden symbols for MS-DOS. Not supported on MS-DOS.
  - opt-in MS-DOS into `USE_UNIX_SOCKETS`.
- improve MS-DOS support in autotools.
  - default to `--disable-threaded-resolver` for MS-DOS.
- make sure to use `close_s()` (from Watt-32) with autotools and cmake.
  `Makefile.mk` used it before this patch.
- GHA: add DJGPP cmake (~30s) and autotools (~60s) build jobs.
  Also build tests and examples with cmake.
- improve AmigaOS support in autotools:
  - configure: detect `CloseSocket()` when it's a macro.
  - configure: fix `IoctlSocket` detection on AmigaOS.
  - curl-amissl.m4: pass AmiSSL libs to tests/servers.
- add AmigaOS3 support to cmake:
  - cmake: fix `HAVE_IOCTLSOCKET_CAMEL` and
    `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` detections.
  - set necessary system libs.
  - add AmiSSL support.
  - inet_ntop, inet_pton: fix using it for AmigaOS. cmake detects them,
    and they did not compile with AmigaOS.
  - cmake: better sync `gethostname` detection with autotools.
    Fixes detection for AmigaOS, where `gethostname` is a macro.
  - cmake: fix `sys/utime.h` detection on AmigaOS.
  - cmake: force-disable `getaddrinfo` for AmigaOS.
  - cmake: tweak threading and static/shared default for AmigaOS.
  - cmake: rely on manual variable `AMIGA` to enable the platform.
- GHA: add AmigaOS cmake and autotools (~45s) jobs.
  Also build tests and examples with cmake.
- INSTALL: update MS-DOS and AmigaOS build instructions.
- amigaos: fix `-Wpointer-sign` and
  `zero or negative size array '_args'` in `Printf()`.
- amigaos: fix `-Wpointer-sign`
- amigaos: fix `-Wredundant-decls` `errno` and `h_errno`.
- amigaos: brute-force silence `lseek()` size warnings.
- amigaos: server/resolve: silence `-Wdiscarded-qualifiers`.
- amigaos: server/resolve: fix `-Wpointer-sign`.
- amigaos: fix `CURL_SA_FAMILY_T` type.
- nonblock: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
  `ioctl` is also detected, but fails when used. Make the above override
  it for a successful build.
  Authored-by: Darren Banfi
  Fixes #15537
  Closes #15603
- tftpd: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
- tftpd: tidy-up conditional code.
- curl: set stack size to 16384 for AmigaOS3/4
  Overriding the default 4096.
  Suggested-by: Darren Banfi
  Ref: https://github.com/curl/curl/pull/15543#issuecomment-2498783123
  Ref: https://wiki.amigaos.net/wiki/Controlling_Application_Stack
- functypes.h: fix `SEND_QUAL_ARG2` for AmigaOS.
- tftp: add missing cast in sendto() call for AmigaOS.
- getinfo: fix warning with AmigaOS.
- tool_operate: silence warning with AmigaOS
- amigaos: fix building libtests due to missing `RLIMIT_NOFILE`.
- curl_gethostname: silence warning for AmigaOS.
- ftp: silence `-Wtype-limits` for AmigaOS.
- libtest: fix timeval initialization for AmigaOS.
- examples: fix `timeval` initialization for AmigaOS.
- examples: silence warning for AmigaOS.
- configure: fix IPv6 detection for cross-builds.
- netrc: fix to build with AmigaOS cleanly.
- buildinfo: detect and add `DOS` tag for MS-DOS builds.
- buildinfo: add `AMIGA` to buildinfo.txt in auttools.
- build: move `USE_WATT32` macro definition to cmake/configure.

Non-MS-DOS/AmigeOS-specific tidy-ups:

- configure: sync `sa_family_t` detection with cmake.
- configure: sync `ADDRESS_FAMILY` detection signals with cmake.
- doh: use `CURL_SA_FAMILY_T`.
- lib: drop mingw-specific `CURL_SA_FAMILY_T` workaround.
- cmake: extend instead of override check-specific
  configurations/requirements.
  This allows to honor global requirements added earlier.
  Necessary for AmigaOS for example.
- cmake: omit warning on disabled IPv6 for MS-DOS and AmigaOS.
  No IPv6 support on these platforms. Also sync with autotools.
- lib1960: use libcurl `inet_pton()` wrapper.
- cmake: detect LibreSSL (to match autotools).
- cmake: say the specific OpenSSL flavour detected.
- hostip: add missing `HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID` guard.
- lib: simplify classic mac feature guards.

Follow-up to a8861b6ccd #9764

Closes #15543
2024-12-16 23:20:55 +01:00
Daniel Stenberg
607bec04d9
INSTALL.md: sync with my brag slide "104 operating systems"
Minoca, Nintendo 3DS and Maemo were missing.

Rename OS/400 to IBM I

Closes #15755
2024-12-16 23:09:11 +01:00
Viktor Szakats
39c06f7883
cmake: enable CURL_USE_PKGCONFIG for cross-MINGW
Enable `CURL_USE_PKGCONFIG` by default for MinGW cross-builds.

Note: This may cause fallouts in certain envs where `pkg-config` picks
up native packages.

Follow-up to e1ab01d1bd #14658
Follow-up to c555ab469d #14575

Closes #15005
2024-12-16 22:16:06 +01:00
Viktor Szakats
b9895b9d3f
cmake: namespace functions and macros
Prefix (or suffix) curl-defined macro and function names with `curl`.
To avoid collisions with non-curl CMake and to make them recognizable
as curl-specific calls.

Closes #15498
2024-12-16 21:55:00 +01:00
Viktor Szakats
b4aa93114c
cmake: replace unset(VAR) with set(VAR "") for init
Also add cleanup `unset()`s where missing.

Reported-by: Kai Pastor
Bug: https://github.com/curl/curl/pull/15255#issuecomment-2458659626
Follow-up to 8b09138083 #14610

Closes #15497
2024-12-16 21:12:51 +01:00
Daniel Stenberg
0af5ce162f
variable.md: mention --expand-variable for variables to variables
To assign a variable using contents from another variable.

Closes #15752
2024-12-16 08:17:22 +01:00
Daniel Stenberg
6eb86e428d
ws-docs: remove the outdated texts saying ws support is experimental
It stopped being experimental in 8.10.0

Fixes #15749
Reported-by: Mohammed Sadiq
Closes #15751
2024-12-15 18:14:52 +01:00
Daniel Stenberg
9294ca8916
variable.md: show function use with examples
Closes #15743
2024-12-14 17:00:23 +01:00
Daniel Stenberg
cd80716cb6
docs/BUGS.md: remove leading space from a link 2024-12-14 14:22:38 +01:00
Daniel Stenberg
7347ddc9e2
tool_operate: make --etag-compare always accept a non-existing file
Consider it a blank etag. It allows for more use cases when the file
just might not have been created yet.

Closes #15737
2024-12-13 15:48:27 +01:00
Daniel Stenberg
a300879b63
curl: return error if etag options are used with multiple URLs
And document it.

Add tests 484 and 485

Fixes #15729
Reported-by: Tamir Duberstein
Closes #15731
2024-12-13 10:11:26 +01:00
Daniel Stenberg
c2ac9ea1ee
checksrc: ban use of sscanf()
Using sscanf() is not a (security) problem in itself, but we strongly
discorage using it for parsing input since it is hard to use right, easy
to mess up and often makes for sloppy error checking.

Allow it in examples and tests

Closes #15687
2024-12-13 09:43:05 +01:00
Daniel Stenberg
d5c738c608
strparse: string parsing helper functions
Designed to aid converting off from sscanf parsers. sscanf is hard to
use right, easy to mess up and often makes for sloppy error checking.

The new parsers allow more exact and pedandic parsing.

This new set of functions should be possible to use (and extend) and
switch over other libcurl parser code to use going forward.

Adapts the following to use the new functions:

- altsvc.c
- hsts.c
- http_aws_sigv4.c

Bonus: fewer memory copies, fewer stack buffers.

Test: Unit test1664

Docs: docs/internals/STRPARSE.md

Closes #15692
2024-12-12 16:00:52 +01:00