By default the API inhibits empty queries and fragments extracted.
Unless this new flag is set.
This also makes the behavior more consistent: without it set, zero
length queries and fragments are considered not present in the URL. With
the flag set, they are returned as a zero length strings if they were in
fact present in the URL.
This applies when extracting the individual query and fragment
components and for the full URL.
Closes#13396
I implemented the IDN functions for macOS and iOS using Unicode
libraries coming with macOS and iOS.
Builds and runs here on macOS 14.2.1. Also verified to load and
run on older macOS version 10.13.
Build requires macOS SDK 13 or equivalent.
Set `-DUSE_APPLE_IDN=ON` CMake option to enable it.
With autotools and other build tools, set these manual options:
```
CPPFLAGS=-DUSE_APPLE_IDN
LIBS=-licucore
```
Completes TODO 1.6.
TODO: add autotools option and feature-detection.
Refs: #5330#5371
Co-authored-by: Viktor Szakats
Closes#13246
The bounds of the size parameter were not specified, and nor was it
specified how to disable the maximum file size check.
The documentation also incorrectly stated that CURLOPT_MAXFILESIZE
always returns CURLE_OK and that CURLOPT_MAXFILESIZE_LARGE only returns
CURLE_OK or CURLE_UNKNOWN_OPTION.
It also did not mention what the default value is, which is zero. This
commit updates the documentation to make note of all these things.
Closes#13372
It's important to set `CURLOPT_NOPROGRESS` to `0` if you want your
transfer callback function, set by `CURLOPT_XFERINFOFUNCTION`, getting
called. To emphasize this to the users, add this to the code example.
Closes#13348
Rename custom target to namespaced (unique) names to avoid colliding
with 3rd-party projects (e.g. libzip) built together with curl.
Reported-by: hammlee96 on github
Fixes#13324Closes#13326
... no need to use an absolute path, that makes the build unncessarily
fail if invoked using a different mount point. managen now takes options
to find the input files.
Update test1478 to provide the dir arguments to managen
Closes#13281
- add `CURL_TRC_READ()` and `CURL_TRC_WRITE()`
- use in generic client writers and readers, as well
as http headers, chunking and websockets
Closes#13223
This time limit the number of files per command to avoid exceeding
limitations of certain OS/shell envs.
Such known env is Windows with the `cmd.exe` shell, which features an
8K command-line length limit to this day.
Allowlisting `UNIX` to have no limit and using a limit of 200 for other
envs to be safe. If there is a way to detect `cmd.exe` and/or we know
which precise envs are sensitive to this, we can tweak these conditions
further.
Even with the low limit, this patch reduces external commands by 200x,
making builds much faster.
Ref: #127622620aa930b (initial)
Ref: #13047f03c85635f (revert)
Reviewed-by: Daniel Stenberg
Closes#13207
- install `mk-ca-bundle.pl` like autotools does.
- generate and install `mk-ca-bundle.1` and `curl-config.1` like
autotools. This fixes tests 1140 and 1173.
Reported-by: Dan Fandrich
Fixes#13194
- add option `BUILD_MISC_DOCS` to control building the above two
manpages. Enabled by default.
- appveyor: stop disabling tests 1140 and 1173.
Reviewed-by: Daniel Stenberg
Closes#13197
Instead of use 'docs/*.md' in dep5. For clarity and avoiding a wide-
matching wildcard.
+ Remove mention of old files from .reuse/dep5
+ add info to .github/dependabot.yml
+ make scripts/copyright.pl warn on non-matching patterns
Closes#13245
This commit updates the optional rustls-ffi librustls dependency from
0.12.0 to 0.13.0. This version is based on the latest available rustls
release (0.23.4).
The breaking API changes from 0.12.0 to 0.13.0 are in API surface unused
by curl, so this is an in-place update without any code changes.
The `RUSTLS.md` documentation is updated to reflect the new version in
use, and to clarify that `cbindgen` isn't required to build `librustls`
- it's only used by developers to update the vendored `rustls.h` header
file maintained upstream.
Closes#13238
This is the step that was not done and caused the 8.7.0 mishap (it
lacked the correctly generated hugehelp file).
Remove the mention of the copyright script as this is verified by a CI
job these days: the REUSE one.
Closes#13216
make -C docs/cmdline-opts install depends on all-am, which in turn
depends on $(MANS), unconditionally defined to be $(man_MANS).
As with CLEANFILES, only add curl.1 to man_MANS when BUILD_DOCS is true
so we don't try to build curl.1 unnecessarily.
Closes#13198
Lots of organizations distribute curl packages to end users. This is a
collection of pointers to where to learn more about curl on and with
each distro.
Assisted-by: Alan Coopersmith
Assisted-by: Andrew Kaster
Assisted-by: Andy Fiddaman
Assisted-by: Arjan van de Ven
Assisted-by: Brian Clemens
Assisted-by: chrysos349 on github
Assisted-by: Dan Fandrich
Assisted-by: Dan McDonald
Assisted-by: Gaelan Steele
Assisted-by: graywolf on github
Assisted-by: Jan Macku
Assisted-by: John Marshall
Assisted-by: Jonathan Perkin
Assisted-by: Kevin Daudt
Assisted-by: Marcus Müller
Assisted-by: Michał Górny
Assisted-by: Outvi V
Assisted-by: Ross Burton
Assisted-by: Sean Molenaar
Assisted-by: Till Wegmüller
Assisted-by: Viktor Szakats
Assisted-by: Winni Neessen
Closes#13178
Remove the PROTOCOLS section from the source files completely and
instead generate them based on the header data in the curldown files.
It also generates TLS backend information for options marked for TLS as
protocol.
Closes#13175
All man pages that are listed to be for TLS now must also specify
exactly what TLS backends the option works for, or use All if they all
work.
cd2nroff makes sure this is done and that the listed backends exist.
Closes#13168
- CURLINFO_TLS_SESSION.md: remove mention of NSS
- CURLINFO_TLS_SSL_PTR.md: remove NSS leftover
- CURLOPT_CAINFO.md: drop mention of backends not supporting this
- CURLOPT_CAPATH.md: wolfSSL also supports this
Closes#13166
The mandatory header now has a mandatory list of protocols for which the
manpage is relevant.
Most man pages already has a "PROTOCOLS" section, but this introduces a
stricter way to specify the relevant protocols.
cd2nroff verifies that at least one protocol is mentioned (which can be
`*`).
This information is not used just yet, but A) the PROTOCOLS section can
now instead get generated and get a unified wording across all manpages
and B) this allows us to more reliably filter/search for protocol
specific manpages/options.
Closes#13166
Use imperative mood consistently for the first sentence describing an
option.
"Set this" instead "tell curl to set" or "this sets..."
Plus some extra cleanups and rephrasing.
Closes#13106
Since it is no longer needed for building tool_hugehelp.c and all the
docs is available in readable markdown format in the tarball, the peeps
that don't want to build the manpage still do good.
Removing it also fixes the complexity of out-of-tree builds when the
curl.1 exists in the source tree.
Letting CMake figure out where libraries are located gives you full
paths. When generating libcurl.pc and curl-config, getting libraries as
full paths is unusual when one expects to get a list of -l<libname>.
To meet expectations, an effort is made to convert the full paths into
-l<libname>, possibly with -L<libdir> before it.
Fixes#6169Fixes#12748Closes#12930
All CR and LF bytes should be stripped, as documented, and all other
bytes are inluded in the data. Starting now, it also excludes null bytes
as they would otherwise also cut the data short.
Reported-by: Simon K
Fixes#13063Closes#13064
Create ASCII version of manpage without nroff
- build src/tool_hugegelp.c from the ascii manpage
- move the the manpage and the ascii version build to docs/cmdline-opts
- remove all use of nroff from the build process
- should make the build entirely reproducible (by avoiding nroff)
- partly reverts 2620aa9 to build libcurl option man pages one by one
in cmake because the appveyor builds got all crazy until I did
The ASCII version of the manpage
- is built with gen.pl, just like the manpage is
- has a right-justified column making the appearance similar to the previous
version
- uses a 4-space indent per level (instead of the old version's 7)
- does not do hyphenation of words (which nroff does)
History
We first made the curl build use nroff for building the hugehelp file in
December 1998, for curl 5.2.
Closes#13047
- update client reader documentation
- client reader, add rewind capabilities
- tell creader to rewind on next start
- Curl_client_reset() will keep reader for future rewind if requested
- add Curl_client_cleanup() for freeing all resources independent of
rewinds
- add Curl_client_start() to trigger rewinds
- move rewind code from multi.c to sendf.c and make part of
"cr-in"'s implementation
- http, move the "resume_from" handling into the client readers
- the setup of a HTTP request is reshuffled to follow:
* determine method, target, auth negotiation
* install the client reader(s) for the request, including crlf
conversions and "chunked" encoding
* apply ranges to client reader
* concat request headers, upgrades, cookies, etc.
* complete request by determining Content-Length of installed
readers in combination with method
* send
- add methods for client readers to
* return the overall length they will generate (or -1 when unknown)
* return the amount of data on the CLIENT level, so that
expect-100 can decide if it want to apply itself
* set a "resume_from" offset or fail if unsupported
- struct HTTP has become largely empty now
- rename `Client_reader_*` to `Curl_creader_*`
Closes#13026
The function that replaces occurances of "--longoption" with "-Z,
--longoption" etc with the proper highlight applied, no longer loops
over the options.
Closes#13041
This fixes miscellaneous typos and duplicated words in the docs, lib
and test comments and a few user facing errorstrings.
Author: RainRat on Github
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Dan Fandrich <dan@coneharvesters.com>
Closes: #13019
- replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to
clarify when and at what level they operate
- send/recv of transfer related data is now done via
`Curl_xfer_send()/Curl_xfer_recv()` which no longer has
socket/socketindex as parameter. It decides on the transfer
setup of `conn->sockfd` and `conn->writesockfd` on which
connection filter chain to operate.
- send/recv on a specific connection filter chain is done via
`Curl_conn_send()/Curl_conn_recv()` which get the socket index
as parameter.
- rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for
naming consistency
- clarify that the special CURLE_AGAIN hangling to return
`CURLE_OK` with length 0 only applies to `Curl_xfer_send()`
and CURLE_AGAIN is returned by all other send() variants.
- fix a bug in websocket `curl_ws_recv()` that mixed up data
when it arrived in more than a single chunk (to be made
into a sperate PR, also)
Added as documented [in
CLIENT-READER.md](5b1f31dfba/docs/CLIENT-READERS.md).
- old `Curl_buffer_send()` completely replaced by new `Curl_req_send()`
- old `Curl_fillreadbuffer()` replaced with `Curl_client_read()`
- HTTP chunked uploads are now formatted in a client reader added when
needed.
- FTP line-end conversions are done in a client reader added when
needed.
- when sending requests headers, remaining buffer space is filled with
body data for sending in "one go". This is independent of the request
body size. Resolves#12938 as now small and large requests have the
same code path.
Changes done to test cases:
- test513: now fails before sending request headers as this initial
"client read" triggers the setup fault. Behaves now the same as in
hyper build
- test547, test555, test1620: fix the length check in the lib code to
only fail for reads *smaller* than expected. This was a bug in the
test code that never triggered in the old implementation.
Closes#12969
The curldown conversion accidentally replaced daniel@haxx.se with
just daniel.se. This reverts back to the proper email address in
the curldown docs as well as in a few other stray places where it
was incorrect (while unrelated to curldown).
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Closes: #12997
Returns 1 if the previous transfer used a proxy, otherwise 0. Useful to
for example determine if a `NOPROXY` pattern matched the hostname or
not.
Extended test 970 and 972
- refs #12397 where it is dicussed how to en-/disable verbose output
of DoH operations
- introducing `struct curl_trc_feat` to track a curl feature for
tracing
- adding `data->state.feat` optionally pointing to the feature a
transfer belongs to
- adding trace functions and verbosity checks on features
- using trace feature in DoH code
- documenting `doh` as feature for `--trace-config`
Closes#12411
- when data arrived in several chunks, the collection into
the passed buffer always started at offset 0, overwriting
the data already there.
adding test_20_07 to verify fix
- debug environment var CURL_WS_CHUNK_SIZE can be used to
influence the buffer chunk size used for en-/decoding.
Closes#12945
