Commit Graph

2707 Commits

Author SHA1 Message Date
Daniel Stenberg
8d7cda1f92
curl: return error when asked to use an unsupported HTTP version
When one of the following options are used but the libcurl in use does
not support it:

--http2
--http2-prior-knowledge
--proxy-http2

Closes #11440
2023-07-15 19:02:20 +02:00
Daniel Stenberg
e53df4cf08
tool_parsecfg: accept line lengths up to 10M
Bumped from 100K set in 47dd957daf

Reported-by: Antoine du Hamel
Fixes #11431
Closes #11435
2023-07-13 15:16:41 +02:00
Emanuele Torre
8484ad09da
tool_writeout_json: fix encoding of control characters
Control characters without a special escape sequence e.g. %00 or %06
were being encoded as "u0006" instead of "\u0006".

Ref: https://github.com/curl/trurl/pull/214#discussion_r1257487858
Closes #11414
2023-07-09 18:57:01 +02:00
Brian Nixon
933aedcde8
tool_easysrc.h: correct easysrc_perform for CURL_DISABLE_LIBCURL_OPTION
Closes #11398
2023-07-09 18:50:16 +02:00
Stefan Eissing
f06cc4f85e
tool: add curl command line option --trace-ids
- added and documented --trace-ids to prepend (after the timestamp)
  the transfer and connection identifiers to each verbose log line
- format is [n-m] with `n` being the transfer id and `m` being the
  connection id. In case there is not valid connection id, print 'x'.
- Log calls with a handle that has no transfer id yet, are written
  without any ids.

Closes #11185
2023-06-12 23:53:09 +02:00
Stefan Eissing
e024d5665d
lib: add CURLINFO_CONN_ID and CURLINFO_XFER_ID
- add an `id` long to Curl_easy, -1 on init
- once added to a multi (or its own multi), it gets
  a non-negative number assigned by the connection cache
- `id` is unique among all transfers using the same
  cache until reaching LONG_MAX where it will wrap
  around. So, not unique eternally.
- CURLINFO_CONN_ID returns the connection id attached to
  data or, if none present, data->state.lastconnect_id
- variables and type declared in tool for write out

Closes #11185
2023-06-12 23:53:00 +02:00
Daniel Stenberg
9ec099a2bc
tool_operate: allow cookie lines up to 8200 bytes
Since this option might set multiple cookies in the same line, it does
not make total sense to cap this at 4096 bytes, which is the limit for a
single cookie name or value.

Closes #11303
2023-06-12 14:15:37 +02:00
Viktor Szakats
3f8fc25720
cmake: add support for "unity" builds
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.

It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.

Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.

Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
  These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
  `statename`.
- rename `ssl_backend_data` structure to unique names for each
  TLS-backend, along with the `ssl_connect_data` struct member
  referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
  builds when compiling `lib/ldap.c`. To support "unity" builds, we had
  to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
  `schannel.h` option) _globally_. This caused an indirect inclusion of
  Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
  as well. This requires `[P]UNICODE_STRING` types, which is apperantly
  not defined automatically (as seen with both MSVS and mingw-w64).
  This patch includes `<subauth.h>` to fix it.
  Ref: https://github.com/curl/curl/runs/13987772013
  Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
  trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
  is built in static mode.

KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
  https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250

Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes #11095
2023-06-07 13:06:08 +00:00
Boris Verkhovskiy
b832cab112
tool_getparam: fix comment
Closes #11253
2023-06-05 20:10:36 +02:00
Raito Bezarius
0a75964d0d
haproxy: add --haproxy-clientip flag to spoof client IPs
CURLOPT_HAPROXY_CLIENT_IP in the library

Closes #10779
2023-06-05 20:08:37 +02:00
Daniel Stenberg
9ad23c38e5
curl: add --ca-native and --proxy-ca-native
These are two boolean options to ask curl to use the native OS's CA
store when verifying TLS servers. For peers and for proxies
respectively.

They currently only have an effect for curl on Windows when built to use
OpenSSL for TLS.

Closes #11049
2023-06-05 20:07:11 +02:00
Daniel Stenberg
fff65550bc
CURLOPT_MAIL_RCPT_ALLOWFAILS: replace CURLOPT_MAIL_RCPT_ALLLOWFAILS
Deprecate the name using three Ls and prefer the name with two.

Replaces #10047
Closes #11218
2023-06-03 23:29:04 +02:00
Daniel Stenberg
bfa7006424
test447: test PUTting a file that grows
... and have curl trim the end when it reaches the expected total amount
of bytes instead of over-sending.

Reported-by: JustAnotherArchivist on github
Closes #11223
2023-06-01 13:43:31 +02:00
Daniel Stenberg
1f85420a28
curl: count uploaded data to stop at the originally given size
Closes #11223
Fixes #11222
Reported-by: JustAnotherArchivist on github
2023-06-01 13:43:28 +02:00
Daniel Stenberg
6661bd588d
tool: remove exclamation marks from error/warning messages 2023-06-01 08:19:21 +02:00
Daniel Stenberg
741f7ed4bc
tool: use errorf() for error output
Convert a number of fprintf() calls.
2023-06-01 08:19:11 +02:00
Daniel Stenberg
6d45b9ca9c
tool: remove newlines from all helpf/notef/warnf/errorf calls
Make voutf() always add one.

Closes #11226
2023-06-01 08:18:21 +02:00
Daniel Stenberg
0807fd72f9
tool_urlglob: use curl_off_t instead of longs
To handle more globs better (especially on Windows)

Closes #11224
2023-05-31 08:14:41 +02:00
Stefan Eissing
64dedb45b5
curl: cache the --trace-time value for a second
- caches HH:MM:SS computed and reuses it for logging during
  the same second.
- common function for plain log line start formatting

Closes #11211
2023-05-28 10:22:53 +02:00
Daniel Stenberg
127eb0d83a
misc: fix spelling mistakes
Reported-by: musvaage on github
Fixes #11171
Closes #11172
2023-05-23 10:42:09 +02:00
Jay Satiro
6f93d5f604
lib: fix conversion warnings with gcc on macOS 2023-05-21 14:02:31 +02:00
Emanuele Torre
f198d33e8d
checksrc: disallow spaces before labels
Out of 415 labels throughout the code base, 86 of those labels were
not at the start of the line. Which means labels always at the start of
the line is the favoured style overall with 329 instances.

Out of the 86 labels not at the start of the line:
* 75 were indented with the same indentation level of the following line
* 8 were indented with exactly one space
* 2 were indented with one fewer indentation level then the following
  line
* 1 was indented with the indentation level of the following line minus
  three space (probably unintentional)

Co-Authored-By: Viktor Szakats

Closes #11134
2023-05-18 20:45:04 +02:00
Daniel Stenberg
39a33fcac0
tool_operate: refuse (--data or --form) and --continue-at combo
libcurl assumes that a --continue-at resumption is done to continue an
upload using the read callback and neither --data nor --form use
that and thus won't do what the user wants. Whatever the user wants
with this strange combination.

Add test 426 to verify.

Reported-by: Smackd0wn on github
Fixes #11081
Closes #11083
2023-05-08 14:10:44 +02:00
Jay Satiro
30ba26d9c3 tool_cb_hdr: Fix 'Location:' formatting for early VTE terminals
- Disable hyperlink formatting for the 'Location:' header value in VTE
  0.48.1 and earlier, since it is buggy in some of those versions.

Prior to this change those terminals may show the location header value
as gibberish or show it twice.

Ref: https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda#backward-compatibility

Fixes https://github.com/curl/curl/issues/10428
Closes https://github.com/curl/curl/pull/11071
2023-05-05 02:52:23 -04:00
Daniel Stenberg
9ce7eee070
checksrc: find bad indentation in conditions without open brace
If the previous line starts with if/while/for AND ends with a closed
parenthesis and there's an equal number of open and closed parentheses
on that line, verify that this line is indented $indent more steps, if
not a cpp line.

Also adjust the fall-out from this fix.

Closes #11054
2023-04-28 23:11:00 +02:00
Emanuele Torre
7f712399d5
checksrc: check for spaces before the colon of switch labels
Closes #11047
2023-04-27 23:26:50 +02:00
Daniel Stenberg
d567cca1de
checksrc: fix SPACEBEFOREPAREN for conditions starting with "*"
The open paren check wants to warn for spaces before open parenthesis
for if/while/for but also for any function call. In order to avoid
catching function pointer declarations, the logic allows a space if the
first character after the open parenthesis is an asterisk.

I also spotted what we did not include "switch" in the check but we should.

This check is a little lame, but we reduce this problem by not allowing
that space for if/while/for/switch.

Reported-by: Emanuele Torre
Closes #11044
2023-04-27 17:24:47 +02:00
Daniel Stenberg
8ff820388f
src/tool_operhlp.c: fix value stored to 'uerr' is never read
Ref: https://github.com/curl/curl/pull/10974#issuecomment-1510461343
Reported-by: Viktor Szakats
Closes #10982
2023-04-17 09:07:07 +02:00
Daniel Stenberg
54ac447b11
curl: do NOT append file name to path for upload when there's a query
Added test 425 to verify.

Reported-by: Dirk Rosenkranz
Bug: https://curl.se/mail/archive-2023-04/0008.html
Closes #10969
2023-04-15 10:52:33 +02:00
Daniel Stenberg
e39754f6a2
curl: add --proxy-http2
For trying HTTP/2 with an HTTPS proxy.

Closes #10926
2023-04-14 10:39:23 +02:00
Daniel Stenberg
808cb31756
tool_writeout: add URL component variables
Output specific components from the used URL. The following variables
are added for this purpose:

  url.scheme, url.user, url.password, url.options, url.host, url.port,
  url.path, url.query, url.fragment, url.zoneid

Add the following for outputting parts of the "effective URL":

 urle.scheme, urle.user, urle.password, urle.options, urle.host, urle.port,
 urle.path, urle.query, urle.fragment, urle.zoneid

Added test 423 and 424 to verify.

Closes #10853
2023-04-04 14:42:44 +02:00
Daniel Stenberg
c0a9f905e6
tool_operate: pass a long as CURLOPT_HEADEROPT argument
Closes #10798
2023-03-22 13:51:37 +01:00
Daniel Stenberg
e2452cf60e
tool_getparam: error if --next is used without a prior URL
Reported-by: 積丹尼 Dan Jacobson
Ref: https://github.com/curl/curl/pull/10769#discussion_r1137895629

Closes #10782
2023-03-17 14:07:10 +01:00
Marcel Raad
ff5c3455ce
tool_operate: silence unused parameter warning
`global` is only used in the `my_setopt` macro version without
`CURL_DISABLE_LIBCURL_OPTION` since commit 4774decf10a.

Closes https://github.com/curl/curl/pull/10752
2023-03-13 13:43:26 +01:00
Jay Satiro
2f17a9b654 tool: improve --stderr handling
- freopen stderr with the user-specified file (--stderr file) instead of
  using a separate 'errors' stream.

- In tool_setup.h override stdio.h's stderr macro as global variable
  tool_stderr.

Both freopen and overriding the stderr macro are necessary because if
the user-specified filename is "-" then stdout is assigned to
tool_stderr and no freopen takes place. See the PR for more information.

Ref: https://github.com/curl/curl/issues/10491

Closes https://github.com/curl/curl/pull/10673
2023-03-12 00:58:40 -05:00
Viktor Szakats
079079b2fd
src: silence wmain() warning for all build methods
llvm/clang and gcc doesn't recognize the wmain() function in Unicode
Windows builds:

llvm/clang:
```
../../src/tool_main.c:239:5: warning: no previous prototype for function 'wmain' [-Wmissing-prototypes]
int wmain(int argc, wchar_t *argv[])
    ^
1 warning generated.
```

gcc:
```
../../src/tool_main.c:239:5: warning: no previous prototype for 'wmain' [-Wmissing-prototypes]
  239 | int wmain(int argc, wchar_t *argv[])
      |     ^~~~~
```

Before this patch, we already silenced it with CMake. This patch moves
the silencing to the source, so that it applies to all build tools.

Bug: https://github.com/curl/curl/issues/7229#issuecomment-1464806651

Reviewed-by: Marcel Raad
Closes #10744
2023-03-11 15:21:43 +00:00
Daniel Stenberg
842be672c8
tool_writeout_json. fix the output for duplicate header names
Header entries with index != 0 are handled at the index 0 level so they
should then be skipped when iterated over.

Reported-by: Boris Okunskiy
Fixes #10704
Closes #10707
2023-03-09 00:13:41 +01:00
Grisha Levit
1dd929674e
tool: dump headers even if file is write-only
The fixes in #10079 brought a (seemingly unrelated) change of open mode
from `wb`/`ab` to `wb+`/`ab+` for the headerfile. This makes it no
longer possible to write the header file to e.g. a pipe, like:

    curl -D >(grep ...) file:///dev/null

Which presently results in `Warning: Failed to open /dev/fd/63`

See #10079
Closes #10675
2023-03-06 11:14:37 +01:00
Daniel Stenberg
835682661c
misc: remove support for curl_off_t < 8 bytes
Closes #10597
2023-02-24 17:05:33 +01:00
Jay Satiro
d9b7f6e750
curl: make -w's %{stderr} use the file set with --stderr
Reported-by: u20221022 on github
Fixes #10491
Closes #10569
2023-02-23 10:05:13 +01:00
Daniel Stenberg
60e13311f4
tool_operate: propagate error codes for missing URL after --next
Fixes #10558
Reported-by: u20221022 on github
Closes #10580
2023-02-22 08:44:45 +01:00
Daniel Stenberg
780ff2c95c
tool_getparam: don't add a new node for just --no-remote-name
Unless --remote-name-all is used.

Fixes #10564
Reported-by: u20221022 on github
Closes #10582
2023-02-22 08:43:13 +01:00
Daniel Stenberg
475207c1c8
tool_progress: shut off progress meter for --silent in parallel
Reported-by: finkjsc on github
Fixes #10573
Closes #10579
2023-02-21 16:11:20 +01:00
Daniel Stenberg
1c9cfb7af3
tool_operate: avoid fclose(NULL) on bad header dump file
Fixes #10570
Reported-by: Jérémy Rabasco
Closes #10571
2023-02-21 10:34:29 +01:00
Jay Satiro
41dfb7f516 tool_operate: fix scanbuild compiler warning
Prior to this change Azure CI scanbuild warned of a potential NULL
pointer string passed to strtol when CURLDEBUG enabled, even though the
way the code was written it wouldn't have happened.

Bug: https://github.com/curl/curl/commit/5479d991#r101159711
Reported-by: Marcel Raad

Closes https://github.com/curl/curl/pull/10559
2023-02-18 19:02:40 -05:00
Daniel Stenberg
6841f2ed5f
curl: make --silent work stand-alone
- renamed the struct field to 'silent' to match the cmdline option
- make --show-error toggle independently of --silent
- make --silent independent of ->noprogress as well

By doing this, the three options --silent, --no-progress-meter and
--show-error should work independently of each other and also work with
and without '--no-' prefix as documented.

Reported-by: u20221022 on github
Fixes #10535
Closes #10536
2023-02-17 09:22:49 +01:00
Daniel Stenberg
5479d9916e
tool_operate: allow debug builds to set buffersize
Using the CURL_BUFFERSIZE environment variable.

Closes #10532
2023-02-16 14:36:24 +01:00
Daniel Stenberg
ce24ea373f
tool_getparam: make --get a true boolean
To match how it is documented in the man page.

Fixes #10527
Reported-by: u20221022 on github
Closes #10531
2023-02-16 10:53:59 +01:00
Daniel Stenberg
95fe2bba74
tool_operate: move the 'updated' variable
This was already done by Dan Fandrich in the previous PR but somehow I
lost that fixup.

Follow-up to 349c5391f2
2023-02-07 12:16:13 +01:00
Dan Fandrich
df3722a46c
tool_operate: Fix error codes during DOS filename sanitize
It would return CURLE_URL_MALFORMAT in an OOM condition.

Closes #10414
2023-02-07 08:20:19 +01:00