Commit Graph

66 Commits

Author SHA1 Message Date
Daniel Stenberg
59831f8061
tool_operate: split up the huge single_transfer into sub functions
- split up in a few smaller and easier to read functions
- simplify several sections
- avoid superfluous extra allocations
- remove unused debug code

Closes #15385
2024-10-23 23:12:54 +02:00
Daniel Stenberg
fbf5d507ce
lib/src: white space edits to comply better with code style
... as checksrc now finds and complains about these.

Closes #14921
2024-09-19 14:59:12 +02:00
Viktor Szakats
c04504885d
src: fix potential macro confusion in cmake unity builds
Sources used `lib/curlx.h` with both `ENABLE_CURLX_PRINTF` set and unset
before including it.

In a cmake "unity" batch where the first included source had it unset,
the next sources did not get the macros requested with
`ENABLE_CURLX_PRINTF` because `lib/curl.x` had already been included
without them.

Fix it by by making the macros enabled permanently and globally for
internal sources, and dropping `ENABLE_CURLX_PRINTF`.

This came up while testing unity builds with smaller batches. The full,
default unity build where all `src` is bundled up in a single unit, was
not affected.

Fixes:
```
$ cmake -B build -DCMAKE_UNITY_BUILD=ON -DCMAKE_UNITY_BUILD_BATCH_SIZE=15
$ make -C build
...
curl/src/tool_getparam.c: In function ‘getparameter’:
curl/src/tool_getparam.c:2409:11: error: implicit declaration of function ‘msnprintf’; did you mean ‘vsnprintf’? [-Wimplicit-function-declaration]
 2409 |           msnprintf(buffer, sizeof(buffer), "%" CURL_FORMAT_CURL_OFF_T "-",
      |           ^~~~~~~~~
      |           vsnprintf
curl/src/tool_getparam.c:2409:11: warning: nested extern declaration of ‘msnprintf’ [-Wnested-externs]
[...]
```

Reported-by: Daniel Stenberg
Bug: https://github.com/curl/curl/pull/14626#issuecomment-2301663491

Closes #14632
2024-08-22 10:45:04 +02:00
Daniel Stenberg
5603204448
curl: fix the -w urle.* variables
urle.scheme, urle.user, urle.password and urle.options mistakenly
operated on the original URL instead of the *effective* (last) URL.

Add test 474 to verify.

Reported-by: Gruber Glass
Fixes #14550
Closes #14560
2024-08-15 14:01:24 +02:00
Alex Snast
136504195a
getinfo: add CURLINFO_POSTTRANSFER_TIME_T
Returns the time, in microseconds, from the start until the last byte is
sent by libcurl (i.e. the request is sent off).

Closes #14189
2024-08-15 09:02:58 +02:00
Daniel Stenberg
c074ba64a8
code: language cleanup in comments
Based on the standards and guidelines we use for our documentation.

 - expand contractions (they're => they are etc)
 - host name = > hostname
 - file name => filename
 - user name = username
 - man page => manpage
 - run-time => runtime
 - set-up => setup
 - back-end => backend
 - a HTTP => an HTTP
 - Two spaces after a period => one space after period

Closes #14073
2024-07-01 22:58:55 +02:00
Daniel Stenberg
d69ee3ef83
curl: support -w '%{num_retries}
Suggested-by: Jay Guerette
Ref: https://github.com/curl/curl/discussions/13901
Closes #13910
2024-06-11 09:07:43 +02:00
Daniel Stenberg
0a3c78f6b6
tool_writeout: get certinfo only when needing it
Removes a fairly expensive libcurl call when not necessary

Closes #13914
2024-06-10 23:12:12 +02:00
Daniel Stenberg
4464c9f2c6
tool_writeout: bsearch the variable name
As the list of variable names grows, doing a simple loop to find the
name get increasingly worse. This switches to a bsearch.

Also: do a case sensitive check for the variable name. The names have
not been documented to be case insensitive and there is no point in
having them so.

Closes #13914
2024-06-10 23:12:05 +02:00
Viktor Szakats
1a89538347
src: tidy up types, add necessary casts
Cherry-picked from #13489
Closes #13614
2024-05-17 12:32:04 +02:00
Daniel Stenberg
ac208ccf6f
write-out: add '%{proxy_used}'
Returns 1 if the previous transfer used a proxy, otherwise 0. Useful to
for example determine if a `NOPROXY` pattern matched the hostname or
not.

Extended test 970 and 972
2024-02-22 08:38:53 +01:00
Viktor Szakats
e5bb88b8f8
tool: use our own stderr variable
Earlier this year we changed our own stderr variable to use the standard
name `stderr` (to avoid bugs where someone is using `stderr` instead of
the curl-tool specific variable). This solution needed to override the
standard `stderr` symbol via the preprocessor. This in turn didn't play
well with unity builds and caused curl tool to crash or stay silent due
to an uninitialized stderr. This was a hard to find issue, fixed by
manually breaking out one file from the unity sources.

To avoid two these two tricks, this patch implements a different
solution: Restore using our own local variable for our stderr output and
leave `stderr` as-is. To avoid using `stderr` by mistake, add a
`checksrc` rule (based on logic we already used in lib for `strerror`)
that detects any `stderr` use in `src` and points to using our own
variable instead: `tool_stderr`.

Follow-up to 06133d3e9b
Follow-up to 2f17a9b654

Closes #11958
2023-09-28 10:50:56 +00:00
Daniel Stenberg
1032f56efa
curl: make %output{} in -w specify a file to write to
It can be used multiple times. Use %output{>>name} to append.

Add docs. Test 990 and 991 verify.

Idea: #11400
Suggested-by: ed0d2b2ce19451f2
Closes #11416
2023-07-31 23:10:59 +02:00
Stefan Eissing
e024d5665d
lib: add CURLINFO_CONN_ID and CURLINFO_XFER_ID
- add an `id` long to Curl_easy, -1 on init
- once added to a multi (or its own multi), it gets
  a non-negative number assigned by the connection cache
- `id` is unique among all transfers using the same
  cache until reaching LONG_MAX where it will wrap
  around. So, not unique eternally.
- CURLINFO_CONN_ID returns the connection id attached to
  data or, if none present, data->state.lastconnect_id
- variables and type declared in tool for write out

Closes #11185
2023-06-12 23:53:00 +02:00
Daniel Stenberg
808cb31756
tool_writeout: add URL component variables
Output specific components from the used URL. The following variables
are added for this purpose:

  url.scheme, url.user, url.password, url.options, url.host, url.port,
  url.path, url.query, url.fragment, url.zoneid

Add the following for outputting parts of the "effective URL":

 urle.scheme, urle.user, urle.password, urle.options, urle.host, urle.port,
 urle.path, urle.query, urle.fragment, urle.zoneid

Added test 423 and 424 to verify.

Closes #10853
2023-04-04 14:42:44 +02:00
Jay Satiro
2f17a9b654 tool: improve --stderr handling
- freopen stderr with the user-specified file (--stderr file) instead of
  using a separate 'errors' stream.

- In tool_setup.h override stdio.h's stderr macro as global variable
  tool_stderr.

Both freopen and overriding the stderr macro are necessary because if
the user-specified filename is "-" then stdout is assigned to
tool_stderr and no freopen takes place. See the PR for more information.

Ref: https://github.com/curl/curl/issues/10491

Closes https://github.com/curl/curl/pull/10673
2023-03-12 00:58:40 -05:00
Jay Satiro
d9b7f6e750
curl: make -w's %{stderr} use the file set with --stderr
Reported-by: u20221022 on github
Fixes #10491
Closes #10569
2023-02-23 10:05:13 +01:00
Daniel Stenberg
2bc1d775f5
copyright: update all copyright lines and remove year ranges
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205
2023-01-03 09:19:21 +01:00
Daniel Stenberg
c6aa19c1da
writeout: add %{certs} and %{num_certs}
Let users get the server certificate chain using the command line

Closes #10019
2022-12-27 22:41:17 +01:00
Daniel Stenberg
7be53774c4
tool_operate: reduce errorbuffer allocs
- parallel transfers: only alloc and keep errorbuffers in memory for
  actual "live" transfers and not for the ones in the pending queue

- serial transfers: reuse the same fixed buffer for all transfers, not
  allocated at all.

Closes #9394
2022-09-01 09:25:00 +02:00
Daniel Stenberg
ddfd41c9b2
tool_writeout: fix enumerated type mixed with another type
Closes #9179
2022-07-23 13:39:23 +02:00
Daniel Stenberg
0ef4f087cd
curl: writeout: fix repeated header outputs
The function stored a terminating zero into the buffer for convenience,
but when on repeated calls that would cause problems. Starting now, the
passed in buffer is not modified.

Reported-by: highmtworks on github
Fixes #9150
Closes #9152
2022-07-17 23:29:02 +02:00
max.mehl
ad9bc5976d
copyright: make repository REUSE compliant
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869
2022-06-13 09:13:00 +02:00
Daniel Stenberg
e7793cb57b
curl/header_json: output the header names in lowercase
To better allow json[“header”].

Reported-by: Peter Korsgaard
Bug: https://daniel.haxx.se/blog/2022/03/24/easier-header-picking-with-curl/comment-page-1/#comment-25878
Closes #8633
2022-03-25 11:24:27 +01:00
Daniel Stenberg
4133a69f2d
curl: add %{header_json} support in -w handling
Outputs all response headers as a JSON object.
2022-03-22 08:24:25 +01:00
Daniel Stenberg
2d45137e1e
curl: add %header{name} support in -w handling
Outputs the response header 'name'
2022-03-22 08:24:25 +01:00
Daniel Stenberg
cdde5414fc
writeout: fix %{http_version} for HTTP/3
Output "3" properly when HTTP/3 was used.

Reported-by: Bernat Mut
Fixes #8072
Closes #8092
2021-12-03 23:00:37 +01:00
Jay Satiro
2f78be51eb
tool_writeout: fix the HTTP_CODE json output
Update test 970 accordingly.

Reported-by: Michal Rus
Fixes #6905
Closes #6906
2021-04-17 11:03:30 +02:00
Viktor Szakats
44872aefc2
http: add support to read and store the referrer header
- add CURLINFO_REFERER libcurl option
- add --write-out '%{referer}' command-line option
- extend --xattr command-line option to fill user.xdg.referrer.url extended
  attribute with the referrer (if there was any)

Closes #6591
2021-02-19 13:57:19 +00:00
Jay Satiro
65ca229461 tool_writeout: refactor write-out and write-out json
- Deduplicate the logic used by write-out and write-out json.

Rather than have separate writeLong, writeString, etc, logic for
each of write-out and write-out json instead have respective shared
functions that can output either format and a 'use_json' parameter to
indicate whether it is json that is output.

This will make it easier to maintain. Rather than have to go through
two sets of logic now we only have to go through one.

- Support write-out %{errormsg} and %{exitcode} in json.

- Clarify in the doc that %{exitcode} is the exit code of the transfer.

Prior to this change it just said "The numerical exitcode" which
implies it's the exit code of the tool, and it's not necessarily that.

Closes https://github.com/curl/curl/pull/6544
2021-02-09 02:48:27 -05:00
Daniel Stenberg
553f286a8f
tool_writeout: make %{errormsg} blank for no errors
Closes #6539
2021-01-28 07:55:25 +01:00
Daniel Stenberg
44a504e5e4
writeout: fix NULL dereference for "this url"
Detected by torture test 1029

Follow-up to 7a90ddf88f

Closes #6374
2020-12-26 13:23:19 +01:00
Daniel Stenberg
7a90ddf88f
curl: add variables to --write-out
In particular, these ones can help a user to create its own error
message when one or transfers fail.

writeout: add 'onerror', 'url', 'urlnum', 'exitcode', 'errormsg'

onerror - lets a user only show the rest on non-zero exit codes

url - the input URL used for this transfer

urlnum - the numerical URL counter (0 indexed) for this transfer

exitcode - the numerical exit code for the transfer

errormsg - obvious

Reported-by: Earnestly on github
Fixes #6199
Closes #6207
2020-12-21 16:38:48 +01:00
Daniel Stenberg
bc7ecc71c0
tooĺ_writeout: fix the -w time output units
Fix regression from commit fc813f80e1 (#6248) that changed the unit
to microseconds instead of seconds with fractions

Reported-by: 不确定
Fixes #6321
Closes #6322
2020-12-15 08:09:29 +01:00
Daniel Stenberg
fc813f80e1
tool_writeout: use off_t getinfo-types instead of doubles
Commit 3b80d3ca46 (June 2017) introduced getinfo replacement
variables that use curl_off_t instead of doubles. Switch the --write-out
function over to use them.

Closes #6248
2020-11-26 08:13:51 +01:00
Daniel Stenberg
4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Daniel Stenberg
0b4c4145aa
tool_writeout: protect fputs() from NULL
When the code was changed to do fputs() instead of fprintf() it got
sensitive for NULL pointers; add checks for that.

Follow-up from 0c1e767e83

Closes #5963
2020-09-15 22:52:51 +02:00
anio
0c1e767e83
tool_writeout: add new writeout variable, %{num_headers}
This variable gives the number of headers.

Closes #5947
2020-09-15 00:37:08 +02:00
Daniel Stenberg
f5e6907d52
curl: add %{method} to the -w variables
Gets the CURLINFO_EFFECTIVE_METHOD from libcurl.

Added test 1197 to verify.
2020-07-14 17:53:45 +02:00
Daniel Stenberg
032e838b73
terminology: call them null-terminated strings
Updated terminology in docs, comments and phrases to refer to C strings
as "null-terminated". Done to unify with how most other C oriented docs
refer of them and what users in general seem to prefer (based on a
single highly unscientific poll on twitter).

Reported-by: coinhubs on github
Fixes #5598
Closes #5608
2020-06-28 00:31:24 +02:00
Michael Kaufmann
884de1a763 writeout_json: Fix data type issues
Load long values correctly (e.g. for http_code).

Use curl_off_t (not long) for:
- size_download (CURLINFO_SIZE_DOWNLOAD_T)
- size_upload (CURLINFO_SIZE_UPLOAD_T)

The unit for these values is bytes/second, not microseconds:
- speed_download (CURLINFO_SPEED_DOWNLOAD_T)
- speed_upload (CURLINFO_SPEED_UPLOAD_T)

Fixes #5131
Closes #5152
2020-03-27 23:32:25 +01:00
Mathias Gumz
04c03416e6
writeout: support to generate JSON output
This commit adds support to generate JSON via the writeout feature:

    -w "%{json}"

It leverages the existing infrastructure as much as possible. Thus,
generating the JSON on STDERR is possible by:

    -w "%{stderr}%{json}"

This implements a variant of
https://github.com/curl/curl/wiki/JSON#--write-out-json.

Closes #4870
2020-03-17 15:01:28 +01:00
Daniel Stenberg
eb63b27b6b
curl: have -w's 'http_version' show '3' for HTTP/3
Closes #4196
2019-08-07 12:51:52 +02:00
Daniel Stenberg
bda4ef417a
curl: correct the switch() logic in ourWriteOut
Follow-up to e431daf013, as I did the wrong correction for a compiler
warning. It should be a break and not a fall-through.

Pointed-out-by: Frank Gevaerts
2018-11-07 11:15:20 +01:00
Frank Gevaerts
e431daf013
curl: add %{stderr} and %{stdout} for --write-out
Closes #3115
2018-11-07 11:09:55 +01:00
Marian Klymov
c45360d463
cppcheck: fix warnings
- Get rid of variable that was generating false positive warning
(unitialized)

- Fix issues in tests

- Reduce scope of several variables all over

etc

Closes #2631
2018-06-11 11:14:48 +02:00
Daniel Stenberg
8e65877870 curl: check for end of input in writeout backslash handling
Reported-by: Brian Carpenter

Added test 1442 to verify
2017-03-25 23:48:57 +01:00
Dan Fandrich
1890d59905 tool_writeout: fixed a buffer read overrun on --write-out
If a % ended the statement, the string's trailing NUL would be skipped
and memory past the end of the buffer would be accessed and potentially
displayed as part of the --write-out output. Added tests 1440 and 1441
to check for this kind of condition.

Reported-by: Brian Carpenter
2017-03-12 08:28:31 +01:00
Daniel Stenberg
1c3e8bbfed checksrc: warn for assignments within if() expressions
... they're already frowned upon in our source code style guide, this
now enforces the rule harder.
2016-12-14 01:29:44 +01:00
Daniel Stenberg
b228d2952b checksrc: stricter no-space-before-paren enforcement
In order to make the code style more uniform everywhere
2016-12-13 23:39:11 +01:00