Commit Graph

49 Commits

Author SHA1 Message Date
Daniel Stenberg
f0d7318193
strparse: provide access functions
To access the string and the length without having to directly use the
struct field names. Gives more freedom, flexbility and keeps
implementation specifics out of users' code.

Closes #16386
2025-02-19 12:17:32 +01:00
Daniel Stenberg
076444ec46
lib: simplify more white space loops
Since the ISBLANK() and ISSPACE() macros check for specific matches,
there is no point in using while(*ptr && ISSPACE(*ptr)) etc, as the
'*ptr' check is then superfluous.

Closes #16363
2025-02-17 13:17:18 +01:00
Daniel Stenberg
b4538ec522
strparse: switch to curl_off_t as base data type
- add hex and octal parsers to the Curl_str_* family
- make curlx_strtoofft use these parsers
- remove all use of strtol() and strtoul() in library code
- generally use Curl_str_* more than strtoofft, for stricter parsing
- supports 64-bit universally, instead of 'long' which differs in size
  between platforms

Extended the unit test 1664 to verify hex and octal parsing.

Closes #16336
2025-02-15 21:58:48 +01:00
Daniel Stenberg
92611f2a56
strparse: switch the API to work on 'const char *'
The functions are not meant to touch the input anyway.

Closes #16316
2025-02-13 11:16:04 +01:00
Daniel Stenberg
d5c738c608
strparse: string parsing helper functions
Designed to aid converting off from sscanf parsers. sscanf is hard to
use right, easy to mess up and often makes for sloppy error checking.

The new parsers allow more exact and pedandic parsing.

This new set of functions should be possible to use (and extend) and
switch over other libcurl parser code to use going forward.

Adapts the following to use the new functions:

- altsvc.c
- hsts.c
- http_aws_sigv4.c

Bonus: fewer memory copies, fewer stack buffers.

Test: Unit test1664

Docs: docs/internals/STRPARSE.md

Closes #15692
2024-12-12 16:00:52 +01:00
Daniel Stenberg
5ea61a0b54
hsts: support "implied LWS" properly around max-age
Adjust test 780 to verify.

Reported-by: newfunction
Closes #15330
2024-10-18 11:42:42 +02:00
Daniel Stenberg
a94973805d
hsts: improve subdomain handling
- on load, only replace existing HSTS entries if there is a full host
  match

- on matching, prefer a full host match and secondary the longest tail
  subdomain match

Closes #15210
2024-10-09 13:48:08 +02:00
Daniel Stenberg
60d8663afb
hsts: avoid the local buffer and memcpy on lookup
Closes #15190
2024-10-08 12:49:17 +02:00
Daniel Stenberg
ba235ab269
llist: remove direct struct accesses, use only functions
- Turned them all into functions to also do asserts etc.

- The llist related structs got all their fields renamed in order to make
  sure no existing code remains using direct access.

- Each list node struct now points back to the list it "lives in", so
  Curl_node_remove() no longer needs the list pointer.

- Rename the node struct and some of the access functions.

- Added lots of ASSERTs to verify API being used correctly

- Fix some cases of API misuse

Add docs/LLIST.md documenting the internal linked list API.

Closes #14485
2024-08-12 13:18:10 +02:00
Daniel Stenberg
c074ba64a8
code: language cleanup in comments
Based on the standards and guidelines we use for our documentation.

 - expand contractions (they're => they are etc)
 - host name = > hostname
 - file name => filename
 - user name = username
 - man page => manpage
 - run-time => runtime
 - set-up => setup
 - back-end => backend
 - a HTTP => an HTTP
 - Two spaces after a period => one space after period

Closes #14073
2024-07-01 22:58:55 +02:00
Viktor Szakats
fc8e0dee30
build: untangle UNITTESTS and DEBUGBUILD macros
- fix `DEBUGBUILD` guards that should be `UNITTESTS`, in libcurl code
  used by unit tests.
- fix guards for libcurl functions used in unit tests only.
- sync `UNITTEST` attribute between declarations and definitions.
- drop `DEBUGBUILD` guard from test `unit2600`.
- fix guards for libcurl HSTS code used by both a unit test (`unit1660`)
  and `test0446`.
- update an existing AppVeyor CI job to test the issues fixed.

This fixes building tests with `CURLDEBUG` enabled but `DEBUGBUILD`
disabled. This can happen when building tests with CMake with
`ENABLE_DEBUG=ON` in Release config, or with `ENABLE_CURLDEBUG=ON`
and _without_ `ENABLE_DEBUG=ON`. Possibly also with autotools
when using `--enable-curldebug` without `--enable-debug`.

Test results:
- before:
  https://ci.appveyor.com/project/curlorg/curl/builds/49835609
  https://ci.appveyor.com/project/curlorg/curl/builds/49898529/job/k8qpbs8idby70smw
  https://github.com/curl/curl/actions/runs/9259078835/job/25470318167?pr=13798#step:13:821
- after: https://ci.appveyor.com/project/curlorg/curl/builds/49839255
  (the two failures are unrelated, subject to PR #13705)

Ref: #13592 (issue discovery)
Ref: #13689 (CI testing this PR with `DEBUGBUILD`/`CURLDEBUG` combinations)
Closes #13694
2024-05-27 21:15:50 +02:00
Daniel Gustafsson
d84a95de11 hsts: explicitly skip blank lines
Keep blank lines or lines containing only whitespace to make it all
the way to the more expensive sscanf call in hsts_add.

Closes: #13603
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2024-05-14 10:19:41 +02:00
Daniel Gustafsson
4d96873a4d hsts: Remove single-use single-line function
The hsts_entry() function contains of a single line and is only
used in a single place in the code, so move the allocation into
hsts_create instead to improve code readability. C code usually
don't use the factory abstraction for object creation, and this
small example wasn't following our usual code style.

Closes: #13604
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2024-05-13 09:07:30 +02:00
Stefan Eissing
c296abd42d
llist: add Curl_llist_append()
- use for better readability in all places where the "insert_next"
  actually performs an append to the list
- add some tests in unit1300

Closes #13336
2024-04-11 09:00:51 +02:00
Daniel Stenberg
142ac257b3
lib: convert Curl_get_line to use dynbuf
Create the line in a dynbuf. Aborts the reading of the file on
errors. Avoids having to always allocate maximum amount from the
start. Avoids direct malloc.

Closes #12846
2024-02-07 09:22:10 +01:00
Daniel Stenberg
24ae4a07f3
hsts: remove assert for zero length domain
A zero length domain can happen if the HSTS parser is given invalid
input data which is not unheard of and is done by the fuzzer.

Follow-up from cfe7902111

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65661

Closes #12676
2024-01-10 13:58:14 +01:00
Daniel Stenberg
cfe7902111
lib: add debug log outputs for CURLE_BAD_FUNCTION_ARGUMENT
Closes #12658
2024-01-08 22:48:24 +01:00
Daniel Stenberg
7c992dd9f8
lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding
Since the copy does not stop at a null byte, let's not call it anything
that makes you think it works like the common strndup() function.

Based on feedback from Jay Satiro, Stefan Eissing and Patrick Monnerat

Closes #12490
2023-12-08 17:22:33 +01:00
Daniel Stenberg
289b486ffa
urldata: move hstslist from 'set' to 'state'
To make it work properly with curl_easy_duphandle(). This, because
duphandle duplicates the entire 'UserDefined' struct by plain copy while
'hstslist' is a linked curl_list of file names. This would lead to a
double-free when the second of the two involved easy handles were
closed.

Closes #12315
2023-11-13 15:36:24 +01:00
Sam James
bc8509a748
misc: fix -Walloc-size warnings
GCC 14 introduces a new -Walloc-size included in -Wextra which gives:

```
src/tool_operate.c: In function ‘add_per_transfer’:
src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ for type ‘struct per_transfer’ with size ‘480’ [-Walloc-size]
  213 |   p = calloc(sizeof(struct per_transfer), 1);
      |     ^
src/var.c: In function ‘addvariable’:
src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type ‘struct var’ with size ‘32’ [-Walloc-size]
  361 |   p = calloc(sizeof(struct var), 1);
      |     ^
```

The calloc prototype is:
```
void *calloc(size_t nmemb, size_t size);
    ```

So, just swap the number of members and size arguments to match the
prototype, as we're initialising 1 struct of size `sizeof(struct
...)`. GCC then sees we're not doing anything wrong.

Closes #12292
2023-11-11 23:35:47 +01:00
Daniel Stenberg
d3b3ba35a5
lib: add and use Curl_strndup()
The Curl_strndup() function is similar to memdup(), but copies 'n' bytes
then adds a terminating null byte ('\0').

Closes #12251
2023-11-02 20:35:20 +01:00
Daniel Stenberg
8c058a820a
hsts: skip single-dot hostname
Reported-by: Maksymilian Arciemowicz

Closes #12247
2023-11-02 13:17:45 +01:00
Viktor Szakats
3f8fc25720
cmake: add support for "unity" builds
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.

It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.

Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.

Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
  across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
  These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
  `statename`.
- rename `ssl_backend_data` structure to unique names for each
  TLS-backend, along with the `ssl_connect_data` struct member
  referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
  builds when compiling `lib/ldap.c`. To support "unity" builds, we had
  to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
  `schannel.h` option) _globally_. This caused an indirect inclusion of
  Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
  as well. This requires `[P]UNICODE_STRING` types, which is apperantly
  not defined automatically (as seen with both MSVS and mingw-w64).
  This patch includes `<subauth.h>` to fix it.
  Ref: https://github.com/curl/curl/runs/13987772013
  Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
  trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
  is built in static mode.

KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
  https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250

Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes #11095
2023-06-07 13:06:08 +00:00
Emanuele Torre
f198d33e8d
checksrc: disallow spaces before labels
Out of 415 labels throughout the code base, 86 of those labels were
not at the start of the line. Which means labels always at the start of
the line is the favoured style overall with 329 instances.

Out of the 86 labels not at the start of the line:
* 75 were indented with the same indentation level of the following line
* 8 were indented with exactly one space
* 2 were indented with one fewer indentation level then the following
  line
* 1 was indented with the indentation level of the following line minus
  three space (probably unintentional)

Co-Authored-By: Viktor Szakats

Closes #11134
2023-05-18 20:45:04 +02:00
Daniel Stenberg
d567cca1de
checksrc: fix SPACEBEFOREPAREN for conditions starting with "*"
The open paren check wants to warn for spaces before open parenthesis
for if/while/for but also for any function call. In order to avoid
catching function pointer declarations, the logic allows a space if the
first character after the open parenthesis is an asterisk.

I also spotted what we did not include "switch" in the check but we should.

This check is a little lame, but we reduce this problem by not allowing
that space for if/while/for/switch.

Reported-by: Emanuele Torre
Closes #11044
2023-04-27 17:24:47 +02:00
Daniel Stenberg
2bc1d775f5
copyright: update all copyright lines and remove year ranges
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205
2023-01-03 09:19:21 +01:00
Daniel Stenberg
ca02a77f05
hsts: handle adding the same host name again
It will then use the largest expire time of the two entries.
2022-12-27 15:22:32 +01:00
Daniel Stenberg
076a2f6291
share: add sharing of HSTS cache among handles
Closes #10138
2022-12-27 15:22:01 +01:00
Daniel Stenberg
3f039dfd6f
strcase: use curl_str(n)equal for case insensitive matches
No point in having two entry points for the same functions.

Also merged the *safe* function treatment into these so that they can
also be used when one or both pointers are NULL.

Closes #9837
2022-11-01 17:01:26 +01:00
Daniel Stenberg
279834dd45
misc: remove duplicated include files
Closes #9796
2022-10-26 11:27:59 +02:00
Daniel Stenberg
6f9fb7ec2d
misc: ISSPACE() => ISBLANK()
Instances of ISSPACE() use that should rather use ISBLANK(). I think
somewhat carelessly used because it sounds as if it checks for space or
whitespace, but also includes %0a to %0d.

For parsing purposes, we should only accept what we must and not be
overly liberal. It leads to surprises and surprises lead to bad things.

Closes #9432
2022-09-06 08:34:30 +02:00
Daniel Stenberg
d64115d7bb
hsts: use Curl_fopen() 2022-06-26 11:04:02 +02:00
max.mehl
ad9bc5976d
copyright: make repository REUSE compliant
Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869
2022-06-13 09:13:00 +02:00
Daniel Stenberg
3c8b969de6
links: update dead links
The wiki pages are gone, remove and link to more long-living docs.

Closes #8897
2022-05-23 08:40:13 +02:00
Daniel Stenberg
fae6fea209
hsts: ignore trailing dots when comparing hosts names
CVE-2022-30115

Reported-by: Axel Chong
Bug: https://curl.se/docs/CVE-2022-30115.html
Closes #8821
2022-05-09 12:54:39 +02:00
lucas
519b62ccd0
docs: update IETF links to use datatracker
The tools.ietf.org domain has been deprecated a while now, with the
links being redirected to datatracker.ietf.org.

Rather than make people eat that redirect time, this change switches the
URL to a more canonical source.

Closes #8317
2022-01-21 23:06:36 +01:00
Daniel Stenberg
da2598f823
hsts: CURLSTS_FAIL from hsts read callback should fail transfer
... and have CURLE_ABORTED_BY_CALLBACK returned.

Extended test 1915 to verify.

Reported-by: Jonathan Cardoso
Fixes #7726
Closes #7729
2021-09-16 12:36:12 +02:00
Daniel Stenberg
54f6d836d8
hsts: handle unlimited expiry
When setting a blank expire string, meaning unlimited, curl would pass
TIME_T_MAX to getime_r() when creating the output, while on 64 bit
systems such a large value cannot be convetered to a tm struct making
curl to exit the loop with an error instead. It can't be converted
because the year it would represent doesn't fit in the 'int tm_year'
field!

Starting now, unlimited expiry is instead handled differently by using a
human readable expiry date spelled out as "unlimited" instead of trying
to use a distant actual date.

Test 1660 and 1915 have been updated to help verify this change.

Reported-by: Jonathan Cardoso
Fixes #7720
Closes #7721
2021-09-15 14:37:36 +02:00
Daniel Stenberg
8822ecf7a1
Curl_hsts_loadcb: don't attempt to load if hsts wasn't inited
Reported-by: Jonathan Cardoso
Fixes #7710
Closes #7711
2021-09-12 23:43:51 +02:00
Daniel Stenberg
1c1d9f1aff
hsts: ignore numberical IP address hosts
Also, use a single function library-wide for detecting if a given hostname is
a numerical IP address.

Reported-by: Harry Sintonen
Fixes #7146
Closes #7149
2021-05-30 19:49:40 +02:00
Daniel Stenberg
d71ff2b9db
hsts: enable by default
No longer considered experimental.

Closes #6700
2021-04-19 08:22:16 +02:00
Daniel Gustafsson
419f0f5534 hsts: remove unused defines
MAX_HSTS_SUBLEN and MAX_HSTS_SUBLENSTR were unused from the initial commit,
and mostly likely leftovers from early development.  Remove as they're not
used for anything.

Closes #6741
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2021-03-14 23:18:27 +01:00
Daniel Stenberg
f6d13090d4
copyright: fix missing year (range) updates 2021-01-29 14:35:13 +01:00
Fabian Keil
4f61fd87b2
misc: fix typos
Bug: https://curl.se/mail/lib-2021-01/0063.html
Closes #6434
2021-01-11 15:12:25 +01:00
Daniel Stenberg
6ee57c8bdd
hsts: remove debug code leftovers
Closes #6175
2020-11-06 08:22:14 +01:00
Daniel Gustafsson
085f91036c hsts: Remove pointless call to free in errorpath
The line variable will always be NULL in the error path, so remove
the free call since it's pointless.

Closes #6170
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2020-11-05 09:42:38 +01:00
Daniel Stenberg
4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Daniel Stenberg
2cfc4ed983
hsts: add read/write callbacks
- read/write callback options
- man pages for the 4 new setopts
- test 1915 verifies the callbacks

Closes #5896
2020-11-03 16:08:48 +01:00
Daniel Stenberg
7385610d0c
hsts: add support for Strict-Transport-Security
- enable in the build (configure)
- header parsing
- host name lookup
- unit tests for the above
- CI build
- CURL_VERSION_HSTS bit
- curl_version_info support
- curl -V output
- curl-config --features
- CURLOPT_HSTS_CTRL
- man page for CURLOPT_HSTS_CTRL
- curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl)
- man page for --hsts
- save cache to disk
- load cache from disk
- CURLOPT_HSTS
- man page for CURLOPT_HSTS
- added docs/HSTS.md
- fixed --version docs
- adjusted curl_easy_duphandle

Closes #5896
2020-11-03 16:08:42 +01:00