Commit Graph

319 Commits

Author SHA1 Message Date
Viktor Szakats
2ccd1c5889
cmakelint: fix to check root CMakeLists.txt
Closes #15565
2024-11-13 12:45:06 +01:00
Daniel Stenberg
448df98d92
mk-ca-bundle: remove CKA_NSS_SERVER_DISTRUST_AFTER conditions
The script's previous treatment of this meta-data was a
misunderstanding. (Added in 1ebc53df25) The mistrust is not for
the root cert at this date (it would simply be removed from the bundle
then instead) but for created server certificates:

    If a builtin certificate has a CKA_NSS_SERVER_DISTRUST_AFTER
    timestamp before the SCT or NotBefore date of a certificate that
    builtin issued, then clients can elect not to trust it.

That is however information that cannot be provided in the generated PEM
output.

Fixes #15547
Reported-by: Andrew Ayer
Closes #15552
2024-11-12 08:33:57 +01:00
Gabriel Marin
a58584a881
checksrc: add check for spaces around logical AND operators
Closes #15144
2024-10-21 09:35:00 +02:00
Daniel Stenberg
d78e129d50
WebSockets: make support official (non-experimental)
Inverts the configure/cmake options to instead provide options that
disable WebSockets and have them (ws + wss) enabled by default.

Closes #14936
2024-09-27 13:20:25 +02:00
Viktor Szakats
cfae354a9a
codespell: extend checks to more subdirs
- fix issues found.
- fix a few more found locally.

Closes #15072
2024-09-27 10:27:08 +02:00
Viktor Szakats
fcc89619d9
singleuse: make git grep faster, add Apple nm support
- avoid regexp in grep to make it run faster.
- add support for parsing Apple `nm` output:
  - skip leading underscore from function names.
  - pick object name from output.

Closes #15070
2024-09-27 10:27:08 +02:00
Gabriel Marin
cff75acfec
checksrc: Added checks for colon operator in ternary expressions
Closes #14990
2024-09-25 08:01:49 +02:00
Gabriel Marin
a4703dac13
checksrc: fixed typo
Closes #14968
2024-09-22 21:23:08 +02:00
Viktor Szakats
0aece8f66d
tidy-up: indent, whitespace, #error in make files
Replace invalid C with `#error`.

Cherry-picked from #14692
Closes #14997
2024-09-22 09:51:15 +02:00
Viktor Szakats
60c3d04465
autotools: add support for 'unity' builds, enable in CI
Implement the "unity" builds as known from CMake, but for autotools.
It's limited to `lib` and `src` (CMake also supports it in `tests`).

Enable with: `--enable-unity` (disabled by default)

Unity builds speed up builds significantly. Cygwin and Windows builds in
particular, but the effect is noticeable on most systems. It also allows
discovering unity issues with autotools, benefitting also CMake when
building the same combination. In CI it makes turnaround times quicker.

This closes build performance with CMake. autotools still lags behind
because it builds shared and static libcurl in two, separate passes.
CMake does it in one. Manpage compilation isn't batched, it is in CMake.
After unity and test bundle support the slowest parts of the build are
the configuration phase (which is effectively a tedious, non-parallel,
compilation and/or linking of 300+ tiny programs. The next bottleneck
is compiling individual examples and finally test servers (only slow
with autotools).

The autotools implementation is slightly less efficient than CMake,
because 3 sources are permanently excluded while in CMake this isn't
necessary and solved more efficiently while building libtests. There is
also no 'unity' support for tests, making them a less efficient also.

Enable it in CI for most `configure` jobs. Except in GHA/dist (though
it works fine there too), to use the default config there. Also skip for
the Linux AWC-LC job where it made builds time a few seconds longer
(reason undiscovered.)

Autotools test suite builds compared between master -> `--enable-unity`:
- GHA/Linux: 32s -> 12s
  https://github.com/curl/curl/actions/runs/10705668823/job/29681617374
  https://github.com/curl/curl/actions/runs/10742978889/job/29796766297
- GHA/macOS: 37s -> 10s
  https://github.com/curl/curl/actions/runs/10705668813/job/29681632885
  https://github.com/curl/curl/actions/runs/10742978699/job/29796768875
- GHA/FreeBSD: 15m25 -> 10m58 (full workflow time, ~qemu)
  https://github.com/curl/curl/actions/runs/10705668811/job/29681607915
  https://github.com/curl/curl/actions/runs/10742978937/job/29796766115
- GHA/Cygwin: 3m32 -> 1m21
  https://github.com/curl/curl/actions/runs/10705668809/job/29681609965
  https://github.com/curl/curl/actions/runs/10742978645/job/29796756933
- GHA/MSYS2: 2m42 -> 50s
  https://github.com/curl/curl/actions/runs/10705668808/job/29681621166
  https://github.com/curl/curl/actions/runs/10742978662/job/29799739289
- GHA/mingw-w64: 5m32 -> 1m23
  https://github.com/curl/curl/actions/runs/10705668808/job/29681628787
  https://github.com/curl/curl/actions/runs/10742978662/job/29799741568

Closes #14815
2024-09-20 23:53:33 +02:00
Daniel Stenberg
e666a678bd
checksrc: check for spaces around '?', '>' and '<'
Closes #14921
2024-09-19 14:59:12 +02:00
Daniel Stenberg
fbf5d507ce
lib/src: white space edits to comply better with code style
... as checksrc now finds and complains about these.

Closes #14921
2024-09-19 14:59:12 +02:00
Daniel Stenberg
28ca199d89
libcurl-docs: CURLINFO_LOCAL_* work for QUIC as well as TCP
Also, CURLOPT_TCP_FASTOPEN is for TCP only.

Closes #14852
2024-09-11 08:16:28 +02:00
Daniel Stenberg
7020bc9c4a
cd2nroff: only require "added-in" for source "libcurl"
To allow this script to be used by trurl easier.

Closes #14844
2024-09-10 13:49:49 +02:00
Daniel Stenberg
80df6a5c12
checksrc: add STRNCPY as an opt-in rule to detect and error on strncpy
make "lib/.checksrc" enable it

Closes #14830
2024-09-09 13:47:50 +02:00
Dan Fandrich
c72dd0bb14 maketgz: fix RELEASE-TOOLS.md for daily tarballs
The daily snapshots have no associated git tag, so provide a commit hash
instead in these cases. Fix the dpkg detection since the shell would
exit immediately without showing an error message if it weren't found.

Closes #14820
2024-09-07 09:33:09 -07:00
Daniel Stenberg
40017fb323
firefox-db2pem: mention what "certutil" the script uses
... and the Debian package that provides it.

Closes #14806
2024-09-06 08:57:34 +02:00
Daniel Stenberg
8d6db8cd8b
scripts/delta: output bugfixes/day 2024-09-06 08:47:06 +02:00
Daniel Stenberg
5c14d696fe
maketgz: move from / into scripts
Closes #14797
2024-09-05 17:07:41 +02:00
Daniel Stenberg
23e6391c1d
managen: in man output, remove the leading space from examples
Leave that rendering decision to the display tool.

Closes #14735
2024-08-30 10:09:48 +02:00
Daniel Stenberg
c280010d8b
managen: fix superfluous leading blank line in quoted sections
When a markdown quoted section using 4-space indentation was converted
to nroff, managen previously caused a newline to appear after the
leading .nf. This fix makes sure that newline is inserted *before* .nf
as intended.

This is perhaps most notable in the HTML version of rendered manpages if
the quoted sections use different colors or similar.

Closes #14732
2024-08-30 00:21:22 +02:00
Dan Fandrich
59b419f1a5 CI: add a script and job to run cmakelint
A number of checks don't match our style or are buggy and so are disabled.

Co-authored-by: Viktor Szakats <vszakats@users.noreply.github.com>
Fixes #14580
Closes #14665
2024-08-27 08:33:44 -07:00
Daniel Stenberg
23749bfd04
GHA: add a checksrc job
This job unconditionally runs checksrc on ALL .c and .h files present in
git.

checksrc.pl: fixed to look for ".checksrc" in the same directory from
where it loads the file to check so that it an be invoked like this

Closes #14625
2024-08-22 23:43:52 +02:00
Daniel Stenberg
99ba50d9c4
misc: general C style cleanups
... in code that previously was not checksrc'ed

Closes #14625
2024-08-22 23:43:45 +02:00
Daniel Stenberg
0066d169e8
managen: wordwrap long example lines in ASCII output
The entire ASCII version of the manpage word wraps at a fixed column,
while example command lines can easily go wider than so.

This change now makes manage work on wrapping long example command lines
to make them look nicer. And also to avoid triggering the build error
caused by too long lines in the output.

Quoted lines cannot be wrapped, so managen now errors out if they are
"too long". With this addition, the 'maxline' script is removed as it is
no longer needed.

Closes #14543
2024-08-16 08:57:19 +02:00
Viktor Szakats
2edbc229cb
dist: add CI job to detect files missing from distro
Also:
- delete previous, cmake-specific solution.
- move a CI script under `.github`.

Follow-up to a118a6ecdd #14323
Closes #14463
2024-08-10 10:02:49 +02:00
Daniel Stenberg
32f9130ae1
mk-ca-bundle.pl: include a link to the caextract webpage
Makes it easier for users who find the file to find its origin.

Fixes https://github.com/curl/curl-www/issues/374

Closes #14465
2024-08-09 08:20:43 +02:00
Daniel Stenberg
badbd4eb46
manpage: ensure a maximum width for the text version
... using the new script 'maxline' to which we specify the maximum
number of columns we allow any single line to be, or it will cause an
error.

Starting out with a max width at 100 columns.

Bonus: shorten the long line in the --ipfs-gateway section.

Closes #14423
2024-08-08 17:49:37 +02:00
Daniel Stenberg
ed2850456c
configure: fail if PSL is not disabled but not found
Regression since 9b3f67e (shipped in 8.7.0)
Reported-by: Ryan Carsten Schmidt
Fixes #14373
Assisted-by: Viktor Szakats
Closes #14379
2024-08-05 08:33:58 +02:00
Viktor Szakats
f81f351b9a
tidy-up: OS names
Use these words and casing more consistently across text, comments and
one curl tool output:
AIX, ALPN, ANSI, BSD, Cygwin, Darwin, FreeBSD, GitHub, HP-UX, Linux,
macOS, MS-DOS, MSYS, MinGW, NTLM, POSIX, Solaris, UNIX, Unix, Unicode,
WINE, WebDAV, Win32, winbind, WinIDN, Windows, Windows CE, Winsock.

Mostly OS names and a few more.

Also a couple of other minor text fixups.

Closes #14360
2024-08-04 19:17:45 +02:00
Daniel Stenberg
86039e6e42
verify-release: shell script that verifies a release tarball
This script remakes a provided curl release and verifies that the newly
built version is identical to the original file.

Due to bugs in releases up to and including curl 8.9.1, it does not work
on tarballs generated before commit 754acd1a9d.

Closes #14350
2024-08-02 23:05:41 +02:00
Daniel Stenberg
d0afb33959
dmaketgz: only run 'make distclean' if Makefile exists
... so that we can avoid the build failure if we run this in a clean
checkout.

Also remove -it from the docker invoke since it is not interactive and
it needs no TTY. They made the job fail in the CI.
2024-08-02 23:05:41 +02:00
Daniel Stenberg
754acd1a9d
dist: fix reproducible build from release tarball
Make it possible to rebuild an identical copy from a release tarball. It
was previously only possible from a checked out git repository.

- add release-tools.sh to dist
- keep Makefile.dist around to include it in dist
- regenerate tool_huge.c with the new version in dist
- fix the dist CI job to not do make clean like before

Closes #14336
2024-08-02 08:28:56 +02:00
Daniel Stenberg
8a9c22796b
CHANGES: rename to CHANGES.md, no longer generated
- scripts/log2changes.pl was not included in release tarballs, which broke
  reproducible builds

- since log2changes uses git to generate the contents, it makes it difficult
  to generate the same contents later (it would need to be fixed)

- the CHANGES file has outlived its purpose. the main changes are in the
  RELEASE-NOTES, the rest are better tracked directly using git or on GitHub

- put a fixed CHANGES.md in there instead pointing out where the info lives
  now

Closes #14331
2024-08-01 13:37:12 +02:00
Daniel Stenberg
e358a6e542
contrithanks.sh: use -F with -v to match lines as strings
Makes names involving [brackets] work.
2024-07-30 12:17:00 +02:00
Patrick Monnerat
d14149e3ea
os400: build cli manual.
Use PASE perl to run manual generation scripts.

As PASE perl is not aware of all possible input file encoding, convert
all files to UTF-8 upon build start (this might be lengthy).

OS/400 terminal emulation may only offer 76 columns, thus a new -c
parameter has been added to the managen program, defining the allowed
width.

If perl is not available, omit generation and disable online manual.

Closes #14289
2024-07-28 15:59:21 +02:00
Daniel Stenberg
5a488251f7
curldown: fixups
- make DEFAULT sections less repetitive

- make historic mentions use HISTORY

- generate the protocols section on `# %PROTOCOLS%` instead of guessing
  where to put it

- generate the availability section on `# %AVAILABILITY%` instead of
  guessing where to put it

- make the protocols section more verbose

Closes #14227
2024-07-19 17:03:25 +02:00
Daniel Stenberg
8c1d9378ac
curldown: make 'added-in:' a mandatory header field
- generate AVAILABILITY manpage sections automatically - for consistent
  wording

- allows us to double-check against other documumentation (symbols-in-versions
  etc)

- enables proper automation/scripting based on this data

- lots of them were wrong or missing in the manpages

- several of them repeated (sometimes mismatching) backend support info

Add test 1488 to verify "added-in" version numbers against
symbols-in-versions.

Closes #14217
2024-07-18 18:04:09 +02:00
Daniel Stenberg
5eba0a4b37
tests/scripts: call it 'manpage' (single word)
Mostly in comments

Closes #14216
2024-07-18 15:12:09 +02:00
Viktor Szakats
09cdf7e531
cmake: delete unused HAVE_LIBSSH2, HAVE_LIBSOCKET macros
- `HAVE_LIBSSH2`: unused in source. Not defined in CMake.

- `HAVE_LIBSOCKET`: unused in source. Used internally in CMake.

autotools sets them implicitly, so add them to the flag comparison
ignore-list.

Closes #14178
2024-07-16 11:46:26 +02:00
Max Mehl
9104bad820
reuse: switch to REUSE 3.2 and REUSE.toml
- remove scripts/copyright.pl

Closes #14107
2024-07-05 20:47:48 +02:00
Daniel Stenberg
2abfc759b9
cmdline-opts: category cleanup
Option cleanups:

 --get is not upload
 --form* are post
 - added several options into ldap, smtp, imap and pop3
 - shortened the category descriptions in the list

category curl fixes:

 --create-dirs removed from 'curl'
 --ftp-create-dirs removed from 'curl'
 --netrc moved to 'auth' from 'curl'
 --netrc-file moved to 'auth' from 'curl'
 --netrc-optional moved to 'auth' from 'curl'
 --no-buffer moved to 'output' from 'curl'
 --no-clobber removed from 'curl'
 --output removed from 'curl'
 --output-dir removed from 'curl'
 --remove-on-error removed from 'curl'

Add a "global" category:

- Made all "global" options set this category

Add a "deprecated" category:

- Moved the deprecated options to it (maybe they should not be in any
 category long term)

Add a 'timeout' category

- Put a number of appropriate options in it

Add an 'ldap' category

- Put the LDAP related option in there

Remove categories "ECH" and "ipfs"

- They should not be categories. Had only one single option each.

Remove category "misc"

- It should not be a category as it is impossible to know when to browse
  it.

--use-ascii moved to ftp and output
--xattr moved to output
--service-name moved to auth

Managen fixes:

- errors if an option is given a category name that is not already setup
  for in code

- verifies that options set `scope: global` also is put in category
  `global´

Closes #14101
2024-07-05 11:05:50 +02:00
Daniel Stenberg
9784f9acf3
managen: fix removing backticks from subtitles
It erroneously removed them from the wrong variable.

Closes #14081
2024-07-02 07:47:49 +02:00
Sertonix
4fab113d4e
mk-ca-bundle.pl: delay 'curl -V' execution until it is needed
Avoid an `Can't exec "curl"` message when curl is not actually needed.

Closes #14060
2024-06-29 22:26:19 +02:00
Daniel Stenberg
e63e0d60b5
managen: insert final .fi for files ending with a quote
When an individual file ended with a quote (typically an example), the
render function would return without ending the quote correctly with a
".fi" (fill in) in the manpage output.

This made the additional text provided below to render wrongly.

Closes #14048
2024-06-28 12:30:55 +02:00
Daniel Stenberg
2d3a51e367
managen: introduce "Multi: per-URL"
For -O, -o and -T that are used once per specified URL.

Closes #14045
2024-06-28 08:42:47 +02:00
Viktor Szakats
f0d8db152a
tidy-up: use /usr/bin/env perl shebang
Most Perl scripts already used it. Sync up the few outliers.

Closes #14029
2024-06-26 23:58:10 +02:00
Daniel Stenberg
c4187ea5fa
managen: only output .RE for manpage output
For ascii they are just rubbish.

Closes #14025
2024-06-26 15:49:26 +02:00
Daniel Stenberg
c1845dc0e2
managen: fix blank line detection
Follow-up to d14a53eea7 which ruined the output somewhat.

Closes #14017
2024-06-25 16:41:54 +02:00
Daniel Stenberg
473ec8a05d
managen: output tabs for each 8 leading spaces
This replacing of eight leading spaces into tabs was already done for
the embedded uncompressed version in tool_hugehelp.c so it does not save
anything there. But the gzip compressed version ends up almost 2K
smaller.

The output in a terminal should be identical.

Before using TABs:

curl.txt 282492 bytes
curl.txt.gz 73261 bytes

With this change applied:

curl.txt 249382 bytes
curl.txt.gz 71470 bytes

Closes #14016
2024-06-25 16:13:30 +02:00