- Fix `NROFF` auto-detection with certain shell/make-build combinations:
When a non-MSYS2 GNU Make runs inside an MSYS2 shell, Make executes
the detection command as-is via `CreateProcess()`. It fails because
`command` is an `sh` built-in. Ensure to explicitly invoke the shell.
- Initialize user-customizable variables:
Silences a list of warnings when running GNU Make with the option
`--warn-undefined-variables`. Another benefit is that it's now easy
to look up all user-customizable `Makefile.mk` variables by grepping
for ` ?=` in the curl source tree.
Suggested-by: Gisle Vanem
Ref: https://github.com/curl/curl/pull/9764#issuecomment-1330674433
- Fix `MKDIR` invocation:
Avoid a warning and potential issue in envs without forward-slash
support.
Closes#10000
Makes curl always use dot as decimal separator for options,
independently of what the locale says. Makes scripts and command lines
portable.
Updated docs accordingly.
Reported-by: Daniel Faust
Fixes#9969Closes#9972
Commit 3b16575ae9 removed support for
building on Novell Netware, but a few leftover traces remained. This
removes the last bits.
Closes: #9966
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Update bare GNU Make `Makefile.m32` to:
- Move objects into a subdirectory.
- Add support for MS-DOS. Tested with DJGPP.
- Add support for Watt-32 (on MS-DOS).
- Add support for AmigaOS.
- Rename `Makefile.m32` to `Makefile.mk`
- Replace `ARCH` with `TRIPLET`.
- Build `tool_hugehelp.c` proper (when tools are available).
- Drop MS-DOS compatibility macro `USE_ZLIB` (replaced by `HAVE_LIBZ`)
- Add support for `ZLIB_LIBS` to override `-lz`.
- Omit object files when building examples.
- Default `CC` to `gcc` once again, for convenience. (Caveat: compiler
name `cc` cannot be set now.)
- Set `-DCURL_NO_OLDIES` for examples, like autotools does.
- Delete `makefile.dj` files. Notice the configuration details and
defaults are not retained with the new method.
- Delete `makefile.amiga` files. A successful build needs a few custom
options. We're also not retaining all build details from the existing
Amiga make files.
- Rename `Makefile.m32` to `Makefile.mk` to reflect that they are not
Windows/MinGW32-specific anymore.
- Add support for new `CFG` options: `-map`, `-debug`, `-trackmem`
- Set `-DNDEBUG` by default.
- Allow using `-DOS=...` in all `lib/config-*.h` headers, syncing this
with `config-win32.h`.
- Look for zlib parts in `ZLIB_PATH/include` and `ZLIB_PATH/lib`
instead of bare `ZLIB_PATH`.
Note that existing build configurations for MS-DOS and AmigaOS likely
become incompatible with this change.
Example AmigaOS configuration:
```
export CROSSPREFIX=/opt/amiga/bin/m68k-amigaos-
export CC=gcc
export CPPFLAGS='-DHAVE_PROTO_BSDSOCKET_H'
export CFLAGS='-mcrt=clib2'
export LDFLAGS="${CFLAGS}"
export LIBS='-lnet -lm'
make -C lib -f Makefile.mk
make -C src -f Makefile.mk
```
Example MS-DOS configuration:
```
export CROSSPREFIX=/opt/djgpp/bin/i586-pc-msdosdjgpp-
export WATT_PATH=/opt/djgpp/net/watt
export ZLIB_PATH=/opt/djgpp
export OPENSSL_PATH=/opt/djgpp
export OPENSSL_LIBS='-lssl -lcrypt'
export CFG=-zlib-ssl
make -C lib -f Makefile.mk
make -C src -f Makefile.mk
```
Closes#9764
If the run-time libcurl is too old to support feature names, the name
array is created locally from the bit masks. This is the only sequence
left that uses feature bit masks.
Closes#9583
If the URL that -G would try to add a query to could not be parsed, it would
display
curl: (27) Out of memory
It now instead shows:
curl: (2) Could not parse the URL, failed to set query
Reported-by: Alex Xu
Fixes#9889Closes#9892
Prior to this change if the user wanted to signal an error from their
write callbacks they would have to use logic to return a value different
from the number of bytes (nmemb) passed to the callback. Also, the
inclination of some users has been to just return 0 to signal error,
which is incorrect as that may be the number of bytes passed to the
callback.
To remedy this the user can now return CURL_WRITEFUNC_ERROR instead.
Ref: https://github.com/curl/curl/issues/9873
Closes https://github.com/curl/curl/pull/9874
This option adds a piece of data, usually a name + value pair, to the
end of the URL query part. The syntax is identical to that used for
--data-urlencode with one extension:
If the argument starts with a '+' (plus), the rest of the string is
provided as-is unencoded.
This allows users to "build" query parts with options and URL encoding
even when not doing GET requests, which the already provided option -G
(--get) is limited to.
This idea was born in a Twitter thread.
Closes#9691
`windres` is not always auto-detected by autotools when building for
Windows. When this happened, the build failed with a confusing error due
to the empty `RC` command:
```
/bin/bash ../libtool --tag=RC --mode=compile -I../include -DCURL_EMBED_MANIFEST -i curl.rc -o curl.o
[...]
Usage: /sandbox/curl/libtool [OPTION]... [MODE-ARG]...
Try 'libtool --help' for more information.
libtool: error: unrecognised option: '-I../include'
```
Improve this by verifying if `RC` is set, and fail with a clear error
otherwise.
Follow-up to 6de7322c03
Ref: https://curl.se/mail/lib-2022-10/0049.html
Reported-by: Thomas Glanzmann
Closes#9781
The read callback can timeout if there's nothing to read within the
given maximum period. Example use case is when doing "curl -m 3
telnet://example.com" or anything else that expects input on stdin or
similar that otherwise would "hang" until something happens and then not
respect the timeout.
This fixes KNOWN_BUG 8.1, first filed in July 2009.
Bug: https://sourceforge.net/p/curl/bugs/846/Closes#9815
- Replace `Github` with `GitHub`.
- Replace `windows` with `Windows`
- Replace `advice` with `advise` where a verb is used.
- A few fixes on removing repeated words.
- Replace `a HTTP` with `an HTTP`
Closes#9802
`./configure --enable-shared --disable-static` fails when trying to link
a shared `curl.exe`, due to `libtool` magically changing the output
filename of `windres` to one that it doesn't find when linking:
```
/bin/sh ../libtool --tag=RC --mode=compile windres -I../../curl/include -DCURL_EMBED_MANIFEST -i ../../curl/src/curl.rc -o curl.o
libtool: compile: windres -I../../curl/include -DCURL_EMBED_MANIFEST -i ../../curl/src/curl.rc -o .libs/curl.o
[...]
CCLD curl.exe
clang: error: no such file or directory: 'curl.o'
```
Let's resolve this by skipping `libtool` and calling `windres` directly
when building `src` (aka `curl.exe`). Leave `lib` unchanged, as it does
need the `libtool` magic. This solution is compatible with building
a static `curl.exe`.
This build scenario is not CI-tested.
While here, delete an obsolete comment about a permanent `libtool`
warning that we've resolved earlier.
Regression from 6de7322c03
Reported-by: Christoph Reiter
Fixes#9803Closes#9805
Enable them in `lib/makefile.amiga` and `src/makefile.amiga` instead.
This allows builds without openssl and/or zlib. E.g. with the
<https://github.com/bebbo/amiga-gcc> cross-compiler.
Reviewed-by: Daniel Stenberg
Closes#9762
In some circumstances when doing parallel transfers, the
single_transfer_cleanup() would not be called and then 'inglob' could
leak.
Test 496 verifies
Reported-by: Trail of Bits
Closes#9749
1 - consider the transfer handled at once when in the function, to avoid
the same list entry to get added more than once in rare error
situations
2 - set the ERRORBUFFER for the handle first after it has been added
successfully
Reported-by: Trail of Bits
Closes#9729
The two defines MAX_FILE2MEMORY and MAX_FILE2STRING define the largest
strings accepted when loading files into memory, but as the size is
later used as input to functions that take the size as 'int' as
argument, the sizes must not be larger than INT_MAX.
These two new assert()s make the code error out if someone would bump
the sizes without this consideration.
Reported-by Trail of Bits
Closes#9719
fcntl() can (in theory) return a non-zero number for success, so a
better test for error is checking for -1 explicitly.
Follow-up to 41e1b30ea1
Mentioned-by: Dominik Klemba
Closes#9708
To avoid URL tricks, use the URL parser for this.
This update changes curl's behavior slightly in that it will ignore the
possible query part from the URL and only use the file name from the
actual path from the URL. I consider it a bugfix.
"curl -O localhost/name?giveme-giveme" will now save the output in the
local file named 'name'
Updated test 1210 to verify
Assisted-by: Jay Satiro
Closes#9684
If the main_checkfds function cannot create new file descriptors in an
attempt to detect of stdin, stdout or stderr are closed.
Also changed the check to use fcntl() to check if the descriptors are
open, which avoids superfluously calling pipe() if they all already are.
Follow-up to facfa19cdd
Reported-by: Trail of Bits
Closes#9663
This is a bit shorter and a lot safer.
Substrings of unescaped characters are added by a single call to reduce
overhead.
Extend test 1465 to handle more kind of escapes.
Closes#9653
C string hexadecimal-escaped characters may have more than 2 digits.
This results in a wrong C compiler interpretation of a 2-digit escaped
character when followed by an hex digit character.
The solution retained here is to represent such characters as 3-digit
octal escapes.
Adjust and extend test 1465 for this case.
Closes#9643
In a recent commit I mistakenly deleted this logic, after seeing a
reference to a filename ending with `.cvs` and thinking it must have
been long gone. Turns out this is an existing file. Restore the rule
and the necessary `COPY` definitions with it.
The restored logic is required for a successful build on a bare source
tree (as opposed to a source release tarball).
Also shorten an existing condition similar to the one added in this
patch.
Regression since 07a0047882Closes#9645
After this patch, we reduce the three copies of most `Makefile.m32`
logic to one. This now resides in `lib/Makefile.m32`. It makes future
updates easier, the code shorter, with a small amount of added
complexity.
`Makefile.m32` reduction:
| | bytes | LOC total | blank | comment | code |
|-------------------|-------:|----------:|-------:|---------:|------:|
| 7.85.0 | 34772 | 1337 | 79 | 192 | 1066 |
| before this patch | 17601 | 625 | 62 | 106 | 457 |
| after this patch | 11680 | 392 | 52 | 104 | 236 |
Details:
- Change rules to create objects for the `v*` subdirs in the `lib` dir.
This allows to use a shared compile rule and assumes that filenames
are not (and will not be) colliding across these directories.
`Makefile.m32` now also stores a list of these subdirs. They are
changing rarely though.
- Sync as much as possible between the three `Makefile.m32` scripts'
rules and their source/target sections.
- After this patch `CPPFLAGS` are all applied to the `src` sources once
again. This matches the behaviour of cmake/autotools. Only zlib ones
are actually required there.
- Use `.rc` names from `Makefile.inc` instead of keeping a duplicate.
- Change examples to link `libcurl.dll` by default. This makes building
trivial, even as a cross-build:
`CC=x86_64-w64-mingw32-gcc make -f Makefile.m32`
To run them, you need to move/copy or add-to-path `libcurl.dll`.
You can select static mode via `CFG=-static`.
- List more of the `Makefile.m32` config variables.
- Drop `.rc` support from examples. It made it fragile without much
benefit.
- Include a necessary system lib for the `externalsocket.c` example.
- Exclude unnecessary systems libs when building in `-dyn` mode.
Closes#9642
This patch overhauls `Makefile.m32` scripts, fixing a list of quirks,
making its behaviour and customization envvars align better with other
build systems, aiming for less code, that is easier to read, use and
maintain.
Details:
- Rename customization envvars:
`CURL_CC` -> `CC`
`CURL_RC` -> `RC`
`CURL_AR` -> `AR`
`CURL_LDFLAG_EXTRAS_DLL` -> `CURL_LDFLAGS_LIB`
`CURL_LDFLAG_EXTRAS_EXE` -> `CURL_LDFLAGS_BIN`
- Drop `CURL_STRIP` and `CURL_RANLIB`. These tools are no longer used.
- Accept `CFLAGS`, `CPPFLAGS`, `RCFLAGS`, `LDFLAGS` and `LIBS` envvars.
- Drop `CURL_CFLAG_EXTRAS`, `CURL_LDFLAG_EXTRAS`, `CURL_RCFLAG_EXTRAS` in
favor of the above.
- Do not automatically enable `zlib` with `libssh2`. `zlib` is optional
with `libssh2`.
- Omit unnecessary `CPPFLAGS` options when building `curl.exe` and
examples.
- Drop support for deprecated `-winssl` `CFG` option. Use `-schannel`
instead.
- Avoid late evaluation where not necessary (`=` -> `:=`).
- Drop support for `CURL_DLL_A_SUFFIX` to override the implib suffix.
Instead, use the standard naming scheme by default: `libcurl.dll.a`.
The toolchain recognizes the name, and selects it automatically when
asking for a `-shared` vs. `-static` build.
- Stop applying `strip` to `libcurl.a`. Follow-up from
16a58e9f93. There was no debug info to
strip since then.
- Stop setting `-O3`, `-W`, `-Wall` options. You can add these to
`CFLAGS` as desired.
- Always enable `-DCURL_DISABLE_OPENSSL_AUTO_LOAD_CONFIG` with OpenSSL,
to avoid that vulnerability on Windows.
- Add `-lbrotlicommon` to `LIBS` when using `brotli`.
- Do not enable `-nghttp3` without `-ngtcp2`.
- `-ssh2` and `-rtmp` options no longer try to auto-select a TLS-backend.
You need to set the backend explicitly. This scales better and avoids
issues with certain combinations (e.g. `libssh2` + `wolfssl` with no
`schannel`).
- Default to OpenSSL TLS-backend with `ngtcp2`. Possible to override via
`NGTCP2_LIBS`.
- Old, alternate method of enabling components (e.g. `SSH2=1`) no longer
supported.
- Delete `SPNEGO` references. They were no-ops.
- Drop support for Win9x environments.
- Allow setting `OPENSSL_LIBS` independently from `OPENSSL_LIBPATH`.
- Support autotools/CMake `libssh2` builds by default.
- Respect `CURL_DLL_SUFFIX` in `-dyn` mode when building `curl.exe` and
examples.
- Assume standard directory layout with `LIBCARES_PATH`. (Instead of the
long gone embedded one.)
- Stop static linking with c-ares by default. Add
`CPPFLAGS=-DCARES_STATICLIB` to enable it.
- Reorganize internal layout to avoid redundancy and emit clean diffs
between src/lib and example make files.
- Delete unused variables.
- Code cleanups/rework.
- Comment and indentation fixes.
Closes#9632
- Drop auto-detection of OpenSSL 1.0.2 and earlier. Now always defaulting
to OpenSSL 1.1.0 and later, LibreSSL and BoringSSL.
- Drop `Invalid path to OpenSSL package` detection. OpenSSL has been
using a standard file layout since 1.1.0, so this seems unnecessary
now.
- Drop special logic to enable Novell LDAP SDK support.
- Drop special logic to enable OpenLDAP LDAP SDK support. This seems
to be distinct from native OpenLDAP, with support implemented inside
`lib/ldap.c` (vs. `lib/openldap.c`) back when the latter did not exist
yet in curl.
- Add `-lwldap32` only if there is no other LDAP library (either native
OpenLDAP, or SDKs above) present.
- Update `doc/INSTALL.md` accordingly.
After this patch, it's necessary to make configration changes when using
OpenSSL 1.0.2 or earlier, or the two LDAP SDKs.
OpenSSL 1.0.2 and earlier:
```
export OPENSSL_INCLUDE = <path-to-openssl>/outinc
export OPENSSL_LIBPATH = <path-to-openssl>/out
export OPENSSL_LIBS = -lssl32 -leay32 -lgdi32
```
Novell LDAP SDK, previously enabled via `USE_LDAP_NOVELL=1`:
```
export CURL_CFLAG_EXTRAS = -I<path-to-sdk>/inc -DCURL_HAS_NOVELL_LDAPSDK
export CURL_LDFLAG_EXTRAS = -L<path-to-sdk>/lib/mscvc -lldapsdk -lldapssl -lldapx
```
OpenLDAP LDAP SDK, previously enabled via `USE_LDAP_OPENLDAP=1`:
```
export CURL_CFLAG_EXTRAS = -I<path-to-sdk>/include -DCURL_HAS_OPENLDAP_LDAPSDK
export CURL_LDFLAG_EXTRAS = -L<path-to-sdk>/lib -lldap -llber
```
I haven't tested these scenarios, and in general we recommend using
a recent OpenSSL release. Also, WinLDAP (the Windows default) and
OpenLDAP (via `-DUSE_OPENLDAP`) are the LDAP options actively worked on
in curl.
Closes#9631
- Add `-lcrypt32` once, and add it always for simplicity.
- Delete broken link and reference to the pre-Vista WinIDN add-on.
MS no longer distribute it.
- Delete related `WINIDN_PATH` option. IDN is a system lib since Vista.
- Sync `LIBCARES_PATH` default with the rest of dependencies.
- Delete version numbers from dependency path defaults.
- `libgsasl` package is now called `gsasl`.
- Delete `libexpat` and `libxml2` references. No longer used by curl.
- Delete `Edit the path below...` comments. We recommend to predefine
those envvars instead.
- `libcares.a` is not an internal dependency anymore. Stop using it as
such.
- `windres` `--include-dir` -> `-I`, `-F` -> `--target=` for readability.
- Delete `STRIP`, `CURL_STRIP`, `AR` references from `src/Makefile.m32`.
They were never used.
- Stop to `clean` some objects twice in `src/Makefile.m32`.
- Delete cvs-specific leftovers.
- Finish resource support in examples make file.
- Delete `-I<root>/lib` from examples make file.
- Fix copyright start year in examples make file.
- Delete duplicate `ftpuploadresume` input in examples make file.
- Sync OpenSSL lib order, `SYNC` support, `PROOT` use, dependency path
defaults, variables names and other internal bits between the three
make files.
- `lib/Makefile.m32` accepted custom options via `DLL_LIBS` envvar. This
was lib-specific and possibly accidental. Use `CURL_LDFLAG_EXTRAS_DLL`
envvar for the same effect.
- Fix linking `curl.exe` and examples to wrong static libs with
auto-detected OpenSSL 1.0.2 or earlier.
- Add `-lgdi32` for OpenSSL 1.0.2 and earlier only.
- Add link to Novell LDAP SDK and use a relative default path. Latest
version is from 2016, linked to an outdated OpenSSL 1.0.1.
- Whitespace and comment cleanups.
TODO in a next commit:
Delete built-in detection/logic for OpenSSL 1.0.2 and earlier, the Novell
LDAP SDK and the other LDAP SDK (which is _not_ OpenLDAP). Write up the
necessary custom envvars to configure them.
Closes#9616
The introduction of CURL_DISABLE_MIME came with some additional bugs:
- Disabled MIME is compiled-in anyway if SMTP and/or IMAP is enabled.
- CURLOPT_MIMEPOST, CURLOPT_MIME_OPTIONS and CURLOPT_HTTPHEADER are
conditioned on HTTP, although also needed for SMTP and IMAP MIME mail
uploads.
In addition, the CURLOPT_HTTPHEADER and --header documentation does not
mention their use for MIME mail.
This commit fixes the problems above.
Closes#9610
Replace bit mask protocol sets by null-terminated arrays of protocol
tokens. These are the addresses of the protocol names returned by
curl_version_info().
Protocol names are sorted case-insensitively before output to satisfy CI
tests matches consistency.
The protocol list returned by curl_version_info() is augmented with all
RTMP protocol variants.
Test 1401 adjusted for new alpha ordered output.
Closes#9546
After this update autotools builds will compile and link `.rc` resources
to Windows executables. Bringing this feature on par with CMake and
Makefile.m32 builds. And also making it unnecessary to improvise these
steps manually, while monkey patching build files, e.g. [0].
You can customize the resource compiler via the `RC` envvar, and its
options via `RCFLAGS`.
This harmless warning may appear throughout the build, even though the
autotools manual documents [1] `RC` as a valid tag, and it fails when
omitting one:
`libtool: error: ignoring unknown tag RC`
[0] 535f19060d/curl-autotools.sh (L376-L382)
[1] https://www.gnu.org/software/libtool/manual/html_node/Tags.htmlCloses#9521
As they are now rejected by the library, take care of not passing
disabled protocol names to CURLOPT_PROTOCOLS_STR and
CURLOPT_REDIR_PROTOCOLS_STR.
Rather than using the CURLPROTO_* constants, dynamically assign protocol
numbers based on the order they are listed by curl_version_info().
New type proto_set_t implements prototype bit masks: it should therefore
be large enough to accomodate all library-enabled protocols. If not,
protocol numbers beyond the bit count of proto_set_t are recognized but
"inaccessible": when used, a warning is displayed and the value is
ignored. Should proto_set_t overflows, enabled protocols are reordered to
force those having a public CURLPROTO_* representation to be accessible.
Code has been added to subordinate RTMP?* protocols to the presence of
RTMP in the enabled protocol list, being returned by curl_version_info()
or not.
