Before this patch `pkg-config`-based detection was ignored, and used
solely as a path hint for native detection.
- fix `pkg_search_module()` result prefix to match what code expects:
`_GSS` (was: `_GSS_PKG`). Update variable that were in sync with old
prefix.
- update the pkg-config codepath to use `_GSS_MODULE_NAME` to detect
GSS flavour. This requires CMake 3.16.
Otherwise fall back to the old method. (The old method doesn't seem to
work anymore (?) as of CMake 3.30.1. Documented
`<prefix>_<modulename>_VERSION` variable is defined, but empty.)
- update the pkg-config codepath to use `_GSS_VERSION` set by CMake.
Resort to the old code when this variable is empty. (The old code
doesn't seem to work anymore (?) as of CMake 3.30.1)
- fix pkg-config codepath to set the documented result variables.
- align native detection variable names with those generated by
`pkg_search_module()` in the pkg-config codepath.
- GHA/macos: enable GSS Heimdal in a cmake job.
Uses the native detection.
- GHA/linux: enable GSS Heimdal in cmake and autotools jobs.
CMake uses `pkg-config`-based detection.
- suppress test 2077 and 2078 results on Linux + Heimdal.
```
FAIL-IGNORED 2077: 'curl --fail --negotiate to unauthenticated service fails' HTTP, HTTP GET, GSS-API
FAIL-IGNORED 2078: 'curl --negotiate should not send empty POST request only' HTTP, HTTP GET, GSS-API
```
Failing with valgrind errors in both autotools and cmake builds:
https://github.com/curl/curl/actions/runs/10282222581/job/28453472068?pr=14430#step:38:3638https://github.com/curl/curl/actions/runs/10282222581/job/28453473398?pr=14430#step:38:7831Closes#14430
- use documented flavour of `xcrun` option.
- show SDK version with a dedicated command.
(Sometimes the SDK path is a symlink and doesn't tell the version.
This is not at the moment the case in CI, but handle it anyway.)
- align group header with reality.
Preinstalled vs. installed Homebrew packages can be recognized
by their directory timestamps. Installed ones have a current date.
Closes#14434
- make `--with-apple-idn` override libidn2, in sync with cmake and
`lib/curl_setup.h`.
- sync detection function name with cmake.
- limit AppleIDN feature check to Darwin.
(also drop !WinIDN precondition check.)
Follow-up to 8de8fe8c98#14401Closes#14419
- nghttp2 is now on by default, adjust build configuration accordingly.
Follow-up to 87aa4ebd82#14136
- GHA/macos: disable libidn2 in combination builds, syncing with
autotools. Otherwise it's now auto-detected via pkg-config by default.
Follow-up to f43adc2c49#14137Closes#14364
- run tests via `make test-ci` instead of `make check` with autotools.
- add `x86_64` job for FreeBSD, with tests.
It matches the existing Cirrus CI job, with these differences:
- finishes 3x faster (thanks to parallel tests enabled).
- librtmp is not enabled because it's slated for removal by FreeBSD.
(already past the removal deadline, thought the package still
installs.)
- DICT and TELNET servers fail to start. Couldn't figure out why.
It means skipping test 1450 and 1452.
- it runs more tests, e.g. websockets and ip6-localhost.
- no `pkg update -f`.
- it misses the `CRYPTOGRAPHY_DONT_BUILD_RUST=1`, `pkg delete curl`,
`chmod 777`, `sudo -u nobody` and `sysctl net.inet.tcp.blackhole`
tricks. The latter is the default in these runners, the others did
not affect results.
- set `-j0` for tests in the NetBSD job. Flaky otherwise.
Closes#14244
`nettle` is a direct dependency of curl, when building with GnuTLS.
Add a new `Find` module to detect it.
Also:
- GHA/macos: drop `nettle` hack no longer necessary.
- add `nettle` to `libcurl.pc`.
- also add `nettle` to `libcurl.pc` in autotools builds.
Follow-up to 781242ffa4#11967Closes#14285
This PR began as an attempt to drop GCC support, after repeated reports
on fallouts when trying to use it on macOS.
Then it transformed into a 3-week project turning up the issues causing
the fallouts, ending up including llvm and all available Xcode / macOS
SDK, macOS runner image, build tools and compiler vendors and versions.
Accumulating 400 sub-commits.
I developed and tested all fixes under this PR, then merged them as
separate patches.
This PR retained CI jobs updates, extensively reworking and extending
them: [1]
At first it seemed GCC and the Apple SDK is "naturally" growing more
incompatible, as Apple added further non-standard features to their
headers. This is partly true, but reality is more complicated.
Besides some issues local to curl, there were bugs in Apple SDK
headers, Homebrew GCC builds, feature missing in the old llvm version
pre-installed on GitHub CI runner images, and subtle incompatibilities
between GCC and llvm/clang when handling language extensions.
Resulting compiler errors seldom pointed to a useful direction, and
internet search was silent about these issues too. Thus, I had to peel
them off layer by layer, using trial and error, and by recognizing
patterns of failures accross 150-200 builds combinations. Exposing
configure logs, and curl_config.h in the CI logs helped too.
1. GCC header compatibility layer ("hack" as GCC calls it)
The toughest issue is GCC's built-in compatibility layer:
https://github.com/gcc-mirror/gcc/tree/master/fixincludes
This patch layer is further patched by a "Darwin compatibility" project
applied on top by Homebrew GCC via:
https://github.com/iains/gcc-12-branchhttps://github.com/iains/gcc-13-branchhttps://github.com/iains/gcc-14-branch
The hack layer is designed in a way that breaks more builds than it
fixes, esp. in context of GHA runners. The idea is to build GCC
specifically for the SDK for the target macOS version. The problem with
this approach is that the Xcode + SDK installed on the local/CI machine
often does not match with the SDK used on while building GCC on
Homebrew's build machines. In these cases the GCC compatibility layer
turns into an "uncompatibility" layer and consistently breaks builds.
curl cannot offer a fix for this, because the solution (I found) is to
patch the toolchain on the local machine. I implemented this for our CI
builds and curl-for-win. In other case the user must do this patching
manually, or choose a compatible GCC + Xcode/SDK combination.
An upstream fix doesn't seem trivial either, because the issue is
ingrained in the compatibility layer's design. Offering an `-fapplesdk`
(or recognizing `-target`) option and/or fixing them within the compiler
would seem like a more robust option, and also how mainline llvm solves
this.
Here's a table summarizing the GCC + SDK combinations and curl build
failures: [2]
More info: https://github.com/curl/curl/issues/10356#issuecomment-2222734103db135f8d72#14119 macos: add workaround for gcc, non-c-ares, IPv6, compile error
Ref: e2db3c475f
Ref: f5c58d7fef
2. Homebrew GCC's `availability` extension
A recent minor Homebrew GCC upgrade caused major breakage. The "Darwin
compatibility" patch applied to GCC implemented the `availability`
compiler attribute in GCC. Apple SDK detected this and enabled using
them, but as it turns out GCC accepts compiler attributes with slightly
different rules than llvm/clang, and how the Apple SDK uses them,
breaking builds.
Affected Homebrew GCC versions are: 12.4.0, 13.3.0 and 14.1.0.
Possibly tracked here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108796
More info: https://github.com/llvm/llvm-project/issues/81767
Commit implementing the `availability` macro:
gcc-12: fd5530b7cb
gcc-13: cb7e4eca68
gcc-14: ff62a10886
That applied to Homebrew GCC (12.4.0):
b904223d98 (diff-89dd0b4176eca7fcc24b591943509bf8a8d6ea904d71e5dfcd6b78fed62fc574R44-R48)
Ref: #13700
More info: https://github.com/curl/curl/pull/14091#issuecomment-2222703468e91fcbac7d#14155 macos: undo `availability` macro enabled by Homebrew gcc
3. Proprietary Apple SDK macros
Apple SDK expects certain macros predefined by the compiler. Missing
them may causes odd issues. Mainline llvm is keeping up with Apple
clang, but it needs a fresh version, while the one installed on GitHub
runners is old (v15). I patched these in `lib/curl_setup.h`.
baa3270846#14134 build: fix llvm 16 or older + Xcode 15 or newer, and gcc
4. Apple SDK header bug
Without certain predefined macros, SDK headers can take a codepath where
it mis-defines its own `TARGET_OS_OSX` macro, which make it break its
own headers later. I patched it in `lib/curl_setup.h`.
ff784af461#14159 build: fix llvm 17 and older + macOS SDK 14.4 and newer
5. `TargetConditionals.h` requires `sys/types.h`
Fixed in curl. It caused feature-detection failurs with autotools, and
could break builds in certain configurations.
e1f6192939#14130 configure: fix `SystemConfiguration` detection
6. Differences between autotools and CMake compiler options
Fixed it by syncing compiler warning options.
59cadacfcc#14128 build: sync warning options between autotools, cmake & compilers
7. Differences between autotools and CMake dependency detection
Fixed it by improving detection of libidn2, with some more fixes
pending for the next feature window.
f43adc2c49#14137 cmake: detect `libidn2` also via `pkg-config`
Ref: #14136 cmake: detect `nghttp2` via `pkg-config`, enable by default
8. libidn2 detection bug with CMake
Fixed the root cause and also the trigger in the CI config.
764fbabf6e#14175 cmake: fix builds with detected libidn2 lib but undetected header
9. Suppressed compiler warnings inside Apple-specific curl code
Fixed these warnings, which allowed to stop silencing them.
b05dc7eb35#14122 sectransp: fix `HAVE_BUILTIN_AVAILABLE` checks to not emit warnings
5fa534b0da#14162 sectransp: fix clang compiler warnings, stop silencing them
10. CMake mis-detecting a CA bundle path on macOS
d2ef6255f4#14182 cmake: sync CA bundle/path detection with autotools
11. Failure to build tests with LibreSSL or wolfSSL with CMake
Fixed by dropping unnecessary includes, makign test builds dependent
on dependency headers.
3765d75ce4#14172 cmake: fix building `unit1600` due to missing `ssl/openssl.h`
12. curl tests with CMake
curl's CMake was missing bits for running the C preprocessor accurately.
It made tests 1119 and 1167 fail. I implemented the missing bits.
efc2c5184d#14124 tests: include current directory when running test Perl commands
c09db8b51b#14129 cmake: create `configurehelp.pm` like autotools does
67cc1e3400#14125 test1119: adapt for `.md` input
13. GCC missing `__builtin_available()` support
curl source code assumes this is available to enable certain codepaths.
It's also intermixed with monotonic timer support.
14. Monotonic timer support with GCC
Detected by GCC, while it probably shouldn't be. llvm/clang detects it
depending on target OS version. I've been playing with this, but so far
without a conclusion or fix.
15. Runtime/test failures with GCC
I couldn't find the reason for most of this. A bunch of RTSP tests fail
with GCC. SecureTransport + HTTP/2 is failing a bunch of tests. With
OpenSSL it fails two of those. SecureTransport builds also fail one DoH
test.
16. Runtime/test failure in llvm/clang
AppleIDN support received a fix with two more remaining.
fd0250869f#14179#14176 IDN: fix ß with AppleIDN
17. Other issues found and fixed while working on this:
2c15aa5765 GHA/macos: delete misplaced `CFLAGS`, drop redundant CMake option
80fb7c0bef#14126 configure: limit `SystemConfiguration` test to non-c-ares, IPv6 builds
cfd6f43d6c#14127 build: tidy up `__builtin_available` feature checks (Apple)
bae5553599#14174 runtests: show name and keywords for failed tests in summary
09cdf7e531#14178 cmake: delete unused `HAVE_LIBSSH2`, `HAVE_LIBSOCKET` macros
d3595c74fa#14186 configure: CA bundle/path detection fixes
58772b0e08#14187 runtests: set `SOURCE_DATE_EPOCH` to fix failing around midnight
18f1cd7a77#14183 tests: sync feature names with `curl -V`
4c22d97be7#14181 build: use `#error` instead of invalid syntax
Pending merges:
- #14185 runtests: fold test details for GitHub CI runs
- #14197 cmake: grab-bag of tidy-ups
- #14196 configure: limit `__builtin_available` test to Darwin
Summary:
In general GCC doesn't seem to be a good fit with curl and macOS for
now. These "lucky" combinations (GitHub Actions runner) will build out
of the box now: macos-14 + Xcode 15.0.1 + gcc-11, gcc-12, gcc-14. The
rest builds with the ugly workaround in place, but all this still leaves
some runtime issues.
More info and links in the commit messages and source code.
[1]: This PR:
- add info about target OS version requirements per feature, with OS
names and release years.
- stop using `-Wno-deprecated-declarations` to suppress warnings.
- use `LDFLAGS=-w` to suppress 'object file was built for newer macOS
version than being linked' warnings.
(there were tens of thousands of them in some jobs)
- allow overriding Xcode version in all jobs.
- improve job names.
- abbreviate CMake as CM, autotools as AM for more compact job names.
- shorten job names by using `!` instead of `no-` and `non-`.
- bump parellel tests to 10 (from 5).
- drop using `--enable-maintainer-mode` `./configure` option.
- add gcc-12 no-ssl, autotools job with tests, ignore failing test
results. (It's not yet clear why gcc-12 builds have different runtime
results than clang/llvm ones.)
- add comments with OS names and release years next to version numbers,
e.g. 10.15 # Catalina (2019)
- fix broken gcc-12 SecureTransport build.
- show compiler, Xcode, SDK, gcc hack SDK versions, Homebrew
preinstalled packages and C compiler predefined macros for each job.
Useful for debugging all the strange problems these builds might have.
- merge brew bundle and install steps.
- move step names to the top.
- dump configure log for both cmake and autotools also for successful
builds. Useful for debugging.
- dump curl_config.h in short (sorted #defines) and full form.
- add support for the mainline llvm compiler.
- set sysroot for gcc and llvm.
- add timeout for cmake jobs.
- add new job matrix: combinations
It supports building all possible compiler, runner image, Xcode/SDK
combinations, with cmake and autotools, target OS versions and with or
without SecureTransport. It's quick. GHA limits the maximum number of
matrix jobs at 256.
I used this as a test-rig to fix the macOS build fallouts with gcc and
llvm.
I settled with 16 jobs, trying to maximize fallout coverage.
- implement hack to make Homebrew gcc work with all available SDKs.
- add handy mini-table about Xcode / SDK versions, OS names, years for
each GHA images, with the defaults.
- add tests for cmake jobs.
- make cmake config hack to link GnuTLS less intrusive.
- stop ignoring test 1452, seems fine now.
- fix to enable libpsl in autotools builds.
- enable libpsl in cmake builds.
- add an llvm job with tests (both autotools and cmake).
- delete similar macOS jobs from Circle CI. GHA is now arm64 too.
[2]: Homebrew GCC vs GHA runner images vs curl builds:
```
macOS Xcode gcc gcc SDK hacks Xcode SDK SDK major Build Compile
(*def) (Homebrew) (CommandLineTools) versions error
-------- -------- ---------- ------------------ ---------- --------- ----- ---------------------
macos-12 13.1 GCC 11.4.0 MacOSX12 MacOSX12.0
macos-12 13.2.1 GCC 11.4.0 MacOSX12 MacOSX12.1
macos-12 13.3.1 GCC 11.4.0 MacOSX12 MacOSX12.3
macos-12 13.4.1 GCC 11.4.0 MacOSX12 MacOSX12.3
macos-12 14.0.1 GCC 11.4.0 MacOSX12 MacOSX12.3
macos-12 14.1 GCC 11.4.0 MacOSX12 MacOSX13.0 MISMATCH FAIL /Applications/Xcode_14.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/os/object.h:275:1: error: expected ';' before 'extern'
macos-12 *14.2 GCC 11.4.0 MacOSX12 MacOSX13.1 MISMATCH FAIL /Applications/Xcode_14.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/os/object.h:275:1: error: expected ';' before 'extern'
macos-13 14.1 GCC 11.4.0 MacOSX13 MacOSX13.0
macos-13 14.2 GCC 11.4.0 MacOSX13 MacOSX13.1
macos-13 14.3.1 GCC 11.4.0 MacOSX13 MacOSX13.3
macos-13 *15.0.1 GCC 11.4.0 MacOSX13 MacOSX14.0 MISMATCH FAIL /Applications/Xcode_15.0.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:103:1: error: unknown type name 'dispatch_queue_t'
macos-13 15.1 GCC 11.4.0 MacOSX13 MacOSX14.2 MISMATCH FAIL /Applications/Xcode_15.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:103:1: error: unknown type name 'dispatch_queue_t'
macos-13 15.2 GCC 11.4.0 MacOSX13 MacOSX14.2 MISMATCH FAIL /Applications/Xcode_15.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:103:1: error: unknown type name 'dispatch_queue_t'
macos-14 14.3.1 GCC 11.4.0 MacOSX14 MacOSX13.3 MISMATCH FAIL /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or more data types in declaration specifiers
macos-14 *15.0.1 GCC 11.4.0 MacOSX14 MacOSX14.0
macos-14 15.1 GCC 11.4.0 MacOSX14 MacOSX14.2
macos-14 15.2 GCC 11.4.0 MacOSX14 MacOSX14.2
macos-14 15.3 GCC 11.4.0 MacOSX14 MacOSX14.4
macos-14 15.4 GCC 11.4.0 MacOSX14 MacOSX14.5
macos-14 16.0 GCC 11.4.0 MacOSX14 MacOSX15.0 MISMATCH FAIL /opt/homebrew/Cellar/gcc@11/11.4.0/lib/gcc/11/gcc/aarch64-apple-darwin23/11/include-fixed/stdio.h:83:8: error: unknown type name 'FILE'
macos-12 13.1 GCC 12.4.0 MacOSX12 MacOSX12.0
macos-12 13.2.1 GCC 12.4.0 MacOSX12 MacOSX12.1
macos-12 13.3.1 GCC 12.4.0 MacOSX12 MacOSX12.3
macos-12 13.4.1 GCC 12.4.0 MacOSX12 MacOSX12.3
macos-12 14.0.1 GCC 12.4.0 MacOSX12 MacOSX12.3
macos-12 14.1 GCC 12.4.0 MacOSX12 MacOSX13.0 MISMATCH FAIL /Applications/Xcode_14.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/os/object.h:275:1: error: expected ';' before 'extern'
macos-12 *14.2 GCC 12.4.0 MacOSX12 MacOSX13.1 MISMATCH FAIL /Applications/Xcode_14.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/os/object.h:275:1: error: expected ';' before 'extern'
macos-13 14.1 GCC 12.4.0 MacOSX13 MacOSX13.0
macos-13 14.2 GCC 12.4.0 MacOSX13 MacOSX13.1
macos-13 14.3.1 GCC 12.4.0 MacOSX13 MacOSX13.3
macos-13 *15.0.1 GCC 12.4.0 MacOSX13 MacOSX14.0 MISMATCH FAIL /Applications/Xcode_15.0.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:103:1: error: unknown type name 'dispatch_queue_t'
macos-13 15.1 GCC 12.4.0 MacOSX13 MacOSX14.2 MISMATCH FAIL /Applications/Xcode_15.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:103:1: error: unknown type name 'dispatch_queue_t'
macos-13 15.2 GCC 12.4.0 MacOSX13 MacOSX14.2 MISMATCH FAIL /Applications/Xcode_15.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:103:1: error: unknown type name 'dispatch_queue_t'
macos-14 14.3.1 GCC 12.4.0 MacOSX14 MacOSX13.3 MISMATCH
macos-14 *15.0.1 GCC 12.4.0 MacOSX14 MacOSX14.0
macos-14 15.1 GCC 12.4.0 MacOSX14 MacOSX14.2
macos-14 15.2 GCC 12.4.0 MacOSX14 MacOSX14.2
macos-14 15.3 GCC 12.4.0 MacOSX14 MacOSX14.4
macos-14 15.4 GCC 12.4.0 MacOSX14 MacOSX14.5
macos-14 16.0 GCC 12.4.0 MacOSX14 MacOSX15.0 MISMATCH FAIL /opt/homebrew/Cellar/gcc@12/12.4.0/lib/gcc/12/gcc/aarch64-apple-darwin23/12/include-fixed/stdio.h:83:8: error: unknown type name 'FILE'
macos-12 13.1 GCC 13.3.0 MacOSX12 MacOSX12.0
macos-12 13.2.1 GCC 13.3.0 MacOSX12 MacOSX12.1
macos-12 13.3.1 GCC 13.3.0 MacOSX12 MacOSX12.3
macos-12 13.4.1 GCC 13.3.0 MacOSX12 MacOSX12.3
macos-12 14.0.1 GCC 13.3.0 MacOSX12 MacOSX12.3
macos-12 14.1 GCC 13.3.0 MacOSX12 MacOSX13.0 MISMATCH FAIL /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or more data types in declaration specifiers
macos-12 *14.2 GCC 13.3.0 MacOSX12 MacOSX13.1 MISMATCH FAIL /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or more data types in declaration specifiers
macos-13 14.1 GCC 13.3.0 MacOSX13 MacOSX13.0
macos-13 14.2 GCC 13.3.0 MacOSX13 MacOSX13.1
macos-13 14.3.1 GCC 13.3.0 MacOSX13 MacOSX13.3
macos-13 *15.0.1 GCC 13.3.0 MacOSX13 MacOSX14.0 MISMATCH FAIL /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or more data types in declaration specifiers
macos-13 15.1 GCC 13.3.0 MacOSX13 MacOSX14.2 MISMATCH FAIL /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or more data types in declaration specifiers
macos-13 15.2 GCC 13.3.0 MacOSX13 MacOSX14.2 MISMATCH FAIL /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or more data types in declaration specifiers
macos-14 14.3.1 GCC 13.3.0 MacOSX14 MacOSX13.3 MISMATCH FAIL /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or more data types in declaration specifiers
macos-14 *15.0.1 GCC 13.3.0 MacOSX14 MacOSX14.0 FAIL /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or more data types in declaration specifiers
macos-14 15.1 GCC 13.3.0 MacOSX14 MacOSX14.2 FAIL /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or more data types in declaration specifiers
macos-14 15.2 GCC 13.3.0 MacOSX14 MacOSX14.2 FAIL /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or more data types in declaration specifiers
macos-14 15.3 GCC 13.3.0 MacOSX14 MacOSX14.4
macos-14 15.4 GCC 13.3.0 MacOSX14 MacOSX14.5
macos-14 16.0 GCC 13.3.0 MacOSX14 MacOSX15.0 MISMATCH FAIL /opt/homebrew/Cellar/gcc@13/13.3.0/lib/gcc/13/gcc/aarch64-apple-darwin23/13/include-fixed/stdio.h:83:8: error: unknown type name 'FILE'
macos-12 13.1 GCC 14.1.0 MacOSX12 MacOSX12.0
macos-12 13.2.1 GCC 14.1.0 MacOSX12 MacOSX12.1
macos-12 13.3.1 GCC 14.1.0 MacOSX12 MacOSX12.3
macos-12 13.4.1 GCC 14.1.0 MacOSX12 MacOSX12.3
macos-12 14.0.1 GCC 14.1.0 MacOSX12 MacOSX12.3
macos-12 14.1 GCC 14.1.0 MacOSX12 MacOSX13.0 MISMATCH FAIL /Applications/Xcode_14.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/os/object.h:275:1: error: expected ';' before 'extern'
macos-12 *14.2 GCC 14.1.0 MacOSX12 MacOSX13.1 MISMATCH FAIL /Applications/Xcode_14.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/os/object.h:275:1: error: expected ';' before 'extern'
macos-13 14.1 GCC 14.1.0 MacOSX13 MacOSX13.0
macos-13 14.2 GCC 14.1.0 MacOSX13 MacOSX13.1
macos-13 14.3.1 GCC 14.1.0 MacOSX13 MacOSX13.3
macos-13 *15.0.1 GCC 14.1.0 MacOSX13 MacOSX14.0 MISMATCH FAIL /Applications/Xcode_15.0.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:70:1: error: type defaults to 'int' in declaration of 'DISPATCH_DECL_FACTORY_CLASS_SWIFT' [-Wimplicit-int]
macos-13 15.1 GCC 14.1.0 MacOSX13 MacOSX14.2 MISMATCH FAIL /Applications/Xcode_15.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:70:1: error: type defaults to 'int' in declaration of 'DISPATCH_DECL_FACTORY_CLASS_SWIFT' [-Wimplicit-int]
macos-13 15.2 GCC 14.1.0 MacOSX13 MacOSX14.2 MISMATCH FAIL /Applications/Xcode_15.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:70:1: error: type defaults to 'int' in declaration of 'DISPATCH_DECL_FACTORY_CLASS_SWIFT' [-Wimplicit-int]
macos-14 14.3.1 GCC 14.1.0 MacOSX14 MacOSX13.3 MISMATCH
macos-14 *15.0.1 GCC 14.1.0 MacOSX14 MacOSX14.0
macos-14 15.1 GCC 14.1.0 MacOSX14 MacOSX14.2
macos-14 15.2 GCC 14.1.0 MacOSX14 MacOSX14.2
macos-14 15.3 GCC 14.1.0 MacOSX14 MacOSX14.4
macos-14 15.4 GCC 14.1.0 MacOSX14 MacOSX14.5
macos-14 16.0 GCC 14.1.0 MacOSX14 MacOSX15.0 MISMATCH FAIL /opt/homebrew/Cellar/gcc/14.1.0_1/lib/gcc/current/gcc/aarch64-apple-darwin23/14/include-fixed/stdio.h:83:8: error: unknown type name 'FILE'
```
Source: https://github.com/curl/curl/actions/runs/9883956647/job/27299564218
This commit fixes earlier commit
1e75edd372, reverted in
41a7e0dcc9681afd91e066411bcee4f369c23366, where I cut the commit
message in half by accident. The patch itself is identical.
Closes#14097
This PR began as an attempt to drop GCC support, after repeated reports
on fallouts when trying to use it on macOS.
Then it transformed into a 3-week project turning up the issues causing
the fallouts, ending up including llvm and all available Xcode / macOS
SDK, macOS runner image, build tools and compiler vendors and versions.
Accumulating 400 sub-commits.
I developed and tested all fixes under this PR, then merged them as
separate patches.
This PR retained CI jobs updates, extensively reworking and extending
them: [1]
At first it seemed GCC and the Apple SDK is "naturally" growing more
incompatible, as Apple added further non-standard features to their
headers. This is partly true, but reality is more complicated.
Besides some issues local to curl, there were bugs in Apple SDK
headers, Homebrew GCC builds, feature missing in the old llvm version
pre-installed on GitHub CI runner images, and subtle incompatibilities
between GCC and llvm/clang when handling language extensions.
Resulting compiler errors seldom pointed to a useful direction, and
internet search was silent about these issues too. Thus, I had to peel
them off layer by layer, using trial and error, and by recognizing
patterns of failures accross 150-200 builds combinations. Exposing
configure logs, and curl_config.h in the CI logs helped too.
1. GCC header compatibility layer ("hack" as GCC calls it)
The toughest issue is GCC's built-in compatibility layer:
https://github.com/gcc-mirror/gcc/tree/master/fixincludes
This patch layer is further patched by a "Darwin compatibility" project
applied on top by Homebrew GCC via:
https://github.com/iains/gcc-12-branchhttps://github.com/iains/gcc-13-branchhttps://github.com/iains/gcc-14-branch
The hack layer is designed in a way that breaks more builds than it
fixes, esp. in context of GHA runners. The idea is to build GCC
specifically for the SDK for the target macOS version. The problem with
this approach is that the Xcode + SDK installed on the local/CI machine
often does not match with the SDK used on while building GCC on
Homebrew's build machines. In these cases the GCC compatibility layer
turns into an "uncompatibility" layer and consistently breaks builds.
curl cannot offer a fix for this, because the solution (I found) is to
patch the toolchain on the local machine. I implemented this for our CI
builds and curl-for-win. In other case the user must do this patching
manually, or choose a compatible GCC + Xcode/SDK combination.
An upstream fix doesn't seem trivial either, because the issue is
ingrained in the compatibility layer's design. Offering an `-fapplesdk`
(or recognizing `-target`) option and/or fixing them within the compiler
would seem like a more robust option, and also how mainline llvm solves
this.
Here's a table summarizing the GCC + SDK combinations and curl build
failures: [2]
More info: https://github.com/curl/curl/issues/10356#issuecomment-2222734103db135f8d72#14119 macos: add workaround for gcc, non-c-ares, IPv6, compile error
Ref: e2db3c475f
Ref: f5c58d7fef
2. Homebrew GCC's `availability` extension
A recent minor Homebrew GCC upgrade caused major breakage. The "Darwin
compatibility" patch applied to GCC implemented the `availability`
compiler attribute in GCC. Apple SDK detected this and enabled using
them, but as it turns out GCC accepts compiler attributes with slightly
different rules than llvm/clang, and how the Apple SDK uses them,
breaking builds.
Affected Homebrew GCC versions are: 12.4.0, 13.3.0 and 14.1.0.
Possibly tracked here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108796
More info: https://github.com/llvm/llvm-project/issues/81767
Commit implementing the `availability` macro:
gcc-12: fd5530b7cb
gcc-13: cb7e4eca68
gcc-14: ff62a10886
That applied to Homebrew GCC (12.4.0):
b904223d98 (diff-89dd0b4176eca7fcc24b591943509bf8a8d6ea904d71e5dfcd6b78fed62fc574R44-R48)
Ref: #13700
More info: https://github.com/curl/curl/pull/14091#issuecomment-2222703468e91fcbac7d#14155 macos: undo `availability` macro enabled by Homebrew gcc
3. Proprietary Apple SDK macros
Apple SDK expects certain macros predefined by the compiler. Missing
them may causes odd issues. Mainline llvm is keeping up with Apple
clang, but it needs a fresh version, while the one installed on GitHub
runners is old (v15). I patched these in `lib/curl_setup.h`.
baa3270846#14134 build: fix llvm 16 or older + Xcode 15 or newer, and gcc
4. Apple SDK header bug
Without certain predefined macros, SDK headers can take a codepath where
it mis-defines its own `TARGET_OS_OSX` macro, which make it break its
own headers later. I patched it in `lib/curl_setup.h`.
ff784af461#14159 build: fix llvm 17 and older + macOS SDK 14.4 and newer
5. `TargetConditionals.h` requires `sys/types.h`
Fixed in curl. It caused feature-detection failurs with autotools, and
could break builds in certain configurations.
e1f6192939#14130 configure: fix `SystemConfiguration` detection
6. Differences between autotools and CMake compiler options
Fixed it by syncing compiler warning options.
59cadacfcc#14128 build: sync warning options between autotools, cmake & compilers
7. Differences between autotools and CMake dependency detection
Fixed it by improving detection of libidn2, with some more fixes
pending for the next feature window.
f43adc2c49#14137 cmake: detect `libidn2` also via `pkg-config`
Ref: #14136 cmake: detect `nghttp2` via `pkg-config`, enable by default
8. libidn2 detection bug with CMake
Fixed the root cause and also the trigger in the CI config.
764fbabf6e#14175 cmake: fix builds with detected libidn2 lib but undetected header
9. Suppressed compiler warnings inside Apple-specific curl code
Fixed these warnings, which allowed to stop silencing them.
b05dc7eb35#14122 sectransp: fix `HAVE_BUILTIN_AVAILABLE` checks to not emit warnings
5fa534b0da#14162 sectransp: fix clang compiler warnings, stop silencing them
10. CMake mis-detecting a CA bundle path on macOS
d2ef6255f4#14182 cmake: sync CA bundle/path detection with autotools
11. Failure to build tests with LibreSSL or wolfSSL with CMake
Fixed by dropping unnecessary includes, makign test builds dependent
on dependency headers.
3765d75ce4#14172 cmake: fix building `unit1600` due to missing `ssl/openssl.h`
12. curl tests with CMake
curl's CMake was missing bits for running the C preprocessor accurately.
It made tests 1119 and 1167 fail. I implemented the missing bits.
efc2c5184d#14124 tests: include current directory when running test Perl commands
c09db8b51b#14129 cmake: create `configurehelp.pm` like autotools does
67cc1e3400#14125 test1119: adapt for `.md` input
13. GCC missing `__builtin_available()` support
curl source code assumes this is available to enable certain codepaths.
It's also intermixed with monotonic timer support.
14. Monotonic timer support with GCC
Detected by GCC, while it probably shouldn't be. llvm/clang detects it
depending on target OS version. I've been playing with this, but so far
without a conclusion or fix.
15. Runtime/test failures with GCC
I couldn't find the reason for most of this. A bunch of RTSP tests fail
with GCC. SecureTransport + HTTP/2 is failing a bunch of tests. With
OpenSSL it fails two of those. SecureTransport builds also fail one DoH
test.
16. Runtime/test failure in llvm/clang
AppleIDN support received a fix with two more remaining.
fd0250869f#14179#14176 IDN: fix ß with AppleIDN
17. Other issues found and fixed while working on this:
2c15aa5765 GHA/macos: delete misplaced `CFLAGS`, drop redundant CMake option
80fb7c0bef#14126 configure: limit `SystemConfiguration` test to non-c-ares, IPv6 builds
cfd6f43d6c#14127 build: tidy up `__builtin_available` feature checks (Apple)
bae5553599#14174 runtests: show name and keywords for failed tests in summary
09cdf7e531#14178 cmake: delete unused `HAVE_LIBSSH2`, `HAVE_LIBSOCKET` macros
d3595c74fa#14186 configure: CA bundle/path detection fixes
58772b0e08#14187 runtests: set `SOURCE_DATE_EPOCH` to fix failing around midnight
18f1cd7a77#14183 tests: sync feature names with `curl -V`
4c22d97be7#14181 build: use `#error` instead of invalid syntax
Pending merge:
Simplify making clean builds by silencing deprecation warnings inside
the example code where these may occur.
Drop related build tweaks/comments from GHA jobs.
Example warning:
```
curl/docs/examples/postit2-formadd.c:65:16: error: 'CURLFORM_COPYNAME' is deprecated: since 7.56.0. Use curl_mime_name() [-Werror=deprecated-declarations]
65 | CURLFORM_COPYNAME, "sendfile",
| ^~~~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/9841099503/job/27166970904#step:10:829Closes#14123
With macOS there is a long-term struggle with deprecation warnings.
In curl they occur with LDAP, SecureTransport and in docs/examples.
There are three ways to fix them:
- by CFLAGS `-Wno-deprecated-declarations` as a workaround.
- by CFLAGS `-mmacosx-version-min` set to a version where the the
feature was not deprecated.
- by CMake option `-DCMAKE_OSX_DEPLOYMENT_TARGET=`.
In GHA CMake jobs, all three were used, and `-mmacosx-version-min` was
set in a bogus way. Delete that bogus option, and delete the lone,
redundant CMake option too.
In a future commit I might replace the suppression option to properly
setting the target OS.
Follow-up to dfdd978f7c#13491
Cherry-picked from #14097
- support detecting wolfSSL via pkg-config (like autotools.)
- detect wolfSSL version.
- detect `HAVE_WOLFSSL_DES_ECB_ENCRYPT`.
(needs e.g. `--enable-curl` when building wolfSSL)
- detect `HAVE_WOLFSSL_FULL_BIO` and enable HTTPS-proxy feature.
(needs e.g. `--enable-opensslall` when building wolfSSL)
- fix to show `HTTPS-proxy` in cmake feature list.
Ref: 55807e6c05#9962
- fix to show `NTLM` in cmake feature list.
- fix to show `smb` and `smbs` in cmake protocol list.
- add wolfSSL CMake job to GHA (for macOS).
- fix mqtt and wolfSSL symbol clash.
```
./curl/lib/mqtt.c: In function 'mqtt_doing':
./curl/lib/mqtt.c:746:17: error: declaration of 'byte' shadows a global declaration [-Werror=shadow]
746 | unsigned char byte;
| ^~~~
/opt/homebrew/Cellar/wolfssl/5.7.0_1/include/wolfssl/wolfcrypt/types.h:85:36: note: shadowed declaration is here
85 | typedef unsigned char byte;
| ^~~~
```
- format `FindWolfSSL.cmake` closer to neighbours.
Closes#14064
- bump rest of the workflows (windows, macos, distrocheck).
- non-native virtualized envs have 2 CPUs, bump down accordingly.
(for `vmactions/omnios-vm` it's just a guess.)
- bump all to nproc + 1.
Follow-up to e838b341a0#12927Closes#13807
For better build performance. Dependency tracking causes a build
overhead while compiling to help a subsequent build, but in CI there is
never one and the extra work is discarded.
Closes#13794
You can enable it with `-DBUILD_EXAMPLES=ON`.
To match autotools' `make examples` feature.
Windows (static) builds not tested.
Also enable examples in a pair of CI jobs.
Apply related updates to the macOS CI workflow:
- drop unused `CXX` envs.
- drop no longer needed `-Wno-error=undef -Wno-error=conversion` flags.
- pass `-Wno-deprecated-declarations` to GCC too (for `BUILD_EXAMPLES`).
- document why `-Wno-deprecated-declarations` is necessary.
Closes#13491
- based on #13478 with additions from #13476
- make homebrew install path flexible
- fix OpenSSL pkgconfig files libdir
- add path to --with-libssh2 target
- disable gcc securetransport due to linker
errors (missing symbols), probably because
the os version is no longer low enough
Assisted-by: Viktor Szakats
Closes#13479
I implemented the IDN functions for macOS and iOS using Unicode
libraries coming with macOS and iOS.
Builds and runs here on macOS 14.2.1. Also verified to load and
run on older macOS version 10.13.
Build requires macOS SDK 13 or equivalent.
Set `-DUSE_APPLE_IDN=ON` CMake option to enable it.
With autotools and other build tools, set these manual options:
```
CPPFLAGS=-DUSE_APPLE_IDN
LIBS=-licucore
```
Completes TODO 1.6.
TODO: add autotools option and feature-detection.
Refs: #5330#5371
Co-authored-by: Viktor Szakats
Closes#13246
Unity mode is not supported by CMake v3.7.2 used in linux-old, but
enable it anyway for consistency and to kick in automatically once
migrating to a newer old Linux in the future.
Also:
- replace `CMAKE_COMPILE_WARNING_AS_ERROR` with `CURL_WERROR`.
- delete default build option `PICKY_COMPILER=ON`.
Closes#13277
PowerShell works (after a steep development curve), but one property of
it stuck and kept causing unresolvable usability issues: With
`$ErrorActionPreference=Stop`, it does abort on failures, but shows only
the first line of the error message. In `Continue` mode, it shows the
full error message, but doesn't stop on all errors. Another issue is
PowerShell considering any stderr output as if the command failed (this
has been improved in 7.2 (2021-Nov), but fixed versions aren't running
in CI and will not be for a long time in all test images.)
Thus, we're going with bash.
Also:
- use `-j2` with autotools tests, making them finish 5-15 minutes per
job faster.
- omit `POSIX_PATH_PREFIX`.
- use `WINDIR`.
- prefer forward slashes.
Follow-up to: 75078a415d#11999
Ref: #12444Fixes#12560Closes#12572
- set `HAVE_LDAP_URL_PARSE` if `ldap_url_parse` function exists.
Before this patch we set it based it on the presence of `stricmp`,
which correctly enabled it on e.g. Windows, but was inaccurate for
other platforms.
- always set `HAVE_LDAP_SSL` if an LDAP backend is detected and
LDAPS is not explicitly disabled. This mimics autotools behaviour.
Previously we set it only for Windows LDAP. After this fix, LDAPS is
correctly enabled in default macOS builds.
- enable LDAP[S] for a CMake macOS CI job. Target OS X 10.9 (Mavericks)
to avoid deprecation warnings for LDAP API.
- always detect `HAVE_LDAP_SSL_H`, even with LDAPS explicitly disabled.
This doesn't make much sense, but let's do it to sync behaviour with
autotools.
- fix benign typo in variable name.
Ref: #11964 (effort to sync cmake detections with autotools)
Closes#12006
With new option `CURL_DISABLE_SRP=ON` to force-disable it.
To match existing option and detection logic in autotools.
Also:
- fix detecting GnuTLS.
We assume `nettle` as a GnuTLS dependency.
- add CMake GnuTLS CI job.
- bump AppVeyor CMake OpenSSL MSVC job to OpenSSL 1.1.1 (from 1.0.2)
TLS-SRP fails to detect with 1.0.2 due to an OpenSSL header bug.
- fix compiler warning when building with GnuTLS and disabled TLS-SRP.
- fix comment typos, whitespace.
Ref: #11964Closes#11967
This patch adds the ability to build a static and shared libcurl library
in a single build session. It also adds an option to select which one to
use when building the curl executable.
New build options:
- `BUILD_STATIC_LIBS`. Default: `OFF`.
Enabled automatically if `BUILD_SHARED_LIBS` is `OFF`.
- `BUILD_STATIC_CURL`. Default: `OFF`.
Requires `BUILD_STATIC_LIBS` enabled.
Enabled automatically if building static libcurl only.
- `STATIC_LIB_SUFFIX`. Default: empty.
- `IMPORT_LIB_SUFFIX`. Default: `_imp` if implib filename would collide
with static lib name (typically with MSVC) in Windows builds.
Otherwise empty.
Also:
- Stop setting the `CURL_STATICLIB` macro via `curl_config.h`, and pass
it directly to the compiler. This also allows to delete a condition
from `tests/server/CMakeLists.txt`.
- Complete a TODO by following the logic used in autotools (also for
`LIBCURL_NO_SHARED`), and set `-DCURL_STATICLIB` in `Cflags:` of
`libcurl.pc` for _static-only_ curl builds.
- Convert an existing CI test to build both shared and static libcurl.
Closes#11505
If brew install/update links openssl into /usr/local, it will be found
before anything we add with `-isystem path` to CPP/LDLFAGS. Get rid of
that by unlinking the keg.
Fixes#11413Closes#11436
Most CI services provide at least two cores, so enable parallel make
jobs to take advantage of that for builds. Some dependencies aren't safe
to build in parallel so leave those as-is. Also, rename a few
workflows to eliminate duplicate names and provide a better idea what
they're about.
Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.
You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.
It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.
Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.
Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
`statename`.
- rename `ssl_backend_data` structure to unique names for each
TLS-backend, along with the `ssl_connect_data` struct member
referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
builds when compiling `lib/ldap.c`. To support "unity" builds, we had
to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
`schannel.h` option) _globally_. This caused an indirect inclusion of
Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
as well. This requires `[P]UNICODE_STRING` types, which is apperantly
not defined automatically (as seen with both MSVS and mingw-w64).
This patch includes `<subauth.h>` to fix it.
Ref: https://github.com/curl/curl/runs/13987772013
Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
is built in static mode.
KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250
Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_buildCloses#11095
Also skip builds on non-Windows platforms when only Windows build files
have changed.
This should reduce the number of useless builds and the associated
waiting time and chance of spurious failures, freeing resources for
new PRs.
Closes#10742
