luo980/curl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
A command line tool and library for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET, TFTP, WS and WSS. libcurl offers a myriad of powerful features
34,225 Commits 20 Branches 224 Tags 202 MiB
C 74.8%
Perl 8.1%
M4 4.8%
Python 4.3%
CMake 2.7%
Other 5.3%
1bf774df57
Go to file
Viktor Szakats 1bf774df57
GHA/windows: mitigate drastic runtests perf drop under MSYS2 
Today GHA Windows runner images (all versions) deployed an upgrade
(20250127.1.0 -> 20250203.1.0) that upgraded the default MSYS2, which
now seems to feature the October 2024 issue that caused curl runtests
run times increasing ~2.5x. It also causes test987 to fail, and vcpkg
jobs hitting their time limits and fail. Reliability also got a hit.

In October this issue came with a Git for Windows upgrade, and likely
the MSYS2 runtime update within it. It affected vcpkg jobs only, and
I mitigated it by switching them to use the default MSYS2 shell and
runtime (at `C:\msys64`):

5f9411f953 #15380

After today's update this mitigation no longer works. The issue also
affects `dl-mingw` jobs now, though to a lesser extent than vcpkg ones.

Tried switching back to Git for Windows which received several updates
since October, but the performance issue is still present.

I managed to mitigate the slowdown in vcpkg by lowering test parallelism
to `-j4` (from `-j8`), after which the jobs are about *half the speed*
than before, and fit their time limits. `dl-mingw` builds run slower by
1-1.5 minutes per job, they were already using `-j4`.

Example jobs:

Before (ALL GOOD):
https://github.com/curl/curl/actions/runs/13167230443/job/36750175428 installed MSYS2, mingw (-j8): 3m50s (OK)
https://github.com/curl/curl/actions/runs/13167230443/job/36750158662 default MSYS2, dl-mingw (-j4): 4m22s (OK)
https://github.com/curl/curl/actions/runs/13167230443/job/36750163392 default MSYS2, vcpkg (-j8): 3m27s (OK)
runner: https://github.com/actions/runner-images/blob/win22/20250127.1/images/windows/Windows2022-Readme.md
C:\msys64:
System: MSYS_NT-10.0-20348 fv-az1115-916 3.5.4-0bc1222b.x86_64 2024-12-05 09:27 UTC x86_64 Msys
https://github.com/msys2/msys2-runtime/commit/0bc1222b

After:
https://github.com/curl/curl/actions/runs/13186498273/job/36809747078 installed MSYS2, mingw (-j8): 3m48s (OK)
https://github.com/curl/curl/actions/runs/13186498273/job/36809728481 default MSYS2, dl-mingw (-j4): 5m56s (SLOW)
https://github.com/curl/curl/actions/runs/13186498273/job/36809736429 default MSYS2, vcpkg (-j8): 9m1s (SLOW)
runner: https://github.com/actions/runner-images/blob/win22/20250203.1/images/windows/Windows2022-Readme.md
C:\msys64:
System: MSYS_NT-10.0-20348 fv-az1115-498 3.5.7-2644508f.x86_64 2025-01-30 09:08 UTC x86_64 Msys
https://github.com/msys2/msys2-runtime/commit/2644508f

windows-2025 image:
C:\msys64:
System: MSYS_NT-10.0-26100 fv-az2043-515 3.5.7-2644508f.x86_64 2025-01-30 09:08 UTC x86_64 Msys

windows-2019 image:
C:\msys64:
System: MSYS_NT-10.0-17763 fv-az1434-677 3.5.7-2644508f.x86_64 2025-01-30 09:08 UTC x86_64 Msys

This PR:
final: https://github.com/curl/curl/actions/runs/13186498273/job/36809736429 GfW, vcpkg (*-j4*): ~7m (SLOW)
test: https://github.com/curl/curl/actions/runs/13187992987/job/36814644852?pr=16217, GfW, vcpkg (-j8): ~11m (SLOWER)

Before and after (unused) Git for Windows (SLOW as tested in this PR):
C:\Program Files\Git
System: MINGW64_NT-10.0-20348 fv-az1760-186 3.5.4-395fda67.x86_64 2024-11-25 09:49 UTC x86_64 Msys
https://github.com/msys2/msys2-runtime/commit/395fda67 (fork)

Before and after (used) MSYS2 installed via msys2/setup-msys2 (OK):
D:\a\_temp\msys64
System: MINGW64_NT-10.0-20348 fv-az836-378 3.5.4-0bc1222b.x86_64 2024-12-05 09:27 UTC x86_64 Msys

Perl pipe issue report from October, still open:
https://github.com/msys2/msys2-runtime/issues/230
ARM deadlock fixed by GfW 2.47.1(1), but for x86_64, on a quick glance:
290bea94d0
Possibly interesting:
https://github.com/msys2/msys2-autobuild/issues/62

Closes #16217
2025-02-07 02:44:09 +01:00
.circleci GHA/linux: enable wolfSSH in a wolfSSL job 2025-01-17 00:20:54 +01:00
.github GHA/windows: mitigate drastic runtests perf drop under MSYS2 2025-02-07 02:44:09 +01:00
CMake cmake: add integration tests, run them in CI 2025-02-07 00:15:48 +01:00
docs libssh: drop support for libssh older than 0.9.0 2025-02-06 22:27:13 +01:00
include RELEASE-NOTES: synced 2025-02-06 10:30:07 +01:00
lib ldap: drop support for legacy Novell LDAP SDK 2025-02-06 23:45:05 +01:00
LICENSES copyright: update all copyright lines and remove year ranges 2023-01-03 09:19:21 +01:00
m4 build: drop macro used to enable -Wsign-conversion warnings in CI 2025-02-03 22:28:08 +01:00
packages build: add tool_hugehelp.c into IBMi build 2025-02-06 14:53:36 +01:00
plan9 copyright: update all copyright lines and remove year ranges 2023-01-03 09:19:21 +01:00
projects tidy-up: .gitignore lines mostly 2025-01-27 20:59:46 +01:00
scripts cmake: add integration tests, run them in CI 2025-02-07 00:15:48 +01:00
src tool_ssls: switch to tool-specific get_line function 2025-02-06 22:25:35 +01:00
tests cmake: add integration tests, run them in CI 2025-02-07 00:15:48 +01:00
winbuild tidy-up: .gitignore lines mostly 2025-01-27 20:59:46 +01:00
.dir-locals.el copyright: update all copyright lines and remove year ranges 2023-01-03 09:19:21 +01:00
.git-blame-ignore-revs copyright: update all copyright lines and remove year ranges 2023-01-03 09:19:21 +01:00
.gitattributes winbuild: MS-DOS batch tidy-ups 2024-07-02 19:26:15 +02:00
.gitignore tidy-up: .gitignore lines mostly 2025-01-27 20:59:46 +01:00
.mailmap ECH: update APIs to those agreed with OpenSSL maintainers 2025-01-10 08:36:25 +01:00
acinclude.m4 configure: do not inline 'dnl' comments [ci skip] 2025-01-17 02:38:10 +01:00
appveyor.sh build: drop tool_hugehelp.c.cvs, tidy up macros, drop buildconf.bat 2025-01-26 14:22:49 +01:00
appveyor.yml appveyor: bump to OpenSSL 3.4 2025-01-21 12:42:33 +01:00
buildconf copyright: update all copyright lines and remove year ranges 2023-01-03 09:19:21 +01:00
CHANGES.md CHANGES: rename to CHANGES.md, no longer generated 2024-08-01 13:37:12 +02:00
CMakeLists.txt cmake: drop LDAP_DEPRECATED=1 macro, to sync with autotools 2025-02-07 00:07:38 +01:00
configure.ac configure/cmake: set asyn-rr a feature only if httpsrr is enabled 2025-02-06 11:42:17 +01:00
COPYING COPYING: bump copyright year range to 1996 - 2025 2025-01-01 21:12:12 +01:00
curl-config.in curl-config: tidy up, optimize 2024-12-24 01:21:38 +01:00
Dockerfile Dockerfile: Update debian:bookworm-slim Docker digest to b73bf02 2024-12-03 09:25:09 +01:00
GIT-INFO.md build: drop tool_hugehelp.c.cvs, tidy up macros, drop buildconf.bat 2025-01-26 14:22:49 +01:00
libcurl.pc.in configure: do not echo most inherited LDFLAGS to config files 2024-11-14 09:55:45 +01:00
Makefile.am cmake: add integration tests, run them in CI 2025-02-07 00:15:48 +01:00
README docs: minor grammar fixes 2022-09-29 10:44:12 +02:00
README.md docs: use lowercase curl and libcurl 2025-01-02 17:15:54 +01:00
RELEASE-NOTES RELEASE-NOTES: synced 2025-02-06 10:30:07 +01:00
renovate.json GHA: drop "3" from openssl names and keys 2024-10-23 19:36:03 +02:00
REUSE.toml projects/Windows: remove wolfSSL from legacy projects 2024-12-22 03:10:35 -05:00
SECURITY.md docs: Clarify OpenSSF Best Practices vs Scorecard 2024-08-22 11:50:20 +02:00

README.md

curl logo

curl is a command-line tool for transferring data specified with URL syntax. Learn how to use curl by reading the manpage or everything curl.

Find out how to install curl by reading the INSTALL document.

libcurl is the library curl is using to do its job. It is readily available to be used by your software. Read the libcurl manpage to learn how.

Open Source

curl is Open Source and is distributed under an MIT-like license.

Contact

Contact us on a suitable mailing list or use GitHub issues/ pull requests/ discussions.

All contributors to the project are listed in the THANKS document.

Commercial support

For commercial support, maybe private and dedicated help with your problems or applications using (lib)curl visit the support page.

Website

Visit the curl website for the latest news and downloads.

Source code

Download the latest source from the Git server:

git clone https://github.com/curl/curl.git

Security problems

Report suspected security problems via our HackerOne page and not in public.

Notice

curl contains pieces of source code that is Copyright (c) 1998, 1999 Kungliga Tekniska Högskolan. This notice is included here to comply with the distribution terms.

Backers

Thank you to all our backers 🙏 Become a backer.

Sponsors

Support this project by becoming a sponsor.