Add support for single-block binaries that contain all libtests and unit tests respectively. Enable with: - autotools: `--enable-test-bundles` - cmake: `-DCURL_TEST_BUNDLES=ON` (They are compatible with `--enable-unity` and `-DCMAKE_UNITY_BUILD=ON` options, for further speed-up.) Makes libtests and unit tests build _fast_, needing little disk space even in static mode. Similar to CMake unity mode, but with a custom script, also supporting autotools builds. The price is having to deal with symbols/macros colliding between `lib*.c` and `unit*.c` sources. Maybe with naming conventions or other solutions this can be improved gradually and reduce the need for manual intervention by `mk-bundle.mk`. I've included a script that does the bulk of detecting name collisions. Also: - CI: enable test bundles. - CI: build tests in more jobs. - lib2305: fix FILE handle leak. - unit1661: fix memleak found by torture test by releasing the `bufref` structure in `unit_stop()` that was allocated in `unit_setup()`. ``` test 1661...[bufref unit tests] Leak detected: memory still allocated: 13 bytes allocated by /home/runner/work/curl/curl/tests/unit/unit1661.c:70 1661: torture FAILED: function number 1 in test. ``` Ref: https://github.com/curl/curl/actions/runs/10967279334/job/30456745290?pr=14772#step:8:41 Similar test suite builds with autotools default and cmake+bundle+unity: - GHA/Linux: 33s vs 7s https://github.com/curl/curl/actions/runs/10705668823/job/29681617374 - GHA/macOS 34s vs 2s https://github.com/curl/curl/actions/runs/10705668813/job/29681632885 - GHA/FreeBSD: 15m25 vs 6m21 (full workflow time, ~qemu) https://github.com/curl/curl/actions/runs/10705668811/job/29681607915 - GHA/Cygwin: 9m52 vs 32s https://github.com/curl/curl/actions/runs/10705668809/job/29681609965 - GHA/MSYS2: 3m52 vs 14s https://github.com/curl/curl/actions/runs/10705668808/job/29681624295 - GHA/mingw-w64: 5m45 vs 30s https://github.com/curl/curl/actions/runs/10705668808/job/29681628787 Autotools test suite builds compared between master -> `--enable-test-bundles`: - GHA/Linux: 33s -> 9s (run tests: 22m23 -> 20m44) https://github.com/curl/curl/actions/runs/10710030193/job/29695932185 https://github.com/curl/curl/actions/runs/10967831456/job/30458220344 - GHA/macOS: 25s -> 4s (run tests: 2m58 -> 2m24) https://github.com/curl/curl/actions/runs/10710030195/job/29695938444 https://github.com/curl/curl/actions/runs/10967831452/job/30458225762 - GHA/non-native (FreeBSD): 4m8 -> 3m12 (full workflow time, ~qemu) https://github.com/curl/curl/actions/runs/10710030198/job/29695928401 https://github.com/curl/curl/actions/runs/10967831458/job/30458212692 - GHA/Cygwin: 9m25 -> 1m9 (run tests: 9m19 -> 3m28) https://github.com/curl/curl/actions/runs/10710030212/job/29695928213 https://github.com/curl/curl/actions/runs/10967831453/job/30458213268 - GHA/MSYS2: 3m54 -> 32s (run tests: 6m3 -> 3m59) https://github.com/curl/curl/actions/runs/10710030190/job/29704850591 https://github.com/curl/curl/actions/runs/10967831449/job/30459280005 - GHA/mingw-w64: 5m42 -> 1m5 (run tests: 7m41 -> 5m36) https://github.com/curl/curl/actions/runs/10710030190/job/29704852058 https://github.com/curl/curl/actions/runs/10967831449/job/30459280862 - Azure MSYS2 mingw64 openssl: 38m55 -> 11m58 https://dev.azure.com/daniel0244/curl/_build/results?buildId=25546&view=logs&j=b58b8c59-0f61-52e9-0f9e-fad562a1e77f&t=0f9230a7-3b10-53ca-9938-700ece377c5e https://dev.azure.com/daniel0244/curl/_build/results?buildId=25547&view=logs&jobId=39473db1-3945-55d5-deb5-c218fad88dce&j=b58b8c59-0f61-52e9-0f9e-fad562a1e77f&t=0f9230a7-3b10-53ca-9938-700ece377c5e - Azure Ubuntu default: 2m15 -> 55s (all build) https://dev.azure.com/daniel0244/curl/_build/results?buildId=25546&view=logs&j=9d58b9ac-e1e6-53b6-f83a-1f9f1d912522&t=a6b38d83-e7cf-5a9b-c762-a178412717b7 https://dev.azure.com/daniel0244/curl/_build/results?buildId=25547&view=logs&jobId=39473db1-3945-55d5-deb5-c218fad88dce&j=9d58b9ac-e1e6-53b6-f83a-1f9f1d912522&t=a6b38d83-e7cf-5a9b-c762-a178412717b7 Cmake test suite builds compared between master -> `-DCURL_TEST_BUNDLES=ON` + unity: - GHA/Linux: 29s -> 7s (run tests: 4m50 -> 4m57, 20m43 -> 20m45) https://github.com/curl/curl/actions/runs/10710030193/job/29695941814 https://github.com/curl/curl/actions/runs/10705668823/job/29681622201 - GHA/Linux old: 44s -> 13s (bundle+no unity) (run tests: 5m5 -> 5m6) https://github.com/curl/curl/actions/runs/10718264094/job/29719794727 https://github.com/curl/curl/actions/runs/10718653175/job/29721009613 - GHA/macOS: 32s -> 2s (run tests: 2m43 -> 2m40) https://github.com/curl/curl/actions/runs/10710030195/job/29695931956 https://github.com/curl/curl/actions/runs/10705668813/job/29681638937 - GHA/non-native (*BSD): inconclusive (full workflow time, ~qemu) https://github.com/curl/curl/actions/runs/10710030198 https://github.com/curl/curl/actions/runs/10705668811 - GHA/Cygwin: 3m9 -> 32s https://github.com/curl/curl/actions/runs/10710030212/job/29695929075 https://github.com/curl/curl/actions/runs/10705668809/job/29681609965 - GHA/MSYS2: 2m24 -> 14s https://github.com/curl/curl/actions/runs/10710030190/job/29704850996 https://github.com/curl/curl/actions/runs/10705668808/job/29681624295 - GHA/mingw-w64: 3m56 -> 30s (run tests: 4m2 -> 3m52) https://github.com/curl/curl/actions/runs/10710030190/job/29704852219 https://github.com/curl/curl/actions/runs/10705668808/job/29681631393 - GHA/mingw-w64-old: 7m19 -> 1m44 (run tests: 3m30 -> 2m53) https://github.com/curl/curl/actions/runs/10710030190/job/29704849763 https://github.com/curl/curl/actions/runs/10705668808/job/29681622329 - GHA/MSVC: 3m22 -> 13s (run tests: 9m43 -> 4m22) https://github.com/curl/curl/actions/runs/10710030190/job/29704850411 https://github.com/curl/curl/actions/runs/10705668808/job/29681623313 - AppVeyor CI MSVC 2008: 4m3 -> 45s (full build) - AppVeyor CI MSVC 2010: 2m56 -> 1m8 (full build) - AppVeyor CI MSVC 2022: 10m19 -> 2m23 (full build) https://ci.appveyor.com/project/curlorg/curl/builds/50538455 https://ci.appveyor.com/project/curlorg/curl/builds/50536558 - AppVeyor CI total build time: 10m30 (master) -> 6m48 (unity) -> 4m5 (bundle) -> 3m24 (bundle+unity) -> 5m7 (bundle+unity+all jobs building tests) Closes #14772 |
||
|---|---|---|
| .. | ||
| .gitignore | ||
| CMakeLists.txt | ||
| curlcheck.h | ||
| Makefile.am | ||
| Makefile.inc | ||
| README.md | ||
| unit1300.c | ||
| unit1302.c | ||
| unit1303.c | ||
| unit1304.c | ||
| unit1305.c | ||
| unit1307.c | ||
| unit1308.c | ||
| unit1309.c | ||
| unit1323.c | ||
| unit1330.c | ||
| unit1394.c | ||
| unit1395.c | ||
| unit1396.c | ||
| unit1397.c | ||
| unit1398.c | ||
| unit1399.c | ||
| unit1600.c | ||
| unit1601.c | ||
| unit1602.c | ||
| unit1603.c | ||
| unit1604.c | ||
| unit1605.c | ||
| unit1606.c | ||
| unit1607.c | ||
| unit1608.c | ||
| unit1609.c | ||
| unit1610.c | ||
| unit1611.c | ||
| unit1612.c | ||
| unit1614.c | ||
| unit1615.c | ||
| unit1616.c | ||
| unit1620.c | ||
| unit1621.c | ||
| unit1650.c | ||
| unit1651.c | ||
| unit1652.c | ||
| unit1653.c | ||
| unit1654.c | ||
| unit1655.c | ||
| unit1656.c | ||
| unit1660.c | ||
| unit1661.c | ||
| unit1663.c | ||
| unit2600.c | ||
| unit2601.c | ||
| unit2602.c | ||
| unit2603.c | ||
| unit2604.c | ||
| unit3200.c | ||
| unit3205.c | ||
Unit tests
The goal is to add tests for all functions in libcurl. If functions are too big and complicated, we should split them into smaller and testable ones.
Build Unit Tests
./configure --enable-debug is required for the unit tests to build. To
enable unit tests, there will be a separate static libcurl built that will be
used exclusively for linking unit test programs. Just build everything as
normal, and then you can run the unit test cases as well.
Run Unit Tests
Unit tests are run as part of the regular test suite. If you have built
everything to run unit tests, to can do 'make test' at the root level. Or you
can cd tests and make and then invoke individual unit tests with
./runtests.pl NNNN where NNNN is the specific test number.
Debug Unit Tests
If a specific test fails you will get told. The test case then has output left
in the %LOGDIR subdirectory, but most importantly you can re-run the test again
using gdb by doing ./runtests.pl -g NNNN. That is, add a -g to make it
start up gdb and run the same case using that.
Write Unit Tests
We put tests that focus on an area or a specific function into a single C
source file. The source file should be named unitNNNN.c where NNNN is a
previously unused number.
Add your test to tests/unit/Makefile.inc (if it is a unit test). Add your
test data file name to tests/data/Makefile.am
You also need a separate file called tests/data/testNNNN (using the same
number) that describes your test case. See the test1300 file for inspiration
and the tests/FILEFORMAT.md documentation.
For the actual C file, here's a simple example:
#include "curlcheck.h"
#include "a libcurl header.h" /* from the lib dir */
static CURLcode unit_setup( void )
{
/* whatever you want done first */
return CURLE_OK;
}
static void unit_stop( void )
{
/* done before shutting down and exiting */
}
UNITTEST_START
/* here you start doing things and checking that the results are good */
fail_unless( size == 0 , "initial size should be zero" );
fail_if( head == NULL , "head should not be initiated to NULL" );
/* you end the test code like this: */
UNITTEST_STOP