c9b95c0bb3
When libcurl discards a connection there are two phases this may go through: "shutdown" and "closing". If a connection is aborted, the shutdown phase is skipped and it is closed right away. The connection filters attached to the connection implement the phases in their `do_shutdown()` and `do_close()` callbacks. Filters carry now a `shutdown` flags next to `connected` to keep track of the shutdown operation. Filters are shut down from top to bottom. If a filter is not connected, its shutdown is skipped. Notable filters that *do* something during shutdown are HTTP/2 and TLS. HTTP/2 sends the GOAWAY frame. TLS sends its close notify and expects to receive a close notify from the server. As sends and receives may EAGAIN on the network, a shutdown is often not successful right away and needs to poll the connection's socket(s). To facilitate this, such connections are placed on a new shutdown list inside the connection cache. Since managing this list requires the cooperation of a multi handle, only the connection cache belonging to a multi handle is used. If a connection was in another cache when being discarded, it is removed there and added to the multi's cache. If no multi handle is available at that time, the connection is shutdown and closed in a one-time, best-effort attempt. When a multi handle is destroyed, all connection still on the shutdown list are discarded with a final shutdown attempt and close. In curl debug builds, the environment variable `CURL_GRACEFUL_SHUTDOWN` can be set to make this graceful with a timeout in milliseconds given by the variable. The shutdown list is limited to the max number of connections configured for a multi cache. Set via CURLMOPT_MAX_TOTAL_CONNECTIONS. When the limit is reached, the oldest connection on the shutdown list is discarded. - In multi_wait() and multi_waitfds(), collect all connection caches involved (each transfer might carry its own) into a temporary list. Let each connection cache on the list contribute sockets and POLLIN/OUT events it's connections are waiting for. - in multi_perform() collect the connection caches the same way and let them peform their maintenance. This will make another non-blocking attempt to shutdown all connections on its shutdown list. - for event based multis (multi->socket_cb set), add the sockets and their poll events via the callback. When `multi_socket()` is invoked for a socket not known by an active transfer, forward this to the multi's cache for processing. On closing a connection, remove its socket(s) via the callback. TLS connection filters MUST NOT send close nofity messages in their `do_close()` implementation. The reason is that a TLS close notify signals a success. When a connection is aborted and skips its shutdown phase, the server needs to see a missing close notify to detect something has gone wrong. A graceful shutdown of FTP's data connection is performed implicitly before regarding the upload/download as complete and continuing on the control connection. For FTP without TLS, there is just the socket close happening. But with TLS, the sent/received close notify signals that the transfer is complete and healthy. Servers like `vsftpd` verify that and reject uploads without a TLS close notify. - added test_19_* for shutdown related tests - test_19_01 and test_19_02 test for TCP RST packets which happen without a graceful shutdown and should no longer appear otherwise. - add test_19_03 for handling shutdowns by the server - add test_19_04 for handling shutdowns by curl - add test_19_05 for event based shutdowny by server - add test_30_06/07 and test_31_06/07 for shutdown checks on FTP up- and downloads. Closes #13976 |
||
|---|---|---|
| .. | ||
| clients | ||
| testenv | ||
| .gitignore | ||
| config.ini.in | ||
| conftest.py | ||
| Makefile.am | ||
| README.md | ||
| requirements.txt | ||
| scorecard.py | ||
| test_01_basic.py | ||
| test_02_download.py | ||
| test_03_goaway.py | ||
| test_04_stuttered.py | ||
| test_05_errors.py | ||
| test_06_eyeballs.py | ||
| test_07_upload.py | ||
| test_08_caddy.py | ||
| test_09_push.py | ||
| test_10_proxy.py | ||
| test_11_unix.py | ||
| test_12_reuse.py | ||
| test_13_proxy_auth.py | ||
| test_14_auth.py | ||
| test_15_tracing.py | ||
| test_16_info.py | ||
| test_17_ssl_use.py | ||
| test_18_methods.py | ||
| test_19_shutdown.py | ||
| test_20_websockets.py | ||
| test_30_vsftpd.py | ||
| test_31_vsftpds.py | ||
The curl HTTP Test Suite
This is an additional test suite using a combination of Apache httpd and nghttpx servers to perform various tests beyond the capabilities of the standard curl test suite.
Usage
The test cases and necessary files are in tests/http. You can invoke pytest from there or from the top level curl checkout and it will find all tests.
curl> pytest
platform darwin -- Python 3.9.15, pytest-6.2.0, py-1.10.0, pluggy-0.13.1
rootdir: /Users/sei/projects/curl
collected 5 items
tests/http/test_01_basic.py .....
Pytest takes arguments. -v increases its verbosity and can be used several times. -k <expr> can be used to run only matching test cases. The expr can be something resembling a python test or just a string that needs to match test cases in their names.
curl> pytest -vv -k test_01_02
runs all test cases that have test_01_02 in their name. This does not have to be the start of the name.
Depending on your setup, some test cases may be skipped and appear as s in the output. If you run pytest verbose, it will also give you the reason for skipping.
Prerequisites
You will need:
- a recent Python, the
cryptographymodule and, of course,pytest - an apache httpd development version. On Debian/Ubuntu, the package
apache2-devhas this. - a local
curlproject build - optionally, a
nghttpxwith HTTP/3 enabled or h3 test cases will be skipped.
Configuration
Via curl's configure script you may specify:
--with-test-nghttpx=<path-of-nghttpx>if you have nghttpx to use somewhere outside your$PATH.--with-test-httpd=<httpd-install-path>if you have an Apache httpd installed somewhere else. On Debian/Ubuntu it will otherwise look into/usr/binand/usr/sbinto find those.--with-test-caddy=<caddy-install-path>if you have a Caddy web server installed somewhere else.--with-test-vsftpd=<vsftpd-install-path>if you have a vsftpd ftp server installed somewhere else.
Usage Tips
Several test cases are parameterized, for example with the HTTP version to use. If you want to run a test with a particular protocol only, use a command line like:
curl> pytest -k "test_02_06 and h2"
Several test cases can be repeated, they all have the repeat parameter (install pytest-repeat module). To make this work, you have to start pytest in the test directory itself (for some unknown reason). Like in:
curl/tests/http> pytest -k "test_02_06 and h2" --count=100
which then runs this test case a hundred times. In case of flaky tests, you can make pytest stop on the first one with:
curl/tests/http> pytest -k "test_02_06 and h2" --count=100 --maxfail=1
which allow you to inspect output and log files for the failed run. Speaking of log files, the verbosity of pytest is also used to collect curl trace output. If you specify -v three times, the curl command is started with --trace:
curl/tests/http> pytest -vvv -k "test_02_06 and h2" --count=100 --maxfail=1
all of curl's output and trace file are found in tests/http/gen/curl.
Writing Tests
There is a lot of pytest documentation with examples. No use in repeating that here. Assuming you are somewhat familiar with it, it is useful how this general test suite is setup. Especially if you want to add test cases.
Servers
In conftest.py 3 "fixtures" are defined that are used by all test cases:
env: the test environment. It is an instance of classtestenv/env.py:Env. It holds all information about paths, availability of features (HTTP/3!), port numbers to use, domains and SSL certificates for those.httpd: the Apache httpd instance, configured and started, then stopped at the end of the test suite. It has sites configured for the domains fromenv. It also loads a local modulemod_curltest?and makes it available in certain locations. (more on mod_curltest below).nghttpx: an instance of nghttpx that provides HTTP/3 support.nghttpxproxies those requests to thehttpdserver. In a direct mapping, so you may access all the resources under the same path as with HTTP/2. Only the port number used for HTTP/3 requests will be different.
pytest manages these fixture so that they are created once and terminated before exit. This means you can Ctrl-C a running pytest and the server will shutdown. Only when you brutally chop its head off, might there be servers left
behind.
Test Cases
Tests making use of these fixtures have them in their parameter list. This tells pytest that a particular test needs them, so it has to create them. Since one can invoke pytest for just a single test, it is important that a test references the ones it needs.
All test cases start with test_ in their name. We use a double number scheme to group them. This makes it ease to run only specific tests and also give a short mnemonic to communicate trouble with others in the project. Otherwise you are free to name test cases as you think fitting.
Tests are grouped thematically in a file with a single Python test class. This is convenient if you need a special "fixture" for several tests. "fixtures" can have "class" scope.
There is a curl helper class that knows how to invoke curl and interpret its output. Among other things, it does add the local CA to the command line, so that SSL connections to the test servers are verified. Nothing prevents anyone from running curl directly, for specific uses not covered by the CurlClient class.
mod_curltest
The module source code is found in testenv/mod_curltest. It is compiled using the apxs command, commonly provided via the apache2-dev package. Compilation is quick and done once at the start of a test run.
The module adds 2 "handlers" to the Apache server (right now). Handler are pieces of code that receive HTTP requests and generate the response. Those handlers are:
curltest-echo: hooked up on the path/curltest/echo. This one echoes a request and copies all data from the request body to the response body. Useful for simulating upload and checking that the data arrived as intended.curltest-tweak: hooked up on the path/curltest/tweak. This handler is more of a Swiss army knife. It interprets parameters from the URL query string to drive its behavior.status=nnn: generate a response with HTTP status codennn.chunks=n: generatenchunks of data in the response body, defaults to 3.chunk_size=nnn: each chunk should containnnnbytes of data. Maximum is 16KB right now.chunkd_delay=duration: waitdurationtime between writing chunksdelay=duration: waitdurationtime to send the response headersbody_error=(timeout|reset): produce an error after the first chunk in the response bodyid=str: addstrin the response headerrequest-id
duration values are integers, optionally followed by a unit. Units are:
d: days (probably not useful here)h: hoursmi: minutess: seconds (the default)ms: milliseconds
As you can see, mod_curltest's tweak handler allow to simulate many kinds of responses. An example of its use is test_03_01 where responses are delayed using chunk_delay. This gives the response a defined duration and the test uses that to reload httpd in the middle of the first request. A graceful reload in httpd lets ongoing requests finish, but will close the connection afterwards and tear down the serving process. The following request need then to open a new connection. This is verified by the test case.