luo980/curl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

curl/var: change byte offset syntax for assignments

Browse Source 
Follow-up to 40c264db61 after discussions on IRC.

The new style is

  name[0-99]=contents

and

  name[0-99]@filename

A) This does not cause the same problems with old curl versions trying
the new syntax as this way will cause old curls just fail with syntax
error and not risk using the wrong file.

B) Adds the same byte range support for "normal" assigns, which the
previous syntax did not. Thus lets a user get a partial content of a
variable etc.

Added test 790 and 791 to verify non-file assigns with ranges.

Closes #15862
This commit is contained in:
Daniel Stenberg 2024-12-30 13:17:39 +01:00
parent e7b90dadb2
commit c2d37463b9
No known key found for this signature in database
GPG Key ID: 5CC908FDB71E12C2
11 changed files with 167 additions and 42 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
docs/cmdline-opts/variable.md
View File

@ -36,14 +36,20 @@ the environment variable is not set, use --variable %name=content or
--variable %name@content. Note that on some systems - but not all -
environment variables are case insensitive.
Added in curl 8.12.0: when getting contents from a file, you can request to
get a byte range from it by appending ";[start-end]" to the filename, where
*start* and *end* are byte offsets to include from the file. For example,
asking for offset "2-10" means offset two to offset ten, including the byte
offset 10, meaning 9 bytes in total. "2-2" means a single byte at offset 2.
Not providing a second number implies to the end of the file. The start offset
cannot be larger than the end offset. Asking for a range that is outside of
the file size makes the variable contents empty.
Added in curl 8.12.0: you can get a byte range from the source by appending
`[start-end]` to the variable name, where *start* and *end* are byte offsets
to include from the contents. For example, asking for offset "2-10" means
offset two to offset ten, inclusive, resulting in 9 bytes in total. `2-2`
means a single byte at offset 2. Not providing a second number implies to the
end of data. The start offset cannot be larger than the end offset. Asking for
a range that is outside of the file size makes the variable contents empty.
For example, getting the first one hundred bytes from a given file:
curl --variable "fraction[0-99]@filename"
Given a byte range that has no data results in an empty string. Asking for a
range that is larger than the content makes curl use the piece of the data
that exists.
To assign a variable using contents from another variable, use
--expand-variable. Like for example assigning a new variable using contents

61
src/var.c
View File

@ -389,6 +389,8 @@ ParameterError setvariable(struct GlobalConfig *global,
bool import = FALSE;
char *ge = NULL;
char buf[MAX_VAR_LEN];
curl_off_t startoffset = 0;
curl_off_t endoffset = CURL_OFF_T_MAX;
if(*input == '%') {
import = TRUE;
@ -423,43 +425,37 @@ ParameterError setvariable(struct GlobalConfig *global,
clen = strlen(ge);
}
}
if(*line == '[') {
/* is there a byte range specified? [num-num] */
if(ISDIGIT(line[1])) {
char *endp;
if(curlx_strtoofft(&line[1], &endp, 10, &startoffset) || (*endp != '-'))
return PARAM_VAR_SYNTAX;
else {
char *p = endp + 1; /* pass the '-' */
if(*p != ']') {
if(curlx_strtoofft(p, &endp, 10, &endoffset) || (*endp != ']'))
return PARAM_VAR_SYNTAX;
line = &endp[1]; /* pass the ']' */
}
else
line = &p[1]; /* pass the ']' */
}
if(startoffset > endoffset)
return PARAM_VAR_SYNTAX;
}
}
if(content)
;
else if(*line == '@') {
/* read from file or stdin */
FILE *file;
bool use_stdin;
char *range;
struct dynbuf fname;
curl_off_t startoffset = 0;
curl_off_t endoffset = CURL_OFF_T_MAX;
line++;
Curl_dyn_init(&fname, MAX_FILENAME);
/* is there a byte range specified? ;[num-num] */
range = strstr(line, ";[");
if(range && ISDIGIT(range[2])) {
char *p = range;
char *endp;
if(curlx_strtoofft(&p[2], &endp, 10, &startoffset) || (*endp != '-'))
return PARAM_VAR_SYNTAX;
else {
p = endp + 1; /* pass the '-' */
if(*p != ']') {
if(curlx_strtoofft(p, &endp, 10, &endoffset) || (*endp != ']'))
return PARAM_VAR_SYNTAX;
}
}
if(startoffset > endoffset)
return PARAM_VAR_SYNTAX;
/* create a dynbuf for the filename without the range */
if(Curl_dyn_addn(&fname, line, (range - line)))
return PARAM_NO_MEM;
/* point to the new file name buffer */
line = Curl_dyn_ptr(&fname);
}
use_stdin = !strcmp(line, "-");
if(use_stdin)
file = stdin;
@ -485,9 +481,20 @@ ParameterError setvariable(struct GlobalConfig *global,
}
else if(*line == '=') {
line++;
clen = strlen(line);
/* this is the exact content */
content = (char *)line;
clen = strlen(line);
if(startoffset || (endoffset != CURL_OFF_T_MAX)) {
if(startoffset >= (curl_off_t)clen)
clen = 0;
else {
/* make the end offset no larger than the last byte */
if(endoffset >= (curl_off_t)clen)
endoffset = clen - 1;
clen = (size_t)(endoffset - startoffset) + 1;
content += startoffset;
}
}
}
else {
warnf(global, "Bad --variable syntax, skipping: %s", input);

2
tests/data/Makefile.am
View File

@ -110,7 +110,7 @@ test727 test728 test729 test730 test731 test732 test733 test734 test735 \
test736 test737 test738 test739 test740 test741 test742 \
\
test780 test781 test782 test783 test784 test785 test786 test787 test788 \
test789 \
test789 test790 test791 \
\
test799 test800 test801 test802 test803 test804 test805 test806 test807 \
test808 test809 test810 test811 test812 test813 test814 test815 test816 \

2
tests/data/test784
View File

@ -35,7 +35,7 @@ http
--variable with a file byte range
</name>
<command>
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable name"@%LOGDIR/in%TESTNUMBER;[5-15]" --expand-data '{{name}}'
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable "name[5-15]@%LOGDIR/in%TESTNUMBER" --expand-data '{{name}}'
</command>
<file name="%LOGDIR/in%TESTNUMBER">
On the first Monday of the month of April, 1625, the market town of Meung

2
tests/data/test785
View File

@ -35,7 +35,7 @@ http
--variable with a file byte range without end
</name>
<command>
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable name"@%LOGDIR/in%TESTNUMBER;[5-]" --expand-data '{{name}}'
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable "name[5-]@%LOGDIR/in%TESTNUMBER" --expand-data '{{name}}'
</command>
<file name="%LOGDIR/in%TESTNUMBER">
On the first Monday of the month of April, 1625, the market town of Meung

2
tests/data/test786
View File

@ -35,7 +35,7 @@ http
--variable with a file byte range, reading from stdin
</name>
<command>
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable name"@-;[5-15]" --expand-data '{{name}}'
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable "name[5-15]@-" --expand-data '{{name}}'
</command>
<stdin>
On the first Monday of the month of April, 1625, the market town of Meung

2
tests/data/test787
View File

@ -21,7 +21,7 @@ http
--variable with a file byte range, bad range
</name>
<command>
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable name"@&LOGDIR/fooo;[15-14]" --expand-data '{{name}}'
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable "name[15-14]@&LOGDIR/fooo" --expand-data '{{name}}'
</command>
</client>

2
tests/data/test788
View File

@ -35,7 +35,7 @@ http
--variable with a file and single-byte byte range
</name>
<command>
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable name"@%LOGDIR/in%TESTNUMBER;[15-15]" --expand-data '{{name}}'
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable "name[15-15]@%LOGDIR/in%TESTNUMBER" --expand-data '{{name}}'
</command>
<file name="%LOGDIR/in%TESTNUMBER">
On the first Monday of the month of April, 1625, the market town of Meung

2
tests/data/test789
View File

@ -35,7 +35,7 @@ http
--variable with a file and byte range out of file
</name>
<command>
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable name"@%LOGDIR/in%TESTNUMBER;[75-85]" --expand-data '{{name}}'
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable "name[75-85]@%LOGDIR/in%TESTNUMBER" --expand-data '{{name}}'
</command>
<file name="%LOGDIR/in%TESTNUMBER">
On the first Monday of the month of April, 1625, the market town of Meung

56
tests/data/test790 Normal file
View File

@ -0,0 +1,56 @@
<testcase>
<info>
<keywords>
HTTP
--variable
</keywords>
</info>
#
# Server-side
<reply>
<data crlf="yes">
HTTP/1.1 200 OK
Date: Tue, 09 Nov 2010 14:49:00 GMT
Server: test-server/fake
Last-Modified: Tue, 13 Jun 2000 12:10:00 GMT
ETag: "21025-dc7-39462498"
Accept-Ranges: bytes
Content-Length: 6
Connection: close
Content-Type: text/html
Funny-head: yesyes
-foo-
</data>
</reply>
#
# Client-side
<client>
<server>
http
</server>
<name>
--variable with a byte range using plain assign
</name>
<command>
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable "name[5-9]=0123456789abcdef" --expand-data '{{name}}'
</command>
</client>
#
# Verify data after the test has been "shot"
<verify>
<protocol crlf="yes" nonewline="yes">
POST /%TESTNUMBER HTTP/1.1
Host: %HOSTIP:%HTTPPORT
User-Agent: curl/%VERSION
Accept: */*
Content-Length: 5
Content-Type: application/x-www-form-urlencoded
56789
</protocol>
</verify>
</testcase>

56
tests/data/test791 Normal file
View File

@ -0,0 +1,56 @@
<testcase>
<info>
<keywords>
HTTP
--variable
</keywords>
</info>
#
# Server-side
<reply>
<data crlf="yes">
HTTP/1.1 200 OK
Date: Tue, 09 Nov 2010 14:49:00 GMT
Server: test-server/fake
Last-Modified: Tue, 13 Jun 2000 12:10:00 GMT
ETag: "21025-dc7-39462498"
Accept-Ranges: bytes
Content-Length: 6
Connection: close
Content-Type: text/html
Funny-head: yesyes
-foo-
</data>
</reply>
#
# Client-side
<client>
<server>
http
</server>
<name>
--variable with a byte range using plain assign, out of range
</name>
<command>
http://%HOSTIP:%HTTPPORT/%TESTNUMBER --variable "name[10-30]=0123456789abcdef" --expand-data '{{name}}'
</command>
</client>
#
# Verify data after the test has been "shot"
<verify>
<protocol crlf="yes" nonewline="yes">
POST /%TESTNUMBER HTTP/1.1
Host: %HOSTIP:%HTTPPORT
User-Agent: curl/%VERSION
Accept: */*
Content-Length: 6
Content-Type: application/x-www-form-urlencoded
abcdef
</protocol>
</verify>
</testcase>