From 2bbf7d5c8cd070cc8541698fe72136328bc18eae Mon Sep 17 00:00:00 2001 From: Michal Artazov Date: Tue, 28 Apr 2020 23:29:22 +0200 Subject: [PATCH] unix: fix int overflow when copying large files I was getting some weird results when copying a 5GB file on Alpine Linux on Raspberry Pi. Out of the 5GB only 1.1GB would get copied and the process would finish without an error. After some digging I found out that there's a problem that some data types have smaller number of bytes on Alpine Linux on Raspberry Pi than on other platforms apparently. When getting the size of the file in bytes, stat holds the size in off_t data type, like this: struct stat { ... off_t st_size; /* total size, in bytes */ ... }; In my case, off_t has 8 bytes which is enough to hold a value up to some exabytes. The problem is that it gets assigned to bytes_to_send variable, which is size_t. In my case, size_t is only 4 bytes, which is only good for about 4GB. If the file is any larger, there's an overflow when assigning it from stat to bytes_to_send. That's easy to fix, I just changed the data type of bytes_to_send to off_t. However there's more. The other 2 variables - in_offset and bytes_written also have to be able to hold the size of the entire file, therefore it makes sense to change them to off_t as well. The last problem is that bytes_to_send is passed down to uv_fs_sendfile() which converts it to size_t again. I could go and change the types everywhere across the whole codebase to off_t but that could break other things, so it seams a bit too much. A much better solution is to have a proxy variable bytes_chunk that is size_t type and copy as much bytes as possible at a time that can fit into size_t. That way it will work the same as before on other platforms, where size_t is more than 4 bytes. PR-URL: https://github.com/libuv/libuv/pull/2758 Reviewed-By: Ben Noordhuis --- src/unix/fs.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/src/unix/fs.c b/src/unix/fs.c index 1b8d2ad1..7cb5ef33 100644 --- a/src/unix/fs.c +++ b/src/unix/fs.c @@ -1110,9 +1110,10 @@ static ssize_t uv__fs_copyfile(uv_fs_t* req) { int dst_flags; int result; int err; - size_t bytes_to_send; - int64_t in_offset; - ssize_t bytes_written; + off_t bytes_to_send; + off_t in_offset; + off_t bytes_written; + size_t bytes_chunk; dstfd = -1; err = 0; @@ -1211,7 +1212,10 @@ static ssize_t uv__fs_copyfile(uv_fs_t* req) { bytes_to_send = src_statsbuf.st_size; in_offset = 0; while (bytes_to_send != 0) { - uv_fs_sendfile(NULL, &fs_req, dstfd, srcfd, in_offset, bytes_to_send, NULL); + bytes_chunk = SSIZE_MAX; + if (bytes_to_send < (off_t) bytes_chunk) + bytes_chunk = bytes_to_send; + uv_fs_sendfile(NULL, &fs_req, dstfd, srcfd, in_offset, bytes_chunk, NULL); bytes_written = fs_req.result; uv_fs_req_cleanup(&fs_req);