Make ThreadSanitizer stop complaining about the static variables that
libuv uses to record the presence (or lack) of system calls and other
kernel features.
Fixes: https://github.com/libuv/libuv/issues/2884
PR-URL: https://github.com/libuv/libuv/pull/2886
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Jameson Nash <vtjnash@gmail.com>
Exposes the original system error of the filesystem syscalls. Adds a new
uv_fs_get_system_error which returns orignal errno on Linux or
GetLastError on Windows.
Ref: https://github.com/libuv/libuv/issues/2348
PR-URL: https://github.com/libuv/libuv/pull/2810
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
I was getting some weird results when copying a 5GB file on Alpine
Linux on Raspberry Pi. Out of the 5GB only 1.1GB would get copied and
the process would finish without an error. After some digging I found
out that there's a problem that some data types have smaller number of
bytes on Alpine Linux on Raspberry Pi than on other platforms
apparently.
When getting the size of the file in bytes, stat holds the size in
off_t data type, like this:
struct stat {
...
off_t st_size; /* total size, in bytes */
...
};
In my case, off_t has 8 bytes which is enough to hold a value up to
some exabytes. The problem is that it gets assigned to bytes_to_send
variable, which is size_t. In my case, size_t is only 4 bytes, which
is only good for about 4GB. If the file is any larger, there's an
overflow when assigning it from stat to bytes_to_send. That's easy
to fix, I just changed the data type of bytes_to_send to off_t.
However there's more.
The other 2 variables - in_offset and bytes_written also have to be
able to hold the size of the entire file, therefore it makes sense to
change them to off_t as well.
The last problem is that bytes_to_send is passed down to
uv_fs_sendfile() which converts it to size_t again. I could go and
change the types everywhere across the whole codebase to off_t but
that could break other things, so it seams a bit too much. A much
better solution is to have a proxy variable bytes_chunk that is
size_t type and copy as much bytes as possible at a time that can
fit into size_t. That way it will work the same as before on other
platforms, where size_t is more than 4 bytes.
PR-URL: https://github.com/libuv/libuv/pull/2758
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Modeled after FreeBSD's `reallocf(3)`: a version of `realloc(3)` that
frees the memory when reallocation fails, simplifying error handling in
many cases.
PR-URL: https://github.com/libuv/libuv/pull/2735
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
uv_fs_copyfile() calls fchmod() to change the target file's permissions
to the source file's permissions but that operation errors with EPERM on
CIFS shares unless they are mounted with the "noperm" option.
Since UNIX-style permissions don't make sense for CIFS anyway, let's
handle the error in libuv by recognizing that it's a CIFS share and
continuing when that is the case.
The same logic probably applies to (a subset of) FUSE file systems but
those haven't been whitelisted yet.
Fixes: https://github.com/libuv/libuv/issues/2596
Refs: https://github.com/nodejs/node/issues/31170
PR-URL: https://github.com/libuv/libuv/pull/2597
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
Look up the "mkostemp" symbol once instead of on every call
to uv_fs_mkstemp().
PR-URL: https://github.com/libuv/libuv/pull/2564
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
Fixes: https://github.com/libuv/libuv/issues/2563
PR-URL: https://github.com/libuv/libuv/pull/2564
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
statx is not implemented on RHEL 7 and trying to call it from within a
docker container might result in a return value of 1 (which is not an
expected value according to statx man page, expected values are 0 or -1)
and errno is set to 0, in which case assume statx is not implemented.
Above behaviour has been seen on RHEL 7 running on a s390.
PR-URL: https://github.com/libuv/libuv/pull/2529
Refs: https://bugzilla.redhat.com/show_bug.cgi?id=1759152
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
The Unix/macOS uv__fs_copyfile() implementation falls back to
using uv_fs_sendfile(). This commit refactors the error handling
to use the sendfile() req's result field, which is an ssize_t
instead of using the return value, which is an int. The int
value was coming back as a negative number for some large files.
Refs: https://github.com/nodejs/node/issues/30085
PR-URL: https://github.com/libuv/libuv/pull/2533
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
This commit fixes uv_fs_copyfile() in cases where an unknown error
occurs when copy-on-write is requested by setting
UV_FS_COPYFILE_FICLONE. The original approach tried to catch some of
the errors raised by the ioctl() call, assuming that sendfile() would
also fail in those cases. This is not necessarily true, as some
variants of ioctl() also raise EINVAL (some maybe EBADF), but sendfile()
works just fine.
This patch reverses the logic, falling back to sendfile() in any
case where ioctl() returns an error. In other words, it tries much
harder to make uv_fs_copyfile() work.
Related to that, the original approach returned UV_ENOTSUP
unconditionally in cases where ioctl() failed and
UV_FS_COPYFILE_FICLONE_FORCE was set. However, ioctl() may have
failed for other reasons than being not supported. The function
now returns the actual error raised by ioctl(), leaving it to the
caller to deal with it.
Fixes: https://github.com/libuv/libuv/issues/2483
PR-URL: https://github.com/libuv/libuv/pull/2514
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Platforms that support O_CLOEXEC have supported it for a long time
and don't need feature detection. Libuv can just assume it works as
advertised.
The feature detection was broken for old Linux kernels because they
don't report EINVAL for unknown open() flags, they simply open the
file without setting the FD_CLOEXEC flag.
Libuv could fix that by checking afterwards with fcntl() if the flag
was actually set but it doesn't support kernels that old in the first
place so let's simplify the logic.
Fixes: https://github.com/libuv/libuv/issues/2450
PR-URL: https://github.com/libuv/libuv/pull/2454
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
This commit fixes a warning about uv__fs_preadv() being
unused on BSD.
PR-URL: https://github.com/libuv/libuv/pull/2367
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
The fallback for systems that lack preadv() only filled the first
buffer.
This commit rectifies that to fill all (or at least as many as possible)
buffers.
Fixes: https://github.com/libuv/libuv/issues/2332
PR-URL: https://github.com/libuv/libuv/pull/2338
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Jameson Nash <vtjnash@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
Make uv_cwd() do what the documentation says it did when the destination
buffer is too small: report UV_ENOBUFS and set the `size` in/out param
to the size of the path including the trailing nul byte.
Fixes: https://github.com/libuv/libuv/issues/2333
PR-URL: https://github.com/libuv/libuv/pull/2335
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
F_BARRIERFSYNC isn't defined when building on that platform.
PR-URL: https://github.com/libuv/libuv/pull/2334
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
This commit add support for Haiku, an open-source operating system
inspired by BeOS.
PR-URL: https://github.com/libuv/libuv/pull/2301
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Fall back to F_BARRIERFSYNC if F_FULLFSYNC is not supported by the file
system, only fall back to fsync() if both fcntls fail.
F_BARRIERFSYNC should be at least as safe as fsync() because it's fsync
coupled with a barrier.
PR-URL: https://github.com/libuv/libuv/pull/2317
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
The close() system call tests for thread cancellation first. That has
the unfortunate side effect of aborting the sytem call with EINTR
without actually closing the file descriptor when the thread is in
the "cancel" state. Work around that by calling close$NOCANCEL().
This might well qualify as an academic bug because approximately no one
uses thread cancellation but let's aim for correctness anyway.
PR-URL: https://github.com/libuv/libuv/pull/2291
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Explicitly initialize uv_stat_t fields st_flags and st_gen when using
statx as uv__to_stat does when statx is not available. This makes
valgrind happier.
PR-URL: https://github.com/libuv/libuv/pull/2263
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jameson Nash <vtjnash@gmail.com>
close() is interruptible, meaning that a signal delivered when the
close() system call is executing, can result in a return value of
-1 with errno set to EINTR or EINPROGRESS. A survey of supported
Unices suggests the file descriptor is in fact closed and so there
is nothing to do except ignore the error.
Libuv already handled it correctly for internal file descriptors
throug uv__close() and uv__close_nocheckstdio() but uv_fs_close()
did not until now.
PR-URL: https://github.com/libuv/libuv/pull/2236
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Jameson Nash <vtjnash@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
Using copyfile(3) on macOS apparently results in situations where file
permissions are ignored. Using the same code for other UNIX-like
operating systems seems to fix the issue.
Refs: https://github.com/nodejs/node/issues/26936#issuecomment-476992597
PR-URL: https://github.com/libuv/libuv/pull/2233
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Kernels > 4.11 support the statx() system call that lets one retrieve
the birth time of a file. Teach libuv about it.
Fixes: https://github.com/libuv/libuv/issues/2152
PR-URL: https://github.com/libuv/libuv/pull/2184
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
On IBM i PASE, EOPNOTSUPP is returned when reading a directory instead
of EISDIR, like (seemingly) every other platform which doesn't support
reading directories. To ensure compatibility with software expecting
EISDIR, we map the EOPNOTSUPP to EISDIR when the fd passed in was a
directory.
This is a partial revert of 25a3894, but scoped to PASE and the fstat
call is moved to the end so it's out of the fast path.
Refs: https://github.com/nodejs/node/issues/25433
Fixes: https://github.com/libuv/libuv/issues/2147
PR-URL: https://github.com/libuv/libuv/pull/2148
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Apple's released source code at https://opensource.apple.com/ shows at
least three different error codes being returned when a filesystem does
not support F_FULLFSYNC.
fcntl() is implemented in xnu-4903.221.2 in bsd/kern/kern_descrip.c,
where it delegates to fcntl_nocancel(). The documentation for
fcntl_nocancel() mentions error codes for some operations, but does not
include F_FULLFSYNC. The F_FULLSYNC branch in fcntl_nocancel() calls
VNOP_IOCTL(_, F_FULLSYNC, NULL, 0, _), whose return value sets the error
code.
VNOP_IOCTL() is implemented in bsd/vfs/kpi_vfs.c and calls the ioctl
function in the vnode's operation vector. The filesystem-level function
names follow the pattern _vnop_ioctl() for all the instances in
opensource code -- {hfs,msdosfs,nfs,ntfs,smbfs,webdav,zfs}_vnop_ioctl().
hfs-407.30.1, msdosfs-229.200.3, and nfs in xnu-4903.221.2 handle
F_FULLFSYNC. ntfs-94.200.1 and smb-759.40.1 do not handle F_FULLFSYNC,
and the default branch returns ENOSUP. webdav-380.200.1 also does not
handle F_FULLFSYNC, but the default branch returns EINVAL. zfs-59 also
does not handle F_FULLSYNC, and its default branch returns ENOTTY.
From a different angle, Apple's ntfs-94.200.1 includes utility code that
uses fcntl(F_FULLFSYNC) and falls back to fsync() just like this patch
does, supporting the hypothesis that there is no good way to detect lack
of F_FULLFSYNC support.
PR-URL: https://github.com/libuv/libuv/pull/2135
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Joran Dirk Greef <joran@ronomon.com>
Some platforms (e.g. GNU/Hurd) do not define PATH_MAX. Add a few other
variants and a fallback constant. Also use alternatives where possible:
* For readlink(), use lstat() to read the length of the link first.
If it is not a symlink, report EINVAL before trying to allocate.
If the size reports as zero, fall back one of the PATH_MAX variants.
* For realpath(), POSIX 2008 allows us to pass a NULL buffer
to tell it to malloc() internally.
This patch was inspired by downstream patches in Debian packaging.
PR-URL: https://github.com/libuv/libuv/pull/2008
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Bug-Debian: https://bugs.debian.org/897061
Bug-Debian: https://bugs.debian.org/909011
Bug-Ubuntu: https://bugs.launchpad.net/bugs/1792647
Inspired-by: Mauricio Faria de Oliveira <mfo@canonical.com>
Inspired-by: Samuel Thibault <sthibault@debian.org>
Remove the artificial EISDIR that was generated when trying to
uv_fs_read() from a file descriptor that refers to a directory.
We don't do that on the BSDs either (where reading from a directory
is allowed) and it introduces an extra stat() call for every read.
Refs: https://github.com/libuv/libuv/pull/2023#issuecomment-427759265
PR-URL: https://github.com/libuv/libuv/pull/2025
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Libuv was allocating PATH_MAX+1 bytes to reserve space for the trailing
nul byte.
On platforms like Linux, where PATH_MAX is fixed at 4096, that meant we
were allocating two pages (as the page size is normally also 4096), even
though the second page was almost never used.
Change that to allocate PATH_MAX bytes and only resize when readlink()
actually writes that many bytes, which is practically never.
PR-URL: https://github.com/libuv/libuv/pull/2009
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
This is now unnecessary as of the previous commit.
PR-URL: https://github.com/libuv/libuv/pull/1940
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Previously both uv_fs_utimes and uv_fs_futimes would fall back to utimes() if untimesat() wasn't supported.
Since utimesat() has been supported since linux kernel 2.6.22, and libuv supports at minimum 2.6.32, this code can be removed.
PR-URL: https://github.com/libuv/libuv/pull/1940
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
This should improve uv_fs_utime resolution and reliability, as
utime(2)'s precision is left more to the implementing platform than the
newer but well supported alternatives.
Related to https://github.com/nodejs/node/issues/22070
PR-URL: https://github.com/libuv/libuv/pull/1940
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
If `nthreads / 2` (rounded up) DNS calls are outstanding,
queue more work of that kind instead of letting it take over
more positions in the thread pool, blocking other work
such as the (usually much faster) file system I/O or
user-scheduled work.
Fixes: https://github.com/nodejs/node/issues/8436
PR-URL: https://github.com/libuv/libuv/pull/1845
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
