GitLab

* NEWS: Record release date.

* tests/ls/hyperlink.sh: If the hostname or any part of
the absolute path would be changed due to URL encoding,
the test would fail.  Therefore simplify to remove
these components of the URL from consideration.

* .gitignore: Remove lines indicated by sc_gitignore_redundant
in a freshly checked out repo.

* gnulib: The only change in this gnulib update
is the tagging of the fts-tests module as longrunning,
which gnulib-tool currently implicitly excludes.
This test was seen to take about 20s and 285MB.
Reported by Assaf Gordon on space restricted VMs.

* src/tty.c (main): Don't distinguish ENOTTY from other errors,
because isatty() doesn't portably distinguish errors.
Solaris returns ENOENT for all input errors for example.
Musl also returns ENOENT, and ENODEV may be returned as disscussed at:
http://openwall.com/lists/musl/2017/04/06/6
* tests/misc/tty.sh: Adjust accordingly.

* tests/ln/sf-1.sh: Generate specific length with space padding
which is supported.
Reported by Assaf Gordon on Alpine Linux.

On some setups the root:object_r:tmp_t context is invalid.
This does indicate a limitation in the test framework,
but for now we'll relax this to skipping the tests.
The tests still run on a Fedora 25 system for example.

* tests/cp/cp-a-selinux.sh: Upon chcon error, skip rather than ERROR.
* tests/install/install-Z-selinux.sh: Likewise.
* tests/misc/chcon.sh: Likewise.
* tests/misc/runcon-no-reorder.sh: Likewise.
* tests/misc/selinux.sh: Likewise.
* tests/mkdir/restorecon.sh: Likewise.

* src/expr.c (eval6): Free memory allocated by mbs_logical_substr().

Introduced in v8.27-47-ga9f2be5b.  Detected by Coverity Analysis:

Error: RESOURCE_LEAK:
src/expr.c:851: leaked_storage: Variable "s" going out of scope
leaks the storage it points to.
849|             char *s = mbs_logical_substr (l->u.s, pos, len);
850|             v = str_value (s);
851|->         }
852|         freev (l);
853|         freev (i1);

* gnulib: The only change included in this update
it the added check for the presence of <linux/fs.h>
which is not present on Alpine Linux by default.

* src/tty.c (main): All systems mention that isatty()
man return EINVAL as well as (the POSIX compliant) ENOTTY.
Also Centos 6 was seen to return EINVAL from ttyname().
* tests/misc/tty.sh: Fix a test issue where we assume
standard input is always a valid tty.
Reported by Assaf Gordon on OpenSolaris 5.10 and 5.11,
and Centos 6.5

This reverts commit v8.27-97-g8cb06d4b because
the setsid() fallback was not implemented correctly
and disabling the ioctl was not a complete solution
to the security issue of the child being passed
the tty of the parent.

Given runcon is not really a sandbox command,
the advice is to use `runcon ... setsid ...`
to avoid this particular issue.

We only use one of statfs or statvfs for `stat -f`
and on the BSDs we use statfs which doesn't have the
f_namelen member.  However on OpenBSD and later FreeBSD
systems statfs does provide f_namemax, so use that.

* NEWS: Mention the improvement for OpenBSD and FreeBSD.
* m4/stat-prog.m4: Check for f_namemax in the statfs struct.
* src/stat.c: Return '?' rather than '*' when we can't
determine the max length of the file system.
* tests/ln/sf-1.sh: This test was failing on all BSDs
due to '*' being returned for the max length which
caused the test to attempt to create 1Mi+1 names.
The test now uses a short name when we can't determine
the max name length to use.

Reported by Assaf Gordon on various BSD based systems.

* src/stat.c (human_fstype): This file system is used
to manage AppArmor policy in the Linux kernel.

* bootstrap: Sync timestamp update.

Similar to the issue with SELinux sandbox (CVE-2016-7545),
children of runcon can inject arbitrary input to the terminal
that would be run at the originating terminal privileges.

The new libseccomp dependency is widely available and used
on modern SELinux systems, but is not available by default
on older systems like RHEL6 etc.

* m4/jm-macros.m4: Check for libseccomp and
warn if unavailable on selinux supporting systems.
* src/local.mk: Link runcon with -lseccomp.
* src/runcon.c (disable_tty_inject): A new function to
disable use of the TIOCSTI using libseccomp, or with setsid()
where libseccomp is unavailable.
* tests/misc/runcon-no-inject.sh: A new test that uses
python to make the TIOCSTI call, and ensure that doesn't succeed.
* tests/local.mk: Reference the new test
* NEWS: Mention the fix.
Addresses http://bugs.gnu.org/24541

Terminals such as iTerm2 and VTE based terminals
(as of version 0.49.1), support hyperlinks when
passed terminals codes as described at:
https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda

* src/ls.c (gobble_file): Allocate an absolute file name to output.
(quote_name): Output the absolute name with the appropriate codes.
(file_escape): A new function to encode file names as per rfc8089.
(main): Handle the new option and call the file_escape_init() helper.
Disable --dired when --hyperlink is specified.
(print_dir): Get the absolute file name here too, so that the
directory name can be linkified.
* NEWS: Mention the new feature.
* tests/ls/hyperlink.sh: Add a new test.
* tests/local.mk: Reference the new test.
* doc/coreutils.texi (ls invocation): Describe --hyperlink.

This saves about 0.5MB uncompressed from the tarball.

* Makefile.am: Following on from v8.26-34-g2c64bc87
update the oldest documented version to 8.18 which
is now about 5 years old.  Also remove older ChangeLogs
that were previously thought to be for changes not
in the git history, but are adequately recorded upon review.
* build-aux/ChangeLog-2007: Remove file.
* lib/ChangeLog-2007: Likewise.
* m4/ChangeLog-2007: Likewise.

This is useful when chaining with other commands that run commands in a
different context, while avoiding using the shell to cd, and thus
having to consider shell quoting the chained command.

* NEWS (New features): Document the new option.
* doc/coreutils.texi (env invocation): Likewise.
* src/env.c (usage): Likewise.
(main): Implement the new option.
* tests/misc/env.sh: Test the new option.

* tests/sample-test: Use framework_error_ rather than fail=1
* tests/chown/deref.sh: Likewise.
* tests/chown/preserve-root.sh: Likewise.
* tests/cp/src-base-dot.sh: Likewise.
* tests/dd/unblock-sync.sh: Likewise.
* tests/du/2g.sh: Likewise.
* tests/du/inacc-dest.sh: Likewise.
* tests/du/one-file-system.sh: Likewise.
* tests/fmt/goal-option.sh: Likewise.
* tests/ln/hard-backup.sh: Likewise.
* tests/ls/color-dtype-dir.sh: Likewise.
* tests/ls/m-option.sh: Likewise.
* tests/ls/stat-dtype.sh: Likewise.
* tests/ls/time-style-diag.sh: Likewise.
* tests/ls/x-option.sh: Likewise.
* tests/misc/chcon.sh: Likewise.
* tests/misc/nohup.sh: Likewise.
* tests/misc/od-N.sh: Likewise.
* tests/misc/sort-compress.sh: Likewise.
* tests/misc/tac-continue.sh: Likewise.
* tests/misc/time-style.sh: Likewise.
* tests/mv/backup-dir.sh: Likewise.
* tests/mv/dir2dir.sh: Likewise.
* tests/rm/dir-no-w.sh: Likewise.
* tests/rm/dir-nonrecur.sh: Likewise.
* tests/rm/inaccessible.sh: Likewise.
* tests/rm/interactive-always.sh: Likewise.
* tests/rm/interactive-once.sh: Likewise.
* tests/rm/rm3.sh: Likewise.
* tests/rm/v-slash.sh: Likewise.
* tests/touch/relative.sh: Likewise.

When systemd is configured to automount a remote file system - see
'man systemd.automount(5)', then the mount point is initially
mounted by systemd with the file system type "autofs".
When the resource is used later on, then the wanted file system is
mounted over that mount point on demand.
'df -l' triggered systemd to mount the file system because it called
stat() on the mount point.
Instead of single-casing "autofs" targets, we can avoid stat()ing
all dummy file systems (which includes "autofs"), because those are
skipped later on in get_dev() anyway.

*src/df.c (filter_mount_list): Also skip dummy file systems unless
the -a option or a specific target are given.
* NEWS: Mention the fix.
Co-authored-by: Bernhard Voelker <mail@bernhard-voelker.de>

Fixes http://bugzilla.suse.com/show_bug.cgi?id=1043059

* doc/coreutils.texi (Realpath usage examples): New section.

The 'readlink' node has '@findex realpath' in it. This results in
    info doc/coreutils.info realpath
incorrectly jumping to the 'readlink' node (instead of the 'realpath'
node). Change it to @cindex instead.

* doc/coreutils.texi (readlink): Change '@findex realpath' to @cindex.

* src/realpath.c (usage): Explicitly say 'DIR' instead of 'FILE' for
--relative-{to,base} parameters, to avoid giving the impression
that regular files can be used as relative base.
* doc/coreutils.texi (realpath): Same.

* src/ls.c (gobble_file): Disable the optimization to avoid quoting
if the symlink target itself needs quoting.  This was introduced
with the quoting alignment adjustments in v8.25-106-g01971c0e
* tests/ls/symlink-quote.sh: Add a test.
* tests/local.mk: Reference the test.
* NEWS: Mention the fix.

commit v8.27-44-g18f6b22f was too aggressive in
only allowing inotify use with regular files. This will
support responsive processing of `tail -f fifo | ...`

* src/tail.c (any_non_regular): Adjust to allow FIFOs
since inotify supports these well.
* tests/tail-2/inotify-only-regular.sh: Adjust comment.

* src/sort.c: Don't include stdio--.h as fopen() is no longer used.

* tests/misc/seq-epipe.sh: Remove stale comment.
* tests/misc/sort-debug-warn.sh: musl doesn't indicate a set_locale()
failure with missing locales, so avoid a test portion in that case.
* tests/misc/wc-files0.sh: Avoid a bug on older ash implementations.
Addresses http://bugs.gnu.org/28054

Problem reported by Lukas Zachar at:
http://bugzilla.redhat.com/1482445
* src/ptx.c (line_width, gap_size, maximum_word_length)
(reference_max_width, half_line_width, before_max_width)
(keyafter_max_width, truncation_string_length, compare_words)
(compare_occurs, search_table, find_occurs_in_text, print_spaces)
(fix_output_parameters, define_all_fields):
Use ptrdiff_t, not int, for object offsets and sizes.
(WORD, OCCURS): Use ptrdiff_t, not short int.
(WORD_TABLE, number_of_occurs, generate_all_output):
Prefer ptrdiff_t to size_t where either will do.
(total_line_count, file_line_count, OCCURS, fix_output_parameters)
(define_all_fields):
Use intmax_t, not int, for line counts.
(DELTA): Remove.  All uses changed.
(OCCURS, find_occurs_in_text, fix_output_parameters):
Use int, not size_t, for file indexes.
(tail_truncation, before_truncation, keyafter_truncation)
(head_truncation, search_table, define_all_fields)
(generate_all_output):
Use bool for booleans.
(digest_word_file, find_occurs_in_text):
Use x2nrealloc instead of checking for overflow by hand.
(find_occurs_in_text, fix_output_parameters, define_all_fields):
Omit unnecessary cast.
(fix_output_parameters): Don’t assume integers fit in 11 digits.
(fix_output_parameters, define_all_fields):
Use sprintf return value rather than calling strlen.
(define_all_fields): Do not rely on sprintf to generate a string
that may contain more than INT_MAX bytes.
(main): Use xstrtoimax, not xstrtoul.
Use xnmalloc to catch integer overflow.

* src/nohup.c: Do not include cloexec.h.
(main): Use fcntl rather than dup + set_cloexec_flag.

POSIX says sigprocmask has unspecified behavior in a multithreaded
program like ‘sort’.
* src/sort.c (pthread_sigmask) [GNULIB_defined_pthread_functions]:
New macro, for use when ‘sort’ is not multithreaded.
(cs_enter, cs_leave): Use it.  Pass address, not value, as
this is typically a tad faster.  All callers changed.

* src/sort.c (move_fd): Rename from move_fd_or_die,
since it no longer can die.

Use O_CLOEXEC when creating file descriptors, so that subsidiary
processes do not inherit file descriptors that they do not need.
This is helpful for ‘sort’, as it is a multithreaded program that
forks and execs.
* bootstrap.conf (gnulib_modules): Add mkostemp, open, pipe2.
* src/sort.c (create_temp_file): Open temporary file with O_CLOEXEC.
(stream_open): Open the stream with O_CLOEXEC.
(pipe_fork): Create the pipe with O_CLOEXEC.
(check_output): Open the output file with O_CLOEXEC.
(main): Use xfopen/xfclose to handle --files0-from, so that
O_CLOEXEC is used properly.  This is simpler anyway.
* tests/misc/sort-files0-from.pl: Adjust to change in diagnostic
wording.

* src/operand2sig.c (operand2sig): AIX uses a different bit pattern
in the returned status from the wait() functions and from shells.
Therefore hardcode the selection of the lower bits of the number.
* NEWS: Mention the fix.

* configure.ac: Set USE_XLC_INCLUDE when __xlc__ is defined.
* src/local.mk: Use it to select the appropriate include option.
Reported by Michael Felt.

* tests/ln/sf-1.sh: Limit the symlink size to 1MiB
to avoid memory exhaustion seen on NFS on AIX, giving:
  + printf '%0*d' 4294967296 0
  + ./tests/ln/sf-1.sh: line 38: printf: warning: 0: Result too large
* tests/id/setgid.sh: Skip the test when the adjusted gid
would equal 4294967295, as that's reserved on AIX.
Reported by Michael Felt.

* src/sort.c (main): Don't assume hard_LC_COLLATE implies
a successful setting of the locale as musl defaults to
UTF8 when failing to set the specified locale.
* tests/misc/sort-debug-warn.sh: Adjust for the now
separated locale debug info and map the musl specific
message back to the common case.
Addresses https://bugs.gnu.org/28054

* src/seq.c (io_error): Use the same error message as would
be generated at exit time when closing the stdout stream.
The inconsistency was added with commit v8.25-26-gc92585b1.
This was noticed due to an inconsistency in the expected
error message generated by seq on musl libc.
Addresses https://bugs.gnu.org/28054

* tests/misc/printf-surprise.sh: With musl libc the
large printf format does succeed, outputting data.
To avoid SIGPIPE being generated we ignore that signal
and then handle the subsequent EPIPE error.
Addresses https://bugs.gnu.org/28054

* configure.ac: Disable some new warnings to avoid false positives.
Building with warnings enabled and latest gcc would evoke build
failure without these changes.  Disable the following in coreutils
proper: -Wformat-overflow=2 -Wformat-truncation=2, and
disable these for gnulib: -Wformat-truncation=2 -Wduplicated-branches