GitLab

* NEWS: Record release date.

* tests/df/no-mtab-status: Include <mntent.h> in test program, so
that the getmntent hack compilation fails on Solaris, as it
should, since it's not compatible with Solaris.  Reported by
Stefano Lattarini in <http://bugs.gnu.org/12225>.

* tests/split/filter: Use xz -1 when compressing, to minimize
memory usage.  Otherwise, xz could fail due to insufficient
virtual memory on a system with very little free memory.

* tests/misc/sort-u-FMR: New file.
* tests/Makefile.am (TESTS): Add it.
* tests/misc/sort: Add the test here, too.
* NEWS (Bug fixes): Mention it.

* tests/init.cfg (require_valgrind_): New function...
* tests/misc/sort-stale-thread-mem: ...extracted from here.

This also fixes a free-memory-read (FMR) bug: when fillbuf's realloc
of buf->buf frees the buffer into which saved_line.text points,
the processing of that just-read longer line includes comparison
against the saved line in freed memory.
* src/sort.c (overlap): Remove.
(fillbuf): Do not try to copy saved lines, as that is too risky
in the presence of parallelism, reallocated buffers, etc.
(sort): Invalidate any saved line before sorting a new batch.

sort -u could omit one or more lines of expected output.
This bug arose because sort recorded the most recently printed line via
reference, and if you were unlucky, the storage for that line would be
reused (overwritten) as additional input was read into memory.  If you
were doubly unlucky, the new value of the "saved" line would not only
match the very next line, but if that next line were also the first in
a series of identical, not-yet-printed lines, then the corrupted "saved"
line value would result in the omission of all matching lines.

* src/sort.c (saved_line): New static/global, renamed and moved from...
(write_unique): ...here.  Old name was "saved", which was too generic
for its new role as file-scoped global.
(fillbuf): With --unique, when we're about to read into a buffer that
overlaps the saved "preceding" line (saved_line), copy the line's .text
member to a realloc'd-as-needed temporary buffer and adjust the line's
key-defining members if they're set.
(overlap): New function.
* tests/misc/sort: New tests.
* NEWS (Bug fixes): Mention it.
* THANKS.in: Update.
Bug introduced via commit v8.5-89-g9face836.
Reported by Rasmus Borup Hansen in
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/23173/focus=24647

* tests/Coreutils.pm (_compare_files): Reverse diff arguments so
that we invoke diff -c $expected $actual, which is consistent with
how init.sh-using tests invoke "compare exp out".

* src/sort.c (fillbuf): Fix comment typo.  x2nrealloc no longer
doubles the size of its input buffer.

* src/remove.c (rm_fts): s/can not/cannot/

Add new option to rm (-d/--dir), which allows removal of
empty directories, while still safely disallowing removal
of non-empty ones.

This improves compatibility with Mac OS X and BSD systems,
which honor the -d option.

* src/remove.c (rm_fts): Remove empty directories when requested.
* src/remove.h (rm_options) [remove_empty_directories]: New member.
* src/rm.c (long_opts, usage, main): Update usage and option parsing.
(rm_option_init): Initialize the new member.
* src/mv.c (rm_option_init): Initialize the new member.
* tests/rm/d-1: New test case - successfully delete empty dir.
* tests/rm/d-2: New test case - refuse to delete nonempty dir.
* tests/Makefile.am (TESTS): Add them.

* src/df.c (main): Add conditions to fail when the mount list cannot
be read: this includes the cases when a file name argument is given
and any of -a, -l, -t or -x is used.
* doc/coreutils.texi: Document the additional error conditions.
* tests/df/no-mtab-status: Add a new test.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.

* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.

* NEWS: Record release date.

* tests/init.cfg (require_mount_list_): A new function
to ensure we can read the list of file systems.
(require_local_dir_): Call the above function, as otherwise
the check is invalid.
* tests/df/total-unprocessed: Ensure df can read the
list of mounted file systems so that --local can be honored.

* tests/misc/printf-surprise: A VM size of 10,000KiB was too
little in which to run "env printf ..." on FreeBSD 9.0-p3.
Increase it to 15,000.

* src/Makefile.am (sort_LDADD): Sort uses euidaccess, which may require
whatever library configure deemed necessary to resolve the eaccess
function, but no one told sort to link with that library.
(sort_LDADD): Add $(LIB_EACCESS).

When the combination of the file system options with given files or
devices does not lead to output, "df --total" would exit successfully
although it should not.

Examples:
  $ df --total --type=xfs /          # when / is not an XFS file system
  $ df --total --local -t nfs  DIR   # nfs is remote per se ...
  $ df --total -t qwerty /dev/sdb5   # typo in file system type

Furthermore, "df --total" would not print the error message "no file
systems processed" when the file argument does not exist or is otherwise
not accessible.

Example:
  $ df --total __not_exist__

These 2 bugs are present since --total was added by commit
v6.12-166-gea2887bb.

* src/df.c (get_dev): Do not set file_systems_processed to true when
force_fsu is true, i.e. when the row for the "total" line is processed.
(main): Don't print totals unless we've processed a file system.
Also only print the "no FS processed" message if there was no
preceding diagnostic.
* tests/df/total-unprocessed: Add a new test.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.

Improved-by: Jim Meyering

* src/truncate.c (main): For a user who makes the mistake of
using a non-seekable file as a reference for the desired length,
truncate would open that file, attempt to seek to its end, but
upon seek failure would neglect to close the file descriptor.
Close the file descriptor even when lseek fails.
In addition, ignore failure to close that reference FD, since as
long as the lseek succeeds, a close failure doesn't matter.
Coverity spotted the potential FD leak.

Improved-by: Pádraig Brady.

* tests/init.cfg (require_ulimit_): Raise VM limit from 10MiB to
20MiB, to accommodate overhead of a valgrind-wrapped date program.
Also declare this function's local variables "local".

* src/split.c (lines_rr) [IF_LINT]: Plug a harmless leak.
(main) [IF_LINT]: Free a usually-small (~70KB) buffer
just before exit, mainly to take this off the radar of
leak-detecting tools.

Improved-by: Pádraig Brady.

* src/tail.c (tail_forever): Close FD to avoid leak after a
failed fstat.

We carry local adjustments for a few gnulib modules via the
patches in gl/.  Nearly all of those patches had become stale
due to evolution of the originals in gnulib.

To refresh them, first make sure you have no local changes in gl/
or in the gnulib submodule, then run "make refresh-gnulib-patches".

* src/tail.c (check_fspec): Save fstat-induced errno *before*
calling close_fd, not after.  Otherwise, the close could well
clobber the global errno, making tail print an invalid diagnostic.
This could happen only with tail -f, and even then, only when
a valid file descriptor were to provoke fstat failure.

Add a test and NEWS entry for a bug inadvertently fixed in
a refactoring in commit v8.9-32-gd4db0cb1

* tests/misc/join (v2-format): Add a new test.
* THANKS.in: Add the reporter.
* NEWS: Mention the old bug.
* cfg.mk (old_NEWS_hash): Update.

Reported-by: Jean-Pierre Tosoni

* man/hostid.x: Add gethostid(3) to SEE ALSO section.
Addresses http://bugs.gnu.org/12023

* man/sort.x: Add SEE ALSO section with entry uniq(1).
* man/uniq.x: Add sort(1) to SEE ALSO section.

* src/system.h (stzncpy): Add "restrict" attribute to each pointer
parameter and note in the comment that the buffers must not overlap.

* src/remove.c (cache_stat_ok, is_nondir_lstat): Remove unused
functions.

* cfg.mk (exclude_file_name_regexp--sc_file_system): Sync this
exemption regexp to match renamed tests/df/df-P.  This avoids
a "make syntax-check" failure.

* src/remove.c (cache_statted, is_dir_lstat): Remove unused
static-inlined functions.
* THANKS.in: Remove my name from this list, now that (with this
commit) it is included automatically.
Copyright-paperwork-exempt: Yes

* tests/cp/fiemap-perf: Skip the test on ext2 file systems,
as we do for ext3.  Also skip the test if we can't create
a 1TiB file, which might not be supported on certain file systems.
Signed-off-by: Pádraig Brady <P@draigBrady.com>

* test/Makefile.am: Reference the moved tests.

It's awkward to read and problematic for scripts when
control characters like '\n' are output.

Note other fields are already handled with mbsalign,
which converts non printable chars to the replacement char.
A caveat to note with that, is the replacement char takes
a place in the field and so possibly truncates the field
if it was the widest field in the records.

Note a more general replacement function, that
handles all printable, or non white space characters,
would require more sophisticated support for various
encodings, and the complexity vs benefit was not
deemed beneficial enough at present.
Perhaps in future a more general replacement function
could be shared between the various utilities.

Note <space> is unaffected in any field,
which could impact scripts processing the output.
However any of the number fields at least could have
spaces considering `LANG=fr_FR df -B\'1`, so it's
probably best to leave spaces, which also allows
scripts to handle mount points with spaces without change.

* src/df.c (hide_problematic_chars): Replace control chars with '?'.
* tests/df/problematic-chars: Add a new root only test.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.

* src/system.h (stzncpy): New function.
* src/pinky.c (print_entry): Use stzncpy, not stpncpy.
The latter does not NUL-terminate.  I assumed that strncpy was
the only function with such a horrible API.  Today I learned that
stpncpy also may not NUL-terminate its result.
The bugs were introduced in commit v8.17-48-gf79263da.
* src/who.c (print_user): Likewise.
Thanks to Erik Auerswald for spotting my error.

* src/pinky.c (print_entry): Remove unwarranted uses of strncpy.
Instead, use stpcpy and stpncpy.
* src/who.c (print_user): Likewise.
* cfg.mk: Remove strncpy exemptions.

* src/shred.c: Avoid gcc -Wstrict-overflow warning.
Addresses http://bugs.gnu.org/11927

* src/stty.c (usage): Disambiguate explanation of -parodd.
* THANKS.in: Add reporter.
Reported by Michael Stummvoll