aboutsummaryrefslogtreecommitdiffstats
path: root/tests
Commit message (Collapse)AuthorAgeFilesLines
* Improve too large physical ashift handlingAlexander Motin2022-09-082-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When iterating through children physical ashifts for vdev, prefer ones above the maximum logical ashift, that we can actually use, but within the administrator defined maximum. When selecting top-level vdev ashift, do not set it to the defined maximum in case physical ashift is even higher, but just ignore one. Using the maximum does not prevent misaligned writes, but reduces space efficiency. Since ZFS tries to write data sequentially and aggregates the writes, in many cases large misanigned writes may be not as bad as the space penalty otherwise. Allow internal physical ashifts for vdevs higher than SHIFT_MAX. May be one day allocator or aggregation could benefit from that. Reduce zfs_vdev_max_auto_ashift default from 16 (64KB) to 14 (16KB), so that ZFS may still use bigger ashifts up to SHIFT_MAX (64KB), but only if it really has to or explicitly told to, but not as an "optimization". There are some read-intensive NVMe SSDs that report Preferred Write Alignment of 64KB, and attempt to build RAIDZ2 of those leads to a space inefficiency that can't be justified. Instead these changes make ZFS fall back to logical ashift of 12 (4KB) by default and only warn user that it may be suboptimal for performance. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes #13798
* Add Linux posix_fadvise supportFinix19792022-09-0811-2/+249
| | | | | | | | | | | | | | | | | The purpose of this PR is to accepts fadvise ioctl from userland to do read-ahead by demand. It could dramatically improve sequential read performance especially when primarycache is set to metadata or zfs_prefetch_disable is 1. If the file is mmaped, generic_fadvise is also called for page cache read-ahead besides dmu_prefetch. Only POSIX_FADV_WILLNEED and POSIX_FADV_SEQUENTIAL are supported in this PR currently. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Finix Yan <[email protected]> Closes #13694
* Add zfs.sync.snapshot_renameAndriy Gapon2022-09-023-0/+70
| | | | | | | | | Only the single snapshot rename is provided. The recursive or more complex rename can be scripted. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Andriy Gapon <[email protected]> Closes #13802
* Add zilstat script to report zil kstats in a user friendly mannerAmeer Hamza2022-09-025-2/+43
| | | | | | | | | | | | Added a python script to process both global and per dataset zil kstats and report them in a user friendly manner similar to arcstat and dbufstat. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Richard Elling <[email protected]> Signed-off-by: Ameer Hamza <[email protected]> Closes #13704
* ZTS: zvol_stress: fix race condition with zinject usageChristian Schwarz2022-08-251-0/+1
| | | | | | | | | | | | | | | | | In automated ZTS runs, I'd occasionally hit log_fail "Expected to see some write errors" because there weren't any write errors. The reason is that we're not syncing the zpool before `zinject -c`. If the writes by `dd` aren't synced out at the time `zinject -c` runs, they will not hit an error and we'll hit the log_fail above. Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Christian Schwarz <[email protected]> Closes #13793
* Fix zpool status in case of unloaded keysGeorge Amanakis2022-08-225-1/+82
| | | | | | | | | | When scrubbing an encrypted filesystem with unloaded key still report an error in zpool status. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alek Pinchuk <[email protected]> Signed-off-by: George Amanakis <[email protected]> Closes #13675 Closes #13717
* Enable relatime by defaultGeorge Melikov2022-08-125-10/+12
| | | | | | | | | | Linux sets relatime on mount by default for any file system, but relatime=off in ZFS disables it explicitly. Let's be consistent with other file systems on Linux. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: George Melikov <[email protected]> Closes #13614
* ZTS: Fix zpool_expand_001_posTony Hutter2022-08-091-47/+54
| | | | | | | | | | | | | | `zpool_expand_001_pos` was often failing due to not seeing autoexpand commands in the `zpool history`. During testing, I found this to be unreliable (sometimes the "online" wouldn't appear in `zpool history`) and unnecessary, as we could simply check that the pool increased in size. This commit revamps the test to check for the expanded pool size and corresponding new free space. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes #13743
* zpool: fix redundancy check after vdev removalStéphane Lesimple2022-08-042-1/+59
| | | | | | | | | | | | | | The presence of indirect vdevs was confusing get_redundancy(), which considered a pool with e.g. only mirror top-level vdevs and at least one indirect vdev (due to the removal of a previous vdev) as already having a broken redundancy, which is not the case. This lead to the possibility of compromising the redundancy of a pool by adding mismatched vdevs without requiring the use of `-f`, and with no visible notice or warning. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Stéphane Lesimple <[email protected]> Closes #13705 Closes #13711
* Add snapshots_changed as propertyUmer Saleem2022-08-025-2/+145
| | | | | | | | | | | | | | | | | | | Make dd_snap_cmtime property persistent across mount and unmount operations by storing in ZAP and restore the value from ZAP on hold into dd_snap_cmtime instead of updating it. Expose dd_snap_cmtime as 'snapshots_changed' property that provides a mechanism to quickly determine whether snapshot list for dataset has changed without having to mount a dataset or iterate the snapshot list. It specifies the time at which a snapshot for a dataset was last created or deleted. This allows us to be more efficient how often we query snapshots. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Umer Saleem <[email protected]> Closes #13635
* Implement a new type of zfs receive: corrective receive (-c)Alek P2022-07-285-1/+390
| | | | | | | | | | | | | | This type of recv is used to heal corrupted data when a replica of the data already exists (in the form of a send file for example). With the provided send stream, corrective receive will read from disk blocks described by the WRITE records. When any of the reads come back with ECKSUM we use the data from the corresponding WRITE record to rewrite the corrupted block. Reviewed-by: Paul Dagnelie <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Paul Zuchowski <[email protected]> Signed-off-by: Alek Pinchuk <[email protected]> Closes #9372
* ZTS: Fix io_uring support checkBrian Behlendorf2022-07-261-1/+1
| | | | | | | | | | | | Not all Linux distribution kernels enable io_uring support by default. Update the run time check to verify that the booted kernel was built with CONFIG_IO_URING=y. Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Tony Nguyen <[email protected]> Co-authored-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13648 Closes #13685
* ZTS: Fix occasional inherit_001_pos.ksh failureBrian Behlendorf2022-07-251-1/+1
| | | | | | | | | | | | | The mountpoint may still be busy when the `zfs unmount -a` command is run causing an unexpected failure. Retry the unmount a couple of times since it should not remain busy for long. 19:10:50.29 NOTE: Reading state from .../inheritance/state021.cfg 19:10:50.32 cannot unmount '/TESTPOOL': pool or dataset is busy 19:10:50.32 ERROR: zfs unmount -a exited 1 Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13686
* Replace dead opensolaris.org license linkTino Reichardt2022-07-111271-1271/+1271
| | | | | | | | | The commit replaces all findings of the link: http://www.opensolaris.org/os/licensing with this one: https://opensource.org/licenses/CDDL-1.0 Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tino Reichardt <[email protected]> Closes #13619
* tests: modernise zdb_decompressнаб2022-06-291-9/+7
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13348
* Enable -Wwrite-stringsнаб2022-06-2917-201/+126
| | | | | | | | Also, fix leak from ztest_global_vars_to_zdb_args() Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13348
* Scrub mirror children without BPsBrian Behlendorf2022-06-235-23/+164
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When scrubbing a raidz/draid pool, which contains a replacing or sparing mirror with multiple online children, only one child will be read. This is not normally a serious concern because the DTL records are used to determine where a good copy of the data is. As long as the data can be read from one child the mirror vdev will use it to repair gaps in any of its children. Furthermore, even if the data which was read is corrupt the raidz code will detect this and issue its own repair I/O to correct the damage in the mirror vdev. However, in the scenario where the DTL is wrong due to silent data corruption (say due to overwriting one child) and the scrub happens to read from a child with good data, then the other damaged mirror child will not be detected nor repaired. While this is possible for both raidz and draid vdevs, it's most pronounced when using draid. This is because by default the zed will sequentially rebuild a draid pool to a distributed spare, and the distributed spare half of the mirror is always preferred since it delivers better performance. This means the damaged half of the mirror will go undetected even after scrubbing. For system administrations this behavior is non-intuitive and in a worst case scenario could result in the only good copy of the data being unknowingly detached from the mirror. This change resolves the issue by reading all replacing/sparing mirror children when scrubbing. When the BP isn't available for verification, then compare the data buffers from each child. They must all be identical, if not there's silent damage and an error is returned to prompt the top-level vdev to issue a repair I/O to rewrite the data on all of the mirror children. Since we can't tell which child was wrong a checksum error is logged against the replacing or sparing mirror vdev. Reviewed-by: Mark Maybee <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13555
* Add Linux namespace delegation supportWill Andrews2022-06-107-1/+291
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows ZFS datasets to be delegated to a user/mount namespace Within that namespace, only the delegated datasets are visible Works very similarly to Zones/Jailes on other ZFS OSes As a user: ``` $ unshare -Um $ zfs list no datasets available $ echo $$ 1234 ``` As root: ``` # zfs list NAME ZONED MOUNTPOINT containers off /containers containers/host off /containers/host containers/host/child off /containers/host/child containers/host/child/gchild off /containers/host/child/gchild containers/unpriv on /unpriv containers/unpriv/child on /unpriv/child containers/unpriv/child/gchild on /unpriv/child/gchild # zfs zone /proc/1234/ns/user containers/unpriv ``` Back to the user namespace: ``` $ zfs list NAME USED AVAIL REFER MOUNTPOINT containers 129M 47.8G 24K /containers containers/unpriv 128M 47.8G 24K /unpriv containers/unpriv/child 128M 47.8G 128M /unpriv/child ``` Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Will Andrews <[email protected]> Signed-off-by: Allan Jude <[email protected]> Signed-off-by: Mateusz Piotrowski <[email protected]> Co-authored-by: Allan Jude <[email protected]> Co-authored-by: Mateusz Piotrowski <[email protected]> Sponsored-by: Buddy <https://buddy.works> Closes #12263
* zvol: Support blk-mq for better performanceTony Hutter2022-06-0912-14/+534
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add support for the kernel's block multiqueue (blk-mq) interface in the zvol block driver. blk-mq creates multiple request queues on different CPUs rather than having a single request queue. This can improve zvol performance with multithreaded reads/writes. This implementation uses the blk-mq interfaces on 4.13 or newer kernels. Building against older kernels will fall back to the older BIO interfaces. Note that you must set the `zvol_use_blk_mq` module param to enable the blk-mq API. It is disabled by default. In addition, this commit lets the zvol blk-mq layer process whole `struct request` IOs at a time, rather than breaking them down into their individual BIOs. This reduces dbuf lock contention and overhead versus the legacy zvol submit_bio() codepath. sequential dd to one zvol, 8k volblocksize, no O_DIRECT: legacy submit_bio() 292MB/s write 453MB/s read this commit 453MB/s write 885MB/s read It also introduces a new `zvol_blk_mq_chunks_per_thread` module parameter. This parameter represents how many volblocksize'd chunks to process per each zvol thread. It can be used to tune your zvols for better read vs write performance (higher values favor write, lower favor read). Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ahelenia Ziemiańska <[email protected]> Reviewed-by: Tony Nguyen <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes #13148 Issue #12483
* Introduce BLAKE3 checksums as an OpenZFS featureTino Reichardt2022-06-0814-15/+619
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit adds BLAKE3 checksums to OpenZFS, it has similar performance to Edon-R, but without the caveats around the latter. Homepage of BLAKE3: https://github.com/BLAKE3-team/BLAKE3 Wikipedia: https://en.wikipedia.org/wiki/BLAKE_(hash_function)#BLAKE3 Short description of Wikipedia: BLAKE3 is a cryptographic hash function based on Bao and BLAKE2, created by Jack O'Connor, Jean-Philippe Aumasson, Samuel Neves, and Zooko Wilcox-O'Hearn. It was announced on January 9, 2020, at Real World Crypto. BLAKE3 is a single algorithm with many desirable features (parallelism, XOF, KDF, PRF and MAC), in contrast to BLAKE and BLAKE2, which are algorithm families with multiple variants. BLAKE3 has a binary tree structure, so it supports a practically unlimited degree of parallelism (both SIMD and multithreading) given enough input. The official Rust and C implementations are dual-licensed as public domain (CC0) and the Apache License. Along with adding the BLAKE3 hash into the OpenZFS infrastructure a new benchmarking file called chksum_bench was introduced. When read it reports the speed of the available checksum functions. On Linux: cat /proc/spl/kstat/zfs/chksum_bench On FreeBSD: sysctl kstat.zfs.misc.chksum_bench This is an example output of an i3-1005G1 test system with Debian 11: implementation 1k 4k 16k 64k 256k 1m 4m edonr-generic 1196 1602 1761 1749 1762 1759 1751 skein-generic 546 591 608 615 619 612 616 sha256-generic 240 300 316 314 304 285 276 sha512-generic 353 441 467 476 472 467 426 blake3-generic 308 313 313 313 312 313 312 blake3-sse2 402 1289 1423 1446 1432 1458 1413 blake3-sse41 427 1470 1625 1704 1679 1607 1629 blake3-avx2 428 1920 3095 3343 3356 3318 3204 blake3-avx512 473 2687 4905 5836 5844 5643 5374 Output on Debian 5.10.0-10-amd64 system: (Ryzen 7 5800X) implementation 1k 4k 16k 64k 256k 1m 4m edonr-generic 1840 2458 2665 2719 2711 2723 2693 skein-generic 870 966 996 992 1003 1005 1009 sha256-generic 415 442 453 455 457 457 457 sha512-generic 608 690 711 718 719 720 721 blake3-generic 301 313 311 309 309 310 310 blake3-sse2 343 1865 2124 2188 2180 2181 2186 blake3-sse41 364 2091 2396 2509 2463 2482 2488 blake3-avx2 365 2590 4399 4971 4915 4802 4764 Output on Debian 5.10.0-9-powerpc64le system: (POWER 9) implementation 1k 4k 16k 64k 256k 1m 4m edonr-generic 1213 1703 1889 1918 1957 1902 1907 skein-generic 434 492 520 522 511 525 525 sha256-generic 167 183 187 188 188 187 188 sha512-generic 186 216 222 221 225 224 224 blake3-generic 153 152 154 153 151 153 153 blake3-sse2 391 1170 1366 1406 1428 1426 1414 blake3-sse41 352 1049 1212 1174 1262 1258 1259 Output on Debian 5.10.0-11-arm64 system: (Pi400) implementation 1k 4k 16k 64k 256k 1m 4m edonr-generic 487 603 629 639 643 641 641 skein-generic 271 299 303 308 309 309 307 sha256-generic 117 127 128 130 130 129 130 sha512-generic 145 165 170 172 173 174 175 blake3-generic 81 29 71 89 89 89 89 blake3-sse2 112 323 368 379 380 371 374 blake3-sse41 101 315 357 368 369 364 360 Structurally, the new code is mainly split into these parts: - 1x cross platform generic c variant: blake3_generic.c - 4x assembly for X86-64 (SSE2, SSE4.1, AVX2, AVX512) - 2x assembly for ARMv8 (NEON converted from SSE2) - 2x assembly for PPC64-LE (POWER8 converted from SSE2) - one file for switching between the implementations Note the PPC64 assembly requires the VSX instruction set and the kfpu_begin() / kfpu_end() calls on PowerPC were updated accordingly. Reviewed-by: Felix Dörre <[email protected]> Reviewed-by: Ahelenia Ziemiańska <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tino Reichardt <[email protected]> Co-authored-by: Rich Ercolani <[email protected]> Closes #10058 Closes #12918
* Replace EXTRA_DIST with dist_noinst_DATABrian Behlendorf2022-05-263-4/+4
| | | | | | | | | | | | | | | The EXTRA_DIST variable is ignored when used in the FALSE conditional of a Makefile.am. This results in the `make dist` target omitting these files from the generated tarball unless CONFIG_USER is defined. This issue can be avoided by switching to use the dist_noinst_DATA variable which is handled as expected by autoconf. This change also adds support for --with-config=dist as an alias for --with-config=srpm and updates the GitHub workflows to use it. Reviewed-by: Ahelenia Ziemiańska <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13459 Closes #13505
* Add make regen-tests to regenerate the test bundleнаб2022-05-181-1/+27
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13447
* tests: add zfs_unshare_008_pos checking whitespace escapingнаб2022-05-125-1/+73
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13165
* tests: many_fds: simplify, moderniseнаб2022-05-111-34/+20
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13411
* Adding ZTS test for O_APPENDBrian Atkinson2022-05-1113-25/+296
| | | | | | | | | | | | | | | Commit 63b18e4 fixed an issue in zpl_aio_write() to make sure that kiocb->ki_pos was updated correctly when opening a file with O_APPEND. Adding a test to verify O_APPEND functionality with lseek can make sure that all other distros/kernel versions also have the correct behavior. Also moved the threadappends_001_pos test into this append test directory in functional ZTS directory. This way the two append tests are together for organization purposes. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Brian Atkinson <[email protected]> Closes #13424
* autoconf: use include directives instead of recursing down test dataнаб2022-05-10176-2686/+1952
| | | | | | | | | | | | | | We drop /multiple/ seconds off the generation, a dozen off a clean rebuild, 185 files, and trivialise the distribution, which can now be trivially generated via the provided snippets Dist diff: -zfs-2.1.99/tests/zfs-tests/tests/functional/pam/utilities.kshlib +zfs-2.1.99/tests/zfs-tests/tests/functional/pam/utilities.kshlib.in Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13316
* autoconf: use include directives instead of recursing down tests (mostly)наб2022-05-1015-229/+163
| | | | | | | | | | | | | | | Only down to tests/zfs-tests/tests, but pull out C programs into the main Makefile ‒ this means we get correct dependency tracking for all programs (and parallelise across them) dist diff: -zfs-2.1.99/tests/zfs-tests/tests/stress/ -zfs-2.1.99/tests/zfs-tests/tests/stress/Makefile.am -zfs-2.1.99/tests/zfs-tests/tests/stress/Makefile.in Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13316
* autoconf: use include directives instead of recursing down etcнаб2022-05-101-0/+1
| | | | | | | | | | dist diff: -zfs-2.1.99/etc/systemd/system/50-zfs.preset.in +zfs-2.1.99/etc/systemd/system/50-zfs.preset Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13316
* Move test-runner.1 into top-level man/наб2022-05-103-298/+1
| | | | | | | | | | | dist delta: +zfs-2.1.99/man/man1/test-runner.1 -zfs-2.1.99/tests/test-runner/man/ -zfs-2.1.99/tests/test-runner/man/test-runner.1 Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13316
* autoconf: use include directives instead of recursing down manнаб2022-05-101-3/+0
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13316
* autoconf: use include directives instead of recursing down cmdнаб2022-05-105-2/+8
| | | | | | | | | | | | | | | | | No installation diff, dist lost -zfs-2.1.99/cmd/fsck_zfs/fsck.zfs which was distributed erroneously, since it's generated Also clean gitrev on clean Also add -e 'any possible bashisms' to default checkbashisms flags, and fully parallelise it and shellcheck, and it works out-of-tree, too Also align the Release in the dist META file correctly Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13316
* autoconf: use include directives instead of recursing down libнаб2022-05-103-26/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As a bonus, this also adds zfs-mount-generator (previously undescended down) and libzstd (not included) to CppCheck As a bonus bonus, abigail rules work out-of-tree, too Against current trunk: $ diff -U0 ./destdir.listing ~/store/code/zfs/destdir.listing -destdir/usr/local/include/libspl/sscanf.h $ diff --color -U0 ./zfs-2.1.99.tar.gz.listing ../oot/zfs-2.1.99.tar.gz.listing | grep -v @@ | grep -v /Makefile -zfs-2.1.99/config/Abigail.am -zfs-2.1.99/lib/libspl/include/util/ -zfs-2.1.99/lib/libspl/include/util/sscanf.h $ diff --color -U0 ./zfs-2.1.99.tar.gz.listing ../oot/zfs-2.1.99.tar.gz.listing | grep -v @@ | grep /Makefile -zfs-2.1.99/lib/libavl/Makefile.in -zfs-2.1.99/lib/libefi/Makefile.in -zfs-2.1.99/lib/libicp/Makefile.in -zfs-2.1.99/lib/libnvpair/Makefile.in -zfs-2.1.99/lib/libshare/Makefile.in -zfs-2.1.99/lib/libspl/include/Makefile.in -zfs-2.1.99/lib/libspl/include/os/freebsd/Makefile.am -zfs-2.1.99/lib/libspl/include/os/freebsd/Makefile.in -zfs-2.1.99/lib/libspl/include/os/freebsd/sys/Makefile.am -zfs-2.1.99/lib/libspl/include/os/freebsd/sys/Makefile.in -zfs-2.1.99/lib/libspl/include/os/linux/Makefile.am -zfs-2.1.99/lib/libspl/include/os/linux/Makefile.in -zfs-2.1.99/lib/libspl/include/os/linux/sys/Makefile.am -zfs-2.1.99/lib/libspl/include/os/linux/sys/Makefile.in -zfs-2.1.99/lib/libspl/include/os/Makefile.am -zfs-2.1.99/lib/libspl/include/os/Makefile.in -zfs-2.1.99/lib/libspl/include/rpc/Makefile.am -zfs-2.1.99/lib/libspl/include/rpc/Makefile.in -zfs-2.1.99/lib/libspl/include/sys/dktp/Makefile.am -zfs-2.1.99/lib/libspl/include/sys/dktp/Makefile.in -zfs-2.1.99/lib/libspl/include/sys/Makefile.am -zfs-2.1.99/lib/libspl/include/sys/Makefile.in -zfs-2.1.99/lib/libspl/include/util/Makefile.am -zfs-2.1.99/lib/libspl/include/util/Makefile.in -zfs-2.1.99/lib/libspl/Makefile.in -zfs-2.1.99/lib/libtpool/Makefile.in -zfs-2.1.99/lib/libunicode/Makefile.in -zfs-2.1.99/lib/libuutil/Makefile.in -zfs-2.1.99/lib/libzfsbootenv/Makefile.in -zfs-2.1.99/lib/libzfs_core/Makefile.in -zfs-2.1.99/lib/libzfs/Makefile.in -zfs-2.1.99/lib/libzpool/Makefile.in -zfs-2.1.99/lib/libzstd/Makefile.in -zfs-2.1.99/lib/libzutil/Makefile.in -zfs-2.1.99/lib/Makefile.in Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13316
* Speed up WB_SYNC_NONE when a WB_SYNC_ALL occurs simultaneouslyShaan Nobee2022-05-038-3/+223
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Page writebacks with WB_SYNC_NONE can take several seconds to complete since they wait for the transaction group to close before being committed. This is usually not a problem since the caller does not need to wait. However, if we're simultaneously doing a writeback with WB_SYNC_ALL (e.g via msync), the latter can block for several seconds (up to zfs_txg_timeout) due to the active WB_SYNC_NONE writeback since it needs to wait for the transaction to complete and the PG_writeback bit to be cleared. This commit deals with 2 cases: - No page writeback is active. A WB_SYNC_ALL page writeback starts and even completes. But when it's about to check if the PG_writeback bit has been cleared, another writeback with WB_SYNC_NONE starts. The sync page writeback ends up waiting for the non-sync page writeback to complete. - A page writeback with WB_SYNC_NONE is already active when a WB_SYNC_ALL writeback starts. The WB_SYNC_ALL writeback ends up waiting for the WB_SYNC_NONE writeback. The fix works by carefully keeping track of active sync/non-sync writebacks and committing when beneficial. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Shaan Nobee <[email protected]> Closes #12662 Closes #12790
* Default zfs_max_recordsize to 16MRich Ercolani2022-04-284-4/+4
| | | | | | | | | | | | | | | | | | | | Increase the default allowed maximum recordsize from 1M to 16M. As described in the zfs(4) man page, there are significant costs which need to be considered before using very large blocks. However, there are scenarios where they make good sense and it should no longer be necessary to artificially restrict their use behind a module option. Note that for 32-bit platforms we continue to leave this restriction in place due to the limited virtual address space available (256-512MB). On these systems only a handful of blocks could be cached at any one time severely impacting performance and potentially stability. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Rich Ercolani <[email protected]> Closes #12830 Closes #13302
* Improve zpool status output, list all affected datasetsGeorge Amanakis2022-04-255-1/+156
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, determining which datasets are affected by corruption is a manual process. The primary difficulty in reporting the list of affected snapshots is that since the error was initially found, the snapshot where the error originally occurred in, may have been deleted. To solve this issue, we add the ID of the head dataset of the original snapshot which the error was detected in, to the stored error report. Then any time a filesystem is deleted, the errors associated with it are deleted as well. Any time a clone promote occurs, we modify reports associated with the original head to refer to the new head. The stored error reports are identified by this head ID, the birth time of the block which the error occurred in, as well as some information about the error itself are also stored. Once this information is stored, we can find the set of datasets affected by an error by walking back the list of snapshots in the given head until we find one with the appropriate birth txg, and then traverse through the snapshots of the clone family, terminating a branch if the block was replaced in a given snapshot. Then we report this information back to libzfs, and to the zpool status command, where it is displayed as follows: pool: test state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A scan: scrub repaired 0B in 00:00:00 with 800 errors on Fri Dec 3 08:27:57 2021 config: NAME STATE READ WRITE CKSUM test ONLINE 0 0 0 sdb ONLINE 0 0 1.58K errors: Permanent errors have been detected in the following files: test@1:/test.0.0 /test/test.0.0 /test/1clone/test.0.0 A new feature flag is introduced to mark the presence of this change, as well as promotion and backwards compatibility logic. This is an updated version of #9175. Rebase required fixing the tests, updating the ABI of libzfs, updating the man pages, fixing bugs, fixing the error returns, and updating the old on-disk error logs to the new format when activating the feature. Reviewed-by: Matthew Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Mark Maybee <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Co-authored-by: TulsiJain <[email protected]> Signed-off-by: George Amanakis <[email protected]> Closes #9175 Closes #12812
* tests: cli_user: zfs_001_neg: print the problematic linesнаб2022-04-251-1/+1
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13352
* Corrected oversight in ZERO_RANGE behaviorRich Ercolani2022-04-207-17/+165
| | | | | | | | | | | | | | It turns out, no, in fact, ZERO_RANGE and PUNCH_HOLE do have differing semantics in some ways - in particular, one requires KEEP_SIZE, and the other does not. Also added a zero-range test to catch this, corrected a flaw that made the punch-hole test succeed vacuously, and a typo in file_write. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Rich Ercolani <[email protected]> Closes #13329 Closes #13338
* linux: module: weld all but spl.ko into zfs.koнаб2022-04-203-8/+6
| | | | | | | | | | | | | | Originally it was thought it would be useful to split up the kmods by functionality. This would allow external consumers to only load what was needed. However, in practice we've never had a case where this functionality would be needed, and conversely managing multiple kmods can be awkward. Therefore, this change merges all but the spl.ko kmod in to a single zfs.ko kmod. Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13274
* tests: cmd: draid: remove unused and undocumented -vнаб2022-04-131-5/+2
| | | | | | | | Found with -Wunused-but-set-variable on Clang trunk Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13304
* ZTS: Retry auto_spare_multiple.kshBrian Behlendorf2022-04-131-0/+1
| | | | | | | | | | The auto_spare_multiple.ksh test may incorrectly fail for a similar reason as the auto_spare_shared.ksh test. Add it to known list of exceptions which should be retried to prevent failures in the CI. Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13318
* ZTS: Retry redundancy_draid_spare[1,3].kshBrian Behlendorf2022-04-132-5/+2
| | | | | | | | | | | | The redundancy_draid_spare1.ksh and redundancy_draid_spare3.ksh test cases are a little to strict for the sequential resilver case. While unlikely it is possible that a handful of correctable checksum errors will be reported resulting in a test failure. Update the zts-report.py script to allow this the test case to be retried if requested. Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13318
* tests: zts-report: add #13215 to known list for FreeBSDнаб2022-04-011-0/+1
| | | | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13259
* tests: zts-report: issue numbers are numbersнаб2022-04-011-34/+34
| | | | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13259
* tests: clean out more temporary filesнаб2022-04-0116-46/+44
| | | | | | | | | | | | What remains is a bunch of anonymous untraceable /tmp/tmp.XXXXXXXXXX files and bak.root.receive.staff1.3835 from an error branch, testdir.1, testdir.3, and testroot454470 (with children) in testroot Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13259
* tests: remove unused functionsнаб2022-04-0110-809/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As found by git -C tests/ grep ^function | grep -vFe '.lua:' -e '.zcp:' | while IFS=":$IFS" read -r _ _ fn _; do [ $(git -C tests/ grep -wF $fn | head -2 | wc -l) -eq 1 ] && echo $fn; done after all rounds this comes out to, sorted: check_slog_state chgusr_exec cksum_files cleanup_pools compare_modes count_ACE dataset_set_defaultproperties ds_is_snapshot get_ACE get_group get_min get_mode get_owner get_rand_checksum get_rand_checksum_any get_rand_large_recsize get_rand_recsize get_user_group getitem indirect_vdev_mapping_size is_dilos log_noresult log_notinuse log_other log_timed_out log_uninitiated log_warning num_jobs_by_cpu plus_sign_check_l plus_sign_check_v record_cksum rwx_node seconds_mmp_waits_for_activity set_cur_usr setup_mirrors setup_raidzs showshares_smb zfs_zones_setup This, of course, doesn't catch recursive ones, or ones that log with their own function name as a prefix, but Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13259
* tests: include: use already-set $UNAME instead of shelling out to uname each ↵наб2022-04-013-76/+90
| | | | | | | | | | time Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13259
* tests: pam: use absolute path to module .soнаб2022-04-015-11/+13
| | | | | | | | | | | | This is a valid configuration and both (a) skips the tests if it's unbuilt/not installed and (b) makes it work even if installed outside the system directory (like in /u/l/l/s instead of /l/s) Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13259
* tests: zts-report: simplifyнаб2022-04-011-25/+12
| | | | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13259
* tests: zts-report: prune unused reasonsнаб2022-04-011-9/+0
| | | | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13259
* tests: zts-report: only known-SKIP zfs_unshare_002_pos on Linuxнаб2022-04-011-1/+1
| | | | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13259
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-30 03:51:40 +0000