aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* autoconf: Pretend `CONFIG_MODULES` is always onDamian Szuberski2022-04-262-26/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Unconditionally inject `CONFIG_MODULES` make variable and `#define CONFIG_MODULES` to Kbuild in `ZFS_LINUX_COMPILE` autoconf function to emulate loadable kernel modules support. This allows OpenZFS to perform Linux checks despite `CONFIG_MODULES=n` in the actual Linux config. - Add `ZFS_AC_KERNEL_CONFIG_MODULES` check which encompasses the logic from `ZFS_AC_KERNEL_TEST_MODULE` with additional diagnostic messages to the user - Removed `ZFS_AC_KERNEL_TEST_MODULE` as it merely duplicates every check in `ZFS_AC_KERNEL_CONFIG_DEFINED` - Moved `ZFS_AC_MODULE_SYMVERS` after `ZFS_AC_KERNEL_CONFIG_DEFINED` so the user has a chance to see the proper diagnostic from the steps before. A workaround for Linux's ``` commit 3e3005df73b535cb849cf4ec8075d6aa3c460f68 Author: Masahiro Yamada <[email protected]> Date: Wed Mar 31 22:38:03 2021 +0900 kbuild: unify modules(_install) for in-tree and external modules If you attempt to build or install modules ('make modules(_install)' with CONFIG_MODULES disabled, you will get a clear error message, but nothing for external module builds. Factor out the modules and modules_install rules into the common part, so you will get the same error message when you try to build external modules with CONFIG_MODULES=n. Signed-off-by: Masahiro Yamada <[email protected]> ``` Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: szubersk <[email protected]> Closes #10832 Closes #13361
* Improve log spacemap load timeAlexander Motin2022-04-2611-147/+257
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previous flushing algorithm limited only total number of log blocks to the minimum of 256K and 4x number of metaslabs in the pool. As result, system with 1500 disks with 1000 metaslabs each, touching several new metaslabs each TXG could grow spacemap log to huge size without much benefits. We've observed one of such systems importing pool for about 45 minutes. This patch improves the situation from five sides: - By limiting maximum period for each metaslab to be flushed to 1000 TXGs, that effectively limits maximum number of per-TXG spacemap logs to load to the same number. - By making flushing more smooth via accounting number of metaslabs that were touched after the last flush and actually need another flush, not just ms_unflushed_txg bump. - By applying zfs_unflushed_log_block_pct to the number of metaslabs that were touched after the last flush, not all metaslabs in the pool. - By aggressively prefetching per-TXG spacemap logs up to 16 TXGs in advance, making log spacemap load process for wide HDD pool CPU-bound, accelerating it by many times. - By reducing zfs_unflushed_log_block_max from 256K to 128K, reducing single-threaded by nature log processing time from ~10 to ~5 minutes. As further optimization we could skip bumping ms_unflushed_txg for metaslabs not touched since the last flush, but that would be an incompatible change, requiring new pool feature. Reviewed-by: Matthew Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Alexander Motin <[email protected]> Sponsored-By: iXsystems, Inc. Closes #12789
* Improve zpool status output, list all affected datasetsGeorge Amanakis2022-04-2521-194/+1219
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, determining which datasets are affected by corruption is a manual process. The primary difficulty in reporting the list of affected snapshots is that since the error was initially found, the snapshot where the error originally occurred in, may have been deleted. To solve this issue, we add the ID of the head dataset of the original snapshot which the error was detected in, to the stored error report. Then any time a filesystem is deleted, the errors associated with it are deleted as well. Any time a clone promote occurs, we modify reports associated with the original head to refer to the new head. The stored error reports are identified by this head ID, the birth time of the block which the error occurred in, as well as some information about the error itself are also stored. Once this information is stored, we can find the set of datasets affected by an error by walking back the list of snapshots in the given head until we find one with the appropriate birth txg, and then traverse through the snapshots of the clone family, terminating a branch if the block was replaced in a given snapshot. Then we report this information back to libzfs, and to the zpool status command, where it is displayed as follows: pool: test state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A scan: scrub repaired 0B in 00:00:00 with 800 errors on Fri Dec 3 08:27:57 2021 config: NAME STATE READ WRITE CKSUM test ONLINE 0 0 0 sdb ONLINE 0 0 1.58K errors: Permanent errors have been detected in the following files: test@1:/test.0.0 /test/test.0.0 /test/1clone/test.0.0 A new feature flag is introduced to mark the presence of this change, as well as promotion and backwards compatibility logic. This is an updated version of #9175. Rebase required fixing the tests, updating the ABI of libzfs, updating the man pages, fixing bugs, fixing the error returns, and updating the old on-disk error logs to the new format when activating the feature. Reviewed-by: Matthew Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Mark Maybee <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Co-authored-by: TulsiJain <[email protected]> Signed-off-by: George Amanakis <[email protected]> Closes #9175 Closes #12812
* tests: cli_user: zfs_001_neg: print the problematic linesнаб2022-04-251-1/+1
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13352
* man: zfs-send.8: fix -X synopses and descriptionнаб2022-04-252-88/+45
| | | | | | | | Also clean up the horrendously verbose -X handling in zfs_main() Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13352
* zvol_wait: Ignore locked zvolsRichard Laager2022-04-221-4/+6
| | | | | | | | | | | | "When an encrypted zvol is locked the zfs-volume-wait service does not start. The /sbin/zvol_wait should not wait for links when the volume has property keystatus=unavailable." -- https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1888405 Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Damian Szuberski <[email protected]> Thanks: James Dingwall <[email protected]> Signed-off-by: Richard Laager <[email protected]> Closes #10662
* Linux 5.18 compat: kobj_type.default_attrs replaced with default_groupsнаб2022-04-223-17/+61
| | | | | | | | Upstream-commit: cdb4f26a63c391317e335e6e683a614358e70aeb ("kobject: kobj_type: remove default_attrs") Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13357
* linux: module: zfs: sysfs: constify types and attrsнаб2022-04-221-5/+5
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13357
* scripts: zfs.sh: explicitly unload all modules via rmmodнаб2022-04-211-4/+7
| | | | | | | | | | modprobe -r only works for depmodded modules, but this also means we have to re-iterate legacy modules, and in the right order Reviewed-by: Brian Atkinson <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13356
* scripts: zfs.sh: explicitly ignore unloaded modules when unloadingнаб2022-04-211-1/+1
| | | | | | | Reviewed-by: Brian Atkinson <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13356
* Strengthen Linux kernel capabilities detectionDamian Szuberski2022-04-212-4/+31
| | | | | | | | | | | | | | - Add `CONFIG_BLOCK` Linux config requirement to `ZFS_AC_KERNEL_CONFIG_DEFINED`. OpenZFS won't compile without that block device support due to large amount of functional dependencies on it. - Remove dependency on `groups_alloc()` in `ZFS_AC_KERNEL_SRC_GROUP_INFO_GID` to circumvent the missing stub in Linux 4.X kernel headers. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: szubersk <[email protected]> Closes #13351
* contrib: dracut: remove getargbool polyfillнаб2022-04-201-14/+0
| | | | | | | | | It was originally released in dracut 008 in February 2011; we can probably drop it now Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13291
* Add dracut.zfs.7наб2022-04-203-0/+281
| | | | | | | | | | Thorough documentation with a dracut.bootup(7)-style flowchart, dracut.cmdline(7)-style cmdline listing, and per-file docs like the old README Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13291
* contrib: dracut: zfs-needshutdown: don't listнаб2022-04-201-1/+1
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13291
* contrib: dracut: zfs-{rollback,snapshot}-bootfs: order after key loadingнаб2022-04-202-8/+4
| | | | | | | | This fixes at least one race I got with an encrypted root Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13291
* contrib: dracut: don't require essentials to be under the same encrootнаб2022-04-202-32/+40
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13291
* contrib: dracut: inline single-use import_pool, move single-use ask_for_passwordнаб2022-04-202-82/+72
| | | | | | | | | Also don't set ROOTFS_MOUNTED; the final mention was removed in dracut 011 from July 2011 Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13291
* contrib: dracut: zfs-lib: remove find_bootfsнаб2022-04-202-27/+2
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13291
* contrib: dracut: zfs-lib: simplify ask_for_passwordнаб2022-04-202-50/+13
| | | | | | | | | The only user is mount-zfs.sh (non-systemd systems), so reduce it to what it needs Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13291
* contrib; dracut: flatten zfs-load-key, simplify zfs-env-bootfsнаб2022-04-203-45/+36
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13291
* contrib; dracut: centralise root= parsing, actually support root=sнаб2022-04-206-119/+108
| | | | | | | | | | | | | | | | | | | | | | | | | So far, everything parsed root= manually, which meant that while zfs-parse.sh was updated, and supposedly supported + -> ' ' conversion, it meant nothing Instead, centralise parsing, and allow: root= root=zfs root=zfs: root=zfs:AUTO root=ZFS=data/set root=zfs:data/set root=zfs:ZFS=data/set (as a side-effect; allowed but undocumented) rootfstype=zfs AND root=data/set <=> root=data/set rootfstype=zfs AND root= <=> root=zfs:AUTO So rootfstype=zfs /also/ behaves as expected, and + decoding works Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13291
* contrib: dracut: parse-zfs: stop pretending we support FILESYSTEM=наб2022-04-201-4/+1
| | | | | | | | | | | | | It was added in the original ae26d0465a ("Add dracut support") commit in 2011, and was then broken a bit later with the advent of dracut-zfs-generator, or maybe earlier as part of other churn Either way, it's broken, and has been in 2.0+ as well, and no-one complained. Stop pretending we support it at all Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13291
* contrib: dracut: parse-zfs: drop initqueue-finished for i/fнаб2022-04-201-3/+0
| | | | | | | | | The switch was released in dracut 009 in March 2011, we can safely get rid of the compatibility hook Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13291
* Corrected oversight in ZERO_RANGE behaviorRich Ercolani2022-04-208-21/+171
| | | | | | | | | | | | | | It turns out, no, in fact, ZERO_RANGE and PUNCH_HOLE do have differing semantics in some ways - in particular, one requires KEEP_SIZE, and the other does not. Also added a zero-range test to catch this, corrected a flaw that made the punch-hole test succeed vacuously, and a typo in file_write. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Rich Ercolani <[email protected]> Closes #13329 Closes #13338
* FreeBSD: Fix translation from ABD to physical pagesAlexander Motin2022-04-201-1/+5
| | | | | | | | | | | | | In hypothetical case of non-linear ABD with single segment, multiple to page size but not aligned to it, vdev_geom_fill_unmap_cb() could fill one page less into bio_ma array. I am not sure it is exploitable, but better to be safe than sorry. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reported-by: Mark Johnston <[email protected]> Signed-off-by: Alexander Motin <[email protected]> Closes #13345
* man: ... -> … againнаб2022-04-205-26/+30
| | | | | | | | zfs-program.8 is left, but that's literal Lua syntax Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13255
* rpm -> deb doesn't fail when optional packages are missingDamian Szuberski2022-04-201-2/+2
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: szubersk <[email protected]> Closes #13331 Closes #13336
* Document zfs inherit -S's interaction with noninheritable propertiesнаб2022-04-201-2/+3
| | | | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Damian Szuberski <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #11894 Closes #13335
* zpool_history_unpack: return correct errno on nvlist_unpack failureLow-power2022-04-201-2/+3
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Damian Szuberski <[email protected]> Signed-off-by: WHR <[email protected]> Closes #13321
* linux: module: uninstall legacy modules on (un)installationнаб2022-04-201-2/+6
| | | | | | | | | | This can be reverted once we're sure nobody's using them anymore (post-3.0 release?) Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13274
* scripts: zfs.sh: unload zfs with dependenciesнаб2022-04-201-51/+8
| | | | | | | Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13274
* scripts: zfs.sh: make usage make senseнаб2022-04-201-31/+27
| | | | | | | | | We don't pass the arguments as arguments Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13274
* scripts: zfs.sh: remove catнаб2022-04-201-3/+2
| | | | | | | Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13274
* linux: module: weld all but spl.ko into zfs.koнаб2022-04-2040-749/+518
| | | | | | | | | | | | | | Originally it was thought it would be useful to split up the kmods by functionality. This would allow external consumers to only load what was needed. However, in practice we've never had a case where this functionality would be needed, and conversely managing multiple kmods can be awkward. Therefore, this change merges all but the spl.ko kmod in to a single zfs.ko kmod. Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13274
* Improve the inline descriptions of the ARC module parametersAllan Jude2022-04-201-10/+10
| | | | | | | | These are displayed as the descriptions of the sysctl's on FreeBSD Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Allan Jude <[email protected]> Closes #13334
* Fix style checking error introduced by zfs-mount changesPaul Dagnelie2022-04-191-3/+1
| | | | | | | Reviewed-by: Matthew Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Nguyen <[email protected]> Signed-off-by: Paul Dagnelie <[email protected]> Closes #13347
* Linux 5.17 compat: GENHD_FL_EXT_DEVT / GENHD_FL_NO_PART_SCANBrian Behlendorf2022-04-193-13/+68
| | | | | | | | | | | | | | As of the 5.17 kernel the GENHD_FL_EXT_DEVT flag has been removed and the GENHD_FL_NO_PART_SCAN flag renamed GENHD_FL_NO_PART. Update zvol_alloc() to set GENHD_FL_NO_PART for the newer kernels which is sufficient. The behavior for prior kernels remains unchanged. 1ebe2e5f ("block: remove GENHD_FL_EXT_DEVT") 46e7eac6 ("block: rename GENHD_FL_NO_PART_SCAN to GENHD_FL_NO_PART") Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13294 Closes #13297
* init.d/zfs-mount: Don't fsck or mount/umount fstab entriesomni2022-04-151-82/+0
| | | | | | | | | This is better handled by existing OS toolset. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Damian Szuberski <[email protected]> Signed-off-by: omni <[email protected]> Issue #7374 Closes #12780
* Fix 'zpool history' sometimes fails without reporting any errorLow-power2022-04-151-9/+11
| | | | | | | | | | | | | | | | The corresponding function 'zpool_get_history' in libzfs would printing an error messages only when the ioctl call failed. Add missing error reporting, specifically memory allocation failures and error from 'zpool_history_unpack'. Also avoid possibly reading of uninitialized 'err' variable in case the requested offset pasts EOF. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Damian Szuberski <[email protected]> Signed-off-by: WHR <[email protected]> Issue #13322 Closes #13320
* libzfs: import: zpool_clear_label: bool for boolean statusнаб2022-04-131-9/+5
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13304
* libzfs: import: zpool_clear_label: don't allocate another time for L2ARC headerнаб2022-04-131-11/+7
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13304
* libzfs: import: zpool_clear_label: actually fail if clearing l2arc header failsнаб2022-04-131-10/+10
| | | | | | | | Found with -Wunused-but-set-variable on Clang trunk Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13304
* module: zfs: vdev_removal: remove unused num_indirectнаб2022-04-131-3/+0
| | | | | | | | | Found with -Wunused-but-set-variable on Clang trunk Fixes: a1d477c24c ("OpenZFS 7614, 9064 - zfs device evacuation/removal") Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13304
* tests: cmd: draid: remove unused and undocumented -vнаб2022-04-131-5/+2
| | | | | | | | Found with -Wunused-but-set-variable on Clang trunk Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13304
* zfs(8): remove errors reported by mandoc 1.14.6szubersk2022-04-131-3/+4
| | | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ahelenia Ziemiańska <[email protected]> Signed-off-by: szubersk <[email protected]> Closes #13315
* zfs(8): add requirements towards fs namesszubersk2022-04-131-6/+25
| | | | | | | | | | | Provide explicit requirements towards file system naming convention in OpenZFS man pages. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ahelenia Ziemiańska <[email protected]> Signed-off-by: szubersk <[email protected]> Mitigates #13310 Closes #13315
* ZTS: Retry auto_spare_multiple.kshBrian Behlendorf2022-04-131-0/+1
| | | | | | | | | | The auto_spare_multiple.ksh test may incorrectly fail for a similar reason as the auto_spare_shared.ksh test. Add it to known list of exceptions which should be retried to prevent failures in the CI. Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13318
* ZTS: Retry redundancy_draid_spare[1,3].kshBrian Behlendorf2022-04-132-5/+2
| | | | | | | | | | | | The redundancy_draid_spare1.ksh and redundancy_draid_spare3.ksh test cases are a little to strict for the sequential resilver case. While unlikely it is possible that a handful of correctable checksum errors will be reported resulting in a test failure. Update the zts-report.py script to allow this the test case to be retried if requested. Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13318
* FreeBSD: Return Mach error codes from VOP_(GET|PUT)PAGESMark Johnston2022-04-131-8/+7
| | | | | | | | | | | FreeBSD's memory management system uses its own error numbers and gets confused when these VOPs return EIO. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reported-by: Peter Holm <[email protected]> Signed-off-by: Mark Johnston <[email protected]> Closes #13311
* FreeBSD: Parameterize ZFS_ENTER/ZFS_VERIFY_VP with an error codeMark Johnston2022-04-132-15/+19
| | | | | | | | | | | | For legacy reasons, a couple of VOPs have to return error numbers that don't come from the usual errno namespace. To handle the cases where ZFS_ENTER or ZFS_VERIFY_ZP fail, we need to be able to override the default error return value of EIO. Extend the macros to permit this. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Mark Johnston <[email protected]> Closes #13311
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-01 17:56:30 +0000