summaryrefslogtreecommitdiffstats
path: root/contrib
Commit message (Collapse)AuthorAgeFilesLines
* dracut: use /bin/sh instead of bash as the intepreterĐoàn Trần Công Danh2020-11-287-19/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Despite that dracut has a hard dependency on bash, its modules doesn't, dracut only has a hard dependency on bash for module-setup (on a fully usable machine). Inside initramfs, dracut allows users choose from a list of handful other shells, e.g. bash, busybox, dash, mkfsh. In fact, my local machine's initramfs is being built with dash, and it's functional for a very long time. Before 64025fa3a (Silence 'make checkbashisms', 2020-08-20), we also allows our users to have that right, too. Let's fix the problem 'make checkbashisms' reported and allows our users to have that right, again. For 'plymouth' case, let's simply run the command inside the if instead of checking for the existence of command before running it, because the status is also failture if plymouth is unavailable. While we're at it, let's remove an unnecessary fork for grep in zfs-generator.sh.in and its following complicated 'if elif fi' with a simple 'case ... esac'. To support this change, also exclude 90zfs from "make checkbashisms" because the current CI infrastructure ships an old version of "checkbashisms", which complains about "command -v", while the current latest "checkbashisms" thinks it's fine. In the near future, we can revert that change to "Makefile.am" when CI infrastructure is updated. Reviewed-by: Gabriel A. Devenyi <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Đoàn Trần Công Danh <[email protected]> Closes #11244
* pam_zfs_key: accommodate different dataset naming schemecragw2020-11-221-0/+54
| | | | | | | | | | | | | | | | Name of dataset for user home directory may vary from the expected $homes_prefix/$username, if different naming scheme is being used. We can use property mountpoint to specify the dataset for $username as long as its value is identical to passwd's pw_dir. For example: NAME PROPERTY VALUE rpool/home/myuser_123456 mountpoint /home/myuser Reviewed-by: Felix Dörre <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Crag Wang <[email protected]> Closes #11165
* dracut/mount-zfs.sh: quote expansion on zpool testнаб2020-11-191-1/+1
| | | | | | | | | Bring over some of the improvements from dracut/zfs-load-key.sh, shellcheck is slightly quieter as well Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #11198
* dracut/zfs-load-key.sh: simplify import loop, quote variable assignmentsнаб2020-11-191-7/+5
| | | | | | | | | | | | | | The loop now has a less confusing condition and properly uses systemctl(1) is-failed's return code instead of that entire mess The assignments could turn into "var=val program" if encryptionroot or keylocation had whitespace in them As a bonus, this (mostly) silences shellcheck Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #11198
* config/dracut/90zfs: handle cases where hostid(1) returns all zerosÉrico Rolim2020-11-141-1/+7
| | | | | | | | | | | | | | | | | | | | | On systems with musl libc, hostid(1) always prints "00000000", which will cause improper behavior when the 90zfs module is configured in a dracut initramfs. Work around this by copying the host /etc/hostid if the file exists, and otherwise only write /etc/hostid if hostid(1) returns something meaningful. This avoids zgenhostid creating a random /etc/hostid for the initramfs, which could lead to errors when trying to import the pool if spl_hostid isn't defined in the kernel command line. Furthermore, tag the /etc/hostid file as hostonly, since it is system specific and shouldn't be taken into account when trying to use an initramfs generated in one system to boot into a different system. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Georgy Yakovlev <[email protected]> Co-authored-by: Andrew J. Hesford <[email protected]> Signed-off-by: Érico Rolim <[email protected]> Closes #11174 Closes #11189
* initramfs: zfsunlock hook breaks /usr/binPavel Zakharov2020-11-101-1/+1
| | | | | | | | | The copy_exec() function expects that the full path of the target file is passed rather than just the directory, and will take care of creating the underlying directories if they don't exist. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Pavel Zakharov <[email protected]> Closes #11162
* Replace ZFS on Linux references with OpenZFSBrian Behlendorf2020-10-083-3/+4
| | | | | | | | | | | | | This change updates the documentation to refer to the project as OpenZFS instead ZFS on Linux. Web links have been updated to refer to https://github.com/openzfs/zfs. The extraneous zfsonlinux.org web links in the ZED and SPL sources have been dropped. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #11007
* Add intel_QAT patchesKjeld Schouten-Lebbing2020-09-305-0/+129
| | | | | | | | | | | Add community compatibility patches for Intel QAT Due to incompatibility with higher kernel versions. Also includes basic instructions. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Kjeld Schouten-Lebbing <[email protected]> Closes #10961 Closes #10962
* contrib/initramfs: fix shellcheck and checkbashisms errors with shebangнаб2020-09-221-87/+80
| | | | | | | Reviewed-by: Gabriel A. Devenyi <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #10908 Closes #10917
* cmd/zgenhostid: replace with simple c implementationGeorgy Yakovlev2020-09-162-11/+4
| | | | | | | | | | | | | | | | | | It was discovered that dracut scripts and zgenhostid always generate little-endian /etc/hostid. This commit provides simple endianess-aware binary and updates the scripts to use it. New features include: -f flag to force overwrite. -o flag to write to different file (for dracut) accepting both 0x01234567 and 01234567 values as input Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Olaf Faaland <[email protected]> Signed-off-by: Georgy Yakovlev <[email protected]> Closes #10887 Closes #10925
* Remove vestigial settings related to initramfsChris McDonough2020-08-221-3/+1
| | | | | | | | | | | | Remove ZFS_POOL_IMPORT, ZFS_INITRD_PRE_MOUNTROOT_SLEEP, ZFS_INITRD_POST_MODPROBE_SLEEP, and ZFS_INITRD_ADDITIONAL_DATASETS features from etc/defaults/zfs.in. These features no longer work. Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Chris McDonough <[email protected]> Closes #9126 Closes #10757
* Silence 'make checkbashisms'Brian Behlendorf2020-08-207-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit d2bce6d03 added the 'make checkbashisms' target but did not resolve all of the bashisms in the scripts. This commit doesn't resolve them all either but it does fix up a few, and it excludes the others so 'make checkstyle' no longer prints warnings. It's a small step in the right direction. * Dracut is Linux specific and itself depends on bash. Therefore all dracut support scripts can be bash specific, update their shebang accordingly. * zed-functions.sh, zfs-import, zfs-mount, zfs-zed, smart paxcheck.sh, make_gitrev.sh - these scripts were excuded from the check until they can be updated and properly tested. * zfsunlock - only whole values for sleep are allowed. * vdev_id - removed unneeded locals; use && instead of -a. * dkms.mkconf, dkms.postbuil - use || instead of -o. Reviewed-by: InsanePrawn <[email protected]> Reviewed-by: Gabriel A. Devenyi <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10755
* Change the error handling for invalid property valuesAllan Jude2020-08-012-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | ZFS recv should return a useful error message when an invalid index property value is provided in the send stream properties nvlist With a compression= property outside of the understood range: Before: ``` receiving full stream of zof/zstd_send@send2 into testpool/recv@send2 internal error: Invalid argument Aborted (core dumped) ``` Note: the recv completes successfully, the abort() is likely just to make it easier to track the unexpected error code. After: ``` receiving full stream of zof/zstd_send@send2 into testpool/recv@send2 cannot receive compression property on testpool/recv: invalid property value received 28.9M stream in 1 seconds (28.9M/sec) ``` Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Allan Jude <[email protected]> Closes #10631
* pyzfs: Add missing entry to zfs_errnoAllan Jude2020-07-311-0/+1
| | | | | | | This was causing all later errno's to have the incorrect value. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Allan Jude <[email protected]> Closes #10649
* Centralize variable substitutionArvind Sankar2020-07-144-53/+10
| | | | | | | | | | | | A bunch of places need to edit files to incorporate the configured paths i.e. bindir, sbindir etc. Move this logic into a common file. Create arc_summary by copying arc_summary[23] as appropriate at build time instead of install time. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Arvind Sankar <[email protected]> Closes #10559
* Use abs_top_builddir when referencing librariesArvind Sankar2020-07-101-4/+4
| | | | | | | | | | | | | | | | | | | | | libtool stores absolute paths in the dependency_libs component of the .la files. If the Makefile for a dependent library refers to the libraries by relative path, some libraries end up duplicated on the link command line. As an example, libzfs specifies libzfs_core, libnvpair and libuutil as dependencies to be linked in. The .la file for libzfs_core also specifies libnvpair, but using an absolute path, with the result that libnvpair is present twice in the linker command line for producing libzfs. While the only thing this causes is to slightly slow down the linking, we can avoid it by using absolute paths everywhere, including for convenience libraries just for consistency. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Arvind Sankar <[email protected]> Closes #10538
* Clean up lib dependenciesArvind Sankar2020-07-101-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | libzutil is currently statically linked into libzfs, libzfs_core and libzpool. Avoid the unnecessary duplication by removing it from libzfs and libzpool, and adding libzfs_core to libzpool. Remove a few unnecessary dependencies: - libuutil from libzfs_core - libtirpc from libspl - keep only libcrypto in libzfs, as we don't use any functions from libssl - librt is only used for clock_gettime, however on modern systems that's in libc rather than librt. Add a configure check to see if we actually need librt - libdl from raidz_test Add a few missing dependencies: - zlib to libefi and libzfs - libuuid to zpool, and libuuid and libudev to zed - libnvpair uses assertions, so add assert.c to provide aok and libspl_assertf Sort the LDADD for programs so that libraries that satisfy dependencies come at the end rather than the beginning of the linker command line. Revamp the configure tests for libaries to use FIND_SYSTEM_LIBRARY instead. This can take advantage of pkg-config, and it also avoids polluting LIBS. List all the required dependencies in the pkgconfig files, and move the one for libzfs_core into the latter's directory. Install pkgconfig files in $(libdir)/pkgconfig on linux and $(prefix)/libdata/pkgconfig on FreeBSD, instead of /usr/share/pkgconfig, as the more correct location for library .pc files. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Arvind Sankar <[email protected]> Closes #10538
* Add device rebuild featureBrian Behlendorf2020-07-031-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The device_rebuild feature enables sequential reconstruction when resilvering. Mirror vdevs can be rebuilt in LBA order which may more quickly restore redundancy depending on the pools average block size, overall fragmentation and the performance characteristics of the devices. However, block checksums cannot be verified as part of the rebuild thus a scrub is automatically started after the sequential resilver completes. The new '-s' option has been added to the `zpool attach` and `zpool replace` command to request sequential reconstruction instead of healing reconstruction when resilvering. zpool attach -s <pool> <existing vdev> <new vdev> zpool replace -s <pool> <old vdev> <new vdev> The `zpool status` output has been updated to report the progress of sequential resilvering in the same way as healing resilvering. The one notable difference is that multiple sequential resilvers may be in progress as long as they're operating on different top-level vdevs. The `zpool wait -t resilver` command was extended to wait on sequential resilvers. From this perspective they are no different than healing resilvers. Sequential resilvers cannot be supported for RAIDZ, but are compatible with the dRAID feature being developed. As part of this change the resilver_restart_* tests were moved in to the functional/replacement directory. Additionally, the replacement tests were renamed and extended to verify both resilvering and rebuilding. Original-patch-by: Isaac Huang <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: John Poduska <[email protected]> Co-authored-by: Mark Maybee <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10349
* Fixes for make distArvind Sankar2020-06-263-12/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reduce the usage of EXTRA_DIST. If files are conditionally included in _SOURCES, _HEADERS etc, automake is smart enough to dist all files that could possibly be included, but this does not apply to EXTRA_DIST, resulting in make dist depending on the configuration. Add some files that were missing altogether in various Makefile's. The changes to disted files in this commit (excluding deleted files): +./cmd/zed/agents/README.md +./etc/init.d/README.md +./lib/libspl/os/freebsd/getexecname.c +./lib/libspl/os/freebsd/gethostid.c +./lib/libspl/os/freebsd/getmntany.c +./lib/libspl/os/freebsd/mnttab.c -./lib/libzfs/libzfs_core.pc -./lib/libzfs/libzfs.pc +./lib/libzfs/os/freebsd/libzfs_compat.c +./lib/libzfs/os/freebsd/libzfs_fsshare.c +./lib/libzfs/os/freebsd/libzfs_ioctl_compat.c +./lib/libzfs/os/freebsd/libzfs_zmount.c +./lib/libzutil/os/freebsd/zutil_compat.c +./lib/libzutil/os/freebsd/zutil_device_path_os.c +./lib/libzutil/os/freebsd/zutil_import_os.c +./module/lua/README.zfs +./module/os/linux/spl/README.md +./tests/README.md +./tests/zfs-tests/tests/functional/cli_root/zfs_clone/zfs_clone_rm_nested.ksh +./tests/zfs-tests/tests/functional/cli_root/zfs_send/zfs_send_encrypted_unloaded.ksh +./tests/zfs-tests/tests/functional/inheritance/README.config +./tests/zfs-tests/tests/functional/inheritance/README.state +./tests/zfs-tests/tests/functional/rsend/rsend_016_neg.ksh +./tests/zfs-tests/tests/perf/fio/sequential_readwrite.fio Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Arvind Sankar <[email protected]> Closes #10501
* pam: implement a zfs_key pam modulefelixdoerre2020-06-244-1/+776
| | | | | | | | | | | | | | | | | Implements a pam module for automatically loading zfs encryption keys for home datasets. The pam module: - loads a zfs key and mounts the dataset when a session opens. - unmounts the dataset and unloads the key when the session closes. - when the user is logged on and changes the password, the module changes the encryption key. Reviewed-by: Richard Laager <[email protected]> Reviewed-by: @jengelh <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Felix Dörre <[email protected]> Closes #9886 Closes #9903
* Drop unnecessary srcdir pathsArvind Sankar2020-06-245-17/+17
| | | | | | | | | | There's no need to specify the srcdir explicitly in _HEADERS and EXTRA_DIST. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Arvind Sankar <[email protected]> Closes #10493
* Merge bash_completions changes from upstreamJoao Carlos Mendes Luis2020-06-161-24/+114
| | | | | | | | | | | | The current bash_completion contrib code in openzfs is very old, and some changes have been added since. The original repo is at https://github.com/Aneurin/zfs-bash I've been using the original @Aneurin code since my first deploy of ZoL. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: João Carlos Mendes Luís <[email protected]> Closes #10456
* bash_completion: add missing attributesGrischa Zengel2020-06-101-3/+3
| | | | | | | | | | | | | | There a some attributes missing which are shown in man pages: zfs list -t type A comma-separated list of types to display, where type is one of filesystem, snapshot, volume, *bookmark*, or all. For example, specifying -t snapshot displays only snapshots. zfs get -s source A comma-separated list of sources to display. Those properties coming from a source other than those in this list are ignored. Each source must be one of the following: local, default, inherited, temporary, *received*, and none. The default value is all sources. zfs get -t type A comma-separated list of types to display, where type is one of filesystem, snapshot, volume, bookmark, or all. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Grischa Zengel <[email protected]> Closes #10418
* Remove unnecessary references to slaveryMatthew Ahrens2020-06-101-13/+13
| | | | | | | | | | | | | | | | | | | | | | The horrible effects of human slavery continue to impact society. The casual use of the term "slave" in computer software is an unnecessary reference to a painful human experience. This commit removes all possible references to the term "slave". Implementation notes: The zpool.d/slaves script is renamed to dm-deps, which uses the same terminology as `dmsetup deps`. References to the `/sys/class/block/$dev/slaves` directory remain. This directory name is determined by the Linux kernel. Although `dmsetup deps` provides the same information, it unfortunately requires elevated privileges, whereas the `/sys/...` directory is world-readable. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #10435
* Add bootfs.snapshot and bootfs.rollback kernel parametersgregory-lee-bartholomew2020-05-296-2/+66
| | | | | | | | | | | | Unlike other filesystems, snapshots and rollbacks of bootfs need to be done from a rescue environment. This patch makes it possible to snap- shot or rollback the bootfs simply by specifying bootfs.snapshot or bootfs.rollback on the kernel command line. The operation will be performed by dracut just before bootfs is mounted. Reviewed-by: Antonio Russo <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Gregory Bartholomew <[email protected]> Closes #10198
* Change zfsunlock for better busybox compatibilityRichard Laager2020-05-101-1/+1
| | | | | | | | | | | | | | It turns out that there are two versions of Busybox, at least on Ubuntu 18.04. If you have the busybox-static package installed, you get a busybox that supports `ps a` and `head`. If you only have busybox-initramfs, you don't. Either way, you have `awk`. This change should also make this compatible with GNU ps, if you somehow end up with that in the initramfs environment. Reviewed-by: Tom Caputi <[email protected]> Reviewed-by: Andrey Prokopenko <[email protected]> Signed-off-by: Richard Laager <[email protected]> Closes #10307
* Unlock encrypted root partition over SSHAndrey Prokopenko2020-05-078-3/+92
| | | | | | | | | | | | | This commit add a new feature for Debian-based distributions to unlock encrypted root partition over SSH. This feature is very handy on headless NAS or VPS cloud servers. To use this feature, you will need to install the dropbear-initramfs package. Reviewed-By: Brian Behlendorf <[email protected]> Reviewed-By: Tom Caputi <[email protected]> Signed-off-by: Andrey Prokopenko <[email protected]> Signed-off-by: Richard Laager <[email protected]> Closes #10027
* Rework README.initramfs.markdownRichard Laager2020-05-071-94/+74
| | | | | | | | | | | | | This file is listed as being in Markdown format, but it didn't really use much Markdown. I have added a fair amount of formatting. I have reordered and reworded things to improve the flow of the text. Reviewed-By: Andrey Prokopenko <[email protected]> Reviewed-By: Brian Behlendorf <[email protected]> Reviewed-By: Tom Caputi <[email protected]> Signed-off-by: Richard Laager <[email protected]> Closes #10027
* Cleanup contrib/initramfs automakeRichard Laager2020-05-075-27/+14
| | | | | | | | | | | | | | | | | | | | | The initramfs hook scripts depend on Makefile. This way, if the substitution code is changed, they should update. This brings it in line with etc/init.d (which was modified to match the example in the automake docs). The initramfs hook script cleaning now matches etc/init.d. There was a mix of SUBDIRS recursion and custom install rules for files in subdirectories. This was duplicated for the "hooks" and "scripts" subdirectories. Now everything uses SUBDIRS. I fixed the substitution of DEFAULT_INITCONF_DIR for hooks/zfs. Reviewed-By: Andrey Prokopenko <[email protected]> Reviewed-By: Brian Behlendorf <[email protected]> Reviewed-By: Tom Caputi <[email protected]> Signed-off-by: Richard Laager <[email protected]> Closes #10027
* Add FreeBSD support to OpenZFSMatthew Macy2020-04-141-1/+4
| | | | | | | | | | | | | | | | | | Add the FreeBSD platform code to the OpenZFS repository. As of this commit the source can be compiled and tested on FreeBSD 11 and 12. Subsequent commits are now required to compile on FreeBSD and Linux. Additionally, they must pass the ZFS Test Suite on FreeBSD which is being run by the CI. As of this commit 1230 tests pass on FreeBSD and there are no unexpected failures. Reviewed-by: Sean Eric Fagan <[email protected]> Reviewed-by: Jorgen Lundman <[email protected]> Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Co-authored-by: Ryan Moeller <[email protected]> Signed-off-by: Matt Macy <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #898 Closes #8987
* Fix cstyle warningsBrian Behlendorf2020-03-171-1/+1
| | | | | | | Fix minor cstyle warnings accidentally introduced by 7145123b. Reviewed-by: Paul Dagnelie <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10143
* Separate warning for incomplete and corrupt streamsPaul Dagnelie2020-03-174-3/+12
| | | | | | | | | | This change adds a separate return code to zfs_ioc_recv that is used for incomplete streams, in addition to the existing return code for streams that contain corruption. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Matthew Ahrens <[email protected]> Signed-off-by: Paul Dagnelie <[email protected]> Closes #10122
* Fix zfs-functions packaging bugRichard Laager2020-03-101-11/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This fixes a bug where the generated zfs-functions was being included along with original zfs-functions.in in the make dist tarball. This caused an unfortunate series of events during build/packaging that resulted in the RPM-installed /etc/zfs/zfs-functions listing the paths as: ZFS="/usr/local/sbin/zfs" ZED="/usr/local/sbin/zed" ZPOOL="/usr/local/sbin/zpool" When they should have been: ZFS="/sbin/zfs" ZED="/sbin/zed" ZPOOL="/sbin/zpool" This affects init.d (non-systemd) distros like CentOS 6. /etc/default/zfs and /etc/zfs/zfs-functions are also used by the initramfs, so they need to be built even when init.d support is not. They have been moved to the (new) etc/default and (existing) etc/zfs source directories, respectively. Fixes: #9443 Co-authored-by: Tony Hutter <[email protected]> Signed-off-by: Richard Laager <[email protected]>
* initramfs: Eliminate substitutionsRichard Laager2020-03-102-26/+2
| | | | | | | These are now handled in zfs-functions, so this is all duplicative and unnecessary. Signed-off-by: Richard Laager <[email protected]>
* pyzfs: Add constants for platform-specific errnosRyan Moeller2020-02-273-6/+25
| | | | | | | | | FreeBSD doesn't have EBADE, ECHRNG, or ETIME. Add constants for these and set them appropriately for the platform. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10061
* Implement bookmark copyingChristian Schwarz2020-02-115-25/+113
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This feature allows copying existing bookmarks using zfs bookmark fs#target fs#newbookmark There are some niche use cases for such functionality, e.g. when using bookmarks as markers for replication progress. Copying redaction bookmarks produces a normal bookmark that cannot be used for redacted send (we are not duplicating the redaction object). ZCP support for bookmarking (both creation and copying) will be implemented in a separate patch based on this work. Overview: - Terminology: - source = existing snapshot or bookmark - new/bmark = new bookmark - Implement bookmark copying in `dsl_bookmark.c` - create new bookmark node - copy source's `zbn_phys` to new's `zbn_phys` - zero-out redaction object id in copy - Extend existing bookmark ioctl nvlist schema to accept bookmarks as sources - => `dsl_bookmark_create_nvl_validate` is authoritative - use `dsl_dataset_is_before` check for both snapshot and bookmark sources - Adjust CLI - refactor shortname expansion logic in `zfs_do_bookmark` - Update man pages - warn about redaction bookmark handling - Add test cases - CLI - pyyzfs libzfs_core bindings Reviewed-by: Matt Ahrens <[email protected]> Reviewed-by: Paul Dagnelie <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Christian Schwarz <[email protected]> Closes #9571
* bash scripts: use /usr/bin/env for bash shebangsGraham Christensen2020-02-103-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Not all systems / distros have a `/bin/bash`, and these scripts are more difficult to run at development time. For example, my system is NixOS which doesn't have a /bin/bash. This is not a problem for NixOS building ZFS as a package: the build environment automatically replaces these shebangs with corrected paths. The problem is much more annoying at development time: either the scripts don't run, or I correct them for my local machine and deal with a perpetually dirty work tree. Before committing this patch I confirmed there are existing scripts which use `/usr/bin/env` to locate bash, so I am thinking this is a safe transformation. There are a handful of other shebangs in this repository which don't work on my system. This patch is useful on its own specifically for `commitcheck.sh`, otherwise I can't validate my commits before submission. Here are the remaining shebangs which NixOS systems won't have: 1274 #!/bin/ksh -p 91 #!/bin/ksh 89 #! /bin/ksh -p 2 #!/bin/sed -f 1 #!/usr/bin/perl -w 1 #!/usr/bin/ksh 1 #!/bin/nawk -f plus this which will create an invalid shebang in `tests/zfs-tests/tests/functional/mv_files/mv_files_common.kshlib`: echo "#!/bin/ksh" > $TEST_BASE_DIR/exitsZero.ksh I chose to leave those alone for now, and gauge the interest in this much smaller patch first. The fixes for these are easy enough by simply using `/usr/bin/env ksh`: 91 #!/bin/ksh 1 #!/usr/bin/ksh The fix for the other set is much trickier. Quoting the GNU coreutils manual: Most operating systems (e.g. GNU/Linux, BSDs) treat all text after the first space as a single argument. When using env in a script it is thus not possible to specify multiple arguments. and not all `env`'s support arguments. Mine (GNU Coreutils 8.31) does, though this feature is new since April 2018, GNU Coreutils 8.30: https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=668306ed86c8c79b0af0db8b9c882654ebb66db2 and worse, requires the -S argument: -S, --split-string=S process and split S into separate arguments; used to pass multiple arguments on shebang lines Example: $ seq 1 2 | $(nix-build '<nixpkgs>' -A coreutils)/bin/env "sort -nr" /nix/[...]-coreutils-8.31/bin/env: ‘sort -nr’: No such file or directory /nix/[...]-coreutils-8.31/bin/env: use -[v]S to pass options in shebang lines $ seq 1 2 | $(nix-build '<nixpkgs>' -A coreutils)/bin/env "-S sort -nr" 2 1 GNU Coreutils says FreeBSD's `env` does, though I wonder if FreeBSD's would be unhappy with the `-S`: https://www.gnu.org/software/coreutils/manual/html_node/env-invocation.html#env-invocation BusyBox v1.30.1 does not, and does not have a `-S`-like option: $ seq 1 2 | $(nix-build '<nixpkgs>' -A busybox)/bin/env "sort -nr" env: can't execute 'sort -nr': No such file or directory Toybox 0.8.1 also does not, and also does not have a `-S` option: $ seq 1 2 | $(nix-build '<nixpkgs>' -A toybox)/bin/env "sort -nr" env: exec sort -nr: No such file or directory --- At any rate, if this patch merges and the remaining ~1,500 are updated, the much larger patch should probably include a checkstyle-like test asserting all new shebangs use `/usr/bin/env`. I also don't mind dealing with NixOS weirdness if the project would prefer that. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Graham Christensen <[email protected]> Closes #9893
* zfs-load-key.sh: ${ZFS} is not the zfs binaryBen Cordero2019-12-291-1/+1
| | | | | | | | | | | | | | | | A change[1] was merged yesterday that should refer to the zfs binary in the initramfs, but is actually an unset shell variable. This commit changes this line to call `zfs` directly like the surrounding code. [1]: cb5b875b273235a4a3ed28e16f416d5bb8865166 Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Garrett Fields <[email protected]> Reviewed-by: Richard Laager <[email protected]> Signed-off-by: Ben Cordero <[email protected]> Closes #9780
* In initramfs, do not prompt if keylocation is "file://"sam-lunt2019-12-262-7/+20
| | | | | | | | | | | | If the encryption key is stored in a file, the initramfs should not prompt for the password. For example, this could be the case if the boot partition is stored on removable media that is only present at boot time Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Garrett Fields <[email protected]> Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Kjeld Schouten <[email protected]> Signed-off-by: Sam Lunt <[email protected]> Closes #9764
* Exchanged two "${ZFS} get -H -o value" commandsGarrett Fields2019-12-181-2/+2
| | | | | | | | | Initramfs uses "get_fs_value()" elsewhere. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Kjeld Schouten-Lebbing <[email protected]> Signed-off-by: Garrett Fields <[email protected]> Closes #9736
* Force systems with kernel option "quiet" to display prompt for passwordGarrett Fields2019-12-171-0/+4
| | | | | | | | | | | | | | | | | | On systems that utilize TTY for password entry, if the kernel option "quiet" is set, the system would appear to freeze on a blank screen, when in fact it is waiting for password entry from the user. Since TTY is the fallback method, this has no effect on systemd or plymouth password prompting. By temporarily setting "printk" to "7", running the command, then resuming with the original "printk" state, the user can see the password prompt. Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Garrett Fields <[email protected]> Closes #9731
* initramfs: setup keymapping and video for promptsRichard Laager2019-12-161-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | From Steve Langasek <[email protected]>: > The poorly-named 'FRAMEBUFFER' option in initramfs-tools controls > whether the console_setup and plymouth scripts are included and used > in the initramfs. These are required for any initramfs which will be > prompting for user input: console_setup because without it the user's > configured keymap will not be set up, and plymouth because you are > not guaranteed to have working video output in the initramfs without > it (e.g. some nvidia+UEFI configurations with the default GRUB > behavior). > The zfs initramfs script may need to prompt the user for passphrases > for encrypted zfs datasets, and we don't know definitively whether > this is the case or not at the time the initramfs is constructed (and > it's difficult to dynamically populate initramfs config variables > anyway), therefore the zfs-initramfs package should just set > FRAMEBUFFER=yes in a conf snippet the same way that the > cryptsetup-initramfs package does > (/usr/share/initramfs-tools/conf-hooks.d/cryptsetup). https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1856408 Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Kjeld Schouten <[email protected]> Signed-off-by: Steve Langasek <[email protected]> Signed-off-by: Richard Laager <[email protected]> Closes #9723
* Skip loading already loaded keyWitaut Bajaryn2019-11-083-5/+15
| | | | | | | | | | | | | | | Don't ask for the password / try to load the key if the key for the encryptionroot is already loaded. The user might have loaded the key manually or by other means before the scripts get called. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tom Caputi <[email protected]> Reviewed-by: Richard Laager <[email protected]> Signed-off-by: Witaut Bajaryn <[email protected]> Closes #9495 Closes #9529
* Add tracepoints for taskq entry lifetime eventsPrakash Surya2019-11-014-2/+69
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds some new DTRACE_PROBE* endpoints so that we can observe taskq latencies on a system. Additionally, a new "taskqlatency.bt" script is added to do this observation via "bpftrace". Lastly, a "zfs-trace.sh" script is added to wrap "bpftrace" with the proper options required to run and use "taskqlatency.bt". For example, with these changes in place, a user can run the following: $ cd ./contrib/bpftrace $ sudo ./zfs-trace.sh taskqlatency.bt Attaching 6 probes... ^C Here's some example output, showing latency information for time spent executing the taskq entry's function: @exec_lat_us[dp_sync_taskq, userquota_updates_task]: [2, 4) 5 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [4, 8) 0 | | [8, 16) 1 |@@@@@@@@@@ | [16, 32) 2 |@@@@@@@@@@@@@@@@@@@@ | @exec_lat_us[z_wr_int_h, zio_execute]: [8, 16) 16 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [16, 32) 2 |@@@@@@ | @exec_lat_us[z_wr_iss_h, zio_execute]: [16, 32) 4 |@@@@@@@@@@@@@@@@ | [32, 64) 13 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [64, 128) 1 |@@@@ | @exec_lat_us[z_ioctl_int, zio_execute]: [2, 4) 1 |@@@@ | [4, 8) 11 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [8, 16) 8 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | @exec_lat_us[dp_sync_taskq, sync_dnodes_task]: [2, 4) 1 |@@@@@@ | [4, 8) 7 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [8, 16) 8 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [16, 32) 2 |@@@@@@@@@@@@@ | [32, 64) 4 |@@@@@@@@@@@@@@@@@@@@@@@@@@ | [64, 128) 1 |@@@@@@ | [128, 256) 0 | | [256, 512) 1 |@@@@@@ Here's some example output, showing latency information for time spent waiting on the taskq, prior to starting execution of entry's function: @queue_lat_us[dp_sync_taskq]: [2, 4) 1 |@@@@ | [4, 8) 7 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [8, 16) 2 |@@@@@@@@ | [16, 32) 3 |@@@@@@@@@@@@@ | [32, 64) 12 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [64, 128) 6 |@@@@@@@@@@@@@@@@@@@@@@@@@@ | [128, 256) 0 | | [256, 512) 1 |@@@@ | @queue_lat_us[z_wr_iss]: [4, 8) 4 |@@@@ | [8, 16) 13 |@@@@@@@@@@@@@@@ | [16, 32) 6 |@@@@@@@ | [32, 64) 2 |@@ | [64, 128) 12 |@@@@@@@@@@@@@@ | [128, 256) 15 |@@@@@@@@@@@@@@@@@@ | [256, 512) 33 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [512, 1K) 27 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [1K, 2K) 7 |@@@@@@@@ | [2K, 4K) 14 |@@@@@@@@@@@@@@@@ | [4K, 8K) 14 |@@@@@@@@@@@@@@@@ | [8K, 16K) 23 |@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [16K, 32K) 43 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| @queue_lat_us[z_wr_int]: [2, 4) 10 |@@@@@ | [4, 8) 71 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [8, 16) 88 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [16, 32) 50 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [32, 64) 65 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [64, 128) 43 |@@@@@@@@@@@@@@@@@@@@@@@@@ | [128, 256) 19 |@@@@@@@@@@@ | [256, 512) 3 |@ | [512, 1K) 1 | | Reviewed by: Brad Lewis <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Prakash Surya <[email protected]> Closes #9525
* dracut/zfs-load-key.sh: properly remove prefixesalaviss2019-10-301-1/+1
| | | | | | | | | | | Removes the 'ZFS=' prefix from $BOOTFS instead of $root. This makes sure that the 'zfs:' prefix remains stripped so that users with 'root=zfs:dataset' cmdline can have key loaded on boot again. Reviewed-by: Garrett Fields <[email protected]> Reviewed-by: Dacian Reece-Stremtan <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Hiếu Lê <[email protected]> Closes #9520
* Fix contrib/zcp/Makefile.amBrian Behlendorf2019-10-301-1/+1
| | | | | | | | | Remove the stray leading + from the Makefile. This was preventing the autosnap.lua channel program from being properly included by `make dist`. Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #9527
* Fix for zfs-dracut regressiondacianstremtan2019-10-011-6/+7
| | | | | | | | | | | | Line 31 and 32 overwrote the ${root} variable which broke mount-zfs.sh We have create a new variable for the dataset instead of overwriting the ${root} variable in zfs-load-key.sh${root} variable in zfs-load-key.sh Reviewed-by: Kash Pande <[email protected]> Reviewed-by: Garrett Fields <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Dacian Reece-Stremtan <[email protected]> Closes #8913 Closes #9379
* Fix typos in contrib/Andrea Gelmini2019-08-306-21/+21
| | | | | | | Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Andrea Gelmini <[email protected]> Closes #9235
* Fix Plymouth passphrase prompt in initramfs scriptRichard Allen2019-08-271-8/+8
| | | | | | | | | | | | | | Entering the ZFS encryption passphrase under Plymouth wasn't working because in the ZFS initrd script, Plymouth was calling zfs via "--command", which wasn't passing through the filesystem argument to zfs load-key properly (it was passing through the single quotes around the filesystem name intended to handle spaces literally, which zfs load-key couldn't understand). Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Garrett Fields <[email protected]> Signed-off-by: Richard Allen <[email protected]> Issue #9193 Closes #9202
* Restore :: in Makefile.amRyan Moeller2019-08-264-0/+7
| | | | | | | | | | | | The double-colon looked like a typo, but it's actually an obscure feature. Rules with :: may appear multiple times and are run independently of one another in the order they appear. The use of :: for distclean-local was conventional, not accidental. Add comments to indicate the intentional use of double-colon rules. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #9210
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-16 10:06:35 +0000