summaryrefslogtreecommitdiffstats
path: root/lib
Commit message (Collapse)AuthorAgeFilesLines
* Update references to nonexistent man pages in codeRyan Moeller2020-10-301-5/+5
| | | | | | | | Refer to the correct section or alternative for FreeBSD and Linux. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #11132
* Share zfs_fsync, zfs_read, zfs_write, et al between Linux and FreeBSDMatthew Macy2020-10-211-1/+0
| | | | | | | | | | The zfs_fsync, zfs_read, and zfs_write function are almost identical between Linux and FreeBSD. With a little refactoring they can be moved to the common code which is what is done by this commit. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #11078
* Cross-platform acltypeRyan Moeller2020-10-131-25/+41
| | | | | | | | | | | | | | | | | The acltype property is currently hidden on FreeBSD and does not reflect the NFSv4 style ZFS ACLs used on the platform. This makes it difficult to observe that a pool imported from FreeBSD on Linux has a different type of ACL that is being ignored, and vice versa. Add an nfsv4 acltype and expose the property on FreeBSD. Make the default acltype nfsv4 on FreeBSD. Setting acltype to an unhanded style is treated the same as setting it to off. The ACLs will not be removed, but they will be ignored. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10520
* FreeBSD: make adjustments for the standalone environmentWarner Losh2020-10-131-4/+0
| | | | | | | | | | | In FreeBSD, there are three compile environments that are supported: user land, the kernel and the bootloader / standalone. Adjust the headers to compile in the standalone environment. Limit kernel-only items from view when _STANDALONE is defined. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Warner Losh <[email protected]> Closes #10998
* FreeBSD: Improve libzfs_error_init messagesRyan Moeller2020-10-133-11/+17
| | | | | | | | | | | | | | | | | | It is a common mistake to have failed to autoload the module due to permission issues when running a ZFS command as a user. "Operation not permitted" is an unhelpfully vague error message. Use a thread-local message buffer to format a nicer error message. We can infer that loading the kernel module failed if the module is not loaded. This can be extended with heuristics for other errors in the future. While looking at this stuff, remove an unused thread-local message buffer found in libspl and remove some inaccurate verbiage from the comment on libzfs_load_module. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #11033
* Replace ZFS on Linux references with OpenZFSBrian Behlendorf2020-10-082-4/+4
| | | | | | | | | | | | | This change updates the documentation to refer to the project as OpenZFS instead ZFS on Linux. Web links have been updated to refer to https://github.com/openzfs/zfs. The extraneous zfsonlinux.org web links in the ZED and SPL sources have been dropped. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #11007
* libzfs_sendrecv: zfs_send: remove unused pipefd and tid variablesChristian Schwarz2020-10-081-18/+1
| | | | | | | | | | | fixup of 196bee4 On gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1-1), the code removed caused `-Wmaybe-uninitialized` errors. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Christian Schwarz <[email protected]> Closes #11021
* Fix EIO after resuming receive of new dataset over an existing oneAlan Somers2020-10-021-2/+4
| | | | | | | | | | | | | | | | | | | When resuming an interrupted ZFS send stream that creates a new dataset with the same name as an existing dataset, if the existing dataset is accessed after the failed receive, then after the subsequent successful receive it will return EIO. This happens because nothing mounts the new dataset, leaving the old, no longer valid dataset still mounted. This commit fixes zfs receive to always unmount and remount the destination, regardless of whether the stream is a new stream or a resumed stream. Sponsored by: Axcient Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Alan Somers <[email protected]> External-issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=249579 Closes #10995 Closes #10999
* zpool command complains about /etc/exports.dGeorge Wilson2020-09-252-43/+64
| | | | | | | | | | | | | | | If the /etc/exports.d directory does not exist, then we should only create it when we're performing an action which already requires root privileges. This commit moves the directory creation to the enable/disable code path which ensures that we have the appropriate privileges. Reviewed-by: Richard Elling <[email protected]> Reviewed-by: Matthew Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: George Wilson <[email protected]> Closes #10785 Closes #10934
* FreeBSD: Add support for procfs_listMatthew Macy2020-09-231-0/+1
| | | | | | | | | | The procfs_list interface is required by several kstats. Implement this functionality for FreeBSD to provide access to these kstats. Reviewed-by: Allan Jude <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10890
* libzfs: Don't leak buf if nvlist is too largeAllan Jude2020-09-182-0/+4
| | | | | | | | | | | | | Resolves FreeBSD Coverity defect: CID 1432398: Resource leaks (RESOURCE_LEAK) libzfs: don't leak hdl if there is an error reading env var Resolves FreeBSD Coverity defect: CID 1432395: Resource leaks (RESOURCE_LEAK) Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Allan Jude <[email protected]> Closes #10882
* pool may become suspended during device expansionGeorge Wilson2020-09-172-33/+129
| | | | | | | | | | | | | | | | | | | | When expanding a device zfs needs to rescan the partition table to get the correct size. This can only happen when we're in the kernel and requires the device to be closed. As part of the rescan, udev is notified and the device links are removed and recreated. This leave a window where the vdev code may try to reopen the device before udev has recreated the link. If that happens, then the pool may end up in a suspended state. To correct this, we leverage the BLKPG_RESIZE_PARTITION ioctl which allows the partition information to be modified even while it's in use. This ioctl also does not remove the device link associated with the zfs data partition so it eliminates the race condition that can occur in the kernel. Reviewed-by: Pavel Zakharov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: George Wilson <[email protected]> Closes #10897
* libzfsbootenv: lzbe_nvlist_set needs to store bootenv version VB_NVLISTToomas Soome2020-09-171-0/+18
| | | | | | | | A small bug did slip into initial libzfsbootenv; while storing nvlist in nvlist, we should make sure the bootenv is using VB_NVLIST format. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Toomas Soome <[email protected]> Closes #10937
* zfs label bootenv should store data as nvlistToomas Soome2020-09-1510-22/+598
| | | | | | | | | | | | | nvlist does allow us to support different data types and systems. To encapsulate user data to/from nvlist, the libzfsbootenv library is provided. Reviewed-by: Arvind Sankar <[email protected]> Reviewed-by: Allan Jude <[email protected]> Reviewed-by: Paul Dagnelie <[email protected]> Reviewed-by: Igor Kozhukhov <[email protected]> Signed-off-by: Toomas Soome <[email protected]> Closes #10774
* libzutil depends on libnvpairMatthew Ahrens2020-09-122-2/+7
| | | | | | | | | | | | libzutil depends on libnvpair, but this dependency is undeclared in the build system. Therefore it isn't possible to make a new command that depends on libzutil, but does not (directly) depend on libnvpair. This commit makes this dependency explicit. Reviewed-by: Brian Behlendorf <[email protected]> Reivewed-by: Ryan Moeller <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #10915
* Display pbkdf2iters property as plain numberFabio Buso2020-09-081-2/+3
| | | | | | | | | The pbkdf2iters property is an iteration counter and should be displayed as plain number rather than in binary unit. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Fabio Buso <[email protected]> Closes #10871
* libshare: Add missing headers for nfs.calaviss2020-09-041-0/+1
| | | | | | | | | | On musl libc, zfs failed to compile due to the missing <fcntl.h> include, which is required for `open()` per POSIX. This commit add the missing <fcntl.h> include. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Hiếu Lê <[email protected]> Closes #10880
* Spruce up pkg-config files for libzfs/libzfs_coreRyan Moeller2020-09-042-4/+4
| | | | | | | | | | | | Several of the listed library dependencies are not relevant on FreeBSD. Have ./configure save libraries that are found via pkg-config as ${LIB}_PC and use the configured automake variables instead of hard coded names so we only get what was actually needed. While here, update the URL to point at the OpenZFS Github repo. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10869
* Fixes for running FreeBSD buildworld on Linux/macOS hostsAlexander Richardson2020-09-031-0/+3
| | | | | | | | | | | Adding an #ifdef __FreeBSD__ to a FreeBSD-specific header may seem odd, but these headers are used on non-FreeBSD systems during the bootstrap tools phase. Originally submitted downstream as https://reviews.freebsd.org/D26193 Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Alex Richardson <[email protected]> Closes #10863
* Fix -Werror,-Wmacro-redefined in limits.hAlexander Richardson2020-09-011-0/+5
| | | | | | | | | | | Those macros are also defined by the compiler-provided float.h which will be included later on (at least in the FreeBSD buildworld case) and triggers these -Werror warnings. Including <float.h> first and only defining the macros when DBL_DIG/FLT_DIG is missing fixes this problem. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Alex Richardson <[email protected]> Closes #10864
* Add 'zfs rename -u' to rename without remountingRyan Moeller2020-09-013-11/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | Allow to rename file systems without remounting if it is possible. It is possible for file systems with 'mountpoint' property set to 'legacy' or 'none' - we don't have to change mount directory for them. Currently such file systems are unmounted on rename and not even mounted back. This introduces layering violation, as we need to update 'f_mntfromname' field in statfs structure related to mountpoint (for the dataset we are renaming and all its children). In my opinion it is worth it, as it allow to update FreeBSD in even cleaner way - in ZFS-only configuration root file system is ZFS file system with 'mountpoint' property set to 'legacy'. If root dataset is named system/rootfs, we can snapshot it (system/rootfs@upgrade), clone it (system/oldrootfs), update FreeBSD and if it doesn't boot we can boot back from system/oldrootfs and rename it back to system/rootfs while it is mounted as /. Before it was not possible, because unmounting / was not possible. Authored by: Pawel Jakub Dawidek <[email protected]> Reviewed-by: Allan Jude <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported by: Matt Macy <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10839
* libspl: Provide platform-specific zone implementationsRyan Moeller2020-08-314-43/+50
| | | | | | | | | | | | | | | | | FreeBSD has the concept of jails, a precursor to Solaris's zones, which can be mapped to the required zones interface with relative ease. The previous ZFS implementation in FreeBSD did so, and we should continue to provide an appropriate implementation in OpenZFS as well. Move lib/libspl/zone.c into platform code and adopt the correct implementation for FreeBSD. While here, prune unused code. Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10851
* Fix definition of BLKGETSIZE64 on FreeBSDAlexander Richardson2020-08-271-5/+1
| | | | | | | | | The matching ioctl is DIOCGMEDIASIZE. Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Alex Richardson <[email protected]> Closes #10818
* zpool: Change base URL for ZFS messages to openzfs-docsRyan Moeller2020-08-261-1/+2
| | | | | | | Reviewed-by: George Melikov <[email protected]> Reviewed-by: Kjeld Schouten <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10820
* Don't assert on nvlists larger than SPA_MAXBLOCKSIZEAllan Jude2020-08-252-2/+43
| | | | | | | | | | | | | | | | | | | | | | | | | Originally we asserted that all reads are less than SPA_MAXBLOCKSIZE However, nvlists are not ZFS records, and are not limited to SPA_MAXBLOCKSIZE. Add a new environment variable, ZFS_SENDRECV_MAX_NVLIST, to allow the user to specify the maximum size of the nvlist that can be sent or received. Default value: 4 * SPA_MAXBLOCKSIZE (64 MB) Modify libzfs send routines to return a useful error if the send stream will generate an nvlist that is beyond the maximum size. Modify libzfs recv routines to add an explicit error message if the nvlist is too large, rather than abort()ing. Move the change the assert() to only trigger on data records Reviewed-by: Paul Dagnelie <[email protected]> Reviewed-by: Kjeld Schouten <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Matthew Ahrens <[email protected]> Signed-off-by: Allan Jude <[email protected]> Closes #9616
* Avoid symbol collision with in-kernel zstdlibSebastian Gottschall2020-08-241-0/+6
| | | | | | | | | | | For Linux, when zfs is compiled as an in kernel static variant and the in kernel zstd library is compiled statically into the kernel a symbol collision will occur. This wrapper header renames all of the relevant zstd functions to avoid this problem. Reviewed-by: Kjeld Schouten <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Sebastian Gottschall <[email protected]> Closes #10775
* Appease GCC sprintf warnings found on Fedora 32/GCC 10.0.1Chris McDonough2020-08-242-2/+2
| | | | | | | | | Increase the size of DDT_NAMELEN and MNT_LINE_MAX to appease GCC snprintf truncation warnings. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Chris McDonough <[email protected]> Closes #10712 Closes #10766
* libzstd: Don't warn about stack frame size in userspaceRyan Moeller2020-08-231-11/+3
| | | | | | | | | | | | | | With the current way CFLAGS are modified in libzstd, CFLAGS passed on the make command line will cause the CFLAGS in the Makefile for zstd.c to be discarded, but not AM_CFLAGS. This causes a smaller frame size limit to be used, and the build fails. We don't need to worry about stack frame sizes in userspace. Drop the extra flags. Reviewed-by: Kjeld Schouten <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10773
* Import vdev ashift optimization from FreeBSDRyan Moeller2020-08-211-35/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Many modern devices use physical allocation units that are much larger than the minimum logical allocation size accessible by external commands. Two prevalent examples of this are 512e disk drives (512b logical sector, 4K physical sector) and flash devices (512b logical sector, 4K or larger allocation block size, and 128k or larger erase block size). Operations that modify less than the physical sector size result in a costly read-modify-write or garbage collection sequence on these devices. Simply exporting the true physical sector of the device to ZFS would yield optimal performance, but has two serious drawbacks: 1. Existing pools created with devices that have different logical and physical block sizes, but were configured to use the logical block size (e.g. because the OS version used for pool construction reported the logical block size instead of the physical block size) will suddenly find that the vdev allocation size has increased. This can be easily tolerated for active members of the array, but ZFS would prevent replacement of a vdev with another identical device because it now appears that the smaller allocation size required by the pool is not supported by the new device. 2. The device's physical block size may be too large to be supported by ZFS. The optimal allocation size for the vdev may be quite large. For example, a RAID controller may export a vdev that requires read-modify-write cycles unless accessed using 64k aligned/sized requests. ZFS currently has an 8k minimum block size limit. Reporting both the logical and physical allocation sizes for vdevs solves these problems. A device may be used so long as the logical block size is compatible with the configuration. By comparing the logical and physical block sizes, new configurations can be optimized and administrators can be notified of any existing pools that are sub-optimal. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Co-authored-by: Matthew Macy <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10619
* Add zstd support to zfsMichael Niewöhner2020-08-204-3/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR adds two new compression types, based on ZStandard: - zstd: A basic ZStandard compression algorithm Available compression. Levels for zstd are zstd-1 through zstd-19, where the compression increases with every level, but speed decreases. - zstd-fast: A faster version of the ZStandard compression algorithm zstd-fast is basically a "negative" level of zstd. The compression decreases with every level, but speed increases. Available compression levels for zstd-fast: - zstd-fast-1 through zstd-fast-10 - zstd-fast-20 through zstd-fast-100 (in increments of 10) - zstd-fast-500 and zstd-fast-1000 For more information check the man page. Implementation details: Rather than treat each level of zstd as a different algorithm (as was done historically with gzip), the block pointer `enum zio_compress` value is simply zstd for all levels, including zstd-fast, since they all use the same decompression function. The compress= property (a 64bit unsigned integer) uses the lower 7 bits to store the compression algorithm (matching the number of bits used in a block pointer, as the 8th bit was borrowed for embedded block pointers). The upper bits are used to store the compression level. It is necessary to be able to determine what compression level was used when later reading a block back, so the concept used in LZ4, where the first 32bits of the on-disk value are the size of the compressed data (since the allocation is rounded up to the nearest ashift), was extended, and we store the version of ZSTD and the level as well as the compressed size. This value is returned when decompressing a block, so that if the block needs to be recompressed (L2ARC, nop-write, etc), that the same parameters will be used to result in the matching checksum. All of the internal ZFS code ( `arc_buf_hdr_t`, `objset_t`, `zio_prop_t`, etc.) uses the separated _compress and _complevel variables. Only the properties ZAP contains the combined/bit-shifted value. The combined value is split when the compression_changed_cb() callback is called, and sets both objset members (os_compress and os_complevel). The userspace tools all use the combined/bit-shifted value. Additional notes: zdb can now also decode the ZSTD compression header (flag -Z) and inspect the size, version and compression level saved in that header. For each record, if it is ZSTD compressed, the parameters of the decoded compression header get printed. ZSTD is included with all current tests and new tests are added as-needed. Per-dataset feature flags now get activated when the property is set. If a compression algorithm requires a feature flag, zfs activates the feature when the property is set, rather than waiting for the first block to be born. This is currently only used by zstd but can be extended as needed. Portions-Sponsored-By: The FreeBSD Foundation Co-authored-by: Allan Jude <[email protected]> Co-authored-by: Brian Behlendorf <[email protected]> Co-authored-by: Sebastian Gottschall <[email protected]> Co-authored-by: Kjeld Schouten-Lebbing <[email protected]> Co-authored-by: Michael Niewöhner <[email protected]> Signed-off-by: Allan Jude <[email protected]> Signed-off-by: Allan Jude <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Sebastian Gottschall <[email protected]> Signed-off-by: Kjeld Schouten-Lebbing <[email protected]> Signed-off-by: Michael Niewöhner <[email protected]> Closes #6247 Closes #9024 Closes #10277 Closes #10278
* libzfs_core: Initialize fail_ioc_cmd to ZFS_IOC_LASTRyan Moeller2020-08-181-2/+2
| | | | | | | | | FreeBSD numbers `ZFS_IOC_*` starting at 0, so pick a different sentinel value to avoid unintentionally messing with `ZFS_IOC_POOL_CREATE` ioctls. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10729
* Remove unused `zpool_is_bootable`George Melikov2020-08-181-11/+0
| | | | | | | | | | | | | Otherwise compiler errors with: ``` libzfs_pool.c:449:1: error: 'zpool_is_bootable' defined but not used [-Werror=unused-function] ``` Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: George Melikov <[email protected]> Closes #10734
* Remove GRUB restrictionsRichard Laager2020-08-171-19/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The GRUB restrictions are based around the pool's bootfs property. Given the current situation where GRUB is not staying current with OpenZFS pool features, having either a non-ZFS /boot or a separate pool with limited features are pretty much the only long-term answers for GRUB support. Only the second case matters in this context. For the restrictions to be useful, the bootfs property would have to be set on the boot pool, because that is where we need the restrictions, as that is the pool that GRUB reads from. The documentation for bootfs describes it as pointing to the root pool. That's also how it's used in the initramfs. ZFS does not allow setting bootfs to point to a dataset in another pool. (If it did, it'd be difficult-to-impossible to enforce these restrictions cross-pool). Accordingly, bootfs is pretty much useless for GRUB scenarios moving forward. Even for users who have only one pool, the existing restrictions for GRUB are incomplete. They don't prevent you from enabling the unsupported checksums, for example. For that reason, I have ripped out all the GRUB restrictions. A little longer-term, I think extending the proposed features=portable system to define a features=grub is a much more useful approach. The user could set that on the boot pool at creation, and things would Just Work. Reviewed-by: Paul Dagnelie <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Richard Laager <[email protected]> Closes #8627
* FreeBSD: Fix module autoloading when built in baseMatthew Macy2020-08-111-2/+12
| | | | | | | | | | | The KMOD name is "zfs" instead of "openzfs" when building in FreeBSD. Define a ZFS_KMOD symbol as "zfs" when IN_BASE is defined, otherwise "openzfs". Reviewed-by: Brian Behlendorf <[email protected]> Co-authored-by: Ryan Moeller <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10699
* Remove KMC_NOMAGAZINEMatthew Ahrens2020-08-051-1/+0
| | | | | | | | | Remove dead code to make the implementation easier to understand. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matt Ahrens <[email protected]> Closes #10650
* Remove KMC_NOHASHMatthew Ahrens2020-08-051-2/+0
| | | | | | | | | Remove dead code to make the implementation easier to understand. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matt Ahrens <[email protected]> Closes #10650
* FreeBSD: Fix `zfs jail` and add a testRyan Moeller2020-08-011-1/+1
| | | | | | | | | zfs_jail was not using zfs_ioctl so failed to map the IOC number correctly. Use zfs_ioctl to perform the jail ioctl and add a test case for FreeBSD. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10658
* Change the error handling for invalid property valuesAllan Jude2020-08-011-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | ZFS recv should return a useful error message when an invalid index property value is provided in the send stream properties nvlist With a compression= property outside of the understood range: Before: ``` receiving full stream of zof/zstd_send@send2 into testpool/recv@send2 internal error: Invalid argument Aborted (core dumped) ``` Note: the recv completes successfully, the abort() is likely just to make it easier to track the unexpected error code. After: ``` receiving full stream of zof/zstd_send@send2 into testpool/recv@send2 cannot receive compression property on testpool/recv: invalid property value received 28.9M stream in 1 seconds (28.9M/sec) ``` Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Allan Jude <[email protected]> Closes #10631
* Changes to make openzfs build within FreeBSD buildworldMatthew Macy2020-07-315-5/+15
| | | | | | | | | A collection of header changes to enable FreeBSD to build with vendored OpenZFS. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10635
* When encountering EZFS_UNKNOWN, print the error text buffer anywayAllan Jude2020-07-311-1/+2
| | | | | | | | | Rather than just saying there was an internal error, provide any context we might have to the user to help them understand the issue. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Allan Jude <[email protected]> Closes #10632
* Rename refcount.h to zfs_refcount.hMatthew Macy2020-07-291-1/+1
| | | | | | | | | Renamed to avoid conflicting with refcount.h when a different implementation is already provided by the platform. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10620
* Prefix zfs internal endian checks with _ZFSMatthew Macy2020-07-284-151/+44
| | | | | | | | | | | FreeBSD defines _BIG_ENDIAN BIG_ENDIAN _LITTLE_ENDIAN LITTLE_ENDIAN on every architecture. Trying to do cross builds whilst hiding this from ZFS has proven extremely cumbersome. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10621
* FreeBSD: Fixes required to build ZFS on PowerPCMatthew Macy2020-07-251-2/+2
| | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10622
* Annotate unused parameters on inline definitions as suchKyle Evans2020-07-231-3/+4
| | | | | | | | | | | | | | * libspl: umem: These are obviously and intentionally unused; annotate them as such to appease -Wunused-parameter builds that include this header. * sys/dmu.h: In this case, clear_on_evict_dbufp is only used for ZFS_DEBUG builds, so annotate it as __maybe_unused to appease -Wunused-parameter. Reviewed-By: Brian Behlendorf <[email protected]> Signed-off-by: Kyle Evans <[email protected]> Closes #10606
* libzfs: const'ify path argument to zfs_path_to_zhandleKyle Evans2020-07-221-1/+1
| | | | | | | | | | | | | | | | | zfs_path_to_zhandle has no need to mutate the path argument, most notably: - zfs_open takes path as const - getextmntent takes path as const - fprintf most clearly doesn't need to mutate it It's hard to foresee any reason that libzfs could conceivably want to mutate it in the future, either, so const'ify it. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Matt Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Kyle Evans <[email protected]> Closes #10605
* FreeBSD: zfs commands backward compatibilityMatthew Macy2020-07-152-8/+64
| | | | | | | | | Update the zfs commands such that they're backwards compatible with the version of ZFS is the base FreeBSD. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10542
* Disable -Wl,-z,defs for ASAN buildsJoao Carlos Mendes Luis2020-07-145-5/+25
| | | | | | | | | | | Commit af65916 added -Wl,-z,defs for the shared libraries. This apparently does not work in some cases with --enable-asan, so only add it for non-ASAN builds. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: João Carlos Mendes Luis <[email protected]> Signed-off-by: Arvind Sankar <[email protected]> Closes #10557 Closes #10560
* Fixing gang ABD child removal race conditionBrian Atkinson2020-07-141-0/+1
| | | | | | | | | | | | | | | | | | | | On linux the list debug code has been setting off a failure when checking that the node->next->prev value is pointing back at the node. At times this check evaluates to 0xdead. When removing a child from a gang ABD we must acquire the child's abd_mtx to make sure that the same ABD is not being added to another gang ABD while it is being removed from a gang ABD. This fixes a race condition when checking if an ABDs link is already active and part of another gang ABD before adding it to a gang. Added additional debug code for the gang ABD in abd_verify() to make sure each child ABD has active links. Also check to make sure another gang ABD is not added to a gang ABD. Reviewed-by: Serapheim Dimitropoulos <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Matt Ahrens <[email protected]> Signed-off-by: Brian Atkinson <[email protected]> Closes #10511
* Remove dependency on sharetab file and refactor sharing logicGeorge Wilson2020-07-1315-1620/+1168
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | == Motivation and Context The current implementation of 'sharenfs' and 'sharesmb' relies on the use of the sharetab file. The use of this file is os-specific and not required by linux or freebsd. Currently the code must maintain updates to this file which adds complexity and presents a significant performance impact when sharing many datasets. In addition, concurrently running 'zfs sharenfs' command results in missing entries in the sharetab file leading to unexpected failures. == Description This change removes the sharetab logic from the linux and freebsd implementation of 'sharenfs' and 'sharesmb'. It still preserves an os-specific library which contains the logic required for sharing NFS or SMB. The following entry points exist in the vastly simplified libshare library: - sa_enable_share -- shares a dataset but may not commit the change - sa_disable_share -- unshares a dataset but may not commit the change - sa_is_shared -- determine if a dataset is shared - sa_commit_share -- notify NFS/SMB subsystem to commit the shares - sa_validate_shareopts -- determine if sharing options are valid The sa_commit_share entry point is provided as a performance enhancement and is not required. The sa_enable_share/sa_disable_share may commit the share as part of the implementation. Libshare provides a framework for both NFS and SMB but some operating systems may not fully support these protocols or all features of the protocol. NFS Operation: For linux, libshare updates /etc/exports.d/zfs.exports to add and remove shares and then commits the changes by invoking 'exportfs -r'. This file, is automatically read by the kernel NFS implementation which makes for better integration with the NFS systemd service. For FreeBSD, libshare updates /etc/zfs/exports to add and remove shares and then commits the changes by sending a SIGHUP to mountd. SMB Operation: For linux, libshare adds and removes files in /var/lib/samba/usershares by calling the 'net' command directly. There is no need to commit the changes. FreeBSD does not support SMB. == Performance Results To test sharing performance we created a pool with an increasing number of datasets and invoked various zfs actions that would enable and disable sharing. The performance testing was limited to NFS sharing. The following tests were performed on an 8 vCPU system with 128GB and a pool comprised of 4 50GB SSDs: Scale testing: - Share all filesystems in parallel -- zfs sharenfs=on <dataset> & - Unshare all filesystems in parallel -- zfs sharenfs=off <dataset> & Functional testing: - share each filesystem serially -- zfs share -a - unshare each filesystem serially -- zfs unshare -a - reset sharenfs property and unshare -- zfs inherit -r sharenfs <pool> For 'zfs sharenfs=on' scale testing we saw an average reduction in time of 89.43% and for 'zfs sharenfs=off' we saw an average reduction in time of 83.36%. Functional testing also shows a huge improvement: - zfs share -- 97.97% reduction in time - zfs unshare -- 96.47% reduction in time - zfs inhert -r sharenfs -- 99.01% reduction in time Reviewed-by: Matt Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Bryant G. Ly <[email protected]> Signed-off-by: George Wilson <[email protected]> External-Issue: DLPX-68690 Closes #1603 Closes #7692 Closes #7943 Closes #10300
* filesystem_limit/snapshot_limit is incorrectly enforced against rootMatthew Ahrens2020-07-111-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The filesystem_limit and snapshot_limit properties limit the number of filesystems or snapshots that can be created below this dataset. According to the manpage, "The limit is not enforced if the user is allowed to change the limit." Two types of users are allowed to change the limit: 1. Those that have been delegated the `filesystem_limit` or `snapshot_limit` permission, e.g. with `zfs allow USER filesystem_limit DATASET`. This works properly. 2. A user with elevated system privileges (e.g. root). This does not work - the root user will incorrectly get an error when trying to create a snapshot/filesystem, if it exceeds the `_limit` property. The problem is that `priv_policy_ns()` does not work if the `cred_t` is not that of the current process. This happens when `dsl_enforce_ds_ss_limits()` is called in syncing context (as part of a sync task's check func) to determine the permissions of the corresponding user process. This commit fixes the issue by passing the `task_struct` (typedef'ed as a `proc_t`) to syncing context, and then using `has_capability()` to determine if that process is privileged. Note that we still need to pass the `cred_t` to syncing context so that we can check if the user was delegated this permission with `zfs allow`. This problem only impacts Linux. Wrappers are added to FreeBSD but it continues to use `priv_check_cred()`, which works on arbitrary `cred_t`. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #8226 Closes #10545
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-30 03:56:56 +0000