aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* StaleBot TweaksKjeld Schouten-Lebbing2020-08-262-2/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Add Status: Triage Needed to bug reports Currently "Type: Defect" is auto added. Adding a triage tag, makes sure all issues are reviewed by a maintainer It also opens up some options to priorities defects in the near future. - Prevent future StaleBot Spam StaleBot will limit itself to 6 actions per hour This should prevent future floods of StaleBot activity (aka Spam) - StaleBot: Ignore issues that are being worked on Ignore the following Issues: - tagged: "Status: Work in Progress" - Having a maintainer assigned - Being part of a project - Having a milestone tag - Rename Ignore "Type: Understood" to "Bot: Not Stale" This Commits changes the general ignore tag for StaleBot from: "Type: Understood" to "Bot: Not Stale" Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Kjeld Schouten-Lebbing <[email protected]> Closes #10813
* Introduce limit on size of L2ARC headersAlexander Motin2020-08-252-5/+36
| | | | | | | | | | | | | | | | | Since L2ARC buffers are not evicted on memory pressure, too large amount of headers on system with irrationally large L2ARC can render it slow or even unusable. This change limits L2ARC writes and rebuild if unevictable L2ARC-only headers reach dangerous level. While there, call arc_adapt() on L2ARC rebuild, so that it could properly grow arc_c, reflecting potentially significant ARC size increase and avoiding slow growth with hopeless eviction attempts later when "overflow" is detected. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reported-by: Richard Elling <[email protected]> Signed-off-by: Alexander Motin <[email protected]> Closes #10765
* Tag 2.0.0-rc1zfs-2.0.0-rc1Brian Behlendorf2020-08-251-4/+4
| | | | | | | | | | | | | | | | | New features: - Unified code base for Linux and FreeBSD - Redacted 'zfs send/recv' - Persistent L2ARC - Sequential resilvering - ZSTD Compression - Log spacemaps - Fast clone deletion - Sectional zfs/zpool man pages - Added 'zpool wait' subcommand - Improved 'zfs share' scalability - Improved AES-GCM encryption performance Signed-off-by: Brian Behlendorf <[email protected]>
* Don't assert on nvlists larger than SPA_MAXBLOCKSIZEAllan Jude2020-08-253-2/+44
| | | | | | | | | | | | | | | | | | | | | | | | | Originally we asserted that all reads are less than SPA_MAXBLOCKSIZE However, nvlists are not ZFS records, and are not limited to SPA_MAXBLOCKSIZE. Add a new environment variable, ZFS_SENDRECV_MAX_NVLIST, to allow the user to specify the maximum size of the nvlist that can be sent or received. Default value: 4 * SPA_MAXBLOCKSIZE (64 MB) Modify libzfs send routines to return a useful error if the send stream will generate an nvlist that is beyond the maximum size. Modify libzfs recv routines to add an explicit error message if the nvlist is too large, rather than abort()ing. Move the change the assert() to only trigger on data records Reviewed-by: Paul Dagnelie <[email protected]> Reviewed-by: Kjeld Schouten <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Matthew Ahrens <[email protected]> Signed-off-by: Allan Jude <[email protected]> Closes #9616
* Mark lua setjmp/longjmp for powerpc weaksterlingjensen2020-08-251-5/+5
| | | | | | | | Linux already defines setjmp/longjmp for powerpc, which leads to duplicate symbols in a statically linked build. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Sterlng Jensen <[email protected]> Closes #10795
* Export dmu_offset_next() symbolBrian Behlendorf2020-08-251-0/+1
| | | | | | | | Export the dmu_offset_next() symbol for use by Lustre. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10796
* man: Canonicalize .TH usageRyan Moeller2020-08-2418-22/+22
| | | | | | | | | | | | | * Use all caps for document title. * Remove section name as it can be inferred from the section number. * Name "OpenZFS" as the document source. * Bump modification date. While here, fixed trailing whitespace reported by igor. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10792
* Fix inability to destroy snapshot used over NFSyouzhongyang2020-08-241-0/+19
| | | | | | | | | | | | The cache of struct svc_export and struct svc_expkey by nfsd and rpc.mountd for the snapshot holds references to the mount point. We need to flush them out before unmounting, otherwise umount would fail with EBUSY. Reviewed-by: Don Brady <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Youzhong Yang <[email protected]> Closes #6000 Closes #10783
* Avoid symbol collision with in-kernel zstdlibSebastian Gottschall2020-08-247-17/+491
| | | | | | | | | | | For Linux, when zfs is compiled as an in kernel static variant and the in kernel zstd library is compiled statically into the kernel a symbol collision will occur. This wrapper header renames all of the relevant zstd functions to avoid this problem. Reviewed-by: Kjeld Schouten <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Sebastian Gottschall <[email protected]> Closes #10775
* Add Stale-botKjeld Schouten-Lebbing2020-08-241-0/+17
| | | | | | | | | | | | | | | This file configures the following stale-bot: https://github.com/apps/stale It is set to mark issues as "Stale" after 365 days It is also set to auto-close the issue 90 days after. Please be aware that this issue also requires- The listed stale-bot being added to the repo. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Kjeld Schouten-Lebbing <[email protected]> Closes #10778
* Appease GCC sprintf warnings found on Fedora 32/GCC 10.0.1Chris McDonough2020-08-243-3/+3
| | | | | | | | | Increase the size of DDT_NAMELEN and MNT_LINE_MAX to appease GCC snprintf truncation warnings. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Chris McDonough <[email protected]> Closes #10712 Closes #10766
* ZTS: Improve block_device_wait on FreeBSDRyan Moeller2020-08-241-1/+3
| | | | | | | | | | | | | | FreeBSD doesn't have an equivalent to udevadm settle, so we have been resorting to a three second sleep to wait for device changes to take effect. This is far from ideal. We are mainly waiting for volmode=geom zvols to appear in /dev, so as a hack, reading the geom config will have the desired effect of quiescing the geom state. Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10768
* Improve documentation of zpool import -d/-c vs -sChris McDonough2020-08-231-3/+4
| | | | | | | | | | | Specify that, by default, zpool import uses the libblkid cache on Linux and geom on FreeBSD, and only scans when -d/-s is provided. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Chris McDonough <[email protected]> Closes #7656 Closes #10771
* CI checkstyle: add linter + rename job + install latest flake8George Melikov2020-08-231-2/+7
| | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: George Melikov <[email protected]> Closes #10784
* ZFS performance tests should clean up NFS mountTony Nguyen2020-08-231-1/+2
| | | | | | | | | | | This change umounts client's NFS mount after each test so we can avoid two sporadic issues: 1) client NFS stale mount and 2) zpool export and zpool destroy failed due to dataset busy Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Nguyen <[email protected]> Closes #10767
* Add seperate issue for questionsKjeld Schouten-Lebbing2020-08-233-2/+39
| | | | | | | | | | | | | A big portion of issues are of "Type: Question". This PR adds a separate issue template for those. It also automatically adds the "Type: Question" tag. in addition it adds "Type: Defect" to all bug reports by default Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Kjeld Schouten-Lebbing <[email protected]> Closes #10779
* libzstd: Don't warn about stack frame size in userspaceRyan Moeller2020-08-231-11/+3
| | | | | | | | | | | | | | With the current way CFLAGS are modified in libzstd, CFLAGS passed on the make command line will cause the CFLAGS in the Makefile for zstd.c to be discarded, but not AM_CFLAGS. This causes a smaller frame size limit to be used, and the build fails. We don't need to worry about stack frame sizes in userspace. Drop the extra flags. Reviewed-by: Kjeld Schouten <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10773
* Prevent zfs_acl_chmod() if aclmode restricted and ACL inheritedAndrew2020-08-222-0/+8
| | | | | | | | | | | | | | | | In absence of inheriting entry for owner@, group@, or everyone@, zfs_acl_chmod() is called to set these. This can cause confusion for Samba admins who do not expect these entries to appear on newly created files and directories once they have been stripped from from the parent directory. When aclmode is set to "restricted", chmod is prevented on non-trivial ACLs. It is not a stretch to assume that in this case the administrator does not want ZFS to add the missing special entries. Add check for this aclmode, and if an inherited entry is present skip zfs_acl_chmod(). Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Andrew Walker <[email protected]> Closes #10748
* Update issue templateKjeld Schouten-Lebbing2020-08-222-0/+43
| | | | | | | | | | | | | | | Github has started using a new issue templating structure. This commit moves the current template and adds one additional one. - Moves issue template to new issue-template folder - Adds feature request template - removes the following warning when viewing issue template Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Kjeld Schouten-Lebbing <[email protected]> Closes #10759
* ZTS: Remove leftover variable namesRyan Moeller2020-08-221-3/+0
| | | | | | | | | These were overlooked when use of `local` was removed to satisfy checkbashisms. Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10762
* Remove vestigial settings related to initramfsChris McDonough2020-08-222-38/+1
| | | | | | | | | | | | Remove ZFS_POOL_IMPORT, ZFS_INITRD_PRE_MOUNTROOT_SLEEP, ZFS_INITRD_POST_MODPROBE_SLEEP, and ZFS_INITRD_ADDITIONAL_DATASETS features from etc/defaults/zfs.in. These features no longer work. Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Chris McDonough <[email protected]> Closes #9126 Closes #10757
* Make formatting of dedup values string consistentClint Armstrong2020-08-221-2/+2
| | | | | | | | | | | All other prop values return options separated by ` | `, dedup values do not, they are separated by `, `. This change makes the dedup value formatting consistent with other properties. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Clint Armstrong <[email protected]> Closes #10761
* Import vdev ashift optimization from FreeBSDRyan Moeller2020-08-2123-99/+326
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Many modern devices use physical allocation units that are much larger than the minimum logical allocation size accessible by external commands. Two prevalent examples of this are 512e disk drives (512b logical sector, 4K physical sector) and flash devices (512b logical sector, 4K or larger allocation block size, and 128k or larger erase block size). Operations that modify less than the physical sector size result in a costly read-modify-write or garbage collection sequence on these devices. Simply exporting the true physical sector of the device to ZFS would yield optimal performance, but has two serious drawbacks: 1. Existing pools created with devices that have different logical and physical block sizes, but were configured to use the logical block size (e.g. because the OS version used for pool construction reported the logical block size instead of the physical block size) will suddenly find that the vdev allocation size has increased. This can be easily tolerated for active members of the array, but ZFS would prevent replacement of a vdev with another identical device because it now appears that the smaller allocation size required by the pool is not supported by the new device. 2. The device's physical block size may be too large to be supported by ZFS. The optimal allocation size for the vdev may be quite large. For example, a RAID controller may export a vdev that requires read-modify-write cycles unless accessed using 64k aligned/sized requests. ZFS currently has an 8k minimum block size limit. Reporting both the logical and physical allocation sizes for vdevs solves these problems. A device may be used so long as the logical block size is compatible with the configuration. By comparing the logical and physical block sizes, new configurations can be optimized and administrators can be notified of any existing pools that are sub-optimal. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Co-authored-by: Matthew Macy <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10619
* Remove hard coded "Linux" OS from manpagesRyan Moeller2020-08-2165-62/+92
| | | | | | | | | | | | | | | The recommended practice for `.Os` on FreeBSD is to not specify any arguments. The correct OS name is used automatically. Oddly enough, on the Linux distro I tested this on (CentOS 7), the man pager defaulted to displaying "BSD" as the OS rather than "Linux". To accommodate this, tack " Linux" back on in an install hook on Linux. This is much simpler than removing it for FreeBSD when vendored in the base system. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10760
* Silence 'make checkbashisms'Brian Behlendorf2020-08-2011-20/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit d2bce6d03 added the 'make checkbashisms' target but did not resolve all of the bashisms in the scripts. This commit doesn't resolve them all either but it does fix up a few, and it excludes the others so 'make checkstyle' no longer prints warnings. It's a small step in the right direction. * Dracut is Linux specific and itself depends on bash. Therefore all dracut support scripts can be bash specific, update their shebang accordingly. * zed-functions.sh, zfs-import, zfs-mount, zfs-zed, smart paxcheck.sh, make_gitrev.sh - these scripts were excuded from the check until they can be updated and properly tested. * zfsunlock - only whole values for sleep are allowed. * vdev_id - removed unneeded locals; use && instead of -a. * dkms.mkconf, dkms.postbuil - use || instead of -o. Reviewed-by: InsanePrawn <[email protected]> Reviewed-by: Gabriel A. Devenyi <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10755
* 'zfs share -a' should clean noauto exportsDon Brady2020-08-204-2/+92
| | | | | | | | | | | | | | | | | | This is a follow on to PR #10688 where `zfs share -a` allows the sharing of canmount=noauto datasets if they are mounted. However, when a dataset with canmount=noauto is not mounted, the command should also purge any existing entries from the exports file. Otherwise, after a reboot, the nfs server attempts to export the underlying mountpath, not the dataset. This can lead to a hard hang for existing client mounts. Instead of just skipping the adding of an export if not mounted and canmount=noauto, have it also remove an existing export of the dataset so that, after a reboot, we don't export an unmounted dataset. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Wilson <[email protected]> Signed-off-by: Don Brady <[email protected]> Closes #10747
* Fix indentation in dnode_free_range()Matthew Ahrens2020-08-201-7/+7
| | | | | | Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #10744
* FreeBSD: 11.x arc_stats compatibilityMatthew Macy2020-08-203-1/+16
| | | | | | | | | Removing other_size from arc_stats breaks top in 11.x jails running on HEAD. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10745
* Add zstd support to zfsMichael Niewöhner2020-08-2069-115/+2666
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR adds two new compression types, based on ZStandard: - zstd: A basic ZStandard compression algorithm Available compression. Levels for zstd are zstd-1 through zstd-19, where the compression increases with every level, but speed decreases. - zstd-fast: A faster version of the ZStandard compression algorithm zstd-fast is basically a "negative" level of zstd. The compression decreases with every level, but speed increases. Available compression levels for zstd-fast: - zstd-fast-1 through zstd-fast-10 - zstd-fast-20 through zstd-fast-100 (in increments of 10) - zstd-fast-500 and zstd-fast-1000 For more information check the man page. Implementation details: Rather than treat each level of zstd as a different algorithm (as was done historically with gzip), the block pointer `enum zio_compress` value is simply zstd for all levels, including zstd-fast, since they all use the same decompression function. The compress= property (a 64bit unsigned integer) uses the lower 7 bits to store the compression algorithm (matching the number of bits used in a block pointer, as the 8th bit was borrowed for embedded block pointers). The upper bits are used to store the compression level. It is necessary to be able to determine what compression level was used when later reading a block back, so the concept used in LZ4, where the first 32bits of the on-disk value are the size of the compressed data (since the allocation is rounded up to the nearest ashift), was extended, and we store the version of ZSTD and the level as well as the compressed size. This value is returned when decompressing a block, so that if the block needs to be recompressed (L2ARC, nop-write, etc), that the same parameters will be used to result in the matching checksum. All of the internal ZFS code ( `arc_buf_hdr_t`, `objset_t`, `zio_prop_t`, etc.) uses the separated _compress and _complevel variables. Only the properties ZAP contains the combined/bit-shifted value. The combined value is split when the compression_changed_cb() callback is called, and sets both objset members (os_compress and os_complevel). The userspace tools all use the combined/bit-shifted value. Additional notes: zdb can now also decode the ZSTD compression header (flag -Z) and inspect the size, version and compression level saved in that header. For each record, if it is ZSTD compressed, the parameters of the decoded compression header get printed. ZSTD is included with all current tests and new tests are added as-needed. Per-dataset feature flags now get activated when the property is set. If a compression algorithm requires a feature flag, zfs activates the feature when the property is set, rather than waiting for the first block to be born. This is currently only used by zstd but can be extended as needed. Portions-Sponsored-By: The FreeBSD Foundation Co-authored-by: Allan Jude <[email protected]> Co-authored-by: Brian Behlendorf <[email protected]> Co-authored-by: Sebastian Gottschall <[email protected]> Co-authored-by: Kjeld Schouten-Lebbing <[email protected]> Co-authored-by: Michael Niewöhner <[email protected]> Signed-off-by: Allan Jude <[email protected]> Signed-off-by: Allan Jude <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Sebastian Gottschall <[email protected]> Signed-off-by: Kjeld Schouten-Lebbing <[email protected]> Signed-off-by: Michael Niewöhner <[email protected]> Closes #6247 Closes #9024 Closes #10277 Closes #10278
* Import ZStandard v1.4.5Michael Niewöhner2020-08-207-2/+30110
| | | | | | | | | | | | | | | | | | | | | | | | | | ZStandard is a modern, high performance, general compression algorithm. It provides similar or better compression levels to GZIP, but with much better performance. ZStandard provides a large selection of compression levels to allow a storage administrator to select the preferred performance/compression trade-off. This commit imports the unmodified ZStandard single-file library which will be used by ZFS. The implementation of this new library is done with future updates of zstd in mind. For this reason we integrated the code in a way, that does not require modifications to the library. For more details, see `module/zstd/README.md`. The library is excluded from codecov calculation and cppcheck as unaltered dependencies do not need full codecov or cppcheck. Co-authored-by: Allan Jude <[email protected]> Co-authored-by: Kjeld Schouten-Lebbing <[email protected]> Co-authored-by: Michael Niewöhner <[email protected]> Signed-off-by: Allan Jude <[email protected]> Signed-off-by: Kjeld Schouten-Lebbing <[email protected]> Signed-off-by: Michael Niewöhner <[email protected]>
* Linux 5.7 compat: Include linux/sched.h in spl/sys/mutex.hPavel Snajdr2020-08-191-0/+1
| | | | | | | | | struct task_struct is needed for lockdep_off() in mutex.h This has popped up after e616cb8daadf (in linux-5.7-rc7). Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Pavel Snajdr <[email protected]> Closes #10741
* FreeBSD: Add option to rewind checkpoint while importing root poolMariusz Zaborski2020-08-193-3/+6
| | | | | | | | This option is used by FreeBSD boot loader. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Mariusz Zaborski <[email protected]> Closes #10738
* ZED: Do not offline a missing device if no spare is availableBrian Behlendorf2020-08-182-24/+39
| | | | | | | | | | | | | | | | | | | | | | | | Due to commit d48091d a removed device is now explicitly offlined by the ZED if no spare is available, rather than the letting ZFS detect it as UNAVAIL. This broke auto-replacing of whole-disk devices, as described in issue #10577. In short, when a new device is reinserted in the same slot, the ZED will try to ONLINE it without letting ZFS recreate the necessary partition table. This change simply avoids setting the device OFFLINE when removed if no spare is available (or if spare_on_remove is false). This change has been left minimal to allow it to be backported to 0.8.x release. The auto_offline_001_pos ZTS test has been updated accordingly. Some follow up work is planned to update the ZED so it transitions the vdev to a REMOVED state. This is a state which has always existed but there is no current interface the ZED can use to accomplish this. Therefore it's being left to a follow up PR. Reviewed-by: Gionatan Danti <[email protected]> Co-authored-by: Gionatan Danti <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10577 Closes #10730
* Fix ARC aggsum access after arc_state_fini()Brian Behlendorf2020-08-181-8/+8
| | | | | | | | | | | | | | Commit 85ec5cbae updated abd_update_scatter_stats() such that it calls arc_space_consume() and arc_space_return() when updating the scatter stats. This requires that the global aggsum value for the ARC be initialized. Normally this is not an issue, however during module unload the l2arc_do_free_on_write() function was called in l2arc_cleanup() after arc_state_fini() destroyed the aggsum values. We can resolve this issue by performing l2arc_do_free_on_write() slightly earlier in arc_fini(). Reviewed-by: Matthew Ahrens <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10739
* libzfs_core: Initialize fail_ioc_cmd to ZFS_IOC_LASTRyan Moeller2020-08-181-2/+2
| | | | | | | | | FreeBSD numbers `ZFS_IOC_*` starting at 0, so pick a different sentinel value to avoid unintentionally messing with `ZFS_IOC_POOL_CREATE` ioctls. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10729
* FreeBSD: Fix UNIX permissions checkingMatthew Macy2020-08-187-99/+174
| | | | | | Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10727
* Add define to enable autotrim to default to onMatthew Macy2020-08-182-2/+8
| | | | | | | | | | | | In FreeBSD trim has defaulted to on for several years. In order to minimize POLA violations on import it's important to maintain this default when importing vendored openzfs in to FreeBSD base. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10719
* Make zc_nvlist_src_size limit tunableRyan Moeller2020-08-186-4/+51
| | | | | | | | | | | | | | | | | | We limit the size of nvlists passed to the kernel so a user cannot make the kernel do an unreasonably large allocation. On FreeBSD this limit was 128 kiB, which turns out to be a bit too small when doing some operations involving a large number of datasets or snapshots, for example replication. Make this limit tunable, with a platform-specific auto default. Linux keeps its limit at KMALLOC_MAX_SIZE. FreeBSD uses 1/4 of the system limit on user wired memory, which allows it to scale depending on system configuration. Reviewed-by: Matt Macy <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Issue #6572 Closes #10706
* Remove unused `zpool_is_bootable`George Melikov2020-08-181-11/+0
| | | | | | | | | | | | | Otherwise compiler errors with: ``` libzfs_pool.c:449:1: error: 'zpool_is_bootable' defined but not used [-Werror=unused-function] ``` Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: George Melikov <[email protected]> Closes #10734
* Remove GRUB restrictionsRichard Laager2020-08-175-89/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The GRUB restrictions are based around the pool's bootfs property. Given the current situation where GRUB is not staying current with OpenZFS pool features, having either a non-ZFS /boot or a separate pool with limited features are pretty much the only long-term answers for GRUB support. Only the second case matters in this context. For the restrictions to be useful, the bootfs property would have to be set on the boot pool, because that is where we need the restrictions, as that is the pool that GRUB reads from. The documentation for bootfs describes it as pointing to the root pool. That's also how it's used in the initramfs. ZFS does not allow setting bootfs to point to a dataset in another pool. (If it did, it'd be difficult-to-impossible to enforce these restrictions cross-pool). Accordingly, bootfs is pretty much useless for GRUB scenarios moving forward. Even for users who have only one pool, the existing restrictions for GRUB are incomplete. They don't prevent you from enabling the unsupported checksums, for example. For that reason, I have ripped out all the GRUB restrictions. A little longer-term, I think extending the proposed features=portable system to define a features=grub is a much more useful approach. The user could set that on the boot pool at creation, and things would Just Work. Reviewed-by: Paul Dagnelie <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Richard Laager <[email protected]> Closes #8627
* ZTS: ztest may cause mmp tests failuresBrian Behlendorf2020-08-171-0/+2
| | | | | | | | | | | | The mmp_exported_import and mmp_inactive_import tests depend on ztest simulating an active pool. If ztest unexpectedly terminates due to an unrelated issue the test case will fail. Since ztest is not yet 100% reliable I've added these tests to the maybe exception list. They can be removed when the issues with ztest are resolved or if the test cases are updated to handle these unexpected failures. Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10726
* Include scatter_chunk_waste in arc_sizeMatthew Ahrens2020-08-176-11/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The ARC caches data in scatter ABD's, which are collections of pages, which are typically 4K. Therefore, the space used to cache each block is rounded up to a multiple of 4K. The ABD subsystem tracks this wasted memory in the `scatter_chunk_waste` kstat. However, the ARC's `size` is not aware of the memory used by this round-up, it only accounts for the size that it requested from the ABD subsystem. Therefore, the ARC is effectively using more memory than it is aware of, due to the `scatter_chunk_waste`. This impacts observability, e.g. `arcstat` will show that the ARC is using less memory than it effectively is. It also impacts how the ARC responds to memory pressure. As the amount of `scatter_chunk_waste` changes, it appears to the ARC as memory pressure, so it needs to resize `arc_c`. If the sector size (`1<<ashift`) is the same as the page size (or larger), there won't be any waste. If the (compressed) block size is relatively large compared to the page size, the amount of `scatter_chunk_waste` will be small, so the problematic effects are minimal. However, if using 512B sectors (`ashift=9`), and the (compressed) block size is small (e.g. `compression=on` with the default `volblocksize=8k` or a decreased `recordsize`), the amount of `scatter_chunk_waste` can be very large. On a production system, with `arc_size` at a constant 50% of memory, `scatter_chunk_waste` has been been observed to be 10-30% of memory. This commit adds `scatter_chunk_waste` to `arc_size`, and adds a new `waste` field to `arcstat`. As a result, the ARC's memory usage is more observable, and `arc_c` does not need to be adjusted as frequently. Reviewed-by: Pavel Zakharov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Wilson <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #10701
* Remove KMC_KMEM and KMC_VMEMMatthew Ahrens2020-08-175-151/+10
| | | | | | | | | | | | | | | | | `KMC_KMEM` and `KMC_VMEM` are now unused since all SPL-implemented caches are `KMC_KVMEM`. KMC_KMEM: Given the default value of `spl_kmem_cache_kmem_limit`, we don't use kmalloc to back the SPL caches, instead we use kvmalloc (KMC_KVMEM). The flag, module parameter, /proc entries, and associated code are removed. KMC_VMEM: This flag is not used, and kvmalloc() is always preferable to vmalloc(). The flag, /proc entries, and associated code are removed. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #10673
* FreeBSD: fix the build with Clang 11Ryan Moeller2020-08-176-2/+12
| | | | | | | | | | | | * Cast void * to uintptr_t before casting to boolean_t. * Avoid clashing definition of __asm when not on Linux to prevent duplicate __volatile__. This was already done in some places but not all. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Matt Macy <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10723
* FreeBSD: fix merge error in zfs_acl_ids_createMatthew Macy2020-08-171-27/+16
| | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10721
* Fix typo in btree.cSerapheim Dimitropoulos2020-08-171-2/+2
| | | | | | | Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Matthew Ahrens <[email protected]> Signed-off-by: Serapheim Dimitropoulos <[email protected]> Closes #10725
* FreeBSD: fallback to /boot/ to look for zpool.cacheMatthew Macy2020-08-172-1/+5
| | | | | | | | | Up until now zpool.cache has always lived in /boot on FreeBSD. For the sake of compatibility fallback to /boot if zpool.cache isn't found in /etc/zfs. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10720
* Fix reporting of L2ARC writes in arc_summary3George Amanakis2020-08-171-3/+3
| | | | | | | | | | arc_summary3 reports L2ARC writes in bytes. However, the related arc_stat is reported as hits. arc_summary2 report this correctly. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: George Amanakis <[email protected]> Closes #10717
* Fix l2arc_dev_rebuild_start thread nameRyan Moeller2020-08-171-4/+5
| | | | | | | | | | | | | | | `thread_create` on FreeBSD stringifies the argument passed as the thread function to create a name for the thread. The thread name for `l2arc_dev_rebuild_start` ended up with `(void (*)(void *))` in it. Change the type signature so the function does not need to be cast when creating the thread. Rename the function to `l2arc_dev_rebuild_thread` for clarity and consistency, as well. Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Amanakis <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10716
* FreeBSD: Create taskq threads in appropriate procRyan Moeller2020-08-174-19/+27
| | | | | | | | Stepping stone toward re-enabling spa_thread on FreeBSD. Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10715
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-04 21:43:48 +0000