aboutsummaryrefslogtreecommitdiffstats
path: root/module
Commit message (Collapse)AuthorAgeFilesLines
* zfsvfs_setup(): zap_stats_t may have undefined content when accessedAllan Jude2020-06-031-3/+3
| | | | Signed-off-by: Allan Jude <[email protected]>
* FreeBSD: Simplify zvol and fix lockingRyan Moeller2020-06-031-66/+25
| | | | | | | | | | | | | | | zvol_geom_bio_strategy should handle its own use of the zvol suspend reader lock and ensure the zilog exists when needed. A few other places using the zvol zilog should use the suspend reader lock as well. Simplify consumers of zvol_geom_bio_strategy, fix the locking, and while in here, use the boolean_t constants with doread. Reviewed-by: Matt Macy <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10381
* Periodically update ARC kstatsRyan Moeller2020-06-031-2/+2
| | | | | | | | | | | | | | FreeBSD needs arc_adjust_zthr to run periodically for kstats to be updated. A comment in the code suggests this may have been the original intent in illumos as well: https://github.com/openzfs/zfs/blob/c946d5a91329b075fb9bda1ac703a2e85139cf1c/module/zfs/arc.c#L4697-L4700 Create the thread with a 1 second timer. Reviewed-by: Matt Macy <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10371
* Restore avl_update() calls and related functionsJorgen Lundman2020-06-031-0/+61
| | | | | | | | | The macOS kmem implementation uses avl_update() and related functions. These same function exist in the Solaris AVL code but were removed because they were unused. Restore them. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Jorgen Lundman <[email protected]> Closes #10390
* Fix crypto build on FreeBSD HEADMatthew Macy2020-05-301-3/+1
| | | | | | | | Update API usage to reflect recent change. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10384
* Memory leak in dsl_destroy_snapshots_nvl error caseJorgen Lundman2020-05-261-0/+2
| | | | | | | | The dsl_destroy_snapshots_nvl() function has an early error out, and temporary nvlists were not freed. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Jorgen Lundman <[email protected]> Closes #10366
* Gang ABD TypeBrian Atkinson2020-05-205-103/+611
| | | | | | | | | | | | | | | Adding the gang ABD type, which allows for linear and scatter ABDs to be chained together into a single ABD. This can be used to avoid doing memory copies to/from ABDs. An example of this can be found in vdev_queue.c in the vdev_queue_aggregate() function. Reviewed-by: Matthew Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Co-authored-by: Brian <[email protected]> Co-authored-by: Mark Maybee <[email protected]> Signed-off-by: Brian Atkinson <[email protected]> Closes #10069
* Use boot_ncpus in place of max_ncpus in taskq_createDeHackEd2020-05-203-6/+6
| | | | | | | | | | | | | Due to hotplug support or BIOS bugs sometimes max_ncpus can be an absurdly high value. I have a system with 32 cores/threads but reports max_ncpus == 440. This many threads potentially cripples the system during arc_prune floods for example. boot_ncpus is the number of working CPUs when called so use that instead. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: DHE <[email protected]> Closes #10282
* freebsd: Correct the order of arguments to copyin() for Q_SETQUOTARyan Moeller2020-05-191-1/+1
| | | | | | | | | | | | Sponsored by: DARPA External-issue: https://reviews.freebsd.org/D24656 FreeBSD-commit: freebsd/freebsd@a431c095d32df45a31faad8382b9bc712480e27e Authored by: jhb <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Ryan Moeller <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10344
* freebsd: return EISDIR for read(2) on directoriesKyle Evans2020-05-161-0/+6
| | | | | | | | | | | | | This is arguably a change for internal consistency within OpenZFS, as the Linux implementation will reject read(2) on directories with EISDIR. It's not unreasonable for read(2) to do something here on FreeBSD, but we don't currently copy out anything useful anyways so start rejecting it with the appropriate error. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Kyle Evans <[email protected]> Closes #10338
* Fix ZVOL_DIRRyan Moeller2020-05-161-0/+1
| | | | | | | | | | We only use ZVOL_DIR on FreeBSD, and on FreeBSD it isn't correct. Move the definition to the file where it is needed, and define it as /dev/zvol/. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10337
* Fix error handling in receive_writer_thread()Matthew Ahrens2020-05-141-1/+2
| | | | | | | | | | | | | | | | | | | | | | If `receive_writer_thread()` gets an error from `receive_process_record()`, it should be saved in `rwa->err` so that we will stop processing records, and the main thread will notice that the receive has failed. When an error is first encountered, this happens correctly. However, if there are more records to dequeue, the next time through the loop we will reset `rwa->err` to zero, allowing us to try to process the following record (2 after the failed record). Depending on what types of records remain, we may incorrectly complete the receive "successfully", but without actually having processed all the records. The fix is to only set `rwa->err` if we got a *non-zero* error. This bug was introduced by #10099 "Improve zfs receive performance by batching writes". Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Paul Dagnelie <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #10320
* Fix VN_OPEN_INVFS typoyparitcher2020-05-141-1/+1
| | | | | | | | | The VN_OPEN_INVFS literal is in the wrong field. Reviewed-by: Matt Macy <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: yparitcher <[email protected]> Closes #10322
* Fix abd_enter/exit_critical wrappersBrian Behlendorf2020-05-143-26/+2
| | | | | | | | | | | | | | | | Commit fc551d7 introduced the wrappers abd_enter_critical() and abd_exit_critical() to mark critical sections. On Linux these are implemented with the local_irq_save() and local_irq_restore() macros which set the 'flags' argument when saving. By wrapping them with a function the local variable is no longer set by the macro and is no longer properly restored. Convert abd_enter_critical() and abd_exit_critical() to macros to resolve this issue and ensure the flags are properly restored. Reviewed-by: Matthew Ahrens <[email protected]> Reviewed-by: Brian Atkinson <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10332
* Upstream: add missing thread_exit()Jorgen Lundman2020-05-144-0/+8
| | | | | | | | Undo FreeBSD wrapper for thread_create() added to call thread_exit. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Jorgen Lundman <[email protected]> Closes #10314
* remove unneeded member drc_err of dmu_recv_cookie_tMatthew Ahrens2020-05-141-7/+5
| | | | | | | | The member drc_err of dmu_recv_cookie_t is used only locally in receive_read, so we can replace it with a local variable. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #10319
* Resilver restarts unnecessarily when it encounters errorsJohn Poduska2020-05-132-2/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | When a resilver finishes, vdev_dtl_reassess is called to hopefully excise DTL_MISSING (amongst other things). If there are errors during the resilver, they are tracked in DTL_SCRUB, as spelled out in the block comment in vdev.c. DTL_SCRUB is in-core only, so it can only be used if the pool was online for the whole resilver. This state is tracked with the spa_scrub_started flag, which only gets set when the scan is initialized. Unfortunately, this flag gets cleared right before vdev_dtl_reassess gets called, so if there are any errors during the scan, DTL_MISSING will never get excised and the resilver will just continually restart. This fix simply moves clearing that flag until after the call to vdev_dtl_reasses. In addition, if a pool is imported and already has scn_errors > 0, this change will restart the resilver immediately instead of doing the rest of the scan and then restarting it from the beginning. On the other hand, if scn_errors == 0 at import, then no errors have been encountered so far, so the spa_scrub_started flag can be safely set. A test has been added to verify that resilver does not restart when relevant DTL's are available. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Paul Zuchowski <[email protected]> Signed-off-by: John Poduska <[email protected]> Closes #10291
* Combine OS-independent ABD Code into Common Source FileBrian Atkinson2020-05-108-2021/+1454
| | | | | | | | | | | | | | | | | Reorganizing ABD code base so OS-independent ABD code has been placed into a common abd.c file. OS-dependent ABD code has been left in each OS's ABD source files, and these source files have been renamed to abd_os. The OS-independent ABD code is now under: module/zfs/abd.c With the OS-dependent code in: module/os/linux/zfs/abd_os.c module/os/freebsd/zfs/abd_os.c Reviewed-by: Matthew Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Brian Atkinson <[email protected]> Closes #10293
* Improvements on persistent L2ARCGeorge Amanakis2020-05-071-75/+159
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Functional changes: We implement refcounts of log blocks and their aligned size on the cache device along with two corresponding arcstats. The refcounts are reflected in the header of the device and provide valuable information as to whether log blocks are accounted for correctly. These are dynamically adjusted as log blocks are committed/evicted. zdb also uses this information in the device header and compares it to the corresponding values as reported by dump_l2arc_log_blocks() which emulates l2arc_rebuild(). If the refcounts saved in the device header report higher values, zdb exits with an error. For this feature to work correctly there should be no active writes on the device. This is also employed in the tests of persistent L2ARC. We extend the structure of the cache device header by adding the two new variables mirroring the refcounts after the existing variables to preserve backward compatibility in terms of persistent L2ARC. 1) a new arcstat "l2_log_blk_asize" and refcount "l2ad_lb_asize" which reflect the total aligned size of log blocks on the device. This is also reflected in the header of the cache device as "dh_lb_asize". 2) a new arcstat "l2arc_log_blk_count" and refcount "l2ad_lb_count" which reflect the total number of L2ARC log blocks present on cache devices. It is also reflected in the header of the cache device as "dh_lb_count". In l2arc_rebuild_vdev() if the amount of committed log entries in a log block is 0 and the device header is valid we update the device header. This will facilitate trimming of the whole device in this case when TRIM for L2ARC is implemented. Improve loop protection in l2arc_rebuild() by using the starting offset of the payload of each log block instead of the starting offset of the log block. If the zio in l2arc_write_buffers() fails, restore the lbps array in the header of the device to its previous state in l2arc_write_done(). If l2arc_rebuild() ends the rebuild process without restoring any L2ARC log blocks in ARC and without any other error, this means that the lbps array in the header is pointing to non-existent or invalid log blocks. Reset the device header in this case. In l2arc_rebuild() change the zfs_dbgmsg messages to spa_history_log_internal() making them user visible with zpool history command. Non-functional changes: Make the first test in persistent L2ARC use `zdb -lll` to increase coverage in `zdb.c`. Rename psize with asize when referring to log blocks, since L2ARC_SET_PSIZE stores the vdev aligned size for log blocks. Also rename dh_log_blk_entries to dh_log_entries to make it clear that it is a mirror of l2ad_log_entries. Added comments for both changes. Fix inaccurate comments for example in l2arc_log_blk_restore(). Add asserts at the end in l2arc_evict() and l2arc_write_buffers(). Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: George Amanakis <[email protected]> Closes #10228
* Add support for boot environment data to be stored in the labelPaul Dagnelie2020-05-074-10/+214
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Modern bootloaders leverage data stored in the root filesystem to enable some of their powerful features. GRUB specifically has a grubenv file which can store large amounts of configuration data that can be read and written at boot time and during normal operation. This allows sysadmins to configure useful features like automated failover after failed boot attempts. Unfortunately, due to the Copy-on-Write nature of ZFS, the standard behavior of these tools cannot handle writing to ZFS files safely at boot time. We need an alternative way to store data that allows the bootloader to make changes to the data. This work is very similar to work that was done on Illumos to enable similar functionality in the FreeBSD bootloader. This patch is different in that the data being stored is a raw grubenv file; this file can store arbitrary variables and values, and the scripting provided by grub is powerful enough that special structures are not required to implement advanced behavior. We repurpose the second padding area in each label to store the grubenv file, protected by an embedded checksum. We add two ioctls to get and set this data, and libzfs_core and libzfs functions to access them more easily. There are no direct command line interfaces to these functions; these will be added directly to the bootloader utilities. Reviewed-by: Pavel Zakharov <[email protected]> Reviewed-by: Matthew Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Paul Dagnelie <[email protected]> Closes #10009
* Enable splitting mirrors with indirect vdevsGeorge Amanakis2020-05-062-4/+11
| | | | | | | | | | | When a top-level vdev is removed from a pool it is converted to an indirect vdev. Until now splitting such mirrored pools was not possible with zpool split. This patch enables handling of indirect vdevs and splitting of those pools with zpool split. Reviewed-by: Matthew Ahrens <[email protected]> Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: George Amanakis <[email protected]> Closes #10283
* taskq: Don't leak system_delay_taskq on FreeBSDRyan Moeller2020-05-051-0/+1
| | | | | | | | | Adds a missing taskq_destroy() call. Reported by: Jorgen Lundman <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10292
* Avoid the GEOM topology lock recursion when autoexpanding a poolRyan Moeller2020-05-041-3/+10
| | | | | | | | | | | | | | | | | | The steps to reproduce the problem: mdconfig -a -t swap -s 3g -u 0 gpart create -s GPT md0 gpart add -t freebsd-zfs -s 1g md0 zpool create -o autoexpand=on foo md0p1 gpart resize -i 1 -s 2g md0 Authored by: pjd <[email protected]> FreeBSD-commit: freebsd/freebsd@bccd2db598ede073d6d06781a5fd3b119c08aa81 Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Ported-by: Ryan Moeller <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10270
* Update FreeBSD SPL atomicsRyan Moeller2020-05-041-35/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sync up with the following changes from FreeBSD: ZFS: add emulation of atomic_swap_64 and atomic_load_64 Some 32-bit platforms do not provide 64-bit atomic operations that ZFS requires, either in userland or at all. We emulate those operations for those platforms using a mutex. That is not entirely correct and it's very efficient. Besides, the loads are plain loads, so torn values are possible. Nevertheless, the emulation seems to work for some definition of work. This change adds atomic_swap_64, which is already used in ZFS code, and atomic_load_64 that can be used to prevent torn reads. Authored by: avg <[email protected]> FreeBSD-commit: freebsd/freebsd@3458e5d1e6354123ec2b0953d29f98126aa442e cleanup of illumos compatibility atomics atomic_cas_32 is implemented using atomic_fcmpset_32 on all platforms. Ditto for atomic_cas_64 and atomic_fcmpset_64 on platforms that have it. The only exception is sparc64 that provides MD atomic_cas_32 and atomic_cas_64. This is slightly inefficient as fcmpset reports whether the operation updated the target and that information is not needed for cas. Nevertheless, there is less code to maintain and to add for new platforms. Also, the operations are done inline now as opposed to function calls before. atomic_add_64_nv is implemented using atomic_fetchadd_64 on platforms that provide it. casptr, cas32, atomic_or_8, atomic_or_8_nv are completely removed as they have no users. atomic_mtx that is used to emulate 64-bit atomics on platforms that lack them is defined only on those platforms. As a result, platform specific opensolaris_atomic.S files have lost most of their code. The only exception is i386 where the compat+contrib code provides 64-bit atomics for userland use. That code assumes availability of cmpxchg8b instruction. FreeBSD does not have that assumption for i386 userland and does not provide 64-bit atomics. Hopefully, this can and will be fixed. Authored by: avg <[email protected]> FreeBSD-commit: freebsd/freebsd@e9642c209b4413f6afb41d3b2607c51d80a1a34 emulate illumos membar_producer with atomic_thread_fence_rel membar_producer is supposed to be a store-store barrier. Also, in the code that FreeBSD has ported from illumos membar_producer is used only with regular stores to regular memory (with respect to caching). We do not have an MI primitive for the store-store barrier, so atomic_thread_fence_rel is the closest we have as it provides (load | store) -> store barrier. Previously, membar_producer was an empty function call on all 32-bit arm-s, 32-bit powerpc, riscv and all mips variants. I think that it was inadequate. On other platforms, such as amd64, arm64, i386, powerpc64, sparc64, membar_producer was implemented using stronger primitives than required for a store-store barrier with respect to regular memory access. For example, it used sfence on amd64 and lock-ed nop in i386 (despite TSO). On powerpc64 we now use recommended lwsync instead of eieio. On sparc64 FreeBSD uses TSO mode. On arm64/aarch64 we now use dmb sy instead of dmb ish. Not sure if this is an improvement, actually. After this change we can drop opensolaris_atomic.S for aarch64, amd64, powerpc64 and sparc64 as all required atomic operations have either direct or light-weight mapping to FreeBSD native atomic operations. Discussed with: kib Authored by: avg <[email protected]> FreeBSD-commit: freebsd/freebsd@50cdda62fced8d21e45858e01dc375a10f1749e fix up r353340, don't assume that fcmpset has strong semantics fcmpset can have two kinds of semantics, weak and strong. For practical purposes, strong semantics means that if fcmpset fails then the reported current value is always different from the expected value. Weak semantics means that the reported current value may be the same as the expected value even though fcmpset failed. That's a so called "sporadic" failure. I originally implemented atomic_cas expecting strong semantics, but many platforms actually have weak one. Reported by: pkubaj (not confirmed if same issue) Discussed with: kib, mjg Authored by: avg <[email protected]> FreeBSD-commit: freebsd/freebsd@238787c74e737e271f17330fbad900acc35651c [PowerPC] [MIPS] Implement 32-bit kernel emulation of atomic64 operations This is a lock-based emulation of 64-bit atomics for kernel use, split off from an earlier patch by jhibbits. This is needed to unblock future improvements that reduce the need for locking on 64-bit platforms by using atomic updates. The implementation allows for future integration with userland atomic64, but as that implies going through sysarch for every use, the current status quo of userland doing its own locking may be for the best. Submitted by: jhibbits (original patch), kevans (mips bits) Reviewed by: jhibbits, jeff, kevans Authored by: bdragon <[email protected]> Differential Revision: https://reviews.freebsd.org/D22976 FreeBSD-commit: freebsd/freebsd@db39dab3a896b3d98e588736e9a2b4ddaeb31f1 Remove sparc64 kernel support Remove all sparc64 specific files Remove all sparc64 ifdefs Removee indireeect sparc64 ifdefs Authored by: imp <[email protected]> FreeBSD-commit: freebsd/freebsd@48b94864c51253da92e4444f0074eec36ef391f Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Ported-by: Ryan Moeller <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10250
* OpenZFS 6765 - zfs_zaccess_delete() comments do not accuratelyPaul B. Henson2020-04-301-22/+21
| | | | | | | | | | | | | | | | | | reflect delete permissions for ACLs Authored by: Kevin Crowe <[email protected]> Reviewed by: Gordon Ross <[email protected]> Reviewed by: Yuri Pankov <[email protected]> Reviewed by: Brian Behlendorf <[email protected]> Approved by: Richard Lowe <[email protected]> Ported-by: Paul B. Henson <[email protected]> Porting Notes: * Only comments are updated OpenZFS-issue: https://www.illumos.org/issues/6765 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/da412744bc Closes #10266
* OpenZFS 6762 - POSIX write should imply DELETE_CHILD on directoriesPaul B. Henson2020-04-301-80/+147
| | | | | | | | | | | | | | | - and some additional considerations Authored by: Kevin Crowe <[email protected]> Reviewed by: Gordon Ross <[email protected]> Reviewed by: Yuri Pankov <[email protected]> Reviewed by: Brian Behlendorf <[email protected]> Approved by: Richard Lowe <[email protected]> Ported-by: Paul B. Henson <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/6762 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1eb4e906ec Closes #10266
* OpenZFS 8984 - fix for 6764 breaks ACL inheritancePaul B. Henson2020-04-301-11/+29
| | | | | | | | | | | | | | Authored by: Dominik Hassler <[email protected]> Reviewed by: Sam Zaydel <[email protected]> Reviewed by: Paul B. Henson <[email protected]> Reviewed by: Prakash Surya <[email protected]> Reviewed by: Brian Behlendorf <[email protected]> Approved by: Matthew Ahrens <[email protected]> Ported-by: Paul B. Henson <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/8984 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/e9bacc6d1a Closes #10266
* OpenZFS 6764 - zfs issues with inheritance flags during chmod(2)Paul B. Henson2020-04-301-85/+79
| | | | | | | | | | | | | | | with aclmode=passthrough Authored by: Albert Lee <[email protected]> Reviewed by: Gordon Ross <[email protected]> Reviewed by: Yuri Pankov <[email protected]> Reviewed by: Brian Behlendorf <[email protected]> Approved by: Richard Lowe <[email protected]> Ported-by: Paul B. Henson <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/6764 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/de0f1ddb59 Closes #10266
* OpenZFS 3254 - add support in zfs for aclmode=restrictedPaul B. Henson2020-04-302-7/+8
| | | | | | | | | | | | | Authored-by: Paul B. Henson <[email protected]> Reviewed by: Albert Lee <[email protected]> Reviewed by: Gordon Ross <[email protected]> Reviewed by: Brian Behlendorf <[email protected]> Approved by: Richard Lowe <[email protected]> Ported-by: Paul B. Henson <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/3254 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/71dbfc287c Closes #10266
* OpenZFS 742 - Resurrect the ZFS "aclmode" property OpenZFS 664 - Umask ↵Paul B. Henson2020-04-303-73/+115
| | | | | | | | | | | | | | | | | | | | | | | masking "deny" ACL entries OpenZFS 279 - Bug in the new ACL (post-PSARC/2010/029) semantics Porting notes: * Updated zfs_acl_chmod to take 'boolean_t isdir' as first parameter rather than 'zfsvfs_t *zfsvfs' * zfs man pages changes mixed between zfs and new zfsprops man pages Reviewed by: Aram Hvrneanu <[email protected]> Reviewed by: Gordon Ross <[email protected]> Reviewed by: Robert Gordon <[email protected]> Reviewed by: [email protected] Reviewed by: Brian Behlendorf <[email protected]> Approved by: Garrett D'Amore <[email protected]> Ported-by: Paul B. Henson <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/742 OpenZFS-issue: https://www.illumos.org/issues/664 OpenZFS-issue: https://www.illumos.org/issues/279 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/a3c49ce110 Closes #10266
* Add longjmp support for Thumb-2Brian Behlendorf2020-04-291-1/+18
| | | | | | | | | | | When a Thumb-2 kernel is being used, then longjmp must be implemented using the Thumb-2 instruction set in module/lua/setjmp/setjmp_arm.S. Original-patch-by: @jsrlabs Reviewed-by: @awehrfritz Signed-off-by: Brian Behlendorf <[email protected]> Closes #7408 Closes #9957 Closes #9967
* Add missing zfs_refcount_destroy() in key_mapping_rele()George Amanakis2020-04-281-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Otherwise when running with reference_tracking_enable=TRUE mounting and unmounting an encrypted dataset panics with: Call Trace: dump_stack+0x66/0x90 slab_err+0xcd/0xf2 ? __kmalloc+0x174/0x260 ? __kmem_cache_shutdown+0x158/0x240 __kmem_cache_shutdown.cold+0x1d/0x115 shutdown_cache+0x11/0x140 kmem_cache_destroy+0x210/0x230 spl_kmem_cache_destroy+0x122/0x3e0 [spl] zfs_refcount_fini+0x11/0x20 [zfs] spa_fini+0x4b/0x120 [zfs] zfs_kmod_fini+0x6b/0xa0 [zfs] _fini+0xa/0x68c [zfs] __x64_sys_delete_module+0x19c/0x2b0 do_syscall_64+0x5b/0x1a0 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Reviewed-By: Brian Behlendorf <[email protected]> Reviewed-By: Tom Caputi <[email protected]> Signed-off-by: George Amanakis <[email protected]> Closes #10246
* Fix zlib leak on FreeBSDRyan Moeller2020-04-281-1/+1
| | | | | | | | | | | | | zlib_inflateEnd was accidentally a wrapper for inflateInit instead of inflateEnd, and hilarity ensues. Fix the typo so we free memory instead of allocating more. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Matthew Ahrens <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10225 Closes #10252
* Fix missing ivset guid with resumed raw base recvTom Caputi2020-04-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | This patch corrects a bug introduced in 61152d1069. When resuming a raw base receive, the dmu_recv code always sets drc->drc_fromsnapobj to the object ID of the previous snapshot. For incrementals, this is correct, but for base sends, this should be left at 0. The presence of this ID eventually allows a check to run which determines whether or not the incoming stream and the previous snapshot have matching IVset guids. This check fails becuase it is not meant to run when there is no previous snapshot. When it does fail, the user receives an error stating that the incoming stream has the problem outlined in errata 4. This patch corrects this issue by simply ensuring drc->drc_fromsnapobj is left as 0 for base receives. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Tom Caputi <[email protected]> Closes #10234 Closes #10239
* Remove deduplicated send/receive codeMatthew Ahrens2020-04-234-377/+19
| | | | | | | | | | | | | | | | | | | | | | | | | Deduplicated send streams (i.e. `zfs send -D` and `zfs receive` of such streams) are deprecated. Deduplicated send streams can be received by first converting them to non-deduplicated with the `zstream redup` command. This commit removes the code for sending and receiving deduplicated send streams. `zfs send -D` will now print a warning, ignore the `-D` flag, and generate a regular (non-deduplicated) send stream. `zfs receive` of a deduplicated send stream will print an error message and fail. The resulting code simplification (especially in the kernel's support for receiving dedup streams) should help enable future performance enhancements. Several new tests are added which leverage `zstream redup`. Reviewed-by: Paul Dagnelie <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Issue #7887 Issue #10117 Issue #10156 Closes #10212
* Use a struct to organize metaslab-group-allocator fieldsMatthew Ahrens2020-04-222-53/+51
| | | | | | | | | | | | | | | | | | | | Each metaslab group (of which there is one per top-level vdev) has several (4, by default) "metaslab group allocators". Each "allocator" has its own metaslab that it prefers to allocate from (the "primary" allocator), and each can perform allocations concurrently with the other allocators. In addition to the primary metaslab, there are several other fields that need to be tracked separately for each allocator. These are currently stored as several arrays in the metaslab_group_t, each array indexed by allocator number. This change organizes all the metaslab-group-allocator-specific fields into a new struct, metaslab_group_allocator_t. The metaslab_group_t now needs only one array indexed by the allocator number - which contains the metaslab_group_allocator_t's. Reviewed-by: Paul Dagnelie <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #10213
* Fix zfs send progress reportingMatthew Ahrens2020-04-201-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The progress of a send is supposed to be reported by `zfs send -v`, but it is not. This works by creating a new user thread (with pthread_create()) which does ZFS_IOC_SEND_PROGRESS ioctls to check how much progress has been made. This IOCTL finds the specified send (since there may be multiple concurrent sends in the system). The IOCTL also checks that the specified send was started by the current process. On Linux, different threads of the same process are represented as different `struct task_struct`s (and, confusingly, have different PID's). To check if if two threads are in the same process, we need to check if they have the same `struct task_struct:group_leader`. We used to to this correctly, but it was inadvertently changed by 30af21b02569 (Redacted Send) to simply check if the current `struct task_struct` is the one that started the send. This commit changes the code back to checking if the send was started by a `struct task_struct` with the same `group_leader` as the calling thread. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Chris Wedgwood <[email protected]> Reviewed-by: Paul Dagnelie <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #10215 Closes #10216
* Use new FreeBSD API to largely eliminate object lockingMatthew Macy2020-04-172-39/+40
| | | | | | | | Propagate changes in HEAD that mostly eliminate object locking. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10205
* Persistent L2ARC minor fixesGeorge Amanakis2020-04-171-8/+6
| | | | | | | | | | | Minor fixes on persistent L2ARC improving code readability and fixing a typo in zdb.c when byte-swapping a log block. It also improves the pesist_l2arc_007_pos.ksh test by giving it more time to retrieve log blocks on the cache device. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Adam D. Moss <[email protected]> Signed-off-by: George Amanakis <[email protected]> Closes #10210
* Update FreeBSD tunablesRyan Moeller2020-04-154-40/+67
| | | | | | | | Remove some obsolete legacy compat, rename some misnamed, and add some missing tunables for FreeBSD. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10203
* Add FreeBSD support to OpenZFSMatthew Macy2020-04-1456-4/+33464
| | | | | | | | | | | | | | | | | | Add the FreeBSD platform code to the OpenZFS repository. As of this commit the source can be compiled and tested on FreeBSD 11 and 12. Subsequent commits are now required to compile on FreeBSD and Linux. Additionally, they must pass the ZFS Test Suite on FreeBSD which is being run by the CI. As of this commit 1230 tests pass on FreeBSD and there are no unexpected failures. Reviewed-by: Sean Eric Fagan <[email protected]> Reviewed-by: Jorgen Lundman <[email protected]> Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Co-authored-by: Ryan Moeller <[email protected]> Signed-off-by: Matt Macy <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #898 Closes #8987
* Disable user space reference trackingBrian Behlendorf2020-04-131-5/+6
| | | | | | | | | | | | | | The memory and cpu cost of reference count tracking with the current implementation is significant. For this reason it has always been disabled by default for the kmods. Apply this same default to user space so ztest doesn't always incur this performance penalty. Our intention is to re-enable this by default for ztest once the code has been optimized. Since we expect to at some point provide a FUSE implementation we wouldn't want this enabled by default for libzpool. Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10189
* zvol_write() can use dmu_tx_hold_write_by_dnode()Matthew Ahrens2020-04-101-1/+1
| | | | | | | | | | | | | We can improve the performance of writes to zvols by using dmu_tx_hold_write_by_dnode() instead of dmu_tx_hold_write(). This reduces lock contention on the first block of the dnode object, and also reduces the amount of CPU needed. The benefit will be highest with multi-threaded async writes (i.e. writes that don't call zil_commit()). Reviewed-by: Jorgen Lundman <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Nguyen <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #10184
* Persistent L2ARCGeorge Amanakis2020-04-104-50/+1415
| | | | | | | | | | | | | | | | | | | | | This commit makes the L2ARC persistent across reboots. We implement a light-weight persistent L2ARC metadata structure that allows L2ARC contents to be recovered after a reboot. This significantly eases the impact a reboot has on read performance on systems with large caches. Reviewed-by: Matthew Ahrens <[email protected]> Reviewed-by: George Wilson <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Co-authored-by: Saso Kiselkov <[email protected]> Co-authored-by: Jorgen Lundman <[email protected]> Co-authored-by: George Amanakis <[email protected]> Ported-by: Yuxuan Shui <[email protected]> Signed-off-by: George Amanakis <[email protected]> Closes #925 Closes #1823 Closes #2672 Closes #3744 Closes #9582
* Don't ignore zfs_arc_max below allmem/32Ryan Moeller2020-04-092-14/+30
| | | | | | | | | | | Set arc_c_min before arc_c_max so that when zfs_arc_min is set lower than the default allmem/32 zfs_arc_max can also be set lower. Add warning messages when tunables are being ignored. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10157 Closes #10158
* Add separate field for indicating that spa is in middle of splitMatthew Macy2020-04-091-0/+2
| | | | | | | | | | | By default it's not possible to open a device already owned by an active vdev. It's necessary to make an exception to this for vdev split. The FreeBSD platform code will make an exception if spa_is splitting is set to to true. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10178
* Linux 5.7 compat: blk_alloc_queue()Brian Behlendorf2020-04-091-24/+24
| | | | | | | | | | | | | | | | | | Commit https://github.com/torvalds/linux/commit/3d745ea5 simplified the blk_alloc_queue() interface by updating it to take the request queue as an argument. Add a wrapper function which accepts the new arguments and internally uses the available interfaces. Other minor changes include increasing the Linux-Maximum to 5.6 now that 5.6 has been released. It was not bumped to 5.7 because this release has not yet been finalized and is still subject to change. Added local 'struct zvol_state_os *zso' variable to zvol_alloc. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10181 Closes #10187
* Use vn_io_fault_uiomove on FreeBSD to avoid potential deadlockMatthew Macy2020-04-081-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added to prevent a possible deadlock, the following comments from FreeBSD explain the issue. The comment describing vn_io_fault_uiomove: /* * Helper function to perform the requested uiomove operation using * the held pages for io->uio_iov[0].iov_base buffer instead of * copyin/copyout. Access to the pages with uiomove_fromphys() * instead of iov_base prevents page faults that could occur due to * pmap_collect() invalidating the mapping created by * vm_fault_quick_hold_pages(), or pageout daemon, page laundry or * object cleanup revoking the write access from page mappings. * * Filesystems specified MNTK_NO_IOPF shall use vn_io_fault_uiomove() * instead of plain uiomove(). */ This used for vn_io_fault which has the following motivation: /* * The vn_io_fault() is a wrapper around vn_read() and vn_write() to * prevent the following deadlock: * * Assume that the thread A reads from the vnode vp1 into userspace * buffer buf1 backed by the pages of vnode vp2. If a page in buf1 is * currently not resident, then system ends up with the call chain * vn_read() -> VOP_READ(vp1) -> uiomove() -> [Page Fault] -> * vm_fault(buf1) -> vnode_pager_getpages(vp2) -> VOP_GETPAGES(vp2) * which establishes lock order vp1->vn_lock, then vp2->vn_lock. * If, at the same time, thread B reads from vnode vp2 into buffer buf2 * backed by the pages of vnode vp1, and some page in buf2 is not * resident, we get a reversed order vp2->vn_lock, then vp1->vn_lock. * * To prevent the lock order reversal and deadlock, vn_io_fault() does * not allow page faults to happen during VOP_READ() or VOP_WRITE(). * Instead, it first tries to do the whole range i/o with pagefaults * disabled. If all pages in the i/o buffer are resident and mapped, * VOP will succeed (ignoring the genuine filesystem errors). * Otherwise, we get back EFAULT, and vn_io_fault() falls back to do * i/o in chunks, with all pages in the chunk prefaulted and held * using vm_fault_quick_hold_pages(). * * Filesystems using this deadlock avoidance scheme should use the * array of the held pages from uio, saved in the curthread->td_ma, * instead of doing uiomove(). A helper function * vn_io_fault_uiomove() converts uiomove request into * uiomove_fromphys() over td_ma array. * * Since vnode locks do not cover the whole i/o anymore, rangelocks * make the current i/o request atomic with respect to other i/os and * truncations. */ Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matt Macy <[email protected]> Closes #10177
* Finish refactoring for ZFS_MODULE_PARAM_CALLRyan Moeller2020-04-071-1/+1
| | | | | | | | | | | | | | | Linux and FreeBSD have different parameters for tunable proc handler. This has prevented FreeBSD from implementing the ZFS_MODULE_PARAM_CALL macro. To complete the sharing of ZFS_MODULE_PARAM_CALL declarations, create per-platform definitions of the parameter list, ZFS_MODULE_PARAM_ARGS. With the declarations wired up we discovered an incorrect scope prefix for spa_slop_shift, so this is now fixed. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10179
* Add 'zfs wait' commandPaul Dagnelie2020-04-016-10/+234
| | | | | | | | | | | | | | | | | | | | | | Add a mechanism to wait for delete queue to drain. When doing redacted send/recv, many workflows involve deleting files that contain sensitive data. Because of the way zfs handles file deletions, snapshots taken quickly after a rm operation can sometimes still contain the file in question, especially if the file is very large. This can result in issues for redacted send/recv users who expect the deleted files to be redacted in the send streams, and not appear in their clones. This change duplicates much of the zpool wait related logic into a zfs wait command, which can be used to wait until the internal deleteq has been drained. Additional wait activities may be added in the future. Reviewed-by: Matthew Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Gallagher <[email protected]> Signed-off-by: Paul Dagnelie <[email protected]> Closes #9707
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-24 10:45:02 +0000