aboutsummaryrefslogtreecommitdiffstats
path: root/module
Commit message (Collapse)AuthorAgeFilesLines
* Undo c89 workarounds to match with upstreamDon Brady2017-11-0447-762/+423
| | | | | | | | | With PR 5756 the zfs module now supports c99 and the remaining past c89 workarounds can be undone. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Don Brady <[email protected]> Closes #6816
* OpenZFS 8081 - Compiler warnings in zdbBrian Behlendorf2017-10-277-61/+95
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix compiler warnings in zdb. With these changes, FreeBSD can compile zdb with all compiler warnings enabled save -Wunused-parameter. usr/src/cmd/zdb/zdb.c usr/src/cmd/zdb/zdb_il.c usr/src/uts/common/fs/zfs/sys/sa.h usr/src/uts/common/fs/zfs/sys/spa.h Fix numerous warnings, including: * const-correctness * shadowing global definitions * signed vs unsigned comparisons * missing prototypes, or missing static declarations * unused variables and functions * Unreadable array initializations * Missing struct initializers usr/src/cmd/zdb/zdb.h Add a header file to declare common symbols usr/src/lib/libzpool/common/sys/zfs_context.h usr/src/uts/common/fs/zfs/arc.c usr/src/uts/common/fs/zfs/dbuf.c usr/src/uts/common/fs/zfs/spa.c usr/src/uts/common/fs/zfs/txg.c Add a function prototype for zk_thread_create, and ensure that every callback supplied to this function actually matches the prototype. usr/src/cmd/ztest/ztest.c usr/src/uts/common/fs/zfs/sys/zil.h usr/src/uts/common/fs/zfs/zfs_replay.c usr/src/uts/common/fs/zfs/zvol.c Add a function prototype for zil_replay_func_t, and ensure that every function of this type actually matches the prototype. usr/src/uts/common/fs/zfs/sys/refcount.h Change FTAG so it discards any constness of __func__, necessary since existing APIs expect it passed as void *. Porting Notes: - Many of these fixes have already been applied to Linux. For consistency the OpenZFS version of a change was applied if the warning was addressed in an equivalent but different fashion. Reviewed by: Matthew Ahrens <[email protected]> Reviewed by: Prakash Surya <[email protected]> Authored by: Alan Somers <[email protected]> Approved by: Richard Lowe <[email protected]> Ported-by: Brian Behlendorf <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/8081 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/843abe1b8a Closes #6787
* ZFS send fails to dump objects larger than 128PiBLOLi2017-10-263-17/+22
| | | | | | | | | | | When dumping objects larger than 128PiB it's possible for do_dump() to miscalculate the FREE_RECORD offset due to an integer overflow condition: this prevents the receiving end from correctly restoring the dumped object. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Fabian Grünbichler <[email protected]> Signed-off-by: loli10K <[email protected]> Closes #6760
* OpenZFS 8558, 8602 - lwp_create() returns EAGAINBrian Behlendorf2017-10-262-9/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 8558 lwp_create() returns EAGAIN on system with more than 80K ZFS filesystems On a system with more than 80K ZFS filesystems, we've seen cases where lwp_create() will start to fail by returning EAGAIN. The problem being, for each of those 80K ZFS filesystems, a taskq will be created for each dataset as part of the ZIL for each dataset. Porting Notes: - The new nomem taskq kstat was dropped. - Added module options and documentation for new tunings zfs_zil_clean_taskq_nthr_pct, zfs_zil_clean_taskq_minalloc, zfs_zil_clean_taskq_maxalloc, and zfs_sync_taskq_batch_pct. Reviewed by: George Wilson <[email protected]> Reviewed by: Sebastien Roy <[email protected]> Approved by: Robert Mustacchi <[email protected]> Authored by: Prakash Surya <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Chris Dunlop <[email protected]> Ported-by: Brian Behlendorf <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/8558 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/216d772 8602 remove unused "dp_early_sync_tasks" field from "dsl_pool" structure Reviewed by: Serapheim Dimitropoulos <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Approved by: Robert Mustacchi <[email protected]> Authored by: Prakash Surya <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Chris Dunlop <[email protected]> Ported-by: Brian Behlendorf <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/8602 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/2bcb545 Closes #6779
* Added no_scrub_restart flag to zpool reopenArkadiusz Bubała2017-10-261-9/+33
| | | | | | | | | | | | | | | | | | | | | | | Added -n flag to zpool reopen that allows a running scrub operation to continue if there is a device with Dirty Time Log. By default if a component device has a DTL and zpool reopen is executed all running scan operations will be restarted. Added functional tests for `zpool reopen` Tests covers following scenarios: * `zpool reopen` without arguments, * `zpool reopen` with pool name as argument, * `zpool reopen` while scrubbing, * `zpool reopen -n` while scrubbing, * `zpool reopen -n` while resilvering, * `zpool reopen` with bad arguments. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tom Caputi <[email protected]> Signed-off-by: Arkadiusz Bubała <[email protected]> Closes #6076 Closes #6746
* Emit history events for 'zpool create'Brian Behlendorf2017-10-232-20/+23
| | | | | | | | | | | | | | | | | | | History commands and events were being suppressed for the 'zpool create' command since the history object did not yet exist. Create the object earlier so this history doesn't get lost. Split the pool_destroy event in to pool_destroy and pool_export so they may be distinguished. Updated events_001_pos and events_002_pos test cases. They now check for the expected history events and were reworked to be more reliable. Reviewed-by: Nathaniel Clark <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6712 Closes #6486
* Support integration with new QAT productswli52017-10-201-3/+5
| | | | | | | | | | | | Support integration with new QAT products: Intel(R) C62x Chipset, or Atom(R) C3000 Processor Product Family SoC: 1. Detect new file name in auto-conf. 2. Change MAX_INSTANCES to 48. 3. Change "num_inst" to U16 to clean a build warning. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Weigang Li <[email protected]> Closes #6767
* Remove vn_rename and vn_remove dependencyBrian Behlendorf2017-10-191-9/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | The only place vn_rename and vn_remove are used is when writing out an updated pool configuration file. By truncating the file instead of renaming and removing it we can avoid having to implement these interfaces entirely. Functionally an empty cache file is treated the same as a missing cache file. This is particularly advantageous because the Linux kernel has never provided a way to reliably implement vn_rename and vn_remove. The cachefile_004_pos.ksh test case was updated to understand that an empty cache file is the same as a missing one. The zfs-import-* systemd service files were not updated to use ConditionFileNotEmpty in place of ConditionPathExists. This means that after exporting all pools and rebooting new pools will not the scanned for on the next boot. This small change should not impact normal usage since pools are not exported as part of a normal shutdown. Documentation was updated accordingly. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Arkadiusz Bubała <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes zfsonlinux/spl#648 Closes #6753
* Fix ASSERT in dmu_free_long_object_raw()Tom Caputi2017-10-181-3/+4
| | | | | | | | | This small patch fixes an issue where dmu_free_long_object_raw() calls dnode_hold() after freeing the dnode a line above. Reviewed-by: Jorgen Lundman <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tom Caputi <[email protected]> Closes #6766
* Post-Encryption FollowupBrian Behlendorf2017-10-1315-329/+518
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR includes fixes for bugs and documentation issues found after the encryption patch was merged and general code improvements for long-term maintainability. Reviewed-by: Jorgen Lundman <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tom Caputi <[email protected]> Issue #6526 Closes #6639 Closes #6703 Cloese #6706 Closes #6714 Closes #6595
| * Fix for #6714Tom Caputi2017-10-111-2/+2
| | | | | | | | | | | | This 2 line patch fixes a possible integer overflow reported by grsec. Signed-off-by: Tom Caputi <[email protected]>
| * Fix for #6706Tom Caputi2017-10-112-18/+72
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch resolves an issue where raw sends would fail to send encryption parameters if the wrapping key was unloaded and reloaded before the data was sent and the dataset wass not an encryption root. The code attempted to lookup the values from the wrapping key which was not being initialized upon reload. This change forces the code to lookup the correct value from the encryption root's DSL Crypto Key. Unfortunately, this issue led to the on-disk DSL Crypto Key for some non-encryption root datasets being left with zeroed out encryption parameters. However, this should not present a problem since these values are never looked at and are overrwritten upon changing keys. This patch also fixes an issue where raw, resumable sends were not being cleaned up appropriately if an invalid DSL Crypto Key was received. Signed-off-by: Tom Caputi <[email protected]>
| * Fix for #6703Tom Caputi2017-10-111-18/+6
| | | | | | | | | | | | | | | | | | This patch resolves an issue where spa_keystore_change_key_sync_impl() incorrectly recursed into clone DSL Directories while recursively rewrapping encryption keys. Clones share keys with their origins, so this logic was incorrect. Signed-off-by: Tom Caputi <[email protected]>
| * Fixes for #6639Tom Caputi2017-10-116-61/+143
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Several issues were uncovered by running stress tests with zfs encryption and raw sends in particular. The issues and their associated fixes are as follows: * arc_read_done() has the ability to chain several requests for the same block of data via the arc_callback_t struct. In these cases, the ARC would only use the first request's dsobj from the bookmark to decrypt the data. This is problematic because the first request might be a prefetch zio which is able to handle the key not being loaded, while the second might use a different key that it is sure will work. The fix here is to pass the dsobj with each individual arc_callback_t so that each request can attempt to decrypt the data separately. * DRR_FREE and DRR_FREEOBJECT records in a send file were not having their transactions properly tagged as raw during raw sends, which caused a panic when the dbuf code attempted to decrypt these blocks. * traverse_prefetch_metadata() did not properly set ZIO_FLAG_SPECULATIVE when issuing prefetch IOs. * Added a few asserts and code cleanups to ensure these issues are more detectable in the future. Signed-off-by: Tom Caputi <[email protected]>
| * Encryption patch follow-upTom Caputi2017-10-1111-229/+293
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * PBKDF2 implementation changed to OpenSSL implementation. * HKDF implementation moved to its own file and tests added to ensure correctness. * Removed libzfs's now unnecessary dependency on libzpool and libicp. * Ztest can now create and test encrypted datasets. This is currently disabled until issue #6526 is resolved, but otherwise functions as advertised. * Several small bug fixes discovered after enabling ztest to run on encrypted datasets. * Fixed coverity defects added by the encryption patch. * Updated man pages for encrypted send / receive behavior. * Fixed a bug where encrypted datasets could receive DRR_WRITE_EMBEDDED records. * Minor code cleanups / consolidation. Signed-off-by: Tom Caputi <[email protected]>
| * Relax ASSERT for #6526Tom Caputi2017-10-111-1/+2
| | | | | | | | | | | | | | | | This patch resolves a minor issue where an ASSERT in metaslab_passivate() that only applies to non weight-based metaslabs was erroneously applied to all metaslabs. Signed-off-by: Tom Caputi <[email protected]>
* | SPARC optimizations for Encode()KireinaHoro2017-10-121-5/+17
| | | | | | | | | | | | | | | | Normally a SPARC processor runs in big endian mode. Save the extra labor needed for little endian machines when the target is a big endian one (sparc). Signed-off-by: Pengcheng Xu <[email protected]>
* | SPARC optimizations for SHA1Transform()KireinaHoro2017-10-121-1/+164
|/ | | | | | | | | | | Passing arguments explicitly into SHA1Transform() increases the number of registers abailable to the compiler, hence leaving more local and out registers available. The missing symbol of sha1_consts[], which prevents compiling on SPARC, is added back, which speeds up the process of utilizing the relative constants. This should fix #6738. Signed-off-by: Pengcheng Xu <[email protected]>
* Fix coverity defects: CID 147474Tobin Harding2017-10-101-1/+2
| | | | | | | | | | | | | CID 147474: Logically dead code (DEADCODE) Remove ternary operator and return `error` directly. Currently return value is derived from a ternary operator. The conditional is always true. The ternary operator is therefore redundant i.e dead code. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tobin C. Harding <[email protected]> Closes #6723
* Skip FREEOBJECTS for objects which can't existFabian Grünbichler2017-10-101-0/+16
| | | | | | | | | | | | | | | | | | When sending an incremental stream based on a snapshot, the receiving side must have the same base snapshot. Thus we do not need to send FREEOBJECTS records for any objects past the maximum one which exists locally. This allows us to send incremental streams (again) to older ZFS implementations (e.g. ZoL < 0.7) which actually try to free all objects in a FREEOBJECTS record, instead of bailing out early. Reviewed by: Paul Dagnelie <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Fabian Grünbichler <[email protected]> Closes #5699 Closes #6507 Closes #6616
* Free objects when receiving full stream as cloneFabian Grünbichler2017-10-101-1/+54
| | | | | | | | | | | | | All objects after the last written or freed object are not supposed to exist after receiving the stream. Free them accordingly, as if a freeobjects record for them had been included in the stream. Reviewed by: Paul Dagnelie <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Fabian Grünbichler <[email protected]> Closes #5699 Closes #6507 Closes #6616
* Fix ARC behavior on 32-bit systemsBrian Behlendorf2017-10-101-27/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With the addition of the ABD changes consumption of the virtual address space has been greatly reduced. This exposed an issue on CONFIG_HIGHMEM systems where free memory was being calculated incorrectly. Functionally this didn't cause any major problems prior to ABD because a lack of available virtual address space was used as an indicator of low memory. This patch makes the following changes to address the issue and in the process realigns the code further with OpenZFS. There are no substantive changes in behavior for 64-bit systems. * Added CONFIG_HIGHMEM case to the arc_all_memory() and arc_free_memory() functions to only consider low memory pages on CONFIG_HIGHMEM systems. * The arc_free_memory() function was updated to return bytes instead of pages to be consistent with the other helper functions. In user space we make up some reasonable values since currently only testing is performed in this context. * Adds three new values to the arcstats kstat to provide visibility in to the ARC's assessment of the memory situation: memory_all_bytes, memory_free_bytes, and memory_available_bytes. * Added kmem_reap() call to arc_available_memory() for 32-bit builds to realign code with OpenZFS. * Reduced size of test file in /async_destroy_001_pos.ksh to speed up test case. Multiple txgs are still required. * Move vdevs used by zpool_clear_001_pos and zpool_upgrade_002_pos to TEST_BASE_DIR location to speed up test cases. Reviewed-by: David Quigley <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #5352 Closes #6734
* Fix abdstats kstat on 32-bit systemsBrian Behlendorf2017-10-061-2/+2
| | | | | | | | When decrementing the struct_size and scatter_chunk_waste kstats the value needs to be cast to an int on 32-bit systems. Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6721
* Remove unnecessary equality checkTobin Harding2017-10-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | Currently `if` statement includes an assignment (from a function return value) and a equality check. The parenthesis are in the incorrect place, currently the code clobbers the function return value because of this. We can fix this by simplifying the `if` statement. `if (foo != 0)` can be more succinctly expressed as `if (foo)` Remove the equality check, add parenthesis to correct the statement. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Chris Dunlop <[email protected]> Signed-off-by: Tobin C. Harding <[email protected]> Closes #6685 Close #6719
* Use linear abd in vdev_copy_uberblocks()Isaac Huang2017-10-051-1/+1
| | | | | | | | | | | The vdev_copy_uberblocks() function should use abd_alloc_linear() to allocate ub_abd, because abd_to_buf(ub_abd)) is used later. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Isaac Huang <[email protected]> Closes #6718 Closes #6713
* Remove dead code from AVL treeBrian Behlendorf2017-10-051-71/+4
| | | | | | | | | | | The avl_update_* functions are never used by ZFS and are therefore being removed. They're barely even used in Illumos. Additionally, simplify avl_add() by using a VERIFY which produces exactly the same behavior under Linux. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6716
* receive_freeobjects() skips freeing some objectsNed Bass2017-10-021-4/+2
| | | | | | | | | | | | | | | | | | | | | | | When receiving a FREEOBJECTS record, receive_freeobjects() incorrectly skips a freed object in some cases. Specifically, this happens when the first object in the range to be freed doesn't exist, but the second object does. This leaves an object allocated on disk on the receiving side which is unallocated on the sending side, which may cause receiving subsequent incremental streams to fail. The bug was caused by an incorrect increment of the object index variable when current object being freed doesn't exist. The increment is incorrect because incrementing the object index is handled by a call to dmu_object_next() in the increment portion of the for loop statement. Add test case that exposes this bug. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Closes #6694 Closes #6695
* Update the default for zfs_txg_historyAlek P2017-09-291-2/+2
| | | | | | | | | | | | | | It's often useful to have access to txg history for debugging purposes. This patch changes the default from 0 to 100 TXGs worth of history preserved. Reviewed by: Matthew Ahrens <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed by: Richard Elling <[email protected]> Reviewed by: Prakash Surya <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Alek Pinchuk <[email protected]> Closes #6691
* Scale the dbuf cache with arc_cchrisrd2017-09-292-8/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | Commit d3c2ae1 introduced a dbuf cache with a default size of the minimum of 100M or 1/32 maximum ARC size. (These figures may be adjusted using dbuf_cache_max_bytes and dbuf_cache_max_shift.) The dbuf cache is counted as metadata for the purposes of ARC size calculations. On a 1GB box the ARC maximum size defaults to c_max 493M which gives a dbuf cache default minimum size of 15.4M, and the ARC metadata defaults to minimum 16M. I.e. the dbuf cache is an significant proportion of the minimum metadata size. With other overheads involved this actually means the ARC metadata doesn't get down to the minimum. This patch dynamically scales the dbuf cache to the target ARC size instead of statically scaling it to the maximum ARC size. (The scale is still set by dbuf_cache_max_shift and the maximum size is still fixed by dbuf_cache_max_bytes.) Using the target ARC size rather than the current ARC size is done to help the ARC reach the target rather than simply focusing on the current size. Reviewed-by: Chunwei Chen <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Chris Dunlop <[email protected]> Issue #6506 Closes #6561
* Fix printk() calls missing log levelDeHackEd2017-09-251-3/+4
| | | | | | | Reviewed-by: George Melikov <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: DHE <[email protected]> Closes #6672
* Reimplement vdev_random_leaf and rename itOlaf Faaland2017-09-221-36/+26
| | | | | | | | | | | | | | | | | | | | | | | | | Rename it as mmp_random_leaf() since it is defined in mmp.c. The earlier implementation could end up spinning forever if a pool had a vdev marked writeable, none of whose children were writeable. It also did not guarantee that if a writeable leaf vdev existed, it would be found. Reimplement to recursively walk the device tree to select the leaf. It searches the entire tree, so that a return value of (NULL) indicates there were no usable leaves in the pool; all were either not writeable or had pending mmp writes. It still chooses the starting child randomly at each level of the tree, so if the pool's devices are healthy, the mmp writes go to random leaves with an even distribution. This was verified by testing using zfs_multihost_history enabled. Reviewed by: Thomas Caputi <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Olaf Faaland <[email protected]> Closes #6631 Closes #6665
* Increase default arc_c_minBrian Behlendorf2017-09-201-3/+4
| | | | | | | | | | | | | | | Increase the default arc_c_min value to which whichever is larger, either 32M or 1/32 of total system memory. This is advantageous for systems with more than 1G of memory where performance issues may occur when the ARC is allowed to collapse below a minimum size. At the same time we want to use the bare minimum value which is still functional so the filesystem can be used in very low memory environments. Reviewed-by: Tim Chase <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6659
* Export symbol dmu_tx_mark_netfree()Brian Behlendorf2017-09-201-0/+1
| | | | | | | | | | | | This symbol is needed by Lustre for the same reason it was needed by the ZPL. It should have been exported when the original patch was merged. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Alex Zhuravlev <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6660
* misc: fix meaningless valuesFeng Sun2017-09-192-2/+3
| | | | | | | Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tim Chase <[email protected]> Signed-off-by: Feng Sun <[email protected]> Closes #6658
* Correct cppcheck errorsGiuseppe Di Natale2017-09-191-1/+0
| | | | | | | | | | | | | ZFS buildbot STYLE builder was moved to Ubuntu 17.04 which has a newer version of cppcheck. Handle the new cppcheck errors. uu_* functions removed in this commit were unused and effectively dead code. They are now retired. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Giuseppe Di Natale <[email protected]> Closes #6653
* Linux 3.14 compat: IO acct, global_page_state, etcGiuseppe Di Natale2017-09-163-10/+42
| | | | | | | | | | | | | | | | | | | | | | generic_start_io_acct/generic_end_io_acct in the master branch of the linux kernel requires that the request_queue be provided. Move the logic from freemem in the spl to arc_free_memory in arc.c. Do this so we can take advantage of global_page_state interface checks in zfs. Upstream kernel replaced struct block_device with struct gendisk in struct bio. Determine if the function bio_set_dev exists during configure and have zfs use that if it exists. bio_set_dev https://github.com/torvalds/linux/commit/74d4699 global_node_page_state https://github.com/torvalds/linux/commit/75ef718 io acct https://github.com/torvalds/linux/commit/d62e26b Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Giuseppe Di Natale <[email protected]> Closes #6635
* Modifying XATTRs doesnt change the ctimeGaurav Kumar2017-09-132-2/+17
| | | | | | | | | Changing any metadata, should modify the ctime. Reviewed-by: Chunwei Chen <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: gaurkuma <[email protected]> Closes #3644 Closes #6586
* Fix false config_cache_write eventsArkadiusz Bubała2017-09-111-0/+5
| | | | | | | | | | | | On pool import when the old cache file is removed the ereport.fs.zfs.config_cache_write event is generated. Because zpool export always removes cache file it happens every export - import sequence. Reviewed-by: George Melikov <[email protected]> Reviewed-by: loli10K <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Arkadiusz Bubała <[email protected]> Closes #6617
* Fix volume WR_INDIRECT log replayBrian Behlendorf2017-09-081-6/+17
| | | | | | | | | | | | | | | | | | | The portion of the zvol_replay_write() handler responsible for replaying indirect log records for some reason never existed. As a result indirect log records were not being correctly replayed. This went largely unnoticed since the majority of zvol log records were of the type WR_COPIED or WR_NEED_COPY prior to OpenZFS 7578. This patch updates zvol_replay_write() to correctly handle these log records and adds a new test case which verifies volume replay to prevent any regression. The existing test case which verified replay on filesystem was renamed slog_replay_fs.ksh for clarity. Reviewed-by: George Melikov <[email protected]> Reviewed-by: loli10K <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6603 Closes #6615
* Revert "Handle new dnode size in incremental..."Brian Behlendorf2017-09-071-5/+3
| | | | | | | | | | | This reverts commit 65dcb0f67a4d72ee4e1e534703db5caacf1ec85f until a comprehensive fix is finalized. The stricter interior dnode detection in 4c5b89f59e4e5c8f5b4680040118ebde09598bbe and the new test case added by this patch revealed a issue with resizing dnodes when receiving an incremental backup stream. Signed-off-by: Brian Behlendorf <[email protected]> Issue #6576
* Improved dnode allocation and dmu_hold_impl()Olaf Faaland2017-09-053-241/+352
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactor dmu_object_alloc_dnsize() and dnode_hold_impl() to simplify the code, fix errors introduced by commit dbeb879 (PR #6117) interacting badly with large dnodes, and improve performance. * When allocating a new dnode in dmu_object_alloc_dnsize(), update the percpu object ID for the core's metadnode chunk immediately. This eliminates most lock contention when taking the hold and creating the dnode. * Correct detection of the chunk boundary to work properly with large dnodes. * Separate the dmu_hold_impl() code for the FREE case from the code for the ALLOCATED case to make it easier to read. * Fully populate the dnode handle array immediately after reading a block of the metadnode from disk. Subsequently the dnode handle array provides enough information to determine which dnode slots are in use and which are free. * Add several kstats to allow the behavior of the code to be examined. * Verify dnode packing in large_dnode_008_pos.ksh. Since the test is purely creates, it should leave very few holes in the metadnode. * Add test large_dnode_009_pos.ksh, which performs concurrent creates and deletes, to complement existing test which does only creates. With the above fixes, there is very little contention in a test of about 200,000 racing dnode allocations produced by tests 'large_dnode_008_pos' and 'large_dnode_009_pos'. name type data dnode_hold_dbuf_hold 4 0 dnode_hold_dbuf_read 4 0 dnode_hold_alloc_hits 4 3804690 dnode_hold_alloc_misses 4 216 dnode_hold_alloc_interior 4 3 dnode_hold_alloc_lock_retry 4 0 dnode_hold_alloc_lock_misses 4 0 dnode_hold_alloc_type_none 4 0 dnode_hold_free_hits 4 203105 dnode_hold_free_misses 4 4 dnode_hold_free_lock_misses 4 0 dnode_hold_free_lock_retry 4 0 dnode_hold_free_overflow 4 0 dnode_hold_free_refcount 4 57 dnode_hold_free_txg 4 0 dnode_allocate 4 203154 dnode_reallocate 4 0 dnode_buf_evict 4 23918 dnode_alloc_next_chunk 4 4887 dnode_alloc_race 4 0 dnode_alloc_next_block 4 18 The performance is slightly improved for concurrent creates with 16+ threads, and unchanged for low thread counts. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Olaf Faaland <[email protected]> Closes #5396 Closes #6522 Closes #6414 Closes #6564
* Handle new dnode size in incremental backup streamNed Bass2017-09-051-3/+5
| | | | | | | | | | | | | | | | | When receiving an incremental backup stream, call dmu_object_reclaim_dnsize() if an object's dnode size differs between the incremental source and target. Otherwise it may appear that a dnode which has shrunk is still occupying slots which are in fact free. This will cause a failure to receive new objects that should occupy the now-free slots. Add a test case to verify that an incremental stream containing objects with changed dnode sizes can be received without error. This test case fails without this change. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Closes #6366 Closes #6576
* Trim new line from zfs_vdev_schedulerBrian Behlendorf2017-09-051-18/+53
| | | | | | | | | Add a helper function to trim the tailing new line. While we're here use this new hook to immediately apply the new scheduler. Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #3356 Closes #6573
* Retire send space estimation via ZFS_IOC_SENDLOLi2017-08-311-0/+5
| | | | | | | | | | | | | | Add a small wrapper around libzfs_core`lzc_send_space() to libzfs so that every legacy ZFS_IOC_SEND consumer, along with their userland counterpart estimate_ioctl(), can leverage ZFS_IOC_SEND_SPACE to request send space estimation. The legacy functionality in zfs_ioc_send() is left untouched for compatibility purposes. Reviewed by: Thomas Caputi <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: loli10K <[email protected]> Closes #6029
* OpenZFS 2976 - remove useless offsetof() macrosRichard Lowe2017-08-301-3/+2
| | | | | | | | | | | | | | | Authored by: Richard Lowe <[email protected]> Reviewed by: Josef 'Jeff' Sipek <[email protected]> Reviewed by: Igor Kozhukhov <[email protected]> Reviewed by: Andy Stormont <[email protected]> Approved by: Dan McDonald <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/2976 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/5c5f137 Closes #6582
* dmu_objset: release bonus buffer in failure pathGvozden Neskovic2017-08-301-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Reported by kmemleak during testing of a new patch: ``` unreferenced object 0xffff9f1c12e38800 (size 1024): comm "z_upgrade", pid 17842, jiffies 4296870904 (age 8746.268s) backtrace: kmemleak_alloc+0x7a/0x100 __kmalloc_node+0x26c/0x510 range_tree_create+0x39/0xa0 [zfs] dmu_zfetch_init+0x73/0xe0 [zfs] dnode_create+0x12c/0x3b0 [zfs] dnode_hold_impl+0x1096/0x1130 [zfs] dnode_hold+0x23/0x30 [zfs] dmu_bonus_hold_impl+0x6b/0x370 [zfs] dmu_bonus_hold+0x1e/0x30 [zfs] dmu_objset_space_upgrade+0x114/0x310 [zfs] dmu_objset_userobjspace_upgrade_cb+0xd8/0x150 [zfs] dmu_objset_upgrade_task_cb+0x136/0x1e0 [zfs] kthread+0x119/0x150 ``` Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Gvozden Neskovic <[email protected]> Closes #6575
* OpenZFS 7028 - avl_destroy_nodes supports emptying, not just destroying, an ↵Eli Rosenthal2017-08-301-3/+4
| | | | | | | | | | | | | | | | | avl tree Authored by: Eli Rosenthal <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Reviewed by: Paul Dagnelie <[email protected]> Reviewed by: Igor Kozhukhov <[email protected]> Approved by: Gordon Ross <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/7028 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/86f617e Closes #6583
* OpenZFS 6447 - handful of nvpair cleanupsSteve Dougherty2017-08-301-0/+2
| | | | | | | | | | | | | | | | Authored by: Steve Dougherty <[email protected]> Reviewed by: Josef 'Jeff' Sipek <[email protected]> Reviewed by: Andy Stormont <[email protected]> Reviewed by: Garrett D'Amore <[email protected]> Approved by: Robert Mustacchi <[email protected]> Reviewed-by: Don Brady <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/6447 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/759e89b Closes #6581
* OpenZFS 5778 - nvpair_type_is_array() does not recognize DATA_TYPE_INT8_ARRAYAndriy Gapon2017-08-301-0/+1
| | | | | | | | | | | | | | | Authored by: Andriy Gapon <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Reviewed by: Prakash Surya <[email protected]> Approved by: Dan McDonald <[email protected]> Reviewed-by: Don Brady <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/5778 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/bf4d553 Closes #6580
* OpenZFS 7261 - nvlist code should enforce name length limitMatthew Ahrens2017-08-301-0/+2
| | | | | | | | | | | | | | | | Authored by: Matthew Ahrens <[email protected]> Reviewed by: Sebastien Roy <[email protected]> Reviewed by: George Wilson <[email protected]> Reviewed by: Robert Mustacchi <[email protected]> Approved by: Dan McDonald <[email protected]> Reviewed-by: Don Brady <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/7261 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/48dd5e6 Closes #6579
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-16 21:51:18 +0000