summaryrefslogtreecommitdiffstats
path: root/module
Commit message (Collapse)AuthorAgeFilesLines
* Linux 3.14 compat: IO acct, global_page_state, etcGiuseppe Di Natale2017-09-163-10/+42
| | | | | | | | | | | | | | | | | | | | | | generic_start_io_acct/generic_end_io_acct in the master branch of the linux kernel requires that the request_queue be provided. Move the logic from freemem in the spl to arc_free_memory in arc.c. Do this so we can take advantage of global_page_state interface checks in zfs. Upstream kernel replaced struct block_device with struct gendisk in struct bio. Determine if the function bio_set_dev exists during configure and have zfs use that if it exists. bio_set_dev https://github.com/torvalds/linux/commit/74d4699 global_node_page_state https://github.com/torvalds/linux/commit/75ef718 io acct https://github.com/torvalds/linux/commit/d62e26b Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Giuseppe Di Natale <[email protected]> Closes #6635
* Modifying XATTRs doesnt change the ctimeGaurav Kumar2017-09-132-2/+17
| | | | | | | | | Changing any metadata, should modify the ctime. Reviewed-by: Chunwei Chen <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: gaurkuma <[email protected]> Closes #3644 Closes #6586
* Fix false config_cache_write eventsArkadiusz Bubała2017-09-111-0/+5
| | | | | | | | | | | | On pool import when the old cache file is removed the ereport.fs.zfs.config_cache_write event is generated. Because zpool export always removes cache file it happens every export - import sequence. Reviewed-by: George Melikov <[email protected]> Reviewed-by: loli10K <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Arkadiusz Bubała <[email protected]> Closes #6617
* Fix volume WR_INDIRECT log replayBrian Behlendorf2017-09-081-6/+17
| | | | | | | | | | | | | | | | | | | The portion of the zvol_replay_write() handler responsible for replaying indirect log records for some reason never existed. As a result indirect log records were not being correctly replayed. This went largely unnoticed since the majority of zvol log records were of the type WR_COPIED or WR_NEED_COPY prior to OpenZFS 7578. This patch updates zvol_replay_write() to correctly handle these log records and adds a new test case which verifies volume replay to prevent any regression. The existing test case which verified replay on filesystem was renamed slog_replay_fs.ksh for clarity. Reviewed-by: George Melikov <[email protected]> Reviewed-by: loli10K <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6603 Closes #6615
* Revert "Handle new dnode size in incremental..."Brian Behlendorf2017-09-071-5/+3
| | | | | | | | | | | This reverts commit 65dcb0f67a4d72ee4e1e534703db5caacf1ec85f until a comprehensive fix is finalized. The stricter interior dnode detection in 4c5b89f59e4e5c8f5b4680040118ebde09598bbe and the new test case added by this patch revealed a issue with resizing dnodes when receiving an incremental backup stream. Signed-off-by: Brian Behlendorf <[email protected]> Issue #6576
* Improved dnode allocation and dmu_hold_impl()Olaf Faaland2017-09-053-241/+352
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactor dmu_object_alloc_dnsize() and dnode_hold_impl() to simplify the code, fix errors introduced by commit dbeb879 (PR #6117) interacting badly with large dnodes, and improve performance. * When allocating a new dnode in dmu_object_alloc_dnsize(), update the percpu object ID for the core's metadnode chunk immediately. This eliminates most lock contention when taking the hold and creating the dnode. * Correct detection of the chunk boundary to work properly with large dnodes. * Separate the dmu_hold_impl() code for the FREE case from the code for the ALLOCATED case to make it easier to read. * Fully populate the dnode handle array immediately after reading a block of the metadnode from disk. Subsequently the dnode handle array provides enough information to determine which dnode slots are in use and which are free. * Add several kstats to allow the behavior of the code to be examined. * Verify dnode packing in large_dnode_008_pos.ksh. Since the test is purely creates, it should leave very few holes in the metadnode. * Add test large_dnode_009_pos.ksh, which performs concurrent creates and deletes, to complement existing test which does only creates. With the above fixes, there is very little contention in a test of about 200,000 racing dnode allocations produced by tests 'large_dnode_008_pos' and 'large_dnode_009_pos'. name type data dnode_hold_dbuf_hold 4 0 dnode_hold_dbuf_read 4 0 dnode_hold_alloc_hits 4 3804690 dnode_hold_alloc_misses 4 216 dnode_hold_alloc_interior 4 3 dnode_hold_alloc_lock_retry 4 0 dnode_hold_alloc_lock_misses 4 0 dnode_hold_alloc_type_none 4 0 dnode_hold_free_hits 4 203105 dnode_hold_free_misses 4 4 dnode_hold_free_lock_misses 4 0 dnode_hold_free_lock_retry 4 0 dnode_hold_free_overflow 4 0 dnode_hold_free_refcount 4 57 dnode_hold_free_txg 4 0 dnode_allocate 4 203154 dnode_reallocate 4 0 dnode_buf_evict 4 23918 dnode_alloc_next_chunk 4 4887 dnode_alloc_race 4 0 dnode_alloc_next_block 4 18 The performance is slightly improved for concurrent creates with 16+ threads, and unchanged for low thread counts. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Olaf Faaland <[email protected]> Closes #5396 Closes #6522 Closes #6414 Closes #6564
* Handle new dnode size in incremental backup streamNed Bass2017-09-051-3/+5
| | | | | | | | | | | | | | | | | When receiving an incremental backup stream, call dmu_object_reclaim_dnsize() if an object's dnode size differs between the incremental source and target. Otherwise it may appear that a dnode which has shrunk is still occupying slots which are in fact free. This will cause a failure to receive new objects that should occupy the now-free slots. Add a test case to verify that an incremental stream containing objects with changed dnode sizes can be received without error. This test case fails without this change. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Closes #6366 Closes #6576
* Trim new line from zfs_vdev_schedulerBrian Behlendorf2017-09-051-18/+53
| | | | | | | | | Add a helper function to trim the tailing new line. While we're here use this new hook to immediately apply the new scheduler. Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #3356 Closes #6573
* Retire send space estimation via ZFS_IOC_SENDLOLi2017-08-311-0/+5
| | | | | | | | | | | | | | Add a small wrapper around libzfs_core`lzc_send_space() to libzfs so that every legacy ZFS_IOC_SEND consumer, along with their userland counterpart estimate_ioctl(), can leverage ZFS_IOC_SEND_SPACE to request send space estimation. The legacy functionality in zfs_ioc_send() is left untouched for compatibility purposes. Reviewed by: Thomas Caputi <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: loli10K <[email protected]> Closes #6029
* OpenZFS 2976 - remove useless offsetof() macrosRichard Lowe2017-08-301-3/+2
| | | | | | | | | | | | | | | Authored by: Richard Lowe <[email protected]> Reviewed by: Josef 'Jeff' Sipek <[email protected]> Reviewed by: Igor Kozhukhov <[email protected]> Reviewed by: Andy Stormont <[email protected]> Approved by: Dan McDonald <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/2976 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/5c5f137 Closes #6582
* dmu_objset: release bonus buffer in failure pathGvozden Neskovic2017-08-301-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Reported by kmemleak during testing of a new patch: ``` unreferenced object 0xffff9f1c12e38800 (size 1024): comm "z_upgrade", pid 17842, jiffies 4296870904 (age 8746.268s) backtrace: kmemleak_alloc+0x7a/0x100 __kmalloc_node+0x26c/0x510 range_tree_create+0x39/0xa0 [zfs] dmu_zfetch_init+0x73/0xe0 [zfs] dnode_create+0x12c/0x3b0 [zfs] dnode_hold_impl+0x1096/0x1130 [zfs] dnode_hold+0x23/0x30 [zfs] dmu_bonus_hold_impl+0x6b/0x370 [zfs] dmu_bonus_hold+0x1e/0x30 [zfs] dmu_objset_space_upgrade+0x114/0x310 [zfs] dmu_objset_userobjspace_upgrade_cb+0xd8/0x150 [zfs] dmu_objset_upgrade_task_cb+0x136/0x1e0 [zfs] kthread+0x119/0x150 ``` Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Gvozden Neskovic <[email protected]> Closes #6575
* OpenZFS 7028 - avl_destroy_nodes supports emptying, not just destroying, an ↵Eli Rosenthal2017-08-301-3/+4
| | | | | | | | | | | | | | | | | avl tree Authored by: Eli Rosenthal <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Reviewed by: Paul Dagnelie <[email protected]> Reviewed by: Igor Kozhukhov <[email protected]> Approved by: Gordon Ross <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/7028 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/86f617e Closes #6583
* OpenZFS 6447 - handful of nvpair cleanupsSteve Dougherty2017-08-301-0/+2
| | | | | | | | | | | | | | | | Authored by: Steve Dougherty <[email protected]> Reviewed by: Josef 'Jeff' Sipek <[email protected]> Reviewed by: Andy Stormont <[email protected]> Reviewed by: Garrett D'Amore <[email protected]> Approved by: Robert Mustacchi <[email protected]> Reviewed-by: Don Brady <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/6447 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/759e89b Closes #6581
* OpenZFS 5778 - nvpair_type_is_array() does not recognize DATA_TYPE_INT8_ARRAYAndriy Gapon2017-08-301-0/+1
| | | | | | | | | | | | | | | Authored by: Andriy Gapon <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Reviewed by: Prakash Surya <[email protected]> Approved by: Dan McDonald <[email protected]> Reviewed-by: Don Brady <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/5778 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/bf4d553 Closes #6580
* OpenZFS 7261 - nvlist code should enforce name length limitMatthew Ahrens2017-08-301-0/+2
| | | | | | | | | | | | | | | | Authored by: Matthew Ahrens <[email protected]> Reviewed by: Sebastien Roy <[email protected]> Reviewed by: George Wilson <[email protected]> Reviewed by: Robert Mustacchi <[email protected]> Approved by: Dan McDonald <[email protected]> Reviewed-by: Don Brady <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/7261 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/48dd5e6 Closes #6579
* OpenZFS 8375 - Kernel memory leak in nvpair codeMatthew Ahrens2017-08-301-2/+4
| | | | | | | | | | | | | | | | | Authored by: Matthew Ahrens <[email protected]> Reviewed by: Pavel Zakharov <[email protected]> Reviewed by: George Wilson <[email protected]> Reviewed by: Prashanth Sreenivasa <[email protected]> Reviewed by: Robert Mustacchi <[email protected]> Approved by: Dan McDonald <[email protected]> Reviewed-by: Don Brady <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/8375 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/843c211 Closes #6578
* Enhance comments for large dnode projectMatthew Ahrens2017-08-293-4/+16
| | | | | | | | | | | | | Fix a few nits in the comments from large dnodes. Also import some of the commit message as a comment in the code, making it more accessible. Reviewed-by: @rottegift Reviewed-by: George Melikov <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Wilson <[email protected]> Signed-off-by: Matt Ahrens <[email protected]> Closes #6551
* Linux 4.8+ compatibility fix for vm statsdbavatar2017-08-241-0/+5
| | | | | | | | | | | | vm_node_stat must be used instead of vm_zone_stat. Unfortunately the old code still compiles potentially leading to silent failure of arc_evictable_memory() AKAMAI: CR 3816601: Regression in zfs dropcache test Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Chunwei Chen <[email protected]> Signed-off-by: Debabrata Banerjee <[email protected]> Closes #6528
* dbuf_cons: deduplicate multilist_link_init()chrisrd2017-08-241-1/+0
| | | | | | | | | | | Remove harmless duplicate multilist_link_init() introduced by commit d3c2ae1. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: loli10K <[email protected]> Signed-off-by: Chris Dunlop <[email protected]> Closes #6552
* OpenZFS 8414 - Implemented zpool scrub pause/resumeAlek P2017-08-241-1/+0
| | | | | | | | | | | | | | | Authored by: Alek Pinchuk <[email protected]> Reviewed by: George Melikov <[email protected]> Reviewed by: Brian Behlendorf <[email protected]> Reviewed by: Brad Lewis <[email protected]> Reviewed by: Serapheim Dimitropoulos <[email protected]> Reviewed by: Matt Ahrens <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Approved by: Dan McDonald <[email protected]> Ported-by: Alek Pinchuk <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/8414 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/c29616076 Closes #6538
* Send / Recv Fixes following b52563Tom Caputi2017-08-232-36/+70
| | | | | | | | | | | | | | | | | | | | | | This patch fixes several issues discovered after the encryption patch was merged: * Fixed a bug where encrypted datasets could attempt to receive embedded data records. * Fixed a bug where dirty records created by the recv code wasn't properly setting the dr_raw flag. * Fixed a typo where a dmu_tx_commit() was changed to dmu_tx_abort() * Fixed a few error handling bugs unrelated to the encryption patch in dmu_recv_stream() Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tom Caputi <[email protected]> Closes #6512 Closes #6524 Closes #6545
* Fix zfs_ioc_pool_sync should not use fnvlistChunwei Chen2017-08-211-3/+9
| | | | | | | | | Use fnvlist on user input would allow user to easily panic zfs. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Alek Pinchuk <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Closes #6529
* vdev_mirror: kstat observables for preferred vdevGvozden Neskovic2017-08-212-6/+78
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Gvozden Neskovic <[email protected]> Closes #6461
* vdev_mirror: load balancing fixesGvozden Neskovic2017-08-212-34/+23
| | | | | | | | | | | | | | | | vdev_queue: - Track the last position of each vdev, including the io size, in order to detect linear access of the following zio. - Remove duplicate `vq_lastoffset` vdev_mirror: - Correctly calculate the zio offset (signedness issue) - Deprecate `vdev_queue_register_lastoffset()` - Add `VDEV_LABEL_START_SIZE` to zio offset of leaf vdevs Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Gvozden Neskovic <[email protected]> Closes #6461
* Fix range locking in ZIL commit codepathLOLi2017-08-213-12/+44
| | | | | | | | | | | | | | | | | | | | | | | Since OpenZFS 7578 (1b7c1e5) if we have a ZVOL with logbias=throughput we will force WR_INDIRECT itxs in zvol_log_write() setting itx->itx_lr offset and length to the offset and length of the BIO from zvol_write()->zvol_log_write(): these offset and length are later used to take a range lock in zillog->zl_get_data function: zvol_get_data(). Now suppose we have a ZVOL with blocksize=8K and push 4K writes to offset 0: we will only be range-locking 0-4096. This means the ASSERTion we make in dbuf_unoverride() is no longer valid because now dmu_sync() is called from zilog's get_data functions holding a partial lock on the dbuf. Fix this by taking a range lock on the whole block in zvol_get_data(). Reviewed-by: Chunwei Chen <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: loli10K <[email protected]> Closes #6238 Closes #6315 Closes #6356 Closes #6477
* Fix remounting snapshots read-writeLOLi2017-08-171-1/+9
| | | | | | | | | | | | It's not enough to preserve/restore MS_RDONLY on the superblock flags to avoid remounting a snapshot read-write: be explicit about our intentions to the VFS layer so the readonly bit is updated correctly in do_remount_sb(). Reviewed-by: Chunwei Chen <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: loli10K <[email protected]> Closes #6510 Closes #6515
* Retire legacy test infrastructureBrian Behlendorf2017-08-154-1309/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Removed zpios kmod, utility, headers and man page. * Removed unused scripts zpios-profile/*, zpios-test/*, zpool-config/*, smb.sh, zpios-sanity.sh, zpios-survey.sh, zpios.sh, and zpool-create.sh. * Removed zfs-script-config.sh.in. When building 'make' generates a common.sh with in-tree path information from the common.sh.in template. This file and sourced by the test scripts and used for in-tree testing, it is not included in the packages. When building packages 'make install' uses the same template to create a new common.sh which is appropriate for the packaging. * Removed unused functions/variables from scripts/common.sh.in. Only minimal path information and configuration environment variables remain. * Removed unused scripts from scripts/ directory. * Remaining shell scripts in the scripts directory updated to cleanly pass shellcheck and added to checked scripts. * Renamed tests/test-runner/cmd/ to tests/test-runner/bin/ to match install location name. * Removed last traces of the --enable-debug-dmu-tx configure options which was retired some time ago. Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6509
* Add corruption failure option to zinject(8)Don Brady2017-08-142-7/+49
| | | | | | | | | | | Added a 'corrupt' error option that will flip a bit in the data after a read operation. This is useful for generating checksum errors at the device layer (in a mirror config for example). It is also used to validate the diagnosis of checksum errors from the zfs diagnosis engine. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Don Brady <[email protected]> Closes #6345
* Native Encryption for ZFS on LinuxTom Caputi2017-08-1442-875/+8531
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change incorporates three major pieces: The first change is a keystore that manages wrapping and encryption keys for encrypted datasets. These commands mostly involve manipulating the new DSL Crypto Key ZAP Objects that live in the MOS. Each encrypted dataset has its own DSL Crypto Key that is protected with a user's key. This level of indirection allows users to change their keys without re-encrypting their entire datasets. The change implements the new subcommands "zfs load-key", "zfs unload-key" and "zfs change-key" which allow the user to manage their encryption keys and settings. In addition, several new flags and properties have been added to allow dataset creation and to make mounting and unmounting more convenient. The second piece of this patch provides the ability to encrypt, decyrpt, and authenticate protected datasets. Each object set maintains a Merkel tree of Message Authentication Codes that protect the lower layers, similarly to how checksums are maintained. This part impacts the zio layer, which handles the actual encryption and generation of MACs, as well as the ARC and DMU, which need to be able to handle encrypted buffers and protected data. The last addition is the ability to do raw, encrypted sends and receives. The idea here is to send raw encrypted and compressed data and receive it exactly as is on a backup system. This means that the dataset on the receiving system is protected using the same user key that is in use on the sending side. By doing so, datasets can be efficiently backed up to an untrusted system without fear of data being compromised. Reviewed by: Matthew Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Jorgen Lundman <[email protected]> Signed-off-by: Tom Caputi <[email protected]> Closes #494 Closes #5769
* Fix NULL pointer when O_SYNC read in snapshotChunwei Chen2017-08-111-1/+3
| | | | | | | | | | When doing read on a file open with O_SYNC, it will trigger zil_commit. However for snapshot, there's no zil, so we shouldn't be doing that. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Closes #6478 Closes #6494
* Allow longer SPA names in statsgaurkuma2017-08-112-15/+21
| | | | | | | | | | | | The pool name can be 256 chars long. Today, in /proc/spl/kstat/zfs/ the name is limited to < 32 characters. This change is to allows bigger pool names. Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: loli10K <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: gaurkuma <[email protected]> Closes #6481
* Simplify threads, mutexs, cvs and rwlocksBrian Behlendorf2017-08-116-12/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Simplify threads, mutexs, cvs and rwlocks * Update the zk_thread_create() function to use the same trick as Illumos. Specifically, cast the new pthread_t to a void pointer and return that as the kthread_t *. This avoids the issues associated with managing a wrapper structure and is safe as long as the callers never attempt to dereference it. * Update all function prototypes passed to pthread_create() to match the expected prototype. We were getting away this with before since the function were explicitly cast. * Replaced direct zk_thread_create() calls with thread_create() for code consistency. All consumers of libzpool now use the proper wrappers. * The mutex_held() calls were converted to MUTEX_HELD(). * Removed all mutex_owner() calls and retired the interface. Instead use MUTEX_HELD() which provides the same information and allows the implementation details to be hidden. In this case the use of the pthread_equals() function. * The kthread_t, kmutex_t, krwlock_t, and krwlock_t types had any non essential fields removed. In the case of kthread_t and kcondvar_t they could be directly typedef'd to pthread_t and pthread_cond_t respectively. * Removed all extra ASSERTS from the thread, mutex, rwlock, and cv wrapper functions. In practice, pthreads already provides the vast majority of checks as long as we check the return code. Removing this code from our wrappers help readability. * Added TS_JOINABLE state flag to pass to request a joinable rather than detached thread. This isn't a standard thread_create() state but it's the least invasive way to pass this information and is only used by ztest. TEST_ZTEST_TIMEOUT=3600 Chunwei Chen <[email protected]> Reviewed-by: Tom Caputi <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #4547 Closes #5503 Closes #5523 Closes #6377 Closes #6495
* zio_dva_throttle_done() should allow zinjected ZIOsanjeevbagewadi2017-08-101-1/+2
| | | | | | | | | | | | | | | | If fault injection is enabled, the ZIO_FLAG_IO_RETRY could be set by zio_handle_device_injection() to generate the FMA events and update stats. Hence, ignore the flag and process such zios. A better fix would be to add another flag in the zio_t to indicate that the zio is failed because of a zinject rule. However, considering the fact that we do this in debug bits, we could do with the crude check using the global flag zio_injection_enabled which is set to 1 when zinject records are added. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Sanjeev Bagewadi <[email protected]> Closes #6383 Closes #6384
* Add libtpool (thread pools)Brian Behlendorf2017-08-099-97/+133
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | OpenZFS provides a library called tpool which implements thread pools for user space applications. Porting this library means the zpool utility no longer needs to borrow the kernel mutex and taskq interfaces from libzpool. This code was updated to use the tpool library which behaves in a very similar fashion. Porting libtpool was relatively straight forward and minimal modifications were needed. The core changes were: * Fully convert the library to use pthreads. * Updated signal handling. * lmalloc/lfree converted to calloc/free * Implemented portable pthread_attr_clone() function. Finally, update the build system such that libzpool.so is no longer linked in to zfs(8), zpool(8), etc. All that is required is libzfs to which the zcommon soures were added (which is the way it always should have been). Removing the libzpool dependency resulted in several build issues which needed to be resolved. * Moved zfeature support to module/zcommon/zfeature_common.c * Moved ratelimiting to to module/zfs/zfs_ratelimit.c * Moved get_system_hostid() to lib/libspl/gethostid.c * Removed use of cmn_err() in zcommon source * Removed dprintf_setup() call from zpool_main.c and zfs_main.c * Removed highbit() and lowbit() * Removed unnecessary library dependencies from Makefiles * Removed fletcher-4 kstat in user space * Added sha2 support explicitly to libzfs * Added highbit64() and lowbit64() to zpool_util.c Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6442
* zv_suspend_lock in zvol_open()/zvol_release()Boris Protopopov2017-08-091-23/+41
| | | | | | | Acquire zv_suspend_lock on first open and last close only. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Boris Protopopov <[email protected]> Closes #6342
* Add debug log entries for failed receive recordsNed Bass2017-08-081-7/+100
| | | | | | | | | | | Log contents of a receive record if an error occurs while writing it out to the pool. This may help determine the cause when backup streams are rejected as invalid. Reviewed by: Matthew Ahrens <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Closes #6465
* Fix dnode allocation raceBrian Behlendorf2017-08-085-66/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When performing concurrent object allocations using the new multi-threaded allocator and large dnodes it's possible to allocate overlapping large dnodes. This case should have been handled by detecting an error returned by dnode_hold_impl(). But that logic only checked the returned dnp was not-NULL, and the dnp variable was not reset to NULL when retrying. Resolve this issue by properly checking the return value of dnode_hold_impl(). Additionally, it was possible that dnode_hold_impl() would misreport a dnode as free when it was in fact in use. This could occurs for two reasons: * The per-slot zrl_lock must be held over the entire critical section which includes the alloc/free until the new dnode is assigned to children_dnodes. Additionally, all of the zrl_lock's in the range must be held to protect moving dnodes. * The dn->dn_ot_type cannot be solely relied upon to check the type. When allocating a new dnode its type will be DMU_OT_NONE after dnode_create(). Only latter when dnode_allocate() is called will it transition to the new type. This means there's a window when allocating where it can mistaken for a free dnode. Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Ned Bass <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Olaf Faaland <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6414 Closes #6439
* Use SET_ERROR for constant non-zero return codesNed Bass2017-08-0218-46/+46
| | | | | | | | | | | | | Update many return and assignment statements to follow the convention of using the SET_ERROR macro when returning a hard-coded non-zero value from a function. This aids debugging by recording the error codes in the debug log. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Signed-off-by: Ned Bass <[email protected]> Closes #6441
* Only record zio->io_delay on reads and writesTony Hutter2017-08-021-1/+1
| | | | | | | | | | | | | | | | | | | While investigating https://github.com/zfsonlinux/zfs/issues/6425 I noticed that ioctl ZIOs were not setting zio->io_delay correctly. They would set the start time in zio_vdev_io_start(), but never set the end time in zio_vdev_io_done(), since ioctls skip it and go straight to zio_done(). This was causing spurious "delayed IO" events to appear, which would eventually get rate-limited and displayed as "Missed events" messages in zed. To get around the problem, this patch only sets zio->io_delay for read and write ZIOs, since that's all we care about anyway. Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes #6425 Closes #6440
* Fix volmode=none property behavior at import timeLOLi2017-07-311-0/+3
| | | | | | | | | | | At import time spa_import() calls zvol_create_minors() directly: with the current implementation we have no way to avoid device node creation when volmode=none. Fix this by enforcing volmode=none directly in zvol_alloc(). Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: loli10K <[email protected]> Closes #6426
* zfs promote|rename .../%recv should be an errorLOLi2017-07-281-2/+10
| | | | | | | | | | | | | | If we are in the middle of an incremental 'zfs receive', the child .../%recv will exist. If we run 'zfs promote' .../%recv, it will "work", but then zfs gets confused about the status of the new dataset. Attempting to do this promote should be an error. Similarly renaming .../%recv datasets should not be allowed. Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: loli10K <[email protected]> Closes #4843 Closes #6339
* OpenZFS 7915 - checks in l2arc_evict could use some cleaning upAndriy Gapon2017-07-281-15/+9
| | | | | | | | | | | | | Authored by: Andriy Gapon <[email protected]> Reviewed by: Dan Kimmel <[email protected]> Reviewed by: Prakash Surya <[email protected]> Approved by: Matthew Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/7915 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/836a00c Closes #6375
* OpenZFS 8373 - TXG_WAIT in ZIL commit pathAndriy Gapon2017-07-281-1/+18
| | | | | | | | | | | | | | Authored by: Andriy Gapon <[email protected]> Reviewed by: Prakash Surya <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Approved by: Dan McDonald <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/8373 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/7f04961 Closes #6403
* OpenZFS 8508 - Mounting a zpool on 32-bit platforms panicsGiuseppe Di Natale2017-07-261-1/+1
| | | | | | | | | | | | Authored by: Justin Hibbits <[email protected]> Reviewed by: Matt Ahrens <[email protected]> Approved by: Dan McDonald <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: Giuseppe Di Natale <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/8508 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/15fc257 Closes #6404
* Add line info and SET_ERROR() to ZFS debug logNed Bass2017-07-251-7/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Redefine the SET_ERROR macro in terms of __dprintf() so the error return codes get logged as both tracepoint events (if tracepoints are enabled) and as ZFS debug log entries. This also allows us to use the same definition of SET_ERROR() in kernel and user space. Define a new debug flag ZFS_DEBUG_SET_ERROR=512 that may be bitwise or'd into zfs_flags. Setting this flag enables both dprintf() and SET_ERROR() messages in the debug log. That is, setting ZFS_DEBUG_SET_ERROR and ZFS_DEBUG_DPRINTF|ZFS_DEBUG_SET_ERROR are equivalent (this was done for sake of simplicity). Leaving ZFS_DEBUG_SET_ERROR unset suppresses the SET_ERROR() messages which helps avoid cluttering up the logs. To enable SET_ERROR() logging, run: echo 1 > /sys/module/zfs/parameters/zfs_dbgmsg_enable echo 512 > /sys/module/zfs/parameters/zfs_flags Remove the zfs_set_error_class tracepoints event class since SET_ERROR() now uses __dprintf(). This sacrifices a bit of granularity when selecting individual tracepoint events to enable but it makes the code simpler. Include file, function, and line number information in debug log entries. The information is now added to the message buffer in __dprintf() and as a result the zfs_dprintf_class tracepoints event class was changed from a 4 parameter interface to a single parameter. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Closes #6400
* Some additional send stream validity checkingNed Bass2017-07-251-8/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | Check in the DMU whether an object record in a send stream being received contains an unsupported dnode slot count, and return an error if it does. Failure to catch an unsupported dnode slot count would result in a panic when the SPA attempts to increment the reference count for the large_dnode feature and the pool has the feature disabled. This is not normally an issue for a well-formed send stream which would have the DMU_BACKUP_FEATURE_LARGE_DNODE flag set if it contains large dnodes, so it will be rejected as unsupported if the required feature is disabled. This change adds a missing object record field validation. Add missing stream feature flag checks in dmu_recv_resume_begin_check(). Consolidate repetitive comment blocks in dmu_recv_begin_check(). Update zstreamdump to print the dnode slot count (dn_slots) for an object record when running in verbose mode. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Signed-off-by: Ned Bass <[email protected]> Closes #6396
* Fix 'zpool clear' on suspended poolsBrian Behlendorf2017-07-251-1/+2
| | | | | | | | | | | | | | | | | 'zpool clear' should be able to resume I/O on suspended, but otherwise healthy, pools. 4a283c7 accidentally introduced a new code path where we call txg_wait_synced() on the suspended pool before we had the chance to resume I/O via zio_resume(): this results in the 'zpool clear' command hanging indefinitely, waiting for a TXG that cannot be synced. Fix this by avoiding the call to txg_wait_synced(). Reviewed-by: George Melikov <[email protected]> Reviewed-by: loli10K <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6399
* Report MMP_STATE_NO_HOSTID immediatelyOlaf Faaland2017-07-251-6/+6
| | | | | | | | | | | | There is no need to perform the activity check before detecting that the user must set the system hostid, because the pool's multihost property is on, but spa_get_hostid() returned 0. The initial call to vdev_uberblock_load() provided the information required. Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Olaf Faaland <[email protected]> Closes #6388
* Add callback for zfs_multihost_intervalOlaf Faaland2017-07-251-1/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | Add a callback to wake all running mmp threads when zfs_multihost_interval is changed. This is necessary when the interval is changed from a very large value to a significantly lower one, while pools are imported that have the multihost property enabled. Without this commit, the mmp thread does not wake up and detect the new interval until after it has waited the old multihost interval time. A user monitoring mmp writes via the provided kstat would be led to believe that the changed setting did not work. Added a test in the ZTS under mmp to verify the new functionality is working. Added a test to ztest which starts and stops mmp threads, and calls into the code to signal sleeping mmp threads, to test for deadlocks or similar locking issues. Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Olaf Faaland <[email protected]> Closes #6387
* Release SCL_STATE in map_write_done()Olaf Faaland2017-07-251-5/+6
| | | | | | | | | | | | | | | | | The config lock must be held for the duration of the MMP write. Since the I/Os are executed via map_nowait(), the done function is the only place where we know the write has completed. Since SCL_STATE is taken as reader, overlapping I/Os do not create a deadlock. The refcount is simply increased when new I/Os are queued and decreased when I/Os complete. Test case added which exercises the probe IO call path to verify the fix and prevent a regression. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Olaf Faaland <[email protected]> Closes #6394
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-16 05:11:09 +0000