aboutsummaryrefslogtreecommitdiffstats
path: root/config
Commit message (Collapse)AuthorAgeFilesLines
* OpenZFS 7290 - ZFS test suite needs to control what utilities it can runJohn Wren Kennedy2017-04-062-186/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | Authored by: John Wren Kennedy <[email protected]> Reviewed by: Dan Kimmel <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Reviewed by: Dan McDonald <[email protected]> Approved by: Gordon Ross <[email protected]> Ported-by: Brian Behlendorf <[email protected]> Ported-by: George Melikov <[email protected]> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
* glibc 2.5 compat: use correct header for makedev() et al.Olaf Faaland2017-03-312-0/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In glibc 2.5, makedev(), major(), and minor() are defined in sys/sysmacros.h. They are also defined in types.h for backward compatability, but using these definitions triggers a compile warning. This breaks the ZFS build, as it builds with -Werror. autoconf email threads indicate these macros may be defined in sys/mkdev.h in some cases. This commit adds configure checks to detect where makedev() is defined: sys/sysmacros.h sys/mkdev.h It assumes major() and minor() are defined in the same place. The libspl types.h then includes sys/sysmacros.h (preferred) or sys/mkdev.h (2nd choice) if one of those defines makedev(). This is done before including the system types.h. An alternative would be to remove uses of major, minor, and makedev, instead comparing the st_dev returned from stat64. These configure checks would then be unnecessary. This change revealed that __NORETURN was being defined unnecessarily in libspl/include/sys/sysmacros.h. That definition is removed. The files in which __NORETURN are used all include types.h, and so all will get the definition provided by feature_tests.h Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Olaf Faaland <[email protected]> Closes #5945
* GZIP compression offloading with QAT acceleratorwli52017-03-222-0/+93
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implement the hardware accelerator method in GZIP compression in ZFS. When the ZFS pool is enabled GZIP compression, the compression API will be automatically transferred to the hardware accelerator to free up CPU resource and speed up the compression time. * To enable Intel QAT hardware acceleration in ZOL you need to have QAT hardware and the driver installed: * QAT hardware DH8950: http://ark.intel.com/products/79483/Intel-QuickAssist-Adapter-8950 * QAT driver: https://01.org/intel-quickassist-technology * Start QAT driver in your system: service qat_service start * Enable QAT in ZFS, e.g.: ./configure --with-qat=<qat-driver-path>/QAT1.6 make * Set GZIP compression in ZFS dataset: zfs set compression = gzip <dataset> * Get QAT hardware statistics by: cat /proc/spl/kstat/zfs/qat * To disable QAT in ZFS: insmod zfs.ko zfs_qat_disable=1 Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Jinshan Xiong <[email protected]> Signed-off-by: Weigang Li <[email protected]> Closes #5846
* Linux 4.11 compat: iops.getattr and friendsOlaf Faaland2017-03-202-0/+68
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In torvalds/linux@a528d35, there are changes to the getattr family of functions, struct kstat, and the interface of inode_operations .getattr. The inode_operations .getattr and simple_getattr() interface changed to: int (*getattr) (const struct path *, struct dentry *, struct kstat *, u32 request_mask, unsigned int query_flags) The request_mask argument indicates which field(s) the caller intends to use. Fields the caller has not specified via request_mask may be set in the returned struct anyway, but their values may be approximate. The query_flags argument indicates whether the filesystem must update the attributes from the backing store. Currently both fields are ignored. It is possible that getattr-related functions within zfs could be optimized based on the request_mask. struct kstat includes new fields: u32 result_mask; /* What fields the user got */ u64 attributes; /* See STATX_ATTR_* flags */ struct timespec btime; /* File creation time */ Fields attribute and btime are cleared; the result_mask reflects this. These appear to be optional based on simple_getattr() and vfs_getattr() within the kernel, which take the same approach. Reviewed-by: Chunwei Chen <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Olaf Faaland <[email protected]> Closes #5875
* [icp] fpu and asm cleanup for linuxGvozden Neskovic2017-03-072-0/+20
| | | | | | | | | | | | | | Properly annotate functions and data section so that objtool does not complain when CONFIG_STACK_VALIDATION and CONFIG_FRAME_POINTER are enabled. Pass KERNELCPPFLAGS to assembler. Use kfpu_begin()/kfpu_end() to protect SIMD regions in Linux kernel. Reviewed-by: Tom Caputi <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Gvozden Neskovic <[email protected]> Closes #5872 Closes #5041
* OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_spaceBrian Behlendorf2017-03-071-24/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reviewed by: Steve Gonczi <[email protected]> Reviewed by: George Wilson <[email protected]> Reviewed by: Pavel Zakharov <[email protected]> Ported-by: Brian Behlendorf <[email protected]> Background information: This assertion about tx_space_* verifies that we are not dirtying more stuff than we thought we would. We “need” to know how much we will dirty so that we can check if we should fail this transaction with ENOSPC/EDQUOT, in dmu_tx_assign(). While the transaction is open (i.e. between dmu_tx_assign() and dmu_tx_commit() — typically less than a millisecond), we call dbuf_dirty() on the exact blocks that will be modified. Once this happens, the temporary accounting in tx_space_* is unnecessary, because we know exactly what blocks are newly dirtied; we call dnode_willuse_space() to track this more exact accounting. The fundamental problem causing this bug is that dmu_tx_hold_*() relies on the current state in the DMU (e.g. dn_nlevels) to predict how much will be dirtied by this transaction, but this state can change before we actually perform the transaction (i.e. call dbuf_dirty()). This bug will be fixed by removing the assertion that the tx_space_* accounting is perfectly accurate (i.e. we never dirty more than was predicted by dmu_tx_hold_*()). By removing the requirement that this accounting be perfectly accurate, we can also vastly simplify it, e.g. removing most of the logic in dmu_tx_count_*(). The new tx space accounting will be very approximate, and may be more or less than what is actually dirtied. It will still be used to determine if this transaction will put us over quota. Transactions that are marked by dmu_tx_mark_netfree() will be excepted from this check. We won’t make an attempt to determine how much space will be freed by the transaction — this was rarely accurate enough to determine if a transaction should be permitted when we are over quota, which is why dmu_tx_mark_netfree() was introduced in 2014. We also won’t attempt to give “credit” when overwriting existing blocks, if those blocks may be freed. This allows us to remove the do_free_accounting logic in dbuf_dirty(), and associated routines. This logic attempted to predict what will be on disk when this txg syncs, to know if the overwritten block will be freed (i.e. exists, and has no snapshots). OpenZFS-issue: https://www.illumos.org/issues/7793 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/3704e0a Upstream bugs: DLPX-32883a Closes #5804 Porting notes: - DNODE_SIZE replaced with DNODE_MIN_SIZE in dmu_tx_count_dnode(), Using the default dnode size would be slightly better. - DEBUG_DMU_TX wrappers and configure option removed. - Resolved _by_dnode() conflicts these changes have not yet been applied to OpenZFS.
* Fix loop device becomes read-onlyChunwei Chen2017-03-061-1/+21
| | | | | | | | | | | | | Commit 933ec99 removes read and write from f_op because the vfs layer will select iter_write or aio_write automatically. However, for Linux <= 4.0, loop_set_fd will actually check f_op->write and set read-only if not exists. This patch add them back and use the generic do_sync_{read,write} for aio_{read,write} and new_sync_{read,write} for {read,write}_iter. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Closes #5776 Closes #5855
* Add auto-online test for ZED/FMA as part of the ZTSSydney Vanda2017-02-281-0/+3
| | | | | | | | | | | | | | | | | | | | | | Automated auto-online test to go along with ZED FMA integration (PR 4673) auto_online_001.pos works with real devices (sd- and mpath) and with non-real block devices (loop) by adding a scsi_debug device to the pool Note: In order for test group to run, ZED must not currently be running. Kernel 3.16.37 or higher needed for scsi_debug to work properly If timeout occurs on test using a scsi_debug device (error noticed on Ubuntu system), a reboot might be needed in order for test to pass. (more investigation into this) Also suppressed output from is_real_device/is_loop_device/is_mpath_device - was making the log file very cluttered with useless error messages "ie /dev/mapper/sdc is not a block device" from previous patch Reviewed-by: Don Brady <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: David Quigley <[email protected]> Signed-off-by: Sydney Vanda <[email protected]> Closes #5774
* Allow c99 code to compileMatthew Ahrens2017-02-082-0/+2
| | | | | | | | | | | | Add the appropriate compiler flags to accept c99 code. This will help to minimize differences with upstream, and aid porting changes. One change was necessary in zvol.c because the DEFINE_IDA() macro does not work with the new compiler flags. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Chunwei Chen <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #5756
* Add -Wno-declaration-after-statement to KERNELCPPFLAGSBrian Behlendorf2017-01-281-0/+1
| | | | | | | | | | | | | | | | Disable the warnings regarding ISO C90 forbidding mixed declarations and code. While this functionality was introduced as part of C99 gcc does allow this in C90 mode as an extension. https://gcc.gnu.org/onlinedocs/gcc/Mixed-Declarations.html#Mixed-Declarations Allowing this usage helps minimize the changes required when porting patches from OpenZFS. The downside here is that this functionality is specific to gcc. Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #5686
* Retire .write/.read file operationsChunwei Chen2017-01-272-0/+22
| | | | | | | | | | | | | | | The .write/.read file operations callbacks can be retired since support for .read_iter/.write_iter and .aio_read/.aio_write has been added. The vfs_write()/vfs_read() entry functions will select the correct interface for the kernel. This is desirable because all VFS write/read operations now rely on common code. This change also add the generic write checks to make sure that ulimits are enforced correctly on write. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Closes #5587 Closes #5673
* Fix spellingka72017-01-033-3/+3
| | | | | | | | | Reviewed-by: Brian Behlendorf <[email protected] Reviewed-by: Giuseppe Di Natale <[email protected]>> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Haakan T Johansson <[email protected]> Closes #5547 Closes #5543
* 4.10 compat - BIO flag changes and othersTim Chase2016-12-303-4/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | [bio] The req_op enum was changed to req_opf. Update the "Linux 4.8 API" autotools checks to use an int to determine whether the various REQ_OP values are defined. This should work properly on kernels >= 4.8. [bio] bio_set_op_attrs() is now an inline function and can't be detected with #ifdef. Add a configure check to determine whether bio_set_op_attrs() is defined. Move the local definition of it from vdev_disk.c to blkdev_compat.h for consistency with other related compability shims. [bio] The read/write flags and their modifiers, including WRITE_FLUSH, WRITE_FUA and WRITE_FLUSH_FUA have been removed from fs.h. Add the new bio_set_flush() compatibility wrapper to replace VDEV_WRITE_FLUSH_FUA and set the flags appropriately for each supported kernel version. [vfs] The generic_readlink() function has been made static. If .readlink in inode_operations is NULL, generic_readlink() is used. [zol typo] Completely unrelated to 4.10 compat, fix a typo in the check for REQ_OP_SECURE_ERASE so that the proper macro is defined: s/HAVE_REQ_OP_SECURE_DISCARD/HAVE_REQ_OP_SECURE_ERASE/ Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Chunwei Chen <[email protected]> Signed-off-by: Tim Chase <[email protected]> Closes #5499
* Use inode_set_flags when availableChunwei Chen2016-12-162-0/+19
| | | | Signed-off-by: Chunwei Chen <[email protected]>
* Add -c to zpool iostat & status to run commandTony Hutter2016-11-292-16/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds a command (-c) option to zpool status and zpool iostat. The -c option allows you to run an arbitrary command on each vdev and display the first line of output in zpool status/iostat. The environment vars VDEV_PATH and VDEV_UPATH are set to the vdev's path and "underlying path" before running the command. For device mapper, multipath, or partitioned vdevs, VDEV_UPATH is the actual underlying /dev/sd* disk. This can be useful if the command you're running requires a /dev/sd* device. The patch also uses /sys/block/<dev>/slaves/ to lookup the underlying device instead of using libdevmapper. This not only removes the libdevmapper requirement at build time, but also allows you to resolve device mapper devices without being root. This means that UDEV_UPATH get set correctly when running zpool status/iostat as an unprivileged user. Example: $ zpool status -c 'echo I am $VDEV_PATH, $VDEV_UPATH' NAME STATE READ WRITE CKSUM mypool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 mpatha ONLINE 0 0 0 I am /dev/mapper/mpatha, /dev/sdc sdb ONLINE 0 0 0 I am /dev/sdb1, /dev/sdb Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes #5368
* Allow zfs unshare <protocol> -aLOLi2016-11-291-0/+1
| | | | | | | | | | | | | | | | | | Allow `zfs unshare <protocol> -a` command to share or unshare all datasets of a given protocol, nfs or smb. Additionally, enable most of ZFS Test Suite zfs_share/zfs_unshare test cases. To work around some Illumos-specific functionalities ($SHARE/$UNSHARE) some function wrappers were added around them. Finally, fix and issue in smb_is_share_active() that would leave SMB shares exported when invoking 'zfs unshare -a' Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Turbo Fredriksson <[email protected]> Signed-off-by: loli10K <[email protected]> Closes #3238 Closes #5367
* Kernel 4.9 compat: file_operations->aio_fsync removalDeHackEd2016-11-152-0/+22
| | | | | | | Linux kernel commit 723c038475b78 removed this field. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: DHE <[email protected]> Closes #5393
* Fix ZFS_AC_KERNEL_SET_CACHED_ACL_USABLE checkGvozden Neskovic2016-11-091-2/+2
| | | | | | | | | | | | | Pass `ACL_TYPE_ACCESS` for type parameter of `set_cached_acl()` and `forget_cached_acl()` to avoid removal of dead code after BUG() in compile time. Tested on 3.2.0 kernel. Introduced in 3779913 Reviewed-by: Massimo Maggi <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Chunwei Chen <[email protected]> Signed-off-by: Gvozden Neskovic <[email protected]> Closes #5378
* Linux 3.14 compat: assign inode->set_acltuxoko2016-11-092-0/+26
| | | | | | | | | | | | | Linux 3.14 introduces inode->set_acl(). Normally, acl modification will come from setxattr, which will handle by the acl xattr_handler, and we already handles that well. However, nfsd will directly calls inode->set_acl or return error if it doesn't exists. Reviewed-by: Tim Chase <[email protected]> Reviewed-by: Massimo Maggi <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Closes #5371 Closes #5375
* Use set_cached_acl and forget_cached_acl when possibleChunwei Chen2016-11-072-2/+32
| | | | | | | | | Originally, these two function are inline, so their usability is tied to posix_acl_release. However, since Linux 3.14, they became EXPORT_SYMBOL, so we can always use them. In this patch, we create an independent test for these two functions so we can use them when possible. Signed-off-by: Chunwei Chen <[email protected]>
* Replace ISAINFO with is_32bit functionBrian Behlendorf2016-11-071-1/+0
| | | | | | | | | The isainfo(1) utility was used by the ZFS Test Suite to determine when running on a 32-bit platform. This non-portable check has been replaced with an is_32bit helper function which uses getconf(1). The getconf(1) utility is available for Linux, FreeBSD, and Illumos. Signed-off-by: Brian Behlendorf <[email protected]>
* Add support for O_TMPFILEChunwei Chen2016-11-042-0/+24
| | | | | | | | | | | | | | | Linux 3.11 add O_TMPFILE to open(2), which allow creating an unlinked file on supported filesystem. It's basically doing open(2) and unlink(2) atomically. The filesystem support is added through i_op->tmpfile. We basically copy the create operation except we get rid of the link and name related stuff and add the new node to unlinked set. We also add support for linkat(2) to link tmpfile. However, since all previous file operation will skip ZIL, we force a txg_wait_synced to make sure we are sync safe. Signed-off-by: Chunwei Chen <[email protected]>
* Fix lookup_bdev() on UbuntuHajo Möller2016-10-261-5/+17
| | | | | | | | | | | | Ubuntu added support for checking inode permissions to lookup_bdev() in kernel commit 193fb6a2c94fab8eb8ce70a5da4d21c7d4023bee (merged in 4.4.0-6.21). Upstream bug: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1636517 This patch adds a test for Ubuntu's variant of lookup_bdev() to configure and calls the function in the correct way. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Hajo Möller <[email protected]> Closes #5336
* Fix taskq creation failure in vdev_open_children()Brian Behlendorf2016-10-241-0/+1
| | | | | | | | | | | | | | When creating and destroying pools in tight loop it's possible to exhaust the number of allowed threads on a system. This results in taskq_create() failling and a NULL dereference. Resolve the issue by falling back to opening the vdevs all synchronously. Reviewed-by: Denys Rtveliashvili <[email protected]> Reviewed-by: Håkan Johansson <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes zfsonlinux/spl#521 Closes #4637
* Linux 4.9 compat: inode_change_ok() renamed setattr_prepare()Brian Behlendorf2016-10-202-0/+24
| | | | | | | | | | | | In torvalds/linux@31051c8 the inode_change_ok() function was renamed setattr_prepare() and updated to take a dentry ratheri than an inode. Update the code to call the setattr_prepare() and add a wrapper function which call inode_change_ok() for older kernels. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Requires-spl: refs/pull/581/head
* Linux 4.9 compat: remove iops->{set,get,remove}xattrChunwei Chen2016-10-202-0/+26
| | | | | | | | In Linux 4.9, torvalds/linux@fd50eca, iops->{set,get,remove}xattr and generic_{set,get,remove}xattr are removed. xattr operations will directly go through sb->s_xattr. Signed-off-by: Chunwei Chen <[email protected]>
* Linux 4.9 compat: iops->rename() wants flagsChunwei Chen2016-10-202-0/+26
| | | | | | | In Linux 4.9, torvalds/linux@2773bf0, iops->rename() and iops->rename2() are merged together into iops->rename(), it now wants flags. Signed-off-by: Chunwei Chen <[email protected]>
* Multipath autoreplace, control enclosure LEDs, event rate limitingTony Hutter2016-10-192-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | 1. Enable multipath autoreplace support for FMA. This extends FMA autoreplace to work with multipath disks. This requires libdevmapper to be installed at build time. 2. Turn on/off fault LEDs when VDEVs become degraded/faulted/online Set ZED_USE_ENCLOSURE_LEDS=1 in zed.rc to have ZED turn on/off the enclosure LED for a drive when a drive becomes FAULTED/DEGRADED. Your enclosure must be supported by the Linux SES driver for this to work. The enclosure LED scripts work for multipath devices as well. The scripts will clear the LED when the fault is cleared. 3. Rate limit ZIO delay and checksum events so as not to flood ZED ZIO delay and checksum events are rate limited to 5/sec in the zfs module. Reviewed-by: Richard Laager <[email protected]> Reviewed by: Don Brady <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes #2449 Closes #3017 Closes #5159
* Explicit block device plugging when submitting multiple BIOsIsaac Huang2016-09-292-0/+22
| | | | | | | | | Without plugging, the default 'noop' scheduler will not merge the BIOs which are part of a large ZIO. Reviewed-by: Andreas Dilger <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Isaac Huang <[email protected]> Closes #5181
* Fix automatically generated release numberBrian Behlendorf2016-09-211-0/+8
| | | | | | | | | | | | | | | When building from the head of a branch a release number is automatically generated with `git describe` using the last tag on that branch as the base. For this to work the last tag on the branch needs to be predictable given the current META file. This logic was accidentally broken when an -rcX tag was added to the branch. Update it to search for a VERSION or VERSION-RELEASE tag. Reviewed-by: Chris Siebenmann <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #5105 Closes #5140
* Change /etc/mtab to /proc/self/mountsslashdd2016-09-201-1/+1
| | | | | | | | | | | Fix misleading error message: "The /dev/zfs device is missing and must be created.", if /etc/mtab is missing. Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Eric Desrochers <[email protected]> Closes #4680 Closes #5029
* OpenZFS - Performance regression suite for zfstestJohn Wren Kennedy2016-09-081-4/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Author: John Wren Kennedy <[email protected]> Reviewed by: Prakash Surya <[email protected]> Reviewed by: Dan Kimmel <[email protected]> Reviewed by: Matt Ahrens <[email protected]> Reviewed by: Paul Dagnelie <[email protected]> Reviewed by: Don Brady <[email protected]> Reviewed by: Richard Elling <[email protected]> Reviewed by: Brian Behlendorf <[email protected]> Reviewed-by: David Quigley <[email protected]> Approved by: Richard Lowe <[email protected]> Ported-by: Don Brady <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/6950 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/dcbf3bd6 Delphix-commit: https://github.com/delphix/delphix-os/commit/978ed49 Closes #4929 ZFS Test Suite Performance Regression Tests This was pulled into OpenZFS via the compressed arc featureand was separated out in zfsonlinux as a separate pull request from PR-4768. It originally came in as QA-4903 in Delphix-OS from John Kennedy. Expected Usage: $ DISKS="sdb sdc sdd" zfs-tests.sh -r perf-regression.run Porting Notes: 1. Added assertions in the setup script to make sure required tools (fio, mpstat, ...) are present. 2. For the config.json generation in perf.shlib used arcstats and other binaries instead of dtrace to query the values. 3. For the perf data collection: - use "zpool iostat -lpvyL" instead of the io.d dtrace script (currently not collecting zfs_read/write latency stats) - mpstat and iostat take different arguments - prefetch_io.sh is a placeholder that uses arcstats instead of dtrace 4. Build machines require fio, mdadm and sysstat pakage (YMMV). Future Work: - Need a way to measure zfs_read and zfs_write latencies per pool. - Need tools to takes two sets of output and display/graph the differences - Bring over additional regression tests from Delphix
* Real disk partitioning now enabled in test suite for LinuxSydney Vanda2016-09-081-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When using real devices, specify DISKS="sdb sdc sdd" opposed to /dev/sdb in zfs-tests.sh - otherwise errors with directory names and disk names registering as "/dev//dev/sdb" for some tests. The same goes for mpath: DISK="mpatha mpathad mpathb" Expected Usage: $ DISKS="sdb sdc sdd" zfs-tests.sh SLICE_PREFIX is now set as "p" for a loop device (ie loop0p2) or "" for a real device (ie sdb2), or either for multipath devices (ie mpatha1 or mpath1p1) instead of only "p" by default. Note that kpartx partitioning is not currently supported in this patch (ie "partx") and may need to be disabled on Debian distributions. Functions added for determining test directory (/dev or /dev/mapper) as well as slice prefix are determined and exported mostly in the cfg file of each test group directory. Currently zpools cannot be created on whole mpath devices that have been partitioned. In order to fix this tests have either been revised to use a partition instead, or if there is a size constraint and the pool needs to be created on the whole disk, partitions are then deleted if the device is a multipath device. This functionality is added to default_cleanup() or to individual cleanup scripts if a non-default cleanup method is used. The max partitions is currently set at 8 to account for all of the tests thus far. Patch changes are generally encompassed in "if is_linux" construct. Signed-off-by: Sydney Vanda <[email protected]> Reviewed-by: John Salinas <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: David Quigley <[email protected]> Closes #4447 Closes #4964 Closes #5074
* Linux compat: Grsecurity kernelGvozden Neskovic2016-08-222-0/+31
| | | | | | | | | | | API Change: Module parameter set/get methods take const parameter in Grsecurity kernel v4.7.1 Signed-off-by: Gvozden Neskovic <[email protected]> Signed-off-by: Jason Zaman <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #4997 Closes #5001
* Add support for AVX-512 family of instruction setsGvozden Neskovic2016-08-161-0/+189
| | | | | | | | | | | | | This patch adds compiler and runtime tests (user and kernel) for following instruction sets: avx512f, avx512cd, avx512er, avx512pf, avx512bw, avx512dq, avx512vl, avx512ifma, avx512vbmi. note: Linux support for AVX-512F (Foundation) instruction set started with linux v3.15 Signed-off-by: Gvozden Neskovic <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #4952
* Use file_dentry and file_inode wrappersChen Haiquan2016-08-112-0/+21
| | | | | | | | | | | | Fix bugs due to kernel change in torvalds/linux@4bacc9c9234c ("overlayfs: Make f_path always point to the overlay and f_inode to the underlay"). This problem crashes system when use zfs as a layer of overlayfs. Signed-off-by: Chen Haiquan <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #4914 Closes #4935
* Linux 4.8 compat: Fix removal of bio->bi_rw memberBrian Behlendorf2016-08-112-0/+71
| | | | | | | | | | | | | | | | | | All users of bio->bi_rw have been replaced with compatibility wrappers. This allows the kernel specific logic to be abstracted away, and for each of the supported cases to be documented with the wrapper. The updated interfaces are as follows: * void blk_queue_set_write_cache(struct request_queue *, bool, bool) * boolean_t bio_is_flush(struct bio *) * boolean_t bio_is_fua(struct bio *) * boolean_t bio_is_discard(struct bio *) * boolean_t bio_is_secure_erase(struct bio *) * VDEV_WRITE_FLUSH_FUA Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Closes #4951
* Linux 4.7 compat: fix zpl_get_acl returns invalid acl pointerChunwei Chen2016-08-092-0/+21
| | | | | | | | | | | | | | | | | Starting from Linux 4.7, get_acl will set acl cache pointer to temporary sentinel value before calling i_op->get_acl. Therefore we can't compare against ACL_NOT_CACHED and return. Since from Linux 3.14, get_acl already check the cache for us, so we disable this in zpl_get_acl. Linux 4.7 also does set_cached_acl for us so we disable it in zpl_get_acl. Signed-off-by: Chunwei Chen <[email protected]> Signed-off-by: Nikolay Borisov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #4944 Closes #4946
* Linux 4.8 compat: posix_acl_valid()Brian Behlendorf2016-08-082-0/+25
| | | | | | | | | | | | | | The posix_acl_valid() function has been updated to require a user namespace. Filesystem callers should normally provide the user_ns from the super block associcated with the ACL; the zpl_posix_acl_valid() wrapper has been added for this purpose. See https://github.com/torvalds/linux/commit/0d4d717f for complete details. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Nikolay Borisov <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Closes #4922
* Retire HAVE_CURRENT_UMASK and HAVE_POSIX_ACL_CACHINGBrian Behlendorf2016-08-082-41/+0
| | | | | | | | | | Remove ZFS_AC_KERNEL_CURRENT_UMASK and ZFS_AC_KERNEL_POSIX_ACL_CACHING configure checks, all supported kernel provide this functionality. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Nikolay Borisov <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Closes #4922
* Linux 4.8 compat: new s_user_ns member of struct super_blockNikolay Borisov2016-08-082-0/+22
| | | | | | | | | | | | | Kernel 4.8 paved the way to enabling mounting a file system inside a non-init user namespace. To facilitate this a s_user_ns member was added holding the userns in which the filesystem's instance was mounted. This enables doing the uid/gid translation relative to this particular username space and not the default init_user_ns. Signed-off-by: Nikolay Borisov <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #4928
* Linux 4.8 compat: submit_bio()Brian Behlendorf2016-07-292-0/+21
| | | | | | | | | | | | | The rw argument has been removed from submit_bio/submit_bio_wait. Callers are now expected to set bio->bi_rw instead of passing it in. See https://github.com/torvalds/linux/commit/4e49ea4a for complete details. Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #4892 Issue #4899
* Fix sync behavior for disk vdevsTim Chase2016-07-252-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prior to b39c22b, which was first generally available in the 0.6.5 release as b39c22b, ZoL never actually submitted synchronous read or write requests to the Linux block layer. This means the vdev_disk_dio_is_sync() function had always returned false and, therefore, the completion in dio_request_t.dr_comp was never actually used. In b39c22b, synchronous ZIO operations were translated to synchronous BIO requests in vdev_disk_io_start(). The follow-on commits 5592404 and aa159af fixed several problems introduced by b39c22b. In particular, 5592404 introduced the new flag parameter "wait" to __vdev_disk_physio() but under ZoL, since vdev_disk_physio() is never actually used, the wait flag was always zero so the new code had no effect other than to cause a bug in the use of the dio_request_t.dr_comp which was fixed by aa159af. The original rationale for introducing synchronous operations in b39c22b was to hurry certains requests through the BIO layer which would have otherwise been subject to its unplug timer which would increase the latency. This behavior of the unplug timer, however, went away during the transition of the plug/unplug system between kernels 2.6.32 and 2.6.39. To handle the unplug timer behavior on 2.6.32-2.6.35 kernels the BIO_RW_UNPLUG flag is used as a hint to suppress the plugging behavior. For kernels 2.6.36-2.6.38, the REQ_UNPLUG macro will be available and ise used for the same purpose. Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #4858
* Check whether the kernel supports i_uid/gid_read/write helpersNikolay Borisov2016-07-252-0/+23
| | | | | | | | | | | | | Since the concept of a kuid and the need to translate from it to ordinary integer type was added in kernel version 3.5 implement necessary plumbing to be able to detect this condition during compile time. If the kernel doesn't support the kuid then just fall back to directly accessing the respective struct inode's members Signed-off-by: Nikolay Borisov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #4685 Issue #227
* Illumos Crypto Port module added to enable native encryption in zfsTom Caputi2016-07-204-23/+23
| | | | | | | | | | | | | | | | | | | | A port of the Illumos Crypto Framework to a Linux kernel module (found in module/icp). This is needed to do the actual encryption work. We cannot use the Linux kernel's built in crypto api because it is only exported to GPL-licensed modules. Having the ICP also means the crypto code can run on any of the other kernels under OpenZFS. I ended up porting over most of the internals of the framework, which means that porting over other API calls (if we need them) should be fairly easy. Specifically, I have ported over the API functions related to encryption, digests, macs, and crypto templates. The ICP is able to use assembly-accelerated encryption on amd64 machines and AES-NI instructions on Intel chips that support it. There are place-holder directories for similar assembly optimizations for other architectures (although they have not been written). Signed-off-by: Tom Caputi <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #4329
* Add configure result for xattr_handlerChunwei Chen2016-07-121-0/+2
| | | | | | Signed-off-by: Chunwei Chen <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #4828
* OpenZFS 6314 - buffer overflow in dsl_dataset_nameIgor Kozhukhov2016-06-282-22/+0
| | | | | | | | | | | Reviewed by: George Wilson <[email protected]> Reviewed by: Prakash Surya <[email protected]> Reviewed by: Igor Kozhukhov <[email protected]> Approved by: Dan McDonald <[email protected]> Ported-by: Brian Behlendorf <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/6314 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/d6160ee
* OpenZFS 2605, 6980, 6902Matthew Ahrens2016-06-281-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 2605 want to resume interrupted zfs send Reviewed by: George Wilson <[email protected]> Reviewed by: Paul Dagnelie <[email protected]> Reviewed by: Richard Elling <[email protected]> Reviewed by: Xin Li <[email protected]> Reviewed by: Arne Jansen <[email protected]> Approved by: Dan McDonald <[email protected]> Ported-by: kernelOfTruth <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/2605 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/9c3fd12 6980 6902 causes zfs send to break due to 32-bit/64-bit struct mismatch Reviewed by: Paul Dagnelie <[email protected]> Reviewed by: George Wilson <[email protected]> Approved by: Robert Mustacchi <[email protected]> Ported by: Brian Behlendorf <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/6980 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/ea4a67f Porting notes: - All rsend and snapshop tests enabled and updated for Linux. - Fix misuse of input argument in traverse_visitbp(). - Fix ISO C90 warnings and errors. - Fix gcc 'missing braces around initializer' in 'struct send_thread_arg to_arg =' warning. - Replace 4 argument fletcher_4_native() with 3 argument version, this change was made in OpenZFS 4185 which has not been ported. - Part of the sections for 'zfs receive' and 'zfs send' was rewritten and reordered to approximate upstream. - Fix mktree xattr creation, 'user.' prefix required. - Minor fixes to newly enabled test cases - Long holds for volumes allowed during receive for minor registration.
* Linux 4.7 compat: handler->set() takes both dentry and inodeBrian Behlendorf2016-06-011-29/+54
| | | | | | | | | Counterpart to fd4c7b7, the same approach was taken to resolve the compatibility issue. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Closes #4717 Issue #4665
* Linux 4.7 compat: use iterate_shared for concurrent readdirChunwei Chen2016-05-201-14/+35
| | | | | | | | | | | | | Register iterate_shared if it exists so the kernel will used shared lock and allowing concurrent readdir. Also, use shared lock when doing llseek with SEEK_DATA or SEEK_HOLE to allow concurrent seeking. Signed-off-by: Chunwei Chen <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #4664 Closes #4665
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-22 15:22:00 +0000