aboutsummaryrefslogtreecommitdiffstats
path: root/config
Commit message (Collapse)AuthorAgeFilesLines
* Add kernel module auto-loadingBrian Behlendorf2018-03-132-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Historically a dynamic misc minor number was registered for the /dev/zfs device in order to prevent minor number collisions. This was fine but it prevented us from being able to use the kernel module auto-loaded which requires a known reserved value. Resolve this issue by adding a configure test to find an available misc minor number which can then be used in MODULE_ALIAS_MISCDEV at build time. By adding this alias the zfs kmod is added to the list of known static-nodes and the systemd-tmpfiles-setup-dev service will create a /dev/zfs character device at boot time. This in turn allows us to update the 90-zfs.rules file to make it aware this is a static node. The upshot of this is that whenever a process (zpool, zfs, zed) opens the /dev/zfs the kmods will be automatic loaded. This even works for unprivileged users so there is no longer a need to manually load the modules at boot time. As an additional bonus the zed now no longer needs to start after the zfs-import.service since it will trigger the module load. In the unlikely event the minor number we selected conflicts with another out of tree unregistered minor number the code falls back to dynamically allocating it. In this case the modules again must be manually loaded. Note that due to the change in the method of registering the minor number the zimport.sh test case may incorrectly fail when the static node for the installed packages is created instead of the dynamic one. This issue will only transiently impact zimport.sh for this single commit when we transition and are mixing and matching methods. Reviewed-by: Fabian Grünbichler <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> TEST_ZIMPORT_SKIP="yes" Closes #7287
* Fix zfs-kmod builds when using rpm >= 4.14LOLi2018-03-091-0/+1
| | | | | | | | | | | With rpm-software-management/rpm@5e94633 a package version containing invalid characters (most commonly a double '-') causes the kmod package generation to terminate with an error. This change takes advantage of the newly introduced rpm macro "_wrong_version_format_terminate_build" to allow kmod packages to be built. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: loli10K <[email protected]> Closes #7284
* Take user namespaces into account in policy checksWolfgang Bumiller2018-03-072-0/+68
| | | | | | | | | | | | | | | | | | | | | | | | | Change file related checks to use user namespaces and make sure involved uids/gids are mappable in the current namespace. Note that checks without file ownership information will still not take user namespaces into account, as some of these should be handled via 'zfs allow' (otherwise root in a user namespace could issue commands such as `zpool export`). This also adds an initial user namespace regression test for the setgid bit loss, with a user_ns_exec helper usable in further tests. Additionally, configure checks for the required user namespace related features are added for: * ns_capable * kuid/kgid_has_mapping() * user_ns in cred_t Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Wolfgang Bumiller <[email protected]> Closes #6800 Closes #7270
* Linux 4.16 compat: get_disk_and_module()Giuseppe Di Natale2018-03-052-0/+20
| | | | | | | | | | As of https://github.com/torvalds/linux/commit/fb6d47a, get_disk() is now get_disk_and_module(). Add a configure check to determine if we need to use get_disk_and_module(). Reviewed-by: loli10K <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Giuseppe Di Natale <[email protected]> Closes #7264
* Fix free memory calculation on v3.14+chrisrd2018-02-233-23/+110
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Provide infrastructure to auto-configure to enum and API changes in the global page stats used for our free memory calculations. arc_free_memory has been broken since an API change in Linux v3.14: 2016-07-28 v4.8 599d0c95 mm, vmscan: move LRU lists to node 2016-07-28 v4.8 75ef7184 mm, vmstat: add infrastructure for per-node vmstats These commits moved some of global_page_state() into global_node_page_state(). The API change was particularly egregious as, instead of breaking the old code, it silently did the wrong thing and we continued using global_page_state() where we should have been using global_node_page_state(), thus indexing into the wrong array via NR_SLAB_RECLAIMABLE et al. There have been further API changes along the way: 2017-07-06 v4.13 385386cf mm: vmstat: move slab statistics from zone to node counters 2017-09-06 v4.14 c41f012a mm: rename global_page_state to global_zone_page_state ...and various (incomplete, as it turns out) attempts to accomodate these changes in ZoL: 2017-08-24 2209e409 Linux 4.8+ compatibility fix for vm stats 2017-09-16 787acae0 Linux 3.14 compat: IO acct, global_page_state, etc 2017-09-19 661907e6 Linux 4.14 compat: IO acct, global_page_state, etc The config infrastructure provided here resolves these issues going back to the original API change in v3.14 and is robust against further Linux changes in this area. Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Chris Dunlop <[email protected]> Closes #7170
* Linux 4.16 compat: use correct *_dec_and_test()Tony Hutter2018-02-222-0/+21
| | | | | | | | | | Use refcount_dec_and_test() on 4.16+ kernels, atomic_dec_and_test() on older kernels. https://lwn.net/Articles/714974/ Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes: #7179 Closes: #7211
* Fix config issues: frame size and headerschrisrd2018-02-1511-11/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. With various (debug and/or tracing?) kernel options enabled it's possible for 'struct inode' and 'struct super_block' to exceed the default frame size, leaving errors like this in config.log: build/conftest.c:116:1: error: the frame size of 1048 bytes is larger than 1024 bytes [-Werror=frame-larger-than=] Fix this by removing the frame size warning for config checks 2. Without the correct headers included, it's possible for declarations to be missed, leaving errors like this in the config.log: build/conftest.c:131:14: error: ‘struct nameidata’ declared inside parameter list [-Werror] Fix this by adding appropriate headers. Note: Both these issues can result in silent config failures because the compile failure is taken to mean "this option is not supported by this kernel" rather than "there's something wrong with the config test". This can lead to something merely annoying (compile failures) to something potentially serious (miscompiled or misused kernel primitives or functions). E.g. the fixes included here resulted in these additional defines in zfs_config.h with linux v4.14.19: Also, drive-by whitespace fixes in config/* files which don't mention "GNU" (those ones look to be imported from elsewhere so leave them alone). Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: Chris Dunlop <[email protected]> Closes #7169
* Linux 4.16 compat: inode_set_iversion()Brian Behlendorf2018-02-082-0/+20
| | | | | | | | | | | | | | | A new interface was added to manipulate the version field of an inode. Add a inode_set_iversion() wrapper for older kernels and use the new interface when available. The i_version field was dropped from the trace point due to the switch to an atomic64_t i_version type. Reviewed-by: Olaf Faaland <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Chunwei Chen <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #7148
* Fix default libdir for Debian/UbuntuBrian Behlendorf2018-02-051-0/+13
| | | | | | | | | | | | | The distribution provided architecture specific RPM macro files for x86_64 and other architectures on Debian/Ubuntu specify the wrong default libdir install location. When building deb packages override _lib with the correct location. Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: loli10K <[email protected]> Reviewed by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #7083 Closes #7101
* Fix Debian packaging on ARMv7/ARM64LOLi2018-01-181-3/+6
| | | | | | | | | | | | | | | When building packages on Debian-based systems specify the target architecture used by 'alien' to convert .rpm packages into .deb: this avoids detecting an incorrect value which results in the following errors: <package>.aarch64.rpm is for architecture aarch64 ; the package cannot be built on this system <package>.armv7l.rpm is for architecture armel ; the package cannot be built on this system Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: loli10K <[email protected]> Closes #7046 Closes #7058
* Support -fsanitize=address with --enable-asanBrian Behlendorf2018-01-109-149/+226
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When --enable-asan is provided to configure then build all user space components with fsanitize=address. For kernel support use the Linux KASAN feature instead. https://github.com/google/sanitizers/wiki/AddressSanitizer When using gcc version 4.8 any test case which intentionally generates a core dump will fail when using --enable-asan. The default behavior is to disable core dumps and only newer versions allow this behavior to be controled at run time with the ASAN_OPTIONS environment variable. Additionally, this patch includes some build system cleanup. * Rules.am updated to set the minimum AM_CFLAGS, AM_CPPFLAGS, and AM_LDFLAGS. Any additional flags should be added on a per-Makefile basic. The --enable-debug and --enable-asan options apply to all user space binaries and libraries. * Compiler checks consolidated in always-compiler-options.m4 and renamed for consistency. * -fstack-check compiler flag was removed, this functionality is provided by asan when configured with --enable-asan. * Split DEBUG_CFLAGS in to DEBUG_CFLAGS, DEBUG_CPPFLAGS, and DEBUG_LDFLAGS. * Moved default kernel build flags in to module/Makefile.in and split in to ZFS_MODULE_CFLAGS and ZFS_MODULE_CPPFLAGS. These flags are set with the standard ccflags-y kbuild mechanism. * -Wframe-larger-than checks applied only to binaries or libraries which include source files which are built in both user space and kernel space. This restriction is relaxed for user space only utilities. * -Wno-unused-but-set-variable applied only to libzfs and libzpool. The remaining warnings are the result of an ASSERT using a variable when is always declared. * -D_POSIX_PTHREAD_SEMANTICS and -D__EXTENSIONS__ dropped because they are Solaris specific and thus not needed. * Ensure $GDB is defined as gdb by default in zloop.sh. Signed-off-by: DHE <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #7027
* Fix incompatibility with Reiser4 patched kernelsRichard Yao2018-01-091-3/+11
| | | | | | | | | | | | | | | | | | | | | | | | In ZFSOnLinux, our sources and build system are self contained such that we do not need to make changes to the Linux kernel sources. Reiser4 on the other hand exists solely as a kernel tree patch and opts to make changes to the kernel rather than adapt to it. After Linux 4.1 made a VFS change that replaced new_sync_read with do_sync_read, Reiser4's maintainer decided to modify the kernel VFS to export the old function. This caused our autotools check to misidentify the kernel API as predating Linux 4.1 on kernels that have been patched with Reiser4 support, which breaks our build. Reiser4 really should be patched to stop doing this, but lets modify our check to be more strict to help the affected users of both filesystems. Also, we were not checking the types of arguments and return value of new_sync_read() and new_sync_write() . Lets fix that too. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Richard Yao <[email protected]> Closes #6241 Closes #7021
* Fix --with-systemd on Debian-based distributions (#6963)LOLi2017-12-172-5/+17
| | | | | | | | | | These changes propagate the "--with-systemd" configure option to the RPM spec file, allowing Debian-based distributions to package systemd-related files. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: loli10K <[email protected]> Closes #6591 Closes #6963
* Suppress incorrect objtool warningsBrian Behlendorf2017-12-071-1/+24
| | | | | | | | | | | | Suppress incorrect warnings from versions of objtool which are not aware of x86 EVEX prefix instructions used for AVX512. module/zfs/vdev_raidz_math_avx512bw.o: warning: objtool: <func+offset>: can't find jump dest instruction at .text Reviewed-by: Don Brady <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6928
* Support integration with new QAT productswli52017-10-201-2/+2
| | | | | | | | | | | | Support integration with new QAT products: Intel(R) C62x Chipset, or Atom(R) C3000 Processor Product Family SoC: 1. Detect new file name in auto-conf. 2. Change MAX_INSTANCES to 48. 3. Change "num_inst" to U16 to clean a build warning. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Weigang Li <[email protected]> Closes #6767
* Add DKMS package on Debian-based distributionsNeal Gompa (ニール・ゴンパ)2017-10-151-5/+10
| | | | | | | | | | | | | | * config/deb.am: Enable building DKMS packages for Debian * rpm/generic/zfs-dkms.spec.in: Adjust spec to be Debian-compatible * Condition kernel-devel Req to RPM distros * Adjust the DKMS Req to have a minimum of a version only * Ensure that --rpm_safe_upgrade isn't used on non-RPM distros * config/deb.am: Drop CONFIG_KERNEL and CONFIG_USER guards * Makefile.am: Add pkg-dkms target Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Neal Gompa <[email protected]> Closes #6044 Closes #6731
* Encryption patch follow-upTom Caputi2017-10-112-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * PBKDF2 implementation changed to OpenSSL implementation. * HKDF implementation moved to its own file and tests added to ensure correctness. * Removed libzfs's now unnecessary dependency on libzpool and libicp. * Ztest can now create and test encrypted datasets. This is currently disabled until issue #6526 is resolved, but otherwise functions as advertised. * Several small bug fixes discovered after enabling ztest to run on encrypted datasets. * Fixed coverity defects added by the encryption patch. * Updated man pages for encrypted send / receive behavior. * Fixed a bug where encrypted datasets could receive DRR_WRITE_EMBEDDED records. * Minor code cleanups / consolidation. Signed-off-by: Tom Caputi <[email protected]>
* Fix "--enable-code-coverage" debug buildBrian Behlendorf2017-09-221-1/+1
| | | | | | | When --enable-code-coverage is provided it should not result in NDEBUG being defined. This is controlled by --enable-debug. Signed-off-by: Brian Behlendorf <[email protected]> Closes #6674
* Add support for "--enable-code-coverage" optionPrakash Surya2017-09-223-31/+266
| | | | | | | | | | | | | | | | | | This change adds support for a new option that can be passed to the configure script: "--enable-code-coverage". Further, the "--enable-gcov" option has been removed, as this new option provides the same functionality (plus more). When using this new option the following make targets are available: * check-code-coverage * code-coverage-capture * code-coverage-clean Note: these make targets can only be run from the root of the project. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Prakash Surya <[email protected]> Closes #6670
* Linux 3.14 compat: IO acct, global_page_state, etcGiuseppe Di Natale2017-09-163-5/+56
| | | | | | | | | | | | | | | | | | | | | | generic_start_io_acct/generic_end_io_acct in the master branch of the linux kernel requires that the request_queue be provided. Move the logic from freemem in the spl to arc_free_memory in arc.c. Do this so we can take advantage of global_page_state interface checks in zfs. Upstream kernel replaced struct block_device with struct gendisk in struct bio. Determine if the function bio_set_dev exists during configure and have zfs use that if it exists. bio_set_dev https://github.com/torvalds/linux/commit/74d4699 global_node_page_state https://github.com/torvalds/linux/commit/75ef718 io acct https://github.com/torvalds/linux/commit/d62e26b Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Giuseppe Di Natale <[email protected]> Closes #6635
* Make "-fno-inline" compile option more accessiblePrakash Surya2017-09-151-1/+2
| | | | | | | | | | | When functions are inlined, it can make the system much more difficult to instrument using tools such as ftrace, BPF, crash, etc. Thus, to aid development and increase the system's observability, when the "--enable-debuginfo" flag is specified, the "-fno-inline" compilation option will be used for both userspace and kernel modules. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Prakash Surya <[email protected]> Closes #6605
* Add configure option to enable gcov analysisBrian Behlendorf2017-09-151-0/+31
| | | | | | | | | * Add configure option to enable gcov analysis. * Includes a few minor ctime fixes. * Add codecov.yml configuration. Reviewed-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6642
* Implement --enable-debuginfo to force debuginfoRichard Yao2017-08-291-1/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Inspection of a Ubuntu 14.04 x64 system revealed that the config file used to build the kernel image differs from the config file used to build kernel modules by the presence of CONFIG_DEBUG_INFO=y: This in itself is insufficient to show that the kernel is built with debuginfo, but a cursory analysis of the debuginfo provided and the size of the kernel strongly suggests that it was built with CONFIG_DEBUG_INFO=y while the modules were not. Installing linux-image-$(uname -r)-dbgsym had no obvious effect on the debuginfo provided by either the modules or the kernel. The consequence is that issue reports from distributions such as Ubuntu and its derivatives build kernel modules without debuginfo contain nonsensical backtraces. It is therefore desireable to force generation of debuginfo, so we implement --enable-debuginfo. Since the build system can build both userspace components and kernel modules, the generic --enable-debuginfo option will force debuginfo for both. However, it also supports --enable-debuginfo=kernel and --enable-debuginfo=user for finer grained control. Enabling debuginfo for the kernel modules works by injecting CONFIG_DEBUG_INFO=y into the make environment. This is enables generation of debuginfo by the kernel build systems on all Linux kernels, but the build environment is slightly different int hat CONFIG_DEBUG_INFO has not been in the CPP. Adding -DCONFIG_DEBUG_INFO would fix that, but it would also cause build failures on kernels where CONFIG_DEBUG_INFO=y is already set. That would complicate its use in DKMS environments that support a range of kernels and is therefore undesireable. We could write a compatibility shim to enable CONFIG_DEBUG_INFO only when it is explicitly disabled, but we forgo doing that because it is unnecessary. Nothing in ZoL or the kernel uses CONFIG_DEBUG_INFO in the CPP at this time and that is unlikely to change. Enabling debuginfo for the userspace components is done by injecting -g into CPPFLAGS. This is not necessary because the build system honors the environment's CPPFLAGS by appending them to the actual CPPFLAGS used, but it is supported for consistency. Reviewed-by: Chunwei Chen <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Richard Yao <[email protected]> Closes #2734
* Make --enable-debug fail when given bogus argsRichard Yao2017-08-291-18/+25
| | | | | | | | | | | | | | | Currently, bogus options to --enable-debug become --disable-debug. That means that passing --enable-debug=true is analogous to --disable-debug, but the result is counterintuitive. We switch to AS_CASE to allow us to fail when given a bogus option. Also, we modify the text printed to clarify that --enable-debug enables assertions. Reviewed-by: Chunwei Chen <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Richard Yao <[email protected]> Closes #2734
* Linux 4.8+ compatibility fix for vm statsdbavatar2017-08-242-0/+23
| | | | | | | | | | | | vm_node_stat must be used instead of vm_zone_stat. Unfortunately the old code still compiles potentially leading to silent failure of arc_evictable_memory() AKAMAI: CR 3816601: Regression in zfs dropcache test Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Chunwei Chen <[email protected]> Signed-off-by: Debabrata Banerjee <[email protected]> Closes #6528
* Fix autoconf detection of super_setup_bdi_nameJustin Bedő2017-07-251-1/+2
| | | | | | | | | | | | | | The previous autoconf test for the presence of super_setup_bdi_name() uses an invocation with an incorrect type signature, producing a warning by the compiler when the test is run. This gets elevated to an error when compiling with -Werror=format-security, causing autoconf to falsely infer super_setup_bdi_name() is not present. This updates the testing code to match the invocation used in include/linux/vfs_compat.h. Reviewed-by: loli10K <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Justin Bedo <[email protected]> Closes #6398
* Linux 4.13 compat: bio->bi_status and blk_status_tBrian Behlendorf2017-07-232-0/+23
| | | | | | | | | Commit torvalds/linux@4e4cbee9. The bio->bi_error field was replaced with bio->bi_status which is an enum that describes all possible error types. Reviewed-by: Chunwei Chen <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6351
* Prevent dependencies on Debianized packagesAntonio Russo2017-07-071-1/+16
| | | | | | | | | | | | | | Call dpkg-shlibdeps with arguments excluding the Debianized packages lib{uutil1,nvpair1,zfs2,zpool2}linux from the auto-generated dependencies of generated .debs. A shim dh_shlibdeps that calls the real dh_shlibdeps with corresponding arguments is installed into a temporary directory, which is in turn pre-pended to the PATH for the alien call, working around alien's inability to directly alter the dependencies of its output debs. Resolves #6106. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Antonio Russo <[email protected]> Closes #6309 Closes #6106
* GCC 7.1 fixesTony Hutter2017-06-282-0/+23
| | | | | | | | | | | GCC 7.1 with will warn when we're not checking the snprintf() return code in cases where the buffer could be truncated. This patch either checks the snprintf return code (where applicable), or simply disables the warnings (ztest.c). Reviewed-by: Chunwei Chen <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes #6253
* Fix RHEL 7.4 bio_set_op_attrs build errorTony Hutter2017-06-271-1/+1
| | | | | | | | | | | | On RHEL 7.4, include/linux/bio.h now includes a macro for bio_set_op_attrs that conflicts with the ifndef in ZFS include/linux/blkdev_compat.h. This patch fixes the build. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes #6234 Closes #6271
* config: allow --with-linux without --with-linux-objChunwei Chen2017-05-251-1/+2
| | | | | | | | | | Don't use `uname -r` to determine kernel build directory when the user specified kernel source with --with-linux. Otherwise, the user is forced to use --with-linux-obj even if they are the same directory, which is very counterintuitive. Signed-off-by: Chunwei Chen <[email protected]> Requires-spl: refs/pull/617/head
* Linux 4.12 compat: CURRENT_TIME removedBrian Behlendorf2017-05-102-0/+20
| | | | | | | | Linux 4.9 added current_time() as the preferred interface to get the filesystem time. CURRENT_TIME was retired in Linux 4.12. Reviewed-by: Chunwei Chen <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6114
* Enable Linux read-ahead for a single page on ZVOLsRichard Yao2017-05-042-0/+21
| | | | | | | | | | | | | | | | | | | | Linux has read-ahead logic designed to accelerate sequential workloads. ZFS has its own read-ahead logic called zprefetch that operates on both ZVOLs and datasets. Having two prefetchers active at the same time can cause overprefetching, which unnecessarily reduces IOPS performance on CoW filesystems like ZFS. Testing shows that entirely disabling the Linux prefetch results in a significant performance penalty for reads while commensurate benefits are seen in random writes. It appears that read-ahead benefits are inversely proportional to random write benefits, and so a single page of Linux-layer read-ahead appears to offer the middle ground for both workloads. Reviewed-by: Chunwei Chen <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Richard Yao <[email protected]> Issue #5902
* Linux 4.12 compat: super_setup_bdi_name()Brian Behlendorf2017-05-023-39/+57
| | | | | | | | | | All filesystems were converted to dynamically allocated BDIs. The destruction of backing_dev_info structures is handled as part of super block destruction. Refactor the code to abstract away the details of creating and destroying a BDI. Reviewed-by: Chunwei Chen <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #6089
* OpenZFS 7290 - ZFS test suite needs to control what utilities it can runJohn Wren Kennedy2017-04-062-186/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | Authored by: John Wren Kennedy <[email protected]> Reviewed by: Dan Kimmel <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Reviewed by: Dan McDonald <[email protected]> Approved by: Gordon Ross <[email protected]> Ported-by: Brian Behlendorf <[email protected]> Ported-by: George Melikov <[email protected]> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
* glibc 2.5 compat: use correct header for makedev() et al.Olaf Faaland2017-03-312-0/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In glibc 2.5, makedev(), major(), and minor() are defined in sys/sysmacros.h. They are also defined in types.h for backward compatability, but using these definitions triggers a compile warning. This breaks the ZFS build, as it builds with -Werror. autoconf email threads indicate these macros may be defined in sys/mkdev.h in some cases. This commit adds configure checks to detect where makedev() is defined: sys/sysmacros.h sys/mkdev.h It assumes major() and minor() are defined in the same place. The libspl types.h then includes sys/sysmacros.h (preferred) or sys/mkdev.h (2nd choice) if one of those defines makedev(). This is done before including the system types.h. An alternative would be to remove uses of major, minor, and makedev, instead comparing the st_dev returned from stat64. These configure checks would then be unnecessary. This change revealed that __NORETURN was being defined unnecessarily in libspl/include/sys/sysmacros.h. That definition is removed. The files in which __NORETURN are used all include types.h, and so all will get the definition provided by feature_tests.h Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Olaf Faaland <[email protected]> Closes #5945
* GZIP compression offloading with QAT acceleratorwli52017-03-222-0/+93
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implement the hardware accelerator method in GZIP compression in ZFS. When the ZFS pool is enabled GZIP compression, the compression API will be automatically transferred to the hardware accelerator to free up CPU resource and speed up the compression time. * To enable Intel QAT hardware acceleration in ZOL you need to have QAT hardware and the driver installed: * QAT hardware DH8950: http://ark.intel.com/products/79483/Intel-QuickAssist-Adapter-8950 * QAT driver: https://01.org/intel-quickassist-technology * Start QAT driver in your system: service qat_service start * Enable QAT in ZFS, e.g.: ./configure --with-qat=<qat-driver-path>/QAT1.6 make * Set GZIP compression in ZFS dataset: zfs set compression = gzip <dataset> * Get QAT hardware statistics by: cat /proc/spl/kstat/zfs/qat * To disable QAT in ZFS: insmod zfs.ko zfs_qat_disable=1 Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Jinshan Xiong <[email protected]> Signed-off-by: Weigang Li <[email protected]> Closes #5846
* Linux 4.11 compat: iops.getattr and friendsOlaf Faaland2017-03-202-0/+68
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In torvalds/linux@a528d35, there are changes to the getattr family of functions, struct kstat, and the interface of inode_operations .getattr. The inode_operations .getattr and simple_getattr() interface changed to: int (*getattr) (const struct path *, struct dentry *, struct kstat *, u32 request_mask, unsigned int query_flags) The request_mask argument indicates which field(s) the caller intends to use. Fields the caller has not specified via request_mask may be set in the returned struct anyway, but their values may be approximate. The query_flags argument indicates whether the filesystem must update the attributes from the backing store. Currently both fields are ignored. It is possible that getattr-related functions within zfs could be optimized based on the request_mask. struct kstat includes new fields: u32 result_mask; /* What fields the user got */ u64 attributes; /* See STATX_ATTR_* flags */ struct timespec btime; /* File creation time */ Fields attribute and btime are cleared; the result_mask reflects this. These appear to be optional based on simple_getattr() and vfs_getattr() within the kernel, which take the same approach. Reviewed-by: Chunwei Chen <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Olaf Faaland <[email protected]> Closes #5875
* [icp] fpu and asm cleanup for linuxGvozden Neskovic2017-03-072-0/+20
| | | | | | | | | | | | | | Properly annotate functions and data section so that objtool does not complain when CONFIG_STACK_VALIDATION and CONFIG_FRAME_POINTER are enabled. Pass KERNELCPPFLAGS to assembler. Use kfpu_begin()/kfpu_end() to protect SIMD regions in Linux kernel. Reviewed-by: Tom Caputi <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Gvozden Neskovic <[email protected]> Closes #5872 Closes #5041
* OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_spaceBrian Behlendorf2017-03-071-24/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reviewed by: Steve Gonczi <[email protected]> Reviewed by: George Wilson <[email protected]> Reviewed by: Pavel Zakharov <[email protected]> Ported-by: Brian Behlendorf <[email protected]> Background information: This assertion about tx_space_* verifies that we are not dirtying more stuff than we thought we would. We “need” to know how much we will dirty so that we can check if we should fail this transaction with ENOSPC/EDQUOT, in dmu_tx_assign(). While the transaction is open (i.e. between dmu_tx_assign() and dmu_tx_commit() — typically less than a millisecond), we call dbuf_dirty() on the exact blocks that will be modified. Once this happens, the temporary accounting in tx_space_* is unnecessary, because we know exactly what blocks are newly dirtied; we call dnode_willuse_space() to track this more exact accounting. The fundamental problem causing this bug is that dmu_tx_hold_*() relies on the current state in the DMU (e.g. dn_nlevels) to predict how much will be dirtied by this transaction, but this state can change before we actually perform the transaction (i.e. call dbuf_dirty()). This bug will be fixed by removing the assertion that the tx_space_* accounting is perfectly accurate (i.e. we never dirty more than was predicted by dmu_tx_hold_*()). By removing the requirement that this accounting be perfectly accurate, we can also vastly simplify it, e.g. removing most of the logic in dmu_tx_count_*(). The new tx space accounting will be very approximate, and may be more or less than what is actually dirtied. It will still be used to determine if this transaction will put us over quota. Transactions that are marked by dmu_tx_mark_netfree() will be excepted from this check. We won’t make an attempt to determine how much space will be freed by the transaction — this was rarely accurate enough to determine if a transaction should be permitted when we are over quota, which is why dmu_tx_mark_netfree() was introduced in 2014. We also won’t attempt to give “credit” when overwriting existing blocks, if those blocks may be freed. This allows us to remove the do_free_accounting logic in dbuf_dirty(), and associated routines. This logic attempted to predict what will be on disk when this txg syncs, to know if the overwritten block will be freed (i.e. exists, and has no snapshots). OpenZFS-issue: https://www.illumos.org/issues/7793 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/3704e0a Upstream bugs: DLPX-32883a Closes #5804 Porting notes: - DNODE_SIZE replaced with DNODE_MIN_SIZE in dmu_tx_count_dnode(), Using the default dnode size would be slightly better. - DEBUG_DMU_TX wrappers and configure option removed. - Resolved _by_dnode() conflicts these changes have not yet been applied to OpenZFS.
* Fix loop device becomes read-onlyChunwei Chen2017-03-061-1/+21
| | | | | | | | | | | | | Commit 933ec99 removes read and write from f_op because the vfs layer will select iter_write or aio_write automatically. However, for Linux <= 4.0, loop_set_fd will actually check f_op->write and set read-only if not exists. This patch add them back and use the generic do_sync_{read,write} for aio_{read,write} and new_sync_{read,write} for {read,write}_iter. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Closes #5776 Closes #5855
* Add auto-online test for ZED/FMA as part of the ZTSSydney Vanda2017-02-281-0/+3
| | | | | | | | | | | | | | | | | | | | | | Automated auto-online test to go along with ZED FMA integration (PR 4673) auto_online_001.pos works with real devices (sd- and mpath) and with non-real block devices (loop) by adding a scsi_debug device to the pool Note: In order for test group to run, ZED must not currently be running. Kernel 3.16.37 or higher needed for scsi_debug to work properly If timeout occurs on test using a scsi_debug device (error noticed on Ubuntu system), a reboot might be needed in order for test to pass. (more investigation into this) Also suppressed output from is_real_device/is_loop_device/is_mpath_device - was making the log file very cluttered with useless error messages "ie /dev/mapper/sdc is not a block device" from previous patch Reviewed-by: Don Brady <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: David Quigley <[email protected]> Signed-off-by: Sydney Vanda <[email protected]> Closes #5774
* Allow c99 code to compileMatthew Ahrens2017-02-082-0/+2
| | | | | | | | | | | | Add the appropriate compiler flags to accept c99 code. This will help to minimize differences with upstream, and aid porting changes. One change was necessary in zvol.c because the DEFINE_IDA() macro does not work with the new compiler flags. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Chunwei Chen <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #5756
* Add -Wno-declaration-after-statement to KERNELCPPFLAGSBrian Behlendorf2017-01-281-0/+1
| | | | | | | | | | | | | | | | Disable the warnings regarding ISO C90 forbidding mixed declarations and code. While this functionality was introduced as part of C99 gcc does allow this in C90 mode as an extension. https://gcc.gnu.org/onlinedocs/gcc/Mixed-Declarations.html#Mixed-Declarations Allowing this usage helps minimize the changes required when porting patches from OpenZFS. The downside here is that this functionality is specific to gcc. Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #5686
* Retire .write/.read file operationsChunwei Chen2017-01-272-0/+22
| | | | | | | | | | | | | | | The .write/.read file operations callbacks can be retired since support for .read_iter/.write_iter and .aio_read/.aio_write has been added. The vfs_write()/vfs_read() entry functions will select the correct interface for the kernel. This is desirable because all VFS write/read operations now rely on common code. This change also add the generic write checks to make sure that ulimits are enforced correctly on write. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Chunwei Chen <[email protected]> Closes #5587 Closes #5673
* Fix spellingka72017-01-033-3/+3
| | | | | | | | | Reviewed-by: Brian Behlendorf <[email protected] Reviewed-by: Giuseppe Di Natale <[email protected]>> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Haakan T Johansson <[email protected]> Closes #5547 Closes #5543
* 4.10 compat - BIO flag changes and othersTim Chase2016-12-303-4/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | [bio] The req_op enum was changed to req_opf. Update the "Linux 4.8 API" autotools checks to use an int to determine whether the various REQ_OP values are defined. This should work properly on kernels >= 4.8. [bio] bio_set_op_attrs() is now an inline function and can't be detected with #ifdef. Add a configure check to determine whether bio_set_op_attrs() is defined. Move the local definition of it from vdev_disk.c to blkdev_compat.h for consistency with other related compability shims. [bio] The read/write flags and their modifiers, including WRITE_FLUSH, WRITE_FUA and WRITE_FLUSH_FUA have been removed from fs.h. Add the new bio_set_flush() compatibility wrapper to replace VDEV_WRITE_FLUSH_FUA and set the flags appropriately for each supported kernel version. [vfs] The generic_readlink() function has been made static. If .readlink in inode_operations is NULL, generic_readlink() is used. [zol typo] Completely unrelated to 4.10 compat, fix a typo in the check for REQ_OP_SECURE_ERASE so that the proper macro is defined: s/HAVE_REQ_OP_SECURE_DISCARD/HAVE_REQ_OP_SECURE_ERASE/ Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Chunwei Chen <[email protected]> Signed-off-by: Tim Chase <[email protected]> Closes #5499
* Use inode_set_flags when availableChunwei Chen2016-12-162-0/+19
| | | | Signed-off-by: Chunwei Chen <[email protected]>
* Add -c to zpool iostat & status to run commandTony Hutter2016-11-292-16/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds a command (-c) option to zpool status and zpool iostat. The -c option allows you to run an arbitrary command on each vdev and display the first line of output in zpool status/iostat. The environment vars VDEV_PATH and VDEV_UPATH are set to the vdev's path and "underlying path" before running the command. For device mapper, multipath, or partitioned vdevs, VDEV_UPATH is the actual underlying /dev/sd* disk. This can be useful if the command you're running requires a /dev/sd* device. The patch also uses /sys/block/<dev>/slaves/ to lookup the underlying device instead of using libdevmapper. This not only removes the libdevmapper requirement at build time, but also allows you to resolve device mapper devices without being root. This means that UDEV_UPATH get set correctly when running zpool status/iostat as an unprivileged user. Example: $ zpool status -c 'echo I am $VDEV_PATH, $VDEV_UPATH' NAME STATE READ WRITE CKSUM mypool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 mpatha ONLINE 0 0 0 I am /dev/mapper/mpatha, /dev/sdc sdb ONLINE 0 0 0 I am /dev/sdb1, /dev/sdb Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes #5368
* Allow zfs unshare <protocol> -aLOLi2016-11-291-0/+1
| | | | | | | | | | | | | | | | | | Allow `zfs unshare <protocol> -a` command to share or unshare all datasets of a given protocol, nfs or smb. Additionally, enable most of ZFS Test Suite zfs_share/zfs_unshare test cases. To work around some Illumos-specific functionalities ($SHARE/$UNSHARE) some function wrappers were added around them. Finally, fix and issue in smb_is_share_active() that would leave SMB shares exported when invoking 'zfs unshare -a' Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Turbo Fredriksson <[email protected]> Signed-off-by: loli10K <[email protected]> Closes #3238 Closes #5367
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-18 09:33:45 +0000