aboutsummaryrefslogtreecommitdiffstats
path: root/tests/test-runner/bin
Commit message (Collapse)AuthorAgeFilesLines
* special device removal space accounting fixesMatthew Ahrens2020-12-231-2/+0
| | | | | | | | | | | | | | | | The space in special devices is not included in spa_dspace (or dsl_pool_adjustedsize(), or the zfs `available` property). Therefore there is always at least as much free space in the normal class, as there is allocated in the special class(es). And therefore, there is always enough free space to remove a special device. However, the checks for free space when removing special devices did not take this into account. This commit corrects that. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Don Brady <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #11329
* ZTS: ztest may cause mmp tests failuresBrian Behlendorf2020-08-171-0/+2
| | | | | | | | | | | | The mmp_exported_import and mmp_inactive_import tests depend on ztest simulating an active pool. If ztest unexpectedly terminates due to an unrelated issue the test case will fail. Since ztest is not yet 100% reliable I've added these tests to the maybe exception list. They can be removed when the issues with ztest are resolved or if the test cases are updated to handle these unexpected failures. Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10726
* ZTS: zvol_misc_volmode is flaky on FreeBSDRyan Moeller2020-07-311-0/+1
| | | | | | | Mark this as a known issue. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10655
* Linux 4.10 compat: has_capability()Brian Behlendorf2020-07-191-0/+2
| | | | | | | | | | | | Stock kernels older than 4.10 do not export the has_capability() function which is required by commit e59a377. To avoid breaking the build on older kernels revert to the safe legacy behavior and return EACCES when privileges cannot be checked. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Matt Ahrens <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10565 Closes #10573
* Update zts-report.py with additional testsBrian Behlendorf2020-07-151-0/+4
| | | | | | | | | | | | | | | The following test cases have been observed to fail frequently enough to be a problem when reporting CI results. Until they can be updated to be entirely reliable add them to the zts-report.py script. alloc_class/alloc_class_011_neg cli_root/zpool_import/zpool_import_012_pos mmp/mmp_on_uberblocks rsend/send_partial_dataset Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10578
* Centralize variable substitutionArvind Sankar2020-07-144-15/+9
| | | | | | | | | | | | A bunch of places need to edit files to incorporate the configured paths i.e. bindir, sbindir etc. Move this logic into a common file. Create arc_summary by copying arc_summary[23] as appropriate at build time instead of install time. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Arvind Sankar <[email protected]> Closes #10559
* Remove dependency on sharetab file and refactor sharing logicGeorge Wilson2020-07-131-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | == Motivation and Context The current implementation of 'sharenfs' and 'sharesmb' relies on the use of the sharetab file. The use of this file is os-specific and not required by linux or freebsd. Currently the code must maintain updates to this file which adds complexity and presents a significant performance impact when sharing many datasets. In addition, concurrently running 'zfs sharenfs' command results in missing entries in the sharetab file leading to unexpected failures. == Description This change removes the sharetab logic from the linux and freebsd implementation of 'sharenfs' and 'sharesmb'. It still preserves an os-specific library which contains the logic required for sharing NFS or SMB. The following entry points exist in the vastly simplified libshare library: - sa_enable_share -- shares a dataset but may not commit the change - sa_disable_share -- unshares a dataset but may not commit the change - sa_is_shared -- determine if a dataset is shared - sa_commit_share -- notify NFS/SMB subsystem to commit the shares - sa_validate_shareopts -- determine if sharing options are valid The sa_commit_share entry point is provided as a performance enhancement and is not required. The sa_enable_share/sa_disable_share may commit the share as part of the implementation. Libshare provides a framework for both NFS and SMB but some operating systems may not fully support these protocols or all features of the protocol. NFS Operation: For linux, libshare updates /etc/exports.d/zfs.exports to add and remove shares and then commits the changes by invoking 'exportfs -r'. This file, is automatically read by the kernel NFS implementation which makes for better integration with the NFS systemd service. For FreeBSD, libshare updates /etc/zfs/exports to add and remove shares and then commits the changes by sending a SIGHUP to mountd. SMB Operation: For linux, libshare adds and removes files in /var/lib/samba/usershares by calling the 'net' command directly. There is no need to commit the changes. FreeBSD does not support SMB. == Performance Results To test sharing performance we created a pool with an increasing number of datasets and invoked various zfs actions that would enable and disable sharing. The performance testing was limited to NFS sharing. The following tests were performed on an 8 vCPU system with 128GB and a pool comprised of 4 50GB SSDs: Scale testing: - Share all filesystems in parallel -- zfs sharenfs=on <dataset> & - Unshare all filesystems in parallel -- zfs sharenfs=off <dataset> & Functional testing: - share each filesystem serially -- zfs share -a - unshare each filesystem serially -- zfs unshare -a - reset sharenfs property and unshare -- zfs inherit -r sharenfs <pool> For 'zfs sharenfs=on' scale testing we saw an average reduction in time of 89.43% and for 'zfs sharenfs=off' we saw an average reduction in time of 83.36%. Functional testing also shows a huge improvement: - zfs share -- 97.97% reduction in time - zfs unshare -- 96.47% reduction in time - zfs inhert -r sharenfs -- 99.01% reduction in time Reviewed-by: Matt Ahrens <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Bryant G. Ly <[email protected]> Signed-off-by: George Wilson <[email protected]> External-Issue: DLPX-68690 Closes #1603 Closes #7692 Closes #7943 Closes #10300
* filesystem_limit/snapshot_limit is incorrectly enforced against rootMatthew Ahrens2020-07-111-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The filesystem_limit and snapshot_limit properties limit the number of filesystems or snapshots that can be created below this dataset. According to the manpage, "The limit is not enforced if the user is allowed to change the limit." Two types of users are allowed to change the limit: 1. Those that have been delegated the `filesystem_limit` or `snapshot_limit` permission, e.g. with `zfs allow USER filesystem_limit DATASET`. This works properly. 2. A user with elevated system privileges (e.g. root). This does not work - the root user will incorrectly get an error when trying to create a snapshot/filesystem, if it exceeds the `_limit` property. The problem is that `priv_policy_ns()` does not work if the `cred_t` is not that of the current process. This happens when `dsl_enforce_ds_ss_limits()` is called in syncing context (as part of a sync task's check func) to determine the permissions of the corresponding user process. This commit fixes the issue by passing the `task_struct` (typedef'ed as a `proc_t`) to syncing context, and then using `has_capability()` to determine if that process is privileged. Note that we still need to pass the `cred_t` to syncing context so that we can check if the user was delegated this permission with `zfs allow`. This problem only impacts Linux. Wrappers are added to FreeBSD but it continues to use `priv_check_cred()`, which works on arbitrary `cred_t`. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #8226 Closes #10545
* pam: implement a zfs_key pam modulefelixdoerre2020-06-241-0/+1
| | | | | | | | | | | | | | | | | Implements a pam module for automatically loading zfs encryption keys for home datasets. The pam module: - loads a zfs key and mounts the dataset when a session opens. - unmounts the dataset and unloads the key when the session closes. - when the user is logged on and changes the password, the module changes the encryption key. Reviewed-by: Richard Laager <[email protected]> Reviewed-by: @jengelh <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Felix Dörre <[email protected]> Closes #9886 Closes #9903
* Fix check for sed --in-placeArvind Sankar2020-06-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The test added in commit 4313a5b4c51e ("Detect if sed supports --in-place") doesn't work at least on my system (autoconfig-2.69). The issue is that SED has already been found and cached before this function is evaluated, with the result that the test is completely skipped. ... checking for a sed that does not truncate output... /usr/bin/sed ... checking for sed --in-place... (cached) /usr/bin/sed The first test is executed by libtool.m4. This looks to have been around in libtool for at least 15 years or so, not sure why this was not encountered at the time of the original commit. Fix this by caching the value of the ac_inplace flag rather than the path to SED. Also use $SED and add AC_REQUIRE to ensure that we use the sed that was located by the standard configure test. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Arvind Sankar <[email protected]> Closes #10493
* Update zts-report.py with additional testsBrian Behlendorf2020-06-221-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The following test cases may still occasionally fail and are being added to the "maybe" list for Linux until they can be updated to be entirely reliable. cli_root/zfs_rename/zfs_rename_002_pos.ksh cli_root/zpool_reopen/zpool_reopen_003_pos.ksh refreserv/refreserv_raidz These 6 tests consistently fail only on Fedora 31+, the failures are related to the kernel rescanning the partition table on loopback devices which is no longer reliable unless partprobe is used. In order to enable the Fedora bot by default they are also being added to the list until the tests can be updated. Any significant regression in functionality covered by these tests will still be detected by the FreeBSD builders. alloc_class/alloc_class_009_pos alloc_class/alloc_class_010_pos cli_root/zpool_expand/zpool_expand_001_pos cli_root/zpool_expand/zpool_expand_005_pos rsend/rsend_007_pos rsend/rsend_010_pos rsend/rsend_011_pos snapshot/rollback_003_pos Signed-off-by: Brian Behlendorf <[email protected]> Closes #10489
* flake8 E741 variable name warningBrian Behlendorf2020-05-141-3/+3
| | | | | | | | | | | | | | | Update the zts-report.py script to conform to the flake8 E741 rule. "Variables named I, O, and l can be very hard to read. This is because the letter I and the letter l are easily confused, and the letter O and the number 0 can be easily confused." - https://www.flake8rules.com/rules/E741.html Reviewed-by: George Melikov <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: John Kennedy <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #10323
* Add FreeBSD support to OpenZFSMatthew Macy2020-04-141-2/+3
| | | | | | | | | | | | | | | | | | Add the FreeBSD platform code to the OpenZFS repository. As of this commit the source can be compiled and tested on FreeBSD 11 and 12. Subsequent commits are now required to compile on FreeBSD and Linux. Additionally, they must pass the ZFS Test Suite on FreeBSD which is being run by the CI. As of this commit 1230 tests pass on FreeBSD and there are no unexpected failures. Reviewed-by: Sean Eric Fagan <[email protected]> Reviewed-by: Jorgen Lundman <[email protected]> Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Co-authored-by: Ryan Moeller <[email protected]> Signed-off-by: Matt Macy <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #898 Closes #8987
* ZTS: Fix and change testcase cache_010_negalex2020-04-131-1/+0
| | | | | | | | | | | | | | | | Commit 379ca9c removed the requirement on aux devices to be block devices only but the test case cache_010_neg was not updated, making it fail consistently. This change changes the test to check that cache devices _can_ be anything that presents a block interface. The testcase is renamed to cache_010_pos and the exceptions for known failure removed from the test runner. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reported-by: Richard Elling <[email protected]> Signed-off-by: Alex John <[email protected]> Closes #10172
* ZTS: Update flaky tests in zts-reportRyan Moeller2020-03-131-7/+20
| | | | | | | | | | | | | | Some tests which pass on FreeBSD but fail on Linux had been put in the "maybe" set. Move these back to "known" under an "if Linux" check so the expected outcome is clear. Add some tests that have been found to be flaky on FreeBSD stable/12 to the "maybe" set. Reviewed-by: John Kennedy <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10120
* ZTS: Test boundary conditions in alloc_class_012Ryan Moeller2020-03-121-0/+2
| | | | | | | | | | | | | | | Issue #9142 describes an error in the checks for device removal that can prevent removal of special allocation class vdevs in some situations. Enhance alloc_class/alloc_class_012_pos to check situations where this bug occurs. Update zts-report with knowledge of issue #9142. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10116 Issue #9142
* ZTS: Add a failsafe callback to run after each testRyan Moeller2020-03-101-48/+114
| | | | | | | | | | | | | | | | | | | Tests that get killed do not have an opportunity to clean up. There are many bad states this can leave the system in, but of particular gravity is when zinject has been used to induce bad behavior for one or more of the test disks. Create a failsafe mechanism in test-runner.py that runs a callback script after every test. The script is common to all tests so all tests benefit from the protection. Add an obligatory `zinject -c all` to clear all zinject state after every test case is run. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10096
* ZTS: Update zts-report exceptions for FreeBSDRyan Moeller2020-03-061-2/+3
| | | | | | | | | | The new zfs_sync_trim_* tests are skipped on FreeBSD. Both of the previously failing tests are now passing. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10105
* ZTS: Add zts-report exceptions for FreeBSDBrian Behlendorf2020-03-041-5/+12
| | | | | | | | | | | | | | | | | | There are three tests we expect to fail only on FreeBSD. * link_count never exits and eventually times out: - @amotin tells me this test is probably not applicable to us - Skip on FreeBSD * userobj feature does not activate immediately after pool upgrade - low impact; we are aware of this issue * removal does not appear to condense on export - low impact; we are aware of this issue Additionally removal_with_zdb passes on FreeBSD, so it is moved to "maybe". Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10093
* ZTS: Test the correct filesystem_limits behaviorBrian Behlendorf2020-03-041-0/+2
| | | | | | | | | | | | | | | | | | | | See issue #8226: Property filesystem_limit does not work as documented There have been previous attempts to fix the behavior on Linux, but so far the issue is still open. See PRs #8228, #8280. The existing tests pass for the incorrect behavior. This is a problem on FreeBSD; we are failing the tests because we implement the feature correctly. I have adapted the tests based on the work by @loli10k in #8280 and extended the changes to fix the snapshot_limit test as well. Linux now fails these tests, so entries linking to the issue have been added to the "maybe" group in zts-report.py. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10082
* ZTS: Change issue URL template to OpenZFS orgRyan Moeller2020-03-021-1/+1
| | | | | | Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10081
* ZTS: Fix zfs_receive_004_negMatthew Ahrens2020-02-271-1/+0
| | | | | | | | | | | | | | | | | | | | | `zfs recv` of an incremental stream that already exists is ignored, with a message like: receiving incremental stream of pool/fs@incsnap into pool/fs@incsnap snap testpool/testfs@incsnap already exists; ignoring And the command exits successfully (exit code 0). The zfs_receive_004_neg test is expecting that a this case will fail, with nonzero exit code. The fix is to remove this specific command from the test case. This lets us check that the remaining commands do in fact fail. Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #10055
* ZTS: Adapt casenorm tests for FreeBSDRyan Moeller2020-02-261-4/+4
| | | | | | | | | | | | | | | | Several casenorm tests pass on FreeBSD but are expected to fail on Linux. Move the passing tests from "fail" to "maybe" so that passing on FreeBSD is not unexpected. Invert platform logic so FreeBSD doesn't use illumos-only zlook. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Igor Kozhukhov <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #10050
* ZTS: Move more tests out of common.runRyan Moeller2020-01-271-3/+0
| | | | | | | | | | | | | | | These tests won't run on all platforms as currently implemented: * add_nested_replacing_spare (needs zed) * fault (needs zed) * mmp (needs multihost_history) * umount_unlink_drained (needs procfs) * zpool_expand (needs udev events and zed) * zpool_reopen (needs scsi_debug) * zvol_swap_003_pos (needs to modify vfstab) * zvol_swap_00[56]_pos (needs swaphigh/swaplen) Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #9871
* ZTS: Get xattr tests running on FreeBSDRyan Moeller2020-01-231-3/+0
| | | | | | | | | | This mostly involves reworking platform checks to make illumos the exception (thanks to their unusual way of exposing xattrs). Other platforms are able to take advantage of the recently added xattr wrappers in libtest. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #9872
* ZTS: Enable zpool_create_008_pos.kshBrian Behlendorf2020-01-171-1/+0
| | | | | | | | | | | | | | | | Remove the blkid version check from zpool_create_008_pos.ksh so the test case will not be skipped. All versions of blkid tested by the CI are either new enough to not suffer from this issue, or have been patched as is the case with CentOS 7 (libblkid-2.23.2-61). Additionally, add a block_device_wait after device partitioning to ensure the expected partitions will exist. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #9853
* ZTS: Remove obsolete zts-report.py exceptionsBrian Behlendorf2020-01-131-28/+0
| | | | | | | | | | | | The disk_reason and udev_reason exceptions can be removed since they apply to now unsupported kernel versions (<v3.10). The checks in the test cases were kept for the purposes of documentation and as useful sanity checks for the test environment. Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #9828
* ZTS: Move dumpdev tests to sunos.runRyan Moeller2020-01-031-10/+0
| | | | | | | | | | | | | | | | | Neither FreeBSD nor Linux support dumping to zvols. DilOS still uses these tests, so the files are kept and the tests have been relocated to sunos.run. An `is_illumos` function was added to libtest.shlib to eliminate some awkward platform checks. A few functions that are not expected to be used outside of illumos have been sanitized of extraneous FreeBSD adaptations. Reviewed-by: Igor Kozhukhov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #9794
* Fix test pattern in zts-report.pyRyan Moeller2019-12-201-1/+2
| | | | | | | | | | The pattern was not updated to match when the test output changed to include a platform identifier for platform specific tests. Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Kjeld Schouten <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #9750
* ZTS: Casenorm fix unicode interpretationGeorge Melikov2019-11-191-8/+0
| | | | | | | | | | | | | | | | Use `printf` to properly interpret unicode characters. Illumos uses a utility called `zlook` to allow additional flags to be provided to readdir and lookup for testing. This functionality could be ported to Linux, but even without it several of the tests can be enabled by instead using the standard `test` command. Additional, work is required to enable the remaining test cases. Reviewed-by: Igor Kozhukhov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: George Melikov <[email protected]> Issue #7633 Closes #8812
* Partially revert 5a6ac4cBrian Behlendorf2019-11-181-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | Reinstate the zpl_revalidate() functionality to resolve a regression where dentries for open files during a rollback are not invalidated. The unrelated functionality for automatically unmounting .zfs/snapshots was not reverted. Nor was the addition of shrink_dcache_sb() to the zfs_resume_fs() function. This issue was not immediately caught by the CI because the test case intended to catch it was included in the list of ZTS tests which may occasionally fail for unrelated reasons. Remove all of the rollback tests from this list to help identify the frequency of any spurious failures. The rollback_003_pos.ksh test case exposes a real issue with the long standing code which needs to be investigated. Regardless, it has been enable with a small workaround in the test case itself. Reviewed-by: Matt Ahrens <[email protected]> Reviewed-by: Pavel Snajdr <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #9587 Closes #9592
* Detect if sed supports --in-placeRyan Moeller2019-10-161-1/+1
| | | | | | | | | | | | | | Not all versions of sed have the --in-place flag. Detect support for the flag during ./configure and provide a fallback mechanism for those systems where sed's behavior differs. The autoconf variable ${ac_inplace} can be used to choose the correct flags for editing a file in place with sed. Replace violating usages in Makefile.am with ${ac_inplace}. Reviewed-by: Chris Dunlop <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #9463
* Move platform independent tests to a shared runfileRyan Moeller2019-10-091-48/+63
| | | | | | | | | | | | | | | | | | | Tests that aren't limited to running on Linux can be moved to a common runfile to be shared with other platforms. The test runner and wrapper script are enhanced to allow specifying multiple runfiles as a comma-separated list. The default runfiles are now "common.run,PLATFORM.run" where PLATFORM is determined at run time. Sections in runfiles that share a path with another runfile can append a colon separator and an identifier to the path in the section name, ie `[tests/functional/atime:Linux]`, to avoid overriding the tests specified by other runfiles. Reviewed-by: Jorgen Lundman <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #9391
* Canonicalize Python shebangsRyan Moeller2019-09-123-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | /usr/bin/env python3 is the suggested[1] shebang for Python in general (likewise for python2) and is conventional across platforms. This eases development on systems where python is not installed in /usr/bin (FreeBSD for example) and makes it possible to develop in virtual environments (venv) for isolating dependencies. Many packaging guidelines discourage the use of /usr/bin/env, but since this is the canonical way of writing shebangs in the Python community, many packaging scripts are already equipped to handle substituting the appropriate absolute path to python automatically. Some RPM package builders lacking brp-mangle-shebangs need a small fallback mechanism in the package spec to stamp the appropriate shebang on installed Python scripts. [1]: https://docs.python.org/3/using/unix.html?#miscellaneous Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #9314
* Fix typosAndrea Gelmini2019-09-022-3/+3
| | | | | | | Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Richard Laager <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Andrea Gelmini <[email protected]> Closes #9251
* Remove code for zfs remapMatthew Ahrens2019-06-241-1/+0
| | | | | | | | | | | | | | | | The "zfs remap" command was disabled by 6e91a72fe3ff8bb282490773bd687632f3e8c79d, because it has little utility and introduced some tricky bugs. This commit removes the code for it, the associated ZFS_IOC_REMAP ioctl, and tests. Note that the ioctl and property will remain, but have no functionality. This allows older software to fail gracefully if it attempts to use these, and avoids a backwards incompatibility that would be introduced if we renumbered the later ioctls/props. Reviewed-by: Tom Caputi <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Matthew Ahrens <[email protected]> Closes #8944
* test-runner.py: change shebang to python3Stoiko Ivanov2019-05-281-1/+1
| | | | | | | | | | | | | | | | | In commit 6e72a5b9b61066146deafda39ab8158c559f5f15 python scripts which work with python2 and python3 changed the shebang from /usr/bin/python to /usr/bin/python3. This gets adapted by the build-system on systems which don't provide python3. This commit changes test-runner.py to also use /usr/bin/python3, enabling the change during buildtime and fixing a minor lintian issue for those Debian packages, which depend on a specific python version (python3/python2). Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: loli10K <[email protected]> Reviewed by: John Kennedy <[email protected]> Signed-off-by: Stoiko Ivanov <[email protected]> Closes #8803
* Allow zfs-tests to recover from hibernationAlek P2019-04-111-19/+70
| | | | | | | | | | | | | | | | When a system sleeps during a zfs-test, the time spent hibernating is counted against the test's runtime even though the test can't and isn't running. This patch tries to detect timeouts due to hibernation and reruns tests that timed out due to system sleeping. In this version of the patch, the existing behavior of returning non-zero when a test was killed is preserved. With this patch applied we still return nonzero and we also automatically rerun the test we suspect of being killed due to system hibernation. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed by: John Kennedy <[email protected]> Signed-off-by: Alek Pinchuk <[email protected]> Closes #8575
* Add TRIM supportBrian Behlendorf2019-03-291-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | UNMAP/TRIM support is a frequently-requested feature to help prevent performance from degrading on SSDs and on various other SAN-like storage back-ends. By issuing UNMAP/TRIM commands for sectors which are no longer allocated the underlying device can often more efficiently manage itself. This TRIM implementation is modeled on the `zpool initialize` feature which writes a pattern to all unallocated space in the pool. The new `zpool trim` command uses the same vdev_xlate() code to calculate what sectors are unallocated, the same per- vdev TRIM thread model and locking, and the same basic CLI for a consistent user experience. The core difference is that instead of writing a pattern it will issue UNMAP/TRIM commands for those extents. The zio pipeline was updated to accommodate this by adding a new ZIO_TYPE_TRIM type and associated spa taskq. This new type makes is straight forward to add the platform specific TRIM/UNMAP calls to vdev_disk.c and vdev_file.c. These new ZIO_TYPE_TRIM zios are handled largely the same way as ZIO_TYPE_READs or ZIO_TYPE_WRITEs. This makes it possible to largely avoid changing the pipieline, one exception is that TRIM zio's may exceed the 16M block size limit since they contain no data. In addition to the manual `zpool trim` command, a background automatic TRIM was added and is controlled by the 'autotrim' property. It relies on the exact same infrastructure as the manual TRIM. However, instead of relying on the extents in a metaslab's ms_allocatable range tree, a ms_trim tree is kept per metaslab. When 'autotrim=on', ranges added back to the ms_allocatable tree are also added to the ms_free tree. The ms_free tree is then periodically consumed by an autotrim thread which systematically walks a top level vdev's metaslabs. Since the automatic TRIM will skip ranges it considers too small there is value in occasionally running a full `zpool trim`. This may occur when the freed blocks are small and not enough time was allowed to aggregate them. An automatic TRIM and a manual `zpool trim` may be run concurrently, in which case the automatic TRIM will yield to the manual TRIM. Reviewed-by: Jorgen Lundman <[email protected]> Reviewed-by: Tim Chase <[email protected]> Reviewed-by: Matt Ahrens <[email protected]> Reviewed-by: George Wilson <[email protected]> Reviewed-by: Serapheim Dimitropoulos <[email protected]> Contributions-by: Saso Kiselkov <[email protected]> Contributions-by: Tim Chase <[email protected]> Contributions-by: Chunwei Chen <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #8419 Closes #598
* flake8 passbunder20152019-02-042-21/+21
| | | | | | | | | | F632 use ==/!= to compare str, bytes, and int literals Reviewed-by: Håkan Johansson <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: bunder2015 <[email protected]> Closes #8368
* test-runner: python3 supportJohn Wren Kennedy2019-01-061-95/+100
| | | | | | | | | | | Updated to be compatible with Python 2.6, 2.7, 3.5 or newer. Reviewed-by: John Ramsden <[email protected]> Reviewed-by: Neal Gompa <[email protected]> Reviewed-by: loli10K <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: John Wren Kennedy <[email protected]> Closes #8096
* pyzfs: python3 support (build system)Brian Behlendorf2019-01-063-1/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Almost all of the Python code in the respository has been updated to be compatibile with Python 2.6, Python 3.4, or newer. The only exceptions are arc_summery3.py which requires Python 3, and pyzfs which requires at least Python 2.7. This allows us to maintain a single version of the code and support most default versions of python. This change does the following: * Sets the default shebang for all Python scripts to python3. If only Python 2 is available, then at install time scripts which are compatible with Python 2 will have their shebangs replaced with /usr/bin/python. This is done for compatibility until Python 2 goes end of life. Since only the installed versions are changed this means Python 3 must be installed on the system for test-runner when testing in-tree. * Added --with-python=<2|3|3.4,etc> configure option which sets the PYTHON environment variable to target a specific python version. By default the newest installed version of Python will be used or the preferred distribution version when creating pacakges. * Fixed --enable-pyzfs configure checks so they are run when --enable-pyzfs=check and --enable-pyzfs=yes. * Enabled pyzfs for Python 3.4 and newer, which is now supported. * Renamed pyzfs package to python<VERSION>-pyzfs and updated to install in the appropriate site location. For example, when building with --with-python=3.4 a python34-pyzfs will be created which installs in /usr/lib/python3.4/site-packages/. * Renamed the following python scripts according to the Fedora guidance for packaging utilities in /bin - dbufstat.py -> dbufstat - arcstat.py -> arcstat - arc_summary.py -> arc_summary - arc_summary3.py -> arc_summary3 * Updated python-cffi package name. On CentOS 6, CentOS 7, and Amazon Linux it's called python-cffi, not python2-cffi. For Python3 it's called python3-cffi or python3x-cffi. * Install one version of arc_summary. Depending on the version of Python available install either arc_summary2 or arc_summary3 as arc_summary. The user output is only slightly different. Reviewed-by: John Ramsden <[email protected]> Reviewed-by: Neal Gompa <[email protected]> Reviewed-by: loli10K <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #8096
* ZTS: Fix and reenable zfs_rename testsTom Caputi2018-11-071-2/+0
| | | | | | | | | | | | | | | zfs_rename_006_pos has been flaky in the past because it was missing a call to block_device_wait to ensure the zvols it creates are present before running dd. Whenever this this happened, zfs_rename_009_neg would also fail because the first test would leak a zvol clone that it did not know how to clean up. This patch fixes the root cause and reenables the test. It also fixes some minor grammar errors. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tom Caputi <[email protected]> Closes #5647 Closes #5648 Closes #8088
* ZTS: Fix test zfs_mount_006_posPaul Zuchowski2018-11-071-1/+0
| | | | | | | | | | | For Linux, place a file in the mount point folder so it will be considered "busy". Fix the while loop so it doesn't rm in directories above the testdir. Add Linux-specific code to test overlay on|off. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Paul Zuchowski <[email protected]> Closes #4990 Closes #8081
* ZTS: Fix posix ACL tests that should passPaul Zuchowski2018-10-311-2/+0
| | | | | | | | | Make sure tests have proper include files. Make sure underlying "chmod" style permissions don't interfere with ACLs. Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Paul Zuchowski <[email protected]> Closes #8069
* Fix flake8 "invalid escape sequence 'x'" warningBrian Behlendorf2018-10-241-2/+3
| | | | | | | | | | | | | | | | From, https://lintlyci.github.io/Flake8Rules/rules/W605.html As of Python 3.6, a backslash-character pair that is not a valid escape sequence now generates a DeprecationWarning. Although this will eventually become a SyntaxError, that will not be for several Python releases. Note 'float_pobj' was simply removed from arcstat.py since it was entirely unused. Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Richard Elling <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #8056
* Fix "zfs destroy" when "sharenfs=on" is usedPrakash Surya2018-10-031-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When using "zfs destroy" on a dataset that is using "sharenfs=on" and has been automatically exported (by libzfs), the dataset will not be automatically unexported as it should be. This workflow appears to have been broken by this commit: 3fd3e56cfd543d7d7a1bf502bfc0db6e24139668 In that change, the "zfs_unmount" function was modified to use the "mnt.mnt_special" field when determining the mount point that is being unmounted, rather than "mnt.mnt_mountp". As a result, when "mntpt" is passed into "zfs_unshare_proto", it's value is now the dataset name rather than the mountpoint. Thus, when this value is used with the "is_shared" function (via "zfs_unshare_proto") it will not find a match (since that function assumes it'll be passed the mountpoint) and incorrectly reports that the dataset is not shared. This can be easily reproduced with the following commands: $ sudo zpool create tank xvdb $ sudo zfs create -o sharenfs=on tank/fish $ sudo zfs destroy tank/fish $ sudo zfs list -r tank NAME USED AVAIL REFER MOUNTPOINT tank 97.5K 7.27G 24K /tank $ sudo exportfs /tank/fish <world> $ sudo cat /etc/dfs/sharetab /tank/fish - nfs rw,crossmnt At this point, the "tank/fish" filesystem doesn't exist, but it's still listed as exported when looking at "exportfs" and "/etc/dfs/sharetab". Also note, this change brings us back in-sync with the illumos code, as it pertains to this one line; on illumos, "mnt.mnt_mountp" is used. Reviewed by: loli10K <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Co-authored-by: George Wilson <[email protected]> Signed-off-by: Prakash Surya <[email protected]> Issue #6143 Closes #7941
* ZTS: Fix snapshot_009_pos, snapshot_010_posBrian Behlendorf2018-10-011-0/+2
| | | | | | | | | | | | | | | | Mitigate the likelihood of the newly created volumes being busy when the 'zfs destroy -r' is issued by waiting for udev to settle. Since this is not a iron clad fix I've added the test case to the known list of possible failures and referenced issue #7961. Finally, in the case this test does fail fix the cleanup logic so subsequent tests won't incorrectly fail. Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: John Kennedy <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #7961 Closes #7962
* Fix flake 8 style warningsGregor Kopka2018-09-262-35/+42
| | | | | | | | | | | | | | Ran zts-report.py and test-runner.py from ./tests/test-runner/bin/ through the 2to3 (https://docs.python.org/2/library/2to3.html). Checked the result, fixed: - 'maxint' -> 'maxsize' that 2to3 missed. - 'cmp=' parameter for a 'sorted()' with a 'key=' version. - try/except wrapping of configparser import as there are still python 2.7 systems that lack a compatibility shim Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Gregor Kopka <[email protected]> Closes #7925 Closes #7952
* Revert "Fix flake 8 style warnings"Brian Behlendorf2018-09-242-37/+35
| | | | | | | | This reverts commit b8fd4310c54444eecb66140d99a6156f4353b29b which accidentally introduced a regression for some versions of python. Signed-off-by: Brian Behlendorf <[email protected]> Issue #7929
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-16 21:07:43 +0000