summaryrefslogtreecommitdiffstats
path: root/tests/test-runner
Commit message (Collapse)AuthorAgeFilesLines
* Allow test-runner to filter test groups by tagGiuseppe Di Natale2017-12-061-16/+40
| | | | | | | | | | | | | | Enable test-runner to accept a list of tags to identify which test groups the user wishes to run. Also allow test-runner to perform multiple iterations of a test run. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Wren Kennedy <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Giuseppe Di Natale <[email protected]> Closes #6788
* Correct flake8 errors after STYLE builder updateGiuseppe Di Natale2017-11-201-5/+5
| | | | | | | | | Fix new flake8 errors related to bare excepts and ambiguous variable names due to a STYLE builder update. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Giuseppe Di Natale <[email protected]> Closes #6776
* Exit test-runner with non-zero if tests are KILLEDGiuseppe Di Natale2017-07-071-0/+4
| | | | | | | | | | fe46eeb introduced non-zero exit codes to test-runner. A non-zero exit code should be returned when test-runner decided to kill a test and mark it as KILLED. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Giuseppe Di Natale <[email protected]> Closes #6325
* On failure tests-runner should do non-zero exitAlek P2017-06-301-3/+7
| | | | | | | | | | | | Right now test runner will always exit(0). It's helpful to have zfs-tests.sh provide different exit values depending on if everything passed or not. We can then use common shell cmds to run tests until failure. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Signed-off-by: Alek Pinchuk <[email protected]> Closes #6285
* Print fail messages before callbacks in test suiteGiuseppe Di Natale2017-06-301-4/+6
| | | | | | | | | | | Reorder operations in _endlog so failure messages get printed prior to performing callbacks and cleanup. This helps clarify why a test failed and places the message closer to the point of incident in the resulting logs. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Giuseppe Di Natale <[email protected]> Closes #6281
* OpenZFS 7503 - zfs-test should tail ::zfs_dbgmsg on test failureBrian Behlendorf2017-04-121-1/+21
| | | | | | | | | | | | | | | | | | | Authored by: Pavel Zakharov <[email protected]> Reviewed by: John Kennedy <[email protected]> Reviewed by: Matt Ahrens <[email protected]> Approved by: Gordon Ross <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed-by: George Melikov <[email protected]> Ported-by: Brian Behlendorf <[email protected]> Porting Notes: - Enable internal log for DEBUG builds and in zfs-tests.sh. - callbacks/zfs_dbgmsg.ksh - Dump interal log via kstat. - callbacks/zfs_dmesg.ksh - Dump dmesg log. - default.cfg - 'Test Suite Specific Commands' dropped. OpenZFS-issue: https://www.illumos.org/issues/7503 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/55a1300 Closes #6002
* OpenZFS 7290 - ZFS test suite needs to control what utilities it can runJohn Wren Kennedy2017-04-061-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | Authored by: John Wren Kennedy <[email protected]> Reviewed by: Dan Kimmel <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Reviewed by: Dan McDonald <[email protected]> Approved by: Gordon Ross <[email protected]> Ported-by: Brian Behlendorf <[email protected]> Ported-by: George Melikov <[email protected]> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
* OpenZFS 7248 - large block support breaks rsend_009_posJohn Wren Kennedy2017-02-151-35/+30
| | | | | | | | | | | | | | | | | 7249 rsend_015_pos produces false failures due to race 7250 testrunner can miss options specific to individual tests in runfiles Authored by: John Wren Kennedy <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Reviewed by: Paul Dagnelie <[email protected]> Reviewed by: Igor Kozhukhov <[email protected]> Approved by: Robert Mustacchi <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Ported-by: George Melikov <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/7248 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/f9a78bf Closes #5799
* OpenZFS 6642 - testrunner output can be displayed in the wrong orderJohn Wren Kennedy2017-02-111-4/+6
| | | | | | | | | | | | | | | | | | | | 6643 zfstest should enforce the required privileges before running. Reviewed by: George Wilson <[email protected]> Reviewed by: Jonathan Mackenzie <[email protected]> Reviewed by: Yuri Pankov <[email protected]> Approved by: Robert Mustacchi <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Giuseppe Di Natale <[email protected]> Reviewed by: John Kennedy <[email protected]> Ported-by: Brian Behlendorf <[email protected] OpenZFS-issue: https://www.illumos.org/issues/6642 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/412db4e Closes #5777 Porting notes: - The 6643 changes were dropped a different version of this script is used to configure the environment under Linux.
* Change location of current symlink created by test-runnerGiuseppe Di Natale2016-10-241-1/+2
| | | | | | | | | | | | | | | | test-runner should be creating the current symlink in the directory above the output directory. In a previous commit, the current symlink was placed in the current working directory, which could be inaccessible. It is more likely that the output directory is always accessible. This is needed because without this there's no deterministic way to get the path to ZFS Test Suite results until after the test suite has started. This makes it difficult for buildbot to follow the log file. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Giuseppe Di Natale <[email protected]> Closes #5314
* Create a symlink to current test-runner outputGiuseppe Di Natale2016-10-181-0/+8
| | | | | | | | | | Generate a symlink in the current working directory to test-runner.py output. This will make it easier for the ZFS buildbot to collect logs. Reviewed by: John Kennedy <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Giuseppe Di Natale <[email protected]> Closes #5293
* Correct style in test-runnerGiuseppe Di Natale2016-10-061-22/+21
| | | | | | | Correct test-runner.py so it passes flake8 python style checking. Signed-off-by: Giuseppe Di Natale <[email protected]>
* OpenZFS - Performance regression suite for zfstestJohn Wren Kennedy2016-09-081-4/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Author: John Wren Kennedy <[email protected]> Reviewed by: Prakash Surya <[email protected]> Reviewed by: Dan Kimmel <[email protected]> Reviewed by: Matt Ahrens <[email protected]> Reviewed by: Paul Dagnelie <[email protected]> Reviewed by: Don Brady <[email protected]> Reviewed by: Richard Elling <[email protected]> Reviewed by: Brian Behlendorf <[email protected]> Reviewed-by: David Quigley <[email protected]> Approved by: Richard Lowe <[email protected]> Ported-by: Don Brady <[email protected]> OpenZFS-issue: https://www.illumos.org/issues/6950 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/dcbf3bd6 Delphix-commit: https://github.com/delphix/delphix-os/commit/978ed49 Closes #4929 ZFS Test Suite Performance Regression Tests This was pulled into OpenZFS via the compressed arc featureand was separated out in zfsonlinux as a separate pull request from PR-4768. It originally came in as QA-4903 in Delphix-OS from John Kennedy. Expected Usage: $ DISKS="sdb sdc sdd" zfs-tests.sh -r perf-regression.run Porting Notes: 1. Added assertions in the setup script to make sure required tools (fio, mpstat, ...) are present. 2. For the config.json generation in perf.shlib used arcstats and other binaries instead of dtrace to query the values. 3. For the perf data collection: - use "zpool iostat -lpvyL" instead of the io.d dtrace script (currently not collecting zfs_read/write latency stats) - mpstat and iostat take different arguments - prefetch_io.sh is a placeholder that uses arcstats instead of dtrace 4. Build machines require fio, mdadm and sysstat pakage (YMMV). Future Work: - Need a way to measure zfs_read and zfs_write latencies per pool. - Need tools to takes two sets of output and display/graph the differences - Bring over additional regression tests from Delphix
* Add log_must_{retry,busy} helpersBrian Behlendorf2016-08-301-0/+75
| | | | | | | | | | | | | | | Add helpers which automatically retry the provided command when the error message matches the provided keyword. This provides an easy way to handle the asynchronous nature of some ZFS commands. For example, the `zfs destroy` command may need to be retried in the case where the block device is unexpected busy. This can be accomplished as follows: log_must_busy $ZFS destroy ... Signed-off-by: Brian Behlendorf <[email protected]> Issue #5002
* Add `zfs allow` and `zfs unallow` supportBrian Behlendorf2016-06-071-3/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ZFS allows for specific permissions to be delegated to normal users with the `zfs allow` and `zfs unallow` commands. In addition, non- privileged users should be able to run all of the following commands: * zpool [list | iostat | status | get] * zfs [list | get] Historically this functionality was not available on Linux. In order to add it the secpolicy_* functions needed to be implemented and mapped to the equivalent Linux capability. Only then could the permissions on the `/dev/zfs` be relaxed and the internal ZFS permission checks used. Even with this change some limitations remain. Under Linux only the root user is allowed to modify the namespace (unless it's a private namespace). This means the mount, mountpoint, canmount, unmount, and remount delegations cannot be supported with the existing code. It may be possible to add this functionality in the future. This functionality was validated with the cli_user and delegation test cases from the ZFS Test Suite. These tests exhaustively verify each of the supported permissions which can be delegated and ensures only an authorized user can perform it. Two minor bug fixes were required for test-running.py. First, the Timer() object cannot be safely created in a `try:` block when there is an unconditional `finally` block which references it. Second, when running as a normal user also check for scripts using the both the .ksh and .sh suffixes. Finally, existing users who are simulating delegations by setting group permissions on the /dev/zfs device should revert that customization when updating to a version with this change. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes #362 Closes #434 Closes #4100 Closes #4394 Closes #4410 Closes #4487
* Add the ZFS Test SuiteBrian Behlendorf2016-03-168-0/+1686
Add the ZFS Test Suite and test-runner framework from illumos. This is a continuation of the work done by Turbo Fredriksson to port the ZFS Test Suite to Linux. While this work was originally conceived as a stand alone project integrating it directly with the ZoL source tree has several advantages: * Allows the ZFS Test Suite to be packaged in zfs-test package. * Facilitates easy integration with the CI testing. * Users can locally run the ZFS Test Suite to validate ZFS. This testing should ONLY be done on a dedicated test system because the ZFS Test Suite in its current form is destructive. * Allows the ZFS Test Suite to be run directly in the ZoL source tree enabled developers to iterate quickly during development. * Developers can easily add/modify tests in the framework as features are added or functionality is changed. The tests will then always be in sync with the implementation. Full documentation for how to run the ZFS Test Suite is available in the tests/README.md file. Warning: This test suite is designed to be run on a dedicated test system. It will make modifications to the system including, but not limited to, the following. * Adding new users * Adding new groups * Modifying the following /proc files: * /proc/sys/kernel/core_pattern * /proc/sys/kernel/core_uses_pid * Creating directories under / Notes: * Not all of the test cases are expected to pass and by default these test cases are disabled. The failures are primarily due to assumption made for illumos which are invalid under Linux. * When updating these test cases it should be done in as generic a way as possible so the patch can be submitted back upstream. Most existing library functions have been updated to be Linux aware, and the following functions and variables have been added. * Functions: * is_linux - Used to wrap a Linux specific section. * block_device_wait - Waits for block devices to be added to /dev/. * Variables: Linux Illumos * ZVOL_DEVDIR "/dev/zvol" "/dev/zvol/dsk" * ZVOL_RDEVDIR "/dev/zvol" "/dev/zvol/rdsk" * DEV_DSKDIR "/dev" "/dev/dsk" * DEV_RDSKDIR "/dev" "/dev/rdsk" * NEWFS_DEFAULT_FS "ext2" "ufs" * Many of the disabled test cases fail because 'zfs/zpool destroy' returns EBUSY. This is largely causes by the asynchronous nature of device handling on Linux and is expected, the impacted test cases will need to be updated to handle this. * There are several test cases which have been disabled because they can trigger a deadlock. A primary example of this is to recursively create zpools within zpools. These tests have been disabled until the root issue can be addressed. * Illumos specific utilities such as (mkfile) should be added to the tests/zfs-tests/cmd/ directory. Custom programs required by the test scripts can also be added here. * SELinux should be either is permissive mode or disabled when running the tests. The test cases should be updated to conform to a standard policy. * Redundant test functionality has been removed (zfault.sh). * Existing test scripts (zconfig.sh) should be migrated to use the framework for consistency and ease of testing. * The DISKS environment variable currently only supports loopback devices because of how the ZFS Test Suite expects partitions to be named (p1, p2, etc). Support must be added to generate the correct partition name based on the device location and name. * The ZFS Test Suite is part of the illumos code base at: https://github.com/illumos/illumos-gate/tree/master/usr/src/test Original-patch-by: Turbo Fredriksson <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Olaf Faaland <[email protected]> Closes #6 Closes #1534
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-15 17:48:27 +0000