diff options
Diffstat (limited to 'tests')
-rw-r--r-- | tests/test-runner/Makefile.am | 2 | ||||
-rw-r--r-- | tests/test-runner/man/Makefile.am | 1 | ||||
-rw-r--r-- | tests/test-runner/man/test-runner.1 | 296 |
3 files changed, 1 insertions, 298 deletions
diff --git a/tests/test-runner/Makefile.am b/tests/test-runner/Makefile.am index db3d96614..4d4b639db 100644 --- a/tests/test-runner/Makefile.am +++ b/tests/test-runner/Makefile.am @@ -1 +1 @@ -SUBDIRS = bin include man +SUBDIRS = bin include diff --git a/tests/test-runner/man/Makefile.am b/tests/test-runner/man/Makefile.am deleted file mode 100644 index 503a5694f..000000000 --- a/tests/test-runner/man/Makefile.am +++ /dev/null @@ -1 +0,0 @@ -dist_man_MANS = test-runner.1 diff --git a/tests/test-runner/man/test-runner.1 b/tests/test-runner/man/test-runner.1 deleted file mode 100644 index b823aaa3e..000000000 --- a/tests/test-runner/man/test-runner.1 +++ /dev/null @@ -1,296 +0,0 @@ -.\" -.\" This file and its contents are supplied under the terms of the -.\" Common Development and Distribution License ("CDDL"), version 1.0. -.\" You may only use this file in accordance with the terms of version -.\" 1.0 of the CDDL. -.\" -.\" A full copy of the text of the CDDL should have accompanied this -.\" source. A copy of the CDDL is also available via the Internet at -.\" http://www.illumos.org/license/CDDL. -.\" -.\" Copyright (c) 2012 by Delphix. All rights reserved. -.\" -.Dd May 26, 2021 -.Dt RUN 1 -.Os -. -.Sh NAME -.Nm run -.Nd find, execute, and log the results of tests -.Sh SYNOPSIS -.Nm -.Op Fl dgq -.Op Fl o Ar outputdir -.Op Fl pP Ar script -.Op Fl t seconds -.Op Fl uxX Ar username -.Ar pathname Ns No … -.Pp -.Nm -.Fl w Ar runfile -.Op Fl gq -.Op Fl o Ar outputdir -.Op Fl pP Ar script -.Op Fl t seconds -.Op Fl uxX Ar username -.Ar pathname Ns No … -.Pp -.Nm -.Fl c Ar runfile -.Op Fl dq -.Pp -.Nm -.Op Fl h -. -.Sh DESCRIPTION -.Nm -command has three basic modes of operation. -With neither -.Fl c -nor -.Fl w , -.Nm -processes the arguments provided on -the command line, adding them to the list for this run. -If a specified -.Ar pathname -is an executable file, it is added as a test. -If a specified -.Ar pathname -is a directory, the behavior depends upon the presence of -.Fl g . -If -.Fl g -is specified, the directory is treated as a test group. -See the section on -.Sy Test Groups -below. -Without -.Fl g , -.Nm -simply descends into the directory looking for executable files. -The tests are then executed, and the results are logged. -.Pp -With -.Fl w , -.Nm -finds tests in the manner described above. -Rather than executing the tests and logging the results, the test configuration -is stored in a -.Ar runfile , -which can be used in future invocations, or edited -to modify which tests are executed and which options are applied. -Options included on the command line with -.Fl w -become defaults in the -.Ar runfile . -.Pp -With -.Fl c , -.Nm -parses a -.Ar runfile , -which can specify a series of tests and test groups to be executed. -The tests are then executed, and the results are logged. -. -.Ss Test Groups -A test group is comprised of a set of executable files, all of which exist in -one directory. -The options specified on the command line or in a -.Ar runfile -apply to individual tests in the group. -The exception is options pertaining to pre and post scripts, which act on all tests as a group. -Rather than running before and after each test, -these scripts are run only once each at the start and end of the test group. -.Ss Test Execution -The specified tests run serially, and are typically assigned results according -to exit values. -Tests that exit zero and non-zero are marked -.Sy PASS -and -.Sy FAIL , -respectively. -When a pre script fails for a test group, only the post script is executed, -and the remaining tests are marked -.Sy SKIPPED . -Any test that exceeds -its -.Ar timeout -is terminated, and marked -.Sy KILLED . -.Pp -By default, tests are executed with the credentials of the -.Nm -script. -Executing tests with other credentials is done via -.Xr sudo 1m , -which must -be configured to allow execution without prompting for a password. -Environment variables from the calling shell are available to individual tests. -During test execution, the working directory is changed to -.Ar outputdir . -. -.Ss Output Logging -By default, -.Nm -will print one line on standard output at the conclusion -of each test indicating the test name, result and elapsed time. -Additionally, for each invocation of -.Nm , -a directory is created using the ISO 8601 date format. -Within this directory is a file named -.Sy log -containing all the -test output with timestamps, and a directory for each test. -Within the test directories, there is one file each for standard output, -standard error and merged output. -The default location for the -.Ar outputdir -is -.Pa /var/tmp/test_results . -.Ss "Runfiles" -The -.Ar runfile -is an INI-style configuration file that describes a test run. -The file has one section named -.Sy DEFAULT , -which contains configuration option -names and their values in -.Sy name No = Ar value -format. -The values in this section apply to all the subsequent sections, -unless they are also specified there, in which case the default is overridden. -The remaining section names are the absolute pathnames of files and directories, -describing tests and test groups respectively. -The legal option names are: -.Bl -tag -width "tests = ['filename', …]" -.It Sy outputdir No = Ar pathname -The name of the directory that holds test logs. -.It Sy pre No = Ar script -Run -.Ar script -prior to the test or test group. -.It Sy pre_user No = Ar username -Execute the pre script as -.Ar username . -.It Sy post No = Ar script -Run -.Ar script -after the test or test group. -.It Sy post_user No = Ar username -Execute the post script as -.Ar username . -.It Sy quiet No = Sy True Ns | Ns Sy False -If -.Sy True , -only the results summary is printed to standard out. -.It Sy tests No = [ Ns Ar 'filename' , No … ] -Specify a list of -.Ar filenames -for this test group. -Only the basename of the absolute path is required. -This option is only valid for test groups, and each -.Ar filename -must be single quoted. -.It Sy timeout No = Ar n -A timeout value of -.Ar n -seconds. -.It Sy user No = Ar username -Execute the test or test group as -.Ar username . -.El -. -.Sh OPTIONS -.Bl -tag -width "-o outputdir" -.It Fl c Ar runfile -Specify a -.Ar runfile -to be consumed by the run command. -.It Fl d -Dry run mode. -Execute no tests, but print a description of each test that would have been run. -.It Fl m -Enable kmemleak reporting (Linux only) -.It Fl g -Create test groups from any directories found while searching for tests. -.It Fl o Ar outputdir -Specify the directory in which to write test results. -.It Fl p Ar script -Run -.Ar script -prior to any test or test group. -.It Fl P Ar script -Run -.Ar script -after any test or test group. -.It Fl q -Print only the results summary to the standard output. -.It Fl s Ar script -Run -.Ar script -as a failsafe after any test is killed. -.It Fl S Ar username -Execute the failsafe script as -.Ar username . -.It Fl t Ar n -Specify a timeout value of -.Ar n -seconds per test. -.It Fl u Ar username -Execute tests or test groups as -.Ar username . -.It Fl w Ar runfile -Specify the name of the -.Ar runfile -to create. -.It Fl x Ar username -Execute the pre script as -.Ar username . -.It Fl X Ar username -Execute the post script as -.Ar username . -.El -. -.Sh EXAMPLES -.Bl -tag -width "-h" -.It Sy Example 1 : No Running ad-hoc tests. -This example demonstrates the simplest invocation of -.Nm . -.Bd -literal -.No % Nm run Ar my-tests -Test: /home/jkennedy/my-tests/test-01 [00:02] [PASS] -Test: /home/jkennedy/my-tests/test-02 [00:04] [PASS] -Test: /home/jkennedy/my-tests/test-03 [00:01] [PASS] - -Results Summary -PASS 3 - -Running Time: 00:00:07 -Percent passed: 100.0% -Log directory: /var/tmp/test_results/20120923T180654 -.Ed -.It Sy Example 2 : No Creating a Ar runfile No for future use. -This example demonstrates creating a -.Ar runfile -with non-default options. -.Bd -literal -.No % Nm run Fl p Ar setup Fl x Ar root Fl g Fl w Ar new-tests.run Ar new-tests -.No % Nm cat Pa new-tests.run -[DEFAULT] -pre = setup -post_user = -quiet = False -user = -timeout = 60 -post = -pre_user = root -outputdir = /var/tmp/test_results - -[/home/jkennedy/new-tests] -tests = ['test-01', 'test-02', 'test-03'] -.Ed -.El -. -.Sh SEE ALSO -.Xr sudo 1m |