aboutsummaryrefslogtreecommitdiffstats
path: root/tests
diff options
context:
space:
mode:
Diffstat (limited to 'tests')
-rw-r--r--tests/test-runner/Makefile.am2
-rw-r--r--tests/test-runner/man/Makefile.am1
-rw-r--r--tests/test-runner/man/test-runner.1296
3 files changed, 1 insertions, 298 deletions
diff --git a/tests/test-runner/Makefile.am b/tests/test-runner/Makefile.am
index db3d96614..4d4b639db 100644
--- a/tests/test-runner/Makefile.am
+++ b/tests/test-runner/Makefile.am
@@ -1 +1 @@
-SUBDIRS = bin include man
+SUBDIRS = bin include
diff --git a/tests/test-runner/man/Makefile.am b/tests/test-runner/man/Makefile.am
deleted file mode 100644
index 503a5694f..000000000
--- a/tests/test-runner/man/Makefile.am
+++ /dev/null
@@ -1 +0,0 @@
-dist_man_MANS = test-runner.1
diff --git a/tests/test-runner/man/test-runner.1 b/tests/test-runner/man/test-runner.1
deleted file mode 100644
index b823aaa3e..000000000
--- a/tests/test-runner/man/test-runner.1
+++ /dev/null
@@ -1,296 +0,0 @@
-.\"
-.\" This file and its contents are supplied under the terms of the
-.\" Common Development and Distribution License ("CDDL"), version 1.0.
-.\" You may only use this file in accordance with the terms of version
-.\" 1.0 of the CDDL.
-.\"
-.\" A full copy of the text of the CDDL should have accompanied this
-.\" source. A copy of the CDDL is also available via the Internet at
-.\" http://www.illumos.org/license/CDDL.
-.\"
-.\" Copyright (c) 2012 by Delphix. All rights reserved.
-.\"
-.Dd May 26, 2021
-.Dt RUN 1
-.Os
-.
-.Sh NAME
-.Nm run
-.Nd find, execute, and log the results of tests
-.Sh SYNOPSIS
-.Nm
-.Op Fl dgq
-.Op Fl o Ar outputdir
-.Op Fl pP Ar script
-.Op Fl t seconds
-.Op Fl uxX Ar username
-.Ar pathname Ns No …
-.Pp
-.Nm
-.Fl w Ar runfile
-.Op Fl gq
-.Op Fl o Ar outputdir
-.Op Fl pP Ar script
-.Op Fl t seconds
-.Op Fl uxX Ar username
-.Ar pathname Ns No …
-.Pp
-.Nm
-.Fl c Ar runfile
-.Op Fl dq
-.Pp
-.Nm
-.Op Fl h
-.
-.Sh DESCRIPTION
-.Nm
-command has three basic modes of operation.
-With neither
-.Fl c
-nor
-.Fl w ,
-.Nm
-processes the arguments provided on
-the command line, adding them to the list for this run.
-If a specified
-.Ar pathname
-is an executable file, it is added as a test.
-If a specified
-.Ar pathname
-is a directory, the behavior depends upon the presence of
-.Fl g .
-If
-.Fl g
-is specified, the directory is treated as a test group.
-See the section on
-.Sy Test Groups
-below.
-Without
-.Fl g ,
-.Nm
-simply descends into the directory looking for executable files.
-The tests are then executed, and the results are logged.
-.Pp
-With
-.Fl w ,
-.Nm
-finds tests in the manner described above.
-Rather than executing the tests and logging the results, the test configuration
-is stored in a
-.Ar runfile ,
-which can be used in future invocations, or edited
-to modify which tests are executed and which options are applied.
-Options included on the command line with
-.Fl w
-become defaults in the
-.Ar runfile .
-.Pp
-With
-.Fl c ,
-.Nm
-parses a
-.Ar runfile ,
-which can specify a series of tests and test groups to be executed.
-The tests are then executed, and the results are logged.
-.
-.Ss Test Groups
-A test group is comprised of a set of executable files, all of which exist in
-one directory.
-The options specified on the command line or in a
-.Ar runfile
-apply to individual tests in the group.
-The exception is options pertaining to pre and post scripts, which act on all tests as a group.
-Rather than running before and after each test,
-these scripts are run only once each at the start and end of the test group.
-.Ss Test Execution
-The specified tests run serially, and are typically assigned results according
-to exit values.
-Tests that exit zero and non-zero are marked
-.Sy PASS
-and
-.Sy FAIL ,
-respectively.
-When a pre script fails for a test group, only the post script is executed,
-and the remaining tests are marked
-.Sy SKIPPED .
-Any test that exceeds
-its
-.Ar timeout
-is terminated, and marked
-.Sy KILLED .
-.Pp
-By default, tests are executed with the credentials of the
-.Nm
-script.
-Executing tests with other credentials is done via
-.Xr sudo 1m ,
-which must
-be configured to allow execution without prompting for a password.
-Environment variables from the calling shell are available to individual tests.
-During test execution, the working directory is changed to
-.Ar outputdir .
-.
-.Ss Output Logging
-By default,
-.Nm
-will print one line on standard output at the conclusion
-of each test indicating the test name, result and elapsed time.
-Additionally, for each invocation of
-.Nm ,
-a directory is created using the ISO 8601 date format.
-Within this directory is a file named
-.Sy log
-containing all the
-test output with timestamps, and a directory for each test.
-Within the test directories, there is one file each for standard output,
-standard error and merged output.
-The default location for the
-.Ar outputdir
-is
-.Pa /var/tmp/test_results .
-.Ss "Runfiles"
-The
-.Ar runfile
-is an INI-style configuration file that describes a test run.
-The file has one section named
-.Sy DEFAULT ,
-which contains configuration option
-names and their values in
-.Sy name No = Ar value
-format.
-The values in this section apply to all the subsequent sections,
-unless they are also specified there, in which case the default is overridden.
-The remaining section names are the absolute pathnames of files and directories,
-describing tests and test groups respectively.
-The legal option names are:
-.Bl -tag -width "tests = ['filename', …]"
-.It Sy outputdir No = Ar pathname
-The name of the directory that holds test logs.
-.It Sy pre No = Ar script
-Run
-.Ar script
-prior to the test or test group.
-.It Sy pre_user No = Ar username
-Execute the pre script as
-.Ar username .
-.It Sy post No = Ar script
-Run
-.Ar script
-after the test or test group.
-.It Sy post_user No = Ar username
-Execute the post script as
-.Ar username .
-.It Sy quiet No = Sy True Ns | Ns Sy False
-If
-.Sy True ,
-only the results summary is printed to standard out.
-.It Sy tests No = [ Ns Ar 'filename' , No … ]
-Specify a list of
-.Ar filenames
-for this test group.
-Only the basename of the absolute path is required.
-This option is only valid for test groups, and each
-.Ar filename
-must be single quoted.
-.It Sy timeout No = Ar n
-A timeout value of
-.Ar n
-seconds.
-.It Sy user No = Ar username
-Execute the test or test group as
-.Ar username .
-.El
-.
-.Sh OPTIONS
-.Bl -tag -width "-o outputdir"
-.It Fl c Ar runfile
-Specify a
-.Ar runfile
-to be consumed by the run command.
-.It Fl d
-Dry run mode.
-Execute no tests, but print a description of each test that would have been run.
-.It Fl m
-Enable kmemleak reporting (Linux only)
-.It Fl g
-Create test groups from any directories found while searching for tests.
-.It Fl o Ar outputdir
-Specify the directory in which to write test results.
-.It Fl p Ar script
-Run
-.Ar script
-prior to any test or test group.
-.It Fl P Ar script
-Run
-.Ar script
-after any test or test group.
-.It Fl q
-Print only the results summary to the standard output.
-.It Fl s Ar script
-Run
-.Ar script
-as a failsafe after any test is killed.
-.It Fl S Ar username
-Execute the failsafe script as
-.Ar username .
-.It Fl t Ar n
-Specify a timeout value of
-.Ar n
-seconds per test.
-.It Fl u Ar username
-Execute tests or test groups as
-.Ar username .
-.It Fl w Ar runfile
-Specify the name of the
-.Ar runfile
-to create.
-.It Fl x Ar username
-Execute the pre script as
-.Ar username .
-.It Fl X Ar username
-Execute the post script as
-.Ar username .
-.El
-.
-.Sh EXAMPLES
-.Bl -tag -width "-h"
-.It Sy Example 1 : No Running ad-hoc tests.
-This example demonstrates the simplest invocation of
-.Nm .
-.Bd -literal
-.No % Nm run Ar my-tests
-Test: /home/jkennedy/my-tests/test-01 [00:02] [PASS]
-Test: /home/jkennedy/my-tests/test-02 [00:04] [PASS]
-Test: /home/jkennedy/my-tests/test-03 [00:01] [PASS]
-
-Results Summary
-PASS 3
-
-Running Time: 00:00:07
-Percent passed: 100.0%
-Log directory: /var/tmp/test_results/20120923T180654
-.Ed
-.It Sy Example 2 : No Creating a Ar runfile No for future use.
-This example demonstrates creating a
-.Ar runfile
-with non-default options.
-.Bd -literal
-.No % Nm run Fl p Ar setup Fl x Ar root Fl g Fl w Ar new-tests.run Ar new-tests
-.No % Nm cat Pa new-tests.run
-[DEFAULT]
-pre = setup
-post_user =
-quiet = False
-user =
-timeout = 60
-post =
-pre_user = root
-outputdir = /var/tmp/test_results
-
-[/home/jkennedy/new-tests]
-tests = ['test-01', 'test-02', 'test-03']
-.Ed
-.El
-.
-.Sh SEE ALSO
-.Xr sudo 1m