OpenZFS - Performance regression suite for zfstest

Author: John Wren Kennedy <[email protected]>
Reviewed by: Prakash Surya <[email protected]>
Reviewed by: Dan Kimmel <[email protected]>
Reviewed by: Matt Ahrens <[email protected]>
Reviewed by: Paul Dagnelie <[email protected]>
Reviewed by: Don Brady <[email protected]>
Reviewed by: Richard Elling <[email protected]>
Reviewed by: Brian Behlendorf <[email protected]>
Reviewed-by: David Quigley <[email protected]>
Approved by: Richard Lowe <[email protected]>
Ported-by: Don Brady <[email protected]>

OpenZFS-issue: https://www.illumos.org/issues/6950
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/dcbf3bd6
Delphix-commit: https://github.com/delphix/delphix-os/commit/978ed49
Closes #4929

ZFS Test Suite Performance Regression Tests

This was pulled into OpenZFS via the compressed arc featureand was
separated out in zfsonlinux as a separate pull request from PR-4768.
It originally came in as QA-4903 in Delphix-OS from John Kennedy.

Expected Usage:

$ DISKS="sdb sdc sdd" zfs-tests.sh -r perf-regression.run

Porting Notes:
1. Added assertions in the setup script to make sure required tools
   (fio, mpstat, ...) are present.
2. For the config.json generation in perf.shlib used arcstats and
    other binaries instead of dtrace to query the values.
3. For the perf data collection:
   - use "zpool iostat -lpvyL" instead of the io.d dtrace script
    (currently not collecting zfs_read/write latency stats)
   - mpstat and iostat take different arguments
   - prefetch_io.sh is a placeholder that uses arcstats instead of
     dtrace
4. Build machines require fio, mdadm and sysstat pakage (YMMV).

Future Work:
   - Need a way to measure zfs_read and zfs_write latencies per pool.
   - Need tools to takes two sets of output and display/graph the
     differences
   - Bring over additional regression tests from Delphix

1 files changed, 11 insertions, 4 deletions

diff --git a/tests/test-runner/cmd/test-runner.py b/tests/test-runner/cmd/test-runner.py
index ad1afff80..4d31ddceb 100755
--- a/tests/test-runner/cmd/test-runner.py
+++ b/tests/test-runner/cmd/test-runner.py
@@ -26,6 +26,7 @@ from select import select
 from subprocess import PIPE
 from subprocess import Popen
 from sys import argv
+from sys import maxint
 from sys import exit
 from threading import Timer
 from time import time
@@ -121,13 +122,16 @@ class Cmd(object):
     def __init__(self, pathname, outputdir=None, timeout=None, user=None):
         self.pathname = pathname
         self.outputdir = outputdir or 'BASEDIR'
-        self.timeout = timeout or 60
+        self.timeout = timeout
         self.user = user or ''
         self.killed = False
         self.result = Result()
 
+        if self.timeout == None:
+	    self.timeout = 60
+
     def __str__(self):
-        return "Pathname: %s\nOutputdir: %s\nTimeout: %s\nUser: %s\n" % (
+        return "Pathname: %s\nOutputdir: %s\nTimeout: %d\nUser: %s\n" % (
                 self.pathname, self.outputdir, self.timeout, self.user)
 
     def kill_cmd(self, proc):
@@ -213,6 +217,9 @@ class Cmd(object):
 
         self.result.starttime = time()
         proc = Popen(privcmd, stdout=PIPE, stderr=PIPE)
+        # Allow a special timeout value of 0 to mean infinity
+        if int(self.timeout) == 0:
+            self.timeout = maxint
         t = Timer(int(self.timeout), self.kill_cmd, [proc])
 
         try:
@@ -301,7 +308,7 @@ class Test(Cmd):
             pre_user = ' (as %s)' % (self.pre_user)
         if len(self.post_user):
             post_user = ' (as %s)' % (self.post_user)
-        return "Pathname: %s\nOutputdir: %s\nTimeout: %s\nPre: %s%s\nPost: " \
+        return "Pathname: %s\nOutputdir: %s\nTimeout: %d\nPre: %s%s\nPost: " \
                "%s%s\nUser: %s\n" % (self.pathname, self.outputdir,
                 self.timeout, self.pre, pre_user, self.post, post_user,
                 self.user)
@@ -376,7 +383,7 @@ class TestGroup(Test):
             pre_user = ' (as %s)' % (self.pre_user)
         if len(self.post_user):
             post_user = ' (as %s)' % (self.post_user)
-        return "Pathname: %s\nOutputdir: %s\nTests: %s\nTimeout: %s\n" \
+        return "Pathname: %s\nOutputdir: %s\nTests: %s\nTimeout: %d\n" \
                "Pre: %s%s\nPost: %s%s\nUser: %s\n" % (self.pathname,
                 self.outputdir, self.tests, self.timeout, self.pre, pre_user,
                 self.post, post_user, self.user)