Distributed Spare (dRAID) Feature

This patch adds a new top-level vdev type called dRAID, which stands
for Distributed parity RAID.  This pool configuration allows all dRAID
vdevs to participate when rebuilding to a distributed hot spare device.
This can substantially reduce the total time required to restore full
parity to pool with a failed device.

A dRAID pool can be created using the new top-level `draid` type.
Like `raidz`, the desired redundancy is specified after the type:
`draid[1,2,3]`.  No additional information is required to create the
pool and reasonable default values will be chosen based on the number
of child vdevs in the dRAID vdev.

    zpool create <pool> draid[1,2,3] <vdevs...>

Unlike raidz, additional optional dRAID configuration values can be
provided as part of the draid type as colon separated values. This
allows administrators to fully specify a layout for either performance
or capacity reasons.  The supported options include:

    zpool create <pool> \
        draid[<parity>][:<data>d][:<children>c][:<spares>s] \
        <vdevs...>

    - draid[parity]       - Parity level (default 1)
    - draid[:<data>d]     - Data devices per group (default 8)
    - draid[:<children>c] - Expected number of child vdevs
    - draid[:<spares>s]   - Distributed hot spares (default 0)

Abbreviated example `zpool status` output for a 68 disk dRAID pool
with two distributed spares using special allocation classes.

```
  pool: tank
 state: ONLINE
config:

    NAME                  STATE     READ WRITE CKSUM
    slag7                 ONLINE       0     0     0
      draid2:8d:68c:2s-0  ONLINE       0     0     0
        L0                ONLINE       0     0     0
        L1                ONLINE       0     0     0
        ...
        U25               ONLINE       0     0     0
        U26               ONLINE       0     0     0
        spare-53          ONLINE       0     0     0
          U27             ONLINE       0     0     0
          draid2-0-0      ONLINE       0     0     0
        U28               ONLINE       0     0     0
        U29               ONLINE       0     0     0
        ...
        U42               ONLINE       0     0     0
        U43               ONLINE       0     0     0
    special
      mirror-1            ONLINE       0     0     0
        L5                ONLINE       0     0     0
        U5                ONLINE       0     0     0
      mirror-2            ONLINE       0     0     0
        L6                ONLINE       0     0     0
        U6                ONLINE       0     0     0
    spares
      draid2-0-0          INUSE     currently in use
      draid2-0-1          AVAIL
```

When adding test coverage for the new dRAID vdev type the following
options were added to the ztest command.  These options are leverages
by zloop.sh to test a wide range of dRAID configurations.

    -K draid|raidz|random - kind of RAID to test
    -D <value>            - dRAID data drives per group
    -S <value>            - dRAID distributed hot spares
    -R <value>            - RAID parity (raidz or dRAID)

The zpool_create, zpool_import, redundancy, replacement and fault
test groups have all been updated provide test coverage for the
dRAID feature.

Co-authored-by: Isaac Huang <[email protected]>
Co-authored-by: Mark Maybee <[email protected]>
Co-authored-by: Don Brady <[email protected]>
Co-authored-by: Matthew Ahrens <[email protected]>
Co-authored-by: Brian Behlendorf <[email protected]>
Reviewed-by: Mark Maybee <[email protected]>
Reviewed-by: Matt Ahrens <[email protected]>
Reviewed-by: Tony Hutter <[email protected]>
Signed-off-by: Brian Behlendorf <[email protected]>
Closes #10102 

3 files changed, 48 insertions, 10 deletions

diff --git a/scripts/Makefile.am b/scripts/Makefile.am
index 9d3994752..af7c36c8f 100644
--- a/scripts/Makefile.am
+++ b/scripts/Makefile.am
@@ -36,6 +36,7 @@ export ZPOOL_SCRIPT_DIR=$$CMD_DIR/zpool/zpool.d
 export ZPOOL_SCRIPTS_PATH=$$CMD_DIR/zpool/zpool.d
 export CONTRIB_DIR=@abs_top_builddir@/contrib
 export LIB_DIR=@abs_top_builddir@/lib
+export SYSCONF_DIR=@abs_top_builddir@/etc
 
 export INSTALL_UDEV_DIR=@udevdir@
 export INSTALL_UDEV_RULE_DIR=@udevruledir@
diff --git a/scripts/zfs-helpers.sh b/scripts/zfs-helpers.sh
index 02b492200..f4edd48e8 100755
--- a/scripts/zfs-helpers.sh
+++ b/scripts/zfs-helpers.sh
@@ -166,6 +166,8 @@ if [ "${INSTALL}" = "yes" ]; then
 	    "$INSTALL_UDEV_RULE_DIR/90-zfs.rules"
 	install "$CMD_DIR/zpool/zpool.d" \
 	    "$INSTALL_SYSCONF_DIR/zfs/zpool.d"
+	install "$SYSCONF_DIR/zfs/draid.d" \
+	    "$INSTALL_SYSCONF_DIR/zfs/draid.d"
 	install "$CONTRIB_DIR/pyzfs/libzfs_core" \
 	    "$INSTALL_PYTHON_DIR/libzfs_core"
 	# Ideally we would install these in the configured ${libdir}, which is
@@ -185,6 +187,7 @@ else
 	remove "$INSTALL_UDEV_RULE_DIR/69-vdev.rules"
 	remove "$INSTALL_UDEV_RULE_DIR/90-zfs.rules"
 	remove "$INSTALL_SYSCONF_DIR/zfs/zpool.d"
+	remove "$INSTALL_SYSCONF_DIR/zfs/draid.d"
 	remove "$INSTALL_PYTHON_DIR/libzfs_core"
 	remove "/lib/libzfs_core.so"
 	remove "/lib/libnvpair.so"
diff --git a/scripts/zloop.sh b/scripts/zloop.sh
index 3d9baaf0e..bbe326aa0 100755
--- a/scripts/zloop.sh
+++ b/scripts/zloop.sh
@@ -18,6 +18,7 @@
 #
 # Copyright (c) 2015 by Delphix. All rights reserved.
 # Copyright (C) 2016 Lawrence Livermore National Security, LLC.
+# Copyright (c) 2017, Intel Corporation.
 #
 
 BASE_DIR=$(dirname "$0")
@@ -246,27 +247,60 @@ while [[ $timeout -eq 0 ]] || [[ $curtime -le $((starttime + timeout)) ]]; do
 	or_die rm -rf "$workdir"
 	or_die mkdir "$workdir"
 
-	# switch between common arrangements & fully randomized
-	if [[ $((RANDOM % 2)) -eq 0 ]]; then
-		mirrors=2
-		raidz=0
+	# switch between three types of configs
+	# 1/3 basic, 1/3 raidz mix, and 1/3 draid mix
+	choice=$((RANDOM % 3))
+
+	# ashift range 9 - 15
+	align=$(((RANDOM % 2) * 3 + 9))
+
+	# randomly use special classes
+	class="special=random"
+
+	if [[ $choice -eq 0 ]]; then
+		# basic mirror only
 		parity=1
+		mirrors=2
+		draid_data=0
+		draid_spares=0
+		raid_children=0
 		vdevs=2
-	else
-		mirrors=$(((RANDOM % 3) * 1))
+		raid_type="raidz"
+	elif [[ $choice -eq 1 ]]; then
+		# fully randomized mirror/raidz (sans dRAID)
 		parity=$(((RANDOM % 3) + 1))
-		raidz=$((((RANDOM % 9) + parity + 1) * (RANDOM % 2)))
+		mirrors=$(((RANDOM % 3) * 1))
+		draid_data=0
+		draid_spares=0
+		raid_children=$((((RANDOM % 9) + parity + 1) * (RANDOM % 2)))
 		vdevs=$(((RANDOM % 3) + 3))
+		raid_type="raidz"
+	else
+		# fully randomized dRAID (sans mirror/raidz)
+		parity=$(((RANDOM % 3) + 1))
+		mirrors=0
+		draid_data=$(((RANDOM % 8) + 3))
+		draid_spares=$(((RANDOM % 2) + parity))
+		stripe=$((draid_data + parity))
+		extra=$((draid_spares + (RANDOM % 4)))
+		raid_children=$(((((RANDOM % 4) + 1) * stripe) + extra))
+		vdevs=$((RANDOM % 3))
+		raid_type="draid"
 	fi
-	align=$(((RANDOM % 2) * 3 + 9))
-	runtime=$((RANDOM % 100))
+
+	# run from 30 to 120 seconds
+	runtime=$(((RANDOM % 90) + 30))
 	passtime=$((RANDOM % (runtime / 3 + 1) + 10))
 
+	zopt="$zopt -K $raid_type"
 	zopt="$zopt -m $mirrors"
-	zopt="$zopt -r $raidz"
+	zopt="$zopt -r $raid_children"
+	zopt="$zopt -D $draid_data"
+	zopt="$zopt -S $draid_spares"
 	zopt="$zopt -R $parity"
 	zopt="$zopt -v $vdevs"
 	zopt="$zopt -a $align"
+	zopt="$zopt -C $class"
 	zopt="$zopt -T $runtime"
 	zopt="$zopt -P $passtime"
 	zopt="$zopt -s $size"