OpenZFS 9112 - Improve allocation performance on high-end systems

Overview
========

We parallelize the allocation process by creating the concept of
"allocators". There are a certain number of allocators per metaslab
group, defined by the value of a tunable at pool open time.  Each
allocator for a given metaslab group has up to 2 active metaslabs; one
"primary", and one "secondary". The primary and secondary weight mean
the same thing they did in in the pre-allocator world; primary metaslabs
are used for most allocations, secondary metaslabs are used for ditto
blocks being allocated in the same metaslab group.  There is also the
CLAIM weight, which has been separated out from the other weights, but
that is less important to understanding the patch.  The active metaslabs
for each allocator are moved from their normal place in the metaslab
tree for the group to the back of the tree. This way, they will not be
selected for use by other allocators searching for new metaslabs unless
all the passive metaslabs are unsuitable for allocations.  If that does
happen, the allocators will "steal" from each other to ensure that IOs
don't fail until there is truly no space left to perform allocations.

In addition, the alloc queue for each metaslab group has been broken
into a separate queue for each allocator. We don't want to dramatically
increase the number of inflight IOs on low-end systems, because it can
significantly increase txg times. On the other hand, we want to ensure
that there are enough IOs for each allocator to allow for good
coalescing before sending the IOs to the disk.  As a result, we take a
compromise path; each allocator's alloc queue max depth starts at a
certain value for every txg. Every time an IO completes, we increase the
max depth. This should hopefully provide a good balance between the two
failure modes, while not dramatically increasing complexity.

We also parallelize the spa_alloc_tree and spa_alloc_lock, which cause
very similar contention when selecting IOs to allocate. This
parallelization uses the same allocator scheme as metaslab selection.

Performance Results
===================

Performance improvements from this change can vary significantly based
on the number of CPUs in the system, whether or not the system has a
NUMA architecture, the speed of the drives, the values for the various
tunables, and the workload being performed. For an fio async sequential
write workload on a 24 core NUMA system with 256 GB of RAM and 8 128 GB
SSDs, there is a roughly 25% performance improvement.

Future Work
===========

Analysis of the performance of the system with this patch applied shows
that a significant new bottleneck is the vdev disk queues, which also
need to be parallelized.  Prototyping of this change has occurred, and
there was a performance improvement, but more work needs to be done
before its stability has been verified and it is ready to be upstreamed.

Authored by: Paul Dagnelie <[email protected]>
Reviewed by: Matthew Ahrens <[email protected]>
Reviewed by: George Wilson <[email protected]>
Reviewed by: Serapheim Dimitropoulos <[email protected]>
Reviewed by: Alexander Motin <[email protected]>
Reviewed by: Brian Behlendorf <[email protected]>
Approved by: Gordon Ross <[email protected]>
Ported-by: Paul Dagnelie <[email protected]>
Signed-off-by: Paul Dagnelie <[email protected]>

Porting Notes:
* Fix reservation test failures by increasing tolerance.

OpenZFS-issue: https://illumos.org/issues/9112
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/3f3cc3c3
Closes #7682

3 files changed, 19 insertions, 3 deletions

diff --git a/tests/zfs-tests/tests/functional/reservation/reservation.cfg b/tests/zfs-tests/tests/functional/reservation/reservation.cfg
index c12ad072c..b4f9af938 100644
--- a/tests/zfs-tests/tests/functional/reservation/reservation.cfg
+++ b/tests/zfs-tests/tests/functional/reservation/reservation.cfg
@@ -29,7 +29,7 @@
 #
 
 export RESV_DELTA=5242880
-export RESV_TOLERANCE=5242880	# Acceptable limit (5MB) for diff in space stats
+export RESV_TOLERANCE=10485760	# Acceptable limit (10MB) diff in space stats
 export RESV_SIZE=52428800	# Default reservation size (50MB)
 export RESV_FREE_SPACE=52428800	# Amount of space (50MB) to leave free in a pool
 export RESV_NUM_FS=10		# Number of filesystems to create
diff --git a/tests/zfs-tests/tests/functional/reservation/reservation_004_pos.sh b/tests/zfs-tests/tests/functional/reservation/reservation_004_pos.sh
index a396aae4b..f8342ff29 100755
--- a/tests/zfs-tests/tests/functional/reservation/reservation_004_pos.sh
+++ b/tests/zfs-tests/tests/functional/reservation/reservation_004_pos.sh
@@ -117,6 +117,12 @@ for obj in $OBJ_LIST ; do
 	new_space_avail=`get_prop available $TESTPOOL`
 	new_space_used=`get_prop used $TESTPOOL`
 
+	#
+	# Recent changes to metaslab logic have caused these tests to expand
+	# outside of their previous tolerance. If this is discovered to be a
+	# bug, rather than a side effect of some interactions, the reservation
+	# should be halved again.
+	#
 	log_must within_limits $space_used $new_space_used $RESV_TOLERANCE
 	log_must within_limits $space_avail $new_space_avail $RESV_TOLERANCE
 done
diff --git a/tests/zfs-tests/tests/functional/slog/slog_014_pos.ksh b/tests/zfs-tests/tests/functional/slog/slog_014_pos.ksh
index 7e1872893..0ec96ae1e 100755
--- a/tests/zfs-tests/tests/functional/slog/slog_014_pos.ksh
+++ b/tests/zfs-tests/tests/functional/slog/slog_014_pos.ksh
@@ -26,7 +26,7 @@
 #
 
 #
-# Copyright (c) 2013, 2016 by Delphix. All rights reserved.
+# Copyright (c) 2013, 2018 by Delphix. All rights reserved.
 #
 
 . $STF_SUITE/tests/functional/slog/slog.kshlib
@@ -50,9 +50,19 @@ for type in "mirror" "raidz" "raidz2"; do
 		log_must zpool create $TESTPOOL $type $VDEV $spare $SDEV \
 			log $LDEV
 
+                # Create a file to be corrupted
+                dd if=/dev/urandom of=/$TESTPOOL/filler bs=1024k count=50
+
+                #
+                # Ensure the file has been synced out before attempting to
+                # corrupt its contents.
+                #
+                sync
+
+		#
 		# Corrupt a pool device to make the pool DEGRADED
-		dd if=/dev/urandom of=/$TESTPOOL/filler bs=1024k count=50
 		# The oseek value below is to skip past the vdev label.
+		#
 		if is_linux; then
 			log_must dd if=/dev/urandom of=$VDIR/a bs=1024k \
 			   seek=4 conv=notrunc count=50