Dump unique configurations and Uberblocks in zdb -lu

For zdb -l, detect when the configuration nvlist in some label l (l>0)
is the same as a configuration already dumped.  If so, do not dump it.

Make a similar check when dumping Uberblocks for zdb -lu.  Check whether
a label already dumped contains an identical Uberblock.  If so, do not
dump the Uberblock.

When dumping a configuration or Uberblock, state which labels it is
found in (0-3), for example: labels = 1 2 3

Detecting redundant uberblocks or configurations is accomplished by
calculating checksums of the uberblocks and the packed nvlists
containing the configuration.

If there is nothing unique to be dumped for a label (ie the
configuration and uberblocks have checksums matching those already
dumped) print nothing for that label.

With additional l's or u's, increase verbosity as follows:

-l      Dump each unique configuration only once.
        Indicate which labels it appears in.
-ll     In addition, dump label space usage stats.
-lll    Dump every configuration, unique or not.

-u      Dump each unique, valid, uberblock only once.
        Indicate which labels it appears in.
-uu     In addition, state which slots are invalid.
-uuu    Dump every uberblock, unique or not.
-uuuu   Dump the uberblock blockpointer (used to be -uuu)

Make exit values conform to the manual page.  Failing to unpack a
configuration nvlist is considered an error, as well as failing to open
or read from the device.

Add three tests, zdb_00{3,4,5}_pos to verify the above functionality.

An example of the output:
	------------------------------------
	LABEL 0
	------------------------------------
	    version: 5000
	    name: 'pool'
	    state: 1
	    txg: 880
	    < ... redacted ... >
	    features_for_read:
		com.delphix:hole_birth
		com.delphix:embedded_data
	    labels = 0
	    Uberblock[0]
		magic = 0000000000bab10c
		version = 5000
		txg = 0
		guid_sum = 3038694082047428541
		timestamp = 1487715500 UTC = Tue Feb 21 14:18:20 2017
		labels = 0 1 2 3
	    Uberblock[4]
		magic = 0000000000bab10c
		version = 5000
		txg = 772
		guid_sum = 9045970794941528051
		timestamp = 1487727291 UTC = Tue Feb 21 17:34:51 2017
		labels = 0
	    < ... redacted ... >
	------------------------------------
	LABEL 1
	------------------------------------
	    version: 5000
	    name: 'pool'
	    state: 1
	    txg: 14
	    < ... redacted ... >
		com.delphix:embedded_data
	    labels = 1 2 3
	    Uberblock[4]
		magic = 0000000000bab10c
		version = 5000
		txg = 4
		guid_sum = 7793930272573252584
		timestamp = 1487727521 UTC = Tue Feb 21 17:38:41 2017
		labels = 1 2 3
	    < ... redacted ... >

Reviewed-by: Tim Chase <[email protected]>
Reviewed-by: Don Brady <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Olaf Faaland <[email protected]>
Closes #5738 

4 files changed, 204 insertions, 1 deletions

diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/Makefile.am b/tests/zfs-tests/tests/functional/cli_root/zdb/Makefile.am
index c5bd009f4..51170fbc8 100644
--- a/tests/zfs-tests/tests/functional/cli_root/zdb/Makefile.am
+++ b/tests/zfs-tests/tests/functional/cli_root/zdb/Makefile.am
@@ -1,4 +1,7 @@
 pkgdatadir = $(datadir)/@PACKAGE@/zfs-tests/tests/functional/cli_root/zdb
 dist_pkgdata_SCRIPTS = \
 	zdb_001_neg.ksh \
-	zdb_002_pos.ksh
+	zdb_002_pos.ksh \
+	zdb_003_pos.ksh \
+	zdb_004_pos.ksh \
+	zdb_005_pos.ksh
diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_003_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_003_pos.ksh
new file mode 100755
index 000000000..d801828c1
--- /dev/null
+++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_003_pos.ksh
@@ -0,0 +1,58 @@
+#!/bin/ksh
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2017 by Lawrence Livermore National Security, LLC.
+#
+
+. $STF_SUITE/include/libtest.shlib
+
+#
+# Description:
+# zdb will not produce redundant dumps of configurations
+#
+# Strategy:
+# 1. Create a pool with two vdevs
+# 2. Copy label 1 from the first vdev to the second vdev
+# 3. Collect zdb -l output for both vdevs
+# 4. Verify that the correct number of configs is dumped for each
+#
+
+log_assert "Verify zdb does not produce redundant dumps of configurations"
+log_onexit cleanup
+
+function cleanup
+{
+	datasetexists $TESTPOOL && destroy_pool $TESTPOOL
+}
+
+verify_runnable "global"
+verify_disk_count "$DISKS" 2
+
+config_count=(1 2)
+set -A DISK $DISKS
+
+default_mirror_setup_noexit $DISKS
+log_must $DD if=/dev/${DISK[0]} of=/dev/${DISK[1]} bs=1K count=256 conv=notrunc
+
+for x in 0 1 ; do
+	config_count=$($ZDB -l $DEV_RDSKDIR/${DISK[$x]} | $GREP -c features_for_read)
+	(( $? != 0)) && log_fail "failed to get config_count from DISK[$x]"
+	log_note "vdev $x: message_count $config_count"
+	[ $config_count -ne ${config_count[$x]} ] && \
+		log_fail "zdb produces an incorrect number of configuration dumps."
+done
+
+cleanup
+
+log_pass "zdb produces unique dumps of configurations."
diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_004_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_004_pos.ksh
new file mode 100755
index 000000000..09e2194d8
--- /dev/null
+++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_004_pos.ksh
@@ -0,0 +1,78 @@
+#!/bin/ksh
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2017 by Lawrence Livermore National Security, LLC.
+#
+
+. $STF_SUITE/include/libtest.shlib
+
+#
+# Description:
+# zdb will not produce redundant dumps of uberblocks
+#
+# Strategy:
+# 1. Create a pool with two vdevs, A and B
+# 2. Offline vdev A
+# 3. Do some I/O
+# 4. Export the pool
+# 5. Copy label 1 from vdev A to vdev B
+# 6. Collect zdb -lu output for vdev B
+# 7. Verify labels 0 and 1 have unique Uberblocks, but 2 and 3 have none
+#
+
+log_assert "Verify zdb produces unique dumps of uberblocks"
+log_onexit cleanup
+
+function cleanup
+{
+	datasetexists $TESTPOOL && destroy_pool $TESTPOOL
+	for DISK in $DISKS; do
+		$ZPOOL labelclear -f $DEV_RDSKDIR/$DISK
+	done
+}
+
+verify_runnable "global"
+verify_disk_count "$DISKS" 2
+set -A DISK $DISKS
+
+default_mirror_setup_noexit $DISKS
+log_must $ZPOOL offline $TESTPOOL ${DISK[0]}
+log_must $DD if=/dev/urandom of=$TESTDIR/testfile bs=1K count=2
+log_must $ZPOOL export $TESTPOOL
+log_must $DD if=$DEV_RDSKDIR/${DISK[0]} of=$DEV_RDSKDIR/${DISK[1]} bs=1K count=256 conv=notrunc
+
+ubs=$($ZDB -lu $DEV_RDSKDIR/${DISK[1]} | $GREP -e LABEL -e Uberblock -e 'labels = ')
+log_note "vdev 1: ubs $ubs"
+
+ub_dump_counts=$($ZDB -lu $DEV_RDSKDIR/${DISK[1]} | \
+	$AWK '	/LABEL/	{label=$NF; blocks[label]=0};
+		/Uberblock/ {blocks[label]++};
+		END {print blocks[0],blocks[1],blocks[2],blocks[3]}')
+(( $? != 0)) && log_fail "failed to get ub_dump_counts from DISK[1]"
+log_note "vdev 1: ub_dump_counts $ub_dump_counts"
+
+set -A dump_count $ub_dump_counts
+for label in 0 1 2 3; do
+	if [[ $label -lt 2 ]]; then
+ 		[[ ${dump_count[$label]} -eq 0 ]] && \
+		    log_fail "zdb incorrectly dumps duplicate uberblocks"
+	else
+ 		[[ ${dump_count[$label]} -ne 0 ]] && \
+		    log_fail "zdb incorrectly dumps duplicate uberblocks"
+	fi
+done
+
+cleanup
+
+log_pass "zdb produces unique dumps of uberblocks"
diff --git a/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_005_pos.ksh b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_005_pos.ksh
new file mode 100755
index 000000000..ba9788510
--- /dev/null
+++ b/tests/zfs-tests/tests/functional/cli_root/zdb/zdb_005_pos.ksh
@@ -0,0 +1,64 @@
+#!/bin/ksh
+
+#
+# This file and its contents are supplied under the terms of the
+# Common Development and Distribution License ("CDDL"), version 1.0.
+# You may only use this file in accordance with the terms of version
+# 1.0 of the CDDL.
+#
+# A full copy of the text of the CDDL should have accompanied this
+# source.  A copy of the CDDL is also available via the Internet at
+# http://www.illumos.org/license/CDDL.
+#
+
+#
+# Copyright (c) 2017 by Lawrence Livermore National Security, LLC.
+#
+
+. $STF_SUITE/include/libtest.shlib
+
+#
+# Description:
+# zdb -l exit codes are correct
+#
+# Strategy:
+# 1. Create a pool
+# 2. Overwrite label 0 on vdev[1] with dd
+# 3. Create an empty file
+# 3. Run zdb -l on vdev[0] and verify exit value 0
+# 4. Run zdb -l on vdev[1] and verify exit value 1
+# 5. Run zdb -l on the empty file and verify exit value 2
+#
+
+log_assert "Verify zdb -l exit codes are correct"
+log_onexit cleanup
+
+function cleanup
+{
+	datasetexists $TESTPOOL && destroy_pool $TESTPOOL
+}
+
+verify_runnable "global"
+verify_disk_count "$DISKS" 2
+
+set -A DISK $DISKS
+
+default_mirror_setup_noexit $DISKS
+log_must $DD if=/dev/zero of=$DEV_RDSKDIR/${DISK[1]} bs=1K count=256 conv=notrunc
+log_must $TRUNCATE -s 0 $TEMPFILE
+
+$ZDB -l $DEV_RDSKDIR/${DISK[0]}
+[[ $? -ne 0 ]] &&
+	log_fail "zdb -l exit codes are incorrect."
+
+$ZDB -l $DEV_RDSKDIR/${DISK[1]}
+[[ $? -ne 1 ]] &&
+	log_fail "zdb -l exit codes are incorrect."
+
+$ZDB -l $TEMPFILE
+[[ $? -ne 2 ]] &&
+	log_fail "zdb -l exit codes are incorrect."
+
+cleanup
+
+log_pass "zdb -l exit codes are correct."