Fix L2ARC reads when compressed ARC disabled

When reading compressed blocks from the L2ARC, with
compressed ARC disabled, arc_hdr_size() returns
LSIZE rather than PSIZE, but the actual read is PSIZE.
This causes l2arc_read_done() to compare the checksum
against the wrong size, resulting in checksum failure.

This manifests as an increase in the kstat l2_cksum_bad
and the read being retried from the main pool, making the
L2ARC ineffective.

Add new L2ARC tests with Compressed ARC enabled/disabled

Blocks are handled differently depending on the state of the
zfs_compressed_arc_enabled tunable.

If a block is compressed on-disk, and compressed_arc is enabled:
- the block is read from disk
- It is NOT decompressed
- It is added to the ARC in its compressed form
- l2arc_write_buffers() may write it to the L2ARC (as is)
- l2arc_read_done() compares the checksum to the BP (compressed)

However, if compressed_arc is disabled:
- the block is read from disk
- It is decompressed
- It is added to the ARC (uncompressed)
- l2arc_write_buffers() will use l2arc_apply_transforms() to
  recompress the block, before writing it to the L2ARC
- l2arc_read_done() compares the checksum to the BP (compressed)
- l2arc_read_done() will use l2arc_untransform() to uncompress it

This test writes out a test file to a pool consisting of one disk
and one cache device, then randomly reads from it. Since the arc_max
in the tests is low, this will feed the L2ARC, and result in reads
from the L2ARC.

We compare the value of the kstat l2_cksum_bad before and after
to determine if any blocks failed to survive the trip through the
L2ARC.

Sponsored-by: The FreeBSD Foundation
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Allan Jude <[email protected]>
Closes #10693 

1 files changed, 2 insertions, 1 deletions

diff --git a/tests/runfiles/common.run b/tests/runfiles/common.run
index 931eee4fa..dcfd1cc91 100644
--- a/tests/runfiles/common.run
+++ b/tests/runfiles/common.run
@@ -536,7 +536,8 @@ user =
 tags = ['functional', 'cli_user', 'zpool_status']
 
 [tests/functional/compression]
-tests = ['compress_001_pos', 'compress_002_pos', 'compress_003_pos']
+tests = ['compress_001_pos', 'compress_002_pos', 'compress_003_pos',
+    'l2arc_compressed_arc', 'l2arc_compressed_arc_disabled']
 tags = ['functional', 'compression']
 
 [tests/functional/cp_files]