OpenZFS 8835 - Speculative prefetch in ZFS not working for misaligned reads

In case of misaligned I/O sequential requests are not detected as such
due to overlaps in logical block sequence:

    dmu_zfetch(fffff80198dd0ae0, 27347, 9, 1)
    dmu_zfetch(fffff80198dd0ae0, 27355, 9, 1)
    dmu_zfetch(fffff80198dd0ae0, 27363, 9, 1)
    dmu_zfetch(fffff80198dd0ae0, 27371, 9, 1)
    dmu_zfetch(fffff80198dd0ae0, 27379, 9, 1)
    dmu_zfetch(fffff80198dd0ae0, 27387, 9, 1)

This patch makes single block overlap to be counted as a stream hit,
improving performance up to several times.

Authored by: Alexander Motin <[email protected]>
Approved by: Gordon Ross <[email protected]>
Reviewed by: Matthew Ahrens <[email protected]>
Reviewed by: Allan Jude <[email protected]>
Reviewed by: Gvozden Neskovic <[email protected]>
Reviewed by: George Melikov <[email protected]>
Ported-by: Brian Behlendorf <[email protected]>

OpenZFS-issue: https://www.illumos.org/issues/8835
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/aab6dd482a
Closes #7062

1 files changed, 19 insertions, 5 deletions

diff --git a/module/zfs/dmu_zfetch.c b/module/zfs/dmu_zfetch.c
index ea9afa282..a2bf80993 100644
--- a/module/zfs/dmu_zfetch.c
+++ b/module/zfs/dmu_zfetch.c
@@ -226,19 +226,33 @@ dmu_zfetch(zfetch_t *zf, uint64_t blkid, uint64_t nblks, boolean_t fetch_data)
 
 	rw_enter(&zf->zf_rwlock, RW_READER);
 
+	/*
+	 * Find matching prefetch stream.  Depending on whether the accesses
+	 * are block-aligned, first block of the new access may either follow
+	 * the last block of the previous access, or be equal to it.
+	 */
 	for (zs = list_head(&zf->zf_stream); zs != NULL;
 	    zs = list_next(&zf->zf_stream, zs)) {
-		if (blkid == zs->zs_blkid) {
+		if (blkid == zs->zs_blkid || blkid + 1 == zs->zs_blkid) {
 			mutex_enter(&zs->zs_lock);
 			/*
 			 * zs_blkid could have changed before we
 			 * acquired zs_lock; re-check them here.
 			 */
-			if (blkid != zs->zs_blkid) {
-				mutex_exit(&zs->zs_lock);
-				continue;
+			if (blkid == zs->zs_blkid) {
+				break;
+			} else if (blkid + 1 == zs->zs_blkid) {
+				blkid++;
+				nblks--;
+				if (nblks == 0) {
+					/* Already prefetched this before. */
+					mutex_exit(&zs->zs_lock);
+					rw_exit(&zf->zf_rwlock);
+					return;
+				}
+				break;
 			}
-			break;
+			mutex_exit(&zs->zs_lock);
 		}
 	}