4 files changed, 96 insertions, 2 deletions
diff --git a/man/man4/zfs.4 b/man/man4/zfs.4
index 20bb95c1a..1f6be3963 100644
--- a/man/man4/zfs.4
+++ b/man/man4/zfs.4
@@ -291,6 +291,14 @@ Default dnode block size as a power of 2.
 .It Sy zfs_default_ibs Ns = Ns Sy 17 Po 128 KiB Pc Pq int
 Default dnode indirect block size as a power of 2.
 .
+.It Sy zfs_dio_enabled Ns = Ns Sy 0 Ns | Ns 1 Pq int
+Enable Direct I/O.
+If this setting is 0, then all I/O requests will be directed through the ARC
+acting as though the dataset property
+.Sy direct
+was set to
+.Sy disabled .
+.
 .It Sy zfs_history_output_max Ns = Ns Sy 1048576 Ns B Po 1 MiB Pc Pq u64
 When attempting to log an output nvlist of an ioctl in the on-disk history,
 the output will not be stored if it is larger than this size (in bytes).
@@ -416,6 +424,26 @@ May be increased up to
 .Sy ASHIFT_MAX Po 16 Pc ,
 but this may negatively impact pool space efficiency.
 .
+.It Sy zfs_vdev_direct_write_verify Ns = Ns Sy Linux 1 | FreeBSD 0 Pq uint
+If non-zero, then a Direct I/O write's checksum will be verified every
+time the write is issued and before it is commited to the block pointer.
+In the event the checksum is not valid then the I/O operation will return EIO.
+This module parameter can be used to detect if the
+contents of the users buffer have changed in the process of doing a Direct I/O
+write.
+It can also help to identify if reported checksum errors are tied to Direct I/O
+writes.
+Each verify error causes a
+.Sy dio_verify
+zevent.
+Direct Write I/O checkum verify errors can be seen with
+.Nm zpool Cm status Fl d .
+The default value for this is 1 on Linux, but is 0 for
+.Fx
+because user pages can be placed under write protection in
+.Fx
+before the Direct I/O write is issued.
+.
 .It Sy zfs_vdev_min_auto_ashift Ns = Ns Sy ASHIFT_MIN Po 9 Pc Pq uint
 Minimum ashift used when creating new top-level vdevs.
 .
@@ -1093,6 +1121,9 @@ This will smoothly handle between ten times and a tenth of this number.
 .Pp
 .Sy zfs_delay_scale No \(mu Sy zfs_dirty_data_max Em must No be smaller than Sy 2^64 .
 .
+.It Sy zfs_dio_write_verify_events_per_second Ns = Ns Sy 20 Ns /s Pq uint
+Rate limit Direct I/O write verify events to this many per second.
+.
 .It Sy zfs_disable_ivset_guid_check Ns = Ns Sy 0 Ns | Ns 1 Pq int
 Disables requirement for IVset GUIDs to be present and match when doing a raw
 receive of encrypted datasets.
diff --git a/man/man7/zfsprops.7 b/man/man7/zfsprops.7
index f7026119b..fa228e9bd 100644
--- a/man/man7/zfsprops.7
+++ b/man/man7/zfsprops.7
@@ -1039,6 +1039,48 @@ See the
 section of
 .Xr zfsconcepts 7 .
 .It Xo
+.Sy direct Ns = Ns Sy disabled Ns | Ns Sy standard Ns | Ns Sy always
+.Xc
+Controls the behavior of Direct I/O requests
+.Pq e.g. Dv O_DIRECT .
+The
+.Sy standard
+behavior for Direct I/O requests is to bypass the ARC when possible.
+These requests will not be cached and performance will be limited by the
+raw speed of the underlying disks
+.Pq Dv this is the default .
+.Sy always
+causes every properly aligned read or write to be treated as a direct request.
+.Sy disabled
+causes the O_DIRECT flag to be silently ignored and all direct requests will
+be handled by the ARC.
+This is the default behavior for OpenZFS 2.2 and prior releases.
+.Pp
+Bypassing the ARC requires that a direct request be correctly aligned.
+For write requests the starting offset and size of the request must be
+.Sy recordsize Ns
+-aligned, if not then the unaligned portion of the request will be silently
+redirected through the ARC.
+For read requests there is no
+.Sy recordsize
+alignment restriction on either the starting offset or size.
+All direct requests must use a page-aligned memory buffer and the request
+size must be a multiple of the page size or an error is returned.
+.Pp
+Concurrently mixing buffered and direct requests to overlapping regions of
+a file can decrease performance.
+However, the resulting file will always be coherent.
+For example, a direct read after a buffered write will return the data
+from the buffered write.
+Furthermore, if an application uses
+.Xr mmap 2
+based file access then in order to maintain coherency all direct requests
+are converted to buffered requests while the file is mapped.
+Currently Direct I/O is not supported with zvols.
+If dedup is enabled on a dataset, Direct I/O writes will not check for
+deduplication.
+Deduplication and Direct I/O writes are currently incompatible.
+.It Xo
 .Sy dnodesize Ns = Ns Sy legacy Ns | Ns Sy auto Ns | Ns Sy 1k Ns | Ns
 .Sy 2k Ns | Ns Sy 4k Ns | Ns Sy 8k Ns | Ns Sy 16k
 .Xc
diff --git a/man/man8/zpool-events.8 b/man/man8/zpool-events.8
index ef20ef4e0..234612bae 100644
--- a/man/man8/zpool-events.8
+++ b/man/man8/zpool-events.8
@@ -98,6 +98,17 @@ This can be an indicator of problems with the underlying storage device.
 The number of delay events is ratelimited by the
 .Sy zfs_slow_io_events_per_second
 module parameter.
+.It Sy dio_verify
+Issued when there was a checksum verify error after a Direct I/O write has been
+issued.
+This event can only take place if the module parameter
+.Sy zfs_vdev_direct_write_verify
+is not set to zero.
+See
+.Xr zfs 4
+for more details on the
+.Sy zfs_vdev_direct_write_verify
+module paramter.
 .It Sy config
 Issued every time a vdev change have been done to the pool.
 .It Sy zpool
@@ -408,8 +419,9 @@ ZIO_STAGE_VDEV_IO_DONE:0x00400000:RW--XT
 ZIO_STAGE_VDEV_IO_ASSESS:0x00800000:RW--XT
 
 ZIO_STAGE_CHECKSUM_VERIFY:0x01000000:R-----
+ZIO_STAGE_DIO_CHECKSUM_VERIFY:0x02000000:-W----
 
-ZIO_STAGE_DONE:0x02000000:RWFCXT
+ZIO_STAGE_DONE:0x04000000:RWFCXT
 .TE
 .
 .Sh I/O FLAGS
diff --git a/man/man8/zpool-status.8 b/man/man8/zpool-status.8
index b40faeb99..868fc4414 100644
--- a/man/man8/zpool-status.8
+++ b/man/man8/zpool-status.8
@@ -36,7 +36,7 @@
 .Sh SYNOPSIS
 .Nm zpool
 .Cm status
-.Op Fl DegiLpPstvx
+.Op Fl dDegiLpPstvx
 .Op Fl T Sy u Ns | Ns Sy d
 .Op Fl c Op Ar SCRIPT1 Ns Oo , Ns Ar SCRIPT2 Oc Ns …
 .Oo Ar pool Oc Ns …
@@ -81,6 +81,15 @@ to display vdevs in flat hierarchy instead of nested vdev objects.
 Specify
 .Sy --json-pool-key-guid
 to set pool GUID as key for pool objects instead of pool names.
+.It Fl d
+Display the number of Direct I/O write checksum verify errors that have occured
+on a top-level VDEV.
+See
+.Sx zfs_vdev_direct_write_verify
+in
+.Xr zfs 4
+for details about the conditions that can cause Direct I/O write checksum
+verify failures to occur.
 .It Fl D
 Display a histogram of deduplication statistics, showing the allocated
 .Pq physically present on disk