diff options
Diffstat (limited to 'man/man7')
| -rw-r--r-- | man/man7/zfsconcepts.7 | 206 | ||||
| -rw-r--r-- | man/man7/zfsprops.7 | 2067 | ||||
| -rw-r--r-- | man/man7/zpool-features.7 | 842 | ||||
| -rw-r--r-- | man/man7/zpoolconcepts.7 | 512 | ||||
| -rw-r--r-- | man/man7/zpoolprops.7 | 412 |
5 files changed, 4039 insertions, 0 deletions
diff --git a/man/man7/zfsconcepts.7 b/man/man7/zfsconcepts.7 new file mode 100644 index 000000000..f958035f7 --- /dev/null +++ b/man/man7/zfsconcepts.7 @@ -0,0 +1,206 @@ +.\" +.\" CDDL HEADER START +.\" +.\" The contents of this file are subject to the terms of the +.\" Common Development and Distribution License (the "License"). +.\" You may not use this file except in compliance with the License. +.\" +.\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +.\" or http://www.opensolaris.org/os/licensing. +.\" See the License for the specific language governing permissions +.\" and limitations under the License. +.\" +.\" When distributing Covered Code, include this CDDL HEADER in each +.\" file and include the License file at usr/src/OPENSOLARIS.LICENSE. +.\" If applicable, add the following below this CDDL HEADER, with the +.\" fields enclosed by brackets "[]" replaced with your own identifying +.\" information: Portions Copyright [yyyy] [name of copyright owner] +.\" +.\" CDDL HEADER END +.\" +.\" Copyright (c) 2009 Sun Microsystems, Inc. All Rights Reserved. +.\" Copyright 2011 Joshua M. Clulow <[email protected]> +.\" Copyright (c) 2011, 2019 by Delphix. All rights reserved. +.\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved. +.\" Copyright (c) 2014, Joyent, Inc. All rights reserved. +.\" Copyright (c) 2014 by Adam Stevko. All rights reserved. +.\" Copyright (c) 2014 Integros [integros.com] +.\" Copyright 2019 Richard Laager. All rights reserved. +.\" Copyright 2018 Nexenta Systems, Inc. +.\" Copyright 2019 Joyent, Inc. +.\" +.Dd June 30, 2019 +.Dt ZFSCONCEPTS 7 +.Os +. +.Sh NAME +.Nm zfsconcepts +.Nd overview of ZFS concepts +. +.Sh DESCRIPTION +.Ss ZFS File System Hierarchy +A ZFS storage pool is a logical collection of devices that provide space for +datasets. +A storage pool is also the root of the ZFS file system hierarchy. +.Pp +The root of the pool can be accessed as a file system, such as mounting and +unmounting, taking snapshots, and setting properties. +The physical storage characteristics, however, are managed by the +.Xr zpool 8 +command. +.Pp +See +.Xr zpool 8 +for more information on creating and administering pools. +.Ss Snapshots +A snapshot is a read-only copy of a file system or volume. +Snapshots can be created extremely quickly, and initially consume no additional +space within the pool. +As data within the active dataset changes, the snapshot consumes more data than +would otherwise be shared with the active dataset. +.Pp +Snapshots can have arbitrary names. +Snapshots of volumes can be cloned or rolled back, visibility is determined +by the +.Sy snapdev +property of the parent volume. +.Pp +File system snapshots can be accessed under the +.Pa .zfs/snapshot +directory in the root of the file system. +Snapshots are automatically mounted on demand and may be unmounted at regular +intervals. +The visibility of the +.Pa .zfs +directory can be controlled by the +.Sy snapdir +property. +.Ss Bookmarks +A bookmark is like a snapshot, a read-only copy of a file system or volume. +Bookmarks can be created extremely quickly, compared to snapshots, and they +consume no additional space within the pool. +Bookmarks can also have arbitrary names, much like snapshots. +.Pp +Unlike snapshots, bookmarks can not be accessed through the filesystem in any way. +From a storage standpoint a bookmark just provides a way to reference +when a snapshot was created as a distinct object. +Bookmarks are initially tied to a snapshot, not the filesystem or volume, +and they will survive if the snapshot itself is destroyed. +Since they are very light weight there's little incentive to destroy them. +.Ss Clones +A clone is a writable volume or file system whose initial contents are the same +as another dataset. +As with snapshots, creating a clone is nearly instantaneous, and initially +consumes no additional space. +.Pp +Clones can only be created from a snapshot. +When a snapshot is cloned, it creates an implicit dependency between the parent +and child. +Even though the clone is created somewhere else in the dataset hierarchy, the +original snapshot cannot be destroyed as long as a clone exists. +The +.Sy origin +property exposes this dependency, and the +.Cm destroy +command lists any such dependencies, if they exist. +.Pp +The clone parent-child dependency relationship can be reversed by using the +.Cm promote +subcommand. +This causes the +.Qq origin +file system to become a clone of the specified file system, which makes it +possible to destroy the file system that the clone was created from. +.Ss "Mount Points" +Creating a ZFS file system is a simple operation, so the number of file systems +per system is likely to be numerous. +To cope with this, ZFS automatically manages mounting and unmounting file +systems without the need to edit the +.Pa /etc/fstab +file. +All automatically managed file systems are mounted by ZFS at boot time. +.Pp +By default, file systems are mounted under +.Pa /path , +where +.Ar path +is the name of the file system in the ZFS namespace. +Directories are created and destroyed as needed. +.Pp +A file system can also have a mount point set in the +.Sy mountpoint +property. +This directory is created as needed, and ZFS automatically mounts the file +system when the +.Nm zfs Cm mount Fl a +command is invoked +.Po without editing +.Pa /etc/fstab +.Pc . +The +.Sy mountpoint +property can be inherited, so if +.Em pool/home +has a mount point of +.Pa /export/stuff , +then +.Em pool/home/user +automatically inherits a mount point of +.Pa /export/stuff/user . +.Pp +A file system +.Sy mountpoint +property of +.Sy none +prevents the file system from being mounted. +.Pp +If needed, ZFS file systems can also be managed with traditional tools +.Po +.Nm mount , +.Nm umount , +.Pa /etc/fstab +.Pc . +If a file system's mount point is set to +.Sy legacy , +ZFS makes no attempt to manage the file system, and the administrator is +responsible for mounting and unmounting the file system. +Because pools must +be imported before a legacy mount can succeed, administrators should ensure +that legacy mounts are only attempted after the zpool import process +finishes at boot time. +For example, on machines using systemd, the mount option +.Pp +.Nm x-systemd.requires=zfs-import.target +.Pp +will ensure that the zfs-import completes before systemd attempts mounting +the filesystem. +See +.Xr systemd.mount 5 +for details. +.Ss Deduplication +Deduplication is the process for removing redundant data at the block level, +reducing the total amount of data stored. +If a file system has the +.Sy dedup +property enabled, duplicate data blocks are removed synchronously. +The result +is that only unique data is stored and common components are shared among files. +.Pp +Deduplicating data is a very resource-intensive operation. +It is generally recommended that you have at least 1.25 GiB of RAM +per 1 TiB of storage when you enable deduplication. +Calculating the exact requirement depends heavily +on the type of data stored in the pool. +.Pp +Enabling deduplication on an improperly-designed system can result in +performance issues (slow IO and administrative operations). +It can potentially lead to problems importing a pool due to memory exhaustion. +Deduplication can consume significant processing power (CPU) and memory as well +as generate additional disk IO. +.Pp +Before creating a pool with deduplication enabled, ensure that you have planned +your hardware requirements appropriately and implemented appropriate recovery +practices, such as regular backups. +Consider using the +.Sy compression +property as a less resource-intensive alternative. diff --git a/man/man7/zfsprops.7 b/man/man7/zfsprops.7 new file mode 100644 index 000000000..3f3ddcebf --- /dev/null +++ b/man/man7/zfsprops.7 @@ -0,0 +1,2067 @@ +.\" +.\" CDDL HEADER START +.\" +.\" The contents of this file are subject to the terms of the +.\" Common Development and Distribution License (the "License"). +.\" You may not use this file except in compliance with the License. +.\" +.\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +.\" or http://www.opensolaris.org/os/licensing. +.\" See the License for the specific language governing permissions +.\" and limitations under the License. +.\" +.\" When distributing Covered Code, include this CDDL HEADER in each +.\" file and include the License file at usr/src/OPENSOLARIS.LICENSE. +.\" If applicable, add the following below this CDDL HEADER, with the +.\" fields enclosed by brackets "[]" replaced with your own identifying +.\" information: Portions Copyright [yyyy] [name of copyright owner] +.\" +.\" CDDL HEADER END +.\" +.\" Copyright (c) 2009 Sun Microsystems, Inc. All Rights Reserved. +.\" Copyright 2011 Joshua M. Clulow <[email protected]> +.\" Copyright (c) 2011, 2019 by Delphix. All rights reserved. +.\" Copyright (c) 2011, Pawel Jakub Dawidek <[email protected]> +.\" Copyright (c) 2012, Glen Barber <[email protected]> +.\" Copyright (c) 2012, Bryan Drewery <[email protected]> +.\" Copyright (c) 2013, Steven Hartland <[email protected]> +.\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved. +.\" Copyright (c) 2014, Joyent, Inc. All rights reserved. +.\" Copyright (c) 2014 by Adam Stevko. All rights reserved. +.\" Copyright (c) 2014 Integros [integros.com] +.\" Copyright (c) 2016 Nexenta Systems, Inc. All Rights Reserved. +.\" Copyright (c) 2014, Xin LI <[email protected]> +.\" Copyright (c) 2014-2015, The FreeBSD Foundation, All Rights Reserved. +.\" Copyright 2019 Richard Laager. All rights reserved. +.\" Copyright 2018 Nexenta Systems, Inc. +.\" Copyright 2019 Joyent, Inc. +.\" Copyright (c) 2019, Kjeld Schouten-Lebbing +.\" +.Dd May 24, 2021 +.Dt ZFSPROPS 7 +.Os +. +.Sh NAME +.Nm zfsprops +.Nd native and user-defined properties of ZFS datasets +. +.Sh DESCRIPTION +Properties are divided into two types, native properties and user-defined +.Po or +.Qq user +.Pc +properties. +Native properties either export internal statistics or control ZFS behavior. +In addition, native properties are either editable or read-only. +User properties have no effect on ZFS behavior, but you can use them to annotate +datasets in a way that is meaningful in your environment. +For more information about user properties, see the +.Sx User Properties +section, below. +. +.Ss Native Properties +Every dataset has a set of properties that export statistics about the dataset +as well as control various behaviors. +Properties are inherited from the parent unless overridden by the child. +Some properties apply only to certain types of datasets +.Pq file systems, volumes, or snapshots . +.Pp +The values of numeric properties can be specified using human-readable suffixes +.Po for example, +.Sy k , +.Sy KB , +.Sy M , +.Sy Gb , +and so forth, up to +.Sy Z +for zettabyte +.Pc . +The following are all valid +.Pq and equal +specifications: +.Li 1536M, 1.5g, 1.50GB . +.Pp +The values of non-numeric properties are case sensitive and must be lowercase, +except for +.Sy mountpoint , +.Sy sharenfs , +and +.Sy sharesmb . +.Pp +The following native properties consist of read-only statistics about the +dataset. +These properties can be neither set, nor inherited. +Native properties apply to all dataset types unless otherwise noted. +.Bl -tag -width "usedbyrefreservation" +.It Sy available +The amount of space available to the dataset and all its children, assuming that +there is no other activity in the pool. +Because space is shared within a pool, availability can be limited by any number +of factors, including physical pool size, quotas, reservations, or other +datasets within the pool. +.Pp +This property can also be referred to by its shortened column name, +.Sy avail . +.It Sy compressratio +For non-snapshots, the compression ratio achieved for the +.Sy used +space of this dataset, expressed as a multiplier. +The +.Sy used +property includes descendant datasets, and, for clones, does not include the +space shared with the origin snapshot. +For snapshots, the +.Sy compressratio +is the same as the +.Sy refcompressratio +property. +Compression can be turned on by running: +.Nm zfs Cm set Sy compression Ns = Ns Sy on Ar dataset . +The default value is +.Sy off . +.It Sy createtxg +The transaction group (txg) in which the dataset was created. +Bookmarks have the same +.Sy createtxg +as the snapshot they are initially tied to. +This property is suitable for ordering a list of snapshots, +e.g. for incremental send and receive. +.It Sy creation +The time this dataset was created. +.It Sy clones +For snapshots, this property is a comma-separated list of filesystems or volumes +which are clones of this snapshot. +The clones' +.Sy origin +property is this snapshot. +If the +.Sy clones +property is not empty, then this snapshot can not be destroyed +.Po even with the +.Fl r +or +.Fl f +options +.Pc . +The roles of origin and clone can be swapped by promoting the clone with the +.Nm zfs Cm promote +command. +.It Sy defer_destroy +This property is +.Sy on +if the snapshot has been marked for deferred destroy by using the +.Nm zfs Cm destroy Fl d +command. +Otherwise, the property is +.Sy off . +.It Sy encryptionroot +For encrypted datasets, indicates where the dataset is currently inheriting its +encryption key from. +Loading or unloading a key for the +.Sy encryptionroot +will implicitly load / unload the key for any inheriting datasets (see +.Nm zfs Cm load-key +and +.Nm zfs Cm unload-key +for details). +Clones will always share an +encryption key with their origin. +See the +.Sx Encryption +section of +.Xr zfs-load-key 8 +for details. +.It Sy filesystem_count +The total number of filesystems and volumes that exist under this location in +the dataset tree. +This value is only available when a +.Sy filesystem_limit +has been set somewhere in the tree under which the dataset resides. +.It Sy keystatus +Indicates if an encryption key is currently loaded into ZFS. +The possible values are +.Sy none , +.Sy available , +and +.Sy unavailable . +See +.Nm zfs Cm load-key +and +.Nm zfs Cm unload-key . +.It Sy guid +The 64 bit GUID of this dataset or bookmark which does not change over its +entire lifetime. +When a snapshot is sent to another pool, the received snapshot has the same GUID. +Thus, the +.Sy guid +is suitable to identify a snapshot across pools. +.It Sy logicalreferenced +The amount of space that is +.Qq logically +accessible by this dataset. +See the +.Sy referenced +property. +The logical space ignores the effect of the +.Sy compression +and +.Sy copies +properties, giving a quantity closer to the amount of data that applications +see. +However, it does include space consumed by metadata. +.Pp +This property can also be referred to by its shortened column name, +.Sy lrefer . +.It Sy logicalused +The amount of space that is +.Qq logically +consumed by this dataset and all its descendents. +See the +.Sy used +property. +The logical space ignores the effect of the +.Sy compression +and +.Sy copies +properties, giving a quantity closer to the amount of data that applications +see. +However, it does include space consumed by metadata. +.Pp +This property can also be referred to by its shortened column name, +.Sy lused . +.It Sy mounted +For file systems, indicates whether the file system is currently mounted. +This property can be either +.Sy yes +or +.Sy no . +.It Sy objsetid +A unique identifier for this dataset within the pool. +Unlike the dataset's +.Sy guid , No the Sy objsetid +of a dataset is not transferred to other pools when the snapshot is copied +with a send/receive operation. +The +.Sy objsetid +can be reused (for a new dataset) after the dataset is deleted. +.It Sy origin +For cloned file systems or volumes, the snapshot from which the clone was +created. +See also the +.Sy clones +property. +.It Sy receive_resume_token +For filesystems or volumes which have saved partially-completed state from +.Nm zfs Cm receive Fl s , +this opaque token can be provided to +.Nm zfs Cm send Fl t +to resume and complete the +.Nm zfs Cm receive . +.It Sy redact_snaps +For bookmarks, this is the list of snapshot guids the bookmark contains a redaction +list for. +For snapshots, this is the list of snapshot guids the snapshot is redacted with +respect to. +.It Sy referenced +The amount of data that is accessible by this dataset, which may or may not be +shared with other datasets in the pool. +When a snapshot or clone is created, it initially references the same amount of +space as the file system or snapshot it was created from, since its contents are +identical. +.Pp +This property can also be referred to by its shortened column name, +.Sy refer . +.It Sy refcompressratio +The compression ratio achieved for the +.Sy referenced +space of this dataset, expressed as a multiplier. +See also the +.Sy compressratio +property. +.It Sy snapshot_count +The total number of snapshots that exist under this location in the dataset +tree. +This value is only available when a +.Sy snapshot_limit +has been set somewhere in the tree under which the dataset resides. +.It Sy type +The type of dataset: +.Sy filesystem , +.Sy volume , +.Sy snapshot , +or +.Sy bookmark . +.It Sy used +The amount of space consumed by this dataset and all its descendents. +This is the value that is checked against this dataset's quota and reservation. +The space used does not include this dataset's reservation, but does take into +account the reservations of any descendent datasets. +The amount of space that a dataset consumes from its parent, as well as the +amount of space that is freed if this dataset is recursively destroyed, is the +greater of its space used and its reservation. +.Pp +The used space of a snapshot +.Po see the +.Sx Snapshots +section of +.Xr zfsconcepts 7 +.Pc +is space that is referenced exclusively by this snapshot. +If this snapshot is destroyed, the amount of +.Sy used +space will be freed. +Space that is shared by multiple snapshots isn't accounted for in this metric. +When a snapshot is destroyed, space that was previously shared with this +snapshot can become unique to snapshots adjacent to it, thus changing the used +space of those snapshots. +The used space of the latest snapshot can also be affected by changes in the +file system. +Note that the +.Sy used +space of a snapshot is a subset of the +.Sy written +space of the snapshot. +.Pp +The amount of space used, available, or referenced does not take into account +pending changes. +Pending changes are generally accounted for within a few seconds. +Committing a change to a disk using +.Xr fsync 2 +or +.Sy O_SYNC +does not necessarily guarantee that the space usage information is updated +immediately. +.It Sy usedby* +The +.Sy usedby* +properties decompose the +.Sy used +properties into the various reasons that space is used. +Specifically, +.Sy used No = +.Sy usedbychildren No + +.Sy usedbydataset No + +.Sy usedbyrefreservation No + +.Sy usedbysnapshots . +These properties are only available for datasets created on +.Nm zpool +.Qo version 13 Qc +pools. +.It Sy usedbychildren +The amount of space used by children of this dataset, which would be freed if +all the dataset's children were destroyed. +.It Sy usedbydataset +The amount of space used by this dataset itself, which would be freed if the +dataset were destroyed +.Po after first removing any +.Sy refreservation +and destroying any necessary snapshots or descendents +.Pc . +.It Sy usedbyrefreservation +The amount of space used by a +.Sy refreservation +set on this dataset, which would be freed if the +.Sy refreservation +was removed. +.It Sy usedbysnapshots +The amount of space consumed by snapshots of this dataset. +In particular, it is the amount of space that would be freed if all of this +dataset's snapshots were destroyed. +Note that this is not simply the sum of the snapshots' +.Sy used +properties because space can be shared by multiple snapshots. +.It Sy userused Ns @ Ns Ar user +The amount of space consumed by the specified user in this dataset. +Space is charged to the owner of each file, as displayed by +.Nm ls Fl l . +The amount of space charged is displayed by +.Nm du No and Nm ls Fl s . +See the +.Nm zfs Cm userspace +command for more information. +.Pp +Unprivileged users can access only their own space usage. +The root user, or a user who has been granted the +.Sy userused +privilege with +.Nm zfs Cm allow , +can access everyone's usage. +.Pp +The +.Sy userused Ns @ Ns Ar ... +properties are not displayed by +.Nm zfs Cm get Sy all . +The user's name must be appended after the +.Sy @ +symbol, using one of the following forms: +.Bl -bullet -compact -offset 4n +.It +POSIX name +.Pq Qq joe +.It +POSIX numeric ID +.Pq Qq 789 +.It +SID name +.Pq Qq joe.smith@mydomain +.It +SID numeric ID +.Pq Qq S-1-123-456-789 +.El +.Pp +Files created on Linux always have POSIX owners. +.It Sy userobjused Ns @ Ns Ar user +The +.Sy userobjused +property is similar to +.Sy userused +but instead it counts the number of objects consumed by a user. +This property counts all objects allocated on behalf of the user, +it may differ from the results of system tools such as +.Nm df Fl i . +.Pp +When the property +.Sy xattr Ns = Ns Sy on +is set on a file system additional objects will be created per-file to store +extended attributes. +These additional objects are reflected in the +.Sy userobjused +value and are counted against the user's +.Sy userobjquota . +When a file system is configured to use +.Sy xattr Ns = Ns Sy sa +no additional internal objects are normally required. +.It Sy userrefs +This property is set to the number of user holds on this snapshot. +User holds are set by using the +.Nm zfs Cm hold +command. +.It Sy groupused Ns @ Ns Ar group +The amount of space consumed by the specified group in this dataset. +Space is charged to the group of each file, as displayed by +.Nm ls Fl l . +See the +.Sy userused Ns @ Ns Ar user +property for more information. +.Pp +Unprivileged users can only access their own groups' space usage. +The root user, or a user who has been granted the +.Sy groupused +privilege with +.Nm zfs Cm allow , +can access all groups' usage. +.It Sy groupobjused Ns @ Ns Ar group +The number of objects consumed by the specified group in this dataset. +Multiple objects may be charged to the group for each file when extended +attributes are in use. +See the +.Sy userobjused Ns @ Ns Ar user +property for more information. +.Pp +Unprivileged users can only access their own groups' space usage. +The root user, or a user who has been granted the +.Sy groupobjused +privilege with +.Nm zfs Cm allow , +can access all groups' usage. +.It Sy projectused Ns @ Ns Ar project +The amount of space consumed by the specified project in this dataset. +Project is identified via the project identifier (ID) that is object-based +numeral attribute. +An object can inherit the project ID from its parent object (if the +parent has the flag of inherit project ID that can be set and changed via +.Nm chattr Fl /+P +or +.Nm zfs project Fl s ) +when being created. +The privileged user can set and change object's project +ID via +.Nm chattr Fl p +or +.Nm zfs project Fl s +anytime. +Space is charged to the project of each file, as displayed by +.Nm lsattr Fl p +or +.Nm zfs project . +See the +.Sy userused Ns @ Ns Ar user +property for more information. +.Pp +The root user, or a user who has been granted the +.Sy projectused +privilege with +.Nm zfs allow , +can access all projects' usage. +.It Sy projectobjused Ns @ Ns Ar project +The +.Sy projectobjused +is similar to +.Sy projectused +but instead it counts the number of objects consumed by project. +When the property +.Sy xattr Ns = Ns Sy on +is set on a fileset, ZFS will create additional objects per-file to store +extended attributes. +These additional objects are reflected in the +.Sy projectobjused +value and are counted against the project's +.Sy projectobjquota . +When a filesystem is configured to use +.Sy xattr Ns = Ns Sy sa +no additional internal objects are required. +See the +.Sy userobjused Ns @ Ns Ar user +property for more information. +.Pp +The root user, or a user who has been granted the +.Sy projectobjused +privilege with +.Nm zfs allow , +can access all projects' objects usage. +.It Sy volblocksize +For volumes, specifies the block size of the volume. +The +.Sy blocksize +cannot be changed once the volume has been written, so it should be set at +volume creation time. +The default +.Sy blocksize +for volumes is 8 Kbytes. +Any power of 2 from 512 bytes to 128 Kbytes is valid. +.Pp +This property can also be referred to by its shortened column name, +.Sy volblock . +.It Sy written +The amount of space +.Sy referenced +by this dataset, that was written since the previous snapshot +.Pq i.e. that is not referenced by the previous snapshot . +.It Sy written Ns @ Ns Ar snapshot +The amount of +.Sy referenced +space written to this dataset since the specified snapshot. +This is the space that is referenced by this dataset but was not referenced by +the specified snapshot. +.Pp +The +.Ar snapshot +may be specified as a short snapshot name +.Pq just the part after the Sy @ , +in which case it will be interpreted as a snapshot in the same filesystem as +this dataset. +The +.Ar snapshot +may be a full snapshot name +.Pq Ar filesystem Ns @ Ns Ar snapshot , +which for clones may be a snapshot in the origin's filesystem +.Pq or the origin of the origin's filesystem, etc. +.El +.Pp +The following native properties can be used to change the behavior of a ZFS +dataset. +.Bl -tag -width "" +.It Xo +.Sy aclinherit Ns = Ns Sy discard Ns | Ns Sy noallow Ns | Ns +.Sy restricted Ns | Ns Sy passthrough Ns | Ns Sy passthrough-x +.Xc +Controls how ACEs are inherited when files and directories are created. +.Bl -tag -compact -offset 4n -width "passthrough-x" +.It Sy discard +does not inherit any ACEs. +.It Sy noallow +only inherits inheritable ACEs that specify +.Qq deny +permissions. +.It Sy restricted +default, removes the +.Sy write_acl +and +.Sy write_owner +permissions when the ACE is inherited. +.It Sy passthrough +inherits all inheritable ACEs without any modifications. +.It Sy passthrough-x +same meaning as +.Sy passthrough , +except that the +.Sy owner@ , group@ , No and Sy everyone@ +ACEs inherit the execute permission only if the file creation mode also requests +the execute bit. +.El +.Pp +When the property value is set to +.Sy passthrough , +files are created with a mode determined by the inheritable ACEs. +If no inheritable ACEs exist that affect the mode, then the mode is set in +accordance to the requested mode from the application. +.Pp +The +.Sy aclinherit +property does not apply to POSIX ACLs. +.It Xo +.Sy aclmode Ns = Ns Sy discard Ns | Ns Sy groupmask Ns | Ns +.Sy passthrough Ns | Ns Sy restricted Ns +.Xc +Controls how an ACL is modified during chmod(2) and how inherited ACEs +are modified by the file creation mode: +.Bl -tag -compact -offset 4n -width "passthrough" +.It Sy discard +default, deletes all +.Sy ACEs +except for those representing +the mode of the file or directory requested by +.Xr chmod 2 . +.It Sy groupmask +reduces permissions granted in all +.Sy ALLOW +entries found in the +.Sy ACL +such that they are no greater than the group permissions specified by +.Xr chmod 2 . +.It Sy passthrough +indicates that no changes are made to the ACL other than creating or updating +the necessary ACL entries to represent the new mode of the file or directory. +.It Sy restricted +will cause the +.Xr chmod 2 +operation to return an error when used on any file or directory which has +a non-trivial ACL whose entries can not be represented by a mode. +.Xr chmod 2 +is required to change the set user ID, set group ID, or sticky bits on a file +or directory, as they do not have equivalent ACL entries. +In order to use +.Xr chmod 2 +on a file or directory with a non-trivial ACL when +.Sy aclmode +is set to +.Sy restricted , +you must first remove all ACL entries which do not represent the current mode. +.El +.It Sy acltype Ns = Ns Sy off Ns | Ns Sy nfsv4 Ns | Ns Sy posix +Controls whether ACLs are enabled and if so what type of ACL to use. +When this property is set to a type of ACL not supported by the current +platform, the behavior is the same as if it were set to +.Sy off . +.Bl -tag -compact -offset 4n -width "posixacl" +.It Sy off +default on Linux, when a file system has the +.Sy acltype +property set to off then ACLs are disabled. +.It Sy noacl +an alias for +.Sy off +.It Sy nfsv4 +default on +.Fx , +indicates that NFSv4-style ZFS ACLs should be used. +These ACLs can be managed with the +.Xr getfacl 1 +and +.Xr setfacl 1 . +The +.Sy nfsv4 +ZFS ACL type is not yet supported on Linux. +.It Sy posix +indicates POSIX ACLs should be used. +POSIX ACLs are specific to Linux and are not functional on other platforms. +POSIX ACLs are stored as an extended +attribute and therefore will not overwrite any existing NFSv4 ACLs which +may be set. +.It Sy posixacl +an alias for +.Sy posix +.El +.Pp +To obtain the best performance when setting +.Sy posix +users are strongly encouraged to set the +.Sy xattr Ns = Ns Sy sa +property. +This will result in the POSIX ACL being stored more efficiently on disk. +But as a consequence, all new extended attributes will only be +accessible from OpenZFS implementations which support the +.Sy xattr Ns = Ns Sy sa +property. +See the +.Sy xattr +property for more details. +.It Sy atime Ns = Ns Sy on Ns | Ns Sy off +Controls whether the access time for files is updated when they are read. +Turning this property off avoids producing write traffic when reading files and +can result in significant performance gains, though it might confuse mailers +and other similar utilities. +The values +.Sy on +and +.Sy off +are equivalent to the +.Sy atime +and +.Sy noatime +mount options. +The default value is +.Sy on . +See also +.Sy relatime +below. +.It Sy canmount Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Sy noauto +If this property is set to +.Sy off , +the file system cannot be mounted, and is ignored by +.Nm zfs Cm mount Fl a . +Setting this property to +.Sy off +is similar to setting the +.Sy mountpoint +property to +.Sy none , +except that the dataset still has a normal +.Sy mountpoint +property, which can be inherited. +Setting this property to +.Sy off +allows datasets to be used solely as a mechanism to inherit properties. +One example of setting +.Sy canmount Ns = Ns Sy off +is to have two datasets with the same +.Sy mountpoint , +so that the children of both datasets appear in the same directory, but might +have different inherited characteristics. +.Pp +When set to +.Sy noauto , +a dataset can only be mounted and unmounted explicitly. +The dataset is not mounted automatically when the dataset is created or +imported, nor is it mounted by the +.Nm zfs Cm mount Fl a +command or unmounted by the +.Nm zfs Cm unmount Fl a +command. +.Pp +This property is not inherited. +.It Xo +.Sy checksum Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Sy fletcher2 Ns | Ns +.Sy fletcher4 Ns | Ns Sy sha256 Ns | Ns Sy noparity Ns | Ns +.Sy sha512 Ns | Ns Sy skein Ns | Ns Sy edonr +.Xc +Controls the checksum used to verify data integrity. +The default value is +.Sy on , +which automatically selects an appropriate algorithm +.Po currently, +.Sy fletcher4 , +but this may change in future releases +.Pc . +The value +.Sy off +disables integrity checking on user data. +The value +.Sy noparity +not only disables integrity but also disables maintaining parity for user data. +This setting is used internally by a dump device residing on a RAID-Z pool and +should not be used by any other dataset. +Disabling checksums is +.Em NOT +a recommended practice. +.Pp +The +.Sy sha512 , +.Sy skein , +and +.Sy edonr +checksum algorithms require enabling the appropriate features on the pool. +.Fx +does not support the +.Sy edonr +algorithm. +.Pp +Please see +.Xr zpool-features 7 +for more information on these algorithms. +.Pp +Changing this property affects only newly-written data. +.It Xo +.Sy compression Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Sy gzip Ns | Ns +.Sy gzip- Ns Ar N Ns | Ns Sy lz4 Ns | Ns Sy lzjb Ns | Ns Sy zle Ns | Ns Sy zstd Ns | Ns +.Sy zstd- Ns Ar N Ns | Ns Sy zstd-fast Ns | Ns Sy zstd-fast- Ns Ar N +.Xc +Controls the compression algorithm used for this dataset. +.Pp +Setting compression to +.Sy on +indicates that the current default compression algorithm should be used. +The default balances compression and decompression speed, with compression ratio +and is expected to work well on a wide variety of workloads. +Unlike all other settings for this property, +.Sy on +does not select a fixed compression type. +As new compression algorithms are added to ZFS and enabled on a pool, the +default compression algorithm may change. +The current default compression algorithm is either +.Sy lzjb +or, if the +.Sy lz4_compress +feature is enabled, +.Sy lz4 . +.Pp +The +.Sy lz4 +compression algorithm is a high-performance replacement for the +.Sy lzjb +algorithm. +It features significantly faster compression and decompression, as well as a +moderately higher compression ratio than +.Sy lzjb , +but can only be used on pools with the +.Sy lz4_compress +feature set to +.Sy enabled . +See +.Xr zpool-features 7 +for details on ZFS feature flags and the +.Sy lz4_compress +feature. +.Pp +The +.Sy lzjb +compression algorithm is optimized for performance while providing decent data +compression. +.Pp +The +.Sy gzip +compression algorithm uses the same compression as the +.Xr gzip 1 +command. +You can specify the +.Sy gzip +level by using the value +.Sy gzip- Ns Ar N , +where +.Ar N +is an integer from 1 +.Pq fastest +to 9 +.Pq best compression ratio . +Currently, +.Sy gzip +is equivalent to +.Sy gzip-6 +.Po which is also the default for +.Xr gzip 1 +.Pc . +.Pp +The +.Sy zstd +compression algorithm provides both high compression ratios and good performance. +You can specify the +.Sy zstd +level by using the value +.Sy zstd- Ns Ar N , +where +.Ar N +is an integer from 1 +.Pq fastest +to 19 +.Pq best compression ratio . +.Sy zstd +is equivalent to +.Sy zstd-3 . +.Pp +Faster speeds at the cost of the compression ratio can be requested by +setting a negative +.Sy zstd +level. +This is done using +.Sy zstd-fast- Ns Ar N , +where +.Ar N +is an integer in [1-9,10,20,30,...,100,500,1000] which maps to a negative +.Sy zstd +level. +The lower the level the faster the compression - +.Ar 1000 No provides the fastest compression and lowest compression ratio. +.Sy zstd-fast +is equivalent to +.Sy zstd-fast-1 . +.Pp +The +.Sy zle +compression algorithm compresses runs of zeros. +.Pp +This property can also be referred to by its shortened column name +.Sy compress . +Changing this property affects only newly-written data. +.Pp +When any setting except +.Sy off +is selected, compression will explicitly check for blocks consisting of only +zeroes (the NUL byte). +When a zero-filled block is detected, it is stored as +a hole and not compressed using the indicated compression algorithm. +.Pp +Any block being compressed must be no larger than 7/8 of its original size +after compression, otherwise the compression will not be considered worthwhile +and the block saved uncompressed. +Note that when the logical block is less than +8 times the disk sector size this effectively reduces the necessary compression +ratio; for example, 8kB blocks on disks with 4kB disk sectors must compress to 1/2 +or less of their original size. +.It Xo +.Sy context Ns = Ns Sy none Ns | Ns +.Ar SELinux-User : Ns Ar SElinux-Role : Ns Ar Selinux-Type : Ns Ar Sensitivity-Level +.Xc +This flag sets the SELinux context for all files in the file system under +a mount point for that file system. +See +.Xr selinux 8 +for more information. +.It Xo +.Sy fscontext Ns = Ns Sy none Ns | Ns +.Ar SELinux-User : Ns Ar SElinux-Role : Ns Ar Selinux-Type : Ns Ar Sensitivity-Level +.Xc +This flag sets the SELinux context for the file system file system being +mounted. +See +.Xr selinux 8 +for more information. +.It Xo +.Sy defcontext Ns = Ns Sy none Ns | Ns +.Ar SELinux-User : Ns Ar SElinux-Role : Ns Ar Selinux-Type : Ns Ar Sensitivity-Level +.Xc +This flag sets the SELinux default context for unlabeled files. +See +.Xr selinux 8 +for more information. +.It Xo +.Sy rootcontext Ns = Ns Sy none Ns | Ns +.Ar SELinux-User : Ns Ar SElinux-Role : Ns Ar Selinux-Type : Ns Ar Sensitivity-Level +.Xc +This flag sets the SELinux context for the root inode of the file system. +See +.Xr selinux 8 +for more information. +.It Sy copies Ns = Ns Sy 1 Ns | Ns Sy 2 Ns | Ns Sy 3 +Controls the number of copies of data stored for this dataset. +These copies are in addition to any redundancy provided by the pool, for +example, mirroring or RAID-Z. +The copies are stored on different disks, if possible. +The space used by multiple copies is charged to the associated file and dataset, +changing the +.Sy used +property and counting against quotas and reservations. +.Pp +Changing this property only affects newly-written data. +Therefore, set this property at file system creation time by using the +.Fl o Sy copies Ns = Ns Ar N +option. +.Pp +Remember that ZFS will not import a pool with a missing top-level vdev. +Do +.Em NOT +create, for example a two-disk striped pool and set +.Sy copies Ns = Ns Ar 2 +on some datasets thinking you have setup redundancy for them. +When a disk fails you will not be able to import the pool +and will have lost all of your data. +.Pp +Encrypted datasets may not have +.Sy copies Ns = Ns Ar 3 +since the implementation stores some encryption metadata where the third copy +would normally be. +.It Sy devices Ns = Ns Sy on Ns | Ns Sy off +Controls whether device nodes can be opened on this file system. +The default value is +.Sy on . +The values +.Sy on +and +.Sy off +are equivalent to the +.Sy dev +and +.Sy nodev +mount options. +.It Xo +.Sy dedup Ns = Ns Sy off Ns | Ns Sy on Ns | Ns Sy verify Ns | Ns +.Sy sha256 Ns Oo , Ns Sy verify Oc Ns | Ns Sy sha512 Ns Oo , Ns Sy verify Oc Ns | Ns Sy skein Ns Oo , Ns Sy verify Oc Ns | Ns +.Sy edonr , Ns Sy verify +.Xc +Configures deduplication for a dataset. +The default value is +.Sy off . +The default deduplication checksum is +.Sy sha256 +(this may change in the future). +When +.Sy dedup +is enabled, the checksum defined here overrides the +.Sy checksum +property. +Setting the value to +.Sy verify +has the same effect as the setting +.Sy sha256 , Ns Sy verify . +.Pp +If set to +.Sy verify , +ZFS will do a byte-to-byte comparison in case of two blocks having the same +signature to make sure the block contents are identical. +Specifying +.Sy verify +is mandatory for the +.Sy edonr +algorithm. +.Pp +Unless necessary, deduplication should +.Em not +be enabled on a system. +See the +.Sx Deduplication +section of +.Xr zfsconcepts 7 . +.It Xo +.Sy dnodesize Ns = Ns Sy legacy Ns | Ns Sy auto Ns | Ns Sy 1k Ns | Ns +.Sy 2k Ns | Ns Sy 4k Ns | Ns Sy 8k Ns | Ns Sy 16k +.Xc +Specifies a compatibility mode or literal value for the size of dnodes in the +file system. +The default value is +.Sy legacy . +Setting this property to a value other than +.Sy legacy No requires the Sy large_dnode No pool feature to be enabled. +.Pp +Consider setting +.Sy dnodesize +to +.Sy auto +if the dataset uses the +.Sy xattr Ns = Ns Sy sa +property setting and the workload makes heavy use of extended attributes. +This +may be applicable to SELinux-enabled systems, Lustre servers, and Samba +servers, for example. +Literal values are supported for cases where the optimal +size is known in advance and for performance testing. +.Pp +Leave +.Sy dnodesize +set to +.Sy legacy +if you need to receive a send stream of this dataset on a pool that doesn't +enable the +.Sy large_dnode +feature, or if you need to import this pool on a system that doesn't support the +.Sy large_dnode No feature. +.Pp +This property can also be referred to by its shortened column name, +.Sy dnsize . +.It Xo +.Sy encryption Ns = Ns Sy off Ns | Ns Sy on Ns | Ns Sy aes-128-ccm Ns | Ns +.Sy aes-192-ccm Ns | Ns Sy aes-256-ccm Ns | Ns Sy aes-128-gcm Ns | Ns +.Sy aes-192-gcm Ns | Ns Sy aes-256-gcm +.Xc +Controls the encryption cipher suite (block cipher, key length, and mode) used +for this dataset. +Requires the +.Sy encryption +feature to be enabled on the pool. +Requires a +.Sy keyformat +to be set at dataset creation time. +.Pp +Selecting +.Sy encryption Ns = Ns Sy on +when creating a dataset indicates that the default encryption suite will be +selected, which is currently +.Sy aes-256-gcm . +In order to provide consistent data protection, encryption must be specified at +dataset creation time and it cannot be changed afterwards. +.Pp +For more details and caveats about encryption see the +.Sx Encryption +section of +.Xr zfs-load-key 8 . +.It Sy keyformat Ns = Ns Sy raw Ns | Ns Sy hex Ns | Ns Sy passphrase +Controls what format the user's encryption key will be provided as. +This property is only set when the dataset is encrypted. +.Pp +Raw keys and hex keys must be 32 bytes long (regardless of the chosen +encryption suite) and must be randomly generated. +A raw key can be generated with the following command: +.Dl # Nm dd Sy if=/dev/urandom bs=32 count=1 Sy of= Ns Pa /path/to/output/key +.Pp +Passphrases must be between 8 and 512 bytes long and will be processed through +PBKDF2 before being used (see the +.Sy pbkdf2iters +property). +Even though the encryption suite cannot be changed after dataset creation, +the keyformat can be with +.Nm zfs Cm change-key . +.It Xo +.Sy keylocation Ns = Ns Sy prompt Ns | Ns Sy file:// Ns Ar /absolute/file/path Ns | Ns Sy https:// Ns Ar address Ns | Ns Sy http:// Ns Ar address +.Xc +Controls where the user's encryption key will be loaded from by default for +commands such as +.Nm zfs Cm load-key +and +.Nm zfs Cm mount Fl l . +This property is only set for encrypted datasets which are encryption roots. +If unspecified, the default is +.Sy prompt . +.Pp +Even though the encryption suite cannot be changed after dataset creation, the +keylocation can be with either +.Nm zfs Cm set +or +.Nm zfs Cm change-key . +If +.Sy prompt +is selected ZFS will ask for the key at the command prompt when it is required +to access the encrypted data (see +.Nm zfs Cm load-key +for details). +This setting will also allow the key to be passed in via the standard input stream, +but users should be careful not to place keys which should be kept secret on +the command line. +If a file URI is selected, the key will be loaded from the +specified absolute file path. +If an HTTPS or HTTP URL is selected, it will be GETted using +.Xr fetch 3 , +libcurl, or nothing, depending on compile-time configuration and run-time availability. +The +.Sy SSL_CA_CERT_FILE +environment variable can be set to set the location +of the concatenated certificate store. +The +.Sy SSL_CA_CERT_PATH +environment variable can be set to override the location +of the directory containing the certificate authority bundle. +The +.Sy SSL_CLIENT_CERT_FILE +and +.Sy SSL_CLIENT_KEY_FILE +environment variables can be set to configure the path +to the client certificate and its key. +.It Sy pbkdf2iters Ns = Ns Ar iterations +Controls the number of PBKDF2 iterations that a +.Sy passphrase +encryption key should be run through when processing it into an encryption key. +This property is only defined when encryption is enabled and a keyformat of +.Sy passphrase +is selected. +The goal of PBKDF2 is to significantly increase the +computational difficulty needed to brute force a user's passphrase. +This is accomplished by forcing the attacker to run each passphrase through a +computationally expensive hashing function many times before they arrive at the +resulting key. +A user who actually knows the passphrase will only have to pay this cost once. +As CPUs become better at processing, this number should be +raised to ensure that a brute force attack is still not possible. +The current default is +.Sy 350000 +and the minimum is +.Sy 100000 . +This property may be changed with +.Nm zfs Cm change-key . +.It Sy exec Ns = Ns Sy on Ns | Ns Sy off +Controls whether processes can be executed from within this file system. +The default value is +.Sy on . +The values +.Sy on +and +.Sy off +are equivalent to the +.Sy exec +and +.Sy noexec +mount options. +.It Sy filesystem_limit Ns = Ns Ar count Ns | Ns Sy none +Limits the number of filesystems and volumes that can exist under this point in +the dataset tree. +The limit is not enforced if the user is allowed to change the limit. +Setting a +.Sy filesystem_limit +to +.Sy on +a descendent of a filesystem that already has a +.Sy filesystem_limit +does not override the ancestor's +.Sy filesystem_limit , +but rather imposes an additional limit. +This feature must be enabled to be used +.Po see +.Xr zpool-features 7 +.Pc . +.It Sy special_small_blocks Ns = Ns Ar size +This value represents the threshold block size for including small file +blocks into the special allocation class. +Blocks smaller than or equal to this +value will be assigned to the special allocation class while greater blocks +will be assigned to the regular class. +Valid values are zero or a power of two from 512B up to 1M. +The default size is 0 which means no small file blocks +will be allocated in the special class. +.Pp +Before setting this property, a special class vdev must be added to the +pool. +See +.Xr zpoolconcepts 7 +for more details on the special allocation class. +.It Sy mountpoint Ns = Ns Pa path Ns | Ns Sy none Ns | Ns Sy legacy +Controls the mount point used for this file system. +See the +.Sx Mount Points +section of +.Xr zfsconcepts 7 +for more information on how this property is used. +.Pp +When the +.Sy mountpoint +property is changed for a file system, the file system and any children that +inherit the mount point are unmounted. +If the new value is +.Sy legacy , +then they remain unmounted. +Otherwise, they are automatically remounted in the new location if the property +was previously +.Sy legacy +or +.Sy none , +or if they were mounted before the property was changed. +In addition, any shared file systems are unshared and shared in the new +location. +.It Sy nbmand Ns = Ns Sy on Ns | Ns Sy off +Controls whether the file system should be mounted with +.Sy nbmand +.Pq Non-blocking mandatory locks . +This is used for SMB clients. +Changes to this property only take effect when the file system is umounted and +remounted. +Support for these locks is scarce and not described by POSIX. +.It Sy overlay Ns = Ns Sy on Ns | Ns Sy off +Allow mounting on a busy directory or a directory which already contains +files or directories. +This is the default mount behavior for Linux and +.Fx +file systems. +On these platforms the property is +.Sy on +by default. +Set to +.Sy off +to disable overlay mounts for consistency with OpenZFS on other platforms. +.It Sy primarycache Ns = Ns Sy all Ns | Ns Sy none Ns | Ns Sy metadata +Controls what is cached in the primary cache +.Pq ARC . +If this property is set to +.Sy all , +then both user data and metadata is cached. +If this property is set to +.Sy none , +then neither user data nor metadata is cached. +If this property is set to +.Sy metadata , +then only metadata is cached. +The default value is +.Sy all . +.It Sy quota Ns = Ns Ar size Ns | Ns Sy none +Limits the amount of space a dataset and its descendents can consume. +This property enforces a hard limit on the amount of space used. +This includes all space consumed by descendents, including file systems and +snapshots. +Setting a quota on a descendent of a dataset that already has a quota does not +override the ancestor's quota, but rather imposes an additional limit. +.Pp +Quotas cannot be set on volumes, as the +.Sy volsize +property acts as an implicit quota. +.It Sy snapshot_limit Ns = Ns Ar count Ns | Ns Sy none +Limits the number of snapshots that can be created on a dataset and its +descendents. +Setting a +.Sy snapshot_limit +on a descendent of a dataset that already has a +.Sy snapshot_limit +does not override the ancestor's +.Sy snapshot_limit , +but rather imposes an additional limit. +The limit is not enforced if the user is allowed to change the limit. +For example, this means that recursive snapshots taken from the global zone are +counted against each delegated dataset within a zone. +This feature must be enabled to be used +.Po see +.Xr zpool-features 7 +.Pc . +.It Sy userquota@ Ns Ar user Ns = Ns Ar size Ns | Ns Sy none +Limits the amount of space consumed by the specified user. +User space consumption is identified by the +.Sy userspace@ Ns Ar user +property. +.Pp +Enforcement of user quotas may be delayed by several seconds. +This delay means that a user might exceed their quota before the system notices +that they are over quota and begins to refuse additional writes with the +.Er EDQUOT +error message. +See the +.Nm zfs Cm userspace +command for more information. +.Pp +Unprivileged users can only access their own groups' space usage. +The root user, or a user who has been granted the +.Sy userquota +privilege with +.Nm zfs Cm allow , +can get and set everyone's quota. +.Pp +This property is not available on volumes, on file systems before version 4, or +on pools before version 15. +The +.Sy userquota@ Ns Ar ... +properties are not displayed by +.Nm zfs Cm get Sy all . +The user's name must be appended after the +.Sy @ +symbol, using one of the following forms: +.Bl -bullet -compact -offset 4n +.It +POSIX name +.Pq Qq joe +.It +POSIX numeric ID +.Pq Qq 789 +.It +SID name +.Pq Qq joe.smith@mydomain +.It +SID numeric ID +.Pq Qq S-1-123-456-789 +.El +.Pp +Files created on Linux always have POSIX owners. +.It Sy userobjquota@ Ns Ar user Ns = Ns Ar size Ns | Ns Sy none +The +.Sy userobjquota +is similar to +.Sy userquota +but it limits the number of objects a user can create. +Please refer to +.Sy userobjused +for more information about how objects are counted. +.It Sy groupquota@ Ns Ar group Ns = Ns Ar size Ns | Ns Sy none +Limits the amount of space consumed by the specified group. +Group space consumption is identified by the +.Sy groupused@ Ns Ar group +property. +.Pp +Unprivileged users can access only their own groups' space usage. +The root user, or a user who has been granted the +.Sy groupquota +privilege with +.Nm zfs Cm allow , +can get and set all groups' quotas. +.It Sy groupobjquota@ Ns Ar group Ns = Ns Ar size Ns | Ns Sy none +The +.Sy groupobjquota +is similar to +.Sy groupquota +but it limits number of objects a group can consume. +Please refer to +.Sy userobjused +for more information about how objects are counted. +.It Sy projectquota@ Ns Ar project Ns = Ns Ar size Ns | Ns Sy none +Limits the amount of space consumed by the specified project. +Project space consumption is identified by the +.Sy projectused@ Ns Ar project +property. +Please refer to +.Sy projectused +for more information about how project is identified and set/changed. +.Pp +The root user, or a user who has been granted the +.Sy projectquota +privilege with +.Nm zfs allow , +can access all projects' quota. +.It Sy projectobjquota@ Ns Ar project Ns = Ns Ar size Ns | Ns Sy none +The +.Sy projectobjquota +is similar to +.Sy projectquota +but it limits number of objects a project can consume. +Please refer to +.Sy userobjused +for more information about how objects are counted. +.It Sy readonly Ns = Ns Sy on Ns | Ns Sy off +Controls whether this dataset can be modified. +The default value is +.Sy off . +The values +.Sy on +and +.Sy off +are equivalent to the +.Sy ro +and +.Sy rw +mount options. +.Pp +This property can also be referred to by its shortened column name, +.Sy rdonly . +.It Sy recordsize Ns = Ns Ar size +Specifies a suggested block size for files in the file system. +This property is designed solely for use with database workloads that access +files in fixed-size records. +ZFS automatically tunes block sizes according to internal algorithms optimized +for typical access patterns. +.Pp +For databases that create very large files but access them in small random +chunks, these algorithms may be suboptimal. +Specifying a +.Sy recordsize +greater than or equal to the record size of the database can result in +significant performance gains. +Use of this property for general purpose file systems is strongly discouraged, +and may adversely affect performance. +.Pp +The size specified must be a power of two greater than or equal to +.Ar 512B +and less than or equal to +.Ar 128kB . +If the +.Sy large_blocks +feature is enabled on the pool, the size may be up to +.Ar 1MB . +See +.Xr zpool-features 7 +for details on ZFS feature flags. +.Pp +Changing the file system's +.Sy recordsize +affects only files created afterward; existing files are unaffected. +.Pp +This property can also be referred to by its shortened column name, +.Sy recsize . +.It Sy redundant_metadata Ns = Ns Sy all Ns | Ns Sy most +Controls what types of metadata are stored redundantly. +ZFS stores an extra copy of metadata, so that if a single block is corrupted, +the amount of user data lost is limited. +This extra copy is in addition to any redundancy provided at the pool level +.Pq e.g. by mirroring or RAID-Z , +and is in addition to an extra copy specified by the +.Sy copies +property +.Pq up to a total of 3 copies . +For example if the pool is mirrored, +.Sy copies Ns = Ns 2 , +and +.Sy redundant_metadata Ns = Ns Sy most , +then ZFS stores 6 copies of most metadata, and 4 copies of data and some +metadata. +.Pp +When set to +.Sy all , +ZFS stores an extra copy of all metadata. +If a single on-disk block is corrupt, at worst a single block of user data +.Po which is +.Sy recordsize +bytes long +.Pc +can be lost. +.Pp +When set to +.Sy most , +ZFS stores an extra copy of most types of metadata. +This can improve performance of random writes, because less metadata must be +written. +In practice, at worst about 100 blocks +.Po of +.Sy recordsize +bytes each +.Pc +of user data can be lost if a single on-disk block is corrupt. +The exact behavior of which metadata blocks are stored redundantly may change in +future releases. +.Pp +The default value is +.Sy all . +.It Sy refquota Ns = Ns Ar size Ns | Ns Sy none +Limits the amount of space a dataset can consume. +This property enforces a hard limit on the amount of space used. +This hard limit does not include space used by descendents, including file +systems and snapshots. +.It Sy refreservation Ns = Ns Ar size Ns | Ns Sy none Ns | Ns Sy auto +The minimum amount of space guaranteed to a dataset, not including its +descendents. +When the amount of space used is below this value, the dataset is treated as if +it were taking up the amount of space specified by +.Sy refreservation . +The +.Sy refreservation +reservation is accounted for in the parent datasets' space used, and counts +against the parent datasets' quotas and reservations. +.Pp +If +.Sy refreservation +is set, a snapshot is only allowed if there is enough free pool space outside of +this reservation to accommodate the current number of +.Qq referenced +bytes in the dataset. +.Pp +If +.Sy refreservation +is set to +.Sy auto , +a volume is thick provisioned +.Po or +.Qq not sparse +.Pc . +.Sy refreservation Ns = Ns Sy auto +is only supported on volumes. +See +.Sy volsize +in the +.Sx Native Properties +section for more information about sparse volumes. +.Pp +This property can also be referred to by its shortened column name, +.Sy refreserv . +.It Sy relatime Ns = Ns Sy on Ns | Ns Sy off +Controls the manner in which the access time is updated when +.Sy atime Ns = Ns Sy on +is set. +Turning this property on causes the access time to be updated relative +to the modify or change time. +Access time is only updated if the previous +access time was earlier than the current modify or change time or if the +existing access time hasn't been updated within the past 24 hours. +The default value is +.Sy off . +The values +.Sy on +and +.Sy off +are equivalent to the +.Sy relatime +and +.Sy norelatime +mount options. +.It Sy reservation Ns = Ns Ar size Ns | Ns Sy none +The minimum amount of space guaranteed to a dataset and its descendants. +When the amount of space used is below this value, the dataset is treated as if +it were taking up the amount of space specified by its reservation. +Reservations are accounted for in the parent datasets' space used, and count +against the parent datasets' quotas and reservations. +.Pp +This property can also be referred to by its shortened column name, +.Sy reserv . +.It Sy secondarycache Ns = Ns Sy all Ns | Ns Sy none Ns | Ns Sy metadata +Controls what is cached in the secondary cache +.Pq L2ARC . +If this property is set to +.Sy all , +then both user data and metadata is cached. +If this property is set to +.Sy none , +then neither user data nor metadata is cached. +If this property is set to +.Sy metadata , +then only metadata is cached. +The default value is +.Sy all . +.It Sy setuid Ns = Ns Sy on Ns | Ns Sy off +Controls whether the setuid bit is respected for the file system. +The default value is +.Sy on . +The values +.Sy on +and +.Sy off +are equivalent to the +.Sy suid +and +.Sy nosuid +mount options. +.It Sy sharesmb Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Ar opts +Controls whether the file system is shared by using +.Sy Samba USERSHARES +and what options are to be used. +Otherwise, the file system is automatically shared and unshared with the +.Nm zfs Cm share +and +.Nm zfs Cm unshare +commands. +If the property is set to on, the +.Xr net 8 +command is invoked to create a +.Sy USERSHARE . +.Pp +Because SMB shares requires a resource name, a unique resource name is +constructed from the dataset name. +The constructed name is a copy of the +dataset name except that the characters in the dataset name, which would be +invalid in the resource name, are replaced with underscore (_) characters. +Linux does not currently support additional options which might be available +on Solaris. +.Pp +If the +.Sy sharesmb +property is set to +.Sy off , +the file systems are unshared. +.Pp +The share is created with the ACL (Access Control List) "Everyone:F" ("F" +stands for "full permissions", i.e. read and write permissions) and no guest +access (which means Samba must be able to authenticate a real user, system +passwd/shadow, LDAP or smbpasswd based) by default. +This means that any additional access control +(disallow specific user specific access etc) must be done on the underlying file system. +.It Sy sharenfs Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Ar opts +Controls whether the file system is shared via NFS, and what options are to be +used. +A file system with a +.Sy sharenfs +property of +.Sy off +is managed with the +.Xr exportfs 8 +command and entries in the +.Pa /etc/exports +file. +Otherwise, the file system is automatically shared and unshared with the +.Nm zfs Cm share +and +.Nm zfs Cm unshare +commands. +If the property is set to +.Sy on , +the dataset is shared using the default options: +.Dl sec=sys,rw,crossmnt,no_subtree_check +.Pp +See +.Xr exports 5 +for the meaning of the default options. +Otherwise, the +.Xr exportfs 8 +command is invoked with options equivalent to the contents of this property. +.Pp +When the +.Sy sharenfs +property is changed for a dataset, the dataset and any children inheriting the +property are re-shared with the new options, only if the property was previously +.Sy off , +or if they were shared before the property was changed. +If the new property is +.Sy off , +the file systems are unshared. +.It Sy logbias Ns = Ns Sy latency Ns | Ns Sy throughput +Provide a hint to ZFS about handling of synchronous requests in this dataset. +If +.Sy logbias +is set to +.Sy latency +.Pq the default , +ZFS will use pool log devices +.Pq if configured +to handle the requests at low latency. +If +.Sy logbias +is set to +.Sy throughput , +ZFS will not use configured pool log devices. +ZFS will instead optimize synchronous operations for global pool throughput and +efficient use of resources. +.It Sy snapdev Ns = Ns Sy hidden Ns | Ns Sy visible +Controls whether the volume snapshot devices under +.Pa /dev/zvol/ Ns Aq Ar pool +are hidden or visible. +The default value is +.Sy hidden . +.It Sy snapdir Ns = Ns Sy hidden Ns | Ns Sy visible +Controls whether the +.Pa .zfs +directory is hidden or visible in the root of the file system as discussed in +the +.Sx Snapshots +section of +.Xr zfsconcepts 7 . +The default value is +.Sy hidden . +.It Sy sync Ns = Ns Sy standard Ns | Ns Sy always Ns | Ns Sy disabled +Controls the behavior of synchronous requests +.Pq e.g. fsync, O_DSYNC . +.Sy standard +is the POSIX-specified behavior of ensuring all synchronous requests +are written to stable storage and all devices are flushed to ensure +data is not cached by device controllers +.Pq this is the default . +.Sy always +causes every file system transaction to be written and flushed before its +system call returns. +This has a large performance penalty. +.Sy disabled +disables synchronous requests. +File system transactions are only committed to stable storage periodically. +This option will give the highest performance. +However, it is very dangerous as ZFS would be ignoring the synchronous +transaction demands of applications such as databases or NFS. +Administrators should only use this option when the risks are understood. +.It Sy version Ns = Ns Ar N Ns | Ns Sy current +The on-disk version of this file system, which is independent of the pool +version. +This property can only be set to later supported versions. +See the +.Nm zfs Cm upgrade +command. +.It Sy volsize Ns = Ns Ar size +For volumes, specifies the logical size of the volume. +By default, creating a volume establishes a reservation of equal size. +For storage pools with a version number of 9 or higher, a +.Sy refreservation +is set instead. +Any changes to +.Sy volsize +are reflected in an equivalent change to the reservation +.Pq or Sy refreservation . +The +.Sy volsize +can only be set to a multiple of +.Sy volblocksize , +and cannot be zero. +.Pp +The reservation is kept equal to the volume's logical size to prevent unexpected +behavior for consumers. +Without the reservation, the volume could run out of space, resulting in +undefined behavior or data corruption, depending on how the volume is used. +These effects can also occur when the volume size is changed while it is in use +.Pq particularly when shrinking the size . +Extreme care should be used when adjusting the volume size. +.Pp +Though not recommended, a +.Qq sparse volume +.Po also known as +.Qq thin provisioned +.Pc +can be created by specifying the +.Fl s +option to the +.Nm zfs Cm create Fl V +command, or by changing the value of the +.Sy refreservation +property +.Po or +.Sy reservation +property on pool version 8 or earlier +.Pc +after the volume has been created. +A +.Qq sparse volume +is a volume where the value of +.Sy refreservation +is less than the size of the volume plus the space required to store its +metadata. +Consequently, writes to a sparse volume can fail with +.Er ENOSPC +when the pool is low on space. +For a sparse volume, changes to +.Sy volsize +are not reflected in the +.Sy refreservation . +A volume that is not sparse is said to be +.Qq thick provisioned . +A sparse volume can become thick provisioned by setting +.Sy refreservation +to +.Sy auto . +.It Sy volmode Ns = Ns Sy default Ns | Ns Sy full Ns | Ns Sy geom Ns | Ns Sy dev Ns | Ns Sy none +This property specifies how volumes should be exposed to the OS. +Setting it to +.Sy full +exposes volumes as fully fledged block devices, providing maximal +functionality. +The value +.Sy geom +is just an alias for +.Sy full +and is kept for compatibility. +Setting it to +.Sy dev +hides its partitions. +Volumes with property set to +.Sy none +are not exposed outside ZFS, but can be snapshotted, cloned, replicated, etc, +that can be suitable for backup purposes. +Value +.Sy default +means that volumes exposition is controlled by system-wide tunable +.Sy zvol_volmode , +where +.Sy full , +.Sy dev +and +.Sy none +are encoded as 1, 2 and 3 respectively. +The default value is +.Sy full . +.It Sy vscan Ns = Ns Sy on Ns | Ns Sy off +Controls whether regular files should be scanned for viruses when a file is +opened and closed. +In addition to enabling this property, the virus scan service must also be +enabled for virus scanning to occur. +The default value is +.Sy off . +This property is not used by OpenZFS. +.It Sy xattr Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Sy sa +Controls whether extended attributes are enabled for this file system. +Two styles of extended attributes are supported: either directory based +or system attribute based. +.Pp +The default value of +.Sy on +enables directory based extended attributes. +This style of extended attribute imposes no practical limit +on either the size or number of attributes which can be set on a file. +Although under Linux the +.Xr getxattr 2 +and +.Xr setxattr 2 +system calls limit the maximum size to 64K. +This is the most compatible +style of extended attribute and is supported by all ZFS implementations. +.Pp +System attribute based xattrs can be enabled by setting the value to +.Sy sa . +The key advantage of this type of xattr is improved performance. +Storing extended attributes as system attributes +significantly decreases the amount of disk IO required. +Up to 64K of data may be stored per-file in the space reserved for system attributes. +If there is not enough space available for an extended attribute +then it will be automatically written as a directory based xattr. +System attribute based extended attributes are not accessible +on platforms which do not support the +.Sy xattr Ns = Ns Sy sa +feature. +OpenZFS supports +.Sy xattr Ns = Ns Sy sa +on both +.Fx +and Linux. +.Pp +The use of system attribute based xattrs is strongly encouraged for users of +SELinux or POSIX ACLs. +Both of these features heavily rely on extended +attributes and benefit significantly from the reduced access time. +.Pp +The values +.Sy on +and +.Sy off +are equivalent to the +.Sy xattr +and +.Sy noxattr +mount options. +.It Sy jailed Ns = Ns Sy off Ns | Ns Sy on +Controls whether the dataset is managed from a jail. +See +.Xr zfs-jail 8 +for more information. +Jails are a +.Fx +feature and are not relevant on other platforms. +The default value is +.Sy off . +.It Sy zoned Ns = Ns Sy on Ns | Ns Sy off +Controls whether the dataset is managed from a non-global zone. +Zones are a Solaris feature and are not relevant on other platforms. +The default value is +.Sy off . +.El +.Pp +The following three properties cannot be changed after the file system is +created, and therefore, should be set when the file system is created. +If the properties are not set with the +.Nm zfs Cm create +or +.Nm zpool Cm create +commands, these properties are inherited from the parent dataset. +If the parent dataset lacks these properties due to having been created prior to +these features being supported, the new file system will have the default values +for these properties. +.Bl -tag -width "" +.It Xo +.Sy casesensitivity Ns = Ns Sy sensitive Ns | Ns +.Sy insensitive Ns | Ns Sy mixed +.Xc +Indicates whether the file name matching algorithm used by the file system +should be case-sensitive, case-insensitive, or allow a combination of both +styles of matching. +The default value for the +.Sy casesensitivity +property is +.Sy sensitive . +Traditionally, +.Ux +and POSIX file systems have case-sensitive file names. +.Pp +The +.Sy mixed +value for the +.Sy casesensitivity +property indicates that the file system can support requests for both +case-sensitive and case-insensitive matching behavior. +Currently, case-insensitive matching behavior on a file system that supports +mixed behavior is limited to the SMB server product. +For more information about the +.Sy mixed +value behavior, see the "ZFS Administration Guide". +.It Xo +.Sy normalization Ns = Ns Sy none Ns | Ns Sy formC Ns | Ns +.Sy formD Ns | Ns Sy formKC Ns | Ns Sy formKD +.Xc +Indicates whether the file system should perform a +.Sy unicode +normalization of file names whenever two file names are compared, and which +normalization algorithm should be used. +File names are always stored unmodified, names are normalized as part of any +comparison process. +If this property is set to a legal value other than +.Sy none , +and the +.Sy utf8only +property was left unspecified, the +.Sy utf8only +property is automatically set to +.Sy on . +The default value of the +.Sy normalization +property is +.Sy none . +This property cannot be changed after the file system is created. +.It Sy utf8only Ns = Ns Sy on Ns | Ns Sy off +Indicates whether the file system should reject file names that include +characters that are not present in the +.Sy UTF-8 +character code set. +If this property is explicitly set to +.Sy off , +the normalization property must either not be explicitly set or be set to +.Sy none . +The default value for the +.Sy utf8only +property is +.Sy off . +This property cannot be changed after the file system is created. +.El +.Pp +The +.Sy casesensitivity , +.Sy normalization , +and +.Sy utf8only +properties are also new permissions that can be assigned to non-privileged users +by using the ZFS delegated administration feature. +. +.Ss Temporary Mount Point Properties +When a file system is mounted, either through +.Xr mount 8 +for legacy mounts or the +.Nm zfs Cm mount +command for normal file systems, its mount options are set according to its +properties. +The correlation between properties and mount options is as follows: +.Bl -tag -compact -offset Ds -width "rootcontext=" +.It Sy atime +atime/noatime +.It Sy canmount +auto/noauto +.It Sy devices +dev/nodev +.It Sy exec +exec/noexec +.It Sy readonly +ro/rw +.It Sy relatime +relatime/norelatime +.It Sy setuid +suid/nosuid +.It Sy xattr +xattr/noxattr +.It Sy nbmand +mand/nomand +.It Sy context Ns = +context= +.It Sy fscontext Ns = +fscontext= +.It Sy defcontext Ns = +defcontext= +.It Sy rootcontext Ns = +rootcontext= +.El +.Pp +In addition, these options can be set on a per-mount basis using the +.Fl o +option, without affecting the property that is stored on disk. +The values specified on the command line override the values stored in the +dataset. +The +.Sy nosuid +option is an alias for +.Sy nodevices , Ns Sy nosetuid . +These properties are reported as +.Qq temporary +by the +.Nm zfs Cm get +command. +If the properties are changed while the dataset is mounted, the new setting +overrides any temporary settings. +. +.Ss User Properties +In addition to the standard native properties, ZFS supports arbitrary user +properties. +User properties have no effect on ZFS behavior, but applications or +administrators can use them to annotate datasets +.Pq file systems, volumes, and snapshots . +.Pp +User property names must contain a colon +.Pq Qq Sy \&: +character to distinguish them from native properties. +They may contain lowercase letters, numbers, and the following punctuation +characters: colon +.Pq Qq Sy \&: , +dash +.Pq Qq Sy - , +period +.Pq Qq Sy \&. , +and underscore +.Pq Qq Sy _ . +The expected convention is that the property name is divided into two portions +such as +.Ar module : Ns Ar property , +but this namespace is not enforced by ZFS. +User property names can be at most 256 characters, and cannot begin with a dash +.Pq Qq Sy - . +.Pp +When making programmatic use of user properties, it is strongly suggested to use +a reversed DNS domain name for the +.Ar module +component of property names to reduce the chance that two +independently-developed packages use the same property name for different +purposes. +.Pp +The values of user properties are arbitrary strings, are always inherited, and +are never validated. +All of the commands that operate on properties +.Po Nm zfs Cm list , +.Nm zfs Cm get , +.Nm zfs Cm set , +and so forth +.Pc +can be used to manipulate both native properties and user properties. +Use the +.Nm zfs Cm inherit +command to clear a user property. +If the property is not defined in any parent dataset, it is removed entirely. +Property values are limited to 8192 bytes. diff --git a/man/man7/zpool-features.7 b/man/man7/zpool-features.7 new file mode 100644 index 000000000..83ca91175 --- /dev/null +++ b/man/man7/zpool-features.7 @@ -0,0 +1,842 @@ +.\" +.\" Copyright (c) 2012, 2018 by Delphix. All rights reserved. +.\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved. +.\" Copyright (c) 2014, Joyent, Inc. All rights reserved. +.\" The contents of this file are subject to the terms of the Common Development +.\" and Distribution License (the "License"). You may not use this file except +.\" in compliance with the License. You can obtain a copy of the license at +.\" usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensing. +.\" +.\" See the License for the specific language governing permissions and +.\" limitations under the License. When distributing Covered Code, include this +.\" CDDL HEADER in each file and include the License file at +.\" usr/src/OPENSOLARIS.LICENSE. If applicable, add the following below this +.\" CDDL HEADER, with the fields enclosed by brackets "[]" replaced with your +.\" own identifying information: +.\" Portions Copyright [yyyy] [name of copyright owner] +.\" Copyright (c) 2019, Klara Inc. +.\" Copyright (c) 2019, Allan Jude +.\" Copyright (c) 2021, Colm Buckley <[email protected]> +.\" +.Dd May 31, 2021 +.Dt ZPOOL-FEATURES 7 +.Os +. +.Sh NAME +.Nm zpool-features +.Nd description of ZFS pool features +. +.Sh DESCRIPTION +ZFS pool on-disk format versions are specified via "features" which replace +the old on-disk format numbers (the last supported on-disk format number is 28). +To enable a feature on a pool use the +.Nm zpool Cm upgrade , +or set the +.Sy feature Ns @ Ns Ar feature-name +property to +.Sy enabled . +Please also see the +.Sx Compatibility feature sets +section for information on how sets of features may be enabled together. +.Pp +The pool format does not affect file system version compatibility or the ability +to send file systems between pools. +.Pp +Since most features can be enabled independently of each other, the on-disk +format of the pool is specified by the set of all features marked as +.Sy active +on the pool. +If the pool was created by another software version +this set may include unsupported features. +. +.Ss Identifying features +Every feature has a GUID of the form +.Ar com.example : Ns Ar feature-name . +The reversed DNS name ensures that the feature's GUID is unique across all ZFS +implementations. +When unsupported features are encountered on a pool they will +be identified by their GUIDs. +Refer to the documentation for the ZFS +implementation that created the pool for information about those features. +.Pp +Each supported feature also has a short name. +By convention a feature's short name is the portion of its GUID which follows the +.Sq \&: +(i.e. +.Ar com.example : Ns Ar feature-name +would have the short name +.Ar feature-name ) , +however a feature's short name may differ across ZFS implementations if +following the convention would result in name conflicts. +. +.Ss Feature states +Features can be in one of three states: +.Bl -tag -width "disabled" +.It Sy active +This feature's on-disk format changes are in effect on the pool. +Support for this feature is required to import the pool in read-write mode. +If this feature is not read-only compatible, +support is also required to import the pool in read-only mode +.Pq see Sx Read-only compatibility . +.It Sy enabled +An administrator has marked this feature as enabled on the pool, but the +feature's on-disk format changes have not been made yet. +The pool can still be imported by software that does not support this feature, +but changes may be made to the on-disk format at any time +which will move the feature to the +.Sy active +state. +Some features may support returning to the +.Sy enabled +state after becoming +.Sy active . +See feature-specific documentation for details. +.It Sy disabled +This feature's on-disk format changes have not been made and will not be made +unless an administrator moves the feature to the +.Sy enabled +state. +Features cannot be disabled once they have been enabled. +.El +.Pp +The state of supported features is exposed through pool properties of the form +.Sy feature Ns @ Ns Ar short-name . +. +.Ss Read-only compatibility +Some features may make on-disk format changes that do not interfere with other +software's ability to read from the pool. +These features are referred to as +.Dq read-only compatible . +If all unsupported features on a pool are read-only compatible, +the pool can be imported in read-only mode by setting the +.Sy readonly +property during import (see +.Xr zpool-import 8 +for details on importing pools). +. +.Ss Unsupported features +For each unsupported feature enabled on an imported pool, a pool property +named +.Sy unsupported Ns @ Ns Ar feature-name +will indicate why the import was allowed despite the unsupported feature. +Possible values for this property are: +.Bl -tag -width "readonly" +.It Sy inactive +The feature is in the +.Sy enabled +state and therefore the pool's on-disk +format is still compatible with software that does not support this feature. +.It Sy readonly +The feature is read-only compatible and the pool has been imported in +read-only mode. +.El +. +.Ss Feature dependencies +Some features depend on other features being enabled in order to function. +Enabling a feature will automatically enable any features it depends on. +. +.Ss Compatibility feature sets +It is sometimes necessary for a pool to maintain compatibility with a +specific on-disk format, by enabling and disabling particular features. +The +.Sy compatibility +feature facilitates this by allowing feature sets to be read from text files. +When set to +.Sy off +(the default), compatibility feature sets are disabled +(i.e. all features are enabled); when set to +.Sy legacy , +no features are enabled. +When set to a comma-separated list of filenames +(each filename may either be an absolute path, or relative to +.Pa /etc/zfs/compatibility.d +or +.Pa /usr/share/zfs/compatibility.d ) , +the lists of requested features are read from those files, +separated by whitespace and/or commas. +Only features present in all files are enabled. +.Pp +Simple sanity checks are applied to the files: +they must be between 1B and 16kB in size, and must end with a newline character. +.Pp +The requested features are applied when a pool is created using +.Nm zpool Cm create Fl o Sy compatibility Ns = Ns Ar … +and controls which features are enabled when using +.Nm zpool Cm upgrade . +.Nm zpool Cm status +will not show a warning about disabled features which are not part +of the requested feature set. +.Pp +The special value +.Sy legacy +prevents any features from being enabled, either via +.Nm zpool Cm upgrade +or +.Nm zpool Cm set Sy feature Ns @ Ns Ar feature-name Ns = Ns Sy enabled . +This setting also prevents pools from being upgraded to newer on-disk versions. +This is a safety measure to prevent new features from being +accidentally enabled, breaking compatibility. +.Pp +By convention, compatibility files in +.Pa /usr/share/zfs/compatibility.d +are provided by the distribution, and include feature sets +supported by important versions of popular distributions, and feature +sets commonly supported at the start of each year. +Compatibility files in +.Pa /etc/zfs/compatibility.d , +if present, will take precedence over files with the same name in +.Pa /usr/share/zfs/compatibility.d . +.Pp +If an unrecognized feature is found in these files, an error message will +be shown. +If the unrecognized feature is in a file in +.Pa /etc/zfs/compatibility.d , +this is treated as an error and processing will stop. +If the unrecognized feature is under +.Pa /usr/share/zfs/compatibility.d , +this is treated as a warning and processing will continue. +This difference is to allow distributions to include features +which might not be recognized by the currently-installed binaries. +.Pp +Compatibility files may include comments: +any text from +.Sq # +to the end of the line is ignored. +.Pp +.Sy Example : +.Bd -literal -compact -offset 4n +.No example# Nm cat Pa /usr/share/zfs/compatibility.d/grub2 +# Features which are supported by GRUB2 +async_destroy +bookmarks +embedded_data +empty_bpobj +enabled_txg +extensible_dataset +filesystem_limits +hole_birth +large_blocks +lz4_compress +spacemap_histogram + +.No example# Nm zpool Cm create Fl o Sy compatibility Ns = Ns Ar grub2 Ar bootpool Ar vdev +.Ed +.Pp +See +.Xr zpool-create 8 +and +.Xr zpool-upgrade 8 +for more information on how these commands are affected by feature sets. +. +.de feature +.It Sy \\$2 +.Bl -tag -compact -width "READ-ONLY COMPATIBLE" +.It GUID +.Sy \\$1:\\$2 +.if !"\\$4"" \{\ +.It DEPENDENCIES +\fB\\$4\fP\c +.if !"\\$5"" , \fB\\$5\fP\c +.if !"\\$6"" , \fB\\$6\fP\c +.if !"\\$7"" , \fB\\$7\fP\c +.if !"\\$8"" , \fB\\$8\fP\c +.if !"\\$9"" , \fB\\$9\fP\c +.\} +.It READ-ONLY COMPATIBLE +\\$3 +.El +.Pp +.. +. +.ds instant-never \ +.No This feature becomes Sy active No as soon as it is enabled \ +and will never return to being Sy enabled . +. +.ds remount-upgrade \ +.No Each filesystem will be upgraded automatically when remounted, \ +or when a new file is created under that filesystem. \ +The upgrade can also be triggered on filesystems via \ +Nm zfs Cm set Sy version Ns = Ns Sy current Ar fs . \ +No The upgrade process runs in the background and may take a while to complete \ +for filesystems containing large amounts of files. +. +.de checksum-spiel +When the +.Sy \\$1 +feature is set to +.Sy enabled , +the administrator can turn on the +.Sy \\$1 +checksum on any dataset using +.Nm zfs Cm set Sy checksum Ns = Ns Sy \\$1 Ar dset +.Po see Xr zfs-set 8 Pc . +This feature becomes +.Sy active +once a +.Sy checksum +property has been set to +.Sy \\$1 , +and will return to being +.Sy enabled +once all filesystems that have ever had their checksum set to +.Sy \\$1 +are destroyed. +.. +. +.Sh FEATURES +The following features are supported on this system: +.Bl -tag -width Ds +.feature org.zfsonlinux allocation_classes yes +This feature enables support for separate allocation classes. +.Pp +This feature becomes +.Sy active +when a dedicated allocation class vdev (dedup or special) is created with the +.Nm zpool Cm create No or Nm zpool Cm add No commands . +With device removal, it can be returned to the +.Sy enabled +state if all the dedicated allocation class vdevs are removed. +. +.feature com.delphix async_destroy yes +Destroying a file system requires traversing all of its data in order to +return its used space to the pool. +Without +.Sy async_destroy , +the file system is not fully removed until all space has been reclaimed. +If the destroy operation is interrupted by a reboot or power outage, +the next attempt to open the pool will need to complete the destroy +operation synchronously. +.Pp +When +.Sy async_destroy +is enabled, the file system's data will be reclaimed by a background process, +allowing the destroy operation to complete +without traversing the entire file system. +The background process is able to resume +interrupted destroys after the pool has been opened, eliminating the need +to finish interrupted destroys as part of the open operation. +The amount of space remaining to be reclaimed by the background process +is available through the +.Sy freeing +property. +.Pp +This feature is only +.Sy active +while +.Sy freeing +is non-zero. +. +.feature com.delphix bookmarks yes extensible_dataset +This feature enables use of the +.Nm zfs Cm bookmark +command. +.Pp +This feature is +.Sy active +while any bookmarks exist in the pool. +All bookmarks in the pool can be listed by running +.Nm zfs Cm list Fl t Sy bookmark Fl r Ar poolname . +. +.feature com.datto bookmark_v2 no bookmark extensible_dataset +This feature enables the creation and management of larger bookmarks which are +needed for other features in ZFS. +.Pp +This feature becomes +.Sy active +when a v2 bookmark is created and will be returned to the +.Sy enabled +state when all v2 bookmarks are destroyed. +. +.feature com.delphix bookmark_written no bookmark extensible_dataset bookmark_v2 +This feature enables additional bookmark accounting fields, enabling the +.Sy written Ns # Ns Ar bookmark +property (space written since a bookmark) and estimates of +send stream sizes for incrementals from bookmarks. +.Pp +This feature becomes +.Sy active +when a bookmark is created and will be +returned to the +.Sy enabled +state when all bookmarks with these fields are destroyed. +. +.feature org.openzfs device_rebuild yes +This feature enables the ability for the +.Nm zpool Cm attach +and +.Nm zpool Cm replace +commands to perform sequential reconstruction +(instead of healing reconstruction) when resilvering. +.Pp +Sequential reconstruction resilvers a device in LBA order without immediately +verifying the checksums. +Once complete, a scrub is started, which then verifies the checksums. +This approach allows full redundancy to be restored to the pool +in the minimum amount of time. +This two-phase approach will take longer than a healing resilver +when the time to verify the checksums is included. +However, unless there is additional pool damage, +no checksum errors should be reported by the scrub. +This feature is incompatible with raidz configurations. +. +This feature becomes +.Sy active +while a sequential resilver is in progress, and returns to +.Sy enabled +when the resilver completes. +. +.feature com.delphix device_removal no +This feature enables the +.Nm zpool Cm remove +command to remove top-level vdevs, +evacuating them to reduce the total size of the pool. +.Pp +This feature becomes +.Sy active +when the +.Nm zpool Cm remove +command is used +on a top-level vdev, and will never return to being +.Sy enabled . +. +.feature org.openzfs draid no +This feature enables use of the +.Sy draid +vdev type. +dRAID is a variant of raidz which provides integrated distributed +hot spares that allow faster resilvering while retaining the benefits of raidz. +Data, parity, and spare space are organized in redundancy groups +and distributed evenly over all of the devices. +.Pp +This feature becomes +.Sy active +when creating a pool which uses the +.Sy draid +vdev type, or when adding a new +.Sy draid +vdev to an existing pool. +. +.feature org.illumos edonr no extensible_dataset +This feature enables the use of the Edon-R hash algorithm for checksum, +including for nopwrite (if compression is also enabled, an overwrite of +a block whose checksum matches the data being written will be ignored). +In an abundance of caution, Edon-R requires verification when used with +dedup: +.Nm zfs Cm set Sy dedup Ns = Ns Sy edonr , Ns Sy verify +.Po see Xr zfs-set 8 Pc . +.Pp +Edon-R is a very high-performance hash algorithm that was part +of the NIST SHA-3 competition. +It provides extremely high hash performance (over 350% faster than SHA-256), +but was not selected because of its unsuitability +as a general purpose secure hash algorithm. +This implementation utilizes the new salted checksumming functionality +in ZFS, which means that the checksum is pre-seeded with a secret +256-bit random key (stored on the pool) before being fed the data block +to be checksummed. +Thus the produced checksums are unique to a given pool, +preventing hash collision attacks on systems with dedup. +.Pp +.checksum-spiel edonr +.Pp +.Fx does not support the Sy edonr No feature. +. +.feature com.delphix embedded_data no +This feature improves the performance and compression ratio of +highly-compressible blocks. +Blocks whose contents can compress to 112 bytes +or smaller can take advantage of this feature. +.Pp +When this feature is enabled, the contents of highly-compressible blocks are +stored in the block "pointer" itself (a misnomer in this case, as it contains +the compressed data, rather than a pointer to its location on disk). +Thus the space of the block (one sector, typically 512B or 4kB) is saved, +and no additional I/O is needed to read and write the data block. +. +\*[instant-never] +. +.feature com.delphix empty_bpobj yes +This feature increases the performance of creating and using a large +number of snapshots of a single filesystem or volume, and also reduces +the disk space required. +.Pp +When there are many snapshots, each snapshot uses many Block Pointer +Objects (bpobjs) to track blocks associated with that snapshot. +However, in common use cases, most of these bpobjs are empty. +This feature allows us to create each bpobj on-demand, +thus eliminating the empty bpobjs. +.Pp +This feature is +.Sy active +while there are any filesystems, volumes, +or snapshots which were created after enabling this feature. +. +.feature com.delphix enabled_txg yes +Once this feature is enabled, ZFS records the transaction group number +in which new features are enabled. +This has no user-visible impact, but other features may depend on this feature. +.Pp +This feature becomes +.Sy active + as soon as it is enabled and will +never return to being +.Sy enabled . +. +.feature com.datto encryption no bookmark_v2 extensible_dataset +This feature enables the creation and management of natively encrypted datasets. +.Pp +This feature becomes +.Sy active +when an encrypted dataset is created and will be returned to the +.Sy enabled +state when all datasets that use this feature are destroyed. +. +.feature com.delphix extensible_dataset no +This feature allows more flexible use of internal ZFS data structures, +and exists for other features to depend on. +.Pp +This feature will be +.Sy active +when the first dependent feature uses it, and will be returned to the +.Sy enabled +state when all datasets that use this feature are destroyed. +. +.feature com.joyent filesystem_limits yes extensible_dataset +This feature enables filesystem and snapshot limits. +These limits can be used to control how many filesystems and/or snapshots +can be created at the point in the tree on which the limits are set. +.Pp +This feature is +.Sy active +once either of the limit properties has been set on a dataset. +Once activated the feature is never deactivated. +. +.feature com.delphix hole_birth no enabled_txg +This feature has/had bugs, the result of which is that, if you do a +.Nm zfs Cm send Fl i +.Pq or Fl R , No since it uses Fl i +from an affected dataset, the receiving party will not see any checksum +or other errors, but the resulting destination snapshot +will not match the source. +Its use by +.Nm zfs Cm send Fl i +has been disabled by default +.Pq see Sy send_holes_without_birth_time No in Xr zfs 4 . +.Pp +This feature improves performance of incremental sends +.Pq Nm zfs Cm send Fl i +and receives for objects with many holes. +The most common case of hole-filled objects is zvols. +.Pp +An incremental send stream from snapshot +.Sy A No to snapshot Sy B +contains information about every block that changed between +.Sy A No and Sy B . +Blocks which did not change between those snapshots can be +identified and omitted from the stream using a piece of metadata called +the "block birth time", but birth times are not recorded for holes +(blocks filled only with zeroes). +Since holes created after +.Sy A No cannot be distinguished from holes created before Sy A , +information about every hole in the entire filesystem or zvol +is included in the send stream. +.Pp +For workloads where holes are rare this is not a problem. +However, when incrementally replicating filesystems or zvols with many holes +(for example a zvol formatted with another filesystem) a lot of time will +be spent sending and receiving unnecessary information about holes that +already exist on the receiving side. +.Pp +Once the +.Sy hole_birth +feature has been enabled the block birth times +of all new holes will be recorded. +Incremental sends between snapshots created after this feature is enabled +will use this new metadata to avoid sending information about holes that +already exist on the receiving side. +.Pp +\*[instant-never] +. +.feature org.open-zfs large_blocks no extensible_dataset +This feature allows the record size on a dataset to be set larger than 128kB. +.Pp +This feature becomes +.Sy active +once a dataset contains a file with a block size larger than 128kB, +and will return to being +.Sy enabled +once all filesystems that have ever had their recordsize larger than 128kB +are destroyed. +. +.feature org.zfsonlinux large_dnode no extensible_dataset +This feature allows the size of dnodes in a dataset to be set larger than 512B. +. +This feature becomes +.Sy active +once a dataset contains an object with a dnode larger than 512B, +which occurs as a result of setting the +.Sy dnodesize +dataset property to a value other than +.Sy legacy . +The feature will return to being +.Sy enabled +once all filesystems that have ever contained a dnode larger than 512B +are destroyed. +Large dnodes allow more data to be stored in the bonus buffer, +thus potentially improving performance by avoiding the use of spill blocks. +. +.feature com.delphix livelist yes +This feature allows clones to be deleted faster than the traditional method +when a large number of random/sparse writes have been made to the clone. +All blocks allocated and freed after a clone is created are tracked by the +the clone's livelist which is referenced during the deletion of the clone. +The feature is activated when a clone is created and remains +.Sy active +until all clones have been destroyed. +. +.feature com.delphix log_spacemap yes com.delphix:spacemap_v2 +This feature improves performance for heavily-fragmented pools, +especially when workloads are heavy in random-writes. +It does so by logging all the metaslab changes on a single spacemap every TXG +instead of scattering multiple writes to all the metaslab spacemaps. +.Pp +\*[instant-never] +. +.feature org.illumos lz4_compress no +.Sy lz4 +is a high-performance real-time compression algorithm that +features significantly faster compression and decompression as well as a +higher compression ratio than the older +.Sy lzjb +compression. +Typically, +.Sy lz4 +compression is approximately 50% faster on compressible data and 200% faster +on incompressible data than +.Sy lzjb . +It is also approximately 80% faster on decompression, +while giving approximately a 10% better compression ratio. +.Pp +When the +.Sy lz4_compress +feature is set to +.Sy enabled , +the administrator can turn on +.Sy lz4 +compression on any dataset on the pool using the +.Xr zfs-set 8 +command. +All newly written metadata will be compressed with the +.Sy lz4 +algorithm. +.Pp +\*[instant-never] +. +.feature com.joyent multi_vdev_crash_dump no +This feature allows a dump device to be configured with a pool comprised +of multiple vdevs. +Those vdevs may be arranged in any mirrored or raidz configuration. +.Pp +When the +.Sy multi_vdev_crash_dump +feature is set to +.Sy enabled , +the administrator can use +.Xr dumpadm 1M +to configure a dump device on a pool comprised of multiple vdevs. +.Pp +Under +.Fx +and Linux this feature is unused, but registered for compatibility. +New pools created on these systems will have the feature +.Sy enabled +but will never transition to +.Sy active , +as this functionality is not required for crash dump support. +Existing pools where this feature is +.Sy active +can be imported. +. +.feature com.delphix obsolete_counts yes device_removal +This feature is an enhancement of +.Sy device_removal , +which will over time reduce the memory used to track removed devices. +When indirect blocks are freed or remapped, +we note that their part of the indirect mapping is "obsolete" – no longer needed. +.Pp +This feature becomes +.Sy active +when the +.Nm zpool Cm remove +command is used on a top-level vdev, and will never return to being +.Sy enabled . +. +.feature org.zfsonlinux project_quota yes extensible_dataset +This feature allows administrators to account the spaces and objects usage +information against the project identifier (ID). +.Pp +The project ID is an object-based attribute. +When upgrading an existing filesystem, +objects without a project ID will be assigned a zero project ID. +When this feature is enabled, newly created objects inherit +their parent directories' project ID if the parent's inherit flag is set +.Pq via Nm chattr Sy [+-]P No or Nm zfs Cm project Fl s Ns | Ns Fl C . +Otherwise, the new object's project ID will be zero. +An object's project ID can be changed at any time by the owner +(or privileged user) via +.Nm chattr Fl p Ar prjid +or +.Nm zfs Cm project Fl p Ar prjid . +.Pp +This feature will become +.Sy active +as soon as it is enabled and will never return to being +.Sy disabled . +\*[remount-upgrade] +. +.feature com.delphix redaction_bookmarks no bookmarks extensible_dataset +This feature enables the use of redacted +.Nm zfs Cm send Ns s , +which create redaction bookmarks storing the list of blocks +redacted by the send that created them. +For more information about redacted sends, see +.Xr zfs-send 8 . +. +.feature com.delphix redacted_datasets no extensible_dataset +This feature enables the receiving of redacted +.Nm zfs Cm send Ns +streams. which create redacted datasets when received. +These datasets are missing some of their blocks, +and so cannot be safely mounted, and their contents cannot be safely read. +For more information about redacted receives, see +.Xr zfs-send 8 . +. +.feature com.datto resilver_defer yes +This feature allows ZFS to postpone new resilvers if an existing one is already +in progress. +Without this feature, any new resilvers will cause the currently +running one to be immediately restarted from the beginning. +.Pp +This feature becomes +.Sy active +once a resilver has been deferred, and returns to being +.Sy enabled +when the deferred resilver begins. +. +.feature org.illumos sha512 no extensible_dataset +This feature enables the use of the SHA-512/256 truncated hash algorithm +(FIPS 180-4) for checksum and dedup. +The native 64-bit arithmetic of SHA-512 provides an approximate 50% +performance boost over SHA-256 on 64-bit hardware +and is thus a good minimum-change replacement candidate +for systems where hash performance is important, +but these systems cannot for whatever reason utilize the faster +.Sy skein No and Sy edonr +algorithms. +.Pp +.checksum-spiel sha512 +. +.feature org.illumos skein no extensible_dataset +This feature enables the use of the Skein hash algorithm for checksum and dedup. +Skein is a high-performance secure hash algorithm that was a +finalist in the NIST SHA-3 competition. +It provides a very high security margin and high performance on 64-bit hardware +(80% faster than SHA-256). +This implementation also utilizes the new salted checksumming +functionality in ZFS, which means that the checksum is pre-seeded with a +secret 256-bit random key (stored on the pool) before being fed the data +block to be checksummed. +Thus the produced checksums are unique to a given pool, +preventing hash collision attacks on systems with dedup. +.Pp +.checksum-spiel skein +. +.feature com.delphix spacemap_histogram yes +This features allows ZFS to maintain more information about how free space +is organized within the pool. +If this feature is +.Sy enabled , +it will be activated when a new space map object is created, or +an existing space map is upgraded to the new format, +and never returns back to being +.Sy enabled . +. +.feature com.delphix spacemap_v2 yes +This feature enables the use of the new space map encoding which +consists of two words (instead of one) whenever it is advantageous. +The new encoding allows space maps to represent large regions of +space more efficiently on-disk while also increasing their maximum +addressable offset. +.Pp +This feature becomes +.Sy active +once it is +.Sy enabled , +and never returns back to being +.Sy enabled . +. +.feature org.zfsonlinux userobj_accounting yes extensible_dataset +This feature allows administrators to account the object usage information +by user and group. +.Pp +\*[instant-never] +\*[remount-upgrade] +. +.feature com.delphix zpool_checkpoint yes +This feature enables the +.Nm zpool Cm checkpoint +command that can checkpoint the state of the pool +at the time it was issued and later rewind back to it or discard it. +.Pp +This feature becomes +.Sy active +when the +.Nm zpool Cm checkpoint +command is used to checkpoint the pool. +The feature will only return back to being +.Sy enabled +when the pool is rewound or the checkpoint has been discarded. +. +.feature org.freebsd zstd_compress no extensible_dataset +.Sy zstd +is a high-performance compression algorithm that features a +combination of high compression ratios and high speed. +Compared to +.Sy gzip , +.Sy zstd +offers slightly better compression at much higher speeds. +Compared to +.Sy lz4 , +.Sy zstd +offers much better compression while being only modestly slower. +Typically, +.Sy zstd +compression speed ranges from 250 to 500 MB/s per thread +and decompression speed is over 1 GB/s per thread. +.Pp +When the +.Sy zstd +feature is set to +.Sy enabled , +the administrator can turn on +.Sy zstd +compression of any dataset using +.Nm zfs Cm set Sy compress Ns = Ns Sy zstd Ar dset +.Po see Xr zfs-set 8 Pc . +This feature becomes +.Sy active +once a +.Sy compress +property has been set to +.Sy zstd , +and will return to being +.Sy enabled +once all filesystems that have ever had their +.Sy compress +property set to +.Sy zstd +are destroyed. +.El +. +.Sh SEE ALSO +.Xr zpool 8 diff --git a/man/man7/zpoolconcepts.7 b/man/man7/zpoolconcepts.7 new file mode 100644 index 000000000..58132baf5 --- /dev/null +++ b/man/man7/zpoolconcepts.7 @@ -0,0 +1,512 @@ +.\" +.\" CDDL HEADER START +.\" +.\" The contents of this file are subject to the terms of the +.\" Common Development and Distribution License (the "License"). +.\" You may not use this file except in compliance with the License. +.\" +.\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +.\" or http://www.opensolaris.org/os/licensing. +.\" See the License for the specific language governing permissions +.\" and limitations under the License. +.\" +.\" When distributing Covered Code, include this CDDL HEADER in each +.\" file and include the License file at usr/src/OPENSOLARIS.LICENSE. +.\" If applicable, add the following below this CDDL HEADER, with the +.\" fields enclosed by brackets "[]" replaced with your own identifying +.\" information: Portions Copyright [yyyy] [name of copyright owner] +.\" +.\" CDDL HEADER END +.\" +.\" Copyright (c) 2007, Sun Microsystems, Inc. All Rights Reserved. +.\" Copyright (c) 2012, 2018 by Delphix. All rights reserved. +.\" Copyright (c) 2012 Cyril Plisko. All Rights Reserved. +.\" Copyright (c) 2017 Datto Inc. +.\" Copyright (c) 2018 George Melikov. All Rights Reserved. +.\" Copyright 2017 Nexenta Systems, Inc. +.\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. +.\" +.Dd June 2, 2021 +.Dt ZPOOLCONCEPTS 7 +.Os +. +.Sh NAME +.Nm zpoolconcepts +.Nd overview of ZFS storage pools +. +.Sh DESCRIPTION +.Ss Virtual Devices (vdevs) +A "virtual device" describes a single device or a collection of devices +organized according to certain performance and fault characteristics. +The following virtual devices are supported: +.Bl -tag -width "special" +.It Sy disk +A block device, typically located under +.Pa /dev . +ZFS can use individual slices or partitions, though the recommended mode of +operation is to use whole disks. +A disk can be specified by a full path, or it can be a shorthand name +.Po the relative portion of the path under +.Pa /dev +.Pc . +A whole disk can be specified by omitting the slice or partition designation. +For example, +.Pa sda +is equivalent to +.Pa /dev/sda . +When given a whole disk, ZFS automatically labels the disk, if necessary. +.It Sy file +A regular file. +The use of files as a backing store is strongly discouraged. +It is designed primarily for experimental purposes, as the fault tolerance of a +file is only as good as the file system on which it resides. +A file must be specified by a full path. +.It Sy mirror +A mirror of two or more devices. +Data is replicated in an identical fashion across all components of a mirror. +A mirror with +.Em N No disks of size Em X No can hold Em X No bytes and can withstand Em N-1 +devices failing without losing data. +.It Sy raidz , raidz1 , raidz2 , raidz3 +A variation on RAID-5 that allows for better distribution of parity and +eliminates the RAID-5 +.Qq write hole +.Pq in which data and parity become inconsistent after a power loss . +Data and parity is striped across all disks within a raidz group. +.Pp +A raidz group can have single, double, or triple parity, meaning that the +raidz group can sustain one, two, or three failures, respectively, without +losing any data. +The +.Sy raidz1 +vdev type specifies a single-parity raidz group; the +.Sy raidz2 +vdev type specifies a double-parity raidz group; and the +.Sy raidz3 +vdev type specifies a triple-parity raidz group. +The +.Sy raidz +vdev type is an alias for +.Sy raidz1 . +.Pp +A raidz group with +.Em N No disks of size Em X No with Em P No parity disks can hold approximately +.Em (N-P)*X No bytes and can withstand Em P No devices failing without losing data. +The minimum number of devices in a raidz group is one more than the number of +parity disks. +The recommended number is between 3 and 9 to help increase performance. +.It Sy draid , draid1 , draid2 , draid3 +A variant of raidz that provides integrated distributed hot spares which +allows for faster resilvering while retaining the benefits of raidz. +A dRAID vdev is constructed from multiple internal raidz groups, each with +.Em D No data devices and Em P No parity devices. +These groups are distributed over all of the children in order to fully +utilize the available disk performance. +.Pp +Unlike raidz, dRAID uses a fixed stripe width (padding as necessary with +zeros) to allow fully sequential resilvering. +This fixed stripe width significantly effects both usable capacity and IOPS. +For example, with the default +.Em D=8 No and Em 4kB No disk sectors the minimum allocation size is Em 32kB . +If using compression, this relatively large allocation size can reduce the +effective compression ratio. +When using ZFS volumes and dRAID, the default of the +.Sy volblocksize +property is increased to account for the allocation size. +If a dRAID pool will hold a significant amount of small blocks, it is +recommended to also add a mirrored +.Sy special +vdev to store those blocks. +.Pp +In regards to I/O, performance is similar to raidz since for any read all +.Em D No data disks must be accessed. +Delivered random IOPS can be reasonably approximated as +.Sy floor((N-S)/(D+P))*single_drive_IOPS . +.Pp +Like raidzm a dRAID can have single-, double-, or triple-parity. +The +.Sy draid1 , +.Sy draid2 , +and +.Sy draid3 +types can be used to specify the parity level. +The +.Sy draid +vdev type is an alias for +.Sy draid1 . +.Pp +A dRAID with +.Em N No disks of size Em X , D No data disks per redundancy group, Em P +.No parity level, and Em S No distributed hot spares can hold approximately +.Em (N-S)*(D/(D+P))*X No bytes and can withstand Em P +devices failing without losing data. +.It Sy draid Ns Oo Ar parity Oc Ns Oo Sy \&: Ns Ar data Ns Sy d Oc Ns Oo Sy \&: Ns Ar children Ns Sy c Oc Ns Oo Sy \&: Ns Ar spares Ns Sy s Oc +A non-default dRAID configuration can be specified by appending one or more +of the following optional arguments to the +.Sy draid +keyword: +.Bl -tag -compact -width "children" +.It Ar parity +The parity level (1-3). +.It Ar data +The number of data devices per redundancy group. +In general, a smaller value of +.Em D No will increase IOPS, improve the compression ratio, +and speed up resilvering at the expense of total usable capacity. +Defaults to +.Em 8 , No unless Em N-P-S No is less than Em 8 . +.It Ar children +The expected number of children. +Useful as a cross-check when listing a large number of devices. +An error is returned when the provided number of children differs. +.It Ar spares +The number of distributed hot spares. +Defaults to zero. +.El +.It Sy spare +A pseudo-vdev which keeps track of available hot spares for a pool. +For more information, see the +.Sx Hot Spares +section. +.It Sy log +A separate intent log device. +If more than one log device is specified, then writes are load-balanced between +devices. +Log devices can be mirrored. +However, raidz vdev types are not supported for the intent log. +For more information, see the +.Sx Intent Log +section. +.It Sy dedup +A device dedicated solely for deduplication tables. +The redundancy of this device should match the redundancy of the other normal +devices in the pool. +If more than one dedup device is specified, then +allocations are load-balanced between those devices. +.It Sy special +A device dedicated solely for allocating various kinds of internal metadata, +and optionally small file blocks. +The redundancy of this device should match the redundancy of the other normal +devices in the pool. +If more than one special device is specified, then +allocations are load-balanced between those devices. +.Pp +For more information on special allocations, see the +.Sx Special Allocation Class +section. +.It Sy cache +A device used to cache storage pool data. +A cache device cannot be configured as a mirror or raidz group. +For more information, see the +.Sx Cache Devices +section. +.El +.Pp +Virtual devices cannot be nested, so a mirror or raidz virtual device can only +contain files or disks. +Mirrors of mirrors +.Pq or other combinations +are not allowed. +.Pp +A pool can have any number of virtual devices at the top of the configuration +.Po known as +.Qq root vdevs +.Pc . +Data is dynamically distributed across all top-level devices to balance data +among devices. +As new virtual devices are added, ZFS automatically places data on the newly +available devices. +.Pp +Virtual devices are specified one at a time on the command line, +separated by whitespace. +Keywords like +.Sy mirror No and Sy raidz +are used to distinguish where a group ends and another begins. +For example, the following creates a pool with two root vdevs, +each a mirror of two disks: +.Dl # Nm zpool Cm create Ar mypool Sy mirror Ar sda sdb Sy mirror Ar sdc sdd +. +.Ss Device Failure and Recovery +ZFS supports a rich set of mechanisms for handling device failure and data +corruption. +All metadata and data is checksummed, and ZFS automatically repairs bad data +from a good copy when corruption is detected. +.Pp +In order to take advantage of these features, a pool must make use of some form +of redundancy, using either mirrored or raidz groups. +While ZFS supports running in a non-redundant configuration, where each root +vdev is simply a disk or file, this is strongly discouraged. +A single case of bit corruption can render some or all of your data unavailable. +.Pp +A pool's health status is described by one of three states: +.Sy online , degraded , No or Sy faulted . +An online pool has all devices operating normally. +A degraded pool is one in which one or more devices have failed, but the data is +still available due to a redundant configuration. +A faulted pool has corrupted metadata, or one or more faulted devices, and +insufficient replicas to continue functioning. +.Pp +The health of the top-level vdev, such as a mirror or raidz device, +is potentially impacted by the state of its associated vdevs, +or component devices. +A top-level vdev or component device is in one of the following states: +.Bl -tag -width "DEGRADED" +.It Sy DEGRADED +One or more top-level vdevs is in the degraded state because one or more +component devices are offline. +Sufficient replicas exist to continue functioning. +.Pp +One or more component devices is in the degraded or faulted state, but +sufficient replicas exist to continue functioning. +The underlying conditions are as follows: +.Bl -bullet -compact +.It +The number of checksum errors exceeds acceptable levels and the device is +degraded as an indication that something may be wrong. +ZFS continues to use the device as necessary. +.It +The number of I/O errors exceeds acceptable levels. +The device could not be marked as faulted because there are insufficient +replicas to continue functioning. +.El +.It Sy FAULTED +One or more top-level vdevs is in the faulted state because one or more +component devices are offline. +Insufficient replicas exist to continue functioning. +.Pp +One or more component devices is in the faulted state, and insufficient +replicas exist to continue functioning. +The underlying conditions are as follows: +.Bl -bullet -compact +.It +The device could be opened, but the contents did not match expected values. +.It +The number of I/O errors exceeds acceptable levels and the device is faulted to +prevent further use of the device. +.El +.It Sy OFFLINE +The device was explicitly taken offline by the +.Nm zpool Cm offline +command. +.It Sy ONLINE +The device is online and functioning. +.It Sy REMOVED +The device was physically removed while the system was running. +Device removal detection is hardware-dependent and may not be supported on all +platforms. +.It Sy UNAVAIL +The device could not be opened. +If a pool is imported when a device was unavailable, then the device will be +identified by a unique identifier instead of its path since the path was never +correct in the first place. +.El +.Pp +Checksum errors represent events where a disk returned data that was expected +to be correct, but was not. +In other words, these are instances of silent data corruption. +The checksum errors are reported in +.Nm zpool Cm status +and +.Nm zpool Cm events . +When a block is stored redundantly, a damaged block may be reconstructed +(e.g. from raidz parity or a mirrored copy). +In this case, ZFS reports the checksum error against the disks that contained +damaged data. +If a block is unable to be reconstructed (e.g. due to 3 disks being damaged +in a raidz2 group), it is not possible to determine which disks were silently +corrupted. +In this case, checksum errors are reported for all disks on which the block +is stored. +.Pp +If a device is removed and later re-attached to the system, +ZFS attempts online the device automatically. +Device attachment detection is hardware-dependent +and might not be supported on all platforms. +. +.Ss Hot Spares +ZFS allows devices to be associated with pools as +.Qq hot spares . +These devices are not actively used in the pool, but when an active device +fails, it is automatically replaced by a hot spare. +To create a pool with hot spares, specify a +.Sy spare +vdev with any number of devices. +For example, +.Dl # Nm zpool Cm create Ar pool Sy mirror Ar sda sdb Sy spare Ar sdc sdd +.Pp +Spares can be shared across multiple pools, and can be added with the +.Nm zpool Cm add +command and removed with the +.Nm zpool Cm remove +command. +Once a spare replacement is initiated, a new +.Sy spare +vdev is created within the configuration that will remain there until the +original device is replaced. +At this point, the hot spare becomes available again if another device fails. +.Pp +If a pool has a shared spare that is currently being used, the pool can not be +exported since other pools may use this shared spare, which may lead to +potential data corruption. +.Pp +Shared spares add some risk. +If the pools are imported on different hosts, +and both pools suffer a device failure at the same time, +both could attempt to use the spare at the same time. +This may not be detected, resulting in data corruption. +.Pp +An in-progress spare replacement can be cancelled by detaching the hot spare. +If the original faulted device is detached, then the hot spare assumes its +place in the configuration, and is removed from the spare list of all active +pools. +.Pp +The +.Sy draid +vdev type provides distributed hot spares. +These hot spares are named after the dRAID vdev they're a part of +.Po Sy draid1 Ns - Ns Ar 2 Ns - Ns Ar 3 No specifies spare Ar 3 No of vdev Ar 2 , +.No which is a single parity dRAID Pc +and may only be used by that dRAID vdev. +Otherwise, they behave the same as normal hot spares. +.Pp +Spares cannot replace log devices. +. +.Ss Intent Log +The ZFS Intent Log (ZIL) satisfies POSIX requirements for synchronous +transactions. +For instance, databases often require their transactions to be on stable storage +devices when returning from a system call. +NFS and other applications can also use +.Xr fsync 2 +to ensure data stability. +By default, the intent log is allocated from blocks within the main pool. +However, it might be possible to get better performance using separate intent +log devices such as NVRAM or a dedicated disk. +For example: +.Dl # Nm zpool Cm create Ar pool sda sdb Sy log Ar sdc +.Pp +Multiple log devices can also be specified, and they can be mirrored. +See the +.Sx EXAMPLES +section for an example of mirroring multiple log devices. +.Pp +Log devices can be added, replaced, attached, detached and removed. +In addition, log devices are imported and exported as part of the pool +that contains them. +Mirrored devices can be removed by specifying the top-level mirror vdev. +. +.Ss Cache Devices +Devices can be added to a storage pool as +.Qq cache devices . +These devices provide an additional layer of caching between main memory and +disk. +For read-heavy workloads, where the working set size is much larger than what +can be cached in main memory, using cache devices allows much more of this +working set to be served from low latency media. +Using cache devices provides the greatest performance improvement for random +read-workloads of mostly static content. +.Pp +To create a pool with cache devices, specify a +.Sy cache +vdev with any number of devices. +For example: +.Dl # Nm zpool Cm create Ar pool sda sdb Sy cache Ar sdc sdd +.Pp +Cache devices cannot be mirrored or part of a raidz configuration. +If a read error is encountered on a cache device, that read I/O is reissued to +the original storage pool device, which might be part of a mirrored or raidz +configuration. +.Pp +The content of the cache devices is persistent across reboots and restored +asynchronously when importing the pool in L2ARC (persistent L2ARC). +This can be disabled by setting +.Sy l2arc_rebuild_enabled Ns = Ns Sy 0 . +For cache devices smaller than +.Em 1GB , +we do not write the metadata structures +required for rebuilding the L2ARC in order not to waste space. +This can be changed with +.Sy l2arc_rebuild_blocks_min_l2size . +The cache device header +.Pq Em 512B +is updated even if no metadata structures are written. +Setting +.Sy l2arc_headroom Ns = Ns Sy 0 +will result in scanning the full-length ARC lists for cacheable content to be +written in L2ARC (persistent ARC). +If a cache device is added with +.Nm zpool Cm add +its label and header will be overwritten and its contents are not going to be +restored in L2ARC, even if the device was previously part of the pool. +If a cache device is onlined with +.Nm zpool Cm online +its contents will be restored in L2ARC. +This is useful in case of memory pressure +where the contents of the cache device are not fully restored in L2ARC. +The user can off- and online the cache device when there is less memory pressure +in order to fully restore its contents to L2ARC. +. +.Ss Pool checkpoint +Before starting critical procedures that include destructive actions +.Pq like Nm zfs Cm destroy , +an administrator can checkpoint the pool's state and in the case of a +mistake or failure, rewind the entire pool back to the checkpoint. +Otherwise, the checkpoint can be discarded when the procedure has completed +successfully. +.Pp +A pool checkpoint can be thought of as a pool-wide snapshot and should be used +with care as it contains every part of the pool's state, from properties to vdev +configuration. +Thus, certain operations are not allowed while a pool has a checkpoint. +Specifically, vdev removal/attach/detach, mirror splitting, and +changing the pool's GUID. +Adding a new vdev is supported, but in the case of a rewind it will have to be +added again. +Finally, users of this feature should keep in mind that scrubs in a pool that +has a checkpoint do not repair checkpointed data. +.Pp +To create a checkpoint for a pool: +.Dl # Nm zpool Cm checkpoint Ar pool +.Pp +To later rewind to its checkpointed state, you need to first export it and +then rewind it during import: +.Dl # Nm zpool Cm export Ar pool +.Dl # Nm zpool Cm import Fl -rewind-to-checkpoint Ar pool +.Pp +To discard the checkpoint from a pool: +.Dl # Nm zpool Cm checkpoint Fl d Ar pool +.Pp +Dataset reservations (controlled by the +.Sy reservation No and Sy refreservation +properties) may be unenforceable while a checkpoint exists, because the +checkpoint is allowed to consume the dataset's reservation. +Finally, data that is part of the checkpoint but has been freed in the +current state of the pool won't be scanned during a scrub. +. +.Ss Special Allocation Class +Allocations in the special class are dedicated to specific block types. +By default this includes all metadata, the indirect blocks of user data, and +any deduplication tables. +The class can also be provisioned to accept small file blocks. +.Pp +A pool must always have at least one normal +.Pq non- Ns Sy dedup Ns /- Ns Sy special +vdev before +other devices can be assigned to the special class. +If the +.Sy special +class becomes full, then allocations intended for it +will spill back into the normal class. +.Pp +Deduplication tables can be excluded from the special class by unsetting the +.Sy zfs_ddt_data_is_special +ZFS module parameter. +.Pp +Inclusion of small file blocks in the special class is opt-in. +Each dataset can control the size of small file blocks allowed +in the special class by setting the +.Sy special_small_blocks +property to nonzero. +See +.Xr zfsprops 7 +for more info on this property. diff --git a/man/man7/zpoolprops.7 b/man/man7/zpoolprops.7 new file mode 100644 index 000000000..513f02e03 --- /dev/null +++ b/man/man7/zpoolprops.7 @@ -0,0 +1,412 @@ +.\" +.\" CDDL HEADER START +.\" +.\" The contents of this file are subject to the terms of the +.\" Common Development and Distribution License (the "License"). +.\" You may not use this file except in compliance with the License. +.\" +.\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +.\" or http://www.opensolaris.org/os/licensing. +.\" See the License for the specific language governing permissions +.\" and limitations under the License. +.\" +.\" When distributing Covered Code, include this CDDL HEADER in each +.\" file and include the License file at usr/src/OPENSOLARIS.LICENSE. +.\" If applicable, add the following below this CDDL HEADER, with the +.\" fields enclosed by brackets "[]" replaced with your own identifying +.\" information: Portions Copyright [yyyy] [name of copyright owner] +.\" +.\" CDDL HEADER END +.\" +.\" Copyright (c) 2007, Sun Microsystems, Inc. All Rights Reserved. +.\" Copyright (c) 2012, 2018 by Delphix. All rights reserved. +.\" Copyright (c) 2012 Cyril Plisko. All Rights Reserved. +.\" Copyright (c) 2017 Datto Inc. +.\" Copyright (c) 2018 George Melikov. All Rights Reserved. +.\" Copyright 2017 Nexenta Systems, Inc. +.\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. +.\" Copyright (c) 2021, Colm Buckley <[email protected]> +.\" +.Dd May 27, 2021 +.Dt ZPOOLPROPS 7 +.Os +. +.Sh NAME +.Nm zpoolprops +.Nd properties of ZFS storage pools +. +.Sh DESCRIPTION +Each pool has several properties associated with it. +Some properties are read-only statistics while others are configurable and +change the behavior of the pool. +.Pp +The following are read-only properties: +.Bl -tag -width "unsupported@guid" +.It Cm allocated +Amount of storage used within the pool. +See +.Sy fragmentation +and +.Sy free +for more information. +.It Sy capacity +Percentage of pool space used. +This property can also be referred to by its shortened column name, +.Sy cap . +.It Sy expandsize +Amount of uninitialized space within the pool or device that can be used to +increase the total capacity of the pool. +On whole-disk vdevs, this is the space beyond the end of the GPT – +typically occurring when a LUN is dynamically expanded +or a disk replaced with a larger one. +On partition vdevs, this is the space appended to the partition after it was +added to the pool – most likely by resizing it in-place. +The space can be claimed for the pool by bringing it online with +.Sy autoexpand=on +or using +.Nm zpool Cm online Fl e . +.It Sy fragmentation +The amount of fragmentation in the pool. +As the amount of space +.Sy allocated +increases, it becomes more difficult to locate +.Sy free +space. +This may result in lower write performance compared to pools with more +unfragmented free space. +.It Sy free +The amount of free space available in the pool. +By contrast, the +.Xr zfs 8 +.Sy available +property describes how much new data can be written to ZFS filesystems/volumes. +The zpool +.Sy free +property is not generally useful for this purpose, and can be substantially more than the zfs +.Sy available +space. +This discrepancy is due to several factors, including raidz parity; +zfs reservation, quota, refreservation, and refquota properties; and space set aside by +.Sy spa_slop_shift +(see +.Xr zfs 4 +for more information). +.It Sy freeing +After a file system or snapshot is destroyed, the space it was using is +returned to the pool asynchronously. +.Sy freeing +is the amount of space remaining to be reclaimed. +Over time +.Sy freeing +will decrease while +.Sy free +increases. +.It Sy health +The current health of the pool. +Health can be one of +.Sy ONLINE , DEGRADED , FAULTED , OFFLINE, REMOVED , UNAVAIL . +.It Sy guid +A unique identifier for the pool. +.It Sy load_guid +A unique identifier for the pool. +Unlike the +.Sy guid +property, this identifier is generated every time we load the pool (i.e. does +not persist across imports/exports) and never changes while the pool is loaded +(even if a +.Sy reguid +operation takes place). +.It Sy size +Total size of the storage pool. +.It Sy unsupported@ Ns Em guid +Information about unsupported features that are enabled on the pool. +See +.Xr zpool-features 7 +for details. +.El +.Pp +The space usage properties report actual physical space available to the +storage pool. +The physical space can be different from the total amount of space that any +contained datasets can actually use. +The amount of space used in a raidz configuration depends on the characteristics +of the data being written. +In addition, ZFS reserves some space for internal accounting that the +.Xr zfs 8 +command takes into account, but the +.Nm +command does not. +For non-full pools of a reasonable size, these effects should be invisible. +For small pools, or pools that are close to being completely full, these +discrepancies may become more noticeable. +.Pp +The following property can be set at creation time and import time: +.Bl -tag -width Ds +.It Sy altroot +Alternate root directory. +If set, this directory is prepended to any mount points within the pool. +This can be used when examining an unknown pool where the mount points cannot be +trusted, or in an alternate boot environment, where the typical paths are not +valid. +.Sy altroot +is not a persistent property. +It is valid only while the system is up. +Setting +.Sy altroot +defaults to using +.Sy cachefile Ns = Ns Sy none , +though this may be overridden using an explicit setting. +.El +.Pp +The following property can be set only at import time: +.Bl -tag -width Ds +.It Sy readonly Ns = Ns Sy on Ns | Ns Sy off +If set to +.Sy on , +the pool will be imported in read-only mode. +This property can also be referred to by its shortened column name, +.Sy rdonly . +.El +.Pp +The following properties can be set at creation time and import time, and later +changed with the +.Nm zpool Cm set +command: +.Bl -tag -width Ds +.It Sy ashift Ns = Ns Sy ashift +Pool sector size exponent, to the power of +.Sy 2 +(internally referred to as +.Sy ashift ) . +Values from 9 to 16, inclusive, are valid; also, the +value 0 (the default) means to auto-detect using the kernel's block +layer and a ZFS internal exception list. +I/O operations will be aligned to the specified size boundaries. +Additionally, the minimum (disk) +write size will be set to the specified size, so this represents a +space vs. performance trade-off. +For optimal performance, the pool sector size should be greater than +or equal to the sector size of the underlying disks. +The typical case for setting this property is when +performance is important and the underlying disks use 4KiB sectors but +report 512B sectors to the OS (for compatibility reasons); in that +case, set +.Sy ashift Ns = Ns Sy 12 +(which is +.Sy 1<<12 No = Sy 4096 ) . +When set, this property is +used as the default hint value in subsequent vdev operations (add, +attach and replace). +Changing this value will not modify any existing +vdev, not even on disk replacement; however it can be used, for +instance, to replace a dying 512B sectors disk with a newer 4KiB +sectors device: this will probably result in bad performance but at the +same time could prevent loss of data. +.It Sy autoexpand Ns = Ns Sy on Ns | Ns Sy off +Controls automatic pool expansion when the underlying LUN is grown. +If set to +.Sy on , +the pool will be resized according to the size of the expanded device. +If the device is part of a mirror or raidz then all devices within that +mirror/raidz group must be expanded before the new space is made available to +the pool. +The default behavior is +.Sy off . +This property can also be referred to by its shortened column name, +.Sy expand . +.It Sy autoreplace Ns = Ns Sy on Ns | Ns Sy off +Controls automatic device replacement. +If set to +.Sy off , +device replacement must be initiated by the administrator by using the +.Nm zpool Cm replace +command. +If set to +.Sy on , +any new device, found in the same physical location as a device that previously +belonged to the pool, is automatically formatted and replaced. +The default behavior is +.Sy off . +This property can also be referred to by its shortened column name, +.Sy replace . +Autoreplace can also be used with virtual disks (like device +mapper) provided that you use the /dev/disk/by-vdev paths setup by +vdev_id.conf. +See the +.Xr vdev_id 8 +manual page for more details. +Autoreplace and autoonline require the ZFS Event Daemon be configured and +running. +See the +.Xr zed 8 +manual page for more details. +.It Sy autotrim Ns = Ns Sy on Ns | Ns Sy off +When set to +.Sy on +space which has been recently freed, and is no longer allocated by the pool, +will be periodically trimmed. +This allows block device vdevs which support +BLKDISCARD, such as SSDs, or file vdevs on which the underlying file system +supports hole-punching, to reclaim unused blocks. +The default value for this property is +.Sy off . +.Pp +Automatic TRIM does not immediately reclaim blocks after a free. +Instead, it will optimistically delay allowing smaller ranges to be aggregated +into a few larger ones. +These can then be issued more efficiently to the storage. +TRIM on L2ARC devices is enabled by setting +.Sy l2arc_trim_ahead > 0 . +.Pp +Be aware that automatic trimming of recently freed data blocks can put +significant stress on the underlying storage devices. +This will vary depending of how well the specific device handles these commands. +For lower-end devices it is often possible to achieve most of the benefits +of automatic trimming by running an on-demand (manual) TRIM periodically +using the +.Nm zpool Cm trim +command. +.It Sy bootfs Ns = Ns Sy (unset) Ns | Ns Ar pool Ns Op / Ns Ar dataset +Identifies the default bootable dataset for the root pool. +This property is expected to be set mainly by the installation and upgrade programs. +Not all Linux distribution boot processes use the bootfs property. +.It Sy cachefile Ns = Ns Ar path Ns | Ns Sy none +Controls the location of where the pool configuration is cached. +Discovering all pools on system startup requires a cached copy of the +configuration data that is stored on the root file system. +All pools in this cache are automatically imported when the system boots. +Some environments, such as install and clustering, need to cache this +information in a different location so that pools are not automatically +imported. +Setting this property caches the pool configuration in a different location that +can later be imported with +.Nm zpool Cm import Fl c . +Setting it to the value +.Sy none +creates a temporary pool that is never cached, and the +.Qq +.Pq empty string +uses the default location. +.Pp +Multiple pools can share the same cache file. +Because the kernel destroys and recreates this file when pools are added and +removed, care should be taken when attempting to access this file. +When the last pool using a +.Sy cachefile +is exported or destroyed, the file will be empty. +.It Sy comment Ns = Ns Ar text +A text string consisting of printable ASCII characters that will be stored +such that it is available even if the pool becomes faulted. +An administrator can provide additional information about a pool using this +property. +.It Sy compatibility Ns = Ns Sy off Ns | Ns Sy legacy Ns | Ns Ar file Ns Oo , Ns Ar file Oc Ns … +Specifies that the pool maintain compatibility with specific feature sets. +When set to +.Sy off +(or unset) compatibility is disabled (all features may be enabled); when set to +.Sy legacy Ns +no features may be enabled. +When set to a comma-separated list of filenames +(each filename may either be an absolute path, or relative to +.Pa /etc/zfs/compatibility.d +or +.Pa /usr/share/zfs/compatibility.d ) +the lists of requested features are read from those files, separated by +whitespace and/or commas. +Only features present in all files may be enabled. +.Pp +See +.Xr zpool-features 7 , +.Xr zpool-create 8 +and +.Xr zpool-upgrade 8 +for more information on the operation of compatibility feature sets. +.It Sy dedupditto Ns = Ns Ar number +This property is deprecated and no longer has any effect. +.It Sy delegation Ns = Ns Sy on Ns | Ns Sy off +Controls whether a non-privileged user is granted access based on the dataset +permissions defined on the dataset. +See +.Xr zfs 8 +for more information on ZFS delegated administration. +.It Sy failmode Ns = Ns Sy wait Ns | Ns Sy continue Ns | Ns Sy panic +Controls the system behavior in the event of catastrophic pool failure. +This condition is typically a result of a loss of connectivity to the underlying +storage device(s) or a failure of all devices within the pool. +The behavior of such an event is determined as follows: +.Bl -tag -width "continue" +.It Sy wait +Blocks all I/O access until the device connectivity is recovered and the errors +are cleared. +This is the default behavior. +.It Sy continue +Returns +.Er EIO +to any new write I/O requests but allows reads to any of the remaining healthy +devices. +Any write requests that have yet to be committed to disk would be blocked. +.It Sy panic +Prints out a message to the console and generates a system crash dump. +.El +.It Sy feature@ Ns Ar feature_name Ns = Ns Sy enabled +The value of this property is the current state of +.Ar feature_name . +The only valid value when setting this property is +.Sy enabled +which moves +.Ar feature_name +to the enabled state. +See +.Xr zpool-features 7 +for details on feature states. +.It Sy listsnapshots Ns = Ns Sy on Ns | Ns Sy off +Controls whether information about snapshots associated with this pool is +output when +.Nm zfs Cm list +is run without the +.Fl t +option. +The default value is +.Sy off . +This property can also be referred to by its shortened name, +.Sy listsnaps . +.It Sy multihost Ns = Ns Sy on Ns | Ns Sy off +Controls whether a pool activity check should be performed during +.Nm zpool Cm import . +When a pool is determined to be active it cannot be imported, even with the +.Fl f +option. +This property is intended to be used in failover configurations +where multiple hosts have access to a pool on shared storage. +.Pp +Multihost provides protection on import only. +It does not protect against an +individual device being used in multiple pools, regardless of the type of vdev. +See the discussion under +.Nm zpool Cm create . +.Pp +When this property is on, periodic writes to storage occur to show the pool is +in use. +See +.Sy zfs_multihost_interval +in the +.Xr zfs 4 +manual page. +In order to enable this property each host must set a unique hostid. +See +.Xr genhostid 1 +.Xr zgenhostid 8 +.Xr spl 4 +for additional details. +The default value is +.Sy off . +.It Sy version Ns = Ns Ar version +The current on-disk version of the pool. +This can be increased, but never decreased. +The preferred method of updating pools is with the +.Nm zpool Cm upgrade +command, though this property can be used when a specific version is needed for +backwards compatibility. +Once feature flags are enabled on a pool this property will no longer have a +value. +.El |