Project Quota on ZFS

Project quota is a new ZFS system space/object usage accounting
and enforcement mechanism. Similar as user/group quota, project
quota is another dimension of system quota. It bases on the new
object attribute - project ID.

Project ID is a numerical value to indicate to which project an
object belongs. An object only can belong to one project though
you (the object owner or privileged user) can change the object
project ID via 'chattr -p' or 'zfs project [-s] -p' explicitly.
The object also can inherit the project ID from its parent when
created if the parent has the project inherit flag (that can be
set via 'chattr +P' or 'zfs project -s [-p]').

By accounting the spaces/objects belong to the same project, we
can know how many spaces/objects used by the project. And if we
set the upper limit then we can control the spaces/objects that
are consumed by such project. It is useful when multiple groups
and users cooperate for the same project, or a user/group needs
to participate in multiple projects.

Support the following commands and functionalities:

zfs set projectquota@project
zfs set projectobjquota@project

zfs get projectquota@project
zfs get projectobjquota@project
zfs get projectused@project
zfs get projectobjused@project

zfs projectspace

zfs allow projectquota
zfs allow projectobjquota
zfs allow projectused
zfs allow projectobjused

zfs unallow projectquota
zfs unallow projectobjquota
zfs unallow projectused
zfs unallow projectobjused

chattr +/-P
chattr -p project_id
lsattr -p

This patch also supports tree quota based on the project quota via
"zfs project" commands set as following:
zfs project [-d|-r] <file|directory ...>
zfs project -C [-k] [-r] <file|directory ...>
zfs project -c [-0] [-d|-r] [-p id] <file|directory ...>
zfs project [-p id] [-r] [-s] <file|directory ...>

For "df [-i] $DIR" command, if we set INHERIT (project ID) flag on
the $DIR, then the proejct [obj]quota and [obj]used values for the
$DIR's project ID will be shown as the total/free (avail) resource.
Keep the same behavior as EXT4/XFS does.

Reviewed-by: Andreas Dilger <[email protected]>
Reviewed-by  Ned Bass <[email protected]>
Reviewed-by: Matthew Ahrens <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Fan Yong <[email protected]>
TEST_ZIMPORT_POOLS="zol-0.6.1 zol-0.6.2 master"
Change-Id: Ib4f0544602e03fb61fd46a849d7ba51a6005693c
Closes #6290 

2 files changed, 246 insertions, 0 deletions

diff --git a/man/man5/zpool-features.5 b/man/man5/zpool-features.5
index 72a6c57b1..523fd1fd0 100644
--- a/man/man5/zpool-features.5
+++ b/man/man5/zpool-features.5
@@ -640,5 +640,39 @@ are destroyed.
 
 .RE
 
+.sp
+.ne 2
+.na
+\fB\fBproject_quota\fR\fR
+.ad
+.RS 4n
+.TS
+l l .
+GUID	org.zfsonlinux:project_quota
+READ\-ONLY COMPATIBLE	yes
+DEPENDENCIES	extensible_dataset
+.TE
+
+This feature allows administrators to account the spaces and objects usage
+information against the project identifier (ID).
+
+The project ID is new object-based attribute. When upgrading an existing
+filesystem, object without project ID attribute will be assigned a zero
+project ID. After this feature is enabled, newly created object will inherit
+its parent directory's project ID if the parent inherit flag is set (via
+\fBchattr +/-P\fR or \fBzfs project [-s|-C]\fR). Otherwise, the new object's
+project ID will be set as zero. An object's project ID can be changed at
+anytime by the owner (or privileged user) via \fBchattr -p $prjid\fR or
+\fBzfs project -p $prjid\fR.
+
+This feature will become \fBactive\fR as soon as it is enabled and will never
+return to being \fBdisabled\fR. Each filesystem will be upgraded automatically
+when remounted or when new file is created under that filesystem. The upgrade
+can also be triggered on filesystems via `zfs set version=current <pool/fs>`.
+The upgrade process runs in the background and may take a while to complete
+for the filesystems containing a large number of files.
+
+.RE
+
 .SH "SEE ALSO"
 \fBzpool\fR(8)
diff --git a/man/man8/zfs.8 b/man/man8/zfs.8
index 2e1ffc6ea..f42851328 100644
--- a/man/man8/zfs.8
+++ b/man/man8/zfs.8
@@ -145,6 +145,34 @@
 .Oo Fl t Ar type Ns Oo , Ns Ar type Oc Ns ... Oc
 .Ar filesystem Ns | Ns Ar snapshot
 .Nm
+.Cm projectspace
+.Op Fl Hp
+.Oo Fl o Ar field Ns Oo , Ns Ar field Oc Ns ... Oc
+.Oo Fl s Ar field Oc Ns ...
+.Oo Fl S Ar field Oc Ns ...
+.Ar filesystem Ns | Ns Ar snapshot
+.Nm
+.Cm project
+.Oo Fl d Ns | Ns Fl r Ns Oc
+.Ar file Ns | Ns Ar directory Ns ...
+.Nm
+.Cm project
+.Fl C
+.Oo Fl kr Ns Oc
+.Ar file Ns | Ns Ar directory Ns ...
+.Nm
+.Cm project
+.Fl c
+.Oo Fl 0 Ns Oc
+.Oo Fl d Ns | Ns Fl r Ns Oc
+.Op Fl p Ar id
+.Ar file Ns | Ns Ar directory Ns ...
+.Nm
+.Cm project
+.Op Fl p Ar id
+.Oo Fl rs Ns Oc
+.Ar file Ns | Ns Ar directory Ns ...
+.Nm
 .Cm mount
 .Nm
 .Cm mount
@@ -905,6 +933,56 @@ The root user, or a user who has been granted the
 privilege with
 .Nm zfs Cm allow ,
 can access all groups' usage.
+.It Sy projectused Ns @ Ns Em project
+The amount of space consumed by the specified project in this dataset. Project
+is identified via the project identifier (ID) that is object-based numeral
+attribute. An object can inherit the project ID from its parent object (if the
+parent has the flag of inherit project ID that can be set and changed via
+.Nm chattr Fl /+P
+or
+.Nm zfs project Fl s )
+when being created. The privileged user can set and change object's project
+ID via
+.Nm chattr Fl p
+or
+.Nm zfs project Fl s
+anytime. Space is charged to the project of each file, as displayed by
+.Nm lsattr Fl p
+or
+.Nm zfs project .
+See the
+.Sy userused Ns @ Ns Em user
+property for more information.
+.Pp
+The root user, or a user who has been granted the
+.Sy projectused
+privilege with
+.Nm zfs allow ,
+can access all projects' usage.
+.It Sy projectobjused Ns @ Ns Em project
+The
+.Sy projectobjused
+is similar to
+.Sy projectused
+but instead it counts the number of objects consumed by project. When the
+property
+.Sy xattr=on
+is set on a fileset, ZFS will create additional objects per-file to store
+extended attributes. These additional objects are reflected in the
+.Sy projectobjused
+value and are counted against the project's
+.Sy projectobjquota .
+When a filesystem is configured to use
+.Sy xattr=sa
+no additional internal objects are required. See the
+.Sy userobjused Ns @ Ns Em user
+property for more information.
+.Pp
+The root user, or a user who has been granted the
+.Sy projectobjused
+privilege with
+.Nm zfs allow ,
+can access all projects' objects usage.
 .It Sy volblocksize
 For volumes, specifies the block size of the volume.
 The
@@ -1566,6 +1644,27 @@ is similar to
 but it limits number of objects a group can consume. Please refer to
 .Sy userobjused
 for more information about how objects are counted.
+.It Sy projectquota@ Ns Em project Ns = Ns Em size Ns | Ns Sy none
+Limits the amount of space consumed by the specified project. Project
+space consumption is identified by the
+.Sy projectused@ Ns Em project
+property. Please refer to
+.Sy projectused
+for more information about how project is identified and set/changed.
+.Pp
+The root user, or a user who has been granted the
+.Sy projectquota
+privilege with
+.Nm zfs allow ,
+can access all projects' quota.
+.It Sy projectobjquota@ Ns Em project Ns = Ns Em size Ns | Ns Sy none
+The
+.Sy projectobjquota
+is similar to
+.Sy projectquota
+but it limits number of objects a project can consume. Please refer to
+.Sy userobjused
+for more information about how objects are counted.
 .It Sy readonly Ns = Ns Sy on Ns | Ns Sy off
 Controls whether this dataset can be modified.
 The default value is
@@ -3000,6 +3099,114 @@ except that the default types to display are
 .Fl t Sy posixgroup Ns \&, Ns Sy smbgroup .
 .It Xo
 .Nm
+.Cm projectspace
+.Op Fl Hp
+.Oo Fl o Ar field Ns Oo , Ns Ar field Oc Ns ... Oc
+.Oo Fl s Ar field Oc Ns ...
+.Oo Fl S Ar field Oc Ns ...
+.Ar filesystem Ns | Ns Ar snapshot
+.Xc
+Displays space consumed by, and quotas on, each project in the specified
+filesystem or snapshot. This subcommand is identical to
+.Nm zfs Cm userspace ,
+except that the project identifier is numeral, not name. So need neither
+the option
+.Sy -i
+for SID to POSIX ID nor
+.Sy -n
+for numeric ID, nor
+.Sy -t
+for types.
+.It Xo
+.Nm
+.Cm project
+.Oo Fl d Ns | Ns Fl r Ns Oc
+.Ar file Ns | Ns Ar directory Ns ...
+.Xc
+List project identifier (ID) and inherit flag of file(s) or directories.
+.Bl -tag -width "-d"
+.It Fl d
+Show the directory project ID and inherit flag, not its childrens. It will
+overwrite the former specified
+.Fl r
+option.
+.It Fl r
+Show on subdirectories recursively. It will overwrite the former specified
+.Fl d
+option.
+.El
+.It Xo
+.Nm
+.Cm project
+.Fl C
+.Oo Fl kr Ns Oc
+.Ar file Ns | Ns Ar directory Ns ...
+.Xc
+Clear project inherit flag and/or ID on the file(s) or directories.
+.Bl -tag -width "-k"
+.It Fl k
+Keep the project ID unchanged. If not specified, the project ID will be reset
+as zero.
+.It Fl r
+Clear on subdirectories recursively.
+.El
+.It Xo
+.Nm
+.Cm project
+.Fl c
+.Oo Fl 0 Ns Oc
+.Oo Fl d Ns | Ns Fl r Ns Oc
+.Op Fl p Ar id
+.Ar file Ns | Ns Ar directory Ns ...
+.Xc
+Check project ID and inherit flag on the file(s) or directories, report the
+entries without project inherit flag or with different project IDs from the
+specified (via
+.Fl p
+option) value or the target directory's project ID.
+.Bl -tag -width "-0"
+.It Fl 0
+Print file name with a trailing NUL instead of newline (by default), like
+"find -print0".
+.It Fl d
+Check the directory project ID and inherit flag, not its childrens. It will
+overwrite the former specified
+.Fl r
+option.
+.It Fl p
+Specify the referenced ID for comparing with the target file(s) or directories'
+project IDs. If not specified, the target (top) directory's project ID will be
+used as the referenced one.
+.It Fl r
+Check on subdirectories recursively. It will overwrite the former specified
+.Fl d
+option.
+.El
+.It Xo
+.Nm
+.Cm project
+.Op Fl p Ar id
+.Oo Fl rs Ns Oc
+.Ar file Ns | Ns Ar directory Ns ...
+.Xc
+.Bl -tag -width "-p"
+Set project ID and/or inherit flag on the file(s) or directories.
+.It Fl p
+Set the file(s)' or directories' project ID with the given value.
+.It Fl r
+Set on subdirectories recursively.
+.It Fl s
+Set project inherit flag on the given file(s) or directories. It is usually used
+for setup tree quota on the directory target with
+.Fl r
+option specified together. When setup tree quota, by default the directory's
+project ID will be set to all its descendants unless you specify the project
+ID via
+.Fl p
+option explicitly.
+.El
+.It Xo
+.Nm
 .Cm mount
 .Xc
 Displays all ZFS file systems currently mounted.
@@ -3812,6 +4019,11 @@ userprop         other          Allows changing any user property
 userquota        other          Allows accessing any userquota@...
                                 property
 userused         other          Allows reading any userused@... property
+projectobjquota  other          Allows accessing any projectobjquota@...
+                                property
+projectquota     other          Allows accessing any projectquota@... property
+projectobjused   other          Allows reading any projectobjused@... property
+projectused      other          Allows reading any projectused@... property
 
 aclinherit       property
 acltype          property