summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Support the fallocate() file operation.Etienne Dechamps2012-02-0961-0/+447
| | | | | | | | | | | | | | Currently only the (FALLOC_FL_PUNCH_HOLE) flag combination is supported, since it's the only one that matches the behavior of zfs_space(). This makes it pretty much useless in its current form, but it's a start. To support other flag combinations we would need to modify zfs_space() to make it more flexible, or emulate the desired functionality in zpl_fallocate(). Signed-off-by: Brian Behlendorf <[email protected]> Issue #334
* Check permissions in zfs_space().Etienne Dechamps2012-02-091-0/+11
| | | | | | | | | | This isn't done on Solaris because on this OS zfs_space() can only be called with an opened file handle. Since the addition of zpl_truncate_range() this isn't the case anymore, so we need to enforce access rights. Signed-off-by: Brian Behlendorf <[email protected]> Issue #334
* Implement the truncate_range() inode operation.Etienne Dechamps2012-02-091-0/+28
| | | | | | | | | | | | | | | | This operation allows "hole punching" in ZFS files. On Solaris this is done via the vop_space() system call, which maps to the zfs_space() function. So we just need to write zpl_truncate_range() as a wrapper around zfs_space(). Note that this only works for regular files, not ZVOLs. This is currently an insecure implementation without permission checking, although this isn't that big of a deal since truncate_range() isn't even callable from userspace. Signed-off-by: Brian Behlendorf <[email protected]> Issue #334
* Fix zconfig.sh non-optimal alignmentBrian Behlendorf2012-02-092-56/+67
| | | | | | | | | | | | | | | The recent zvol improvements have changed default suggested alignment for zvols from 512b (default) to 8k (zvol blocksize). Because of this the zconfig.sh tests which create paritions are now generating a warning about non-optimal alignments. This change updates the need zconfig.sh tests such that a partition will be properly aligned. In the process, it shifts from using the sfdisk utility to the parted utility to create partitions. It also moves the creation of labels, partitions, and filesystems in to generic functions in common.sh.in. Signed-off-by: Brian Behlendorf <[email protected]>
* Use 32 as the default number of zvol threads.Etienne Dechamps2012-02-081-4/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, the `zvol_threads` variable, which controls the number of worker threads which process items from the ZVOL queues, is set to the number of available CPUs. This choice seems to be based on the assumption that ZVOL threads are CPU-bound. This is not necessarily true, especially for synchronous writes. Consider the situation described in the comments for `zil_commit()`, which is called inside `zvol_write()` for synchronous writes: > itxs are committed in batches. In a heavily stressed zil there will be a > commit writer thread who is writing out a bunch of itxs to the log for a > set of committing threads (cthreads) in the same batch as the writer. > Those cthreads are all waiting on the same cv for that batch. > > There will also be a different and growing batch of threads that are > waiting to commit (qthreads). When the committing batch completes a > transition occurs such that the cthreads exit and the qthreads become > cthreads. One of the new cthreads becomes he writer thread for the batch. > Any new threads arriving become new qthreads. We can easily deduce that, in the case of ZVOLs, there can be a maximum of `zvol_threads` cthreads and qthreads. The default value for `zvol_threads` is typically between 1 and 8, which is way too low in this case. This means there will be a lot of small commits to the ZIL, which is very inefficient compared to a few big commits, especially since we have to wait for the data to be on stable storage. Increasing the number of threads will increase the amount of data waiting to be commited and thus the size of the individual commits. On my system, in the context of VM disk image storage (lots of small synchronous writes), increasing `zvol_threads` from 8 to 32 results in a 50% increase in sequential synchronous write performance. We should choose a more sensible default for `zvol_threads`. Unfortunately the optimal value is difficult to determine automatically, since it depends on the synchronous write latency of the underlying storage devices. In any case, a hardcoded value of 32 would probably be better than the current situation. Having a lot of ZVOL threads doesn't seem to have any real downside anyway. Signed-off-by: Brian Behlendorf <[email protected]> Fixes #392
* Improve ZVOL queue behavior.Etienne Dechamps2012-02-0764-0/+1159
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Linux block device queue subsystem exposes a number of configurable settings described in Linux block/blk-settings.c. The defaults for these settings are tuned for hard drives, and are not optimized for ZVOLs. Proper configuration of these options would allow upper layers (I/O scheduler) to take better decisions about write merging and ordering. Detailed rationale: - max_hw_sectors is set to unlimited (UINT_MAX). zvol_write() is able to handle writes of any size, so there's no reason to impose a limit. Let the upper layer decide. - max_segments and max_segment_size are set to unlimited. zvol_write() will copy the requests' contents into a dbuf anyway, so the number and size of the segments are irrelevant. Let the upper layer decide. - physical_block_size and io_opt are set to the ZVOL's block size. This has the potential to somewhat alleviate issue #361 for ZVOLs, by warning the upper layers that writes smaller than the volume's block size will be slow. - The NONROT flag is set to indicate this isn't a rotational device. Although the backing zpool might be composed of rotational devices, the resulting ZVOL often doesn't exhibit the same behavior due to the COW mechanisms used by ZFS. Setting this flag will prevent upper layers from making useless decisions (such as reordering writes) based on incorrect assumptions about the behavior of the ZVOL. Signed-off-by: Brian Behlendorf <[email protected]>
* Fix synchronicity for ZVOLs.Etienne Dechamps2012-02-0760-3/+433
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | zvol_write() assumes that the write request must be written to stable storage if rq_is_sync() is true. Unfortunately, this assumption is incorrect. Indeed, "sync" does *not* mean what we think it means in the context of the Linux block layer. This is well explained in linux/fs.h: WRITE: A normal async write. Device will be plugged. WRITE_SYNC: Synchronous write. Identical to WRITE, but passes down the hint that someone will be waiting on this IO shortly. WRITE_FLUSH: Like WRITE_SYNC but with preceding cache flush. WRITE_FUA: Like WRITE_SYNC but data is guaranteed to be on non-volatile media on completion. In other words, SYNC does not *mean* that the write must be on stable storage on completion. It just means that someone is waiting on us to complete the write request. Thus triggering a ZIL commit for each SYNC write request on a ZVOL is unnecessary and harmful for performance. To make matters worse, ZVOL users have no way to express that they actually want data to be written to stable storage, which means the ZIL is broken for ZVOLs. The request for stable storage is expressed by the FUA flag, so we must commit the ZIL after the write if the FUA flag is set. In addition, we must commit the ZIL before the write if the FLUSH flag is set. Also, we must inform the block layer that we actually support FLUSH and FUA. Signed-off-by: Brian Behlendorf <[email protected]>
* Support "sync=always" for ZVOLs.Etienne Dechamps2012-02-071-1/+1
| | | | | | | | Currently the "sync=always" property works for regular ZFS datasets, but not for ZVOLs. This patch remedies that. Signed-off-by: Brian Behlendorf <[email protected]> Fixes #374.
* Let libnvpair be linked independently of libzfs.Darik Horn2012-02-074-3/+9
| | | | | | | | | | | | Autoconf will fail to detect the ZoL libnvpair on systems that do not implicitly link library runtime dependencies, which is anything that has the GCC 4.5 DCO update. Build libuutil before libnvpair, and put it on the the LDADD line of the libnvpair automake template. Signed-off-by: Brian Behlendorf <[email protected]> Closes: #560
* Linux 3.3 compat, sops->show_options()Brian Behlendorf2012-02-0359-7/+234
| | | | | | | | | | The second argument of sops->show_options() was changed from a 'struct vfsmount *' to a 'struct dentry *'. Add an autoconf check to detect the API change and then conditionally define the expected interface. In either case we are only interested in the zfs_sb_t. Signed-off-by: Brian Behlendorf <[email protected]> Closes #549
* Cleanup ZFS debug infrastructureBrian Behlendorf2012-02-027-162/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | Historically the internal zfs debug infrastructure has been scattered throughout the code. Since we expect to start making more use of this code this patch performs some cleanup. * Consolidate the zfs debug infrastructure in the zfs_debug.[ch] files. This includes moving the zfs_flags and zfs_recover variables, plus moving the zfs_panic_recover() function. * Remove the existing unused functionality in zfs_debug.c and replace it with code which correctly utilized the spl logging infrastructure. * Remove the __dprintf() function from zfs_ioctl.c. This is dead code, the dprintf() functionality in the kernel relies on the spl log support. * Remove dprintf() from hdr_recl(). This wasn't particularly useful and was missing the required format specifier anyway. * Subsequent patches should unify the dprintf() and zfs_dbgmsg() functions. Signed-off-by: Brian Behlendorf <[email protected]>
* Allow multiple values per directory entryBrian Behlendorf2012-02-022-6/+28
| | | | | | | | | | | | | | | | | | | | When using zfs to back a Lustre filesystem it's advantageous to to store a fid with the object id in the directory zap. The only technical impediment to doing this is that the zpl code expects a single value in the zap per directory entry. This change relaxes that requirement such that multiple entries are allowed provided the first one is the object id. The zpl code will just ignore additional entries. This allows the ZoL count to mount datasets which are being used as Lustre server backends. Once the upstream feature flags support is merged in this change should be updated to a read-only feature. Until this occurs other zfs implementations will not be able to read the zfs filesystems created by Lustre. Signed-off-by: Brian Behlendorf <[email protected]>
* Export symbol zfs_attr_tableBrian Behlendorf2012-01-271-0/+1
| | | | | | | | Export the zfs_attr_table symbol so it may be used by non-zpl consumers which are still interested in writing a zpl compatible dataset (e.g. Lustre). Signed-off-by: Brian Behlendorf <[email protected]>
* Ignore dataset if the dds_type is DMU_OST_OTHERPrakash Surya2012-01-191-0/+3
| | | | | | | | | | | | Since the zpios and potentially other ZFS tests use the DMU_OST_OTHER type to label their datasets, the zpool and zfs commands should gracefully handle this type when it is encountered. This patch modifies the commands' behavior to ignore any datasets with a dds_type of DMU_OST_OTHER. Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #536
* Fix rpm dependenciesBrian Behlendorf2012-01-182-10/+11
| | | | | | | | | | | | | | | | | | | | | | | This change updates the rpm spec files to have strictly correct package dependencies. That means a few things: * The zfs-modules package is now tied to a specific build of the spl-modules packages based on the kernel version. This ensures that the correct spl-modules packages will always get installed and not just the newest. * The zfs package now requires both the zfs-modules and spl packages. Thus a 'yum install zfs' will pull in the minimal set of packages required for a functional system. * The zfs-devel packages now require the zfs package to be installed which is normal behavior for -devel packages. * Remove the redundant distribution release extension. This is already added once because it is part of the kernel package release name. Signed-off-by: Brian Behlendorf <[email protected]>
* Add the release component to headersBrian Behlendorf2012-01-1816-24/+24
| | | | | | | | | When the original build system code was added the release component was accidentally omited from the development header install path. This patch adds the missing path component so it's always clear exactly what release your compiling against. Signed-off-by: Brian Behlendorf <[email protected]>
* Allow GPT+EFI vdev replacement in boot pools.Darik Horn2012-01-181-0/+2
| | | | | | | | | | | | | | Commit zfsonlinux/zfs@57a4eddc4d5e1e6c10d8d7dcf87a9fc27398adcd allows the bootfs property to be set on any pool, but does not accommodate subsequent vdev changes. For example: # zpool replace rpool /dev/sda /dev/sdb operation not supported on this type of pool property 'bootfs' is not supported on EFI labeled devices For non-Solaris builds, disable the check that emits this error. Signed-off-by: Brian Behlendorf <[email protected]>
* Combine libraries: spl, avl, efi, share, unicode.Darik Horn2012-01-1732-485/+154
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | These libraries, which are an artifact of the ZoL development process, conflict with packages that are already in distribution: * libspl: SPL Programming Language * libavl: AVL for Linux * libefi: GRUB And these libraries are potential conflicts: * libshare: the Linux Mount Manager * libunicode: Perl and Python Recompose these five ZoL components into the four libraries that are conventionally provided by Solaris and FreeBSD systems: + libnvpair + libuutil + libzpool + libzfs This change resolves the name conflict, makes ZoL more compatible with existing software that uses autotools to detect ZFS, and allows pkg-zfs to better reflect the official Debian kFreeBSD packaging. Signed-off-by: Brian Behlendorf <[email protected]> Closes: #430
* Allow setting bootfs on any poolRichard Laager2012-01-171-3/+8
| | | | | | | | | | The vdev_is_bootable() restrictions are no longer necessary with recent GRUB2 code. FreeBSD has implemented the same change, except that I moved the Solaris comment to be inside the #ifdef __sun__ block. Signed-off-by: Brian Behlendorf <[email protected]> Issue #317
* Reduce number of zio free threadsNed Bass2012-01-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As described in Issue #458 and #258, unlinking large amounts of data can cause the threads in the zio free wait queue to start spinning. Reducing the number of z_fr_iss threads from a fixed value of 100 to 1 per cpu signficantly reduces contention on the taskq spinlock and improves throughput. Instrumenting the taskq code showed that __taskq_dispatch() can spend a long time holding tq->tq_lock if there are a large number of threads in the queue. It turns out the time spent in wake_up() scales linearly with the number of threads in the queue. When a large number of short work items are dispatched, as seems to be the case with unlink, the worker threads drain the queue faster than the dispatcher can fill it. They then all pile into the work wait queue to wait for new work items. So if 100 threads are in the queue, wake_up() takes about 100 times as long, and the woken threads have to spin until the dispatcher releases the lock. Reducing the number of threads helps with the symptoms, but doesn't get to the root of the problem. It would seem that wake_up() shouldn't scale linearly in time with queue depth, particularly if we are only trying to wake up one thread. In that vein, I tried making all of the waiting processes exclusive to prevent the scheduler from iterating over the entire list, but I still saw the linear time scaling. So further investigation is needed, but in the meantime reducing the thread count is an easy workaround. Signed-off-by: Brian Behlendorf <[email protected]> Issue #258 Issue #458
* Increase link count limit to 2^31-1Brian Behlendorf2012-01-131-3/+3
| | | | | | | | | | Originally, the per-file link limit was set to 65536 because the exact Linux VFS limit was unclear. Internally ZFS is able to support 64-bit link counts. After a more careful investigation the limit can be safely raised to 2^31-1. Signed-off-by: Brian Behlendorf <[email protected]> Closes #514
* Run ZFS_AC_PACMAN only if $VENDOR is "arch"Prakash Surya2012-01-132-82/+87
| | | | | | | | | | | Unfortunately, Arch's package manager `pacman` shares it's name with a popular arcade video game. Thus, in order to refrain from executing the video game when we mean to execute the package manager, ZFS_AC_PACMAN is now only run when $VENDOR is determined to be "arch". Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #517
* Add overlay(-O) mount option supportSuman Chakravartula2012-01-123-4/+15
| | | | | | | | | | | | | | | | | | | Linux supports mounting over non-empty directories by default. In Solaris this is not the case and -O option is required for zfs mount to mount a zfs filesystem over a non-empty directory. For compatibility, I've added support for -O option to mount zfs filesystems over non-empty directories if the user wants to, just like in Solaris. I've defined MS_OVERLAY to record it in the flags variable if the -O option is supplied. The flags variable passes through a few functions and its checked before performing the empty directory check in zfs_mount function. If -O is given, the check is not performed. Signed-off-by: Brian Behlendorf <[email protected]> Closes #473
* Apply the ZoL coding standard to zpl_xattr.cDarik Horn2012-01-121-17/+16
| | | | | | | Make the indenting in the zpl_xattr.c file consistent with the Sun coding standard by removing soft tabs. Signed-off-by: Brian Behlendorf <[email protected]>
* Linux 3.2 compat, security_inode_init_security()Brian Behlendorf2012-01-125-3/+233
| | | | | | | | | | | | | | | | | | The security_inode_init_security() API has been changed to include a filesystem specific callback to write security extended attributes. This was done to support the initialization of multiple LSM xattrs and the EVM xattr. This change updates the code to use the new API when it's available. Otherwise it falls back to the previous implementation. In addition, the ZFS_AC_KERNEL_6ARGS_SECURITY_INODE_INIT_SECURITY autoconf test has been made more rigerous by passing the expected types. This is done to ensure we always properly the detect the correct form for the security_inode_init_security() API. Signed-off-by: Brian Behlendorf <[email protected]> Closes #516
* Treat /dev/vd* as whole disksRichard Laager2012-01-111-0/+6
| | | | | | | Correctly detect /dev/vd devices as whole disks and attempt to create an EFI partition table. Signed-off-by: Brian Behlendorf <[email protected]>
* Avoid using awk in the zpool_id script.Darik Horn2012-01-111-4/+13
| | | | | | | | Some implementations of `awk` incorrectly parse the \< and \> regex symbols, so use a `while read` loop and regular globbing instead. Signed-off-by: Brian Behlendorf <[email protected]> Closes: #259
* Linux 3.1 compat, super_block->s_shrinkBrian Behlendorf2012-01-1165-51/+858
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Linux 3.1 kernel has introduced the concept of per-filesystem shrinkers which are directly assoicated with a super block. Prior to this change there was one shared global shrinker. The zfs code relied on being able to call the global shrinker when the arc_meta_limit was exceeded. This would cause the VFS to drop references on a fraction of the dentries in the dcache. The ARC could then safely reclaim the memory used by these entries and honor the arc_meta_limit. Unfortunately, when per-filesystem shrinkers were added the old interfaces were made unavailable. This change adds support to use the new per-filesystem shrinker interface so we can continue to honor the arc_meta_limit. The major benefit of the new interface is that we can now target only the zfs filesystem for dentry and inode pruning. Thus we can minimize any impact on the caching of other filesystems. In the context of making this change several other important issues related to managing the ARC were addressed, they include: * The dnlc_reduce_cache() function which was called by the ARC to drop dentries for the Posix layer was replaced with a generic zfs_prune_t callback. The ZPL layer now registers a callback to drop these dentries removing a layering violation which dates back to the Solaris code. This callback can also be used by other ARC consumers such as Lustre. arc_add_prune_callback() arc_remove_prune_callback() * The arc_reduce_dnlc_percent module option has been changed to arc_meta_prune for clarity. The dnlc functions are specific to Solaris's VFS and have already been largely eliminated already. The replacement tunable now represents the number of bytes the prune callback will request when invoked. * Less aggressively invoke the prune callback. We used to call this whenever we exceeded the arc_meta_limit however that's not strictly correct since it results in over zeleous reclaim of dentries and inodes. It is now only called once the arc_meta_limit is exceeded and every effort has been made to evict other data from the ARC cache. * More promptly manage exceeding the arc_meta_limit. When reading meta data in to the cache if a buffer was unable to be recycled notify the arc_reclaim thread to invoke the required prune. * Added arcstat_prune kstat which is incremented when the ARC is forced to request that a consumer prune its cache. Remember this will only occur when the ARC has no other choice. If it can evict buffers safely without invoking the prune callback it will. * This change is also expected to resolve the unexpect collapses of the ARC cache. This would occur because when exceeded just the arc_meta_limit reclaim presure would be excerted on the arc_c value via arc_shrink(). This effectively shrunk the entire cache when really we just needed to reclaim meta data. Signed-off-by: Brian Behlendorf <[email protected]> Closes #466 Closes #292
* Move Arch Linux's VENDOR check above Ubuntu'sPrakash Surya2011-12-192-4/+4
| | | | | | | | | | | | | | | If the lsb-release package is installed on an Arch Linux distribution, the configure step will incorrectly detect the running distribution as Ubuntu. This is a result of both distributions providing an /etc/lsb-release file, and the Ubuntu VENDOR check being performed first. Since the Arch Linux test check's for a file more specific to the Arch Linux distribution, moving Arch Linux's VENDOR check above Unbuntu's check provides a quick and easy solution. Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]>
* Add LIBSELINUX to mount_zfs_LDFLAGS.Darik Horn2011-12-162-2/+2
| | | | | | | | | | | | | Regenerating the autotools configuration on Debian and Ubuntu systems causes compilation to fail with this error message: cmd/mount_zfs/../../cmd/mount_zfs/mount_zfs.c:403: undefined reference to `is_selinux_enabled' In the automake template, set "mount_zfs_LDFLAGS = ... $(LIBSELINUX)" so that the /sbin/mount.zfs utility is linked to libselinux. Signed-off-by: Brian Behlendorf <[email protected]>
* Linux 3.2 compat: set_nlink()Darik Horn2011-12-1660-1/+228
| | | | | | | | | | | Directly changing inode->i_nlink is deprecated in Linux 3.2 by commit SHA: bfe8684869601dacfcb2cd69ef8cfd9045f62170 Use the new set_nlink() kernel function instead. Signed-off-by: Brian Behlendorf <[email protected]> Closes: #462
* Update the character class in the zpool man page.Darik Horn2011-12-161-1/+1
| | | | | | | | ZoL and all Solaris derivatives allow pool names to contain the colon and space characters. Update the man page to reflect current behavior. Signed-off-by: Brian Behlendorf <[email protected]> Closes: #438
* Add make rule for building Arch Linux packagesPrakash Surya2011-12-1461-21/+589
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added the necessary build infrastructure for building packages compatible with the Arch Linux distribution. As such, one can now run: $ ./configure $ make pkg # Alternatively, one can run 'make arch' as well on the Arch Linux machine to create two binary packages compatible with the pacman package manager, one for the zfs userland utilities and another for the zfs kernel modules. The new packages can then be installed by running: # pacman -U $package.pkg.tar.xz In addition, source-only packages suitable for an Arch Linux chroot environment or remote builder can also be build using the 'sarch' make rule. NOTE: Since the source dist tarball is created on the fly from the head of the build tree, it's MD5 hash signature will be continually influx. As a result, the md5sum variable was intentionally omitted from the PKGBUILD files, and the '--skipinteg' makepkg option is used. This may or may not have any serious security implications, as the source tarball is not being downloaded from an outside source. Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #491
* Illumos #734: Use taskq_dispatch_ent() interfaceGarrett D'Amore2011-12-145-41/+134
| | | | | | | | | | | | | | | | | | | | | | | | | | | It has been observed that some of the hottest locks are those of the zio taskqs. Contention on these locks can limit the rate at which zios are dispatched which limits performance. This upstream change from Illumos uses new interface to the taskqs which allow them to utilize a prealloc'ed taskq_ent_t. This removes the need to perform an allocation at dispatch time while holding the contended lock. This has the effect of improving system performance. Reviewed by: Albert Lee <[email protected]> Reviewed by: Richard Lowe <[email protected]> Reviewed by: Alexey Zaytsev <[email protected]> Reviewed by: Jason Brian King <[email protected]> Reviewed by: George Wilson <[email protected]> Reviewed by: Adam Leventhal <[email protected]> Approved by: Gordon Ross <[email protected]> References to Illumos issue: https://www.illumos.org/issues/734 Ported-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #482
* Set zvol_major/zvol_threads permissionsBrian Behlendorf2011-12-071-2/+2
| | | | | | | | | | | | The zvol_major and zvol_threads module options were being created with 0 permission bits. This prevented them from being listed in the /sys/module/zfs/parameters/ directory, although they were visible in `modinfo zfs`. This patch fixes the issue by updating the permission bits to 0444. For the moment these options must be read-only because they are used during module initialization. Signed-off-by: Brian Behlendorf <[email protected]> Issue #392
* Update default ARC memory limitsBrian Behlendorf2011-12-051-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the upstream OpenSolaris ZFS code the maximum ARC usage is limited to 3/4 of memory or all but 1GB, whichever is larger. Because of how Linux's VM subsystem is organized these defaults have proven to be too large which can lead to stability issues. To avoid making everyone manually tune the ARC the defaults are being changed to 1/2 of memory or all but 4GB. The rational for this is as follows: * Desktop Systems (less than 8GB of memory) Limiting the ARC to 1/2 of memory is desirable for desktop systems which have highly dynamic memory requirements. For example, launching your web browser can suddenly result in a demand for several gigabytes of memory. This memory must be reclaimed from the ARC cache which can take some time. The user will experience this reclaim time as a sluggish system with poor interactive performance. Thus in this case it is preferable to leave the memory as free and available for immediate use. * Server Systems (more than 8GB of memory) Using all but 4GB of memory for the ARC is preferable for server systems. These systems often run with minimal user interaction and have long running daemons with relatively stable memory demands. These systems will benefit most by having as much data cached in memory as possible. These values should work well for most configurations. However, if you have a desktop system with more than 8GB of memory you may wish to further restrict the ARC. This can still be accomplished by setting the 'zfs_arc_max' module option. Additionally, keep in mind these aren't currently hard limits. The ARC is based on a slab implementation which can suffer from memory fragmentation. Because this fragmentation is not visible from the ARC it may believe it is within the specified limits while actually consuming slightly more memory. How much more memory get's consumed will be determined by how badly fragmented the slabs are. In the long term this can be mitigated by slab defragmentation code which was OpenSolaris solution. Or preferably, using the page cache to back the ARC under Linux would be even better. See issue #75 for the benefits of more tightly integrating with the page cache. This change also fixes a issue where the default ARC max was being set incorrectly for machines with less than 2GB of memory. The constant in the arc_c_max comparison must be explicitly cast to a uint64_t type to prevent overflow and the wrong conditional branch being taken. This failure was typically observed in VMs which are commonly created with less than 2GB of memory. Signed-off-by: Brian Behlendorf <[email protected]> Issue #75
* Quote variables in the zfs.lsb script.Darik Horn2011-12-051-16/+16
| | | | | | | | | For consistency and safety, quote all variables in the zfs.lsb script. This protects in the unlikely case that any of the file names contain whitespace. Signed-off-by: Brian Behlendorf <[email protected]> Issue #439
* Source /etc/default/zfs after setting defaults.Darik Horn2011-12-051-5/+3
| | | | | | | | | | | Let the administrator override all script variables by sourcing the /etc/default/zfs file after the default values are set. The spelling mistake in the old path name makes it unlikely that this bug affected any users. Signed-off-by: Brian Behlendorf <[email protected]> Closes: #371
* Demote the whackbang in the zpool_id script.Darik Horn2011-12-051-1/+1
| | | | | | | | | The zpool_id script is posixly correct and does not use bash features, so change its whackbang from /bin/bash to /bin/sh. Debian policy also stipulates that system scripts be dash compatible. Signed-off-by: Brian Behlendorf <[email protected]>
* Demote egrep to grep in the zpool_id script.Darik Horn2011-12-051-1/+1
| | | | | | | Direct invocation of GNU egrep is deprecated by its man page, and the its argument in the zpool_id script is not an extended expression. Signed-off-by: Brian Behlendorf <[email protected]>
* Quote variables in the zpool_id script.Darik Horn2011-12-051-12/+12
| | | | | | | | | | | For consistency and safety, quote all variables in the zpool_id script. This accomodates a `-c CONFIG` parameter value with whitespace in the path name. Also fix a typo in the usage synopsis for `-h`. Signed-off-by: Brian Behlendorf <[email protected]> Issue #439
* Support path_id changes in udev 174.Darik Horn2011-12-051-1/+9
| | | | | | | | The /lib/udev/path_id helper became a builtin command in the udev 174 release, so test whether path_id is external in the zpool_id script. Signed-off-by: Brian Behlendorf <[email protected]> Closes: #429
* Added comments for libshare's NFS functions.Gunnar Beutner2011-12-051-1/+57
| | | | | | | Some of the functions' purpose wasn't immediately obvious without additional explanations. This commit adds these missing comments. Signed-off-by: Brian Behlendorf <[email protected]>
* Fix configure tests to play nice with GCC 4.6Prakash Surya2011-11-2912-0/+99
| | | | | | | | | | | | As of GCC 4.6, specific kernel 2.6.32 header files do not compile cleanly without warnings. One specific example of this is the arch/x86/include/asm/percpu.h file. Thus, a few of the configure tests were getting hung up on this and the '-Wno-unsued-but-set-variables' compile option had to be introduced. Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #459
* Allow xattrs on symlinksBrian Behlendorf2011-11-291-0/+4
| | | | | | | | | | | | | | | | | | | | | | The Solaris version of ZFS does not allow xattrs to be set on symlinks due to the way they implemented the attropen() system call. Linux however implements xattrs through the lgetxattr() and lsetxattr() system calls which do not have this limitation. The only reason this hasn't always worked under ZFS on Linux is that the xattr handlers were not registered for symlink type inodes. This was done simply to be consistent with the Solaris behavior. Upon futher reflection I believe this should be allowed under Linux. The only ill effect would be that the xattrs on symlinks will not be visible when the pool is imported on a Solaris system. This also has the benefit that it allows for SELinux style security xattr labeling which expects to be able to set xattrs on all inode types. Signed-off-by: Brian Behlendorf <[email protected]> Closes #272
* Implement SA based xattrsBrian Behlendorf2011-11-2812-76/+425
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current ZFS implementation stores xattrs on disk using a hidden directory. In this directory a file name represents the xattr name and the file contexts are the xattr binary data. This approach is very flexible and allows for arbitrarily large xattrs. However, it also suffers from a significant performance penalty. Accessing a single xattr can requires up to three disk seeks. 1) Lookup the dnode object. 2) Lookup the dnodes's xattr directory object. 3) Lookup the xattr object in the directory. To avoid this performance penalty Linux filesystems such as ext3 and xfs try to store the xattr as part of the inode on disk. When the xattr is to large to store in the inode then a single external block is allocated for them. In practice most xattrs are small and this approach works well. The addition of System Attributes (SA) to zfs provides us a clean way to make this optimization. When the dataset property 'xattr=sa' is set then xattrs will be preferentially stored as System Attributes. This allows tiny xattrs (~100 bytes) to be stored with the dnode and up to 64k of xattrs to be stored in the spill block. If additional xattr space is required, which is unlikely under Linux, they will be stored using the traditional directory approach. This optimization results in roughly a 3x performance improvement when accessing xattrs which brings zfs roughly to parity with ext4 and xfs (see table below). When multiple xattrs are stored per-file the performance improvements are even greater because all of the xattrs stored in the spill block will be cached. However, by default SA based xattrs are disabled in the Linux port to maximize compatibility with other implementations. If you do enable SA based xattrs then they will not be visible on platforms which do not support this feature. ---------------------------------------------------------------------- Time in seconds to get/set one xattr of N bytes on 100,000 files ------+--------------------------------+------------------------------ | setxattr | getxattr bytes | ext4 xfs zfs-dir zfs-sa | ext4 xfs zfs-dir zfs-sa ------+--------------------------------+------------------------------ 1 | 2.33 31.88 21.50 4.57 | 2.35 2.64 6.29 2.43 32 | 2.79 30.68 21.98 4.60 | 2.44 2.59 6.78 2.48 256 | 3.25 31.99 21.36 5.92 | 2.32 2.71 6.22 3.14 1024 | 3.30 32.61 22.83 8.45 | 2.40 2.79 6.24 3.27 4096 | 3.57 317.46 22.52 10.73 | 2.78 28.62 6.90 3.94 16384 | n/a 2342.39 34.30 19.20 | n/a 45.44 145.90 7.55 65536 | n/a 2941.39 128.15 131.32* | n/a 141.92 256.85 262.12* Legend: * ext4 - Stock RHEL6.1 ext4 mounted with '-o user_xattr'. * xfs - Stock RHEL6.1 xfs mounted with default options. * zfs-dir - Directory based xattrs only. * zfs-sa - Prefer SAs but spill in to directories as needed, a trailing * indicates overflow in to directories occured. NOTE: Ext4 supports 4096 bytes of xattr name/value pairs per file. NOTE: XFS and ZFS have no limit on xattr name/value pairs per file. NOTE: Linux limits individual name/value pairs to 65536 bytes. NOTE: All setattr/getattr's were done after dropping the cache. NOTE: All tests were run against a single hard drive. Signed-off-by: Brian Behlendorf <[email protected]> Issue #443
* In autoconf v2.68, AC_LANG_PROGRAM must be quotedPrakash Surya2011-11-284-7/+7
| | | | | | | | | | | | | | | | | This change updates the AC_LANG_PROGRAM autoconf macro invocations to be wrapped in quotes. As of autoconf version 2.68, the quotes are necessary to prevent warnings from appearing. Specifically, the autoconf v2.68 Forward Porting Notes specifies: It is important to note that you need to ensure that the call to AC_LANG_SOURCE is quoted and not expanded, otherwise that will cause the warning to appear nonetheless. Finally, because of the additional quoting we can drop the extra quotas used by the ZFS_AC_CONFIG_USER_STACK_GUARD autoconf check. Signed-off-by: Brian Behlendorf <[email protected]> Closes #464
* Allow leading digits in userquota/groupquota namesSuman Chakravartula2011-11-211-39/+28
| | | | | | | | | | | | | | | | | | | While setting/getting userquota and groupquota properties, the input was not treated as a possible username or groupname if it had a leading digit. While useradd in linux recommends the regexp [a-z_][a-z0-9_-]*[$]? , it is not enforced. This causes problem for usernames with leading digits in them. We need to be able to support getting and setting properties for this unconventional but possible input category I've updated the code to validate the username or groupname directly via the API. Also, note that I moved this validation to the beginning before the check for SID names with @. This also supports usernames with @ character in them which are valid. Only when input with @ is not a valid username, it is interpreted as a potential SID name. Signed-off-by: Brian Behlendorf <[email protected]> Closes #428
* Limit maximum ashift value to 12Brian Behlendorf2011-11-112-2/+2
| | | | | | | | | | | | | | | | | While we initially allowed you to set your ashift as large as 17 (SPA_MAXBLOCKSIZE) that is actually unsafe. What wasn't considered at the time is that each uberblock written to the vdev label ring buffer will be of this size. Now the buffer is statically sized to 128k and we need to be able to fit several uberblocks in it. With a large ashift that becomes a problem. Therefore I'm reducing the maximum configurable ashift value to 12. This is large enough for the 4k sector drives and small enough that we can still keep the most recent 32 uberblock in the vdev label ring buffer. Signed-off-by: Brian Behlendorf <[email protected]> Closes #425
* Fix depmod warningBrian Behlendorf2011-11-101-4/+4
| | | | | | | | | | | | The depmod utility from module-init-tools 3.12-pre3 generates a warning when the -e option is used without -E or -F. This was observed under OpenSuse 11.4. To resolve the issue when the exact System.map-* for your kernel cannot be found fallback to a generic safe '/sbin/depmod -a'. WARNING: -e needs -E or -F Signed-off-by: Brian Behlendorf <[email protected]>