summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Add additional state TXG_STATE_WAIT_FOR_SYNC for txg.Igor Lvovsky2014-01-233-8/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | In several cases when digging into kstats we can found two txgs in SYNC state, e.g. txg birth state nreserved nread nwritten ... 985452 258127184872561 C 0 373948416 2376272384 ... 985453 258129016180616 C 0 378173440 28793344 ... 985454 258129016271523 S 0 0 0 ... 985455 258130864245986 S 0 0 0 ... 985456 258130867458851 O 0 0 0 ... However only first txg (985454) is really syncing at this moment. The other one (985455) marked as SYNCED is actually in a post-QUIESCED state and waiting to start sync. So, the new TXG_STATE_WAIT_FOR_SYNC state between TXG_STATE_QUIESCED and TXG_STATE_SYNCED was added to reveal this situation. txg birth state nreserved nread nwritten ... 1086896 235261068743969 C 0 163577856 8437248 ... 1086897 235262870830801 C 0 280625152 822594048 ... 1086898 235264172219064 S 0 0 0 ... 1086899 235264936134407 W 0 0 0 ... 1086900 235264936296156 O 0 0 0 ... Signed-off-by: Igor Lvovsky <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #2075
* Use enum type(zfetch_dirn_t) insteadShen Yan2014-01-231-2/+4
| | | | | | | Fix code with zfetch_dirn_t, which is more readable and clear. Signed-off-by: Brian Behlendorf <[email protected]> Closes #2068
* Allow chown/chgrp when no ACL SAs exist.Tim Chase2014-01-231-0/+11
| | | | | | | | | | | | | | | From the comment in the commit: Some ZFS implementations (ZEVO) create neither a ZNODE_ACL nor a DACL_ACES SA in which case ENOENT is returned from zfs_acl_node_read() when the SA can't be located. Allow chown/chgrp to succeed in these cases rather than returning an error that makes no sense in the context of the caller. Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue zfs-osx/zfs#86 Closes #1911 Closes #2029
* vdev_file_io_start() to use taskq_dispatch(TQ_PUSHPAGE)Ned Bass2014-01-231-1/+1
| | | | | | | | | | | | The vdev_file_io_start() function may be processing a zio that the txg_sync thread is waiting on. In this case it is not safe to perform memory allocations that may generate new I/O since this could cause a deadlock. To avoid this, call taskq_dispatch() with TQ_PUSHPAGE instead of TQ_SLEEP. Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #1928
* Remove incorrect use of EXTRA_DIST for man pagesBrian Behlendorf2014-01-173-7/+4
| | | | | | | | | | Setting the 'dist_' prefix is the correct way to instruct Automake to include these files in the distribution. The EXTRA_DIST variable is reserved for files which are not covered by the automatic rules. http://www.gnu.org/software/automake/manual/automake.html#Basics Signed-off-by: Brian Behlendorf <[email protected]>
* vdev_id: support per-channel slot mappingsNed Bass2014-01-173-81/+65
| | | | | | | | | | | | | | | | | The vdev_id udev helper currently applies slot renumbering rules to every channel (JBOD) in the system. This is too inflexible for systems with non-homogeneous storage topologies. The "slot" keyword now takes an optional third parameter which names a channel to which the mapping will apply. If the third parameter is omitted then the rule applies to all channels. The first-specified rule that can match a slot takes precedence. Therefore a channel-specific rule for a given slot should generally appear before a generic rule for the same slot number. In this way a custom slot mapping can be applied to a particular channel and a default mapping applied to the rest. Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2056
* Use long holds in zvol_set_volsize()Brian Behlendorf2014-01-141-55/+67
| | | | | | | | | | | | | | | | | | | Under Linux the zvol_set_volsize() function was originally written to use dmu_objset_hold()/dmu_objset_rele(). Subsequently, the dmu_objset_own()/dmu_objset_disown() interfaces were added but the ZVOL code wasn't updated to take advantage of them. This was never an issue but after the dsl_pool_config changes the code now takes the config lock twice. The cleanest solution is to shift to using dmu_objset_own() which takes a long hold on the dataset and does not hold the dsl pool lock. This patch also slightly restructures the existing code such that it more closely resembles the upstream Illumos code. Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2039
* Disable GCCs aggressive loop optimizationBrian Behlendorf2014-01-144-2/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GCC >+ 4.8's aggressive loop optimization breaks some of the iterators over the dn_blkptr[] pseudo-array in dnode_phys. Since dn_blkptr[] is defined as a single-element array, GCC believes an iterator can only access index 0 and will unroll the loop into a single iteration. One way to resolve the issue would be to cast the array to a pointer and fix all the iterators that might break. The only loop where it is known to cause a problem is this loop in dmu_objset_write_ready(): for (i = 0; i < dnp->dn_nblkptr; i++) bp->blk_fill += dnp->dn_blkptr[i].blk_fill; In the common case where dn_nblkptr is 3, the loop is only executed a single time and "i" is equal to 1 following the loop. The specific breakage caused by this problem is that the blk_fill of root block pointers wouldn't be set properly when more than one blkptr is in use (when no indrect blocks are needed). The simple reproducing sequence is: zpool create tank /tank.img zdb -ddddd tank 0 Notice that "fill=31", however, there are two L0 indirect blocks with "F=31" and "F=5". The fill count should be 36 rather than 31. This problem causes an assert to be hit in a simple "zdb tank" when built with --enable-debug. However, this approach was not taken because we need to be absolutely sure we catch all instances of this unwanted optimization. Therefore, the build system has been updated to detect if GCC supports the aggressive loop optimization. If it does the optimization will be explicitly disabled using the -fno-aggressive-loop-optimization option. Original-fix-by: Tim Chase <[email protected]> Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2010 Closes #2051
* Properly link zpool command to libblkidRichard Yao2014-01-141-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 31fc19399e597e3391f19f1392ab120f1de0d5f2 incorrectly removed $(LIBBLKID) from cmd/zpool/Makefile.am. This meant that the toolchain was not given -lblkid, which resulted in the following build failure on Ubuntu 13.10: /usr/bin/ld: zpool_vdev.o: undefined reference to symbol 'blkid_put_cache@@BLKID_1.0' /lib/x86_64-linux-gnu/libblkid.so.1: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status That commit reworked various Makefile.am to follow best practices, so we reintroduce $(LIBBLKID) in a manner consistent with that, rather than explicitly reverting the change. Reproduction of this issue was done on a Gentoo Linux system by executing the following commands: zfs create -o mountpoint=/mnt/ubuntu-13.10 rpool/ROOT/ubuntu-13.10 debootstrap --variant=buildd --arch amd64 saucy /mnt/ubuntu-13.10 http://archive.ubuntu.com/ubuntu/ mount -o bind /dev /mnt/ubuntu-13.10/dev/ mount -o bind /proc/ /mnt/ubuntu-13.10/proc/ mount -o bind /sys/ /mnt/ubuntu-13.10/sys/ cp /etc/resolv.conf /mnt/ubuntu-13.10/etc/ (cd /mnt/ubuntu-13.10/root/ && git clone git://github.com/zfsonlinux/zfs.git) chroot /mnt/ubuntu-13.10/ apt-get install git autoconf libtool zlib1g-dev uuid-dev libblkid-dev \#apt-get install alien fakeroot vim cd /root/zfs ./autogen.sh ./configure --with-config=user --prefix=/usr make That will create a Ubuntu 13.10 chroot, fetch the sources and build test. At this point, cmd/zpool/Makefile.am was modified and the following commands were run to verify that the build issue was resolved: git clean -xdf ./autogen.sh ./configure --with-config=user --prefix=/usr make Although it is not shown here, the absence of libblkid-dev enables ZFS to build successfully without the patch. This could explain how this escaped detection until recently. A test without libblkid-dev was done to verify that the patch did not cause a regression in the absence of libblkid: apt-get remove libblkid-dev git clean -xdf ./autogen.sh ./configure --with-config=user --prefix=/usr make Additionally, the commands themselves were tested against my live system from within the chroot to ensure basic functionality. My live system had corresponding kernel modules already installed and basic commands such as `zpool list` and `zfs list` worked without incident. Lastly, this patch was also build tested on Gentoo Linux, where it caused no problems. At time of writing, these steps can be used to reproduce these results on any modern Linux system that has debootstrap installed. On Gentoo, installing debootstrap can be done with `emerge dev-util/debootstrap`. The current ZFSOnLinux HEAD revision as of writing is fd23720ae14dca926800ae70e6c8f4b4f82efc08. Once this is fixed in HEAD, either that revision or another before this fix and after 31fc19399e597e3391f19f1392ab120f1de0d5f2 will be needed to reproduce this issue. Lastly, it remains to be seen why the toolchains on the systems performing regression tests did not catch this. This is not a ZFS-specific issue, but it is something that we will want to explore in the future. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2038
* Prevent duplicate mnttab cache entriesBrian Behlendorf2014-01-141-0/+13
| | | | | | | | | | | | | Under Linux its possible to mount the same filesystem multiple times in the namespace. This can be done either with bind mounts or simply with multiple mount points. Unfortunately, the mnttab cache code is implemented using an AVL tree which does not support duplicate entries. To avoid this issue this patch updates the code to check for a duplicate entry before adding a new one. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Michael Martin <[email protected]> Closes #2041
* Drain iput taskq outside z_teardown_lockBrian Behlendorf2014-01-091-8/+8
| | | | | | | | | | | | | | It's unsafe to drain the iput taskq while holding the z_teardown_lock as a writer. This is because when the last reference on an inode is dropped it may still have pages which need to be written to disk. This will be done through zpl_writepages which will acquire the z_teardown_lock as a reader in ZFS_ENTER. Therefore, if we're holding the lock as a writer in zfs_sb_teardown the unmount will deadlock. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Chris Dunlop <[email protected]> Closes #1988
* Force LZ4_FORCE_SW_BITCOUNT for SparcBrian Behlendorf2014-01-091-0/+3
| | | | | | | | | | | | | | This change was proposed for Sparc but it's not clear to me why it's required. Proper support exists in the lz4 code to detect the endianness and the required builtins are available for gcc. Still I'm including the patch because it will only impact Sparc and it may resolve a case which hasn't occured to me. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: marku89 <[email protected]> Issue #1700
* Fix zfs_getattr_fast typesBrian Behlendorf2014-01-091-1/+6
| | | | | | | | | | | | | On Sparc sp->blksize will be a 64-bit value which is then cast incorrectly to a 32-bit value. For big endian systems this results in an incorrect value for sp->blksize. To resolve the problem local variables of the correct size are used and then assigned to sp->blksize. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: marku89 <[email protected]> Issue #1700
* Fix nvlist 'Bus Error' for SparcBrian Behlendorf2014-01-091-2/+4
| | | | | | | | | | | | | The mis-aligned memory accesses in nvpair_native_embedded() and nvpair_native_embedded_array() will cause a 'Bus Error' for architectures such as Sparc which not fully byte addressible. To avoid this issue care is taken to avoid dereferencing the potentially mis-aligned packed nvlist_t. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: marku89 <[email protected]> Issue #1700
* Use local variable to read zp->z_modeBrian Behlendorf2014-01-092-2/+5
| | | | | | | | | | | | | | | | | | When accessing the zp->z_mode through the SA bulk interface we expect that 64-bits are available to hold the result. However, on 32-bit platforms mode_t will only be 32-bits so we cannot pass it to SA_ADD_BULK_ATTR(). Instead a local uint64_t variable must be used and the result assigned to zp->z_mode. This went unnoticed on 32-bit little endian platforms because the bytes happen to end up in the correct 32-bits. But on big endian platforms like Sparc the zp->z_mode will always end up set to zero. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: marku89 <[email protected]> Issue #1700
* Define the needed ISA types for SparcBrian Behlendorf2014-01-092-2/+31
| | | | | | | | | | Add the minimum required ISA types to support the Sparc architecture. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: marku89 <[email protected]> Issue #1700
* Add ddt, ddt_entry, and l2arc_hdr cachesJohn Layman2014-01-074-16/+42
| | | | | | | | | | | Back the allocations for ddt tables+entries and l2arc headers with kmem caches. This will reduce the cost of allocating these commonly used structures and allow for greater visibility of them through the /proc/spl/kmem/slab interface. Signed-off-by: John Layman <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1893
* Remove unconditional sharetab updateBrian Behlendorf2014-01-071-8/+0
| | | | | | | | | | | | | | | Removes the unconditional sharetab update when running any zfs command. This means the sharetab might become out of date if users are manually adding/removing shares with exportfs. But we shouldn't punish all callers to zfs in order to handle that unlikely case. In the unlikely event we observe issues because of this it can always be added back to just the share/unshare call paths where we need an up to date sharetab. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Turbo Fredriksson <[email protected]> Signed-off-by: Chris Dunlop <[email protected]> Issue #845
* Enable /etc/mtab cache to improve performanceBrian Behlendorf2014-01-071-1/+1
| | | | | | | | | | | | | | | | | | | | | Re-enable the /etc/mtab cache to prevent the zfs command from having to repeatedly open and read from the /etc/mtab file. Instead an AVL tree of the mounted filesystems is created and used to vastly speed up lookups. This means that if non-zfs filesystems are mounted concurrently the 'zfs mount' will not immediately detect them. In practice that will rarely happen and even if it does the absolute worst case would be a failed mount. This was originally disabled out of an abundance of paranoia. NOTE: There may still be some parts of the code which do not consult the mtab cache. They should be updated to check the mtab cache as they as discovered to be a problem. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Turbo Fredriksson <[email protected]> Signed-off-by: Chris Dunlop <[email protected]> Issue #845
* Add UNSHARING of filesystems and EXPORTING poolsTurbo Fredriksson2014-01-071-0/+11
| | | | | | | | | | As a 'stop' action ensure the filesystem is unshared before it is unmounted, just in case. Additionally, export the pool so it may be cleanly imported by a different host. Signed-off-by: Turbo Fredriksson <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2003
* Fix the creation of ZPOOL_HIST_CMD pool history entries.Tim Chase2014-01-072-9/+21
| | | | | | | | | | | | | | | | | | | | | Move the libzfs_fini() after the zpool_log_history() call so the ZPOOL_HIST_CMD entry can get written. Fix the handling of saved_poolname in zfsdev_ioctl() which was broken as part of the stack-reduction work in a16878805388c4d96cb8a294de965071d138a47b. Since ZoL destroys the TSD data in which the previously successful ioctl()'s pool name is stored following every vop, the ZFS_IOC_LOG_HISTORY ioctl has a very important restriction: it can only successfully write a long entry following a successful ioctl() if no intervening vops have been performed. Some of zfs subcommands do perform intervening vops and to do the logging themselves. At the moment, the "create" and "clone" subcommands have been modified appropriately. Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1998
* Properly handle updates of variably-sized SA entries.Tim Chase2013-12-201-6/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | During the update process in sa_modify_attrs(), the sizes of existing variably-sized SA entries are obtained from sa_lengths[]. The case where a variably-sized SA was being replaced neglected to increment the index into sa_lengths[], so subsequent variable-length SAs would be rewritten with the wrong length. This patch adds the missing increment operation so all variably-sized SA entries are stored with their correct lengths. Previously, a size-changing update of a variably-sized SA that occurred when there were other variably-sized SAs in the bonus buffer would cause the subsequent SAs to be corrupted. The most common case in which this would occur is when a mode change caused the ZPL_DACL_ACES entry to change size when a ZPL_DXATTR (SA xattr) entry already existed. The following sequence would have caused a failure when xattr=sa was in force and would corrupt the bonus buffer: open(filename, O_WRONLY | O_CREAT, 0600); ... lsetxattr(filename, ...); /* create xattr SA */ chmod(filename, 0650); /* enlarges the ACL */ Signed-off-by: Chris Dunlop <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1978
* Register correct handlers for nvlist_{dup,pack,unpack}Brian Behlendorf2013-12-201-38/+19
| | | | | | | | | | | This change is related to commit 81eaf15 which ensured the correct allocation handlers were installed for nvlist_alloc(). The nvlist functions nvlist_dup(), nvlist_pack(), and nvlist_unpack() suffer from the same issue and have been updated accordingly. Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #1937
* Add full SELinux supportMatthew Thode2013-12-1910-82/+134
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Four new dataset properties have been added to support SELinux. They are 'context', 'fscontext', 'defcontext' and 'rootcontext' which map directly to the context options described in mount(8). When one of these properties is set to something other than 'none'. That string will be passed verbatim as a mount option for the given context when the filesystem is mounted. For example, if you wanted the rootcontext for a filesystem to be set to 'system_u:object_r:fs_t' you would set the property as follows: $ zfs set rootcontext="system_u:object_r:fs_t" storage-pool/media This will ensure the filesystem is automatically mounted with that rootcontext. It is equivalent to manually specifying the rootcontext with the -o option like this: $ zfs mount -o rootcontext=system_u:object_r:fs_t storage-pool/media By default all four contexts are set to 'none'. Further information on SELinux contexts is detailed in mount(8) and selinux(8) man pages. Signed-off-by: Matthew Thode <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Richard Yao <[email protected]> Closes #1504
* cstyle: Resolve C style issuesMichael Kjorling2013-12-18165-1945/+2129
| | | | | | | | | | | | | | | | | | The vast majority of these changes are in Linux specific code. They are the result of not having an automated style checker to validate the code when it was originally written. Others were caused when the common code was slightly adjusted for Linux. This patch contains no functional changes. It only refreshes the code to conform to style guide. Everyone submitting patches for inclusion upstream should now run 'make checkstyle' and resolve any warning prior to opening a pull request. The automated builders have been updated to fail a build if when 'make checkstyle' detects an issue. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1821
* cstyle: Allow spaces in all commentsBrian Behlendorf2013-12-181-2/+2
| | | | | | | | | | Update the cstyle.pl script to allow pictures in all comments not just header comments. Recent changes from Illumos such as d3cc8b1 have relocated various pictures in the standard block comments to make the code more readable. Signed-off-by: Brian Behlendorf <[email protected]> Issue #1821
* cstyle: Exclude several files from 'make checkstyle'Brian Behlendorf2013-12-181-1/+2
| | | | | | | | | The zfs_config.h header and *.mod.c files are both products of the build process. They must be excluded from the style check because they are not part of the pristine source. Signed-off-by: Brian Behlendorf <[email protected]> Issue #1821
* Illumos #4208John Wren Kennedy2013-12-181-2/+2
| | | | | | | | | | | | | | | | | 4208 Typo in zfs_main.c: "posxiuser" Reviewed by: Sonu Pillai <[email protected]> Reviewed by: Will Guyette <[email protected]> Reviewed by: Eric Diven <[email protected]> Reviewed by: Christopher Siden <[email protected]> Approved by: Richard Lowe <[email protected]> References: https://www.illumos.org/issues/4208 illumos/illumos-gate@f38cb554a534c6df738be3f4d23327e69888e634 Ported-by: Brian Behlendorf <[email protected]> Closes #1986
* Add zfs_send_corrupt_data module optionTurbo Fredriksson2013-12-182-0/+16
| | | | | | | | | Tuning setting to ignore read/checksum errors when sending data. Signed-off-by: Turbo Fredriksson <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1982 Issue #1897
* Cause zfs.spec to place dracut files properlyAaron Fineman2013-12-181-1/+1
| | | | | | | | | This is an extension of commit ffb2111. As the fedora conditional has been added, this allows centos/rhel-6 to fall back to the proper directory (/usr/share/dracut) Signed-off-by: Brian Behlendorf <[email protected]> Closes #1984
* Handle acl flags from util-linux mount commandrenelson2013-12-183-3/+19
| | | | | | | | | | Add acl, noacl and posixacl to option_map, avoiding ENOENT error case when mount from util-linux-2.24 execs mount.zfs with any of those flags Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: renelson <[email protected]> Issue #1968
* Fix grammar in parse_options() error messagerenelson2013-12-171-1/+1
| | | | | | | | | A minor grammar error was corrected in in the parse_options() error handling for the ENOENT case. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: renelson <[email protected]> Issue #1968
* Fix z_sync_cnt decrement in zfs_closeChunwei Chen2013-12-172-10/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | The comment in zfs_close states that "Under Linux the zfs_close() hook is not symmetric with zfs_open()". This is not true. zfs_open/zfs_close is associated with every successful struct file creation/deletion, which should always be balanced. Here is an example of what's wrong: Process A B open(O_SYNC) z_sync_cnt = 1 open(O_SYNC) z_sync_cnt = 2 close() z_sync_cnt = 0 So z_sync_cnt is 0 even if B still has the file with O_SYNC. Also moves the generic_file_open call before zfs_open to ensure that in the case generic_file_open fails z_sync_cnt is not incremented. This is safe because generic_file_open has no side effects. Signed-off-by: Chunwei Chen <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #1962
* Silence e2fsck warning in zconfig.shBrian Behlendorf2013-12-161-11/+16
| | | | | | | | | | | | | | | | | | | When running zconfig.sh test 7 and 8 cause the following warning to be printed to the console. It's caused because we're snapshoting a mounted ext2 filesystem which is not in a 'clean' state. This is to be expected since we have no guarentees about the on-disk consistency of the filesystem. EXT2-fs warning: mounting unchecked fs, running e2fsck is recommended To silence the warning and preserve the intent of these test cases they have been updated to unmount the filesystem prior to snapshoting them. This ensures the ext2 filesystem is in a consistent state when the snapshot is taken. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Closes #1972
* cstyle: zvol.cBrian Behlendorf2013-12-161-109/+113
| | | | | | | | | | | Update zvol.c to conform to the style guidelines, verified by running cstyle.pl on the source file. This patch contains no functional changes. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Tim Chase <[email protected]> Issue #1821
* Update zfs(8) Snapshots sectionBrian Behlendorf2013-12-161-1/+1
| | | | | | | | | | | | | The Snapshots section of the zfs(8) man page is incorrect and should have been updated as part of #1312. Snapshots of volumes can be accessed independently and their visibility is determined by the 'snapdev=hidden|visible' property. This is analogous to the existing 'snapdir=hidden|visible' property. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Tim Chase <[email protected]> Closes #1921
* Sync /dev/zfs ioctl orderingBrian Behlendorf2013-12-162-4/+25
| | | | | | | | | | | | | | | | | | | | | | In order to minimize any future disruption caused by the addition and removal /dev/zfs ioctls this patch makes the following changes. 1) Sync ZoL's ioctl ordering such that it matches Illumos. For historic reasons the ZFS_IOC_DESTROY_SNAPS and ZFS_IOC_POOL_REGUID ioctls were out of order. 2) Move Linux and FreeBSD specific ioctls in to their own reserved ranges. This allows us to preserve the existing ordering when new ioctls are added by either Illumos or FreeBSD. When an ioctl is no longer needed it should be retired in place. This change alters the ZFS user/kernel ABI so make sure you rebuild both your user and kernel modules. However, it should allow for a much stabler interface going forward. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Closes #1973
* Remove ZFC_IOC_*_MINOR ioctl()sBrian Behlendorf2013-12-1612-456/+167
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Early versions of ZFS coordinated the creation and destruction of device minors from userspace. This was inherently racy and in late 2009 these ioctl()s were removed leaving everything up to the kernel. This significantly simplified the code. However, we never picked up these changes in ZoL since we'd already significantly adjusted this code for Linux. This patch aims to rectify that by finally removing ZFC_IOC_*_MINOR ioctl()s and moving all the functionality down in to the kernel. Since this cleanup will change the kernel/user ABI it's being done in the same tag as the previous libzfs_core ABI changes. This will minimize, but not eliminate, the disruption to end users. Once merged ZoL, Illumos, and FreeBSD will basically be back in sync in regards to handling ZVOLs in the common code. While each platform must have its own custom zvol.c implemenation the interfaces provided are consistent. NOTES: 1) This patch introduces one subtle change in behavior which could not be easily avoided. Prior to this change callers of 'zfs create -V ...' were guaranteed that upon exit the /dev/zvol/ block device link would be created or an error returned. That's no longer the case. The utilities will no longer block waiting for the symlink to be created. Callers are now responsible for blocking, this is why a 'udev_wait' call was added to the 'label' function in scripts/common.sh. 2) The read-only behavior of a ZVOL now solely depends on if the ZVOL_RDONLY bit is set in zv->zv_flags. The redundant policy setting in the gendisk structure was removed. This both simplifies the code and allows us to safely leverage set_disk_ro() to issue a KOBJ_CHANGE uevent. See the comment in the code for futher details on this. 3) Because __zvol_create_minor() and zvol_alloc() may now be called in a sync task they must use KM_PUSHPAGE. References: illumos/illumos-gate@681d9761e8516a7dc5ab6589e2dfe717777e1123 Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Tim Chase <[email protected]> Closes #1969
* Illumos #4121 vdev_label_init read onlyGeorge Wilson2013-12-121-1/+1
| | | | | | | | | | | | | | | | 4121 vdev_label_init should treat request as succeeded when pool is read only Reviewed by: Christopher Siden <[email protected]> Reviewed by: Matthew Ahrens <[email protected]> Reviewed by: Saso Kiselkov <[email protected]> Approved by: Richard Lowe <[email protected]> References: https://www.illumos.org/issues/4121 illumos/illumos-gate@973c78e94bf9634782164382c9e291bf81161fa5 Ported-by: Brian Behlendorf <[email protected]> Closes #1863
* Fix atime handling.Tim Chase2013-12-122-3/+2
| | | | | | | | | | | | | | | | | | | | | | | Previously, the atime-modifying vnops called ZFS_ACCESSTIME_STAMP() followed by zfs_inode_update() to update the atime. However, since atimes are cached in the znode for delayed writing, the zfs_inode_update() function would effectively ignore the cached atime by reading it from the SA. This commit moves the updating of the atime in the inode into zfs_tstamp_update_setup() which is called by the ZFS_ACCESSTIME_STAMP() macro and eliminates the call to zfs_inode_update() in the atime-modifying vnops. It's possible the same thing could have been done directly in zfs_inode_update() but I wasn't sure that it was safe in all cases where it is called. The effect is that atime handling is as if "strictatime" were selected; even if the filesystem is mounted with "relatime". Signed-off-by: Brian Behlendorf <[email protected]> Issue #1949
* Fix zstream_t incorrect typeShen Yan2013-12-101-1/+1
| | | | | | | | | | | | | The DMU zfetch code organizes streams with lists not avl trees. A avl_node_t was mistakenly used for a list_node_t in the zstream_t type. This is incorrect (but harmless) and when unnoticed because: 1) The list functions explicitly cast the value preventing a warning, 2) sizeof(avl_node_t) >= sizeof(list_node_t) so no overrun occurs, and 3) The calculated offset is the same regardless of the type. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1946
* Remove MAX when initializing arc_c_maxdavid.chen2013-12-101-1/+1
| | | | | | | | | | | | | | The MAX when initializing arc_c_max doesn't make any sense because it hasn't been set anywhere before. Though, arc_c_max should be implicitly set to zero when initializing arc_stats, so the MAX doesn't make any difference. The MAX was mistakenly left if place when the Illumos default values were changed for Linux. Signed-off-by: david.chen <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1941
* Fix multipath bug in vdev_id caused by inconsistent field numberingSimon Guest2013-12-101-2/+4
| | | | | | | | | | | | | | | | | | The bug is caused by multipath output like this: 35000c50056bd77a7 dm-15 HP,MB3000FCWDH size=2.7T features='0' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=0 status=active | `- 2:0:16:0 sdq 65:0 active undef running `-+- policy='round-robin 0' prio=0 status=enabled `- 4:0:52:0 sdfp 130:176 active undef running Note that the pipe symbols mean that the field numbering is different between the sdq and sdfp lines. The fix edits out the pipe symbols. Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1692
* Revert "Use directory xattrs for symlinks"Ned Bass2013-12-101-4/+0
| | | | | | | | | | | This reverts commit 6a7c0ccca44ad02c476a111d8f7911fc8b12fff7. A proper fix for Issue #1648 was landed under Issue #1890, so this is no longer needed. Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1648
* sa_find_sizes() may compute wrong SA header sizeJames Pan2013-12-101-24/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Under the right conditions sa_find_sizes() will compute an incorrect size of the system attribute (SA) header. This causes a failed assertion when the SA_HDR_SIZE_MATCH_LAYOUT() test returns false, and may lead to corruption of SA data. The bug presents itself when there are more than two variable-length SAs of just the right size to fit in the bonus buffer of a dnode. The existing logic fails to account for the SA header space needed to store the sizes of all the variable-length SAs. A reproducer was possible on Linux by setting the xattr=sa dataset property and storing xattrs on symbolic links (Issue #1648). Note the corrupt link target name: $ zfs set xattr=sa tank/fish $ cd /tank/fish $ ln -fs 12345678901234567 link $ setfattr -n trusted.0000000000000000000 -v 0x000000000000000000000000 -h link $ setfattr -n trusted.1111111111111111111 -v 0x000000000000000000000000 -h link $ ls -l link lrwxrwxrwx 1 root root 17 Dec 6 15:40 link -> 90123456701234567 Commit 6a7c0ccca44ad02c476a111d8f7911fc8b12fff7 worked around this bug by forcing xattr's on symlinks to be stored in directory format. This change implements a proper fix, so the workaround can now be reverted. The reference link below contains a reproducer for FreeBSD. References: http://lists.open-zfs.org/pipermail/developer/2013-November/000306.html Ported-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1890
* Update init script to allow verbose mountsTurbo Fredriksson2013-12-061-1/+6
| | | | | | | | | | Allow verbose mounts to make is easier to monitor progress when mounting a large number of filesystems. This functionality is disabled by default. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1929
* Update init script to allow /dev/disk/by-id importTurbo Fredriksson2013-12-061-2/+13
| | | | | | | | | | | | Many people prefer to use by-id at import time instead of using the cache file. This can be a much better solution than the cache file in some environments so we're adding some infrastructure to allow it. This functionality is disabled by default. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1929
* Fix 'zfs diff' shares errorBrian Behlendorf2013-12-061-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | When creating a dataset with ZoL a zsb->z_shares_dir ZAP object will not be created because shares are unimplemented. Instead ZoL just sets zsb->z_shares_dir to zero to indicate there are no shares. However, if you import a pool which was created with a different ZFS implementation then the shares ZAP object may exist. Code was added to handle this case but it clearly wasn't sufficiently tested with other ZFS pools. There was a bug in the zpl_shares_getattr() function which passed the wrong inode to zfs_getattr_fast() for the case where are shares ZAP object does exist. This causes an EIO to be returned to stat64() which in turn causes 'zfs diff' to fail. This fix is the pass the correct inode after a sucessful zfs_zget(). Additionally, only put away the references if we were able to get one. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Graham Booker <https://github.com/gbooker> Signed-off-by: timemaster67 <https://github.com/timemaster67> Closes #1426 Closes #481
* Add module versioningBrian Behlendorf2013-12-066-0/+6
| | | | | | | | | | | | | | | | | | | | | | | Use the standard Linux MODULE_VERSION macro to expose the installed zavl, znvpair, zunicode, zcommon, zfs, and zpios module versions. This will also automatically add a checksum of the .c files and headers in "srcversion". See: /sys/module/zavl/version /sys/module/zavl/srcversion /sys/module/znvpair/version /sys/module/znvpair/srcversion /sys/module/zunicode/version /sys/module/zunicode/srcversion /sys/module/zcommon/version /sys/module/zcommon/srcversion /sys/module/zfs/version /sys/module/zfs/srcversion /sys/module/zpios/version /sys/module/zpios/srcversion Signed-off-by: Brian Behlendorf <[email protected]> Closes #1923
* Illumos #4045 write throttle & i/o scheduler performance workMatthew Ahrens2013-12-0638-846/+1943
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 4045 zfs write throttle & i/o scheduler performance work 1. The ZFS i/o scheduler (vdev_queue.c) now divides i/os into 5 classes: sync read, sync write, async read, async write, and scrub/resilver. The scheduler issues a number of concurrent i/os from each class to the device. Once a class has been selected, an i/o is selected from this class using either an elevator algorithem (async, scrub classes) or FIFO (sync classes). The number of concurrent async write i/os is tuned dynamically based on i/o load, to achieve good sync i/o latency when there is not a high load of writes, and good write throughput when there is. See the block comment in vdev_queue.c (reproduced below) for more details. 2. The write throttle (dsl_pool_tempreserve_space() and txg_constrain_throughput()) is rewritten to produce much more consistent delays when under constant load. The new write throttle is based on the amount of dirty data, rather than guesses about future performance of the system. When there is a lot of dirty data, each transaction (e.g. write() syscall) will be delayed by the same small amount. This eliminates the "brick wall of wait" that the old write throttle could hit, causing all transactions to wait several seconds until the next txg opens. One of the keys to the new write throttle is decrementing the amount of dirty data as i/o completes, rather than at the end of spa_sync(). Note that the write throttle is only applied once the i/o scheduler is issuing the maximum number of outstanding async writes. See the block comments in dsl_pool.c and above dmu_tx_delay() (reproduced below) for more details. This diff has several other effects, including: * the commonly-tuned global variable zfs_vdev_max_pending has been removed; use per-class zfs_vdev_*_max_active values or zfs_vdev_max_active instead. * the size of each txg (meaning the amount of dirty data written, and thus the time it takes to write out) is now controlled differently. There is no longer an explicit time goal; the primary determinant is amount of dirty data. Systems that are under light or medium load will now often see that a txg is always syncing, but the impact to performance (e.g. read latency) is minimal. Tune zfs_dirty_data_max and zfs_dirty_data_sync to control this. * zio_taskq_batch_pct = 75 -- Only use 75% of all CPUs for compression, checksum, etc. This improves latency by not allowing these CPU-intensive tasks to consume all CPU (on machines with at least 4 CPU's; the percentage is rounded up). --matt APPENDIX: problems with the current i/o scheduler The current ZFS i/o scheduler (vdev_queue.c) is deadline based. The problem with this is that if there are always i/os pending, then certain classes of i/os can see very long delays. For example, if there are always synchronous reads outstanding, then no async writes will be serviced until they become "past due". One symptom of this situation is that each pass of the txg sync takes at least several seconds (typically 3 seconds). If many i/os become "past due" (their deadline is in the past), then we must service all of these overdue i/os before any new i/os. This happens when we enqueue a batch of async writes for the txg sync, with deadlines 2.5 seconds in the future. If we can't complete all the i/os in 2.5 seconds (e.g. because there were always reads pending), then these i/os will become past due. Now we must service all the "async" writes (which could be hundreds of megabytes) before we service any reads, introducing considerable latency to synchronous i/os (reads or ZIL writes). Notes on porting to ZFS on Linux: - zio_t gained new members io_physdone and io_phys_children. Because object caches in the Linux port call the constructor only once at allocation time, objects may contain residual data when retrieved from the cache. Therefore zio_create() was updated to zero out the two new fields. - vdev_mirror_pending() relied on the depth of the per-vdev pending queue (vq->vq_pending_tree) to select the least-busy leaf vdev to read from. This tree has been replaced by vq->vq_active_tree which is now used for the same purpose. - vdev_queue_init() used the value of zfs_vdev_max_pending to determine the number of vdev I/O buffers to pre-allocate. That global no longer exists, so we instead use the sum of the *_max_active values for each of the five I/O classes described above. - The Illumos implementation of dmu_tx_delay() delays a transaction by sleeping in condition variable embedded in the thread (curthread->t_delay_cv). We do not have an equivalent CV to use in Linux, so this change replaced the delay logic with a wrapper called zfs_sleep_until(). This wrapper could be adopted upstream and in other downstream ports to abstract away operating system-specific delay logic. - These tunables are added as module parameters, and descriptions added to the zfs-module-parameters.5 man page. spa_asize_inflation zfs_deadman_synctime_ms zfs_vdev_max_active zfs_vdev_async_write_active_min_dirty_percent zfs_vdev_async_write_active_max_dirty_percent zfs_vdev_async_read_max_active zfs_vdev_async_read_min_active zfs_vdev_async_write_max_active zfs_vdev_async_write_min_active zfs_vdev_scrub_max_active zfs_vdev_scrub_min_active zfs_vdev_sync_read_max_active zfs_vdev_sync_read_min_active zfs_vdev_sync_write_max_active zfs_vdev_sync_write_min_active zfs_dirty_data_max_percent zfs_delay_min_dirty_percent zfs_dirty_data_max_max_percent zfs_dirty_data_max zfs_dirty_data_max_max zfs_dirty_data_sync zfs_delay_scale The latter four have type unsigned long, whereas they are uint64_t in Illumos. This accommodates Linux's module_param() supported types, but means they may overflow on 32-bit architectures. The values zfs_dirty_data_max and zfs_dirty_data_max_max are the most likely to overflow on 32-bit systems, since they express physical RAM sizes in bytes. In fact, Illumos initializes zfs_dirty_data_max_max to 2^32 which does overflow. To resolve that, this port instead initializes it in arc_init() to 25% of physical RAM, and adds the tunable zfs_dirty_data_max_max_percent to override that percentage. While this solution doesn't completely avoid the overflow issue, it should be a reasonable default for most systems, and the minority of affected systems can work around the issue by overriding the defaults. - Fixed reversed logic in comment above zfs_delay_scale declaration. - Clarified comments in vdev_queue.c regarding when per-queue minimums take effect. - Replaced dmu_tx_write_limit in the dmu_tx kstat file with dmu_tx_dirty_delay and dmu_tx_dirty_over_max. The first counts how many times a transaction has been delayed because the pool dirty data has exceeded zfs_delay_min_dirty_percent. The latter counts how many times the pool dirty data has exceeded zfs_dirty_data_max (which we expect to never happen). - The original patch would have regressed the bug fixed in zfsonlinux/zfs@c418410, which prevented users from setting the zfs_vdev_aggregation_limit tuning larger than SPA_MAXBLOCKSIZE. A similar fix is added to vdev_queue_aggregate(). - In vdev_queue_io_to_issue(), dynamically allocate 'zio_t search' on the heap instead of the stack. In Linux we can't afford such large structures on the stack. Reviewed by: George Wilson <[email protected]> Reviewed by: Adam Leventhal <[email protected]> Reviewed by: Christopher Siden <[email protected]> Reviewed by: Ned Bass <[email protected]> Reviewed by: Brendan Gregg <[email protected]> Approved by: Robert Mustacchi <[email protected]> References: http://www.illumos.org/issues/4045 illumos/illumos-gate@69962b5647e4a8b9b14998733b765925381b727e Ported-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1913
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-27 03:29:55 +0000