aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Disable arc_p adapt dampener by defaultPrakash Surya2014-02-212-2/+23
| | | | | | | | | | | It's unclear why adjustments to arc_p need to be dampened as they are in arc_adjust. With that said, it's removal significantly improves the arc's ability to "warm up" to a given workload. Thus, I'm disabling by default until its usefulness is better understood. Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #2110
* Allow "arc_p" to drop to zero or grow to "arc_c"Prakash Surya2014-02-212-24/+4
| | | | | | | | | | | | | | | | | | Setting a limit on the minimum value of "arc_p" has been shown to have detrimental effects on the arc hit rate for certain "metadata" intensive workloads. Specifically, this has been exhibited with a workload that constantly dirties new "metadata" but also frequently touches a "small" amount of mfu data (e.g. mkdir's). What is seen is that the new anon data throttles the mfu list to a negligible size (because arc_p > anon + mru in arc_get_data_buf), even though the mfu ghost list receives a constant stream of hits. To remedy this, arc_p is now allowed to drop to zero if the algorithm deems it necessary. Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #2110
* Disable aggressive arc_p growth by defaultPrakash Surya2014-02-212-1/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For specific workloads consisting mainly of mfu data and new anon data buffers, the aggressive growth of arc_p found in the arc_get_data_buf() function can have detrimental effects on the mfu list size and ghost list hit rate. Running a workload consisting of two processes: * Process 1 is creating many small files * Process 2 is tar'ing a directory consisting of many small files I've seen arc_p and the mru grow to their maximum size, while the mru ghost list receives 100K times fewer hits than the mfu ghost list. Ideally, as the mfu ghost list receives hits, arc_p should be driven down and the size of the mfu should increase. Given the specific workload I was testing with, the mfu list size should grow to a point where almost no mfu ghost list hits would occur. Unfortunately, this does not happen because the newly dirtied anon buffers constancy drive arc_p to its maximum value and keep it there (effectively prioritizing the mru list and starving the mfu list down to a negligible size). The logic to increment arc_p from within the arc_get_data_buf() function was introduced many years ago in this upstream commit: commit 641fbdae3a027d12b3c3dcd18927ccafae6d58bc Author: maybee <none@none> Date: Wed Dec 20 15:46:12 2006 -0800 6505658 target MRU size (arc.p) needs to be adjusted more aggressively and since I don't fully understand the motivation for the change, I am reluctant to completely remove it. As a way to test out how it's removal might affect performance, I've disabled that code by default, but left it tunable via a module option. Thus, if its removal is found to be grossly detrimental for certain workloads, it can be re-enabled on the fly, without a code change. Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #2110
* Adjust arc_p based on "bytes" in arc_shrinkPrakash Surya2014-02-211-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | In an attempt to prevent arc_c from collapsing "too fast", the arc_shrink() function was updated to take a "bytes" parameter by this change: commit 302f753f1657c05a4287226eeda1f53ae431b8a7 Author: Brian Behlendorf <[email protected]> Date: Tue Mar 13 14:29:16 2012 -0700 Integrate ARC more tightly with Linux Unfortunately, that change failed to make a similar change to the way that arc_p was updated. So, there still exists the possibility for arc_p to collapse to near 0 when the kernel start calling the arc's shrinkers. This change attempts to fix this, by decrementing arc_p by the "bytes" parameter in the same way that arc_c is updated. In addition, a minimum value of arc_p is attempted to be maintained, similar to the way a minimum arc_p value is maintained in arc_adapt(). Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #2110
* Set zfs_arc_min to 4MBBrian Behlendorf2014-02-211-6/+3
| | | | | | | | | | | | | | | | | | | | | Decrease the mimimum ARC size from 1/32 of total system memory (or 64MB) to a much smaller 4MB. 1) Large systems with over a 1TB of memory are being deployed and reserving 1/32 of this memory (32GB) as the mimimum requirement is overkill. 2) Tiny systems like the raspberry pi may only have 256MB of memory in which case 64MB is far too large. The ARC should be reclaimable if the VFS determines it needs the memory for some other purpose. If you want to ensure the ARC is never completely reclaimed due to memory pressure you may still set a larger value with zfs_arc_min. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Prakash Surya <[email protected]> Issue #2110
* Merge branch 'issue-2094'Brian Behlendorf2014-02-2115-14/+640
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch stack was designed to accomplish the following: * Cleanly address the pool incompatibility which was accidentally introduced post-0.6.2 but pre-0.6.3. This required adding infrastructure which can be used from now on to notify system administrators of errata which affect their pool. * Add additional automated regression testing to ensure this type of compatibility issue is caught prior to the change being merged. Signed-off-by: Brian Behlendorf <[email protected]> Closes #2094
| * Add erratum for issue #2094Richard Yao2014-02-214-0/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ZoL commit 1421c89 unintentionally changed the disk format in a forward- compatible, but not backward compatible way. This was accomplished by adding an entry to zbookmark_t, which is included in a couple of on-disk structures. That lead to the creation of pools with incorrect dsl_scan_phys_t objects that could only be imported by versions of ZoL containing that commit. Such pools cannot be imported by other versions of ZFS or past versions of ZoL. The additional field has been removed by the previous commit. However, affected pools must be imported and scrubbed using a version of ZoL with this commit applied. This will return the pools to a state in which they may be imported by other implementations. The 'zpool import' or 'zpool status' command can be used to determine if a pool is impacted. A message similar to one of the following means your pool must be scrubbed to restore compatibility. $ zpool import pool: zol-0.6.2-173 id: 1165955789558693437 state: ONLINE status: Errata #1 detected. action: The pool can be imported using its name or numeric identifier, however there is a compatibility issue which should be corrected by running 'zpool scrub' see: http://zfsonlinux.org/msg/ZFS-8000-ER config: ... $ zpool status pool: zol-0.6.2-173 state: ONLINE scan: pool compatibility issue detected. see: https://github.com/zfsonlinux/zfs/issues/2094 action: To correct the issue run 'zpool scrub'. config: ... If there was an async destroy in progress 'zpool import' will prevent the pool from being imported. Further advice on how to proceed will be provided by the error message as follows. $ zpool import pool: zol-0.6.2-173 id: 1165955789558693437 state: ONLINE status: Errata #2 detected. action: The pool can not be imported with this version of ZFS due to an active asynchronous destroy. Revert to an earlier version and allow the destroy to complete before updating. see: http://zfsonlinux.org/msg/ZFS-8000-ER config: ... Pools affected by the damaged dsl_scan_phys_t can be detected prior to an upgrade by running the following command as root: zdb -dddd poolname 1 | grep -P '^\t\tscan = ' | sed -e 's;scan = ;;' | wc -w Note that `poolname` must be replaced with the name of the pool you wish to check. A value of 25 indicates the dsl_scan_phys_t has been damaged. A value of 24 indicates that the dsl_scan_phys_t is normal. A value of 0 indicates that there has never been a scrub run on the pool. The regression caused by the change to zbookmark_t never made it into a tagged release, Gentoo backports, Ubuntu, Debian, Fedora, or EPEL stable respositorys. Only those using the HEAD version directly from Github after the 0.6.2 but before the 0.6.3 tag are affected. This patch does have one limitation that should be mentioned. It will not detect errata #2 on a pool unless errata #1 is also present. It expected this will not be a significant problem because pools impacted by errata #2 have a high probably of being impacted by errata #1. End users can ensure they do no hit this unlikely case by waiting for all asynchronous destroy operations to complete before updating ZoL. The presence of any background destroys on any imported pools can be checked by running `zpool get freeing` as root. This will display a non-zero value for any pool with an active asynchronous destroy. Lastly, it is expected that no user data has been lost as a result of this erratum. Original-patch-by: Tim Chase <[email protected]> Reworked-by: Brian Behlendorf <[email protected]> Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #2094
| * Add generic errata infrastructureBrian Behlendorf2014-02-217-9/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | From time to time it may be necessary to inform the pool administrator about an errata which impacts their pool. These errata will by shown to the administrator through the 'zpool status' and 'zpool import' output as appropriate. The errata must clearly describe the issue detected, how the pool is impacted, and what action should be taken to resolve the situation. Additional information for each errata will be provided at http://zfsonlinux.org/msg/ZFS-8000-ER. To accomplish the above this patch adds the required infrastructure to allow the kernel modules to notify the utilities that an errata has been detected. This is done through the ZPOOL_CONFIG_ERRATA uint64_t which has been added to the pool configuration nvlist. To add a new errata the following changes must be made: * A new errata identifier must be assigned by adding a new enum value to the zpool_errata_t type. New enums must be added to the end to preserve the existing ordering. * Code must be added to detect the issue. This does not strictly need to be done at pool import time but doing so will make the errata visible in 'zpool import' as well as 'zpool status'. Once detected the spa->spa_errata member should be set to the new enum. * If possible code should be added to clear the spa->spa_errata member once the errata has been resolved. * The show_import() and status_callback() functions must be updated to include an informational message describing the errata. This should include an action message describing what an administrator should do to address the errata. * The documentation at http://zfsonlinux.org/msg/ZFS-8000-ER must be updated to describe the errata. This space can be used to provide as much additional information as needed to fully describe the errata. A link to this documentation will be automatically generated in the output of 'zpool import' and 'zpool status'. Original-idea-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Richard Yao <[email protected] Issue #2094
| * Use expected zpool_status_t typeBrian Behlendorf2014-02-211-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Both the zpool_import_status() and zpool_get_status() functions return the zpool_status_t enum. This explicit type should be used rather than the more generic int type. This patch makes no functional change and should only be considered code cleanup. It happens to have been done in the context of #2094 because that's when I noticed this issue. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Richard Yao <[email protected] Issue #2094
| * Revert changes to zbookmark_tRichard Yao2014-02-212-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 1421c89142376bfd41e4de22ed7c7846b9e41f95 added a field to zbookmark_t that unintentinoally caused a disk format change. This negatively affected backward compatibility and platform portability. Therefore, this field is being removed. The function that field permitted is left unimplemented until a later patch that will reimplement the field in a way that does not affect the disk format. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #2094
| * Add zimport.sh compatibility test scriptBrian Behlendorf2014-02-214-0/+500
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Verify that an assortment of known good reference pools can be imported using different versions of the ZoL code. By default references pools for the major ZFS implementation will be checked against the most recent ZoL tags and the master development branch. Alternate tags or branches may be verified with the '-s <src-tag> option. Passing the keyword "installed" will instruct the script to test whatever version is installed. Preferentially a reference pool is used for all tests. However, if one does not exist and the pool-tag matches one of the src-tags then a new reference pool will be created using binaries from that source build. This is particularly useful when you need to test your changes before opening a pull request. New reference pools may be added by placing a bzip2 compressed tarball of the pool in the scripts/zpool-example directory and then passing the -p <pool-tag> option. To increase the test coverage reference pools should be collected for all the major ZFS implementations. Having these pools easily available is also helpful to the developers. Care should be taken to run these tests with a kernel supported by all the listed tags. Otherwise build failure will cause false positives. EXAMPLES: The following example will verify the zfs-0.6.2 tag, the master branch, and the installed zfs version can correctly import the listed pools. Note there is no reference pool available for master and installed but because binaries are available one is automatically constructed. The working directory is also preserved between runs (-k) preventing the need to rebuild from source for multiple runs. zimport.sh -k -f /var/tmp/zimport \ -s "zfs-0.6.1 zfs-0.6.2 master installed" \ -p "all master installed" --------------------- ZFS on Linux Source Versions -------------- zfs-0.6.1 zfs-0.6.2 master 0.6.2-180 ----------------------------------------------------------------- Clone SPL Skip Skip Skip Skip Clone ZFS Skip Skip Skip Skip Build SPL Skip Skip Skip Skip Build ZFS Skip Skip Skip Skip ----------------------------------------------------------------- zevo-1.1.1 Pass Pass Pass Pass zol-0.6.1 Pass Pass Pass Pass zol-0.6.2-173 Fail Fail Pass Pass zol-0.6.2 Pass Pass Pass Pass master Fail Fail Pass Pass installed Pass Pass Pass Pass Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Richard Yao <[email protected]> Issue #2094
* Propagate errors when registering "relatime" property callback.Tim Chase2014-02-121-1/+1
| | | | | | | | | | | | Various errors can occur when registering property callbacks. As the author's comments indicate, the code is very paranoid about preserving the first-seen error when registering callbacks. This patch causes an error seen while registering the "relatime" callback to not clobber a previously-seen error. Reported-by: Jorgen Lundman <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2117
* Add default FILEDIR path to zpool-config scriptsBrian Behlendorf2014-02-1216-61/+45
| | | | | | | | | | | | | | | | Allow the caller of the zpool-create.sh script to override the default path where file vdevs are created. This allows for greated flexibilty when scripting. Additionally, update the default path from /tmp/ to /var/tmp/ because these days /tmp/ is likely a ramdisk. Even though these files are sparse they may grow large in which case they should be backed by a physical device. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2120
* Add explicit Conflicts for zfs-fuse packagesRalf Ertzinger2014-02-101-0/+4
| | | | | | | | | | | | zfs-fuse provides the same commands and man page names as ZoL. Changing the names on either side would make each incompatible with all existing documentation about ZFS. Providing bit identical files is not possible due to differing codebases. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Turbo Fredriksson <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1866
* Fix zconfig.sh test 9Brian Behlendorf2014-02-101-0/+1
| | | | | | | | | | Commit ba6a240 adjusted the behavior of 'zfs create -V'. The caller is no longer guaranteed that udev will have finished creating the /dev/ entries by the time to command exits. It is therefore required that we explicitly block waiting for udev to settle for this test to run reliably. Signed-off-by: Brian Behlendorf <[email protected]>
* Add systemd unit files for ZFS startupRalf Ertzinger2014-02-0518-3/+203
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds systemd unit files replacing the functionality offered by the SysV init script found in etc/init.d. It has been developed and tested on Fedora 19, Fedora 20 and openSuSE 13.1. Four unit files and one target are offered. zfs-import-cache.service: Import pools from /etc/zfs/zpool.cache. This unit will wait for udev to settle. zfs-import-scan.service: Import pools by scanning /dev/disk/by-id for zvols. This unit will only run if /etc/zfs/zpool.cache is not present. This unit will wait for udev to settle zfs-mount.service: Mount ZFS native filesystems. It contains a dependency to be loaded before local-fs.target. zfs-share.service: Share NFS/SMB filesystems. This unit contains a dependency that will cause it to be restarted whenever the smb or nfs-server unit is restarted, restoring the shares added. zfs.target: This target pulls in the other units in order to start ZFS. It's the only unit that can be enabled/disabled, all other services are static and pulled in by dependencies. It will honour zfs=off and zfs=no options on the kernel command line. Signed-off-by: Brian Behlendorf <[email protected]> Closes #2108
* Fix corrupted l2_asize in arcstatsBrian Behlendorf2014-02-051-1/+1
| | | | | | | | | | | | | | | | | | | Commit e0b0ca9 accidentally corrupted the l2_asize displayed in arcstats. This was caused by changing the l2arc_buf_hdr.b_asize member from an int to uint32_t type. There are places in the code where this field is cast to a uint64_t resulting in the b_hits member being treated as part of b_asize. To resolve the issue the type has been changed to a uint64_t, and the b_hits member is placed after the enum to prevent the size of the structure from increasing. This is a good example of exactly why it's a bad idea to use ambiguous types (int) in these structures. Signed-off-by: DHE <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1990
* 4188 assertion failed in dmu_tx_hold_free(): dn_datablkshift != 0Matthew Ahrens2014-01-311-1/+8
| | | | | | | | | | | | | | Reviewed by: George Wilson <[email protected]> Reviewed by: Christopher Siden <[email protected]> Approved by: Garrett D'Amore <[email protected]> Refences: https://www.illumos.org/issues/4188 illumos/illumos-gate@bb411a08b05466bfe0c7095b6373bbc1587e259a Ported-by: Chris Dunlop <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2091
* Illumos 4504 traverse_visitbp: visit group before userMatthew Ahrens2014-01-291-6/+6
| | | | | | | | | | | | | | | | 4504 traverse_visitbp: visit DMU_GROUPUSED_OBJECT before DMU_USERUSED_OBJECT Reviewed by: Christopher Siden <[email protected]> Reviewed by: George Wilson <[email protected]> References: https://illumos.org/issues/4504 http://code.delphix.com/illumos-4504 http://svnweb.freebsd.org/base?view=revision&revision=260812 Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Tim Chase <[email protected]> Closes #2079
* Implement relatime.Tim Chase2014-01-297-10/+110
| | | | | | | | | | | | Add the "relatime" property. When set to "on", a file's atime will only be updated if the existing atime at least a day old or if the existing ctime or mtime has been updated since the last access. This behavior is compatible with the Linux "relatime" mount option. Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2064 Closes #1917
* Fix error message in zpiosPatrik Greco2014-01-291-2/+2
| | | | | | | | The chunksize must always be strictly smaller than the regionsize. Signed-off-by: Andrew Uselton <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2072
* Call gethrtime() only once per new txg creationCyril Plisko2014-01-233-5/+5
| | | | | | | | | | | | | | | | | | | | | | | When transitioning current open TXG into QUIESCE state and opening a new one txg_quiesce() calls gethrtime(): - to mark the birth time of the new TXG - to record the SPA txg history kstat - implicitely inside spa_txg_history_add() These timestamps are practically the same, so that the first one can be used instead of the other two. The only visible difference is that inside spa_txg_history_add() the time spent in kmem_zalloc() will be counted towards the opened TXG. Since at this point the new TXG already exists (tx->tx_open_txg has been already incremented) it is actually a correct accounting. In any case this extra work is only happening when spa_txg_history kstat is activated (i.e. zfs_txg_history > 0) and doesn't affect the normal processing in any way. Signed-off-by: Cyril Plisko <[email protected]> Issue #2075
* Add additional state TXG_STATE_WAIT_FOR_SYNC for txg.Igor Lvovsky2014-01-233-8/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | In several cases when digging into kstats we can found two txgs in SYNC state, e.g. txg birth state nreserved nread nwritten ... 985452 258127184872561 C 0 373948416 2376272384 ... 985453 258129016180616 C 0 378173440 28793344 ... 985454 258129016271523 S 0 0 0 ... 985455 258130864245986 S 0 0 0 ... 985456 258130867458851 O 0 0 0 ... However only first txg (985454) is really syncing at this moment. The other one (985455) marked as SYNCED is actually in a post-QUIESCED state and waiting to start sync. So, the new TXG_STATE_WAIT_FOR_SYNC state between TXG_STATE_QUIESCED and TXG_STATE_SYNCED was added to reveal this situation. txg birth state nreserved nread nwritten ... 1086896 235261068743969 C 0 163577856 8437248 ... 1086897 235262870830801 C 0 280625152 822594048 ... 1086898 235264172219064 S 0 0 0 ... 1086899 235264936134407 W 0 0 0 ... 1086900 235264936296156 O 0 0 0 ... Signed-off-by: Igor Lvovsky <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #2075
* Use enum type(zfetch_dirn_t) insteadShen Yan2014-01-231-2/+4
| | | | | | | Fix code with zfetch_dirn_t, which is more readable and clear. Signed-off-by: Brian Behlendorf <[email protected]> Closes #2068
* Allow chown/chgrp when no ACL SAs exist.Tim Chase2014-01-231-0/+11
| | | | | | | | | | | | | | | From the comment in the commit: Some ZFS implementations (ZEVO) create neither a ZNODE_ACL nor a DACL_ACES SA in which case ENOENT is returned from zfs_acl_node_read() when the SA can't be located. Allow chown/chgrp to succeed in these cases rather than returning an error that makes no sense in the context of the caller. Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue zfs-osx/zfs#86 Closes #1911 Closes #2029
* vdev_file_io_start() to use taskq_dispatch(TQ_PUSHPAGE)Ned Bass2014-01-231-1/+1
| | | | | | | | | | | | The vdev_file_io_start() function may be processing a zio that the txg_sync thread is waiting on. In this case it is not safe to perform memory allocations that may generate new I/O since this could cause a deadlock. To avoid this, call taskq_dispatch() with TQ_PUSHPAGE instead of TQ_SLEEP. Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #1928
* Remove incorrect use of EXTRA_DIST for man pagesBrian Behlendorf2014-01-173-7/+4
| | | | | | | | | | Setting the 'dist_' prefix is the correct way to instruct Automake to include these files in the distribution. The EXTRA_DIST variable is reserved for files which are not covered by the automatic rules. http://www.gnu.org/software/automake/manual/automake.html#Basics Signed-off-by: Brian Behlendorf <[email protected]>
* vdev_id: support per-channel slot mappingsNed Bass2014-01-173-81/+65
| | | | | | | | | | | | | | | | | The vdev_id udev helper currently applies slot renumbering rules to every channel (JBOD) in the system. This is too inflexible for systems with non-homogeneous storage topologies. The "slot" keyword now takes an optional third parameter which names a channel to which the mapping will apply. If the third parameter is omitted then the rule applies to all channels. The first-specified rule that can match a slot takes precedence. Therefore a channel-specific rule for a given slot should generally appear before a generic rule for the same slot number. In this way a custom slot mapping can be applied to a particular channel and a default mapping applied to the rest. Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2056
* Use long holds in zvol_set_volsize()Brian Behlendorf2014-01-141-55/+67
| | | | | | | | | | | | | | | | | | | Under Linux the zvol_set_volsize() function was originally written to use dmu_objset_hold()/dmu_objset_rele(). Subsequently, the dmu_objset_own()/dmu_objset_disown() interfaces were added but the ZVOL code wasn't updated to take advantage of them. This was never an issue but after the dsl_pool_config changes the code now takes the config lock twice. The cleanest solution is to shift to using dmu_objset_own() which takes a long hold on the dataset and does not hold the dsl pool lock. This patch also slightly restructures the existing code such that it more closely resembles the upstream Illumos code. Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2039
* Disable GCCs aggressive loop optimizationBrian Behlendorf2014-01-144-2/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GCC >+ 4.8's aggressive loop optimization breaks some of the iterators over the dn_blkptr[] pseudo-array in dnode_phys. Since dn_blkptr[] is defined as a single-element array, GCC believes an iterator can only access index 0 and will unroll the loop into a single iteration. One way to resolve the issue would be to cast the array to a pointer and fix all the iterators that might break. The only loop where it is known to cause a problem is this loop in dmu_objset_write_ready(): for (i = 0; i < dnp->dn_nblkptr; i++) bp->blk_fill += dnp->dn_blkptr[i].blk_fill; In the common case where dn_nblkptr is 3, the loop is only executed a single time and "i" is equal to 1 following the loop. The specific breakage caused by this problem is that the blk_fill of root block pointers wouldn't be set properly when more than one blkptr is in use (when no indrect blocks are needed). The simple reproducing sequence is: zpool create tank /tank.img zdb -ddddd tank 0 Notice that "fill=31", however, there are two L0 indirect blocks with "F=31" and "F=5". The fill count should be 36 rather than 31. This problem causes an assert to be hit in a simple "zdb tank" when built with --enable-debug. However, this approach was not taken because we need to be absolutely sure we catch all instances of this unwanted optimization. Therefore, the build system has been updated to detect if GCC supports the aggressive loop optimization. If it does the optimization will be explicitly disabled using the -fno-aggressive-loop-optimization option. Original-fix-by: Tim Chase <[email protected]> Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2010 Closes #2051
* Properly link zpool command to libblkidRichard Yao2014-01-141-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 31fc19399e597e3391f19f1392ab120f1de0d5f2 incorrectly removed $(LIBBLKID) from cmd/zpool/Makefile.am. This meant that the toolchain was not given -lblkid, which resulted in the following build failure on Ubuntu 13.10: /usr/bin/ld: zpool_vdev.o: undefined reference to symbol 'blkid_put_cache@@BLKID_1.0' /lib/x86_64-linux-gnu/libblkid.so.1: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status That commit reworked various Makefile.am to follow best practices, so we reintroduce $(LIBBLKID) in a manner consistent with that, rather than explicitly reverting the change. Reproduction of this issue was done on a Gentoo Linux system by executing the following commands: zfs create -o mountpoint=/mnt/ubuntu-13.10 rpool/ROOT/ubuntu-13.10 debootstrap --variant=buildd --arch amd64 saucy /mnt/ubuntu-13.10 http://archive.ubuntu.com/ubuntu/ mount -o bind /dev /mnt/ubuntu-13.10/dev/ mount -o bind /proc/ /mnt/ubuntu-13.10/proc/ mount -o bind /sys/ /mnt/ubuntu-13.10/sys/ cp /etc/resolv.conf /mnt/ubuntu-13.10/etc/ (cd /mnt/ubuntu-13.10/root/ && git clone git://github.com/zfsonlinux/zfs.git) chroot /mnt/ubuntu-13.10/ apt-get install git autoconf libtool zlib1g-dev uuid-dev libblkid-dev \#apt-get install alien fakeroot vim cd /root/zfs ./autogen.sh ./configure --with-config=user --prefix=/usr make That will create a Ubuntu 13.10 chroot, fetch the sources and build test. At this point, cmd/zpool/Makefile.am was modified and the following commands were run to verify that the build issue was resolved: git clean -xdf ./autogen.sh ./configure --with-config=user --prefix=/usr make Although it is not shown here, the absence of libblkid-dev enables ZFS to build successfully without the patch. This could explain how this escaped detection until recently. A test without libblkid-dev was done to verify that the patch did not cause a regression in the absence of libblkid: apt-get remove libblkid-dev git clean -xdf ./autogen.sh ./configure --with-config=user --prefix=/usr make Additionally, the commands themselves were tested against my live system from within the chroot to ensure basic functionality. My live system had corresponding kernel modules already installed and basic commands such as `zpool list` and `zfs list` worked without incident. Lastly, this patch was also build tested on Gentoo Linux, where it caused no problems. At time of writing, these steps can be used to reproduce these results on any modern Linux system that has debootstrap installed. On Gentoo, installing debootstrap can be done with `emerge dev-util/debootstrap`. The current ZFSOnLinux HEAD revision as of writing is fd23720ae14dca926800ae70e6c8f4b4f82efc08. Once this is fixed in HEAD, either that revision or another before this fix and after 31fc19399e597e3391f19f1392ab120f1de0d5f2 will be needed to reproduce this issue. Lastly, it remains to be seen why the toolchains on the systems performing regression tests did not catch this. This is not a ZFS-specific issue, but it is something that we will want to explore in the future. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2038
* Prevent duplicate mnttab cache entriesBrian Behlendorf2014-01-141-0/+13
| | | | | | | | | | | | | Under Linux its possible to mount the same filesystem multiple times in the namespace. This can be done either with bind mounts or simply with multiple mount points. Unfortunately, the mnttab cache code is implemented using an AVL tree which does not support duplicate entries. To avoid this issue this patch updates the code to check for a duplicate entry before adding a new one. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Michael Martin <[email protected]> Closes #2041
* Drain iput taskq outside z_teardown_lockBrian Behlendorf2014-01-091-8/+8
| | | | | | | | | | | | | | It's unsafe to drain the iput taskq while holding the z_teardown_lock as a writer. This is because when the last reference on an inode is dropped it may still have pages which need to be written to disk. This will be done through zpl_writepages which will acquire the z_teardown_lock as a reader in ZFS_ENTER. Therefore, if we're holding the lock as a writer in zfs_sb_teardown the unmount will deadlock. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Chris Dunlop <[email protected]> Closes #1988
* Force LZ4_FORCE_SW_BITCOUNT for SparcBrian Behlendorf2014-01-091-0/+3
| | | | | | | | | | | | | | This change was proposed for Sparc but it's not clear to me why it's required. Proper support exists in the lz4 code to detect the endianness and the required builtins are available for gcc. Still I'm including the patch because it will only impact Sparc and it may resolve a case which hasn't occured to me. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: marku89 <[email protected]> Issue #1700
* Fix zfs_getattr_fast typesBrian Behlendorf2014-01-091-1/+6
| | | | | | | | | | | | | On Sparc sp->blksize will be a 64-bit value which is then cast incorrectly to a 32-bit value. For big endian systems this results in an incorrect value for sp->blksize. To resolve the problem local variables of the correct size are used and then assigned to sp->blksize. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: marku89 <[email protected]> Issue #1700
* Fix nvlist 'Bus Error' for SparcBrian Behlendorf2014-01-091-2/+4
| | | | | | | | | | | | | The mis-aligned memory accesses in nvpair_native_embedded() and nvpair_native_embedded_array() will cause a 'Bus Error' for architectures such as Sparc which not fully byte addressible. To avoid this issue care is taken to avoid dereferencing the potentially mis-aligned packed nvlist_t. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: marku89 <[email protected]> Issue #1700
* Use local variable to read zp->z_modeBrian Behlendorf2014-01-092-2/+5
| | | | | | | | | | | | | | | | | | When accessing the zp->z_mode through the SA bulk interface we expect that 64-bits are available to hold the result. However, on 32-bit platforms mode_t will only be 32-bits so we cannot pass it to SA_ADD_BULK_ATTR(). Instead a local uint64_t variable must be used and the result assigned to zp->z_mode. This went unnoticed on 32-bit little endian platforms because the bytes happen to end up in the correct 32-bits. But on big endian platforms like Sparc the zp->z_mode will always end up set to zero. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: marku89 <[email protected]> Issue #1700
* Define the needed ISA types for SparcBrian Behlendorf2014-01-092-2/+31
| | | | | | | | | | Add the minimum required ISA types to support the Sparc architecture. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: marku89 <[email protected]> Issue #1700
* Add ddt, ddt_entry, and l2arc_hdr cachesJohn Layman2014-01-074-16/+42
| | | | | | | | | | | Back the allocations for ddt tables+entries and l2arc headers with kmem caches. This will reduce the cost of allocating these commonly used structures and allow for greater visibility of them through the /proc/spl/kmem/slab interface. Signed-off-by: John Layman <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1893
* Remove unconditional sharetab updateBrian Behlendorf2014-01-071-8/+0
| | | | | | | | | | | | | | | Removes the unconditional sharetab update when running any zfs command. This means the sharetab might become out of date if users are manually adding/removing shares with exportfs. But we shouldn't punish all callers to zfs in order to handle that unlikely case. In the unlikely event we observe issues because of this it can always be added back to just the share/unshare call paths where we need an up to date sharetab. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Turbo Fredriksson <[email protected]> Signed-off-by: Chris Dunlop <[email protected]> Issue #845
* Enable /etc/mtab cache to improve performanceBrian Behlendorf2014-01-071-1/+1
| | | | | | | | | | | | | | | | | | | | | Re-enable the /etc/mtab cache to prevent the zfs command from having to repeatedly open and read from the /etc/mtab file. Instead an AVL tree of the mounted filesystems is created and used to vastly speed up lookups. This means that if non-zfs filesystems are mounted concurrently the 'zfs mount' will not immediately detect them. In practice that will rarely happen and even if it does the absolute worst case would be a failed mount. This was originally disabled out of an abundance of paranoia. NOTE: There may still be some parts of the code which do not consult the mtab cache. They should be updated to check the mtab cache as they as discovered to be a problem. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Turbo Fredriksson <[email protected]> Signed-off-by: Chris Dunlop <[email protected]> Issue #845
* Add UNSHARING of filesystems and EXPORTING poolsTurbo Fredriksson2014-01-071-0/+11
| | | | | | | | | | As a 'stop' action ensure the filesystem is unshared before it is unmounted, just in case. Additionally, export the pool so it may be cleanly imported by a different host. Signed-off-by: Turbo Fredriksson <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #2003
* Fix the creation of ZPOOL_HIST_CMD pool history entries.Tim Chase2014-01-072-9/+21
| | | | | | | | | | | | | | | | | | | | | Move the libzfs_fini() after the zpool_log_history() call so the ZPOOL_HIST_CMD entry can get written. Fix the handling of saved_poolname in zfsdev_ioctl() which was broken as part of the stack-reduction work in a16878805388c4d96cb8a294de965071d138a47b. Since ZoL destroys the TSD data in which the previously successful ioctl()'s pool name is stored following every vop, the ZFS_IOC_LOG_HISTORY ioctl has a very important restriction: it can only successfully write a long entry following a successful ioctl() if no intervening vops have been performed. Some of zfs subcommands do perform intervening vops and to do the logging themselves. At the moment, the "create" and "clone" subcommands have been modified appropriately. Signed-off-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1998
* Properly handle updates of variably-sized SA entries.Tim Chase2013-12-201-6/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | During the update process in sa_modify_attrs(), the sizes of existing variably-sized SA entries are obtained from sa_lengths[]. The case where a variably-sized SA was being replaced neglected to increment the index into sa_lengths[], so subsequent variable-length SAs would be rewritten with the wrong length. This patch adds the missing increment operation so all variably-sized SA entries are stored with their correct lengths. Previously, a size-changing update of a variably-sized SA that occurred when there were other variably-sized SAs in the bonus buffer would cause the subsequent SAs to be corrupted. The most common case in which this would occur is when a mode change caused the ZPL_DACL_ACES entry to change size when a ZPL_DXATTR (SA xattr) entry already existed. The following sequence would have caused a failure when xattr=sa was in force and would corrupt the bonus buffer: open(filename, O_WRONLY | O_CREAT, 0600); ... lsetxattr(filename, ...); /* create xattr SA */ chmod(filename, 0650); /* enlarges the ACL */ Signed-off-by: Chris Dunlop <[email protected]> Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1978
* Register correct handlers for nvlist_{dup,pack,unpack}Brian Behlendorf2013-12-201-38/+19
| | | | | | | | | | | This change is related to commit 81eaf15 which ensured the correct allocation handlers were installed for nvlist_alloc(). The nvlist functions nvlist_dup(), nvlist_pack(), and nvlist_unpack() suffer from the same issue and have been updated accordingly. Signed-off-by: Ned Bass <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #1937
* Add full SELinux supportMatthew Thode2013-12-1910-82/+134
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Four new dataset properties have been added to support SELinux. They are 'context', 'fscontext', 'defcontext' and 'rootcontext' which map directly to the context options described in mount(8). When one of these properties is set to something other than 'none'. That string will be passed verbatim as a mount option for the given context when the filesystem is mounted. For example, if you wanted the rootcontext for a filesystem to be set to 'system_u:object_r:fs_t' you would set the property as follows: $ zfs set rootcontext="system_u:object_r:fs_t" storage-pool/media This will ensure the filesystem is automatically mounted with that rootcontext. It is equivalent to manually specifying the rootcontext with the -o option like this: $ zfs mount -o rootcontext=system_u:object_r:fs_t storage-pool/media By default all four contexts are set to 'none'. Further information on SELinux contexts is detailed in mount(8) and selinux(8) man pages. Signed-off-by: Matthew Thode <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Richard Yao <[email protected]> Closes #1504
* cstyle: Resolve C style issuesMichael Kjorling2013-12-18165-1945/+2129
| | | | | | | | | | | | | | | | | | The vast majority of these changes are in Linux specific code. They are the result of not having an automated style checker to validate the code when it was originally written. Others were caused when the common code was slightly adjusted for Linux. This patch contains no functional changes. It only refreshes the code to conform to style guide. Everyone submitting patches for inclusion upstream should now run 'make checkstyle' and resolve any warning prior to opening a pull request. The automated builders have been updated to fail a build if when 'make checkstyle' detects an issue. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1821
* cstyle: Allow spaces in all commentsBrian Behlendorf2013-12-181-2/+2
| | | | | | | | | | Update the cstyle.pl script to allow pictures in all comments not just header comments. Recent changes from Illumos such as d3cc8b1 have relocated various pictures in the standard block comments to make the code more readable. Signed-off-by: Brian Behlendorf <[email protected]> Issue #1821
* cstyle: Exclude several files from 'make checkstyle'Brian Behlendorf2013-12-181-1/+2
| | | | | | | | | The zfs_config.h header and *.mod.c files are both products of the build process. They must be excluded from the style check because they are not part of the pristine source. Signed-off-by: Brian Behlendorf <[email protected]> Issue #1821
* Illumos #4208John Wren Kennedy2013-12-181-2/+2
| | | | | | | | | | | | | | | | | 4208 Typo in zfs_main.c: "posxiuser" Reviewed by: Sonu Pillai <[email protected]> Reviewed by: Will Guyette <[email protected]> Reviewed by: Eric Diven <[email protected]> Reviewed by: Christopher Siden <[email protected]> Approved by: Richard Lowe <[email protected]> References: https://www.illumos.org/issues/4208 illumos/illumos-gate@f38cb554a534c6df738be3f4d23327e69888e634 Ported-by: Brian Behlendorf <[email protected]> Closes #1986
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-22 14:49:38 +0000