aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Restructure zfs_readdir() to fix regressionsRichard Yao2013-10-291-15/+22
| | | | | | | | | | | | | | | | | | | | | | This does the following: 1. It creates a uint8_t type value, which is initialized to DT_DIR on dot directories and ZFS_DIRENT_TYPE(zap.za_first_integer) otherwise. This resolves a regression where we return unintialized values as the directory entry type on dot directories. This was accidentally introduced by commit 8170d281263e52ff33d7fba93ab625196844df36. 2. It restructures zfs_readdir() code to use `uint64_t offset` like Illumos instead of `loff_t *pos`. This resolves a regression where negative ZAP cursors were treated as if they were dot directories. 3. It restructures the function to more closely match the structure of zfs_readdir() on Illumos and removes the unused variable outcount, which was only used on Illumos. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1750
* Add -p switch to "zpool get"Ralf Ertzinger2013-10-282-10/+40
| | | | | | | | This works the same as the -p switch to "zfs get", displaying full resolution values for appropriate attributes. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1813
* Introduce zpool_get_prop_literal interfaceRalf Ertzinger2013-10-282-3/+19
| | | | | | | | | | | | | | | | | | | | | This change introduces zpool_get_prop_literal. It's an expanded version of zpool_get_prop taking one additional boolean parameter. With this parameter set to B_FALSE it will behave identically to zpool_get_prop. Setting it to B_TRUE will return full precision numbers for the following properties: ZPOOL_PROP_SIZE ZPOOL_PROP_ALLOCATED ZPOOL_PROP_FREE ZPOOL_PROP_FREEING ZPOOL_PROP_EXPANDSZ ZPOOL_PROP_ASHIFT Also introduced is a wrapper function for zpool_get_prop making it use zpool_get_prop_literal in the background. Signed-off-by: Brian Behlendorf <[email protected]> Issue #1813
* Corrected "zfs list -t <type>" syntaxSteven Hartland2013-10-252-4/+4
| | | | | | | in man page and in command help. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1805
* Merge branch 'kstat'Brian Behlendorf2013-10-2527-326/+1653
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | This branch updates several of the zfs kstats to take advantage of the improved raw kstat functionality. In addition, two new kstats and a script called dbufstat.py are introduced. Updated+New Kstats * dbufs - Stats for all dbufs in the dbuf_hash * <pool>/txgs - Stats for the last N txgs synced to disk * <pool>/reads - Stats for rhe last N reads issues by the ARC * <pool>/dmu_tx_assign - Histogram of tx assign times Signed-off-by: Brian Behlendorf <[email protected]>
| * Add dbufstat.py commandBrian Behlendorf2013-10-255-6/+544
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The dbufstat.py command was added to provide a conveniant way to easily determine what ZFS is caching. The script consumes the raw /proc/spl/kstat/zfs/dbufs kstat data can consolidates it in to a more human readable form. This was designed primarily as a tool to aid developers but it may also be useful for advanced users who want more visibility in to what the ARC is caching. When run without options dbufstat.py will default to showing a list of all objects with at least one buffer present in the cache. The total cache space consumed by that object will be printed on the right along with the object type. Similar to the arcstats.py command the -x option may used to display additional fields. Two other modes of operation are also supported by dbufstat.py and the expectation is additional display modes may be added as needed. The -t option will summerize the total number of bytes cached for each object type, and the -b option will show every dbuf currently cached. The script was designed to be consistent with arcstat.py and includes most of the same options and funcationality. Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]>
| * Add visibility in to cached dbufsBrian Behlendorf2013-10-259-8/+368
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently there is no mechanism to inspect which dbufs are being cached by the system. There are some coarse counters in arcstats by they only give a rough idea of what's being cached. This patch aims to improve the current situation by adding a new dbufs kstat. When read this new kstat will walk all cached dbufs linked in to the dbuf_hash. For each dbuf it will dump detailed information about the buffer. It will also dump additional information about the referenced arc buffer and its related dnode. This provides a more complete view in to exactly what is being cached. With this generic infrastructure in place utilities can be written to post-process the data to understand exactly how the caching is working. For example, the data could be processed to show a list of all cached dnodes and how much space they're consuming. Or a similar list could be generated based on dnode type. Many other ways to interpret the data exist based on what kinds of questions you're trying to answer. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Prakash Surya <[email protected]>
| * Add visibility in to dmu_tx_assign timesBrian Behlendorf2013-10-253-0/+113
| | | | | | | | | | | | | | | | | | | | | | This change adds a new kstat to gain some visibility into the amount of time spent in each call to dmu_tx_assign. A histogram is exported via the new dmu_tx_assign file. The information contained in this histogram is the frequency dmu_tx_assign took to complete given an interval range. Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]>
| * Add visibility in to txg sync behaviorBrian Behlendorf2013-10-253-1/+322
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change is an attempt to add visibility in to how txgs are being formed on a system, in real time. To do this, a list was added to the in memory SPA data structure for a pool, with each element on the list corresponding to txg. These entries are then exported through the kstat interface, which can then be interpreted in userspace. For each txg, the following information is exported: * Unique txg number (uint64_t) * The time the txd was born (hrtime_t) (*not* wall clock time; relative to the other entries on the list) * The current txg state ((O)pen/(Q)uiescing/(S)yncing/(C)ommitted) * The number of reserved bytes for the txg (uint64_t) * The number of bytes read during the txg (uint64_t) * The number of bytes written during the txg (uint64_t) * The number of read operations during the txg (uint64_t) * The number of write operations during the txg (uint64_t) * The time the txg was closed (hrtime_t) * The time the txg was quiesced (hrtime_t) * The time the txg was synced (hrtime_t) Note that while the raw kstat now stores relative hrtimes for the open, quiesce, and sync times. Those relative times are used to calculate how long each state took and these deltas and printed by output handlers. Signed-off-by: Brian Behlendorf <[email protected]>
| * Add visibility in to arc_readPrakash Surya2013-10-2511-13/+310
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change is an attempt to add visibility into the arc_read calls occurring on a system, in real time. To do this, a list was added to the in memory SPA data structure for a pool, with each element on the list corresponding to a call to arc_read. These entries are then exported through the kstat interface, which can then be interpreted in userspace. For each arc_read call, the following information is exported: * A unique identifier (uint64_t) * The time the entry was added to the list (hrtime_t) (*not* wall clock time; relative to the other entries on the list) * The objset ID (uint64_t) * The object number (uint64_t) * The indirection level (uint64_t) * The block ID (uint64_t) * The name of the function originating the arc_read call (char[24]) * The arc_flags from the arc_read call (uint32_t) * The PID of the reading thread (pid_t) * The command or name of thread originating read (char[16]) From this exported information one can see, in real time, exactly what is being read, what function is generating the read, and whether or not the read was found to be already cached. There is still some work to be done, but this should serve as a good starting point. Specifically, dbuf_read's are not accounted for in the currently exported information. Thus, a follow up patch should probably be added to export these calls that never call into arc_read (they only hit the dbuf hash table). In addition, it might be nice to create a utility similar to "arcstat.py" to digest the exported information and display it in a more readable format. Or perhaps, log the information and allow for it to be "replayed" at a later time. Signed-off-by: Prakash Surya <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]>
| * Revert "Add txgs-<pool> kstat file"Brian Behlendorf2013-10-254-231/+1
| | | | | | | | This reverts commit e95853a331529a6cb96fdf10476c53441e59f4e1.
| * Revert "Add new kstat for monitoring time in dmu_tx_assign"Brian Behlendorf2013-10-253-72/+0
|/ | | | | | This reverts commit 92334b14ec378b1693573b52c09816bbade9cf3e. Signed-off-by: Brian Behlendorf <[email protected]>
* Increase default udev wait timeBrian Behlendorf2013-10-223-3/+8
| | | | | | | | | | | | | | | | | | When creating a new pool, or adding/replacing a disk in an existing pool, partition tables will be automatically created on the devices. Under normal circumstances it will take less than a second for udev to create the expected device files under /dev/. However, it has been observed that if the system is doing heavy IO concurrently udev may take far longer. If you also throw in some cheap dodgy hardware it may take even longer. To prevent zpool commands from failing due to this the default wait time for udev is being increased to 30 seconds. This will have no impact on normal usage, the increase timeout should only be noticed if your udev rules are incorrectly configured. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1646
* Linux 3.11 compat: Rename LZ4 symbolsRichard Yao2013-10-223-5/+5
| | | | | | | | | | Linus Torvalds merged LZ4 into Linux 3.11. This causes a conflict whenever CONFIG_LZ4_DECOMPRESS=y or CONFIG_LZ4_COMPRESS=y are set in the kernel's .config. We rename the symbols to avoid the conflict. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1789
* Dedup-related documentation additions for zpool and zdb.Tim Chase2013-10-223-10/+40
| | | | | | | | | | | | | Document the "-D" and "-T" options and the optional interval and count or "zpool status". Also for zpool's man page, use a consistent order for the various "-T" options to match the program's help output. Document the effect of additional "-D" options for zdb. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1786
* Add missing dsl pool configuration lockTim Chase2013-10-221-1/+3
| | | | | | | | | | | | | | | The semantics introduced by the restructured sync task of illumos 3464 require this lock when calling dmu_snapshot_list_next(). The pool is locked/unlocked for each iteration to reduce the chance of long-running locks. This was accidentally missed when doing the original port because ZoL's control directory code is Linux-specific and is in a different file than in illumos. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1785
* Illumos #3552George Wilson2013-10-181-10/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 3552 condensing one space map burns 3 seconds of CPU in spa_sync() thread (fix race condition) References: https://www.illumos.org/issues/3552 illumos/illumos-gate@03f8c366886542ed249a15d755ae78ea4e775d9d Ported-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Porting notes: This fixes an upstream regression that was introduced in commit zfsonlinux/zfs@e51be06697762215dc3b679f8668987034a5a048, which ported the Illumos 3552 changes. This fix was added to upstream rather quickly, but at the time of the port, no one spotted it and the race was rare enough that it passed our regression tests. I discovered this when comparing our metaslab.c to the illumos metaslab.c. Without this change it is possible for metaslab_group_alloc() to consume a large amount of cpu time. Since this occurs under a mutex in a rcu critical section the kernel will log this to the console as a self-detected cpu stall as follows: INFO: rcu_sched self-detected stall on CPU { 0} (t=60000 jiffies g=11431890 c=11431889 q=18271) Closes #1687 Closes #1720 Closes #1731 Closes #1747
* Fix libzfs_core changes to follow GNU libtool guidelinesRichard Yao2013-10-104-4/+4
| | | | | | | | | | | | | | | | | | | | The GNU libtool documentation states to start with a version of 0:0:0, rather than 1:1:0. Illumos uses the name libzfs_core.so.1, so to be consistent, we should go with 1:0:0. http://www.gnu.org/software/libtool/manual/libtool.html#Updating-version-info The GNU libtool documentation also provides guidence on how the version information should be incremented. Doing this does a SONAME bump of the libzfs and libzpool libraries. This is particularly important on Gentoo because a SONAME bump enables portage to retain the older libraries until any packages that link to them are rebuilt. The main example of this is GRUB2's grub2-mkconfig, which will break unless it is rebuilt against the new libraries. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #1751
* Generate libraries with correct DT_NEEDED entriesRichard Yao2013-10-1013-15/+15
| | | | | | | | | | | | | | | | | | | | Libraries that depend on other libraries should list them in ELF's DT_NEEDED field so that programs linking to them do not need to specify those libraries unless they depend on them as well. This is not the case in the current code and the consequence is that anything that needs a library must know its dependencies. This is fragile and caused GRUB2's configure script to break when a dependency was added on libblkid in libzfs. This resolves that problem by using LIBADD/LDADD to specify libraries in Makefile.am instead of LDFLAGS. This ensures that proper DT_NEEDED entries are generated and prevents GRUB2's configure script from breaking in the presence of a libblkid dependency. This also removes unneeded dependencies from various files. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #1751
* Fix libblkid supportRichard Yao2013-10-102-8/+27
| | | | | | | | | | | | | | | | | | | | | | | | | libblkid support is dormant because the autotools check is broken and liblkid identifies ZFS vdevs as "zfs_member", not "zfs". We fix that with a few changes: First, we fix the libblkid autotools check to do a few things: 1. Make a 64MB file, which is the minimum size ZFS permits. 2. Make 4 fake uberblock entries to make libblkid's check succeed. 3. Return 0 upon success to make autotools use the success case. 4. Include stdlib.h to avoid implicit declration of free(). 5. Check for "zfs_member", not "zfs" 6. Make --with-blkid disable autotools check (avoids Gentoo sandbox violation) 7. Pass '-lblkid' correctly using LIBS not LDFLAGS. Second, we change the libblkid support to scan for "zfs_member", not "zfs". This makes --with-blkid work on Gentoo. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Issue #1751
* Update detach section of zpool(8)Neil Stockbridge2013-10-101-1/+1
| | | | | | | | | The detach section of the zpool(8) man page now suggests the offline command. Using offline may be more appropriate for certain situations. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1776
* Export symbols dsl_pool_config_{enter,exit}Ned Bass2013-10-101-0/+3
| | | | | | | | | | These are needed by consumers (i.e. Lustre) who wish to use the dsl_prop_register() interface to register callbacks when pool properties of interest change. This interface requires that the DSL pool configuration lock is held when called. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1762
* Fix memory leak false positive in log_internal()Brian Behlendorf2013-10-091-2/+4
| | | | | | | | | | | | | | When building the spl with --enable-debug-kmem-tracking a memory leak is detected in log_internal(). This happens to be a false positive because the memory was freed using strfree() instead of kmem_free(). All kmem_alloc()'s must be released with kmem_free() to ensure correct accounting. SPL: kmem leaked 135/5641311 bytes address size data func:line ffff8800cba7cd80 135 ZZZZZZZZZZZZZZZZ log_internal:456 Signed-off-by: Brian Behlendorf <[email protected]>
* Update drive databaseRichard Yao2013-10-091-0/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add Corsair Force GS drive (obtained from drive_id) Add Kingston HyperX 3K (obtained from drive_id) Add OCZ Vertex 4 drive (obtained from drive_id) Add Samsung SM843T enterprise drive (obtained from drive_id) Add entries for additional sizes of Intel 320/330/335/520 series Add Cruical C400 (obtained from Illumos user's sd.conf) Add Toshiba SSD (obtained from Illumos user's sd.conf) Add Samsung's first SLC SSD (obtained from drive_id) Add OCZ Core Series (obtained from drive_id) Add Intel DC S3700 (obtained from drive_id) Notes: 1. The drive identifer obtained for the Samsung SM843T was MZ7WD480. The rest were extrapolated. The additional entries were checked with Google to verify that such drives exist in the wild. 2. The additional entries for Intel drives were extrapolated from existing entries. The additional entries were checked with Google to verify that such drives exist in the wild. 3. The "ATA C400-MTFDDAC512M" and "ATA TOSHIBA THNSNH51" entries are from the sd.conf of gcbirzan on freenode. Additional entries were extrapolated from them and checked with Google. 4. I obtained the Samsung MCCOE64G entry from an actual drive. The Samsung MCCOE32G entry was extrapolated from it and checked with Google. 5. I obtained the SSDSC2BA10 from a 100GB Intel DC S3700 drive and extrapolated the entries for the additional models. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1752
* Export addition dsl_prop_* symbolsBrian Behlendorf2013-09-251-0/+6
| | | | | | | | The recent sync task restructuring in 13fe019 introduced several new symbols which should be exported for use by consumers such as Lustre. Signed-off-by: Brian Behlendorf <[email protected]>
* Allocate the ioctl "output" nvlist with KM_PUSHPAGE.Tim Chase2013-09-251-1/+1
| | | | | | | | | | Some ZFS errors such as certain snapshot failures can occur in the sync task context. Because they may require additional memory allocations, the initial nvlist must be allocated with KM_PUSHPAGE. Signed-off-by: Brian Behlendorf <[email protected]> Issue #1746 Issue #1737
* Fix several new KM_SLEEP warningsTim Chase2013-09-253-4/+4
| | | | | | | | | A handful of allocations now occur in the sync path and need to use KM_PUSHPAGE. These were introduced by commit 13fe019. Signed-off-by: Brian Behlendorf <[email protected]> Issue #1746 Issue #1737
* Fix spa_deadman() TQ_SLEEP warningBrian Behlendorf2013-09-252-2/+2
| | | | | | | | | | The spa_deadman() and spa_sync() functions can both be run in the spa_sync context and therefore should use TQ_PUSHPAGE instead of TQ_SLEEP. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1734 Closes #1749
* Removing unneeded mutex for reading vq_pending_tree sizeGregorKopka2013-09-251-8/+1
| | | | | | | | | | | | | | | | Locking mutex &vq->vq_lock in vdev_mirror_pending is unneeded: * no data is modified * only vq_pending_tree is read * in case garbage is returned (eg. vq_pending_tree being updated while the read is made) the worst case would be that a single read could be queued on a mirror side which more busy than thought The benefit of this change is streamlining of the code path since it is taken for *every* mirror member on *every* read. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1739
* Reduce the stack usage of dsl_dataset_remove_clones_keyKohsuke Kawaguchi2013-09-251-7/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | dataset_remove_clones_key does recursion, so if the recursion goes deep it can overrun the linux kernel stack size of 8KB. I have seen this happen in the actual deployment, and subsequently confirmed it by running a test workload on a custom-built kernel that uses 32KB stack. See the following stack trace as an example of the case where it would have run over the 8KB stack kernel: Depth Size Location (42 entries) ----- ---- -------- 0) 11192 72 __kmalloc+0x2e/0x240 1) 11120 144 kmem_alloc_debug+0x20e/0x500 2) 10976 72 dbuf_hold_impl+0x4a/0xa0 3) 10904 120 dbuf_prefetch+0xd3/0x280 4) 10784 80 dmu_zfetch_dofetch.isra.5+0x10f/0x180 5) 10704 240 dmu_zfetch+0x5f7/0x10e0 6) 10464 168 dbuf_read+0x71e/0x8f0 7) 10296 104 dnode_hold_impl+0x1ee/0x620 8) 10192 16 dnode_hold+0x19/0x20 9) 10176 88 dmu_buf_hold+0x42/0x1b0 10) 10088 144 zap_lockdir+0x48/0x730 11) 9944 128 zap_cursor_retrieve+0x1c4/0x2f0 12) 9816 392 dsl_dataset_remove_clones_key.isra.14+0xab/0x190 13) 9424 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 14) 9032 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 15) 8640 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 16) 8248 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 17) 7856 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 18) 7464 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 19) 7072 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 20) 6680 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 21) 6288 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 22) 5896 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 23) 5504 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 24) 5112 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 25) 4720 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 26) 4328 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 27) 3936 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 28) 3544 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 29) 3152 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 30) 2760 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 31) 2368 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 32) 1976 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 33) 1584 392 dsl_dataset_remove_clones_key.isra.14+0x10c/0x190 34) 1192 232 dsl_dataset_destroy_sync+0x311/0xf60 35) 960 72 dsl_sync_task_group_sync+0x12f/0x230 36) 888 168 dsl_pool_sync+0x48b/0x5c0 37) 720 184 spa_sync+0x417/0xb00 38) 536 184 txg_sync_thread+0x325/0x5b0 39) 352 48 thread_generic_wrapper+0x7a/0x90 40) 304 128 kthread+0xc0/0xd0 41) 176 176 ret_from_fork+0x7c/0xb0 This change reduces the stack usage in dsl_dataset_remove_clones_key by allocating structures in heap, not in stack. This is not a fundamental fix, as one can create an arbitrary large data set that runs over any fixed size stack, but this will make the problem far less likely. Signed-off-by: Brian Behlendorf <[email protected]> Signed-off-by: Kohsuke Kawaguchi <[email protected]> Closes #1726
* Fix zpl_mknod() return valuesBrian Behlendorf2013-09-131-1/+1
| | | | | | | | | | | The zpl_mknod() function was incorrectly negating its return value. This doesn't cause any problems in the success case, but it does prevent us from returning the correct error code for a failure. The implementation of this function is now consistent with all the other zpl_* functions. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1717
* Fix uninitialized variablesBrian Behlendorf2013-09-131-2/+2
| | | | | | | | | When compiling on an ARM device using gcc 4.7.3 several variables in the zfs_obj_to_path_impl() function were flagged as uninitialized. To resolve the warnings explicitly initialize them to zero. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1716
* Stop runtime pointer modifications in autotools checksRichard Yao2013-09-132-16/+24
| | | | | | | | | | | | | c38367c73f592ca9729ba0d5e70b5e3bc67e0745 was meant to eliminate runtime function pointer modifications in autotools checks because they were prone to false negatives on kernels hardened by the PaX project. Unfortunately, I missed the xattr_handler and super_block->s_bdi autotools checks. Recent changes to PaX constified xattr_handler->get/set, which lead me to discover this oversight. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1433
* Fix dmu_objset_find_dp() KM_SLEEP warningTim Chase2013-09-111-1/+1
| | | | | | | | | After the restructuring in 13fe019 The 'zfs rename' command will result in a KM_SLEEP being called in the sync context. This may deadlock due to reclaim so it was changed to KM_PUSHPAGE. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1711
* Illumos #3464Matthew Ahrens2013-09-0486-5916/+6298
| | | | | | | | | | | | | | | | | 3464 zfs synctask code needs restructuring Reviewed by: Dan Kimmel <[email protected]> Reviewed by: Adam Leventhal <[email protected]> Reviewed by: George Wilson <[email protected]> Reviewed by: Christopher Siden <[email protected]> Approved by: Garrett D'Amore <[email protected]> References: https://www.illumos.org/issues/3464 illumos/illumos-gate@3b2aab18808792cbd248a12f1edf139b89833c13 Ported-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1495
* Illumos #2882, #2883, #2900Matthew Ahrens2013-09-0456-1514/+2873
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 2882 implement libzfs_core 2883 changing "canmount" property to "on" should not always remount dataset 2900 "zfs snapshot" should be able to create multiple, arbitrary snapshots at once Reviewed by: George Wilson <[email protected]> Reviewed by: Chris Siden <[email protected]> Reviewed by: Garrett D'Amore <[email protected]> Reviewed by: Bill Pijewski <[email protected]> Reviewed by: Dan Kruchinin <[email protected]> Approved by: Eric Schrock <[email protected]> References: https://www.illumos.org/issues/2882 https://www.illumos.org/issues/2883 https://www.illumos.org/issues/2900 illumos/illumos-gate@4445fffbbb1ea25fd0e9ea68b9380dd7a6709025 Ported-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1293 Porting notes: WARNING: This patch changes the user/kernel ABI. That means that the zfs/zpool utilities built from master are NOT compatible with the 0.6.2 kernel modules. Ensure you load the matching kernel modules from master after updating the utilities. Otherwise the zfs/zpool commands will be unable to interact with your pool and you will see errors similar to the following: $ zpool list failed to read pool configuration: bad address no pools available $ zfs list no datasets available Add zvol minor device creation to the new zfs_snapshot_nvl function. Remove the logging of the "release" operation in dsl_dataset_user_release_sync(). The logging caused a null dereference because ds->ds_dir is zeroed in dsl_dataset_destroy_sync() and the logging functions try to get the ds name via the dsl_dataset_name() function. I've got no idea why this particular code would have worked in Illumos. This code has subsequently been completely reworked in Illumos commit 3b2aab1 (3464 zfs synctask code needs restructuring). Squash some "may be used uninitialized" warning/erorrs. Fix some printf format warnings for %lld and %llu. Apply a few spa_writeable() changes that were made to Illumos in illumos/illumos-gate.git@cd1c8b8 as part of the 3112, 3113, 3114 and 3115 fixes. Add a missing call to fnvlist_free(nvl) in log_internal() that was added in Illumos to fix issue 3085 but couldn't be ported to ZoL at the time (zfsonlinux/zfs@9e11c73) because it depended on future work.
* Tag zfs-0.6.2zfs-0.6.2Brian Behlendorf2013-08-224-1/+7
| | | | | | META file and release log updated. Signed-off-by: Brian Behlendorf <[email protected]>
* Use directory xattrs for symlinksBrian Behlendorf2013-08-221-0/+4
| | | | | | | | | | | | | There is currently a subtle bug in the SA implementation which can crop up which prevents us from safely using multiple variable length SAs in one object. Fortunately, the only existing use case for this are symlinks with SA based xattrs. Therefore, until the root cause in the SA code can be identified and fixed we prevent adding SA xattrs to symlinks. Signed-off-by: Brian Behlendorf <[email protected]> Issue #1468
* Revert "Evict meta data from ghost lists + l2arc headers"Brian Behlendorf2013-08-221-17/+1
| | | | | | | | This reverts commit fadd0c4da1e2ccd6014800d8b1a0fd117dd323e8 which introduced a regression in honoring the meta limit. Signed-off-by: Brian Behlendorf <[email protected]> Close #1660
* Implement database to workaround misreported physical sector sizesRichard Yao2013-08-221-4/+129
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This implements vdev_bdev_database_check(). It alters the detected sector size of any device listed in a database of drives known to lie about their physical sector sizes. This is based on "6931570 Add flash devices' VID/PID to disk table to advertising 4K physical sector size" from Open Solaris and on sg_simple4.c from sg3_utils. About two dozen lines are taken from sg_simple4.c, which is GPLv2 licensed. However, sg_simple4.c is analogous to a Hello World program and is safe for us to use. We requested that Douglas Gilbert, the author of sg_simple4.c, confirm that this is the case. A cutdown version of his response is as follows: ``` I would consider a SCSI INQUIRY example using the Linux sg driver interface (also written by me) as the equivalent of an "hello world" program in C. ``` The database was created with the help of the freenode and ZFSOnLinux communities. Some notes: 1. The following drives both were confirmed to lie via reports in IRC and they contain capacity information in their identifiers: INTEL SSDSA2M080 INTEL SSDSA2M160 M4-CT256M4SSD2 WDC WD15EARS-00S WDC WD15EARS-00Z WDC WD20EARS-00M The identifiers for different capacity models were extrapolated and added under the assumption that those models also lie. Google was used to verify that the extrapolated drive identifiers existed prior to their inclusion. 2. The OCZ-VERTEX2 3.5 identifer applies to two drives that differ solely in page size (and slightly in capacity). One uses 4096-byte pages and the other uses 8192-byte pages. Both are set to use 8192-byte pages. We could detect the page size by checking the capacity, but that would unnecessarily complicate the code. 3. It is possible for updated drive firmware to correctly report the sector size. There were reports of a few advanced format drives doing that. One report stated that the vendor changed the identification string while another was unclear on this. Both reports involved WDC models. 4. Google was used to determine the size of pages in the listed flash devices. Reports of 8192-byte pages took precedence over reports of 4096-byte pages. 5. Devices behind USB adapters can have their identification strings altered. Identification strings obtained across USB adapters are omitted and no attempt is made to correct for alterations made by USB adapters when doing comparisons against the database. Two entries in the Open Solaris database that appear to have been altered by a USB adapter were omitted. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1652
* Linux 3.11 compat: fops->iterate()Richard Yao2013-08-158-124/+260
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit torvalds/linux@2233f31aade393641f0eaed43a71110e629bb900 replaced ->readdir() with ->iterate() in struct file_operations. All filesystems must now use the new ->iterate method. To handle this the code was reworked to use the new ->iterate interface. Care was taken to keep the majority of changes confined to the ZPL layer which is already Linux specific. However, minor changes were required to the common zfs_readdir() function. Compatibility with older kernels was accomplished by adding versions of the trivial dir_emit* helper functions. Also the various *_readdir() functions were reworked in to wrappers which create a dir_context structure to pass to the new *_iterate() functions. Unfortunately, the new dir_emit* functions prevent us from passing a private pointer to the filldir function. The xattr directory code leveraged this ability through zfs_readdir() to generate the list of xattr names. Since we can no longer use zfs_readdir() a simplified zpl_xattr_readdir() function was added to perform the same task. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1653 Issue #1591
* Fix z_wr_iss_h zio_execute() import hangBrian Behlendorf2013-08-151-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | Because we need to be more frugal about our stack usage under Linux. The __zio_execute() function was modified to re-dispatch zios to a ZIO_TASKQ_ISSUE thread when we're in a context which is known to be stack heavy. Those two contexts are the sync thread and what ever thread is performing spa initialization. Unfortunately, this change introduced an unlikely bug which can result in a zio being re-dispatched indefinitely and never being executed. If during spa initialization we handle a zio with ZIO_PRIORITY_NOW it will be moved to the high priority queue. When __zio_execute() is called again for the zio it will mis- interpret the context and re-dispatch it again. The system will get stuck spinning re-dispatching the zio and making no forward progress. To fix this rare issue __zio_execute() has been updated not to re-dispatch zios on either the ZIO_TASKQ_ISSUE or ZIO_TASKQ_ISSUE_HIGH task queues. In practice this issue was rarely reported and can usually be fixed by rebooting the system and importing the pool again. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1455
* Don't specifically open /etc/mtab - it is done in libzfs_init()Turbo Fredriksson2013-08-151-8/+3
| | | | | | | a few lines further down and we can share the open file handle. Signed-off-by: Brian Behlendorf <[email protected]> Issue #1498
* No point in rewind() mtab in zfs_unshare_proto(). We're not reallyTurbo Fredriksson2013-08-151-1/+0
| | | | | | | | reading the file, but instead use libzfs_mnttab_find() which does the nessesary freopen() for us. Signed-off-by: Brian Behlendorf <[email protected]> Issue #1498
* Use setmntent() OR fopen()Turbo Fredriksson2013-08-151-0/+4
| | | | | | | | For the same reasons it's used in libzfs_init(), this was just overlooked because zinject gets minimal use. Signed-off-by: Brian Behlendorf <[email protected]> Issue #1498
* Fix for re-reading /etc/mtab in zfs_is_mounted()John Layman2013-08-141-4/+15
| | | | | | | | | | | | When /etc/mtab is updated on Linux it's done atomically with rename(2). A new mtab is written, the existing mtab is unlinked, and the new mtab is renamed to /etc/mtab. This means that we must close the old file and open the new file to get the updated contents. Using rewind(3) will just move the file pointer back to the start of the file, freopen(3) will close and open the file. Signed-off-by: Brian Behlendorf <[email protected]> Closes #1611
* Illumos #3098 zfs userspace/groupspace failYuri Pankov2013-08-142-22/+26
| | | | | | | | | | | | | | 3098 zfs userspace/groupspace fail without saying why when run as non-root Reviewed by: Eric Schrock <[email protected]> Approved by: Richard Lowe <[email protected]> References: https://www.illumos.org/issues/3098 illumos/illumos-gate@70f56fa69343b013f47e010537cff8ef3a7a40a5 Ported-by: Tim Chase <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1596
* Illumos #3618 ::zio dcmd does not show timestamp dataMatthew Ahrens2013-08-124-13/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 3618 ::zio dcmd does not show timestamp data Reviewed by: Adam Leventhal <[email protected]> Reviewed by: George Wilson <[email protected]> Reviewed by: Christopher Siden <[email protected]> Reviewed by: Garrett D'Amore <[email protected]> Approved by: Dan McDonald <[email protected]> References: http://www.illumos.org/issues/3618 illumos/illumos-gate@c55e05cb35da47582b7afd38734d2f0d9c6deb40 Notes on porting to ZFS on Linux: The original changeset mostly deals with mdb ::zio dcmd. However, in order to provide the requested functionality it modifies vdev and zio structures to keep the timing data in nanoseconds instead of ticks. It is these changes that are ported over in the commit in hand. One visible change of this commit is that the default value of 'zfs_vdev_time_shift' tunable is changed: zfs_vdev_time_shift = 6 to zfs_vdev_time_shift = 29 The original value of 6 was inherited from OpenSolaris and was subotimal - since it shifted the raw tick value - it didn't compensate for different tick frequencies on Linux and OpenSolaris. The former has HZ=1000, while the latter HZ=100. (Which itself led to other interesting performance anomalies under non-trivial load. The deadline scheduler delays the IO according to its priority - the lower priority the further the deadline is set. The delay is measured in units of "shifted ticks". Since the HZ value was 10 times higher, the delay units were 10 times shorter. Thus really low priority IO like resilver (delay is 10 units) and scrub (delay is 20 units) were scheduled much sooner than intended. The overall effect is that resilver and scrub IO consumed more bandwidth at the expense of the other IO.) Now that the bookkeeping is done is nanoseconds the shift behaves correctly for any tick frequency (HZ). Ported-by: Cyril Plisko <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1643
* Linux 3.8 compat: Support CONFIG_UIDGID_STRICT_TYPE_CHECKSRichard Yao2013-08-093-7/+7
| | | | | | | | | | | | | | | | When CONFIG_UIDGID_STRICT_TYPE_CHECKS is enabled uid_t/git_t are replaced by kuid_t/kgid_t, which are structures instead of integral types. This causes any code that uses an integral type to fail to build. The User Namespace functionality introduced in Linux 3.8 requires CONFIG_UIDGID_STRICT_TYPE_CHECKS, so we could not build against any kernel that supported it. We resolve this by converting between the new kuid_t/kgid_t structures and the original uid_t/gid_t types. Signed-off-by: Richard Yao <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #1589
* Evict meta data from ghost lists + l2arc headersBrian Behlendorf2013-08-091-1/+17
| | | | | | | | | | | | | | | | | When the meta limit is exceeded the ARC evicts some meta data buffers from the mfu+mru lists. Unfortunately, for meta data heavy workloads it's possible for these buffers to accumulate on the ghost lists if arc_c doesn't exceed arc_size. To handle this case arc_adjust_meta() has been entended to explicitly evict meta data buffers from the ghost lists in proportion to what was evicted from the mfu+mru lists. If this is insufficient we request that the VFS release some inodes and dentries. This will result in the release of some dnodes which are counted as 'other' metadata. Signed-off-by: Brian Behlendorf <[email protected]>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-21 17:50:56 +0000