summaryrefslogtreecommitdiffstats
path: root/scripts
Commit message (Collapse)AuthorAgeFilesLines
* Add lustre zpios-test workloadBrian Behlendorf2010-11-083-20/+88
| | | | | | | | The lustre zpios-test simulates a reasonable lustre workload. It will create 128 threads, the same as a Lustre OSS, and then 4096 individual objects. Each objects is 16MiB in size and will be written/read in 1MiB from a random thread. This is fundamentally how we expect Lustre to behave for large IO intensive workloads.
* Replace custom zpool configs with generic configsBrian Behlendorf2010-11-0817-276/+261
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To streamline testing I have in the past added several custom configs to the zpool-config directory. This change reverts those custom configs and replaces them with three generic config which can do the same thing. The generic config behavior can be set by setting various environment variables when calling either the zpool-create.sh or zpios.sh scripts. For example if you wanted to create and test a single 4-disk Raid-Z2 configuration using disks [A-D]1 with dedicated ZIL and L2ARC devices you could run the following. $ ZIL="log A2" L2ARC="cache B2" RANKS=1 CHANNELS=4 LEVEL=2 \ zpool-create.sh -c zpool-raidz $ zpool status tank pool: tank state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 A1 ONLINE 0 0 0 B1 ONLINE 0 0 0 C1 ONLINE 0 0 0 D1 ONLINE 0 0 0 logs A2 ONLINE 0 0 0 cache B2 ONLINE 0 0 0 errors: No known data errors
* Add zconfig test for adding and removing vdevsNed Bass2010-10-221-0/+101
| | | | | | | | | | | | | This test performs a sanity check of the zpool add and remove commands. It tests adding and removing both a cache disk and a log disk to and from a zpool. Usage of both a shorthand device path and a full path is covered. The test uses a scsi_debug device as the disk to be added and removed. This is done so that zpool will see it as a whole disk and partition it, which it does not currently done for loopback devices. We want to verify that the manipulation done to whole disks paths to hide the parition information does not break the add/remove interface. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
* Add zfault zpool configurations and testsBrian Behlendorf2010-10-1222-66/+2069
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Eleven new zpool configurations were added to allow testing of various failure cases. The first 5 zpool configurations leverage the 'faulty' md device type which allow us to simuluate IO errors at the block layer. The last 6 zpool configurations leverage the scsi_debug module provided by modern kernels. This device allows you to create virtual scsi devices which are backed by a ram disk. With this setup we can verify the full IO stack by injecting faults at the lowest layer. Both methods of fault injection are important to verifying the IO stack. The zfs code itself also provides a mechanism for error injection via the zinject command line tool. While we should also take advantage of this appraoch to validate the code it does not address any of the Linux integration issues which are the most concerning. For the moment we're trusting that the upstream Solaris guys are running zinject and would have caught internal zfs logic errors. Currently, there are 6 r/w test cases layered on top of the 'faulty' md devices. They include 3 writes tests for soft/transient errors, hard/permenant errors, and all writes error to the device. There are 3 matching read tests for soft/transient errors, hard/permenant errors, and fixable read error with a write. Although for this last case zfs doesn't do anything special. The seventh test case verifies zfs detects and corrects checksum errors. In this case one of the drives is extensively damaged and by dd'ing over large sections of it. We then ensure zfs logs the issue and correctly rebuilds the damage. The next test cases use the scsi_debug configuration to injects error at the bottom of the scsi stack. This ensures we find any flaws in the scsi midlayer or our usage of it. Plus it stresses the device specific retry, timeout, and error handling outside of zfs's control. The eighth test case is to verify that the system correctly handles an intermittent device timeout. Here the scsi_debug device drops 1 in N requests resulting in a retry either at the block level. The ZFS code does specify the FAILFAST option but it turns out that for this case the Linux IO stack with still retry the command. The FAILFAST logic located in scsi_noretry_cmd() does no seem to apply to the simply timeout case. It appears to be more targeted to specific device or transport errors from the lower layers. The ninth test case handles a persistent failure in which the device is removed from the system by Linux. The test verifies that the failure is detected, the device is made unavailable, and then can be successfully re-add when brought back online. Additionally, it ensures that errors and events are logged to the correct places and the no data corruption has occured due to the failure.
* Add FAILFAST supportBrian Behlendorf2010-10-125-0/+5
| | | | | | | | | | | | | | | | | | | | ZFS works best when it is notified as soon as possible when a device failure occurs. This allows it to immediately start any recovery actions which may be needed. In theory Linux supports a flag which can be set on bio's called FAILFAST which provides this quick notification by disabling the retry logic in the lower scsi layers. That's the theory at least. In practice is turns out that while the flag exists you oddly have to set it with the BIO_RW_AHEAD flag. And even when it's set it you may get retries in the low level drivers decides that's the right behavior, or if you don't get the right error codes reported to the scsi midlayer. Unfortunately, without additional kernels patchs there's not much which can be done to improve this. Basically, this just means that it may take 2-3 minutes before a ZFS is notified properly that a device has failed. This can be improved and I suspect I'll be submitting patches upstream to handle this.
* Add missing Makefile.in from zpool_layout commitBrian Behlendorf2010-09-171-0/+520
| | | | | The scripts/zpool-layout/Makefile.in file generated by autogen.sh was accidentally omitted from the previous commit. Add it.
* Add [-m map] option to zpool_layoutBrian Behlendorf2010-09-175-2/+158
| | | | | | | | | | | | | | | | | By default the zpool_layout command would always use the slot number assigned by Linux when generating the zdev.conf file. This is a reasonable default there are cases when it makes sense to remap the slot id assigned by Linux using your own custom mapping. This commit adds support to zpool_layout to provide a custom slot mapping file. The file contains in the first column the Linux slot it and in the second column the custom slot mapping. By passing this map file with '-m map' to zpool_config the mapping will be applied when generating zdev.conf. Additionally, two sample mapping have been added which reflect different ways to map the slots in the dragon drawers.
* Wait up to timeout seconds for udev devicezfs-0.5.1Brian Behlendorf2010-09-113-6/+29
| | | | | | | | | | Occasional failures were observed in zconfig.sh because udev could be delayed for a few seconds. To handle this the wait_udev function has been added to wait for timeout seconds for an expected device before returning an error. By default callers currently use a 30 seconds timeout which should be much longer than udev ever needs but not so long to worry the test suite is hung.
* Reduce volume size in zconfig.shBrian Behlendorf2010-09-101-9/+9
| | | | | Due to occasional ENOSPC failures on certain platforms I've reduced the size of the ZVOL from 400M to 300M for the zvol+ext2 clone tests.
* Support custom build directories and move includesBrian Behlendorf2010-09-0810-274/+2016
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | One of the neat tricks an autoconf style project is capable of is allow configurion/building in a directory other than the source directory. The major advantage to this is that you can build the project various different ways while making changes in a single source tree. For example, this project is designed to work on various different Linux distributions each of which work slightly differently. This means that changes need to verified on each of those supported distributions perferably before the change is committed to the public git repo. Using nfs and custom build directories makes this much easier. I now have a single source tree in nfs mounted on several different systems each running a supported distribution. When I make a change to the source base I suspect may break things I can concurrently build from the same source on all the systems each in their own subdirectory. wget -c http://github.com/downloads/behlendorf/zfs/zfs-x.y.z.tar.gz tar -xzf zfs-x.y.z.tar.gz cd zfs-x-y-z ------------------------- run concurrently ---------------------- <ubuntu system> <fedora system> <debian system> <rhel6 system> mkdir ubuntu mkdir fedora mkdir debian mkdir rhel6 cd ubuntu cd fedora cd debian cd rhel6 ../configure ../configure ../configure ../configure make make make make make check make check make check make check This change also moves many of the include headers from individual incude/sys directories under the modules directory in to a single top level include directory. This has the advantage of making the build rules cleaner and logically it makes a bit more sense.
* Remove scripts/common.shBrian Behlendorf2010-09-011-443/+0
| | | | | | | | | This script is now dynamically generated at configure time from scripts/common.sh.in. This change was made by commit 26e61dd074df64f9e1d779273efd56fa9d92cdc5 but we accidentally kept the common.sh file around. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
* Add initial autoconf productsBrian Behlendorf2010-08-311-0/+549
| | | | | | | Add the initial products from autogen.sh. These products will be updated incrementally after this point as development occurs. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
* Add linux zpios supportBrian Behlendorf2010-08-3120-0/+1794
| | | | | | Linux kernel implementation of PIOS test app. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
* Add linux kernel device supportBrian Behlendorf2010-08-313-31/+531
| | | | | | | | | | | | | | | | This branch contains the majority of the changes required to cleanly intergrate with Linux style special devices (/dev/zfs). Mainly this means dropping all the Solaris style callbacks and replacing them with the Linux equivilants. This patch also adds the onexit infrastructure needed to track some minimal state between ioctls. Under Linux it would be easy to do this simply using the file->private_data. But under Solaris they apparent need to pass the file descriptor as part of the ioctl data and then perform a lookup in the kernel. Once again to keep code change to a minimum I've implemented the Solaris solution. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
* Add build systemBrian Behlendorf2010-08-3131-0/+1915
| | | | | | Add autoconf style build infrastructure to the ZFS tree. This includes autogen.sh, configure.ac, m4 macros, some scripts/*, and makefiles for all the core ZFS components.
* Update core ZFS code from build 121 to build 141.Brian Behlendorf2010-05-281-18/+17
|
* Update nvpair's to include nv_alloc_fixed supportBrian Behlendorf2010-04-291-0/+1
|
* Add Solaris FMA style supportBrian Behlendorf2010-04-291-3/+5
|
* Rename update-zfs.sh -> zfs-update.sh for consistencyBrian Behlendorf2010-03-111-0/+0
|
* Pull in latest man pages as part of update-zfs.shBrian Behlendorf2009-12-111-3/+27
| | | | | | | | | | | The script has been updated to download the latest documentations packages for Solaris and extract the needed ZFS man pages. These will still need a little markup to handle changes between the Solaris and Linux versions of ZFS. Howver, they should be pretty minor I've tried hard to keep the interface the same. In additional to the script update the zdb, zfs, and zpool man pages have been added to the repo.
* Remove zvol.c when updating in update-zfs.sh Linux version available.Brian Behlendorf2009-11-151-0/+1
|
* Script update-zfs.sh updated to include libefi libraryBrian Behlendorf2009-10-091-0/+6
|
* Add unicode libraryBrian Behlendorf2009-01-051-0/+9
|
* Remove zdump, it's an unrelateds command which I added simply due to the z* ↵Brian Behlendorf2009-01-051-4/+0
| | | | command convention
* Remove zcommon reference merged in to zpoolBrian Behlendorf2008-12-121-3/+0
|
* Moving lib/libspl to linux-libspl branchBrian Behlendorf2008-12-111-11/+0
|
* Script updatesBrian Behlendorf2008-12-111-2/+8
|
* Move library files to libBrian Behlendorf2008-12-111-5/+7
|
* Fix typoBrian Behlendorf2008-12-111-1/+1
|
* Add a few missing filesBrian Behlendorf2008-12-111-7/+9
|
* Move the world out of /zfs/ and seperate out module build treeBrian Behlendorf2008-12-111-49/+53
|
* Remove libumem, we will try and remove this dependency entirely. If we ↵Brian Behlendorf2008-12-101-7/+0
| | | | can't then the best move will simply be to use the official library, or build it as a convenience library
* Move vmem* to libumemBrian Behlendorf2008-12-091-4/+3
|
* Temporarily move taskq+util to libzpool until that directory is broken in to ↵Brian Behlendorf2008-12-091-3/+3
| | | | lib+module
* Minor tweak to update scriptBrian Behlendorf2008-12-081-1/+1
|
* Add userspace zfs_context fileBrian Behlendorf2008-12-031-0/+1
|
* Rebase to OpenSolaris b103, in the process we are removing any code which ↵Brian Behlendorf2008-12-031-38/+129
| | | | did not originate from the OpenSolaris source. These changes will be reintroduced in topic branches for easier tracking
* Finish removing all non-upstream bits from masterBrian Behlendorf2008-12-0112-1133/+0
|
* Removed build system from master branch, will relocate to linux-zfs-branchBrian Behlendorf2008-12-011-8/+0
|
* Update zpios for trivial workloadBrian Behlendorf2008-11-261-12/+6
|
* Initial Linux ZFS GIT RepoBrian Behlendorf2008-11-2014-0/+1206