aboutsummaryrefslogtreecommitdiffstats
path: root/tests/zfs-tests
Commit message (Collapse)AuthorAgeFilesLines
* ZTS: Fix send_partial_dataset.kshBrian Behlendorf2022-03-131-12/+10
| | | | | | | | | | | | | The send_partial_dataset test verifies that partial send streams can be resumed. This test may occasionally fail with a "token is corrupt" error if the `mess_send_file` truncates a send stream below the size of the DRR_BEGIN record. Update this function to set a minimum size to ensure there is at least an intact DDR_BEGIN record which allows for the receiving dataset to be created. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Rich Ercolani <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13177
* Add physical device size to SIZE column in 'zpool list -v'Akash B2022-03-081-1/+5
| | | | | | | | | | | | | Add physical device size/capacity only for physical devices in 'zpool list -v' instead of displaying "-" in the SIZE column. This would make it easier to see the individual device capacity and to determine which spares are large enough to replace which devices. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Dipak Ghosh <[email protected]> Signed-off-by: Akash B <[email protected]> Closes #12561 Closes #13106
* ZTS: /dev/null: accept no substitutesRich Ercolani2022-03-0815-42/+34
| | | | | | | | | | | Instead of writing to "devnull" and rming it later, just > /dev/null to not have to cleanup later. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Paul Dagnelie <[email protected]> Co-authored-by: Rich Ercolani <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13133
* Revert "ZTS: Avoid piping send directly to /dev/null"наб2022-03-081-3/+1
| | | | | | | | | | This reverts commit 1a79f7e86021c5de33d3518dd9a0f14f924ee345. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Paul Dagnelie <[email protected]> Reviewed-by: Rich Ercolani <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13133
* Cleaning up a couple of ZTS tests setup scriptsBrian Atkinson2022-03-082-3/+4
| | | | | | | | | | | | With the zfs_destroy ZTS test case the setup script needed to call default_setup_noexit so compression could be turned off. Also, added log_must to setting compression off in the reservation setup script for turning off compression. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Rich Ercolani <[email protected]> Signed-off-by: Brian Atkinson <[email protected]> Closes #13173
* Added noexit variant for Raidz setup in ZTS testsBrian Atkinson2022-03-083-7/+8
| | | | | | | | | | | | | | | | | | | The regular default_raidz_setup function in the ZFS test suite called log_pass after creating the zpool. However, with compression now being on by default 56fa4aa, there is no way to turn compression off in the setup.ksh scripts when creating a raidz VDEV. The addition of the function default_raidz_setup_noexit allows for a raidz VDEV to be created, additional zfs property settings to be applied and for the setup.ksh script itself to call log_pass. With the addition of default_raidz_setup_noexit some stray log_pass calls were removed from any setup.ksh scripts that call default_raidz_setup. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Rich Ercolani <[email protected]> Signed-off-by: Brian Atkinson <[email protected]> Closes #13173
* Fix ENOSPC when unlinking multiple files from full poolBrian Behlendorf2022-03-082-1/+62
| | | | | | | | | | | | | | | | | | | | | | | | When unlinking multiple files from a pool at 100% capacity, it was possible for ENOSPC to be returned after the first unlink. e.g. rm -f /mnt/fs/test1.0.0 /mnt/fs/test1.1.0 /mnt/fs/test1.2.0 rm: cannot remove '/mnt/fs/test1.1.0': No space left on device rm: cannot remove '/mnt/fs/test1.2.0': No space left on device After waiting for the pending deferred frees from the first unlink to be processed the remaining files can then be unlinked. This is caused by the quota limit in dsl_dir_tempreserve_impl() being temporarily decreased to the allocatable pool capacity less any deferred free space. This is resolved using the existing mechanism of returning ERESTART when over quota as long as we know enough space will shortly be available after processing the pending deferred frees. Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13172
* Expose additional file level attributesUmer Saleem2022-03-0717-43/+652
| | | | | | | | | | | | | | | | | | | | | ZFS allows to update and retrieve additional file level attributes for FreeBSD. This commit allows additional file level attributes to be updated and retrieved for Linux. These include the flags stored in the upper half of z_pflags only. Two new IOCTLs have been added for this purpose. ZFS_IOC_GETDOSFLAGS can be used to retrieve the attributes, while ZFS_IOC_SETDOSFLAGS can be used to update the attributes. Attributes that are allowed to be updated include ZFS_IMMUTABLE, ZFS_APPENDONLY, ZFS_NOUNLINK, ZFS_ARCHIVE, ZFS_NODUMP, ZFS_SYSTEM, ZFS_HIDDEN, ZFS_READONLY, ZFS_REPARSE, ZFS_OFFLINE and ZFS_SPARSE. Flags can be or'd together while calling ZFS_IOC_SETDOSFLAGS. Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Umer Saleem <[email protected]> Closes #13118
* Use _Noreturn (C11; GNU89) properlyAlejandro Colomar2022-03-042-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A function that returns with no value is a different thing from a function that doesn't return at all. Those are two orthogonal concepts, commonly confused. pthread_create(3) expects a pointer to a start routine that has a very precise prototype: void *(*start_routine)(void *); However, other thread functions, such as kernel ones, expect: void (*start_routine)(void *); Providing a different one is incorrect, and has only been working because the ABIs happen to produce a compatible function. We should use '_Noreturn void', since it's the natural type, and then provide a '_Noreturn void *' wrapper for pthread functions. For consistency, replace most cases of __NORETURN or __attribute__((noreturn)) by _Noreturn. _Noreturn is understood by -std=gnu89, so it should be safe to use everywhere. Ref: https://github.com/openzfs/zfs/pull/13110#discussion_r808450136 Ref: https://software.codidact.com/posts/285972 Reviewed-by: Brian Behlendorf <[email protected]> Co-authored-by: Ahelenia Ziemiańska <[email protected]> Signed-off-by: Alejandro Colomar <[email protected]> Closes #13120
* Default to ON for compressionRich Ercolani2022-03-0336-39/+119
| | | | | | | | | A simple change, but so many tests break with it, and those are the majority of this. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Rich Ercolani <[email protected]> Closes #13078
* ZTS: Fix import_devices_missing.kshBrian Behlendorf2022-03-021-2/+2
| | | | | | | | | | Related to commit 90b77a036. Retry the `zpool export` if the pool is "busy" indicating there is a process accessing the mount point. This can happen after an import, allowing it to be retried will avoid spurious test failures. Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13169
* ZTS: switch to rsync for directory diffsAleksa Sarai2022-03-0115-68/+148
| | | | | | | | | | | | | | | | | | | | | | | | | | | | While "diff -r" is the most straightforward way of comparing directory trees for differences, it has two major issues: * File metadata is not compared, which means that subtle bugs may be missed even if a test is written that exercises the buggy behaviour. * diff(1) doesn't know how to compare special files -- it assumes they are always different, which means that a test using diff(1) on special files will always fail (resulting in such tests not being added). rsync can be used in a very similar manner to diff (with the -ni flags), but has the additional benefit of being able to detect and resolve many more differences between directory trees. In addition, rsync has a standard set of features and flags while diffs feature set depends on whether you're using GNU or BSD binutils. Note that for several of the test cases we expect that file timestamps will not match. For example, the ctime for a file creation or modify event is stored in the intent log but not the mtime. Thus when replaying the log the correct ctime is set but the current mtime is used. This is the expected behavior, so to prevent these tests from failing, there's a replay_directory_diff function which ignores those kinds of changes. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Aleksa Sarai <[email protected]> Closes #12588
* log xattr=sa create/remove/update to ZILJitendra Patidar2022-02-224-1/+161
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As such, there are no specific synchronous semantics defined for the xattrs. But for xattr=on, it does log to ZIL and zil_commit() is done, if sync=always is set on dataset. This provides sync semantics for xattr=on with sync=always set on dataset. For the xattr=sa implementation, it doesn't log to ZIL, so, even with sync=always, xattrs are not guaranteed to be synced before xattr call returns to caller. So, xattr can be lost if system crash happens, before txg carrying xattr transaction is synced. This change adds xattr=sa logging to ZIL on xattr create/remove/update and xattrs are synced to ZIL (zil_commit() done) for sync=always. This makes xattr=sa behavior similar to xattr=on. Implementation notes: The actual logging is fairly straight-forward and does not warrant additional explanation. However, it has been 14 years since we last added new TX types to the ZIL [1], hence this is the first time we do it after the introduction of zpool features. Therefore, here is an overview of the feature activation and deactivation workflow: 1. The feature must be enabled. Otherwise, we don't log the new record type. This ensures compatibility with older software. 2. The feature is activated per-dataset, since the ZIL is per-dataset. 3. If the feature is enabled and dataset is not for zvol, any append to the ZIL chain will activate the feature for the dataset. Likewise for starting a new ZIL chain. 4. A dataset that doesn't have a ZIL chain has the feature deactivated. We ensure (3) by activating on the first zil_commit() after the feature was enabled. Since activating the features requires waiting for txg sync, the first zil_commit() after enabling the feature will be slower than usual. The downside is that this is really a conservative approximation: even if we never append a 'TX_SETSAXATTR' to the ZIL chain, we pay the penalty for feature activation. The upside is that the user is in control of when we pay the penalty, i.e., upon enabling the feature. We ensure (4) by hooking into zil_sync(), where ZIL destroy actually happens. One more piece on feature activation, since it's spread across multiple functions: zil_commit() zil_process_commit_list() if lwb == NULL // first zil_commit since zil_open zil_create() if no log block pointer in ZIL header: if feature enabled and not active: // CASE 1 enable, COALESCE txg wait with dmu_tx that allocated the log block else // log block was allocated earlier than this zil_open if feature enabled and not active: // CASE 2 enable, EXPLICIT txg wait else // already have an in-DRAM LWB if feature enabled and not active: // this happens when we enable the feature after zil_create // CASE 3 enable, EXPLICIT txg wait [1] https://github.com/illumos/illumos-gate/commit/da6c28aaf62fa55f0fdb8004aa40f88f23bf53f0 Reviewed-by: Matthew Ahrens <[email protected]> Reviewed-by: Christian Schwarz <[email protected]> Reviewed-by: Ahelenia Ziemiańska <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Jitendra Patidar <[email protected]> Closes #8768 Closes #9078
* ZTS: Retry in import_rewind_config_changed.kshBrian Behlendorf2022-02-201-9/+49
| | | | | | | | | | | | | | | | | | | As explained by the disclaimer in the test case, "This test can fail since nothing guarantees that old MOS blocks aren't overwritten." This behavior is expected and correct, but results in a flaky test case which is problematic for the CI. The best we can do to resolve this is to retry the sub-test which failed when the MOS blocks have clearly been overwritten. When testing failures were rare enough that a single retry should normally be sufficient. However, we allow up to five for good measure. Reviewed by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13119
* ZTS: Fix vdev_zaps_004_pos.kshBrian Behlendorf2022-02-171-0/+1
| | | | | | | | | | When attaching a vdev to a mirror wait for the resilver to complete before invoking `zdb` to inspect the pool. This ensures the pool is essentially idle which allows `zdb` to open the imported pool reliably. Reviewed-by: John Kennedy <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13112 Closes #6935
* Enable encrypted raw sending to pools with greater ashiftGeorge Amanakis2022-02-162-0/+194
| | | | | | | | | | | | | | | | | | | | | | Raw sending from pool1/encrypted with ashift=9 to pool2/encrypted with ashift=12 results to failure when mounting pool2/encrypted (Input/Output error). Notably, the opposite, raw sending from a greater ashift to a lower one does not fail. This happens because zio_compress_write() falsely checks only ZIO_FLAG_RAW_COMPRESS and not ZIO_FLAG_RAW_ENCRYPT which is also set in encrypted raw send streams. In this case it rounds up the psize and if not equal to the zio->io_size it modifies the block by zeroing out the extra bytes. Because this happens in a SA attr. registration object (type=46), the decryption fails upon mounting the filesystem, and zpool status falsely reports an error. Fix this by checking both ZIO_FLAG_RAW_COMPRESS and ZIO_FLAG_RAW_ENCRYPT before deciding whether to zero-pad a block. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: George Amanakis <[email protected]> Closes #13067 Closes #13074
* Cross-platform xattr user namespace compatibilityRyan Moeller2022-02-153-1/+95
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ZFS on Linux originally implemented xattr namespaces in a way that is incompatible with other operating systems. On illumos, xattrs do not have namespaces. Every xattr name is visible. FreeBSD has two universally defined namespaces: EXTATTR_NAMESPACE_USER and EXTATTR_NAMESPACE_SYSTEM. The system namespace is used for protected FreeBSD-specific attributes such as MAC labels and pnfs state. These attributes have the namespace string "freebsd:system:" prefixed to the name in the encoding scheme used by ZFS. The user namespace is used for general purpose user attributes and obeys normal access control mechanisms. These attributes have no namespace string prefixed, so xattrs written on illumos are accessible in the user namespace on FreeBSD, and xattrs written to the user namespace on FreeBSD are accessible by the same name on illumos. Linux has several xattr namespaces. On Linux, ZFS encodes the namespace in the xattr name for every namespace, including the user namespace. As a consequence, an xattr in the user namespace with the name "foo" is stored by ZFS with the name "user.foo" and therefore appears on FreeBSD and illumos to have the name "user.foo" rather than "foo". Conversely, none of the xattrs written on FreeBSD or illumos are accessible on Linux unless the name happens to be prefixed with one of the Linux xattr namespaces, in which case the namespace is stripped from the name. This makes xattrs entirely incompatible between Linux and other platforms. We want to make the encoding of user namespace xattrs compatible across platforms. A critical requirement of this compatibility is for xattrs from existing pools from FreeBSD and illumos to be accessible by the same names in the user namespace on Linux. It is also necessary that existing pools with xattrs written by Linux retain access to those xattrs by the same names on Linux. Making user namespace xattrs from Linux accessible by the correct names on other platforms is important. The handling of other namespaces is not required to be consistent. Add a fallback mechanism for listing and getting xattrs to treat xattrs as being in the user namespace if they do not match a known prefix. Do not allow setting or getting xattrs with a name that is prefixed with one of the namespace names used by ZFS on supported platforms. Allow choosing between legacy illumos and FreeBSD compatibility and legacy Linux compatibility with a new tunable. This facilitates replication and migration of pools between hosts with different compatibility needs. The tunable controls whether or not to prefix the namespace to the name. If the xattr is already present with the alternate prefix, remove it so only the new version persists. By default the platform's existing convention is used. Reviewed-by: Christian Schwarz <[email protected]> Reviewed-by: Ahelenia Ziemiańska <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #11919
* Rename fallthrough to zfs_fallthroughJorgen Lundman2022-02-151-1/+1
| | | | | | | | Unfortunately macOS has obj-C keyword "fallthrough" in the OS headers. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Damian Szuberski <[email protected]> Signed-off-by: Jorgen Lundman <[email protected]> Closes #13097
* ZTS: Fix checkpoint_ro_rewind.kshBrian Behlendorf2022-02-131-2/+2
| | | | | | | | | | Related to commit 90b77a036. Retry the `zpool export` if the pool is "busy" indicating there is a process accessing the mount point. This can happen after an import and allowing it to be retried will avoid spurious test failures. Reviewed by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13092
* ZTS: Fix zpool_expand_001_posBrian Behlendorf2022-02-131-2/+2
| | | | | | | | | | | | | | | | | The dRAID section of the zpool_expand_001_pos test would reliably fail because the calculated expansion size assumed the dRAID top-level vdev was created with a distributed spare. Create the vdev as expected to resolve the test failure. This test case flaw was accidentally caused by changing the default number of dRAID distributed spares from one to zero while dRAID was being developed. Additionally, remove zpool_expand_005_pos from the list of possible faulty tests. It appears to be passing consistently in my testing. Reviewed by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13091
* ZTS: Fix problem with zdb_objset_id testPaul Zuchowski2022-02-111-14/+16
| | | | | | | | | Use large numbers for datasets with numeric names to avoid name and id collisions. Sporadic test failures were observed when the test would create $TESTPOOL/100 with an objset ID of 100. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Paul Zuchowski <[email protected]> Closes #13087
* ZTS: Fix zvol_misc_volmode testBrian Behlendorf2022-02-091-18/+31
| | | | | | | | | | | | Changing volmode may need to remove minors, which could be open, so call udev_wait() before we "zfs set volmode=<value>". This ensures no udev process has the zvol open (i.e. blkid) and the kernel zvol_remove_minor_impl() function won't skip removing the in use device. Reviewed-by: John Kennedy <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13075
* Receive checks should allow unencrypted child datasetsAttila Fülöp2022-02-093-5/+91
| | | | | | | | | | | | | | | | | | | | dmu_recv_begin_check() unconditionally sets the DS_HOLD_FLAG_DECRYPT flag before calling dsl_dataset_hold_flags(). If the key on the receiving side isn't loaded or the send stream contains embedded blocks, the receive check fails for a stream which is perfectly valid and could be received without any problem. This seems like a remnant of the initial design, where unencrypted datasets below encrypted ones weren't allowed. Add a condition to set `DS_HOLD_FLAG_DECRYPT` only for encrypted datasets, modify an existing test to detect this regression and add a test for raw replication streams. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Amanakis <[email protected]> Co-authored-by: George Amanakis <[email protected]> Signed-off-by: Attila Fülöp <[email protected]> Closes #13033 Closes #13076
* Linux 5.16 compat: don't use XSTATE_XSAVE to save FPU stateAttila Fülöp2022-02-093-0/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | Linux 5.16 moved XSTATE_XSAVE and XSTATE_XRESTORE out of our reach, so add our own XSAVE{,OPT,S} code and use it for Linux 5.16. Please note that this differs from previous behavior in that it won't handle exceptions created by XSAVE an XRSTOR. This is sensible for three reasons. - Exceptions during XSAVE and XRSTOR can only occur if the feature is not supported or enabled or the memory operand isn't aligned on a 64 byte boundary. If this happens something else went terribly wrong, and it may be better to stop execution. - Previously we just printed a warning and didn't handle the fault, this is arguable for the above reason. - All other *SAVE instruction also don't handle exceptions, so this at least aligns behavior. Finally add a test to catch such a regression in the future. Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Attila Fülöp <[email protected]> Closes #13042 Closes #13059
* ZTS: Update enospc_002_pos test caseBrian Behlendorf2022-02-041-3/+0
| | | | | | | | | | | The on-disk cost of creating a snapshot or bookmark is sufficiently low that it is difficult to make it reliably fail even when the pool is "full". In order to avoid false positives remove these two checks from the test case. Reviewed-by: George Melikov <[email protected]> Reviewed-by: John Kennedy <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #13060
* Fix clearing set-uid and set-gid bits on a file when replying a writePawel Jakub Dawidek2022-02-037-79/+179
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | POSIX requires that set-uid and set-gid bits to be removed when an unprivileged user writes to a file and ZFS does that during normal operation. The problem arrises when the write is stored in the ZIL and replayed. During replay we have no access to original credentials of the process doing the write, so zfs_write() will be performed with the root credentials. When root is doing the write set-uid and set-gid bits are not removed from the file. To correct that, log a separate TX_SETATTR entry that removed those bits on first write to such file. Idea from: Christian Schwarz Add test for ZIL replay of setuid/setgid clearing. Improve various edge cases when clearing setid bits: - The setid bits can be readded during a single write, so make sure to check for them on every chunk write. - Log TX_SETATTR record at most once per transaction group (if the setid bits are keep coming back). - Move zfs_log_setattr() outside of zp->z_acl_lock. Reviewed-by: Dan McDonald <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Co-authored-by: Christian Schwarz <[email protected]> Signed-off-by: Pawel Jakub Dawidek <[email protected]> Closes #13027
* Add `--enable-asan` and `--enable-ubsan` switchesDamian Szuberski2022-02-0312-9/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | `configure` now accepts `--enable-asan` and `--enable-ubsan` switches which results in passing `-fsanitize=address` and `-fsanitize=undefined`, respectively, to the compiler. Those flags are enabled in GitHub workflows for ZTS and zloop. Errors reported by both instrumentations are corrected, except for: - Memory leak reporting is (temporarily) suppressed. The cost of fixing them is relatively high compared to the gains. - Checksum computing functions in `module/zcommon/zfs_fletcher*` have UBSan errors suppressed. It is completely impractical to enforce 64-byte payload alignment there due to performance impact. - There's no ASan heap poisoning in `module/zstd/lib/zstd.c`. A custom memory allocator is used there rendering that measure unfeasible. - Memory leaks detection has to be suppressed for `cmd/zvol_id`. `zvol_id` is run by udev with the help of `ptrace(2)`. Tracing is incompatible with memory leaks detection. Reviewed-by: Ahelenia Ziemiańska <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: szubersk <[email protected]> Closes #12928
* Add enumerated vdev names to 'zpool iostat -v' and 'zpool list -v'Akash B2022-02-032-2/+2
| | | | | | | | | | | | This commit adds enumerated names to disambiguate between the different vdevs. Previously only 'zpool status' showed enumerated vdev names, now 'zpool list -v' and 'zpool iostat -v' also shows the enumerated vdev names. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Dipak Ghosh <[email protected]> Signed-off-by: Akash B <[email protected]> Closes #12510 Closes #13031
* Report dnodes with faulty bonuslenGeorge Amanakis2022-02-032-0/+162
| | | | | | | | | | | | | | | | | | | In files created/modified before 4254acb there may be a corruption of xattrs which is not reported during scrub and normal send/receive. It manifests only as an error when raw sending/receiving. This happens because currently only the raw receive path checks for discrepancies between the dnode bonus length and the spill pointer flag. In case we encounter a dnode whose bonus length is greater than the predicted one, we should report an error. Modify in this regard dnode_sync() with an assertion at the end, dump_dnode() to error out, dsl_scan_recurse() to report errors during a scrub, and zstream to report a warning when dumping. Also added a test to verify spill blocks are sent correctly in a raw send. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: George Amanakis <[email protected]> Closes #12720 Closes #13014
* Fix trivial calloc(3) arguments orderTomohiro Kusumi2022-02-021-1/+1
| | | | | | | Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Tomohiro Kusumi <[email protected]> Closes #13052
* ZTS: explicitly strip whitespace for broken wc(1) implementationsнаб2022-01-283-1/+3
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13032
* tests: simplify check_bg_procs_limit_num(), inline into setupнаб2022-01-264-57/+18
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #12979
* tests: prune remaining xargs(1), add missing zfs-project -c0 noteнаб2022-01-267-13/+9
| | | | | | | | | | -c0 suppresses diagnoses ‒ it's not just -c but with NULs; cf. http://build.zfsonlinux.org/builders/Debian%2010%20x86_64%20%28TEST%29/builds/10605/steps/shell_4/logs/log search for the second "zfs project -s -p" instance Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #12979
* tests: simplify find_vfstab_dev()наб2022-01-262-32/+2
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #12979
* Introduce a flag to skip comparing the local mac when raw sendingGeorge Amanakis2022-01-212-1/+121
| | | | | | | | | | | | | | | | | | | | | | | | | Raw receiving a snapshot back to the originating dataset is currently impossible because of user accounting being present in the originating dataset. One solution would be resetting user accounting when raw receiving on the receiving dataset. However, to recalculate it we would have to dirty all dnodes, which may not be preferable on big datasets. Instead, we rely on the os_phys flag OBJSET_FLAG_USERACCOUNTING_COMPLETE to indicate that user accounting is incomplete when raw receiving. Thus, on the next mount of the receiving dataset the local mac protecting user accounting is zeroed out. The flag is then cleared when user accounting of the raw received snapshot is calculated. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: George Amanakis <[email protected]> Closes #12981 Closes #10523 Closes #11221 Closes #11294 Closes #12594 Issue #11300
* Fix problem with zdb -dPaul Zuchowski2022-01-202-4/+40
| | | | | | | | | | | | | | | | | | zdb -d <pool>/<objset ID> does not work when other command line arguments are included i.e. zdb -U <cachefile> -d <pool>/<objset ID> This change fixes the command line parsing to handle this situation. Also fix issue where zdb -r <dataset> <file> does not handle the root <dataset> of the pool. Introduce -N option to force <objset ID> to be interpreted as a numeric objsetID. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Rich Ercolani <[email protected]> Reviewed-by: Tony Nguyen <[email protected]> Signed-off-by: Paul Zuchowski <[email protected]> Closes #12845 Closes #12944
* Removed Python 2 and Python 3.5- supportDamian Szuberski2022-01-135-20/+4
| | | | | | | | | | | | Deprecation of Python versions below 3.6 gives opportunity to unify the build and install requirements for OpenZFS packages. The minimal supported Python version is 3.6 as this is the most recent Python package CentOS/RHEL 7 users can get. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Rich Ercolani <[email protected]> Reviewed-by: John Kennedy <[email protected]> Signed-off-by: szubersk <[email protected]> Closes #12925
* ZTS: normalize on use of `sync_pool` and `sync_all_pools`Allan Jude2022-01-0695-118/+148
| | | | | | | | | | | | - Replaces use of manual `zpool sync` - Don't use `log_must sync_pool` as `sync_pool` uses it internally - Replace many (but not all) uses of `sync` with `sync_pool` This makes the tests more consistent, and makes searching easier. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Allan Jude <[email protected]> Closes #12894
* Verify dRAID empty sectorsBrian Behlendorf2022-01-041-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | Verify that all empty sectors are zero filled before using them to calculate parity. Failure to do so can result in incorrect parity columns being generated and written to disk if the contents of an empty sector are non-zero. This was possible because the checksum only protects the data portions of the buffer, not the empty sector padding. This issue has been addressed by updating raidz_parity_verify() to check that all dRAID empty sectors are zero filled. Any sectors which are non-zero will be fixed, repair IO issued, and a checksum error logged. They can then be safely used to verify the parity. This specific type of damage is unlikely to occur since it requires a disk to have silently returned bad data, for an empty sector, while performing a scrub. However, if a pool were to have been damaged in this way, scrubbing the pool with this change applied will repair both the empty sector and parity columns as long as the data checksum is valid. Checksum errors will be reported in the `zpool status` output for any repairs which are made. Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Mark Maybee <[email protected]> Reviewed-by: Brian Atkinson <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #12857
* ZTS: Fix enospc_002_pos.ksh againBrian Behlendorf2021-12-231-1/+1
| | | | | | | | | | | | | | | | | | | This is a follow up commit for e03a41a60 which aimed to resolve this same test failure. The core "problem" here is that it takes very little space to perform a clone/snapshot/bookmark, which means if we want these commands to reliably fail the pool must truely have exhausted all free space. This commit increases the number of fill iterations to try and consume every block which we can. This still can't guarantee the clone/snapshot/bookmark will fail, but it significantly improves the odds. The exception was kept since it's still not a sure thing. Reviewed-by: George Melikov <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Igor Kozhukhov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #12903
* ZTS: Fix rollback_003_pos.kshBrian Behlendorf2021-12-222-7/+17
| | | | | | | | | | | | | | | | | | Under Linux when rolling back a mounted filesystem negative dentries may not be dropped from the cache. This can result in an ENOENT being incorrectly returned on first access. Issuing a `df` before the unmount results in the negative dentries being invalidated and side steps the issue. This is solely a workaround for the test case on Linux and not correct behavior. The core issue of invalidating negative dentries needs to be handled with a kernel side change. This is being tracked as issue #6143. Reviewed-by: George Melikov <[email protected]> Reviewed-by: John Kennedy <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #12898 Issue #6143
* ZTS: Fix refreserv_raidz.kshBrian Behlendorf2021-12-221-1/+1
| | | | | | | | | | The rerefreserv_raidz test was failing on Linux because the sync being issued doesn't guarantee a pool sync. Switch to using the sync_pool function and remove the ZTS exception for Linux. Reviewed-by: George Melikov <[email protected]> Reviewed-by: John Kennedy <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #12897
* zfs-test/mmap_seek: fix build on muslGeorgy Yakovlev2021-12-211-0/+4
| | | | | | | | The build on musl needs linux/fs.h for SEEK_DATA and friends, and sys/sysmacros.h for P2ROUNDUP. Add the needed headers. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Georgy Yakovlev <[email protected]> Closes #12891
* tests: fix unused, remove argsusedнаб2021-12-2114-21/+18
| | | | | | Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #12835
* ZTS: speed up rsend testsBrian Behlendorf2021-12-213-11/+3
| | | | | | | | | | | | | | | | | | | | | With some minor tweaks several of rsend tests can be sped up considerably without significantly reducing test coverage. * send-c_verify_ratio: ~120s -> ~60s * send_realloc_*_files: ~330s -> ~65s For the send_realloc* tests this also has the advantage of removing (most of) the linux/freebsd conditional logic. Note that for this test more passes, and thus more incremental send/recvs, are preferable to a larger number of files. Total run time of the rsend test group was reduced from roughly 20 to 11 minutes in an environment similar to what's used by the CI. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #12876
* ZTS: rsend_007_pos failuresBrian Behlendorf2021-12-211-0/+1
| | | | | | | | | | | | | | | | | | | | | | | The rsend_007_pos test reliably fails on Linux in the cleanup function. This is caused by an unmount error when attempting to recursively destroy the newly received datasets. Invoking `df` prior to the `zfs destroy` interestingly avoids the unmont error. Why this should matter is unclear and should be investigated. However, this minor tweak may allow us to remove the ZTS rsend exceptions. The subsequent rsend_010_pos and rsend_011_pos failures were a result of this initial failure. The other "maybe" failures I was unable to reproduce and have not been recently observed in the master branch. Reviewed-by: Tony Nguyen <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #5665 Closes #6086 Closes #6087 Closes #6446 Closes #12876
* Linux: Implement FS_IOC_GETVERSIONRyan Moeller2021-12-1711-0/+207
| | | | | | | | | | Provide access to file generation number on Linux. Add test coverage. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ahelenia Ziemiańska <[email protected]> Signed-off-by: Ryan Moeller <[email protected]> Closes #12856
* ZTS: alloc_class.ksh must wait for the process to exitBrian Behlendorf2021-12-171-0/+1
| | | | | | | | | | | | | | The alloc_class_* tests may fail on Linux with an EBUSY error if `zfs destroy` is run before the `dd` process has had a chance to terminate. Wait on the pid after the `kill -9` to make sure. When testing I didn't observe any failures for the alloc_class tests. Remove them from the exceptions list, the CI was used to verify the tests pass on all platforms. Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Rich Ercolani <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes #12873
* ZTS: Avoid piping send directly to /dev/nullRich Ercolani2021-12-171-1/+3
| | | | | | | | | | | | Unfortunately, #11445 means while we fail gracefully now, we still fail, unless people want to implement a complex workaround just to support /dev/null. So let's just use the cheap workaround in a test for now. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Signed-off-by: Rich Ercolani <[email protected]> Closes #12872
* ZTS: Fix zpool_reopen_[1-5] on Fedora 35Tony Hutter2021-12-171-2/+2
| | | | | | | | | | | | | The zpool_reopen_[1-5] tests are failing Fedora 35 with: zpool_reopen_001_pos.ksh[64]: log_must[67]: log_pos[270]: wait_for_resilver_end[98]: wait_for_action: line 71: func: is read only Renaming 'func' -> 'funct' fixes the issue. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes #12871
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-27 16:50:01 +0000