Add `--enable-asan` and `--enable-ubsan` switches

`configure` now accepts `--enable-asan` and `--enable-ubsan` switches
which results in passing `-fsanitize=address`
and `-fsanitize=undefined`, respectively, to the compiler. Those
flags are enabled in GitHub workflows for ZTS and zloop. Errors
reported by both instrumentations are corrected, except for:

- Memory leak reporting is (temporarily) suppressed. The cost of
  fixing them is relatively high compared to the gains.

- Checksum computing functions in `module/zcommon/zfs_fletcher*`
  have UBSan errors suppressed. It is completely impractical
  to enforce 64-byte payload alignment there due to performance
  impact.

- There's no ASan heap poisoning in `module/zstd/lib/zstd.c`. A custom
  memory allocator is used there rendering that measure
  unfeasible.

- Memory leaks detection has to be suppressed for `cmd/zvol_id`.
  `zvol_id` is run by udev with the help of `ptrace(2)`. Tracing is
  incompatible with memory leaks detection.

Reviewed-by: Ahelenia Ziemiańska <[email protected]>
Reviewed-by: George Melikov <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: szubersk <[email protected]>
Closes #12928 

1 files changed, 15 insertions, 0 deletions

diff --git a/cmd/zvol_id/zvol_id_main.c b/cmd/zvol_id/zvol_id_main.c
index 22f2e848c..929a1a6e7 100644
--- a/cmd/zvol_id/zvol_id_main.c
+++ b/cmd/zvol_id/zvol_id_main.c
@@ -35,6 +35,21 @@
 #include <sys/zfs_znode.h>
 #include <sys/fs/zfs.h>
 
+#if defined(ZFS_ASAN_ENABLED)
+/*
+ * zvol_id is invoked by udev with the help of ptrace()
+ * making sanitized binary with leak detection croak
+ * because of tracing mechanisms collision
+ */
+extern const char *__asan_default_options(void);
+
+const char *__asan_default_options(void) {
+	return ("abort_on_error=true:halt_on_error=true:"
+		"allocator_may_return_null=true:disable_coredump=false:"
+		"detect_stack_use_after_return=true:detect_leaks=false");
+}
+#endif
+
 static int
 ioctl_get_msg(char *var, int fd)
 {