configure.ac: add --enable-sanitize option

Enable code sanitizers by adding -fsanitize=$foo flags for the compiler
and linker.

In addition, this also disables checking for undefined symbols: running
the address sanitizer requires additional symbols which should be provided
by a preloaded libasan.so (preloaded for hooking into malloc & friends
globally), and the undefined symbols check gets tripped up by that.

Running the tests works normally via `make check`, but shows additional
failures with the address sanitizer due to memory leaks that seem to be
mostly leaks in the tests themselves. I believe those failures should
really be fixed. In the mean-time, you can set

export ASAN_OPTIONS=detect_leaks=0

to only check for more serious error types.

v2:
- fail reasonably when an unsupported sanitize flag is given (Eric Engestrom)

Reviewed-by: Bartosz Tomczyk <[email protected]> (v1)
Reviewed-by: Eric Engestrom <[email protected]>
Reviewed-by: Emil Velikov <[email protected]>

1 files changed, 27 insertions, 1 deletions

diff --git a/configure.ac b/configure.ac
index 7246c6017a2..957991cef7f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -515,6 +515,12 @@ AC_ARG_ENABLE([profile],
     [enable_profile=no]
 )
 
+AC_ARG_ENABLE([sanitize],
+    [AS_HELP_STRING([--enable-sanitize@<:@=address|undefined@:>@],
+        [enable code sanitizer @<:@default=disabled@:>@])],
+    [enable_sanitize="$enableval"],
+    [enable_sanitize=no])
+
 if test "x$enable_profile" = xyes; then
     DEFINES="$DEFINES -DPROFILE"
     if test "x$GCC" = xyes; then
@@ -550,6 +556,21 @@ else
    DEFINES="$DEFINES -DNDEBUG"
 fi
 
+if test "x$enable_sanitize" != xno; then
+    if test "x$enable_profile" = xyes; then
+        AC_MSG_WARN([Sanitize and Profile are enabled at the same time])
+    fi
+
+    CFLAGS="$CFLAGS -fsanitize=$enable_sanitize"
+    CXXFLAGS="$CXXFLAGS -fsanitize=$enable_sanitize"
+    LDFLAGS="$LDFLAGS -fsanitize=$enable_sanitize"
+
+    AC_LINK_IFELSE(
+        [AC_LANG_SOURCE([int main(){return 0;}])],
+        [],
+        [AC_MSG_FAILURE([sanitize flags '$enable_sanitize' not supported])])
+fi
+
 dnl
 dnl Check if linker supports -Bsymbolic
 dnl
@@ -590,7 +611,12 @@ case "$host_os" in
 openbsd* | darwin* )
     LD_NO_UNDEFINED="" ;;
 *)
-    LD_NO_UNDEFINED="-Wl,--no-undefined" ;;
+    if test "x$enable_sanitize" = xno; then
+        LD_NO_UNDEFINED="-Wl,--no-undefined"
+    else
+        LD_NO_UNDEFINED=""
+    fi
+    ;;
 esac
 
 AC_SUBST([LD_NO_UNDEFINED])