From a485ad0bcdcab865e14a54133a271198c86e41ab Mon Sep 17 00:00:00 2001 From: Roland Scheidegger Date: Wed, 13 Dec 2017 03:33:21 +0100 Subject: gallivm: fix an issue with NaNs with seamless cube filtering Cube texture wrapping is a bit special since the values (post face projection) always are within [0,1], so we took advantage of that and omitted some clamps. However, we can still get NaNs (either because the coords already had NaNs, or the face projection generated them), and in fact we didn't handle them quite safely. I've seen -INT_MAX + 1 been propagated through as the final int coord value, albeit I didn't observe a crash. (Not quite a coincidence, since any stride mul with -INT_MAX or -INT_MAX+1 will turn up as a small positive number - nevertheless, I'd rather not try my luck, I'm not entirely sure it can't really turn up negative neither due to seamless coord swapping, plus ifloor of a NaN is not guaranteed to return -INT_MAX by any standard. And we kill off NaNs similarly with ordinary texture wrapping too.) So kill off the NaNs by using the common max against zero method. Reviewed-by: Brian Paul Reviewed-by: Jose Fonseca --- src/gallium/auxiliary/gallivm/lp_bld_sample_soa.c | 11 +++++++++++ 1 file changed, 11 insertions(+) (limited to 'src') diff --git a/src/gallium/auxiliary/gallivm/lp_bld_sample_soa.c b/src/gallium/auxiliary/gallivm/lp_bld_sample_soa.c index def731e9d95..6b1509c7cfa 100644 --- a/src/gallium/auxiliary/gallivm/lp_bld_sample_soa.c +++ b/src/gallium/auxiliary/gallivm/lp_bld_sample_soa.c @@ -1130,6 +1130,17 @@ lp_build_sample_image_linear(struct lp_build_sample_context *bld, */ /* should always have normalized coords, and offsets are undefined */ assert(bld->static_sampler_state->normalized_coords); + /* + * The coords should all be between [0,1] however we can have NaNs, + * which will wreak havoc. In particular the y1_clamped value below + * can be -INT_MAX (on x86) and be propagated right through (probably + * other values might be bogus in the end too). + * So kill off the NaNs here. + */ + coords[0] = lp_build_max_ext(coord_bld, coords[0], coord_bld->zero, + GALLIVM_NAN_RETURN_OTHER_SECOND_NONNAN); + coords[1] = lp_build_max_ext(coord_bld, coords[1], coord_bld->zero, + GALLIVM_NAN_RETURN_OTHER_SECOND_NONNAN); coord = lp_build_mul(coord_bld, coords[0], flt_width_vec); /* instead of clamp, build mask if overflowed */ coord = lp_build_sub(coord_bld, coord, half); -- cgit v1.2.3