diff options
author | Kenneth Graunke <[email protected]> | 2016-08-15 22:26:42 -0700 |
---|---|---|
committer | Kenneth Graunke <[email protected]> | 2016-08-17 17:26:11 -0700 |
commit | 01e99cba043084be7477e3a801029bfee581ab87 (patch) | |
tree | de4346ea7a1c2c8cc7b075831422e773813392de /src/compiler/nir/nir_search_helpers.h | |
parent | 0ff57446e3786243c6d752c91be2108595f2663e (diff) |
mesa: Fix uf10_to_f32() scale factor in the E == 0 and M != 0 case.
GL_EXT_packed_float, 2.1.B Unsigned 10-Bit Floating-Point Numbers:
0.0, if E == 0 and M == 0,
2^-14 * (M / 32), if E == 0 and M != 0,
2^(E-15) * (1 + M/32), if 0 < E < 31,
INF, if E == 31 and M == 0, or
NaN, if E == 31 and M != 0,
In the second case (E == 0 and M != 0), we were multiplying the mantissa
by 2^-20, when we should have been multiplying by 2^-19 (which is
2^(-14 + -5), or 2^-14 * 2^-5, or 2^-14 / 32).
The previous section defines the formula for 11-bit numbers, which is:
2^-14 * (M / 64), if E == 0 and M != 0,
In other words, we had accidentally copy and pasted the 11-bit code
to the 10-bit case, and neglected to change the exponent.
Fixes dEQP-GLES3.functional.pbo.renderbuffer.r11f_g11f_b10f_triangles
when run with surface dimensions of 1536x1152 or 1920x1080.
Cc: [email protected]
References: https://code.google.com/p/chrome-os-partner/issues/detail?id=56244
Signed-off-by: Kenneth Graunke <[email protected]>
Reviewed-by: Stephane Marchesin <[email protected]>
Reviewed-by: Antia Puentes <[email protected]>
Diffstat (limited to 'src/compiler/nir/nir_search_helpers.h')
0 files changed, 0 insertions, 0 deletions