mesa/vbo: Fix scaling issue in 10-bit signed normalized packing.

For the 10-bit components, the divisor was incorrect.  A 10-bit signed
integer can represent -2^9 through 2^9 - 1, which leads to the following
ranges:

       (float)value.x          -> [ -512,  511]
2.0F * (float)value.x          -> [-1024, 1022]
2.0F * (float)value.x + 1.0F   -> [-1023, 1023]

So dividing by 511 would incorrectly scale it to approximately:
[-2.001956947, 2.001956947].  To correctly scale to [-1.0, 1.0], we need
to divide by 1023.

This correctly implements the desktop GL rules.  ES 3.0 has different
rules, but those will be implemented in a separate patch.

Signed-off-by: Kenneth Graunke <[email protected]>
Tested-by: Marek Olšák <[email protected]>

1 files changed, 1 insertions, 1 deletions

diff --git a/src/mesa/vbo/vbo_attrib_tmp.h b/src/mesa/vbo/vbo_attrib_tmp.h
index 6bc53bab3d2..9a4df695d39 100644
--- a/src/mesa/vbo/vbo_attrib_tmp.h
+++ b/src/mesa/vbo/vbo_attrib_tmp.h
@@ -117,7 +117,7 @@ static inline float conv_i10_to_norm_float(int i10)
 {
    struct attr_bits_10 val;
    val.x = i10;
-   return (2.0F * (float)val.x + 1.0F) * (1.0F  / 511.0F);
+   return (2.0F * (float)val.x + 1.0F) * (1.0F  / 1023.0F);
 }
 
 static inline float conv_i2_to_norm_float(int i2)