From 605f964d5cc7016fc74e0563829fa794da845c20 Mon Sep 17 00:00:00 2001 From: Kenneth Graunke Date: Mon, 13 Aug 2012 00:35:41 -0700 Subject: mesa: Use GLdouble for depthMax in final unpack conversions. The final step of _mesa_unpack_depth_span is to take the temporary GLfloat depth values and convert them to the desired format. When converting to GL_UNSIGNED_INTEGER with depthMax > 0xffffff, we use double-precision math to avoid overflow and precision problems. Or at least that's the idea. Unfortunately GLdouble z = depthValues[i] * (GLfloat) depthMax; actually causes single-precision multiplication, since both operands are GLfloats. Casting depthMax to GLdouble causes the scaling to be done with double-precision math. Fixes a regression in oglconform's depth-stencil basic.read.ds test since c60ac7b17993d28af65b04f9bbbf3ee74c35358c, where the expected and actual values differed slightly. For example, 0xcfa7a6 vs. 0xcfa7a4. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=49772 Signed-off-by: Kenneth Graunke --- src/mesa/main/pack.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'src/mesa/main/pack.c') diff --git a/src/mesa/main/pack.c b/src/mesa/main/pack.c index 83192c1574c..672a467aa1c 100644 --- a/src/mesa/main/pack.c +++ b/src/mesa/main/pack.c @@ -4913,7 +4913,7 @@ _mesa_unpack_depth_span( struct gl_context *ctx, GLuint n, else { /* need to use double precision to prevent overflow problems */ for (i = 0; i < n; i++) { - GLdouble z = depthValues[i] * (GLfloat) depthMax; + GLdouble z = depthValues[i] * (GLdouble) depthMax; if (z >= (GLdouble) 0xffffffff) zValues[i] = 0xffffffff; else -- cgit v1.2.3