glsl: don't allow non-flat integral types in varying structs/arrays.

In the GLSL 1.30 spec, section 4.3.6 ("Outputs") says:

    "If a vertex output is a signed or unsigned integer or integer
    vector, then it must be qualified with the interpolation qualifier
    flat."

The GLSL ES 3.00 spec further clarifies, in section 4.3.6 ("Output
Variables"):

    "Vertex shader outputs that are, *or contain*, signed or unsigned
    integers or integer vectors must be qualified with the
    interpolation qualifier flat."

(Emphasis mine.)

The language in the GLSL ES 3.00 spec is clearly correct and should be
applied to all shading language versions, since varyings that contain
ints can't be interpolated, regardless of which shading language
version is in use.

(Note that in GLSL 1.50 the restriction is changed to apply to
fragment shader inputs rather than vertex shader outputs, to
accommodate the fact that in the presence of geometry shaders, vertex
shader outputs are not necessarily interpolated.  That will be
addressed by a future patch).

NOTE: This is a candidate for stable branches.

Reviewed-by: Ian Romanick <[email protected]>
Reviewed-by: Anuj Phogat <[email protected]>
(cherry picked from commit 93c913485ea16d09b2d753c872607e79c0b68f6b)

3 files changed, 30 insertions, 6 deletions

diff --git a/src/glsl/ast_to_hir.cpp b/src/glsl/ast_to_hir.cpp
index 668973d4c2a..2ff44ada777 100644
--- a/src/glsl/ast_to_hir.cpp
+++ b/src/glsl/ast_to_hir.cpp
@@ -2829,9 +2829,9 @@ ast_declarator_list::hir(exec_list *instructions,
        *    flat."
        *
        * From section 4.3.4 of the GLSL 3.00 ES spec:
-       *    "Fragment shader inputs that are signed or unsigned integers or
-       *    integer vectors must be qualified with the interpolation qualifier
-       *    flat."
+       *    "Fragment shader inputs that are, or contain, signed or unsigned
+       *    integers or integer vectors must be qualified with the
+       *    interpolation qualifier flat."
        *
        * Since vertex outputs and fragment inputs must have matching
        * qualifiers, these two requirements are equivalent.
@@ -2839,12 +2839,12 @@ ast_declarator_list::hir(exec_list *instructions,
       if (state->is_version(130, 300)
           && state->target == vertex_shader
           && state->current_function == NULL
-          && var->type->is_integer()
+          && var->type->contains_integer()
           && var->mode == ir_var_shader_out
           && var->interpolation != INTERP_QUALIFIER_FLAT) {
 
-         _mesa_glsl_error(&loc, state, "If a vertex output is an integer, "
-                          "then it must be qualified with 'flat'");
+         _mesa_glsl_error(&loc, state, "If a vertex output is (or contains) "
+                          "an integer, then it must be qualified with 'flat'");
       }
 
 
diff --git a/src/glsl/glsl_types.cpp b/src/glsl/glsl_types.cpp
index 4a2c8790790..02d85b8f6e0 100644
--- a/src/glsl/glsl_types.cpp
+++ b/src/glsl/glsl_types.cpp
@@ -156,6 +156,24 @@ glsl_type::contains_sampler() const
    }
 }
 
+
+bool
+glsl_type::contains_integer() const
+{
+   if (this->is_array()) {
+      return this->fields.array->contains_integer();
+   } else if (this->is_record()) {
+      for (unsigned int i = 0; i < this->length; i++) {
+	 if (this->fields.structure[i].type->contains_integer())
+	    return true;
+      }
+      return false;
+   } else {
+      return this->is_integer();
+   }
+}
+
+
 gl_texture_index
 glsl_type::sampler_index() const
 {
diff --git a/src/glsl/glsl_types.h b/src/glsl/glsl_types.h
index b0db2bf11ce..8cfd8dd021d 100644
--- a/src/glsl/glsl_types.h
+++ b/src/glsl/glsl_types.h
@@ -360,6 +360,12 @@ struct glsl_type {
    }
 
    /**
+    * Query whether or not type is an integral type, or for struct and array
+    * types, contains an integral type.
+    */
+   bool contains_integer() const;
+
+   /**
     * Query whether or not a type is a float type
     */
    bool is_float() const