summaryrefslogtreecommitdiffstats
path: root/src/panfrost
Commit message (Collapse)AuthorAgeFilesLines
* pan/midgard: Do not repeatedly spill same valueAlyssa Rosenzweig2019-10-161-2/+14
| | | | | | | It doesn't make sense. You already spilled it once, and it didn't help. Don't try again, or you'll end up in a loop. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Fix memory corruption in register spillingAlyssa Rosenzweig2019-10-161-2/+2
| | | | | | | Essentially an off-by-one error ... bit of an edge case, but seems to occur in some glamor shaders. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Use 16-bit liveness masksAlyssa Rosenzweig2019-10-163-15/+14
| | | | | | We'll want liveness per-byte, so we need to accomodate up to 16 bytes. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Fix mir_mask_of_read_components with dot productsAlyssa Rosenzweig2019-10-151-5/+5
| | | | Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add perspective ops to mir_get_swizzleAlyssa Rosenzweig2019-10-151-0/+6
| | | | | | I really need to just make this a table.. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Don't try to propagate swizzles to branchesAlyssa Rosenzweig2019-10-151-2/+2
| | | | Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Allow non-contiguous masks in UBO loweringAlyssa Rosenzweig2019-10-151-7/+2
| | | | | | | We don't really need to impose this condition, but we do need to cope with the slightly more general case. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Report read mask for branch argumentsAlyssa Rosenzweig2019-10-151-0/+4
| | | | | | Conditionals in particular read values. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Replace mir_is_live_after with new passAlyssa Rosenzweig2019-10-031-57/+15
| | | | | | | | Now that we have live_out calculated per block as metadata, calculating liveness of an instruction at a given point in the program becomes O(n) to the size of the block worst-case, rather than O(n) the program. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Calculate temp_count for livenessAlyssa Rosenzweig2019-10-032-1/+3
| | | | | | This needs to be correct or the analysis fails. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Invalidate liveness for mir_is_live_afterAlyssa Rosenzweig2019-10-033-0/+6
| | | | | | | | | | Callers should have liveness info ready. Ideally we'd have a nice metadata tracking framework like NIR to handle this automatically, but for now this will allow us to make forward progress... when we're about to do something with liveness, invalidate everything ahead to force a clean calculation. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Begin tracking liveness metadataAlyssa Rosenzweig2019-10-034-5/+39
| | | | | | | This will allow us to explicitly invalidate liveness analysis results so we can cache liveness results. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Don't try to OR live_in of successorsAlyssa Rosenzweig2019-10-031-6/+2
| | | | | | | | By definition, once liveness analysis has occurred: live_out = OR {succ} succ->live_in Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Move RA's liveness analysis into midgard_liveness.cAlyssa Rosenzweig2019-10-033-122/+129
| | | | | | | | | | | There are unfortunately two distinct liveness analysis passes in the compiler right now -- one good (but complex) pass used by RA based on solving data flow equations, and one awful (but simple) pass used for dead code elimination and bundling based on an abstract walk of the AST. Let's move RA's pass into shared code so we can work on unifying. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add mir_calculate_temp_count helperAlyssa Rosenzweig2019-10-032-0/+19
| | | | | | | This allows us to fill in ctx->temp_count explicitly, even if we haven't squished down the MIR. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Remove mir_has_multiple_writesAlyssa Rosenzweig2019-10-033-18/+0
| | | | | | | We already enforce this with the SSA/register distinction in the backend. There is no need to duplicate this logic merely for an assert. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Allow scheduling conditions with constantsAlyssa Rosenzweig2019-09-301-4/+10
| | | | | | | | Now that we have constant adjustment logic abstracted, we can do this safely. Along with the csel inversion patch, this allows many more common csel ops to inline their condition in the bundle. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add csel invert optimizationAlyssa Rosenzweig2019-09-303-0/+27
| | | | Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add mir_flip helperAlyssa Rosenzweig2019-09-303-10/+21
| | | | | | | Useful for various operations on both commutative and anticommutative ops. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Tightly pack 32-bit constantsAlyssa Rosenzweig2019-09-301-16/+113
| | | | | | | If we can reuse constant slots from other instructions, we would like to do so to include more instructions per bundle. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Allow writeout to see into the futureAlyssa Rosenzweig2019-09-301-1/+40
| | | | | | | | If an instruction could be scheduled to vmul to satisfy the writeout conditions, let's do that and save an instruction+cycle per fragment shader. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Allow 6 instructions per bundleAlyssa Rosenzweig2019-09-301-2/+3
| | | | | | We never had a scheduler good enough to hit this case before! :) Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Only one conditional per bundle allowedAlyssa Rosenzweig2019-09-301-0/+16
| | | | | | There's no r32 to save ya after you use up r31 :) Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Schedule to smul/saddAlyssa Rosenzweig2019-09-301-0/+5
| | | | Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Extend choose_instruction for scalar unitsAlyssa Rosenzweig2019-09-301-0/+4
| | | | Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Don't double check SCALAR unitsAlyssa Rosenzweig2019-09-301-4/+0
| | | | Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Use new schedulerAlyssa Rosenzweig2019-09-303-678/+130
| | | | | | | | We still emit in-order but we switch to using the bundles created from the new scheduler, which will allow greater flexibility and room for out-of-order optimization. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add distance metric to choose_instructionAlyssa Rosenzweig2019-09-301-0/+14
| | | | | | | | | | | | | | We require chosen instructions to be "close", to avoid ballooning register pressure. This is a kludge that will go away once we have proper liveness tracking in the scheduler, but for now it prevents a lot of needless spilling. v2: Lower threshold to 6 (from 8). Schedule is hurt, but a few shaders that spilled excessively are fixed. Signed-off-by: Alyssa Rosenzweig <[email protected]> Derp
* pan/midgard: Add mir_choose_alu helperAlyssa Rosenzweig2019-09-301-0/+24
| | | | | | Based on a given unit. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Implement load/store pairingAlyssa Rosenzweig2019-09-301-55/+12
| | | | | | | We can bundle two load/store together. This eliminates the need for explicit load/store pairing in a prepass, as well. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Extend csel_swizzle to branchesAlyssa Rosenzweig2019-09-303-5/+10
| | | | | | | | Conditions for branches don't have a swizzle explicitly in the emitted binary, but they do implicitly get swizzled in whatever instruction wrote r31, so we need to handle that. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add helpers for scheduling conditionalsAlyssa Rosenzweig2019-09-301-0/+146
| | | | | | | | | | | | | Conditional instructions (csel and conditional branches) require their condition to be written to a special condition pipeline register (r31.w for scalar, r31.xyzw for vector). However, pipeline registers are live only for the duration of a single bundle. As such, the logic to schedule conditionals correct is surprisingly complex. Essentially, we see if we could stuff the conditional within the same bundle as the csel/branch without breaking anything; if we can, we do that. If we can't, we add a dummy move to make room. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Implement predicate->unitAlyssa Rosenzweig2019-09-301-0/+9
| | | | | | This allows ALUs to select for each unit of the bundle separately. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add predicate->excludeAlyssa Rosenzweig2019-09-301-4/+14
| | | | | | | | | | | | | | | A bit of a kludge but allows setting an implicit dependency of synthetic conditional moves on the actual condition, fixing code generated like: vmul.feq r0, .. sadd.imov r31, .., r0 vadd.fcsel [...] The imov runs simultaneous with feq so it gets garbage results, but it's too late to add an actual dependency practically speaking, since the new synthetic imov doesn't have a node associated. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add constant intersection filtersAlyssa Rosenzweig2019-09-301-0/+55
| | | | | | | | | | In the future, we will want to keep track of which components of constants of various sizes correspond to which parts of the bundle constants, like in the old scheduler. For now, let's just stub it out for a simple rule of one instruction with embedded constants per bundle. We can eventually do better, of course. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Remove csel constant unit forceAlyssa Rosenzweig2019-09-301-3/+0
| | | | Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add mir_schedule_texture/ldst/alu helpersAlyssa Rosenzweig2019-09-301-0/+190
| | | | | | | We don't actually do any scheduling here yet, but add per-tag helpers to consume an instruction, print it, pop it off the worklist. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add mir_choose_bundle helperAlyssa Rosenzweig2019-09-301-0/+25
| | | | | | | | | It's not always obvious what the optimal bundle type should be. Let's break out the logic to decide. Currently set for purely in-order operation. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add mir_update_worklist helperAlyssa Rosenzweig2019-09-301-0/+39
| | | | | | | | After we've chosen an instruction, popped it off, and processed it, it's time to update the worklist, removing that instruction from the dependency graph to allow its dependents to be put onto the worklist. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add mir_choose_instruction stubAlyssa Rosenzweig2019-09-301-0/+55
| | | | | | | | | | | | | In the future, this routine will implement the core scheduling logic to decide which instruction out of the worklist will be scheduled next, in a way that minimizes cycle count and register pressure. In the present, we are more interested in replicating in-order scheduling with the much-more-powerful out-of-order model. So rather than discriminating by a register pressure estimate, we simply choose the latest possible instruction in the worklist. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Initialize worklistAlyssa Rosenzweig2019-09-301-0/+17
| | | | | | This flows naturally from the dependency graph Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Calculate dependency graphAlyssa Rosenzweig2019-09-302-0/+131
| | | | Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add flatten_mir helperAlyssa Rosenzweig2019-09-301-0/+22
| | | | | | | We would like to flatten a linked list of midgard_instructions into an array of midgard_instruction pointers on the heap. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Squeeze indices before schedulingAlyssa Rosenzweig2019-09-301-0/+1
| | | | | | This allows node_count to be correct while scheduling. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Fix component count handling for ldstAlyssa Rosenzweig2019-09-302-37/+37
| | | | | | | It's not based on the writemask and it can't be inferred; it's just intrinsic to the op itself. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* pan/midgard: Add missing parans in SWIZZLE definitionAlyssa Rosenzweig2019-09-301-1/+1
| | | | Signed-off-by: Alyssa Rosenzweig <[email protected]>
* Revert "panfrost: Rework midgard_pair_load_store() to kill the nested ↵Boris Brezillon2019-09-191-29/+34
| | | | | | | | | | foreach loop" There's a missing prev_ldst = NULL; assignment in the new logic, but even with this fixed it seems to regress some applications, so let's revert the change until we find the real problem. This reverts commit c9bebae2877e55cdcd94f9f9f3f6805238caeb28.
* panfrost: Rework midgard_pair_load_store() to kill the nested foreach loopBoris Brezillon2019-09-131-34/+29
| | | | | | | | | | | | | | | | | | | | | | | mir_foreach_instr_in_block_safe() is based on list_for_each_entry_safe() which is designed to protect against removal of the current entry, but removing the entry placed just after the current one will lead to a use-after-free situation. Luckily, the midgard_pair_load_store() logic guarantees that the instruction being removed (if any) is never placed just after ins which in turn guarantees that the hidden __next variable always points to a valid object. Took me a bit of time to realize that this code was safe, so I'm suggesting to get rid of the inner mir_foreach_instr_in_block_from() loop and rework the code so that the removed instruction is always the current one (which is what the list_for_each_entry_safe() API was initially designed for). While at it, we also get rid of the unecessary insert(ins)/remove(ins) dance by simply moving the instruction around. Signed-off-by: Boris Brezillon <[email protected]> Reviewed-by: Alyssa Rosenzweig <[email protected]>
* panfrost: Fix a list_assert() in schedule_block()Boris Brezillon2019-09-131-4/+6
| | | | | | | | | list_for_each_entry() does not allow modifying the current item pointer. Let's rework the skip-instructions logic in schedule_block() to not break this rule. Signed-off-by: Boris Brezillon <[email protected]> Reviewed-by: Alyssa Rosenzweig <[email protected]>
* nir: allow specifying filter callback in lower_alu_to_scalarVasily Khoruzhick2019-09-062-2/+2
| | | | | | | | | | | | | Set of opcodes doesn't have enough flexibility in certain cases. E.g. Utgard PP has vector conditional select operation, but condition is always scalar. Lowering all the vector selects to scalar increases instruction number, so we need a way to filter only those ops that can't be handled in hardware. Reviewed-by: Qiang Yu <[email protected]> Reviewed-by: Eric Anholt <[email protected]> Reviewed-by: Jason Ekstrand <[email protected]> Signed-off-by: Vasily Khoruzhick <[email protected]>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-19 16:41:28 +0000