aboutsummaryrefslogtreecommitdiffstats
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* vl/compositor: implement error handlingNayan Deshmukh2017-01-032-3/+12
| | | | | | | pipe_buffer_map and pipe_buffer_create may return NULL Signed-off-by: Nayan Deshmukh <[email protected]> Reviewed-by: Christian König <[email protected]>
* i965/vec4: enable ARB_gpu_shader_fp64 for HaswellIago Toral Quiroga2017-01-031-0/+3
| | | | Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: adjust spilling costs for 64-bit registers.Iago Toral Quiroga2017-01-031-2/+13
| | | | Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: prevent spilling of DOUBLE_TO_SINGLE destinationIago Toral Quiroga2017-01-031-0/+12
| | | | | | | | | | | | | | | | | | | | | FROM_DOUBLE opcodes are setup so that they use a dst register with a size of 2 even if they only produce a single-precison result (this is so that the opcode can use the larger register to produce a 64-bit aligned intermediary result as required by the hardware during the conversion process). This creates a problem for spilling though, because when we attempt to emit a spill for the dst we see a 32-bit destination and emit a scratch write that allocates a single spill register, making the intermediary writes go beyond the size of the allocation. Prevent this by avoiding to spill the destination register of these opcodes. Alternatively, we can avoid this by splitting the opcode in two: one that produces a 64-bit aligned result and one that takes the 64-bit aligned result as input and produces a 32-bit result from it. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: avoid spilling of registers that mix 32-bit and 64-bit accessIago Toral Quiroga2017-01-031-0/+24
| | | | | | | | | | | | | | | | | | | | When 64-bit registers are (un)spilled, we need to execute data shuffling code before writing to or after reading from memory. If we have instructions that operate on 64-bit data via 32-bit instructions, (un)spills for the register produced by 32-bit instructions will not do data shuffling at all (because we only see a normal 32-bit istruction seemingly operating on 32-bit data). This means that subsequent reads with that register using DF access will unshuffle data read from memory that was never adequately shuffled when it was written. Fixing this would require to identify which 32-bit instructions write 64-bit data and emit spill instructions only when the full 64-bit data has been written (by multiple 32-bit instructions writing to different offsets of the same register) and always emit 64-bit unspills whenever 64-bit data is read, even when the instruction uses a 32-bit type to read from them. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: support basic spilling of 64-bit registersIago Toral Quiroga2017-01-031-6/+28
| | | | | | | | | | | | | | | The current spilling code can't spill vgrf allocations larger than 1 but SIMD4x2 doubles require 2 vgrfs, so we need to permit this case (which is handled properly for DF data types by emitting 2 scratch messages and doing data shuffling). We accomplish this by not auto-disabling spilling for vgrf allocations with a size of 2, and then disable spilling on any register with an offset != 0B (which indicates array access). Disable spilling of partial DF reads/writes because these don't read/write data for both logical threads and our scratch messages for 64-bit data need data for both threads to be present. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: run scalarize_df() after spillingIago Toral Quiroga2017-01-031-0/+18
| | | | | | | | Spilling of 64-bit data requires data shuffling for the corresponding scratch read/write messages. This produces unsupported swizzle regions and writemasks that we need to scalarize. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: prevent src/dst hazards during 64-bit register allocationIago Toral Quiroga2017-01-031-1/+7
| | | | | | | | | | | | | | 8-wide compressed DF operations are executed as two separate 4-wide DF operations. In that scenario, we have to be careful when we allocate register space for their operands to prevent the case where the first half of the instruction overwrites the source of the second half. To do this we mark compressed instructions as having hazards to make sure that ther register allocators assigns a register regions for the destination that does not overlap with the region assigned for any of its source operands. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4/scalarize_df: support more swizzles via vstride=0Iago Toral Quiroga2017-01-033-21/+51
| | | | | | | | | | | | | | | | By exploiting gen7's hardware decompression bug with vstride=0 we gain the capacity to support additional swizzle combinations. This also fixes ZW writes from X/Y channels like in: mov r2.z:df r0.xxxx:df Because DF regions use 2-wide rows with a vstride of 2, the region generated for the source would be r0<2,2,1>.xyxy:DF, which is equivalent to r0.xxzz, so we end up writing r0.z in r2.z instead of r0.x. Using a vertical stride of 0 in these cases we get to replicate the XX swizzle and write what we want. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4/scalarize_df: do not scalarize swizzles that we can support nativelyIago Toral Quiroga2017-01-033-25/+112
| | | | | | | | | | | | | | Certain swizzles like XYZW can be supported by translating only the first two 64-bit swizzle channels to 32-bit channels. This happens with swizzles such that the first two logical components, when translated to 32-bit channels and replicated across the second dvec2 row, select the same channels specified by the 3rd and 4th logical swizzle components. Notice that this opens up the possibility that some instructions are not scalarized and can end up with XY or ZW 32-bit writemasks. Make sure we always scalarize in such cases. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: split instructions that read 64-bit interleaved attributesIago Toral Quiroga2017-01-031-2/+26
| | | | | | | | | | Stages that use interleaved attributes generate regions with a vstride=0 that can hit the gen7 hardware decompression bug. v2: - Make static the function and fix indent (Matt) Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: dump subnr for FIXED_GRFIago Toral Quiroga2017-01-031-1/+1
| | | | | | | | This came in handy when debugging the payload setup for Tess Eval, since it prints correct subnr for attributes that can be loaded in the second half of a register. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4/tes: consider register offsets during attribute setupIago Toral Quiroga2017-01-031-2/+2
| | | | Reviewed-by: Matt Turner <[email protected]>
* i965/vec4/tes: fix setup_payload() for 64bit data typesIago Toral Quiroga2017-01-031-1/+20
| | | | | | | | | | | | | | | | | | Use a width of 2 with 64-bit attributes. Also, if we have a dvec3/4 attribute that gets split across two registers such that components XY are stored in the second half of a register and components ZW are stored in the first half of the next, we need to fix regioning for any instruction that reads components Z/W of the attribute. Notice this also means that we can't support sources that read cross-dvec2 swizzles (like XZ for example). v2: don't assert that we have a single channel swizzle in the case that we have to fix up Z/W access on the first half of the next register. We can handle any swizzle that does not cross dvec2 boundaries, which the double scalarization pass should have prevented anyway. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4/tes: fix input loading for 64bit data typesIago Toral Quiroga2017-01-031-17/+55
| | | | | | v2: use byte_offset() instead of offset() Reviewed-by: Matt Turner <[email protected]>
* i965/vec4/tcs: fix outputs for 64-bit dataIago Toral Quiroga2017-01-031-2/+29
| | | | | | v2: use byte_offset() instead of offset() Reviewed-by: Matt Turner <[email protected]>
* i965/vec4/tcs: fix input loading for 64-bit dataIago Toral Quiroga2017-01-031-4/+30
| | | | | | v2: use byte_offset() instead of offset() Reviewed-by: Matt Turner <[email protected]>
* i965/vec4/gs: fix input loading for 64bit dataSamuel Iglesias Gonsálvez2017-01-031-17/+34
| | | | | | | | | v2 (Iago): - Adapt 64-bit path to component packing changes. Signed-off-by: Samuel Iglesias Gonsálvez <[email protected]> Signed-off-by: Iago Toral Quiroga <[email protected]> Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: fix store output for 64-bit typesIago Toral Quiroga2017-01-031-2/+25
| | | | | | | | | We need to shuffle the data before it is written to the URB. Also, dvec3/4 need two vec4 slots. v2: use byte_offset() instead of offset(). Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: fix attribute setup for doublesIago Toral Quiroga2017-01-031-7/+14
| | | | Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: fix indentation in lower_attributes_to_hw_regs()Iago Toral Quiroga2017-01-031-8/+8
| | | | Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: make emit_pull_constant_load support 64-bit loadsIago Toral Quiroga2017-01-032-55/+50
| | | | | | | | | | | This way callers don't need to know about 64-bit particularities and we reuse some code. v2: - use byte_offset() instead of offset() - only mark the surface as used once Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: fix move_push_constants_to_pull_constants() for 64-bit dataIago Toral Quiroga2017-01-031-4/+19
| | | | | | v2: adapt to changes in offset() Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: fix indentation in move_push_constants_to_pull_constants()Iago Toral Quiroga2017-01-031-30/+30
| | | | Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: fix move_uniform_array_access_to_pull_constant() for 64-bit dataIago Toral Quiroga2017-01-031-2/+18
| | | | | | v2: adapt to changes in offset() Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: fix scratch writes for 64bit dataIago Toral Quiroga2017-01-031-9/+55
| | | | | | | | | | Mostly the same stuff as usual: we ned to shuffle the data before we write and we need to emit two 32-bit write messages (with appropriate 32-bit writemask channels set) for a full dvec4 scratch write. v2: use byte_offset() instead of offset(). Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: fix scratch reads for 64bit dataIago Toral Quiroga2017-01-031-2/+14
| | | | | | | | | v2: Setup for a 64-bit scratch read by checking the type size of the correct register v3: Use byte_offset() instead of offset() Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: fix scratch offset for 64bit dataIago Toral Quiroga2017-01-031-6/+16
| | | | | | | | | A vec4 is 16 bytes and a dvec4 is 32 bytes so for doubles we have to multiply the reladdr by 2. The reg_offset part is in units of 16 bytes and is used to select the low/high 16-byte chunk of a full dvec4, so we don't want to multiply that part of the address. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: do not split scratch read/write opcodesIago Toral Quiroga2017-01-031-0/+9
| | | | | | | | 64-bit scratch read/writes require to shuffle data around so we need to have access to the full 64-bit data. We will do the right thing for these when we emit the messages. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: Do not use DepCtrl with 64-bit instructionsIago Toral Quiroga2017-01-031-1/+13
| | | | | | | | | | The BDW PRM says that it is not supported, but it seems that gen7 is also affected, since doing DepCtrl on double-float instructions leads to GPU hangs in some cases, which is probably not surprising knowing that this is not supported in new hardware iterations. The SKL PRMs do not mention this restriction, so it is probably fine. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: extend the DWORD multiply DepCtrl restriction to all gen8 platformsIago Toral Quiroga2017-01-031-3/+6
| | | | | | | v2: - Add Broxton as Intel's internal PRMs says that it is needed (Matt). Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: don't copy propagate misaligned registersSamuel Iglesias Gonsálvez2017-01-031-0/+3
| | | | | | | | This means we would copy propagate partial reads or writes and that can affect the result. Signed-off-by: Samuel Iglesias Gonsálvez <[email protected]> Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: don't propagate single-precision uniforms into 4-wide instructionsIago Toral Quiroga2017-01-031-0/+11
| | | | | | | | Otherwise we end up producing code that violates the register region restriction that says that when execsize == width and hstride != 0 the vstride can't be 0. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: Prevent copy propagation from violating pre-gen8 restrictionsIago Toral Quiroga2017-01-031-0/+7
| | | | | | | | In gen < 8 instructions that write more than one register need to read more than one register too. Make sure we don't break that restriction by copy propagating from a uniform. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: prevent copy-propagation from values with a different type sizeIago Toral Quiroga2017-01-031-0/+7
| | | | | | | Because the meaning of the swizzles and writemasks involved is different, so replacing the source would lead to different semantics. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: don't constant propagate 64-bit immediatesConnor Abbott2017-01-031-0/+7
| | | | | | | v2: Also check if the instruction source target is 64-bit. (Samuel) Signed-off-by: Samuel Iglesias Gonsálvez <[email protected]> Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: Fix SSBO stores for 64-bit dataIago Toral Quiroga2017-01-031-8/+32
| | | | | | | | | | In this case we need to shuffle the 64-bit data before we write it to memory, source from reg_offset + 1 to write components Z and W and consider that each DF channel is twice as big. v2: use byte_offset() instead of offset(). Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: Fix SSBO loads for 64-bit dataIago Toral Quiroga2017-01-031-4/+29
| | | | | | | | | | Same requirements as for UBO loads. v2: - use byte_offset() instead of offset() (Iago) - keep the const. offset as an immediate like the original code did (Juan) Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: Fix UBO loads for 64-bit dataIago Toral Quiroga2017-01-031-15/+34
| | | | | | | | | | | | | | | We need to emit 2 32-bit load messages to load a full dvec4. If only 1 or 2 double components are needed dead-code-elimination will remove the second one. We also need to shuffle the result of the 32-bit messages to form valid 64-bit SIMD4x2 data. v2: - use byte_offset() instead of offset() (Iago) - keep the const. offset as an immediate like the original code did (Juan) Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: Add a shuffle_64bit_data helperIago Toral Quiroga2017-01-032-0/+76
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | SIMD4x2 64bit data is stored in register space like this: r0.0:DF x0 y0 z0 w0 r1.0:DF x1 y1 z1 w1 When we need to write data such as this to memory using 32-bit write messages we need to shuffle it in this fashion: r0.0:DF x0 y0 x1 y1 r0.1:DF z0 w0 z1 w1 and emit two 32-bit write messages, one for r0.0 at base_offset and another one for r0.1 at base_offset+16. We also need to do the inverse operation when we read using 32-bit messages to produce valid SIMD4x2 64bit data from the data read. We can achieve this by aplying the exact same shuffling to the data read, although we need to apply different channel enables since the layout of the data is reversed. This helper implements the data shuffling logic and we will use it in various places where we read and write 64bit data from/to memory. v2 (Curro): - Use the writemask helper and don't assert on the original writemask being XYZW. - Use the Vec4 IR builder to simplify the implementation. v3 (Iago): - Use byte_offset() instead of offset(). v3: - Fix typo (Matt) - Clarify the example and fix indention (Matt). Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: support multiple dispatch widths and groups in the IR builder.Iago Toral Quiroga2017-01-031-2/+37
| | | | Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: Lower 64-bit MADIago Toral Quiroga2017-01-032-0/+45
| | | | | | | | | | | | | | | | | | | The previous patch made sure that we do not generate MAD instructions for any NIR's 64-bit ffma, but there is nothing preventing i965 from producing MAD instructions as a result of lowerings or optimization passes. This patch makes sure that any 64-bit MAD produced inside the driver after translating from NIR is also converted to MUL+ADD before we generate code. v2: - Use a copy constructor to copy all relevant instruction fields from the original mad into the add and mul instructions v3: - Rename the lowering and fix commit log (Matt) Signed-off-by: Samuel Iglesias Gonsálvez <[email protected]> Reviewed-by: Matt Turner <[email protected]>
* i965/vec4/nir: do not emit 64-bit MADIago Toral Quiroga2017-01-031-5/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | RepCtrl=1 does not work with 64-bit operands so we need to use RepCtrl=0. In that situation, the regioning generated for the sources seems to be equivalent to <4,4,1>:DF, so it will only work for components XY, which means that we have to move any other swizzle to a temporary so that we can source from channel X (or Y) in MAD and we also need to split the instruction (we are already scalarizing DF instructions but there is room for improvement and with MAD would be more restricted in that area) Also, it seems that MAD operations like this only write proper output for channels X and Y, so writes to Z and W also need to be done to a temporary using channels X/Y and then move that to channels Z or W of the actual dst. As a result the code we produce for native 64-bit MAD instructions is rather bad, and much worse than just emitting MUL+ADD. For reference, a simple case of a fully scalarized dvec4 MAD operation requires 15 instructions if we use native MAD and 8 instructions if we emit ADD+MUL instead. There are some improvements that we can do to the emission of MAD that might bring the instruction count down in some cases, but it comes at the expense of a more complex implementation so it does not seem worth it, at least initially. This patch makes translation of NIR's 64-bit FMMA instructions produce MUL+ADD instead of MAD. Currently, there is nothing else in the vec4 backend that emits MAD instructions, so this is sufficient and it helps optimization passes see MUL+ADD from the get go. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: Skip swizzle to subnr in 3src instructions with DF operandsIago Toral Quiroga2017-01-031-1/+4
| | | | | | | | | | | | | | We make scalar sources in 3src instructions use subnr instead of swizzles because they don't really use swizzles. With doubles it is more complicated because we use vstride=0 in more scenarios in which they don't produce scalar regions. Also RepCtrl=1 is not allowed with 64-bit operands, so we should avoid this. v2: Fix typo (Matt) Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: fix indentation in pack_uniform_registersIago Toral Quiroga2017-01-031-15/+15
| | | | Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: fix pack_uniform_registers for doublesIago Toral Quiroga2017-01-031-2/+9
| | | | | | We need to consider the fact that dvec3/4 require two vec4 slots. Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: teach register coalescing about 64-bitIago Toral Quiroga2017-01-031-3/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | Specifically, at least for now, we don't want to deal with the fact that channel sizes for fp64 instructions are twice the size, so prevent coalescing from instructions with a different type size. Also, we should check that if we are coalescing a register from another MOV we should be writing the same amount of data in both operations, otherwise we end up wiring more or less than the original instruction. This can happen, for example, when we have split fp64 MOVs with an exec size of 4 that only write one register each and then a MOV with exec size of 8 that reads both. We want to avoid the pass to think that it can coalesce from the first split MOV alone. Ideally we would like the pass to see that it can coalesce from both split MOVs instead, but for now we keep it simple. Finally, the pass doesn't support coalescing of multiple registers but in the case of normal SIMD4x2 double-precision instructions they naturally write two registers (one per vertex) and there is no reason why we should not allow coalescing in this case. Change the restriction to bail if we see instructions that write more than 8 channels, where the channels can be 32-bit or 64-bit. v2: - Make sure that scan_inst and inst write the same amount of data. Reviewed-by: Matt Turner <[email protected]>
* i965/disasm: fix subreg for dst in Align16 modeIago Toral Quiroga2017-01-031-1/+1
| | | | | | | | | | | | There is a single bit for this, so it is a binary 0 or 1 meaning offset 0B or 16B respectively. v2: - Since brw_inst_dst_da16_subreg_nr() is known to be 1, remove it from the expression (Curro) Reviewed-by: Francisco Jerez <[email protected]> Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: implement access to DF source components Z/WIago Toral Quiroga2017-01-031-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The general idea is that with 32-bit swizzles we cannot address DF components Z/W directly, so instead we select the region that starts at the the 16B offset into the register and use X/Y swizzles. The above, however, has the caveat that we can't do that without violating register region restrictions unless we probably do some sort of SIMD splitting. Alternatively, we can accomplish what we need without SIMD splitting by exploiting the gen7 hardware decompression bug for instructions with a vstride=0. For example, an instruction like this: mov(8) r2.x:DF r0.2<0>xyzw:DF Activates the hardware bug and produces this region: Component: x0 y0 z0 w0 x1 y1 z1 w1 Register: r0.2 r0.3 r0.2 r0.3 r1.2 r1.3 r1.2 r1.3 Where r0.2 and r0.3 are r0.z:DF for the first vertex of the SIMD4x2 execution and r1.2 and r1.3 are the same for the second vertex. Using this to our advantage we can select r0.z:DF by doing r0.2<0,2,1>.xyxy and r0.w by doing r0.2<0,2,1>.zwzw without needing to split the instruction. Of course, this only works for gen7, but that is the only hardware platform were we implement align16/fp64 at the moment. v2: Adapted to the fact that we now do this after converting to hardware registers (Iago) Reviewed-by: Matt Turner <[email protected]>
* i965/vec4: translate 64-bit swizzles to 32-bitIago Toral Quiroga2017-01-032-3/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The hardware can only operate with 32-bit swizzles, which is a rather limiting restriction. However, the idea is not to expose this to the optimization passes, which would be a mess to deal with. Instead, we let the bulk of the vec4 backend ignore this fact and we fix the swizzles right at codegen time. At the moment the pass only needs to handle single value swizzles thanks to the scalarization pass that runs before it. Notice that this only works for X/Y swizzles. We will add support for Z/W swizzles in the next patch, since they need a bit more work. v2 (Sam): - Do not expand swizzle of 64-bit immediate values. v3: - Do this after translation to hardware registers instead of doing it right before so we don't need the force_vstride0 flag (Curro). - Squashed patch that included FIXED_GRF in the list of register files that need this translation (Iago). - Remove swizzle assignments for VGRF and UNIFORM files in convert_to_hw_regs(), they will be set by apply_logical_swizzle() (Iago). Signed-off-by: Samuel Iglesias Gonsálvez <[email protected]> Reviewed-by: Matt Turner <[email protected]>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-03 15:20:53 +0000