summaryrefslogtreecommitdiffstats
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* panfrost: Fixup vertex offsets to prevent shadow copyAlyssa Rosenzweig2019-04-192-43/+67
| | | | | | | | | | | Mali attribute buffers have to be 64-byte aligned. However, Gallium enforces no such requirement; for unaligned buffers, we were previously forced to create a shadow copy (slow!). To prevent this, we instead use the offseted buffer's address with the lower bits masked off, and then add those masked off bits to the src_offset. Proof of correctness included, possibly for the opportunity to say "QED" unironically. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* panfrost: Track BO lifetime with jobs and reference countsAlyssa Rosenzweig2019-04-199-37/+207
| | | | | | | | | | | | | | | | | | | | This (fairly large) patch continues work surrounding the panfrost_job abstraction to improve job lifetime management. In particular, we add infrastructure to track which BOs are used by a particular job (currently limited to the vertex buffer BOs), to reference count these BOs, and to automatically manage the BOs memory based on the reference count. This set of changes serves as a code cleanup, as a way of future proofing for allowing flushing BOs, and immediately as a bugfix to workaround the missing reference counting for vertex buffer BOs. Meanwhile, there are a few cleanups to vertex buffer handling code itself, so in the short-term, this allows us to remove the costly VBO staging workaround, since this patch addresses the underlying causes. v2: Use pipe_reference for BO reference counting, rather than managing it ourselves. Don't duplicate hash-table key removal. Fix vertex buffer counting. Signed-off-by: Alyssa Rosenzweig <[email protected]>
* anv/nir: Add a central helper for figuring out SSBO address formatsJason Ekstrand2019-04-193-57/+98
| | | | Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* nir: Add helpers for getting the type of an address formatJason Ekstrand2019-04-191-0/+33
| | | | Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Implement VK_EXT_descriptor_indexingJason Ekstrand2019-04-195-2/+93
| | | | | | | | | Now that everything is in place to do bindless for all resource types except input attachments and UBOs, VK_EXT_descriptor_indexing is "trivial". Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Put binding flags in descriptor set layoutsJason Ekstrand2019-04-192-0/+19
| | | | | Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Use bindless handles for imagesJason Ekstrand2019-04-195-4/+63
| | | | | Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* intel/fs: Add support for bindless image load/store/atomicJason Ekstrand2019-04-193-8/+72
| | | | | Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Use bindless textures and samplersJason Ekstrand2019-04-196-31/+228
| | | | | | | | | | This commit changes anv to put bindless handles and sampler pointers into the descriptor buffer and use those instead of bindful when we run out of binding table space. This "spilling" of descriptors allows to to advertise an almost unbounded number of images and samplers. Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Pass the plane into lower_tex_derefJason Ekstrand2019-04-191-7/+5
| | | | | Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Use write_image_view to initialize immutable samplersJason Ekstrand2019-04-191-5/+13
| | | | | | | | | Instead of setting it manually, call the helper. When setting descriptor sets becomes more complicated than just setting some struct values, this will keep immutable sampler handling correct. Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Count the number of planes in each descriptor bindingJason Ekstrand2019-04-192-3/+19
| | | | | Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* intel/fs: Add support for bindless texture opsJason Ekstrand2019-04-195-10/+86
| | | | | | | | | | | | | | We add two new texture sources for bindless surface and sampler handles. Bindless surface handles are expected to be pre-shifted so that the 20-bit surface state table index is in the top 20 bits of the 32-bit handle. This lets us avoid any extra shifts in the shader. Bindless sampler handles are 32-byte aligned byte offsets from general state base address. We use 32-byte aligned instead of 16-byte aligned to avoid having to use more indirect messages than needed. It means we can't tightly pack samplers but that's probably not a big deal. Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* intel,nir: Lower TXD with a bindless samplerJason Ekstrand2019-04-193-0/+9
| | | | | | | | | When we have a bindless sampler, we need an instruction header. Even in SIMD8, this pushes the instruction over the sampler message size maximum of 11 registers. Instead, we have to lower TXD to TXL. Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Implement VK_KHR_shader_atomic_int64Jason Ekstrand2019-04-198-5/+51
| | | | | Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Implement SSBOs bindings with GPU addresses in the descriptor BOJason Ekstrand2019-04-196-35/+347
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit adds a new way for ANV to do SSBO bindings by just passing a GPU address in through the descriptor buffer and using the A64 messages to access the GPU address directly. This means that our variable pointers are now "real" pointers instead of a vec2(BTI, offset) pair. This carries a few of advantages: 1. It lets us support a virtually unbounded number of SSBO bindings. 2. It lets us implement VK_KHR_shader_atomic_int64 which we couldn't implement before because those atomic messages are only available in the bindless A64 form. 3. It's way better than messing around with bindless handles for SSBOs which is the only other option for VK_EXT_descriptor_indexing. 4. It's more future looking, maybe? At the least, this is what NVIDIA does (they don't have binding based SSBOs at all). This doesn't a priori mean it's better, it just means it's probably not terrible. The big disadvantage, of course, is that we have to start doing our own bounds checking for robustBufferAccess again have to push in dynamic offsets. Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Lower some SSBO operations in apply_pipeline_layoutJason Ekstrand2019-04-191-2/+212
| | | | | | | | | | | | In order to avoid the potential overhead of A64 operations on all SSBO ops, we look for those SSBO ops where we can get to the descriptor set from the SSBO access operation and lower those to a binding-table approach. When robustBufferAccess is enabled, this lets the hardware do the bounds checking for us. It also avoids some potentially expensive 64-bit integer calculations. Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Add a has_a64_buffer_access to anv_physical_deviceJason Ekstrand2019-04-194-6/+11
| | | | | | | | This is more descriptive and a bit nicer than checking for gen >= 8 && use_softpin everywhere. Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* intel/nir: Re-run int64 lowering in postprocess_nirJason Ekstrand2019-04-191-0/+1
| | | | | | | | | | | We're about to start doing 64-bit pointer calculations in ANV. They will get applied after brw_preprocess_nir which is where we currently do 64-bit integer arithmetic lowering. Because we're adding 64-bit integer arithmetic after the initial lowering has happened, we need to lower again. Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* nir/lower_io: Expose some explicit I/O lowering helpersJason Ekstrand2019-04-192-42/+65
| | | | | Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv/pipeline: Add skeleton support for spilling to bindlessJason Ekstrand2019-04-194-27/+122
| | | | | | | | | | If the number of surfaces or samplers exceeds what we can put in a table, we will want to spill out to bindless. There is no bindless support yet but this gets us the basic framework that will be used by later commits. Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv/pipeline: Sort bindings by most used firstJason Ekstrand2019-04-191-40/+95
| | | | | | | | | This commit just sorts the bindings by how often they're used vs the array size of the binding. This will let us make more nuanced decisions about what goes in the binding table vs. what to make bindless. Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Add a #define for the max binding table sizeJason Ekstrand2019-04-193-4/+16
| | | | | | | | | This also fixes a bug where we mis-calculate maximum binding table sizes and may return true in vkGetDescriptorSetLayoutSupport even for sets too large to fit in a binding table. Fixes: ddc40691221 "anv: Implement VK_KHR_maintenance3" Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Put image params in the descriptor set buffer on gen8 and earlierJason Ekstrand2019-04-196-124/+109
| | | | | | | | | | | | This is really where they belong; not push constants. The one downside here is that we can't push them anymore for compute shaders. However, that's a general problem and we should figure out how to push descriptor sets for compute shaders. This lets us bump MAX_IMAGES to 64 on BDW and earlier platforms because we no longer have to worry about push constant overhead limits. Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* anv: Make all VkDeviceMemory BOs resident permanentlyJason Ekstrand2019-04-194-46/+48
| | | | | | | | | | | | | | | | | | | | | We spend a lot of time in the driver adding things to hash sets to track residency. The reality is that a properly built Vulkan app uses large memory objects and sub-allocates from them. In a typical frame, most of if not all of those allocations are going to be resident for the entire frame so we're really not saving ourselves much by tracking fine-grained residency. Just throwing everything in the validation list does make it a little bit more expensive inside the kernel to walk the list and ensure that all our VA is in order. However, without relocations, the overhead of that is pretty small. If we ever do run into a memory pressure situation where the fine- grained residency could even potentially help, we would likely be swapping one page out to make room for another within the draw call and performance is totally lost at that point. We're better off swapping out other apps and just letting ours run a whole frame. Reviewed-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* freedreno/ir3: fix const assertRob Clark2019-04-191-1/+0
| | | | | Fixes: fe8c57e859d freedreno/ir3: use nir_src_as_uint in a few places Signed-off-by: Rob Clark <[email protected]>
* gallium/auxiliary/vl: Fix a couple of warningsKristian H. Kristensen2019-04-192-8/+2
| | | | | | | | Remove unused functions and mark unhandled default case with unreachable. Signed-off-by: Kristian H. Kristensen <[email protected]> Reviewed-by: Emil Velikov <[email protected]>
* egl/dri2: Mark potentially unused 'display' variable with MAYBE_UNUSEDKristian H. Kristensen2019-04-191-1/+1
| | | | | | | | Sometimes there is no X11 platform. Signed-off-by: Kristian H. Kristensen <[email protected]> Reviewed-by: Ian Romanick <[email protected]> Reviewed-by: Emil Velikov <[email protected]>
* ralloc: Fully qualify non-virtual destructor callKristian H. Kristensen2019-04-191-1/+1
| | | | | | | | | | | | This suppresses warning about calling a non-virtual destructor in a non-final class with virtual functions: src/compiler/glsl/ast.h:53:4: warning: destructor called on non-final 'ast_node' that has virtual functions but non-virtual destructor [-Wdelete-non-virtual-dtor] DECLARE_LINEAR_ZALLOC_CXX_OPERATORS(ast_node); Signed-off-by: Kristian H. Kristensen <[email protected]> Reviewed-by: Ian Romanick <[email protected]> Reviewed-by: Emil Velikov <[email protected]>
* nir_opcodes.py: Saturate to expression that doesn't overflowKristian H. Kristensen2019-04-192-1/+3
| | | | | | | | | | | | | | | Compiler warns about overflow when assigning UINT64_MAX to something smaller than a uin64_t: src/compiler/nir/nir_constant_expressions.c:16909:50: warning: implicit conversion from 'unsigned long long' to 'uint1_t' (aka 'unsigned char') changes value from 18446744073709551615 to 255 [-Wconstant-conversion] uint1_t dst = (src0 + src1) < src0 ? UINT64_MAX : (src0 + src1); ~~~ ^~~~~~~~~~ Shift UINT64_MAX down to the appropriate maximum value for the type being assigned to. Signed-off-by: Kristian H. Kristensen <[email protected]> Reviewed-by: Jason Ekstrand <[email protected]>
* glsl_to_nir: Initialize debug variableKristian H. Kristensen2019-04-191-1/+1
| | | | | | | | If we want to assert on found == true when the loop exits early, we need to initialize it to false. Signed-off-by: Kristian H. Kristensen <[email protected]> Reviewed-by: Ian Romanick <[email protected]>
* tgsi: Mark tgsi_strings_check() unusedKristian H. Kristensen2019-04-191-1/+1
| | | | | | | It's there to hold the static asserts, don't warning about it being unused. Signed-off-by: Kristian H. Kristensen <[email protected]>
* anv: limit URB reconfigurations when using blorpLionel Landwerlin2019-04-193-3/+11
| | | | | | | | | | | If the last graphics pipeline bound to the command buffer has enough space in its VS URB entries for Blorp then avoid reconfiguring the URB partitions. v2: s/0/MESA_SHADER_VERTEX/ (Caio) Signed-off-by: Lionel Landwerlin <[email protected]> Reviewed-by: Caio Marcelo de Oliveira Filho <[email protected]>
* intel/devinfo: add basic sanity tests on device databaseLionel Landwerlin2019-04-192-0/+45
| | | | | | | | | v2: #undef NDEBUG (Eric) Use inc_include & inc_src (Eric) Signed-off-by: Lionel Landwerlin <[email protected]> Reviewed-by: Eric Engestrom <[email protected]> Reviewed-by: Anuj Phogat [email protected]
* intel/devinfo: fix missing num_thread_per_eu on ICLLionel Landwerlin2019-04-191-6/+2
| | | | | | | | | | | There was an assumption that num_thread_per_eu would be set in the Gen8 features. Since this is mostly the same of all gen8->11 (except GEN9_LP that overwrites it) let's just factor it out. Signed-off-by: Lionel Landwerlin <[email protected]> Cc: [email protected] Acked-by: Eric Engestrom <[email protected]> Reviewed-by: Anuj Phogat [email protected]
* nir: Use the nir_builder _imm helpers in setting up deref offsets.Eric Anholt2019-04-191-4/+3
| | | | | | | | | | When looking at the dEQP nested_struct_array_dynamic_index_fragment code after lowering, I was horrified at the amount of adding and multiplying by 0 we were doing. The builder _imm helpers handle that for you so that the following optimization passes have less work to do. Plus, it's easier to read. Reviewed-by: Jason Ekstrand <[email protected]>
* nir: Fix deref offset calculation for structs.Eric Anholt2019-04-191-1/+1
| | | | | | | | | | We were calcuating the offset for the field within the struct, and just dropping it on the floor. Fixes a regression in KHR-GLES3.shaders.struct.local.nested_struct_array_dynamic_index_fragment and a few of its friends since the scratch lowering commit. Fixes: e8e159e9df40 ("nir/deref: Add helpers for getting offsets") Reviewed-by: Jason Ekstrand <[email protected]>
* lima: enable nir fsign lowering in ppirErico Nunes2019-04-191-0/+1
| | | | | | | | The mali utgard pp doesn't support a sign instruction. Use the nir lowering function for fsign to implement fsign in ppir. Signed-off-by: Erico Nunes <[email protected]> Reviewed-by: Eric Anholt <[email protected]>
* nir/algebraic: add lowering for fsignErico Nunes2019-04-192-0/+4
| | | | | | | | | | | The mali utgard pp doesn't support a sign instruction. In the ARM offline shader compiler, the sign function is implemented using sub(gt(0.0, a), lt(0.0, a)). This is a generic optimization, so implement it in the nir level when lower_fsign is set, alongside the lowering for isign. Signed-off-by: Erico Nunes <[email protected]> Reviewed-by: Eric Anholt <[email protected]>
* lima/ppir: support ppir_op_ceilErico Nunes2019-04-193-0/+14
| | | | | | | | | Add a few missing ppir_op_ceil enum handling entries to implement nir_op_fceil in lima ppir. Signed-off-by: Erico Nunes <[email protected]> Reviewed-by: Vasily Khoruzhick <[email protected]> Reviewed-by: Qiang Yu <[email protected]>
* radv: Support VK_EXT_inline_uniform_block.Bas Nieuwenhuizen2019-04-195-15/+124
| | | | | | | | | | | | | | | | | | | Basically just reserve the memory in the descriptor sets. On the shader side we construct a buffer descriptor, since AFAIU VGPR indexing on 32-bit pointers in LLVM is still broken. This fully supports update after bind and variable descriptor set sizes. However, the limits are somewhat arbitrary and are mostly about finding a reasonable division of a 2 GiB max memory size over the set. v2: - rebased on top of master (Samuel) - remove the loading resources rework (Samuel) - only load UBO descriptors if it's a pointer (Samuel) - use LLVMBuildPtrToInt to avoid IR failures (Samuel) Reviewed-by: Bas Nieuwenhuizen <[email protected]> (v2)
* ac/nir: use the new raw/struct SSBO atomic intrisics for comp_swapSamuel Pitoiset2019-04-191-2/+1
| | | | | | | | | | | | This is actually fixed now. This change requires LLVM r358579. Make sure to have it in your tree, otherwise the following piglit will hang: tests/spec/arb_shader_storage_buffer_object/execution/ssbo-atomicCompSwap-int.shader_test Signed-off-by: Samuel Pitoiset <[email protected]> Reviewed-by: Timothy Arceri <[email protected]>
* ac/nir: only use the new raw/struct SSBO atomic intrinsics with LLVM 9+Samuel Pitoiset2019-04-191-1/+4
| | | | | | | | They are buggy with older LLVM version, see r358579. Fixes: 78c551aca1c ("ac/nir: use new LLVM 8 intrinsics for SSBO atomics except cmpswap") Signed-off-by: Samuel Pitoiset <[email protected]> Reviewed-by: Timothy Arceri <[email protected]>
* ac/nir: only use the new raw/struct image atomic intrinsics with LLVM 9+Samuel Pitoiset2019-04-191-1/+4
| | | | | | | | | They are buggy with LLVM 8 because they weren't marked as source of divergence, see r358579. Fixes: dd0172e865f ("radv: Use structured intrinsics instead of indexing workaround for GFX9.")" Signed-off-by: Samuel Pitoiset <[email protected]> Reviewed-by: Timothy Arceri <[email protected]>
* iris: Be less aggressive at postdraw work skippingKenneth Graunke2019-04-181-28/+36
| | | | | | | | | | | | | | | | We empty the cache sets when flushing the batch, at which point we need to add any framebuffer related BOs even though the bindings haven't changed. So, we now do the cache set tracking unconditionally. For now, we continue skipping resolve work based on the same conditions in the predraw functions - the thinking is if we didn't trigger resolves, there's nothing to update here. Time will tell if this works. Partly reverts commit 365886ebe1a54f893b688b457553eead6aa572ea, and fixes Unigine Valley rendering on Gen9+. Drops drawoverhead scores by about 10-12%. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=110353
* intel/fs: Account for live range lengths in spill costsJason Ekstrand2019-04-181-1/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current register allocator has a concept of "spill benefit" which is based on the number of nodes with which a given node interferes. The idea is that you want to spill stuff with high interference because those are the most likely registers to help when spilling. However, this fails to take into account the length of the live range so the allocator frequently picks "cheap" (not many uses) registers which are actually very short lived and so spilling them doesn't help with the pressure situation. This commit takes into account the length of the live range to make long-lived registers more likely to get spilled than short-lived ones. This encourages the spill chooser to choose slightly larger registers which will affect a larger area of the program and hopefully we have to spill fewer of them to get the same reduction in over-all register pressure. Shader-db results on Kaby Lake: total spills in shared programs: 23664 -> 12050 (-49.08%) spills in affected programs: 19243 -> 7629 (-60.35%) helped: 296 HURT: 8 total fills in shared programs: 32028 -> 25139 (-21.51%) fills in affected programs: 20378 -> 13489 (-33.81%) helped: 295 HURT: 16 Of course, most of that is in Deus Ex... Shader-db results on Kaby Lake (without Deus Ex): total spills in shared programs: 6479 -> 5834 (-9.96%) spills in affected programs: 3231 -> 2586 (-19.96%) helped: 40 HURT: 4 total fills in shared programs: 17165 -> 17099 (-0.38%) fills in affected programs: 6951 -> 6885 (-0.95%) helped: 40 HURT: 7 Even without Deus Ex, the spill help is pretty respectable. The worst hurt shaders were one compute shader in Aztec Ruins and one fragment shader in KSP that were each hurt by around 13% fill 9% spill. VkPipeline-db results on Kaby Lake: total spills in shared programs: 9149 -> 8069 (-11.80%) spills in affected programs: 5197 -> 4117 (-20.78%) helped: 27 HURT: 16 total fills in shared programs: 26390 -> 25477 (-3.46%) fills in affected programs: 12662 -> 11749 (-7.21%) helped: 24 HURT: 22 The Vulkan results were decidedly more mixed but we don't have nearly as many apps in that database yet. Reviewed-by: Kenneth Graunke <[email protected]> Reviewed-by: Matt Turner <[email protected]>
* virgl/vtest: bump up protocol version + support encoded transfersGurchetan Singh2019-04-183-3/+12
| | | | | | | | This more accurately reflects what the drm winsys does. Signed-off-by: Gurchetan Singh <[email protected]> Reviewed-By: Gert Wollny <[email protected]> Reviewed-By: Piotr Rak <[email protected]>
* virgl/vtest: wait after issuing a transfer getGurchetan Singh2019-04-181-2/+3
| | | | | | | | | | | | Otherwise, there's artifacts when running Unigine Valley with protocol version 2. We can get away with not waiting for most buffers, but let's be conservative. Signed-off-by: Gurchetan Singh <[email protected]> Reviewed-By: Gert Wollny <[email protected]> Reviewed-By: Piotr Rak <[email protected]>
* virgl/vtest: modify sending and receiving data for shared memoryGurchetan Singh2019-04-181-4/+35
| | | | | | | | We need to copy the shared memory region to the display target. Signed-off-by: Gurchetan Singh <[email protected]> Reviewed-By: Gert Wollny <[email protected]> Reviewed-By: Piotr Rak <[email protected]>
* virgl/vtest: receive and handle shared memory fdGurchetan Singh2019-04-182-7/+55
| | | | | | | | | | | | | | | | The only tricky part is with protocol 0 we can either have a display target or resource backing store. With protocol 2 we can have both. Make the map/unmap functions only deal with the resource backing store. v2: Handle MSAA texture case. v3: spelling v4: Fix dangling else (@prak) v5: mmap --> os_mmap (@prak) + added comments (@gerddie) Signed-off-by: Gurchetan Singh <[email protected]> Reviewed-By: Gert Wollny <[email protected]> Reviewed-By: Piotr Rak <[email protected]>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-28 12:24:55 +0000