Here are a few environment variable debug environment variables specific to ANV:
If defined to
false, this will disable the generated indirect draw optimization in ANV. This will only affect Gfx11+.
If defined to
false, this will disable pipeline caching, forcing ANV to reparse and recompile any VkShaderModule (SPIRV) it is given.
If defined to
true, this will prevent usage of self modifying command buffers to implement
vkCmdExecuteCommands. As a result of this, it will also disable VK_KHR_performance_query.
If defined to
true, this forces all descriptor sets to use the internal Bindless model.
If defined to
true, this disables support for timeline semaphores.
If defined to
true, this forces ANV to always do kernel relocations in command buffers. This should only have an effect on hardware that doesn’t support soft-pinning (Ivybridge, Haswell, Cherryview).
Specifies up to how many view shaders can be lowered to handle VK_KHR_multiview. Beyond this number, multiview is implemented using instanced rendering. If unspecified, the value default to
Here is the ANV bindless binding model that was implemented for the descriptor indexing feature of Vulkan 1.2 :
The HW binding table is generated when the draw or dispatch commands are emitted. Here are the types of entries one can find in the binding table :
The currently bound descriptor sets, one entry per descriptor set (our limit is 8).
For dynamic buffers, one entry per dynamic buffer.
For draw commands, render target entries if needed.
The entries of the HW binding table for descriptor sets are RENDER_SURFACE_STATE similar to what you would have for a normal uniform buffer. The shader will emit reads this buffer first to get the information it needs to access a surface/sampler/etc… and then emits the appropriate message using the information gathered from the descriptor set buffer.
Each binding type entry gets an associated structure in memory
This is the information read by the shader.
Binding tables are arrays of 32bit offset entries referencing surface states. This is how shaders can refer to binding table entry to read or write a surface. For example fragment shaders will often refer to entry 0 as the first render target.
The way binding tables are managed is fairly awkward.
Each shader stage must have its binding table programmed through
a corresponding instruction
3DSTATE_BINDING_TABLE_POINTERS_* (each stage has its own).
The value programmed in the
instructions is not a 64bit pointer but an offset from the address
STATE_BASE_ADDRESS::Surface State Base Address or
3DSTATE_BINDING_TABLE_POOL_ALLOC::Binding Table Pool Base Address
(available on Gfx11+). The offset value in
3DSTATE_BINDING_TABLE_POINTERS_* is also limited to a few bits
(not a full 32bit value), meaning that as we use more and more binding
tables we need to reposition
STATE_BASE_ADDRESS::Surface State Base
Address to make space for new binding table arrays.
To make things even more awkward, the binding table entries are also
STATE_BASE_ADDRESS::Surface State Base Address so as
STATE_BASE_ADDRESS::Surface State Base Address we need
add that offsets to the binding table entries.
The way with deal with this is that we allocate 4Gb of address space (since the binding table entries can address 4Gb of surface state elements). We reserve the first gigabyte exclusively to binding tables, so that anywhere we position our binding table in that first gigabyte, it can always refer to the surface states in the next 3Gb.
Descriptor Set Memory Layout¶
Here is a representation of how the descriptor set bindings, with each elements in each binding is mapped to a the descriptor set memory :
Each Binding in the descriptor set is allocated an array of
anv_*_descriptor data structure. The type of
used for a binding is selected based on the
The value of
is a byte offset from the descriptor set memory to the associated
anv_descriptor_set_binding_layout::array_size is the
anv_*_descriptor elements in the descriptor set memory
from that offset for the binding.
Pipeline state emission¶
Vulkan initially started by baking as much state as possible in pipelines. But extension after extension, more and more state has become potentially dynamic.
ANV tries to limit the amount of time an instruction has to be packed to reprogram part of the 3D pipeline state. The packing is happening in 2 places :
genX_pipeline.cwhere the non dynamic state is emitted in the pipeline batch. Chunks of the batches are copied into the command buffer as a result of calling
vkCmdBindPipeline(), depending on what changes from the previously bound graphics pipeline
genX_gfx_state.cwhere the dynamic state is added to already packed instructions from
The rule to know where to emit an instruction programming the 3D pipeline is as follow :
If any field of the instruction can be made dynamic, it should be emitted in
Otherwise, the instruction can be emitted in
When a piece of state programming is dynamic, it should have a
corresponding field in
anv_gfx_dynamic_state and the
genX(cmd_buffer_flush_gfx_runtime_state) function should be
updated to ensure we minimize the amount of time an instruction should
be emitted. Each instruction should have a associated
ANV_GFX_STATE_* mask so that the dynamic emission code can tell
when to re-emit an instruction.
Generated indirect draws optimization¶
Indirect draws have traditionally been implemented on Intel HW by
loading the indirect parameters from memory into HW registers using
the command streamer’s
MI_LOAD_REGISTER_MEM instruction before
dispatching a draw call to the 3D pipeline.
On recent products, it was found that the command streamer is showing as performance bottleneck, because it cannot dispatch draw calls fast enough to keep the 3D pipeline busy.
The solution to this problem is to change the way we deal with
indirect draws. Instead of loading HW registers with values using the
command streamer, we generate entire set of
instructions using a shader. The generated instructions contain the
entire draw call parameters. This way the command streamer executes
3DPRIMITIVE instructions and doesn’t do any data loading from
memory or touch HW registers, feeding the 3D pipeline as fast as it
In ANV this implemented by using a side batch buffer. When ANV
encounters the first indirect draws, it generates a jump into the side
batch, the side batch contains a draw call using a generation shader
for each indirect draw. We keep adding on more generation draws into
the batch until we have to stop due to command buffer end, secondary
command buffer calls or a barrier containing the access flag
VK_ACCESS_INDIRECT_COMMAND_READ_BIT. The side batch buffer jump
back right after the instruction where it was called.