Base object structs

The Vulkan runtime code provides a set of base object structs which must be used if you want your driver to take advantage of any of the runtime code. There are other base structs for various things which are not covered here but those are optional. The ones covered here are the bare minimum set which form the core of the Vulkan runtime code:

As one might expect, vk_instance is the required base struct for implementing VkInstance, vk_physical_device is required for VkPhysicalDevice, and vk_device for VkDevice. Everything else must derive from vk_object_base or from some struct that derives from vk_object_base.

vk_object_base

The root base struct for all Vulkan objects is vk_object_base. Every object exposed to the client through the Vulkan API must inherit from vk_object_base by having a vk_object_base or some struct that inherits from vk_object_base as the driver struct’s first member. Even though we have container_of() and use it liberally, the vk_object_base should be the first member as there are a few places, particularly in the logging framework, where we use void pointers to avoid casting and this only works if the address of the driver struct is the same as the address of the vk_object_base.

The standard pattern for defining a Vulkan object inside a driver looks something like this:

struct drv_sampler {
   struct vk_object_base base;

   /* Driver fields */
};

VK_DEFINE_NONDISP_HANDLE_CASTS(drv_sampler, base, VkSampler,
                               VK_OBJECT_TYPE_SAMPLER);

Then, to the object in a Vulkan entrypoint,

VKAPI_ATTR void VKAPI_CALL drv_DestroySampler(
    VkDevice                                    _device,
    VkSampler                                   _sampler,
    const VkAllocationCallbacks*                pAllocator)
{
   VK_FROM_HANDLE(drv_device, device, _device);
   VK_FROM_HANDLE(drv_sampler, sampler, _sampler);

   if (!sampler)
      return;

   /* Tear down the sampler */

   vk_object_free(&device->vk, pAllocator, sampler);
}

The VK_DEFINE_NONDISP_HANDLE_CASTS() macro defines a set of type-safe cast functions called drv_sampler_from_handle() and drv_sampler_to_handle() which cast a VkSampler to and from a struct drv_sampler *. Because compile-time type checking with Vulkan handle types doesn’t always work in C, the _from_handle() helper uses the provided VkObjectType to assert at runtime that the provided handle is the correct type of object. Both cast helpers properly handle NULL and VK_NULL_HANDLE as inputs. The VK_FROM_HANDLE() macro provides a convenient way to declare a drv_foo pointer and initialize it from a VkFoo handle in one smooth motion.

struct vk_object_base

Base struct for all Vulkan objects

VkObjectType type

Type of this object

This is used for runtime type checking when casting to and from Vulkan handle types since compile-time type checking doesn’t always work.

struct vk_device *device

Pointer to the device in which this object exists, if any

This is NULL for instances and physical devices but should point to a valid vk_device for almost everything else. (There are a few WSI objects that don’t inherit from a device.)

struct vk_instance

Pointer to the instance in which this object exists

This is NULL for device level objects as it’s main purpose is to make the instance allocator reachable for freeing data owned by instance level objects.

void vk_object_base_init(struct vk_device *device, struct vk_object_base *base, VkObjectType obj_type)

Initialize a vk_base_object

Parameters:

  • device – [in] The vk_device this object was created from or NULL

  • base – [out] The vk_object_base to initialize

  • obj_type – [in] The VkObjectType of the object being initialized

void vk_object_base_finish(struct vk_object_base *base)

Tear down a vk_object_base

Parameters:

  • base – [out] The vk_object_base being torn down

VK_DEFINE_HANDLE_CASTS(__driver_type, __base, __VkType, __VK_TYPE)

Define handle cast macros for the given dispatchable handle type

For a given driver_struct, this defines driver_struct_to_handle() and driver_struct_from_handle() helpers which provide type-safe (as much as possible with Vulkan handle types) casts to and from the driver_struct type. As an added layer of protection, these casts use the provided VkObjectType to assert that the object is of the correct type when running with a debug build.

Parameters:

  • __driver_type – The name of the driver struct; it is assumed this is the name of a struct type and struct will be prepended automatically

  • __base – The name of the vk_base_object member

  • __VkType – The Vulkan object type such as VkImage

  • __VK_TYPE – The VkObjectType corresponding to __VkType, such as VK_OBJECT_TYPE_IMAGE

VK_DEFINE_NONDISP_HANDLE_CASTS(__driver_type, __base, __VkType, __VK_TYPE)

Define handle cast macros for the given non-dispatchable handle type

For a given driver_struct, this defines driver_struct_to_handle() and driver_struct_from_handle() helpers which provide type-safe (as much as possible with Vulkan handle types) casts to and from the driver_struct type. As an added layer of protection, these casts use the provided VkObjectType to assert that the object is of the correct type when running with a debug build.

Parameters:

  • __driver_type – The name of the driver struct; it is assumed this is the name of a struct type and struct will be prepended automatically

  • __base – The name of the vk_base_object member

  • __VkType – The Vulkan object type such as VkImage

  • __VK_TYPE – The VkObjectType corresponding to __VkType, such as VK_OBJECT_TYPE_IMAGE

VK_FROM_HANDLE(__driver_type, __name, __handle)

Declares a __driver_type pointer which represents __handle

Parameters:

  • __driver_type – The name of the driver struct; it is assumed this is the name of a struct type and struct will be prepended automatically

  • __name – The name of the declared pointer

  • __handle – The Vulkan object handle with which to initialize __name

vk_instance

struct vk_instance

Base struct for all VkInstance implementations

This contains data structures necessary for detecting enabled extensions, handling entrypoint dispatch, and implementing vkGetInstanceProcAddr(). It also contains data copied from the VkInstanceCreateInfo such as the application information.

VkAllocationCallbacks alloc

Allocator used when creating this instance

This is used as a fall-back for when a NULL pAllocator is passed into a device-level create function such as vkCreateImage().

struct vk_app_info app_info

VkInstanceCreateInfo::pApplicationInfo

const struct vk_instance_extension_table *supported_extensions

Table of all supported instance extensions

This is the static const struct passed by the driver as the supported_extensions parameter to vk_instance_init().

struct vk_instance_extension_table enabled_extensions

Table of all enabled instance extensions

This is generated automatically as part of vk_instance_init() from VkInstanceCreateInfo::ppEnabledExtensionNames.

struct vk_instance_dispatch_table dispatch_table

Instance-level dispatch table

struct [anonymous]

List of all physical devices and callbacks

This is used for automatic physical device creation, deletion and enumeration.

VkResult (*enumerate)(struct vk_instance *instance)

Enumerate physical devices for this instance

The driver can implement this callback for custom physical device enumeration. The returned value must be a valid return code of vkEnumeratePhysicalDevices.

Note that the loader calls vkEnumeratePhysicalDevices of all installed ICDs and fails device enumeration when any of the calls fails. The driver should return VK_SUCCESS when it does not find any compatible device.

If this callback is not set, try_create_for_drm will be used for enumeration.

VkResult (*try_create_for_drm)(struct vk_instance *instance, struct _drmDevice *device, struct vk_physical_device **out)

Try to create a physical device for a drm device

The returned value must be a valid return code of vkEnumeratePhysicalDevices, or VK_ERROR_INCOMPATIBLE_DRIVER. When VK_ERROR_INCOMPATIBLE_DRIVER is returned, the error and the drm device are silently ignored.

void (*destroy)(struct vk_physical_device *pdevice)

Handle the destruction of a physical device

This callback has to be implemented when using common physical device management. The device pointer and any resource allocated for the device should be freed here.

uint64_t trace_mode

Enabled tracing modes

bool trace_per_submit

Whether the capture mode is per-submit.

VkResult vk_instance_init(struct vk_instance *instance, const struct vk_instance_extension_table *supported_extensions, const struct vk_instance_dispatch_table *dispatch_table, const VkInstanceCreateInfo *pCreateInfo, const VkAllocationCallbacks *alloc)

Initialize a vk_instance

Along with initializing the data structures in vk_instance, this function validates the Vulkan version number provided by the client and checks that every extension specified by VkInstanceCreateInfo::ppEnabledExtensionNames is actually supported by the implementation and returns VK_ERROR_EXTENSION_NOT_PRESENT if an unsupported extension is requested.

Parameters:

  • instance – [out] The instance to initialize

  • supported_extensions – [in] Table of all instance extensions supported by this instance

  • dispatch_table – [in] Instance-level dispatch table

  • pCreateInfo – [in] VkInstanceCreateInfo pointer passed to vkCreateInstance()

  • alloc – [in] Allocation callbacks used to create this instance; must not be NULL

void vk_instance_finish(struct vk_instance *instance)

Tears down a vk_instance

Parameters:

  • instance – [out] The instance to tear down

Once a driver has a vk_instance, implementing all the various instance-level vkGet*ProcAddr() entrypoints is trivial:

VKAPI_ATTR PFN_vkVoidFunction VKAPI_CALL
drv_GetInstanceProcAddr(VkInstance _instance,
                        const char *pName)
{
   VK_FROM_HANDLE(vk_instance, instance, _instance);
   return vk_instance_get_proc_addr(instance,
                                    &drv_instance_entrypoints,
                                    pName);
}

PUBLIC VKAPI_ATTR PFN_vkVoidFunction VKAPI_CALL
vk_icdGetInstanceProcAddr(VkInstance instance,
                          const char *pName)
{
   return drv_GetInstanceProcAddr(instance, pName);
}
PFN_vkVoidFunction vk_instance_get_proc_addr(const struct vk_instance *instance, const struct vk_instance_entrypoint_table *entrypoints, const char *name)

Implementation of vkGetInstanceProcAddr()

PFN_vkVoidFunction vk_instance_get_proc_addr_unchecked(const struct vk_instance *instance, const char *name)

Unchecked version of vk_instance_get_proc_addr

This is identical to vk_instance_get_proc_addr() except that it doesn’t check whether extensions are enabled before returning function pointers. This is useful in window-system code where we may use extensions without the client explicitly enabling them.

PFN_vkVoidFunction vk_instance_get_physical_device_proc_addr(const struct vk_instance *instance, const char *name)

Implementation of vk_icdGetPhysicalDeviceProcAddr()

We also provide an implementation of vkEnumerateInstanceExtensionProperties() which can be used similarly:

VKAPI_ATTR VkResult VKAPI_CALL
drv_EnumerateInstanceExtensionProperties(const char *pLayerName,
                                         uint32_t *pPropertyCount,
                                         VkExtensionProperties *pProperties)
{
   if (pLayerName)
      return vk_error(NULL, VK_ERROR_LAYER_NOT_PRESENT);

   return vk_enumerate_instance_extension_properties(
      &instance_extensions, pPropertyCount, pProperties);
}
VkResult vk_enumerate_instance_extension_properties(const struct vk_instance_extension_table *supported_extensions, uint32_t *pPropertyCount, VkExtensionProperties *pProperties)

Implementation of vkEnumerateInstanceExtensionProperties()

vk_physical_device

struct vk_physical_device

Base struct for all VkPhysicalDevice implementations

struct vk_instance *instance

Instance which is the parent of this physical device

struct vk_device_extension_table supported_extensions

Table of all supported device extensions

This table is initialized from the supported_extensions parameter passed to vk_physical_device_init() if not NULL. If a NULL extension table is passed, all extensions are initialized to false and it’s the responsibility of the driver to populate the table. This may be useful if the driver’s physical device initialization order is such that extension support cannot be determined until significant physical device setup work has already been done.

struct vk_features supported_features

Table of all supported features

This table is initialized from the supported_features parameter passed to vk_physical_device_init() if not NULL. If a NULL features table is passed, all features are initialized to false and it’s the responsibility of the driver to populate the table. This may be useful if the driver’s physical device initialization order is such that feature support cannot be determined until significant physical device setup work has already been done.

struct vk_properties properties

Table of all physical device properties which is initialized similarly to supported_features

struct vk_physical_device_dispatch_table dispatch_table

Physical-device-level dispatch table

struct disk_cache *disk_cache

Disk cache, or NULL

struct wsi_device *wsi_device

WSI device, or NULL

const struct vk_sync_type *const *supported_sync_types

A null-terminated array of supported sync types, in priority order

The common implementations of VkFence and VkSemaphore use this list to determine what vk_sync_type to use for each scenario. The list is walked and the first vk_sync_type matching their criterion is taken. For instance, VkFence requires that it not be a timeline and support reset and CPU wait. If an external handle type is requested, that is considered just one more criterion.

const struct vk_pipeline_cache_object_ops *const *pipeline_cache_import_ops

A null-terminated array of supported pipeline cache object types

The common implementation of VkPipelineCache uses this to remember the type of objects stored in the cache and deserialize them immediately when importing the cache. If an object type isn’t in this list, then it will be loaded as a raw data object and then deserialized when we first look it up. Deserializing immediately avoids a copy but may be more expensive for objects that aren’t hit.

VkResult vk_physical_device_init(struct vk_physical_device *physical_device, struct vk_instance *instance, const struct vk_device_extension_table *supported_extensions, const struct vk_features *supported_features, const struct vk_properties *properties, const struct vk_physical_device_dispatch_table *dispatch_table)

Initialize a vk_physical_device

Parameters:

  • physical_device – [out] The physical device to initialize

  • instance – [in] The instance which is the parent of this physical device

  • supported_extensions – [in] Table of all device extensions supported by this physical device

  • supported_features – [in] Table of all features supported by this physical device

  • dispatch_table – [in] Physical-device-level dispatch table

void vk_physical_device_finish(struct vk_physical_device *physical_device)

Tears down a vk_physical_device

Parameters:

  • physical_device – [out] The physical device to tear down

vk_device

struct vk_device

Base struct for VkDevice

VkAllocationCallbacks alloc

Allocator used to create this device

This is used as a fall-back for when a NULL pAllocator is passed into a device-level create function such as vkCreateImage().

struct vk_physical_device

Pointer to the physical device

struct vk_device_extension_table enabled_extensions

Table of enabled extensions

struct vk_features enabled_features

Table of enabled features

struct vk_device_dispatch_table dispatch_table

Device-level dispatch table

const struct vk_device_dispatch_table *command_dispatch_table

Command dispatch table

This is used for emulated secondary command buffer support. To use emulated (trace/replay) secondary command buffers:

  1. Provide your “real” command buffer dispatch table here. Because this doesn’t get populated by vk_device_init(), the driver will have to add the vk_common entrypoints to this table itself.

  2. Add vk_enqueue_unless_primary_device_entrypoint_table to your device level dispatch table.

const struct vk_command_buffer_ops *command_buffer_ops

Command buffer vtable when using the common command pool

const struct vk_device_shader_ops *shader_ops

Shader vtable for VK_EXT_shader_object and common pipelines

const struct vk_acceleration_structure_build_ops *as_build_ops

Acceleration structure build vtable for common BVH building.

void (*write_buffer_cp)(VkCommandBuffer cmdbuf, VkDeviceAddress addr, void *data, uint32_t size)

Write data to a buffer from the command processor. This is simpler than setting up a staging buffer and faster for small writes, but is not meant for larger amounts of data. p data is owned by the caller and the driver is expected to write it out directly to the command stream as part of an immediate write packet.

VkResult (*capture_trace)(VkQueue queue)

Driver provided callback for capturing traces

Triggers for this callback are:

  • Keyboard input (F12)

  • Creation of a trigger file

  • Reaching the trace frame

VkResult (*check_status)(struct vk_device *device)

Checks the status of this device

This is expected to return either VK_SUCCESS or VK_ERROR_DEVICE_LOST. It is called before vk_queue::driver_submit and after every non-trivial wait operation to ensure the device is still around. This gives the driver a hook to ask the kernel if its device is still valid. If the kernel says the device has been lost, it MUST call vk_device_set_lost().

This function may be called from any thread at any time.

VkResult (*create_sync_for_memory)(struct vk_device *device, VkDeviceMemory memory, bool signal_memory, struct vk_sync **sync_out)

Creates a vk_sync that wraps a memory object

This is always a one-shot object so it need not track any additional state. Since it’s intended for synchronizing between processes using implicit synchronization mechanisms, no such tracking would be valid anyway.

If signal_memory is set, the resulting vk_sync will be used to signal the memory object from a queue via vk_queue_submit::signals. The common code guarantees that, by the time vkQueueSubmit() returns, the signal operation has been submitted to the kernel via the driver’s vk_queue::driver_submit hook. This means that any vkQueueSubmit() call which needs implicit synchronization may block.

If signal_memory is not set, it can be assumed that memory object already has a signal operation pending from some other process and we need only wait on it.

struct vk_pipeline_cache

Implicit pipeline cache, or NULL

enum vk_device_timeline_mode

An enum describing how timeline semaphores work

enumerator VK_DEVICE_TIMELINE_MODE_NONE

Timeline semaphores are not supported

enumerator VK_DEVICE_TIMELINE_MODE_EMULATED

Timeline semaphores are emulated with vk_timeline

In this mode, timeline semaphores are emulated using vk_timeline which is a collection of binary semaphores, one per time point. These timeline semaphores cannot be shared because the data structure exists entirely in userspace. These timelines are virtually invisible to the driver; all it sees are the binary vk_syncs, one per time point.

To handle wait-before-signal, we place all vk_queue_submits in the queue’s submit list in vkQueueSubmit() and call vk_device_flush() at key points such as the end of vkQueueSubmit() and vkSemaphoreSignal(). This ensures that, as soon as a given submit’s dependencies are fully resolvable, it gets submitted to the driver.

enumerator VK_DEVICE_TIMELINE_MODE_ASSISTED

Timeline semaphores are a kernel-assisted emulation

In this mode, timeline semaphores are still technically an emulation in the sense that they don’t support wait-before-signal natively. Instead, all GPU-waitable objects support a CPU wait-for-pending operation which lets the userspace driver wait until a given event on the (possibly shared) vk_sync is pending. The event is “pending” if a job has been submitted to the kernel (possibly from a different process) which will signal it. In vkQueueSubit, we use this wait mode to detect waits which are not yet pending and, the first time we do, spawn a thread to manage the queue. That thread waits for each submit’s waits to all be pending before submitting to the driver queue.

We have to be a bit more careful about a few things in this mode. In particular, we can never assume that any given wait operation is pending. For instance, when we go to export a sync file from a binary semaphore, we need to first wait for it to be pending. The spec guarantees that the vast majority of these waits return almost immediately, but we do need to insert them for correctness.

enumerator VK_DEVICE_TIMELINE_MODE_NATIVE

Timeline semaphores are 100% native

In this mode, wait-before-signal is natively supported by the underlying timeline implementation. We can submit-and-forget and assume that dependencies will get resolved for us by the kernel. Currently, this isn’t supported by any Linux primitives.

enum vk_queue_submit_mode submit_mode

Per-device submit mode

This represents the device-wide submit strategy which may be different from the per-queue submit mode. See vk_queue.submit.mode for more details.

VkResult vk_device_init(struct vk_device *device, struct vk_physical_device *physical_device, const struct vk_device_dispatch_table *dispatch_table, const VkDeviceCreateInfo *pCreateInfo, const VkAllocationCallbacks *alloc)

Initialize a vk_device

Along with initializing the data structures in vk_device, this function checks that every extension specified by VkInstanceCreateInfo::ppEnabledExtensionNames is actually supported by the physical device and returns VK_ERROR_EXTENSION_NOT_PRESENT if an unsupported extension is requested. It also checks all the feature struct chained into the pCreateInfo->pNext chain against the features returned by vkGetPhysicalDeviceFeatures2 and returns VK_ERROR_FEATURE_NOT_PRESENT if an unsupported feature is requested.

Parameters:

  • device – [out] The device to initialize

  • physical_device – [in] The physical device

  • dispatch_table – [in] Device-level dispatch table

  • pCreateInfo – [in] VkDeviceCreateInfo pointer passed to vkCreateDevice()

  • alloc – [in] Allocation callbacks passed to vkCreateDevice()

void vk_device_finish(struct vk_device *device)

Tears down a vk_device

Parameters:

  • device – [out] The device to tear down