Command Pools

The Vulkan runtime code provides a common VkCommandPool implementation which makes managing the lifetimes of command buffers and recycling their internal state easier. To use the common command pool a driver needs to fill out a vk_command_buffer_ops struct and set the command_buffer_ops field of vk_device.

struct vk_command_buffer_ops

Command buffer ops

VkResult (*create)(struct vk_command_pool*, VkCommandBufferLevel, struct vk_command_buffer**)

Creates a command buffer

Used by the common command pool implementation. This function MUST call vk_command_buffer_finish(). Notably, this function does not receive any additional parameters such as the level. The level will be set by vk_common_AllocateCommandBuffers() and the driver must not rely on it until vkBeginCommandBuffer() time.

void (*reset)(struct vk_command_buffer*, VkCommandBufferResetFlags)

Resets the command buffer

Used by the common command pool implementation. This function MUST call vk_command_buffer_reset(). Unlike vkResetCommandBuffer(), this function does not have a return value because it may be called on destruction paths.

void (*destroy)(struct vk_command_buffer*)

Destroys the command buffer

Used by the common command pool implementation. This function MUST call vk_command_buffer_finish().

By reducing the entirety of command buffer lifetime management to these three functions, much of the complexity of command pools can be implemented in common code, providing better, more consistent behavior across Mesa.

Command Buffer Recycling

The common command pool provides automatic command buffer recycling as long as the driver uses the common vkAllocateCommandBuffers() and vkFreeCommandBuffers() implementations. The driver must also provide the reset function pointer in vk_command_buffer_ops.

With the common command buffer pool, when the client calls vkFreeCommandBuffers(), the command buffers are not immediately freed. Instead, they are reset with VK_COMMAND_BUFFER_RESET_RELEASE_RESOURCES_BIT, their base object is recycled, and they are added to a free list inside the pool. When the client then calls vkAllocateCommandBuffers(), we check the free list and return a recycled command buffer, if any are available. This provides some basic command buffer pooling without the driver doing any additional work.

Custom command pools

If a driver wishes to recycle at a finer granularity than whole command buffers, they can do so by providing their own command pool implementation which wraps vk_command_pool. The common use-case here is if the driver wants to pool command-buffer-internal objects at a finer granularity than whole command buffers. The command pool provides a place where things like GPU command buffers or upload buffers can be cached without having to take a lock.

When implementing a custom command pool, drivers need only implement three entrypoints:

  • vkCreateCommandPool()

  • vkDestroyCommandPool()

  • vkTrimCommandPool()

All of the other entrypoints will be handled by common code so long as the driver’s command pool derives from vk_command_pool.

The driver implementation of the command buffer recycle() function should respect VK_COMMAND_BUFFER_RESET_RELEASE_RESOURCES_BIT and, when set, return any recyclable resources to the command pool. This may be set by the client when it calls vkResetCommandBuffer(), come from a whole-pool reset via VK_COMMAND_POOL_RESET_RELEASE_RESOURCES_BIT, or come from the common command buffer code when a command buffer is recycled.

The driver’s implementation of vkTrimCommandPool() should free any resources that have been cached within the command pool back to the device or back to the OS. It must also call vk_command_pool_trim() to allow the common code to free any recycled command buffers.

Reference

struct vk_command_pool

Command buffer vtable for command buffers allocated from this pool

VkCommandPoolCreateFlags flags

VkCommandPoolCreateInfo::flags

uint32_t queue_family_index

VkCommandPoolCreateInfo::queueFamilyIndex

VkAllocationCallbacks alloc

Allocator passed to vkCreateCommandPool()

bool recycle_command_buffers

True if we should recycle command buffers

struct list_head command_buffers

List of all command buffers

struct list_head free_command_buffers

List of freed command buffers for trimming.

VkResult vk_command_pool_init(struct vk_device *device, struct vk_command_pool *pool, const VkCommandPoolCreateInfo *pCreateInfo, const VkAllocationCallbacks *pAllocator)

Initialize a vk_command_pool

Parameters:

  • device – [in] The Vulkan device

  • pool – [out] The command pool to initialize

  • pCreateInfo – [in] VkCommandPoolCreateInfo pointer passed to vkCreateCommandPool()

  • pAllocator – [in] Allocation callbacks passed to vkCreateCommandPool()

void vk_command_pool_finish(struct vk_command_pool *pool)

Tear down a vk_command_pool

Parameters:

  • pool – [inout] The command pool to tear down

void vk_command_pool_trim(struct vk_command_pool *pool, VkCommandPoolTrimFlags flags)

Trim a vk_command_pool

This discards any resources that may be cached by the common vk_command_pool code. For driver-implemented command pools, drivers should call this function inside their vkTrimCommandPool() implementation. This should be called before doing any driver-specific trimming in case it ends up returning driver-internal resources to the pool.

Parameters:

  • pool – [inout] The command pool to trim

  • flags – [in] Flags controling the trim operation