Vulkan backend & ggml-vulkan-shaders.hpp #7323
-
I am just wondering how the contents of ggml-vulkan-shaders.hpp was generated? I would like to learn about how the vulkan backend works. If I understand correctly, this is compiled spirv shader code. Why store the compiled code as bytes here, rather than storing source and generating the binary at build time? Would this improve maintainability & transparency? PS. I am fairly new to llama.cpp, GPUs, vulkan, shaders and ML in general, so apologies if my questions are somewhat confused. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
The shaders are generated using the You can see the rationale behind most of the design decisions in the initial PR #2059. |
Beta Was this translation helpful? Give feedback.
The shaders are generated using the
ggml_vk_generate_shaders.py
script. You shouldn't need to use it unless the source is modified. The shaders are included to smooth out compilation and runtime.You can see the rationale behind most of the design decisions in the initial PR #2059.