Jeff Bolz cbb88c4050 vulkan: Optimize mul_mat_vec p021 and nc shaders (llama/12505) ... * tests: add mul_mat perf/functional tests for p021/nc vulkan shaders * vulkan: Optimize mul_mat_vec p021 and nc shaders. These shaders are used in attention calculations, and when the KV cache grows large they start to dominate the run time. For the nc shader (which is called with large 'k' dimension), use unrolling and vector loads. For the p021 shader (which is called with large 'm' and small 'k' dimensions), take advantage of grouped query attention to reuse loads from the A matrix for the whole group, and reduce the number of workgroups (too much overhead from tiny dispatches). Using subgroupAdd in the p021 shader also helps, use that conditionally.		2025-03-27 11:06:03 +02:00
..
cmake	cmake: Comment out GGML_BIN_DIR for now (ggml/1139)	2025-03-27 11:06:03 +02:00
include	llama: Add support for RWKV v7 architecture (llama/12412)	2025-03-27 11:06:03 +02:00
src	vulkan: Optimize mul_mat_vec p021 and nc shaders (llama/12505)	2025-03-27 11:06:03 +02:00
.gitignore	whisper : reorganize source code + improve CMake (#2256)	2024-06-26 19:34:09 +03:00
CMakeLists.txt	SYCL: using graphs is configurable by environment variable and compile option (llama/12371)	2025-03-27 11:06:03 +02:00