extern/whisper.cpp
1
0
Fork 1
mirror of https://github.com/ggerganov/whisper.cpp.git synced 2025-08-18 16:58:34 +02:00
Code Issues Packages Projects Releases Wiki Activity

CPU/CUDA: Gemma 2 FlashAttention support (llama/8542)

Browse Source 
* CPU/CUDA: Gemma 2 FlashAttention support

* apply logit_softcap to scale in kernel

* disable logit softcapping tests on Metal

* remove metal check
This commit is contained in:
Johannes Gäßler
2024-08-24 21:34:59 +02:00
committed by Georgi Gerganov
parent 9b16ddd3a5
commit 24d8534bd8
10 changed files with 304 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
ggml/include/ggml.h
View File

@@ -1807,7 +1807,8 @@ extern "C" {
struct ggml_tensor * v,
struct ggml_tensor * mask,
float scale,
float max_bias);
float max_bias,
float logit_softcap);
GGML_API void ggml_flash_attn_ext_set_prec(
struct ggml_tensor * a,