4 Commits

Author	SHA1	Message	Date
Georgi Gerganov	3623bd58f2	ggml : fix I8MM Q4_1 scaling factor conversion (llama/10562) ... ggml-ci	2024-12-08 20:14:35 +02:00
Georgi Gerganov	4ca1e72fe0	ggml : fix row condition for i8mm kernels (llama/10561) ... ggml-ci	2024-12-08 20:14:35 +02:00
Eve	3216efef2e	AVX BF16 and single scale quant optimizations (llama/10212) ... * use 128 bit loads (i've tried 256->128 to death and its slower) * double accumulator * avx bf16 vec dot * +3% q4_0 inference * +7% tg +5% pp compared to master * slower f16c version, kep for reference * 256b version, also slow. i tried :) * revert f16 * faster with madd * split to functions * Q8_0 and IQ4_NL, 5-7% faster * fix potential overflow (performance reduced) * 16 bit add for q4_0 only * merge	2024-11-20 21:00:08 +02:00
Diego Devesa	746bf2596f	ggml : build backends as libraries (llama/10256) ... * ggml : build backends as libraries --------- Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> Co-authored-by: R0CKSTAR <xiaodong.ye@mthreads.com>	2024-11-20 21:00:08 +02:00