whisper.cpp/examples/talk-llama
Georgi Gerganov 5fd1bdd7fc
whisper : add GPU support via cuBLAS (#834)
* make : add WHISPER_CUBLAS

* make : fix CUBLAS build

* whisper : disable Flash Attention + adjust memory buffers

* whisper : remove old commented code

* readme : add cuBLAS instructions

* cmake : add WHISPER_CUBLAS option

* gitignore : ignore build-cublas
2023-04-30 12:14:33 +03:00
..
prompts talk-llama : add alpaca support (#668) 2023-03-29 23:01:14 +03:00
.gitignore talk, talk-llama : add basic example script for eleven-labs tts (#728) 2023-04-14 19:53:58 +03:00
CMakeLists.txt whisper : add GPU support via cuBLAS (#834) 2023-04-30 12:14:33 +03:00
eleven-labs.py talk, talk-llama : add basic example script for eleven-labs tts (#728) 2023-04-14 19:53:58 +03:00
llama_internal.h talk-llama : update to latest llama.cpp (improved performance) 2023-04-10 22:59:13 +03:00
llama_util.h talk-llama : update to latest llama.cpp (improved performance) 2023-04-10 22:59:13 +03:00
llama.cpp talk-llama : update to latest llama.cpp (improved performance) 2023-04-10 22:59:13 +03:00
llama.h talk-llama : update to latest llama.cpp (improved performance) 2023-04-10 22:59:13 +03:00
README.md talk-llama : add discussion link 2023-03-28 10:11:34 +03:00
speak.sh talk, talk-llama : add basic example script for eleven-labs tts (#728) 2023-04-14 19:53:58 +03:00
talk-llama.cpp talk-llama : correct default speak.sh path (#720) 2023-04-14 19:36:09 +03:00

talk-llama

Talk with an LLaMA AI in your terminal

Demo Talk

Building

The talk-llama tool depends on SDL2 library to capture audio from the microphone. You can build it like this:

# Install SDL2 on Linux
sudo apt-get install libsdl2-dev

# Install SDL2 on Mac OS
brew install sdl2

# Build the "talk-llama" executable
make talk-llama

# Run it
./talk-llama -mw ./models/ggml-small.en.bin -ml ../llama.cpp/models/13B/ggml-model-q4_0.bin -p "Georgi" -t 8
  • The -mw argument specifies the Whisper model that you would like to use. Recommended base or small for real-time experience
  • The -ml argument specifies the LLaMA model that you would like to use. Read the instructions in https://github.com/ggerganov/llama.cpp for information about how to obtain a ggml compatible LLaMA model

TTS

For best experience, this example needs a TTS tool to convert the generated text responses to voice. You can use any TTS engine that you would like - simply edit the speak.sh script to your needs. By default, it is configured to use MacOS's say, but you can use whatever you wish.

Discussion

If you have any feedback, please let "us" know in the following discussion: https://github.com/ggerganov/whisper.cpp/discussions/672?converting=1