llama.cpp Install on Windows: Run GGUF Models Fast, by Laxmi Kant | KGP Talkie on YouTube. 13 min 9 s.
- Video length
- 13 min 9 s
- Time to do it
- 30 minutes
- Difficulty
- Medium
- Steps
- 10
What you need
Steps
Open the llama.cpp GitHub page and go to the Latest release. Click the build link and find the Windows list. Pick x64 for a normal PC, then CUDA 13 or CUDA 12 for NVIDIA, Vulkan for AMD or Intel graphics, or CPU if you have no card.
Extract the zip to a simple folder like C:\llama. If you chose a CUDA build, download the separate CUDA DLL zip and extract those files into the same folder, or the program will not start.
Open Command Prompt in that folder and run llama-cli with --list-devices. Check that your graphics card appears. If it says none, you picked the CPU build or the wrong zip.
Download a model straight from Hugging Face by passing -hf and the model name to llama-cli. Choose a Q4_K_M file for a good balance of size and quality on most PCs.
Chat in the terminal once the model loads. Type your prompt and press Enter. If you already use LM Studio, point llama-cli at those GGUF files with -m, and add --mmproj for vision models.
Start llama-server to get a browser chat page. If you see "couldn't bind HTTP server socket", the port is taken, so add --port 8081 or another free number.
Open the address the server prints, usually http://127.0.0.1:8080, in your browser. Use the built-in web UI to chat and to ask questions about an uploaded screenshot.
Test the OpenAI-compatible API with curl.exe. Put your JSON request in a body.json file and post it to the server's chat completions endpoint. Use curl.exe, not plain curl, in older PowerShell.
Read the real speed from the prompt eval and eval lines the server prints. Those tokens-per-second numbers tell you how the model actually performs on your hardware.
On a PC without a graphics card, use the CPU build, which is under 50 MB. Add --reasoning off to get answers in seconds instead of waiting on long thinking output.
Tips and common mistakes
Stop the server with Ctrl+C before starting a new one, or the port stays busy and you get the bind error again.
The two options you will use most are -c to set context size and -ngl to control how many layers go to the GPU.
Download a newer release zip and extract it over the old folder to update, keeping your models in a separate folder.
CUDA builds extract to about 700 MB, so keep them on a drive with free space.
Questions
Do I need a graphics card to run llama.cpp?
No. The CPU build runs on any 64-bit Windows PC, just slower, around 16 tokens per second on a desktop processor in the video. NVIDIA cards use the CUDA build and AMD or Intel graphics use Vulkan.
Why does llama-server say it couldn't bind the HTTP server socket?
Another program is already using port 8080. Start the server again with --port and a different number, such as 8081, then open that address in your browser.
What does Q4_K_M mean in a model name?
It is a 4-bit quantization level, a compressed version of the model. It keeps quality decent while making the file small enough to run on ordinary hardware.
This guide was drafted from the video's details and general task knowledge. It has not been checked against the full video. Check the video and relevant product instructions before following a procedure. Report a correction.
