highperfocused/ollama
1
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2025-11-11 03:57:46 +01:00
Code Issues Packages Projects Releases Wiki Activity

logs: fix bogus "0 MiB free" log line (#12590)

Browse Source 
On the llama runner, after the recent GGML bump a new log line reports
incorrect 0 MiB free after our patch to remove memory from the props.  This
adjusts the llama.cpp code to fetch the actual free memory of the active device.
This commit is contained in:
Daniel Hiltgen
2025-10-14 11:26:28 -07:00
committed by GitHub
parent 2aba569a2a
commit 850da848c5
2 changed files with 28 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
llama/llama.cpp/src/llama.cpp vendored
View File

@@ -267,10 +267,12 @@ static struct llama_model * llama_model_load_from_file_impl(
for (auto * dev : model->devices) {
ggml_backend_dev_props props;
ggml_backend_dev_get_props(dev, &props);
size_t memory_free, memory_total;
ggml_backend_dev_memory(dev, &memory_free, &memory_total);
LLAMA_LOG_INFO("%s: using device %s (%s) (%s) - %zu MiB free\n", __func__,
ggml_backend_dev_name(dev), ggml_backend_dev_description(dev),
props.device_id ? props.device_id : "unknown id",
props.memory_free/1024/1024);
memory_free/1024/1024);
}
const int status = llama_model_load(path_model, splits, *model, params);