Daniel Hiltgen 1fdb351c37

New engine: vision models and auto-fallback (#9113) ...

* Include unified vision layers in memory prediction

For newer vision models with a single gguf, include
the projection estimates.

* Adjust CLI to handle both styles of vision model metadata

* Wire up new tokenizers for new engine

If we're loading the new engine, utilize the new model
text processor instead of calling into cgo wrappers for
llama.cpp.  This also cleans up some tech debt from the
older tokenization flow for the C++ server which was
no longer used.

This also adjusts the grammar handling logic to pass
through to the new engine instead of utilizing the cgo
schema to grammar call.

* Lay foundation for auto selection of new engine

2025-03-04 09:03:46 -08:00

..

ggml

New engine: vision models and auto-fallback (#9113)

2025-03-04 09:03:46 -08:00

util/bufioutil

next ollama runner (#7913)

2025-02-13 16:31:21 -08:00