mirror of
https://github.com/ollama/ollama.git
synced 2025-04-14 14:49:25 +02:00
1feff61977
When computing the size of the cache for sliding window attention, we don't need to multiple the batch size by the number of parallel sequences - the batch size is constant. This also simplifies the check for whether to allocate the cache size based on capacity or window size as the batch size is already incorporated into the capacity when handled by the runner.