Daniel Hiltgen 31f64183dc perf: build graph for next batch in parallel to keep GPU busy ...

This refactors the main run loop of the ollama runner to perform the main GPU
intensive tasks (Compute+Floats) in a go routine so we can prepare the next
batch in parallel to reduce the amount of time the GPU stalls waiting for the
next batch of work.

2025-08-19 12:33:03 -07:00

..

backend

perf: build graph for next batch in parallel to keep GPU busy

2025-08-19 12:33:03 -07:00

nn

…

backend.go

perf: build graph for next batch in parallel to keep GPU busy

2025-08-19 12:33:03 -07:00