4 Commits

Author	SHA1	Message	Date
Daniel Hiltgen	171796791f	Adjust mmap logic for cuda windows for faster model load ... On Windows, recent llama.cpp changes make mmap slower in most cases, so default to off. This also implements a tri-state for use_mmap so we can detect the difference between a user provided value of true/false, or unspecified.	2024-06-17 16:54:30 -07:00
Michael Yang	e40145a39d	lint	2024-06-04 11:13:30 -07:00
Jackie Li	af47413dba	Add MarshalJSON to Duration (#3284) ... --------- Co-authored-by: Patrick Devine <patrick@infrahq.com>	2024-05-06 15:59:18 -07:00
Patrick Devine	47cfe58af5	Default Keep Alive environment variable (#3094) ... --------- Co-authored-by: Chris-AS1 <8493773+Chris-AS1@users.noreply.github.com>	2024-03-13 13:29:40 -07:00