mirror of
https://github.com/ollama/ollama.git
synced 2025-11-11 21:17:41 +01:00
* TEMPORARY: Update the llama.cpp upstream to my fork's Granite Four branch
This will be redone once my branch is merged upstream in llama.cpp
* feat: Update all patches
There are a number that are no longer needed at all:
- 0003-embeddings: Embeddings entirely overhauled on master
- 0008-ensure-KV-cache-is-fully-defragmented: KV caching entirely
overhauled on master
- 0019-metal-add-mean-kernel-14267: Merged upstream
- 0020-CUDA-add-mean-operation-14313: Merged upstream
* feat: Sync llama.cpp and ggml
* fix: Update rsync-filter for all moved/new/removed files
* fix: Add files missing from sync
* fix: Update ggml rsync-filter for new ggml-cpu/arch subdirs
* fix: Add ggml files missing from sync
* fix: Narrow llama.cpp rsync-filter to not include mtmd main tool cpp files
* fix: Remove mtmd main cpp files
* fix: Add missing include in sampling_ext.cpp
* fix: Update llama.go to use mtmd instead of clip/llava
* fix: Add patch for mtmd_input_text
* chore: Ignore *.patched in the patch directory
* fix: Fix support for arch-specific ggml-cpu source files with new arrangement
In https://github.com/ggml-org/llama.cpp/pull/13892, all arch-specific
implementations were split out into a nested tree structure under
ggml-cpu/arch. This conflicts with standard CGO layout where all
arch-specific source files are expected to live in the same directory as
the parent go module and use suffixes based on GOOS and GOARCH. As such,
there were really two options for getting this to work:
1. Add a patch on top of the GGML sync to rearrange the files to match the
GO layout convention
2. Use CGO directives to conditionally include the nested source files in
the compilation units
This commit does (2) in order to minimize the set of changes needed on top
of the upstream file layout. To get this to work, there are two key things
needed:
1. In cpu.go, #cgo directives are added to explicitly set __${GOARCH}__ in
the preprocessor directives
2. In arch-impls.c|cpp, use an #ifdef | #elif defined | #endif chain to
explicitly include the .c|.cpp files for the given architecture from the
nested directory
* fix: Use mtmd_helper to correctly load the bitmap for the image
* fix: Apply patch for mtmd_text_input
* fix: Add missing stb to llama.cpp rsync-filter
* fix: Add sync'ed stb vendored header
* fix: Use c++17 and include vendor for go wrapper modules
* fix: Update patch 0015 for upstream implementation of uuid
* feat: Bump to the latest tip of the branch
* fix: Update patches for bump
* feat: Bump back to the cenral repo and point at the latest master
This includes granite 4 and a number of other model architectures!
* fix: Revert changes to ggml export GPU UUID patch
* fix: Add patch for GGML_VERSION and GGML_COMMIT constants
* feat: Sync all patched code
* build: Include cmake/common.cmake in ggml sync
* build: Add top-level include for GNUINstallDirs in CMakeLists.txt
This is used to populate CMAKE_INSTALL_BINDIR
* fix: Add a patch to avoid power throttling API on non-msvc windows builds
* fix: Sync patch changes for ggml-cpu.c
* feat: Bump llama.cpp to 4a4f42
This picks up support for Kimi K2 and PLaMO-2
* feat: Sync llama.cpp
* fix: Handle multi-chunk image encodings from mtmd
* fix: Re-number patches after merge with `main`
* feat: Bump to 41e78c in the makefile
* fix: Fix Solar and argsort/copy patches after bump
* fix: Remove Gemma3n CUDA Graphs patch
It was implemented upstream:
https://github.com/ggml-org/llama.cpp/pull/14741
* feat: Sync llama.cpp / ggml after latest bump
* build: Remove unnecessary CFLAGS definitions in cpu.go
* fix: Remove unnecessary additions in the rsync-filter
* fix: Remove unused vendored code for chat template parsing
* Revert "fix: Remove Gemma3n CUDA Graphs patch"
This reverts commit d724caced3.
* fix: Update 0020 CUDA Graphs for gemma3n to keep both llama.cpp and ollama fixes
https://github.com/ollama/ollama/pull/11195#issuecomment-3137312394
* fix: Sync ggml-cuda.cu after keeping both style cuda graph fixes for gemma3n
* unwind mxfp4 patch
Prepare to bump ggml with their impl for mxfp4
* bump
* fix windows build error
* Convert tensors at load time
Repack the mxfp4 tensors as ggmls kernels expect them to be.
* convert mlp bf16 to f32
* buffer the conversion better
* reshape earlier
* openai swiglu
* add ids
* split qkv, gate_up
* fix nested alt tags
* fast attention
* remove debug messages
* fix lint
* remove redundant test
* remap values only if source/target are different
* add back i32->i32 copy
* refactor cpu quants
* clean up vendor
* update patch instructions
* clean up patches
* remove webgpu
* update mem
* also handle gpt-oss
* revert convert changes
---------
Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
Co-authored-by: Gabe Goodhart <ghart@us.ibm.com>
Co-authored-by: Daniel Hiltgen <daniel@ollama.com>
188 lines
6.5 KiB
C++
Vendored
188 lines
6.5 KiB
C++
Vendored
// __ _____ _____ _____
|
|
// __| | __| | | | JSON for Modern C++
|
|
// | | |__ | | | | | | version 3.12.0
|
|
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
|
|
//
|
|
// SPDX-FileCopyrightText: 2013 - 2025 Niels Lohmann <https://nlohmann.me>
|
|
// SPDX-License-Identifier: MIT
|
|
|
|
#ifndef INCLUDE_NLOHMANN_JSON_FWD_HPP_
|
|
#define INCLUDE_NLOHMANN_JSON_FWD_HPP_
|
|
|
|
#include <cstdint> // int64_t, uint64_t
|
|
#include <map> // map
|
|
#include <memory> // allocator
|
|
#include <string> // string
|
|
#include <vector> // vector
|
|
|
|
// #include <nlohmann/detail/abi_macros.hpp>
|
|
// __ _____ _____ _____
|
|
// __| | __| | | | JSON for Modern C++
|
|
// | | |__ | | | | | | version 3.12.0
|
|
// |_____|_____|_____|_|___| https://github.com/nlohmann/json
|
|
//
|
|
// SPDX-FileCopyrightText: 2013 - 2025 Niels Lohmann <https://nlohmann.me>
|
|
// SPDX-License-Identifier: MIT
|
|
|
|
|
|
|
|
// This file contains all macro definitions affecting or depending on the ABI
|
|
|
|
#ifndef JSON_SKIP_LIBRARY_VERSION_CHECK
|
|
#if defined(NLOHMANN_JSON_VERSION_MAJOR) && defined(NLOHMANN_JSON_VERSION_MINOR) && defined(NLOHMANN_JSON_VERSION_PATCH)
|
|
#if NLOHMANN_JSON_VERSION_MAJOR != 3 || NLOHMANN_JSON_VERSION_MINOR != 12 || NLOHMANN_JSON_VERSION_PATCH != 0
|
|
#warning "Already included a different version of the library!"
|
|
#endif
|
|
#endif
|
|
#endif
|
|
|
|
#define NLOHMANN_JSON_VERSION_MAJOR 3 // NOLINT(modernize-macro-to-enum)
|
|
#define NLOHMANN_JSON_VERSION_MINOR 12 // NOLINT(modernize-macro-to-enum)
|
|
#define NLOHMANN_JSON_VERSION_PATCH 0 // NOLINT(modernize-macro-to-enum)
|
|
|
|
#ifndef JSON_DIAGNOSTICS
|
|
#define JSON_DIAGNOSTICS 0
|
|
#endif
|
|
|
|
#ifndef JSON_DIAGNOSTIC_POSITIONS
|
|
#define JSON_DIAGNOSTIC_POSITIONS 0
|
|
#endif
|
|
|
|
#ifndef JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON
|
|
#define JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON 0
|
|
#endif
|
|
|
|
#if JSON_DIAGNOSTICS
|
|
#define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS _diag
|
|
#else
|
|
#define NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS
|
|
#endif
|
|
|
|
#if JSON_DIAGNOSTIC_POSITIONS
|
|
#define NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS _dp
|
|
#else
|
|
#define NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS
|
|
#endif
|
|
|
|
#if JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON
|
|
#define NLOHMANN_JSON_ABI_TAG_LEGACY_DISCARDED_VALUE_COMPARISON _ldvcmp
|
|
#else
|
|
#define NLOHMANN_JSON_ABI_TAG_LEGACY_DISCARDED_VALUE_COMPARISON
|
|
#endif
|
|
|
|
#ifndef NLOHMANN_JSON_NAMESPACE_NO_VERSION
|
|
#define NLOHMANN_JSON_NAMESPACE_NO_VERSION 0
|
|
#endif
|
|
|
|
// Construct the namespace ABI tags component
|
|
#define NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c) json_abi ## a ## b ## c
|
|
#define NLOHMANN_JSON_ABI_TAGS_CONCAT(a, b, c) \
|
|
NLOHMANN_JSON_ABI_TAGS_CONCAT_EX(a, b, c)
|
|
|
|
#define NLOHMANN_JSON_ABI_TAGS \
|
|
NLOHMANN_JSON_ABI_TAGS_CONCAT( \
|
|
NLOHMANN_JSON_ABI_TAG_DIAGNOSTICS, \
|
|
NLOHMANN_JSON_ABI_TAG_LEGACY_DISCARDED_VALUE_COMPARISON, \
|
|
NLOHMANN_JSON_ABI_TAG_DIAGNOSTIC_POSITIONS)
|
|
|
|
// Construct the namespace version component
|
|
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch) \
|
|
_v ## major ## _ ## minor ## _ ## patch
|
|
#define NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT(major, minor, patch) \
|
|
NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT_EX(major, minor, patch)
|
|
|
|
#if NLOHMANN_JSON_NAMESPACE_NO_VERSION
|
|
#define NLOHMANN_JSON_NAMESPACE_VERSION
|
|
#else
|
|
#define NLOHMANN_JSON_NAMESPACE_VERSION \
|
|
NLOHMANN_JSON_NAMESPACE_VERSION_CONCAT(NLOHMANN_JSON_VERSION_MAJOR, \
|
|
NLOHMANN_JSON_VERSION_MINOR, \
|
|
NLOHMANN_JSON_VERSION_PATCH)
|
|
#endif
|
|
|
|
// Combine namespace components
|
|
#define NLOHMANN_JSON_NAMESPACE_CONCAT_EX(a, b) a ## b
|
|
#define NLOHMANN_JSON_NAMESPACE_CONCAT(a, b) \
|
|
NLOHMANN_JSON_NAMESPACE_CONCAT_EX(a, b)
|
|
|
|
#ifndef NLOHMANN_JSON_NAMESPACE
|
|
#define NLOHMANN_JSON_NAMESPACE \
|
|
nlohmann::NLOHMANN_JSON_NAMESPACE_CONCAT( \
|
|
NLOHMANN_JSON_ABI_TAGS, \
|
|
NLOHMANN_JSON_NAMESPACE_VERSION)
|
|
#endif
|
|
|
|
#ifndef NLOHMANN_JSON_NAMESPACE_BEGIN
|
|
#define NLOHMANN_JSON_NAMESPACE_BEGIN \
|
|
namespace nlohmann \
|
|
{ \
|
|
inline namespace NLOHMANN_JSON_NAMESPACE_CONCAT( \
|
|
NLOHMANN_JSON_ABI_TAGS, \
|
|
NLOHMANN_JSON_NAMESPACE_VERSION) \
|
|
{
|
|
#endif
|
|
|
|
#ifndef NLOHMANN_JSON_NAMESPACE_END
|
|
#define NLOHMANN_JSON_NAMESPACE_END \
|
|
} /* namespace (inline namespace) NOLINT(readability/namespace) */ \
|
|
} // namespace nlohmann
|
|
#endif
|
|
|
|
|
|
/*!
|
|
@brief namespace for Niels Lohmann
|
|
@see https://github.com/nlohmann
|
|
@since version 1.0.0
|
|
*/
|
|
NLOHMANN_JSON_NAMESPACE_BEGIN
|
|
|
|
/*!
|
|
@brief default JSONSerializer template argument
|
|
|
|
This serializer ignores the template arguments and uses ADL
|
|
([argument-dependent lookup](https://en.cppreference.com/w/cpp/language/adl))
|
|
for serialization.
|
|
*/
|
|
template<typename T = void, typename SFINAE = void>
|
|
struct adl_serializer;
|
|
|
|
/// a class to store JSON values
|
|
/// @sa https://json.nlohmann.me/api/basic_json/
|
|
template<template<typename U, typename V, typename... Args> class ObjectType =
|
|
std::map,
|
|
template<typename U, typename... Args> class ArrayType = std::vector,
|
|
class StringType = std::string, class BooleanType = bool,
|
|
class NumberIntegerType = std::int64_t,
|
|
class NumberUnsignedType = std::uint64_t,
|
|
class NumberFloatType = double,
|
|
template<typename U> class AllocatorType = std::allocator,
|
|
template<typename T, typename SFINAE = void> class JSONSerializer =
|
|
adl_serializer,
|
|
class BinaryType = std::vector<std::uint8_t>, // cppcheck-suppress syntaxError
|
|
class CustomBaseClass = void>
|
|
class basic_json;
|
|
|
|
/// @brief JSON Pointer defines a string syntax for identifying a specific value within a JSON document
|
|
/// @sa https://json.nlohmann.me/api/json_pointer/
|
|
template<typename RefStringType>
|
|
class json_pointer;
|
|
|
|
/*!
|
|
@brief default specialization
|
|
@sa https://json.nlohmann.me/api/json/
|
|
*/
|
|
using json = basic_json<>;
|
|
|
|
/// @brief a minimal map-like container that preserves insertion order
|
|
/// @sa https://json.nlohmann.me/api/ordered_map/
|
|
template<class Key, class T, class IgnoredLess, class Allocator>
|
|
struct ordered_map;
|
|
|
|
/// @brief specialization that maintains the insertion order of object keys
|
|
/// @sa https://json.nlohmann.me/api/ordered_json/
|
|
using ordered_json = basic_json<nlohmann::ordered_map>;
|
|
|
|
NLOHMANN_JSON_NAMESPACE_END
|
|
|
|
#endif // INCLUDE_NLOHMANN_JSON_FWD_HPP_
|