Changelog

Generated from the LlamaNexus repo's own CHANGELOG.md - see source for the definitive version. Run node scripts/sync-changelogs.mjs to refresh this page.

[0.1.21] - 2026-07-18

Added

• Add POST /api/remove-model to permanently delete a downloaded model: removes its GGUF file(s) from the Hugging Face cache (resolving through the snapshot symlink to the real blob, so the actual disk space is freed - not just a dangling symlink), and its router.preset.ini ctx-size override section, if it has one. Best-effort unloads it first so a currently-loaded model isn't left served from a file that's about to disappear. Restarts llama-server afterward (same reason a successful pull does) so its router - which only scans --models-dir/--models-preset once at its own startup - stops reporting the removed model in /openai/v1/models//ollama/api/tags and no longer allows loading it. Multi-shard quants have every shard removed together, same grouping /ollama/api/tags already uses

[0.1.20] - 2026-07-15

Added

• Add a --version flag that prints the version and exits immediately, without requiring a subcommand or printing the usual startup banner. No short flag, since -v is already --verbose

[0.1.19] - 2026-07-13

Fixed

• Loading a big model could fail with a 502 llama-server unavailable at almost exactly 5 minutes in: the HTTP client used to forward /openai/v1/chat/completions to llama-server had a ResponseHeaderTimeout of 5 minutes, which fires the moment llama-server hasn't sent response headers yet - and it sends none at all until it's done loading the model, so any cold start slower than 5 minutes was cut off regardless of the client's own overall 10-minute Timeout. ResponseHeaderTimeout is removed and the overall timeout - now the only limit - is raised to 15 minutes and made configurable via LLAMANEXUS_CHAT_TIMEOUT_SECONDS (e.g. set it in docker-compose's environment: for very large models or slow hardware)

[0.1.18] - 2026-07-11

Added

• /ollama/api/pull accepts an optional files array alongside model to download an explicit set of exact repo-relative paths as one batch (e.g. every shard of a split GGUF quant) - llama-server is restarted once after the whole batch completes instead of once per file. Progress lines gain optional part/parts fields for a "part 3 of 9"-style UI, aggregated across the batch by hf_progress_download.py

Fixed

• The single-tag download path (repo:TAG) resolved an ambiguous tag match by silently downloading only the first matching file and logging a warning nobody saw - this was a real bug for split-quant repos specifically, since every shard of a quant matches the same tag substring (e.g. all of model-Q4_K_M-00001-of-00009.gguf … -00009-of-00009.gguf match tag Q4_K_M), so only shard 1 was ever fetched. A tag matching multiple files now downloads the whole set together if - and only if - they're all consistent shards of one group (same base name, same "-of-N" total); genuinely ambiguous matches now fail with an error instead of guessing
• ScanHFCacheModels (backing /ollama/api/tags and the llamanexus run CLI) created one bogus entry per shard file for a split quant, and never looked inside per-quant subdirectories at all (some repos, e.g. unsloth's dynamic quants, nest each quant's files in its own folder). It now recursively scans the whole snapshot directory and groups shard files into a single entry per quant, sized as the sum of all shards and pointing at shard 1 (what llama.cpp's own split-file loading expects). Flat, non-sharded files keep the exact same tag they had before this change
• Known limitation, not addressed here: whether llama-server's own --models-dir/--models-preset router (which is what actually resolves a model id to a file for real serving, independent of this proxy) correctly recognizes a downloaded shard set as one loadable model is outside this codebase and unverified

[0.1.17] - 2026-07-10

Fixed

• Dockerfile.base's ROCm GPU_TARGETS auto-selection was wrong on two counts, found by checking ROCm/llvm-project's actual compiler target table (llvm/lib/TargetParser/TargetParser.cpp) across release tags instead of AMD's product-support docs (which only cover officially validated products, not every target the compiler accepts): (1) gfx940/gfx941 (early MI300 preview names) are still valid through ROCm 7.1.1 but were removed starting exactly at 7.2.0 (clang: error: unsupported HIP gpu architecture: gfx940 on 7.2+); (2) gfx950 (CDNA4/MI350) was wrongly gated to "7.x only" - it's actually been present since exactly 6.4.0, not 7.0. GPU_TARGETS now does a real version comparison (sort -V-based, handles 2- and 3-part version strings) against both boundaries instead of a bare 7* string match

[0.1.16] - 2026-07-10

Fixed

• Dockerfile.vulkan's final image failed with error: externally-managed-environment on pip3 install huggingface_hub: Ubuntu 24.04 (which only this image's runtime stage uses, unlike the 22.04-based CUDA/ROCm runtimes) enforces PEP 668 by default. Added --break-system-packages (verified it installs cleanly) - safe here since this is a throwaway single-purpose container, not a real host Python environment

[0.1.15] - 2026-07-10

Fixed

• The Vulkan backend failed to configure with Could not find a package configuration file provided by "SPIRV-Headers": ggml-vulkan's CMakeLists.txt does a REQUIRED find_package(SPIRV-Headers CONFIG) that libvulkan-dev/glslc alone don't satisfy. Added the spirv-headers apt package (confirmed it installs SPIRV-HeadersConfig.cmake) to Dockerfile.base's Vulkan-only install step

[0.1.14] - 2026-07-09

Fixed

• build-base.sh cuda-<version> failed with a confusing nvidia/cuda:<version>-devel-ubuntu22.04: not found if given a two-part version like cuda-12.2 — nvidia/cuda images are only ever tagged with the full three-part patch version (e.g. 12.2.0), unlike rocm/dev-ubuntu-22.04 which also publishes two-part tags. build-base.sh now validates the CUDA version format up front and fails fast with a clear message pointing at the right tag format, instead of a cryptic Docker registry error later

[0.1.13] - 2026-07-09

Fixed

• build-base.sh cuda-<version>/rocm-<version> failed with unknown flag: --build-arg CUDA_VERSION: its version_build_args helper used echo "--build-arg" "CUDA_VERSION=...", which joins both onto a single line, so mapfile read them as one array element instead of two, and Docker saw --build-arg CUDA_VERSION=... glued together as one malformed argument. Changed to printf '%s\n' ... so each argument lands on its own line

[0.1.12] - 2026-07-09

Added

• build-base.sh now accepts a specific CUDA/ROCm version right on the command line, e.g. ./build-base.sh cuda-12.4 or ./build-base.sh rocm-7.0 — the version suffix is passed through as --build-arg CUDA_VERSION=.../ROCM_VERSION=... to both the base and (with --full) final image builds, and whatever's typed becomes the image tag verbatim (cuda-12.4 → makershop/llamanexus:cuda-12.4-base)
• Dockerfile.base's ROCm GPU_TARGETS now auto-selects based on ROCM_VERSION: gfx950 (CDNA4/MI350-series) is added automatically for ROCm 7.x, since it isn't a recognized compile target on 6.x's compiler — still overridable via --build-arg GPU_TARGETS=...

Changed — breaking

• Rename the ROCm/HIP backend from hip to rocm everywhere: Dockerfile.hip → Dockerfile.rocm, --build-arg BACKEND=hip → BACKEND=rocm, image tags hip-base/hip-beta → rocm-base/rocm-beta. Matches cuda/rocm/vulkan all being platform names, rather than mixing in hip (which is really just the API within ROCm)
• Dockerfile.cuda/Dockerfile.rocm/Dockerfile.vulkan now take a BASE_IMAGE build arg (default makershop/llamanexus:<backend>-base) instead of hardcoding that tag, so build-base.sh --full can point the final image at a version-specific base tag like makershop/llamanexus:cuda-12.4-base; Dockerfile.cuda's runtime stage also now takes CUDA_VERSION instead of hardcoding 12.2.0

[0.1.11] - 2026-07-09

Fixed

• The CUDA backend's base/final images had roughly doubled in size (~51GB→99GB base, ~4GB→6GB final) after CUDA_ARCHITECTURES defaulted to an explicit "61;70;75;80;86;89;90" list: passing bare architecture numbers (no -real/-virtual suffix) tells nvcc to compile both full machine code and PTX for every one of them, whereas llama.cpp's own ggml-cuda/CMakeLists.txt default (used when CMAKE_CUDA_ARCHITECTURES isn't set at all) is a much leaner mix — mostly PTX-only (-virtual) with fully-compiled code (-real) for just the one or two architectures most likely to matter. CUDA_ARCHITECTURES now defaults to empty, so nothing is passed to cmake and llama.cpp's own default takes over; set --build-arg CUDA_ARCHITECTURES=... explicitly only if you want to narrow the build to specific card(s)

[0.1.10] - 2026-07-09

Fixed

• Linking ggml-rpc-server (and other targets) failed with relocation R_X86_64_32 against '.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE on the HIP backend: Ubuntu's toolchain defaults to PIE executables, which need every linked object to be position-independent, but the static libs weren't being built with -fPIC. Added -DCMAKE_POSITION_INDEPENDENT_CODE=ON to all three backends' cmake configure step in Dockerfile.base.

[0.1.9] - 2026-07-09

Fixed

• The HIP backend (Dockerfile.base --build-arg BACKEND=hip) failed to build against rocm/dev-ubuntu-22.04:6.2: it's missing hipBLAS/rocBLAS (only bundled in ROCm's -complete image tags, not this plain one) — CMake Error ... Could not find a package configuration file provided by "hipblas" — now installed explicitly via a HIP-only hipblas-dev/rocblas-dev apt step
• Even with hipBLAS installed, ROCm 6.2 still failed with error: unknown type name '__hip_fp8_e4m3': llama.cpp's ggml-cuda/vendors/hip.h gates that type on HIP_VERSION >= 60200000 (ROCm 6.2.0), but rocm/dev-ubuntu-22.04:6.2 doesn't actually ship the hip/hip_fp8.h header that type comes from, despite nominally satisfying that check — bumped the default ROCM_VERSION (in both Dockerfile.base and Dockerfile.hip) from 6.2 to 6.4, safely past the boundary

[0.1.8] - 2026-07-09

Added

• Add Vulkan as a build backend (Dockerfile.base --build-arg BACKEND=vulkan, plus Dockerfile.vulkan for the final image) alongside CUDA and ROCm/HIP, for any Vulkan-capable GPU (Nvidia/AMD/Intel)
• Add build-base-metal.sh to build llama.cpp with Metal (-DGGML_METAL=ON) and the llamanexus binary natively on macOS — Metal can't be built or used inside a Docker container (no macOS base image exists, and Metal needs the actual macOS SDK)
• Add build-base.sh, wrapping Dockerfile.base: builds one backend or all three (no argument), and its --full flag also builds the final runnable image on top of the base — replaces the old single-file Dockerfile.full, now removed

Changed

• Combine the separate CUDA/HIP base Dockerfiles into one Dockerfile.base, selecting the backend via --build-arg BACKEND=cuda|hip|vulkan (default cuda)
• Reference the RPC worker binary by its real llama.cpp name, ggml-rpc-server, everywhere (Dockerfiles, main.go, build-base-metal.sh) instead of renaming it to rpc-server on install

Fixed

• hf_progress_download.py's path was hardcoded to /app/hf_progress_download.py (Docker-only); it's now resolved relative to the running llamanexus binary first, so a native (non-Docker) install (e.g. via build-base-metal.sh) works too
• Dockerfile.base's default CUDA_ARCHITECTURES=all failed with nvcc fatal: Unsupported gpu architecture 'compute_' on Ubuntu 22.04, whose cmake apt package (3.22) predates CMake 3.23's special all value — now defaults to an explicit architecture list (Pascal through Hopper)

[0.1.7] - 2026-07-09

Added

• GET /openai/v1/models now reports each model's n_ctx_train — its trained context length, read directly from GGUF metadata (*.context_length) — separate from context_length (the actual --ctx-size the router applied). Read via a small dependency-free GGUF metadata-header parser (gguf.go) that only reads the metadata block, not the tensor data, and caches results per file path. Best-effort matched against the router's own model ids (only attached when the match is unambiguous, so a repo with multiple quant files never risks reporting the wrong file's value).

[0.1.6] - 2026-07-08

Fixed

• POST /ollama/api/pull now waits for llama-server to finish restarting (up to 60s) before the request completes, instead of returning as soon as the restart was triggered. Previously, a client that queried /openai/v1/models or issued a chat completion right after a successful pull could hit llama-server mid-restart and find the freshly downloaded model missing, appearing to require a full LlamaNexus restart to "fix" — restarting llama-server was always sufficient, the response just wasn't waiting for it to finish.
• POST /openai/v1/chat/completions now normalizes the request's model field (stripping a trailing .gguf/:latest) the same way /ollama/api/chat already did, before forwarding to llama-server. Previously a model downloaded and requested by its exact repo:filename.gguf form — in particular one whose quant tag isn't one of the 4 hardcoded short tags ScanHFCacheModels recognizes (Q4_K_M, Q8_0, Q5_K_M, Q4_0), so it falls back to the bare filename without .gguf — got rejected by the router with model ... not found even though it was present and loaded.

[0.1.5] - 2026-07-07

Added

• serve now unconditionally broadcasts its own presence via UDP heartbeat (type server-heartbeat) for client discovery — e.g. the VS Code extension's server picker. Advertises the main API address (--port), resolved the same way as the existing RPC-worker heartbeat: --advertise-addr flag → LLAMANEXUS_ADVERTISE_ADDR env → auto-detected outbound IP.
• New --discovery-port flag (default 50050) controls the port this new client-facing heartbeat is broadcast on.

Changed — breaking

• The RPC-worker heartbeat's discovery port flag is renamed from --discovery-port to --rpc-discovery-port (same default, 50051, same behavior otherwise). Scripts/containers that pass --discovery-port expecting the old RPC-worker-discovery meaning need to update to --rpc-discovery-port — the --discovery-port name now means the new client-discovery port above.

[0.1.4] - 2026-07-06

Added

• Runtime RPC worker management, backing the VS Code extension's new RPC worker panel:
  • GET /status now reports a mode field (single / manual / discovery) alongside the existing discovery/workers/roster/version.
  • POST /api/rpc/mode ({"mode":"single"|"rpc"}) — toggles between single-node mode and whichever RPC source was last active (auto-discovery if it was ever enabled, otherwise the remembered manual --rpc roster). Discovery is paused/resumed rather than forgotten, so toggling back re-enables it.
  • POST /api/rpc/workers ({"addr":"host:port"}) — adds a worker to the manual roster and restarts llama-server with it. Rejected while auto-discovery is active (that roster is heartbeat-managed).
  • POST /api/rpc/workers/remove ({"addr":"host:port"}) — removes a worker from the manual roster, falling back to single-node mode if the roster becomes empty.
  • All of the above go through a new RPCManager that owns mode/roster state, replacing the old fire-and-forget --rpc/--discovery locals; nothing is persisted to disk, so a real restart of LlamaNexus reverts to its own CLI flags.

[0.1.3] - 2026-07-06

Added

• POST /api/set-ctx-size — dedicated endpoint to change a model's ctx-size (writes router.preset.ini, restarts llama-server, and waits for the router to come back up before responding). Lets external tools trigger the existing context-size-override mechanism without faking a chat completion request the way the num_ctx-in-body trick requires.
• POST /api/eject-model — dedicated endpoint to unload a model, forwarding straight to llama-server's router /models/unload. Lets external tools eject a model without faking an Ollama-style keep_alive: 0 request.
• GET /openai/v1/models now reports two new fields per model:
  • context_length — the model's real, currently-applied ctx-size, sourced from its router.preset.ini per-model override or the [*] global default. Previously this endpoint returned no context-size information at all.
  • loaded — whether the router currently has that model loaded, sourced from the same live status /ollama/api/ps already exposed.
• ModelPresetManager.GetGlobalCtxSize() — reads the [*] section's default ctx-size, used as the fallback for context_length above when a model has no explicit override.
• VS Code extension (llama-nexus/): new companion extension for chatting with LlamaNexus directly from the editor. Highlights:
  • Streaming chat panel with markdown rendering, per-code-block Copy/Insert actions, and a live context-usage bar sized from the new context_length field (falling back to a manual setting when unavailable).
  • Tool/function-calling support: the model can call read_file, patch_file (targeted snippet replacement, to avoid re-emitting whole files), write_file, and list_files against the open workspace, all confined to the workspace root, with a confirmation prompt before any write.
  • /ctx-size <size> and /model slash commands, the latter showing a live-loaded indicator per model and an eject action, both backed by the two new endpoints above.
  • See llama-nexus/README.md for full details and settings.

Fixed

• N/A for this release — all changes above are additive.

Notes

• If you rely on tool/function calling from the VS Code extension (or any other OpenAI-compatible client), launch llama-server with --jinja so it renders the model's real chat template. Without it, some models (observed with Qwen3-Coder) can leak tool calls as literal text on later turns of a conversation instead of structured JSON.

[0.1.0] - 2026-07-01

• Initial release