Skip to content

Fix missing llama-server binary causing "binary not found" error#13

Open
IvanBrazza wants to merge 1 commit into
alpine-docker:masterfrom
IvanBrazza:fix/llama-server-binary-missing
Open

Fix missing llama-server binary causing "binary not found" error#13
IvanBrazza wants to merge 1 commit into
alpine-docker:masterfrom
IvanBrazza:fix/llama-server-binary-missing

Conversation

@IvanBrazza

Copy link
Copy Markdown

Summary

  • Ollama 0.30+ moved inference out-of-process into a standalone llama-server binary plus supporting shared libraries (libllama-server-impl.so, libllama.so, libmtmd.so, etc.), all installed together under /usr/lib/ollama/. This Dockerfile only ever copied libggml-cpu-* and libggml-base.so, so llama-server was never present in the built image, breaking every model run with llama-server binary not found.
  • Copy the whole /usr/lib/ollama directory instead of hand-picking files, and prune the GPU backend subdirs (cuda_v*, rocm_v*, vulkan, cuda_jetpack*) in a separate build stage so the deleted GPU bytes never end up in the final image's layers (a naive rm -rf in the final stage bloats the image to several GB due to layer history).

Fixes #12

Test plan

  • Built the image locally and confirmed /usr/lib/ollama contains llama-server and all its dependent .so files
  • Ran the container, pulled nomic-embed-text, and ran ollama run nomic-embed-text helloworld — got a real embedding back (previously failed with the binary-not-found error)
  • Confirmed final image size stays close to the original ~85MB (measured ~105MB) rather than ballooning with GPU backend libraries

🤖 Generated with Claude Code

Ollama 0.30+ moved inference out-of-process into a standalone
llama-server binary plus supporting shared libraries
(libllama-server-impl.so, libllama.so, libmtmd.so, etc.), all
installed together under /usr/lib/ollama/. This Dockerfile only ever
copied libggml-cpu-* and libggml-base.so, so llama-server was never
present in the built image, breaking every model run.

Copy the whole /usr/lib/ollama directory instead of hand-picking
files, and prune the GPU backend subdirs (cuda_v*, rocm_v*, vulkan,
cuda_jetpack*) in a separate build stage so the deleted GPU bytes
never end up in the final image's layers. Verified the rebuilt image
stays close to the original ~85MB size and successfully serves a
model end-to-end.

Fixes alpine-docker#12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Error

1 participant