llamafile is a framework designed to simplify the distribution and execution of large language models (LLMs) using single-file executables. It combines llama.cpp with Cosmopolitan Libc into one cohesive tool, enabling users to run LLMs locally without complex installations.
Key Features:
Creates single-file executables that run on most computers.
Supports various models like LLaVA and Mistral for diverse applications.
Offers both a web UI chat interface and an OpenAI-compatible API endpoint.
Compatible across platforms, including Windows, macOS, and Linux.
Provides command-line interfaces for direct interaction with models.
Audience & Benefits:
Ideal for developers seeking to deploy models without complex setups and end-users desiring easy access to powerful LLMs. Both benefit from reduced complexity, streamlined workflows, enhanced privacy through local execution, and versatile model support for varied applications.
llamafile can be installed via winget, ensuring a seamless experience for integrating and running advanced language models efficiently and securely.
README
llamafile
> We want to hear from you!
Mozilla.ai recently adopted the llamafile project, and we're planning an approach for codebase modernization. Please share what you find most valuable about llamafile and what would make it more useful for your work.
Read more via the blog and add your voice to the discussion here.
Our goal is to make open LLMs much more
accessible to both developers and end users. We're doing that by
combining llama.cpp with Cosmopolitan Libc into one
framework that collapses all the complexity of LLMs down to
a single-file executable (called a "llamafile") that runs
locally on most computers, with no installation.
# Download an example model (LLaVA 1.5 7B)
curl -LO https://huggingface.co/Mozilla/llava-v1.5-7b-llamafile/resolve/main/llava-v1.5-7b-q4.llamafile
# Make it executable (macOS/Linux/BSD)
chmod +x llava-v1.5-7b-q4.llamafile
# Run it (opens browser automatically)
./llava-v1.5-7b-q4.llamafile
Windows users: Rename the file to add .exe extension before running.
Documentation
Check the full documentation in the docs/ folder or online at mozilla-ai.github.io/llamafile, or directly jump into one of the following subsections:
While the llamafile project is Apache 2.0-licensed, our changes
to llama.cpp are licensed under MIT (just like the llama.cpp project
itself) so as to remain compatible and upstreamable in the future,
should that be desired.
The llamafile logo on this page was generated with the assistance of DALL·E 3.
