TL;DR : Dalai llama & alpaca
Dalai
IBrowse, Search, and Manage Stablediffusion Images in One Place
What ?
LLaMA (Large Language Model Meta Artificial-Intelligence) developed by Meta (ex. Facebook).
LLaMa need trained models to work. "Unfortunately", some got leaked, and from those, multiple models emerged, fine-tuned by the community.
- We have 4 models : 7B, 13B, 30B and 65B. 'B' for Billions of parameters.
Alpaca model is a fine-tuned version of the LLaMA model capable of following instructions ...
Dalai is the library that run all of this locally on your PC, with a Web-UI.
Requirements:
RAM
- 7B => ~4 GB
- 13B => ~8 GB
- 30B => ~16 GB
- 65B => ~32 GB
Disk space
Alpaca
- 7B => ~4.21GB
- 13B => ~8.14GB
Llama
- 7B
- Full: The model takes up 31.17GB
- Quantized: 4.21GB
- 13B
- Full: The model takes up 60.21GB
- Quantized: 4.07GB * 2 = 8.14GB
- 30B
- Full: The model takes up 150.48GB
- Quantized: 5.09GB * 4 = 20.36GB
- 65B
- Full: The model takes up 432.64GB
- Quantized: 5.11GB * 8 = 40.88GB
Commands (with Docker compose)
git clone https://github.com/cocktailpeanut/dalai.git
cd dalai
docker compose build
docker compose run dalai npx dalai alpaca install 7B
docker compose up -dCan change alpaca to 'llama', and '7B' by another model
There are no 30B and 65B Alpaca models
Downloaded model goes in ./models
App (Web UI)
Web UI on local HTTP port 3000 : http://127.0.0.1:3000/ or http://localhost:3000
