LLaMA (Large Language Model Meta Artificial-Intelligence) developed by Meta (ex. Facebook).
LLaMa need trained models to work. "Unfortunately", some got leaked, and from those, multiple models emerged, fine-tuned by the community.
- We have 4 models : 7B, 13B, 30B and 65B. 'B' for Billions of parameters.
Dalai is the library that run all of this locally on your PC, with a Web-UI.
- 7B => ~4 GB
- 13B => ~8 GB
- 30B => ~16 GB
- 65B => ~32 GB
- 7B => ~4.21GB
- 13B => ~8.14GB
- Full: The model takes up 31.17GB
- Quantized: 4.21GB
- Full: The model takes up 60.21GB
- Quantized: 4.07GB * 2 = 8.14GB
- Full: The model takes up 150.48GB
- Quantized: 5.09GB * 4 = 20.36GB
- Full: The model takes up 432.64GB
- Quantized: 5.11GB * 8 = 40.88GB
Commands (with Docker compose)
git clone https://github.com/cocktailpeanut/dalai.git cd dalai docker compose build docker compose run dalai npx dalai alpaca install 7B docker compose up -d
Can change alpaca to 'llama', and '7B' by another model
There are no 30B and 65B Alpaca models
Downloaded model goes in