| Component | Specification |
|---|---|
| VRAM | 80GB |
| Storage | 500GB (minimum) 1TB (recommended) |
| RAM | 128GB+ (recommended) |
| Example GPUs | A100 (recommended) H100 (overkill) |
| vCPU's | 12 (minimum) 18+ (recommended) |
Highly depends on what it is you want to run, which models etc. I'd recommend doing some research into the models and some experiements.
As a rule of thumb;
| Model Type | VRAM Requirement |
|---|---|
| Llama 70B & Mixtral-8x7B | >=50GB |
| Image generation models | ~8GB |