Just a stranger trying things.

  • 2 Posts
  • 40 Comments
Joined 2 years ago
cake
Cake day: July 16th, 2023

help-circle







  • To run the full 671B sized model (404GB in size), you would need more than 404GB of combined GPU memory and standard memory (and that’s only to run it, you would most probably want it all to be GPU memory to make it run fast).

    With 24GB of GPU memory, the largest model which would fit from the R1 series would be the 32b-qwen-distill-q4_K_M (20GB in size) available at ollama (and possibly elsewhere).