#local-llm

Articles tagged with #local-llm

Running Large MoE LLMs on Modest Hardware with FreeToken
A new open-source inference engine designed to run MoE models on hardware that doesn't have enough VRAM to hold them
Sep 10, 202615 min read14
Using a mac mini (2012) for LLM inference with KoboldCpp
or... "necessity is the mother of invention"
Sep 9, 202614 min read34