TheCornCollector@piefed.zip to LocalLLaMA@sh.itjust.worksEnglish · 1 month agoQwen3.6 27B releasedhuggingface.coexternal-linkmessage-square23linkfedilinkarrow-up165arrow-down11file-text
arrow-up164arrow-down1external-linkQwen3.6 27B releasedhuggingface.coTheCornCollector@piefed.zip to LocalLLaMA@sh.itjust.worksEnglish · 1 month agomessage-square23linkfedilinkfile-text
minus-squareAbrinoxus@thelemmy.clublinkfedilinkEnglisharrow-up2·1 month agoKoboldcpp is an easy way to get into running local llm:s they have executables for linux, mac, windows on their github and a simple gui to load a model
minus-squaresexual_tomato@lemmy.dbzer0.comlinkfedilinkEnglisharrow-up4·1 month agoI use ollama to run it, litellm to put an openai API in front of it, and use it via any app that can talk to an open AI API.
minus-squareAbrinoxus@thelemmy.clublinkfedilinkEnglisharrow-up1·1 month agoNp. Kobold has a very active discord as well (i think matrix too)
minus-squarevenusaur@lemmy.worldlinkfedilinkEnglisharrow-up2·1 month agoI ended up starting with llama.cpp. I’ll check out Kobold too.
Koboldcpp is an easy way to get into running local llm:s they have executables for linux, mac, windows on their github and a simple gui to load a model
I use ollama to run it, litellm to put an openai API in front of it, and use it via any app that can talk to an open AI API.
Thanks!
Np. Kobold has a very active discord as well (i think matrix too)
I ended up starting with llama.cpp. I’ll check out Kobold too.