Just been able to run Deepseek and Qwen AI on my local machine and have been testing them out.
It's just 5 minutes of browsing,
1) Install Ollama
It's a tool for running AI models locally on your machine.
Download it from here: https://ollama.com/download
2) Next, need to pull and run the DeepSeek R1 / Qwen AI model locally.
Ollama provides various model sizes—larger models generally mean more advanced AI capabilities, but they require a more powerful GPU.
1.5B version (smallest):
ollama run deepseek-r1:1.5b
8B version:
ollama run deepseek-r1:8b
14B version:
ollama run deepseek-r1:14b
32B version:
ollama run deepseek-r1:32b
70B version (biggest/smartest):
ollama run deepseek-r1:70b
Maybe start with a smaller model first to test the waters. Just open a terminal and run:
ollama run deepseek-r1:8b
Once it's pulled, the model will run locally on your machine. Simple as that!
Note: The bigger versions (like 32B and 70B) need some serious GPU power. Start small and work your way up based on your hardware!
**For Qwen Ai, use command below:
ollama run qwen2.5:3b
3) Set up Chatbox - a client for AI models
Chatbox: a free powerful desktop interface that works with most models.
Download here: https://chatboxai.app
In Chatbox, go to settings and switch the model provider to Ollama Api. Since we're running models locally, can ignore the built-in cloud AI options - no license key or payment is needed!
Then set up the Ollama API host - the default setting is http://127.0.0.1:11434, which should work right out of the box. That's it! Just pick the model and hit save. Now you're all set and ready to chat with your locally running Deepseek R1!
R1-7B was fast but relatively incompetent - the results came quick but were virtually worthless, with some pretty easy to see mistakes.
The R1-8B model took in many cases 5-10 minutes just to think through the answer, before even generating a response. It wasn't terrible - and the response was verifiably better/more accurate,
My config:
Ryzen 5700x3d with 32 GB Ram and Arc A770
