I've been trying out different AI models locally to see how they fare in helping me code.

When it comes to testing these models, these were all used through the Continue extension on Visual Studio Code. I utilized the below models for coding help with React (JavaScript), Django (Python), and occasionally Bash scripting.

If you'd like to see how I set up AI models locally, and configured the Continue extension on VS Code, visit the Part 1 page for instructions.

Best Models for Coding

Of the ones that I tried, I found the best to be codellama:7b, deepseek-coder, and qwen2.5-coder:3b.

Codellama

The 7b version is the lightest that there is for Codellama, so right off the bat, it's already going to be slower than the other models.

While it was a lot slower, it was still helpful having this model handy for more complex coding tasks. I felt this model was best at understanding what I was trying to get at; with the least amount of explanation. I also felt this model gave the most accurate code to what I was prompting it.

As mentioned though, this is a pretty heavy model. Even with a 3080TI, responses were pretty slow. If I prompted for a larger bit of code, or a longer explanation, completing the response would take some time.

While it's the more accurate model of the ones that we'll go through, I use this one the least because of how slow responses can be.

Qwen Coder

The qwen2.5-coder:3b model is the one that I ended up using the most. I really like this model as it's right in the middle ground. It's gets the job done, helps me with coding questions, gives mostly accurate answers, and is lightweight.

Be sure to check out the page linked for Qwen Coder. There are even heavier and lighter models that you can use depending on your machine's specs. For my setup, I found that the 3b model was perfect. Responses were generally good, and they were very fast.

The one thing I did notice with this model is that it seems, just in my experience, that prompts need to be pretty clear; instructions need to specify exactly what you're going for.

DeepSeek Coder

I don't really use deepseek-coder much anymore. I still put it here though because it was able to understand instructions very well, but there was one major flaw. The code that it would give in the Continue extension on VS Code was not formatted correctly at all. I'm not sure if this is just an issue for me, or if this is an issue with the Continue extension, but I couldn't make much use of the code that it was supplying.

It was a real shame that I couldn't get this to work properly because this was by far the fastest model, and it seemed to analyze context given, and understand instructions very well. I would definitely encourage trying this model if you get the chance. Please let me know in the comments if you're facing similar issues.