I’d like to set up a local coding assistant so that I can stop using Google to ask complex questions to for search results.

I really don’t know what I’m doing or if there’s anything that’s available that respects privacy. I don’t necessarily trust search results for this kind of query either.

I want to run it on my desktop, Ryzen 7 5800xt + Radeon RX 6950xt + 32gb of RAM. I don’t need or expect data center performance out of this thing. I’m also a strict Sublime user so I’d like to avoid VS Code suggestions as much as possible.

My coding laptop is an oooooold MacBook Air so I’d like something that can be ran on my desktop and used from my laptop if possible. No remote access needed, just to use from the same home network.

Something like LM Studio and Qwen sounds like it’s what I’m looking for, but since I’m unfamiliar with what exists I figured I would ask for Lemmy’s opinion.

Is LM Studio + Qwen a good combo for my needs? Are there alternatives?

I’m on Lemmy Connect and can’t see comments from other instances when I’m logged in, but to whomever melted down from this question your relief is in my very first sentence:

to ask complex questions to for search results.

  • Coolcoder360@lemmy.worldEnglish
    121·
    28 days ago

    I’ve not found them useful yet for more than basic things. I tried Ollama, it let’s you run locally, has simple setup, stays out of the way.

  • perry@aussie.zoneEnglish
    5·
    27 days ago

    Qwen coder model from Huggingface, following the instructions there to run it in llama.cpp. Once that’s up: OpenCode and use the custom OpenAI API to connect it.

    You’ll get far better results than trying to use other local options out of the box.

    There may be better models potentially but I’ve found Qwen 2.5 etc to be pretty fantastic overall, and definitely a fine option beside Claude/ChatGPT/Gemini. I’ve tested the lot and it’s usually far more down to instruction and AGENTS.md instructions/layout than it is down to just the model.

    • 70k32@sh.itjust.worksEnglish
      1·
      27 days ago

      This. Llama.cpp with Vulkan backend running in docker-compose, some Qwen3-Coder quantization from huggingface and pointing Opencode to that local setup with a OpenAI-compatible is working great for me.

  • TomAwezome@lemmy.worldEnglish
    32·
    28 days ago

    I get good mileage out of the Jan client and Void editor, various models will work but Jan-4B tends to do OK, maybe a Meta-Llama model could do alright too. The Jan client has settings where you can start up a local OpenAI-compatible server, and Void can be configured to point to that localhost URL+port and specific models. If you want to go the extra mile for privacy and you’re on a Linux distro, install firejail from your package manager and run both Void and Jan inside the same namespace with outside networking disabled so it only can talk on localhost. E.g.: firejail --noprofile --net=none --name=nameGoesHere Jan and firejail --noprofile --net=none --join=nameGoesHere void, where one of them sets up the namespace (–name=) and the other one joins the namespace (–join=)

    • IMALlama@lemmy.worldEnglish
      112·
      28 days ago

      Straight up vibe coding is a horrible idea, but I’ll happily take tools to reduce mundane tasks.

      The project I’m currently working on leans on Temporal for durable execution. We define the activities and workflows in protobufs and utilize codegen for all the boring boiler plate stuff. The project hasa number of http endpoints that are again defined in protos, along with their inputs and outputs. Again, lots of code gen. Is code gen making me less creative or degrading my skills? I don’t think so. It sure makes the output more consistent and reduces the opportunity for errors.

      If I engage gen AI during development, which isn’t very often, my prompts are very targeted and the scope is narrow. However, I’ve found that gen AI is great for writing and modifying tests and with a little prompting you can get pretty solid unit test coverage for a verity of different scenarios. In the case of the software I write at work the creativity is in the actual code and the unit tests are often pretty repetitive (happy path, bad input 1…n, no result, mock an error at this step, etc). Once you know how to do that there’s no reason not to offload it IMO.

    • MasterBlaster@lemmy.worldEnglish
      31·
      27 days ago

      We can’t ignore this. We need to know how it is done if we want to earn salaries. Reality rarely makes a dent in the corporate herd until years later.

      By then, careers are obliterated.

      There are ways to protect your mind in the meantime.

        • MasterBlaster@lemmy.worldEnglish
          2·
          24 days ago

          Use AI, learn the methods, tools, uses. Then keep working through the process yourself like you do now with AI as a partner who sometimes flakes out.

          Remind yourself of the tenents of critical thinking regularly. Never just accept what AI tells you. seek proof behind the answers. Think through those answers to ensure you understand the logic behind them.

          Fight complacency.