Open-source ChatGPT Alternative | Jan
jan.ai
Jan runs 100% offline on your computer, utilizes open-source AI models, prioritizes privacy, and is highly customizable.
  • @Comprehensive49@lemmygrad.ml
    7•9 months ago

    This isn’t really a significant player in the space yet. This field of local LLM frontends is extremely active right now, with many new projects popping up and user counts shifting around. Here’s a good article that reviews the most prominent ones right now, with rankings for different uses: https://matilabs.ai/2024/02/07/run-llms-locally/

    TLDR from the article:

    Having tried all of these tools, I find they are trying to solve for a few different problems. So, depending on what you are looking to do, here are my conclusions:

    • If you are looking to develop an AI application, and you have a Mac or Linux machine, Ollama is great because it’s very easy to set up, easy to work with, and fast.
    • If you are looking to chat locally with documents, GPT4All is the best out of the box solution that is also easy to set up
    • If you are looking for advanced control and insight into neural networks and machine learning, as well as the widest range of model support, you should try transformers
    • In terms of speed, I think Ollama or llama.cpp are both very fast
    • If you are looking to work with a CLI tool, llm is clean and easy to set up
    • If you want to use Google Cloud, you should look into localllm
    • For native support for roleplay and gaming (adding characters, persistent stories), the best choices are going to be textgen-webui by Oobabooga, and koboldcpp. Alternatively, you can use ollama with custom UIs such as ollama-webui
    • ☆ Yσɠƚԋσʂ ☆OP
      5•9 months ago

      I’ve been using GPT4All locally, and that does work pretty well. It’s worth noting that a lot of these will work with the same models. So, the difference is mostly in the UI ergonomics.

      • @Comprehensive49@lemmygrad.ml
        4•9 months ago

        Good point. I’ve been dabbling with Khoj myself, which connects with Obsidian, the Markdown notemaking software I use. This lets the AI draw from my notes on the fly.

        I’m waiting to build a PC with the RTX 5090 or something before diving deeper into local AI. The integrated GPU I’m using right now is just too slow.