Hello, I made a local AI Chat app that is as easy to operate as Spotify.
I know that doesn’t appeal to a lot of you because you’re mostly hardcore programmers, but this is for the people that aren’t as savvy and just have a normal MacBook. It’s running an 80B A3B Qwen model on any M-series Mac. Even on an 8GB Mac it gets 5-9 tok/s, so it’s not a proof of concept, it’s an actual program.
It has its own SQLite memory system. The 8GB version has fewer features than the 16GB and above, but even the 8GB will chat at faster than reading speed with zero internet, never touch a cloud, and synthesize info between (small) documents.
The 16GB or more version is actually pretty robust—you can upload as many docs as you want and it will cross-reference all the info.
I know not a ton of you will have use for it as a daily program, but I’m thinking about doctors' offices and lawyers' offices that can’t upload their files to the cloud. This would get them ChatGPT-level AI for free. Right now companies are having to spend $5000 and a monthly service fee just to get local AI, and this will do it for the price of a used MacBook Air with 8GB.
I really would like brutal feedback (about the actual operation of the program, not the idea) and to have people test it out to see if there are bugs that I'm not catching. Don't roast me on the code, there are some messes in there from failed experiments that I will clean up later. It's Mac Notarized and safe, and any info you want to know is at yanahchat.com
giovanniwants•39m ago
I know that doesn’t appeal to a lot of you because you’re mostly hardcore programmers, but this is for the people that aren’t as savvy and just have a normal MacBook. It’s running an 80B A3B Qwen model on any M-series Mac. Even on an 8GB Mac it gets 5-9 tok/s, so it’s not a proof of concept, it’s an actual program.
It has its own SQLite memory system. The 8GB version has fewer features than the 16GB and above, but even the 8GB will chat at faster than reading speed with zero internet, never touch a cloud, and synthesize info between (small) documents.
The 16GB or more version is actually pretty robust—you can upload as many docs as you want and it will cross-reference all the info.
I know not a ton of you will have use for it as a daily program, but I’m thinking about doctors' offices and lawyers' offices that can’t upload their files to the cloud. This would get them ChatGPT-level AI for free. Right now companies are having to spend $5000 and a monthly service fee just to get local AI, and this will do it for the price of a used MacBook Air with 8GB.
I really would like brutal feedback (about the actual operation of the program, not the idea) and to have people test it out to see if there are bugs that I'm not catching. Don't roast me on the code, there are some messes in there from failed experiments that I will clean up later. It's Mac Notarized and safe, and any info you want to know is at yanahchat.com
Also want any advice on how to get your app noticed and off the ground for any of you that have been at this stage before. Thanks! Github https://github.com/giovanniwants/Yanah/releases/tag/v0.2.1