Build your own Siri locally and on-device

https://thehyperplane.substack.com/p/build-your-own-siri-locally-on-device

124•andreeamiclaus•7h ago

Comments

caust1c•5h ago

So build your own crappy agent-assistant?

In earnest though, I'm certain we'll see a community replacement of Siri by end-of-year if the iPhone permissions model allows it or there's some workaround. IDK what the limitations are here but I'm eagerly awaiting the community to step in where Siri has failed.

Ancapistani•5h ago

The assistant is only half the story here. This looks like a great, well-defined tutorial project to learn how to put this stuff together locally.

andrewmcwatters•5h ago

Crappy? Dude, Siri at one point couldn't even tell you what today's date was. The bar is on the ground.

4ndrewl•4h ago

Think Different

0cf8612b2e1e•4h ago

I do not know if it is because I have been trained to make simple requests, but there are only a half dozen things I would verbally ask a robot.

- time of day

- calendar date

- weather

- set a timer

- simple math calculation

That’s 90% of the functionality right there.

andrewmcwatters•3h ago

Funny you mention this. My usage is the same. I suspect we’ve all been trained to expect little-to-nothing from these assistants.

jihadjihad•2h ago

And incredibly it manages to get those wrong a non-trivial amount of the time.

dang•4h ago

> So build your own crappy agent-assistant?

"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." - https://news.ycombinator.com/newsguidelines.html

(Your comment would be fine without that first bit.)

thedeep_mind•5h ago

This is great, thanks for putting this together.

Haven't followed it through yet, but does this model run successfully on an iPhone?

My 9 year old ran a Qwen 0.6B model using ollama quite well, anything else was too slow to offer a good UX.

SparkyMcUnicorn•2h ago

MLC[0] indicates that it can run models in the 8B range on iOS, but 1-3B sounds more reasonable to me.

[0] https://llm.mlc.ai/docs/deploy/ios.html#bring-your-own-model

parpfish•2h ago

Oh, a nine year old PHONE.

I was thinking there was a fourth grader out there deploying models when at that age I was still learning multiplication tables.

NetOpWibby•1h ago

My son just turned 9 today so I was like, "Wow! I wonder if my kid would be interested in doing this?"

mrcwinn•4h ago

Cool project and nice write-up!

mystified5016•4h ago

Does Apple even allow you to replace Siri with another assistant? For the longest time on android, all non-Google assistants were crippled by not being able to listen in the background or use the assistant hardkey, gestures, or shortcuts. I'm not sure if the Google assistant still has privileges others don't, but I wouldn't be surprised in the least.

jedisct1•3h ago

More or less. This is what Perplexity does.

bronco21016•1h ago

I saw an article about this and downloaded the Perplexity app but I was unable to figure out if this was true? Do I need a paid tier? I just quickly worked through the free sign up and couldn't sort it out. The demo looked really slick. Is it worth pursuing?

matthewfcarlson•2h ago

Part of the problem is the wake word “hey siri” is actually handed by a separate coprocessor (AOP) with the model compiled into the firmware. While anything is technically possible, it isn’t as simple as just letting the google app run in the background since the AP is asleep when any of these gesture happen. You could probably setup the action button on the side to open an assistant, but that’s going to be a less pleasant experience (app might not be open, etc).

Details are listed below

https://machinelearning.apple.com/research/hey-siri

kimixa•1h ago

Same with android phones - a super-specific hardcoded phrase is much easier to work in the power budgets required for an "always on" part of the device.

It's why a manufacturer (like Samsung) can change that sort of thing on their devices, but it's not realistically something an end user (or even an app) can customize in software. It's not some "arbitrary" limitation.

layer8•1h ago

I think people would be fine with having to call it Siri if only they could replace the actual assistant.

dangus•7m ago

I presume you could pretty easily use new-ish action button to run a custom shortcut that brings up an alternative assistant app.

catapart•4h ago

Man, I'd really love it if this were just a product/app I could download and use a UI to configure/teach.

But this guide gives me what I need to make that, I think, so a big thank you for this!

worldsayshi•3h ago

I love the idea and I would like to build something like this. But the few attempts i have made using whisper locally has so far been underwhelming. Has anyone gotten results with small whisper models that are good enough for a use case like this?

Maybe I've just had a bad microphone.

jtr1•2h ago

I’ve noticed recently (maybe I missed an announcement) that Siri now functions locally for at least some commands. Try putting an Apple watch in airplane mode and asking it to set a timer or reminder

cadamsdotcom•1h ago

Why haven’t Apple taken a look at the data then hardcoded handlers for the top ~1000 usages???

FastVLM: Dramatically Faster Vision Language Model from Apple

Air Traffic Control

A conversation about AI for science with Jason Pruet

Understanding LucasArts' iMUSE System

The Barbican

How to avoid P hacking

FedRAMP 20x – One Month in and Moving Fast

Can you trust that permission pop-up on macOS?

HealthBench – An evaluation for AI systems and human health

RIP Usenix ATC

Wtfis: Passive hostname, domain and IP lookup tool for non-robots

Launch HN: ParaQuery (YC X25) – GPU Accelerated Spark/SQL

Build your own Siri locally and on-device

A community-led fork of Organic Maps

University of Texas-led team solves a big problem for fusion energy

Reviving a modular cargo bike design from the 1930s

Ruby 3.5 Feature: Namespace on read

NASA Study Reveals Venus Crust Surprise

Writing N-body gravity simulations code in Python

Show HN: Lumoar – Free SOC 2 tool for SaaS startups

The Beam

Policy of Transience

Continuous glucose monitors reveal variable glucose responses to the same meals

Demonstrably Secure Software Supply Chains with Nix

Legion Health (YC S21) is hiring engineers to help fix mental health with AI

Universe expected to decay in 10⁷⁸ years, much sooner than previously thought

The Acid King (2001)

Why GADTs matter for performance (2015)

Show HN: Airweave – Let agents search any app

Has anyone coined the term “fast tech” yet?

Build your own Siri locally and on-device

Comments

FastVLM: Dramatically Faster Vision Language Model from Apple

Air Traffic Control

A conversation about AI for science with Jason Pruet

Understanding LucasArts' iMUSE System

The Barbican

How to avoid P hacking

FedRAMP 20x – One Month in and Moving Fast

Can you trust that permission pop-up on macOS?

HealthBench – An evaluation for AI systems and human health

RIP Usenix ATC

Wtfis: Passive hostname, domain and IP lookup tool for non-robots

Launch HN: ParaQuery (YC X25) – GPU Accelerated Spark/SQL

Build your own Siri locally and on-device

A community-led fork of Organic Maps

University of Texas-led team solves a big problem for fusion energy

Reviving a modular cargo bike design from the 1930s

Ruby 3.5 Feature: Namespace on read

NASA Study Reveals Venus Crust Surprise

Writing N-body gravity simulations code in Python

Show HN: Lumoar – Free SOC 2 tool for SaaS startups

The Beam

Policy of Transience

Continuous glucose monitors reveal variable glucose responses to the same meals

Demonstrably Secure Software Supply Chains with Nix

Legion Health (YC S21) is hiring engineers to help fix mental health with AI

Universe expected to decay in 10⁷⁸ years, much sooner than previously thought

The Acid King (2001)

Why GADTs matter for performance (2015)

Show HN: Airweave – Let agents search any app

Has anyone coined the term “fast tech” yet?