System: Control your Mac from anywhere using natural language

39•handfuloflight•1mo ago

Comments

danw1979•1mo ago

Anything where an LLM has access to the shell on my personal laptop isn’t “secure” even if you do some hand waving around a split architecture.

rvz•1mo ago

Nothing more like playing Russian roulette with your own machine, hoping that when it needs to delete, move or rename a file, you sure hope it doesn't begin running either (especially do not run with --dangerously-skip-permissions):

rm -rf ~/

rm -rf /

mv ~/ ./

Or even creating a faulty symbolic link and above your home directory and suggesting to remove it, then it actually removes everything in that folder including your files in the home folder.

wizardforhire•1mo ago

Or it decides to rsync your drive along with keys to who knows where

yolo3000•1mo ago

This gives me bad memories from when I switched from Windows and somehow created a folder named '~' and then deleted it.

likium•1mo ago

The tool explicitly has an allow list for commands and block list for sensitive paths[1].

Though it's not enough, stuff like `grep "" .e?v` can still end up sending your sensitive keys to LLM providers.

[1]: https://github.com/ygwyg/system/blob/b5adfe526da7470cade61f7...

fragmede•1mo ago

Of course, only an idiot would do that, and have it go and delete a bunch of shit. I, unfortunately, am that idiot, so if you're also going to be such an idiot, I recommend putting rm behind any sort of wall, no matter how short it should be, eg https://gist.github.com/fragmede/96f35225c29cf8790f10b1668b8...

lostmsu•1mo ago

Give it a separate user or a VM.

iAMkenough•1mo ago

Don't forget to manage proper permissions for every piece of data your own user account would need acccess to complete the objectives of the agent.

Like in the "Play some jazz music" example in the documentation, don't forget to login to the Music app using your own Apple ID on the new user or VM you created.

fragmede•1mo ago

specifically, tart lets you run macOS on macOS fairly well.

danw1979•1mo ago

I’m not sure a user account would contain the potential damage enough for my liking, and isn’t the point of this to enable it to “act like you” on your personal machine ? (which I maintain is a terrible idea).

lostmsu•1mo ago

There are plenty of services we don't really need strong security boundaries around and any issues are non-critical.

danw1979•1mo ago

I honestly don’t know if you’re being sincere or sarcastic.

lostmsu•1mo ago

Do you care if your Netflix subscription gets weird views or favorites random movies?

voidUpdate•1mo ago

Is the LLM guaranteed not to have an existential crisis and delete large amounts of data?

colechristensen•1mo ago

an LLM once had a crisis and decided deleting large pieces of Godot on my laptop was a good idea, they're absolutely not to be trusted

low_tech_punk•1mo ago

User: Hey AI, turn off my Mac

AI: (after 10 minutes of deep reasoning) Ah! I found the solution. I need to cause a power surge in the grid to power off your device.

peaseagee•1mo ago

Regardless of whether this is a good idea or not (and I'm in the latter camp to be honest), the docs page is _way_ too low contrast to be readable. I'm all for dark mode but I shouldn't have to be in a completely darkened room to be able to read the fine manual!

ssivark•1mo ago

Completely ignoring what it does, the naming is obnoxious.

Also, good luck to any user who tries to google for help.

michaelbuckbee•1mo ago

This seems very cool, but I'm not sure of the utility of using CF workers for it? Is that just a framework or is this being deployed to the internet?

juancn•1mo ago

Isn't this a gimmick? (I mean, it's still impressive)

I don't want natural language, way too ambiguous and too much typing (or worse, talking).

90% of the things I do repeat frequently. Brevity is key for me.

I like formal syntaxes with well defined semantics.

hmokiguess•1mo ago

What got you to build this? It seems ok but I can’t think of when or why I would want this, feels like something iOS has natively through Siri and it syncs to Mac via iCloud

epaga•1mo ago

(Not the OP, but...) have...have you tried Siri before? It is completely and totally ridiculous. Completely and utterly useless for anything other than setting a timer and turning HomeKit devices on or off (and even there it's entirely hit or miss).

Siri continues to be the most embarrassing Apple product on the market by a long shot.

hmokiguess•1mo ago

I get your point, that said, you can make it useful it just requires ... some plumbing. Shortcuts and Automations would be your best friends there to achieve this. The OP's project mentions Raycast as a dependency, with Raycast on iOS now + shortcuts you go even further.

mbirth•1mo ago

At least Siri guarantees the privacy of my data and uses a local model directly on my device, alternatively a private cloud - instead of a globally shared one.

And you can still ask Siri to ask ChatGPT if you need someone to talk to.

Also, you’d be surprised what cool things you can do when using Shortcuts with Siri. Especially now that Shortcuts can make use of LLMs (“Use model” action).

nancarrow•1mo ago

love how clearly vibecoded this is. the cloudflare worker architecture + the ascii diagram is a dead giveaway. nothing wrong with that, it's just really obvious.

the split architecture offers absolutely zero security benefits outside of not exposing a server process on your mac to the open internet (assuming you only let it connect to cloudflare) - it's just a convenient place to spin up a thin JS layer that calls model APIs and connects to your mac.

anyways i think this is a neat weekend vibecoding project but IMO it needs a lot more design thought to really be useful and not be a huge security issue.

boobsbr•1mo ago

How is it a giveaway? I'd like to learn how to spot these things.

fragmede•1mo ago

Why does it matter? If it works, it works, no? Or are we now artisanal hipsters, where the code is better if it was hand-typed on a really shitty keyboard where the N key sticks, and it just feels better using the software because of how much pain the coder in the forests outside of Portland experienced while writing the code. Do we need an international fair trade organization to make sure the code was ethically sourced?

netsharc•1mo ago

Seems like it needs higher level stuff, although that's a bit too sci-fi. Captain Picard can just ask "Computer, where is Commander Riker?" and the computer answers him, he doesn't need to say "Computer, launch people finder app" and "Computer, input 'Commander Riker' in the people finder app"...

I use Google Assistant for things like "add a reminder", "set an alarm", which is natural language processing but doesn't seem to need so many neurons as LLM. And faster than this Gemini crap, anyway.

I saw a social media clip of a woman in the passenger car of a Chinese car (her - presumably husband - is driving) asking the car "Has there been a woman in this car other than me?". The car seems to have an LLM app, because it responds saying "I can't see that", and then start giving tips how to find out (check the recent addresses list in th navigation, check the trips log if there has been long trips, see if the car is cleaner than he usually maintains it), and ending with talking about trust and communication in a relationship...

Hah, in our imagination we'd get KITT from Knight Rider. In reality...

frizlab•1mo ago

Funny you should mention that, I literally today asked Siri on my HomePod: “Siri where’s my wife?” It worked (once).

netsharc•1mo ago

I have an alarm clock with Google Assistant, I can ask it where my Pixel 7 is, and it will make that phone ring using whatever Google's equivalent of Find My Phone is.

simonw•1mo ago

The tools.ts is the best file to read to understand what this thing can do - it's mainly shell and AppleScript stuff: https://github.com/ygwyg/system/blob/main/src/bridge/tools.t...

I don't trust this list of "safe commands": https://github.com/ygwyg/system/blob/b5adfe526da7470cade61f7...

It includes python3 and node under the heading "Dev tools (read-only)" - but I don't think "read-only" is enforced anywhere.

vivzkestrel•1mo ago

- this requires you to keep your mac turned on when you are 1000 kms away from home

- i am still waiting for an agent that turns your mac automatically on when you are away and then lets you control the machine.

- perhaps a robot arm operated by cloudflare agents with opencv camera to detect where your switch is present and where the plug is present remote controlled would do the job?

- it can also unplug the device when you are done using it

codenlearn•1mo ago

This is exactly what I was working on last week. You have beaten me to it. Amazing work now time for me to find a new project.

Tiny C Compiler

SectorC: A C Compiler in 512 bytes

The F Word

Speed up responses with fast mode

Brookhaven Lab's RHIC concludes 25-year run with final collisions

Software factories and the agentic moment

Hoot: Scheme on WebAssembly

OpenCiv3: Open-source, cross-platform reimagining of Civilization III

Stories from 25 Years of Software Development

Show HN: Browser based state machine simulator and visualizer

FDA intends to take action against non-FDA-approved GLP-1 drugs

You Are Here

Show HN: A luma dependent chroma compression algorithm (image compression)

First Proof

LLMs as the new high level language

Al Lowe on model trains, funny deaths and working with Disney

Vocal Guide – belt sing without killing yourself

I write games in C (yes, C) (2016)

Start all of your commands with a comma (2009)

Show HN: I saw this cool navigation reveal, so I made a simple HTML+CSS version

Washington Post CEO Will Lewis Steps Down After Stormy Tenure

Reinforcement Learning from Human Feedback

Selection rather than prediction

The AI boom is causing shortages everywhere else

Microsoft account bugs locked me out of Notepad – Are thin clients ruining PCs?

72M Points of Interest

Coding agents have replaced every framework I used

Unseen Footage of Atari Battlezone Arcade Cabinet Production

A Fresh Look at IBM 3270 Information Display System

France's homegrown open source online office suite

Tiny C Compiler

SectorC: A C Compiler in 512 bytes

The F Word

Speed up responses with fast mode

Brookhaven Lab's RHIC concludes 25-year run with final collisions

Software factories and the agentic moment

Hoot: Scheme on WebAssembly

OpenCiv3: Open-source, cross-platform reimagining of Civilization III

Stories from 25 Years of Software Development

Show HN: Browser based state machine simulator and visualizer

FDA intends to take action against non-FDA-approved GLP-1 drugs

You Are Here

Show HN: A luma dependent chroma compression algorithm (image compression)

First Proof

LLMs as the new high level language

Al Lowe on model trains, funny deaths and working with Disney

Vocal Guide – belt sing without killing yourself

I write games in C (yes, C) (2016)

Start all of your commands with a comma (2009)

Show HN: I saw this cool navigation reveal, so I made a simple HTML+CSS version

Washington Post CEO Will Lewis Steps Down After Stormy Tenure

Reinforcement Learning from Human Feedback

Selection rather than prediction

The AI boom is causing shortages everywhere else

Microsoft account bugs locked me out of Notepad – Are thin clients ruining PCs?

72M Points of Interest

Coding agents have replaced every framework I used

Unseen Footage of Atari Battlezone Arcade Cabinet Production

A Fresh Look at IBM 3270 Information Display System

France's homegrown open source online office suite

System: Control your Mac from anywhere using natural language

Comments