frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Launch HN: Freestyle: Sandboxes for AI Coding Agents

https://www.freestyle.sh
73•benswerd•1h ago
We’re Ben and Jacob, cofounders of Freestyle (https://freestyle.sh). We’re building a cloud for Coding Agents.

For the first generation of agents it looked like workflows with minimal tools. 2 years ago we published a package to let AI work in SQL, at that time GPT-4 could write simple scripts. Soon after the first AI App Builders started using AI to make whole websites; we supported that with a serverless deploy system.

But the current generation is going much further, instead of minimal tools and basic serverless apps AI can utilize the full power of a computer (“sandbox”). We’re building sandboxes that are interchangeable with EC2s from your agents perspective, with bonus features:

1. We’ve figured out how to fork a sandbox horizontally without more than a 400ms pause in it. That's not forking the filesystem, we mean forking the whole memory of it. If you’re half way down a browser page with animations running, they’ll be in the same place in all the forks. If you’re running a minecraft server every block and player will be in the same place on the forks. If you’re running a local environment and an error comes up in process that error will be there in all the forks. This works for snapshotting as well, you can save your place and come back weeks later.

2. Our sandboxes start in ~500ms.

Demo: https://www.loom.com/share/8b3d294d515442f296aecde1f42f5524

Compared with other sandboxes, our goal is to be the most powerful. We support full Linux + hardware-virtualization, eBPF, Fuse, etc. We run full Debian with multiple users and we use a systemd init instead of runc. Whatever your AI expects to work on debian should work on these vms, and if it doesn’t send a bug report.

In order to make this possible, we’ve moved to our own bare metal racks. Early in our testing we realized that moving VMs across cloud nodes would not have acceptable performance properties. We asked Google Cloud and AWS for a quote on their bare metal nodes and found that the monthly cost was equivalent to the total cost of the hardware so we did that.

Our goal is to build the necessary infrastructure to replicate the human devloop on the massively multi-tenant scale of AI, so these VMs should be as powerful as the ones you’re used to, while also being available to provision in seconds.

Comments

n2d4•1h ago
Cool! I've been using your API for running sandboxed JS. Nice to see you also support VMs now.

    > we mean forking the whole memory of it
How does this work? Are you copying the entire snapshot, or is this something fancy like copy-on-write memory? If it's the former, doesn't the fork time depend on the size of the machine?
benswerd•1h ago
We're using copy on write with the memory itself. Fork time is completely decoupled from the size of the machine.

Creating snapshots takes a 2-4 second interruption in the VM due to sheer IO that we didn't want here.

Whats especially cool about this approach is not only is fork time O(1) with respect to machine size, but its also O(1) with respect to the amount of forks.

_jayhack_•1h ago
Would love to understand how you compare to other providers like Modal, Daytona, Blaxel, E2B and Vercel. I think most other agent builders will have the same question. Can you provide a feature/performance comparison matrix to make this easier?
benswerd•1h ago
I'm working on an article deep diving into the differences between all of us. I think the goal of Freestyle is to be the most powerful and most EC2 like of the bunch.

Daytona runs on Sysbox (https://github.com/nestybox/sysbox) which is VM-like but when you run low level things it has issues.

Modal is the only provider with GPU support.

I haven't played around with Blaxel personally yet.

E2B/Vercel are both great hardware virtualized "sandboxes"

Freestyle VMS are built based on the feedback our users gave us that things they expected to be able to do on existing sandboxes didn't work. A good example here is Freestyle is the only provider of the above (haven't tested blaxel) that gives users access to the boot disk, or the ability to reboot a VM.

tomComb•1h ago
And fly.io sprites
benswerd•1h ago
Fly.io sprites is the most similar to us of the bunch. They do hardware virtualization as well, have comparable start times and are full Linux. What we call snapshots they call checkpoints.

The big pros of Sprites over us is their advanced networking stack and the Fly.io ecosystem. The big cons are that Sprites are incredibly bare bones — they don't have any templating utilities. I've also heard that Sprites sometimes become unavailable for extended periods of time.

The big pros of Freestyle over Sprites is fork, advanced templating, and IMO a better debugging experience because of our structure.

knowsuchagency•55m ago
Thanks for the thoughtful response. I'm predominantly a self-hoster, but I think your product makes a lot of sense for a wide variety of users and businesses. I'm excited to try out freestyle!
benswerd•49m ago
Self hosting can be doable for constant small/medium size workloads

You can handroll a lot with: https://github.com/nestybox/sysbox?tab=readme-ov-file https://gvisor.dev https://github.com/containers/bubblewrap?tab=readme-ov-file

For hardware virtualized machines it much harder but you can do it via: https://github.com/firecracker-microvm/firecracker/ https://github.com/cloud-hypervisor/cloud-hypervisor

Freestyle/other providers will likely provide better debugging experience but thats something you can probably get past for a lot of workloads.

The time when you/anyone should think about Freestyle/anyone is when the load spikes/the need to create hundreds of VMs in short spikes shows up, or when you're looking for some of the more complex feature sets any given provider has built out (forks, GPUs, network boundaries, etc).

I also highly recommend self hosting anything you do outside of your normal VPC. Sandboxes are the biggest possible attack surface and it is a feature of us that we're not in your cloud; If we mess up security your app is still fine.

indigodaddy•43m ago
This is what I do (my project) for self hosting on a VPS/server:

https://GitHub.com/jgbrwn/vibebin

Also I'm a huge proponent of exe.dev

Obviously your service/approach is different than exe, more like sprites but like you said more targeted/opinionated to AI coding/sandboxing tasks it looks like. Interesting space for sure!

rsyring•49m ago
I'd also be interested in a comparison with exe.dev which I'm currently using.
benswerd•8m ago
Exe.dev is a individual developer oriented service. Freestyle is more oriented at platforms building the next exe.dev.

Thats why our pricing is usage based and we have a much larger API surface.

Fraaaank•1h ago
Your pricing page is broken
benswerd•1h ago
Reviewing this now. our public pricing at www.freestyle.sh/pricing seems to be working, can you point me in a more specific direction?
MarcelinoGMX3C•42m ago
The technical challenges in getting memory forking to deliver those sub-second start and fork times are significant. I've seen the pain of trying to achieve that level of state transfer and rapid provisioning. While "EC2-like" gets the point across for many, going bare metal reveals the practical limits of cloud virtualization for high-performance, complex workloads like these. It shows a real understanding of where cloud abstraction helps and where it just adds overhead.

The cost argument for owning the hardware for this specific use case also makes sense, considering the scale these agent environments will demand. Also worth noting, sandboxes are effectively an open attack surface; architecting them not to be in your main VPC is a sound security decision from the start.

skybrian•42m ago
It doesn't seem very easy to calculate how much it would cost per month to keep a mostly-idle VM running (for example, with a personal web app). The $20/month plan from exe.dev seems more hobbyist-friendly for that. Maybe that's not the intended use, though?
benswerd•30m ago
We're not going after hobbyists. We're building the platform for companies like exe.dev to build on. Thats why its all usage based.

That said, our $50 a month plan can be used as an individual for your coding agents, but I wouldn't recommend it.

indigodaddy•11m ago
Ooof, if you are the middleman platform then it's sure gonna get expensive for the end user
siva7•40m ago
I have so many interesting problems on Ai, sandboxing isn't one of them. It's a pointless excercise yet disproportionately so many people love to to do this. Probably because sandboxing doesn't feel as magic as Agents itself and more like the old times of "traditional" software development.
iterateoften•32m ago
Yeah, idk I guess it’s interesting if you are an engineer looking for something to do,

But like I see multiple sandbox for agents products a week. Way too saturated of a market

benswerd•27m ago
I disagree (as a sandboxing company).

With respect to the market, every single sandbox sucks. I'm not gonna shit talk competitors but there is not a good sandboxing platform out there yet — including me — compared to where we'll be in 6 months.

We've heard all the platforms have consistent uptime, feature completeness, networking and debugging issues. And in our own platform we're not 1/10ths of the way through solving the requests we've gotten.

Next generation of Agents needs computers, and those computers are gonna look really different than "sandboxes" do today.

tcdent•21m ago
I don't think you're wrong, but if you really want to really re-think the approach, building an orchestration layer for Firecracker like every other company in the space is doing is probably not it.
hobofan•25m ago
It is a mostly pointless exercise if the goal is trying to contain negative impact of AI agents (e.g. OpenClaw).

It is a very necessary building block for many common features that can be steered in a more deterministic way, e.g. "code interpreter" feature for data analysis or file creation like commonly seen in chat web UIs.

moezd•13m ago
Believe it or not, once you start working for a regulated industry, it is all you would ever think of. There, people don't care if you are vibing with the latest libraries and harnesses or if it's magic, they care that the entire deployment is in some equivalent of a Faraday cage. Plus, many people just don't appreciate it when their agents go rm -rf / on them.
rasengan•31m ago
Interesting!

We're working on a similar solution at UnixShells.com [1]. We built a VMM that forks, and boots, in < 20ms and is live, serving customers! We have a lot of great tools available, via MIT, on our github repo [2] as well!

[1] https://unixshells.com

[2] https://github.com/unixshells

stingraycharles•25m ago
I’m super interested since it seems like you have given everything a lot of thought and effort but I am not sure I understand it.

When I’m thinking of sandboxes, I’m thinking of isolated execution environments.

What does forking sandboxes bring me? What do your sandboxes in general bring me?

Please take this in the best possible way: I’m missing a use case example that’s not abstract and/or small. What’s the end goal here(

benswerd•18m ago
So isolation is correct. Forking a sandbox gives you multiple exact duplicates of isolated environments.

When your coding agent has 10 ideas for what to do, to evaluate them correctly it needs to be able to evaluate them in isolation.

If you're building a website testing agent and halfway down a website, with a form half filled out a session ongoing, etc and it realizes it wants to test 2 things in isolation, forking is the only way.

We also envision this powering the next generation of devcycles "AI Agent, go try these 10 things and tell me which works best". AI forks the environment 10 times, gets 10 exact copies, does the thing in each of them, evaluates it, then takes the best option.

indigodaddy•13m ago
Yep I can see this especially when the agent is spinning up test servers/smokes and you don't want those conflicting. How do we reconcile all the potential different git hashes though, upstream I guess etc (this might be an easy answer and I'm not super proficient with git so forgive)
benswerd•11m ago
So we recommend branch per fork, merge what you like.

You have to change the branch on each fork individually currently and thats unlikely to change in the short term due to the complexity of git internals, but its not that hard to do yourself `git checkout -b fork-{whateverDiscriminator}`

wsve•12m ago
Agreed, the thing I'd be most interested in is the isolated execution environment you mentioned. Agents running autopilot are powerful. Agents running unsupervised on a machine with developer permissions and certificates where anything could influence the agent to act on an attacker's behalf is terrifying
benswerd•10m ago
I recommend running the agent harness outside of the computer. The mental model I like to use is the computer is a tool the agent is using, and anything in the computer is untrusted.
jeremyjh•3m ago
I would recommend not giving an agent the full run of any computing environment. Do handle fine grained internet access controls and credential injection like OpenShell does?
croes•1m ago
The problem is the agent, which should be treated untrusted. The computer isn’t the problem
stocktech•20m ago
I built something like this at work using plain Docker images. Can you help me understand your value prop a little better?

The memory forking seems like a cool technical achievement, but I don't understand how it benefits me as a user. If I'm delegating the whole thing to the AI anyway, I care more about deterministic builds so that the AI can tackle the problem.

benswerd•15m ago
So first MicroVM != Container, and container is not a secure isolation system. I would not run untrusted containers on your nodes without extra hardening.

The memory forking was originally invented because for AI App Builders and first response driven applications its extremely important that they are instant (difference between running bun dev and the dev server already being running).

However its much more generally applicable, Postgres is a great example of this. You can't fork the filesystem under postgres and get consistency. Same thing with a browser state, a weird server state, or anything that exists in memory. The memory forking gives a huge performance boost while snapshotting whats actually going on at one instant.

jnstrdm05•15m ago
how many seconds to provision are we talking about here? 1 sec vs 60 is a dealbreaker for me, some clarity on that would be nice.
benswerd•13m ago
500ms. Less than 1 second. We're aiming to get that down to 200ms in the next 3 months.
maxmaio•1m ago
Congrats Ben and Jacob!