frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Moving beyond fork() + exec()

https://lwn.net/SubscriberLink/1076018/16f01bbbb8e0d1f0/
68•jwilk•1h ago

Comments

ComputerGuru•1h ago
I'm not surprised Chen's patch was rejected; that's an extremely niche usecase not worth supporting. With my shell developer hat on, I agree with the closing "developers would likely welcome a native implementation that isn't (unlike the current implementation) hiding fork() and exec() under the covers".
smj-edison•49m ago
It sounds like they're interested in the concept though, just not that specific implementation.
sanderjd•40m ago
Yeah this seems like a promising discussion.
hparadiz•1h ago
Maybe tangentially related but I always think it's silly that every linux process has the same libgcc_so.so.1 loaded into memory for each process even though the raw binary for the library is exactly the same so you end up with like 800 copies of libgcc_so.so.1 in memory.

I mean maybe this has been optimized for already and I don't know what I'm talking about but maybe someone with more knowledge about the kernel knows? Is this something we simply can't optimize for because of security implications?

201984•58m ago
Shared libraries (and mmapped files in general) are deduplicated; it's nowhere near as bad as you think. The kernel loads a .so into memory once and then maps that memory into every process that mmaps it.

Editing to add: this deduplication is one of the greatest upsides to dynamic linking. Common libs like libgcc and libc only have to exist in memory once and can stay in CPU caches, whereas if they were statically linked into every binary, each binary would have a copy of that library that wouldn't be shared with anything else and you'd waste a lot of memory.

sjmulder•31m ago
Doesn't the loaded code have to be patched for relocations?
ptspts•24m ago
It does, so not 100% is reused. The patched parts are in different sections though, so the entire .text (code) section ends up being reused.
monocasa•21m ago
Not on modern archs that provide decent support for PIE (position independent executables).
t-3•18m ago
Not if it's position-independent.
monocasa•57m ago
Those mappings by default all go to the same shared memory.

Unices have been sharing executable memory between processes longer than there's been mmap for user space to do the same thing themselves. I remember seeing it in the 2BSD kernel for instance.

Sophira•51m ago
I'm guessing that a big part of the problem with moving away from fork() in general is that each new process needs a copy of the parent process' environment anyway, right?
lokar•48m ago
the environment is not that big
dijit•48m ago
I'm a bit naive, but I don't think that's necessarily a requirement.

It might be commonly held convention, and thus, an assumption, in Linux (and, broadly, UNIX) but I don't think it's true inside VAX or even Windows, so I don't think it's a requirement.

Unless I've missed something (which is totally possible, this is not an area of OS design I've spent much time).

sjmulder•32m ago
Even DOS has environment inheritance!
lanstin•19m ago
But also UID, groups, controlling TTY, process group, capabilities, pipes, shared memory, etc. and the file descriptors while maybe not inherently needed are how a lot of Unix plumbing works.
zerobees•41m ago
The LWN article is incorrect in saying that it "must copy the entire process state (including memory) for the child process". There are some kernel structures and page tables that need to be initialized, plus you need a new stack, but it's not nearly as dramatic as implied. Most of the parent's memory is "incorporated by reference", so to speak.

In fact, if you profile it, in the fork() + execve() model, execve() is far more expensive, because not only does it replace the old process with a new one, but it also involves running the dynamic linker, which opens, parses, and mmaps library files.

It still makes sense to get rid of the fork() overhead if you're going to throw away the cloned process state soon thereafter, but if you wanted to make process execution radically faster, rethinking the exec architecture would probably offer more significant gains.

ktpsns•48m ago
There is lots of discussion on this old API here on hacker news, for instance https://news.ycombinator.com/item?id=31739794
lokar•47m ago
This seems unnecessary to me. In the example, the core of git should be a library yo can link so you don't need to run the binary. That would be better in every way.
sanderjd•41m ago
There are lots of reasons to want to spawn fresh processes, which aren't solved by linking a library.
aerzen•27m ago
Spawning processes should not be on the hot path of any program.
1718627440•12m ago
Why? That's a very useful processing primitive.
lokar•17m ago
Sure, but not many times a second
1718627440•6m ago
But when you use a process, you get tons of things for free, the subtask is invoked in parallel, you get isolation and you can control execution for free. Unless you are already writing a multithreaded program or already accept passing objects in memory, using a process is actually easier to write than using a library.

If I use a library, I also need to start using threads and need to invent some core synchronization mechanism. I essentially are reinventing a small scheduler, when I already get this from the OS for free. Also know any crash in the third-party code will crash the whole program, the third-party code has access to the whole address space. With invoking a process you also have a standardized API implemented by the OS.

sanderjd•43m ago
I just ran into this recently, where I had an obscure bug caused by needing to close more file descriptors in the forked process. "I want a clone of the current process" is just way less common in my experience than "I want a completely new process". It feels crazy that we don't have a way to directly express the latter thing, and can only approximate it by cloning and then fixing things up in post.
dnw•40m ago
What do you mean by "a completely new process"?
sanderjd•37m ago
A process that shares nothing with the process that spawned it.
jerf•25m ago
A thing that makes that complicated is that while you want that conceptually, you don't want that in reality. For instance, if the spawning process is in a container of some sort and it spawned a process that "shares nothing with the process that spawned it", the spawned process would no longer be in that container, because the state of "being in the container" is one of the things it shares with the parent process.

This is just an example of I don't even know how many things a modern-day process will share from its parent.

By "complicated" I do not even remotely mean "unsolvable". I just mean that if you really dig down into what it means to "share nothing" in a modern operating system, it's a lot richer than it was back when fork+exec was a practical solution. There's a lot of fuzzy things that could go either way when you say "shares nothing".

JoBrad•6m ago
debatem1•35m ago
There are a lot of slightly different fork-exec-like things in the concept space and it's hard to imagine one approach satisfying them all. IMO it would be interesting to take an approach analogous-ish to sched_ext_ops where you built the rough flow chart of a combined fork-exec, but with hooks built to enable ebpf to change behavior or skip the bits these sophisticated users don't want/need.
uecker•33m ago
The elegance of the fork() + exec() model is that every kind of configuration can be done after the fork using all the usual APIs. Every attempt to replace it with a combined call that I have seen so far seemed fundamentally poorer because it needs to add all configuration options as parameters to the call and then do this in away that you can extend it later and does not become a mess.
amluto•28m ago
I have the entirely opposite opinion. IMO a big mistake of the UNIXy model is that so much state is preserved across the creation of a process. For example, there are APIs to have a specific thing be fd number 4 so you can run a program and have it find that thing at fd 4. This is weird.

Windows, for all its many, many faults, did not use fork+exec and instead mostly has options for how one creates a process. It wasn’t done elegantly, but it was the right decision.

1718627440•15m ago
Is it weirder, that you can pass an variable precisely into argument 4? You do need to pass information to a subprocess and there needs to be some agreement on what means what. Sure, maybe you could use names instead of fds, but that sounds needlessly complicated.
jonhohle•6m ago
That’s like saying you could use positions to specify function argument access (as in assembly) instead of variable names. File descriptors being numbers that are likely array indexes in a file handle seems like a leaky abstraction. Having a namespace that a parent process share with its children seems like a much cleaner design.
burnt-resistor•
mrkeen•21m ago
> fork() is a relatively expensive system call; it must copy the entire process state (including memory) for the child process. Many optimizations have been made over the years, but a fork is still a fundamentally costly operation. To make things worse, a fork() call is often immediately followed by an exec(), which will discard all of that memory that was so carefully copied for the child.

It's weird to leave out a mention of copy-on-write - the optimisation that means that you don't copy over all the memory.

FooBarWidget•11m ago
It says state. Copy on write still means it's O(number of page table entries) even if you don't copy the contents. It's a well known issue that forking a program with large virtual memory size is slow.
tux3•10m ago
This was left implicit in the article, but what they mean by copying the process state here is the memory management structures. That's mainly the page tables and the VMAs.

That means you have to allocate new pages to hold a copy of all these structures, even if the actual memory pointed by the pages is shared. And walking all those structures to make a copy is still costly.

rom1v•19m ago
Related to the discussion: "A fork() in the road": https://www.microsoft.com/en-us/research/wp-content/uploads/...

> ABSTRACT

> The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog the ways in which fork is a terrible abstraction for the modern programmer to use, describe how it compromises OS implementations, and propose alternatives.

> As the designers and implementers of operating systems, we should acknowledge that fork’s continued existence as a first-class OS primitive holds back systems research, and deprecate it. As educators, we should teach fork as a historical artifact, and not the first process creation mechanism students encounter.

burnt-resistor•17m ago
> "If you are repeatedly creating large processes, you are already doing it wrong. The fix is in user space, not the kernel."

Every couple of years, someone claims they have "the solution" implying everyone else who came before them didn't know what they were doing.

saidinesh5•55m ago
Typically libgcc_so.so is loaded by the linker, which uses an mmap call to map the binary into the address space.

> The kernel keeps track of which file is mapped where, and can detect when a request is made to map an already mapped file again, avoiding physical memory allocation if possible.

Relevant stack overflow answer: https://stackoverflow.com/questions/61950951/linux-shared-li...

mlaretallack•54m ago
In Linux, when a shared lib is loaded by multiple processes, its loaded once and not duplicated in ram. Only if a memory page is modified by the process will the memory be duplicated. (Hope I have explained that correctly)
BoingBoomTschak•44m ago
Eh? Aren't shared libraries actually shared in memory?
sirsinsalot•36m ago
I have a rule for myself. If I think something is silly or stupid, I assume I don't understand it. I usually find I do not understand it, and it no longer seems silly when I do understand it.

In this case too, you think it is silly because you don't understand it. Your assumptions are wrong, making it seem silly.

sanderjd•38m ago
A lot of times you actively don't want the parent environment or any of the memory or file descriptors. And then you have to actively do work to fix all that stuff up after the fork.
That’s how you get zombie processes and memory leaks.
wongarsu•23m ago
The equivalent of CreateProcessW https://learn.microsoft.com/en-us/windows/win32/api/processt...
1718627440•18m ago
But you generally want to communicate with that process, so you do need to setup e.g. file descriptors and stuff, which needs information from the parent process to be passed.
jonhohle•11m ago
Most programming languages abstract this out to be able to connect or drop the 3 standard pipes. Typically this is the only thing that can be shared anyway unless the other program is specifically shared and expects other file handles to be available, in which case fork might be the right system call anyway.
9m ago
You're simply failing to grasp the value of the simplicity, compatibility, and portability of POSIX/*nix. Inventing yet another way to create a process would be complex and break things. It's a-la-carte to enable configuration after fork of the new CoW or non-CoW process but before exec (unless vfork or similar were used). This is the model.

If you want to greenfield re-engineer the world with all new system calls and a totally different execution model, feel free to go right ahead.

fanf2•6m ago
[delayed]

Moving beyond fork() + exec()

https://lwn.net/SubscriberLink/1076018/16f01bbbb8e0d1f0/
73•jwilk•1h ago•52 comments

Benchmarks in Leipzig

https://arxiv.org/abs/2606.05818
60•root-parent•2h ago•26 comments

Nvidia is proposing a beast of a CPU system for Windows PCs

https://twitter.com/lemire/status/2062880075117113739
57•tosh•3h ago•108 comments

AI didn't break the web. The dotcons did – AI just turned up the volume

https://hamishcampbell.com/ai-didnt-break-the-web-the-dotcons-did-ai-just-turned-up-the-volume/
7•speckx•38m ago•0 comments

Google will pay SpaceX $920M per month for compute

https://techcrunch.com/2026/06/05/google-will-pay-spacex-920m-per-month-for-compute/
169•ramanan•4h ago•203 comments

How LLMs work

https://www.0xkato.xyz/how-llms-actually-work/
638•0xkato•2d ago•176 comments

Building Rust Procedural Macros from the Grounds Up

https://www.learnix-os.com/ch02-03-implementing-the-bitfields-proc-macro.html
35•Sagi21805•5d ago•2 comments

Pokemon Emerald Ported to WebAssembly (100k FPS)

https://pokeemerald.com/
100•tripplyons•4h ago•27 comments

Tribute to Jiro Yamada, Automotive Artist (1960-2025) [video]

https://www.youtube.com/watch?v=rJ2gQ5Md60U
18•NaOH•19h ago•1 comments

The new bibliomaniacs

https://engelsbergideas.com/notebook/the-new-bibliomaniacs/
36•RickJWagner•4h ago•27 comments

S&P 500 rejects SpaceX, also blocking entry for OpenAI and Anthropic

https://arstechnica.com/tech-policy/2026/06/sp-500-blocks-fast-spacex-entry-wont-waive-rule-for-u...
1001•maltalex•11h ago•367 comments

Mbodi AI (YC P25) Is Hiring Founding Machine Learning Engineer (Robotics)

https://www.ycombinator.com/companies/mbodi-ai/jobs/WYAcNkX-founding-machine-learning-engineer
1•chitianhao•4h ago

The intracies of modern camera lens repair (2024)

https://salvagedcircuitry.com/sigma-45mm.html
213•transistor-man•15h ago•76 comments

US House lawmakers release draft bill to prohibit state AI rules

https://www.reuters.com/business/us-house-lawmakers-release-draft-bill-regulate-ai-2026-06-04/
52•1vuio0pswjnm7•2h ago•17 comments

Police in England and Wales told to halt AI use in court statements

https://www.ft.com/content/229e5949-3ebc-4151-8a86-a01b5e259241
8•nmstoker•33m ago•0 comments

Splash Is a Colour Format

https://www.todepond.com/lab/splash/
4•tobr•2d ago•0 comments

New method turns ocean water into drinking water, without waste

https://www.rochester.edu/newscenter/what-is-desalination-definition-ocean-water-704732/
452•speckx•1d ago•189 comments

Ask HN: What was your "oh shit" moment with GenAI?

448•andrehacker•1d ago•797 comments

Azure Linux Desktop

https://www.boxofcables.dev/azure-linux-desktop-a-build-2026-mashup-of-wslc-winui-reactor-and-azu...
59•haydenbarnes•8h ago•38 comments

Social Cache Busting

https://www.autodidacts.io/social-cache-busting/
92•surprisetalk•4d ago•30 comments

pg_durable: Microsoft open sources in-database durable execution

https://github.com/microsoft/pg_durable
440•coffeemug•1d ago•101 comments

Pre-Modern Armies for Worldbuilders, Part I: Why They Fight

https://acoup.blog/2026/06/05/collections-pre-modern-armies-for-worldbuilders-part-i-why-they-fight/
130•gostsamo•12h ago•41 comments

Astronauts told to return to ISS after sheltering over air leak repairs

https://www.bbc.com/news/live/c4g44ew3g1kt
413•janpot•1d ago•255 comments

The perils of UUID primary keys in SQLite

https://andersmurphy.com/2026/06/05/the-perils-of-uuid-primary-keys-in-sqlite.html
115•emschwartz•16h ago•69 comments

Show HN: Soft Body Jiggle Physics

https://github.com/xloveee/jiggle-physics
16•vesperance•4d ago•7 comments

Running Python code in a sandbox with MicroPython and WASM

https://simonwillison.net/2026/Jun/6/micropython-in-a-sandbox/
5•theanonymousone•1h ago•1 comments

Gemma 4 QAT models: Optimizing compression for mobile and laptop efficiency

https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gem...
376•theanonymousone•23h ago•114 comments

Mouseless – keyboard-driven control of macOS/Linux/Windows

https://mouseless.click
562•riddley•3d ago•230 comments

The Smart TV in Your LivingRoom Is a Node in the AIScraping Economy

https://blog.includesecurity.com/2026/06/the-smart-tv-in-your-livingroom-is-a-node-in-the-aiscrap...
148•nikcub•6h ago•51 comments

The back cover of C++: The Language raises questions not answered by front cover

https://devblogs.microsoft.com/oldnewthing/20260605-01/?p=112391
132•paulmooreparks•12h ago•43 comments