BTW most of applications is totally fine with a UNIX file apis.
But I agree with you; I'd rather use the thing without excess abstraction, and the standard apis work well enough for most applications. Some things do make sense to do the work to increase performance though.
I used 6.0 because 5.8-5.9 is roughly when io_uring became interesting to use for most use cases with zero copies, prepared buffers and other goodies, and 6.0 is roughly when people finally started being able to craft benchmarks where io_uring implementations beat epoll.
[0] https://ziglang.org/documentation/master/std/#std.Io, or https://ziglang.org/documentation/0.16.0/std/#std.Io after the release
They look very similar so that makes sense, just curious on the order of events.
Also I tried using libxev for a project of mine and found it really broke the zig way of doing things. All these callbacks needed to return disarm/rearm instead of error unions so I had to catch every single error instead of being able to use try.
I could have reworked it further to make try work but found the entire thing very verbose and difficult to use with 6 params for all the callback functions.
Thankfully my use case was such that poll() was more than sufficient and that is part of zigs posix namespace so that was what I went with.
Describing its design, the readme for libxev says "credit to this awesome blog post" and links to the same Tigerbeetle post in this submission.
I think Microsoft fixed that in Windows Vista by providing a higher-level APIs on top of IOCP. See CreateThreadpoolIo, CloseThreadpoolIo, StartThreadpoolIo, and WaitForThreadpoolIoCallbacks WinAPI functions.
It’s wonderful being able to write straightforward code that works fast on every platform with no code changes.
I guess the strength of rust (and zig for now) is that the community has a chance to explore lots of different ways to solve these problems. And the corresponding weakness is that everyone uses different libraries, so it’s a fragmented ecosystem full of libraries that may or may not work together properly.
> Can you get similar performance with QUIC?
I don't know that I've seen benchmarks, but I'd be surprised if you can get similar performance with QUIC. TCP has decades of optimization that you can lean on, UDP for bulk transfer really doesn't. For a lot of applications, server performance from QUIC vs TCP+TLS isn't a big deal, because you'll spend much more server performance on computing what to send than on sending it... For static file serving, I'd be surprised if QUIC is actually competitive, but it still might not be a big deal if your server is overpowered and can hit the NIC limits with either.
However, the highest performance public QUIC implementation benchmarks only get ~10 Gb/s per core. It is unclear to me if this is due to slow QUIC implementations or poor UDP stacks with inadequate buffering and processing.
It's also something I just find fascinating because it's one of the few practical cases where I feel like the compositional approach has what seems to be an insurmountable disadvantage compared to making a single thing more complex. Maybe there are a lot more of them that just aren't obvious to me because the "larger" thing is already so well-established that I wouldn't consider breaking it into smaller pieces because of the inherent advantage from having them combined, but even then it still seems surprising that that gold standard for so long arguably because of how well it worked with things that came after eventually run into change in expectations that it can't adapt to as well as something with intentionally larger scope to include one of those compositional layers.
TCP couples them all in a large monolithic, tangled mess. QUIC, despite being a little more complex, has the layers much less coupled even though it is still a monolithic blob.
A better network protocol design would be actually fully decoupling the layers then building something like QUIC as a composition of those layers. This is high performance and lets you flexibly handle basically the entire gamut of network protocols currently in use.
Apple put in effort to get MPTCP accepted in cellular networks (where they have direct leverage) and having it out there (used by Siri) puts pressure on other networks too. If they did the same thing for Fast Open (SYN with data), it could be big.
Unfortunately, I'm not sure anyone other than Apple is capable of doing it. Nobody else really has leverage against enough carriers to demand they make new TCP patterns work; and not many organizations would want to try adding something to SYNs that might fail. (Also, MPTCP allows session movement, so a new TLS handshake isn't required)
This never happened, did it?
Suppose libex is the alternative.
[1] https://en.wikipedia.org/wiki/Io_uring [2] https://github.com/axboe/liburing/discussions/1047
If it ever integrates with LSMs, then it may be time to give it another look.
formerly_proven•2mo ago
O_NONBLOCK basically doesn't do anything for file-based file-descriptions - a file is always considered "ready" for I/O.
catlifeonmars•2mo ago
gpderetta•2mo ago
But for files data is always available to read (unless the file is empty) or write (unless the disk is full). Even if you somehow interpret readiness as the backing pages being loaded in the page cache, files are random access so which pages (ie which specific offset and length) you are interested in can't be expressed via a simple fd based poll-like API (Linux tried to make splice work for this use case, but it didn't work out).
stingraycharles•2mo ago
(This is a genuine question)
yarosv•2mo ago
WD-42•2mo ago
RantyDave•2mo ago
Begs a question though: are there any NVME "spinny rust" disks?
wtallis•2mo ago
My recollection is that NVMe had some features added specifically for hard drives. I don't know if anyone ever bothered making a hard drive that natively used NVMe over PCIe; the main goal was to enable NVMe over Fabrics to work with both solid state and spinning rust drives, so that it could fully replace iSCSI.
Hackbraten•2mo ago
I’ve experienced deadlocks in well-known programs, because developers who were unaware of this issue did a synchronous round-robin loop over stdout and stderr. [1]
[0]: https://docs.ruby-lang.org/en/master/Open3.html#method-c-pop...
[1]: https://github.com/Homebrew/homebrew-cask/pull/21665