As for Facebook tracking I 100% dislike that.
https://github.com/pkgxdev/pantry/pull/5360#issuecomment-233...
Ruby seems fine for brew. Does this do anything else better? Ruby makes it easy to write recipes for it which is a huge boon for a package manager.
The main reason I find brew to be a bit slow is just a lack of parallelism with regards to downloads and installs. Rather brew does alternating and sequential D, I, D, I, D, I, when I wish it just kept downloading in the background when it is doing the installs. It would cut down the brew upgrade time by 30% or more at the cost of more disk space used during the process.
But this one qualm I have isn't a result of its language implementation at all.
(This is a perverse countereffect: small projects can make performance decisions that Homebrew and other larger projects can't make, because they don't have a large install base that reveals the limitations of those decisions.)
I have heard that before.
Hmm.... I wonder if you can get away with not doing parallel downloads, but just keep the sequential downloads going in the background while it is installing a package? It is the pause in downloads during an install that I find is the main slow down in brew.
I could be wrong, but I believe multiple people, including maintainers, have looked into exactly that :-)
(I also need to correct myself: there is some work ongoing into concurrent downloads[1]. That work hasn't hit `brew install` yet, where I imagine the question of concurrent traffic volume will become more pressing.)
(Among other technical challenges, like updating the P2P broadcast for each new bottle.)
Generating additional metadata at bottle build time doesn't appear to be much of a technical challenge either.
These are asymmetric: brew runs at a point in time, and most people decidedly do not want brew running in the background or blocking while leechers are still being serviced. They want it to exit quickly once the task at hand is done.
> Generating additional metadata at bottle build time doesn't appear to be much of a technical challenge either.
That's not the challenge. The challenge is distributing those updates. My understanding is that there's no standard way to update a torrent file; you re-roll a new file with the changes. That means staggered delivery, which in turn means a long tail of clients that see different, incompatible views of the same majority-equal files.
You should only re-distribute the original file that was downloaded and thus one can just advertise the original torrent that was downloaded.
But as you said earlier, brew is a point in time command and this BitTorrent solution would only really work if brew switched to an always-on service. And I am not sure that many people want to do that, although I am sure some would.
Yes, brew exits when it is done installing, nothing would need to change about that if you used BT protocol to speed up downloads. I'm sure you do have some helpful users who would volunteer to seed their cache though, which would become feasible.
> That's not the challenge. The challenge is distributing those updates.
The metadata goes in the formula alongside the current metadata (URLs and hashes.)
Kinda. You do create a new torrent, but you distribute it in a way that to a swarm member is functionally equivalent to updating an old one. Check out BEP-0039 and BEP-0046 which respectively cover the HTTP and DHT mechanisms for updating torrents:
https://www.bittorrent.org/beps/bep_0039.html
https://www.bittorrent.org/beps/bep_0046.html
If that updated torrent is a BEP-0052 (v2) torrent it will hash per-file, and so the updated v2 torrent will have identical hashes for files which aren't changed: https://www.bittorrent.org/beps/bep_0052.html
This combines with BEP-0038 so the updated torrent can refer to the infohash of the older torrent with which it shares files, so if you already have the old one you only have to download files that have changed: https://www.bittorrent.org/beps/bep_0038.html
(There’s also still the state/seeding problem and its collision with user expectations around brew getting faster, or at least not any slower.)
I could see institutional seeders doing it as a way to donate bandwidth though, like a CDN that's built into the distribution protocol instead of getting load-balanced to Microsoft's nearest PoP when hitting a GitHub `ghcr.io` URI like Homebrew does today. Or even better, use that as an HTTP Seed (BEP-0019) to combine benefits of both :)
(My skepticism around whether this makes sense for Homebrew might be obscuring it, but I’m generally quite a big fan of distributed/P2P protocols, and I strongly believe that existing CDN decencies in packaging ecosystems are a risk that needs mitigating.)
There is no state/seeding problem. The client downloads from the same https url as always but uses peers on an as-available basis to speed things up and reduce load on the origin.
> The client downloads from the same https url as always but uses peers on an as-available basis to speed things up and reduce load on the origin.
So some kind of hybrid scheme, which (to me) implies the worst of both worlds: clients are still going to hammer upstreams on package updates (since client traffic isn’t uniform), and every client pays a bunch of peering overhead that doesn’t pay off until the files are “hot.” In other words, upstreams still need to plan for the same amount of capacity, and clients have to do more work.
(The adjacent thread observes that none of this is necessary if CDNs or other large operators do this between themselves, rather than involving clients. That seems strictly preferable to me.)
And the language doesn’t have much to do with that. This project looks to be someone just toying around with Rust or their own PM. Props for that, but the headline has extra implications on HN.
I recently rewrote a big portion of Atlas [1]. It’s a Nim based dependency manager that clones Nim packages to a `deps/` folder. Initially I was worried about using reference types, etc, for performance reasons. It’s a general habit of mine. Then I remembered that stuff would be negligible compared to the download times and git overhead. Well aside from the SAT solver.
What part of the home brew experience do you find slow?
eg...
% time brew upgrade
brew upgrade 0.75s user 0.16s system 68% cpu 1.337 total
% time brew list
brew list 0.01s user 0.02s system 57% cpu 0.054 total
If this is true, why are the Rust tools, on average, so much faster than the JS (or whatever) tools they replace?
Hint: the answer hasn't just to do with the theoretical technical merits of the underlying platform, but also what type of developer they attract.
To be fair I haven't noticed Brew being as tediously slow as Pip. Maybe I just use it way less.
1. Python packaging, unlike Homebrew, does have a compute-heavy phase in the form of dependency resolution. `uv` can make significant gains over `pip` in that phase.
2. `uv` performs parallel downloads and other operations, in part because of Rust's fearless parallelism.
Homebrew doesn't really have (1), since resolution is just a linearization of the dependency tree. And parallelism of downloads in (2) poses a problem for Homebrew that's been mentioned in other threads (whereas parallelism is not a significant problem for PyPI's CDN-fronted distribution).
This. There's a wave of projects whose only value proposition is this vacuous "let's reinvent the wheel in Rust" sales pitch, where nothing of value is proposed beyond throwing around the Rust buzzword.
It would be interesting to know if there are other goals though, e.g. UX improvements.
There has to be more important reasons to replace a mature widely use project like homebrew.
2. It's a personal project.
3. It's explicitly declared as alpha software.
Doesn’t tell me how it differs. What makes this next generation? Just the programming language?
If it’s just a for fun personal project that no one else is supposed to use, I’m not sure why it’s on HN.
Setting of relative paths for bottle installs is still not perfect, well it works for every bottle I have tested except rust. Getting bottles working 100% is very doable though imo.
Build from source formulae is still pretty f*ed + I do not know if it is really feasible given that the json API lacks information there and a full on Ruby -> Rust transpiler is way out of scope. Will probably settle for automatic build system detection based on archive structure there. + Maybe do my own version of the .rb scripts but in a more general machine readable format, not .rs lol
Casks seem to work but I have only tested some .dmg -> .app ones and .pkg installers so far though. As with bottles 100% doable.
Given that almost all formulae are available as bottles for modern ARM mac this could become a fully featured package manager. Actually didn't think so many people would look at it, started building it for myself because Homebrew just isn't cutting it for what I want.
Started working on a declarative package + system manager for mac because I feel ansible is overkill for one machine and not really made for that and nix-darwin worms itself into the system so deep. Wrapping Brew commands was abysmally slow though so I started working on this and by now I am deep enough in I won't stop xD
Anyway I am grateful for every bug report, Issue and well meaning pull request.
I'm guessing it's that you hoping that it is eventually more performant -- are there specific areas of current brew you have identified as performance bottlenecks likely to eventually benefit from a rust implementation?
Or any more info to share about assumptions/hopes that motivated this or any other motivations?
The performance of apt/dnf in comparison is surreal; but dnf (or at least yum, its predecessor) is written in Python; which has even worse performance characteristics than Ruby.
Clearly something is wrong, I wonder how different they are architecturally.
(There's still low handing fruit, but it's not like it was a few years ago where `brew list` took seconds to run. It now runs nearly instantaneously for me locally, like most of the other happy path commands.)
However, the speed increase coincided with my upgrade to an M-Series laptop, so it's possible I just presumed there was a significant hardware speedup in the time we're talking about.
Thanks to rust just being (slightly, significantly? no idea about ruby's speed) faster + concurrent downloading & pouring of bottles, most "regular" formula installs feel a good bit faster than brew already. Mainly noticeable when installing multiple formulae at once.
Casks, especially those with pkg installers, seem to profit a bit less here.
Performance was a reason, not the main one though, like I said I wanted and still want, to build a declarative package + system managing solution on top. The idea was to get into rust with that. Imo having the base written in the same language instead of wrapping commands also gives more flexibility there.
Another reason is that I never liked the way brew looks and feels. Right now the ui/ux for Sapphire is far from finished, more like a clusterf*k and only the search command really looks the way I want it. Aiming for something modern, clean and information rich without beeing overly verbose. I really like dnf5 and what AerynOS is doing and will probably take some inspiration there.
Like mentioned, Bottles and Casks should be 100% doable and that would cover most package needs on macOS, I do not see why I should also define a new repo and packaging ecosystem when such a big and popular one exists.
Source build capability will probably stay(for easy integration of source building in the system management part later) but not be focused on brew formulae as the ruby dsl would be a horror to parse.
Well and sh*t I am not trying to compete really. This is the first time building something with rust and I really really had no idea what a giant never ending rabbit hole macOS package management is and how massive and complex Brew is.
This went from should I to can I pretty quick for me xD
Maybe that could be a place where sapphire differentiates?
If I may surface one use case: Several years ago I had to manage a bunch of Macs for CI jobs. The build process (Unreal's UAT) didn't support running more than one build process at a time, and Docker was really slow, so I'd hoped to use different user accounts to bypass that and get some parallelization gains. Homebrew made that very difficult with its penchant for system-wide installs. So a feature request: I'd love to see a competitive package manager that limits itself to operating somewhere (overridable) in the user's home directory.
I didn't check, but there is a chance that path is also hardcoded in (some) formulae, so even building from the source might not help here.
Please add knobs for the end user to manually configure this per package and global default before adding autodetection. As a user to is very frustrating to have to patch the package manager to override some well-intentioned automagic which didn't consider my setup or dig through sources to uncover some undocumented assumption. yarn is a cautionary example.
The way I ended up using it was that `brew install` would temporarily install something, without adding it to my Brewfile. And a little `brew add` wrapper would add the package to my Brewfile to keep it on the system permanently. That part with the wrapper could have used some love and would be a nice fit for a new brew-compatible frontend IMO. Maybe you could expand on that for Sapphire, if that also scratches your declarative itch?
I don't see a lot of engineers running Intel Macs anymore. I haven't seen any engineers who still use an Intel model, myself, and for quite some time. Especially when there are Apple Silicon options for well under $1,000 that highly outperform the Intel models.
I'm not a big fan of keeping the Homebrew terminology though. I never know what a formula, keg, cask, cellar, tap or bottle is. Why not keep to the standard terms of package and repository etc? I don't know beer brewing terminology or how beer brewing is analogous to package management, and I honestly wish that it wasn't something which my tools expect me to learn.
Everything you need is there...
I'd agree that the current homebrew terms are inappropriately whimsical and hard to grasp, but you are right in your intuition and goal, IMO.
That is, taking care of the gears first and then carefully adjusting the public API.
No different than the Windows registry, which apparently uses a honeybee / hive metaphor because some Windows dev hated bees and their teammates liked trolling them.
I don't want to memorise their twee names; I'd much rather the name tell me what the entity / operation does by itself.
It’s not that I like boring. But I really like descriptive names. I have other things to do with my time than figuring out what the hell a cask, a tap, or a bottle is. Like solving the problem that requires the damn software.
I wish they’d just call them binaries, macOS packages, packages, gui-packages, etc.
To be clear, all those words are also jargon but they’re reusable concepts across software.
At its core, there are really two parts to Homebrew:
1. There's the client side, i.e. `brew`, which 99.9% of users stick to happy paths (bottle installs, supported platforms) within. These users could be supported with relative ease by a small native-code installer, since the majority of the work done by the installer in the happy path is fetching bottles, exploding them, and doing a bit of relocation.
2. There's literally everything else, i.e. all of the developer, repository, and CI/CD machinery that keeps homebrew-core humming. This is the largely invisible infrastructure that makes `brew install` work smoothly, and it's very hard to RIIR (in a large part because it's tied heavily to the formula DSL, which is arbitrary Ruby).
(1) is a nice experimental space, because Homebrew does (IMO) a decent job of isolating the client-facing side from the complexity of (2). However, (2) is where the meat-and-potatoes of packaging happens, and where Homebrew's differentiators really lie (specifically, in how easy it is to contribute new packages and bump existing ones).
Edit: Another noteworthy aspect here around performance: I mentioned this in another comment[1], but parallel downloads of things like bottles and DMGs is not an architectural limitation of Homebrew itself, but instead a conscious decision to trade some install speed for courtesy towards the services we fetch from (including GitHub itself). Smaller projects can sidestep this because they're not directing nearly the same degree of volume; I think this project will discover if/when its volumes grow that it will need to serialize downloads to avoid being throttled or outright limited.
That doesn't make sense. As you say you're directing a huge volume of traffic so it makes no difference exactly when a user downloads a byte. It all gets smeared out. Only the total amount of data matters and that is unaffected by parallelism.
Homebrew's traffic pattern is not a uniform distribution. Package updates go out, and users download those packages in structured ways: there are spikes for MDM-managed Homebrew installations, spikes for cronjobs and CI/CD systems, spikes at 9AM on different coasts when developers sign into their machines, etc.
(How much this matters is also not uniform: it matters somewhat less for GitHub Packages - they should have a hefty CDN - and it matters somewhat more for Casks, which tend to be large DMGs hosted on individual servers.)
I also feel that there could be a lot of automation in the backend part, catching bugs early (maybe even on local machine before CI run) for example.
RIIR - "Rewrite It In Rust" (maybe obvious in context? sharing in case not)
What discourages me from using Homebrew is the intent and the mindset of its developers and packagers, who, I think, see their goal building an "unstable" distribution, as Debian defines it: "[a distribution that] is run by developers and those who like to live on the edge".
I am not blaming the Homebrew developers for building a Sid rather than Bookworm. Some people want just that. Heck, I used to run Debian Sid myself, but have lost my patience for maintaining my own computers since: I am kept busy enough by fixing the software I write, I don't want to spend more time fixing software I did not.
With many macOS users coming from a different communities than Debian users, I really wonder how well that would go over with the folks whose software was being distributed.
I know why certain software has be run with root privileges, but it's a bit hard for me to come with a reasonable scenario where a software would fail to run properly when installed to a directory that is owned by the same non-root user that launches that software.
Lemme try to repro again when I'm home.
This is how I install Homebrew when I have to, and so far the only issue I ran into is that binary packages are often tagged as installable only to Homebrew's default folder, so Homebrew had to built the to-be-installed software from source instead, resulting in it taking longer and the computer fans spinning louder.
[0] https://docs.brew.sh/Installation#untar-anywhere-unsupported
I know it sounds dumb but uv was smart to go shorter than pip and sapphire feels heavier than brew no matter what it does after typing that.
Something like that you mean?
I am not sure if the lesson is to try harder to avoid offence, or live with the fact that words can have multiple meanings and we can be "professional" enough to ignore some of those meanings in some contexts.
A decent full package manager would support a simple, shell-like DSL like say Alpine or Arch, concurrent and parallel phases (such as downloads/builds/installs), multiple versions, reproducible builds, building from source, build acceleration, security auditing, patch management, and package cryptographic signatures (not hashes or signing of hashes).
Nix is theoretically amazing but the barrier-to-entry and gotchas for popular use make it self-limiting. Simplicity in particular areas is a price that is often paid for popularity.
A couple purely superficial suggestions (echoing some other comments here):
- Lose the Brew terminology, especially if the name of the project isn't a synonym of "brew." - Change the name in general. "Sapphire" makes me think of "Ruby." IMO the obvious name is MacPac :p
Interestingly, I always imagined that a would-be replacement would come written in Swift. I guess I was wrong.
nartho•4h ago
adamnemecek•4h ago
carterschonwald•4h ago
adamnemecek•4h ago
Henchman21•4h ago
israrkhan•4h ago
no_wizard•4h ago
cassianoleal•4h ago
mort96•4h ago
gkfasdfasdf•4h ago
carstenhag•4h ago
StopDisinfo910•4h ago
user82747493•4h ago
StopDisinfo910•4h ago
woodruffw•4h ago
(Beyond anything else, Homebrew's biggest "win" over MacPorts was and probably is still UX and DX. The core technology of a packaging ecosystem is rarely itself the differentiator.)
StopDisinfo910•3h ago
Homebrew is by far the worst package manager I have ever used. I’m still sour it somehow dragged away packagers from solutions which were better in every way by being promoted as the "default" solution.
woodruffw•2h ago
I can't find these anywhere on the official blog, which goes back to the first 1.0 release of Homebrew. Links would be helpful.
Lammy•2h ago
Here are the comparisons to other package managers:
> Packages are brewed in individual, versioned kegs. Then symlinks are created to give a normal POSIX tree. This way the filesystem is the package database. Everything else is now easy. We are made of win.
vs MacPorts registry which used its own homebrewed (lol) Receipts files in 2009, and now uses a SQLite DB: https://guide.macports.org/chunked/internals.registry.html#i...
> I wouldn't worry about it not being root. We don't install anything base enough for it to be a concern (unlike MacPorts or Fink).
vs MacPorts installs to `/opt/local` as root.
> Why Not MacPorts?
> =================
> 1. MacPorts installs its own libz, its own openssl, etc. It is an autarky.
> This makes no sense to me. OS X comes with all that shit.
> 2. MacPorts support Tiger, and PPC. We don't, so things are better optimised.
There is no “Why Not Fink?” section.
And because I didn't know the word autarky: https://en.wiktionary.org/wiki/autarky
woodruffw•2h ago
cstrahan•4h ago
I don’t think you are being fair. This question presupposes that the supposed problems can be solved by iterative changes, rather than being inherent in the chosen design/architecture of the software, which usually requires complete replacement thereof (as well as the leadership thereof, as people who choose poor solutions to problems often can’t appreciate arguments for superior solutions).
(Not that I’m trying to suggest that I agree that homebrew in particular is bad — just speaking generally.)
user82747493•3h ago
lxe•4h ago
cls59•4h ago
Also, using Git + GitHub instead of SVN + Trac was absolutely a winning pick for scaling project participation back in the 2000s.
Lammy•4h ago
https://www.finkproject.org/doc/users-guide/index.php
https://pdb.finkproject.org/pdb/index.php?phpLang=en
(Not a recommendation; seems pretty dead)
whalesalad•4h ago
jrochkind1•4h ago
That was it, that's why I switched, nothing more, nothing less.
cosmic_cheese•4h ago
anarticle•4h ago
frollogaston•3h ago
nottorp•4h ago