This is the way.
The people working on Rust are a finite (probably overextended!) set of people and you can't just add more work to their plate. "Just" making the standard library bigger is probably a non-starter.
I think it'd be great if some group of people took up the very hard work to curate a set of crates that everyone would use and provide a nice façade to them, completely outside of the Rust team umbrella. Then people can start using this Katamari crate to prove out the usefulness of it.
However, many people wouldn't use it. I wouldn't because I simply don't care and am happy adding my dependencies one-by-one with minimal feature sets. Others wouldn't because it doesn't have the mystical blessing/seal-of-approval of the Rust team.
A lot.
Like, a lot a lot a lot. Browse through any programming language that has an open issue tracker for all the closed proposals sometime. Individually, perhaps a whole bunch of good ideas. The union of them? Not so much.
It's unfortunate that the response so far hasn't been very positive
- All included crates can be tested for inter-compatibility
- Release all included crates under a single version, simplifying upgrades
- Sample projects as living documentation to demo integrations and upgrades
- Breaking changes can be held until all affected crates are fixed, then bump all at once
- An achievable, valuable, local goal for code review / crev coverage metrics
We do not need to saddle Rust with garbage that will feel dated like Python's standard library. Cargo does the job just fine. We just need some high quality optional batteries.
Embedded projects are unlikely to need standard library bloat. No_std should be top of mind for everyone.
Something that might make additional libraries feel more first class: if cargo finally got namespaces and if the Rust project took on "@rust/" as the org name to launch officially sanctioned and maintained packages.
Rust, as a systems language, is quite good at working on a variety of systems.
And the systems language remark, I am still looking forward when sorting ABI issues for binary libraries is finally something that doesn't need to go through solutions designed for C and C++.
Python's standard library is a strength, not a weakness. Rust should be so lucky. It's wonderful to have basic functionality which is guaranteed to be there no matter what. Many people work in environments where they can't just YOLO download packages from the Internet, so they have to make do with whatever is in the stdlib or what they can write themselves.
Rust is luckier. It has the correct approach. You can find every battery you need in crates.io.
Python has had monstrosities like urllib, urllib2, http, etc. All pretty much ignored in favor of the external requests library and its kin. The standard library also has inconsistencies in calling conventions and naming conventions and it has to support those *FOREVER*.
The core language should be pristine. Rust is doing it right. Everything else you need is within grasp.
Standard response every time there is some criticism of Rust.
My counter argument is that the "batteries included" approach tends to atrophy and become dead weight.
Your counter seems to be "that's not an argument, that's just Rust hype."
Am I interpreting you correctly? Because I think my argument is salient and correct. I don't want to be stuck with dated APIs from 20 years of cruft in the standard library.
I think what you're suggesting is a great idea for a new standard library layer, you're just not using that label. A set of packages in a Rust namespace, maintained by the same community of folks but under policies that comply with best practices for security and some additional support to meet those best practices. The crates shouldn't be required, so no_std should work just as it would prior to such a collection.
However I rather have cruft that works everywhere the toolchain is fully implemented, instead of playing whack-a-mole with third party libraries when only some platforms are supported.
How much maintenance could you possibly need to load secrets from .env into the environment.
What this means in practice is that the call to invoke dotenv should also be marked as unsafe so that the invoker can ensure safety by placing it at the right place.
If no one is maintaining the crate, that won’t happen and someone might try to load environment variables at a bad time.
whatever the issue is, "setting an env var is unsafe" is so interesting to me that I'm now craving a blog post explaining this
> Achtung! This is a v0.* version! Expect bugs and issues all around. Submitting pull requests and issues is highly encouraged!
ZeroVer https://0ver.org/
But that's not practical for all situations. For example, Web frontend developer culture might be the worst environment, to the point you often can't get many things done in feasible time, if you don't adopt the same reckless practices.
I'm also seeing it now with the cargo-culting of opaque self-hosted AI tools and models. For learning and experimenting, I'd spend more time sufficiently compartmentalizing an individual tool than with using it.
This weekend, I'm dusting off my Rust skills, for a small open source employability project (so I can't invest in expensive dependency management on this one). The main thing thing bothering me isn't allocation management, but the sinking feeling when I watch the cast-of-thousands explosion of transitive dependencies for the UI and async libraries that I want to use. It's only a matter of time before one of those is compromised, if not already, and one is all it takes.
There's literally 1000s of RFCs for rust with only a small handful that are integrated. Having this forest, IMO, makes it hard for any given proposal to really stand out. Further, it makes duplicate effort almost inevitable.
Rust's RFC process is effectively a dead letter box for most.
Devs can add whatever they feel like on their workstations but it will be a sad build server if they get pushed without permission.
Anything else will get abused in the name of expediency and just-this-one-time.
Also, the process for adding a crate/gem/module/library needs to be the same as anything else: license review, code review, subscription to the appropriate mailing list or other announce channel, and assignment of responsibility. All of these except code review can be really, really fast once you have the process going.
All problems are, at least in part, dependency chain management problems.
The dependency trees for most interpreted or source-distributed languages are ridiculous, and review of even a few of those seems practically impossible in a lot of development environments.
A compromised dev machine is also a problem.
If I was to design a Rust 2.0, I'd make it so dependencies need permissions to access IO, or unsafe code, etc.
This is a problem with all languages and actually an area where Rust shines (due to editions). Your pulled in packages will compile as they previously did. This is not true for garbage collected languages (pun intended).
> Out of curiosity I ran toeki a tool for counting lines of code, and found a staggering 3.6 million lines of rust .... How could I ever audit all of that code?
Again, another area where Rust shines. You can audit and most importantly modify the code. This is not that easy if you were using Nodejs where the runtimes are behind node/v8 or whatever. You compile these things (including TLS) yourself and have full control over them. That's why Tokio is huge.
JavaScript is backwards compatible going back effectively forever, as is Java. Rust's unique system is having a way to make breaking changes to the language without breaking old code, not that they prioritize supporting old code indefinitely.
The libraries are a different story—you're likely to have things break under you that rely on older versions of libraries when you update—but I don't see Rust actually having solved that.
> You can audit and most importantly modify the code. This is not that easy if you were using Nodejs where the runtimes are behind node/v8 or whatever.
Node and V8 are open source, which makes the code just as auditable and modifiable as the 3.6 million lines of Rust. Which is to say, both are equally unapproachable.
No language can fix that. However, I've lost count of the times my Python/JavaScript interpretation fails because of something in one of the dependencies. Usually, it's not a JS/Python problem but rather has to do with a Node/Python version update. It always boils down to the "core" issue which is the runtime. That's why I like that Rust give me a "fixed" runtime that I download/compile/package with my program.
> Node and V8 are open source, which makes the code just as auditable and modifiable as the 3.6 million lines of Rust. Which is to say, both are equally unapproachable.
I've recently patched a weird bug under Tokio/Otel and can't imagine doing that with Node/V8 without it being a major hassle. It is relatively straightforward in Rust though requires maintaining your own fork of only the dependency/branch in question.
If I were designing a new language I think I'd be very interested in putting some sort of capability system in so I can confine entire library trees safely, and libraries can volunteer somehow what capabilities they need/offer. I think it would need to be a new language if for no other reason than ecosystems will need to be written with the concept in them from the beginning.
For instance, consider an "image loading library". In most modern languages such libraries almost invariably support loading images from a file, directly, for convenience if nothing else. In a language that supported this concept of capabilities it would be necessary to support loading them from a stream, so either the image library would need you to supply it a stream unconditionally, or if the capability support is more rich, you could say "I don't want you to be able to load files" in your manifest or something and the compiler would block the "LoadFromFile(filename)" function at compile time. Multiply that out over an entire ecosystem and I think this would be hard to retrofit. It's hugely backwards incompatible if it is done correctly, it would be a de facto fork of the entire ecosystem.
I honestly don't see any other solution to this in the long term, except to create a world where the vast majority of libraries become untargetable in supply chain attacks because they can't open sockets or read files and are thus useless to attackers, and we can reduce our attack surface to just the libraries that truly need the deep access. And I think if a language came out with this design, you'd be surprised at how few things need the dangerous permissions.
Even a culture of minimizing dependencies is just delaying the inevitable. We've been seeing Go packages getting supply-chain-attacked and it getting into people's real code bases, and that community is about as hostile to large dependency trees as any can be and still function. It's not good enough.
As long as all library code is compiled/run from source, a compiler/runtime can replace system calls with wrappers that check caller-specific permissions, and it can refuse to compile or insert runtime panics if the language's escape hatches would be used. It can be as safe as the language is safe, so long as you're ok with panics when the rules are broken.
It'd take some work to document and distribute capability profiles for libraries that don't care to support it, but a similar effort was proven possible with TypeScript.
1. Well defined scope
2. Infrequent changes
Nomad has many of these (msgpack, envparse, cli, etc). These dependencies go years without changing so the dependency management burden rapidly approaches zero. This is an especially useful property for “leaf” dependencies with no dependencies of their own.
I wish libraries could advertise their intent to be Mature. I’d choose a Mature protobuf library over one that constantly tweaked its ergonomics and performance. Continual iterative improvement is often a boon, but sometimes it’s not worth the cost.
If you want a mature protobuf implementation you should probably buy one. Expecting some guy/gal on the internet to maintain one for your for free seems ill advised.
I will say I get great satisfaction from the little envparse library I wrote needing near-0 maintenance. It’s a rare treat to be able to consider any project truly done.
Also there are lots of lovely projects maintained at high levels by hobbyists, and plenty of abandonware that was at some point paid for
It is an easy way to get a somewhat OK standard library as the things you add became popular on their own merits at some point.
Once added, the lowest friction path is to just use the standard library; and as it is the standard library you have a slightly better hope someone will care to maintain it. You can still build a better one if needed for your use-case, but the batteries are included for basic usage
tokio is a work-stealing, asynchronous runtime. This is a feature that would be an entire language. Does OP consider it reasonable to audit the entire Go language? or the V8 engine for Node? v8 is ~10x more lines than tokio.
If Cloudflare uses Node, would you expect Cloudflare to audit v8 quarterly?
This is something I've only ever seen cargo do.
This is the cause of so many issues.
And its not like we're at war or trying to cure the next pandemic, we're writing CRUD apps and trying to convince people to click on adds for crap they don't need.
You can ensure that third-party Rust dependencies have been audited by a trusted entity with cargo-vet.
And you should have taken a look at where those 3M locs come from, it's usually from Microsoft's windows-rs crates that are transitively included in your dependencies through default features and build targets of crates built to run on windows.
Isn't the point of a memory safe language to allow programmers to be sloppy without repercussions, i.e., to not think about managing memory and even to not understand how memory works.
Would managing dependencies be any different. Does Rust allow programmers to avoid thinking carefully about selecting dependencies.
No. The point is even the best programmers of unsafe languages regularly introduce both simple and subtle bugs into codebases while being careful about handling memory correctly, and therefore we should use languages that don't even allow those bugs for most every use case. Using these languages still allows crap programmers to waste GBs of correctly allocated and handled memory, and good programmers to write tight, resouce-sipping code.
Dependencies are orthogonal to this.
Are there systems languages that provide memory management but do not default to using third party libraries. If yes, then do these languages make it easier for programmers to avoid dependencies.
You can be _relatively_ sure that you're not introducing memory unsafety by adding a dependency, but you can't be sure that it isn't malware unless you audit it.
It's the difference between a wet mess and a dry one. Rust creates dry messes. It's still a mess.
Meanwhile, the heaviest JavaScript parser implemented in JavaScript is more lightweight.
I decided that I should leave this project alone and spend my time elsewhere.
> Meanwhile, the heaviest JavaScript parser implemented in JavaScript is more lightweight.
The lightest weight javascript program relies on V8 to run, which has multiple orders of magnitude more dependencies. Most of which you have never heard of.
At least cargo makes it easier to get a clearer picture of what the dependencies are for a program.
This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painstakingly included parts of into your size constrained environment (fit it on a floppy). You probably picked apart that library and pulled the bits you needed, integrating them into your builds to be as small as possible.
Today we pile libraries on top of libraries on top of libraries. Its super easy to say `import foolib`, then call `foolib.do_thing()` and just start running. Who knows or cares what all 'foolib' contains.
At each level a caller might need 5% of the functionality of any given dependency. The deeper the dependency tree gets the more waste piles on. Eventually you end up in a world where your simple binary is 500 MiB of code you never actually call, but all you did was take that one dependency to format a number.
In some cases the languages make this worse. Go and Rust, for example, encourage everything for a single package/mod to go in the same file. Adding optional functionality can get ugly when it would require creating new modules, but if you only want to use a tiny part of the module, what do you do?
The only real solution I can think of to deal with this long term is ultra-fine-grained symbols and dependencies. Every function, type, and other top-level language construct needs to declare the set of things it needs to run (other functions, symbols, types, etc). When you depend on that one symbol it can construct, on demand, the exact graph of symbols it needs and dump the rest for any given library. You end up with the minimal set of code for the functionality you need.
Its a terrible idea and I'd hate it, but how else do you address the current setup of effectively building the whole universe of code branching from your dependencies and then dragging it around like a boat anchor of dead code.
The analogy I use is cooking a huge dinner, then throwing out everything but the one side dish you wanted. If you want just the side-dish you should be able to cook just the side-dish.
I just don't listen. Things should be easy. Rust is easy. Don't overthink it
Clarification: Go allows for a very simple multi-file. It’s one feature I really like, because it allows splitting otherwise coherent module into logical parts.
For example, you can't split up a module into foo.rs containing `Foo` and bar.rs containing `Bar`, both in module 'mymod' in such a way that you can `use mymod::Bar and foo.rs is never built/linked.
My point is the granularity of the package/mod encourages course-grained deps, which I argue is a problem.
It should be easy to build and deploy profiling-aware builds (PGO/BOLT) and to get good feedback around time/instructions spent per package, as well as a measure of the ratio of each library that's cold or thrown away at build time.
It's a terrible idea because you're trying to reinvent section splitting + `--gc-sections` at link time, which rust (which the article is about) already does by default.
If you want to use dependencies, I wouldn't be surprised when you realise they also want to use dependencies. But you can put your money/time in the right places. Invest in the dependencies that do things well.
In the absence of a technical solution, all others basically involve someone else having to audit and constantly maintain all that code. If it was pulled into Rust stdlib, that team would be stuck handling it, and making changes to any of that code becomes more difficult.
palata•12h ago
Cargo makes it so simple to add tons of dependencies that it is really hard not to do it. But that does not stop here: even if I try to be careful with adding dependencies, a couple dependencies are likely to pull tens of transitive dependencies each.
"Then don't depend on them", you say. Sure, but that means I won't write my project, because I won't write those things from scratch. I could probably audit the dependency (if it wasn't pulling 50 packages itself), but I can't reasonably write it myself.
It is different with C++: I can often find dependencies that don't pull tens of transitive dependencies in C++. Maybe because it's harder to add dependencies, maybe because the ecosystem is more mature, I don't know.
But it feels like the philosophy in Rust is to pull many small packages, so it doesn't seem like it will change. And that's a pity, because I like Rust-the-language better than C++-the-language. It just feels like I trade "it's not memory-safe" for "you have to pull tons of random code from the Internet".
jampekka•8h ago
ChocolateGod•7h ago
It's part of the reason why software distribution on Linux has been pushed to using containers, removing the point of having shared libraries. I think Google with it's C++ replacement (Carbon) plans on doing it's own system.
skydhash•31m ago
It could be better, but the current solutions (npm, go, python,...) favor only the developers, not the maintainers and packagers.
antonvs•7h ago
You need to think a bit harder about that, to help you decide whether your position is rational.
MeetingsBrowser•38m ago
perrygeo•7h ago
I'm not sure it's a philosophy, more a pragmatic consideration for compilation speeds. Anyone who's done a non-trivial amount of Rust knows that moment when the project gets too big and needs to split into separate crates. It's kinda sad that you can't organize code according to proper abstractions, many times I feel forced to refactor for compiler performance.
X0Refraction•7h ago
I think it makes a good point that some of the difference here is just perception due to dependencies in C/C++ being less immediately visible since they're dynamically loaded. To some degree that is a plus though as you likely trust the maintainers of your OS distribution to provide stable, supported libraries.
As other commenters have said, perhaps this is an area where the Rust maintainers could provide some kind of extended standard library where they don't guarantee backwards compatibility forever, but do provide guarantees about ongoing fixes for security issues.
imtringued•7h ago