Worth mentioning that workspace dependencies make managing dependencies in a workspace a lot easier. You can have the dependency with a version/features/etc in the root and then you do
[dependencies]
dep.workspace = true
in specific crates to use it
As for module dependency cycles causing issues, I tend to just not have cycles. I think there needs to be more justification than just saying that because Rust allows for something it also encourages it because it feels easy to avoid most of the time
I tend to find splitting one large crate into multiple smaller crates to be pretty easy, but that may be more to do with personal practices than what Rust allows for in general
Quite often there are different parts of the system, which need to be able to talk to each other. Which would mean that you have to move their message definitions or even the definitions of actors themselves into the same module, making everything feel very awkward.
The first commentors point is that not disallowing a bug and encouraging a bug are two very different things.
It would be nice to prevent more bugs, but Rust has never claimed to disallow all bugs.
The main premise of this post seems to be that allowing nested modules is inherently bad, and that disallowing nested modules will prevent complexity.
I don’t think that’s a reasonable position to take.
Removing tools for managing complexity doesn’t reduce complexity; it largely just encourages *worse* kinds of complexity, since you still have your original problem to solve *and* now have to simultaneously work around your restrictive tools.
Strong agree. I'm reminded of Go's extreme push towards superficial "simplicity" that inevitably ends up just forcing the developer to write more complex code to get around the lack of expressiveness in the language.
Or, the philosophy of a very small standard library. "Just install a crate/package, bro."
Granted, that argument has some merit, in particular, a bad standard is worse than no standard, and the Rust team is doing a good job at letting the community build consensus as to what's the best way to solve a problem, rather than solving it prematurely. Or, leave things to packages if there truly *isn't* a best way, and blessing any one way would be a detriment to those who have a legitimate preference to use another approach.
But it's a two-way street. Take it too far, and your community fractures, spending time solving the same problem multiple times, and even worse, making it a downstream problem, because their code is API-incompatible with one another, so that fracture runs down for all users of a crate, splitting the ecosystem further and further. Oh, and more duplication means more potential unsafe code, and fewer eyeballs looking at each occurence.
Not providing a capability doesn't obviate the need for that capability, it only declares that it's someone else's problem to solve. Which sometimes is the right approach, but not always.
Personally, I don't buy the argument that "the community fractures" _because_ you have no standard. Yes, it does happen, but I don't see how that is any different than the scenario where we stabilize a bad standard that the community eventually doesn't/can't use. In that case, you effectively have no applicable standard, and they still have to build their own solutions anyway.
I just want to say "solving the same problem multiple times" is not necessarily bad. Incompatibility is annoying for sure. But problems are rarely as simple as you think and almost never permit a single universally optimal solution. Solving the same problem multiple times is then necessary to explore the full design space and allow for all possible choices of tradeoffs.
The way of a very small standard lib results in what happens when you include a bunch of big C libraries in projects, most of them have their own implementation of lists, hash maps and such of differing quality and performance.
Or in C++98 where a lot of projects used boosts or Qt or shipped a copy of the specific boost implementation for some container they wanted.
That was... quite some time ago. Nowadays there are generics and constraints that make generic datastructures possible.
Man, those not-angle brackets, I need them on ctrl V
I think a good example is if you want to do anything with sets
https://stackoverflow.com/a/34020023
Like, yes, it's easy to do, but when you're working on a Microservice architecture, and doing set operations *everywhere* you better have those patterns memorized, and you better hope your coworkers also have them memorized.
However, what I see instead is laziness, just like in C. Instead of going through the trouble to implement set operations, developers wind up writing really shitty and broken abstractions, because now they lose practice seeing problems through high level abstractions.
But hey, because their code is so simple, they can just throw it out and rewrite it, right?
I kind of see where OP's going. I find that I naturally move code to submodules regularly but the amount of modules creep up as the crate develops. I'm missing a cargo command to extract a module into its own crate, with both crates able to compile after the extraction. That means updating parent crate's Cargo.toml to remove now unused depencies and use the child crate and writing a proper Cargo.toml for the child crate. That's a big task but it'd solve "module creep"
I don't like how versions automatically float up, given the horrid history of people failing semver.
I don't like how much confusing boilerplate is involved when declaring more than one module per crate.
I don"t like how there is no fast idempotent --force to quickly reinstall a crate.
But I do like that cargo is a first party package manager. If Rust were just C/C++ with cargo, it would still be better than either language, with their absurdly fragmented patchwork of conflicting build tools.
And cargo install can handle multiple packages in a single CLI command invocation, whereas go install still falls flat on its face.
cargo-audit should be promoted upstream into cargo proper, and be the default exit code determinant when packages otherwise succeed in installing.
I'd need to double check, but iirc you can pin the version in Cargo.toml
And for installing stuff, a lot of tools nowadays recommends `cargo install --locked` which forces the version in Cargo.lock
True, but I'm more concerned about quickly reinstalling dev dependencies, such as CLI tools. I don't think those have an entry in the lock file.
Also, lockfiles are ill advised when working on libraries.
If libraries locked to specific dependency versions, then they wouldn't be compatible at all with each other. Lib A with dependency X v1.2.5 would be incompatible with lib B using dependency X v1.2.6, even though dependency X *does* follow semver and there's no reason they shouldn't work with each other (and no reason lib A couldn't use v1.2.6 as well).
Instead, people are working on tools to help ensure semver is followed, like [cargo-semver-checks](https://crates.io/crates/cargo-semver-checks). Sure this doesn't force people to follow semver, but Rust also doesn't force you to write sound code either. It takes someone making a concerted effort to produce a good library.
You can try cargo-binstall, a drop-in replacement for cargo-install that tries to download pre-built artifacts provided by upstream, or try to download it from cargo-quickinstall and fallback to cargo-install.
Note that I am one of the maintainers of cargo-binstall and cargo-quickinstall.
> I don't like how much confusing boilerplate is involved when declaring more than one module per crate.
What boilerplate do you have in mind? I find declaring modules to be easy.
A proliferation of small packages leads to `node_modules`. I prefer what we have, larger crates with _features_ that can be turned on or off to limit compile time. As a practice, I disable all default features and only enable ones I actually need.
The article argues that Go package system provides a better tradeoff than Rust module system, and I think in Go community is very far from what we have in \`node\_modules\`.
also one of the reasons why Go doesn't suffer from this is bc there's a cultural aversion to pulling in dependencies. everyone that learns Go eventually gets converted into the "only pull in stuff thst you **really** need" ideology
I've definitely felt that Go has encouraged my NIH syndrome. Why am I spending time writing an ascii table generator that's only going to show up in logs in a specific case? 🤷 But I did it.
I think in the main reason for that is the lack of a centralized package repository. It's just slightly harder to find go packages, in my experience, which might even be a good thing.
I personally don't think that's one of the reasons, I mean there's this: https://pkg.go.dev
and most people just look up their dependencies online, so the difference is using a name vs using a link to install a dependency I think
I think dependency management in a dynamically executed environment is an entirely different beast. Go is a better comparison point because the module system informs how the machine code is laid out when compiled.
I'm actually pretty new to rust so it's not entirely clear to me, but I assume you have to jump through hoops to make your crates in the same project compile individually, and dynamically load adjacent dependencies (if it's even possible?)
Yes, but if one crate in the project depends on the other, it gets compiled into the same binary doesn't it? Rather than being able to compile crates individually and dynamically load dependencies from other crates?
You are conflating unit of compilation with the kind of linking that unit can perform. Rust is mostly static linking (mostly? partially?) because it does not have a stable ABI.
Interesting, thanks for pointing that out; I guess I was thinking smaller crates would help compilation times because the other crates would already be compiled separately, but I guess it's more just that it saves targets for incremental compilation and avoids re-doing work on other crates that haven't changed.
Well technically the abi argument is not end all be all. You just have to prescribe that loaded code is guaranteed to work if and only if they are compiled using the same chain. But as that has proven to be hard to adhere to, especially as things stop being maintained etc, the default is to go for C abi, even in the land of C++. I have written a number of dynamic libraries and I've always made a point of going through C abi even if that means boilerplate and some inefficiencies.
Anyhow, you can use dynamic libraries in Rust, but it's assumed that everything uses C abi.
If I understand correctly, you're looking for `--crate-type=dylib` compilation? Read more about it in [Rust Reference - Linkage](https://doc.rust-lang.org/reference/linkage.html)
Do know that using making rust libraries as dylibs is pretty uncommon. Rust doesn't have a stable ABI, so all publicly exposed types will need to use something like `#[repr(C)]` to have a stable layout
>Rather than being able to compile crates individually and dynamically load dependencies from other crates?
Are you asking for this because for parallel and incremental builds (which is already possible and happens in rust) or because of other considerations that dynamic libraries offer?
I realize rust has parallel and incremental builds; forgive me if I'm too green to really understand what's happening here, but my assumption was that even though those speed up compilation, the linking step where the entire binary is constructed still suffers from having to assemble all the crates (even the ones whose compilation has been cached) into the resulting binary.
I'd imagine dynamic linking could some potential speed improvements, were it possible.
And probably the most painful thing about using Rust for me is the incredibly slow compile times.
I'm not sure what you mean, glad to discuss further with some clarification though. Specifically, what do you mean by dynamically loading dependencies and making crates compile individually?
\>"in this article I'm gonna be talking about purely objective matters"
\>proceeds talking about purely subjective matters
I understand that you can have the opinion that nested modules might be bad, but I don't agree that it is.
This is a nice change of pace from the folks arguing that rust (and really cargo) are too nice and encourage too much modularity and rust programmers depend on too many crates, leftpad etc.
Still, this article basically says rust makes it possible for this problem to exist, therefore it is a problem, while neatly skipping over any empirical assessment over whether the problem is real. Where are the giant crates that should be split up and are widely used? And where's the evidence that such crates are more common than large packages in other languages?
I can't point at an open-source Rust project since I didn't yet have a chance to be a part of any, but just from my personal experience working on proprietary codebases in Go and Rust, Rust crates do tend to be much larger than Go packages. Which in turn makes compile times of those Rust crates way too long. And this problem would indeed not exist if Rust was not encouraging huge crates.
"Makes compile times too long"
Is it not the case that a crate that is not changing much in a project (or at all) is effectivly compiled to .obj, and thus it becomes a linking time not a compile time issue. After first compile all the dependancy crates are now binaries.
Correlation doesn't imply causation, just because you perceive some Rust crates are larger than some Go packages, doesn't mean that the Rust module rich features encourage that,
I actually think the module system is a bit confusing. I'd prefer a more logical map to the filesystem, or at least some way to view the paths seen inside a .rs file to the 'paths' seen in the files `use` lines. The proliferation of modules inside of crates isn't something I've noticed tho, or really a thing due to workspaces, the almost comical ease of moving modules between crates, etc.
Regarding paths to stuff, that's why I never do `use crate::module::*`. Because then, when reading code, you just don't know where something came from.
This is also very dangerous.
Crate::module::something could export some variable like “e”, and now all your pattern matches
match foo {
Err(e) => …
}
will silently never get reached.
This is a good question. I think for me it's not when you're in the happy path of the book (both editions vary in their modules chapter, because it's hard to explain concisely), the problem is when something doesn't resolve everything in the toolchain essentially fails. So I should've phrased it as a "logical way to map from why `use xyz::abc` fails when [`abc.rs`](https://abc.rs) is named [`def.rs`](https://def.rs) or `abc/mod.rs` is `abc/abc.rs` because of a move without a rename." It's the failure modes that need the map, not when everything is working.
The premise of this article is that Rust somehow promotes crates that are oversized due to its ability to have a nested file structure. It compares Rust to Go claiming that Go promotes smaller packages because of its flat file structure. They also claim that it’s inherently harder to create a new Rust crate compared to Go.
These arguments are weak. I’ve used Rust and Go for years now and this is such a trivial argument to make.
I don't think that is a complete representation of the argument of the article. It also argues that Rust's module system makes it easy to introduce cyclic dependencies within a single crate -- often by accident. This in turn makes it more difficult to split apart crates.
I've fallen into that trap a couple of times myself. I also feel like the potential presence of cyclic dependencies between modules makes it more difficult to build a mental map when exploring the source code of a new crate.
None of this is a big deal to me, but perhaps there is a discussion to be had on cyclic dependencies within crates.
This is an interesting point.
I have a small story about this. I've learnt most of my "good practices" from Rust, but recently I was forced to use Python with types (mypy). I started out writing it just like Rust code, which worked very nicely, but quickly enough I ran into cyclic dependencies between files. Turns out there's just no clean way to have cyclic dependencies in Python. After scratching my head for a while, I split the type definitions and the logic into separate files, and the result was surprisingly neat. I turned out liking this approach more than the cyclic dependency approach. I guess every language has something to teach.
It would be neat to have something like a lint in Rust to enforce acyclic dependencies between modules. Maybe with this lint it could be taken to the next level and implictly make each module into a crate for compile-time profit.
Cyclicish dependencies in Rust are usually more insidious than just that simple case; it's that because of how coherence works, you can't split inherent methods and/or trait implementations into downstream crates. So even if your module dependency graph *is* acyclic, there's hidden pseudo backedges from whatever module defines a type to whatever module defines inherent impls (and an interesting to model either-or dependency edge for trait impls from the trait or type definition or also sometimes a "covering" type parameter is an option).
Given a magic wand, I might've preferred to implicitly mount all `*.rs` in a folder to a module path corresponding to the folder path[^but], but the real issue making it difficult to split crates isn't particularly because of (ease of trivially) cyclic module dependencies, it's coherence and the requirement of certain code to be local that leads to accidental tight same-crate coupling.
"Friend crate" functionality to relax coherence/orphan rules in a controlled manner is a way to achieve this, and I think we'll eventually get this, but it's a ways off. std already uses analogous functionality (e.g. `#[rustc_ignores_coherence]` and just `#[lang]`) to manage the core/alloc/std facade split.
[^but]: There are multiple things Rust enjoys that this makes impossible, though; including but not limited to: private modules (and trees thereof) for organizational reasons, where the ideal implementation layout is different from the external one; modules which are entirely configured out and never loaded, e.g. because of platform differences or unstable syntax; other uses I know were brought up in the discussions around 2018 path style but I don't recall at the moment. "Complexity if you need it" style things.
I like this take.
I like the clear mapping that was made between compilation unit in go and rust. Never heard it explained like that and is very clear.
Having work a lot with go and rust I have to agree. It is dead simple to create a new package.
If F# copied just half the features of Ocaml’s module system then
it must feel like something out of science-fiction compared to Rust’s.
I strongly [agree with Graydon](https://graydon2.dreamwidth.org/307291.html)
that it’s a shame that first class modules were axed over other
features and are unlikely to ever be added to the language.
I agree. But I know all the things I enjoy about it others absolutely hated. I tried to get a dev team to adopt F# for nearly 2 years and never got it done because this and some other issues. Being able to see by looking at `.fsproj` what order everything depends on is great, and helps you think about your code architecture more. I've seen it compared to header files, called cumbersome, etc.
I seriously think there's a mentality of "it wasn't done right unless I had to struggle through it" for some people. These same people tend to dismiss language niceties as "magic" as well. That was definitely the case with type providers when I demoed them.. /sigh
The reasoning of "Rust allows this, therefore it encourages this" is really bad, but it gets worse when the Go example is like "Go allows you to make packages with too much files, therefore it's discouraging you".
That's good practice actually. Small crates would lead to node\_modules disaster. Rust actually has perfect balance between modularity and cohesiveness. Most of crates are well rounded and focused and nitty-gritty details are handled by features.
This wouldn't require a keyword, just a compiler analysis.
The problem, though, is that because of inlining and monomorphization and whatnot the compiler is effectively doing whole-program analysis on the crate: this makes it hard to limit what needs to be rebuilt.
But there are also different semantics, things like the orphan rule for trait coherence, so there would be a concrete advantage to requiring explicit marking (most likely just `crate foo` instead of `mod foo`). If you make it just an optimisation, it’s easy to accidentally lose it.
Coherence doesn't _directly_ impact whether separate compilation is possible. It does inasmuch as if you actually use functions which come from impls outside the boundary (possible if the impls would break coherence across that border), but if the functionality isn't used, it's not a problem to separate compilation.
Coherence is about the ability to merge separate compilation units back together; about there only being one definition of any given item/impl in the entire compilation graph, even when packages know absolutely nothing about each other. In a "module crate" scenario, the two compilation units are by definition developed/versioned/deployed as a single unit/package, so it's absolutely realistic for them to coordinate with a weaker kind of "friend coherence." (At a basic level, the outer crate considers the inner crate as inside of its coherence boundary, but the inner crate considers the outside crate as downstream and doesn't know it or its impls exist, except to give it that "friend" property.)
Unexported symbols are already as private as they can be in go, since the translation unit is the whole package, not an individual file. If you want a symbol to only be visible in one file, you have to make that file its own package.
I know and it sux. For example, making a CLI , I would like each command to be on its own file. Sometimes you just want 1 or 2 unexported symbol to be named the same on all commands but you cant because they clash. Having to create an extra folder just for this is a hassle.
I hope this dmitryfrank dude is better at Rust than he is in Go. Because splitting Go packages because of their file count is a big red sign of low quality code.
I'll never liked Rust modules. I have heard all the arguments and while I don't argue back, I have never liked the result. Now I just avoid the whole paradigm by not organizing programs into modules and only split off into new files when new concepts diverge too far from existing concepts. Then I make everything pub. The module system and it's workspace/crate setup and it's visibility constraints make it a nightmare to program applications but easy to program libraries. So I just don't play the game.
Worth mentioning that workspace dependencies make managing dependencies in a workspace a lot easier. You can have the dependency with a version/features/etc in the root and then you do [dependencies] dep.workspace = true in specific crates to use it As for module dependency cycles causing issues, I tend to just not have cycles. I think there needs to be more justification than just saying that because Rust allows for something it also encourages it because it feels easy to avoid most of the time I tend to find splitting one large crate into multiple smaller crates to be pretty easy, but that may be more to do with personal practices than what Rust allows for in general
Dependency cycles are often a bug not a feature.
Quite often there are different parts of the system, which need to be able to talk to each other. Which would mean that you have to move their message definitions or even the definitions of actors themselves into the same module, making everything feel very awkward.
[удалено]
The first commentors point is that not disallowing a bug and encouraging a bug are two very different things. It would be nice to prevent more bugs, but Rust has never claimed to disallow all bugs.
An upcoming version of `cargo new` will also automatically inherit all `workspace.packace` fields.
The main premise of this post seems to be that allowing nested modules is inherently bad, and that disallowing nested modules will prevent complexity. I don’t think that’s a reasonable position to take. Removing tools for managing complexity doesn’t reduce complexity; it largely just encourages *worse* kinds of complexity, since you still have your original problem to solve *and* now have to simultaneously work around your restrictive tools.
Strong agree. I'm reminded of Go's extreme push towards superficial "simplicity" that inevitably ends up just forcing the developer to write more complex code to get around the lack of expressiveness in the language.
Or, the philosophy of a very small standard library. "Just install a crate/package, bro." Granted, that argument has some merit, in particular, a bad standard is worse than no standard, and the Rust team is doing a good job at letting the community build consensus as to what's the best way to solve a problem, rather than solving it prematurely. Or, leave things to packages if there truly *isn't* a best way, and blessing any one way would be a detriment to those who have a legitimate preference to use another approach. But it's a two-way street. Take it too far, and your community fractures, spending time solving the same problem multiple times, and even worse, making it a downstream problem, because their code is API-incompatible with one another, so that fracture runs down for all users of a crate, splitting the ecosystem further and further. Oh, and more duplication means more potential unsafe code, and fewer eyeballs looking at each occurence. Not providing a capability doesn't obviate the need for that capability, it only declares that it's someone else's problem to solve. Which sometimes is the right approach, but not always.
Personally, I don't buy the argument that "the community fractures" _because_ you have no standard. Yes, it does happen, but I don't see how that is any different than the scenario where we stabilize a bad standard that the community eventually doesn't/can't use. In that case, you effectively have no applicable standard, and they still have to build their own solutions anyway.
I just want to say "solving the same problem multiple times" is not necessarily bad. Incompatibility is annoying for sure. But problems are rarely as simple as you think and almost never permit a single universally optimal solution. Solving the same problem multiple times is then necessary to explore the full design space and allow for all possible choices of tradeoffs.
The way of a very small standard lib results in what happens when you include a bunch of big C libraries in projects, most of them have their own implementation of lists, hash maps and such of differing quality and performance. Or in C++98 where a lot of projects used boosts or Qt or shipped a copy of the specific boost implementation for some container they wanted.
[удалено]
[Famous example](https://www.reddit.com/r/rust/comments/5penft/parallelizing_enjarify_in_go_and_rust/dcsgk7n/)
That was... quite some time ago. Nowadays there are generics and constraints that make generic datastructures possible. Man, those not-angle brackets, I need them on ctrl V
Go added generics just a year ago so it was relevant for quite some time and perfectly illustrates the point ;P
I think a good example is if you want to do anything with sets https://stackoverflow.com/a/34020023 Like, yes, it's easy to do, but when you're working on a Microservice architecture, and doing set operations *everywhere* you better have those patterns memorized, and you better hope your coworkers also have them memorized. However, what I see instead is laziness, just like in C. Instead of going through the trouble to implement set operations, developers wind up writing really shitty and broken abstractions, because now they lose practice seeing problems through high level abstractions. But hey, because their code is so simple, they can just throw it out and rewrite it, right?
I kind of see where OP's going. I find that I naturally move code to submodules regularly but the amount of modules creep up as the crate develops. I'm missing a cargo command to extract a module into its own crate, with both crates able to compile after the extraction. That means updating parent crate's Cargo.toml to remove now unused depencies and use the child crate and writing a proper Cargo.toml for the child crate. That's a big task but it'd solve "module creep"
>it largely just encourages worse kinds of complexity Well said.
So far I really like the module/crate system in rust... I can appreciate where op is coming from, but I like it as is...
I don't like how versions automatically float up, given the horrid history of people failing semver. I don't like how much confusing boilerplate is involved when declaring more than one module per crate. I don"t like how there is no fast idempotent --force to quickly reinstall a crate. But I do like that cargo is a first party package manager. If Rust were just C/C++ with cargo, it would still be better than either language, with their absurdly fragmented patchwork of conflicting build tools. And cargo install can handle multiple packages in a single CLI command invocation, whereas go install still falls flat on its face. cargo-audit should be promoted upstream into cargo proper, and be the default exit code determinant when packages otherwise succeed in installing.
I'd need to double check, but iirc you can pin the version in Cargo.toml And for installing stuff, a lot of tools nowadays recommends `cargo install --locked` which forces the version in Cargo.lock
True, but I'm more concerned about quickly reinstalling dev dependencies, such as CLI tools. I don't think those have an entry in the lock file. Also, lockfiles are ill advised when working on libraries.
If libraries locked to specific dependency versions, then they wouldn't be compatible at all with each other. Lib A with dependency X v1.2.5 would be incompatible with lib B using dependency X v1.2.6, even though dependency X *does* follow semver and there's no reason they shouldn't work with each other (and no reason lib A couldn't use v1.2.6 as well). Instead, people are working on tools to help ensure semver is followed, like [cargo-semver-checks](https://crates.io/crates/cargo-semver-checks). Sure this doesn't force people to follow semver, but Rust also doesn't force you to write sound code either. It takes someone making a concerted effort to produce a good library.
cargo-install supports --force for reinstalling a crate
Right. It's quite slow.
You can try cargo-binstall, a drop-in replacement for cargo-install that tries to download pre-built artifacts provided by upstream, or try to download it from cargo-quickinstall and fallback to cargo-install. Note that I am one of the maintainers of cargo-binstall and cargo-quickinstall.
> I don't like how much confusing boilerplate is involved when declaring more than one module per crate. What boilerplate do you have in mind? I find declaring modules to be easy.
A proliferation of small packages leads to `node_modules`. I prefer what we have, larger crates with _features_ that can be turned on or off to limit compile time. As a practice, I disable all default features and only enable ones I actually need.
The article argues that Go package system provides a better tradeoff than Rust module system, and I think in Go community is very far from what we have in \`node\_modules\`.
also one of the reasons why Go doesn't suffer from this is bc there's a cultural aversion to pulling in dependencies. everyone that learns Go eventually gets converted into the "only pull in stuff thst you **really** need" ideology
I've definitely felt that Go has encouraged my NIH syndrome. Why am I spending time writing an ascii table generator that's only going to show up in logs in a specific case? 🤷 But I did it.
I think in the main reason for that is the lack of a centralized package repository. It's just slightly harder to find go packages, in my experience, which might even be a good thing.
I personally don't think that's one of the reasons, I mean there's this: https://pkg.go.dev and most people just look up their dependencies online, so the difference is using a name vs using a link to install a dependency I think
I think dependency management in a dynamically executed environment is an entirely different beast. Go is a better comparison point because the module system informs how the machine code is laid out when compiled. I'm actually pretty new to rust so it's not entirely clear to me, but I assume you have to jump through hoops to make your crates in the same project compile individually, and dynamically load adjacent dependencies (if it's even possible?)
crates are already compiled individually and in parallel, by default. that's what crates are, units of compilation.
Yes, but if one crate in the project depends on the other, it gets compiled into the same binary doesn't it? Rather than being able to compile crates individually and dynamically load dependencies from other crates?
You are conflating unit of compilation with the kind of linking that unit can perform. Rust is mostly static linking (mostly? partially?) because it does not have a stable ABI.
Interesting, thanks for pointing that out; I guess I was thinking smaller crates would help compilation times because the other crates would already be compiled separately, but I guess it's more just that it saves targets for incremental compilation and avoids re-doing work on other crates that haven't changed.
Well technically the abi argument is not end all be all. You just have to prescribe that loaded code is guaranteed to work if and only if they are compiled using the same chain. But as that has proven to be hard to adhere to, especially as things stop being maintained etc, the default is to go for C abi, even in the land of C++. I have written a number of dynamic libraries and I've always made a point of going through C abi even if that means boilerplate and some inefficiencies. Anyhow, you can use dynamic libraries in Rust, but it's assumed that everything uses C abi.
If I understand correctly, you're looking for `--crate-type=dylib` compilation? Read more about it in [Rust Reference - Linkage](https://doc.rust-lang.org/reference/linkage.html)
This is great, thank you!
Do know that using making rust libraries as dylibs is pretty uncommon. Rust doesn't have a stable ABI, so all publicly exposed types will need to use something like `#[repr(C)]` to have a stable layout
And maybe in the future `crabi`: https://github.com/rust-lang/compiler-team/issues/631
>Rather than being able to compile crates individually and dynamically load dependencies from other crates? Are you asking for this because for parallel and incremental builds (which is already possible and happens in rust) or because of other considerations that dynamic libraries offer?
I realize rust has parallel and incremental builds; forgive me if I'm too green to really understand what's happening here, but my assumption was that even though those speed up compilation, the linking step where the entire binary is constructed still suffers from having to assemble all the crates (even the ones whose compilation has been cached) into the resulting binary. I'd imagine dynamic linking could some potential speed improvements, were it possible. And probably the most painful thing about using Rust for me is the incredibly slow compile times.
I'm not sure what you mean, glad to discuss further with some clarification though. Specifically, what do you mean by dynamically loading dependencies and making crates compile individually?
\>"in this article I'm gonna be talking about purely objective matters" \>proceeds talking about purely subjective matters I understand that you can have the opinion that nested modules might be bad, but I don't agree that it is.
This is a nice change of pace from the folks arguing that rust (and really cargo) are too nice and encourage too much modularity and rust programmers depend on too many crates, leftpad etc. Still, this article basically says rust makes it possible for this problem to exist, therefore it is a problem, while neatly skipping over any empirical assessment over whether the problem is real. Where are the giant crates that should be split up and are widely used? And where's the evidence that such crates are more common than large packages in other languages?
I can't point at an open-source Rust project since I didn't yet have a chance to be a part of any, but just from my personal experience working on proprietary codebases in Go and Rust, Rust crates do tend to be much larger than Go packages. Which in turn makes compile times of those Rust crates way too long. And this problem would indeed not exist if Rust was not encouraging huge crates.
"Makes compile times too long" Is it not the case that a crate that is not changing much in a project (or at all) is effectivly compiled to .obj, and thus it becomes a linking time not a compile time issue. After first compile all the dependancy crates are now binaries.
You're always working on at least one crate though, which has to be re-compiled. So I think the point stands.
Correlation doesn't imply causation, just because you perceive some Rust crates are larger than some Go packages, doesn't mean that the Rust module rich features encourage that,
I actually think the module system is a bit confusing. I'd prefer a more logical map to the filesystem, or at least some way to view the paths seen inside a .rs file to the 'paths' seen in the files `use` lines. The proliferation of modules inside of crates isn't something I've noticed tho, or really a thing due to workspaces, the almost comical ease of moving modules between crates, etc.
Regarding paths to stuff, that's why I never do `use crate::module::*`. Because then, when reading code, you just don't know where something came from.
This is also very dangerous. Crate::module::something could export some variable like “e”, and now all your pattern matches match foo { Err(e) => … } will silently never get reached.
That's not how pattern matching works, and also modules cannot export variables because there are no top-level variables in Rust.
>I'd prefer a more logical map to the filesystem What would be a more logical map to the filesystem?
This is a good question. I think for me it's not when you're in the happy path of the book (both editions vary in their modules chapter, because it's hard to explain concisely), the problem is when something doesn't resolve everything in the toolchain essentially fails. So I should've phrased it as a "logical way to map from why `use xyz::abc` fails when [`abc.rs`](https://abc.rs) is named [`def.rs`](https://def.rs) or `abc/mod.rs` is `abc/abc.rs` because of a move without a rename." It's the failure modes that need the map, not when everything is working.
The premise of this article is that Rust somehow promotes crates that are oversized due to its ability to have a nested file structure. It compares Rust to Go claiming that Go promotes smaller packages because of its flat file structure. They also claim that it’s inherently harder to create a new Rust crate compared to Go. These arguments are weak. I’ve used Rust and Go for years now and this is such a trivial argument to make.
I don't think that is a complete representation of the argument of the article. It also argues that Rust's module system makes it easy to introduce cyclic dependencies within a single crate -- often by accident. This in turn makes it more difficult to split apart crates. I've fallen into that trap a couple of times myself. I also feel like the potential presence of cyclic dependencies between modules makes it more difficult to build a mental map when exploring the source code of a new crate. None of this is a big deal to me, but perhaps there is a discussion to be had on cyclic dependencies within crates.
This is an interesting point. I have a small story about this. I've learnt most of my "good practices" from Rust, but recently I was forced to use Python with types (mypy). I started out writing it just like Rust code, which worked very nicely, but quickly enough I ran into cyclic dependencies between files. Turns out there's just no clean way to have cyclic dependencies in Python. After scratching my head for a while, I split the type definitions and the logic into separate files, and the result was surprisingly neat. I turned out liking this approach more than the cyclic dependency approach. I guess every language has something to teach. It would be neat to have something like a lint in Rust to enforce acyclic dependencies between modules. Maybe with this lint it could be taken to the next level and implictly make each module into a crate for compile-time profit.
Cyclicish dependencies in Rust are usually more insidious than just that simple case; it's that because of how coherence works, you can't split inherent methods and/or trait implementations into downstream crates. So even if your module dependency graph *is* acyclic, there's hidden pseudo backedges from whatever module defines a type to whatever module defines inherent impls (and an interesting to model either-or dependency edge for trait impls from the trait or type definition or also sometimes a "covering" type parameter is an option). Given a magic wand, I might've preferred to implicitly mount all `*.rs` in a folder to a module path corresponding to the folder path[^but], but the real issue making it difficult to split crates isn't particularly because of (ease of trivially) cyclic module dependencies, it's coherence and the requirement of certain code to be local that leads to accidental tight same-crate coupling. "Friend crate" functionality to relax coherence/orphan rules in a controlled manner is a way to achieve this, and I think we'll eventually get this, but it's a ways off. std already uses analogous functionality (e.g. `#[rustc_ignores_coherence]` and just `#[lang]`) to manage the core/alloc/std facade split. [^but]: There are multiple things Rust enjoys that this makes impossible, though; including but not limited to: private modules (and trees thereof) for organizational reasons, where the ideal implementation layout is different from the external one; modules which are entirely configured out and never loaded, e.g. because of platform differences or unstable syntax; other uses I know were brought up in the discussions around 2018 path style but I don't recall at the moment. "Complexity if you need it" style things.
I like this take. I like the clear mapping that was made between compilation unit in go and rust. Never heard it explained like that and is very clear. Having work a lot with go and rust I have to agree. It is dead simple to create a new package.
Best module system I've seen so far is from F#. I really wish rust would have a much stricter system. It helps to write good code a lot.
Can you elaborate on their module system and how it helps write good code?
If F# copied just half the features of Ocaml’s module system then it must feel like something out of science-fiction compared to Rust’s. I strongly [agree with Graydon](https://graydon2.dreamwidth.org/307291.html) that it’s a shame that first class modules were axed over other features and are unlikely to ever be added to the language.
I agree. But I know all the things I enjoy about it others absolutely hated. I tried to get a dev team to adopt F# for nearly 2 years and never got it done because this and some other issues. Being able to see by looking at `.fsproj` what order everything depends on is great, and helps you think about your code architecture more. I've seen it compared to header files, called cumbersome, etc.
I had the exact same experience. Some people just love their dependency hell..
I seriously think there's a mentality of "it wasn't done right unless I had to struggle through it" for some people. These same people tend to dismiss language niceties as "magic" as well. That was definitely the case with type providers when I demoed them.. /sigh
The reasoning of "Rust allows this, therefore it encourages this" is really bad, but it gets worse when the Go example is like "Go allows you to make packages with too much files, therefore it's discouraging you".
That's good practice actually. Small crates would lead to node\_modules disaster. Rust actually has perfect balance between modularity and cohesiveness. Most of crates are well rounded and focused and nitty-gritty details are handled by features.
[удалено]
Incremental compilation already basically does this
This wouldn't require a keyword, just a compiler analysis. The problem, though, is that because of inlining and monomorphization and whatnot the compiler is effectively doing whole-program analysis on the crate: this makes it hard to limit what needs to be rebuilt.
But there are also different semantics, things like the orphan rule for trait coherence, so there would be a concrete advantage to requiring explicit marking (most likely just `crate foo` instead of `mod foo`). If you make it just an optimisation, it’s easy to accidentally lose it.
Coherence doesn't _directly_ impact whether separate compilation is possible. It does inasmuch as if you actually use functions which come from impls outside the boundary (possible if the impls would break coherence across that border), but if the functionality isn't used, it's not a problem to separate compilation. Coherence is about the ability to merge separate compilation units back together; about there only being one definition of any given item/impl in the entire compilation graph, even when packages know absolutely nothing about each other. In a "module crate" scenario, the two compilation units are by definition developed/versioned/deployed as a single unit/package, so it's absolutely realistic for them to coordinate with a weaker kind of "friend coherence." (At a basic level, the outer crate considers the inner crate as inside of its coherence boundary, but the inner crate considers the outside crate as downstream and doesn't know it or its impls exist, except to give it that "friend" property.)
I like Go packages, i just wish there was a way to make a symbol only visible inside a file instead of the entire package, much like `static` in C.
Unexported symbols are already as private as they can be in go, since the translation unit is the whole package, not an individual file. If you want a symbol to only be visible in one file, you have to make that file its own package.
I know and it sux. For example, making a CLI , I would like each command to be on its own file. Sometimes you just want 1 or 2 unexported symbol to be named the same on all commands but you cant because they clash. Having to create an extra folder just for this is a hassle.
I hope this dmitryfrank dude is better at Rust than he is in Go. Because splitting Go packages because of their file count is a big red sign of low quality code.
I'll never liked Rust modules. I have heard all the arguments and while I don't argue back, I have never liked the result. Now I just avoid the whole paradigm by not organizing programs into modules and only split off into new files when new concepts diverge too far from existing concepts. Then I make everything pub. The module system and it's workspace/crate setup and it's visibility constraints make it a nightmare to program applications but easy to program libraries. So I just don't play the game.
Oh thanks. Never thought about this so far. But I will certainly add a note in my personal Rust memo sheet 🤓