Greg K-H: "Writing new code in Rust is a win for all of us"
Linus response here seems relevant to this context: https://lore.kernel.org/rust-for-linux/CAHk-=wgLbz1Bm8QhmJ4d...
The impression I get from simply reading these various discussions, is that some folks are not convinced that the pain from accepting Rust is worth the gain.
Possibly also that a significant portion of the suggested gain may be achievable via other means.
i.e. bounds checking and some simple (RAII-like) allocation/freeing simplifications may be possible without rust, and that those are (from the various papers arguing for Rust / memory safety elsewhere) the larger proportion of the safety bugs which Rust catches.
Possibly just making clang the required compiler, and adopting these extension may give an easier bang-for-buck: https://clang.llvm.org/docs/BoundsSafety.html
Over and above that, there seem to be various complaints about the readability and aesthetics of Rust code, and a desire not to be subjected to such.
In my honest opinion, it's not a good idea to mix two programming languages into the same monolithic codebase side by side. It would be less problematic if used for different purposes or layers, like frontend and backend. But we know it still creates unpleasant friction when you have to work on both sides on your own. Otherwise, it creates technical AND communication friction if the C devs and Rust devs work separately. As someone who works with embedded systems at times, I can imagine the pain that you have to set up two toolchains (with vastly different build infra beasts like GNU Make and Cargo) and the prolonged build time of CI and edit-compile-run debugging cycles given the notorious slow compile time of the Rust/LLVM compiler.
Regarding the Linux development process: How do Linux maintainers / contributors have time to read these long threads of long posts? Just this one discussion looks like it would take hours to read and these are busy developers.
How does it work? Are there only a few threads that they read? Which ones?
But for new code / drivers, writing them in Rust where these types of bugs just can't happen (or happen much much less) is a win for all of us, why wouldn't we do this? -- greg k-h
We should have seen this post before Hector Martin got so fed up that he decided to resign(to be fair, he probably had other issues as well that contributed).
I was very confused by the lack of an actual response from Linus, he only said that social media brigading is bad, but he didn't give clarity on what would be the way forward on that DMA issue.
I have worked in a similar situation and it was the worst experience of my work life. Being stonewalled is incredibly painful and having weak ambiguous leadership enhances that pain.
If I were a R4L developer, I would stop contributing until Linus codifies the rules around Rust that all maintainers would have to adhere to because it's incredibly frustrating to put a lot of effort into something and to be shut down with no technical justification.
I do not understand how this is supposed to work in practice. If there are "Rust bindings" then the kernel cannot have a freely evolving internal ABI, and the project is doomed to effectively split into the "C" core side and the "Rust" side which is more client oriented. Maybe it will be a net win for Linux for finally stabilizing the internal APIs, and even open the door to other languages and out-of-tree modules. On the other hand, if there are no "Rust bindings" then Rust brings very little to the table.
Community and people are the main issue.
If the people who work on the kernel now don't like that direction then that's a big problem.
The Linux leadership don't seem very focused on the people issues.
Where is the evidence that there is buy in from the actual people doing kernel development now?
Or is it just Linus and Greg as commanders saying "thou shalt".
> The majority of bugs (quantity, not quality/severity) we have are due to the stupid little corner cases in C .... Things like simple overwrites of memory (not that rust can catch all of these by far), error path cleanups, forgetting to check error values, and use-after-free mistakes.
What's the reach here of linters/address san/valgrind?
Or a linter written specifically for the linux kernel? Require (error-path) tests? It feels excessive to plug another language if these are the main arguments? Are there any other arguments for using Rust?
And even without any extra tools to guard against common mistakes, how much effort is solving those bug fixes anyway? Is it an order of magnitude larger than the cognitive load of learning a (not so easy!) language and context-switching continuously between them?
These days, the bugs I generate in my own code are rarely programming errors. They're misunderstandings of the problem I am trying to solve, or misunderstandings of how to fit it into the rest of the (very complex) code.
For example, I cannot even recall the last time I had a double-free bug, though I used to do it often enough.
The emphasis for me is on a language that makes it easy to express algorithms.
This statement was sorely needed for this discussion to move forward. Hopefully the last section fills the needed parties with resolve
It's really all opinions what is better or worse, but i do respect the sentiment that there is some boundary, and on one side of the boundary, Rust makes a lot of sense, and the other side, Rust does not work at all. (managing global mutable resources). It weirds me out a bit there is even such discussions going on in projects like this. It seems obvious and proven at this point and if not that then atleast it should be obvious already for a long time that if you program within some large codebase or ecosystem, you are not the only voice, and you need to learn to collaborate with people with different views as you and make it work.
I really don't like rust, hence instead of wanting to contribute to projects which will inadvertently lead to more and more rust code being brought in, i start my own projects, when i can be the only voice of reason and have my joys of making things segfault :>... Its quite simple. If like me you are stuobborn and unflexible, you are a lone wolf. accept it and move on to be happy :) rather than trying to piss against the wind of change.
The actual project is "lets modernize the internal kernel api surface", and "how tolerable is it to write against this api in rust" is just the best metric at hand to measure the progress.
This is the correct frame for RFL proponents. You're welcome.
I wonder how Microsoft implements rust in their kernel.
As for this issue, it's just a nature of any project, people will come and go regardless, so why not let those C developers leave and keep the rust folks instead? At some point you have to steer the ship and there will always be a group of people unhappy about the course
> the C++ language committee issues seem to be pointing out that everyone better be abandoning that language as soon as possible
What is he referring to?
It's really disappointing to me to see a lot of the negative reactions and comments here. I know it's popular and in vogue now to hate on Rust, but:
Influential people who have worked on the ins and outs of the Linux kernel for years and decades believe that adopting Rust (or at least keeping the Rust experiment going) is worth the pain it will cause.
That's really all that matters. I see people commenting here about how they think RAII isn't suitable for kernel code, or how keeping C and Rust interfaces in sync will slow down important refactoring and changes, or how they think it's unacceptable that some random tiny-usage architecture that Rust/LLVM doesn't support will be left behind, or... whatever.
So what! I'm not a Linux kernel developer or maintainer, and I suspect most (if not all) of the people griping here aren't either. What does it matter to you if Linux adopts Rust? Your life will not be impacted in any way. All that matters is what the maintainers think. They think this is a worthwhile way to spend their time. The people putting in the work get to decide.
> The majority of bugs (quantity, not quality/severity) we have are due to the stupid little corner cases in C that are totally gone in Rust. Things like simple overwrites of memory (not that rust can catch all of these by far), error path cleanups, forgetting to check error values, and use-after-free mistakes. That's why I'm wanting to see Rust get into the kernel, these types of issues just go away, allowing developers and maintainers more time to focus on the REAL bugs that happen (i.e. logic issues, race conditions, etc.)
C committee, are you listening? Hello? Hello? Bueller?
(Unfortunately, if they are listening it is to make more changes on how compilers should take "creative licenses" in making developers shoot themselves in the foot)
Seems to me that everyone is focused on the technical merits, not weighing the effort of learning a new programming language/toolchain/ecosystem for the maintainers appropriately.
Mastering a new programming language to a degree that makes one a competent maintainer is nothing to sneeze at and some maintainers might be unwilling to based on personal interests/motivation, which I'd consider legitimate position.
I think its important to acknowledge that not everyone may feel comfortable talking about their lack of competence/disinterest.
"Rust also gives us the ability to define our in-kernel apis in ways that make them almost impossible to get wrong when using them. We have way too many difficult/tricky apis that require way too much maintainer review just to "ensure that you got this right" that is a combination of both how our apis have evolved over the years"
Funny, that's not Theodore T'so's position. The Rust guys tried to ask about interface semantics and he yelled at them:
undefined
Prediction 2030: Linus Retires and C++ accepted as the primary language for writing the kernel.
Inadvertently, Rust makes working with C++ acceptable.
It’s hard. Most people agree it should have memory safety, but also I’m not looking to become a full scale maintainer either.
I've been using Linux since 2005, and I've loved it in almost every circumstance. But the drama over the last couple of years surrounding Rust in the kernel has really soured me on it, and I'm now very pessimistic about its future. But I think beyond the emotional outbursts of various personalities, I don't think that the problem is which side is "right". Both sides have extremely valid points. I don't think the problem is actually solvable, because managing a 40M+ SLoC codebase is barely tenable in general, and super duper untenable for something that we rely on for security while running in ring 0.
My best hope is for replacement. I think we've finally hit the ceiling of where monolithic kernels can take us. The Linux kernel will continue to make extremely slow progress while it deals with internal politics fighting against an architecture that can only get bigger and less secure over time.
But what could be the replacement? There's a handful of fairly mature microkernels out there, each with extremely immature userspaces. There doesn't seem to be any concerted efforts behind any of them. I have a lot of hope for SeL4, but progress there seems to be slow mostly because the security model has poor ergonomics. I'd love to see some sort of breakout here.
"he C++ language committee issues seem to be pointing out that everyone better be abandoning that language as soon as possible if they wish to have any codebase that can be maintained for any length of time."
I'd love to know where he got this impression. The new C++ features go a long way to helping make the language easier, and safer, to use.
This might be a silly question, but why don't we have something like PR Gate pipelines that ensures it passes before being picked up by a maintainer?
undefined
perhaps someone can point me to a link where i can get information WHY it is so hard to call C from Rust or call into Rust code from C So i do not get the talk because i do not understand the issue.
Pasting the entire thing so people on mobile can read (at least on iPhone readability doesn’t work here:
As someone who has seen almost EVERY kernel bugfix and security issue for the past 15+ years (well hopefully all of them end up in the stable trees, we do miss some at times when maintainers/developers forget to mark them as bugfixes), and who sees EVERY kernel CVE issued, I think I can speak on this topic.
The majority of bugs (quantity, not quality/severity) we have are due to the stupid little corner cases in C that are totally gone in Rust. Things like simple overwrites of memory (not that rust can catch all of these by far), error path cleanups, forgetting to check error values, and use-after-free mistakes. That's why I'm wanting to see Rust get into the kernel, these types of issues just go away, allowing developers and maintainers more time to focus on the REAL bugs that happen (i.e. logic issues, race conditions, etc.)
I'm all for moving our C codebase toward making these types of problems impossible to hit, the work that Kees and Gustavo and others are doing here is wonderful and totally needed, we have 30 million lines of C code that isn't going anywhere any year soon. That's a worthy effort and is not going to stop and should not stop no matter what.
But for new code / drivers, writing them in rust where these types of bugs just can't happen (or happen much much less) is a win for all of us, why wouldn't we do this? C++ isn't going to give us any of that any decade soon, and the C++ language committee issues seem to be pointing out that everyone better be abandoning that language as soon as possible if they wish to have any codebase that can be maintained for any length of time.
Rust also gives us the ability to define our in-kernel apis in ways that make them almost impossible to get wrong when using them. We have way too many difficult/tricky apis that require way too much maintainer review just to "ensure that you got this right" that is a combination of both how our apis have evolved over the years (how many different ways can you use a 'struct cdev' in a safe way?) and how C doesn't allow us to express apis in a way that makes them easier/safer to use. Forcing us maintainers of these apis to rethink them is a GOOD thing, as it is causing us to clean them up for EVERYONE, C users included already, making Linux better overall.
And yes, the Rust bindings look like magic to me in places, someone with very little Rust experience, but I'm willing to learn and work with the developers who have stepped up to help out here. To not want to learn and change based on new evidence (see my point about reading every kernel bug we have.)
Rust isn't a "silver bullet" that will solve all of our problems, but it sure will help in a huge number of places, so for new stuff going forward, why wouldn't we want that?
Linux is a tool that everyone else uses to solve their problems, and here we have developers that are saying "hey, our problem is that we want to write code for our hardware that just can't have all of these types of bugs automatically".
Why would we ignore that?
Yes, I understand our overworked maintainer problem (being one of these people myself), but here we have people actually doing the work!
Yes, mixed language codebases are rough, and hard to maintain, but we are kernel developers dammit, we've been maintaining and strengthening Linux for longer than anyone ever thought was going to be possible. We've turned our development model into a well-oiled engineering marvel creating something that no one else has ever been able to accomplish. Adding another language really shouldn't be a problem, we've handled much worse things in the past and we shouldn't give up now on wanting to ensure that our project succeeds for the next 20+ years. We've got to keep pushing forward when confronted with new good ideas, and embrace the people offering to join us in actually doing the work to help make sure that we all succeed together.
thanks,
greg k-h
> > > > > David Howells did a patch set in 2018 (I believe) to clean up the C code in the kernel so it could be compiled with either C or C++; the patchset wasn't particularly big and mostly mechanical in nature, something that would be impossible with Rust. Even without moving away from the common subset of C and C++ we would immediately gain things like type safe linkage.
> > >
> > > That is great, but that does not give you memory safety and everyone
> > > would still need to learn C++.
> >
> > The point is that C++ is a superset of C, and we would use a subset of C++
> > that is more "C+"-style. That is, most changes would occur in header files,
> > especially early on. Since the kernel uses a lot of inlines and macros,
> > the improvements would still affect most of the existing kernel code,
> > something you simply can't do with Rust.
I have yet to see a compelling argument for allowing a completely new language with a completely different compiler and toolchain into the kernel while continuing to bar C++ entirely, when even just a restricted subset could bring safety- and maintainability-enhancing features today, such as RAII, smart pointers, overloadable functions, namespaces, and templates, and do so using the existing GCC toolchain, which supports even recent vintages of C++ (e.g., C++20) on Linux's targeted platforms.
Greg's response:
> But for new code / drivers, writing them in rust where these types of bugs just can't happen (or happen much much less) is a win for all of us, why wouldn't we do this? C++ isn't going to give us any of that any decade soon, and the C++ language committee issues seem to be pointing out that everyone better be abandoning that language as soon as possible if they wish to have any codebase that can be maintained for any length of time.
side-steps this. Even if Rust is "better," it's much easier to address at least some of C's shortcomings with C++, and it can be done without significantly rewriting existing code, sacrificing platform support, or the incorporation of a new toolchain.
For example, as pointed out (and as Greg ignored), the kernel is replete with macros--a poor substitute for genuine generic programming that offers no type safety and the ever-present possibility for unintended side effects due to repeated evaluation of the arguments, e.g.:
#define MAX(x, y) (((x) > (y)) ? (x) : (y))
One need only be bitten by this kind of bug once to have it color your perception of C, permanently.
Christoph Hellwig seems fun to interact with. He drive-by posts the same, repeated points and seemingly refuses to engage with any replies.
[dead]
[dead]
[dead]
[flagged]
No pain, no gain
What's in Rust that creates drama in every project where it's used?
Aren't these people tired of shaving that yak already? I wish they rather focused on making one (1) decent distro for desktop use.
I think it's becoming apparent that any attempt to progressively re-write a large codebase into a new language is always going to fail. Needs to done ground up new.
Rust changes every few months. It's simply not a mature language or people behind it have no idea what they are doing.
Isn't this a bait and switch, that all the c kernel devs were complaining about? That it wouldn't be just drivers but also all new kernel code? The lack of candor over the goal of R4L and downplaying of other potential solutions should give any maintainer (including potential rust ones) pause.
Anyway, why just stop at rust? If we really care about safety, lets drop the act and go make everyone do formal methods. Frama-C is at least C, has a richer contract language, has heavy static analysis tools before having to go to proofs, is much more proven, and the list goes on. Or, why not add Spark to the codebase if we are okay with mixing langs in the codebase? Its very safe.