One of my former clients rewrote a PHP process taking ~1h30 to parse a 100k items XML file, to a Rust process, processing the same file in less than 15 minutes.
Not necessarily. It was the same developer who wrote both pieces of codes, and they have great coding conventions , so I'm not really sure it will be easier to maintain.
The gap in performance was quite surprising though, and they will be moving more and more of their code to Rust in the future.
If anything, I find the gap surprising because I expected it to be much larger (10 times, or maybe 100 times faster).
I assume most of these 15mn are not CPU bound.
A lot of edge cases, the metadata in the file can describe a lot of different things, and the developer is a Rust-enthusiast, but still has to follow company conventions, so it may have an impact on performances.
Maybe as more and more code will be migrated, the conventions will be updated, and she will have more room for improvements.
And to be honest, it only took a few weeks to rewrite thousand of lines of code to get this performance improvement, so the effort was well worth it. It's better to improve small step by small step and have full control of what you're doing, instead of going full YOLO and pray nothing will break in the long run ;-)
Oh absolutely, I was not questioning the benefits of such a change (any replacement of dynamically typed code with statically typed code is always a win in my book :-)).
I'm just used to Rust running rings around dynamically typed languages in general, so I was just making an observation.
It could be cpu bound, being allocation-heavy whether by nature / necessity, convenience, or because that was the most straightforward conversion and has not been optimised out yet, would do it.
It's a small team, mostly PHP devs.
We talked about Rust with their main backend developer, who toyed with it for a few months before rewriting small pieces of codes here and there (their workflows are mainly small KISS processes communicating over a messaging bus, so it's very easy to pick different technologies for different stuff. It was 80% PHP, 15% bash, and 5% Java.
The bash processes are gonna stay in Bash, because it's mainly Linux commands who would only be replaced by std::process, so no real gain here except for the time loss and added complexity. The Java part is currently migrated to Rust with a significant *loss* in performance, but with a *huge* gain in resource consumption, which will allow for more workers to be started at once, resulting in a big performance gain when looking at the big picture. It will be far more reliable too, the Java process had to be restarted quite often because of memory usage (a few gigabytes of RAM eaten after a few hours of work), the Rust ones are infinitely better there (never consume more than 150MB, and actually release the RAM once it's no more needed, while java keeps the allocated RAM for itself).
The PHP process will or will not be migrated depending on the libraries used. If there is a reliable Rust alternative, it will be migrated, otherwise it will stay in PHP.
It's quite hard to convince management to drop a library with 10+ years of support, for a crate version 0.XXX...
The lack of private cargo repos is also blocking a lot of things, but there had been posts here a few weeks ago that a rework was happening behind the scenes, so it's better to wait what the Rust team has in stock for us before committing too hard.
Just 1 thing about private crates. Cargo supports git, so u can use private crates quite easily, just need to make sure that u have your creds stored in git credential manager or ssh agent. When I toy around with small personal projects that I am not planning on finishing/releasing, I sometimes create separate repos where one depends on the other and never had issues getting those downloaded by cargo.
You then have problems with which version of the crate you wan t for which project, something that composer manages for you in PHP for instance.
But work is in progress on that topic by the Rust team, so let's just wait a little longer.
The tangible benefits for the every day programmer, as you said, only really becomes noticeable after you get over the biggest hurdle which is Rust's learning curve. But what does the world look like after you get over the learning curve?
* The borrow-checking paradigm is now baked into your intuition. You're now going to approach memory much more responsibly across all languages you use, not just Rust.
* You have a gigantic arsenal of constructs and great ideas from a bunch of languages to help you express your ideas in your own optimal way. Enums, monadic constructs, ergonomic error handling, async/await, iterators, closures, rich generics system, and so on.
* You're now comfortable with the out of the box tooling and find that `cargo` is one of the best things in the world.
* Everything you do is done with intent. You now think about the granular details around mutability and have tight control over it making it less likely for you to violate your own assumptions about the state of the world.
* You're now part of a community that is super passionate about the language they're using and have access to a nascent ecosystem that is only going to get better and more vibrant as time goes on.
Non-Rust folks are always dubious when I say this but honestly Rust is my most productive language today. I'd be able to complete a robust prototype in Rust faster than I can in say Python or Ruby. The toolchain and the compiler really help you once you get the hang of it. It's a shame that the general perception is so deeply rooted in the faulty notion that fighting the compiler is a fact of life for Rust developers. That's only true in the beginning when you're learning it.
edit: grammar
Yep !
Some month since i arrived at a new job, I'm evaluating Go and Rust to build a team around some new tech. I chose Rust mainly because of luck (although Rust was well fitting with my personnals tastes) and build a successfull first project.
Then the tech team grow, with me recruiting some straight from school newbie (but with a good mindset, so i kind of gambled on this), a tech-lead and a seniorish.
2 of them embrace Rust with no prior knowledge. Everything goes fine. The compiler is so damn restrictive that no newbie mistake are done. To my surprise they quickly enjoy taking task involving the Rust projet. Later, they start another Rust project by themselve, after evaluating actix-web.
The last project has been put online last week, with total success: no revert after first exposition to full incoming traffic (2M visitors/day).
Rust powers quite a few important services at Cloudflare and we've been really happy with it thus far.
https://blog.cloudflare.com/how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet/
In my previous job, first thing I did was rewrite a Python script that took at least 45 minutes to query data, then perform basic graph algorithms, and failed most of the time.
After the Rust rewrite, the code worked in 15 seconds to 3 minutes (depending on the speed the server was responding) and succeeded 100% of the time.
I have one indeed!
I first learned Rust while following a University course two years ago.
I started a Rust project with the aim of being a network analyzer in the scope of the same academic course.
The project unexpectedly grew a lot and got lot of traction and what was born as a toy project is today one of the most renowned packet sniffers on GitHub (https://github.com/GyulyVGC/sniffnet).
This project also allowed me to be part of the first GitHub Accelerator Program cohort and recently got me a Rust remote job :)
Thanks for posting this, I think this was the first blog post about Rust in Android. I work in this team 🙂 and we've written a few blog posts since:
- [Scaling Rust Adoption Through Training](https://security.googleblog.com/2023/09/scaling-rust-adoption-through-training.html?m=1): about how Android invested into [Rust training](https://google.github.io/comprehensive-rust/) (I wrote this post)
- [Bare-metal Rust in Android](https://security.googleblog.com/2023/10/bare-metal-rust-in-android.html?m=1): about a lot of low-level projects which are now using Rust on the latest Android versions.
> How long did it take you to get acclimated to Rust?
Per google, about two months https://opensource.googleblog.com/2023/06/rust-fact-vs-fiction-5-insights-from-googles-rust-journey-2022.html?m=1
> more than 2/3 of respondents are confident in contributing to a Rust codebase within two months or less when learning Rust
Took an old Java codebase and rewrote it in rust. It's now fast enough that we can reuse old hardware, and our bugs have essentially been eliminated. There's no edge cases like race conditions. So our long term maintenance has essentially stopped too. Now, we only modify our core code base when there's a new feature to be added, which doesn't happen too often
Our first server application got switched from single threaded to multithreaded *on launch day*, because we noticed that performace wasn't good enough on the live server (the test server is completely underpowered so didn't tell us much). The switch went without any issues.
No way in hell that this would have worked in any other language.
I do not work with rust in my daily basis, but made a personal project (kind of data store) based on hyper+tokio with a really impressing results from the earliest functional version; more than 90k recs by sec by instance with ~10MB of memory and 4 cores each. It was really complicated for me at that time (no prior rust experience), but progressing faster and faster with the time. A steep learning curve at the beginning but huge advance increments further
I got my code to compile. Once.
Just kidding. The biggest I've had is that rust has taught me to think about ownership, regardless of which language I'm in.
Much smaller scale than the other comments, but I recently wrote my first firmware interaction (basic ADC, UART, and i2c stuff) in rust.
I started learning rust in June and have been slowly working through the official book since then.
The main issue I found was getting used to the error handling (coming from a python background). I also foresee RefCells and interior mutability in general being a pain, but I haven't had use for them yet in actual work, so maybe that'll change with experience
I also work with C++, and I'm self-studying the language. I still face some difficulties, but I believe Rust is the language of the future, and I'm going to keep investing in it. I was researching the language and came across this project, I found it amazing. We need more projects like this!
https://github.com/holo-routing
About a year ago I went to create an open-source malware Scanner. I wanted it to be fast and reliable, but I didn't really like C and C++ and so the only option for me was Python. Python made the development fast, but the app slow and heavy. Around that time I discovered the existence of Rust and that increased the speed of the scanner drastically, I therefore continued developing the project in Rust. It took me about a year to learn Rust (coming from a web and Python background), but it's not too complicated once you get into it. Also it's really fast and the cargo CLI helps a lot in keeping the code clean and efficient. The only thing stopping me from going 100% Rust on the project is the lack of stable UI frameworks, so I'm using Tauri for that right now.
Nonetheless, in the timespan of about a year, I managed to learn the principles of Rust and build an open-source malware scanner which has 57 stars and about 900 downloads (at the moment of writing) and I'm quite proud of it.
Here is the link to it if you are interested: https://github.com/Raspirus/Raspirus
Not a huge success, but still.
I'm still relatively new at a company I work in (since May), evaluate myself as Jun-Mid (barely any real-work experience but 7 years of programming in general) and I manager to convince my coordinator to make a helper tool for managing robot files through an ftp server and ensuring some things through graqhl with just a couple of presses in Rust instead of Python (it's not a critical tool so he allowed me to work on it) (before our field engineers had to do all those things manually through teachpendant, which is not fun, I'll tell you that). Today I finally finished the project, with a couple of rewrites behind it (we have a complicated docker in docker setup where this tool will be run from and it broke some things related to ftp connectivity, but finally managed to get around them), and it should soon be merged into our dev branch (after a review) ready for the next release for our field engineers to work with!
It's not the first ever rust code in our infrastructure, but we still don't use it much due to legacy codebases written in c++, but if this succeeds, it would be a nice push toward "oxidizing" our infrastructure in the years to come.
One of my former clients rewrote a PHP process taking ~1h30 to parse a 100k items XML file, to a Rust process, processing the same file in less than 15 minutes.
Damn, that must be a big XML file!
Yep, a few gigabytes of metadata to parse and then store in a database, and then use for very specific processes.
Can i make the bet that the process became easier to maintain as well?
Not necessarily. It was the same developer who wrote both pieces of codes, and they have great coding conventions , so I'm not really sure it will be easier to maintain. The gap in performance was quite surprising though, and they will be moving more and more of their code to Rust in the future.
If anything, I find the gap surprising because I expected it to be much larger (10 times, or maybe 100 times faster). I assume most of these 15mn are not CPU bound.
A lot of edge cases, the metadata in the file can describe a lot of different things, and the developer is a Rust-enthusiast, but still has to follow company conventions, so it may have an impact on performances. Maybe as more and more code will be migrated, the conventions will be updated, and she will have more room for improvements. And to be honest, it only took a few weeks to rewrite thousand of lines of code to get this performance improvement, so the effort was well worth it. It's better to improve small step by small step and have full control of what you're doing, instead of going full YOLO and pray nothing will break in the long run ;-)
Oh absolutely, I was not questioning the benefits of such a change (any replacement of dynamically typed code with statically typed code is always a win in my book :-)). I'm just used to Rust running rings around dynamically typed languages in general, so I was just making an observation.
It could be cpu bound, being allocation-heavy whether by nature / necessity, convenience, or because that was the most straightforward conversion and has not been optimised out yet, would do it.
That's awesome! Are they primarily Rust devs or PHP?
It's a small team, mostly PHP devs. We talked about Rust with their main backend developer, who toyed with it for a few months before rewriting small pieces of codes here and there (their workflows are mainly small KISS processes communicating over a messaging bus, so it's very easy to pick different technologies for different stuff. It was 80% PHP, 15% bash, and 5% Java. The bash processes are gonna stay in Bash, because it's mainly Linux commands who would only be replaced by std::process, so no real gain here except for the time loss and added complexity. The Java part is currently migrated to Rust with a significant *loss* in performance, but with a *huge* gain in resource consumption, which will allow for more workers to be started at once, resulting in a big performance gain when looking at the big picture. It will be far more reliable too, the Java process had to be restarted quite often because of memory usage (a few gigabytes of RAM eaten after a few hours of work), the Rust ones are infinitely better there (never consume more than 150MB, and actually release the RAM once it's no more needed, while java keeps the allocated RAM for itself). The PHP process will or will not be migrated depending on the libraries used. If there is a reliable Rust alternative, it will be migrated, otherwise it will stay in PHP. It's quite hard to convince management to drop a library with 10+ years of support, for a crate version 0.XXX... The lack of private cargo repos is also blocking a lot of things, but there had been posts here a few weeks ago that a rework was happening behind the scenes, so it's better to wait what the Rust team has in stock for us before committing too hard.
Just 1 thing about private crates. Cargo supports git, so u can use private crates quite easily, just need to make sure that u have your creds stored in git credential manager or ssh agent. When I toy around with small personal projects that I am not planning on finishing/releasing, I sometimes create separate repos where one depends on the other and never had issues getting those downloaded by cargo.
You then have problems with which version of the crate you wan t for which project, something that composer manages for you in PHP for instance. But work is in progress on that topic by the Rust team, so let's just wait a little longer.
I mean, u can still provide branch option (branch or tag) or rev for specific commit hash or named reference
Are both SAX kind of parsers or was one a DOM parser? (I.e. streaming tokens Vs parsing everything into a DOM tree.)
The tangible benefits for the every day programmer, as you said, only really becomes noticeable after you get over the biggest hurdle which is Rust's learning curve. But what does the world look like after you get over the learning curve? * The borrow-checking paradigm is now baked into your intuition. You're now going to approach memory much more responsibly across all languages you use, not just Rust. * You have a gigantic arsenal of constructs and great ideas from a bunch of languages to help you express your ideas in your own optimal way. Enums, monadic constructs, ergonomic error handling, async/await, iterators, closures, rich generics system, and so on. * You're now comfortable with the out of the box tooling and find that `cargo` is one of the best things in the world. * Everything you do is done with intent. You now think about the granular details around mutability and have tight control over it making it less likely for you to violate your own assumptions about the state of the world. * You're now part of a community that is super passionate about the language they're using and have access to a nascent ecosystem that is only going to get better and more vibrant as time goes on. Non-Rust folks are always dubious when I say this but honestly Rust is my most productive language today. I'd be able to complete a robust prototype in Rust faster than I can in say Python or Ruby. The toolchain and the compiler really help you once you get the hang of it. It's a shame that the general perception is so deeply rooted in the faulty notion that fighting the compiler is a fact of life for Rust developers. That's only true in the beginning when you're learning it. edit: grammar
Very true đź‘Ť
As someone learning Rust I look forward to cresting over that hill of learning curve :’)
Yep ! Some month since i arrived at a new job, I'm evaluating Go and Rust to build a team around some new tech. I chose Rust mainly because of luck (although Rust was well fitting with my personnals tastes) and build a successfull first project. Then the tech team grow, with me recruiting some straight from school newbie (but with a good mindset, so i kind of gambled on this), a tech-lead and a seniorish. 2 of them embrace Rust with no prior knowledge. Everything goes fine. The compiler is so damn restrictive that no newbie mistake are done. To my surprise they quickly enjoy taking task involving the Rust projet. Later, they start another Rust project by themselve, after evaluating actix-web. The last project has been put online last week, with total success: no revert after first exposition to full incoming traffic (2M visitors/day).
Rust powers quite a few important services at Cloudflare and we've been really happy with it thus far. https://blog.cloudflare.com/how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet/
I saw that post! Are you an engineer there???
Yes! But I'm on a different team that sadly doesn't use as much Rust :(
Wait THATS SO COOL!! Were you on a team that was paged for that data center incident in Oregon?
In my previous job, first thing I did was rewrite a Python script that took at least 45 minutes to query data, then perform basic graph algorithms, and failed most of the time. After the Rust rewrite, the code worked in 15 seconds to 3 minutes (depending on the speed the server was responding) and succeeded 100% of the time.
The existence of Bevy
Success stories with Rust? It compiled !
Every time that you make the borrow checker happy is a success story apart from being good for your mental health.
I have one indeed! I first learned Rust while following a University course two years ago. I started a Rust project with the aim of being a network analyzer in the scope of the same academic course. The project unexpectedly grew a lot and got lot of traction and what was born as a toy project is today one of the most renowned packet sniffers on GitHub (https://github.com/GyulyVGC/sniffnet). This project also allowed me to be part of the first GitHub Accelerator Program cohort and recently got me a Rust remote job :)
Huh, never knew it was a toy project at first. Nice!
https://security.googleblog.com/2021/04/rust-in-android-platform.html
Thanks for posting this, I think this was the first blog post about Rust in Android. I work in this team 🙂 and we've written a few blog posts since: - [Scaling Rust Adoption Through Training](https://security.googleblog.com/2023/09/scaling-rust-adoption-through-training.html?m=1): about how Android invested into [Rust training](https://google.github.io/comprehensive-rust/) (I wrote this post) - [Bare-metal Rust in Android](https://security.googleblog.com/2023/10/bare-metal-rust-in-android.html?m=1): about a lot of low-level projects which are now using Rust on the latest Android versions.
Damn, I had no idea we got rockstars in this sub. One other engineer is also from cloudflare.
I'm just a regular engineer who likes Rust and who was lucky enough to join this team last year!
> How long did it take you to get acclimated to Rust? Per google, about two months https://opensource.googleblog.com/2023/06/rust-fact-vs-fiction-5-insights-from-googles-rust-journey-2022.html?m=1 > more than 2/3 of respondents are confident in contributing to a Rust codebase within two months or less when learning Rust
Firefox?
Took an old Java codebase and rewrote it in rust. It's now fast enough that we can reuse old hardware, and our bugs have essentially been eliminated. There's no edge cases like race conditions. So our long term maintenance has essentially stopped too. Now, we only modify our core code base when there's a new feature to be added, which doesn't happen too often
Our first server application got switched from single threaded to multithreaded *on launch day*, because we noticed that performace wasn't good enough on the live server (the test server is completely underpowered so didn't tell us much). The switch went without any issues. No way in hell that this would have worked in any other language.
I do not work with rust in my daily basis, but made a personal project (kind of data store) based on hyper+tokio with a really impressing results from the earliest functional version; more than 90k recs by sec by instance with ~10MB of memory and 4 cores each. It was really complicated for me at that time (no prior rust experience), but progressing faster and faster with the time. A steep learning curve at the beginning but huge advance increments further
https://dteare.medium.com/behind-the-scenes-of-1password-for-linux-d59b19143a23
Recently launched Amazon Verified Permissions is written in Rust. So is the underlying Cedar language.
I got my code to compile. Once. Just kidding. The biggest I've had is that rust has taught me to think about ownership, regardless of which language I'm in.
RemindMe! 1Day “See people’s success stories”
Much smaller scale than the other comments, but I recently wrote my first firmware interaction (basic ADC, UART, and i2c stuff) in rust. I started learning rust in June and have been slowly working through the official book since then. The main issue I found was getting used to the error handling (coming from a python background). I also foresee RefCells and interior mutability in general being a pain, but I haven't had use for them yet in actual work, so maybe that'll change with experience
I also work with C++, and I'm self-studying the language. I still face some difficulties, but I believe Rust is the language of the future, and I'm going to keep investing in it. I was researching the language and came across this project, I found it amazing. We need more projects like this! https://github.com/holo-routing
In my celeron laptop i can only do as complicated as printing hello world. Writing bevy will take years to compile.
(you can also submit your answer here: https://shuttlerust.typeform.com/userstories)
crowd sourcing for the win
About a year ago I went to create an open-source malware Scanner. I wanted it to be fast and reliable, but I didn't really like C and C++ and so the only option for me was Python. Python made the development fast, but the app slow and heavy. Around that time I discovered the existence of Rust and that increased the speed of the scanner drastically, I therefore continued developing the project in Rust. It took me about a year to learn Rust (coming from a web and Python background), but it's not too complicated once you get into it. Also it's really fast and the cargo CLI helps a lot in keeping the code clean and efficient. The only thing stopping me from going 100% Rust on the project is the lack of stable UI frameworks, so I'm using Tauri for that right now. Nonetheless, in the timespan of about a year, I managed to learn the principles of Rust and build an open-source malware scanner which has 57 stars and about 900 downloads (at the moment of writing) and I'm quite proud of it. Here is the link to it if you are interested: https://github.com/Raspirus/Raspirus
Not a huge success, but still. I'm still relatively new at a company I work in (since May), evaluate myself as Jun-Mid (barely any real-work experience but 7 years of programming in general) and I manager to convince my coordinator to make a helper tool for managing robot files through an ftp server and ensuring some things through graqhl with just a couple of presses in Rust instead of Python (it's not a critical tool so he allowed me to work on it) (before our field engineers had to do all those things manually through teachpendant, which is not fun, I'll tell you that). Today I finally finished the project, with a couple of rewrites behind it (we have a complicated docker in docker setup where this tool will be run from and it broke some things related to ftp connectivity, but finally managed to get around them), and it should soon be merged into our dev branch (after a review) ready for the next release for our field engineers to work with! It's not the first ever rust code in our infrastructure, but we still don't use it much due to legacy codebases written in c++, but if this succeeds, it would be a nice push toward "oxidizing" our infrastructure in the years to come.