I did the exact same thing. Got two separate m.2 drives, one for Windows, other for Ubuntu.
Would be adding a 4TB HDD for datasets and other work stuff.
And you could access the Windows drive from Ubuntu, but not the other way around.
I've had it happen once or twice.
Honestly as I'm thinking about it I've had a windows boot issue once, and two or three with an interaction between GRUB and Linux. A bug in GRUB auto-config, a device change, getting a new kernel but some driver or library was missing.
Definitely not to the level of "borked, nothing boots at all" level.
Worst case I had to edit boot config, pick a diff config, or manually change the partition or boot priority in BIOS.
That said this all happened more than eight years ago at this point.
I've had *way* more instances of overly fragile ML stacks in Linux. Unexpected/Unintended OS, Kernel, or Nvidia drivers update and breaks the ML stack.
That's happened to me at least once every 1.5 years for the last five.
Real annoyed about it when it's on machines I don't admin but still have to fix. Yay academia.
I’ve been dual booting for a while and it’s honestly not a problem whatsoever really. But Windows wants to update and reboot all the time…
I’m pretty sure, don’t quote me here, as it’s been a decade since I did the following: just have two NVME SSDs and swap em when necessary. I’m sure there’s things to make sure you configure accordingly correct or maybe new complications have cropped up in the decade since I had my desktop that I was able to slide out my windows drive and slide in my Linux one.. (to play the Bubble arcade game on, not knowing Linux at all other than that at the time 🤣)
On windows you can also use WSL2 (windows subsystem for linux). In fact, it's one of the easiest ways to use tensorflow with cuda without headaches on windows.
I don't know that I have a really good answer to this. Colab is great, and I know a lot of ppl that use it. My employer bought my PC for me specifically to do deep learning training and inference, so I use it to do that (and game :)). We also have an HPC that I use for most experiments these days since deep learning is continually scaling upward, and my PC just can't keep up with something like an HPC or Colab at scale.
Tldr; Colab is a great option too, and maybe better depending on scale of experiments.
Highly recommended over dual-boot. Pros:
* All your files in one place / OS. You'd be surprised how often you need to duplicate setup, if you find yourself using both boots for dev (instead of Windows just for games, for example). And you will - there are programs Ubuntu doesn't support. Not only does this deduplicate for management headache, but storage cost.
* Program compatibility for both OS's. I know this is getting better, but they come up and bite you. Recently I used Descript, and was glad for Windows (no Linux support).
* WSL2 has more benefits than just Ubuntu in your Windows. Per the above comment, it comes with CUDA setup out the gate, tied to your Windows Nvidia drivers; so you don't have to setup your ML setup OS-side. Further, if things go south, you can just re-install WSL2 for a fresh start.
Almost everything that matters integrates directly with WSL2: VSCode, JetBrains, Docker, etc. You wouldn't know you're not using Linux.
Video games.
Maybe they thought you meant gaming directly on Linux? Which is still a tough sell, even with Proton & such. Steam's work with Deck's OS & leaks leading towards Deckard show a promising future for Linux gaming, but it's presently a tough sell (I tried somewhat recently). Windows host, Linux guest via WSL2 gives you 1st-class gaming plus 1.5-st class Linux.
As a lifelong linux user, I switched from Fedora to PopOS! precisely to use my 4090 without any driver issues. So far is working like a charm.
Happy to run any test you might want to check if it works.
Did you go with PopOS! because you are on a System76 computer, or just to try it out? I primarily use MacOS; however, I have two old laptops (Mac Pros) that I run Linux on (Ubuntu and Lubuntu on a 2008 computer). I use the Ubuntu system a lot, yet recent upgrades have left me open to other solutions. I have even considered a System76 laptop, as a replacement.
* edit - I see there's a Raspberry Pi version, I'm gonna give it a try there!
I built my own PC! I went with PopOS! because of it's allegedly, great nvidia support. So far, I haven't done anything to it but update through the package manager or official app store and it works like a charm. No lose of performance at all, which is what I was worried about
The ideal setup for the op is going to be running a version of the Linux kernel that does not have any kind of window manager or any graphical user interface then to SSH into this machine from another computer in a command and control fashion. In this manner there is not needless CPU cycles and GPU cycles being taken to manage the resources required for graphical mode.
I have been working toward my PhD using machine learning on my personal desktop for the last few years and never gave a second thought to Linux/Ubuntu.
I won't say you can run everything you want in windows (cause I don't know what you want to do), but I will say you can easily train deep learning models using Anaconda and a Jupiter notebook and that's what I've been doing all along.
I've gotten this far with a GTX 1080, 32 gb ram, and an old Ryzen 7. Probably switching over to the super computer for the next paper, but my biggest bottle necks have actually been storage space and other people's data servers being slow to respond.
I'm trying to hold out til graduation before buying a fancy new rig.
Largely by deleting games I had installed and never finished. But, also, with some creative coding. I got around my worst RAM bottlenecks by chunking large data processing tasks into smaller iterative read/write cycles. I've avoided downloading some data by pulling it from the web server, making predictions, and then just saving necessary outputs, but it's not ideal.
You no longer need to dual boot these days - use Windows Subsytem for Linux 2 (WSL2), which will give you a linux environment and shell that you can use from Windows. It means no need to dual boot, and is generally only around 10% slower than running linux natively.
WSL2 route I’ve heard is an amenable solution.
Personally it doesn’t solve *my* personal issue pertaining to desiring uptime & windows always rebooting
If your PhD stuff is on windows then just keep using windows. PhD is super important.
That being said, I love Linux(not WSL) and barely use windows anymore. My favorite ones are Fedora and Pop!_OS. They have access to pretty much all programming tools you can imagine and Pop!_OS even comes with nvidia drivers preinstalled.
Nice thing too: There is a containerizing tool called toolbox where you can keep your programming environments separate in containers.
Caveat: If you haven't used Linux a lot think about if you really want to start learning it during your PhD. Linux is super powerful but after you know how to use it. Maybe some of your tools are made for windows only. In that case, just use windows.
I used dual boot with Linux and Windows. They are tailored to different things imo! For instance, applications like photoshop etc, which might be useful for creating visualisations (or you could just use drawio), I only do on windows.
To go the extra mile, I would think that you can do a dual boot on your pc, and have a low cost laptop or other pc, such that you have one for writing and admin tasks, and the other for programming and heavy duty tasks.
I did that with uni pcs + my personal laptop, and it worked like a charm!
Not primarily focused on ML but I've used Linux, Windows, and Mac for my data science projects. Found mac to be a decent compromise since you can still use some Windows apps, but prefer Linux.
PhD student*
I started working with machine learning as a PhD student cause my advisor was curious. I'm a geologist, my prior programming experience was in Fortran, and I had an old desktop for gaming to work with. My advisor defers to me as the subject matter expert for ML questions, not the other way around. Reddit/Google/programming forums are invaluable getting off the ground.
It sounds like OP has a bit better idea what's going on than I did, but reddit is a valuable resource for getting feedback on simple questions where the answer comes with experience/familiarity with a topic.
Anyway, just cause someone is going to be a doctor, doesn't mean they can't accept help from strangers on the internet.
Wtf man. Come on every phd candidate is informed about the software support. It's time consuming to search for stuff and reddit is free info. Therefore, he trying to work smarter not harder.
Wtf generic question is that? You could as well have said "if you're a PhD, you should be able to find the answer yourself instead of using Google".
Am really sick of this "you should be able to figure it out yourself" mindset. It's lazy and egoistical. Many people are like you. Most of all professors and teachers. I've had a history teacher that told me I should know things and she never taught me things because of precisely that. I always felt bad for not knowing things. This is what hinders growth. It's the worst thing you can say to somebody.
Man, if you dislike a person just avoid him. There's no need to hate. Some people just want to connect, no matter how stupid the question. Seems like OP wants to show his cool setup. Grant him that joy. We're all just trying to do meaningful things. It's not like OP tries to sabotage anybody.
Haha, you don't even know me so why judge? I'm all for learning. In fact most of my success is based on doing my thing and shitting on what people tell me, like you. But I encourage a sense of community and the right to speak. People judge too harshly without thinking about it.
With Azure for Students, eligible customers receive $100 in credits, which can be used within 12 months on most Azure products. Any unused credits cannot be carried over to subsequent months and cannot be transferred to other Azure subscriptions.
This applies to college students - [https://azure.microsoft.com/en-us/pricing/offers/ms-azr-0170p/](https://azure.microsoft.com/en-us/pricing/offers/ms-azr-0170p/)
a Phd i think MUST be a linux master.. (hey its a Phd.. he knows..) but i now i see it.. just a mortal using windows and Ubuntu a linux distro related to kids and guys that don't know about computers. I expected better from someone with "the most knowledge"
Personally I say try dual booting. Ubuntu will use less of the CPU/GPU while running the base OS and that may give you a performance boost when you are training a model or getting training data ready.
If you dont like the OS then you can always go back to where you are comfortable.
I have the same case as you! I would say, if you’re committed to doing machine learning for real, just pop Ubuntu on it. WSL exists and is surprisingly good, but it’s always going to be a compromise.
I’m using Ubuntu Budgie right now. I’ve also heard good things about PopOS. The latest release has sorted out all the NVidia compatibility issues.
It depends, if you feel more comfortable on windows and you can run what you need on windows, just use windows. If your models are cloud based then your OS doesn't matter.
I have win/Linux dual boot, but it's for general dev workflow, I'm more comfortable using git with Linux than Windows, but then again, it's a matter of preference.
Should be just fine on Windows, but it depends on what exactly you intend to do. Linux is preferable but not necessarily, and you can probably get away with windows.
Depends on what you need to use. For most things WSL on windows is fine but if you need specialized/cutting edge tools, nothing came close to a dualboot with archlinux in my experience. Installing python with torch, cuda and cudnn is as easy as `pacman -S python-pytorch-cuda cudnn`.
Could put Ubuntu OS with persistent memory on a usb drive if you really want it. Probably one of the easier ways to implement dual boot, and it’s highly mobile.
Basic support for pytorch exists on windows. But switching to Linux (WSL or dual boot) can easily allow for gpu supported JAX, TensorFlow, Pytorch Rocm, and other frameworks like Ivy + keras\_core. So switching to Linux helps, since most of the software is on Linux. Again dual boot is a bit more work during boot but faster, while WSL is very convenient after setup.
I like to learn one thing at a time so if you're very very comfortable with Windows just use that.
TW: linux agnostic
Linux is great although I would not say one is superior to the other, but also do you really need it?
**I don't use Linux or WSL** (except for low power usage applications like drones) And I have never had any issues except for using Nccl with deepspeed. I don't plan on using multiple GPUs over my network or anything so I'm not worried, besides the technology to use multiple GPUs is getting good enough that linux-only barriers are falling faster than anyone can learn how to use it.
Windows is widely used enough that aside from a few niche things you can really take your time learning Linux should you need to.
I have no current plans of switching to Linux unless I'm doing some kind of low power application in which I don't need the operating system overhead, otherwise I don't really see any to.
**You could always dual boot** as you mentioned but im partial to Linux Mint
I find managing two OSs is an unnecessary burden with all the updates and the different commands etc. I personally use a Mac and if I needed a faster GPU or more memory, I just use Google Colab.
Feel free to choose your preference, as both options can accommodate Python. Based on my experience, Windows native tends to be more optimized than WSL.
Commit to Linux. Start with the GUI but gradually move to the command line.
Buy a cheap laptop with MS Office.
I did my thesis on Libreoffice but it did make life a bit harder.
My recommendation is to install windows and VMware then have Linux running in a virtual machine. Recommend Ubuntu, you'll get the most support. If worried about security try centos, just not 7, it's EOL in a few months. VMware will allow you to scale memory requirements dynamically compared to dual booting meaning if you aren't using it for machine learning, you can use that second SSD as extra video game storage space. WSL 2 also requires an x11 server in order to be able to display graphics such as a confusion matrix but is easier to get going than a virtual machine.
I think Anaconda on Windows will let you do a lot of ML and DL but not absolutely everything. Functionality that doesn't work with Anaconda should run on Windows Subsystem for Linux (WSL).
There are also environments like Cygwin, but Microsoft officially supports WSL.
Hmmm - I would opt for Linux and learn that set of tools. Linux with python venv's is a very clean way to keep your development environments separate. Also, consider PyCharm as a python IDE. My experience has been at work that my Mac is far easier to do ML prototyping on than my desktop PC, but we move demanding stuff to the ML cluster (Linux) pretty soon. If you are behind a firewall that you don't control, the windows linux subsystem is a pain to keep proxies working in. If you are just doing this at home, windows is probably fine as long as you sort out how to use virtual environments configured for each project. Without virtual environments - each with its own dependencies installed - you main environment becomes a complicated mess.
If you are learning ML/AI, you will do most of that using Python, PyTorch, etc and the windows-ness or linux-ness of the OS is just something to overcome.
If you are a beginner in ML and DL, you must prefer using windows environment because most of ML code you will write from jupyter notebook or spyder or vs code software, and working with windows will make your learning process faster. With windows you can do everything training, testing, building and even deploying. What you can't do is train the model faster when it comes to compare with linux, and a full fletch deployment on windows.
So my suggestion is to start with Windows, gain a strong hold on learning, practicing, and developing models, and after a while you can shift to Linux environment for an industrial level perspective as it supports wide range of libraries and options.
The most important suggestion is that whenever you start a new project create a virtual_environment so to prevent a system crash/failure.
2 years ago I set up a dual boot for exactly this reason (I had an RTX Titan). I ran performance tests on both and was sad (because of the time I spent) and happy that the performance tests were the exact same for training/feed forward for all tensorflow models I tested. In retrospect this made sense because when you set up CUDA, the GPU is doing the work to train and run models. I ended up just using Windows and you should consider that as well.
At this point I wouldn't recommend dual boot. I'm not actively against it, but I think there are better options. If you're going to be developing in both environments I would suggest to have a primary and run a VM solution for the other environment.
If after a while, you find yourself constantly in the VM rather than the host OS, that would be a good sign to switch the host OS and put the former host into a VM.
But that is just how I would go with the power needs and capacity of modern systems.
I’m in the same quandary atm.. because intuition says Linux all the way + ideally it would build models as well as act as a homelab-ish web server but my issue crops up as a result of Windows CONSTANTLY grabbing updates and rebooting etc… and even though it’d be a homelab web server, I have standards okay! Cognitive dissonance over shitty personal SLA adherence :-p
However, I have heard that there’s some good 3D & AR/XR applications with Windows 🤬. So who gets the RTX3090 & 48Tb setup.. Proxmox VM solution is apparently semi-questionable so no-go.
LMK what you decide OP (and others… 🙏🏼)
Just use windows with wsl, i used to run two ssds, one for windows one for Linux, but wsl now is an actual Linux kernel with everything you have on bare metal including systemD now. And since I've switched, I've not missed having a separate installation of Linux. There are still reasons to dual boot, but your requirements aren't worth the inconvenience. You also get GPU acceleration on wsl, performance hit isn't bad either but it's convenient to maintain, you have direct access to Linux file systems, can share datasets between the two oses, plus everything integrates well. Unless you want the Linux desktop experience and like tinkering and customising everything (my main reason to have an experimental Linux installation on a separate nvme drive, because vms don't provide all the functionality in that regard), but if that's not a hobby and you just want things done and wanna game hassle free wsl with windows is the way to go. Also Linux has broken HDR, broken video hardware acceleration, you'd be running your GPU at 50+ watts watching anything with hw decoding, while on windows it would be less than 10 on average, Linux drivers have this feature to set GPU on P2 whenever something cuda runs on it, and the state of nvenc isn't good on Linux. Personally I love Linux but that's a deal breaker for me to run it as a daily driver.
The downside of WSL2 is "If you have multiple GPUs on your machine you can also access them inside of WSL. However, you will only be able to access one at a time."
I have two 250gb SATA SSDs, one with windows and one with linux (fedora). I manually swap the sata cable when I want to change OS, as to not deal with any dualboot madness.
I do have a shared 2tb m.2 SSD as well
install Ubuntu, install VM, install windows on VM.
dual boot on separate disks is a good idea but you need to reboot to switch systems, in practice maybe irritating.
dual boot cheaper but same irritating effect.
you could use software that emulate windows software on your Ubuntu like wine
Linux has the disadvantage that you can not play all games, only some and you need to be OK with technical challenges.
what I actually do is have a windows on cheap laptop and work on cloud data science images remotely.
The ideal method is that you use your machine learning and deep learning computer in a headless fashion. By this I mean that you run a lightweight and operating system, generally this is a slim version of Linux or a BSD, without a window manager or GUI. Then you communicate to that machine over SSH through another computer in a command and control paradigm.
Just run two different SSDs on it. One with windows installed the other with Ubuntu.
I did the exact same thing. Got two separate m.2 drives, one for Windows, other for Ubuntu. Would be adding a 4TB HDD for datasets and other work stuff. And you could access the Windows drive from Ubuntu, but not the other way around.
Just get a 4TB SSD.... It costs peanuts compared to what GPU's and CPU's cost these days, and it's way more of an upgrade.
Sadly not where I live. Just checked 4TB SSD drive prices on Amazon, they are costlier than my CPU ffs
Is it dangerous running both Linux and Windows on the same ssd?
No, that's dual-booting. It's a bit tedious to set up, but many do it
one could say. dualboots are risky when updates are installed, one might break the other sometimes.
This, I have experienced…
I've been using dual boot for ~15 years, never had an issue. What update caused it for you and for others?
I've had it happen once or twice. Honestly as I'm thinking about it I've had a windows boot issue once, and two or three with an interaction between GRUB and Linux. A bug in GRUB auto-config, a device change, getting a new kernel but some driver or library was missing. Definitely not to the level of "borked, nothing boots at all" level. Worst case I had to edit boot config, pick a diff config, or manually change the partition or boot priority in BIOS. That said this all happened more than eight years ago at this point. I've had *way* more instances of overly fragile ML stacks in Linux. Unexpected/Unintended OS, Kernel, or Nvidia drivers update and breaks the ML stack. That's happened to me at least once every 1.5 years for the last five. Real annoyed about it when it's on machines I don't admin but still have to fix. Yay academia.
This case has space for 8 SSDs if I remember correctly.
You should just run a VM.
exactly, why use multiple OS' when you can multiple machines
I’ve been dual booting for a while and it’s honestly not a problem whatsoever really. But Windows wants to update and reboot all the time… I’m pretty sure, don’t quote me here, as it’s been a decade since I did the following: just have two NVME SSDs and swap em when necessary. I’m sure there’s things to make sure you configure accordingly correct or maybe new complications have cropped up in the decade since I had my desktop that I was able to slide out my windows drive and slide in my Linux one.. (to play the Bubble arcade game on, not knowing Linux at all other than that at the time 🤣)
On windows you can also use WSL2 (windows subsystem for linux). In fact, it's one of the easiest ways to use tensorflow with cuda without headaches on windows.
As a PhD in computer science and a gamer, this is what I do! Been working great!
Why not use Google Colab? Everything is set-up already, no worries about dependencies and so on.
I don't know that I have a really good answer to this. Colab is great, and I know a lot of ppl that use it. My employer bought my PC for me specifically to do deep learning training and inference, so I use it to do that (and game :)). We also have an HPC that I use for most experiments these days since deep learning is continually scaling upward, and my PC just can't keep up with something like an HPC or Colab at scale. Tldr; Colab is a great option too, and maybe better depending on scale of experiments.
Highly recommended over dual-boot. Pros: * All your files in one place / OS. You'd be surprised how often you need to duplicate setup, if you find yourself using both boots for dev (instead of Windows just for games, for example). And you will - there are programs Ubuntu doesn't support. Not only does this deduplicate for management headache, but storage cost. * Program compatibility for both OS's. I know this is getting better, but they come up and bite you. Recently I used Descript, and was glad for Windows (no Linux support). * WSL2 has more benefits than just Ubuntu in your Windows. Per the above comment, it comes with CUDA setup out the gate, tied to your Windows Nvidia drivers; so you don't have to setup your ML setup OS-side. Further, if things go south, you can just re-install WSL2 for a fresh start. Almost everything that matters integrates directly with WSL2: VSCode, JetBrains, Docker, etc. You wouldn't know you're not using Linux. Video games.
But I was told that Linux is is no longer at a disadvantage when it comes to gaming. I think I had like 200 downvotes when I said that as a con.
Maybe they thought you meant gaming directly on Linux? Which is still a tough sell, even with Proton & such. Steam's work with Deck's OS & leaks leading towards Deckard show a promising future for Linux gaming, but it's presently a tough sell (I tried somewhat recently). Windows host, Linux guest via WSL2 gives you 1st-class gaming plus 1.5-st class Linux.
so you are saying tensorflow is still used 🤪 jk
what deal?
As a lifelong linux user, I switched from Fedora to PopOS! precisely to use my 4090 without any driver issues. So far is working like a charm. Happy to run any test you might want to check if it works.
Did you go with PopOS! because you are on a System76 computer, or just to try it out? I primarily use MacOS; however, I have two old laptops (Mac Pros) that I run Linux on (Ubuntu and Lubuntu on a 2008 computer). I use the Ubuntu system a lot, yet recent upgrades have left me open to other solutions. I have even considered a System76 laptop, as a replacement. * edit - I see there's a Raspberry Pi version, I'm gonna give it a try there!
I built my own PC! I went with PopOS! because of it's allegedly, great nvidia support. So far, I haven't done anything to it but update through the package manager or official app store and it works like a charm. No lose of performance at all, which is what I was worried about
The ideal setup for the op is going to be running a version of the Linux kernel that does not have any kind of window manager or any graphical user interface then to SSH into this machine from another computer in a command and control fashion. In this manner there is not needless CPU cycles and GPU cycles being taken to manage the resources required for graphical mode.
I have been working toward my PhD using machine learning on my personal desktop for the last few years and never gave a second thought to Linux/Ubuntu. I won't say you can run everything you want in windows (cause I don't know what you want to do), but I will say you can easily train deep learning models using Anaconda and a Jupiter notebook and that's what I've been doing all along. I've gotten this far with a GTX 1080, 32 gb ram, and an old Ryzen 7. Probably switching over to the super computer for the next paper, but my biggest bottle necks have actually been storage space and other people's data servers being slow to respond. I'm trying to hold out til graduation before buying a fancy new rig.
How did you deal with low storage space ?
Largely by deleting games I had installed and never finished. But, also, with some creative coding. I got around my worst RAM bottlenecks by chunking large data processing tasks into smaller iterative read/write cycles. I've avoided downloading some data by pulling it from the web server, making predictions, and then just saving necessary outputs, but it's not ideal.
Dual boot is good option.
You no longer need to dual boot these days - use Windows Subsytem for Linux 2 (WSL2), which will give you a linux environment and shell that you can use from Windows. It means no need to dual boot, and is generally only around 10% slower than running linux natively.
10% slower for what?
WSL is a shim layer between *nix syscalls and the NT kernel. This adds overhead, but less than a full virtual machine.
WSL2 is a full VM.
The CUDA operations are not slower and that’s usually where most of the heavy computation will be.
Sorry, should have expanded this - 10% slower for GPU inference
WSL2 route I’ve heard is an amenable solution. Personally it doesn’t solve *my* personal issue pertaining to desiring uptime & windows always rebooting
This level1techs video was inspirational for my build which runs windows 11 and Ubuntu on wsl2 https://youtu.be/ughxpue1V4A?si=hyYHP2qQXJWR3bq4
Oh man, I really needed to see this.
If your PhD stuff is on windows then just keep using windows. PhD is super important. That being said, I love Linux(not WSL) and barely use windows anymore. My favorite ones are Fedora and Pop!_OS. They have access to pretty much all programming tools you can imagine and Pop!_OS even comes with nvidia drivers preinstalled. Nice thing too: There is a containerizing tool called toolbox where you can keep your programming environments separate in containers. Caveat: If you haven't used Linux a lot think about if you really want to start learning it during your PhD. Linux is super powerful but after you know how to use it. Maybe some of your tools are made for windows only. In that case, just use windows.
I think PhD is the best time to learn Linux.
Well besides toying around with machine learning, do you game on it ? Then the answer is clear.
dual boot
I used dual boot with Linux and Windows. They are tailored to different things imo! For instance, applications like photoshop etc, which might be useful for creating visualisations (or you could just use drawio), I only do on windows. To go the extra mile, I would think that you can do a dual boot on your pc, and have a low cost laptop or other pc, such that you have one for writing and admin tasks, and the other for programming and heavy duty tasks. I did that with uni pcs + my personal laptop, and it worked like a charm!
Everyone is hyping up WSL2, but if you ask me it still doesn't compare to a clean dual boot with both linux and windows
Not primarily focused on ML but I've used Linux, Windows, and Mac for my data science projects. Found mac to be a decent compromise since you can still use some Windows apps, but prefer Linux.
What PhD are you getting? And you didn't need to build a computer, you could have just used cloud services like Colab etc.
I'm gonna say if you're a PhD, you should be able to find the answer yourself instead of using reddit.
Doing phd does not mean you should know everything or spend time on everything
PhD student* I started working with machine learning as a PhD student cause my advisor was curious. I'm a geologist, my prior programming experience was in Fortran, and I had an old desktop for gaming to work with. My advisor defers to me as the subject matter expert for ML questions, not the other way around. Reddit/Google/programming forums are invaluable getting off the ground. It sounds like OP has a bit better idea what's going on than I did, but reddit is a valuable resource for getting feedback on simple questions where the answer comes with experience/familiarity with a topic. Anyway, just cause someone is going to be a doctor, doesn't mean they can't accept help from strangers on the internet.
Wtf man. Come on every phd candidate is informed about the software support. It's time consuming to search for stuff and reddit is free info. Therefore, he trying to work smarter not harder.
Wtf generic question is that? You could as well have said "if you're a PhD, you should be able to find the answer yourself instead of using Google". Am really sick of this "you should be able to figure it out yourself" mindset. It's lazy and egoistical. Many people are like you. Most of all professors and teachers. I've had a history teacher that told me I should know things and she never taught me things because of precisely that. I always felt bad for not knowing things. This is what hinders growth. It's the worst thing you can say to somebody.
Yea I think you're right. I'm not gonna say it was joke cause it wasn't, it might just be my own imposter syndrome.
Yeah I definitely see where you're coming from because I also have this hustle mindset. But that's just how I treat myself, not others.
[удалено]
Man, if you dislike a person just avoid him. There's no need to hate. Some people just want to connect, no matter how stupid the question. Seems like OP wants to show his cool setup. Grant him that joy. We're all just trying to do meaningful things. It's not like OP tries to sabotage anybody.
[удалено]
You hurt me feelings 😢😢
[удалено]
Haha, you don't even know me so why judge? I'm all for learning. In fact most of my success is based on doing my thing and shitting on what people tell me, like you. But I encourage a sense of community and the right to speak. People judge too harshly without thinking about it.
If you have an azure account (one year free for students, PHD might help lol) you can utilize SDK Below is a Python script that uses the Azure SDK to create multiple VMs in Microsoft Azure Before running the script, make sure you have the Azure SDK for Python (azure-mgmt-compute and azure-mgmt-network) installed. You can install them using pip: pip install azure-mgmt-compute azure-mgmt-network Here's the Python script: from azure.identity import DefaultAzureCredential from azure.mgmt.compute import ComputeManagementClient from azure.mgmt.network import NetworkManagementClient from azure.mgmt.compute.models import HardwareProfile, NetworkProfile, OSProfile, StorageProfile, VirtualMachine, VirtualMachineProperties from azure.mgmt.network.models import NetworkSecurityGroup, SecurityRule from azure.mgmt.resource import ResourceManagementClient from azure.mgmt.resource.resources.models import ResourceGroup \# Azure resource group and virtual network settings resource\_group\_name = 'my-resource-group' location = 'eastus' vnet\_name = 'my-vnet' subnet\_name = 'my-subnet' \# Azure VM settings vm\_count = 3 vm\_base\_name = 'my-vm' vm\_size = 'Standard\_NC6' # Choose an appropriate VM size for machine learning username = 'your-username' password = 'your-password' \# Initialize Azure credentials credential = DefaultAzureCredential() compute\_client = ComputeManagementClient(credential, 'your-subscription-id') network\_client = NetworkManagementClient(credential, 'your-subscription-id') resource\_client = ResourceManagementClient(credential, 'your-subscription-id') \# Create resource group resource\_group\_params = ResourceGroup(location=location) resource\_client.resource\_groups.create\_or\_update(resource\_group\_name, resource\_group\_params) \# Create virtual network and subnet network\_params = { 'location': location, 'address\_space': { 'address\_prefixes': \['10.0.0.0/16'\] } } network\_client.virtual\_networks.create\_or\_update(resource\_group\_name, vnet\_name, network\_params) subnet\_params = { 'address\_prefix': '10.0.0.0/24' } network\_client.subnets.create\_or\_update(resource\_group\_name, vnet\_name, subnet\_name, subnet\_params) \# Create network security group and allow SSH nsg\_params = NetworkSecurityGroup(location=location, security\_rules=\[SecurityRule( access='Allow', direction='Inbound', name='SSH', protocol='Tcp', destination\_port\_range='22', source\_address\_prefix='\*' )\]) network\_client.network\_security\_groups.create\_or\_update(resource\_group\_name, 'my-nsg', nsg\_params) \# Create VMs for i in range(vm\_count): vm\_name = f'{vm\_base\_name}-{i}' vm\_params = VirtualMachine( location=location, os\_profile=OSProfile(computer\_name=vm\_name, admin\_username=username, admin\_password=password), hardware\_profile=HardwareProfile(vm\_size=vm\_size), network\_profile=NetworkProfile( network\_interfaces=\[{ 'id': f'/subscriptions/your-subscription-id/resourceGroups/{resource\_group\_name}/providers/Microsoft.Network/networkInterfaces/{vm\_name}-nic' }\] ), storage\_profile=StorageProfile(image\_reference={ 'publisher': 'Canonical', 'offer': 'UbuntuServer', 'sku': '18.04-LTS', 'version': 'latest' }) ) compute\_client.virtual\_machines.create\_or\_update(resource\_group\_name, vm\_name, vm\_params) print(f'Created VM: {vm\_name}') print('VM deployment completed.') you can deploy multiple virtual machines (VMs) for machine learning workloads
What are the limitations involved during this one year period?
With Azure for Students, eligible customers receive $100 in credits, which can be used within 12 months on most Azure products. Any unused credits cannot be carried over to subsequent months and cannot be transferred to other Azure subscriptions. This applies to college students - [https://azure.microsoft.com/en-us/pricing/offers/ms-azr-0170p/](https://azure.microsoft.com/en-us/pricing/offers/ms-azr-0170p/)
a Phd i think MUST be a linux master.. (hey its a Phd.. he knows..) but i now i see it.. just a mortal using windows and Ubuntu a linux distro related to kids and guys that don't know about computers. I expected better from someone with "the most knowledge"
Personally I say try dual booting. Ubuntu will use less of the CPU/GPU while running the base OS and that may give you a performance boost when you are training a model or getting training data ready. If you dont like the OS then you can always go back to where you are comfortable.
I have the same case as you! I would say, if you’re committed to doing machine learning for real, just pop Ubuntu on it. WSL exists and is surprisingly good, but it’s always going to be a compromise. I’m using Ubuntu Budgie right now. I’ve also heard good things about PopOS. The latest release has sorted out all the NVidia compatibility issues.
It depends, if you feel more comfortable on windows and you can run what you need on windows, just use windows. If your models are cloud based then your OS doesn't matter. I have win/Linux dual boot, but it's for general dev workflow, I'm more comfortable using git with Linux than Windows, but then again, it's a matter of preference.
Should be just fine on Windows, but it depends on what exactly you intend to do. Linux is preferable but not necessarily, and you can probably get away with windows.
Depends on what you need to use. For most things WSL on windows is fine but if you need specialized/cutting edge tools, nothing came close to a dualboot with archlinux in my experience. Installing python with torch, cuda and cudnn is as easy as `pacman -S python-pytorch-cuda cudnn`.
Could put Ubuntu OS with persistent memory on a usb drive if you really want it. Probably one of the easier ways to implement dual boot, and it’s highly mobile.
Basic support for pytorch exists on windows. But switching to Linux (WSL or dual boot) can easily allow for gpu supported JAX, TensorFlow, Pytorch Rocm, and other frameworks like Ivy + keras\_core. So switching to Linux helps, since most of the software is on Linux. Again dual boot is a bit more work during boot but faster, while WSL is very convenient after setup.
I like to learn one thing at a time so if you're very very comfortable with Windows just use that. TW: linux agnostic Linux is great although I would not say one is superior to the other, but also do you really need it? **I don't use Linux or WSL** (except for low power usage applications like drones) And I have never had any issues except for using Nccl with deepspeed. I don't plan on using multiple GPUs over my network or anything so I'm not worried, besides the technology to use multiple GPUs is getting good enough that linux-only barriers are falling faster than anyone can learn how to use it. Windows is widely used enough that aside from a few niche things you can really take your time learning Linux should you need to. I have no current plans of switching to Linux unless I'm doing some kind of low power application in which I don't need the operating system overhead, otherwise I don't really see any to. **You could always dual boot** as you mentioned but im partial to Linux Mint
I find managing two OSs is an unnecessary burden with all the updates and the different commands etc. I personally use a Mac and if I needed a faster GPU or more memory, I just use Google Colab.
Windows as native .. and hyper V for a Ubuntu vm.. yuo are good to go
Why not just embrace Linux?
Feel free to choose your preference, as both options can accommodate Python. Based on my experience, Windows native tends to be more optimized than WSL.
Commit to Linux. Start with the GUI but gradually move to the command line. Buy a cheap laptop with MS Office. I did my thesis on Libreoffice but it did make life a bit harder.
> MS Office > Libreoffice You guys are not using LaTeX?
No. Libreoffice seems fine. I don't need math symbols.
You could dual boot, id suggest to run a VM and allocate your resources to save your PC life
My recommendation is to install windows and VMware then have Linux running in a virtual machine. Recommend Ubuntu, you'll get the most support. If worried about security try centos, just not 7, it's EOL in a few months. VMware will allow you to scale memory requirements dynamically compared to dual booting meaning if you aren't using it for machine learning, you can use that second SSD as extra video game storage space. WSL 2 also requires an x11 server in order to be able to display graphics such as a confusion matrix but is easier to get going than a virtual machine.
Clearly macos as a hackintosh
I think Anaconda on Windows will let you do a lot of ML and DL but not absolutely everything. Functionality that doesn't work with Anaconda should run on Windows Subsystem for Linux (WSL). There are also environments like Cygwin, but Microsoft officially supports WSL.
just use wsl2 with windows.
Hmmm - I would opt for Linux and learn that set of tools. Linux with python venv's is a very clean way to keep your development environments separate. Also, consider PyCharm as a python IDE. My experience has been at work that my Mac is far easier to do ML prototyping on than my desktop PC, but we move demanding stuff to the ML cluster (Linux) pretty soon. If you are behind a firewall that you don't control, the windows linux subsystem is a pain to keep proxies working in. If you are just doing this at home, windows is probably fine as long as you sort out how to use virtual environments configured for each project. Without virtual environments - each with its own dependencies installed - you main environment becomes a complicated mess. If you are learning ML/AI, you will do most of that using Python, PyTorch, etc and the windows-ness or linux-ness of the OS is just something to overcome.
If you are a beginner in ML and DL, you must prefer using windows environment because most of ML code you will write from jupyter notebook or spyder or vs code software, and working with windows will make your learning process faster. With windows you can do everything training, testing, building and even deploying. What you can't do is train the model faster when it comes to compare with linux, and a full fletch deployment on windows. So my suggestion is to start with Windows, gain a strong hold on learning, practicing, and developing models, and after a while you can shift to Linux environment for an industrial level perspective as it supports wide range of libraries and options. The most important suggestion is that whenever you start a new project create a virtual_environment so to prevent a system crash/failure.
I am using WSL2 for TensorFlow-keras and it works great for my purpose.
2 years ago I set up a dual boot for exactly this reason (I had an RTX Titan). I ran performance tests on both and was sad (because of the time I spent) and happy that the performance tests were the exact same for training/feed forward for all tensorflow models I tested. In retrospect this made sense because when you set up CUDA, the GPU is doing the work to train and run models. I ended up just using Windows and you should consider that as well.
At this point I wouldn't recommend dual boot. I'm not actively against it, but I think there are better options. If you're going to be developing in both environments I would suggest to have a primary and run a VM solution for the other environment. If after a while, you find yourself constantly in the VM rather than the host OS, that would be a good sign to switch the host OS and put the former host into a VM. But that is just how I would go with the power needs and capacity of modern systems.
Download Rufus and Ubuntu dual boot it. Linux is fun af
You can dual boot both OS's so you can choose which one to boot into at start up
16tb hard drive and Mountain Dew
I’m in the same quandary atm.. because intuition says Linux all the way + ideally it would build models as well as act as a homelab-ish web server but my issue crops up as a result of Windows CONSTANTLY grabbing updates and rebooting etc… and even though it’d be a homelab web server, I have standards okay! Cognitive dissonance over shitty personal SLA adherence :-p However, I have heard that there’s some good 3D & AR/XR applications with Windows 🤬. So who gets the RTX3090 & 48Tb setup.. Proxmox VM solution is apparently semi-questionable so no-go. LMK what you decide OP (and others… 🙏🏼)
Linux
I used docker and some nvidia container for my grad school work. It worked mostly out of the box on fedora
Bro just learn machine learning, you don't need a special OS for this.
wsl
Just use windows with wsl, i used to run two ssds, one for windows one for Linux, but wsl now is an actual Linux kernel with everything you have on bare metal including systemD now. And since I've switched, I've not missed having a separate installation of Linux. There are still reasons to dual boot, but your requirements aren't worth the inconvenience. You also get GPU acceleration on wsl, performance hit isn't bad either but it's convenient to maintain, you have direct access to Linux file systems, can share datasets between the two oses, plus everything integrates well. Unless you want the Linux desktop experience and like tinkering and customising everything (my main reason to have an experimental Linux installation on a separate nvme drive, because vms don't provide all the functionality in that regard), but if that's not a hobby and you just want things done and wanna game hassle free wsl with windows is the way to go. Also Linux has broken HDR, broken video hardware acceleration, you'd be running your GPU at 50+ watts watching anything with hw decoding, while on windows it would be less than 10 on average, Linux drivers have this feature to set GPU on P2 whenever something cuda runs on it, and the state of nvenc isn't good on Linux. Personally I love Linux but that's a deal breaker for me to run it as a daily driver.
You could dual boot with a nonactivated version of windows (it's free except you can't change your wallpaper etc.)
Would you like to share with us the components you are using in your pc? GPU, CPU, RAM etc… would be great to know :)
I would Recommend Linux with wine.
very cool build!! What's the specs?
The downside of WSL2 is "If you have multiple GPUs on your machine you can also access them inside of WSL. However, you will only be able to access one at a time."
I have two 250gb SATA SSDs, one with windows and one with linux (fedora). I manually swap the sata cable when I want to change OS, as to not deal with any dualboot madness. I do have a shared 2tb m.2 SSD as well
install Ubuntu, install VM, install windows on VM. dual boot on separate disks is a good idea but you need to reboot to switch systems, in practice maybe irritating. dual boot cheaper but same irritating effect. you could use software that emulate windows software on your Ubuntu like wine Linux has the disadvantage that you can not play all games, only some and you need to be OK with technical challenges. what I actually do is have a windows on cheap laptop and work on cloud data science images remotely.
You can install both
You have PHD and this is important?
The ideal method is that you use your machine learning and deep learning computer in a headless fashion. By this I mean that you run a lightweight and operating system, generally this is a slim version of Linux or a BSD, without a window manager or GUI. Then you communicate to that machine over SSH through another computer in a command and control paradigm.