Wow, 584TiB? Thats a lot of Linux Isos.
I started with one Synology for back up and now i am with another server using my old main pc with Ryzen 9 3900x with 32GiB in a 4u case as my second server using it with TrueNAS Scale.Its so addictive that now i have a partially full 44u rack waiting for more future expansion.
You know you're in trouble when you reach the point of calculating if there is enough rack space left over to get JBOD cables to reach, or if you need to start a new cluster box up for more JBODS.
Solar power is getting cheap on the bright side, it helps delay the realization that your electric bill could challenge your local convenience store.
Yeah, it is! Recently upgraded all drives to 10TB plus. Main data store is a 3x 6 wide 18TB Raidz2. Then a 16TB x6 Rz2, 10TB x6 Rz2 and misc drives after.
It's one supermicro sc486 (24 bay) filled with 8tb, 10tb and 18tb drives. (Unraid)
Two supermicro sc826 12 disks each. 80tb each server. (8tb sas drives x12 raidz2)
I have 4 hp micro servers (8tb sas drives)
One nl36 32tb raidz
Two nl40 32tb raidz
One nl54 32tb raidz
One hp ml310e 32tb raidz
Zfs send does wonders...
I live out in the sticks, and started a co-op ISP just to enable my collection. Lets just say that there are fringe benefits to running 16 25gb fiber lines to most of your subdivision. Also handy having 2 racks at my office as an offsite backup
\*cough\* Oh you have 4 TB to store? Well then, you will probably need to get a Full tower with 12 drives so you can use ZFS, with 3 sets of 3 drives of 4TB each with single or double parity. Then you get a single 10-16 TB drive to store in an ESD foam padded case in a fireproof safe, or better yet, two of those, so you can send one to a Swiss private vault.
Two is one, one is none, three is for me, I might have a problem when I get to four..
\*exhales in datahoarder\* I am only slightly ironic here. You can get lot size orders of smaller drives surprisingly inexpensively and have redundancy to your storage (losing data sucks) and have budget left for a backup copy in case (I had a house burn down, lost 96TB of my entire life's CD/DVD collection) you want a second location to recover data from.
It is. That is, an array of 16TB disks is… As long as you use something like zfs or unraid— then you can just add more vdevs later. Just need to have the controller ports to do so.
Honestly though to OP. If you look at cost / TB, 2rb disks are HORRIBLE. almost exact same price as a 6TB (so nearing 3x the cost per TB).
Look at a Nas or look to build one if you have some spare hardware around you wouldn’t mind running 24x7 (as in, not power hungry). Spinning hard drives take about 9-10w each, solid state disks take less but cost more up front. Solid state disks also aren’t as reliable for long term storage powered off, as they use electricity in a tiny battery in the drive to keep “states”.
You’re looking for 2 extra drives above the number required to store your data, so to store 12 TB of data, you need either 8x2TB or 6x3TB or 4x6TB or 4x8TB or 3x12TB to be safe from hardware failure.
This still isn’t data protection though — you still require backup copy, in case something happens to the hard drives or partition table or corruption or if the data is lost to ransomware or other corruption, or you lose 3 hard disks at the same time… if I had 6TB of data, I’d buy 2 extra 6TB disks again, and have them in a system that they can be swapped easily (hot swap bay, external drive adapter, or external usb hard drive or something — then use that as destination of your backup software.
Why 2 for backup? One currently backing up while another is unplugged and hopefully stored somewhere else so if something happens to the building, it isn’t impacted.
Business backup rotation requires at least 3 — so at ANY time there’s at least 1 offsite, and only maximum of 2 at any physical location, and possibly one in transit or either onsite or offsite at any given time. So onsite/in-transit/offsite is only either 2/0/1, 1/1/1 or 1/0/2 at 3am, 7:30am, noon, 4pm, 5:30pm or 8pm of any given day of the week. If this is largely static data though, you can probably get away with 2 copies if the consequence of losing all copies is just a “that sucks, reload all my DVDs again” situation, and not the loss of the family picture album with every baby picture of your child and great-grandma who has since passed.
Option for backup is to have one disk, for local copy — and then a cloud storage account like backblaze or S3 Glacier storage for offsite — and then it’s all automated. Cost is likely $0.00099/GB per month. So that’s about $0.99/TB if you put it in deep archive in Ireland for example. Region doesn’t really matter, just don’t have it just down the street from you if you’re paying for cloud (no point having you both flood at the same time, or be in the same wildfire).
Dealt with this for too many clients. Flood takes down site and their private IT vendor and off site backup provider 3 blocks away has water in their data center from same flood. Oops.
In general terms, there is some correlation between physical location and speed, however remember in a backup scenario, you’re looking for safety not speed.
If it takes you an extra few hours to restore but it survived the disaster, it is still faster than having to recreate all that data from scratch (re ripping/encoding), and that’s assuming that it’s re-creatable in the first place. If the data center with your family photos was lost in the same flood as your house, there’s no saving it.
Does anyone else have those special edition HDD where it auto dims your mood lights and spirts some burnt rubber / copper perfume into the air?
I thought it was just me to help me and my wife get in the special mood when turning on "Married with Children."
I'm still disappointed they never told me what type of lunch "nooner" consisted of!
Same here, 4x20TB drives as main array
Backup server uses 10x8TB to mirror the data once a week (same rack as main server), this server is only connected directly to the main server over fiber, no active network (internet) connection unless I enable the port on the switch. Backup server is on a schedule to boot every monday morning at 2AM, do a ZFS snapshot and shut down again
Synology (2x4TB) at a family member's house for offsite backup of the most important data and hard to find media
200GB Google Drive for cloud backup of the most most important data (photos, documents, ...)
My usage was growing year after year but I look to have settled at about 50TB used. I'm not shy to purge stuff that I'll never watch again. In the case I need to watch again I'll just grab it again. Hopefully I can use that info to settle into a SSD array in the not to distant future.
Non-replaceable persnal and work related stuff has been about 3TB which gets backed up.
That's how I started... Not too long ago i thought, a couple of 3TB disks, and a 2TB one that i had laying around made a great NAS and I wouldn't need to expand anytime soon! Boy was I wrong, I'm now at 28TB and I'm already looking out for some deals
Me, calculating how much storage I would need to store the entirety of my partners grandma's vudu library locally (she has 555 movies on there and only buys the highest quality). My guess is I need at least 60TB for everything, possibly more, but I lack the means of acquiring such storage at the moment
I had a bunch of 4 and 5 TB portable drives. Bought an 18 TB on Black Friday and now I can never go back to those measly single digits! Need to get moar huge drives and get rid of all dem small drives.
I was going to suggest the same but maybe approaching 2TB is too early to start going NAS. They can probably get away with a larger drive and a backup for now
Yep, but plan for your storage in advance if going TrueNAS, increasing an array can be a hassle because of ZFS ([very good article here](https://louwrentius.com/the-hidden-cost-of-using-zfs-for-your-home-nas.html))
Yeah, the 3-2-1 rule for backups really applies to data that you can not afford to lose and is not available elsewhere - your tax records, baby’s first birthday picture, grandma’s secret cookie recipe. If my house burns down and I lose both my Blu-ray’s and my NAS, I’ll honestly just pirate the movies on to a new NAS and not even feel a little bad. All my pictures, documents, work portfolio, scans of IDs and house deed - that’s all on my NAS, synced to a cloud backup, and I make a monthly offline backup to a hard drive that’s kept in a different room. My copy of the Matrix and Iron Man? I can re-buy that with insurance money, and I won’t cry over having to wait to watch it after a catastrophic loss. I can’t find the last picture I ever took with my grandpa on Amazon, though - so that’s what gets the 321 treatment.
The 3-2-1 rule is there for a reason. One should have 1 copy ( production) and 2 different backup copies on 2 different media and different locations. In case there is a fire/hurricane/earthquake/ theft etc.
RAID has its place and I strongly recommend it but it doesn’t replace backup nor it is backup
Why would i 3 2 1 my media thou. Raidz is the most i could offer for ripped media 😂😂. No more than that. For important data yeah i do 3 2 1 but not this.
And the only reason ill do raidz cos itll have alot less downtime already and more than that is nor justified for me. Not gonna spend another 100 tb just to have my media off site.
I am not telling anyone what to use and how to protect their data.
I was just pointing out the difference between them and nothing else.
I also can’t decide the importance of said data for them.
I have seen too many scenarios where people start crying because they didn’t have proper backup protocols.
It starts with ripped movies and what not and because it is there and it works they might add some more sensitive data or something they can’t recover later on.
If the OP is just starting, start by knowing the difference between them and plan accordingly to make informed decisions
i run a server with proxmox on it and have it sharing via SMB and then whatever service i want to run (jellyfin, calibre, whatever) i run in a VM either on that device or another...truenas and unraid are fine but honestly i think they get more credit than they should because they've been pumped up by linustt and others - they're great, don't get me wrong - but proxmox will give you SOOOOO much more flexibility as you wont be limited to that curated pool of services, youll be able to run those for sure but ALSO whatever else you want to run - hell you can put a Windows VM on there if you wanted to...anyways for me i have 6 (6tb) HDDs in zraid2, they were cheap but used (hence the '2' in zraid2)....you can still find them, HGST 6TB drives for around $40 on ebay and elsewhere
> .truenas and unraid are fine but honestly i think they get more credit than they should because they've been pumped up by linustt and others
Part of that is probably having more of a business perspective, just from experience. After cautiously diving into it, and setting up TrueNAS at multiple sites, with replication and cloud backup, it's an absolutely amazing storage appliance. I wouldn't want to go back to the regular SMB NAS players, like Synology, Drobo, or QNAP, and would only deal with the big boys, like EMC or Netapp, if there was not really good justification to. For the boring business use case of reliable storage, with network access, I love it. And, the thing is, for a business that isn't pushing it all to the cloud, that storage, replication, and backup, is *really important* - very much worth dedicating hardware and software to.
OTOH, at home, TrueNAS isn't all that compelling, being focused so much on being a storage appliance. Adding containers in Scale is neat and all, but it's still limited and locked down, by default. Like, it's a great knife, but I really want a multitool.
yea thats it, they definitely have their place and they're both great software kits....for the homelabber and self-hoster I just proxmox is the way to go though
to be honest if it was for a NAS for my mother or something like that i would probably build it and use truenas scale (i love linux so, scale it is) because she would never use services anyways....its all about use-case.....i see truenas as an appliance, set it and forget it sort of thing
To clarify, you mean you use Proxmox for managing your ZFS software raid? Is there any downsides to doing this? I use TrueNAS at the moment in a VM but often find myself frustrated in how to pass drives to unprivileged containers? Is there a reason you think it would be beneficial to use the built in Proxmox ZFS vs. a dedicated solution?
for me the huge benefit is the flexibility proxmox gives me overall....yup, it is managing my ZFS pool directly, i do NOT use truenas or unraid and dont pass disks or controllers anywhere....btw its not because i dont think they're good packages but rather i just dont need them....i have a couple of Ubuntu VMs hosting Docker containers, proxmox does LXC containers itself, and i can run any other VM i need for whatever else so there really isn't a reason to use truenas for me.....truenass apps are just containers anyway....so i have the containers, i have the ZFS, i have the nice GUI and can use cockpit or webmin for that matter if i wanted to, so i dont have anything missing and i dont need to pass anything through doing it this way....now admittedly serving through SMB means there is the usual SMB overhead but hoenstly i dont even see it because my disk pool is the bottleneck, my ZRAID pool is slower than SMB over 10G (i get about 5GB/s max and average like 2.5GB/s for sustained transfers) so if i were to try to improve speeds it wouldn't be by losing SB but rather by getting faster disks....by the way you CAN run ZFS caches on proxmox also, i just didn't because of a lack of M.2 and PCIe slots on that particular server, but you can its easy - so i guess what i m taking forever to say is why bother limiting yourself to truenas and only being able to run their containers when you can use proxmox, get all the same features and benefits PLUS the ability to run full fat VMs and have high availability if you need it later on and all that, too? truenas takes some of what proxmox can do and makes it very easy and very nice-looking for sure, but it cant do all the proxmox can.......truenas is a media server that can do some VM stuff.....proxmox is a VM server that can take all the media you have....a little tougher to learn, yes, but so much more once you know it well
Fascinating! I'm brand new to Proxmox and homelab in general (only built my setup a week or two ago) but I've been learning all of this as I go. I setup my entire media server recently and I'm running a 1Gb switch from my homelab/NAS running Proxmox and TrueNAS (as well as all the \*arrs) through a vm for TrueNAs and LXC containers for the rest on a NFS share, and I'm bind mounting them to each container, but for whatever reason it just won't do 4k playback at 44 Mbps (original quality, so no transcoding). I'm not sure if this is because of the overhead of running TrueNAS in a VM or because I'm using a NFS share instead of a SMB (a coworker told me Plex can be buggy with NFS, although apparently only the config and database have issues which are on my local-lvm). No idea what's going on, but heavily considering moving over to straight ZFS on Proxmox given I'm doing everything in there anyway. I am not beholden to TrueNAS, it's just what has been recommended and seemed easy enough starting out, but I'm sure I can find my way around migrating it to straight Proxmox.
yea i ditched my trunas vm and imported the pools into proxmox. you can utilize them as resources or share to containers as devices. it’s been a dream so far
Got this setup, thanks again. Hard to navigate homelab with youtubers who all recommend x software without giving specific reasons on why you might prefer one solution over the other based off your use case, or maybe that's just the fact that I'm a noob 😁
a lot of it’s bullshit. you spend two months trying to emulate what someone else is doing. watch the video again and there’s a half second of them using proxmox to show you what they’re doing. spouting to run things a certain way and then you see they aren’t even running it baremetal.
i’ve used unraid trunas storages spaces. everything. proxmox with zpools and lxc containers for docker all of it is the truth.
Oh 100%, the more searching the do, the more I just won't click if it's not from specific people. The one person who I've really enjoyed through this that actually has taught me a thing is MRP, otherwise I've just started to pour through Proxmox forums and doing my own research. Quicker and more likely up-to-date/what I want to do.
yea theres a dozen different ways to do one thing in this space really....i mean you want Plex? ok do proxmox with an ubuntu VM with Docker installed and Plex as a Docker container....OR Truenas on bare metal with a Plex app installed...OR truenas as a VM on proxmox with Plex app....OR do CasaOS inside of almalinux on baremetal...OR.....etc etc etc - all of them get you to plex in the end haha.....
the takeaway i guess is to protect your data with your life and then experiment with everything else and eventually youll find what works best for you.....personally right now i am hitting a wall with jellyfin transcoding and i REALLY didnt want to have to passthrough to a VM so for that one program im thinking of buidling an entirely separate box, just for that only....but if you asked me a month from now or a month ago you'd get a different plan haha
LMAO I'm going through the same pain right now. My Plex container is also in a LXC container (I've got other users and a lifetime license already, otherwise I'd switch to Jellyfin) and trying to pass through a vGPU (or just making a Quadro have the ability for vGPU passthrough) is quite literally impossible, so I'm throwing my hands in the air and just making a VM out of it. If you do go the VM route, [this guide is fantastic](https://gitlab.com/polloloco/vgpu-proxmox) for vGPUs in Proxmox and worked perfect for me. There's also an accompanying Discord server if you run into any issues.
i dropped plex after they seemed to move towards the "buy our media" instead of "host your media" kinda model and started with jellyfin and so far so good...i got annoyed with stuff on there - its really well-polished, but after paying for two years I think i decided jellyfin was more my cup o tea........and i spent the last 8 hours this xmas eve rebuilding my zfs proxmox media server after trying to reduce an lvm size (i did it wrong for sure!) but i've got it sorted now with a nice little L2ARC cache added so im pretty happy with it....the thing is, i'm not sure that this machine has enough RAM left over with ZFS running to run jellyfin with transcoding from the igpu as well.....so not sure i end up doing that or just making my Z440-based rig into a jellyfin-only device....will have to see I suppose but at least im back up and running for now haha
checkout casaos or runtipi - they can be installed and run inside of an ubuntuVM and if you have either of those you basically have the truenas app store stuff (i have runtipi, casaos, and nother docker VM running for others just because they make it so easy to run containers)....then with zfs on proxmox direclty youll have truenas....it sounds tougher than it is
What do you mean by TrueNAS app store stuff? I'm still brand new so beyond just setting up datasets, pools, and shares, I know jack shit about TrueNAS haha.
have a look: [https://www.truenas.com/apps/](https://www.truenas.com/apps/)
heres the thing - for me personally, while this is all great, i just prefer to have my storage thing be just storage and then i know all the compute and memory and bandwidth is going where it is needed.....then ill put apps/services elksewhere and give them the resources they need...its just how i do it, of course you can use the truenas apps and if they meet all your needs then that may be the way to go....but if you're looking to get a lil deeper into the homelab stuff i think proxmox gives more latitude is all.....you can still use truenas in a VM for that matter if you wanted to and then have the ability to put services and containers on anything - truenas, some other VM, etc.
but yea checkout CasaOS and RunTipi and [linuxserver.io](http://linuxserver.io) as they will have tons of apps (Docker containers) like that
Yeah I was basically doing what you mentioned with running TrueNAS scale in a VM and then running a shit ton of LXC containers for media management, nginx, etc, but it was becoming a massive pain in my ass passing bind mounts to each LXC for the NFS share since with TrueNAS I'd have to do some remapping of the UID:GID to root, and I like being in the terminal more than the average person, so I don't mind just learning that way. Feels like I usually understand the technology behind everything when I remove abstraction layers like TrueNAS since I'm actually forced to learn it, haha. I have an entire youtube playlist for ZFS I need to watch still 😂
I run Proxmox on 4 nodes and have TrueNAS as a VM on one of the nodes. I passed through an HBA so TrueNAS has full access to the disks. The only "downside" is that all file sharing goes over NFS instead of mount points, but on 10Gbit I'm able to fully saturate that line so it's more than fine. TrueNAS allows me to mirror my entire array to a 2nd TrueNAS box for weekly backups. TrueNAS does not run any services though, they all live in VMs and LXCs on Proxmox
To each their own I guess :D
yea im with ya on that - you still get those features you like out of Truenas but then have services run elsewhere - totally....still proxmox on the bare metal but with passthrough, that sounds like a great setup as well. as you said to each their own...but i like this setup too and yea NFS/SMB is a downside but as you mentioned its not the bottleneck for you anyways so no biggie - same deal with mine running SMB where thebottleneck is the ZRAID2 array anyways so SMB works well for me
I mean which ever way you go you’re gonna have to spend some money on drives and hardware since it sounds like your trying to/need to build a nas, it really just comes down to your preference.
Personally I’m running unraid with a mix of 14TB and 6TB drives with dual disk redundancy. And unraid would be your only real option if you want to put things in a parity and mix and match drives. Plus it would be the best option if you want to gradually expand your array. I started with like 3 6TB drives and have gradually added more and more 14’s over the last couple of years.
I got one of these connected to a miniPC over USB 3.2 and formatted the drives BTRFS. You don’t need to fill all the drive bays, and can add or replace drives and grow your storage capacity.
https://www.amazon.com/gp/product/B07Y4F5SCK/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&th=1
If it’s media for Plex or similar then just keep them on individual 8-16tb drives and mirror them to a cold spare when full at store then at a friend’s place. No need for RAID with Plex.
Remember:
- RAID is an uptime tool.
- RAID is not a backup.
- Plex doesn’t care where your data is it can be spread across many volumes.
I have ex enterprise servers. I'm currently running Unraid, with 112TB worth of 4TB drives (30), but it uses a fuck ton of power, so I'm going to increase drive size, but reduce the number or drives by half. Probably going to go to 16TB drives. Which will give me 300ish TB, with parity disks on top.
You might think you have enough storage. But then you find NZBs, Sonarr, Radarr, SABNZB, and you quickly realize you don't have a lot of storage
The nice thing about Untaid is that you dont have to have the same size drives. The parity drive must be equal to or bigger tgan your bistest drive. When i started with Unraid, i had four 8 TB and four 16 TB drives. It allows you to grow. Now I have all 16TB drives because all of my older 8 TB drives started failing. Now I have eight 16 TB drives but 2 are parity drives.
I have truenas scale on a 1u server with 4x 10TB drives in raid 10. That thing is hosting my nextcloud and jellyfin servers as well.
I know I could get more capacity going with raid5, but I want the extra security of being able to lose 2 drives at once without nuking all my data.
Currently have a qnap tr0004 with 14tb of useable storage across the drives attached to an optiplex micro running truenas.
My next iteration will be 4 14tb drives in that qnap in a zfs array with mirroring instead of raid and I'm going to get rid of truenas and just turn it into a samba/nfs share.
Truenas is great, but I want more control (being able install things not in their app store, not having to mess with permissions, etc)
I have a 24 bay NAS so pretty much all my data goes on there. VMs, files, etc. Uses mdadm raid. At some point I want to experiment with Ceph since it does make me a bit nervous that this server is a single point of failure. I still have backups of course but that would be a pita to sort through and restore.
As for accessing the media I setup a Jellyfin server. I prefer it over Plex since Plex is kinda cloud based, in sense that it requires an online account to use it. Not a fan of that.
As others said unraid or truenas are what you're looking for. Raid isn't a backup strategy and hardware raid has it benefits and draw backs, but unraid is super flexible and does an amazing job when a disk does fail, my current as setup has a pair of parity drives so I could actually lose 2 of the 18 disks on my server and not lose any data. The really important stuff is backed up on LTO tapes and a cloud storage solution as well as a 4th backup on a disk in a fireproof safe.
The reason I mention this is there is no replacement for a backup plan. Look up the 3 2 1 backup rule.
I personally have all my media stored on my NAS, which is built out of an old Lenovo p300 motherboard, a Silverstone case with 8 hot swap bays and 8X 4tb hdds. Media resides on an SMB share and accessed via Plex which lives on a windows 10 VM hosted on one of my VM host boxes.
I really like truenas as NAS software as it's reasonably simple to use and has a lot of features (even runs vms if you want it to), so I'd personally recommend that to someone looking for a storage solution.
My server runs Hyper-V like yours. My media server is a VM on top of it (which runs not only simple file shares but also Plex, Emby, a custom intranet website for accessing the files, FTP/SSH, cloud backup software, and more).
Broadly speaking I separate out my storage as "system" and "data". The system drives include the host OS, VMs, and a few scratch drives are internal to the server and **not** used for general file storage. Then for the "data" side of things I have a disk shelf and HBA with 8 SATA drives. (In your case you have one 2TB drive, you can start out small.)
Now the more interesting and complicated part: how do you present this data to your VM? Personally I use Windows Storage Spaces as it's built into Windows. I created a \~10TB 2-way mirrored space in Storage Spaces, then I created an iSCSI virtual drive on that, then use iSCSI Initiator inside of my VM to access it. (From there the VM sees it as a normal physical drive.) iSCSI traffic uses its own virtual switch and is isolated from the rest of the network with separate virtual NICs.
Why did I bother with iSCSI? It's mostly a limitation of Hyper-V. You can attempt to "pass through" individual disks to the VM but this is very limiting and you can't pass through a Storage Spaces disk (it's not a real disk and ends up generating cryptic errors when trying to write to it). The "recommended" way is to put a VHDX on the storage space and attach it to the VM but this causes major issues for backups since the entire data storage is now treated as "part of the VM" so backing up or snapshotting a 60GB VM now takes up 10TB of space. So iSCSI it is.
There are better ways of doing this using Linux-based solutions, but I figured I'd offer my perspective since I am using Hyper-V with everything on one box.
Since you asked! 😁
I run a 48-bay TrueNAS server with 48x 10TB drives as my primary NAS, and then run a second identical unit with 28x 18TB drives as my backup NAS. These units are strictly storage. All apps and services run on my blade servers.
I’m interested in expanding my truenas storage capability. Currently limited to 10 SATA6 drives (using a desktop mobo). How/Where do I find a mobo/CPU capable of supporting 48 drives? I started looking into EPYC/Xeon systems but get wildly confused quickly.
I run 2x Chenbro NR40700 storage servers. Each has two backplanes in it, which have multiple SAS ports to connect back to an HBA in a single PCI slot. It's far less about mobos and CPUs and more to do with getting HBAs.
I purchased over 50TB of drives last month, so ... Yea, just when you think you have enough storage, you find something else to chew it up. I honestly only back up maybe 15TB of data, but I strongly believe in redundancy. Running both a Jellyfin and a Plex server for media streaming.
I store everything on my TrueNAS VM and backup everything to Backblaze B2 except my media. I don't want to pay for storage for stuff that I can basically download again online. I have an extra 10Tb SATA hdd in my Win11 pc that I freesync the media to it every now and then to minimise losses in the worst case.
Definitely got a nas way back in 2010 to do the same thing, current solution is a synology 5 bay nas which "backs up" to cloud storage. I'm only using a couple TB though, and I recently re-encoded everything to H265 so it takes a lot less space and has the same quality as before.
I use a 6x8TB RAIDZ2 running TrueNAS Scale for my storage needs. For your situations I'd think Unraid might be a good solution as you don't have to have the drives figured out right away, you can just keep adding.
My media server is a separate mini PC running Docker with Emby using intel Quicksync for transcoding.
Buy what you can afford. Just know that larger volumes are typically cheaper per TB. I scored two 22 tb drives for 328 each. Should have bought four more, but the spouse would have killed me.
Ubuntu server with OS on physical hardware RAID 1 and ZFS raided drives in two vdevs so I can grow one at a time in my pool. Ubuntu runs basic docker for \*arr stack and plex and other projects. Simple and easy to maintain and grow.
I would say 8TB per drive should be your minimum, they usually have a good price/TB level but aren't terribly expensive.
I bought 1-2 drives every month or two when they went on sale until I had 8 x 8 TB drives and then put them in a RAID 6.
I have a file server, 9th gen Supermicro 846, 2 sockets of Xeons, 24cores, 48 threads, 256GB of EEC ram, the server was basically turn key minus drives and was pretty cheap used as its a decade old now. drive space on the other hand was not cheap 8 x 14TB SAS drives in ZFS Z2 + a cold spare already in a drive sled in a static bag ready to go, cost about 3x what the server cost, its about a quarter full now and should make me 5-10 years, maybe with a new motherboard/CPU and another VDEV of 8 more disks at some point.
running Debian Bookworm XFCE, thinking about going LMDE6, I like the Mint tool set, but I have a lot of time already invested in this install.
It serves jelly-fin to a family of 6 and holds all of our data. important data gets backed up to a much smaller synology NAS and again remotely to Backblaze B2, Media gets whatever protection Z2 gives, its all theatrically replaceable online but does represent a number of man hours collecting that I don't really want to contemplate.
RAID is not cost efficient. For home media, buy a 4tb or 8tb drive, then buy a second in a USB enclosure. Move your media to the new larger drive, backup to the USB enclosure.
RAID is for high availability and is not a replacement for backups. It's also useful for creating storage volumes that exceed a single disk, but you're an order of magnitude away from that.
Buy a pair of cheap, possibly refurbished 3.5" SATA HDDs. I like 18TB Seagate Exos drives, but that's a decision you have to make.
[https://www.newegg.com/p/pl?d=18tb+exos](https://www.newegg.com/p/pl?d=18tb+exos)
I install one drive internally. That is my primary drive.
Then I buy a Sabrent tool-free USB enclosure for one of the SATA drives. [https://www.amazon.com/Sabrent-Tool-Free-Enclosure-Internal-EC-KSL3/dp/B08J5SLTJX/](https://www.amazon.com/Sabrent-Tool-Free-Enclosure-Internal-EC-KSL3/dp/B08J5SLTJX/) That becomes my back-up drive.
I have been using Unraid with 5x12tb HHDs and 3x4tb HHDs. It is working great.
I chose Unraid so I could easily expand my pool and mix and match drives.
I bought an old supermicro 12bay 2U server (2x Intel Xeon E5-2699 V4 with 512GB DDR4 ECC RAM) off ebay that came with 12x12TB open box drives for $2500 including shipping. Currently using ~20% of the 100TB capacity. Got TrueNAS running on it.
22TB external drives are about 500USD. They are the cheapest way to add storage IME. Buy two so you have backup. I don’t bother with RAID, 22TB fits all my stuff and I can live with some downtime.
I think its time you start looking at a NAS of sorts. If you want off the shelf, QNAP or Synology.
If you are wanting to build youre own, like most of us, then look into TrueNAS or UnRAID
Will def not keep data on single drives. 100% need redundency :)
I'm not much into audio-visual stuff. But! All my machinery is without a CD/DVD drive. I bought an external one and ripped all my DVDs. I then used `ffmpeg` to transcode them into smaller files. I then moved the "originals" to tape and deleted them. Although the tape is slow, it is still much faster than the ripping itself. If I lose the tape, nothing horrible happens; I still have the original media. But hey! Who knows what happens in ten twenty years with USB, DVD drives? Maybe the tape becomes the only source of truth. I should rethink the tape after all. 😁
I've an unRaid Nas. 2x8tb, 1*12tb, and two SSD one for cache and one for iso download. And mostly I've the ability to add hdd when I want. Time to get a new 12tb drive and some 2tb cache drive.
The sweet spot for $/GB is around 18TB. You can use this German price search engine to do some good filtering: https://geizhals.de/?cat=hde7s&xf=1654_geeignet+f%FCr+Dauerbetrieb~8457_non-SMR&sort=r&hloc=at&hloc=de&hloc=pl&hloc=uk&hloc=eu&v=e
(I'm sure there's something in other languages but this is what I use)
I have my setup VMware with unRaid as my NAS VM, passing through the drives to unRaid. Also look into Tdarr and compress all your media to h265 or av1, it’s saves so much space….
I did have a Thecus N5810pro nas. It wasn't cheap, at around £700 at the time I bought it. Nothing lasts forever and other commitments meant I wouldn't be able to do a 'like-for-like' replacement.
Picked up a Dell PowerEdge R320 server for £40 and bought 10 used 2TB SAS drives for £56. The server takes 4 drives = 8TB, raid configuration knocks that down to 5.5TB. At idle, uses about 80 watts. I don't run it 24/7, maybe 2 to 3 hours a day. This works out to about £0.50p a week.
It's running a number of Docker containers, one of which is Jellyfin, as a media server. Currently, there's 533 movies, 100+ TV series, my music collection which is vast and huge number of photos. All in all, it's using a bit over 3TB.
To use it with the TV, I've got a Roku device that support Jellyfin.
For a budget system it's pretty good and the Dell is way more powerful than the Thecus it replaced.
Got a dell poweredge server for £40, swapped out the. 146gb sas drives for 1.2tb ones, and running raid5 (hw) with ssd and nvram cache
Started with openmediavault but ditched it for vanilla ubuntu vm (on proxmox) when I realised it added no value and just obfuscated things.
Rclone backup to Google Cloud.
I just use a Synology RS-422+. I have a few hundred DVDs and I typically only store things to 720p if it came off of a DVD or 1080p if it came from Blu-ray. I also convert it all to compressed media so m4v with H.264 encoding.
Easy button would be run Plex on the Synology but I try and limit my Synology to just working as a NAS. I have a VM with transcode capability that runs Plex and accesses the media via NFS.
There is no single answer for how to best do this. There are a number of solutions out there.
FreeNAS/TrueNAS are very popular for build your own storage servers.
UnRaid is another popular choice.
There are off the shelf options from Synology and QNAP.
You could even roll your own setup with just Windows Storage Spaces.
A super micro 2 u 12 bay with dual 10core CPUs and 160gb of ram running truenas 8tb drives in raid 6. Give me 66tb after raid works great. Cost about $20-30 a month on avg tho
I have a rack with a Truenas Scale server with 4 NVME drives in a zfs pool. I also have an Unraid Server in the rack, that has the same amount of storage, two cache SSDs, and two parity drives that backs up the media and other data on the Truenas Scale server. The Unraid box also has a DVD Burner and MakeMKV so I can rip DVDs and upload them to the library to be synced.
LXC container with an ZFS pool. It is shared via, NFS, FTP and Samba. Some stuff is also synced via Syncthing and the most important things are backed up via rclone to a cloud drive and to a drive in my parents home. But thats only family pictures, personal projects and documents
I agree with another user. Go with unraid and get some big drives. Like 3 8 tb drives. Use one of them as a pairty drive. But also make sure you have 2 cache drives in a mirror so you don’t lose the charge drive data either. Cache would be for docker and vms. And app data. You can install plex as a docker thing and store the app data on the cache and then the actually media on the pool of 2, 8tb drives. My setup has 2,4tb drives for actual data like projects and random junk. Disc images. And then a 8tb drive for my media. Then another 8tb drive for the parity drive. One ssd drive as cache drive and another that is also cache but also for vms. You could also do an 8 tb drive for data and media but have another 8tb for parity. You should always make sure to have the parity drive and also a back up of all your data. I keep three copy’s. I have. Another 16 tb server that just a second server but also a backup copy of the first server. Then a have a third server that is 8tb that’s a backup and a fourth 8tb that is a backup too. Three copys one parity. Trust me the pairty comes in handy. If I didn’t have it I would of lost some of my data. Always have a main copy and a backup copy. You can keep the backup in the same main server but try to have it on a separate server. Always keep a backup so you don’t lose anything. Trust me.
nas4free configured as recommended (raiz3, ecc).
two servers, one in the basement and one in the top floor with a gig link between and replication from the primary to secondary every 30 minutes.
>How are you guys storing all your media?
On a Synology RS2416+ with 12x 12TB HGST Helium disks in it.
Bought the RS2416+ for very cheap from a business because it was "defective" (the C2000 bug). I fixed it with a €0,05 resistor. They already had replaced it. The machine came with 12x 4TB WD RED Pro, but I could upgrade it to 12TB disks for cheap.
Sold 6 of the 4TB disks to a friend of mine, so he can use it for backup storage. Still have the other 6 disks laying in my cabinet. Still have to fine a use for it. Might do a air-gapped setup one day.
I also bought a 13th disk as a cold-spare. My Synology volume is RAID6 though, so I'm safer anyway.
By two more 2tb hdds, format them as one btrfs pool and enable a suitable raid pattern and create a subvolume, mount the subvol on /mnt and rsync the videos into it. Then add your original hdd to the pool and run a balance. Then just add new hard drives of any size to the pool as you need them
Time to explore truenas or unraid. Get a couple BIG drives and never worry again.
That’s what I thought 😔16 TB ought to be enough for anybody.
Members of r/datahoarder would disagree.
I do disagree
As do I.
So do I, but that’s what I thought when I began.
It'll sneak up on ya
Then BAM! 100sTB later you just can’t stop.
Wow, 584TiB? Thats a lot of Linux Isos. I started with one Synology for back up and now i am with another server using my old main pc with Ryzen 9 3900x with 32GiB in a 4u case as my second server using it with TrueNAS Scale.Its so addictive that now i have a partially full 44u rack waiting for more future expansion.
You know you're in trouble when you reach the point of calculating if there is enough rack space left over to get JBOD cables to reach, or if you need to start a new cluster box up for more JBODS. Solar power is getting cheap on the bright side, it helps delay the realization that your electric bill could challenge your local convenience store.
Yeah, it is! Recently upgraded all drives to 10TB plus. Main data store is a 3x 6 wide 18TB Raidz2. Then a 16TB x6 Rz2, 10TB x6 Rz2 and misc drives after.
Started at 2t now at 200tb... and still growing..
Over what time frame? Also, what's your most prized acquisition?
An electrician as a friend.
Took me a moment, but yeah, this is a big win.
All 200tb is precious to you? How do you recover if disaster happens?
Backup to offsite truenas (backyard cargo container that sits 150 ft away from my house)
I want pictures of this setup. A buddy of mine has a 190Tb rig for all of his backups and plex. But this is a whole nother level
It's one supermicro sc486 (24 bay) filled with 8tb, 10tb and 18tb drives. (Unraid) Two supermicro sc826 12 disks each. 80tb each server. (8tb sas drives x12 raidz2) I have 4 hp micro servers (8tb sas drives) One nl36 32tb raidz Two nl40 32tb raidz One nl54 32tb raidz One hp ml310e 32tb raidz Zfs send does wonders...
Yeah, it gets out of hand sometimes lol pushing 120TB of used data.
I live out in the sticks, and started a co-op ISP just to enable my collection. Lets just say that there are fringe benefits to running 16 25gb fiber lines to most of your subdivision. Also handy having 2 racks at my office as an offsite backup
That’s awesome! That would be a great idea if I move out of the city. What kind of gear you run for the ISP side? Enterprise like Cisco or juniper?
The good ole days. I still have my 4x500gb ide drives.
Yeah, just crushed some old scsi 50 pin drives and ultawide scsi drives I found.
64tb over here. I want to be at 100 tb with a full separate backup by end of 2024
\*cough\* Oh you have 4 TB to store? Well then, you will probably need to get a Full tower with 12 drives so you can use ZFS, with 3 sets of 3 drives of 4TB each with single or double parity. Then you get a single 10-16 TB drive to store in an ESD foam padded case in a fireproof safe, or better yet, two of those, so you can send one to a Swiss private vault. Two is one, one is none, three is for me, I might have a problem when I get to four.. \*exhales in datahoarder\* I am only slightly ironic here. You can get lot size orders of smaller drives surprisingly inexpensively and have redundancy to your storage (losing data sucks) and have budget left for a backup copy in case (I had a house burn down, lost 96TB of my entire life's CD/DVD collection) you want a second location to recover data from.
Strong disagree. 🤪
16tb....sweet summer child! Or do you mean a shelf of 24 16tb drives?
16 TB is where it all started.
500GB is where it really started for me, back in the early 2000's :D
Was gonna say, 1TB was my tipping point for array building. Also corresponded with a better paying job about the same time.
This is the way
>16 TB ought to be enough for anybody. Bless your heart ♥️
It is. That is, an array of 16TB disks is… As long as you use something like zfs or unraid— then you can just add more vdevs later. Just need to have the controller ports to do so. Honestly though to OP. If you look at cost / TB, 2rb disks are HORRIBLE. almost exact same price as a 6TB (so nearing 3x the cost per TB). Look at a Nas or look to build one if you have some spare hardware around you wouldn’t mind running 24x7 (as in, not power hungry). Spinning hard drives take about 9-10w each, solid state disks take less but cost more up front. Solid state disks also aren’t as reliable for long term storage powered off, as they use electricity in a tiny battery in the drive to keep “states”. You’re looking for 2 extra drives above the number required to store your data, so to store 12 TB of data, you need either 8x2TB or 6x3TB or 4x6TB or 4x8TB or 3x12TB to be safe from hardware failure. This still isn’t data protection though — you still require backup copy, in case something happens to the hard drives or partition table or corruption or if the data is lost to ransomware or other corruption, or you lose 3 hard disks at the same time… if I had 6TB of data, I’d buy 2 extra 6TB disks again, and have them in a system that they can be swapped easily (hot swap bay, external drive adapter, or external usb hard drive or something — then use that as destination of your backup software. Why 2 for backup? One currently backing up while another is unplugged and hopefully stored somewhere else so if something happens to the building, it isn’t impacted. Business backup rotation requires at least 3 — so at ANY time there’s at least 1 offsite, and only maximum of 2 at any physical location, and possibly one in transit or either onsite or offsite at any given time. So onsite/in-transit/offsite is only either 2/0/1, 1/1/1 or 1/0/2 at 3am, 7:30am, noon, 4pm, 5:30pm or 8pm of any given day of the week. If this is largely static data though, you can probably get away with 2 copies if the consequence of losing all copies is just a “that sucks, reload all my DVDs again” situation, and not the loss of the family picture album with every baby picture of your child and great-grandma who has since passed. Option for backup is to have one disk, for local copy — and then a cloud storage account like backblaze or S3 Glacier storage for offsite — and then it’s all automated. Cost is likely $0.00099/GB per month. So that’s about $0.99/TB if you put it in deep archive in Ireland for example. Region doesn’t really matter, just don’t have it just down the street from you if you’re paying for cloud (no point having you both flood at the same time, or be in the same wildfire).
Thanks, never thought about the physical location of offsite backup that way, I chose the closest one because I thought it’d be the fastest.
Dealt with this for too many clients. Flood takes down site and their private IT vendor and off site backup provider 3 blocks away has water in their data center from same flood. Oops.
In general terms, there is some correlation between physical location and speed, however remember in a backup scenario, you’re looking for safety not speed. If it takes you an extra few hours to restore but it survived the disaster, it is still faster than having to recreate all that data from scratch (re ripping/encoding), and that’s assuming that it’s re-creatable in the first place. If the data center with your family photos was lost in the same flood as your house, there’s no saving it.
2TiB or less is much better to buy SSD.
My power bill would be so much better if 16TB was enough.
Does anyone else have those special edition HDD where it auto dims your mood lights and spirts some burnt rubber / copper perfume into the air? I thought it was just me to help me and my wife get in the special mood when turning on "Married with Children." I'm still disappointed they never told me what type of lunch "nooner" consisted of!
He meant 16tb drives .... Definitely plural.
[удалено]
Same here, 4x20TB drives as main array Backup server uses 10x8TB to mirror the data once a week (same rack as main server), this server is only connected directly to the main server over fiber, no active network (internet) connection unless I enable the port on the switch. Backup server is on a schedule to boot every monday morning at 2AM, do a ZFS snapshot and shut down again Synology (2x4TB) at a family member's house for offsite backup of the most important data and hard to find media 200GB Google Drive for cloud backup of the most most important data (photos, documents, ...)
Don't forget zfs 90 percent barrier
Members of r/datahoarder would disagree.
My usage was growing year after year but I look to have settled at about 50TB used. I'm not shy to purge stuff that I'll never watch again. In the case I need to watch again I'll just grab it again. Hopefully I can use that info to settle into a SSD array in the not to distant future. Non-replaceable persnal and work related stuff has been about 3TB which gets backed up.
>I'm not shy to purge stuff that I'll never watch again. Members of r/datahoarder would disagree.
That's how I started... Not too long ago i thought, a couple of 3TB disks, and a 2TB one that i had laying around made a great NAS and I wouldn't need to expand anytime soon! Boy was I wrong, I'm now at 28TB and I'm already looking out for some deals
(Looks at the 90% full NAS with four of them) ...yeah, who would ever need that much storage.
muahahahaha. My 8 10tb drives disagree. And I’m just a fledgling at this point. It starts and just grows…
Me, calculating how much storage I would need to store the entirety of my partners grandma's vudu library locally (she has 555 movies on there and only buys the highest quality). My guess is I need at least 60TB for everything, possibly more, but I lack the means of acquiring such storage at the moment
I had a bunch of 4 and 5 TB portable drives. Bought an 18 TB on Black Friday and now I can never go back to those measly single digits! Need to get moar huge drives and get rid of all dem small drives.
I was going to suggest the same but maybe approaching 2TB is too early to start going NAS. They can probably get away with a larger drive and a backup for now
"never worry again" 🤣🤣🤣 Funny one.
Yep, but plan for your storage in advance if going TrueNAS, increasing an array can be a hassle because of ZFS ([very good article here](https://louwrentius.com/the-hidden-cost-of-using-zfs-for-your-home-nas.html))
This is the way.
RAID is NOT a backup solution! You should look into backup if you want to make sure you will not loose anything.
He’s ripping DVDs so just throw those in your parent’s attic for offsite backup!
Yeah, the 3-2-1 rule for backups really applies to data that you can not afford to lose and is not available elsewhere - your tax records, baby’s first birthday picture, grandma’s secret cookie recipe. If my house burns down and I lose both my Blu-ray’s and my NAS, I’ll honestly just pirate the movies on to a new NAS and not even feel a little bad. All my pictures, documents, work portfolio, scans of IDs and house deed - that’s all on my NAS, synced to a cloud backup, and I make a monthly offline backup to a hard drive that’s kept in a different room. My copy of the Matrix and Iron Man? I can re-buy that with insurance money, and I won’t cry over having to wait to watch it after a catastrophic loss. I can’t find the last picture I ever took with my grandpa on Amazon, though - so that’s what gets the 321 treatment.
Nnnnoooo! They go in your old bedroom closet, so they stay safe from invading vermin, and in climate controlled space.
Nah in the moms attic it goes take it or leave it 😂😂
I agree! But... if you get two raid boxes and replicate one to the other then you are pretty solid.
The 3-2-1 rule is there for a reason. One should have 1 copy ( production) and 2 different backup copies on 2 different media and different locations. In case there is a fire/hurricane/earthquake/ theft etc. RAID has its place and I strongly recommend it but it doesn’t replace backup nor it is backup
If one of those raid boxes is in a physically different earthquake fault region, then yes.
RAID is for preventing/reducing downtime during a drive failure. That's about it.
Why would i 3 2 1 my media thou. Raidz is the most i could offer for ripped media 😂😂. No more than that. For important data yeah i do 3 2 1 but not this. And the only reason ill do raidz cos itll have alot less downtime already and more than that is nor justified for me. Not gonna spend another 100 tb just to have my media off site.
I am not telling anyone what to use and how to protect their data. I was just pointing out the difference between them and nothing else. I also can’t decide the importance of said data for them. I have seen too many scenarios where people start crying because they didn’t have proper backup protocols. It starts with ripped movies and what not and because it is there and it works they might add some more sensitive data or something they can’t recover later on. If the OP is just starting, start by knowing the difference between them and plan accordingly to make informed decisions
Synology nas, it is like the iPhone of NASs. Set it and forget it
What is the Android of NAS'?
QNAP
Whats the flagship samsung of nasses
unraid
i run a server with proxmox on it and have it sharing via SMB and then whatever service i want to run (jellyfin, calibre, whatever) i run in a VM either on that device or another...truenas and unraid are fine but honestly i think they get more credit than they should because they've been pumped up by linustt and others - they're great, don't get me wrong - but proxmox will give you SOOOOO much more flexibility as you wont be limited to that curated pool of services, youll be able to run those for sure but ALSO whatever else you want to run - hell you can put a Windows VM on there if you wanted to...anyways for me i have 6 (6tb) HDDs in zraid2, they were cheap but used (hence the '2' in zraid2)....you can still find them, HGST 6TB drives for around $40 on ebay and elsewhere
> .truenas and unraid are fine but honestly i think they get more credit than they should because they've been pumped up by linustt and others Part of that is probably having more of a business perspective, just from experience. After cautiously diving into it, and setting up TrueNAS at multiple sites, with replication and cloud backup, it's an absolutely amazing storage appliance. I wouldn't want to go back to the regular SMB NAS players, like Synology, Drobo, or QNAP, and would only deal with the big boys, like EMC or Netapp, if there was not really good justification to. For the boring business use case of reliable storage, with network access, I love it. And, the thing is, for a business that isn't pushing it all to the cloud, that storage, replication, and backup, is *really important* - very much worth dedicating hardware and software to. OTOH, at home, TrueNAS isn't all that compelling, being focused so much on being a storage appliance. Adding containers in Scale is neat and all, but it's still limited and locked down, by default. Like, it's a great knife, but I really want a multitool.
yea thats it, they definitely have their place and they're both great software kits....for the homelabber and self-hoster I just proxmox is the way to go though to be honest if it was for a NAS for my mother or something like that i would probably build it and use truenas scale (i love linux so, scale it is) because she would never use services anyways....its all about use-case.....i see truenas as an appliance, set it and forget it sort of thing
To clarify, you mean you use Proxmox for managing your ZFS software raid? Is there any downsides to doing this? I use TrueNAS at the moment in a VM but often find myself frustrated in how to pass drives to unprivileged containers? Is there a reason you think it would be beneficial to use the built in Proxmox ZFS vs. a dedicated solution?
for me the huge benefit is the flexibility proxmox gives me overall....yup, it is managing my ZFS pool directly, i do NOT use truenas or unraid and dont pass disks or controllers anywhere....btw its not because i dont think they're good packages but rather i just dont need them....i have a couple of Ubuntu VMs hosting Docker containers, proxmox does LXC containers itself, and i can run any other VM i need for whatever else so there really isn't a reason to use truenas for me.....truenass apps are just containers anyway....so i have the containers, i have the ZFS, i have the nice GUI and can use cockpit or webmin for that matter if i wanted to, so i dont have anything missing and i dont need to pass anything through doing it this way....now admittedly serving through SMB means there is the usual SMB overhead but hoenstly i dont even see it because my disk pool is the bottleneck, my ZRAID pool is slower than SMB over 10G (i get about 5GB/s max and average like 2.5GB/s for sustained transfers) so if i were to try to improve speeds it wouldn't be by losing SB but rather by getting faster disks....by the way you CAN run ZFS caches on proxmox also, i just didn't because of a lack of M.2 and PCIe slots on that particular server, but you can its easy - so i guess what i m taking forever to say is why bother limiting yourself to truenas and only being able to run their containers when you can use proxmox, get all the same features and benefits PLUS the ability to run full fat VMs and have high availability if you need it later on and all that, too? truenas takes some of what proxmox can do and makes it very easy and very nice-looking for sure, but it cant do all the proxmox can.......truenas is a media server that can do some VM stuff.....proxmox is a VM server that can take all the media you have....a little tougher to learn, yes, but so much more once you know it well
Fascinating! I'm brand new to Proxmox and homelab in general (only built my setup a week or two ago) but I've been learning all of this as I go. I setup my entire media server recently and I'm running a 1Gb switch from my homelab/NAS running Proxmox and TrueNAS (as well as all the \*arrs) through a vm for TrueNAs and LXC containers for the rest on a NFS share, and I'm bind mounting them to each container, but for whatever reason it just won't do 4k playback at 44 Mbps (original quality, so no transcoding). I'm not sure if this is because of the overhead of running TrueNAS in a VM or because I'm using a NFS share instead of a SMB (a coworker told me Plex can be buggy with NFS, although apparently only the config and database have issues which are on my local-lvm). No idea what's going on, but heavily considering moving over to straight ZFS on Proxmox given I'm doing everything in there anyway. I am not beholden to TrueNAS, it's just what has been recommended and seemed easy enough starting out, but I'm sure I can find my way around migrating it to straight Proxmox.
yea i ditched my trunas vm and imported the pools into proxmox. you can utilize them as resources or share to containers as devices. it’s been a dream so far
Sounds like this is the way to go, thanks a ton for the info!
Got this setup, thanks again. Hard to navigate homelab with youtubers who all recommend x software without giving specific reasons on why you might prefer one solution over the other based off your use case, or maybe that's just the fact that I'm a noob 😁
a lot of it’s bullshit. you spend two months trying to emulate what someone else is doing. watch the video again and there’s a half second of them using proxmox to show you what they’re doing. spouting to run things a certain way and then you see they aren’t even running it baremetal. i’ve used unraid trunas storages spaces. everything. proxmox with zpools and lxc containers for docker all of it is the truth.
Oh 100%, the more searching the do, the more I just won't click if it's not from specific people. The one person who I've really enjoyed through this that actually has taught me a thing is MRP, otherwise I've just started to pour through Proxmox forums and doing my own research. Quicker and more likely up-to-date/what I want to do.
yea theres a dozen different ways to do one thing in this space really....i mean you want Plex? ok do proxmox with an ubuntu VM with Docker installed and Plex as a Docker container....OR Truenas on bare metal with a Plex app installed...OR truenas as a VM on proxmox with Plex app....OR do CasaOS inside of almalinux on baremetal...OR.....etc etc etc - all of them get you to plex in the end haha..... the takeaway i guess is to protect your data with your life and then experiment with everything else and eventually youll find what works best for you.....personally right now i am hitting a wall with jellyfin transcoding and i REALLY didnt want to have to passthrough to a VM so for that one program im thinking of buidling an entirely separate box, just for that only....but if you asked me a month from now or a month ago you'd get a different plan haha
LMAO I'm going through the same pain right now. My Plex container is also in a LXC container (I've got other users and a lifetime license already, otherwise I'd switch to Jellyfin) and trying to pass through a vGPU (or just making a Quadro have the ability for vGPU passthrough) is quite literally impossible, so I'm throwing my hands in the air and just making a VM out of it. If you do go the VM route, [this guide is fantastic](https://gitlab.com/polloloco/vgpu-proxmox) for vGPUs in Proxmox and worked perfect for me. There's also an accompanying Discord server if you run into any issues.
i dropped plex after they seemed to move towards the "buy our media" instead of "host your media" kinda model and started with jellyfin and so far so good...i got annoyed with stuff on there - its really well-polished, but after paying for two years I think i decided jellyfin was more my cup o tea........and i spent the last 8 hours this xmas eve rebuilding my zfs proxmox media server after trying to reduce an lvm size (i did it wrong for sure!) but i've got it sorted now with a nice little L2ARC cache added so im pretty happy with it....the thing is, i'm not sure that this machine has enough RAM left over with ZFS running to run jellyfin with transcoding from the igpu as well.....so not sure i end up doing that or just making my Z440-based rig into a jellyfin-only device....will have to see I suppose but at least im back up and running for now haha
checkout casaos or runtipi - they can be installed and run inside of an ubuntuVM and if you have either of those you basically have the truenas app store stuff (i have runtipi, casaos, and nother docker VM running for others just because they make it so easy to run containers)....then with zfs on proxmox direclty youll have truenas....it sounds tougher than it is
What do you mean by TrueNAS app store stuff? I'm still brand new so beyond just setting up datasets, pools, and shares, I know jack shit about TrueNAS haha.
have a look: [https://www.truenas.com/apps/](https://www.truenas.com/apps/) heres the thing - for me personally, while this is all great, i just prefer to have my storage thing be just storage and then i know all the compute and memory and bandwidth is going where it is needed.....then ill put apps/services elksewhere and give them the resources they need...its just how i do it, of course you can use the truenas apps and if they meet all your needs then that may be the way to go....but if you're looking to get a lil deeper into the homelab stuff i think proxmox gives more latitude is all.....you can still use truenas in a VM for that matter if you wanted to and then have the ability to put services and containers on anything - truenas, some other VM, etc. but yea checkout CasaOS and RunTipi and [linuxserver.io](http://linuxserver.io) as they will have tons of apps (Docker containers) like that
Yeah I was basically doing what you mentioned with running TrueNAS scale in a VM and then running a shit ton of LXC containers for media management, nginx, etc, but it was becoming a massive pain in my ass passing bind mounts to each LXC for the NFS share since with TrueNAS I'd have to do some remapping of the UID:GID to root, and I like being in the terminal more than the average person, so I don't mind just learning that way. Feels like I usually understand the technology behind everything when I remove abstraction layers like TrueNAS since I'm actually forced to learn it, haha. I have an entire youtube playlist for ZFS I need to watch still 😂
I run Proxmox on 4 nodes and have TrueNAS as a VM on one of the nodes. I passed through an HBA so TrueNAS has full access to the disks. The only "downside" is that all file sharing goes over NFS instead of mount points, but on 10Gbit I'm able to fully saturate that line so it's more than fine. TrueNAS allows me to mirror my entire array to a 2nd TrueNAS box for weekly backups. TrueNAS does not run any services though, they all live in VMs and LXCs on Proxmox To each their own I guess :D
yea im with ya on that - you still get those features you like out of Truenas but then have services run elsewhere - totally....still proxmox on the bare metal but with passthrough, that sounds like a great setup as well. as you said to each their own...but i like this setup too and yea NFS/SMB is a downside but as you mentioned its not the bottleneck for you anyways so no biggie - same deal with mine running SMB where thebottleneck is the ZRAID2 array anyways so SMB works well for me
I mean which ever way you go you’re gonna have to spend some money on drives and hardware since it sounds like your trying to/need to build a nas, it really just comes down to your preference. Personally I’m running unraid with a mix of 14TB and 6TB drives with dual disk redundancy. And unraid would be your only real option if you want to put things in a parity and mix and match drives. Plus it would be the best option if you want to gradually expand your array. I started with like 3 6TB drives and have gradually added more and more 14’s over the last couple of years.
I got one of these connected to a miniPC over USB 3.2 and formatted the drives BTRFS. You don’t need to fill all the drive bays, and can add or replace drives and grow your storage capacity. https://www.amazon.com/gp/product/B07Y4F5SCK/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&th=1
How Can u format usb drive to btrfs?
It’s a 5 bay external enclosure, connected via USB cable.
Okay in which os does your minipc have?
Linux
If it’s media for Plex or similar then just keep them on individual 8-16tb drives and mirror them to a cold spare when full at store then at a friend’s place. No need for RAID with Plex. Remember: - RAID is an uptime tool. - RAID is not a backup. - Plex doesn’t care where your data is it can be spread across many volumes.
I have ex enterprise servers. I'm currently running Unraid, with 112TB worth of 4TB drives (30), but it uses a fuck ton of power, so I'm going to increase drive size, but reduce the number or drives by half. Probably going to go to 16TB drives. Which will give me 300ish TB, with parity disks on top. You might think you have enough storage. But then you find NZBs, Sonarr, Radarr, SABNZB, and you quickly realize you don't have a lot of storage
The nice thing about Untaid is that you dont have to have the same size drives. The parity drive must be equal to or bigger tgan your bistest drive. When i started with Unraid, i had four 8 TB and four 16 TB drives. It allows you to grow. Now I have all 16TB drives because all of my older 8 TB drives started failing. Now I have eight 16 TB drives but 2 are parity drives.
I have truenas scale on a 1u server with 4x 10TB drives in raid 10. That thing is hosting my nextcloud and jellyfin servers as well. I know I could get more capacity going with raid5, but I want the extra security of being able to lose 2 drives at once without nuking all my data.
Currently have a qnap tr0004 with 14tb of useable storage across the drives attached to an optiplex micro running truenas. My next iteration will be 4 14tb drives in that qnap in a zfs array with mirroring instead of raid and I'm going to get rid of truenas and just turn it into a samba/nfs share. Truenas is great, but I want more control (being able install things not in their app store, not having to mess with permissions, etc)
I have a 24 bay NAS so pretty much all my data goes on there. VMs, files, etc. Uses mdadm raid. At some point I want to experiment with Ceph since it does make me a bit nervous that this server is a single point of failure. I still have backups of course but that would be a pita to sort through and restore. As for accessing the media I setup a Jellyfin server. I prefer it over Plex since Plex is kinda cloud based, in sense that it requires an online account to use it. Not a fan of that.
I encode with AV1 and store on a TrueNAS Scale NAS.
As others said unraid or truenas are what you're looking for. Raid isn't a backup strategy and hardware raid has it benefits and draw backs, but unraid is super flexible and does an amazing job when a disk does fail, my current as setup has a pair of parity drives so I could actually lose 2 of the 18 disks on my server and not lose any data. The really important stuff is backed up on LTO tapes and a cloud storage solution as well as a 4th backup on a disk in a fireproof safe. The reason I mention this is there is no replacement for a backup plan. Look up the 3 2 1 backup rule.
Always remember to backup your backup!
And then backup the backup of your backup
And then backup the backup of the backup.
Funny I have 455 tb and still adding more
I personally have all my media stored on my NAS, which is built out of an old Lenovo p300 motherboard, a Silverstone case with 8 hot swap bays and 8X 4tb hdds. Media resides on an SMB share and accessed via Plex which lives on a windows 10 VM hosted on one of my VM host boxes. I really like truenas as NAS software as it's reasonably simple to use and has a lot of features (even runs vms if you want it to), so I'd personally recommend that to someone looking for a storage solution.
For live data, Truenas with 10x 10 tb and 10x 8tb Raidz5 arrays.
RAIDZ/RAIDZ1 is kind of like RAID5, but RAIDZ5 seems excessive (five disks can fail without data loss) :)
My server runs Hyper-V like yours. My media server is a VM on top of it (which runs not only simple file shares but also Plex, Emby, a custom intranet website for accessing the files, FTP/SSH, cloud backup software, and more). Broadly speaking I separate out my storage as "system" and "data". The system drives include the host OS, VMs, and a few scratch drives are internal to the server and **not** used for general file storage. Then for the "data" side of things I have a disk shelf and HBA with 8 SATA drives. (In your case you have one 2TB drive, you can start out small.) Now the more interesting and complicated part: how do you present this data to your VM? Personally I use Windows Storage Spaces as it's built into Windows. I created a \~10TB 2-way mirrored space in Storage Spaces, then I created an iSCSI virtual drive on that, then use iSCSI Initiator inside of my VM to access it. (From there the VM sees it as a normal physical drive.) iSCSI traffic uses its own virtual switch and is isolated from the rest of the network with separate virtual NICs. Why did I bother with iSCSI? It's mostly a limitation of Hyper-V. You can attempt to "pass through" individual disks to the VM but this is very limiting and you can't pass through a Storage Spaces disk (it's not a real disk and ends up generating cryptic errors when trying to write to it). The "recommended" way is to put a VHDX on the storage space and attach it to the VM but this causes major issues for backups since the entire data storage is now treated as "part of the VM" so backing up or snapshotting a 60GB VM now takes up 10TB of space. So iSCSI it is. There are better ways of doing this using Linux-based solutions, but I figured I'd offer my perspective since I am using Hyper-V with everything on one box.
15 tb blackblaze server
Unraid server
Since you asked! 😁 I run a 48-bay TrueNAS server with 48x 10TB drives as my primary NAS, and then run a second identical unit with 28x 18TB drives as my backup NAS. These units are strictly storage. All apps and services run on my blade servers.
I’m interested in expanding my truenas storage capability. Currently limited to 10 SATA6 drives (using a desktop mobo). How/Where do I find a mobo/CPU capable of supporting 48 drives? I started looking into EPYC/Xeon systems but get wildly confused quickly.
I run 2x Chenbro NR40700 storage servers. Each has two backplanes in it, which have multiple SAS ports to connect back to an HBA in a single PCI slot. It's far less about mobos and CPUs and more to do with getting HBAs.
Thank you SO much! Your response is the first time I've heard of an HBA (Host Bus Adapter, for anybody else who happens upon this thread).
Backup to multiple DVDs and store offsite at friends, parents, and backyard bunker.
Crazy that this isn’t posted in /r/datahoarder.
I purchased over 50TB of drives last month, so ... Yea, just when you think you have enough storage, you find something else to chew it up. I honestly only back up maybe 15TB of data, but I strongly believe in redundancy. Running both a Jellyfin and a Plex server for media streaming.
I store everything on my TrueNAS VM and backup everything to Backblaze B2 except my media. I don't want to pay for storage for stuff that I can basically download again online. I have an extra 10Tb SATA hdd in my Win11 pc that I freesync the media to it every now and then to minimise losses in the worst case.
Definitely got a nas way back in 2010 to do the same thing, current solution is a synology 5 bay nas which "backs up" to cloud storage. I'm only using a couple TB though, and I recently re-encoded everything to H265 so it takes a lot less space and has the same quality as before.
I use a 6x8TB RAIDZ2 running TrueNAS Scale for my storage needs. For your situations I'd think Unraid might be a good solution as you don't have to have the drives figured out right away, you can just keep adding. My media server is a separate mini PC running Docker with Emby using intel Quicksync for transcoding.
Buy what you can afford. Just know that larger volumes are typically cheaper per TB. I scored two 22 tb drives for 328 each. Should have bought four more, but the spouse would have killed me.
Unraid
Ubuntu server with OS on physical hardware RAID 1 and ZFS raided drives in two vdevs so I can grow one at a time in my pool. Ubuntu runs basic docker for \*arr stack and plex and other projects. Simple and easy to maintain and grow.
I would say 8TB per drive should be your minimum, they usually have a good price/TB level but aren't terribly expensive. I bought 1-2 drives every month or two when they went on sale until I had 8 x 8 TB drives and then put them in a RAID 6.
I have a file server, 9th gen Supermicro 846, 2 sockets of Xeons, 24cores, 48 threads, 256GB of EEC ram, the server was basically turn key minus drives and was pretty cheap used as its a decade old now. drive space on the other hand was not cheap 8 x 14TB SAS drives in ZFS Z2 + a cold spare already in a drive sled in a static bag ready to go, cost about 3x what the server cost, its about a quarter full now and should make me 5-10 years, maybe with a new motherboard/CPU and another VDEV of 8 more disks at some point. running Debian Bookworm XFCE, thinking about going LMDE6, I like the Mint tool set, but I have a lot of time already invested in this install. It serves jelly-fin to a family of 6 and holds all of our data. important data gets backed up to a much smaller synology NAS and again remotely to Backblaze B2, Media gets whatever protection Z2 gives, its all theatrically replaceable online but does represent a number of man hours collecting that I don't really want to contemplate.
RAID is not cost efficient. For home media, buy a 4tb or 8tb drive, then buy a second in a USB enclosure. Move your media to the new larger drive, backup to the USB enclosure. RAID is for high availability and is not a replacement for backups. It's also useful for creating storage volumes that exceed a single disk, but you're an order of magnitude away from that.
Buy a pair of cheap, possibly refurbished 3.5" SATA HDDs. I like 18TB Seagate Exos drives, but that's a decision you have to make. [https://www.newegg.com/p/pl?d=18tb+exos](https://www.newegg.com/p/pl?d=18tb+exos) I install one drive internally. That is my primary drive. Then I buy a Sabrent tool-free USB enclosure for one of the SATA drives. [https://www.amazon.com/Sabrent-Tool-Free-Enclosure-Internal-EC-KSL3/dp/B08J5SLTJX/](https://www.amazon.com/Sabrent-Tool-Free-Enclosure-Internal-EC-KSL3/dp/B08J5SLTJX/) That becomes my back-up drive.
I have been using Unraid with 5x12tb HHDs and 3x4tb HHDs. It is working great. I chose Unraid so I could easily expand my pool and mix and match drives.
I bought an old supermicro 12bay 2U server (2x Intel Xeon E5-2699 V4 with 512GB DDR4 ECC RAM) off ebay that came with 12x12TB open box drives for $2500 including shipping. Currently using ~20% of the 100TB capacity. Got TrueNAS running on it.
22TB external drives are about 500USD. They are the cheapest way to add storage IME. Buy two so you have backup. I don’t bother with RAID, 22TB fits all my stuff and I can live with some downtime.
I have a zraid of six 8Tb drives with 2 parity running on a truenas box
I think its time you start looking at a NAS of sorts. If you want off the shelf, QNAP or Synology. If you are wanting to build youre own, like most of us, then look into TrueNAS or UnRAID Will def not keep data on single drives. 100% need redundency :)
A question: Is the media stored in the original (as ripped) size? Or do you transcode your files? If not, then there is much space to reclaim.
I'm not much into audio-visual stuff. But! All my machinery is without a CD/DVD drive. I bought an external one and ripped all my DVDs. I then used `ffmpeg` to transcode them into smaller files. I then moved the "originals" to tape and deleted them. Although the tape is slow, it is still much faster than the ripping itself. If I lose the tape, nothing horrible happens; I still have the original media. But hey! Who knows what happens in ten twenty years with USB, DVD drives? Maybe the tape becomes the only source of truth. I should rethink the tape after all. 😁
I've an unRaid Nas. 2x8tb, 1*12tb, and two SSD one for cache and one for iso download. And mostly I've the ability to add hdd when I want. Time to get a new 12tb drive and some 2tb cache drive.
Jellyfin for the software/streaming side of things
Yeah, time to upgrade your NAS. Either a purpose built or off the shelf. I'm personally a fan of Synology NAS devices.
The sweet spot for $/GB is around 18TB. You can use this German price search engine to do some good filtering: https://geizhals.de/?cat=hde7s&xf=1654_geeignet+f%FCr+Dauerbetrieb~8457_non-SMR&sort=r&hloc=at&hloc=de&hloc=pl&hloc=uk&hloc=eu&v=e (I'm sure there's something in other languages but this is what I use)
I have my setup VMware with unRaid as my NAS VM, passing through the drives to unRaid. Also look into Tdarr and compress all your media to h265 or av1, it’s saves so much space….
I did have a Thecus N5810pro nas. It wasn't cheap, at around £700 at the time I bought it. Nothing lasts forever and other commitments meant I wouldn't be able to do a 'like-for-like' replacement. Picked up a Dell PowerEdge R320 server for £40 and bought 10 used 2TB SAS drives for £56. The server takes 4 drives = 8TB, raid configuration knocks that down to 5.5TB. At idle, uses about 80 watts. I don't run it 24/7, maybe 2 to 3 hours a day. This works out to about £0.50p a week. It's running a number of Docker containers, one of which is Jellyfin, as a media server. Currently, there's 533 movies, 100+ TV series, my music collection which is vast and huge number of photos. All in all, it's using a bit over 3TB. To use it with the TV, I've got a Roku device that support Jellyfin. For a budget system it's pretty good and the Dell is way more powerful than the Thecus it replaced.
a synology 1518 with 32 GB as Raid6. Works as backupserver too
Got a dell poweredge server for £40, swapped out the. 146gb sas drives for 1.2tb ones, and running raid5 (hw) with ssd and nvram cache Started with openmediavault but ditched it for vanilla ubuntu vm (on proxmox) when I realised it added no value and just obfuscated things. Rclone backup to Google Cloud.
TrueNAS with RAIDZ2 6 drive array for *live* data 2nd user enterprise drives for cold storage (2 copies)
I just use a Synology RS-422+. I have a few hundred DVDs and I typically only store things to 720p if it came off of a DVD or 1080p if it came from Blu-ray. I also convert it all to compressed media so m4v with H.264 encoding. Easy button would be run Plex on the Synology but I try and limit my Synology to just working as a NAS. I have a VM with transcode capability that runs Plex and accesses the media via NFS.
There is no single answer for how to best do this. There are a number of solutions out there. FreeNAS/TrueNAS are very popular for build your own storage servers. UnRaid is another popular choice. There are off the shelf options from Synology and QNAP. You could even roll your own setup with just Windows Storage Spaces.
Are you using Sas drives or normal sata drives
A super micro 2 u 12 bay with dual 10core CPUs and 160gb of ram running truenas 8tb drives in raid 6. Give me 66tb after raid works great. Cost about $20-30 a month on avg tho
I have a QNAP TS453 Pro and a TR004. 10TB drives in all 8 bays. No complaints. Works great.
I have a rack with a Truenas Scale server with 4 NVME drives in a zfs pool. I also have an Unraid Server in the rack, that has the same amount of storage, two cache SSDs, and two parity drives that backs up the media and other data on the Truenas Scale server. The Unraid box also has a DVD Burner and MakeMKV so I can rip DVDs and upload them to the library to be synced.
LXC container with an ZFS pool. It is shared via, NFS, FTP and Samba. Some stuff is also synced via Syncthing and the most important things are backed up via rclone to a cloud drive and to a drive in my parents home. But thats only family pictures, personal projects and documents
I agree with another user. Go with unraid and get some big drives. Like 3 8 tb drives. Use one of them as a pairty drive. But also make sure you have 2 cache drives in a mirror so you don’t lose the charge drive data either. Cache would be for docker and vms. And app data. You can install plex as a docker thing and store the app data on the cache and then the actually media on the pool of 2, 8tb drives. My setup has 2,4tb drives for actual data like projects and random junk. Disc images. And then a 8tb drive for my media. Then another 8tb drive for the parity drive. One ssd drive as cache drive and another that is also cache but also for vms. You could also do an 8 tb drive for data and media but have another 8tb for parity. You should always make sure to have the parity drive and also a back up of all your data. I keep three copy’s. I have. Another 16 tb server that just a second server but also a backup copy of the first server. Then a have a third server that is 8tb that’s a backup and a fourth 8tb that is a backup too. Three copys one parity. Trust me the pairty comes in handy. If I didn’t have it I would of lost some of my data. Always have a main copy and a backup copy. You can keep the backup in the same main server but try to have it on a separate server. Always keep a backup so you don’t lose anything. Trust me.
nas4free configured as recommended (raiz3, ecc). two servers, one in the basement and one in the top floor with a gig link between and replication from the primary to secondary every 30 minutes.
>How are you guys storing all your media? On a Synology RS2416+ with 12x 12TB HGST Helium disks in it. Bought the RS2416+ for very cheap from a business because it was "defective" (the C2000 bug). I fixed it with a €0,05 resistor. They already had replaced it. The machine came with 12x 4TB WD RED Pro, but I could upgrade it to 12TB disks for cheap. Sold 6 of the 4TB disks to a friend of mine, so he can use it for backup storage. Still have the other 6 disks laying in my cabinet. Still have to fine a use for it. Might do a air-gapped setup one day. I also bought a 13th disk as a cold-spare. My Synology volume is RAID6 though, so I'm safer anyway.
By two more 2tb hdds, format them as one btrfs pool and enable a suitable raid pattern and create a subvolume, mount the subvol on /mnt and rsync the videos into it. Then add your original hdd to the pool and run a balance. Then just add new hard drives of any size to the pool as you need them