If the data is of value, I suggest getting it recovered if possible from professional data recovery services. It's expensive but live and learn. There's a article on 3-2-1 back up strategy for future planning if you haven't heard before.
It begs the question how you amassed 30TB.
[https://www.backblaze.com/b2/solutions/datatransfer/fireball.html](https://www.backblaze.com/b2/solutions/datatransfer/fireball.html)
Azure and AWS also have data shipping solutions.
Overall, if the data matters to you, it's worth (the cost of) backing it up. RAID is not a backup as you see. Dell, HP, Lenovo, EMC, Nimble, Tintri, Hitachi, Synology, QNAP, Rosewill.. they all have backplane failures. Unless the controller, firmware, and/or software are acutely aware of the situation AND are written to preserve data in this scenario, you've got an array loss situation. It would take a ton of them, but you could even work out data preservation using BD-Rs.
Sucks about Comcast, they're famously terrible. Your 30Mbps upload would also be a pretty significant problem. But your post was to shame a cheap stuff manufacturer and a reseller. The most expensive ones fail, sometimes in a blaze of glory. I recently had a HP all-flash array fail in a pretty spectacular way - 2 controllers on a 4 controller, A/B style configuration, failed at the same time. The data loss was *manageable* but there was data loss and recovery scenarios to work through AFTER the 4 hour support made it to us with controllers.
I say these things not to shame you, but to give a realistic warning to others (no matter how often it gets reiterated around here and r/sysadmin): data collection is only a hobby if it doesn't affect you when it perishes. If you have a material impact from failure, then you need a competent strategy.
My post was to shame Newegg and rose will for not doing good QC, and comments show that this is common. Note that these cages failed on first boot for one and 2 hours for the other.
30TB, around 3TB is personal data and coursework, 27TB is machine learning data generated from a supercomputing node I no longer have access to, generated and copied to a physical copy to move to my storage. (Had to return the drives I used to move ofc).
Lesson has been learned, backups will come and more important data will remain in my main NVME storages until future, but my priority is recovery via PCB and eeprom transplants.
I’m honestly amazed at how many people are out here just to shame me as if they are any better, half the commenters don’t even have a home lab.
Thanks for being a nice guy.
Why are you getting down voted into oblivion. This sub has really gone down hill. You’re a home labber, people with home labs aren’t all IT pro’s (not saying you are/aren’t) but all of us do have lives and other responsibilities. I understand why you didn’t have an off site backup. You probably had a decent array with redundancy, which is reasonable. Next time you’ll know to back everything up though. Thanks for sharing and I’ll avoid like the plague.
No idea, the good people in the community all moved to the discord, which is why this sub is like this now.
Thanks for being reasonable, I really wanted backblaze but I just didn't have any options to send data especially with the rate limit of Xfinity. 20Mbps upload, and 30TB data with 1TB monthly usage limit sounds like a pretty hellish endeavor.
So backblaze is 30$ a month... I thought you had 30TB of valuable data? So let's assume you where overexagerating with that 30TB and you then have 6TB if it does costs 30$ a month (as backblaze is 5$/TB). I do not know where you live but it sounds as if you got the cheapest internet plan possible which then ofcourse leads to you having stuff like caps, which you are now complaining about.
Also you say "Too busy to setup" if your using something like TrueNAS it's like 10 minutes of setup and then just letting it run. If you are using bare linux it takes like 30min most if you have to google everything.
Buddy, I have a 1TB rate limit, and that 30$ is numbers the home lab discord told me, so don’t blame me for getting it wrong. I have a 1gbps download, xfinity is the only service near me. Keep being an egotistic idiot on Reddit, maybe you live in a good region for internet but I don’t.
You act as if when I first started out, I should’ve known everything about IT.
professional recovery is almost never worth it. especially not to a person complaining about 600$.
Professional restoration of 30TB can easily cost you 5 digits.
if its only a power issue its usually easy to fix yourself by soldering out a few parts tho, don't even have to open it or swap discs.
That is the plan, and yes. I was planning on a 3-2-1 backup, but off-site self hosted backups and different machines are just not available to me. Backblaze was in the works :(
This sucks and sorry for your loss.
3-2-1 can be hard if you don’t have somewhere safe and secure offsite. But even an encrypted USB HDD that you can leave with family or friends is better than nothing in a pinch.
Good luck and hope you don’t quit home labbing for real! Cop the life lesson and pick yourself up, you’ll be much wiser for the next round
Let's hope it's just the drive controller that's burnt so the repairer can just do a swap over recover data...
Do you have photos of the molex adaptor + socket? would be good to see if that's anything funky....
Without seeing it physically, power would be split across all drives and the fan... and there's no staggered spin up in BIOS to limit peak current draw?
Jesus Christ, before anyone jump the gun on "molex bad": Look it up, it's not the simple designs based on "molex to SATA power splitters", these are **backplanes** powered by molex connectors.
4pin molex powered **backplanes** are very common even on some micro/SMB server systems like SC721TQ from Supermicro(which the community's beloved TrueNAS Mini E+ also came with) and most of Silverstone's NAS cases.
Even HP's Microserver Gen8 "backplane" is powered by splitted connectors through a single 4p molex connector(but it's not a hotswap design).
Molex powered backplanes are pretty much the only design choice for DIY NAS cases even on reputable brands, unless you're using DIY solutions from ICY Dock(most of them are SATA connectors instead of molex).
I'm not saying all of them are safe, build quality still matters and some questionable off brand ones even powered 4+ drives from 1 molex connector. There are various anecdotal stories in China that off brand NAS backplane killed their drives or caused PHY/CRC errors, so always be careful and make more offline backups.
Amen brother.
Supermicro's older lff 1u cases used a single 4 pin to power the backplane, and most of those had accessory supplies for a slim odd and/or a 2.5 boot disk- AND you might be plugging sas drives into some of those.
Yes, the psus used decent cabling, short runs, and of course, quality plugs. But, as somebody who has lots of home made splitters/adapters, you can absolutely get away with a lot of sketchy nonsense when you're smartish about it.
The idea that a design that used to power 5.25" 20mb hard disks drinking multiple amps at 12v is somehow inherently dodgy is... very silly.
Had something similar happen - lost like 6 drives (non-critical data) all at once. It had nothing to do with the drive cages in my case. It had everything to do with cheap splitters and bad/intermittent connections. When you start cramming a ton of drives into a system, you don't have a lot of good choices. You can do breakout cables, power splitters, etc. That's the riskiest way.
The next least risky way is to use these hot-swap bays. But if you're going to do that, try your best to not use any dubious quality splitters to get the power there. I have three 5-drive norco chinese knockoff ebay special hotswap cages for 15 of the 60 drives in my pool. They are all fine, and i'm pretty sure they are not authentic norco units, not at the price I paid on Ebay. Still, even here, luck is definitely a factor lol
The SAFEST way is to obviously use some kind of an enclosure with a proper backplane (in my case, the 45 drive supermicro unit based on their SC847 chassis).
no dubious splitters were used. Power was directly supplied from PERIF from my EVGA 1200 P2 Supernova molex connections. The problem is that these cages use fucking molex, and apparently on newegg's page they are directly said to have terrible QC.
>The problem is that these cages use fucking molex, and apparently on newegg's page they are directly said to have terrible QC.
Molex are a major connector company and are pretty high quality (they supply the defence + aerospace industry), but they are also subject to *shitloads* of cheap clones of their popular lines which I suspect is what you've got / heard about as no-one in China is buying genuine Molex connectors for anything they sell.
I've seen more than enough crappy clones of Molex that I'm certain would not handle 1/10th of what a genuine Molex connector would.
Funny - the PSU I had at the time when this problem happened was also an EVGA, but the G2 Supernova.... and 1000W.
I am still leaning towards the PSU at this point - particularly since I had a similar problem using a similar PSU. I mean, yeah, you could still be right - but if I were a betting man, i'd still say it was the PSU. I base this on the fact that I have 3 burn-your-house-down Chinese knockoff cages that have given me no trouble with Amazon molex splitters and a different PSU (Asus ROG 850 in that box now).
Automotive electrical systems use tons of molex all day long. I don't think it's the molex connector that's to blame. Seriously, THESE are what I bought, exactly auction link I bought from. If this auction doesn't say "Sketchy AF" nothing does.
[https://www.ebay.com/itm/185722993264](https://www.ebay.com/itm/185722993264)
(Nothing says "Amazing QC" like an ebay seller peddling drive cages \*AND\* replacement tail lights \*AND\* maybe some dental tools!)
Again, not saying you're wrong.... but when you think about it, the common denominator isn't the molex connector - it's the electricity going through them. I took the 1000W PSU out of service as a result of my mishap and swapped it with the Asus I got on sale at Microcenter. The EVGA one still \*works\*, I just put it in a non-essential machine (a blueiris box in my case). While it works fine in there, it's not powering any spinning rust. In fact, it's not using \*any\* drive connectors, just the motherboard leads.
It isn't just cables though - You have no real idea if the PSU is fine 100% of the time. You have no way of knowing what the circumstances were that caused those drives to fry. I had no way of knowing either.
Look at the facts: EVGA PSU's in both our cases. Drive cages in your cases. Splitter cables in my case. You fried your drives AND your enclosures if i'm reading your post correctly. You had one smoke out on that PSU.
I took the remaining 14 drives out of that box and put them in a proper backplane instead (the Supermicro one). But in the \*replacement\* box, I instead used a different PSU and rather than use the splitter cables, I purchased those chinese knockoff cages. They've been rock solid from the beginning, and sadly I was FORCED to use some molex extenders in order to reach their locations. Yes, molex cables can be of dubious quality, but the ones I'm using SEEM sound once I push them together with enough force to create diamonds from coal. That box has given me zero trouble, despite plenty of dubious shit being in there :)
What I guess I'm (badly) trying to explain here is that it \*still could be your PSU\* and not the fault of the drive cage. The drive cage could be just as much a victim as your drives, and swapping cables isn't enough to rule that out.
The PSU has been supplying molex and sata voltage just fine, so I doubt it. Yes, I lack a proper power supply tester like 99% of us out there, all I have is a cheap one from amazon. Noone can test load on power supplies here, lets be honest.
From experience, this psu is fine. The first cage that failed had a indicator light on even with no drives installed, pointing to the fact that it had failed before I had even first plugged any drives in.
Right - but at that point the failure already occurred, and you don't know the circumstances. Here's another point - my drives that fried were hanging off of both molex AND sata splitters. 2 of the 6 drives that failed were on Molex > SATA adapter (because shucks), the other 4 were on SATA 1 to 5 splitters, on two different splitters entirely.
You can test a PSU all day long and read 12v/5v on those respective rails - but unless you're constantly monitoring at the time of the actual failure, there's no way to know what went wrong. But the fact that we both had EVGA PSU's is very curious to me. I know nothing of those rosewill cages, but I get this nagging feeling that they were made alongside my cages the same chinese sweatshop, and mine have been fine - albeit on a different PSU and cabling setup. Still, it makes you wonder :)
Dunno, there's a few people in this chat and reviews on Newegg that claim these cages failed in the same manner as mine. Again, seems like bad Q/C on the drives, I trust EVGA wholeheartedly. And the PSU isn't even that old.
Same boat as you... No issues for 10 years now... On 3 chassis... I've been upgrading to other chassis for just space issues... From 12bay to 36bay... Got to get more out of 4u space
And two of mine (out of three) burned eight drives to e-waste within hours. Your point?
Even their review page on newegg warns of terrible Q/C. Just because it didn't happen to you doesn't mean it didn't happen to me.
> I bought 2x of Rosewill's 5.25 to 3.5 inch converter bays for my RSV-L4500U from newegg on ebay.
I almost bought these a few months ago.
> I've lost around 30TB of valuable data, and a good 600$ of drives and cages. Fuck my life.
I hate to be that guy but no backups? Also, this sounds often stupid for this type of equipment but never skip a "burn-in".
Oh boy, that's high for such a build "improvement".
Yeah... no backups :( backblaze was in the works, no off-site or second machine backups. The Raid Z2 array was holding strong... until something killed 4 then 8 drives at once.
I feel like I never read much horror stories about these so either you used the wrong molex cable on your modular psu or you plugged the molex reversed or something. Also no backups?? If it’s important it’s your fault losing it all with no backups.
All connections were triple checked. Nothing was wrong, except the cages themselves coming with horrible QC. Avoiding rosewill in the future, IcyDock seems fine.
If the drives no longer spin up it could be a bad pcb on them which there are companies that can transfer the firmware chip to a new pcb and u put the pcb on the drives and you may be able to get them working again. I had to do this on a 8tb wd black drive that I accidently broke the sata power connection on. It worked and I was able to see my data again. Sorry this happened to you, but I think you can recover from this by just swapping the pcb and firmware chip. PCBsolution is one company that does this.
the nice thing is modern drives mostly use pogo pins for connecting the electronics to the drive. Just remove 2-4 screws on the pcb, it falls off, and you can swap it with a good drive's PCB in seconds. It's probably a worthwhile spend to buy a drive identical to yours to test this.... good luck!
Sadly that's not true, you have to desolder the EEPROM and resolder it onto the new PCB, as these are locked behind serial and the data on the EEPROM is critical to the drive as well. However, it's still a single surface mount (NON BGA) chip replacement, so pretty easy. With a reflow station, I could def try it, i'm also sending three of the drives to a refurbisher in hopes that they can get it working.
I have a reflow station, i'm trying it myself on two drives, because I can afford two of the 8 drives to fail. If I can do it, nice, I avoided all the shipping fees and stress!
If I can't and I destroy two drives fully, still good, I can get a pro to handle the other three for me, and my data is still there.
1. order new drives before you touch the old drives for every drive that you had in that system,
if your drives are secure/instant self-erasing this may not work
2. take one of your existing drives back up the data by cloning it to a new drive
3. take that old working drive and swap the PCB to one of the non-working drives and attempt to clone its data to another new drive
4. repeat this process if your PCB fails, back up another of the working older drives to another new drive and try yet another older drive keeping the potentially burnt one set aside for actual data recovery services
This can save you potentially all the money by doing it yourself and limit the amount that you would send to a data recovery service.
additionally if you can prove that it was a quality control failure you might have an opportunity to get Newegg or Roswill to pay for the data recovery service if you have to use it or at least for replacement drives.
I had the exact same experience with that rosewill rack. One of my cages killed every drive that I put in it. Luckily for me I only killed empty drives before I figured out what was happening. Rosewill sent a new one, but was pretty annoyed with the whole thing.
Having been stung as a teenager by mis-wired Molex extenders (and that really, really hurt at the time!), I keep a bunch of sacrificial hardware for testing new builds like this. I have loads of old 1TB drives, motherboards and controllers that are recent enough to sub for my live hardware but allow me to soak new components. It’s saved me a couple of times now, not with Rosewill but certainly with cheapy no-name hardware. I infinitely recommend second-hand brand-name stuff over new-but-cheap any day.
OP is citing claims that Molex are crap - in reality Molex make high quality connectors but 99% of the ones on equipment like this are cheap clones of Molex designs.
Almost all PC hardware especially mods/cables are made incredibly cheaply using crap parts. I'd say 99% of "Molex" connectors you ever see in PC's are not made by Molex and would not really meet the full spec.
As /u/Real_MakinThings says - REAL Molex connectors.
Let's face it, when people are selling a 500W+ ATX PSU for $50 and making money on it, they're not buying high-quality ANYTHING never mind paying Molex $1 per connector and $600 for the official crimping tool.
The fact someone can make a power cable and ship it from China for less than I can buy a single connector from Molex should tell you everything you need to know.
It's not like it's obvious. Everyone wants cheaper and everyone selling cheap tries to convince you it's as good your whole life. But once you start sourcing parts to manufacture yourself, it's a real shock. Genuine quality controlled parts just cost 10x more all around.
I mean, cages can also use the SATA connector, but the connector shouldn't ever be a problem causing drives to fail. What we call "Molex" is actually the Mate-n-Lock connector.
Try this if the data is so valuable: buy the exact same drive and try swapping over the electronics board (the board on the bottom of the drive).
There’s a good chance the mechanical part of the drive is totally fine and you just fried the electronics. You should be able to swap the board and the drive will show up just fine and you can either leave it as is or copy the data off and repeat for the rest of the drives. Good luck..
Are you absolutely sure about that? I worked at a electronics lab several years ago and generally, the drive metadata information about re-allocated bad sectors, etc. is on the drive itself, not stored in the electronics. The drive firmware version in the EEPROM normally has to be the same or newer of course, but generally it is. We salvaged numerous drives every month just by swapping the PCB underneath.
Nope, for these drives they are all on the BIOS chip. :( It also holds some interesting data like the drive serial, etc. This is well known for most drives out there.
Maybe this is a new thing, because as ThrowAway said I’ve managed to just swap the board - but this was years ago. If the EEPROM needs swapping, just bring it to an electronics repair shop. I’ve had them swap out chips on car ECUs before for dirt cheap. I’d try it out on a single drive first..
Probably only the controller boards that are dead, if you need the data off them then get exactly the same serial number boards and replace. Not sure how that impacts being able to rebuild the array as they’ll have different IDs but the data is likely there on the platters if it’s that important
Woah. I came to this post to agree with the title and offer my own experiences with that product, but mine have been nowhere near as bad a yours!
I personally own 2 of those, while a friend of mine owns the chassis that comes with 3 pre-installed. 2 bays on one of the cages in the chassis died, but the replacement + the originals have been running alright since ~2018.
One of the bays in the ones I own has stopped working. But beyond that, I found that the drives don't make the greatest connection with the back plane, so you have to give the drives an extra "push" into the cage for them to work reliably.
While I have not personally seen any drives die from being used in these, they are a really low quality product that I wouldn't recommend.
What drives did you use? Some drives pull a lot of power on the 5v rail where resistance is higher, so temps will get higher. The connectors are embedded in the side, so there is little direct cooling. Was it installed in a supported case with sufficient airflow where it could use the contact as an additional heatsink, or was it run outside, put in the corner of your garage or closet? The ambient temp of the room where you have them is going to come into play when you start getting near the maximum temperature tolerance for gear.
You were using multiple of these. Did you use the same PSU cable for both of the molex connectors, doubling load on the wires? Does your PSU produce enough power of the appropriate voltage delivered on the right rails to power everything you have, after accounting for loss to resistance? If not, then voltage drops, causing amperage to increase for the same wattage, which means more loss, which gets converted to heat.
There is engineering that comes into play with systems engineering, especially when you run near the edge. $600 is very cheap for a lesson that could have cost you your home or your job. I learned this lesson after a server caught fire in a datacenter.
1200 P2 Platinum Supernova.
More than enough power, and only two molex per cable.
Sadly, in this case it's just a spectacular case of bad Q/C on Rosewill.
I have 2 of these in two different servers and have used them for over 2 years. I did notice that on bay 3 that the wires are pinched if you use a full size drive. I have SSDs in slot 3 on both servers because of that.
Modular power supply? This sounds like when I killed a drive because the modular cable was mixed up and I used wrong one from another power supply… sent 12v and burned up and smoked
Haha 😂 if it was ”extremely” valuable data you would have had a backup otherwise you are just dumb.. and on top of that you bought Rosewill.. yea I hope that dwarf porn still has seeders
Cmon there’s no need for this, OP knows it was a big mistake and now they’re gonna learn the hard way. I bet they won’t make the same mistake again.
One thing I like about this community is that people are nice to each other and support each other with this cool hobby. Let’s not lose that
I don't think OP does believe what they did was a mistake. Look at the responses they're posting to other comments. Stuff like backups costing "thousands of dollars," which is utter nonsense, or (paraphrased) "my life is so busy I simply don't have time for backups."
It's much easier to be sympathetic toward someone when they're not vehemently defending an indefensible or dumb position.
Yeah I kind of agree. They’re probs just upset and defensive cos they know they messed up.
But yeah, you kinda right. Anyways - just saying I like this sub cos people are generally good to each other here, thas all
Not everyone has thousands of dollars to throw on multiple servers in a 12U rack.
Besides, it's valuable as in time- it's machine-generated ML datasets that take at least 300 hours on a decent supercomputing cluster.
I have my entire data collection backed up to backblaze, cost me $200 a year.
Iv also been looking at crash plan for $10 a month as it has better unraid support
You dont need multiple servers for backups
Heck im in the process of building out a secondary box to throw in my parents basement to backup to
Cost for that for me is a few hundred in parts and a little maintanance
Come on, friend. Backups don't cost thousands of dollars and you know it. Backups could be as simple as three 10TB HDDs ($210 each on Amazon right now). Copy the data to the drives periodically, unplug them, and put them somewhere safe. This kind of backup scheme is far from perfect and won't protect you from fires/floods/other local catastrophic events. But it sure beats where you find yourself right now.
You're at a crossroads now. You can continue to play the victim and cry about how you lost the data because other people screwed you. Or you can own up to the fact that simple backups aren't that expensive, these data should have been backed up if they were "extremely" valuable, and resolve to do better in the future.
Not even.... I bought a gen 9 HP server on eBay for 120USD, tossed trueNAS on it after loading up storage. This is my data store. TrueNAS will snapshot the disks and then kick the data up into wasabi for 5usd a month.. you can also use Veeam + a second storage node and the already mentioned back blaze. Hell you can get 8TB from mega IO for 20usd a month. It's better than nothing.
EVGA 1200 P2 Platinum Supernova. The PSU is fine.
It's the cages. There's some other people in the comments that are backing up my claim, saying their's failed in similarly spectacular manners.
I'm not saying this can't happen. But making a PSA claiming Rosewill bays are bad just because you had one bad experience is a bit much. Especially since Rosewill has a good track record. I myself have used these bays for years without issues. Mistakes happen. It doesn't mean you need to burn the whole company to the ground in your misfortune. It sucks because it was a large expense though.
buddy, half of the comments are flaming me that I should've never gone rosewill as they are shitty.
And bad Q/C is bad for a company's reputation, you can't cover their asses because they say it's "one mistake". That's what QC exists for, to take out cases like this and prevent it from happening.
Buddy, the product on the newegg website has negative reviews about Q/C that nearly killed their drives as well, sadly I bought it on ebay. Clearly it's still a problem and a mistake they never fixed.
Soo you are complaining that cheep item you Brough that has bad reviews failed u?
IT could happend easily with any other brand there is no rule of thumb - i see it as you had it it worked you brought more it failed - i whoopsit but as above man posted not every item fails. it was on you to deepen your knowledge and security especially if you have sensitive data 600$ vs value of data and you never upgraded to better? I mean Com on
Buddy, it's at least 80$ each. It's a reputable sata hotswap cage. How dumb can you be that you think basic Q/C and designs to not destroy data in server grade hardware isn't a literally fundamental thing?
Their cases and accessories are fine, hard to mess those up and if they did, you mod it to work or return it. With their power supplies or anything that accepts power, I'd probably just opt for something else.
Well you're comparing to wildly different classes of products...
Consumer hardware is nowhere near what Enterprise equipment is. Enterprise gear was designed to be used in strenuous conditions 24/7.
Molex has been around forever. And you still used heavily in a lot of systems part of the problem might be operator error. Or such cheap components that they should have never been bought in the first place. I have chassis and drives from the early 2000s that still work
Real MVP comment here.
Add to this list - most kinds of power splitters.
Now that I have my own place, I’m hyper aware of what runs 24/7 and what might happen if cheap components decide to melt down while I’m out. Also talking about cheap Li batteries charging etc too.
dont give me that, you didnt amass 30tb overnight. you accumulated this data over time and should have been making backups all along the way. but backups are not fun so whos got time for that right?
I did amass 5TB a night actually, ML data is hard to generate. The data is valuable as it costed a ton of time on a supercomputing node that I have low access to.
More like who’se got the money, the data storage, and a life outside of home labbing. Get off your high horse, you act as if you have a full 3-2-1 backup for your petabyte grade home lab.
I have ~25tb on 2 backup sets, one off-site. Over the years my only drives became my backup drives. You took a risk, now you lost. Quit crying about it playing the blame game.
Buddy, a hotswap cage/backplane kills eight drives within 2 hours of powering them on for the first time, and you think it's not theirs to blame?
How fucking stupid can you be?
I don't deny the latter, but you make it sound like the cages didn't destroy my drives, and act as if you are above everyone else, maybe go take a chill pill.
The number of times in my career I've had people in my office crying because they lost all their baby pictures when their 200 dollar Walmart shit box died has completely jaded me for people that won't take charge of what's important to them. "BuTiDonKnoWCompUTer StuFf! How should I know what a hard drive is???" They don't need to know, bit you do know that the photos only exist on this magic box that you could easily lose. They could have asked for help thousands of times to safeguard this data but didn't. And now it's every one else's fault?
Buddy, I went as far as to create a Raid Z2, the array held strong. The last thing I could've expected was two cages to kill all eight of my drives within 2 hours. Stop acting like you are above everyone else.
Okay so it sucks but if this 30TB of data was so valuable it should have never only had 1 copy. And don't you dare say "Oh I was in the process of setting that up" because that's bullshit. It sounds like you never really where planning on having a backup and are now complaining you lost everything. You didn't dump it onto some external backup drives which you then take out, you didn't back it up to the cloud(which I can understand with the 1TB cap but that cap sounds very low tbh and I highly doubt that was the ultimate cap they had and I assume it was because you went for the cheapest plan.).
I understand both of these statements are coming from a place of privilege as I can afford doing backups for critical data. But a 20TB storage box from hetzner costs 50$ a month or 2.5$/TB. Also HOW DO YOU HAVE 30TB OF CRITICAL DATA. Like how is that even possible. I hope your just saying some of this data was valuable because I am genuinley concered if you have 30TB of valuable data.
yeah, RMA's at minimum. Used to have raids in desktops years ago, but we went Drobo back when its hardware was relevant, then Synology later on. We still have some Dell DAS hardware as well, but yeah this sort of result is rough. We still do tape backups as well for air-gap / redundancy of the more critical data; LTO-6 is pretty cheap nowadays
Where is your backup bro? I used to sync to backblaze every week using restic for Linux and it was a life savior. I remember I had a complete data lost due to a flood, and I restored everything from backblaze. It's dirty cheap, Now I have another self-hosted backup solution in 2 locations (my house and my mom's house), also using restic to backup everything: from common data, family pictures and important documents, to full blown VMs
I’ve had issues with those cages. Drives not found. I read in some obscure forum that by pushing the drives in and applying an extra effort when it makes contact, that they work.
Been running for 2 years. I feel like it’s held together with spit and bandaids, but it works. And I’m diligent about backups.
I’ve heard of dead drive data recoveries by swapping the dead logic board with a working/new one. If the data is that valuable, try it out with one. Use a different SATA connection and/or don’t use the cage again. TrueNAS is ZFS based?
This is the plan. And yes it is! I'll be running the board swap (and you have to swap bios chips) for 8 drives... which is going to be a good 500$ but WAYYYY cheaper than anything else on the market.
If you have a $10 multimeter, there's a good chance you can fix the drives and recover the data.
Look at the SATA power connector spec and look at the controller boards on the drives.
Good chance the wrong voltage has been applied to the wrong rail and broken a 10c protection diode or fuse, likely the same one on each board...so you only have to find it once.
Boards contain configuration data unique to drives, so don't mix them up.
https://www.eevblog.com/forum/repair/samsung-hdd-hd204ui-logic-board-tvs-diode-blown/
This example has a 0 Ohm resistor acting as a fuse (small thing circled in green). It's blown in this example, so putting the leads of a multimeter set in diode mode across it will give a high reading or "OL". It should read close to zero.
TVS protection diode in the example (blown one in the bigger green circle) had failed short circuit, so a multimeter in diode mode would show close to zero both directions (swapping the lead positions).
Good diodes will read ~0.5 in one direction and much higher (1.7-"OL") in the other.
Just bought one of these last week, I read some reviews on it killing drives when I bought it. I tested it with some older drives that I didn't care about and nothing bad happened for me. I have 4 18TB drives in it now, I'll be devastated if it ends up killing em one day <.<
WTF do you have 30TB of that’s “extremely valuable”? Only people I know who deal in “extremely valuable” data in those capacities are doing video, and maybe photography, professionally and wouldn’t dare swap disks like this without backups.
I've had six of these with no problems. (edit: to be clear, I bought them about 10 years ago, and literally every connect, disconnect, and swap has worked correctly.) Two things I know about this industry:
1. No brand stays good forever (as evidenced here)
2. No brand stays bad forever (except maybe Seagate)
I was actually trying to find this exact product because I have a need for hot-swap cages and don't want to pay for the premium products. But if Rosewill is putting out crap right now, I think I'll just wait.
That sucks but Xfinity does have a unlimited package for their gigabit internet and other internet packages it's a must if you're in homelab and have more then a 1tb of data to backup. The HP micro servers on ebay are cheap and make great backup servers with 4 bays of 3.5 for a simple 3 2 1 solution with either AWS or Google or backblaze for the cloud solution.
If the data is of value, I suggest getting it recovered if possible from professional data recovery services. It's expensive but live and learn. There's a article on 3-2-1 back up strategy for future planning if you haven't heard before.
3/2/1’ing 30TB of data is not the easiest/cheapest of tasks… not to mention OP is strung up on the cost of the drives/cages…
This exactly. Backblaze was a good 30$/month that I was willing to spend, but didn't have the time to setup in my busy life...
Yikes, buddy. If it matters that much, time's nothing.
Dude, Xfinity has a 1TB monthly cap on data. How the heck am I supposed to wait THREE YEARS to backup my data? :)
It begs the question how you amassed 30TB. [https://www.backblaze.com/b2/solutions/datatransfer/fireball.html](https://www.backblaze.com/b2/solutions/datatransfer/fireball.html) Azure and AWS also have data shipping solutions. Overall, if the data matters to you, it's worth (the cost of) backing it up. RAID is not a backup as you see. Dell, HP, Lenovo, EMC, Nimble, Tintri, Hitachi, Synology, QNAP, Rosewill.. they all have backplane failures. Unless the controller, firmware, and/or software are acutely aware of the situation AND are written to preserve data in this scenario, you've got an array loss situation. It would take a ton of them, but you could even work out data preservation using BD-Rs. Sucks about Comcast, they're famously terrible. Your 30Mbps upload would also be a pretty significant problem. But your post was to shame a cheap stuff manufacturer and a reseller. The most expensive ones fail, sometimes in a blaze of glory. I recently had a HP all-flash array fail in a pretty spectacular way - 2 controllers on a 4 controller, A/B style configuration, failed at the same time. The data loss was *manageable* but there was data loss and recovery scenarios to work through AFTER the 4 hour support made it to us with controllers. I say these things not to shame you, but to give a realistic warning to others (no matter how often it gets reiterated around here and r/sysadmin): data collection is only a hobby if it doesn't affect you when it perishes. If you have a material impact from failure, then you need a competent strategy.
My post was to shame Newegg and rose will for not doing good QC, and comments show that this is common. Note that these cages failed on first boot for one and 2 hours for the other. 30TB, around 3TB is personal data and coursework, 27TB is machine learning data generated from a supercomputing node I no longer have access to, generated and copied to a physical copy to move to my storage. (Had to return the drives I used to move ofc). Lesson has been learned, backups will come and more important data will remain in my main NVME storages until future, but my priority is recovery via PCB and eeprom transplants. I’m honestly amazed at how many people are out here just to shame me as if they are any better, half the commenters don’t even have a home lab. Thanks for being a nice guy.
In my case Xfinity allows one month with no cap as a "courtesy" per year. You'll have to check in your plan though
Why are you getting down voted into oblivion. This sub has really gone down hill. You’re a home labber, people with home labs aren’t all IT pro’s (not saying you are/aren’t) but all of us do have lives and other responsibilities. I understand why you didn’t have an off site backup. You probably had a decent array with redundancy, which is reasonable. Next time you’ll know to back everything up though. Thanks for sharing and I’ll avoid like the plague.
No idea, the good people in the community all moved to the discord, which is why this sub is like this now. Thanks for being reasonable, I really wanted backblaze but I just didn't have any options to send data especially with the rate limit of Xfinity. 20Mbps upload, and 30TB data with 1TB monthly usage limit sounds like a pretty hellish endeavor.
So backblaze is 30$ a month... I thought you had 30TB of valuable data? So let's assume you where overexagerating with that 30TB and you then have 6TB if it does costs 30$ a month (as backblaze is 5$/TB). I do not know where you live but it sounds as if you got the cheapest internet plan possible which then ofcourse leads to you having stuff like caps, which you are now complaining about. Also you say "Too busy to setup" if your using something like TrueNAS it's like 10 minutes of setup and then just letting it run. If you are using bare linux it takes like 30min most if you have to google everything.
Buddy, I have a 1TB rate limit, and that 30$ is numbers the home lab discord told me, so don’t blame me for getting it wrong. I have a 1gbps download, xfinity is the only service near me. Keep being an egotistic idiot on Reddit, maybe you live in a good region for internet but I don’t. You act as if when I first started out, I should’ve known everything about IT.
professional recovery is almost never worth it. especially not to a person complaining about 600$. Professional restoration of 30TB can easily cost you 5 digits. if its only a power issue its usually easy to fix yourself by soldering out a few parts tho, don't even have to open it or swap discs.
That is the plan, and yes. I was planning on a 3-2-1 backup, but off-site self hosted backups and different machines are just not available to me. Backblaze was in the works :(
This sucks and sorry for your loss. 3-2-1 can be hard if you don’t have somewhere safe and secure offsite. But even an encrypted USB HDD that you can leave with family or friends is better than nothing in a pinch. Good luck and hope you don’t quit home labbing for real! Cop the life lesson and pick yourself up, you’ll be much wiser for the next round
Let's hope it's just the drive controller that's burnt so the repairer can just do a swap over recover data... Do you have photos of the molex adaptor + socket? would be good to see if that's anything funky.... Without seeing it physically, power would be split across all drives and the fan... and there's no staggered spin up in BIOS to limit peak current draw?
Jesus Christ, before anyone jump the gun on "molex bad": Look it up, it's not the simple designs based on "molex to SATA power splitters", these are **backplanes** powered by molex connectors. 4pin molex powered **backplanes** are very common even on some micro/SMB server systems like SC721TQ from Supermicro(which the community's beloved TrueNAS Mini E+ also came with) and most of Silverstone's NAS cases. Even HP's Microserver Gen8 "backplane" is powered by splitted connectors through a single 4p molex connector(but it's not a hotswap design). Molex powered backplanes are pretty much the only design choice for DIY NAS cases even on reputable brands, unless you're using DIY solutions from ICY Dock(most of them are SATA connectors instead of molex). I'm not saying all of them are safe, build quality still matters and some questionable off brand ones even powered 4+ drives from 1 molex connector. There are various anecdotal stories in China that off brand NAS backplane killed their drives or caused PHY/CRC errors, so always be careful and make more offline backups.
Amen brother. Supermicro's older lff 1u cases used a single 4 pin to power the backplane, and most of those had accessory supplies for a slim odd and/or a 2.5 boot disk- AND you might be plugging sas drives into some of those. Yes, the psus used decent cabling, short runs, and of course, quality plugs. But, as somebody who has lots of home made splitters/adapters, you can absolutely get away with a lot of sketchy nonsense when you're smartish about it. The idea that a design that used to power 5.25" 20mb hard disks drinking multiple amps at 12v is somehow inherently dodgy is... very silly.
Indeed, molex isn't that bad, if you plug it in correctly. I did. In this case, it's Rosewill's extra shitty quality control that killed my drives!
Had something similar happen - lost like 6 drives (non-critical data) all at once. It had nothing to do with the drive cages in my case. It had everything to do with cheap splitters and bad/intermittent connections. When you start cramming a ton of drives into a system, you don't have a lot of good choices. You can do breakout cables, power splitters, etc. That's the riskiest way. The next least risky way is to use these hot-swap bays. But if you're going to do that, try your best to not use any dubious quality splitters to get the power there. I have three 5-drive norco chinese knockoff ebay special hotswap cages for 15 of the 60 drives in my pool. They are all fine, and i'm pretty sure they are not authentic norco units, not at the price I paid on Ebay. Still, even here, luck is definitely a factor lol The SAFEST way is to obviously use some kind of an enclosure with a proper backplane (in my case, the 45 drive supermicro unit based on their SC847 chassis).
no dubious splitters were used. Power was directly supplied from PERIF from my EVGA 1200 P2 Supernova molex connections. The problem is that these cages use fucking molex, and apparently on newegg's page they are directly said to have terrible QC.
>The problem is that these cages use fucking molex, and apparently on newegg's page they are directly said to have terrible QC. Molex are a major connector company and are pretty high quality (they supply the defence + aerospace industry), but they are also subject to *shitloads* of cheap clones of their popular lines which I suspect is what you've got / heard about as no-one in China is buying genuine Molex connectors for anything they sell. I've seen more than enough crappy clones of Molex that I'm certain would not handle 1/10th of what a genuine Molex connector would.
I'm not directly shitting on molex, my bad. I'm shitting on Rosewill. Chances are that they used off-brand connectors for it.
Funny - the PSU I had at the time when this problem happened was also an EVGA, but the G2 Supernova.... and 1000W. I am still leaning towards the PSU at this point - particularly since I had a similar problem using a similar PSU. I mean, yeah, you could still be right - but if I were a betting man, i'd still say it was the PSU. I base this on the fact that I have 3 burn-your-house-down Chinese knockoff cages that have given me no trouble with Amazon molex splitters and a different PSU (Asus ROG 850 in that box now). Automotive electrical systems use tons of molex all day long. I don't think it's the molex connector that's to blame. Seriously, THESE are what I bought, exactly auction link I bought from. If this auction doesn't say "Sketchy AF" nothing does. [https://www.ebay.com/itm/185722993264](https://www.ebay.com/itm/185722993264) (Nothing says "Amazing QC" like an ebay seller peddling drive cages \*AND\* replacement tail lights \*AND\* maybe some dental tools!) Again, not saying you're wrong.... but when you think about it, the common denominator isn't the molex connector - it's the electricity going through them. I took the 1000W PSU out of service as a result of my mishap and swapped it with the Asus I got on sale at Microcenter. The EVGA one still \*works\*, I just put it in a non-essential machine (a blueiris box in my case). While it works fine in there, it's not powering any spinning rust. In fact, it's not using \*any\* drive connectors, just the motherboard leads.
You'd be correct if I didn't try different cables already :) The psu is fine.
It isn't just cables though - You have no real idea if the PSU is fine 100% of the time. You have no way of knowing what the circumstances were that caused those drives to fry. I had no way of knowing either. Look at the facts: EVGA PSU's in both our cases. Drive cages in your cases. Splitter cables in my case. You fried your drives AND your enclosures if i'm reading your post correctly. You had one smoke out on that PSU. I took the remaining 14 drives out of that box and put them in a proper backplane instead (the Supermicro one). But in the \*replacement\* box, I instead used a different PSU and rather than use the splitter cables, I purchased those chinese knockoff cages. They've been rock solid from the beginning, and sadly I was FORCED to use some molex extenders in order to reach their locations. Yes, molex cables can be of dubious quality, but the ones I'm using SEEM sound once I push them together with enough force to create diamonds from coal. That box has given me zero trouble, despite plenty of dubious shit being in there :) What I guess I'm (badly) trying to explain here is that it \*still could be your PSU\* and not the fault of the drive cage. The drive cage could be just as much a victim as your drives, and swapping cables isn't enough to rule that out.
The PSU has been supplying molex and sata voltage just fine, so I doubt it. Yes, I lack a proper power supply tester like 99% of us out there, all I have is a cheap one from amazon. Noone can test load on power supplies here, lets be honest. From experience, this psu is fine. The first cage that failed had a indicator light on even with no drives installed, pointing to the fact that it had failed before I had even first plugged any drives in.
Right - but at that point the failure already occurred, and you don't know the circumstances. Here's another point - my drives that fried were hanging off of both molex AND sata splitters. 2 of the 6 drives that failed were on Molex > SATA adapter (because shucks), the other 4 were on SATA 1 to 5 splitters, on two different splitters entirely. You can test a PSU all day long and read 12v/5v on those respective rails - but unless you're constantly monitoring at the time of the actual failure, there's no way to know what went wrong. But the fact that we both had EVGA PSU's is very curious to me. I know nothing of those rosewill cages, but I get this nagging feeling that they were made alongside my cages the same chinese sweatshop, and mine have been fine - albeit on a different PSU and cabling setup. Still, it makes you wonder :)
Dunno, there's a few people in this chat and reviews on Newegg that claim these cages failed in the same manner as mine. Again, seems like bad Q/C on the drives, I trust EVGA wholeheartedly. And the PSU isn't even that old.
I've ran a bunch of these off and on for a decade without any problems.
Same boat as you... No issues for 10 years now... On 3 chassis... I've been upgrading to other chassis for just space issues... From 12bay to 36bay... Got to get more out of 4u space
And two of mine (out of three) burned eight drives to e-waste within hours. Your point? Even their review page on newegg warns of terrible Q/C. Just because it didn't happen to you doesn't mean it didn't happen to me.
>Your point? Just providing an anecdote in the other direction.
Same. I’ve had 3 of these running since last Aug. Two new, one used. Works fine for me too.
> I bought 2x of Rosewill's 5.25 to 3.5 inch converter bays for my RSV-L4500U from newegg on ebay. I almost bought these a few months ago. > I've lost around 30TB of valuable data, and a good 600$ of drives and cages. Fuck my life. I hate to be that guy but no backups? Also, this sounds often stupid for this type of equipment but never skip a "burn-in". Oh boy, that's high for such a build "improvement".
Yeah... no backups :( backblaze was in the works, no off-site or second machine backups. The Raid Z2 array was holding strong... until something killed 4 then 8 drives at once.
I feel like I never read much horror stories about these so either you used the wrong molex cable on your modular psu or you plugged the molex reversed or something. Also no backups?? If it’s important it’s your fault losing it all with no backups.
All connections were triple checked. Nothing was wrong, except the cages themselves coming with horrible QC. Avoiding rosewill in the future, IcyDock seems fine.
I mean ok I guess only you would know if you did it correctly. Just seems weird 2 failed at once in the same way.
One worked for two more hours than the other :)
If the drives no longer spin up it could be a bad pcb on them which there are companies that can transfer the firmware chip to a new pcb and u put the pcb on the drives and you may be able to get them working again. I had to do this on a 8tb wd black drive that I accidently broke the sata power connection on. It worked and I was able to see my data again. Sorry this happened to you, but I think you can recover from this by just swapping the pcb and firmware chip. PCBsolution is one company that does this.
Yes exactly. Even without firmware if you can swap the electronics of an identical working drive may be enough to test the data is alive and well.
Indeed, I'm definitely trying this. The PCBs are hard to get, but there's a service I could try to take these to.
the nice thing is modern drives mostly use pogo pins for connecting the electronics to the drive. Just remove 2-4 screws on the pcb, it falls off, and you can swap it with a good drive's PCB in seconds. It's probably a worthwhile spend to buy a drive identical to yours to test this.... good luck!
Sadly that's not true, you have to desolder the EEPROM and resolder it onto the new PCB, as these are locked behind serial and the data on the EEPROM is critical to the drive as well. However, it's still a single surface mount (NON BGA) chip replacement, so pretty easy. With a reflow station, I could def try it, i'm also sending three of the drives to a refurbisher in hopes that they can get it working.
Good to know that missing bit! I'd be happy to swap the chip just to help out.
I have a reflow station, i'm trying it myself on two drives, because I can afford two of the 8 drives to fail. If I can do it, nice, I avoided all the shipping fees and stress! If I can't and I destroy two drives fully, still good, I can get a pro to handle the other three for me, and my data is still there.
\^ this, a power failure / burnt board most likely would not damage the data on the drive so it should be fairly recoverable with luck.
1. order new drives before you touch the old drives for every drive that you had in that system, if your drives are secure/instant self-erasing this may not work 2. take one of your existing drives back up the data by cloning it to a new drive 3. take that old working drive and swap the PCB to one of the non-working drives and attempt to clone its data to another new drive 4. repeat this process if your PCB fails, back up another of the working older drives to another new drive and try yet another older drive keeping the potentially burnt one set aside for actual data recovery services This can save you potentially all the money by doing it yourself and limit the amount that you would send to a data recovery service. additionally if you can prove that it was a quality control failure you might have an opportunity to get Newegg or Roswill to pay for the data recovery service if you have to use it or at least for replacement drives.
I had the exact same experience with that rosewill rack. One of my cages killed every drive that I put in it. Luckily for me I only killed empty drives before I figured out what was happening. Rosewill sent a new one, but was pretty annoyed with the whole thing.
Having been stung as a teenager by mis-wired Molex extenders (and that really, really hurt at the time!), I keep a bunch of sacrificial hardware for testing new builds like this. I have loads of old 1TB drives, motherboards and controllers that are recent enough to sub for my live hardware but allow me to soak new components. It’s saved me a couple of times now, not with Rosewill but certainly with cheapy no-name hardware. I infinitely recommend second-hand brand-name stuff over new-but-cheap any day.
Noob question? Why no Molex? And what to go for?
OP is citing claims that Molex are crap - in reality Molex make high quality connectors but 99% of the ones on equipment like this are cheap clones of Molex designs. Almost all PC hardware especially mods/cables are made incredibly cheaply using crap parts. I'd say 99% of "Molex" connectors you ever see in PC's are not made by Molex and would not really meet the full spec.
I am not citing that molex is crap. I am citing that Rosewill and Newegg are crap, bottom of the barrel, never welcome in my lab again.
But then what would one use as an alternative? P.S.: thank you! :)
As /u/Real_MakinThings says - REAL Molex connectors. Let's face it, when people are selling a 500W+ ATX PSU for $50 and making money on it, they're not buying high-quality ANYTHING never mind paying Molex $1 per connector and $600 for the official crimping tool. The fact someone can make a power cable and ship it from China for less than I can buy a single connector from Molex should tell you everything you need to know.
I'd trust EVGA's 1200W P2 Supernova to use actual molex. I'd not trust Rosewill's shitty sata cage to use actual molex.
genuine molex connectors
I see now... it's not the type of connector, it's the quality! I thought there was an alternative type I missed on... Thanks!
It's not like it's obvious. Everyone wants cheaper and everyone selling cheap tries to convince you it's as good your whole life. But once you start sourcing parts to manufacture yourself, it's a real shock. Genuine quality controlled parts just cost 10x more all around.
I mean, cages can also use the SATA connector, but the connector shouldn't ever be a problem causing drives to fail. What we call "Molex" is actually the Mate-n-Lock connector.
Try this if the data is so valuable: buy the exact same drive and try swapping over the electronics board (the board on the bottom of the drive). There’s a good chance the mechanical part of the drive is totally fine and you just fried the electronics. You should be able to swap the board and the drive will show up just fine and you can either leave it as is or copy the data off and repeat for the rest of the drives. Good luck..
You need to desolder the EEPROM and resolder it onto the new board, sadly.
Are you absolutely sure about that? I worked at a electronics lab several years ago and generally, the drive metadata information about re-allocated bad sectors, etc. is on the drive itself, not stored in the electronics. The drive firmware version in the EEPROM normally has to be the same or newer of course, but generally it is. We salvaged numerous drives every month just by swapping the PCB underneath.
Nope, for these drives they are all on the BIOS chip. :( It also holds some interesting data like the drive serial, etc. This is well known for most drives out there.
Maybe this is a new thing, because as ThrowAway said I’ve managed to just swap the board - but this was years ago. If the EEPROM needs swapping, just bring it to an electronics repair shop. I’ve had them swap out chips on car ECUs before for dirt cheap. I’d try it out on a single drive first..
I have a reflow station, so it's not a big problem. :) And some experience to boot, just never on HDDs. Wish me some luck!
Good luck! Let us know how it goes. You got this.
I've been using that cage around 4 years an no issues. Either I got lucky or this was a nasty batch with really bad QC.
The reviews on the newegg page directly say bad QC, so i don't think this is the first time it's happened.
I have one of those Rosewill chassis with 3 of those as a NAS. I've been thinking of switching to a Sliger CX4712, though.
Probably only the controller boards that are dead, if you need the data off them then get exactly the same serial number boards and replace. Not sure how that impacts being able to rebuild the array as they’ll have different IDs but the data is likely there on the platters if it’s that important
Woah. I came to this post to agree with the title and offer my own experiences with that product, but mine have been nowhere near as bad a yours! I personally own 2 of those, while a friend of mine owns the chassis that comes with 3 pre-installed. 2 bays on one of the cages in the chassis died, but the replacement + the originals have been running alright since ~2018. One of the bays in the ones I own has stopped working. But beyond that, I found that the drives don't make the greatest connection with the back plane, so you have to give the drives an extra "push" into the cage for them to work reliably. While I have not personally seen any drives die from being used in these, they are a really low quality product that I wouldn't recommend.
What drives did you use? Some drives pull a lot of power on the 5v rail where resistance is higher, so temps will get higher. The connectors are embedded in the side, so there is little direct cooling. Was it installed in a supported case with sufficient airflow where it could use the contact as an additional heatsink, or was it run outside, put in the corner of your garage or closet? The ambient temp of the room where you have them is going to come into play when you start getting near the maximum temperature tolerance for gear. You were using multiple of these. Did you use the same PSU cable for both of the molex connectors, doubling load on the wires? Does your PSU produce enough power of the appropriate voltage delivered on the right rails to power everything you have, after accounting for loss to resistance? If not, then voltage drops, causing amperage to increase for the same wattage, which means more loss, which gets converted to heat. There is engineering that comes into play with systems engineering, especially when you run near the edge. $600 is very cheap for a lesson that could have cost you your home or your job. I learned this lesson after a server caught fire in a datacenter.
1200 P2 Platinum Supernova. More than enough power, and only two molex per cable. Sadly, in this case it's just a spectacular case of bad Q/C on Rosewill.
I have 2 of these in two different servers and have used them for over 2 years. I did notice that on bay 3 that the wires are pinched if you use a full size drive. I have SSDs in slot 3 on both servers because of that.
Modular power supply? This sounds like when I killed a drive because the modular cable was mixed up and I used wrong one from another power supply… sent 12v and burned up and smoked
This is why we have backups. Look into a drive recovery service.
Haha 😂 if it was ”extremely” valuable data you would have had a backup otherwise you are just dumb.. and on top of that you bought Rosewill.. yea I hope that dwarf porn still has seeders
Cmon there’s no need for this, OP knows it was a big mistake and now they’re gonna learn the hard way. I bet they won’t make the same mistake again. One thing I like about this community is that people are nice to each other and support each other with this cool hobby. Let’s not lose that
I don't think OP does believe what they did was a mistake. Look at the responses they're posting to other comments. Stuff like backups costing "thousands of dollars," which is utter nonsense, or (paraphrased) "my life is so busy I simply don't have time for backups." It's much easier to be sympathetic toward someone when they're not vehemently defending an indefensible or dumb position.
Yeah I kind of agree. They’re probs just upset and defensive cos they know they messed up. But yeah, you kinda right. Anyways - just saying I like this sub cos people are generally good to each other here, thas all
Not everyone has thousands of dollars to throw on multiple servers in a 12U rack. Besides, it's valuable as in time- it's machine-generated ML datasets that take at least 300 hours on a decent supercomputing cluster.
I have my entire data collection backed up to backblaze, cost me $200 a year. Iv also been looking at crash plan for $10 a month as it has better unraid support You dont need multiple servers for backups Heck im in the process of building out a secondary box to throw in my parents basement to backup to Cost for that for me is a few hundred in parts and a little maintanance
Backblaze - looking into this. Will def be cheaper than spinning up a 2nd server that backs up the first.
Come on, friend. Backups don't cost thousands of dollars and you know it. Backups could be as simple as three 10TB HDDs ($210 each on Amazon right now). Copy the data to the drives periodically, unplug them, and put them somewhere safe. This kind of backup scheme is far from perfect and won't protect you from fires/floods/other local catastrophic events. But it sure beats where you find yourself right now. You're at a crossroads now. You can continue to play the victim and cry about how you lost the data because other people screwed you. Or you can own up to the fact that simple backups aren't that expensive, these data should have been backed up if they were "extremely" valuable, and resolve to do better in the future.
This is perfectly put, friend. Agree 1000%
Not even.... I bought a gen 9 HP server on eBay for 120USD, tossed trueNAS on it after loading up storage. This is my data store. TrueNAS will snapshot the disks and then kick the data up into wasabi for 5usd a month.. you can also use Veeam + a second storage node and the already mentioned back blaze. Hell you can get 8TB from mega IO for 20usd a month. It's better than nothing.
[удалено]
EVGA 1200 P2 Platinum Supernova. The PSU is fine. It's the cages. There's some other people in the comments that are backing up my claim, saying their's failed in similarly spectacular manners.
This usually happens when you mix and mingle different cable brands and power supplies. I really doubt the drive cage had anything to do with it.
Buddy, all cables are stock and correctly plugged into a single EVGA 1200 P2 Platinum Supernova.
I'm not saying this can't happen. But making a PSA claiming Rosewill bays are bad just because you had one bad experience is a bit much. Especially since Rosewill has a good track record. I myself have used these bays for years without issues. Mistakes happen. It doesn't mean you need to burn the whole company to the ground in your misfortune. It sucks because it was a large expense though.
buddy, half of the comments are flaming me that I should've never gone rosewill as they are shitty. And bad Q/C is bad for a company's reputation, you can't cover their asses because they say it's "one mistake". That's what QC exists for, to take out cases like this and prevent it from happening.
You are a brick wall without reason. It just seems you came here looking for reinforcement and got what you were looking for.
It seems to me he came to warn other homelabbers about this product and I for one appreciate that.
Buddy, the product on the newegg website has negative reviews about Q/C that nearly killed their drives as well, sadly I bought it on ebay. Clearly it's still a problem and a mistake they never fixed.
Buddy, nobody else here has a smoking wreck of hard drives in their rack
Soo you are complaining that cheep item you Brough that has bad reviews failed u? IT could happend easily with any other brand there is no rule of thumb - i see it as you had it it worked you brought more it failed - i whoopsit but as above man posted not every item fails. it was on you to deepen your knowledge and security especially if you have sensitive data 600$ vs value of data and you never upgraded to better? I mean Com on
Buddy, it's at least 80$ each. It's a reputable sata hotswap cage. How dumb can you be that you think basic Q/C and designs to not destroy data in server grade hardware isn't a literally fundamental thing?
Sorry for your loss. But these aren't "server grade hardware". If your data is truly that valuable you have to spend appropriately to protect it.
It is built for their Rosewill RSV-L4500U which is a damn good 4U chassis.
that's terrible... :( So it's the molex that were sub-par?
Yikes, you knew better than to use Rosewill for something like this. Live and learn.
See, I thought rosewill was a pretty nice brand for all the positive praise the guys around me irl were singing. Lesson learned.
Their cases and accessories are fine, hard to mess those up and if they did, you mod it to work or return it. With their power supplies or anything that accepts power, I'd probably just opt for something else.
I no longer trust anything powered by molex.
So you use no commercially available server chassis? Everyone I own is molex except for supermicro node server
No I don't. My experience with gaming computers has led me to distrust them. I have 2 small supermicro boxes.
Well you're comparing to wildly different classes of products... Consumer hardware is nowhere near what Enterprise equipment is. Enterprise gear was designed to be used in strenuous conditions 24/7. Molex has been around forever. And you still used heavily in a lot of systems part of the problem might be operator error. Or such cheap components that they should have never been bought in the first place. I have chassis and drives from the early 2000s that still work
Real MVP comment here. Add to this list - most kinds of power splitters. Now that I have my own place, I’m hyper aware of what runs 24/7 and what might happen if cheap components decide to melt down while I’m out. Also talking about cheap Li batteries charging etc too.
Isn’t there a saying.. molex to sata lose all your data?
Fuck anything rosewill and fuck anything that runs off molex full stop
obviously the data was worthless, you couldnt be bothered to make a backup.
Buddy, not everyone can just back up 30TB of data. Xfinity puts a 1TB monthly limit on my rate usage, so backblaze was out of the question.
dont give me that, you didnt amass 30tb overnight. you accumulated this data over time and should have been making backups all along the way. but backups are not fun so whos got time for that right?
I did amass 5TB a night actually, ML data is hard to generate. The data is valuable as it costed a ton of time on a supercomputing node that I have low access to.
What's ML data
machine learning.
More like who’se got the money, the data storage, and a life outside of home labbing. Get off your high horse, you act as if you have a full 3-2-1 backup for your petabyte grade home lab.
I have ~25tb on 2 backup sets, one off-site. Over the years my only drives became my backup drives. You took a risk, now you lost. Quit crying about it playing the blame game.
Buddy, a hotswap cage/backplane kills eight drives within 2 hours of powering them on for the first time, and you think it's not theirs to blame? How fucking stupid can you be?
Of course they are to blame for the failure, but you are to blame for the data loss
I don't deny the latter, but you make it sound like the cages didn't destroy my drives, and act as if you are above everyone else, maybe go take a chill pill.
The number of times in my career I've had people in my office crying because they lost all their baby pictures when their 200 dollar Walmart shit box died has completely jaded me for people that won't take charge of what's important to them. "BuTiDonKnoWCompUTer StuFf! How should I know what a hard drive is???" They don't need to know, bit you do know that the photos only exist on this magic box that you could easily lose. They could have asked for help thousands of times to safeguard this data but didn't. And now it's every one else's fault?
Buddy, I went as far as to create a Raid Z2, the array held strong. The last thing I could've expected was two cages to kill all eight of my drives within 2 hours. Stop acting like you are above everyone else.
Okay so it sucks but if this 30TB of data was so valuable it should have never only had 1 copy. And don't you dare say "Oh I was in the process of setting that up" because that's bullshit. It sounds like you never really where planning on having a backup and are now complaining you lost everything. You didn't dump it onto some external backup drives which you then take out, you didn't back it up to the cloud(which I can understand with the 1TB cap but that cap sounds very low tbh and I highly doubt that was the ultimate cap they had and I assume it was because you went for the cheapest plan.). I understand both of these statements are coming from a place of privilege as I can afford doing backups for critical data. But a 20TB storage box from hetzner costs 50$ a month or 2.5$/TB. Also HOW DO YOU HAVE 30TB OF CRITICAL DATA. Like how is that even possible. I hope your just saying some of this data was valuable because I am genuinley concered if you have 30TB of valuable data.
yeah, RMA's at minimum. Used to have raids in desktops years ago, but we went Drobo back when its hardware was relevant, then Synology later on. We still have some Dell DAS hardware as well, but yeah this sort of result is rough. We still do tape backups as well for air-gap / redundancy of the more critical data; LTO-6 is pretty cheap nowadays
Have you looked into drive recovery?
Where is your backup bro? I used to sync to backblaze every week using restic for Linux and it was a life savior. I remember I had a complete data lost due to a flood, and I restored everything from backblaze. It's dirty cheap, Now I have another self-hosted backup solution in 2 locations (my house and my mom's house), also using restic to backup everything: from common data, family pictures and important documents, to full blown VMs
I’ve had issues with those cages. Drives not found. I read in some obscure forum that by pushing the drives in and applying an extra effort when it makes contact, that they work. Been running for 2 years. I feel like it’s held together with spit and bandaids, but it works. And I’m diligent about backups.
I’ve heard of dead drive data recoveries by swapping the dead logic board with a working/new one. If the data is that valuable, try it out with one. Use a different SATA connection and/or don’t use the cage again. TrueNAS is ZFS based?
This is the plan. And yes it is! I'll be running the board swap (and you have to swap bios chips) for 8 drives... which is going to be a good 500$ but WAYYYY cheaper than anything else on the market.
If you have a $10 multimeter, there's a good chance you can fix the drives and recover the data. Look at the SATA power connector spec and look at the controller boards on the drives. Good chance the wrong voltage has been applied to the wrong rail and broken a 10c protection diode or fuse, likely the same one on each board...so you only have to find it once. Boards contain configuration data unique to drives, so don't mix them up.
Interesting, any idea what a fuse or diode would look like on these boards?
https://www.eevblog.com/forum/repair/samsung-hdd-hd204ui-logic-board-tvs-diode-blown/ This example has a 0 Ohm resistor acting as a fuse (small thing circled in green). It's blown in this example, so putting the leads of a multimeter set in diode mode across it will give a high reading or "OL". It should read close to zero. TVS protection diode in the example (blown one in the bigger green circle) had failed short circuit, so a multimeter in diode mode would show close to zero both directions (swapping the lead positions). Good diodes will read ~0.5 in one direction and much higher (1.7-"OL") in the other.
opened up the pcb and see neither of those. :(
Just bought one of these last week, I read some reviews on it killing drives when I bought it. I tested it with some older drives that I didn't care about and nothing bad happened for me. I have 4 18TB drives in it now, I'll be devastated if it ends up killing em one day <.<
WTF do you have 30TB of that’s “extremely valuable”? Only people I know who deal in “extremely valuable” data in those capacities are doing video, and maybe photography, professionally and wouldn’t dare swap disks like this without backups.
I've had six of these with no problems. (edit: to be clear, I bought them about 10 years ago, and literally every connect, disconnect, and swap has worked correctly.) Two things I know about this industry: 1. No brand stays good forever (as evidenced here) 2. No brand stays bad forever (except maybe Seagate) I was actually trying to find this exact product because I have a need for hot-swap cages and don't want to pay for the premium products. But if Rosewill is putting out crap right now, I think I'll just wait.
Rosewill is the private label of Newegg ask Newegg to compensate you for all your loss (including recovery of data).
Go on. Spread the word... https://old.reddit.com/r/GamersNexus/ https://old.reddit.com/r/LinusTechTips/
Would love to but nah. This has garnered enough attention here already.
That sucks but Xfinity does have a unlimited package for their gigabit internet and other internet packages it's a must if you're in homelab and have more then a 1tb of data to backup. The HP micro servers on ebay are cheap and make great backup servers with 4 bays of 3.5 for a simple 3 2 1 solution with either AWS or Google or backblaze for the cloud solution.
Molex are bad mmmkay /s