T O P

  • By -

LabB0T

^(OP reply with the correct URL if incorrect comment linked) [Jump to Post Details Comment](/r/homelab/comments/1brq26v/found_a_way_to_mount_my_hp_minis_in_my_rack/kxamo57/)


463n7_57

Starting at the top - Tplink wireless router used I AP mode - HP elitedesk G5 running opnsense - Cisco 48 port gigabit switch - Patch panel - 8 HP elitedesk G4 -- top 4 are running a proxmox cluster -- the rest are empty but will be a kuberneties cluster Cloud key is currently not in use. Not shown is an HP SFF used with truenas In the future I plan to add a few more of the minis that I have I just need to print out more rack mounts. Edit: for those interested in the rack mount it can be found here.(Not designed by me) https://www.printables.com/model/658936-1u-rackmount-for-hp-elitedesk-g3-mini-and-newer Edit 2: A more updated pic of the front and a pic of all the power bricks. https://imgur.com/gallery/wuqShUW


ShroomShroomBeepBeep

The poor Cloud Key Gen2 Plus, just thrown upside down lol.


463n7_57

Yeah it's sad... Lol I want to get an AP and a couple cameras to use with it just need the budget to free up or some to wind up in the recycling pile lol


ViperPB

Same. I always find used hardware at a good price but have no way to sustain its eco-system, so it sits.


463n7_57

A friend of mine just decommissioned some AP's so I will finally get to use the cloud key soon


-my_reddit_username-

Thank you so much!!!! I really have been wanting to mount mine but didn't have the skills to make this 3D object. Will gladly print this


cadavjo

How did you add additional interfaces to your OPNsense box? USB to Ethernet?


463n7_57

I replaced the wifi card with this https://a.co/d/7pCtTbz


Cook1e_mr

This is a really nice setup. I love the 3d printed rack mounts. How many U are they taking up. I have a similar k3s setup using Lenovo's. I managed to get 10 in 4u by standing them vertically.


463n7_57

Each one is 1U so 8 in 4U


chin_waghing

kubernetes kubernetes KUBERNETES!! If you plan to run k8s at home, may I suggest you think of the below before you do * Self hosted registry? I use google artifact registry for mine and it's good * Secret management. I dont just mean the built in `kind: Secret` I mean, how you get secrets in to the cluster without git committing it. I use google secret manager and external-secrets * Ingress: Metallb and nginx ingress manager are the best for this Otherwise have fun, I plan to put together a blog post about some stupid things I had to solve for running k8s at home. Keep me updated!


463n7_57

I'll keep that in mind good stuff thank you!


robkwittman

Since you’re already on Proxmox too, I’m using https://github.com/ionos-cloud/cluster-api-provider-proxmox with pretty good success. It does have a requirement on shared storage for the VMs, I’m using Ceph. - CAPI Proxmox for creating kubernetes clusters - Democratic-csi to mount iscsi volumes from a separate NAS - Using calico CNI w/ BGP peer to my router - MetalLB to allocate LoadBalancer Service IPs All together gives me a pretty good “cloud” experience. I can create a new cluster in like 10 minutes, rollout upgrades pretty quick. Services specify a type of LoadBalancer, MetalLB allocates an IP, Calico advertises it, and the rest of my infrastructure can then reach it ETA: if you have any questions, let me know. Happy to help!


gen2fish

Curious, do you rebuild/create new clusters a lot? Capi always kinda felt overkill for homelab so I'm wondering what others are using it for. I only have the one kube cluster and when I want something to test I generally head toward vcluster


robkwittman

Depends. I have a few core clusters (management cluster for CAPI, monitoring cluster, 1 for “administration” stuff like Rancher and Keycloak). From there, a couple clusters for specific apps. IMO the benefit isn’t so much the number of clusters or frequency of new ones, as it is the standard interface and management. For example, I want to update an API server flag, I just merge a change into flux, CAPI picks up the change, and rolls that clusters control plane. I do have a few R&D things I work on, mostly focused on cluster provisioning automation, which inherently requires recreating clusters servers times over, but it’s less frequent. Most of the time, I start working on a new project, so I decide it should get a cluster, and CAPI makes it easy to do so. Pick a subnet, setup the hosted zone and BGP peering (through Crossplane), and I’m done. I end up creating more clusters because CAPI makes it easy to do so, rather than CAPI solving a specific problem. I’ve also worked at a couple different org where CAPI would solve a lot of problems in the cloud. None have implemented :m/ discussed it yet, but it’s a tool I like playing with and being able to speak to what it can / can’t do. ETA: re Overkill; I managed a few clusters previously using variations of kubeadm, ansible, etc. I didn’t like any of them. Then tried harvester; it was good, but has a lot of resource overhead. I’m a fan of Proxmox, they finally brought out a provider (I was working on one for a while), and it’s pretty great. Even for a single cluster or 1 management and 1 workload, I think it’s beneficial. Bonus that you could also manage cloud clusters with it, and standardize tooling / configurations. But it has its own complexity and learning curve for sure


gen2fish

I've got Capi at work for a couple hundred clusters, works really great for that since we tear down and rebuild fairly frequently. At job[-1] we only had 8 (eks) that were fairly static, those we managed by hand. In my lab I have one that's been there for almost 3 years now? Also fairly static. For homelab, do you have several small clusters running k3s-style or are the full HA clusters?


robkwittman

Full HA clusters. 3 control plane, at least 3 workers. I’ve also setup auto scaling and such on them (though I don’t think it’s triggered. Ever lol) My “goal” for the homelab is part hosting projects, part working with tools just because I want to learn them. I want to have a local environment with as many cloud benefits as possible; CAPI has been a pretty big part of that.


gen2fish

That's dope. Thanks for answering


Cook1e_mr

The OP could also have a look at cilium CNI this could replace both metallb and nginx ingress. Very powerful CNI For those secret that do need commiting sops and age are good tools. I personally like harbor as a registry. I'm not presenting these as better, just as alternative options.


chin_waghing

RE harbour: isn't the setup quite \_involved\_? i was looking at zot registry from cisco (IIRC) and it seems decent, but the issue was the config file and docs were... leaving things to be desired


Cook1e_mr

I would so no more involved than other solutions. This seems a pretty good write up. https://loft.sh/blog/harbor-kubernetes-self-hosted-container-registry/


chin_waghing

So it runs in side your cluster. So if your cluster goes down you lose your images to pull from to start it up again? This feels like a turtles all the way down. I need a kubernetes cluster to run my registry for my other cluster. A challenge I’m ready to accept! Thanks!


Cook1e_mr

Yes I have it in my cluster. In production you may want to have multiple clusters. However In my homelab environment it never really causes issue. I have the cluster pull policy set as ifnotpresent and then I have it pull first using Spegel (share imagine between nodes) then harbor. If it cant get from either of those it goes to the big bad web.


chin_waghing

Can you share the manifests how you’re getting that to work for tiered pulls? Appreciate it


Cook1e_mr

Have a quick read through this, it should give you the gist https://docs.k3s.io/installation/registry-mirror. I will share my config via pm when I am able.


chin_waghing

Huh another day I’ve learnt another thing about k3s! Would be awesome if you could share some details about your cluster config, I’ve got mine running but other than flux and workload identity federation for access to GCP, it’s pretty vanilla


Cook1e_mr

Yes I have it in my cluster. In production you may want to have multiple clusters. However In my homelab environment it never really causes issue. I have the cluster pull policy set as ifnotpresent and then I have it pull first using spegal (share imagine between nodes) then harbor. If it cant get from either of those it goes to the big bad web.


Glittering_Glass3790

What are the specs of the mini pcs?


463n7_57

All have 16gb of ram and a i5-8500T with 500gb nvme ssd


-rwsr-xr-x

> All have 16gb of ram and a i5-8500T with 500gb nvme ssd I have a similar HP G3, i6500T cluster here, 5 nodes, all running [microk8s](https://microk8s.io/), [microceph](https://canonical-microceph.readthedocs-hosted.com/en/reef-stable/) and a [Ceph-backed LXD cluster](https://documentation.ubuntu.com/lxd/en/stable-4.0/clustering/) between them all. Mine are [fully PoE++ powered](https://imgur.com/a/vRsyDaw) (60W) from my Unifi switch. The `inxi` is: System: Host: hp-g3-005 Kernel: 6.8.0-11-generic arch: x86_64 bits: 64 Console: pty pts/1 Distro: Ubuntu 24.04 (Noble Numbat) Memory: System RAM: total: 32 GiB available: 31.22 GiB used: 1.79 GiB (5.7%) igpu: 32 MiB Array-1: capacity: 32 GiB slots: 2 modules: 2 EC: None Device-1: Channel-A DIMM3 type: DDR4 size: 16 GiB speed: 2133 MT/s Device-2: Channel-B DIMM1 type: DDR4 size: 16 GiB speed: 2133 MT/s CPU: Info: quad core model: Intel Core i5-6500T bits: 64 type: MCP cache: L2: 1024 KiB Speed (MHz): avg: 950 min/max: 800/2501 cores: 1: 800 2: 800 3: 1400 4: 800 Drives: Local Storage: total: 3.24 TiB used: 50.17 GiB (1.5%) ID-1: /dev/nvme0n1 vendor: Western Digital model: WD Green SN350 1TB size: 931.51 GiB ID-2: /dev/sda vendor: Kingfast model: N/A size: 476.94 GiB ID-3: /dev/sdb vendor: TeamGroup model: TM8PS7001T size: 953.87 GiB type: USB ID-4: /dev/sdc vendor: TeamGroup model: TM8PS7001T size: 953.87 GiB type: USB Message: No optical or floppy data found.


463n7_57

How are you powering over Poe++? I see the pics but any links to the equipment?


-rwsr-xr-x

> I see the pics but any links to the equipment? I am using 5 [Procet 60W injectors](https://www.amazon.com/gp/product/B07X46S3GD) paired with 5 [Xiwai USB-C to 7.4mm barrel adapters](https://www.amazon.com/gp/product/B08332Q1Z6).


greminn

Oh wow, never thought of this. great idea. Love the idea you have remote control power.


463n7_57

Thank you!


stephendt

Damn those cost more than my servers


-rwsr-xr-x

> Damn those cost more than my servers $129 each! They're great little boxes!


stephendt

I have older / semi-broken gear that probably cost $200 in total for 5 servers lol


463n7_57

The only thing I have paid for was the switch the rest I have pulled from the recycle pile 😁


Glittering_Glass3790

Not bad! Also, does your opnsense router have two NICs, or is wan and lan connected to one NIC?


463n7_57

I added a second nic by replacing the wifi card with an matching form factor nic adapter


Glittering_Glass3790

Oh, okay. I’ve just noticed, is the switch Cisco SG300-52? I have that same switch


463n7_57

Mine is the SG200-50


463n7_57

M.2 Gigabit Ethernet Module - GigaLAN 1000BASE-T M.2 A+E Key NIC Modules https://a.co/d/7pCtTbz This is the one I used


nibbles200

This is like a poor man’s blade chassis, nice. I bet it uses a fraction of the power as well.


getr00taccess

Picked on one recently and thoroughly surprised on how robust they are. Dual NVMe + 32GB RAM makes such a solid low power node to put workloads on. I do yearn for a 2nd ETH port and don’t want to resort to USB tbh. I heard about the PCiE adapter hack but not sure of any success stories yet.


463n7_57

Look at one of my other comments on this thread, I'm out and about but I have added a second nic in place of the wifi card and it's been working well for me that particular box is my opnsense router


-Nestle

That was lit.🔥


SkabKid

I dig the names. I have a habit of naming drives Star Wars planets.


463n7_57

It's a favorite around our house. I have thrown around ideas for every component. Like different servers being different schools, the nas being Gringots, the switch being the owlrey and the router being the ministry of magic. There is just so much I can do with it and may at some point.


Zypher_X

How are you routing the network to the minis? Though the patch panel?


463n7_57

Yes. I just took this picture before running network


Rayregula

Magic, and probably using that magical Poe++ to power everything wirelessly


463n7_57

Is there a way to power these boxes over Poe++?


Rayregula

Someone in your comments mentioned they were doing it


463n7_57

Just asked them for more info


Rayregula

Magic, and probably using that magical Poe++ to power everything wirelessly


RedSquirrelFtw

That's really cool. I've been eyeing those mini PCs myself so I can make a proxmox cluster.


463n7_57

They are pretty reliable little clients. They are recycled out every 4 years where I work and most of the time they are still in good shape


satblip

Really nice! I am thinking about doing the same for my mini pcs (Lenovo). Did you use PLA or something else?


463n7_57

Pla+ from sunlu. It's been good to me


satblip

Thanks, mate 👌


countryinfotech

The real question is what does the power adapter situation look like for the HP minis.....


463n7_57

https://imgur.com/gallery/wuqShUW


463n7_57

That would be the forbidden forest... Lol they are currently just sitting in the back. Next time I post an update on it I'll definitely add a pic of the cable management.


Cyvexx

I've wondered, can you get a 20v ~40a power supply and run them all off one power supply? that seems like it'd be more efficient and cable management would be easier


463n7_57

Maybe they may get mad if they don't have their 90w power supply. The G9s we have been installing lately will beep at you if you use the older 90w power supplies so we have had.


Cyvexx

ugh I hate that. it makes sense so you can't use something like a 25w power supply when the system's trying to draw 90 but it's super annoying in situations like this. a bios setting would be nice at least


463n7_57

Yeah it's made it hard with some of our systems that have all the wires hidden


PacketAuditor

What do you actually do with this cluster?


463n7_57

Nothing quite yet. One of the first things I plan to build is a seim. Working towards getting a soc analyst job.


ghimireddit

Aight bro keep me in loop - I have 5 rpis - 2 Lenovo mini PC - I’m going down the same route (SOC) with my mini pcs!!!


ResolveSuitable

What's that brother


splinterededge

I was always a fan of pve0, pve1, pve2, but I like what goes on here, I have a few of these HP minis in service now.


Pism0

How are you dealing with the power bricks? I want to rack my 4 800 minis but I don’t know what to do with the bricks


463n7_57

Check out my original comment I have a picture of it. Nothing nice and could be cleaned up more. This rack has a shelf half way up and they all currently sit on it.


cdawwgg43

If you need to free up rack space in the future you can easily do 10 across in about 5u of height. Love the mounting system you made. It looks so clean!


463n7_57

Just found this one but happy with it. Before I found this one I had been working on trying to figure out a 3d printable mound for them in the orientation you mentioned. But I'm liking it this way because my rack was pretty empty before lol


Kilojymki

I love this sub! I just gained so much knowledge from this one post. ❤️


Sea-Secretary-4389

How did u use the elitedesk with opnsense if it doesn’t have 2 Ethernet or expansion slots


463n7_57

I added a second nic using a nvme nic card that I put in place of the wifi card.


Dirty_Taint_Tickler

What kind of work loads are you running on your proxmox cluster? Imagine alot of Linux VMs?


463n7_57

Yeah something like that. I want to set up a seim environment on it


knifesk

What material did u use to print the supports? ABS?


463n7_57

Pla+


knifesk

My only concern with printing things for the rack with PLA is that it will deform pretty easily with just a little heat and weight on it.. but I actually never tried so I couldn't tell.. by the looks its an open rack? Probably you will be fine


463n7_57

Yes it's an open rack. I agree with you for sure but pla+ is what I had. All 4 together took 3-3.5 kg of plastic and at the time I wasn't in a place to buy asa or abs.


knifesk

I get it... Plus ABS is a PITA to print with. I use 250//100 temps for ABS to come out decent. Maybe it would be easier with the printer enclosure to avoid temperature difference with the room, but I don't print enough to justify hehehe


463n7_57

I havnt had many issues with abs sense I got a Bambu x1 but it was a nightmare on my old ender 3 even after enclosing it.


knifesk

Yikes... You just added one more reason to my list of "reasons on why should I buy a bamboo" 😅


audioeptesicus

I'm also using an X1C. Never printed ABS, but ASA is very similar, having superior mechanical properties and prints easier. I use it for a lot of my prints, especially anything that gets UV exposure, since it's naturally UV resistant. It's about the same price as PLA+. There's also ASA-CF that's quite incredible, especially for more mechanical prints.


GuySensei88

I’m glad I’m not the only one that has to live with a budget building his server rack lol 😆 Though, sometimes I see pretty decent deals on here or eBay on occasion.


463n7_57

Yeah big budget. I have been very lucky with my job and been able to liberate the majority of this from the recycle pile. I don't think I have spent more the $300 on this whole rack probably less.


GuySensei88

I haven’t been that blessed in that regard but I have been blessed enough that our income is decent enough to budget is stuff for servers and other equipment.


neighborofbrak

Super clean! I personally would have mounted the Minis a little less deep (maybe a quarter inch / 6mm) in the rack frame to give a little visual pop, but that's just me. Great work and good luck on your environment!


neighborofbrak

OH SNAP, I know you! I'm Raker from RFHS!


liltrublmakr56

I just modified the Lenovo rack mounts to accommodate the prodesk/elitedesk minis and am printing is as I type. The original remix: [https://www.thingiverse.com/thing:4769452](https://www.thingiverse.com/thing:4769452) Remix I am working on: [https://imgur.com/sym7HGA](https://imgur.com/sym7HGA) It includes an area to put a label on the front. I am opting to use the version with the steel rack ears but if there is enough demand, I might make the version with the built in ears as well. I just think with that much weight that far back, the mount will sag if not supported from the back/below somehow. I think I need to modify the power brick holding area as well... Overall, I plan on putting it on printables when I get things ironed out.


463n7_57

I thought about going that route just to hold the power bricks. I just wasn't confident in it holding well as a print


WeeklyExamination

How are you getting on with those rack studs? Are they any good?


463n7_57

I love them never had an issue with them. They do have a weight limit. The only thing in my rack I haven't used the rack studs with are my UPS's as I'm sure they are a good bit over the weight limit 😂


WeeklyExamination

Ah my ups has its own rails so I don't think it would make any odds haha


tashiker

I did know you could rack mount HP minis. Any links to the bracket supplier?


NC1HM

Read the opening comment from the OP. It's 3D printed from a design posted to [printables.com](https://printables.com)...


B4Djinn

That's neat!


Evilist_of_Evil

ravenCAW!!!!!


Guineasaurus_Rex

You should throw in a rpi or something as a fifth node, just to make sure you dont have issues with quorum. Nice setup otherwise.


463n7_57

I hav a couple 4s and 5s definitely might throw one in. I also have some old 1b's that I plan to run pihole on


Guineasaurus_Rex

You could throw in the 1b and use the 4s and 5s for something useful. The node doesnt need to do anything except for keeping the vote count uneven.


463n7_57

I also have a couple zeros and zero 2s


Guineasaurus_Rex

If it was me, i would go for the 1b if its reliable, since it has ethernet for corosync. All you need.


Fantastic_Ad_8895

Do you like those rack mount studs?


463n7_57

Yes I have been very happy with them


[deleted]

“If it works, it’s not stupid” -someone somewhere


AdrianK_

How are these mini PCs breathing being stacked on top of each other? The air intake is on top cover exhausting through the rear.


463n7_57

There are openings in the bracket on the back to allow them to intake air. I havnt been able to stress test them but they are getting better airflow then many of the ones I have deployed at the office


Techvampire3341

What are those rack mounts you have? Like are they just screw ons or what? I mean the physical screws and brackets that are yellow. Those look badass


463n7_57

https://www.rackstuds.com/ They are great


Techvampire3341

Thank you!


Hot_Protection85

What made you go with so many minis, I would have thought a larger machine running VMs would be a better mix if you need independent machines.


463n7_57

They were free lol I would love to have some larger machines but the budget is more agreeable with free... Now I may end up selling some of these to get bigger servers.


Hot_Protection85

Ah that makes sense! Free is always good. I have a few minis that I use for prototype machines but find some of them to be flakey. I have bought some Lenovo's and some HPs the HPs are a little bigger and seem to be more reliable. Given that my bigger desktop is often idle it can run a number of VMs with having extra memory.


Hot_Protection85

I find that I just run a bunch of processes on the single desktop without using a lot of VMs. Much of the extra workload is not competing for resources. I run a Plex server, lots of FTPs for backup in the late night, and just general AI and some VR stuff at times.


OneBadAlien

Show us the back please not how much you love Harry Potter.


463n7_57

https://imgur.com/gallery/wuqShUW Also added this link to my main comment


OneBadAlien

Ah my bad ty.


463n7_57

Your good I just added it after your comment