Pros and cons of Proxmox in a home lab?

Hi all. I was curious about some of the pros and cons of using Proxmox in a home lab set up. It seems like in most home lab setups it’s overkill. But I feel like there may be something I’m missing. Let’s say I run my home lab on two or three different SBCs. Main server is an x86 i5 machine with 16gigs memory and the others are arm devices with 8 gigs memory. Ample space on all. Wouldn’t Proxmox be overkill here and eat up more system resources than just running base Ubuntu, Debian or other server distro on them all and either running the services needed from binary or docker? Seems like the extra memory needed to run the Proxmox software and then the containers would just kill available memory or CPU availability. Am I wrong in thinking that Proxmox is better suited for when you have a machine with 32gigs or more of memory and some sort of base line powerful cpu?

Pika , (edited )
@Pika@sh.itjust.works avatar

I'm currently running proxmox on a 32 gig server running a ryzen 5600 G, it's going fine the containers don't actually use all that much RAM and personally I'm actually seeing a better benchmarks than I did when I just ran as a Bare Bones Ubuntu server, my biggest issue has actually been a larger IO strain than anything, because it's a lot more IO heavy now since everything's containerized. I think I easily could run it with a lower amount of ram I would just have to turn off some of the more RAM intensive items

As for if I regret changing, no way Jose, I absolutely love the ability of having everything containerized because I can set things up how I want it when I want it and if I end up screwing something up configuration wise or decide that I no longer need that service I can just nuke the container without having to remember well what did I install on this program so I can remove it and do other programs need this dependency to work. Plus while I haven't tinkered as much in this area, you can hard set what resources you want a lot to each instance, so if you have a program like say a pi hole that you know is never going to use x amount of resources to be able to appropriately work you can restrict what it can do so if something does go wrong with it it doesn't use all of your system resources

The biggest con out of it is probably having to figure out how to do the networking side because every container is going to have a different IP address, I found using a web dashboard is my friend because I can have heimdel tell me where all my services are and I just have to click the icon to bring me to the right IP address, it took a lot of work to figure out how it's operational and how to get it working, but the benefits I've gotten of having it is amazing. Just make sure you have a spare disk to temporarily clone partitions to because it's extremly difficult to use existing disks in the machine. I've been slowly going one at a time copying it over to an external drive nuking the and then reinitializing the disc as part of the proxmox lvm and then copying the data back over onto their appropriate image file.

specialseaweed ,

I used Proxmox for awhile, then went to Unraid. I learned a lot using Proxmox but for ease of homelabbing, it’s tough to beat Unraid. It depends on what you’re wanting from your lab.

peregus ,

I run it on a 4GB Fujitsu Futro S920! 😆
All the RAM seems to be used by 3 VMs. Some SWAP is been used, ok, but the Proxmox overhead doesn't seem that much.

TCB13 ,
@TCB13@lemmy.world avatar

C'mon just move to Incus: https://lemmy.world/comment/10896868 :P

peregus ,

No way! For just 1 reason: I will have to learn another new thing and replace it in about 6 servers. I value my time and for now Proxmox is fine.

P.s. Incus seems nice though! NO, stop tempting me!!! I'm already in the rabbit hole with a gazilion of self hosted services and dozens piling up in the to do list 🙈🙈🙈

TCB13 ,
@TCB13@lemmy.world avatar

Well, I understand your POV... but real software freedom instead of messages asking you to buy a license and a questionable kernel is always a good choice :P

kitnaht ,

Does Incus support things like Kernel Samepage Merging? How does it handle Windows VMs? Does the WebUI give a nice and easy novnc window that just works?

TCB13 , (edited )
@TCB13@lemmy.world avatar

Yes ksmtuned is your friend. For VMs it can be managed / enabled like any other Linux Kernel + QEMU/KVM running with KSM enabled.

On LXC containers it may be a bit harder as it depends a LOT, best results if you're using systemd both the host and containers. It may work out all out of the box or you'll have to resort to ksm_wrapper in both the Incus executable and the stuff running inside your containers.

Don't forget that:

KSM only operates on those areas of address space which an application
has advised to be likely candidates for merging, by using the madvise(2)
system call: int madvise(addr, length, MADV_MERGEABLE).
https://www.kernel.org/doc/Documentation/vm/ksm.txt

How does it handle Windows VMs

As one would except from QEMU... https://blog.simos.info/how-to-run-a-windows-virtual-machine-on-incus-on-linux/

Does the WebUI give a nice and easy novnc window

Yes it works fine. https://youtu.be/wqEH_d8LC1k?feature=shared&t=508

possiblylinux127 ,
@possiblylinux127@lemmy.zip avatar

Just wait until you get a few machines. You can live transfer things and dynamically allocate resources.

fortera ,

I use Proxmox/virtualisation because I want to be able to run services within their own OS. I've got a VM dedicated to docker both at home and in my colocation, since a lot of services I'm happy to just chuck on there, but there's others with more complex setups, and other services/systems that just running them in docker isn't an option.

LordCrom ,

I use it on reclaimed hardware ... Works great for me. Has all the features you'd want for a home lab, and I run a few production hosts there as well

rutrum ,
@rutrum@lm.paradisus.day avatar

It seemed nice at first, but one major issue: GPU passthrough was a nightmare. It cant be done in the UI and I didnt understand fully how it worked. There are many different tutorials not by promox that are outdated or may not work. It was frustrating enough I jumped to NixOS. Other hiccups included having to go to the terminal to passthrough drives for openmediavault, but that one was kind of straightforward atleast, and it worked first time.

In hindsight, I didnt actually need to virtualize everything at that level, so I never really had a good use case for it anyway. I use containers over entire VMs.

monkeyman512 ,

I think GPU passthrough has improved since you have used it. Some command line prep work is still necessary, but the passthrough config is done in the GUI.

Swarfega ,

I did it a week ago and it was just a case of passing through the video card. I came across a lot of guides and they were all in the CLI. I assume things have improved or maybe it differs per card. I was just using onboard graphics from an N100 CPU.

philpo ,

A lot of guides are still for Proxmox 7 or even 6 on that matter.

Proxmox 8 has changed a lot in that regard.

possiblylinux127 ,
@possiblylinux127@lemmy.zip avatar

The onboard iGPU doesn't need anything special once you turn on IOMMU. You just click add ePCI device.

possiblylinux127 ,
@possiblylinux127@lemmy.zip avatar

It really depends on the hardware. Also PVE 8 adds some improvements. I would just follow the wiki.

habitualTartare ,

I'm using a commercial desktop with an i5 Sandy bridge. I maxed out to 32Gb of ram only because I'm running trueNAS, debian with containers, and home assistant. Most RAM goes to trueNAS and trueNAS doesn't accurately report ram. For CPU, mostly just task limited but I don't really think thats a proxmox issue.
Obviously it's not going to support an enterprise or even small business but it works for what I need of less than 4 users on my budget.

Proxmox doesn't really ask for much but I probably would recommend docker for your arm devices.

possiblylinux127 ,
@possiblylinux127@lemmy.zip avatar

You need Proxmox

Seriously though it is nice to have

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • random
  • selfhosted@lemmy.world
  • test
  • worldmews
  • mews
  • All magazines
    •