I don’t have a really good reason not to use LXCs right now. I use VMs because that’s what I knew when I started with Proxmox and the Internet seems pretty divided on when each one shines over the other. The goal of my switch to podman was twofold: switch to rootless and use something with better systemd support. I was hacking together unit files for docker using some pretty dumb tricks, none of that is necessary with quadlets though.
What’s the benefit in your eyes for LXC over VM? I don’t run Windows or anything so using the host kernel isn’t an issue for me. I do sometime have problems with OOM kills taking out a VM though, but my understanding is if it were an LXC that kill could have hit a much more important process than my general apps VM.
E: As far as Fedora under IBM… I don’t like it either. I’m relatively prepared to jump back to Debian though, I’ve kept my Andi key playbooks updated for both Fedora and Debian just in case I have to go back.
In my case, I don’t need the isolation of a VM, really I’m just looking to separate the service I’m running into something manageable and easy to move between hosts. I could do a VM for each, but I’d be adding overhead and power requirements without much benefit.
And really that’d all it comes down to for me. Each service is its own LXC, from stuff most self-hosters use, to random one-offs I write. Managing it all stays in ansible for everything, and the structure is quite a bit simpler.
When I do want to bring it elsewhere, I van package it up clean and toss that on a new LXC somewhere else quickly, like an 80 core monster with $16k in GPU thats already getting pushed hard, and knowing it will be of almost no impact to its main job while adding the service it needs.
I do still have VMs, but that is to do things like dealing with windows. Especially specific versions, like a piece of software for some work stuff that requires XP or server 2008 specifically. Its pretty isolated though, not even allowed network access out. All my writes are to a thumb drive if I need to get something out of it (which is uniquely set as the thumb drive its allowed to see).
So nothing that I couldn’t do a bunch of other ways, this is just the structure thats working best for me.
The biggest thing keeping from doing an LXC per app is a poor decision when I first set the lab up, I only gave it a /24 and didn’t separate out iot/user devices/servers so I’m flirting with exhausting the IPs. I’m planning on setting up opnsense soon so that should take care of it. I have a few different servers with apps grouped by type/priority and then running podman for the containers inside. It works well and I probably shouldn’t change it for no real reason.
Ah - yeah ive got trunk to each of the machines in my clusters, 9 vlans total, and of course I can add more whenever this way. I’m a bit of a glutton for naming and numbering structure too, so the purpose of the service determines which VLAN its on. Like Home Assistant has just about its own vlan, with sensors and misc tools in support of it all there. A different one for IoT devices by others (that I will never trust with internet access, so its initiate from another VLAN on the FW only, outbound can’t be initiated from any device on it, etc), one for work thats part of a site-to-site with work, with a few ports on the switch allocated that I can just plug in ad hoc, etc.
Definitely helps to have the range to play it this way!
In an ideal world I have multiple vlans for studf like iot, security cameras, my personal devices, my family’s personal devices, and various ones for lab stuff (externally available apps, critical apps, etc.)
Networking is my biggest neglect and learning it to start fixing things feels pretty daunting when I only have an hour or so some nights to tinker. I’ll get there eventually though.
Its well worth it IMO, makes service segregation so much easier. It may help to toss a router off your main network, and start experimenting that way, give you a decent place to mess things up - which is, again purely my opinion, one of the best ways to learn.
I don’t have a really good reason not to use LXCs right now. I use VMs because that’s what I knew when I started with Proxmox and the Internet seems pretty divided on when each one shines over the other. The goal of my switch to podman was twofold: switch to rootless and use something with better systemd support. I was hacking together unit files for docker using some pretty dumb tricks, none of that is necessary with quadlets though.
What’s the benefit in your eyes for LXC over VM? I don’t run Windows or anything so using the host kernel isn’t an issue for me. I do sometime have problems with OOM kills taking out a VM though, but my understanding is if it were an LXC that kill could have hit a much more important process than my general apps VM.
E: As far as Fedora under IBM… I don’t like it either. I’m relatively prepared to jump back to Debian though, I’ve kept my Andi key playbooks updated for both Fedora and Debian just in case I have to go back.
In my case, I don’t need the isolation of a VM, really I’m just looking to separate the service I’m running into something manageable and easy to move between hosts. I could do a VM for each, but I’d be adding overhead and power requirements without much benefit.
And really that’d all it comes down to for me. Each service is its own LXC, from stuff most self-hosters use, to random one-offs I write. Managing it all stays in ansible for everything, and the structure is quite a bit simpler.
When I do want to bring it elsewhere, I van package it up clean and toss that on a new LXC somewhere else quickly, like an 80 core monster with $16k in GPU thats already getting pushed hard, and knowing it will be of almost no impact to its main job while adding the service it needs.
I do still have VMs, but that is to do things like dealing with windows. Especially specific versions, like a piece of software for some work stuff that requires XP or server 2008 specifically. Its pretty isolated though, not even allowed network access out. All my writes are to a thumb drive if I need to get something out of it (which is uniquely set as the thumb drive its allowed to see).
So nothing that I couldn’t do a bunch of other ways, this is just the structure thats working best for me.
The biggest thing keeping from doing an LXC per app is a poor decision when I first set the lab up, I only gave it a /24 and didn’t separate out iot/user devices/servers so I’m flirting with exhausting the IPs. I’m planning on setting up opnsense soon so that should take care of it. I have a few different servers with apps grouped by type/priority and then running podman for the containers inside. It works well and I probably shouldn’t change it for no real reason.
Ah - yeah ive got trunk to each of the machines in my clusters, 9 vlans total, and of course I can add more whenever this way. I’m a bit of a glutton for naming and numbering structure too, so the purpose of the service determines which VLAN its on. Like Home Assistant has just about its own vlan, with sensors and misc tools in support of it all there. A different one for IoT devices by others (that I will never trust with internet access, so its initiate from another VLAN on the FW only, outbound can’t be initiated from any device on it, etc), one for work thats part of a site-to-site with work, with a few ports on the switch allocated that I can just plug in ad hoc, etc.
Definitely helps to have the range to play it this way!
In an ideal world I have multiple vlans for studf like iot, security cameras, my personal devices, my family’s personal devices, and various ones for lab stuff (externally available apps, critical apps, etc.)
Networking is my biggest neglect and learning it to start fixing things feels pretty daunting when I only have an hour or so some nights to tinker. I’ll get there eventually though.
Its well worth it IMO, makes service segregation so much easier. It may help to toss a router off your main network, and start experimenting that way, give you a decent place to mess things up - which is, again purely my opinion, one of the best ways to learn.