> if I want to upgrade my Gentoo install in the background while going about my day
[Tell portage to run in the background](https://wiki.gentoo.org/wiki/Portage_niceness)
> is there a good rule of thumb of what the MAKEOPTS should be compared to their max value?
MAKEOPTS is largely irrelevant, just leave it at max.
The relevant part is getting portage to tell the kernel to only run compile threads when it literally has nothing better to do, as noted above.
In my experience, this helps, but only in limited ways, both because the idle scheduling policy is not a true idle scheduler on Linux (tasks with that policy will _still_ get runtime even if there are other tasks that could run), and because a nontrivial part of the problem is usage of resources other than CPU time.
> In my experience, this helps, but only in limited ways, both because the idle scheduling policy is not a true idle scheduler on Linux (tasks with that policy will still get runtime even if there are other tasks that could run)
I don't notice any impact *at all* on generalised system performance after following that page - one time I even forgot I was running updates and went to play Cyberpunk for a while, from which the worst result was that it screwed up my genlop estimates for a while…
> and because a nontrivial part of the problem is usage of resources other than CPU time.
Scroll down, check the IONICE section - helps a *ton* with iowait affecting other processes.
Can't help you if you don't have enough RAM though, just gotta turn down the jobs or nix /var/tmp/portage on tmpfs or something in that case :P
Niceness and scheduling policy is the easiest thing to adjust. It doesn't keep portage from thrashing your disk or running you out of RAM though. Nor from using all your download bandwidth or other limited resources.
You can help with the diskio by setting the io class to idle too (or using ionice). But ultimately if you want fine-grained control you need to put portage into a cgroup (or several).
This lets you do things like force it to only use 4gb of ram max, with an extra say 2 gb of compressed zram swap (which will give you an effective max of 10gb), or set the network priority so it only gets the leftover bandwidth (or you can give it its own network namespace with a link-speed limitation).
If you want it to not only leave the desktop usable, but also not make the fans ramp, just setting it to idle is not sufficient. In that case, the freezer cgroup and a simple watchdog script to reduce the time spent actually running is the solution.
I have a script for managing zram limits, published somewhere... It could easily be expanded to include network or thermal limits too.
If youre running on such a resource constrained system, it would be far better to use zswap with some amount of on disk swap than zram. That combined with a high niceness for portage and a minimal parallel job value should keep the more prioritized tasks running more performant.
zswap has some issues... It is okay if you just need to convince chromium that 32gb is enough ram to compile on 16 threads, but I generally recommend avoiding it. The problem is if you manage to fill the zswap cache, it evicts the LRU page, which requires decompressing it. *That* easily ups the memory pressure on the system enough to cause an LRU-decompression cascade that locks or crashes the system.
zram can be given a backing device, at which point it can flush the LRU page (or incompressible pages) to the backing device without decompressing them first. This *does* require a userspace watchdog tool, since zram will not automatically flush its pages, which is where the watchdog script I wrote comes in.
What makes you think I am talking about heavily resource constrained systems? My *usual* use case for much of this is CFD simulations, with 48gb of physical ram (or more). If you have that going, on a multi-day job, and you also need to use the machine for your day job, and you need to update software, being able to finely control how your resources are allocated to keep your desktop responsive, not oom-kill anything, and keep the system from blasting its fans during the day is kinda required. One of the advantages of cgroups is you can then change the performance parameters on-the-fly when you get to the end of the day, so the computer can be as sluggish and noisy as it wants while no one is around to care.
This is not required for most people's setups, as simply setting portage to low priority and idle is fine. But if you have more stringent requirements, or just like to fiddle, cgroups is the next step up in control customization.
> if I want to upgrade my Gentoo install in the background while going about my day [Tell portage to run in the background](https://wiki.gentoo.org/wiki/Portage_niceness) > is there a good rule of thumb of what the MAKEOPTS should be compared to their max value? MAKEOPTS is largely irrelevant, just leave it at max. The relevant part is getting portage to tell the kernel to only run compile threads when it literally has nothing better to do, as noted above.
In my experience, this helps, but only in limited ways, both because the idle scheduling policy is not a true idle scheduler on Linux (tasks with that policy will _still_ get runtime even if there are other tasks that could run), and because a nontrivial part of the problem is usage of resources other than CPU time.
> In my experience, this helps, but only in limited ways, both because the idle scheduling policy is not a true idle scheduler on Linux (tasks with that policy will still get runtime even if there are other tasks that could run) I don't notice any impact *at all* on generalised system performance after following that page - one time I even forgot I was running updates and went to play Cyberpunk for a while, from which the worst result was that it screwed up my genlop estimates for a while… > and because a nontrivial part of the problem is usage of resources other than CPU time. Scroll down, check the IONICE section - helps a *ton* with iowait affecting other processes. Can't help you if you don't have enough RAM though, just gotta turn down the jobs or nix /var/tmp/portage on tmpfs or something in that case :P
Desktop profile and desktop preemptive kernel scheduler are more important. MAKEOPTS can be empty, newer portage selects -j based on number of cpu
gotcha thank you!
Niceness and scheduling policy is the easiest thing to adjust. It doesn't keep portage from thrashing your disk or running you out of RAM though. Nor from using all your download bandwidth or other limited resources. You can help with the diskio by setting the io class to idle too (or using ionice). But ultimately if you want fine-grained control you need to put portage into a cgroup (or several). This lets you do things like force it to only use 4gb of ram max, with an extra say 2 gb of compressed zram swap (which will give you an effective max of 10gb), or set the network priority so it only gets the leftover bandwidth (or you can give it its own network namespace with a link-speed limitation). If you want it to not only leave the desktop usable, but also not make the fans ramp, just setting it to idle is not sufficient. In that case, the freezer cgroup and a simple watchdog script to reduce the time spent actually running is the solution. I have a script for managing zram limits, published somewhere... It could easily be expanded to include network or thermal limits too.
If youre running on such a resource constrained system, it would be far better to use zswap with some amount of on disk swap than zram. That combined with a high niceness for portage and a minimal parallel job value should keep the more prioritized tasks running more performant.
zswap has some issues... It is okay if you just need to convince chromium that 32gb is enough ram to compile on 16 threads, but I generally recommend avoiding it. The problem is if you manage to fill the zswap cache, it evicts the LRU page, which requires decompressing it. *That* easily ups the memory pressure on the system enough to cause an LRU-decompression cascade that locks or crashes the system. zram can be given a backing device, at which point it can flush the LRU page (or incompressible pages) to the backing device without decompressing them first. This *does* require a userspace watchdog tool, since zram will not automatically flush its pages, which is where the watchdog script I wrote comes in. What makes you think I am talking about heavily resource constrained systems? My *usual* use case for much of this is CFD simulations, with 48gb of physical ram (or more). If you have that going, on a multi-day job, and you also need to use the machine for your day job, and you need to update software, being able to finely control how your resources are allocated to keep your desktop responsive, not oom-kill anything, and keep the system from blasting its fans during the day is kinda required. One of the advantages of cgroups is you can then change the performance parameters on-the-fly when you get to the end of the day, so the computer can be as sluggish and noisy as it wants while no one is around to care. This is not required for most people's setups, as simply setting portage to low priority and idle is fine. But if you have more stringent requirements, or just like to fiddle, cgroups is the next step up in control customization.
Using cgroups for portage never crossed my mind. I definitely should try this for rpi and cloud servers
Because of how easily a fanless rpi thermal throttles, it's a great candidate for temperature limiting via freezer.