Does this reduce your concern?

Are you sure that this data is compressed in memory as well as the on-disk encoding? The editing operations themselves surely operate on uncompressed data, and then it's compressed upon being written to disk.

Is earlyoom disabled on this system? It should send SIGTERM to something in this case - which is not the best UX as a hint that an adjustment needs to be made. But I want to make sure there isn't something else going wrong.

I apologize for the possible duplicate message, the past didn't show up for reasons I don't understand. I think it's a bad idea. It is better to make `zram-fraction` equal to 0.8 or 0.9, but not 1.0. For example, if you run Darktable, load a floating-point TIFF image into it, and try to export it, on systems with little RAM, it will fill both RAM itself and zram, and cause the system to freeze completely because the memory is almost incompressible. Also, other data (video editors, for example) which are not compressible can get into RAM.

For example, if you run Darktable, load a floating-point TIFF image into it, and try to export it, on systems with little RAM, it will fill both RAM itself and zram, and cause the system to freeze completely because the memory is almost incompressible.

Also, other data (video editors, for example) which are not compressible can get into RAM.

system hangs with earlyoom and swap on zram, disksize=150% MemTotal,

With disksize=100% MemTotal this should be very rare case.

Scaling ZRAM to full memory size

https://fedoraproject.org/wiki/Changes/Scale_ZRAM_to_full_memory_size == Summary == Fedora 33 [[Changes/SwapOnZRAM|enabled zram by default]]. The size of the virtual swap devices was set so that the amount of memory used for compressed swap pages was limited to a quarter of physical memory in typical scenarios. That size is now increased to half of physical memory (`zram-fraction` becomes 1.0, `max-zram-size` becomes 8 GiB). This allows systems with small amounts to successfully launch the [https://anaconda-installer.readthedocs.io/en/latest/ Anaconda installer] and other programs. == Owner == * Name: [[User:Zbyszek|Zbigniew Jędrzejewski-Szmek]] * Email: zbyszek(a)in.waw.pl * Name: [[User:jwrdegoede|Hans de Goede]] * Email: hdegoede(a)redhat.com == Detailed Description == When ZRAM was enabled by default in Fedora 33, the size of the device (before compression) was limited to fraction 0.5 of RAM or 4 GiB, whichever is less. The reason to limit the fraction to less than 1.0 is that with only incompressible data in memory, whole RAM would be filled by the "compressed" pages, leaving no RAM for normal use. But this concern seems to have been overblown, and there have been no reports of compressed swap taking up too much memory. In real use, we will have at least a few hundred MiB of code in memory, which compresses quite well, so some compression will occur even when working with incompressible data. In typical usage scenarios, we get effective compression of 2:1 or 3:1, which means that the compressed pages use at most between 1/4th and 1/6th of RAM. By increasing the fraction to 1.0, we expect to consign up to half of RAM for compressed pages. Increasing the size of the device is important for small devices. For devices with 1GB of RAM, sizing ZRAM to 1 GiB allows successful installation with Anaconda. In general, it is expected that having more compressed swap will make the device more functional after installation too. This increase is the most important for devices with small amounts of memory (< 1 GiB), but will be beneficial to systems with slightly larger systems (1–8 GiB). Technically, this change is completely trivial, amounting to two lines of configuration. Nevertheless, there is some potential for problems with incompressible workloads. The experience with ZRAM so far makes the Change authors optimistic that no issues will be encountered, but users and other developers should be aware of the change. == Benefit to Fedora == Fedora works better on devices with low RAM. In particular, the installer can function. == Scope == * Proposal owners: change the defaults in `zram-generator-defaults` package to `zram-fraction=1.0`, `max-zram-size=8096`. * Other developers: N/A (not a System Wide Change) * Policies and guidelines: N/A (not a System Wide Change) * Trademark approval: N/A (not needed for this Change) == Upgrade/compatibility impact == Default configuration is changed. Systems with local configuration will not be affected. Systems which had zram-generator disabled will not be affected. == How To Test == Install the new version of the `zram-generator-defaults` package, execute `sudo systemctl daemon-reload && sudo systemctl restart swap-create@zram0`, use `zramctl` to check that the device has the expected size. After some use, check that pages are compressed at least with ration 2:1: <pre> $ zramctl NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT /dev/zram0 lzo-rle 7.6G 4K 74B 12K 4 [SWAP] ^ ^ ^ | \____\_______ the ratio of those values should be more than 2:1 | `------ this should be approx the size of RAM </pre> (Note that the device is destroyed and recreated during restart. This means that all pages will be "swapped in", i.e. decompressed. On machines with low memory, rebooting might be a better option.) == User Experience == Not user visible. == Dependencies == None. == Contingency Plan == * Contingency mechanism: Revert default configuration to previous values. * Contingency deadline: Final freeze, any time really. * Blocks release? No * Blocks product? No == Documentation == None. == Release Notes == The default size of compressed zram devices (`/dev/zram0`) has been increased to 1.0 fraction of RAM, or 8 GiB, whichever is smaller. -- Ben Cotton He / Him / His Senior Program Manager, Fedora & CentOS Stream Red Hat TZ=America/Indiana/Indianapolis _______________________________________________ devel mailing list -- devel(a)lists.fedoraproject.org To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

== Scope == * Proposal owners: change the defaults in `zram-generator-defaults` package to `zram-fraction=1.0`, `max-zram-size=8096`.

> Someone on Fedora Discussion¹ noticed this number and wonders if it's meant > to be 8192. Not that it probably matters very much, but, you know, nice > round numbers and all. Yep, it was. I'll fix the wiki.

That's about changing settings. When restarting the zram service, it has to remove the swap device which forces all the swapped pages back into RAM. This might not be successful if you don't have enough RAM, so if you are using a low memory device, you should reboot instead. Although if you see that you aren't using too much swap, then it could be ok.

Now I'm a bit torn: the code is nice enough, but it seems to be a solution in search of a problem. So I thought I'd try a little crowd-sourcing: Would we have a real use for something like this?

- low memory (4G or less, somewhat subjective)

On Sat, Jan 23, 2021 at 4:29 AM Zbigniew Jędrzejewski-Szmek <zbyszek(a)in.waw.pl&gt; wrote: > > Hi, > > the proposal for Fedora 34 is to use zram-size == 1.0 * ram. > (Which I think is OK for the reasons listed in the Change page [0].) > But the original motivation for this change was boosting the size on > machines with little ram [1]. I wrote an exploratory patch [2] to specify > the size as a formula. From the docs: > > > An alternative way to set the zram device size as a mathematical expressoin > > that can be used instead of 'zram-fraction' and 'max-zram-size'. Basic arithmetic > > operators like '*', '+', '-', '/', are supported, as well as 'min()' and 'max()' > > and the variable 'ram' which specifies size of RAM in megabytes. > > > > Examples: > > > > # this is the same as the default config > > zram-size = min(0.5 * ram, 4096) > > > > # fraction 1.0 for first 4GB, and then fraction 0.5 above that > > zram-size = 1.0 * min(ram, 4096) + 0.5 * max(ram - 4096, 0) > > Now I'm a bit torn: the code is nice enough, but it seems to be a solution > in search of a problem. So I thought I'd try a little crowd-sourcing: > Would we have a real use for something like this? > > (One possible direction: one thing I want to explore next is using zram > or zwap based on whether the machine has a physical swap device. Maybe > such a language would be useful then — with additional variables > specifying e.g. the physical swap size…) I think everything discussed so far is neutral to good for all use cases (all editions and spins) being discussed. But I also think it's a good idea for zram-generator to be a bit more biased toward setups with one or more of: - low memory (4G or less, somewhat subjective) - limited life storage (eMMC, SD Card, USB stick) - slow drive (primarily rotational, but also all of the above) - no swap (zram-based swap is better than no swap) The above categories pretty much mean that the improvements for disk-based swap that are in-progress upstream, aren't likely to be used. That means zram-based swap usage will continue to provide significant benefit.

That's the tl;dr and now it's giant text wall time... The remaining category is "everyone else", i.e. >= 8G RAM, and a reasonably performant SATA SSD or NVMe. This category benefits overall with the swap on zram approach, mainly because swap thrashing is just so terrible. However, I expect the future is a return to disk based swap for two reasons: (1) given highly variable workloads, having 100% eviction efficacy decomplicates memory management and resource control (2) there are upstream improvements happening incrementally that are improving swap performance. e.g. the anonymous memory balancing logic has been totally reworked. Neither zram nor zswap support cgroupvs2. There's work happening on getting zswap cgroups compatible as well as integrating it into memory management rather than having all these different buffet-style add-ons that distros and users have to evaluate and integrate. The swap improvements started happening in kernel 5.8, and I'd say they're opt-in testable [1] for folks using kernel 5.10+ - whereby they can switch back and forth between exclusively zram-based and disk-based swap, to help evaluate what's working better and what isn't. This is not a case of us moving back to disk-based swap soon. There still is no cgroups support in device-mapper, and right now the only way to secure swap is to put it on dm-crypt. One of the benefits of zram-based swap is, it's volatile, so any leaks of personal information can be ignored (at least if the system is powered off). Another issue is dynamically creating/removing swapfiles. We kinda want to avoid partitions because that preallocation siphons away a possibly limited resource that may not get used at all; and also related is still pending work (not yet happening) on Secure Boot and hibernation images which would necessarily need disk based swap. Anyway there's quite a lot of work happening, and even though it isn't ready to be used by default in Fedora, it is a good time for early adopters to do performance testing as this work continues. I anticipate the server and desktop will eventually move away from zram-based swap, but I can't give a time frame for it. [1] For the early adopters who want to experiment with their swap dependent workloads and different configurations: https://github.com/facebookexperimental/resctl-demo [2] One thing not discussed much is where to put the swapfile on Btrfs. This is my current suggestion: btrfs sub create /var/swap chattr +C /var/swap fallocate -l 4G /var/swap/swapfile1 mkswap /var/swap/swapfile1 swapon /var/swap/swapfile1 Be sure to read the limitations in 'man 5 btrfs' - the above takes care of most concerns, the other one is it needs to be a single device Btrfs. Other arrangements are possible. Ping me on irc (cmurf) if you have questions about alternatives. -- Chris Murphy _______________________________________________ devel mailing list -- devel(a)lists.fedoraproject.org To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

On Thu, Jan 28, 2021 at 03:20:38PM +0200, Alexander Bokovoy wrote: > With today's OpenQA tests I can point out that using zram on 2048MB RAM > VMs actually breaks FreeIPA deployment: > https://openqa.fedoraproject.org/tests/763006#step/role_deploy_domain_con... > > OpenQA uses 2048MB RAM for QEMU VMs and this was typically OK for > FreeIPA deployment with integrated CA and DNS server. Not anymore with > zram activated: > > Jan 27 21:17:47 fedora zram_generator::generator[25243]: Creating unit dev-zram0.swap (/dev/zram0 with 1384MB) > > which ends up eating 2/3rds of the whole memory budget and FreeIPA > installer fails: > > 2021-01-28T02:18:31Z DEBUG ipa-server-install was invoked with arguments [] and options: {'unattended': True, 'ip_addresses': None, 'domain_name': 'test.openqa.fedoraproject.org', 'realm_name': 'TEST.OPENQA.FEDORAPROJECT.ORG', 'host_name': None, 'ca_cert > 2021-01-28T02:18:31Z DEBUG IPA version 4.9.1-1.fc34 > 2021-01-28T02:18:31Z DEBUG IPA platform fedora > 2021-01-28T02:18:31Z DEBUG IPA os-release Fedora 34 (Server Edition Prerelease) > 2021-01-28T02:18:31Z DEBUG Available memory is 823529472B > ... > 2021-01-28T02:18:31Z DEBUG The ipa-server-install command failed, exception: ScriptError: Less than the minimum 1.2GB of RAM is available, 0.77GB available > 2021-01-28T02:18:31Z ERROR Less than the minimum 1.2GB of RAM is available, 0.77GB available > 2021-01-28T02:18:31Z ERROR The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information Enabling zram doesn't really "take away memory", because no pre-allocation happens. If there is no physical swap, then adding zram0 should just shown additional swap space, so I don't think it could cause the check to fail. But if there is physical swap, the zram device is used with higher preference than the physical swap. So I think the explanation could be that the VM has a swap partition. Before, some pages would be swapped out to zram, and some would be swapped out to the "real" swap. The fraction of RAM used for compressed zram would be on the order of 25% (zram-fraction=0.5 multiplied by typical compression 2:1). But now the kernel sees more zram swap, so it inserts pages there, taking away more of RAM, instead of saving pages to disk. So more memory (maybe 50% RAM) is used for the unswappable compressed pages. But this shouldn't break things: if there is enough pressure, pages would be swapped out to the physical swap device too. Assuming that this guess is correct, the check that ipa-server-install is doing should be adjusted. It should use the total available memory (ram + all kinds of swap) in the check, and not just available uncompressed pages. Or if it wants to ignore disk-based swap for some reason, it should use ram + zram + in-memory-zwap in the check.

It would be nice to see the output of 'swapon -s' and 'zramctl' and 'free' on that machine. > While we can ask Adam to increase memory in those VMs, 2GB RAM was our > (FreeIPA) recommended lower level target for home deployments with > Celeron or RPI4 systems. Now zram use will force those systems to be > unusable out of the box. That's certainly not the goal. The main goal of the Change is to support machines with less RAM, not require more RAM. Zbyszek _______________________________________________ devel mailing list -- devel(a)lists.fedoraproject.org To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

On Thu, Jan 28, 2021 at 11:04:34AM -0500, Rob Crittenden wrote: > Zbigniew Jędrzejewski-Szmek wrote: > > On Thu, Jan 28, 2021 at 03:20:38PM +0200, Alexander Bokovoy wrote: > >> With today's OpenQA tests I can point out that using zram on 2048MB RAM > >> VMs actually breaks FreeIPA deployment: > >> https://openqa.fedoraproject.org/tests/763006#step/role_deploy_domain_con... > >> > >> OpenQA uses 2048MB RAM for QEMU VMs and this was typically OK for > >> FreeIPA deployment with integrated CA and DNS server. Not anymore with > >> zram activated: > >> > >> Jan 27 21:17:47 fedora zram_generator::generator[25243]: Creating unit dev-zram0.swap (/dev/zram0 with 1384MB) > >> > >> which ends up eating 2/3rds of the whole memory budget and FreeIPA > >> installer fails: > >> > >> 2021-01-28T02:18:31Z DEBUG ipa-server-install was invoked with arguments [] and options: {'unattended': True, 'ip_addresses': None, 'domain_name': 'test.openqa.fedoraproject.org', 'realm_name': 'TEST.OPENQA.FEDORAPROJECT.ORG', 'host_name': None, 'ca_cert > >> 2021-01-28T02:18:31Z DEBUG IPA version 4.9.1-1.fc34 > >> 2021-01-28T02:18:31Z DEBUG IPA platform fedora > >> 2021-01-28T02:18:31Z DEBUG IPA os-release Fedora 34 (Server Edition Prerelease) > >> 2021-01-28T02:18:31Z DEBUG Available memory is 823529472B > >> ... > >> 2021-01-28T02:18:31Z DEBUG The ipa-server-install command failed, exception: ScriptError: Less than the minimum 1.2GB of RAM is available, 0.77GB available > >> 2021-01-28T02:18:31Z ERROR Less than the minimum 1.2GB of RAM is available, 0.77GB available > >> 2021-01-28T02:18:31Z ERROR The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information > > > > Enabling zram doesn't really "take away memory", because no pre-allocation happens. > > If there is no physical swap, then adding zram0 should just shown additional > > swap space, so I don't think it could cause the check to fail. > > But if there is physical swap, the zram device is used with higher preference > > than the physical swap. So I think the explanation could be that the VM has > > a swap partition. Before, some pages would be swapped out to zram, and some would > > be swapped out to the "real" swap. The fraction of RAM used for compressed zram > > would be on the order of 25% (zram-fraction=0.5 multiplied by typical compression 2:1). > > > > But now the kernel sees more zram swap, so it inserts pages there, taking away > > more of RAM, instead of saving pages to disk. So more memory (maybe 50% RAM) is > > used for the unswappable compressed pages. But this shouldn't break things: > > if there is enough pressure, pages would be swapped out to the physical swap device > > too. > > > > Assuming that this guess is correct, the check that ipa-server-install is > > doing should be adjusted. It should use the total available memory (ram + all kinds > > of swap) in the check, and not just available uncompressed pages. > > Or if it wants to ignore disk-based swap for some reason, it should use > > ram + zram + in-memory-zwap in the check. > > For bare metal IPA uses the python3-psutil call: > psutil.virtual_memory.available() > > I don't know how/if psutil reports zram (or cgroup v1 and v2 for that > matter). psutil (in general) reports data from /proc/meminfo; available come from MemAvailable: in that file. This is defined in kernel as: MemAvailable: An estimate of how much memory is available for starting new applications, without swapping. Calculated from MemFree, SReclaimable, the size of the file LRU lists, and the low watermarks in each zone. The estimate takes into account that the system needs some page cache to function well, and that not all reclaimable slab will be reclaimable, due to items being in use. The impact of those factors will vary from system to system. Notice "without swapping" in second line. Next question, how zram impacts reporting of MemAvailable by kernel?

> >> > ram + zram + in-memory-zwap in the check. > >> > >> For bare metal IPA uses the python3-psutil call: > >> psutil.virtual_memory.available() > >> > >> I don't know how/if psutil reports zram (or cgroup v1 and v2 for that > >> matter). > > > > psutil (in general) reports data from /proc/meminfo; available come > > from MemAvailable: in that file. This is defined in kernel as: > > > >MemAvailable: An estimate of how much memory is available for starting new > > applications, without swapping. Calculated from MemFree, > > SReclaimable, the size of the file LRU lists, and the low > > watermarks in each zone. > > The estimate takes into account that the system needs some > > page cache to function well, and that not all reclaimable > > slab will be reclaimable, due to items being in use. The > > impact of those factors will vary from system to system. > > > >Notice "without swapping" in second line. Next question, how zram impacts > >reporting of MemAvailable by kernel? > > This is a good note. If zram breaks kernel API promise to user space > (/proc/meminfo is one such API), how can it be enabled by default. I > also would question enabling zram by default if it does not play along > with cgroups. We do depend on cgroups being properly managed by systemd, > including resource allocation. > > In my opinion, zram enablement in Fedora is quite premature. > It's the default Fedora wide since Fedora 33. It's been used by default in Fedora IoT since the beginning, and in openQA Anaconda tests for even longer than that. What's premature about it?

VM 5.11.0-0.rc5.20210127git2ab38c17aac1.136.fc34.x86_64 [ 0.701792] Memory: 3016992K/4190656K available (43019K kernel code, 11036K rwdata, 27184K rodata, 5036K init, 31780K bss, 1173408K reserved, 0K cma-reserved) Baremetal 5.10.11-200.fc33.x86_64 [ 0.125875] Memory: 12059084K/12493424K available (14345K kernel code, 3465K rwdata, 9704K rodata, 2536K init, 5504K bss, 434080K reserved, 0K cma-reserved)

On Thu, 2021-01-28 at 13:46 -0700, Chris Murphy wrote: > > OK I'm seeing this problem in a VM with > Fedora-Workstation-Live-x86_64-Rawhide-20210128.n.0.iso but I'm not > sure how consistent it is yet. MemTotal is ~3G for a VM that has 4G > allocated. Something's wrong... > > VM 5.11.0-0.rc5.20210127git2ab38c17aac1.136.fc34.x86_64 > [ 0.701792] Memory: 3016992K/4190656K available (43019K kernel > code, 11036K rwdata, 27184K rodata, 5036K init, 31780K bss, 1173408K > reserved, 0K cma-reserved) > > Baremetal 5.10.11-200.fc33.x86_64 > [ 0.125875] Memory: 12059084K/12493424K available (14345K kernel > code, 3465K rwdata, 9704K rodata, 2536K init, 5504K bss, 434080K > reserved, 0K cma-reserved) > > Why is reserved so much higher in the VM case? It clearly sees the 4G > but is delimiting it to 3G for some reason I don't understand. This is > well before the zram module is loaded, by the way. I've filed https://bugzilla.redhat.com/show_bug.cgi?id=1921923 for this. zdzichu suggests https://lkml.org/lkml/2021/1/25/701 may be related.

This is a good note. If zram breaks kernel API promise to user space (/proc/meminfo is one such API), how can it be enabled by default. I also would question enabling zram by default if it does not play along with cgroups. We do depend on cgroups being properly managed by systemd, including resource allocation.

On Thu, 2021-01-28 at 14:16 +0000, Zbigniew Jędrzejewski-Szmek wrote: > On Thu, Jan 28, 2021 at 03:20:38PM +0200, Alexander Bokovoy wrote: > > With today's OpenQA tests I can point out that using zram on 2048MB RAM > > VMs actually breaks FreeIPA deployment: > > https://openqa.fedoraproject.org/tests/763006#step/role_deploy_domain_con... > > > > OpenQA uses 2048MB RAM for QEMU VMs and this was typically OK for > > FreeIPA deployment with integrated CA and DNS server. Not anymore with > > zram activated: > > > > Jan 27 21:17:47 fedora zram_generator::generator[25243]: Creating unit dev-zram0.swap (/dev/zram0 with 1384MB) > > > > which ends up eating 2/3rds of the whole memory budget and FreeIPA > > installer fails: > > > > 2021-01-28T02:18:31Z DEBUG ipa-server-install was invoked with arguments [] and options: {'unattended': True, 'ip_addresses': None, 'domain_name': 'test.openqa.fedoraproject.org', 'realm_name': 'TEST.OPENQA.FEDORAPROJECT.ORG', 'host_name': None, 'ca_cert > > 2021-01-28T02:18:31Z DEBUG IPA version 4.9.1-1.fc34 > > 2021-01-28T02:18:31Z DEBUG IPA platform fedora > > 2021-01-28T02:18:31Z DEBUG IPA os-release Fedora 34 (Server Edition Prerelease) > > 2021-01-28T02:18:31Z DEBUG Available memory is 823529472B > > ... > > 2021-01-28T02:18:31Z DEBUG The ipa-server-install command failed, exception: ScriptError: Less than the minimum 1.2GB of RAM is available, 0.77GB available > > 2021-01-28T02:18:31Z ERROR Less than the minimum 1.2GB of RAM is available, 0.77GB available > > 2021-01-28T02:18:31Z ERROR The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information > > Enabling zram doesn't really "take away memory", because no pre-allocation happens. > If there is no physical swap, then adding zram0 should just shown additional > swap space, so I don't think it could cause the check to fail. > But if there is physical swap, the zram device is used with higher preference > than the physical swap. So I think the explanation could be that the VM has > a swap partition. The openQA test in question runs after, and uses the hard disk from, a test that runs a default Fedora Server install: https://openqa.fedoraproject.org/tests/763650 which, AIUI, should not be creating a swap partition. The logs from the test - https://openqa.fedoraproject.org/tests/763657/file/role_deploy_domain_con... - do not show any swaps being activated other than zram ones. So no, I don't think there is a swap partition.

With today's OpenQA tests I can point out that using zram on 2048MB RAM VMs actually breaks FreeIPA deployment: https://openqa.fedoraproject.org/tests/763006#step/role_deploy_domain_con... OpenQA uses 2048MB RAM for QEMU VMs and this was typically OK for FreeIPA deployment with integrated CA and DNS server. Not anymore with zram activated: Jan 27 21:17:47 fedora zram_generator::generator[25243]: Creating unit dev-zram0.swap (/dev/zram0 with 1384MB)

which ends up eating 2/3rds of the whole memory budget and FreeIPA installer fails:

2021-01-28T02:18:31Z DEBUG ipa-server-install was invoked with arguments [] and options: {'unattended': True, 'ip_addresses': None, 'domain_name': ' test.openqa.fedoraproject.org', 'realm_name': ' TEST.OPENQA.FEDORAPROJECT.ORG', 'host_name': None, 'ca_cert 2021-01-28T02:18:31Z DEBUG IPA version 4.9.1-1.fc34 2021-01-28T02:18:31Z DEBUG IPA platform fedora 2021-01-28T02:18:31Z DEBUG IPA os-release Fedora 34 (Server Edition Prerelease) 2021-01-28T02:18:31Z DEBUG Available memory is 823529472B ... 2021-01-28T02:18:31Z DEBUG The ipa-server-install command failed, exception: ScriptError: Less than the minimum 1.2GB of RAM is available, 0.77GB available.

(One possible direction: one thing I want to explore next is using zram or zwap based on whether the machine has a physical swap device. Maybe such a language would be useful then — with additional variables specifying e.g. the physical swap size…)