On Sat, Feb 06, 2021 at 04:11:46PM +0100, Pavel Raiskup wrote:
I'm trying to overcommit storage space on one our hypervisors (and reclaim space if possible after memory peak times) and it seems like kernel in Fedora 33 isn't doing automatic discard on swap partitions? The swap is enabled by:
$ swapon --discard /dev/sda
The guest (on RHEL 8 HV host) is started with virtio-scsi controller to propagate the trim requests to the host storage, the volumes I tried were both of raw and qcow2 format.
The 'fstrim -av' isn't propagated to SWAP mount points (expected probably). The only thing that helps to shrink the host volume allocated for guest swap is 'swapoff -a && swapon --discard /dev/sda' (but I assume TRIM is sent to the driver in this case, and that the problem isn't in the host libvirt setup).
fstrim will only trim a regular filesystem. AFAIK swap is not involved in any way when you do fstrim. On the other hand swapon --discard is correct, but you may need to check that --discard=pages was selected. I have no idea what particular operations will cause a TRIM to actually be sent though - it could also depend on the kernel version. It seems unlikely that the kernel would send TRIM for every single freed page because that would be very expensive on some devices.
If you want to monitor what requests are seen on the hypervisor side, I'd suggest running this command on the hypervisor:
nbdkit memory 10G --filter=log logfile=/var/tmp/log
Modify the guest's libvirt XML (virsh edit guestname) to add:
<disk type='network' device='disk'> <driver name='qemu' type='raw'/> <source protocol='nbd'> <host name='localhost' port='10809'/> </source> <target dev='sdb' bus='scsi'/> </disk>
(You might have to open port 10809 on the firewall, or else use a Unix domain socket)
Then put the swap on /dev/sdb in the guest.
Then do whatever experiments you need and you should see (or perhaps not see) Trim requests in /var/tmp/log.
You might nee dto play with <driver ... discard="unmap" ...> setting in the libvirt XML too. I can't remember if that's the default or not.
Rich.
But otherwise, freeing the guest memory manually, minimizing the swap usage, etc. (experiments with tmpfs) doesn't result in freeing the allocated storage blocks on host.
Am I doing something wrong, is this expected or is this a bug?
FTR, I tested "kernel-core-5.10.11-200.fc33.x86_64" and "kernel-core-5.10.12-200.fc33.x86_64" but this is not a regression -- I'm experimenting with TRIM in this setup for the first time.
Dmesg output seems to indicate page cluster trim is enabled (c symbol): [ 3936.914260] Adding 33554428k swap on /dev/sda. Priority:-2 extents:1 across:33554428k DscFS
Thanks, Pavel
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org