I tried to set up an aarch64 vm, and not really wanting to set up boot server and such. I just ran. virt-install \ --name Fedora_25_AArch64 --ram 2048 --arch aarch64 \ --disk size=8 --os-variant fedora25 \ --location https://dl.fedoraproject.org/pub/fedora-secondary/releases/25/Everything/... --extra-args "inst.ks=https://pwhalen.fedorapeople.org/kickstarts/Fedora-Minimal-AArch64.ks" It almost worked. I already had qemu-kvm set up for a windows partition for my vehicle obdII software probably with group install virtualization, then I installed qemu-system-arm and qemu-system-aarch64 along with edk2-aarch64 libguestfs-tools-c It bombed out at (I have the full install log including the backtrace but I didn't want to spam the list with it.) It appeared to be going okay for a while. It is using the 4.8.6-300 kernel, is there an SMP bug in it? Or any suggestions?

----- Original Message ----- From: Paul Whalen <pwhalen(a)redhat.com&gt; To: Sean Omalley <omalley_s(a)rocketmail.com&gt; Cc: arm(a)lists.fedoraproject.org Sent: Wednesday, February 8, 2017 10:48 AM Subject: Re: [fedora-arm] Qemu aarch64 install ----- Original Message ----- > I tried to set up an aarch64 vm, and not really wanting to set up boot > server > and such. I just ran. > > > virt-install \ > --name Fedora_25_AArch64 --ram 2048 --arch aarch64 \ > --disk size=8 --os-variant fedora25 \ > --location > https://dl.fedoraproject.org/pub/fedora-secondary/releases/25/Everything/... > --extra-args > "inst.ks=https://pwhalen.fedorapeople.org/kickstarts/Fedora-Minimal-AArch64.ks" > > It almost worked. I already had qemu-kvm set up for a windows partition for > my vehicle obdII software probably with group install virtualization, then > I > installed qemu-system-arm and qemu-system-aarch64 along with edk2-aarch64 > libguestfs-tools-c > > It bombed out at (I have the full install log including the backtrace but I > didn't want to spam the list with it.) It appeared to be going okay for a > while. It is using the 4.8.6-300 kernel, is there an SMP bug in it? Or any > suggestions? Unfortunately you need allocate more ram, I generally use 4096, but you might be able to get by with a little less. Using 2048 I can reproduce the same result. The wiki does use 4096 in the example, but I'll make a note of the crash as well. --- Thanks! I will try that! It stopped exactly in the same place the second time I tried. I added --vcpus 1. When I tried through the Virt-manager gui and tried an http install it got quite a bit further. It installed 312/485, then a similar error, then continued and finally died at 408/485 with a server install, I wasn't sure how to add the anaconda file so it was more packages. I figured it created a swapfile if it needed more ram. It was the same settings I gave win10 and what I typically give to linux x86 guests.

> We lose some ram on aarch64 due to cma allocation, its something that is > actively being looked at. Anaconda also needs a fair amount of ram during > the network install to copy files. Once the install is complete, you should > be able to reduce the ram to 2048. 2G not being enough is pretty excessive. I never had such problems on x86. Only recently (a year ago or so) I started using 2G for guests, 1G used to be enough even for network installs.

I suspect a big factor here is CONFIG_ARM64_64K_PAGES=y. Raspberry pi 3, with standard fedora kernel (64k pages), minimal install (@core), booted to the shell prompt. "free" reports ~122MB of memory as used. Same with a self-compiled kernel (4k pages), I get ~44MB of used memory. That is almost factor three!

> I suspect a big factor here is CONFIG_ARM64_64K_PAGES=y. Raspberry pi > 3, with standard fedora kernel (64k pages), minimal install (@core), > booted to the shell prompt. "free" reports ~122MB of memory as used. > Same with a self-compiled kernel (4k pages), I get ~44MB of used memory. > That is almost factor three! That is almost the complete factor. It's in fact a combination of 64K pages and CMA. The two together mean that the minimum CMA allocated is 512Mb of RAM. It's a known problem and one we're looking at different ways/options to resolve and working with upstream to do so and reconsidering 64K pages in Fedora. We've traditionally gone that route due to large enterprise systems and to be in sync with downstreams like RHELSA and CentOS in that regard.

What was the reason to go 64k pages in the first place?

Sure, with larger pages memory management overhead goes down. But on the other hand the memory footprint goes up, and frankly I'm a bit surprised how much it goes up.

So I'm wondering whenever 64k pages is a net win even on enterprise machines. Did people benchmark this?

> pressure on them to clean this up. RHEL (and Cent) will use 64K no > matter what, but there could be a (short term) case for Fedora having > a cycle or two with a smaller size - I would prefer to avoid that. I think it would be more useful to offer a 4k kernel in addition to the 64k one, similar to how Fedora ships kernel with and without lpae support on armv7.

W dniu 16.02.2017 o 18:59, Gerd Hoffmann pisze: > I think it would be more useful to offer a 4k kernel in addition to > the 64k one, similar to how Fedora ships kernel with and without > lpae support on armv7. Some applications still check PAGE_SIZE only during compilation instead of on runtime. Btrfs was also hardcoding it.