Hi Thomas,
>> > I have looked into the CMA setting issue a bit. This is what I have found so far.
>> >
>> > The rpi4 needs CMA to be in ZONE_DMA (lower 1GB of memory) as this is the only area that the peripherals on the rpi4 can address.
>> >
>> > The DT sets the allowed range to allocate the CMA from (arch/arm/boot/dts/bcm2711.dtsi#L869), but it seems to not work here. What does work is instead to set the offset manually. I replaced "cma=256MB" with "cma=256M@704M" and then it boots. Note that it has to be 256M instead of 256MB.
>>
>> Right, because of this it may be able to be set in config.txt, I seem
>> to remember seeing this somewhere but as we don't support accelerated
>> graphics on the RPi4 I've not looked. I don't believe the
>> unaccelerated graphics uses the CMA so for the current situation you
>> may be able to drop it.
>>
>> If it's an option to set it in config.txt we need to work out if this
>> is a general option that works for all the rpi models or if it's
>> explicitly for the RPi4, if the later we really need to report and get
>> the bug fixed because we aim to produce generic images which "just
>> work" across all the rpi devices, anything else just makes it a
>> support nightmare for people like myself that attempt to support it
>> and it would be less work just not to support the RPi4 at all TBH.
>>
>> > Removing the cma option on the command line was known as a workaround. Without that we would fall back to the build config of 64MB cma which was located at offset 0x38000000. This left 64MB at the end of ZONE_DMA, and I chose offset 704M so that those 64MB would still be free. Not sure if that is needed or not. The crashkernel needs to be in ZONE_DMA as well but it seems to be set to 0 size.
>> >
>> > I have tested on 5.7 rc2 from rawhide.
>> >
>> > This probably belongs in a bug report. What would be the correct place to file that? From what I can tell upstream has been tested with cma settings without problems (as long as the requested CMA size can fit in ZONE_DMA). From that it seems like fedora-specific issue. Not sure though.
>>
>> Not sure what you mean by "upstream" here, we use an almost pure
>> upstream Linus kernel, if you mean the "Raspberry Pi Foundation" and
>> their kernel, that's a downstream fork of Linus's kernel. They also
>> have a lot of other patches and use a different desktop, GNOME from
>> experience and working with them then RPi upstream GPU maintainer we
>> worked out GNOME needed the 256Mb allocation but desktops like XFCE
>> use a lot less (~192Mb if memory serves) and Raspbian uses a light
>> desktop so I suspect they allocate a lot less.
>
>
> I meant upstream as in linus tree. I noticed some comments in the review of rpi4 ustreaming patches where various cma sizes were tested. Thus I suspected that it could be related to downstream patches. If we do not carry any relevant then perhaps the issue could related to setting the cma both in config.txt and on the commandline?
>
> Raspbian uses 256MB via kernel commandline and the config.txt in /boot does not have any setting for cma. The cma starts at 0x1ec00000 so in the lower 1gb. But it is a 4.19 kernel + patches so not really useful to look at for this specific issue.
Just as a follow up to this it looks like Raspbian has just (finally?)
rebased to the 5.4 LTS kernel [1] in their downstream kernel and
looking through the change they've done what I thought would be needed
and provided a means of dealing with CMA via dt-overlays, I wasn't
sure whether they would have done it via a config.txt cma=XX or a
dt-overlay option, I've literally just looked at the diff but it looks
like for F-33 we can investigate dealing with this in the config.txt
rather than kernel command line.
I've literally just looked at the diff here and won't have time to
investigate this until later this month, which actually is probably
just fine to give a few more firmware releases to settle the rebase
down, but if someone wants to start looking further do reach out.
That is quite a commit to dive into :) I will take a look at it.
For info here is what I have found since my last email:
I attached the serial console to check what happens when we hang at boot with the cma setting. We allocate the 256 MB at 0xEC000000. That again leaves 64MB at the end but at the end of the 4GB this time.
With [3] the logic is that users should have full control of the cma allocation when using the setting on the commandline. The idea makes sense but it is unfortunate that specifying only a cma size also leads to ignoring the valid allocation range specified in dt for rpi4. The commit was introduced in 5.6 and thus is not in the raspberrypi kernel. I guess this will still be a problem then if we continue to set the cma on the commandline?
Peter
[1] https://github.com/raspberrypi/firmware/commit/7eff9f6774bb43bfd61e749a0b45ffddc98c2311
[2] https://github.com/raspberrypi/firmware/commit/7eff9f6774bb43bfd61e749a0b45ffddc98c2311#diff-32265084aeae5fd34fbaf894f22fb20fR553