On 21 Aug 2020, at 07:34, Peter Robinson <pbrobinson(a)gmail.com&gt; wrote: Hi Tim, > It's been a while since I've spun up fedoraiot, so I thought I'd see where we'd got to, as I've now got a concrete usecase monitoring some solar panels. Unfortunately, when I put Fedora-IoT-32-20200603.0.aarch64.raw.xz onto an SDCard, the pi (3b+) boots and gets to a login prompt, but I cannot login; neither as `root` with no password on the console, nor ssh'ing as `root` after using the Zezere portal. I'm pretty sure that I'm using the right key (although the documentation is a bit light on what a good key content looks like). The `/portal/devices/` page shows that there's an action to upload a key, with a `cancel runrequest` button, but no indication that more is happening. > > Any hints or tips / troubleshooting approaches? There's details in the docs for key provisioning: https://docs.fedoraproject.org/en-US/iot/ignition/ _______________________________________________ IoT mailing list -- iot(a)lists.fedoraproject.org To unsubscribe send an email to iot-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/iot@lists.fedoraproject.org

Hullo I’m sure that I’m being a bit dim here and that I’ve seen this working. However. Starting with a downloaded image for a Pi (3b+), after a `rpm-ostree upgrade`, I seem unable to deploy layered packages. They seem to install, but after the reboot, they’re not there. As an example, I tried to drop `toolbox` onto the image so that I could build some of my own rpms to run under systemd, using the approach here: https://bit.ly/3b8Ivri. The initial status before installation looks like this: ``` rpm-ostree status State: idle Deployments: * ostree://fedora-iot:fedora/stable/aarch64/iot Version: 32.20200817.0 (2020-08-17T15:28:18Z) Commit: ace3685f47d39f5d4b6065d49554cf9ada92da685541640298b53576895176ea GPGSignature: Valid signature by 97A1AE57C3A2372CCA3A4ABA6C13026D12C944D0 ostree://fedora-iot:fedora/stable/aarch64/iot Version: 32.20200603.0 (2020-06-03T10:45:43Z) Commit: c02bd26925b4e849fd0e53f3645e97b5cb22f47d7614c5a047d6200c64b3421b GPGSignature: Valid signature by 7D22D5867F2A4236474BF7B850CB390B3C3359C4 ``` `sudo rpm-ostree install toolbox` progresses as I’d expect, creating a new bootable deployment: ``` [tim@localhost ~]$ sudo rpm-ostree install toolbox Checking out tree ace3685... done Enabled rpm-md repositories: fedora-cisco-openh264 updates fedora fedora-modular updates-modular rpm-md repo 'fedora-cisco-openh264' (cached); generated: 2020-03-17T20:10:43Z rpm-md repo 'updates' (cached); generated: 2020-08-27T14:18:53Z rpm-md repo 'fedora' (cached); generated: 2020-04-22T22:22:16Z rpm-md repo 'fedora-modular' (cached); generated: 2020-04-22T20:58:40Z rpm-md repo 'updates-modular' (cached); generated: 2020-08-27T14:52:01Z Importing rpm-md... done Resolving dependencies... done Checking out packages... done Running pre scripts... done Running post scripts... done Running posttrans scripts... done Writing rpmdb... done Writing OSTree commit... done ⠉ Staging deployment... Staging deployment... done Freed: 354.4?MB (pkgcache branches: 217) Added: flatpak-session-helper-1.6.5-1.fc32.aarch64 toolbox-0.0.93-1.fc32.aarch64 Run "systemctl reboot" to start a reboot ``` and the status looks ok (ignoring the niggling error): ``` [tim@localhost ~]$ rpm-ostree status error: Timeout was reached [tim@localhost ~]$ rpm-ostree status State: idle Deployments: ostree://fedora-iot:fedora/stable/aarch64/iot Version: 32.20200817.0 (2020-08-17T15:28:18Z) BaseCommit: ace3685f47d39f5d4b6065d49554cf9ada92da685541640298b53576895176ea GPGSignature: Valid signature by 97A1AE57C3A2372CCA3A4ABA6C13026D12C944D0 Diff: 2 added LayeredPackages: toolbox * ostree://fedora-iot:fedora/stable/aarch64/iot Version: 32.20200817.0 (2020-08-17T15:28:18Z) Commit: ace3685f47d39f5d4b6065d49554cf9ada92da685541640298b53576895176ea GPGSignature: Valid signature by 97A1AE57C3A2372CCA3A4ABA6C13026D12C944D0 ostree://fedora-iot:fedora/stable/aarch64/iot Version: 32.20200603.0 (2020-06-03T10:45:43Z) Commit: c02bd26925b4e849fd0e53f3645e97b5cb22f47d7614c5a047d6200c64b3421b GPGSignature: Valid signature by 7D22D5867F2A4236474BF7B850CB390B3C3359C4 ``` But when I reboot, that deployment isn’t the image used and it’s disappeared from rpm-ostree’s world: ``` [tim@localhost ~]$ sudo rpm-ostree status State: idle Deployments: * ostree://fedora-iot:fedora/stable/aarch64/iot Version: 32.20200817.0 (2020-08-17T15:28:18Z) Commit: ace3685f47d39f5d4b6065d49554cf9ada92da685541640298b53576895176ea GPGSignature: Valid signature by 97A1AE57C3A2372CCA3A4ABA6C13026D12C944D0 ostree://fedora-iot:fedora/stable/aarch64/iot Version: 32.20200603.0 (2020-06-03T10:45:43Z) Commit: c02bd26925b4e849fd0e53f3645e97b5cb22f47d7614c5a047d6200c64b3421b GPGSignature: Valid signature by 7D22D5867F2A4236474BF7B850CB390B3C3359C4 ``` Did I forget how this type of layered installation should work?

On 29 Aug 2020, at 12:34, TimC <tim+fedoraproject.org(a)coote.org&gt; wrote: Hi Dusty Is this a likely culprit? ``` -- Reboot -- Aug 28 20:56:24 localhost.localdomain systemd[1]: Finished OSTree Finalize Staged Deployment. Aug 28 21:07:23 localhost.localdomain systemd[1]: Stopping OSTree Finalize Staged Deployment... Aug 28 21:07:23 localhost.localdomain ostree[14561]: Finalizing staged deployment Aug 28 21:07:32 localhost.localdomain ostree[14561]: Copying /etc changes: 10 modified, 0 removed, 34 added Aug 28 21:07:32 localhost.localdomain ostree[14561]: Copying /etc changes: 10 modified, 0 removed, 34 added Aug 28 21:08:30 localhost.localdomain ostree[14561]: ** Aug 28 21:08:30 localhost.localdomain ostree[14561]: OSTree:ERROR:src/libostree/ostree-sysroot-deploy.c:1775:install_deployment_kernel: assertion failed (kernel_layout->bootcsum == bootcsum): ("02ec07fa832b4485887f6dbac19d0311e3> Aug 28 21:08:30 localhost.localdomain ostree[14561]: Bail out! OSTree:ERROR:src/libostree/ostree-sysroot-deploy.c:1775:install_deployment_kernel: assertion failed (kernel_layout->bootcsum == bootcsum): ("02ec07fa832b4485887f6dba> Aug 28 21:08:30 localhost.localdomain systemd[1]: ostree-finalize-staged.service: Control process exited, code=dumped, status=6/ABRT Aug 28 21:08:30 localhost.localdomain systemd[1]: ostree-finalize-staged.service: Failed with result 'core-dump'. Aug 28 21:08:30 localhost.localdomain systemd[1]: Stopped OSTree Finalize Staged Deployment. Aug 28 21:08:30 localhost.localdomain systemd[1]: ostree-finalize-staged.service: Consumed 8.211s CPU time. ``` Note that this is from yesterday’s fail. When I tried again today, I only get this (before reboot): ``` -- Reboot -- Aug 29 11:12:12 localhost.localdomain systemd[1]: Finished OSTree Finalize Staged Deployment. ``` When I run `sudo ostree admin finalize-staged`, it fails, thus: ``` sudo ostree admin finalize-staged Copying /etc changes: 10 modified, 0 removed, 34 added ** OSTree:ERROR:src/libostree/ostree-sysroot-deploy.c:1775:install_deployment_kernel: assertion failed (kernel_layout->bootcsum == bootcsum): ("02ec07fa832b4485887f6dbac19d0311e3039855cc6ebd57aefaafa4a3997d1e" == "fc8629fdb6f5355dd5359350296b619216cbc32e19887511caf75ee9d6aafbad") Bail out! OSTree:ERROR:src/libostree/ostree-sysroot-deploy.c:1775:install_deployment_kernel: assertion failed (kernel_layout->bootcsum == bootcsum): ("02ec07fa832b4485887f6dbac19d0311e3039855cc6ebd57aefaafa4a3997d1e" == "fc8629fdb6f5355dd5359350296b619216cbc32e19887511caf75ee9d6aafbad") Aborted ``` I’d guess that’s a fail originating somehow in my trying to increase the image and filesystem in the download as my first attempts to layer packages just ran out of space. The approach worked with Fedora-IoT 29 and was based on decompressing, appending 0’s to the end of the file and then resizing the filesystem. Would it make sense to confirm the cause by layering a single package onto the original download, if that’s useful? I think that I will need some mechanism to increase the image size, tho’ or I won’t be able to get many tools onto the pi. Tim > On 29 Aug 2020, at 06:02, Dusty Mabe <dusty(a)dustymabe.com&gt; wrote: > > > > On 8/28/20 5:43 PM, TimC wrote: >> Hullo >> >> I’m sure that I’m being a bit dim here and that I’ve seen this working. However. >> >> Starting with a downloaded image for a Pi (3b+), after a `rpm-ostree upgrade`, I seem unable to deploy layered packages. They seem to install, but after the reboot, they’re not there. As an example, I tried to drop `toolbox` onto the image so that I could build some of my own rpms to run under systemd, using the approach here: https://bit.ly/3b8Ivri. The initial status before installation looks like this: >> ``` >> rpm-ostree status >> State: idle >> Deployments: >> * ostree://fedora-iot:fedora/stable/aarch64/iot >> Version: 32.20200817.0 (2020-08-17T15:28:18Z) >> Commit: ace3685f47d39f5d4b6065d49554cf9ada92da685541640298b53576895176ea >> GPGSignature: Valid signature by 97A1AE57C3A2372CCA3A4ABA6C13026D12C944D0 >> >> ostree://fedora-iot:fedora/stable/aarch64/iot >> Version: 32.20200603.0 (2020-06-03T10:45:43Z) >> Commit: c02bd26925b4e849fd0e53f3645e97b5cb22f47d7614c5a047d6200c64b3421b >> GPGSignature: Valid signature by 7D22D5867F2A4236474BF7B850CB390B3C3359C4 >> ``` >> >> `sudo rpm-ostree install toolbox` progresses as I’d expect, creating a new bootable deployment: >> ``` >> [tim@localhost ~]$ sudo rpm-ostree install toolbox >> Checking out tree ace3685... done >> Enabled rpm-md repositories: fedora-cisco-openh264 updates fedora fedora-modular updates-modular >> rpm-md repo 'fedora-cisco-openh264' (cached); generated: 2020-03-17T20:10:43Z >> rpm-md repo 'updates' (cached); generated: 2020-08-27T14:18:53Z >> rpm-md repo 'fedora' (cached); generated: 2020-04-22T22:22:16Z >> rpm-md repo 'fedora-modular' (cached); generated: 2020-04-22T20:58:40Z >> rpm-md repo 'updates-modular' (cached); generated: 2020-08-27T14:52:01Z >> Importing rpm-md... done >> Resolving dependencies... done >> Checking out packages... done >> Running pre scripts... done >> Running post scripts... done >> Running posttrans scripts... done >> Writing rpmdb... done >> Writing OSTree commit... done >> ⠉ Staging deployment... >> Staging deployment... done >> >> Freed: 354.4?MB (pkgcache branches: 217) >> Added: >> flatpak-session-helper-1.6.5-1.fc32.aarch64 >> toolbox-0.0.93-1.fc32.aarch64 >> Run "systemctl reboot" to start a reboot >> >> ``` >> >> and the status looks ok (ignoring the niggling error): >> ``` >> [tim@localhost ~]$ rpm-ostree status >> error: Timeout was reached >> [tim@localhost ~]$ rpm-ostree status >> State: idle >> Deployments: >> ostree://fedora-iot:fedora/stable/aarch64/iot >> Version: 32.20200817.0 (2020-08-17T15:28:18Z) >> BaseCommit: ace3685f47d39f5d4b6065d49554cf9ada92da685541640298b53576895176ea >> GPGSignature: Valid signature by 97A1AE57C3A2372CCA3A4ABA6C13026D12C944D0 >> Diff: 2 added >> LayeredPackages: toolbox >> >> * ostree://fedora-iot:fedora/stable/aarch64/iot >> Version: 32.20200817.0 (2020-08-17T15:28:18Z) >> Commit: ace3685f47d39f5d4b6065d49554cf9ada92da685541640298b53576895176ea >> GPGSignature: Valid signature by 97A1AE57C3A2372CCA3A4ABA6C13026D12C944D0 >> >> ostree://fedora-iot:fedora/stable/aarch64/iot >> Version: 32.20200603.0 (2020-06-03T10:45:43Z) >> Commit: c02bd26925b4e849fd0e53f3645e97b5cb22f47d7614c5a047d6200c64b3421b >> GPGSignature: Valid signature by 7D22D5867F2A4236474BF7B850CB390B3C3359C4 >> ``` >> >> But when I reboot, that deployment isn’t the image used and it’s disappeared from rpm-ostree’s world: >> ``` >> [tim@localhost ~]$ sudo rpm-ostree status >> State: idle >> Deployments: >> * ostree://fedora-iot:fedora/stable/aarch64/iot >> Version: 32.20200817.0 (2020-08-17T15:28:18Z) >> Commit: ace3685f47d39f5d4b6065d49554cf9ada92da685541640298b53576895176ea >> GPGSignature: Valid signature by 97A1AE57C3A2372CCA3A4ABA6C13026D12C944D0 >> >> ostree://fedora-iot:fedora/stable/aarch64/iot >> Version: 32.20200603.0 (2020-06-03T10:45:43Z) >> Commit: c02bd26925b4e849fd0e53f3645e97b5cb22f47d7614c5a047d6200c64b3421b >> GPGSignature: Valid signature by 7D22D5867F2A4236474BF7B850CB390B3C3359C4 >> ``` >> >> Did I forget how this type of layered installation should work? > > > Seems like it should be working. This vaguely sounds like the deployment not getting finalized. > > Does `sudo journalctl -u ostree-finalize-staged.service` show any errors? If it is timing out for > some reason you could try to manually run `sudo ostree admin finalize-staged` *before* you type > reboot to reboot into the new deployment so that you know it finishes. > > Dusty _______________________________________________ IoT mailing list -- iot(a)lists.fedoraproject.org To unsubscribe send an email to iot-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/iot@lists.fedoraproject.org

I’d guess that’s a fail originating somehow in my trying to increase the image and filesystem in the download as my first attempts to layer packages just ran out of space. The approach worked with Fedora-IoT 29 and was based on decompressing, appending 0’s to the end of the file and then resizing the filesystem.