On Thu, Sep 30, 2021 at 4:13 PM Gordon Messmer <gordon.messmer@gmail.com> wrote:
>
> On 9/30/21 12:41, Chris Murphy wrote:
> > Seems likely to me some service is not quitting properly, preventing /
> > from being unmounted
>
>
> If that were the case, there might be information about an exit failure
> in the log.  On the next boot, "journalctl -b -1" might have useful info.
>
> I also wonder if enabling sysrq and using sysrq+'t' would help determine
> which process is stuck:

It might but it's also really verbose, so for starters "systemctl
list-jobs" is the path of least resistance I suspect. Report suggests
the systemd cylon eye, which suggests a job is running and not exiting
as you say, so list the jobs and then do a systemctl status on each
one of them to find out what they are up to.

Well, I finally enabled the shell, and it (as it is intermittent) hung up on shutdown.

lsof showed nothing that I could see.  

The stop job rotates between four mounted volumes, and the lvm2-monitor.service.  
(The lvm2.monitor finally stopped, the volumes have not) 

systemctl status doesn't seem to display much, but the stop job (and cylon eye) keep overwriting everything, making it really hard to see the output.

Perhaps let me let everyone off the hook though.  I was thinking about it, and started digging into the logs.  This system has been upgraded up from at least Fedora 30, my dnf logs start in 2019-06.  (It may even predate that, not sure)

So, as far as dragging people through debugging seems somewhat futile to me, unless collectively you think that it is worthwhile to pursue this.  I'm game if this can help the Fedora community, but the system is working,  I can back up, and do a fresh install after a long time of good service after so many upgrades.  (I've done all of them, and even switched to the beta track when it got close a few times)