On Wed, Nov 27, 2019 at 6:45 PM Fraser Tweedale <ftweedal(a)redhat.com> wrote:
> Hi Patrick,
>
> I want to follow up with this. Did you get things working again?
>
> With the latest packages for both f30 and f31, I upgraded a FreeIPA
> installation from f30 to f31 without encountering any problems.
>
> Perhaps the jss issue caused the system to enter a poor state during
> the initial ipa-server-upgrade, such that subsequent runs of
> ipa-server-upgrade fail, even after upgrading jss.
>
> Anyone else have thoughts on this? Anyone able to reproduce with
> latest packages?
>
> Thanks,
> Fraser
>
> On Tue, Nov 05, 2019 at 03:25:13AM +0800, Patrick Dung via
> FreeIPA-users wrote:
> > The stack trace of my last mail should be hitting this bug:
> >
https://bugzilla.redhat.com/show_bug.cgi?id=1755634
> >
> > On Fri, Nov 1, 2019 at 3:20 PM Patrick Dung <patdung100(a)gmail.com>
> > wrote:
> >
> > > Hello all, 1) Resent, I used reply instead of reply all in the
> > > last mail.
> > >
> > > 2) I tailed the dirsrv log and perform a manual
> > > ipa-server-upgrade. I didn't found connection refused log in
> > > the dirsrv.
> > >
> > > 3) BTW, I had another ipa server that is a replica. Originally
> > > both freeipa server had upgrade problem. On the replica server,
> > > I tried to install jss-4.6.2-2.fc31.x86_64 (according to
> > >
https://bugzilla.redhat.com/show_bug.cgi?id=1766451) The
> > > ipa-server-upgrade is successfully run on the replica server.
> > > But there is problem when I access:
https://replica:8443, the
> > > error message is shown in below. HTTP Status 500 – Internal
> > > Server Error ------------------------------
> > >
> > > *Type* Exception Report
> > >
> > > *Message* org.apache.jasper.JasperException: Unable to compile
> > > class for JSP
> > >
> > > *Description* The server encountered an unexpected condition
> > > that prevented it from fulfilling the request.
> > >
> > > *Exception*
> > >
> > > org.apache.jasper.JasperException:
> > > org.apache.jasper.JasperException: Unable to compile class for
> > > JSP
> > >
>
org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:604)
> > >
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:422)
> > >
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:385)
> > > org.apache.jasper.servlet.JspServlet.service(JspServlet.java:329)
> > > javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
> > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > java.lang.reflect.Method.invoke(Method.java:498)
> > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
> > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
> > > java.security.AccessController.doPrivileged(Native Method)
> > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> > >
> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
> > >
> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:170)
> > > java.security.AccessController.doPrivileged(Native Method)
> > > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
> > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > java.lang.reflect.Method.invoke(Method.java:498)
> > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
> > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
> > > java.security.AccessController.doPrivileged(Native Method)
> > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> > >
> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
> > >
> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
> > >
> > > *Root Cause*
> > >
> > > org.apache.jasper.JasperException: Unable to compile class for
> > > JSP
> > >
> org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:619)
> > >
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:399)
> > >
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:385)
> > > org.apache.jasper.servlet.JspServlet.service(JspServlet.java:329)
> > > javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
> > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > java.lang.reflect.Method.invoke(Method.java:498)
> > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
> > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
> > > java.security.AccessController.doPrivileged(Native Method)
> > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> > >
> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
> > >
> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:170)
> > > java.security.AccessController.doPrivileged(Native Method)
> > > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
> > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > java.lang.reflect.Method.invoke(Method.java:498)
> > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
> > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
> > > java.security.AccessController.doPrivileged(Native Method)
> > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> > >
> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
> > >
> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
> > >
> > > *Root Cause*
> > >
> > > java.security.AccessControlException: access denied
> > > ("java.util.PropertyPermission"
> > > "tolerateIllegalAmbiguousVarargsInvocation" "read")
> > >
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> > >
> java.security.AccessController.checkPermission(AccessController.java:886)
> > > java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
> > >
> java.lang.SecurityManager.checkPropertyAccess(SecurityManager.java:1294)
> > > java.lang.System.getProperty(System.java:717)
> > >
>
org.eclipse.jdt.internal.compiler.impl.CompilerOptions.<init>(CompilerOptions.java:513)
> > >
> org.apache.jasper.compiler.JDTCompiler.generateClass(JDTCompiler.java:483)
> > > org.apache.jasper.compiler.Compiler.compile(Compiler.java:392)
> > > org.apache.jasper.compiler.Compiler.compile(Compiler.java:362)
> > > org.apache.jasper.compiler.Compiler.compile(Compiler.java:346)
> > >
> org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:603)
> > >
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:399)
> > >
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:385)
> > > org.apache.jasper.servlet.JspServlet.service(JspServlet.java:329)
> > > javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
> > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > java.lang.reflect.Method.invoke(Method.java:498)
> > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
> > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
> > > java.security.AccessController.doPrivileged(Native Method)
> > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> > >
> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
> > >
> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:170)
> > > java.security.AccessController.doPrivileged(Native Method)
> > > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
> > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > java.lang.reflect.Method.invoke(Method.java:498)
> > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
> > > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
> > > java.security.AccessController.doPrivileged(Native Method)
> > > javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> > >
> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
> > >
> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
> > >
> > > *Note* The full stack trace of the root cause is available in
> > > the server logs.
> > >
> > >
> > > 4) On the main server, I tried to upgrade jss and run
> > > ipa-server-upgrade but the error is still there. The error is
> > > the same as the original mail a few days ago.
> > >
> > > Thanks,
> > >
> > > Patrick
> > >
> > > On Thu, Oct 31, 2019 at 12:25 PM Patrick Dung
> > > <patdung100(a)gmail.com> wrote:
> > >
> > >> I tail the dirsrv log and perform a manual ipa-server-upgrade.
> > >> I didn't found connection refused log in the dirsrv.
> > >>
> > >> Thanks, Patrick
> > >>
> > >> On Thu, Oct 31, 2019 at 7:43 AM Fraser Tweedale
> > >> <ftweedal(a)redhat.com> wrote:
> > >>
> > >>> Is there anything in the dirsrv log relating to the connection
> > >>> attempt? Connection Refused could in fact be a TLS handshake
> > >>> error (the TLS handshake also includes certificate
> > >>> authentication).
> > >>>
> > >>> Cheers, Fraser
> > >>>
> > >>> On Wed, Oct 30, 2019 at 10:47:54PM +0800, Patrick Dung via
> > >>> FreeIPA-users wrote:
> > >>> > Hello Endi,
> > >>> >
> > >>> > The DS is up.
> > >>> >
> > >>> > $ ldapsearch -LLL -x -H ldaps://home.local.nonet:636 -b
> > >>> > "cn=users,cn=accounts,dc=local,dc=nonet" dn:
> > >>> > cn=users,cn=accounts,dc=local,dc=nonet objectClass: top
> > >>> > objectClass: nsContainer cn: users
> > >>> >
> > >>> > dn: uid=admin,cn=users,cn=accounts,dc=local,dc=nonet
> > >>> > objectClass: top objectClass: person objectClass:
> > >>> > posixaccount objectClass: krbprincipalaux objectClass:
> > >>> > krbticketpolicyaux objectClass: inetuser objectClass:
> > >>> > ipaobject objectClass: ipasshuser objectClass:
> > >>> > ipaSshGroupOfPubKeys uid: admin cn: Administrator sn:
> > >>> > Administrator uidNumber: 700000 gidNumber: 700000
> > >>> > homeDirectory: /home/admin loginShell: /bin/bash gecos:
> > >>> > Administrator
> > >>> >
> > >>> > I had include more logs as attachment.
> > >>> >
> > >>> > Thanks, Patrick
> > >>> >
> > >>> > On Wed, Oct 30, 2019 at 10:23 PM Endi Sukma Dewata
> > >>> > <edewata(a)redhat.com
> > >>> >
> > >>> > wrote:
> > >>> >
> > >>> > > Hi Patrick,
> > >>> > >
> > >>> > > I see that you included the CA debug log:
> > >>> > >
> > >>> > > 2019-10-30 05:03:50 [main] FINE: LdapBoundConnection:
> > >>> > > Connecting to home.local.nonet:636 with client cert
auth
> > >>> > > 2019-10-30 05:03:50 [main] FINE:
> > >>> ldapconn/PKISocketFactory.makeSSLSocket:
> > >>> > > begins 2019-10-30 05:03:50 [main] SEVERE: Unable to
create
> > >>> > > socket: java.net.ConnectException: Connection refused
> > >>> > > (Connection refused) java.net.ConnectException:
Connection
> > >>> > > refused (Connection refused) at
> > >>> > > java.net.PlainSocketImpl.socketConnect(Native Method)
at
> > >>> > >
java.net
> > >>> > >
> .AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> > >>> > > at
java.net
> > >>> > >
> > >>>
> .AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> > >>> > > at
java.net
> > >>> > >
> .AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> > >>> > > at
> > >>> > >
> java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> > >>> > > at java.net.Socket.connect(Socket.java:607) at
> > >>> > > java.net.Socket.connect(Socket.java:556) at
> > >>> > > java.net.Socket.<init>(Socket.java:452)
at
> > >>> > > java.net.Socket.<init>(Socket.java:262)
> > >>> > >
> > >>> > > The stack trace above is incomplete so it's hard to
tell
> > >>> > > exactly
> > >>> where
> > >>> > > in PKI code the exception happened, but the earlier
> > >>> > > message seem to indicate that it's trying to connect
to
> > >>> > > the the DS, so you need to
> > >>> make
> > >>> > > sure the DS is running and accessible.
> > >>> > >
> > >>> > > -- Endi S. Dewata
> > >>> > >
> > >>> > > ----- Original Message -----
> > >>> > > > Hi Patrick,
> > >>> > > >
> > >>> > > > The "javax.ws.rs.ServiceUnavailableException:
Subsystem
> > >>> unavailable"
> > >>> > > > suggests that the CA subsystem got undeployed
> > >>> > > > automatically from Tomcat due to an error in CA
(the
> > >>> > > > Tomcat itself seems to be running just fine).
> > >>> > > >
> > >>> > > > You'll need to check the CA debug log in
> > >>> /var/log/pki/pki-tomcat/ca/*,
> > >>> > > > hopefully it will show the actual problem. Once
it's
> > >>> > > > fixed, you
> > >>> just
> > >>> > > > need to restart Tomcat and the CA should be
redeployed
> > >>> automatically.
> > >>> > > >
> > >>> > > > Also check the upgrade log in
> > >>> > > > /var/log/pki/pki-server-upgrade-* to
> > >>> see
> > >>> > > > if there's any PKI upgrade issue.
> > >>> > > >
> > >>> > > > Hope this helps.
> > >>> > > >
> > >>> > > > -- Endi S. Dewata
> > >>> > > >
> > >>> > > > ----- Original Message -----
> > >>> > > > > Looks like it's the second problem (on
pagure)
> > >>> > > > >
> > >>> > > > > -- Logs begin at Wed 2019-10-30 02:34:10 HKT,
end at
> > >>> > > > > Wed
> > >>> 2019-10-30
> > >>> > > > > 06:28:21 HKT. -- Oct 30 03:39:43
home.local.nonet
> > >>> > > > > systemd[1]: Starting PKI Tomcat
> > >>> Server
> > >>> > > > > pki-tomcat... Oct 30 03:39:44
home.local.nonet
> > >>> > > > > pki-server[57211]:
---------------------------- Oct 30
> > >>> > > > > 03:39:44 home.local.nonet pki-server[57211]:
> > >>> > > > > pki-tomcat
> > >>> instance
> > >>> > > > > migrated Oct 30 03:39:44 home.local.nonet
> > >>> > > > > pki-server[57211]:
---------------------------- Oct 30
> > >>> > > > > 03:39:44 home.local.nonet systemd[1]: Started
PKI
> > >>> > > > > Tomcat
> > >>> Server
> > >>> > > > > pki-tomcat. Oct 30 03:39:44 home.local.nonet
> > >>> > > > > server[57330]: Java virtual
> > >>> machine
> > >>> > > used:
> > >>> > > > > /usr/lib/jvm/jre-1.8.0-openjdk/bin/java Oct
30
> > >>> > > > > 03:39:44 home.local.nonet server[57330]:
classpath
> > >>> > > > > used:
> > >>> > > > >
> > >>> > >
> > >>>
>
/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/lib/java/commons-daemon.jar
> > >>> > > > > Oct 30 03:39:44 home.local.nonet
server[57330]: main
> > >>> > > > > class used:
org.apache.catalina.startup.Bootstrap Oct
> > >>> > > > > 30 03:39:44 home.local.nonet server[57330]:
flags
> > >>> > > > > used:
-Djava.library.path=/usr/lib64/nuxwdog-jni Oct
> > >>> > > > > 30 03:39:44 home.local.nonet server[57330]:
options
> > >>> > > > > used: -Dcatalina.base=/var/lib/pki/pki-tomcat
> > >>> > > -Dcatalina.home=/usr/share/tomcat
> > >>> > > > > -Djava.endorsed.dirs=
> > >>> -Djava.io.tmpdir=/var/lib/pki/pki-tomcat/temp
> > >>> > > > >
> > >>> > >
> > >>>
> -Djava.util.logging.config.file=/var/lib/pki/pki-tomcat/conf/logging.properties
> > >>> > > > >
> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> > >>> > > > > -Djava.security.manager
> > >>> > > > >
> > >>>
-Djava.security.policy==/var/lib/pki/pki-tomcat/conf/catalina.policy
> > >>> > > > > Oct 30 03:39:44 home.local.nonet
server[57330]:
> > >>> > > > > arguments used:
> > >>> start
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]:
> > >>> > > > > WARNING:
> > >>> Exception
> > >>> > > > > processing realm
> > >>> > > > > [com.netscape.cms.tomcat.ProxyRealm@296c31c9]
> > >>> > > > > background
> > >>> process
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]:
> > >>> > > > > javax.ws.rs.ServiceUnavailableException:
Subsystem
> > >>> > > > > unavailable Oct 30 03:40:03 home.local.nonet
> > >>> > > > > server[57330]: at
> > >>> > > > >
> > >>> > >
> > >>>
> com.netscape.cms.tomcat.ProxyRealm.backgroundProcess(ProxyRealm.java:142)
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]: at
> > >>> > > > >
> > >>> > >
> > >>>
> org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1137)
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]: at
> > >>> > > > >
> > >>> > >
> > >>>
>
org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5566)
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]: at
> > >>> > > > >
> > >>> > >
> > >>>
>
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1353)
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]: at
> > >>> > > > >
> > >>> > >
> > >>>
>
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1357)
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]: at
> > >>> > > > >
> > >>> > >
> > >>>
>
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1335)
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]: at
> > >>> > > > >
> > >>>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]: at
> > >>> > > > >
> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]: at
> > >>> > > > >
> > >>> > >
> > >>>
>
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]: at
> > >>> > > > >
> > >>> > >
> > >>>
>
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]: at
> > >>> > > > >
> > >>> > >
> > >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]: at
> > >>> > > > >
> > >>> > >
> > >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]: at
> > >>> > > > >
> > >>> > >
> > >>>
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> > >>> > > > > Oct 30 03:40:03 home.local.nonet
server[57330]: at
> > >>> > > > > java.lang.Thread.run(Thread.java:748)
> > >>> > > > >
> > >>> > > > > The pki-tomcat instance is running. It would
output
> > >>> > > > > error when I
> > >>> browse
> > >>> > > > >
https://home.local.nonet:8443/ca , but
> > >>> > >
https://home.local.nonet:8443/pki/
> > >>> > > > > is ok Instance ID: pki-tomcat Active: True
Unsecure
> > >>> > > > > Port: 8080 Secure Port: 8443 AJP Port: 8009
Tomcat
> > >>> > > > > Port: 8005
> > >>> > > > >
> > >>> > > > > CA Subsystem: Type: Subordinate CA (Security
Domain)
> > >>> > > > > SD Registration URL:
https://home.local.nonet:443
> > >>> > > > > Enabled: True Unsecure URL:
> > >>> > > > >
http://home.local.nonet:8080/ca/ee/ca Secure
Agent
> > >>> > > > > URL:
https://home.local.nonet:8443/ca/agent/ca
Secure
> > >>> > > > > EE URL:
https://home.local.nonet:8443/ca/ee/ca
Secure
> > >>> > > > > Admin URL:
https://home.local.nonet:8443/ca/services
> > >>> > > > > PKI Console URL:
https://home.local.nonet:8443/ca
> > >>> > > > >
> > >>> > > > > Thanks, Patrick
> > >>> > > > >
> > >>> > > > > On Wed, Oct 30, 2019 at 5:44 AM Alex Scheel
<
> > >>> > > > > ascheel(a)redhat.com
> > >>> >
> > >>> > > wrote:
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > You might try checking journalctl output.
> > >>> > > > >
> > >>> > > > > It might be this bug:
> > >>> > >
https://bugzilla.redhat.com/show_bug.cgi?id=1766451
> > >>> > > > >
> > >>> > > > > Otherwise, it is a perfect match for this
bug:
> > >>> > > > >
https://pagure.io/dogtagpki/issue/3111
> > >>> > > > >
> > >>> > > > > Which I'd also like journalctl output on,
if you have
> > >>> > > > > any to
> > >>> share. :)
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > I should have a Bodhi update out tonight yet
for the
> > >>> > > > > issue in
> > >>> the BZ.
> > >>> > > > > Without more information, I'm not sure
we'd know what
> > >>> > > > > cause for the
> > >>> second
> > >>> > > issue
> > >>> > > > > is.
> > >>> > > > >
> > >>> > > > > - Alex
> > >>> > > > >
> > >>> > > > > ----- Original Message -----
> > >>> > > > > > From: "Patrick Dung via
FreeIPA-users" <
> > >>> > > > > > freeipa-users(a)lists.fedorahosted.org >
To:
> > >>> > > > > > freeipa-users(a)lists.fedorahosted.org Cc:
"Patrick
> > >>> > > > > > Dung" < patdung100(a)gmail.com >
Sent: Tuesday,
> > >>> > > > > > October 29, 2019 5:29:09 PM Subject:
[Freeipa-users]
> > >>> > > > > > FreeIPA having problem after
> > >>> upgrading from
> > >>> > > > > > Fedora 30 to 31
> > >>> > > > > >
> > >>> > > > > > Hello,
> > >>> > > > > >
> > >>> > > > > > I got problem upgrading from FC30 to
FC31. Before
> > >>> > > > > > upgrade the FreeIPA in FC30 is running
fine.
> > >>> > > > > >
> > >>> > > > > > After OS upgrade, IPA cannot start and
checked that
> > >>> > > > > > it stuck
> > >>> at the
> > >>> > > CA
> > >>> > > > > > part. I run ipa-server-upgrade manually
but there
> > >>> > > > > > is problem.
> > >>> > > > > >
> > >>> > > > > > 2019-10-29T21:03:58Z DEBUG request GET
> > >>> > > > > >
https://home.local.nonet:8443/ca/rest/account/login
> > >>> > > > > > 2019-10-29T21:03:58Z DEBUG request body
''
> > >>> > > > > > 2019-10-29T21:03:58Z DEBUG response
status 500
> > >>> > > > > > 2019-10-29T21:03:58Z DEBUG response
headers
> > >>> > > > > > Content-Type: text/html;charset=utf-8
> > >>> > > > > > Content-Language: en Content-Length: 2481
Date: Tue,
> > >>> > > > > > 29 Oct 2019 21:03:58 GMT Connection:
close
> > >>> > > > > >
> > >>> > > > > >
> > >>> > > > > > 2019-10-29T21:03:58Z DEBUG response body
(decoded):
> > >>> > > > > > b'<!doctype html><html
lang="en"><head><title>HTTP
> > >>> > > > > > Status 500 \xe2\x80\x93 Internal
> > >>> Server
> > >>> > > > > > Error</title><style
type="text/css">h1
> > >>> > > > > >
> > >>> > >
> > >>>
>
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
> > >>> > > > > > h2
> > >>> > > > > >
> > >>> > >
> > >>>
>
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;}
> > >>> > > > > > h3
> > >>> > > > > >
> > >>> > >
> > >>>
>
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}
> > >>> > > > > > body
> > >>> > > > > >
> > >>> > >
> > >>>
> {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;}
> > >>> > > > > > b
> > >>> > > > > >
> > >>> > >
> > >>>
> {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;}
> > >>> > > > > > p
> > >>> > > > > >
> > >>> > >
> > >>>
> {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}
> > >>> > > > > > a {color:black;} a.name {color:black;}
.line
> > >>> > > > > >
> > >>> > >
> > >>>
>
{height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP
> > >>> > > > > > Status 500 \xe2\x80\x93 Internal Server
> > >>> > > > > > Error</h1><hr
> > >>> class="line"
> > >>> > > > > > /><p><b>Type</b>
Exception
> > >>> > > > > >
Report</p><p><b>Message</b>
> > >>> Subsystem
> > >>> > > > > >
unavailable</p><p><b>Description</b> The server
> > >>> > > > > > encountered an
> > >>> > > unexpected
> > >>> > > > > > condition that prevented it from
fulfilling the
> > >>> > > > > >
request.</p><p><b>Exception</b></p><pre> javax.ws.rs
> > >>> > > > > > .ServiceUnavailableException: Subsystem
> > >>> > > > > >
> > >>> > >
> > >>>
>
unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:150)\n\torg.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:515)\n\tcom.netscape.cms.tomcat.ExternalAuthenticationValve.invoke(ExternalAuthenticationValve.java:82)\n\torg.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)\n\torg.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:678)\n\torg.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)\n\torg.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)\n\torg.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)\n\torg.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860)\n\
> > >>> > > > > >
torg.apache.tomcat.util.net
> > >>> > > > > >
> .NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1589)\n\
> > >>> > > > > >
torg.apache.tomcat.util.net
> > >>> > > > > >
> > >>> > >
> > >>>
>
.SocketProcessorBase.run(SocketProcessorBase.java:49)\n\tjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\torg.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\tjava.lang.Thread.run(Thread.java:748)\n</pre><p><b>Note</b>
> > >>> > > > > > The full stack trace of the root cause is
available
> > >>> > > > > > in the
> > >>> server
> > >>> > > > > > logs.</p><hr
class="line" /><h3>Apache
> > >>> > > Tomcat/9.0.26</h3></body></html>'
> > >>> > > > > > 2019-10-29T21:03:58Z ERROR IPA server
upgrade
> > >>> > > > > > failed: Inspect /var/log/ipaupgrade.log
and run
> > >>> > > > > > command ipa-server-upgrade
> > >>> manually.
> > >>> > > > > > 2019-10-29T21:03:58Z DEBUG File
> > >>> > > > > >
"/usr/lib/python3.7/site-packages/ipapython/admintool.py",
> > >>> line 179,
> > >>> > > in
> > >>> > > > > > execute return_value = self.run() File
> > >>> > > > > >
> > >>> > >
> > >>>
>
"/usr/lib/python3.7/site-packages/ipaserver/install/ipa_server_upgrade.py",
> > >>> > > > > > line 54, in run server.upgrade() File
> > >>> > > > > >
> > >>> > >
> > >>>
> "/usr/lib/python3.7/site-packages/ipaserver/install/server/upgrade.py",
> > >>> > > > > > line 2223, in upgrade
upgrade_configuration() File
> > >>> > > > > >
> > >>> > >
> > >>>
> "/usr/lib/python3.7/site-packages/ipaserver/install/server/upgrade.py",
> > >>> > > > > > line 2093, in upgrade_configuration
> > >>> > > > > > ca_enable_ldap_profile_subsystem(ca)
File
> > >>> > > > > >
> > >>> > >
> > >>>
> "/usr/lib/python3.7/site-packages/ipaserver/install/server/upgrade.py",
> > >>> > > > > > line 414, in
ca_enable_ldap_profile_subsystem
> > >>> > > > > > cainstance.migrate_profiles_to_ldap()
File
> > >>> > >
> "/usr/lib/python3.7/site-packages/ipaserver/install/cainstance.py",
> > >>> > > > > > line 1937, in migrate_profiles_to_ldap
> > >>> > > > > > _create_dogtag_profile(profile_id,
profile_data,
> > >>> overwrite=False)
> > >>> > > > > > File
> > >>> > >
> "/usr/lib/python3.7/site-packages/ipaserver/install/cainstance.py",
> > >>> > > > > > line 1943, in _create_dogtag_profile
with
> > >>> > > > > > api.Backend.ra_certprofile as
profile_api: File
> > >>>
"/usr/lib/python3.7/site-packages/ipaserver/plugins/dogtag.py",
> > >>> > > line
> > >>> > > > > > 1315, in __enter__ raise
> > >>> > > > > >
errors.RemoteRetrieveError(reason=_('Failed to
> > >>> authenticate to
> > >>> > > CA
> > >>> > > > > > REST API'))
> > >>> > > > > >
> > >>> > > > > > 2019-10-29T21:03:58Z DEBUG The
ipa-server-upgrade
> > >>> > > > > > command
> > >>> failed,
> > >>> > > > > > exception: RemoteRetrieveError: Failed
to
> > >>> > > > > > authenticate to CA
> > >>> REST API
> > >>> > > > > > 2019-10-29T21:03:58Z ERROR Unexpected
error - see
> > >>> > > /var/log/ipaupgrade.log
> > >>> > > > > > for details: RemoteRetrieveError: Failed
to
> > >>> > > > > > authenticate to CA REST API
2019-10-29T21:03:58Z
> > >>> > > > > > ERROR The ipa-server-upgrade command
> > >>> failed. See
> > >>> > > > > > /var/log/ipaupgrade.log for more
information
> > >>> > > > > >
> > >>> > > > > > From /var/log/pki/pki-tomcat/ca/debug log
file:
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINE:
LdapAuthInfo:
> > >>> > > > > > init() 2019-10-30 05:03:50 [main] FINE:
> > >>> > > > > > LdapAuthInfo: init begins 2019-10-30
05:03:50 [main]
> > >>> > > > > > FINEST: Getting
> > >>> > > > > >
internaldb.ldapauth.authtype=SslClientAuth
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINE:
LdapAuthInfo: init
> > >>> > > > > > ends 2019-10-30 05:03:50 [main] FINEST:
Property
> > >>> internaldb.errorIfDown
> > >>> > > not
> > >>> > > > > > found 2019-10-30 05:03:50 [main] FINEST:
Getting
> > >>> > > internaldb.errorIfDown=true
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINEST:
Property
> > >>> internaldb.doCloning not
> > >>> > > > > > found 2019-10-30 05:03:50 [main] FINEST:
Getting
> > >>> internaldb.doCloning=true
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINE:
> > >>> > > > > > LdapBoundConnFactory:
> > >>> doCloning:
> > >>> > > true
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINE:
> > >>> > > > > > LdapBoundConnFactory:
> > >>> mininum: 3
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINE:
> > >>> > > > > > LdapBoundConnFactory:
> > >>> maximum: 15
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINE:
> > >>> > > > > > LdapBoundConnFactory: host:
home.local.nonet
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINE:
> > >>> > > > > > LdapBoundConnFactory: port:
> > >>> 636
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINE:
> > >>> > > > > > LdapBoundConnFactory: secure:
> > >>> true
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINE:
> > >>> > > > > > LdapBoundConnFactory:
> > >>> > > authentication: 2
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINE:
> > >>> > > > > > LdapBoundConnFactory:
makeConnection(true)
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINEST:
Getting
> > >>> > > > > >
internaldb.ldapauth.clientCertNickname=subsystemCert
> > >>> cert-pki-ca
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINEST:
Property
> > >>> > > > > > tcp.keepAlive not
> > >>> found
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINEST:
Getting
> > >>> > > > > > tcp.keepAlive=true 2019-10-30 05:03:50
[main] FINE:
> > >>> > > > > > TCP Keep-Alive: true 2019-10-30 05:03:50
[main]
> > >>> > > > > > FINE: LdapBoundConnection:
> > >>> Connecting to
> > >>> > > > > > home.local.nonet:636 with client cert
auth
> > >>> > > > > > 2019-10-30 05:03:50 [main] FINE:
> > >>> > > ldapconn/PKISocketFactory.makeSSLSocket:
> > >>> > > > > > begins 2019-10-30 05:03:50 [main] SEVERE:
Unable to
> > >>> > > > > > create socket: java.net.ConnectException:
Connection
> > >>> > > > > > refused (Connection
> > >>> refused)
> > >>> > > > > > java.net.ConnectException: Connection
refused
> > >>> > > > > > (Connection
> > >>> refused)
> > >>> > > > > > at
java.net.PlainSocketImpl.socketConnect(Native
> > >>> > > > > > Method) at
java.net
> > >>> > > > > >
> > >>>
.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> > >>> > > > > > at
java.net
> > >>> > > > > >
> > >>> > >
> > >>>
> .AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> > >>> > > > > > at
java.net
> > >>> > > > > >
> > >>>
.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> > >>> > > > > > at
> > >>> > > > > >
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> > >>> > > > > > at
java.net.Socket.connect(Socket.java:607) at
> > >>> > > > > > java.net.Socket.connect(Socket.java:556)
at
> > >>> > > > > >
java.net.Socket.<init>(Socket.java:452) at
> > >>> > > > > >
java.net.Socket.<init>(Socket.java:262)
> > >>> > > > > >
> > >>> > > > > > Some error is logged to
/var/log/messages: Oct 30
> > >>> > > > > > 05:26:50 home server[65722]: WARNING:
Exception
> > >>> processing
> > >>> > > realm
> > >>> > > > > >
[com.netscape.cms.tomcat.ProxyRealm@5647a92b]
> > >>> > > > > > background
> > >>> process
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
javax.ws.rs
> > >>> > > > > > .ServiceUnavailableException: Subsystem
> > >>> unavailable
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > >
> > >>> > >
> > >>>
> com.netscape.cms.tomcat.ProxyRealm.backgroundProcess(ProxyRealm.java:142)
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > >
> > >>> > >
> > >>>
> org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1137)
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > >
> > >>> > >
> > >>>
>
org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5566)
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > >
> > >>> > >
> > >>>
>
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1353)
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > >
> > >>> > >
> > >>>
>
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1357)
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > >
> > >>> > >
> > >>>
>
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1335)
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > >
> > >>> > >
> > >>>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > >
> > >>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > >
> > >>> > >
> > >>>
>
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > >
> > >>> > >
> > >>>
>
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > >
> > >>> > >
> > >>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > >
> > >>> > >
> > >>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > >
> > >>> > >
> > >>>
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> > >>> > > > > > Oct 30 05:26:50 home server[65722]:
#011at
> > >>> > > > > > java.lang.Thread.run(Thread.java:748)
> > >>> > > > > >
> > >>> > > > > > I am able to connect to my ldap server
port 636 with
> > >>> > > > > > TLS
> > >>> without
> > >>> > > problem.
> > >>> > > > > >
> > >>> > > > > > Thanks, Patrick
> > >>> > > > > >
> > >>> > > > > >
_______________________________________________
> > >>> > > > > > FreeIPA-users mailing list --
> > >>> freeipa-users(a)lists.fedorahosted.org
> > >>> > > > > > To unsubscribe send an email to
> > >>> > > > > >
freeipa-users-leave(a)lists.fedorahosted.org Fedora
> > >>> > > > > > Code of Conduct:
> > >>> > > > > >
>
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > >>> > > > > > List Guidelines:
> > >>> > >
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > >>> > > > > > List Archives:
> > >>> > > > > >
> > >>> > >
> > >>>
>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
> > >>> > > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > >
_______________________________________________
> > >>> > > > > FreeIPA-users mailing list --
> > >>> freeipa-users(a)lists.fedorahosted.org
> > >>> > > > > To unsubscribe send an email to
> > >>> > > freeipa-users-leave(a)lists.fedorahosted.org
> > >>> > > > > Fedora Code of Conduct:
> > >>> > > > >
>
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > >>> > > > > List Guidelines:
> > >>> > >
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > >>> > > > > List Archives:
> > >>> > > > >
> > >>> > >
> > >>>
>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
> > >>> > > > >
> > >>> > > > _______________________________________________
> > >>> > > > FreeIPA-users mailing list --
> > >>> > > > freeipa-users(a)lists.fedorahosted.org To unsubscribe
send
> > >>> > > > an email to
> > >>> > > freeipa-users-leave(a)lists.fedorahosted.org
> > >>> > > > Fedora Code of Conduct:
> > >>> > > >
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > >>> > > > List Guidelines:
> > >>>
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > >>> > > > List Archives:
> > >>> > > >
> > >>> > >
> > >>>
>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
> > >>> > > >
> > >>> > >
> > >>> > >
> > >>>
> > >>>
> > >>> > _______________________________________________
> > >>> > FreeIPA-users mailing list --
> > >>> > freeipa-users(a)lists.fedorahosted.org To unsubscribe send an
> > >>> > email to
> > >>> freeipa-users-leave(a)lists.fedorahosted.org
> > >>> > Fedora Code of Conduct:
> > >>>
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > >>> > List Guidelines:
> > >>>
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > >>> > List Archives:
> > >>>
>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
> > >>>
> > >>>
>
> > _______________________________________________
> > FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> > To unsubscribe send an email to
> freeipa-users-leave(a)lists.fedorahosted.org
> > Fedora Code of Conduct:
>
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
>
>