7:12 a.m.
Hi all,
I'm trying to set up a replica on CentOS 7, the master is on CentOS 6. Eventually, I
want to retire the CentOS 6 host. I'm following this migration guide:
https://www.freeipa.org/page/Howto/Migration#Migrating_existing_FreeIPA_d...
However, running `ipa-replica-install --setup-ca ./replica-info-replica.fqdn.gpg` always
gets stuck and eventually fails when setting up pki-tomcatd:
Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes
[1/28]: configuring certificate server instance
[2/28]: exporting Dogtag certificate store pin
[3/28]: stopping certificate server instance to update CS.cfg
[4/28]: backing up CS.cfg
[5/28]: disabling nonces
[6/28]: set up CRL publishing
[7/28]: enable PKIX certificate path discovery and validation
[8/28]: starting certificate server instance
[9/28]: configure certmonger for renewals
[10/28]: importing RA certificate from PKCS #12 file
[11/28]: setting audit signing renewal to 2 years
[12/28]: restarting certificate server
[13/28]: authorizing RA to modify profiles
[14/28]: authorizing RA to manage lightweight CAs
[15/28]: Ensure lightweight CAs container exists
[16/28]: Ensuring backward compatibility
[17/28]: configure certificate renewals
[18/28]: configure Server-Cert certificate renewal
[19/28]: Configure HTTP to proxy connections
[20/28]: restarting certificate server
[21/28]: updating IPA configuration
[22/28]: enabling CA instance
[23/28]: exposing CA instance on LDAP
[24/28]: migrating certificate profiles to LDAP
[25/28]: importing IPA certificate profiles
[26/28]: adding default CA ACL
[27/28]: adding 'ipa' CA entry
[28/28]: configuring certmonger renewal for lightweight CAs
Done configuring certificate server (pki-tomcatd).
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.
ipapython.admintool: ERROR CA did not start in 300.0s
ipapython.admintool: ERROR The ipa-replica-install command failed. See
/var/log/ipareplica-install.log for more information
Looking at `ipareplica-install.log`:
2019-07-24T11:14:21Z DEBUG stderr=
2019-07-24T11:14:21Z DEBUG wait_for_open_ports: localhost [8080, 8443] timeout 300
2019-07-24T11:14:21Z DEBUG waiting for port: 8080
2019-07-24T11:14:21Z DEBUG Failed to connect to port 8080 tcp on ::1
2019-07-24T11:14:21Z DEBUG Failed to connect to port 8080 tcp on 127.0.0.1
2019-07-24T11:14:25Z DEBUG SUCCESS: port: 8080
2019-07-24T11:14:25Z DEBUG waiting for port: 8443
2019-07-24T11:14:25Z DEBUG SUCCESS: port: 8443
2019-07-24T11:14:25Z DEBUG Start of pki-tomcatd(a)pki-tomcat.service complete
2019-07-24T11:14:25Z DEBUG Waiting until the CA is running
2019-07-24T11:14:25Z DEBUG request POST http://replica.fqdn:8080/ca/admin/ca/getStatus
2019-07-24T11:14:25Z DEBUG request body ''
2019-07-24T11:14:44Z DEBUG response status 500
2019-07-24T11:14:44Z DEBUG response headers Server: Apache-Coyote/1.1
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 2208
Date: Wed, 24 Jul 2019 11:14:44 GMT
Connection: close
2019-07-24T11:14:44Z DEBUG response body '<html><head><title>Apache
Tomcat/7.0.76 - Error report</title><style><!--H1
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
H2
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;}
H3
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}
BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P
{font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color
: black;}A.name {color : black;}HR {color : #525D76;}--></style>
</head><body><h1>HTTP Status 500 - Subsystem
unavailable</h1><HR size="1"
noshade="noshade"><p><b>type</b> Exception
report</p><p><b>message</b> <u>Subsystem
unavailable</u></p><p><b>description</b> <u>The server
encountered an internal error that prevented it from fulfilling this requ
est.</u></p><p><b>exception</b>
<pre>javax.ws.rs.ServiceUnavailableException: Subsystem
unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:145)\n\torg.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:500)\n\torg.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)\n\torg.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)\n\torg.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)\n\torg.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)\n\torg.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)\n\torg.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)\n\tjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\torg.apache.tomcat.util.threads.TaskThrea
d$WrappingRunnable.run(TaskThread.java:61)\n\tjava.lang.Thread.run(Thread.java:748)\n</pre></p><p><b>note</b>
<u>The full stack trace of the root cause is available in the Apache Tomcat/7.0.76
logs.</u></p><HR size="1"
noshade="noshade"><h3>Apache
Tomcat/7.0.76</h3></body></html>'
2019-07-24T11:14:44Z DEBUG The CA status is: check interrupted due to error: Retrieving CA
status failed with status 500
2019-07-24T11:14:44Z DEBUG Waiting for CA to start...
2019-07-24T11:14:45Z DEBUG request POST http://replica.fqdn:8080/ca/admin/ca/getStatus
2019-07-24T11:14:45Z DEBUG request body ''
2019-07-24T11:14:45Z DEBUG response status 500
2019-07-24T11:14:45Z DEBUG response headers Server: Apache-Coyote/1.1
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 2208
Date: Wed, 24 Jul 2019 11:14:45 GMT
Connection: close
Looking into the log of pki-tomcatd, I see the following:
Internal Database Error encountered: Could not connect to LDAP server host replica.fqdn
port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
[...]
WARNING: Exception processing realm com.netscape.cms.tomcat.ProxyRealm@6ae79124 background
process
javax.ws.rs.ServiceUnavailableException: Subsystem unavailable
at com.netscape.cms.tomcat.ProxyRealm.backgroundProcess(ProxyRealm.java:137)
at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1356)
at org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5958)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1542)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1552)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1552)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1520)
at java.lang.Thread.run(Thread.java:748)
I checked that the pki-tomcatd uses the right certificates, following this guide:
https://floblanc.wordpress.com/2017/09/11/troubleshooting-freeipa-pki-tom...
Everything looked fine, i.e., tomcat uses the correct certificate and can also read the
private key.
Interestingly, during the setup of the replica, the setup is stuck for quite some time
(~30 minutes) in the step " [1/28]: configuring certificate server instance".
In the ns-slapd log, I can see a lot of the following:
INFO - import_monitor_threads - import ipaca: Processed 40105 entries -- average rate
123.8/sec, recent rate 114.0/sec, hit ratio 100%
I'm surprised by the number of entries. I had set up the same host as a replica in a
previous try, but needed to remove it due to another error. May those be left-overs from
the previous replica instance? I didn't see this happening on the first attempt.
Before redoing the setup, I removed the host from the replica set with `ipa-replica-manage
del --force`, from the csreplica with `ipa-csreplica-manage del --force`, and also deleted
the host entry itself with `ipa host-del`. I also uninstalled the freeipa server on the
replica host.
I'm also wondering about the `Authentication failed (48)`, as 48 indicates
LDAP_INAPPROPRIATE_AUTH.
I'm not sure how to debug this. Any help is appreciated!
Kind regards,
Till
7:32 a.m.
Hi,
On Wed, Jul 24, 2019 at 2:13 PM Till Hofmann via FreeIPA-users
<freeipa-users(a)lists.fedorahosted.org> wrote:
Hi all,
I'm trying to set up a replica on CentOS 7, the master is on CentOS 6. Eventually, I
want to retire the CentOS 6 host. I'm following this migration guide:
https://www.freeipa.org/page/Howto/Migration#Migrating_existing_FreeIPA_d...
However, running `ipa-replica-install --setup-ca ./replica-info-replica.fqdn.gpg` always
gets stuck and eventually fails when setting up pki-tomcatd:
Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes
[1/28]: configuring certificate server instance
[2/28]: exporting Dogtag certificate store pin
[3/28]: stopping certificate server instance to update CS.cfg
[4/28]: backing up CS.cfg
[5/28]: disabling nonces
[6/28]: set up CRL publishing
[7/28]: enable PKIX certificate path discovery and validation
[8/28]: starting certificate server instance
[9/28]: configure certmonger for renewals
[10/28]: importing RA certificate from PKCS #12 file
[11/28]: setting audit signing renewal to 2 years
[12/28]: restarting certificate server
[13/28]: authorizing RA to modify profiles
[14/28]: authorizing RA to manage lightweight CAs
[15/28]: Ensure lightweight CAs container exists
[16/28]: Ensuring backward compatibility
[17/28]: configure certificate renewals
[18/28]: configure Server-Cert certificate renewal
[19/28]: Configure HTTP to proxy connections
[20/28]: restarting certificate server
[21/28]: updating IPA configuration
[22/28]: enabling CA instance
[23/28]: exposing CA instance on LDAP
[24/28]: migrating certificate profiles to LDAP
[25/28]: importing IPA certificate profiles
[26/28]: adding default CA ACL
[27/28]: adding 'ipa' CA entry
[28/28]: configuring certmonger renewal for lightweight CAs
Done configuring certificate server (pki-tomcatd).
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.
ipapython.admintool: ERROR CA did not start in 300.0s
ipapython.admintool: ERROR The ipa-replica-install command failed. See
/var/log/ipareplica-install.log for more information
Looking at `ipareplica-install.log`:
2019-07-24T11:14:21Z DEBUG stderr=
2019-07-24T11:14:21Z DEBUG wait_for_open_ports: localhost [8080, 8443] timeout 300
2019-07-24T11:14:21Z DEBUG waiting for port: 8080
2019-07-24T11:14:21Z DEBUG Failed to connect to port 8080 tcp on ::1
2019-07-24T11:14:21Z DEBUG Failed to connect to port 8080 tcp on 127.0.0.1
2019-07-24T11:14:25Z DEBUG SUCCESS: port: 8080
2019-07-24T11:14:25Z DEBUG waiting for port: 8443
2019-07-24T11:14:25Z DEBUG SUCCESS: port: 8443
2019-07-24T11:14:25Z DEBUG Start of pki-tomcatd(a)pki-tomcat.service complete
2019-07-24T11:14:25Z DEBUG Waiting until the CA is running
2019-07-24T11:14:25Z DEBUG request POST http://replica.fqdn:8080/ca/admin/ca/getStatus
2019-07-24T11:14:25Z DEBUG request body ''
2019-07-24T11:14:44Z DEBUG response status 500
2019-07-24T11:14:44Z DEBUG response headers Server: Apache-Coyote/1.1
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 2208
Date: Wed, 24 Jul 2019 11:14:44 GMT
Connection: close
2019-07-24T11:14:44Z DEBUG response body '<html><head><title>Apache
Tomcat/7.0.76 - Error report</title><style><!--H1
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
H2
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;}
H3
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}
BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P
{font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color
: black;}A.name {color : black;}HR {color : #525D76;}--></style>
</head><body><h1>HTTP Status 500 - Subsystem
unavailable</h1><HR size="1"
noshade="noshade"><p><b>type</b> Exception
report</p><p><b>message</b> <u>Subsystem
unavailable</u></p><p><b>description</b> <u>The server
encountered an internal error that prevented it from fulfilling this requ
est.</u></p><p><b>exception</b>
<pre>javax.ws.rs.ServiceUnavailableException: Subsystem
unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:145)\n\torg.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:500)\n\torg.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)\n\torg.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)\n\torg.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)\n\torg.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)\n\torg.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)\n\torg.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)\n\tjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\torg.apache.tomcat.util.threads.TaskThrea
d$WrappingRunnable.run(TaskThread.java:61)\n\tjava.lang.Thread.run(Thread.java:748)\n</pre></p><p><b>note</b>
<u>The full stack trace of the root cause is available in the Apache Tomcat/7.0.76
logs.</u></p><HR size="1"
noshade="noshade"><h3>Apache
Tomcat/7.0.76</h3></body></html>'
2019-07-24T11:14:44Z DEBUG The CA status is: check interrupted due to error: Retrieving
CA status failed with status 500
2019-07-24T11:14:44Z DEBUG Waiting for CA to start...
2019-07-24T11:14:45Z DEBUG request POST http://replica.fqdn:8080/ca/admin/ca/getStatus
2019-07-24T11:14:45Z DEBUG request body ''
2019-07-24T11:14:45Z DEBUG response status 500
2019-07-24T11:14:45Z DEBUG response headers Server: Apache-Coyote/1.1
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 2208
Date: Wed, 24 Jul 2019 11:14:45 GMT
Connection: close
Looking into the log of pki-tomcatd, I see the following:
Internal Database Error encountered: Could not connect to LDAP server host replica.fqdn
port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
[...]
WARNING: Exception processing realm com.netscape.cms.tomcat.ProxyRealm@6ae79124
background process
javax.ws.rs.ServiceUnavailableException: Subsystem unavailable
at com.netscape.cms.tomcat.ProxyRealm.backgroundProcess(ProxyRealm.java:137)
at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1356)
at org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5958)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1542)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1552)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1552)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1520)
at java.lang.Thread.run(Thread.java:748)
I checked that the pki-tomcatd uses the right certificates, following this guide:
https://floblanc.wordpress.com/2017/09/11/troubleshooting-freeipa-pki-tom...
Everything looked fine, i.e., tomcat uses the correct certificate and can also read the
private key.
Interestingly, during the setup of the replica, the setup is stuck for quite some time
(~30 minutes) in the step " [1/28]: configuring certificate server instance".
In the ns-slapd log, I can see a lot of the following:
INFO - import_monitor_threads - import ipaca: Processed 40105 entries -- average rate
123.8/sec, recent rate 114.0/sec, hit ratio 100%
I'm surprised by the number of entries. I had set up the same host as a replica in a
previous try, but needed to remove it due to another error. May those be left-overs from
the previous replica instance? I didn't see this happening on the first attempt.
Before redoing the setup, I removed the host from the replica set with `ipa-replica-manage
del --force`, from the csreplica with `ipa-csreplica-manage del --force`, and also deleted
the host entry itself with `ipa host-del`. I also uninstalled the freeipa server on the
replica host.
Could you count the actual number of requests records in the o=ipaca
suffix and examine them?
Cheers
François
> I'm also wondering about the `Authentication failed (48)`, as 48 indicates
LDAP_INAPPROPRIATE_AUTH.
>
> I'm not sure how to debug this. Any help is appreciated!
>
> Kind regards,
> Till
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
9:03 a.m.
Hi François,
Thanks for the reply!
On 7/24/19 2:32 PM, François Cami wrote:
>
> Interestingly, during the setup of the replica, the setup is stuck for quite some
time (~30 minutes) in the step " [1/28]: configuring certificate server
instance". In the ns-slapd log, I can see a lot of the following:
> INFO - import_monitor_threads - import ipaca: Processed 40105 entries -- average rate
123.8/sec, recent rate 114.0/sec, hit ratio 100%
> I'm surprised by the number of entries. I had set up the same host as a replica
in a previous try, but needed to remove it due to another error. May those be left-overs
from the previous replica instance? I didn't see this happening on the first attempt.
Before redoing the setup, I removed the host from the replica set with `ipa-replica-manage
del --force`, from the csreplica with `ipa-csreplica-manage del --force`, and also deleted
the host entry itself with `ipa host-del`. I also uninstalled the freeipa server on the
replica host.
Could you count the actual number of requests records in the o=ipaca
suffix and examine them?
I'm not exactly sure what you mean (I don't have much experience with
LDAP). Searching for "(objectclass=ipaca*)" gives me 2 results (but I
guess that's not what you meant). On the replica, ns-slapd processed
267358 entries before finishing.
Kind regards,
Till
10:48 a.m.
On 7/24/19 4:03 PM, Till Hofmann wrote:
Hi François,
Thanks for the reply!
On 7/24/19 2:32 PM, François Cami wrote:
>>
>> Interestingly, during the setup of the replica, the setup is stuck for quite some
time (~30 minutes) in the step " [1/28]: configuring certificate server
instance". In the ns-slapd log, I can see a lot of the following:
>> INFO - import_monitor_threads - import ipaca: Processed 40105 entries -- average
rate 123.8/sec, recent rate 114.0/sec, hit ratio 100%
>> I'm surprised by the number of entries. I had set up the same host as a
replica in a previous try, but needed to remove it due to another error. May those be
left-overs from the previous replica instance? I didn't see this happening on the
first attempt. Before redoing the setup, I removed the host from the replica set with
`ipa-replica-manage del --force`, from the csreplica with `ipa-csreplica-manage del
--force`, and also deleted the host entry itself with `ipa host-del`. I also uninstalled
the freeipa server on the replica host.
>
> Could you count the actual number of requests records in the o=ipaca
> suffix and examine them?
I'm not exactly sure what you mean (I don't have much experience with
LDAP). Searching for "(objectclass=ipaca*)" gives me 2 results (but I
guess that's not what you meant). On the replica, ns-slapd processed
267358 entries before finishing.
OK, I was looking in the wrong place. The number of request records in
LDAP is 268721. I'm not sure what exactly I should be looking for, but I
don't see anything unusual.
I'm currently looking into the ldap auth config of tomcat, I noticed
that it looks quite different compared to the master instance.
Kind regards,
Till
10:52 a.m.
On Wed, Jul 24, 2019 at 5:48 PM Till Hofmann <thofmann(a)fedoraproject.org> wrote:
On 7/24/19 4:03 PM, Till Hofmann wrote:
> Hi François,
>
> Thanks for the reply!
>
> On 7/24/19 2:32 PM, François Cami wrote:
>
>>>
>>> Interestingly, during the setup of the replica, the setup is stuck for quite
some time (~30 minutes) in the step " [1/28]: configuring certificate server
instance". In the ns-slapd log, I can see a lot of the following:
>>> INFO - import_monitor_threads - import ipaca: Processed 40105 entries --
average rate 123.8/sec, recent rate 114.0/sec, hit ratio 100%
>>> I'm surprised by the number of entries. I had set up the same host as a
replica in a previous try, but needed to remove it due to another error. May those be
left-overs from the previous replica instance? I didn't see this happening on the
first attempt. Before redoing the setup, I removed the host from the replica set with
`ipa-replica-manage del --force`, from the csreplica with `ipa-csreplica-manage del
--force`, and also deleted the host entry itself with `ipa host-del`. I also uninstalled
the freeipa server on the replica host.
>>
>> Could you count the actual number of requests records in the o=ipaca
>> suffix and examine them?
>
>
> I'm not exactly sure what you mean (I don't have much experience with
> LDAP). Searching for "(objectclass=ipaca*)" gives me 2 results (but I
> guess that's not what you meant). On the replica, ns-slapd processed
> 267358 entries before finishing.
OK, I was looking in the wrong place. The number of request records in
LDAP is 268721. I'm not sure what exactly I should be looking for, but I
don't see anything unusual.
I could be wrong but at 114 entries processed per second, 268721 would
need 37 mins to complete and the timeout is at 5 mins (the 300 seconds
above).
Let me investigate a bit more and I'll get back to you.
Cheers
François
> I'm currently looking into the ldap auth config of tomcat, I noticed
> that it looks quite different compared to the master instance.
>
> Kind regards,
> Till
11:03 a.m.
On Wed, Jul 24, 2019 at 5:52 PM François Cami <fcami(a)redhat.com> wrote:
On Wed, Jul 24, 2019 at 5:48 PM Till Hofmann <thofmann(a)fedoraproject.org> wrote:
>
>
>
> On 7/24/19 4:03 PM, Till Hofmann wrote:
> > Hi François,
> >
> > Thanks for the reply!
> >
> > On 7/24/19 2:32 PM, François Cami wrote:
> >
> >>>
> >>> Interestingly, during the setup of the replica, the setup is stuck for
quite some time (~30 minutes) in the step " [1/28]: configuring certificate server
instance". In the ns-slapd log, I can see a lot of the following:
> >>> INFO - import_monitor_threads - import ipaca: Processed 40105 entries
-- average rate 123.8/sec, recent rate 114.0/sec, hit ratio 100%
> >>> I'm surprised by the number of entries. I had set up the same host
as a replica in a previous try, but needed to remove it due to another error. May those be
left-overs from the previous replica instance? I didn't see this happening on the
first attempt. Before redoing the setup, I removed the host from the replica set with
`ipa-replica-manage del --force`, from the csreplica with `ipa-csreplica-manage del
--force`, and also deleted the host entry itself with `ipa host-del`. I also uninstalled
the freeipa server on the replica host.
> >>
> >> Could you count the actual number of requests records in the o=ipaca
> >> suffix and examine them?
> >
> >
> > I'm not exactly sure what you mean (I don't have much experience with
> > LDAP). Searching for "(objectclass=ipaca*)" gives me 2 results (but
I
> > guess that's not what you meant). On the replica, ns-slapd processed
> > 267358 entries before finishing.
>
> OK, I was looking in the wrong place. The number of request records in
> LDAP is 268721. I'm not sure what exactly I should be looking for, but I
> don't see anything unusual.
I could be wrong but at 114 entries processed per second, 268721 would
need 37 mins to complete and the timeout is at 5 mins (the 300 seconds
above).
Let me investigate a bit more and I'll get back to you.
So it looks like you are hitting an issue Rob is already working on:
https://github.com/freeipa/freeipa/pull/3374
but the fix is WIP.
In the meantime, can you edit ipalib/constants.py in your system?
Around line 175 there should be:
('replication_wait_timeout', 300),
It should be safe to replace 300 with 3600 at replica install time.
If install succeeds then wait at least 1h then shut down services:
# ipactl stop
revert the above change, and restart IPA:
# ipactl start
Please let us know if this helped.
François
> Cheers
> François
>
> > I'm currently looking into the ldap auth config of tomcat, I noticed
> > that it looks quite different compared to the master instance.
> >
> > Kind regards,
> > Till
2:30 p.m.
On 7/24/19 6:03 PM, François Cami wrote:
On Wed, Jul 24, 2019 at 5:52 PM François Cami
<fcami(a)redhat.com> wrote:
>
> On Wed, Jul 24, 2019 at 5:48 PM Till Hofmann <thofmann(a)fedoraproject.org>
wrote:
>>
>>
>>
>> On 7/24/19 4:03 PM, Till Hofmann wrote:
>>> Hi François,
>>>
>>> Thanks for the reply!
>>>
>>> On 7/24/19 2:32 PM, François Cami wrote:
>>>
>>>>>
>>>>> Interestingly, during the setup of the replica, the setup is stuck
for quite some time (~30 minutes) in the step " [1/28]: configuring certificate
server instance". In the ns-slapd log, I can see a lot of the following:
>>>>> INFO - import_monitor_threads - import ipaca: Processed 40105 entries
-- average rate 123.8/sec, recent rate 114.0/sec, hit ratio 100%
>>>>> I'm surprised by the number of entries. I had set up the same
host as a replica in a previous try, but needed to remove it due to another error. May
those be left-overs from the previous replica instance? I didn't see this happening on
the first attempt. Before redoing the setup, I removed the host from the replica set with
`ipa-replica-manage del --force`, from the csreplica with `ipa-csreplica-manage del
--force`, and also deleted the host entry itself with `ipa host-del`. I also uninstalled
the freeipa server on the replica host.
>>>>
>>>> Could you count the actual number of requests records in the o=ipaca
>>>> suffix and examine them?
>>>
>>>
>>> I'm not exactly sure what you mean (I don't have much experience
with
>>> LDAP). Searching for "(objectclass=ipaca*)" gives me 2 results (but
I
>>> guess that's not what you meant). On the replica, ns-slapd processed
>>> 267358 entries before finishing.
>>
>> OK, I was looking in the wrong place. The number of request records in
>> LDAP is 268721. I'm not sure what exactly I should be looking for, but I
>> don't see anything unusual.
>
> I could be wrong but at 114 entries processed per second, 268721 would
> need 37 mins to complete and the timeout is at 5 mins (the 300 seconds
> above).
>
> Let me investigate a bit more and I'll get back to you.
So it looks like you are hitting an issue Rob is already working on:
https://github.com/freeipa/freeipa/pull/3374
but the fix is WIP.
In the meantime, can you edit ipalib/constants.py in your system?
Around line 175 there should be:
('replication_wait_timeout', 300),
It should be safe to replace 300 with 3600 at replica install time.
If install succeeds then wait at least 1h then shut down services:
# ipactl stop
revert the above change, and restart IPA:
# ipactl start
Please let us know if this helped.
Thanks for the suggestion. Unfortunately, that didn't help. The timeout
timer only starts after the last step (28/28), but the syncing already
happens before that, so the timeout doesn't affect that. I see the same
errors as before, just for a longer time.
I could get one step further by modifying the tomcat config so it
wouldn't use the certificate, similar to what they did here:
https://www.redhat.com/archives/freeipa-users/2017-January/msg00216.html
However, that's just a workaround. Also, I immediately get another error:
testLDAPConnection: The specified user cn=Replication Manager
masterAgreement1-replica.fqdn-pki-tomcat,cn=config does not exist
CMSEngine: init(): password test execution failed for replicationdbwith
NO_SUCH_USER. This may not be a latest instance. Ignoring ..
Kind regards,
Till
2:18 a.m.
I added Replication timeout in /usr/share/dirsrv/data/template-dse.ldif on
replica before ipa-replica-install which took care of time consumed for
large data getting replicated.
https://access.redhat.com/documentation/en-us/red_hat_directory_server/10...
You
need to setup timeout as per this link.
On Thu, Jul 25, 2019 at 1:37 AM Till Hofmann via FreeIPA-users <
freeipa-users(a)lists.fedorahosted.org> wrote:
On 7/24/19 6:03 PM, François Cami wrote:
> On Wed, Jul 24, 2019 at 5:52 PM François Cami <fcami(a)redhat.com> wrote:
>>
>> On Wed, Jul 24, 2019 at 5:48 PM Till Hofmann <
thofmann(a)fedoraproject.org> wrote:
>>>
>>>
>>>
>>> On 7/24/19 4:03 PM, Till Hofmann wrote:
>>>> Hi François,
>>>>
>>>> Thanks for the reply!
>>>>
>>>> On 7/24/19 2:32 PM, François Cami wrote:
>>>>
>>>>>>
>>>>>> Interestingly, during the setup of the replica, the setup is
stuck
for quite some time (~30 minutes) in the step " [1/28]: configuring
certificate server instance". In the ns-slapd log, I can see a lot of the
following:
>>>>>> INFO - import_monitor_threads - import ipaca: Processed 40105
entries -- average rate 123.8/sec, recent rate 114.0/sec, hit ratio 100%
>>>>>> I'm surprised by the number of entries. I had set up the
same host
as a replica in a previous try, but needed to remove it due to another
error. May those be left-overs from the previous replica instance? I didn't
see this happening on the first attempt. Before redoing the setup, I
removed the host from the replica set with `ipa-replica-manage del
--force`, from the csreplica with `ipa-csreplica-manage del --force`, and
also deleted the host entry itself with `ipa host-del`. I also uninstalled
the freeipa server on the replica host.
>>>>>
>>>>> Could you count the actual number of requests records in the
o=ipaca
>>>>> suffix and examine them?
>>>>
>>>>
>>>> I'm not exactly sure what you mean (I don't have much experience
with
>>>> LDAP). Searching for "(objectclass=ipaca*)" gives me 2 results
(but I
>>>> guess that's not what you meant). On the replica, ns-slapd
processed
>>>> 267358 entries before finishing.
>>>
>>> OK, I was looking in the wrong place. The number of request records in
>>> LDAP is 268721. I'm not sure what exactly I should be looking for, but
I
>>> don't see anything unusual.
>>
>> I could be wrong but at 114 entries processed per second, 268721 would
>> need 37 mins to complete and the timeout is at 5 mins (the 300 seconds
>> above).
>>
>> Let me investigate a bit more and I'll get back to you.
>
> So it looks like you are hitting an issue Rob is already working on:
> https://github.com/freeipa/freeipa/pull/3374
> but the fix is WIP.
>
> In the meantime, can you edit ipalib/constants.py in your system?
> Around line 175 there should be:
> ('replication_wait_timeout', 300),
>
> It should be safe to replace 300 with 3600 at replica install time.
>
> If install succeeds then wait at least 1h then shut down services:
> # ipactl stop
> revert the above change, and restart IPA:
> # ipactl start
>
> Please let us know if this helped.
>
Thanks for the suggestion. Unfortunately, that didn't help. The timeout
timer only starts after the last step (28/28), but the syncing already
happens before that, so the timeout doesn't affect that. I see the same
errors as before, just for a longer time.
I could get one step further by modifying the tomcat config so it
wouldn't use the certificate, similar to what they did here:
https://www.redhat.com/archives/freeipa-users/2017-January/msg00216.html
However, that's just a workaround. Also, I immediately get another error:
testLDAPConnection: The specified user cn=Replication Manager
masterAgreement1-replica.fqdn-pki-tomcat,cn=config does not exist
CMSEngine: init(): password test execution failed for replicationdbwith
NO_SUCH_USER. This may not be a latest instance. Ignoring ..
Kind regards,
Till
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
--
Thanks & Regards
Arpit Tolani
3:33 a.m.
New subject: ipa-replica-install fails to start pki-tomcatd
Hi Arpit,
On 7/26/19 9:18 AM, Arpit Tolani wrote:
I added Replication timeout in
/usr/share/dirsrv/data/template-dse.ldif
on replica before ipa-replica-install which took care of time consumed
for large data getting replicated.
https://access.redhat.com/documentation/en-us/red_hat_directory_server/10...
need to setup timeout as per this link.
Thanks for the hint, but I really don't think a timeout is the issue
here. After I did what François suggested, the replication went through
(as before) and it then waited for 60 (instead of 5) minutes for
pki-tomcatd to start, which repeatedly fails to authenticate with the
following error:
netscape.ldap.LDAPException: Authentication failed (48)
There's is nothing else going on, so increasing more timeouts will just
increase the time until it fails.
To see if this is related to some leftover config flying around for the
replica host, I set up a second replica host, which now fails much
earlier during setup with the following error:
NSMMReplicationPlugin - acquire_replica - agmt="cn=master.fqdn"
(master:389): Unable to acquire replica: permission denied. The bind dn
"" does not have permission to supply replication updates to the
replica. Will retry later.
I tried this multiple times, always resulting in the same error.
Kind regards,
Till
>
> On Thu, Jul 25, 2019 at 1:37 AM Till Hofmann via FreeIPA-users
> <freeipa-users(a)lists.fedorahosted.org
> <mailto:freeipa-users@lists.fedorahosted.org>> wrote:
>
>
>
> On 7/24/19 6:03 PM, François Cami wrote:
> > On Wed, Jul 24, 2019 at 5:52 PM François Cami <fcami(a)redhat.com
> <mailto:fcami@redhat.com>> wrote:
> >>
> >> On Wed, Jul 24, 2019 at 5:48 PM Till Hofmann
> <thofmann(a)fedoraproject.org <mailto:thofmann@fedoraproject.org>>
wrote:
> >>>
> >>>
> >>>
> >>> On 7/24/19 4:03 PM, Till Hofmann wrote:
> >>>> Hi François,
> >>>>
> >>>> Thanks for the reply!
> >>>>
> >>>> On 7/24/19 2:32 PM, François Cami wrote:
> >>>>
> >>>>>>
> >>>>>> Interestingly, during the setup of the replica, the
setup is
> stuck for quite some time (~30 minutes) in the step " [1/28]:
> configuring certificate server instance". In the ns-slapd log, I can
> see a lot of the following:
> >>>>>> INFO - import_monitor_threads - import ipaca: Processed
40105
> entries -- average rate 123.8/sec, recent rate 114.0/sec, hit ratio 100%
> >>>>>> I'm surprised by the number of entries. I had set up
the same
> host as a replica in a previous try, but needed to remove it due to
> another error. May those be left-overs from the previous replica
> instance? I didn't see this happening on the first attempt. Before
> redoing the setup, I removed the host from the replica set with
> `ipa-replica-manage del --force`, from the csreplica with
> `ipa-csreplica-manage del --force`, and also deleted the host entry
> itself with `ipa host-del`. I also uninstalled the freeipa server on
> the replica host.
> >>>>>
> >>>>> Could you count the actual number of requests records in
the
> o=ipaca
> >>>>> suffix and examine them?
> >>>>
> >>>>
> >>>> I'm not exactly sure what you mean (I don't have much
> experience with
> >>>> LDAP). Searching for "(objectclass=ipaca*)" gives me 2
results
> (but I
> >>>> guess that's not what you meant). On the replica, ns-slapd
> processed
> >>>> 267358 entries before finishing.
> >>>
> >>> OK, I was looking in the wrong place. The number of request
> records in
> >>> LDAP is 268721. I'm not sure what exactly I should be looking
> for, but I
> >>> don't see anything unusual.
> >>
> >> I could be wrong but at 114 entries processed per second, 268721
> would
> >> need 37 mins to complete and the timeout is at 5 mins (the 300
> seconds
> >> above).
> >>
> >> Let me investigate a bit more and I'll get back to you.
> >
> > So it looks like you are hitting an issue Rob is already working on:
> > https://github.com/freeipa/freeipa/pull/3374
> > but the fix is WIP.
> >
> > In the meantime, can you edit ipalib/constants.py in your system?
> > Around line 175 there should be:
> > ('replication_wait_timeout', 300),
> >
> > It should be safe to replace 300 with 3600 at replica install time.
> >
> > If install succeeds then wait at least 1h then shut down services:
> > # ipactl stop
> > revert the above change, and restart IPA:
> > # ipactl start
> >
> > Please let us know if this helped.
> >
>
> Thanks for the suggestion. Unfortunately, that didn't help. The timeout
> timer only starts after the last step (28/28), but the syncing already
> happens before that, so the timeout doesn't affect that. I see the same
> errors as before, just for a longer time.
>
> I could get one step further by modifying the tomcat config so it
> wouldn't use the certificate, similar to what they did here:
> https://www.redhat.com/archives/freeipa-users/2017-January/msg00216.html
>
> However, that's just a workaround. Also, I immediately get another
> error:
> testLDAPConnection: The specified user cn=Replication Manager
> masterAgreement1-replica.fqdn-pki-tomcat,cn=config does not exist
> CMSEngine: init(): password test execution failed for replicationdbwith
> NO_SUCH_USER. This may not be a latest instance. Ignoring ..
>
>
> Kind regards,
> Till
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> <mailto:freeipa-users@lists.fedorahosted.org>
> To unsubscribe send an email to
> freeipa-users-leave(a)lists.fedorahosted.org
> <mailto:freeipa-users-leave@lists.fedorahosted.org>
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
>
>
>
> --
> Thanks & Regards
> Arpit Tolani
>
7:59 a.m.
On 7/24/19 2:12 PM, Till Hofmann via FreeIPA-users wrote:
Hi all,
I'm trying to set up a replica on CentOS 7, the master is on CentOS 6. Eventually, I
want to retire the CentOS 6 host. I'm following this migration guide:
https://www.freeipa.org/page/Howto/Migration#Migrating_existing_FreeIPA_d...
However, running `ipa-replica-install --setup-ca ./replica-info-replica.fqdn.gpg` always
gets stuck and eventually fails when setting up pki-tomcatd:
Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes
[1/28]: configuring certificate server instance
[2/28]: exporting Dogtag certificate store pin
[3/28]: stopping certificate server instance to update CS.cfg
[4/28]: backing up CS.cfg
[5/28]: disabling nonces
[6/28]: set up CRL publishing
[7/28]: enable PKIX certificate path discovery and validation
[8/28]: starting certificate server instance
[9/28]: configure certmonger for renewals
[10/28]: importing RA certificate from PKCS #12 file
[11/28]: setting audit signing renewal to 2 years
[12/28]: restarting certificate server
[13/28]: authorizing RA to modify profiles
[14/28]: authorizing RA to manage lightweight CAs
[15/28]: Ensure lightweight CAs container exists
[16/28]: Ensuring backward compatibility
[17/28]: configure certificate renewals
[18/28]: configure Server-Cert certificate renewal
[19/28]: Configure HTTP to proxy connections
[20/28]: restarting certificate server
[21/28]: updating IPA configuration
[22/28]: enabling CA instance
[23/28]: exposing CA instance on LDAP
[24/28]: migrating certificate profiles to LDAP
[25/28]: importing IPA certificate profiles
[26/28]: adding default CA ACL
[27/28]: adding 'ipa' CA entry
[28/28]: configuring certmonger renewal for lightweight CAs
Done configuring certificate server (pki-tomcatd).
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.
ipapython.admintool: ERROR CA did not start in 300.0s
ipapython.admintool: ERROR The ipa-replica-install command failed. See
/var/log/ipareplica-install.log for more information
Looking at `ipareplica-install.log`:
2019-07-24T11:14:21Z DEBUG stderr=
2019-07-24T11:14:21Z DEBUG wait_for_open_ports: localhost [8080, 8443] timeout 300
2019-07-24T11:14:21Z DEBUG waiting for port: 8080
2019-07-24T11:14:21Z DEBUG Failed to connect to port 8080 tcp on ::1
2019-07-24T11:14:21Z DEBUG Failed to connect to port 8080 tcp on 127.0.0.1
2019-07-24T11:14:25Z DEBUG SUCCESS: port: 8080
2019-07-24T11:14:25Z DEBUG waiting for port: 8443
2019-07-24T11:14:25Z DEBUG SUCCESS: port: 8443
2019-07-24T11:14:25Z DEBUG Start of pki-tomcatd(a)pki-tomcat.service complete
2019-07-24T11:14:25Z DEBUG Waiting until the CA is running
2019-07-24T11:14:25Z DEBUG request POST http://replica.fqdn:8080/ca/admin/ca/getStatus
2019-07-24T11:14:25Z DEBUG request body ''
2019-07-24T11:14:44Z DEBUG response status 500
2019-07-24T11:14:44Z DEBUG response headers Server: Apache-Coyote/1.1
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 2208
Date: Wed, 24 Jul 2019 11:14:44 GMT
Connection: close
2019-07-24T11:14:44Z DEBUG response body '<html><head><title>Apache
Tomcat/7.0.76 - Error report</title><style><!--H1
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
H2
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;}
H3
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}
BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P
{font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color
: black;}A.name {color : black;}HR {color : #525D76;}--></style>
</head><body><h1>HTTP Status 500 - Subsystem
unavailable</h1><HR size="1"
noshade="noshade"><p><b>type</b> Exception
report</p><p><b>message</b> <u>Subsystem
unavailable</u></p><p><b>description</b> <u>The server
encountered an internal error that prevented it from fulfilling this requ
est.</u></p><p><b>exception</b>
<pre>javax.ws.rs.ServiceUnavailableException: Subsystem
unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:145)\n\torg.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:500)\n\torg.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)\n\torg.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)\n\torg.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)\n\torg.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)\n\torg.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)\n\torg.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)\n\tjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\torg.apache.tomcat.util.threads.TaskThrea
d$WrappingRunnable.run(TaskThread.java:61)\n\tjava.lang.Thread.run(Thread.java:748)\n</pre></p><p><b>note</b>
<u>The full stack trace of the root cause is available in the Apache Tomcat/7.0.76
logs.</u></p><HR size="1"
noshade="noshade"><h3>Apache
Tomcat/7.0.76</h3></body></html>'
2019-07-24T11:14:44Z DEBUG The CA status is: check interrupted due to error: Retrieving
CA status failed with status 500
2019-07-24T11:14:44Z DEBUG Waiting for CA to start...
2019-07-24T11:14:45Z DEBUG request POST http://replica.fqdn:8080/ca/admin/ca/getStatus
2019-07-24T11:14:45Z DEBUG request body ''
2019-07-24T11:14:45Z DEBUG response status 500
2019-07-24T11:14:45Z DEBUG response headers Server: Apache-Coyote/1.1
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 2208
Date: Wed, 24 Jul 2019 11:14:45 GMT
Connection: close
Looking into the log of pki-tomcatd, I see the following:
Internal Database Error encountered: Could not connect to LDAP server host replica.fqdn
port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
Hi,
a few things to check on the replica:
- is the ldap server running and listening on port 636?
- is there any SSL error in the log of pki-tomcatd? I recall an issue
6577 [1] with a topology initially deployed as CA-less, then CA got
installed but the admin forgot to run ipa-certupdate on the nodes. As a
result, the CA cert was not put in all the relevant databases and
replica files did not contain the CA cert.
[1] https://pagure.io/freeipa/issue/6577
flo
[...]
WARNING: Exception processing realm com.netscape.cms.tomcat.ProxyRealm@6ae79124
background process
javax.ws.rs.ServiceUnavailableException: Subsystem unavailable
at com.netscape.cms.tomcat.ProxyRealm.backgroundProcess(ProxyRealm.java:137)
at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1356)
at
org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5958)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1542)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1552)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1552)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1520)
at java.lang.Thread.run(Thread.java:748)
I checked that the pki-tomcatd uses the right certificates, following this guide:
https://floblanc.wordpress.com/2017/09/11/troubleshooting-freeipa-pki-tom...
Everything looked fine, i.e., tomcat uses the correct certificate and can also read the
private key.
Interestingly, during the setup of the replica, the setup is stuck for quite some time
(~30 minutes) in the step " [1/28]: configuring certificate server instance".
In the ns-slapd log, I can see a lot of the following:
INFO - import_monitor_threads - import ipaca: Processed 40105 entries -- average rate
123.8/sec, recent rate 114.0/sec, hit ratio 100%
I'm surprised by the number of entries. I had set up the same host as a replica in a
previous try, but needed to remove it due to another error. May those be left-overs from
the previous replica instance? I didn't see this happening on the first attempt.
Before redoing the setup, I removed the host from the replica set with `ipa-replica-manage
del --force`, from the csreplica with `ipa-csreplica-manage del --force`, and also deleted
the host entry itself with `ipa host-del`. I also uninstalled the freeipa server on the
replica host.
I'm also wondering about the `Authentication failed (48)`, as 48 indicates
LDAP_INAPPROPRIATE_AUTH.
I'm not sure how to debug this. Any help is appreciated!
Kind regards,
Till
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
9:31 a.m.
Hi Florence,
Thanks for the pointers!
On 7/24/19 2:59 PM, Florence Blanc-Renaud wrote:
Hi,
a few things to check on the replica:
- is the ldap server running and listening on port 636?
Yes, the server is running and listening to port 636. I can also query
the server, but only after running
`export LDAPTLS_CACERT=/etc/ipa/ca.crt`.
If I don't set the CACERT, then the connection fails with
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)
- is there any SSL error in the log of pki-tomcatd? I recall an
issue
6577 [1] with a topology initially deployed as CA-less, then CA got
installed but the admin forgot to run ipa-certupdate on the nodes. As a
result, the CA cert was not put in all the relevant databases and
replica files did not contain the CA cert.
Do you mean other than this one:
Internal Database Error encountered: Could not connect to LDAP server
host replica.fqdn port 636 Error netscape.ldap.LDAPException:
Authentication failed (48)
I can also see the following error, but that's before pki-tomcatd is
restarted:
testLDAPConnection: The specified user cn=Replication Manager
masterAgreement1-replica.fqdn.de-pki-tomcat,cn=config does not exist
CMSEngine: init(): password test execution failed for replicationdbwith
NO_SUCH_USER. This may not be a latest instance. Ignoring ..
The error in #6577 looks quite similar, but the scenario is different. I
can't just call ipa-certupdate because there is no configured ipa client
on the replica.
I'm also wondering if I could just set up the replica without the CA,
shut down the old master and then use ipa-ca-install to set up the CA on
the new server. Would that work?
Kind regards,
Till
>
> [1] https://pagure.io/freeipa/issue/6577
>
> flo
>
7:55 a.m.
Hi all,
I managed to work around the issue by:
1. Setting up the replica without the CA (i.e., `ipa-replica-install` without
`--setup-ca`)
2. Set up the CA with `ipa-ca-install`. This also failed at some point (because it could
not contact the old master on port 8443), but it seemed to do "enough" so I
could actually ignore the missing steps.
I turned off the original master, verified that I could still log in on the clients and
also tested certificate renewal with `ipa-cacert-manage renew`, which was successful.
I don't know what the missing steps were, I hope this won't bite me in the long
run. Do you have any suggestions what else I could test to verify that the CA is also
working properly?
Thanks for all the help!
Kind regards,
Till
8:06 a.m.
New subject: ipa-replica-install fails to start pki-tomcatd
Till Hofmann via FreeIPA-users wrote:
Hi all,
I managed to work around the issue by:
1. Setting up the replica without the CA (i.e., `ipa-replica-install` without
`--setup-ca`)
2. Set up the CA with `ipa-ca-install`. This also failed at some point (because it could
not contact the old master on port 8443), but it seemed to do "enough" so I
could actually ignore the missing steps.
I turned off the original master, verified that I could still log in on the clients and
also tested certificate renewal with `ipa-cacert-manage renew`, which was successful.
This is not a sufficient test. This command only renews the CA certificate.
I don't know what the missing steps were, I hope this won't
bite me in the long run. Do you have any suggestions what else I could test to verify that
the CA is also working properly?
If you want to test that the CA works then issue a new certificate,
revoke it, ensure it lands in the CRL at next generation.
You also need to set the certificate renewal master and CRL generator.
1729
days inactive
1731
days old
freeipa-users@lists.fedorahosted.org
12 comments
5 participants
participants (5)
-
Arpit Tolani -
Florence Blanc-Renaud -
François Cami -
Rob Crittenden -
Till Hofmann