11:08 p.m.
When I first installed our replica, it worked just fine - I could add a
user and see it on the master server. And vice versa.
I recently went back to take a look and make sure everything was working -
and it's not.
ipactl status shows everything is ok. Munge is up. I can ssh hostname
between machines.
When I look at the ID Views in the interface, I get an "IPA Error 903:
InternalError".
When I do an id <username> I get nosuch user.
I did some googling. In /var log/dirsrv/domain/errors I found this:
[26/Oct/2017:12:31:23.454702287 +1100] - ERR - set_krb5_creds - Could not
get initial credentials for principal [ldap/
vmdr-linuxidm.unix.domain.com(a)UNIX.DOMAIN.COM] in keytab
[FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC for
requested realm)
I can get `kinit admin` working fine. But there's something wrong. I don't
know where to look exactly.
/var/log/httpd/error has this
RuntimeError: Unable to load file /usr/share/ipa/smb.conf.empty
Which is interesting. There's no file /usr/share/ipa/smb.conf.empty but
there is a /usr/share/ipa/smb.conf.template?
Ok, I think I've found the problem:
ipa-replica-conncheck -c -m <master>
Failed to connect to port 7389 tcp on 10.126.18.73
PKI-CA: Directory Service port (7389): FAILED
ERROR: Port check failed! Inaccessible port(s): 7389 (TCP)
On the master, pki-tomcatd is showing as OK, although nmap -sT -O localhost
doesn't show 7389 open.
Where can I look next?
ipa -version
VERSION: 4.5.0, API_VERSION: 2.228
cheers
L.
------
"The antidote to apocalypticism is *apocalyptic civics*. Apocalyptic civics
is the insistence that we cannot ignore the truth, nor should we panic
about it. It is a shared consciousness that our institutions have failed
and our ecosystem is collapsing, yet we are still here — and we are
creative agents who can shape our destinies. Apocalyptic civics is the
conviction that the only way out is through, and the only way through is
together. "
*Greg Bloom* @greggish
https://twitter.com/greggish/status/873177525903609857
3:38 p.m.
Lachlan Musicman via FreeIPA-users wrote:
When I first installed our replica, it worked just fine - I could add
a
user and see it on the master server. And vice versa.
I recently went back to take a look and make sure everything was working
- and it's not.
ipactl status shows everything is ok. Munge is up. I can ssh hostname
between machines.
When I look at the ID Views in the interface, I get an "IPA Error 903:
InternalError".
See /var/log/httpd/error_log for details, there may be a python backtrace.
When I do an id <username> I get nosuch user.
https://docs.pagure.org/SSSD.sssd/users/troubleshooting.html
I did some googling. In /var log/dirsrv/domain/errors I found this:
[26/Oct/2017:12:31:23.454702287 +1100] - ERR - set_krb5_creds - Could
not get initial credentials for principal
[ldap/vmdr-linuxidm.unix.domain.com(a)UNIX.DOMAIN.COM
<mailto:vmdr-linuxidm.unix.domain.com@UNIX.DOMAIN.COM>] in keytab
[FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC for
requested realm)
I can get `kinit admin` working fine. But there's something wrong. I
don't know where to look exactly.
KRB5_TRACE=/dev/stdout kinit admin
See what KDC kinit is using. It should be using the local box because
masters should point only to themselves.
/var/log/httpd/error has this
RuntimeError: Unable to load file /usr/share/ipa/smb.conf.empty
Which is interesting. There's no file /usr/share/ipa/smb.conf.empty but
there is a /usr/share/ipa/smb.conf.template?
Probably need more context.
Ok, I think I've found the problem:
ipa-replica-conncheck -c -m <master>
Failed to connect to port 7389 tcp on 10.126.18.73
PKI-CA: Directory Service port (7389): FAILED
ERROR: Port check failed! Inaccessible port(s): 7389 (TCP)
On the master, pki-tomcatd is showing as OK, although nmap -sT -O
localhost doesn't show 7389 open.
Where can I look next?
ipa -version
VERSION: 4.5.0, API_VERSION: 2.228
It shouldn't be even trying port 7389 with v4.5.0. Very old versions of
IPA used to use two separate 389-ds instances, one for the IPA data and
one for the CA data. They were combined long ago. This could just be a
check in case you had a very old master in which case this is a red herring.
rob
6:32 p.m.
On 27 October 2017 at 07:38, Rob Crittenden <rcritten(a)redhat.com> wrote:
Lachlan Musicman via FreeIPA-users wrote:
>
> When I look at the ID Views in the interface, I get an "IPA Error 903:
> InternalError".
See /var/log/httpd/error_log for details, there may be a python backtrace.
Sure do!
[Thu Oct 26 12:57:25.413102 2017] [:error] [pid 1316] ipa: ERROR:
non-public: RuntimeError: Unable to load file /usr/share/ipa/smb.conf.empty
[Thu Oct 26 12:57:25.413118 2017] [:error] [pid 1316] Traceback (most
recent call last):
[Thu Oct 26 12:57:25.413121 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 367, in
wsgi_execute
[Thu Oct 26 12:57:25.413124 2017] [:error] [pid 1316] result =
command(*args, **options)
[Thu Oct 26 12:57:25.413126 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 447, in __call__
[Thu Oct 26 12:57:25.413128 2017] [:error] [pid 1316] return
self.__do_call(*args, **options)
[Thu Oct 26 12:57:25.413130 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 475, in
__do_call
[Thu Oct 26 12:57:25.413133 2017] [:error] [pid 1316] ret =
self.run(*args, **options)
[Thu Oct 26 12:57:25.413135 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 797, in run
[Thu Oct 26 12:57:25.413137 2017] [:error] [pid 1316] return
self.execute(*args, **options)
[Thu Oct 26 12:57:25.413139 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/plugins/baseldap.py", line
2050, in execute
[Thu Oct 26 12:57:25.413141 2017] [:error] [pid 1316] truncated =
callback(self, ldap, entries, truncated, *args, **options)
[Thu Oct 26 12:57:25.413144 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/plugins/idviews.py", line 1123,
in post_callback
[Thu Oct 26 12:57:25.413146 2017] [:error] [pid 1316] ldap, entries,
truncated, *args, **options)
[Thu Oct 26 12:57:25.413148 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/plugins/idviews.py", line 829,
in post_callback
[Thu Oct 26 12:57:25.413151 2017] [:error] [pid 1316]
self.obj.convert_anchor_to_human_readable_form(entry, **options)
[Thu Oct 26 12:57:25.413153 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/plugins/idviews.py", line 733,
in convert_anchor_to_human_readable_form
[Thu Oct 26 12:57:25.413156 2017] [:error] [pid 1316] anchor
[Thu Oct 26 12:57:25.413158 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/plugins/idviews.py", line 632,
in resolve_anchor_to_object_name
[Thu Oct 26 12:57:25.413161 2017] [:error] [pid 1316] name =
domain_validator.get_trusted_domain_object_from_sid(sid)
[Thu Oct 26 12:57:25.413163 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/dcerpc.py", line 503, in
get_trusted_domain_object_from_sid
[Thu Oct 26 12:57:25.413165 2017] [:error] [pid 1316] attrs=attrs)
[Thu Oct 26 12:57:25.413167 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/dcerpc.py", line 380, in
get_trusted_domain_objects
[Thu Oct 26 12:57:25.413170 2017] [:error] [pid 1316] entries =
self.search_in_dc(domain, filter, attrs, scope, basedn)
[Thu Oct 26 12:57:25.413172 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/dcerpc.py", line 689, in
search_in_dc
[Thu Oct 26 12:57:25.413174 2017] [:error] [pid 1316] info =
self.__retrieve_trusted_domain_gc_list(domain)
[Thu Oct 26 12:57:25.413176 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/dcerpc.py", line 763, in
__retrieve_trusted_domain_gc_list
[Thu Oct 26 12:57:25.413179 2017] [:error] [pid 1316]
os.path.join(paths.USR_SHARE_IPA_DIR, "smb.conf.empty"))
[Thu Oct 26 12:57:25.413181 2017] [:error] [pid 1316] RuntimeError: Unable
to load file /usr/share/ipa/smb.conf.empty
> [26/Oct/2017:12:31:23.454702287 +1100] - ERR - set_krb5_creds - Could
> not get initial credentials for principal
> [ldap/vmdr-linuxidm.unix.domain.com(a)UNIX.DOMAIN.COM
> <mailto:vmdr-linuxidm.unix.domain.com@UNIX.DOMAIN.COM>] in keytab
> [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC for
> requested realm)
>
> I can get `kinit admin` working fine. But there's something wrong. I
> don't know where to look exactly.
KRB5_TRACE=/dev/stdout kinit admin
See what KDC kinit is using. It should be using the local box because
masters should point only to themselves.
Yes, that command makes reference to it's own ip, eg: "Sending TCP request
to stream 10.126.18.129:88"
> /var/log/httpd/error has this
>
> RuntimeError: Unable to load file /usr/share/ipa/smb.conf.empty
>
> Which is interesting. There's no file /usr/share/ipa/smb.conf.empty but
> there is a /usr/share/ipa/smb.conf.template?
Probably need more context.
I've only just realised this is the above error - when I go to ID
View->Default Trust View in the WebUI, I get the above python stacktrace,
but I also get
[Fri Oct 27 10:03:43.466674 2017] [:warn] [pid 5686] [client
10.126.160.47:53715] failed to set perms (3140) on file
(/var/run/ipa/ccaches/admin(a)UNIX.DOMAIN.COM)!, referer:
https://vmdr-linuxidm.unix.domain.com/ipa/ui/
>
> Ok, I think I've found the problem:
>
> ipa-replica-conncheck -c -m <master>
> Failed to connect to port 7389 tcp on 10.126.18.73
> PKI-CA: Directory Service port (7389): FAILED
> ERROR: Port check failed! Inaccessible port(s): 7389 (TCP)
>
>
> On the master, pki-tomcatd is showing as OK, although nmap -sT -O
> localhost doesn't show 7389 open.
>
> Where can I look next?
>
> ipa -version
> VERSION: 4.5.0, API_VERSION: 2.228
It shouldn't be even trying port 7389 with v4.5.0. Very old versions of
IPA used to use two separate 389-ds instances, one for the IPA data and
one for the CA data. They were combined long ago. This could just be a
check in case you had a very old master in which case this is a red
herring.
Ok - I'll ignore then.
cheers
L.
------
"The antidote to apocalypticism is *apocalyptic civics*. Apocalyptic civics
is the insistence that we cannot ignore the truth, nor should we panic
about it. It is a shared consciousness that our institutions have failed
and our ecosystem is collapsing, yet we are still here — and we are
creative agents who can shape our destinies. Apocalyptic civics is the
conviction that the only way out is through, and the only way through is
together. "
*Greg Bloom* @greggish
https://twitter.com/greggish/status/873177525903609857
6:55 p.m.
On 27 October 2017 at 10:32, Lachlan Musicman <datakid(a)gmail.com> wrote:
On 27 October 2017 at 07:38, Rob Crittenden
<rcritten(a)redhat.com> wrote:
> Lachlan Musicman via FreeIPA-users wrote:
> > > When I look at the ID Views in the interface, I get
an "IPA Error 903:
> > InternalError".
> See /var/log/httpd/error_log for details, there may be a
python backtrace.
Sure do!
[Thu Oct 26 12:57:25.413102 2017] [:error] [pid 1316] ipa: ERROR:
non-public: RuntimeError: Unable to load file /usr/share/ipa/smb.conf.empty
[Thu Oct 26 12:57:25.413118 2017] [:error] [pid 1316] Traceback (most
recent call last):
[Thu Oct 26 12:57:25.413121 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 367, in
wsgi_execute
[Thu Oct 26 12:57:25.413124 2017] [:error] [pid 1316] result =
command(*args, **options)
[Thu Oct 26 12:57:25.413126 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 447, in
__call__
[Thu Oct 26 12:57:25.413128 2017] [:error] [pid 1316] return
self.__do_call(*args, **options)
[Thu Oct 26 12:57:25.413130 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 475, in
__do_call
[Thu Oct 26 12:57:25.413133 2017] [:error] [pid 1316] ret =
self.run(*args, **options)
[Thu Oct 26 12:57:25.413135 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 797, in run
[Thu Oct 26 12:57:25.413137 2017] [:error] [pid 1316] return
self.execute(*args, **options)
[Thu Oct 26 12:57:25.413139 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/plugins/baseldap.py", line
2050, in execute
[Thu Oct 26 12:57:25.413141 2017] [:error] [pid 1316] truncated =
callback(self, ldap, entries, truncated, *args, **options)
[Thu Oct 26 12:57:25.413144 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/plugins/idviews.py", line
1123, in post_callback
[Thu Oct 26 12:57:25.413146 2017] [:error] [pid 1316] ldap, entries,
truncated, *args, **options)
[Thu Oct 26 12:57:25.413148 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/plugins/idviews.py", line
829, in post_callback
[Thu Oct 26 12:57:25.413151 2017] [:error] [pid 1316]
self.obj.convert_anchor_to_human_readable_form(entry, **options)
[Thu Oct 26 12:57:25.413153 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/plugins/idviews.py", line
733, in convert_anchor_to_human_readable_form
[Thu Oct 26 12:57:25.413156 2017] [:error] [pid 1316] anchor
[Thu Oct 26 12:57:25.413158 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/plugins/idviews.py", line
632, in resolve_anchor_to_object_name
[Thu Oct 26 12:57:25.413161 2017] [:error] [pid 1316] name =
domain_validator.get_trusted_domain_object_from_sid(sid)
[Thu Oct 26 12:57:25.413163 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/dcerpc.py", line 503, in
get_trusted_domain_object_from_sid
[Thu Oct 26 12:57:25.413165 2017] [:error] [pid 1316] attrs=attrs)
[Thu Oct 26 12:57:25.413167 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/dcerpc.py", line 380, in
get_trusted_domain_objects
[Thu Oct 26 12:57:25.413170 2017] [:error] [pid 1316] entries =
self.search_in_dc(domain, filter, attrs, scope, basedn)
[Thu Oct 26 12:57:25.413172 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/dcerpc.py", line 689, in
search_in_dc
[Thu Oct 26 12:57:25.413174 2017] [:error] [pid 1316] info =
self.__retrieve_trusted_domain_gc_list(domain)
[Thu Oct 26 12:57:25.413176 2017] [:error] [pid 1316] File
"/usr/lib/python2.7/site-packages/ipaserver/dcerpc.py", line 763, in
__retrieve_trusted_domain_gc_list
[Thu Oct 26 12:57:25.413179 2017] [:error] [pid 1316]
os.path.join(paths.USR_SHARE_IPA_DIR, "smb.conf.empty"))
[Thu Oct 26 12:57:25.413181 2017] [:error] [pid 1316] RuntimeError: Unable
to load file /usr/share/ipa/smb.conf.empty
> > [26/Oct/2017:12:31:23.454702287 +1100] - ERR -
set_krb5_creds - Could
> > not get initial credentials for principal
> > [ldap/vmdr-linuxidm.unix.domain.com(a)UNIX.DOMAIN.COM
> > <mailto:vmdr-linuxidm.unix.domain.com@UNIX.DOMAIN.COM>] in keytab
> > [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC for
> > requested realm)
> > > I can get `kinit admin` working fine. But there's
something wrong. I
> > don't know where to look exactly.
> KRB5_TRACE=/dev/stdout kinit admin
> See what KDC kinit is using. It should be using the local
box because
> masters should point only to themselves.
Yes, that command makes reference to it's own ip, eg: "Sending TCP request
to stream 10.126.18.129:88"
> > /var/log/httpd/error has this
> > > RuntimeError: Unable to load file
/usr/share/ipa/smb.conf.empty
> > > Which is interesting. There's no file
/usr/share/ipa/smb.conf.empty but
> > there is a /usr/share/ipa/smb.conf.template?
> Probably need more context.
I've only just realised this is the above error - when I go to ID
View->Default Trust View in the WebUI, I get the above python stacktrace,
but I also get
[Fri Oct 27 10:03:43.466674 2017] [:warn] [pid 5686] [client
10.126.160.47:53715] failed to set perms (3140) on file
(/var/run/ipa/ccaches/admin(a)UNIX.DOMAIN.COM)!, referer:
https://vmdr-linuxidm.unix.domain.com/ipa/ui/
I looked at the notes I made from the out put created when I set the
replica up. It makes reference to
https://vmdr-linuxidm.unix.domain.com/ipa/session/json. When I go to that
page in a browser, I see:
{"result": null, "version": "4.5.0", "error":
{"message": "Missing or
invalid HTTP Referer, missing", "code": 911, "data":
{"referer":
"missing"}, "name": "RefererError"}, "id": null,
"principal":
"admin(a)UNIX.DOMAIN.COM"}
cheers
L.
------
"The antidote to apocalypticism is *apocalyptic civics*. Apocalyptic civics
is the insistence that we cannot ignore the truth, nor should we panic
about it. It is a shared consciousness that our institutions have failed
and our ecosystem is collapsing, yet we are still here — and we are
creative agents who can shape our destinies. Apocalyptic civics is the
conviction that the only way out is through, and the only way through is
together. "
*Greg Bloom* @greggish
https://twitter.com/greggish/status/873177525903609857
7:04 p.m.
On 27 October 2017 at 07:38, Rob Crittenden <rcritten(a)redhat.com> wrote:
Lachlan Musicman via FreeIPA-users wrote:
>
> ipa -version
> VERSION: 4.5.0, API_VERSION: 2.228
It shouldn't be even trying port 7389 with v4.5.0. Very old versions of
IPA used to use two separate 389-ds instances, one for the IPA data and
one for the CA data. They were combined long ago. This could just be a
check in case you had a very old master in which case this is a red
herring.
I went back to take another look at the dirsrv logs after you said this,
and I saw something I didn't see yesterday. I notice that the cn has "meTo"
appended to the start of the master server name.
Is that meant to be that way, or have I mistyped in the wrong window at the
wrong time somewhere?
ERR - NSMMReplicationPlugin - bind_and_check_pwp - agmt="cn=
meTovmpr-linuxidm.unix.domain.com" (vmpr-linuxidm:389) - Replication bind
with GSSAPI auth failed: LDAP error -1 (Can't contact LDAP server) ()
Cheers
L.
------
"The antidote to apocalypticism is *apocalyptic civics*. Apocalyptic civics
is the insistence that we cannot ignore the truth, nor should we panic
about it. It is a shared consciousness that our institutions have failed
and our ecosystem is collapsing, yet we are still here — and we are
creative agents who can shape our destinies. Apocalyptic civics is the
conviction that the only way out is through, and the only way through is
together. "
*Greg Bloom* @greggish
https://twitter.com/greggish/status/873177525903609857
2367
days inactive
2368
days old
freeipa-users@lists.fedorahosted.org
4 comments
2 participants
participants (2)
-
Lachlan Musicman -
Rob Crittenden