looks like sshd is trying to read /home/ouruser/.ssh/authorized_keys and is stuck. Can you read this file from the command line? Is it e.g. on NFS which might not be properly mounted? Does it work if you skip pubkey authentication ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver bye, Sumit

On Tue, Feb 09, 2021 at 11:33:15AM -0500, Robert Kudyba via FreeIPA-users wrote: > > > > looks like sshd is trying to read /home/ouruser/.ssh/authorized_keys and > > is stuck. Can you read this file from the command line? Is it e.g. on > > NFS which might not be properly mounted? > > > > Does it work if you skip pubkey authentication > > > > ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver > > > > bye, > > Sumit > > > > Thanks for the suggestion. What happens is the NIS password works. The > FreeIPA password, which I update with: > ipa user-mod ouruser --setattr "userpassword=xxxx", fails with the below > errors/logs > > Feb 9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086. > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Set > /proc/self/oom_score_adj to 0 > Feb 9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5 > newsock 5 pipe 7 sock 8 > Feb 9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after > dupping: 4, 4 > Feb 9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port 53332 > on 150.108.64.156 port 22 rdomain "" > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Local version string > SSH-2.0-OpenSSH_8.4 > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version > 2.0, remote software version OpenSSH_8.4 > Feb 9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat > OpenSSH* compat 0x04000000 > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid: 74/74 > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types: > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT received > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm: > curve25519-sha256 [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm: > ecdsa-sha2-nistp256 [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server cipher: > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client cipher: > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256 > need=32 dh_need=32 [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256 > need=32 dh_need=32 [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: expecting > SSH2_MSG_KEX_ECDH_INIT [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: rekey out after 4294967296 > blocks [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: Sending SSH2_MSG_EXT_INFO > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: expecting SSH2_MSG_NEWKEYS > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS received > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: rekey in after 4294967296 > blocks [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user > ouruser service ssh-connection method none [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0 > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for > "ouruser" > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST to > "x.x.x.x" > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to > "ssh" > Feb 9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user > ouruser service ssh-connection method keyboard-interactive [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0 > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive devs > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge: > user=ouruser devs= [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam' > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start: > trying authentication method 'pam' [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive for > ouruser from x.x.x.x port 53332 ssh2 [preauth] > Feb 9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth): authentication > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser > Feb 9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): authentication > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser > Feb 9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for > user ouruser: 9 (Authentication service cannot retrieve authentication info) > Feb 9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication failure > for ouruser from x.x.x.x > Feb 9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam for > ouruser from x.x.x.x port 53332 ssh2 > Feb 9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for user > ouruser service ssh-connection method keyboard-interactive [preauth] > Feb 9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1 > [preauth] > Feb 9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive devs > [preauth] > Feb 9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge: > user=ouruser devs= [preauth] > Feb 9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam' > [preauth] > Feb 9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start: > trying authentication method 'pam' [preauth] > Feb 9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive for > ouruser from x.x.x.x port 53332 ssh2 [preauth] > > > Feb 9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086. > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Set > /proc/self/oom_score_adj to 0 > Feb 9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5 > newsock 5 pipe 7 sock 8 > Feb 9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after > dupping: 4, 4 > Feb 9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port 53332 > on 150.108.64.156 port 22 rdomain "" > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Local version string > SSH-2.0-OpenSSH_8.4 > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version > 2.0, remote software version OpenSSH_8.4 > Feb 9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat > OpenSSH* compat 0x04000000 > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid: 74/74 > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types: > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT received > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm: > curve25519-sha256 [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm: > ecdsa-sha2-nistp256 [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server cipher: > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client cipher: > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256 > need=32 dh_need=32 [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256 > need=32 dh_need=32 [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: expecting > SSH2_MSG_KEX_ECDH_INIT [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: rekey out after 4294967296 > blocks [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: Sending SSH2_MSG_EXT_INFO > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: expecting SSH2_MSG_NEWKEYS > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS received > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: rekey in after 4294967296 > blocks [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user > ouruser service ssh-connection method none [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0 > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for > "ouruser" > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST to > "x.x.x.x" > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to > "ssh" > Feb 9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user > ouruser service ssh-connection method keyboard-interactive [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0 > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive devs > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge: > user=ouruser devs= [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam' > [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start: > trying authentication method 'pam' [preauth] > Feb 9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive for > ouruser from x.x.x.x port 53332 ssh2 [preauth] > Feb 9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth): authentication > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser > Feb 9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): authentication > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser > Feb 9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for > user ouruser: 9 (Authentication service cannot retrieve authentication info) > Feb 9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication failure > for ouruser from x.x.x.x > Feb 9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam for > ouruser from x.x.x.x port 53332 ssh2 > Feb 9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for user > ouruser service ssh-connection method keyboard-interactive [preauth] > Feb 9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1 > [preauth] > Feb 9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive devs > [preauth] > Feb 9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge: > user=ouruser devs= [preauth] > Feb 9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam' > [preauth] > Feb 9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start: > trying authentication method 'pam' [preauth] > Feb 9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive for > ouruser from x.x.x.x port 53332 ssh2 [preauth] > > With the NIS password the logs show this: Hi, did you drop what happened before or is this the only debug output for the NIS password?

Feb 9 11:16:57 debug1: do_pam_account: called > Feb 9 11:16:57 ourserver sshd[536226]: debug1: PAM: num PAM env strings 2 > Feb 9 11:16:57 ourserver sshd[536226]: Postponed keyboard-interactive/pam > for cai from 150.108.68.26 port 53646 ssh2 [preauth] > Feb 9 11:16:57 ourserver sshd[536226]: debug1: do_pam_account: called > Feb 9 11:16:57 ourserver sshd[536226]: Accepted keyboard-interactive/pam > for cai from 150.108.68.26 port 53646 ssh2 > Feb 9 11:16:57 ourserver sshd[536226]: debug1: monitor_child_preauth: cai > has been authenticated by privileged process > Feb 9 11:16:57 ourserver sshd[536226]: debug1: monitor_read_log: child log > fd closed > Feb 9 11:16:57 ourserver sshd[536226]: debug1: audit_event: unhandled > event 2 > Feb 9 11:16:57 ourserver sshd[536226]: debug1: temporarily_use_uid: > 5879/200 (e=0/0) > Feb 9 11:16:57 ourserver sshd[536226]: debug1: ssh_gssapi_storecreds: Not > a GSSAPI mechanism > Feb 9 11:16:57 ourserver sshd[536226]: debug1: restore_uid: 0/0 > Feb 9 11:16:57 ourserver sshd[536226]: debug1: SELinux support disabled > Feb 9 11:16:57 ourserver sshd[536226]: debug1: PAM: establishing > credentials > Feb 9 11:16:57 ourserver systemd[536237]: pam_unix(systemd-user:session): > session opened for user cai(uid=5879) by (uid=0) > > What options should be set in /etc/ssh/sshd_config? Is sssd necessary for > this to work with the FreeIPA password

I tried this on another test server, and configured NIS for the users, which are different. Same issue. All the verbose output adds a lot of log noise but I'm hoping it provides a clue. ipactl status Directory Service: RUNNING krb5kdc Service: RUNNING kadmin Service: RUNNING named Service: RUNNING httpd Service: RUNNING ipa-custodia Service: RUNNING pki-tomcatd Service: RUNNING ipa-otpd Service: RUNNING ipa-dnskeysyncd Service: RUNNING ipa: INFO: The ipactl command was successful here is ssh -o PubkeyAuthentication=no -vvv -k ouruser@ourserver OpenSSH_8.4p1, OpenSSL 1.1.1i FIPS  8 Dec 2020 debug1: Reading configuration data /etc/ssh/ssh_config debug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0 debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf debug2: checking match for 'final all' host ourserver originally ourserver debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final' debug2: match not found debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only) debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-] debug3: kex names ok: [curve25519-sha256,curve25519-sha256(a)libssh.org <mailto:curve25519-sha256@libssh.org>,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512] debug1: configuration requests final Match pass debug1: re-parsing configuration debug1: Reading configuration data /etc/ssh/ssh_config debug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0 debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf debug2: checking match for 'final all' host ourserver originally ourserver debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final' debug2: match found debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-] debug3: kex names ok: [curve25519-sha256,curve25519-sha256(a)libssh.org <mailto:curve25519-sha256@libssh.org>,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512] debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts' debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2' debug1: Executing proxy command: exec /usr/bin/sss_ssh_knownhostsproxy -p 22 ourserver debug1: identity file /root/.ssh/id_rsa type 0 debug1: identity file /root/.ssh/id_rsa-cert type -1 debug1: identity file /root/.ssh/id_dsa type -1 debug1: identity file /root/.ssh/id_dsa-cert type -1 debug1: identity file /root/.ssh/id_ecdsa type -1 debug1: identity file /root/.ssh/id_ecdsa-cert type -1 debug1: identity file /root/.ssh/id_ecdsa_sk type -1 debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1 debug1: identity file /root/.ssh/id_ed25519 type -1 debug1: identity file /root/.ssh/id_ed25519-cert type -1 debug1: identity file /root/.ssh/id_ed25519_sk type -1 debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1 debug1: identity file /root/.ssh/id_xmss type -1 debug1: identity file /root/.ssh/id_xmss-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_8.4 debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4 debug1: match: OpenSSH_8.4 pat OpenSSH* compat 0x04000000 debug2: fd 5 setting O_NONBLOCK debug2: fd 4 setting O_NONBLOCK debug1: Authenticating to ourserver:22 as 'ouruser' debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts" debug3: record_hostkey: found key type ECDSA in file /root/.ssh/known_hosts:31 debug3: load_hostkeys: loaded 1 keys from ourserver debug3: hostkeys_foreach: reading file "/var/lib/sss/pubconf/known_hosts" debug3: order_hostkeyalgs: have matching best-preference key type ecdsa-sha2-nistp256-cert-v01(a)openssh.com <mailto:ecdsa-sha2-nistp256-cert-v01@openssh.com>, using HostkeyAlgorithms verbatim debug3: send packet: type 20 debug1: SSH2_MSG_KEXINIT sent debug3: receive packet: type 20 debug1: SSH2_MSG_KEXINIT received debug2: local client KEXINIT proposal debug2: KEX algorithms: curve25519-sha256,curve25519-sha256(a)libssh.org <mailto:curve25519-sha256@libssh.org>,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,ext-info-c debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01(a)openssh.com <mailto:ecdsa-sha2-nistp256-cert-v01@openssh.com>,ecdsa-sha2-nistp384-cert-v01@openssh.com <mailto:ecdsa-sha2-nistp384-cert-v01@openssh.com>,ecdsa-sha2-nistp521-cert-v01@openssh.com <mailto:ecdsa-sha2-nistp521-cert-v01@openssh.com>,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com <mailto:sk-ecdsa-sha2-nistp256-cert-v01@openssh.com>,ssh-ed25519-cert-v01@openssh.com <mailto:ssh-ed25519-cert-v01@openssh.com>,sk-ssh-ed25519-cert-v01@openssh.com <mailto:sk-ssh-ed25519-cert-v01@openssh.com>,rsa-sha2-512-cert-v01@openssh.com <mailto:rsa-sha2-512-cert-v01@openssh.com>,rsa-sha2-256-cert-v01@openssh.com <mailto:rsa-sha2-256-cert-v01@openssh.com>,ssh-rsa-cert-v01@openssh.com <mailto:ssh-rsa-cert-v01@openssh.com>,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com <mailto:sk-ecdsa-sha2-nistp256@openssh.com>,ssh-ed25519,sk-ssh-ed25519@openssh.com <mailto:sk-ssh-ed25519@openssh.com>,rsa-sha2-512,rsa-sha2-256,ssh-rsa debug2: ciphers ctos: aes256-gcm(a)openssh.com <mailto:aes256-gcm@openssh.com>,chacha20-poly1305@openssh.com <mailto:chacha20-poly1305@openssh.com>,aes256-ctr,aes128-gcm@openssh.com <mailto:aes128-gcm@openssh.com>,aes128-ctr debug2: ciphers stoc: aes256-gcm(a)openssh.com <mailto:aes256-gcm@openssh.com>,chacha20-poly1305@openssh.com <mailto:chacha20-poly1305@openssh.com>,aes256-ctr,aes128-gcm@openssh.com <mailto:aes128-gcm@openssh.com>,aes128-ctr debug2: MACs ctos: hmac-sha2-256-etm(a)openssh.com <mailto:hmac-sha2-256-etm@openssh.com>,hmac-sha1-etm@openssh.com <mailto:hmac-sha1-etm@openssh.com>,umac-128-etm@openssh.com <mailto:umac-128-etm@openssh.com>,hmac-sha2-512-etm@openssh.com <mailto:hmac-sha2-512-etm@openssh.com>,hmac-sha2-256,hmac-sha1,umac-128@openssh.com <mailto:umac-128@openssh.com>,hmac-sha2-512 debug2: MACs stoc: hmac-sha2-256-etm(a)openssh.com <mailto:hmac-sha2-256-etm@openssh.com>,hmac-sha1-etm@openssh.com <mailto:hmac-sha1-etm@openssh.com>,umac-128-etm@openssh.com <mailto:umac-128-etm@openssh.com>,hmac-sha2-512-etm@openssh.com <mailto:hmac-sha2-512-etm@openssh.com>,hmac-sha2-256,hmac-sha1,umac-128@openssh.com <mailto:umac-128@openssh.com>,hmac-sha2-512 debug2: compression ctos: none,zlib(a)openssh.com <mailto:zlib@openssh.com>,zlib debug2: compression stoc: none,zlib(a)openssh.com <mailto:zlib@openssh.com>,zlib debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug2: peer server KEXINIT proposal debug2: KEX algorithms: curve25519-sha256,curve25519-sha256(a)libssh.org <mailto:curve25519-sha256@libssh.org>,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256 debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 debug2: ciphers ctos: chacha20-poly1305(a)openssh.com <mailto:chacha20-poly1305@openssh.com>,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com <mailto:aes128-gcm@openssh.com>,aes256-gcm@openssh.com <mailto:aes256-gcm@openssh.com> debug2: ciphers stoc: chacha20-poly1305(a)openssh.com <mailto:chacha20-poly1305@openssh.com>,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com <mailto:aes128-gcm@openssh.com>,aes256-gcm@openssh.com <mailto:aes256-gcm@openssh.com> debug2: MACs ctos: umac-64-etm(a)openssh.com <mailto:umac-64-etm@openssh.com>,umac-128-etm@openssh.com <mailto:umac-128-etm@openssh.com>,hmac-sha2-256-etm@openssh.com <mailto:hmac-sha2-256-etm@openssh.com>,hmac-sha2-512-etm@openssh.com <mailto:hmac-sha2-512-etm@openssh.com>,hmac-sha1-etm@openssh.com <mailto:hmac-sha1-etm@openssh.com>,umac-64@openssh.com <mailto:umac-64@openssh.com>,umac-128@openssh.com <mailto:umac-128@openssh.com>,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: MACs stoc: umac-64-etm(a)openssh.com <mailto:umac-64-etm@openssh.com>,umac-128-etm@openssh.com <mailto:umac-128-etm@openssh.com>,hmac-sha2-256-etm@openssh.com <mailto:hmac-sha2-256-etm@openssh.com>,hmac-sha2-512-etm@openssh.com <mailto:hmac-sha2-512-etm@openssh.com>,hmac-sha1-etm@openssh.com <mailto:hmac-sha1-etm@openssh.com>,umac-64@openssh.com <mailto:umac-64@openssh.com>,umac-128@openssh.com <mailto:umac-128@openssh.com>,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: compression ctos: none,zlib(a)openssh.com <mailto:zlib@openssh.com> debug2: compression stoc: none,zlib(a)openssh.com <mailto:zlib@openssh.com> debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug1: kex: algorithm: curve25519-sha256 debug1: kex: host key algorithm: ecdsa-sha2-nistp256 debug1: kex: server->client cipher: aes256-gcm(a)openssh.com <mailto:aes256-gcm@openssh.com> MAC: <implicit> compression: none debug1: kex: client->server cipher: aes256-gcm(a)openssh.com <mailto:aes256-gcm@openssh.com> MAC: <implicit> compression: none debug1: kex: curve25519-sha256 need=32 dh_need=32 debug1: kex: curve25519-sha256 need=32 dh_need=32 debug3: send packet: type 30 debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug3: receive packet: type 31 debug1: Server host key: ecdsa-sha2-nistp256 SHA256:yVm8V2ODZo0nAuvr9k2ydTJv0RtOgkl8Sp5Mkmp/F0M debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts" debug3: record_hostkey: found key type ECDSA in file /root/.ssh/known_hosts:31 debug3: load_hostkeys: loaded 1 keys from ourserver debug3: hostkeys_foreach: reading file "/var/lib/sss/pubconf/known_hosts" debug1: Host 'ourserver' is known and matches the ECDSA host key. debug1: Found key in /root/.ssh/known_hosts:31 debug3: send packet: type 21 debug2: set_newkeys: mode 1 debug1: rekey out after 4294967296 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug3: receive packet: type 21 debug1: SSH2_MSG_NEWKEYS received debug2: set_newkeys: mode 0 debug1: rekey in after 4294967296 blocks debug1: Will attempt key: /root/.ssh/id_rsa RSA SHA256:Sena4XB1wVt7x+o55Y9EI5WnQIyZ/SLFk+t6tmBFMYA debug1: Will attempt key: /root/.ssh/id_dsa debug1: Will attempt key: /root/.ssh/id_ecdsa debug1: Will attempt key: /root/.ssh/id_ecdsa_sk debug1: Will attempt key: /root/.ssh/id_ed25519 debug1: Will attempt key: /root/.ssh/id_ed25519_sk debug1: Will attempt key: /root/.ssh/id_xmss debug2: pubkey_prepare: done debug3: send packet: type 5 debug3: receive packet: type 7 debug1: SSH2_MSG_EXT_INFO received debug1: kex_input_ext_info: server-sig-algs=&lt;ssh-ed25519,sk-ssh-ed25519(a)openssh.com <mailto:sk-ssh-ed25519@openssh.com>,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com <mailto:sk-ecdsa-sha2-nistp256@openssh.com>,webauthn-sk-ecdsa-sha2-nistp256@openssh.com <mailto:webauthn-sk-ecdsa-sha2-nistp256@openssh.com>> debug3: receive packet: type 6 debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug3: send packet: type 50 debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive debug3: preferred gssapi-with-mic,keyboard-interactive,password debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug1: Unspecified GSS failure.  Minor code may provide more information Server host/ourserver.edu(a)ourserver.edu <mailto:ourserver.edu@ourserver.edu> not found in Kerberos database debug3: send packet: type 50 debug2: we sent a gssapi-with-mic packet, wait for reply debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive debug2: we did not send a packet, disable method debug3: authmethod_lookup keyboard-interactive debug3: remaining preferred: password debug3: authmethod_is_enabled keyboard-interactive debug1: Next authentication method: keyboard-interactive debug2: userauth_kbdint debug3: send packet: type 50 debug2: we sent a keyboard-interactive packet, wait for reply debug3: receive packet: type 60 debug2: input_userauth_info_req debug2: input_userauth_info_req: num_prompts 1 And here are the server logs from /var/log secure and you can see sssd is being used: Feb 10 14:36:24 ourserver sshd[3024290]: debug1: Forked child 3084339. Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Set /proc/self/oom_score_adj to 0 Feb 10 14:36:24 ourserver sshd[3084339]: debug1: rexec start in 4 out 4 newsock 4 pipe 6 sock 7 Feb 10 14:36:24 ourserver sshd[3084339]: debug1: inetd sockets after dupping: 4, 4 Feb 10 14:36:24 ourserver sshd[3084339]: Connection from x.x.x.x port 34160 on 150.108.68.128 port 22 rdomain "" Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Local version string SSH-2.0-OpenSSH_8.4 Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4 Feb 10 14:36:24 ourserver sshd[3084339]: debug1: match: OpenSSH_8.4 pat OpenSSH* compat 0x04000000 Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SELinux support disabled [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: permanently_set_uid: 74/74 [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: list_hostkey_types: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_KEXINIT sent [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_KEXINIT received [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: algorithm: curve25519-sha256 [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: host key algorithm: ecdsa-sha2-nistp256 [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: client->server cipher: aes256-gcm(a)openssh.com <mailto:aes256-gcm@openssh.com> MAC: <implicit> compression: none [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: server->client cipher: aes256-gcm(a)openssh.com <mailto:aes256-gcm@openssh.com> MAC: <implicit> compression: none [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: curve25519-sha256 need=32 dh_need=32 [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kex: curve25519-sha256 need=32 dh_need=32 [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: expecting SSH2_MSG_KEX_ECDH_INIT [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: rekey out after 4294967296 blocks [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_NEWKEYS sent [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: Sending SSH2_MSG_EXT_INFO [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: expecting SSH2_MSG_NEWKEYS [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: SSH2_MSG_NEWKEYS received [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: rekey in after 4294967296 blocks [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: KEX done [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: userauth-request for user ouruser service ssh-connection method none [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: attempt 0 failures 0 [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: connection from x.x.x.x matched 'Address 192.168.0.*,127.0.0.1,10.10.1.*' at line 158 Feb 10 14:36:24 ourserver sshd[3084339]: debug1: PAM: initializing for "ouruser" Feb 10 14:36:24 ourserver sshd[3084339]: debug1: PAM: setting PAM_RHOST to "x.x.x.x" Feb 10 14:36:24 ourserver sshd[3084339]: debug1: PAM: setting PAM_TTY to "ssh" Feb 10 14:36:24 ourserver sshd[3084339]: debug1: userauth-request for user ouruser service ssh-connection method gssapi-with-mic [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: attempt 1 failures 0 [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: userauth-request for user ouruser service ssh-connection method keyboard-interactive [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: attempt 2 failures 0 [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: keyboard-interactive devs  [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: auth2_challenge: user=ouruser devs= [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: kbdint_alloc: devices 'pam' [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: debug1: auth2_challenge_start: trying authentication method 'pam' [preauth] Feb 10 14:36:24 ourserver sshd[3084339]: Postponed keyboard-interactive for ouruser from x.x.x.x port 34160 ssh2 [preauth] Feb 10 14:36:28 ourserver sshd[3084344]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser Feb 10 14:36:28 ourserver proxy_child: pam_unix(sssd-shadowutils:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth): received for user ouruser: 7 (Authentication failure) Feb 10 14:36:33 ourserver sshd[3084339]: error: PAM: Authentication failure for ouruser from x.x.x.x Feb 10 14:36:33 ourserver sshd[3084339]: Failed keyboard-interactive/pam for ouruser from x.x.x.x port 34160 ssh2 Feb 10 14:36:33 ourserver sshd[3084339]: debug1: userauth-request for user ouruser service ssh-connection method keyboard-interactive [preauth] Feb 10 14:36:33 ourserver sshd[3084339]: debug1: attempt 3 failures 1 [preauth] Feb 10 14:36:33 ourserver sshd[3084339]: debug1: keyboard-interactive devs  [preauth] Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge: user=ouruser devs= [preauth] Feb 10 14:36:33 ourserver sshd[3084339]: debug1: kbdint_alloc: devices 'pam' [preauth] Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge_start: trying authentication method 'pam' [preauth] Feb 10 14:36:33 ourserver sshd[3084339]: Postponed keyboard-interactive for ouruser from x.x.x.x port 34160 ssh2 [preauth] I verified the FreeIPA password in both the GUI and via ipa user-mod. The only time the user is able to log in is using the NIS password. ldapsearch -x -D and kinit username work successfully. klist displays the user details correctly. I can see that the installation script edits /etc/ssh/sshd_config with: Include /etc/ssh/sshd_config.d/04-ipa.conf which has: PubkeyAuthentication yes KerberosAuthentication no GSSAPIAuthentication yes UsePAM yes ChallengeResponseAuthentication yes AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys AuthorizedKeysCommandUser nobody When the NIS password is used successfully here are the server logs: Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for user ouruser service ssh-connection method none [preauth] Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 0 failures 0 [preauth] Feb 10 14:56:39 ourserver sshd[3085147]: debug1: connection from x.x.x.x matched 'Address 192.168.0.*,127.0.0.1,10.10.1.*' at line 158 Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: initializing for "ouruser" Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_RHOST to "x.x.x.x" Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_TTY to "ssh" Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for user ouruser service ssh-connection method gssapi-with-mic [preauth] Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 1 failures 0 [preauth] Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for user ouruser service ssh-connection method keyboard-interactive [preauth] Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 2 failures 0 [preauth] Feb 10 14:56:39 ourserver sshd[3085147]: debug1: keyboard-interactive devs  [preauth] Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge: user=ouruser devs= [preauth] Feb 10 14:56:39 ourserver sshd[3085147]: debug1: kbdint_alloc: devices 'pam' [preauth] Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge_start: trying authentication method 'pam' [preauth] Feb 10 14:56:39 ourserver sshd[3085147]: Postponed keyboard-interactive for ouruser from x.x.x.x port 35046 ssh2 [preauth] Feb 10 14:56:42 ourserver sshd[3085152]: debug1: do_pam_account: called Feb 10 14:56:42 ourserver sshd[3085147]: debug1: PAM: num PAM env strings 2 Feb 10 14:56:42 ourserver sshd[3085147]: Postponed keyboard-interactive/pam for ouruser from x.x.x.x port 35046 ssh2 [preauth] Feb 10 14:56:42 ourserver sshd[3085147]: debug1: do_pam_account: called I do see the error that sticks out is " Server host/ourserver.edu(a)ourserver.edu <mailto:ourserver.edu@ourserver.edu> not found in Kerberos database" but we have students that log in from all over the world so do all clients need to be added? iptables, firewalld, and nftables are off and disabled. No hbac rules:

ipa hbacrule-find -------------------- 2 HBAC rules matched --------------------   Rule name: allow_all   User category: all   Host category: all   Service category: all   Description: Allow all users to access any host from any host   Enabled: TRUE   Rule name: allow_systemd-user   User category: all   Host category: all   Description: Allow pam_systemd to run user@.service to create a system user session   Enabled: TRUE ---------------------------- Number of entries returned 2 Am I missing something obvious to regulars? On Tue, Feb 9, 2021 at 12:34 PM Robert Kudyba <rkudyba(a)fordham.edu <mailto:rkudyba@fordham.edu>> wrote: On Tue, Feb 9, 2021 at 12:20 PM Sumit Bose via FreeIPA-users <freeipa-users(a)lists.fedorahosted.org <mailto:freeipa-users@lists.fedorahosted.org>> wrote: On Tue, Feb 09, 2021 at 11:33:15AM -0500, Robert Kudyba via FreeIPA-users wrote: > > > > looks like sshd is trying to read /home/ouruser/.ssh/authorized_keys and > > is stuck. Can you read this file from the command line? Is it e.g. on > > NFS which might not be properly mounted? > > > > Does it work if you skip pubkey authentication > > > >     ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver > > > > bye, > > Sumit > > > > Thanks for the suggestion. What happens is the NIS password works. The > FreeIPA password, which I update with: > ipa user-mod ouruser --setattr "userpassword=xxxx", fails with the below > errors/logs > > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086. > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set > /proc/self/oom_score_adj to 0 > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5 > newsock 5 pipe 7 sock 8 > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after > dupping: 4, 4 > Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port 53332 > on 150.108.64.156 port 22 rdomain "" > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string > SSH-2.0-OpenSSH_8.4 > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version > 2.0, remote software version OpenSSH_8.4 > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat > OpenSSH* compat 0x04000000 > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid: 74/74 > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types: > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT received > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm: > curve25519-sha256 [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm: > ecdsa-sha2-nistp256 [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server cipher: > aes256-gcm(a)openssh.com <mailto:aes256-gcm@openssh.com> MAC: <implicit> compression: none [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client cipher: > aes256-gcm(a)openssh.com <mailto:aes256-gcm@openssh.com> MAC: <implicit> compression: none [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256 > need=32 dh_need=32 [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256 > need=32 dh_need=32 [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting > SSH2_MSG_KEX_ECDH_INIT [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after 4294967296 > blocks [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending SSH2_MSG_EXT_INFO > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting SSH2_MSG_NEWKEYS > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS received > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after 4294967296 > blocks [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user > ouruser service ssh-connection method none [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0 > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for > "ouruser" > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST to > "x.x.x.x" > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to > "ssh" > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user > ouruser service ssh-connection method keyboard-interactive [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0 > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive devs >  [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge: > user=ouruser devs= [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam' > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start: > trying authentication method 'pam' [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive for > ouruser from x.x.x.x port 53332 ssh2 [preauth] > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth): authentication > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): authentication > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for > user ouruser: 9 (Authentication service cannot retrieve authentication info) > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication failure > for ouruser from x.x.x.x > Feb  9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam for > ouruser from x.x.x.x port 53332 ssh2 > Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for user > ouruser service ssh-connection method keyboard-interactive [preauth] > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1 > [preauth] > Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive devs >  [preauth] > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge: > user=ouruser devs= [preauth] > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam' > [preauth] > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start: > trying authentication method 'pam' [preauth] > Feb  9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive for > ouruser from x.x.x.x port 53332 ssh2 [preauth] > > > Feb  9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086. > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Set > /proc/self/oom_score_adj to 0 > Feb  9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5 > newsock 5 pipe 7 sock 8 > Feb  9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after > dupping: 4, 4 > Feb  9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port 53332 > on 150.108.64.156 port 22 rdomain "" > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Local version string > SSH-2.0-OpenSSH_8.4 > Feb  9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version > 2.0, remote software version OpenSSH_8.4 > Feb  9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat > OpenSSH* compat 0x04000000 > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid: 74/74 > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types: > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT received > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm: > curve25519-sha256 [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm: > ecdsa-sha2-nistp256 [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server cipher: > aes256-gcm(a)openssh.com <mailto:aes256-gcm@openssh.com> MAC: <implicit> compression: none [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client cipher: > aes256-gcm(a)openssh.com <mailto:aes256-gcm@openssh.com> MAC: <implicit> compression: none [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256 > need=32 dh_need=32 [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256 > need=32 dh_need=32 [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting > SSH2_MSG_KEX_ECDH_INIT [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey out after 4294967296 > blocks [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: Sending SSH2_MSG_EXT_INFO > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: expecting SSH2_MSG_NEWKEYS > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS received > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: rekey in after 4294967296 > blocks [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user > ouruser service ssh-connection method none [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0 > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for > "ouruser" > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST to > "x.x.x.x" > Feb  9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to > "ssh" > Feb  9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for user > ouruser service ssh-connection method keyboard-interactive [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0 > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive devs >  [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge: > user=ouruser devs= [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam' > [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start: > trying authentication method 'pam' [preauth] > Feb  9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive for > ouruser from x.x.x.x port 53332 ssh2 [preauth] > Feb  9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth): authentication > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x  user=ouruser > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): authentication > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser > Feb  9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for > user ouruser: 9 (Authentication service cannot retrieve authentication info) > Feb  9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication failure > for ouruser from x.x.x.x > Feb  9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam for > ouruser from x.x.x.x port 53332 ssh2 > Feb  9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for user > ouruser service ssh-connection method keyboard-interactive [preauth] > Feb  9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1 > [preauth] > Feb  9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive devs >  [preauth] > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge: > user=ouruser devs= [preauth] > Feb  9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices 'pam' > [preauth] > Feb  9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start: > trying authentication method 'pam' [preauth] > Feb  9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive for > ouruser from x.x.x.x port 53332 ssh2 [preauth] > > With the NIS password the logs show this: Hi, did you drop what happened before or is this the only debug output for the NIS password? The below here are just logs from /var/log/secure for the user that  successfully logs in with his/her NIS password. By "drop what happened before" do you mean the original log snip? Yes I removed those in an attempt to shorten the message content. > Feb  9 11:16:57 debug1: do_pam_account: called > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: num PAM env strings 2 > Feb  9 11:16:57 ourserver sshd[536226]: Postponed keyboard-interactive/pam > for cai from 150.108.68.26 port 53646 ssh2 [preauth] > Feb  9 11:16:57 ourserver sshd[536226]: debug1: do_pam_account: called > Feb  9 11:16:57 ourserver sshd[536226]: Accepted keyboard-interactive/pam > for cai from 150.108.68.26 port 53646 ssh2 > Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_child_preauth: cai > has been authenticated by privileged process > Feb  9 11:16:57 ourserver sshd[536226]: debug1: monitor_read_log: child log > fd closed > Feb  9 11:16:57 ourserver sshd[536226]: debug1: audit_event: unhandled > event 2 > Feb  9 11:16:57 ourserver sshd[536226]: debug1: temporarily_use_uid: > 5879/200 (e=0/0) > Feb  9 11:16:57 ourserver sshd[536226]: debug1: ssh_gssapi_storecreds: Not > a GSSAPI mechanism > Feb  9 11:16:57 ourserver sshd[536226]: debug1: restore_uid: 0/0 > Feb  9 11:16:57 ourserver sshd[536226]: debug1: SELinux support disabled > Feb  9 11:16:57 ourserver sshd[536226]: debug1: PAM: establishing > credentials > Feb  9 11:16:57 ourserver systemd[536237]: pam_unix(systemd-user:session): > session opened for user cai(uid=5879) by (uid=0) > > What options should be set in /etc/ssh/sshd_config? Is sssd necessary for > this to work with the FreeIPA password   Yes, SSSD must be configured and runnnig. ssd does appear to be working fine and in /etc/ipa/ca.crt and the service is running correctly: [domain/ourdomain.edu <http://ourdomain.edu/>] id_provider = ipa ipa_server_mode = True ipa_server = ourdomain.edu <http://ourdomain.edu/> ipa_domain = ourdomain.edu <http://ourdomain.edu/> ipa_hostname = ourdomain.edu <http://ourdomain.edu/> auth_provider = ipa chpass_provider = ipa access_provider = ipa cache_credentials = True ldap_tls_cacert = /etc/ipa/ca.crt krb5_store_password_if_offline = True [sssd] services = nss, pam, ifp, ssh, sudo domains = ourdomain.edu <http://ourdomain.edu/> [nss] homedir_substring = /home memcache_timeout = 600 [ifp] allowed_uids = ipaapi, root systemctl status sssd * sssd.service - System Security Services Daemon      Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled; vendor preset: enabled)      Active: active (running) since Fri 2021-01-29 14:31:34 EST; 1 weeks 3 days ago   _______________________________________________ FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...

What is ourserver.edu? In order to log in using Kerberos/GSSAPI then the machine acting as the server needs to be enrolled as an IPA client so it has a keytab. rob

SSSD'S 'proxy_child' should be only be called if the proxy provider is configured? bye, Sumit

> Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth): authentication > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser > Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth): received for > user ouruser: 7 (Authentication failure) > Feb 10 14:36:33 ourserver sshd[3084339]: error: PAM: Authentication failure > for ouruser from x.x.x.x > Feb 10 14:36:33 ourserver sshd[3084339]: Failed keyboard-interactive/pam > for ouruser from x.x.x.x port 34160 ssh2 > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: userauth-request for user > ouruser service ssh-connection method keyboard-interactive [preauth] > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: attempt 3 failures 1 > [preauth] > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: keyboard-interactive devs > [preauth] > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge: > user=ouruser devs= [preauth] > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: kbdint_alloc: devices > 'pam' [preauth] > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge_start: > trying authentication method 'pam' [preauth] > Feb 10 14:36:33 ourserver sshd[3084339]: Postponed keyboard-interactive for > ouruser from x.x.x.x port 34160 ssh2 [preauth] > > I verified the FreeIPA password in both the GUI and via ipa user-mod. The > only time the user is able to log in is using the NIS password. ldapsearch > -x -D and kinit username work successfully. klist displays the user details > correctly. > > I can see that the installation script edits /etc/ssh/sshd_config with: > Include /etc/ssh/sshd_config.d/04-ipa.conf > > which has: > PubkeyAuthentication yes > KerberosAuthentication no > GSSAPIAuthentication yes > UsePAM yes > ChallengeResponseAuthentication yes > AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys > AuthorizedKeysCommandUser nobody > > When the NIS password is used successfully here are the server logs: > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for user > ouruser service ssh-connection method none [preauth] > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 0 failures 0 > [preauth] > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: connection from x.x.x.x > matched 'Address 192.168.0.*,127.0.0.1,10.10.1.*' at line 158 > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: initializing for > "ouruser" > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_RHOST to > "x.x.x.x" > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_TTY to > "ssh" > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for user > ouruser service ssh-connection method gssapi-with-mic [preauth] > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 1 failures 0 > [preauth] > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for user > ouruser service ssh-connection method keyboard-interactive [preauth] > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 2 failures 0 > [preauth] > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: keyboard-interactive devs > [preauth] > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge: > user=ouruser devs= [preauth] > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: kbdint_alloc: devices > 'pam' [preauth] > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge_start: > trying authentication method 'pam' [preauth] > Feb 10 14:56:39 ourserver sshd[3085147]: Postponed keyboard-interactive for > ouruser from x.x.x.x port 35046 ssh2 [preauth] > Feb 10 14:56:42 ourserver sshd[3085152]: debug1: do_pam_account: called > Feb 10 14:56:42 ourserver sshd[3085147]: debug1: PAM: num PAM env strings 2 > Feb 10 14:56:42 ourserver sshd[3085147]: Postponed keyboard-interactive/pam > for ouruser from x.x.x.x port 35046 ssh2 [preauth] > Feb 10 14:56:42 ourserver sshd[3085147]: debug1: do_pam_account: called > > I do see the error that sticks out is " Server host/ > ourserver.edu(a)ourserver.edu not found in Kerberos database" but we have > students that log in from all over the world so do all clients need to be > added? iptables, firewalld, and nftables are off and disabled. No hbac > rules: > ipa hbacrule-find > -------------------- > 2 HBAC rules matched > -------------------- > Rule name: allow_all > User category: all > Host category: all > Service category: all > Description: Allow all users to access any host from any host > Enabled: TRUE > > Rule name: allow_systemd-user > User category: all > Host category: all > Description: Allow pam_systemd to run user@.service to create a system > user session > Enabled: TRUE > ---------------------------- > Number of entries returned 2 > > Am I missing something obvious to regulars? > > > On Tue, Feb 9, 2021 at 12:34 PM Robert Kudyba <rkudyba(a)fordham.edu&gt; wrote: > > > On Tue, Feb 9, 2021 at 12:20 PM Sumit Bose via FreeIPA-users < > > freeipa-users(a)lists.fedorahosted.org&gt; wrote: > > > >> On Tue, Feb 09, 2021 at 11:33:15AM -0500, Robert Kudyba via FreeIPA-users > >> wrote: > >> > > > >> > > looks like sshd is trying to read /home/ouruser/.ssh/authorized_keys > >> and > >> > > is stuck. Can you read this file from the command line? Is it e.g. on > >> > > NFS which might not be properly mounted? > >> > > > >> > > Does it work if you skip pubkey authentication > >> > > > >> > > ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver > >> > > > >> > > bye, > >> > > Sumit > >> > > > >> > > >> > Thanks for the suggestion. What happens is the NIS password works. The > >> > FreeIPA password, which I update with: > >> > ipa user-mod ouruser --setattr "userpassword=xxxx", fails with the below > >> > errors/logs > >> > > >> > Feb 9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086. > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Set > >> > /proc/self/oom_score_adj to 0 > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5 > >> > newsock 5 pipe 7 sock 8 > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after > >> > dupping: 4, 4 > >> > Feb 9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port > >> 53332 > >> > on 150.108.64.156 port 22 rdomain "" > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Local version string > >> > SSH-2.0-OpenSSH_8.4 > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version > >> > 2.0, remote software version OpenSSH_8.4 > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat > >> > OpenSSH* compat 0x04000000 > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid: > >> 74/74 > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types: > >> > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 > >> [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT > >> received > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm: > >> > curve25519-sha256 [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm: > >> > ecdsa-sha2-nistp256 [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server > >> cipher: > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client > >> cipher: > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256 > >> > need=32 dh_need=32 [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256 > >> > need=32 dh_need=32 [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: expecting > >> > SSH2_MSG_KEX_ECDH_INIT [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: rekey out after > >> 4294967296 > >> > blocks [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: Sending > >> SSH2_MSG_EXT_INFO > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: expecting > >> SSH2_MSG_NEWKEYS > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS > >> received > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: rekey in after > >> 4294967296 > >> > blocks [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for > >> user > >> > ouruser service ssh-connection method none [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0 > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for > >> > "ouruser" > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST > >> to > >> > "x.x.x.x" > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to > >> > "ssh" > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for > >> user > >> > ouruser service ssh-connection method keyboard-interactive [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0 > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive > >> devs > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge: > >> > user=ouruser devs= [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices > >> 'pam' > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start: > >> > trying authentication method 'pam' [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive > >> for > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth] > >> > Feb 9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth): > >> authentication > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x > >> user=ouruser > >> > Feb 9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): > >> authentication > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser > >> > Feb 9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for > >> > user ouruser: 9 (Authentication service cannot retrieve authentication > >> info) > >> > Feb 9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication > >> failure > >> > for ouruser from x.x.x.x > >> > Feb 9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam > >> for > >> > ouruser from x.x.x.x port 53332 ssh2 > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for > >> user > >> > ouruser service ssh-connection method keyboard-interactive [preauth] > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1 > >> > [preauth] > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive > >> devs > >> > [preauth] > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge: > >> > user=ouruser devs= [preauth] > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices > >> 'pam' > >> > [preauth] > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start: > >> > trying authentication method 'pam' [preauth] > >> > Feb 9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive > >> for > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth] > >> > > >> > > >> > Feb 9 11:10:34 ourserver sshd[381563]: debug1: Forked child 536086. > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Set > >> > /proc/self/oom_score_adj to 0 > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 out 5 > >> > newsock 5 pipe 7 sock 8 > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after > >> > dupping: 4, 4 > >> > Feb 9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x port > >> 53332 > >> > on 150.108.64.156 port 22 rdomain "" > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Local version string > >> > SSH-2.0-OpenSSH_8.4 > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol version > >> > 2.0, remote software version OpenSSH_8.4 > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 pat > >> > OpenSSH* compat 0x04000000 > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SELinux support disabled > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: permanently_set_uid: > >> 74/74 > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types: > >> > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 > >> [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT sent > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT > >> received > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm: > >> > curve25519-sha256 [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: host key algorithm: > >> > ecdsa-sha2-nistp256 [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server > >> cipher: > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client > >> cipher: > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256 > >> > need=32 dh_need=32 [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: curve25519-sha256 > >> > need=32 dh_need=32 [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: expecting > >> > SSH2_MSG_KEX_ECDH_INIT [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: rekey out after > >> 4294967296 > >> > blocks [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS sent > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: Sending > >> SSH2_MSG_EXT_INFO > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: expecting > >> SSH2_MSG_NEWKEYS > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS > >> received > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: rekey in after > >> 4294967296 > >> > blocks [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for > >> user > >> > ouruser service ssh-connection method none [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures 0 > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing for > >> > "ouruser" > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_RHOST > >> to > >> > "x.x.x.x" > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting PAM_TTY to > >> > "ssh" > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: userauth-request for > >> user > >> > ouruser service ssh-connection method keyboard-interactive [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures 0 > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: keyboard-interactive > >> devs > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge: > >> > user=ouruser devs= [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: devices > >> 'pam' > >> > [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge_start: > >> > trying authentication method 'pam' [preauth] > >> > Feb 9 11:10:35 ourserver sshd[536086]: Postponed keyboard-interactive > >> for > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth] > >> > Feb 9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth): > >> authentication > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x > >> user=ouruser > >> > Feb 9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): > >> authentication > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser > >> > Feb 9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): received for > >> > user ouruser: 9 (Authentication service cannot retrieve authentication > >> info) > >> > Feb 9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication > >> failure > >> > for ouruser from x.x.x.x > >> > Feb 9 11:10:41 ourserver sshd[536086]: Failed keyboard-interactive/pam > >> for > >> > ouruser from x.x.x.x port 53332 ssh2 > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: userauth-request for > >> user > >> > ouruser service ssh-connection method keyboard-interactive [preauth] > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures 1 > >> > [preauth] > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: keyboard-interactive > >> devs > >> > [preauth] > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge: > >> > user=ouruser devs= [preauth] > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: devices > >> 'pam' > >> > [preauth] > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge_start: > >> > trying authentication method 'pam' [preauth] > >> > Feb 9 11:10:41 ourserver sshd[536086]: Postponed keyboard-interactive > >> for > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth] > >> > > >> > With the NIS password the logs show this: > >> > >> Hi, > >> > >> did you drop what happened before or is this the only debug output for > >> the NIS password? > >> > > > > The below here are just logs from /var/log/secure for the user that > > successfully logs in with his/her NIS password. > > > > By "drop what happened before" do you mean the original log snip? Yes I > > removed those in an attempt to shorten the message content. > > > > > Feb 9 11:16:57 debug1: do_pam_account: called > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: PAM: num PAM env > >> strings 2 > >> > Feb 9 11:16:57 ourserver sshd[536226]: Postponed > >> keyboard-interactive/pam > >> > for cai from 150.108.68.26 port 53646 ssh2 [preauth] > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: do_pam_account: called > >> > Feb 9 11:16:57 ourserver sshd[536226]: Accepted > >> keyboard-interactive/pam > >> > for cai from 150.108.68.26 port 53646 ssh2 > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: monitor_child_preauth: > >> cai > >> > has been authenticated by privileged process > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: monitor_read_log: child > >> log > >> > fd closed > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: audit_event: unhandled > >> > event 2 > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: temporarily_use_uid: > >> > 5879/200 (e=0/0) > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: ssh_gssapi_storecreds: > >> Not > >> > a GSSAPI mechanism > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: restore_uid: 0/0 > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: SELinux support disabled > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: PAM: establishing > >> > credentials > >> > Feb 9 11:16:57 ourserver systemd[536237]: > >> pam_unix(systemd-user:session): > >> > session opened for user cai(uid=5879) by (uid=0) > >> > > >> > What options should be set in /etc/ssh/sshd_config? Is sssd necessary > >> for > >> > this to work with the FreeIPA password > >> > > > > > > Yes, SSSD must be configured and runnnig. ssd does appear to be working > > fine and in /etc/ipa/ca.crt and the service is running correctly: > > > > [domain/ourdomain.edu] > > > > id_provider = ipa > > ipa_server_mode = True > > ipa_server = ourdomain.edu > > ipa_domain = ourdomain.edu > > ipa_hostname = ourdomain.edu > > auth_provider = ipa > > chpass_provider = ipa > > access_provider = ipa > > cache_credentials = True > > ldap_tls_cacert = /etc/ipa/ca.crt > > krb5_store_password_if_offline = True > > [sssd] > > services = nss, pam, ifp, ssh, sudo > > > > domains = ourdomain.edu > > [nss] > > homedir_substring = /home > > memcache_timeout = 600 > > > > [ifp] > > allowed_uids = ipaapi, root > > > > systemctl status sssd > > * sssd.service - System Security Services Daemon > > Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled; vendor > > preset: enabled) > > Active: active (running) since Fri 2021-01-29 14:31:34 EST; 1 weeks 3 > > days ago > > > > > _______________________________________________ > FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org > To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org > Fedora Code of Conduct: https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o... > List Guidelines: https://urldefense.proofpoint.com/v2/url?u=https-3A__fedoraproject.org_wi... > List Archives: https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.fedorahosted.o... _______________________________________________ FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org Fedora Code of Conduct: https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o... List Guidelines: https://urldefense.proofpoint.com/v2/url?u=https-3A__fedoraproject.org_wi... List Archives: https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.fedorahosted.o... Do not reply to spam on the list, report it: https://urldefense.proofpoint.com/v2/url?u=https-3A__pagure.io_fedora-2Di...

One one of the test servers sssd fails to start correctly. Here are the errors and their respective logs, with debug enabled. freeipa-server-4.9.1-1.fc33.x86_64 systemctl status sssd --no-pager -l * sssd.service - System Security Services Daemon Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled; vendor preset: enabled) Active: activating (start) since Tue 2021-02-16 15:15:32 EST; 1s ago Main PID: 809519 (sssd) Tasks: 2 (limit: 9484) Memory: 10.9M CPU: 998ms CGroup: /system.slice/sssd.service |-809519 /usr/sbin/sssd -i --logger=files `-809520 /usr/libexec/sssd/sssd_be --domain implicit_files --uid 0 --gid 0 --logger=files Feb 16 15:15:32 ourdomain.edu systemd[1]: Starting System Security Services Daemon... Feb 16 15:15:32 ourdomain.edu sssd[809519]: NSCD socket was detected and seems to be configured to cache some of the databases controlled by SSSD [passwd,group,netgroup,services]. It is recommended not to run NSCD in parallel with SSSD, unless NSCD is configured not to cache these. Feb 16 15:15:33 ourdomain.edu sssd[809519]: Starting up

However: systemctl status nscd * nscd.service - Name Service Cache Daemon Loaded: loaded (/usr/lib/systemd/system/nscd.service; disabled; vendor preset: disabled) Active: inactive (dead) /var/log/sssd/sssd_ourdomain.edu.log (2021-02-16 15:06:30): [be[ourdomain.edu]] [ipa_init_dyndns] (0x0100): Dynamic DNS updates are off. (2021-02-16 15:06:30): [be[ourdomain.edu]] [setup_tls_config] (0x0020): Unknown value for tls_reqcert 'never (or allow'.

(2021-02-16 15:06:30): [be[ourdomain.edu]] [ipa_init_misc] (0x0020): Unable to get TLS options [22]: Invalid argument (2021-02-16 15:06:30): [be[ourdomain.edu]] [sssm_ipa_init] (0x0020): Unable to init IPA module [22]: Invalid argument (2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_module_run_constructor] (0x0010): Module [ipa] constructor failed [22]: Invalid argument (2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_load_module] (0x0020): Unable to create DP module. (2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_target_init] (0x0010): Unable to load module ipa (2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_load_targets] (0x0020): Unable to load target [id] [80]: Accessing a corrupted shared library. (2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_init_done] (0x0020): Unable to initialize DP targets [1432158209]: Internal Error (2021-02-16 15:06:30): [be[ourdomain.edu]] [dp_terminate_active_requests] (0x0400): Terminating active data provider requests (2021-02-16 15:06:30): [be[ourdomain.edu]] [sbus_server_symlink_remove] (0x0400): Symlink removed [/var/lib/sss/pipes/private/sbus-dp_ourdomain.edu] /var/log/sssd/sssd.log (2021-02-16 15:13:21): [sssd] [monitor_cleanup] (0x0010): Error removing pidfile! (2 [No such file or directory]) (2021-02-16 15:13:22): [sssd] [server_setup] (0x0040): Starting with debug level = 0x0070 (2021-02-16 15:13:22): [sssd] [svc_child_info] (0x0040): Child [809126] (' ourdomain.edu':'%BE_ourdomain.edu') exited with code [3] (2021-02-16 15:13:22): [sssd] [svc_child_info] (0x0040): Child [809127] (' ourdomain.edu':'%BE_ourdomain.edu') exited with code [3] (2021-02-16 15:13:24): [sssd] [svc_child_info] (0x0040): Child [809133] (' ourdomain.edu':'%BE_ourdomain.edu') exited with code [3] (2021-02-16 15:13:27): [sssd] [services_startup_timeout] (0x0020): Providers did not start in time, forcing services startup! (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809137] ('nss':'nss') exited with code [3] (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809139] ('ifp':'ifp') exited with code [3] (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809141] ('sudo':'sudo') exited with code [3] (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809138] ('pam':'pam') exited with code [3] (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809140] ('ssh':'ssh') exited with code [3] (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809142] ('pac':'pac') exited with code [3] (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809143] ('nss':'nss') exited with code [3] (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809144] ('ifp':'ifp') exited with code [3] (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809145] ('sudo':'sudo') exited with code [3] (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809147] ('ssh':'ssh') exited with code [3] (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809146] ('pam':'pam') exited with code [3] (2021-02-16 15:13:27): [sssd] [svc_child_info] (0x0040): Child [809148] ('pac':'pac') exited with code [3] (2021-02-16 15:13:28): [sssd] [svc_child_info] (0x0040): Child [809153] (' ourdomain.edu':'%BE_ourdomain.edu') exited with code [3] (2021-02-16 15:13:28): [sssd] [monitor_restart_service] (0x0010): Process [ ourdomain.edu], definitely stopped! (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0040): Returned with: 1 (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating [pac][809148] (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill [pac][809148]: [No such process] (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating [ssh][809147] (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill [ssh][809147]: [No such process] (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating [pam][809146] (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill [pam][809146]: [No such process] (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating [sudo][809145] (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill [sudo][809145]: [No such process] (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating [ifp][809144] (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill [ifp][809144]: [No such process] (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating [nss][809143] (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Couldn't kill [nss][809143]: [No such process] (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Terminating [implicit_files][809125] (2021-02-16 15:13:28): [sssd] [monitor_quit] (0x0020): Child [implicit_files] exited gracefully (2021-02-16 15:13:28): [sssd] [monitor_cleanup] (0x0010): Error removing pidfile! (2 [No such file or directory]) In /var/log/sssd/sssd_pam.log, /var/log/sssd/sssd_sudo.log, /var/log/sssd/sssd_ifp.log, /var/log/sssd/sssd_pac.log & /var/log/sssd/sssd_nss.log all these logs end with: (2021-02-16 15:08:30): [pam] [server_setup] (0x0040): Starting with debug level = 0x0070 (2021-02-16 15:08:30): [pam] [sss_iface_connect_address] (0x0020): check_file failed for [/var/lib/sss/pipes/private/sbus-dp_ourdomain.edu]. (2021-02-16 15:08:30): [pam] [sss_dp_init] (0x0010): Failed to connect to backend server. (2021-02-16 15:08:30): [pam] [sss_process_init] (0x0010): fatal error setting up backend connector (2021-02-16 15:08:30): [pam] [pam_process_init] (0x0010): sss_process_init() failed /var/log/sssd/sssd_implicit_files.log (2021-02-16 15:12:24): [be[implicit_files]] [sysdb_set_cache_entry_attr] (0x0040): Error: 17 (File exists) (2021-02-16 15:12:24): [be[implicit_files]] [sysdb_set_cache_entry_attr] (0x0040): Error: 17 (File exists) (2021-02-16 15:12:24): [be[implicit_files]] [sysdb_store_new_group] (0x0040): sysdb_add_group failed (while renaming group) for: wireshark@implicit_files [1630]. (2021-02-16 15:12:24): [be[implicit_files]] [sysdb_store_group] (0x0040): Cache update failed: 17 (2021-02-16 15:12:24): [be[implicit_files]] [save_file_group] (0x0040): Could not add group to cache (2021-02-16 15:12:26): [be[implicit_files]] [orderly_shutdown] (0x0010): SIGTERM: killing children (2021-02-16 15:12:26): [be[implicit_files]] [orderly_shutdown] (0x0040): Shutting down (status = 0) (2021-02-16 15:12:26): [be[implicit_files]] [server_setup] (0x0040): Starting with debug level = 0x0070 However ipactl status does not show any errors: Directory Service: RUNNING krb5kdc Service: RUNNING kadmin Service: RUNNING named Service: RUNNING httpd Service: RUNNING ipa-custodia Service: RUNNING pki-tomcatd Service: RUNNING ipa-otpd Service: RUNNING ipa-dnskeysyncd Service: RUNNING ipa: INFO: The ipactl command was successful Anything sticking out? On Thu, Feb 11, 2021 at 4:51 PM Robert Kudyba <rkudyba(a)fordham.edu&gt; wrote: > What is ourserver.edu? In order to log in using Kerberos/GSSAPI then the >> machine acting as the server needs to be enrolled as an IPA client so it >> has a keytab. >> rob > > > OK I added a Fedora server as a client. From ipa host-show > client.ourserver.edu > Host name: client.ourserver.edu > Platform: x86_64 > Operating system: 5.10.13-200.fc33.x86_64 > Principal name: host/client.ourserver.edu(a)OLDDSM.DSM.FORDHAM.EDU > Principal alias: host/client.ourserver.edu(a)OLDDSM.DSM.FORDHAM.EDU > SSH public key fingerprint: SHA256:xxx(ecdsa-sha2-nistp256), > SHA256:xxx (ssh-ed25519), > SHA256:xxx (ssh-rsa) > Password: False > Keytab: True > Managed by: client.ourserver.edu > > ssh -k still fails. > > I created a new user in via ipa user-add and that user also fails even > with a su -: > > su: user testuser does not exist or the user entry does not contain all > the required fields > > Feb 11 15:07:21 ourserver sshd[609424]: debug1: rekey out after 4294967296 > blocks [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: debug1: SSH2_MSG_NEWKEYS sent > [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: debug1: Sending SSH2_MSG_EXT_INFO > [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: debug1: expecting SSH2_MSG_NEWKEYS > [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: debug1: SSH2_MSG_NEWKEYS received > [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: debug1: rekey in after 4294967296 > blocks [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: debug1: KEX done [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: debug1: userauth-request for user > yume service ssh-connection method none [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: debug1: attempt 0 failures 0 > [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: Invalid user testuser from x.x.x.x > port 36264 > Feb 11 15:07:21 ourserver sshd[609424]: debug1: PAM: initializing for > "testuser" > Feb 11 15:07:21 ourserver sshd[609424]: debug1: PAM: setting PAM_RHOST to > "x.x.x.x" > Feb 11 15:07:21 ourserver sshd[609424]: debug1: PAM: setting PAM_TTY to > "ssh" > Feb 11 15:07:21 ourserver sshd[609424]: debug1: userauth-request for user > testuser service ssh-connection method keyboard-interactive [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: debug1: attempt 1 failures 0 > [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: debug1: keyboard-interactive devs > [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: debug1: auth2_challenge: > user=testuser devs= [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: debug1: kbdint_alloc: devices > 'pam' [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: debug1: auth2_challenge_start: > trying authentication method 'pam' [preauth] > Feb 11 15:07:21 ourserver sshd[609424]: Postponed keyboard-interactive for > invalid user testuser from x.x.x.x port 36264 ssh2 [preauth] > Feb 11 15:07:24 ourserver sshd[609424]: Connection closed by invalid user > testuser x.x.x.x port 36264 [preauth] > > are you sure the sssd.conf you have send earlier it the right one? >> SSSD'S 'proxy_child' should be only be called if the proxy provider is >> configured? >> >> bye, >> Sumit >> > > > Sorry, you are correct. I'm testing between 2 test NIS servers to see if > there's a difference in behavior but the same problem, ssh -k never works > with the FreeIPA/Kerberos password > > >> >> > Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth): >> authentication >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser >> > Feb 10 14:36:31 ourserver sshd[3084344]: pam_sss(sshd:auth): received >> for >> > user ouruser: 7 (Authentication failure) >> > Feb 10 14:36:33 ourserver sshd[3084339]: error: PAM: Authentication >> failure >> > for ouruser from x.x.x.x >> > Feb 10 14:36:33 ourserver sshd[3084339]: Failed keyboard-interactive/pam >> > for ouruser from x.x.x.x port 34160 ssh2 >> > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: userauth-request for >> user >> > ouruser service ssh-connection method keyboard-interactive [preauth] >> > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: attempt 3 failures 1 >> > [preauth] >> > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: keyboard-interactive >> devs >> > [preauth] >> > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge: >> > user=ouruser devs= [preauth] >> > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: kbdint_alloc: devices >> > 'pam' [preauth] >> > Feb 10 14:36:33 ourserver sshd[3084339]: debug1: auth2_challenge_start: >> > trying authentication method 'pam' [preauth] >> > Feb 10 14:36:33 ourserver sshd[3084339]: Postponed keyboard-interactive >> for >> > ouruser from x.x.x.x port 34160 ssh2 [preauth] >> > >> > I verified the FreeIPA password in both the GUI and via ipa user-mod. >> The >> > only time the user is able to log in is using the NIS password. >> ldapsearch >> > -x -D and kinit username work successfully. klist displays the user >> details >> > correctly. >> > >> > I can see that the installation script edits /etc/ssh/sshd_config with: >> > Include /etc/ssh/sshd_config.d/04-ipa.conf >> > >> > which has: >> > PubkeyAuthentication yes >> > KerberosAuthentication no >> > GSSAPIAuthentication yes >> > UsePAM yes >> > ChallengeResponseAuthentication yes >> > AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys >> > AuthorizedKeysCommandUser nobody >> > >> > When the NIS password is used successfully here are the server logs: >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for >> user >> > ouruser service ssh-connection method none [preauth] >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 0 failures 0 >> > [preauth] >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: connection from x.x.x.x >> > matched 'Address 192.168.0.*,127.0.0.1,10.10.1.*' at line 158 >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: initializing for >> > "ouruser" >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_RHOST >> to >> > "x.x.x.x" >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: PAM: setting PAM_TTY to >> > "ssh" >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for >> user >> > ouruser service ssh-connection method gssapi-with-mic [preauth] >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 1 failures 0 >> > [preauth] >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: userauth-request for >> user >> > ouruser service ssh-connection method keyboard-interactive [preauth] >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: attempt 2 failures 0 >> > [preauth] >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: keyboard-interactive >> devs >> > [preauth] >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge: >> > user=ouruser devs= [preauth] >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: kbdint_alloc: devices >> > 'pam' [preauth] >> > Feb 10 14:56:39 ourserver sshd[3085147]: debug1: auth2_challenge_start: >> > trying authentication method 'pam' [preauth] >> > Feb 10 14:56:39 ourserver sshd[3085147]: Postponed keyboard-interactive >> for >> > ouruser from x.x.x.x port 35046 ssh2 [preauth] >> > Feb 10 14:56:42 ourserver sshd[3085152]: debug1: do_pam_account: called >> > Feb 10 14:56:42 ourserver sshd[3085147]: debug1: PAM: num PAM env >> strings 2 >> > Feb 10 14:56:42 ourserver sshd[3085147]: Postponed >> keyboard-interactive/pam >> > for ouruser from x.x.x.x port 35046 ssh2 [preauth] >> > Feb 10 14:56:42 ourserver sshd[3085147]: debug1: do_pam_account: called >> > >> > I do see the error that sticks out is " Server host/ >> > ourserver.edu(a)ourserver.edu not found in Kerberos database" but we have >> > students that log in from all over the world so do all clients need to >> be >> > added? iptables, firewalld, and nftables are off and disabled. No hbac >> > rules: >> > ipa hbacrule-find >> > -------------------- >> > 2 HBAC rules matched >> > -------------------- >> > Rule name: allow_all >> > User category: all >> > Host category: all >> > Service category: all >> > Description: Allow all users to access any host from any host >> > Enabled: TRUE >> > >> > Rule name: allow_systemd-user >> > User category: all >> > Host category: all >> > Description: Allow pam_systemd to run user@.service to create a >> system >> > user session >> > Enabled: TRUE >> > ---------------------------- >> > Number of entries returned 2 >> > >> > Am I missing something obvious to regulars? >> > >> > >> > On Tue, Feb 9, 2021 at 12:34 PM Robert Kudyba <rkudyba(a)fordham.edu&gt; >> wrote: >> > >> > > On Tue, Feb 9, 2021 at 12:20 PM Sumit Bose via FreeIPA-users < >> > > freeipa-users(a)lists.fedorahosted.org&gt; wrote: >> > > >> > >> On Tue, Feb 09, 2021 at 11:33:15AM -0500, Robert Kudyba via >> FreeIPA-users >> > >> wrote: >> > >> > > >> > >> > > looks like sshd is trying to read >> /home/ouruser/.ssh/authorized_keys >> > >> and >> > >> > > is stuck. Can you read this file from the command line? Is it >> e.g. on >> > >> > > NFS which might not be properly mounted? >> > >> > > >> > >> > > Does it work if you skip pubkey authentication >> > >> > > >> > >> > > ssh -o PubkeyAuthentication=no -vv -k ouruser@ourserver >> > >> > > >> > >> > > bye, >> > >> > > Sumit >> > >> > > >> > >> > >> > >> > Thanks for the suggestion. What happens is the NIS password works. >> The >> > >> > FreeIPA password, which I update with: >> > >> > ipa user-mod ouruser --setattr "userpassword=xxxx", fails with the >> below >> > >> > errors/logs >> > >> > >> > >> > Feb 9 11:10:34 ourserver sshd[381563]: debug1: Forked child >> 536086. >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Set >> > >> > /proc/self/oom_score_adj to 0 >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 >> out 5 >> > >> > newsock 5 pipe 7 sock 8 >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after >> > >> > dupping: 4, 4 >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x >> port >> > >> 53332 >> > >> > on 150.108.64.156 port 22 rdomain "" >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Local version >> string >> > >> > SSH-2.0-OpenSSH_8.4 >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol >> version >> > >> > 2.0, remote software version OpenSSH_8.4 >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 >> pat >> > >> > OpenSSH* compat 0x04000000 >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SELinux support >> disabled >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: >> permanently_set_uid: >> > >> 74/74 >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types: >> > >> > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 >> > >> [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT >> sent >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT >> > >> received >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm: >> > >> > curve25519-sha256 [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: host key >> algorithm: >> > >> > ecdsa-sha2-nistp256 [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server >> > >> cipher: >> > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client >> > >> cipher: >> > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: >> curve25519-sha256 >> > >> > need=32 dh_need=32 [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: >> curve25519-sha256 >> > >> > need=32 dh_need=32 [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: expecting >> > >> > SSH2_MSG_KEX_ECDH_INIT [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: rekey out after >> > >> 4294967296 >> > >> > blocks [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS >> sent >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: Sending >> > >> SSH2_MSG_EXT_INFO >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: expecting >> > >> SSH2_MSG_NEWKEYS >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS >> > >> received >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: rekey in after >> > >> 4294967296 >> > >> > blocks [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: userauth-request >> for >> > >> user >> > >> > ouruser service ssh-connection method none [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures >> 0 >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing >> for >> > >> > "ouruser" >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting >> PAM_RHOST >> > >> to >> > >> > "x.x.x.x" >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting >> PAM_TTY to >> > >> > "ssh" >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: userauth-request >> for >> > >> user >> > >> > ouruser service ssh-connection method keyboard-interactive >> [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures >> 0 >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: >> keyboard-interactive >> > >> devs >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge: >> > >> > user=ouruser devs= [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: >> devices >> > >> 'pam' >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: >> auth2_challenge_start: >> > >> > trying authentication method 'pam' [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: Postponed >> keyboard-interactive >> > >> for >> > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth] >> > >> > Feb 9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth): >> > >> authentication >> > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x >> > >> user=ouruser >> > >> > Feb 9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): >> > >> authentication >> > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x >> user=ouruser >> > >> > Feb 9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): >> received for >> > >> > user ouruser: 9 (Authentication service cannot retrieve >> authentication >> > >> info) >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication >> > >> failure >> > >> > for ouruser from x.x.x.x >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: Failed >> keyboard-interactive/pam >> > >> for >> > >> > ouruser from x.x.x.x port 53332 ssh2 >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: userauth-request >> for >> > >> user >> > >> > ouruser service ssh-connection method keyboard-interactive >> [preauth] >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures >> 1 >> > >> > [preauth] >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: >> keyboard-interactive >> > >> devs >> > >> > [preauth] >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge: >> > >> > user=ouruser devs= [preauth] >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: >> devices >> > >> 'pam' >> > >> > [preauth] >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: >> auth2_challenge_start: >> > >> > trying authentication method 'pam' [preauth] >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: Postponed >> keyboard-interactive >> > >> for >> > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth] >> > >> > >> > >> > >> > >> > Feb 9 11:10:34 ourserver sshd[381563]: debug1: Forked child >> 536086. >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Set >> > >> > /proc/self/oom_score_adj to 0 >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: rexec start in 5 >> out 5 >> > >> > newsock 5 pipe 7 sock 8 >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: inetd sockets after >> > >> > dupping: 4, 4 >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: Connection from x.x.x.x >> port >> > >> 53332 >> > >> > on 150.108.64.156 port 22 rdomain "" >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Local version >> string >> > >> > SSH-2.0-OpenSSH_8.4 >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: Remote protocol >> version >> > >> > 2.0, remote software version OpenSSH_8.4 >> > >> > Feb 9 11:10:34 ourserver sshd[536086]: debug1: match: OpenSSH_8.4 >> pat >> > >> > OpenSSH* compat 0x04000000 >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SELinux support >> disabled >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: >> permanently_set_uid: >> > >> 74/74 >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: list_hostkey_types: >> > >> > rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 >> > >> [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT >> sent >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_KEXINIT >> > >> received >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: algorithm: >> > >> > curve25519-sha256 [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: host key >> algorithm: >> > >> > ecdsa-sha2-nistp256 [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: client->server >> > >> cipher: >> > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: server->client >> > >> cipher: >> > >> > aes256-gcm(a)openssh.com MAC: <implicit> compression: none [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: >> curve25519-sha256 >> > >> > need=32 dh_need=32 [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kex: >> curve25519-sha256 >> > >> > need=32 dh_need=32 [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: expecting >> > >> > SSH2_MSG_KEX_ECDH_INIT [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: rekey out after >> > >> 4294967296 >> > >> > blocks [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS >> sent >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: Sending >> > >> SSH2_MSG_EXT_INFO >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: expecting >> > >> SSH2_MSG_NEWKEYS >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: SSH2_MSG_NEWKEYS >> > >> received >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: rekey in after >> > >> 4294967296 >> > >> > blocks [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: KEX done [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: userauth-request >> for >> > >> user >> > >> > ouruser service ssh-connection method none [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: attempt 0 failures >> 0 >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: initializing >> for >> > >> > "ouruser" >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting >> PAM_RHOST >> > >> to >> > >> > "x.x.x.x" >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: PAM: setting >> PAM_TTY to >> > >> > "ssh" >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: userauth-request >> for >> > >> user >> > >> > ouruser service ssh-connection method keyboard-interactive >> [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: attempt 1 failures >> 0 >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: >> keyboard-interactive >> > >> devs >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: auth2_challenge: >> > >> > user=ouruser devs= [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: kbdint_alloc: >> devices >> > >> 'pam' >> > >> > [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: debug1: >> auth2_challenge_start: >> > >> > trying authentication method 'pam' [preauth] >> > >> > Feb 9 11:10:35 ourserver sshd[536086]: Postponed >> keyboard-interactive >> > >> for >> > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth] >> > >> > Feb 9 11:10:39 ourserver sshd[536091]: pam_unix(sshd:auth): >> > >> authentication >> > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x >> > >> user=ouruser >> > >> > Feb 9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): >> > >> authentication >> > >> > failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x >> user=ouruser >> > >> > Feb 9 11:10:39 ourserver sshd[536091]: pam_sss(sshd:auth): >> received for >> > >> > user ouruser: 9 (Authentication service cannot retrieve >> authentication >> > >> info) >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: error: PAM: Authentication >> > >> failure >> > >> > for ouruser from x.x.x.x >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: Failed >> keyboard-interactive/pam >> > >> for >> > >> > ouruser from x.x.x.x port 53332 ssh2 >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: userauth-request >> for >> > >> user >> > >> > ouruser service ssh-connection method keyboard-interactive >> [preauth] >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: attempt 2 failures >> 1 >> > >> > [preauth] >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: >> keyboard-interactive >> > >> devs >> > >> > [preauth] >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: auth2_challenge: >> > >> > user=ouruser devs= [preauth] >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: kbdint_alloc: >> devices >> > >> 'pam' >> > >> > [preauth] >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: debug1: >> auth2_challenge_start: >> > >> > trying authentication method 'pam' [preauth] >> > >> > Feb 9 11:10:41 ourserver sshd[536086]: Postponed >> keyboard-interactive >> > >> for >> > >> > ouruser from x.x.x.x port 53332 ssh2 [preauth] >> > >> > >> > >> > With the NIS password the logs show this: >> > >> >> > >> Hi, >> > >> >> > >> did you drop what happened before or is this the only debug output >> for >> > >> the NIS password? >> > >> >> > > >> > > The below here are just logs from /var/log/secure for the user that >> > > successfully logs in with his/her NIS password. >> > > >> > > By "drop what happened before" do you mean the original log snip? Yes >> I >> > > removed those in an attempt to shorten the message content. >> > > >> > > > Feb 9 11:16:57 debug1: do_pam_account: called >> > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: PAM: num PAM env >> > >> strings 2 >> > >> > Feb 9 11:16:57 ourserver sshd[536226]: Postponed >> > >> keyboard-interactive/pam >> > >> > for cai from 150.108.68.26 port 53646 ssh2 [preauth] >> > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: do_pam_account: >> called >> > >> > Feb 9 11:16:57 ourserver sshd[536226]: Accepted >> > >> keyboard-interactive/pam >> > >> > for cai from 150.108.68.26 port 53646 ssh2 >> > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: >> monitor_child_preauth: >> > >> cai >> > >> > has been authenticated by privileged process >> > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: monitor_read_log: >> child >> > >> log >> > >> > fd closed >> > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: audit_event: >> unhandled >> > >> > event 2 >> > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: >> temporarily_use_uid: >> > >> > 5879/200 (e=0/0) >> > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: >> ssh_gssapi_storecreds: >> > >> Not >> > >> > a GSSAPI mechanism >> > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: restore_uid: 0/0 >> > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: SELinux support >> disabled >> > >> > Feb 9 11:16:57 ourserver sshd[536226]: debug1: PAM: establishing >> > >> > credentials >> > >> > Feb 9 11:16:57 ourserver systemd[536237]: >> > >> pam_unix(systemd-user:session): >> > >> > session opened for user cai(uid=5879) by (uid=0) >> > >> > >> > >> > What options should be set in /etc/ssh/sshd_config? Is sssd >> necessary >> > >> for >> > >> > this to work with the FreeIPA password >> > >> >> > > >> > > >> > > Yes, SSSD must be configured and runnnig. ssd does appear to be >> working >> > > fine and in /etc/ipa/ca.crt and the service is running correctly: >> > > >> > > [domain/ourdomain.edu] >> > > >> > > id_provider = ipa >> > > ipa_server_mode = True >> > > ipa_server = ourdomain.edu >> > > ipa_domain = ourdomain.edu >> > > ipa_hostname = ourdomain.edu >> > > auth_provider = ipa >> > > chpass_provider = ipa >> > > access_provider = ipa >> > > cache_credentials = True >> > > ldap_tls_cacert = /etc/ipa/ca.crt >> > > krb5_store_password_if_offline = True >> > > [sssd] >> > > services = nss, pam, ifp, ssh, sudo >> > > >> > > domains = ourdomain.edu >> > > [nss] >> > > homedir_substring = /home >> > > memcache_timeout = 600 >> > > >> > > [ifp] >> > > allowed_uids = ipaapi, root >> > > >> > > systemctl status sssd >> > > * sssd.service - System Security Services Daemon >> > > Loaded: loaded (/usr/lib/systemd/system/sssd.service; enabled; >> vendor >> > > preset: enabled) >> > > Active: active (running) since Fri 2021-01-29 14:31:34 EST; 1 >> weeks 3 >> > > days ago >> > > >> > > >> >> > _______________________________________________ >> > FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org >> > To unsubscribe send an email to >> freeipa-users-leave(a)lists.fedorahosted.org >> > Fedora Code of Conduct: >> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o... >> > List Guidelines: >> https://urldefense.proofpoint.com/v2/url?u=https-3A__fedoraproject.org_wi... >> > List Archives: >> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.fedorahosted.o... >> _______________________________________________ >> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org >> To unsubscribe send an email to >> freeipa-users-leave(a)lists.fedorahosted.org >> Fedora Code of Conduct: >> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o... >> List Guidelines: >> https://urldefense.proofpoint.com/v2/url?u=https-3A__fedoraproject.org_wi... >> List Archives: >> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.fedorahosted.o... >> Do not reply to spam on the list, report it: >> https://urldefense.proofpoint.com/v2/url?u=https-3A__pagure.io_fedora-2Di... >> >

_______________________________________________ FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

> > that's odd, can you check with ps if nscd is running? It is not. > Does /var/run/nscd/socket exists? > Yes it does, I then deleted it. > > (2021-02-16 15:06:30): [be[ourdomain.edu]] [setup_tls_config] (0x0020): > > Unknown value for tls_reqcert 'never (or allow'. > > Can you check your sssd.conf if the 'tls_reqcert' option is set to > 'never (or allow'? I guess you wanted to set it to 'never'. > Nice catch. Bad copy/paste for me as I tried to enable more verbose debug logs. Now sssd at least starts. So new users created in IPA/LDAP can now ssh -k and su correctly. The debug ssh server logs look like this: Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: userauth-request for user tuser service ssh-connection method keyboard-interactive [preauth] Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: attempt 3 failures 1 [preauth] Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: keyboard-interactive devs [preauth] Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: auth2_challenge: user=tuser devs= [preauth] Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: kbdint_alloc: devices 'pam' [preauth] Feb 17 09:44:47 ourdomainsshd[1016078]: debug1: auth2_challenge_start: trying authentication method 'pam' [preauth] Feb 17 09:44:48 ourdomainsshd[1016078]: Postponed keyboard-interactive for tuser from x.x.x.x port 48614 ssh2 [preauth] Feb 17 09:44:52 ourdomainsshd[1016085]: pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.xuser=tuser Feb 17 09:44:52 ourdomainsshd[1016085]: pam_sss(sshd:auth): received for user tuser: 12 (Authentication token is no longer valid; new one required) Feb 17 09:44:52 ourdomainsshd[1016085]: debug1: do_pam_account: called Feb 17 09:44:52 ourdomainsshd[1016085]: pam_sss(sshd:account): User info message: Password expired. Change your password now. Feb 17 09:44:52 ourdomainsshd[1016085]: pam_unix(sshd:chauthtok): user "tuser" does not exist in /etc/passwd Feb 17 09:44:52 ourdomainsshd[1016078]: Postponed keyboard-interactive/pam for tuser from x.x.x.x port 48614 ssh2 [preauth] ssh -vvv has this: debug2: pubkey_prepare: done debug3: send packet: type 5 debug3: receive packet: type 7 debug1: SSH2_MSG_EXT_INFO received debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519, sk-ssh-ed25519(a)openssh.com ,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521, sk-ecdsa-sha2-nistp256(a)openssh.com, webauthn-sk-ecdsa-sha2-nistp256(a)openssh.com&gt; debug3: receive packet: type 6 debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug3: send packet: type 50 debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive debug3: preferred password debug3: authmethod_lookup password debug3: remaining preferred: debug3: authmethod_is_enabled password debug1: Next authentication method: password As for an imported NIS user, only the NIS password continues to allow logins. I also stopped ypserv and ypbind to see if there was any difference and there is not. Here are the ssh server logs for the NIS password that succeeds: Feb 17 09:34:56 ourdomain sshd [1015632]: debug1: userauth-request for user ouruser service ssh-connection method password [preauth] Feb 17 09:34:56 ourdomain sshd [1015632]: debug1: attempt 2 failures 1 [preauth] Feb 17 09:34:56 ourdomain sshd [1015632]: pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser Feb 17 09:34:56 ourdomain sshd [1015632]: pam_sss(sshd:auth): received for user ouruser: 9 (Authentication service cannot retrieve authentication info) Feb 17 09:34:58 ourdomain sshd [1015632]: debug1: PAM: password authentication failed for ouruser: Authentication failure Feb 17 09:34:58 ourdomain sshd [1015632]: Failed password for ouruser from x.x.x.x port 48578 ssh2 Feb 17 09:35:07 ourdomain sshd [1015632]: debug1: userauth-request for user ouruser service ssh-connection method password [preauth] Feb 17 09:35:07 ourdomain sshd [1015632]: debug1: attempt 3 failures 2 [preauth] Feb 17 09:35:08 ourdomain sshd [1015632]: debug1:* PAM: password authentication accepted for ouruser* Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: do_pam_account: called Feb 17 09:35:08 ourdomain sshd [1015632]: Accepted password for ouruser from x.x.x.x port 48578 ssh2 Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: monitor_child_preauth: ouruser has been authenticated by privileged process Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: monitor_read_log: child log fd closed Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: audit_event: unhandled event 2 Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: temporarily_use_uid: 5920/200 (e=0/0) Feb 17 09:35:08 ourdomain sshd [1015632]: debug1:* ssh_gssapi_storecreds: Not a GSSAPI mechanism* Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: restore_uid: 0/0 Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: SELinux support disabled Feb 17 09:35:08 ourdomain sshd [1015632]: debug1: PAM: establishing credentials Feb 17 09:35:08 ourdomain systemd[1015655]: pam_unix(systemd-user:session): session opened for user ouruser (uid=5920) by (uid=0) Feb 17 09:36:38 ourdomain sshd [1015632]: pam_unix(sshd:session): session opened for user ouruser (uid=5920) by (uid=0) Feb 17 09:36:39 ourdomain sshd [1015632]: User child is on pid 1015696 Feb 17 09:36:39 ourdomain sshd [1015696]: debug1: PAM: establishing credentials What sticks out to me is what I bolded, that PAM password auth works and that this is NOT a GSSAPI mechanism. Am I missing a step? Here are the related logs from sssd_ourdomain.edu.log: (2021-02-17 10:45:25): [be[ourdomain.edu]*] [dp_get_account_info_send] (0x0200): Got request for [0x1][BE_REQ_USER][name=pam_usertype_non_existent:@ourdomain.edu <pam_usertype_non_existent%3A(a)ourdomain.edu&gt;]* (2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_attach_req] (0x0400): DP Request [Account #70]: New request. Flags [0x0001]. (2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_attach_req] (0x0400): Number of active DP request: 1 (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_search_user_next_base] (0x0400): Searching for users with base [cn=accounts,dc=ourdomain,dc=edu] (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(uid=pam_usertype_non_existent:)(objectclass=posixAccount)(uid=*)(&(uidNumber=*)(!(uidNumber=0))))][cn=accounts,dc=ourdomain,dc=edu]. (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_get_generic_op_finished] (0x0400): Search result: Success(0), no errmsg set (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_search_user_process] (0x0400): *Search for users, returned 0 results*. (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_search_user_next_base] (0x0400): Searching for users with base [cn=trusts,dc=ourdomain,dc=edu] (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(&(uid=pam_usertype_non_existent:)(objectclass=posixAccount)(uid=*)(&(uidNumber=*)(!(uidNumber=0))))(objectClass=ipaIDObject))][cn=trusts,dc=ourdomain,dc=edu]. (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_get_generic_op_finished] (0x0400): Search result: Success(0), no errmsg set (2021-02-17 10:45:25): [be[ourdomain.edu]] [sdap_search_user_process] (0x0400): *Search for users, returned 0 results*. (2021-02-17 10:45:25): [be[ourdomain.edu]] [sysdb_search_by_name] (0x0400): No such entry (2021-02-17 10:45:25): [be[ourdomain.edu]] [sysdb_delete_user] (0x0400): Error: 2 (No such file or directory) (2021-02-17 10:45:25): [be[ourdomain.edu]] [sysdb_search_by_name] (0x0400): No such entry (2021-02-17 10:45:25): [be[ourdomain.edu]] [ipa_id_get_account_info_orig_done] (0x0080): Object not found, ending request (2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_req_done] (0x0400): DP Request [Account #70]: Request handler finished [0]: Success (2021-02-17 10:45:25): [be[ourdomain.edu]] [_dp_req_recv] (0x0400): DP Request [Account #70]: Receiving request data. (2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_req_destructor] (0x0400): DP Request [Account #70]: Request removed. (2021-02-17 10:45:25): [be[ourdomain.edu]] [dp_req_destructor] (0x0400): Number of active DP request: 0 (2021-02-17 10:45:25): [be[ourdomain.edu]] [sbus_issue_request_done] (0x0400): sssd.dataprovider.getAccountInfo: Success So the log "*name=pam_usertype_non_existent*:" seems rather pertinent. I see from another thread https://freeipa-users.redhat.narkive.com/GYwxOWHr/understanding-the-migra..., I also used --setattr userpassword={crypt}yourencryptedpass. ipa user-show returns "Kerberos keys available: False" on the migrated NIS users. I tested changing passwords of users via the GUI and then ipa user-show returns "Kerberos keys available: True"

Anything stick out here?

_______________________________________________ FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

have you enabled the migration mode with > > ipa config-mod --enable-migration=True > I've tried it with True and False. At what point should this be changed to False? > With this authentication with SSSD should fall back to LDAP > authentication if the Kerberos keys are not available and this would > trigger a creation of the Kerberos keys for the user trying to log in. > The fallback appears to be NIS. The good news is the user can log in to the GUI, i.e.i https://ourserver/ipa/ui and change their password but I do see this error in the Apache error log which sounds like this issue <https://pagure.io/freeipa/issue/7032>;: [Wed Mar 03 13:53:07.526386 2021] [wsgi:error] [pid 16169:tid 16554] [remote xx.xx.xx.xx:63098] ipa: DEBUG: Destroyed connection context.ldap2_140265125387520 [Wed Mar 03 13:53:07.563873 2021] [:warn] [pid 16174:tid 16239] [client xx.xx.xx.xx:63098] failed to set perms (3140) on file (/run/ipa/ccaches/admin(a)OURDOMAIN.EDU-jhCS0U)!, referer: https://ourdomain.edu/ipa/ui/ [Wed Mar 03 13:53:07.564720 2021] [wsgi:error] [pid 16170:tid 16545] [remote xx.xx.xx.xx:63098] ipa: DEBUG: WSGI wsgi_dispatch.__call__: [Wed Mar 03 13:53:07.564838 2021] [wsgi:error] [pid 16170:tid 16545] [remote xx.xx.xx.xx:63098] ipa: DEBUG: WSGI jsonserver_session.__call__: [Wed Mar 03 13:53:07.570164 2021] [:warn] [pid 16174:tid 16285] [client xx.xx.xx.xx:63076] failed to set perms (3140) on file (/run/ipa/ccaches/adminOURDOMAIN.EDU-jhCS0U)!, referer: https://ourdomain.edu/ipa/ui/ Now from ssh -vvv -k this is what we see: debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic *debug1: Next authentication method: gssapi-with-micdebug1: Unspecified GSS failure. Minor code may provide more informationNo Kerberos credentials available (default cache: KEYRING:persistent:0)debug1: Unspecified GSS failure. Minor code may provide more informationNo Kerberos credentials available (default cache: KEYRING:persistent:0)* debug2: we did not send a packet, disable method debug3: authmethod_lookup publickey debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled publickey debug1: Next authentication method: publickey debug1: Offering public key: /ouruser/.ssh/id_rsa RSA SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU debug3: send packet: type 50 debug2: we sent a publickey packet, wait for reply debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password,keyboard-interactive debug1: Trying private key: /ouruser/.ssh/id_dsa debug3: no such identity: /ouruser/.ssh/id_dsa: No such file or directory debug1: Trying private key: /ouruser/.ssh/id_ecdsa debug3: no such identity: /ouruser/.ssh/id_ecdsa: No such file or directory debug1: Trying private key: /ouruser/.ssh/id_ecdsa_sk debug3: no such identity: /ouruser/.ssh/id_ecdsa_sk: No such file or directory debug1: Trying private key: /ouruser/.ssh/id_ed25519 debug3: no such identity: /ouruser/.ssh/id_ed25519: No such file or directory debug1: Trying private key: /ouruser/.ssh/id_ed25519_sk debug3: no such identity: /ouruser/.ssh/id_ed25519_sk: No such file or directory debug1: Trying private key: /ouruser/.ssh/id_xmss debug3: no such identity: /ouruser/.ssh/id_xmss: No such file or directory debug2: we did not send a packet, disable method debug3: authmethod_lookup keyboard-interactive debug3: remaining preferred: password debug3: authmethod_is_enabled keyboard-interactive debug1: Next authentication Is this a clue? Unspecified GSS failure. Minor code may provide more information No Kerberos credentials available (default cache: KEYRING:persistent:0) From the ssh server logs, set to debug: Mar 3 14:00:46 ourserver sshd[79161]: debug1: attempt 0 failures 0 [preauth] Mar 3 14:00:46 ourserver sshd[79161]: debug1: PAM: initializing for "ouruser" Mar 3 14:00:46 ourserver sshd[79161]: debug1: PAM: setting PAM_RHOST to "x.x.x.x" Mar 3 14:00:46 ourserver sshd[79161]: debug1: PAM: setting PAM_TTY to "ssh" Mar 3 14:00:46 ourserver sshd[79161]: debug1: userauth-request for user ouruser service ssh-connection method publickey [preauth] Mar 3 14:00:46 ourserver sshd[79161]: debug1: attempt 1 failures 0 [preauth] Mar 3 14:00:46 ourserver sshd[79161]: debug1: userauth_pubkey: test pkalg rsa-sha2-512 pkblob RSA SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU [preauth] Mar 3 14:00:46 ourserver sshd[79161]: debug1: temporarily_use_uid: 6915/200 (e=0/0) Mar 3 14:00:46 ourserver sshd[79161]: debug1: trying public key file /home/ouruser/.ssh/authorized_keys Mar 3 14:00:46 ourserver sshd[79161]: debug1: Could not open authorized keys '/home/ouruser/.ssh/authorized_keys': No such file or directory Mar 3 14:00:46 ourserver sshd[79161]: debug1: restore_uid: 0/0 Mar 3 14:00:46 ourserver sshd[79161]: debug1: temporarily_use_uid: 99/99 (e=0/0) Mar 3 14:00:46 ourserver sshd[79161]: debug1: restore_uid: 0/0 Mar 3 14:00:46 ourserver sshd[79161]: debug1: temporarily_use_uid: 99/99 (e=0/0) Mar 3 14:00:46 ourserver sshd[79161]: debug1: restore_uid: 0/0 Mar 3 14:00:46 ourserver sshd[79161]: Failed publickey for ouruser from x.x.x.x port 40248 ssh2: RSA SHA256:2ucGhU53Ue6Z8BbwowH5U3ykOoVL8F8oN1NbPUCt2vU Mar 3 14:00:46 ourserver sshd[79161]: debug1: userauth-request for user ouruser service ssh-connection method keyboard-interactive [preauth] Mar 3 14:00:46 ourserver sshd[79161]: debug1: attempt 2 failures 1 [preauth] Mar 3 14:00:46 ourserver sshd[79161]: debug1: keyboard-interactive devs [preauth] Mar 3 14:00:46 ourserver sshd[79161]: debug1: auth2_challenge: user=ouruser devs= [preauth] Mar 3 14:00:46 ourserver sshd[79161]: debug1: kbdint_alloc: devices 'pam' [preauth] Mar 3 14:00:46 ourserver sshd[79161]: debug1: auth2_challenge_start: trying authentication method 'pam' [preauth] Mar 3 14:00:46 ourserver sshd[79161]: Postponed keyboard-interactive for ouruser from x.x.x.x port 40248 ssh2 [preauth] Mar 3 14:00:50 ourserver sshd[79168]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser Mar 3 14:00:50 ourserver sshd[79168]: pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x user=ouruser Mar 3 14:00:50 ourserver sshd[79168]: pam_sss(sshd:auth): received for user ouruser: 9 (Authentication service cannot retrieve authentication info) Mar 3 14:00:52 ourserver sshd[79161]: error: PAM: Authentication failure for ouruser from x.x.x.x Mar 3 14:00:52 ourserver sshd[79161]: Failed keyboard-interactive/pam for ouruser from x.x.x.x port 40248 ssh2 Mar 3 14:00:52 ourserver sshd[79161]: debug1: userauth-request for user ouruser service ssh-connection method keyboard-interactive [preauth] Mar 3 14:00:52 ourserver sshd[79161]: debug1: attempt 3 failures 2 [preauth] Mar 3 14:00:52 ourserver sshd[79161]: debug1: keyboard-interactive devs [preauth] Mar 3 14:00:52 ourserver sshd[79161]: debug1: auth2_challenge: user=ouruser devs= [preauth] Mar 3 14:00:52 ourserver sshd[79161]: debug1: kbdint_alloc: devices 'pam' [preauth] Mar 3 14:00:52 ourserver sshd[79161]: debug1: auth2_challenge_start: trying authentication method 'pam' [preauth] Here are the server logs on a successful login with the NIS password: Mar 3 14:06:09 ourserver sshd[79292]: Accepted keyboard-interactive/pam for ouruser from xx.xx.xx.xx port 40252 ssh2 Mar 3 14:06:09 ourserver sshd[79292]: debug1: monitor_child_preauth: ouruser has been authenticated by privileged process Mar 3 14:06:09 ourserver sshd[79292]: debug1: monitor_read_log: child log fd closed Mar 3 14:06:09 ourserver sshd[79292]: debug1: audit_event: unhandled event 2 Mar 3 14:06:09 ourserver sshd[79292]: debug1: temporarily_use_uid: 6915/200 (e=0/0) Mar 3 14:06:09 ourserver sshd[79292]: debug1: *ssh_gssapi_storecreds: Not a GSSAPI mechanism* Mar 3 14:06:09 ourserver sshd[79292]: debug1: restore_uid: 0/0 Mar 3 14:06:09 ourserver sshd[79292]: debug1: SELinux support disabled Mar 3 14:06:09 ourserver sshd[79292]: debug1: PAM: establishing credentials Mar 3 14:06:09 ourserver systemd[79307]: pam_unix(systemd-user:session): session opened for user ouruser(uid=6915) by (uid=0) Mar 3 14:06:10 ourserver sshd[79292]: pam_unix(sshd:session): session opened for user ouruser(uid=6915) by (uid=0) Mar 3 14:06:10 ourserver sshd[79292]: User child is on pid 79320 Mar 3 14:06:10 ourserver sshd[79320]: debug1: PAM: establishing credentials So it clearly says it's "Not a GSSAPI mechanism". /etc/nsswitch.conf (which is a symbolic link to /etc/authselect/nsswitch.conf) passwd: sss files systemd group: sss files systemd netgroup: sss files automount: sss files services: sss files sudoers: files sss shadow: files nis hosts: files nis mdns4_minimal [NOTFOUND=return] dns myhostname mymachines And /etc/authselect/user-nsswitch.conf has passwd: files nis systemd shadow: files nis group: files nis systemd hosts: files nis mdns4_minimal [NOTFOUND=return] dns myhostname mymachines Why is the Kerberos login failing?

Robert Kudyba via FreeIPA-users wrote: I believe we've made some progress but not quite there yet. Just to recap, any NEW user created via CLI or GUI can connect via ssh. All imported NIS users can only log in with their NIS password. I change the user's password in the UI and check the Password checkbox in User authentication type and click Save. I successfully added a client: ipa host-add-managedby --hosts= ourdomain.edu client.ourdomain.edu Host name: client.ourdomain.edu Platform: x86_64 Operating system: 5.10.9-201.fc33.x86_64 Principal name: host/client.ourdomain.edu(a)OURDOMAIN.EDU Principal alias: host/ client.ourdomain.edu(a)OURDOMAIN.EDU Managed by: client.ourdomain.edu, ourdomain.edu ------------------------- Number of members added 1 ------------------------- [root@ourdomain ~]# ipa-getkeytab -s ourdomain.edu -p host/ client.ourdomain.edu -k /tmp/client.keytab > Keytab successfully retrieved and stored in: /tmp/client.keytab This is why SSSD isn't working. SSSD uses the host keytab in /etc/krb5.keytab and you invalidated it with the above command.

> Based on this SF discussion > < https://urldefense.proofpoint.com/v2/url?u=https-3A__serverfault.com_ques... >, > I changed: in /etc/krb5.conf > default_ccache_name = FILE:/tmp/krb5cc_%{uid} I don't think this is necessary.

> > Keytab successfully retrieved and stored in: /tmp/client.keytab > > This is why SSSD isn't working. SSSD uses the host keytab in > /etc/krb5.keytab and you invalidated it with the above command. > > > OK what do I need to do to fix this? I got this > from https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o... > (which I realize is old), Please do not use these documents. We'd remove them if it were in our control.

sync up the keys.

> Does this have to do with your comment above about SSSD not working? Yes. A keytab is a password and this is effectively a "bad password" error.

Still seeing: preauth (spake) verify failure: Preauthentication failed kvno ldap/ourdomain kvno = 2 kvno http/ourdomain kvno = 1 klist -k Keytab name: FILE:/etc/krb5.keytab KVNO Principal ---- -------------------------------------------------------------------------- 2 host/ourdomain.edu(a)OURDOMAIN.EDU 2 host/ourdomain.edu(a)OURDOMAIN.EDU 2 host/ourdomain.edu(a)OURDOMAIN.EDU 2 host/ourdomain.edu(a)OURDOMAIN.EDU 2 host/ourdomain.edu(a)OURDOMAIN.EDU 2 host/ourdomain.edu(a)OURDOMAIN.EDU 4 host/ourclient.edu(a)OURCLIENT.EDU 4 host/ourclient.edu(a)OURCLIENT.EDU 5 host/ourclient.edu(a)OURCLIENT.EDU 5 host/ourclient.edu(a)OURCLIENT.EDU 6 host/ourclient.edu(a)OURCLIENT.EDU 6 host/ourclient.edu(a)OURCLIENT.EDU 7 host/ourclient.edu(a)OURCLIENT.EDU 7 host/ourclient.edu(a)OURCLIENT.EDU On Thu, Mar 11, 2021 at 3:47 PM Robert Kudyba <rkudyba(a)fordham.edu&gt; wrote: > > > Keytab successfully retrieved and stored in: /tmp/client.keytab >> > >> > This is why SSSD isn't working. SSSD uses the host keytab in >> > /etc/krb5.keytab and you invalidated it with the above command. >> > >> > >> > OK what do I need to do to fix this? I got this >> > from >> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.fedoraproject.o... >> > (which I realize is old), >> >> Please do not use these documents. We'd remove them if it were in our >> control. >> > > Understood. > > > You can re-run your getkeytab command using /etc/krb5.keytab instead to >> sync up the keys. >> > > OK I ran: > > ipa-getkeytab -s ourdomain.edu -p host/client.ourdomain.edu -k > /etc/krb5.keytab > Keytab successfully retrieved and stored in: /etc/krb5.keytab > > Then I scp the keytab to the client /etc directory. Now in the krb5.log I > see: > > Mar 11 15:39:29 ourdomain.edu krb5kdc[369141](info): TGS_REQ (6 etypes > {aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), > camellia256-cts-cmac(26), aes128-cts-hmac-sha1-96(17), > aes128-cts-hmac-sha256-128(19), camellia128-cts-cmac(25)}) 150.108.64.55: > LOOKING_UP_SERVER: authtime 0, etypes {rep=UNSUPPORTED:(0)} > host/client.ourdomain edu(a)OURDOMAIN.EDU for DNS/ > ds-dc31.ds.ourdomain.edu(a)OURDOMAIN.EDU, Server not found in Kerberos > database > Mar 11 15:39:29 ourdomain.edu krb5kdc[369141](info): closing down fd 11 > > ssh logs: > Mar 11 15:39:27 ourdomain sshd[375517]: debug1: userauth-request for user > ouruser service ssh-connection method keyboard-interactive [preauth] > Mar 11 15:39:27 ourdomain sshd[375517]: debug1: attempt 2 failures 1 > [preauth] > Mar 11 15:39:27 ourdomain sshd[375517]: debug1: keyboard-interactive devs > [preauth] > Mar 11 15:39:27 ourdomain sshd[375517]: debug1: auth2_challenge: > user=ouruser devs= [preauth] > Mar 11 15:39:27 ourdomain sshd[375517]: debug1: kbdint_alloc: devices > 'pam' [preauth] > Mar 11 15:39:27 ourdomain sshd[375517]: debug1: auth2_challenge_start: > trying authentication method 'pam' [preauth] > Mar 11 15:39:27 ourdomain sshd[375517]: Postponed keyboard-interactive > for ouruser from x.x.x.x port 44136 ssh2 [preauth] > Mar 11 15:39:30 ourdomain sshd[375524]: pam_unix(sshd:auth): > authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x > user=ouruser > Mar 11 15:39:30 ourdomain sshd[375524]: pam_sss(sshd:auth): > authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=x.x.x.x > user=ouruser > Mar 11 15:39:30 ourdomain sshd[375524]: pam_sss(sshd:auth): received for > user ouruser: 9 (Authentication service cannot retrieve authentication info) > > >> > Does this have to do with your comment above about SSSD not working? >> >> Yes. A keytab is a password and this is effectively a "bad password" >> error. > > > Do I have to restart any services? >

Robert Kudyba wrote: > I'd like to provide an update. I can get ssh -k to work but here's what > I had to do: > 1. I had to run ipa-client-install on another server/computer > 2. I ran kinit ouruser(a)OURDOMAIN.EDU <mailto:ouruser@OURDOMAIN.EDU> > 3. I could then run ssh -k ouruser(a)ourdomain.edu > <mailto:ouruser@ourdomain.edu> and automatically logged in without > needing to enter a password. > > My question is, how does this scale to users, i.e., in our case, > students, who are all over the world using their own laptops? Does every > user client, i.e., computer, need to run ipa-client-install? Am I > missing something? It depends on what the expectations are for these user-owned machines.

If you don't need IPA identities and IPA users won't log into them, then they only need a working krb5.conf and DNS configured on them.

So your students would log into their own controlled machine using their own local account, kinit student123(a)univ.edu and ssh using their credentials. The krb5.conf will tell the student machine how to contact the KDC. That's all that is necessary (beyond working DNS).

> It depends on what the expectations are for these user-owned machines. > > > Only expectation is to be able to log in to a server, get access to > their home directory and be able to do their assignments, e.g., C++, > Java or Python programming. > > > If you don't need IPA identities and IPA users won't log into them, then > they only need a working krb5.conf and DNS configured on them. > > > So each device needs to drop in the krb5.conf file from the FreeIPA > server? How does this work on a Windows client? From the server? I wouldn't. It is likely going to need some hand-tuning depending on your configuration. For example the server is going to have a hardcoded KDC in it. You may or may not want that.

> > So your students would log into their own controlled machine using their > own local account, kinit student123(a)univ.edu > <mailto:student123@univ.edu> and ssh using their > credentials. > > The krb5.conf will tell the student machine how to contact the KDC. > That's all that is necessary (beyond working DNS). > > > I just tried this on another Fedora 33 workstation, dropped in the > /etc/krb5.conf file and all I get is: > kinit: No KCM server found while getting default ccache You can comment the values out in /etc/krb5.conf.d/kcm_default_ccache to change the default ccache type, or comment out the includes in krb5.conf (probably easier).

> I'm puzzled as to what we'd need to tell/provide to a student, who is > enrolled remotely and can't come on campus, to be able to connect to our > server via their Windows or Mac laptop. I don't know about Windows. I used the Windows MIT Kerberos packages a decade or more ago and they worked fine with PuTTY (and IPA with discovery) but whether that applies now or not I have no idea. Mac I think should work similar to Linux: provide a krb5.conf and things should just work. Again, you'll likely have to tweak the configuration depending on what version of MIT Mac ships these days.

Robert Kudyba via FreeIPA-users wrote: > > > On Tue, Mar 16, 2021 at 3:40 PM Rob Crittenden <rcritten(a)redhat.com > <mailto:rcritten@redhat.com>> wrote: > > > It depends on what the expectations are for these user-owned > machines. > > > > > > Only expectation is to be able to log in to a server, get access to > > their home directory and be able to do their assignments, e.g., C++, > > Java or Python programming. > > > > > > If you don't need IPA identities and IPA users won't log into > them, then > > they only need a working krb5.conf and DNS configured on them. > > > > > > So each device needs to drop in the krb5.conf file from the FreeIPA > > server? How does this work on a Windows client? > > From the server? I wouldn't. It is likely going to need some hand-tuning > depending on your configuration. For example the server is going to have > a hardcoded KDC in it. You may or may not want that. > > > So we have to customized the /etc/krb5.conf file that exists on the > server for any student devices. I mean, you don't want to use ipa-client-install which would do all of this for you, and I understand the reasons, but it does mean some additional work on your part. I don't know your network so at most I can make general suggestions, not provide you a full configuration.

In retrospect the default krb5.conf that ships on Fedora provides for includes. I think this is probably your best bet: provide an IPA configuration that resides there and it should co-exist pretty easily with any other configuration. I'm not completely sure about the order of loading and which configuration "wins" when there is conflict. The man page is the place to look. And kcm_default_ccache has instructions on how to enable/run sssd-kcm so that this should work out-of-the-box. That is probably better than having students comment it out, unless you can control the order of what "wins" when there is conflicting configuration.

> So your students would log into their own controlled machine > using their > > own local account, kinit student123(a)univ.edu > <mailto:student123@univ.edu> > > <mailto:student123@univ.edu <mailto:student123@univ.edu>> and > ssh using their > > credentials. > > > > The krb5.conf will tell the student machine how to contact the > KDC. > > That's all that is necessary (beyond working DNS). > > > > > > I just tried this on another Fedora 33 workstation, dropped in the > > /etc/krb5.conf file and all I get is: > > kinit: No KCM server found while getting default ccache > > You can comment the values out in /etc/krb5.conf.d/kcm_default_ccache to > change the default ccache type, or comment out the includes in krb5.conf > (probably easier). > > > OK now I can get any Fedora client to kinit and then ssh. See about for perhaps a less hacky approach than I originally suggested.

> > I'm puzzled as to what we'd need to tell/provide to a student, who is > > enrolled remotely and can't come on campus, to be able to connect > to our > > server via their Windows or Mac laptop. > > I don't know about Windows. I used the Windows MIT Kerberos packages a > decade or more ago and they worked fine with PuTTY (and IPA with > discovery) but whether that applies now or not I have no idea. > > Mac I think should work similar to Linux: provide a krb5.conf and things > should just work. Again, you'll likely have to tweak the configuration > depending on what version of MIT Mac ships these days. > > > kinit --version > > kinit (Heimdal 1.5.1apple1) > > > So my first test with the server krb5.conf file copied into /etc: > > kinit: krb5_get_init_creds: unable to reach any KDC in realm > OURDOMAIN.EDU < https://urldefense.proofpoint.com/v2/url?u=http-3A__OURDOMAIN.EDU&d=D... >, tried 0 KDCs > > > So the first suggestion < https://urldefense.proofpoint.com/v2/url?u=https-3A__apple.stackexchange.... > I > found was to preface kdc = tcp > > Then I made sure the firewall on the Mac was disabled. I also added the > test IPA server & IP into /etc/hosts. I can ping it successfully. > > What else needs to change? It's difficult to troubleshoot in a void. I don't know your network configuration nor what krb5.conf you're using. It sure looks like discovery of the KDC over DNS failed.

> I configured the following in krb5.conf and now at least get prompted > for a password and kinit works!: > [libdefaults] > dns_lookup_kdc = no > dns_lookup_realm = no > > klist > Ticket cache: API:krb5cc > Default principal: ouruser(a)OURDOMAIN.EDU <mailto:ouruser@OURDOMAIN.EDU> > > Valid starting Expires Service principal > 03/18/21 15:17:43 03/19/21 15:17:39 krbtgt/OURDOMAIN.EDU(a)OURDOMAIN.EDU > <mailto:OURDOMAIN.EDU@OURDOMAIN.EDU> I don't know why mac/Windows isn't working. It doesn't look like it is even trying GSSAPI.