I was right I did fix all the ARM issues, but not in 1.3.8, only in 1.4.0.  It was a large change though that required a few patches.  I'll see what I can do about backporting the changes...


On 06/01/2018 05:38 PM, Jonathan Vaughn wrote:
Created https://pagure.io/389-ds-base/issue/49746

I'll get you the build log once it's done building without the patch.

On Fri, Jun 1, 2018 at 3:54 PM, Mark Reynolds <mreynolds@redhat.com> wrote:



On 06/01/2018 04:32 PM, Jonathan Vaughn wrote:
Alright, I think I've got everything* working. (* Not running the CA server on the Arm device, not tested, but from what I've read before I would need to adjust the startup timeout since OpenJDK is so slow).

1) I removed the Arm replica from the cluster and reimaged the device entirely and started over.

2) Rebuilt 389-ds-base using the patch by Mark Reynolds, and replica install succeeded (just as it did with the patch before). I did use the next version (1.3.8) of 389-ds-base because the package installed version bumped up to 1.3.8.1 - patch worked same as on 1.3.7

3) I also had to apply the fix to /etc/httpd/conf.d/ipa.conf from https://pagure.io/freeipa/issue/7337 so that Web UI login could work (I did this previously on the Arm device but still couldn't login - likely just too many things were tried and something else was busted elsewhere).

As far as I can tell everything seems to be working now.

I'm not sure if the patch provided would work as-is or if it would break on non-Arm architectures. 

The patch :

diff --git a/ldap/servers/plugins/replication/repl5_agmt.c b/ldap/servers/plugins/replication/repl5_agmt.c
index d71d3f38b..e0f1f41bd 100644
--- a/ldap/servers/plugins/replication/repl5_agmt.c
+++ b/ldap/servers/plugins/replication/repl5_agmt.c
@@ -3035,7 +3035,7 @@ agmt_update_maxcsn(Replica *r, Slapi_DN *sdn, int op, LDAPMod **mods, CSN *csn)
                 slapi_ch_free_string(&agmt->maxcsn);
                 agmt->maxcsn = slapi_ch_smprintf("%s;%s;%s;%ld;%d;%s", slapi_sdn_get_dn(agmt->replarea),
                                                  slapi_rdn_get_value_by_ref(slapi_rdn_get_rdn(agmt->rdn)), agmt->hostname,
-                                                 agmt->port, agmt->consumerRID, maxcsn);
+                                                 (long)agmt->port, (int)agmt->consumerRID, maxcsn);
             }
             PR_Unlock(agmt->lock);
         }

To get this fixed properly (to avoid building from source), should I open an issue in Pagure.io ? Or will someone else handle that?

Please file ticket:

https://pagure.io/389-ds-base/new_issue

Can you also do me a favor please?  Please build the server again, but without my patch, and send the build output to me.  I want to see if there are any compiler errors/warnings.  The issue is that I fixed all the issues on ARM, including this exact issue you ran into.  Its working for others without issue.  I was wondering if it was perhaps Raspberry's version of ARM  that was causing issues.  Anyway the build output (without my fix) would be very interesting to look at.

Thanks,
Mark


On Wed, May 23, 2018 at 4:04 PM, Jonathan Vaughn <jonathan@creatuity.com> wrote:
Anyone have any further suggestions for troubleshooting this issue with the web UI?

On Fri, May 18, 2018 at 4:01 PM, Jonathan Vaughn <jonathan@creatuity.com> wrote:
httpd error_log

[Fri May 18 15:58:29.546255 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: WSGI wsgi_dispatch.__call__:
[Fri May 18 15:58:29.547217 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: WSGI login_password.__call__:
[Fri May 18 15:58:29.549799 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Obtaining armor in ccache /var/run/ipa/ccaches/armor_14004
[Fri May 18 15:58:29.550812 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Initializing anonymous ccache
[Fri May 18 15:58:29.552268 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Starting external process
[Fri May 18 15:58:29.553377 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: args=/usr/bin/kinit -n -c /var/run/ipa/ccaches/armor_14004 -X X509_anchors=FILE:/var/kerberos/krb5kdc/kdc.crt -X X509_anchors=FILE:/var/lib/ipa-client/pki/kdc-ca-bundle.pem
[Fri May 18 15:58:30.486537 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Process finished, return code=0
[Fri May 18 15:58:30.488137 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: stdout=
[Fri May 18 15:58:30.489128 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: stderr=
[Fri May 18 15:58:30.490955 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Initializing principal admin using password
[Fri May 18 15:58:30.492060 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Using armor ccache /var/run/ipa/ccaches/armor_14004 for FAST webauth
[Fri May 18 15:58:30.493262 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Using enterprise principal
[Fri May 18 15:58:30.494728 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Starting external process
[Fri May 18 15:58:30.495799 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: args=/usr/bin/kinit admin -c /var/run/ipa/ccaches/kinit_14004 -T /var/run/ipa/ccaches/armor_14004 -E
[Fri May 18 15:58:30.721637 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Process finished, return code=0
[Fri May 18 15:58:30.723230 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: stdout=Password for admin@CREATUITY.INTERNAL:
[Fri May 18 15:58:30.723432 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284]
[Fri May 18 15:58:30.724345 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: stderr=
[Fri May 18 15:58:30.725716 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Cleanup the armor ccache
[Fri May 18 15:58:30.727071 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Starting external process
[Fri May 18 15:58:30.728152 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: args=/usr/bin/kdestroy -A -c /var/run/ipa/ccaches/armor_14004
[Fri May 18 15:58:30.798642 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Process finished, return code=0
[Fri May 18 15:58:30.800249 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: stdout=
[Fri May 18 15:58:30.801224 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: stderr=
[Fri May 18 15:58:30.857737 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Starting new HTTP connection (1): ipa-11.creatuity.internal
[Fri May 18 15:58:30.867978 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: http://ipa-11.creatuity.internal:80 "GET /ipa/session/cookie HTTP/1.1" 301 260
[Fri May 18 15:58:30.889415 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: Starting new HTTPS connection (1): ipa-11.creatuity.internal
[Fri May 18 15:58:31.105092 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: https://ipa-11.creatuity.internal:443 "GET /ipa/session/cookie HTTP/1.1" 200 0
[Fri May 18 15:58:31.157295 2018] [wsgi:error] [pid 14003:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: WSGI wsgi_dispatch.__call__:
[Fri May 18 15:58:31.158347 2018] [wsgi:error] [pid 14003:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: WSGI jsonserver_session.__call__:
[Fri May 18 15:58:31.159289 2018] [wsgi:error] [pid 14003:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: no ccache, need login
[Fri May 18 15:58:31.160349 2018] [wsgi:error] [pid 14003:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: 401 Unauthorized need login
[Fri May 18 15:58:31.175232 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: WSGI wsgi_dispatch.__call__:
[Fri May 18 15:58:31.176208 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: WSGI KerberosLogin.__call__:
[Fri May 18 15:58:31.177060 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: no ccache, need login
[Fri May 18 15:58:31.178047 2018] [wsgi:error] [pid 14004:tid 2903716672] [remote 10.10.11.3:59284] ipa: DEBUG: 401 Unauthorized need login


On Fri, May 18, 2018 at 3:53 PM, Rob Crittenden <rcritten@redhat.com> wrote:
Jonathan Vaughn wrote:
> System time appears to be fine.
>
> Here's the httpd access log (nothing in error log):
> 10.10.255.101 - - [18/May/2018:15:26:19 -0500] "GET /ipa/session/cookie
> HTTP/1.1" 301 260 "-" "python-requests/2.18.4"
> 10.10.255.101 - admin@CREATUITY.INTERNAL [18/May/2018:15:26:20 -0500]
> "GET /ipa/session/cookie HTTP/1.1" 200 -
> 10.10.11.3 - - [18/May/2018:15:26:18 -0500] "POST
> /ipa/session/login_password HTTP/1.1" 200 20
> 10.10.11.3 - admin@CREATUITY.INTERNAL [18/May/2018:15:26:20 -0500] "POST
> /ipa/session/json HTTP/1.1" 401 -
> 10.10.11.3 - admin@CREATUITY.INTERNAL [18/May/2018:15:26:20 -0500] "GET
> /ipa/session/login_kerberos?_=1526674892374 HTTP/1.1" 401 -
>

You'll want to look at the error log for more details. For even more
output create /etc/ipa/server.conf with the contents:

[global]
debug = True

and restart httpd

rob

>
> On Fri, May 18, 2018 at 3:23 PM, Jonathan Vaughn <jonathan@creatuity.com
> <mailto:jonathan@creatuity.com>> wrote:
>
>     Actually - may have spoke too soon. I thought I'd signed in to the
>     UI but I was actually in another server's UI. When I try to log in
>     it says that my login has expired... so progress but something else
>     is busted.
>
>     On Fri, May 18, 2018 at 3:20 PM, Jonathan Vaughn
>     <jonathan@creatuity.com <mailto:jonathan@creatuity.com>> wrote:
>
>         Looks like replica install finished just fine, and I can access
>         the the web UI on it. No obvious problems. 
>
>         So it would appear that code change fixed 389DS for ARM. 
>
>         I'm not running the Dogtag CA on the Pi, just on the non-Pi
>         servers, otherwise would need to probably adjust the startup
>         timeout for it so that the install script didn't give up waiting
>         on it. When I was first trying to play with this I tried
>         installing the first replica/master on the Pi, and ran into the
>         Dogtag timeout problem. Even installing Oracle Java instead of
>         the OpenJDK version of Java didn't solve it, because somehow it
>         always picks the OpenJDK java instead of oracle even if I change
>         the alternatives setting to default to the Oracle one... 
>
>         But as far as I can tell otherwise, everything is working as
>         expected.