I started fresh again. I installed and configured gluu-server-3.1.2. Here is the install error log:
```
cat setup_error.log
16:05:31 05/05/18 Installing Gluu Server
16:07:55 05/05/18 Generating RSA private key, 2048 bit long modulus
.................+++
........+++
e is 65537 (0x10001)
16:07:55 05/05/18 writing RSA key
16:07:55 05/05/18 Signature ok
subject=/C=US/ST=PA/L=Media/O=jeffmandel.org/CN=gluu.jeffmandel.org/emailAddress=jeffemandel@jeffmandel.org
Getting Private key
16:07:55 05/05/18 Certificate was added to keystore
16:07:55 05/05/18 Generating RSA private key, 2048 bit long modulus
..................................+++
...............................................+++
e is 65537 (0x10001)
16:07:55 05/05/18 writing RSA key
16:07:55 05/05/18 Signature ok
subject=/C=US/ST=PA/L=Media/O=jeffmandel.org/CN=gluu.jeffmandel.org/emailAddress=jeffemandel@jeffmandel.org
Getting Private key
16:07:55 05/05/18 Certificate was added to keystore
16:07:55 05/05/18 Generating RSA private key, 2048 bit long modulus
...............................................................+++
..+++
e is 65537 (0x10001)
16:07:55 05/05/18 writing RSA key
16:07:55 05/05/18 Signature ok
subject=/C=US/ST=PA/L=Media/O=jeffmandel.org/CN=gluu.jeffmandel.org/emailAddress=jeffemandel@jeffmandel.org
Getting Private key
16:07:56 05/05/18 Certificate was added to keystore
16:07:56 05/05/18 Generating RSA private key, 2048 bit long modulus
................................................................................................................+++
........................................................................................................................................................+++
e is 65537 (0x10001)
16:07:56 05/05/18 writing RSA key
16:07:56 05/05/18 Signature ok
subject=/C=US/ST=PA/L=Media/O=jeffmandel.org/CN=gluu.jeffmandel.org/emailAddress=jeffemandel@jeffmandel.org
Getting Private key
16:07:56 05/05/18 Certificate was added to keystore
16:07:56 05/05/18 Generating RSA private key, 2048 bit long modulus
.............................................................................+++
.....................................................................................+++
e is 65537 (0x10001)
16:07:56 05/05/18 writing RSA key
16:07:56 05/05/18 Signature ok
subject=/C=US/ST=PA/L=Media/O=jeffmandel.org/CN=gluu.jeffmandel.org/emailAddress=jeffemandel@jeffmandel.org
Getting Private key
16:07:57 05/05/18 Certificate was added to keystore
16:07:57 05/05/18 Generating RSA private key, 2048 bit long modulus
..............................................................+++
............................+++
e is 65537 (0x10001)
16:07:57 05/05/18 writing RSA key
16:07:57 05/05/18 Signature ok
subject=/C=US/ST=PA/L=Media/O=jeffmandel.org/CN=localhost/emailAddress=jeffemandel@jeffmandel.org
Getting Private key
16:07:57 05/05/18 Certificate was added to keystore
16:07:57 05/05/18 Generating RSA private key, 2048 bit long modulus
.....+++
.............................................+++
e is 65537 (0x10001)
16:07:57 05/05/18 writing RSA key
16:07:57 05/05/18 Signature ok
subject=/C=US/ST=PA/L=Media/O=jeffmandel.org/CN=localhost/emailAddress=jeffemandel@jeffmandel.org
Getting Private key
16:07:57 05/05/18 Certificate was added to keystore
16:07:57 05/05/18 Entry for alias gluu.jeffmandel.org successfully imported.
Import command completed: 1 entries successfully imported, 0 entries failed or cancelled
16:07:58 05/05/18 Entry for alias gluu.jeffmandel.org successfully imported.
Import command completed: 1 entries successfully imported, 0 entries failed or cancelled
16:07:58 05/05/18 Entry for alias gluu.jeffmandel.org successfully imported.
Import command completed: 1 entries successfully imported, 0 entries failed or cancelled
16:08:28 05/05/18 Running in chroot, ignoring request.
16:08:33 05/05/18 INFO: ext initialised in ${jetty.base}/start.ini
INFO: logging initialised in ${jetty.base}/start.ini
INFO: server initialised (transitively) in ${jetty.base}/start.ini
INFO: http initialised in ${jetty.base}/start.ini
INFO: http-forwarded initialised in ${jetty.base}/start.ini
INFO: servlets initialised in ${jetty.base}/start.ini
INFO: deploy initialised in ${jetty.base}/start.ini
INFO: jsp initialised in ${jetty.base}/start.ini
INFO: websocket initialised in ${jetty.base}/start.ini
MKDIR: ${jetty.base}/logs
MKDIR: ${jetty.base}/webapps
INFO: Base directory was modified
16:08:34 05/05/18 INFO: ext initialised in ${jetty.base}/start.ini
INFO: logging initialised in ${jetty.base}/start.ini
INFO: server initialised (transitively) in ${jetty.base}/start.ini
INFO: http initialised in ${jetty.base}/start.ini
INFO: http-forwarded initialised in ${jetty.base}/start.ini
INFO: deploy initialised in ${jetty.base}/start.ini
INFO: jsp initialised in ${jetty.base}/start.ini
INFO: websocket initialised in ${jetty.base}/start.ini
MKDIR: ${jetty.base}/logs
MKDIR: ${jetty.base}/webapps
INFO: Base directory was modified
```
I can access the appliance and it seems to work. Now I install cred-manager. Here is the log:
```
16:18:05 05/05/18 Removed ./setup_cred_mgr.log
16:18:05 05/05/18 Removed ./setup_cred_mgr_error.log
16:18:05 05/05/18 Installing Credential Manager
16:18:05 05/05/18 ./setup_cred_mgr.properties Properties not found. Interactive setup commencing...
16:19:49 05/05/18 Checking properties
16:19:55 05/05/18 Downloading files
16:19:55 05/05/18 Calculating memory setting for applications
16:19:55 05/05/18 Rendering templates
16:19:55 05/05/18 Rendering template cred-manager
16:19:55 05/05/18 Rendering template cred-manager.json
16:19:55 05/05/18 Configuring Credential manager...
16:19:55 05/05/18 Copied ./output/result/cred-manager.json to /etc/gluu/conf
16:19:55 05/05/18 Copying cred-manager.war into jetty webapps folder...
16:19:55 05/05/18 Installing jetty service cred-manager...
16:19:55 05/05/18 Preparing cred-manager service base folders
16:19:55 05/05/18 Running: /bin/mkdir -p /opt/gluu/jetty/cred-manager
16:19:55 05/05/18 Preparing cred-manager service base configuration
16:19:55 05/05/18 Running: /opt/jre/bin/java -jar /opt/jetty/start.jar jetty.home=/opt/jetty jetty.base=/opt/gluu/jetty/cred-manager --add-to-start=deploy,http,logging,jsp,http-forwarded
16:19:55 05/05/18 INFO: logging initialised in ${jetty.base}/start.ini
INFO: server initialised (transitively) in ${jetty.base}/start.ini
INFO: http initialised in ${jetty.base}/start.ini
INFO: http-forwarded initialised in ${jetty.base}/start.ini
INFO: deploy initialised in ${jetty.base}/start.ini
INFO: jsp initialised in ${jetty.base}/start.ini
MKDIR: ${jetty.base}/logs
MKDIR: ${jetty.base}/webapps
INFO: Base directory was modified
16:19:55 05/05/18 Running: /bin/chown -R jetty:jetty /opt/gluu/jetty/cred-manager
16:19:55 05/05/18 Rendering template cred-manager
16:19:55 05/05/18 Copied ./output/jetty/cred-manager to /etc/default
16:19:55 05/05/18 Running: /bin/chown root:root /etc/default/cred-manager
16:19:55 05/05/18 Copied /opt/jetty/bin/jetty.sh to /etc/init.d/cred-manager
16:19:55 05/05/18 Running: sed -i s/^# Provides: jetty/# Provides: cred-manager/ /etc/init.d/cred-manager
16:19:55 05/05/18 Running: /usr/sbin/update-rc.d cred-manager defaults 60 20
16:19:55 05/05/18 Copied /opt/dist/gluu/cred-manager.war to /opt/gluu/jetty/cred-manager/webapps
16:19:55 05/05/18 Copied /opt/dist/gluu/twilio-7.17.0.jar to /opt/gluu/jetty/oxauth/custom/libs
16:19:55 05/05/18 Running: /bin/chown -R jetty:jetty /opt/gluu/jetty/oxauth/custom/libs
16:19:55 05/05/18 Error reading None
16:19:55 05/05/18 Traceback (most recent call last):
File "/install/community-edition-setup/setup.py", line 763, in readFile
f = open(inFilePath)
IOError: [Errno 2] No such file or directory: '/etc/redhat-release'
16:19:55 05/05/18 Found package '/opt/dist/oxd/oxd-server_3.1.2-8~xenial+Ub16.04_all.deb' for install
16:19:55 05/05/18 Running: /usr/bin/dpkg --install /opt/dist/oxd/oxd-server_3.1.2-8~xenial+Ub16.04_all.deb
16:20:00 05/05/18 Selecting previously unselected package oxd-server.
(Reading database ... 18440 files and directories currently installed.)
Preparing to unpack .../oxd-server_3.1.2-8~xenial+Ub16.04_all.deb ...
Unpacking oxd-server (3.1.2-8~xenial+Ub16.04) ...
Setting up oxd-server (3.1.2-8~xenial+Ub16.04) ...
Processing triggers for systemd (229-4ubuntu16) ...
16:20:00 05/05/18 Running: chmod 644 -R /etc/oxd/oxd-server/oxd-conf.json
16:20:00 05/05/18 Running: /usr/sbin/update-rc.d oxd-server defaults 50 25
16:20:00 05/05/18 Running: chmod +x /opt/oxd-server/bin/oxd-start.sh
16:20:00 05/05/18 Running: /usr/sbin/service oxd-server start
16:20:01 05/05/18 Run: /usr/sbin/service oxd-server start with result code: 0
16:20:01 05/05/18 Running: /bin/chown -R root:jetty /etc/gluu/conf/cred-manager.json
16:20:01 05/05/18 Running: /bin/chmod g+w /etc/gluu/conf/cred-manager.json
16:20:01 05/05/18 Importing LDIF files into LDAP
16:20:01 05/05/18 Importing LDIF file './output/scripts_cred_manager.ldif' into OpenLDAP
16:20:01 05/05/18 Running: /bin/su ldap -c cd /install/community-edition-setup; /opt/symas/bin/slapadd -b o=gluu -f /opt/symas/etc/openldap/slapd.conf -l ./output/scripts_cred_manager.ldif
16:20:01 05/05/18 Running: /usr/sbin/service oxauth restart
16:20:18 05/05/18 Run: /usr/sbin/service oxauth restart with result code: 0
16:20:18 05/05/18 Running: /usr/sbin/service cred-manager start
16:20:28 05/05/18 Run: /usr/sbin/service cred-manager start with result code: 0
16:20:28 05/05/18 Running: mv -f /etc/rc3.d/S*cred-manager /etc/rc3.d/S97cred-manager
16:20:28 05/05/18 Run: mv -f /etc/rc3.d/S*cred-manager /etc/rc3.d/S97cred-manager with result code: 0
16:20:28 05/05/18 Running: mv -f /etc/rc3.d/S*oxd-server /etc/rc3.d/S91oxd-server
16:20:28 05/05/18 Run: mv -f /etc/rc3.d/S*oxd-server /etc/rc3.d/S91oxd-server with result code: 0
16:20:28 05/05/18 Saving properties to ./setup.properties.last
```
I can now access identity and cred-manager. Looking at oxauth.log, I see:
```
grep ERROR oxauth.log
2018-05-05 16:22:11,914 ERROR [qtp2008017533-10] [xdi.oxauth.authorize.ws.rs.AuthorizeRestWebServiceImpl] (AuthorizeRestWebServiceImpl.java:825) - Failed to load session from LDAP by session_id: '168d484a-eb8e-4dce-b3eb-ebf3d90159a4'
```
Troubling, but it's working. I restart oxauth from within the chroot. Still working. I come out of chroot, stop and start the gluu-server-3.1.2 service, and it comes back 503. Looking at oxauth.log again, I see:
```
2018-05-05 16:27:23,036 ERROR [qtp2008017533-16] [org.xdi.oxauth.auth.Authenticator] (Authenticator.java:517) - Failed to get attributes from session
```
Not to be deterred, I try stopping and starting oxd-server, oxauth, and cred-manager. Still 503. In total desperation, I restart identity, then oxauth. Gluu comes roaring back to life. I have confirmed that after a gluu-server-3.1.2 service restart, the only way to bring gluu back to life is to restart identity and then oxauth. I'm guessing reworking /etc/init.d/gluu-server-3.1.2 is the fix.
Hope this helps.