502 Gateway Error - Failed to load configuration from LDAP - Gluu Docker Multi-node

By: Sathish Kumar S . user 26 Nov 2018 at 12:55 a.m. CST

17 Responses
Sathish Kumar S . gravatar
We are implementing Gluu Docker Multi-node (3.1.3 - Master branch) in Swarm (without docker-machine). Note: Single-Node stack works fine in Docker Swarm. We are getting "502 gateway error" in nginx as it could not load https://localhost:8080/identity. As the Oxtrust is not loaded completely the nginx conf not getting updated, this is the cause of issue, ## Log Oxtrust, Log Set 1: ``` Nov 26 08:42:49 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] 2018-11-26 08:42:49.763:INFO:oeja.AnnotationConfiguration:main: Scanning elapsed time=22131ms Nov 26 08:42:50 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] 2018-11-26 08:42:50.217:WARN:oeja.WebServletAnnotation:main: org.jboss.resteasy.plugins.server.servlet.HttpServlet30Dispatcher defines neither @WebServlet .value nor @WebServlet.urlPatterns Nov 26 08:42:54 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] Nov 26, 2018 8:42:54 AM org.richfaces.webapp.ResourceServletContainerInitializer registerServlet Nov 26 08:42:54 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] INFO: Auto-registered servlet ResourceServlet with mapping '/org.richfaces.resources/*' Nov 26 08:43:17 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] 2018-11-26 08:43:17,403 INFO [main] [org.gluu.oxtrust.util.BuildVersion] (BuildVersion.java:82) - Root element :beans Nov 26 08:43:17 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] 2018-11-26 08:43:17,758 INFO [main] [org.xdi.oxauth.model.util.SecurityProviderUtility] (SecurityProviderUtility.java:23) - Adding Bouncy Castle Provider Nov 26 08:43:18 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] 2018-11-26 08:43:18,088 INFO [main] [org.gluu.oxtrust.ldap.service.AppInitializer] (AppInitializer.java:275) - Build date 2018-11-09 20:29. Code revision 7a98d4cb9bbd1ba24f0486fb88c1ef86cdeeab69 on ${git.commit.time}. Build 302 Nov 26 08:43:18 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] 2018-11-26 08:43:18,111 INFO [main] [org.gluu.oxtrust.config.ConfigurationFactory] (ConfigurationFactory.java:134) - Creating oxTrustConfiguration Nov 26 08:43:18 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] 2018-11-26 08:43:18,115 INFO [main] [org.gluu.oxtrust.config.ConfigurationFactory] (ConfigurationFactory.java:420) - ########## ldapFileName = /etc/gluu/ conf/ox-ldap.properties Nov 26 08:43:19 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] 2018-11-26 08:43:19,744 INFO [main] [org.gluu.oxtrust.config.ConfigurationFactory] (ConfigurationFactory.java:357) - Loading configuration from LDAP... Nov 26 08:43:20 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] 2018-11-26 08:43:20,018 INFO [main] [org.gluu.oxtrust.ldap.service.AppInitializer] (AppInitializer.java:310) - Created ldapEntryManager: org.gluu.site.ld ap.OperationsFacade@7ffcd8df Nov 26 08:43:20 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] 2018-11-26 08:43:20,388 ERROR [main] [org.gluu.oxtrust.config.ConfigurationFactory] (ConfigurationFactory.java:405) - Failed to load configuration from LD AP Nov 26 08:43:20 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] org.gluu.site.ldap.persistence.exception.MappingException: Failed to convert json value '{ Nov 26 08:43:20 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] "orgInum":"@!B25B.4280.6C89.D07D!0001!0AB3.44F0", Nov 26 08:43:20 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/554cf7 baaa73[17126] "orgIname":"", ``` Log Set 2: ``` Nov 25 20:05:49 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/683bbdc223d8[17126] 2018-11-25 20:05:49,448 WARN [main] [org.gluu.oxtrust.config.ConfigurationFactory] (ConfigurationFactory.java:381) - Unable to find configuration in LDA , try to load configuration from file system... Nov 25 20:05:49 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/683bbdc223d8[17126] #011at org.gluu.oxtrust.config.ConfigurationFactory.createFromLdap(ConfigurationFactory.java:382) [classes/:?] Nov 25 20:05:49 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/683bbdc223d8[17126] #011at org.gluu.oxtrust.ldap.service.AppInitializer.applicationInitialized(AppInitializer.java:203) [classes/:?] Nov 25 20:05:49 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/683bbdc223d8[17126] 2018-11-25 20:05:49,473 ERROR [main] [org.gluu.oxtrust.config.ConfigurationFactory] (ConfigurationFactory.java:153) - Failed to load configuration from LDAP. Please fix it!!!. Nov 25 20:05:49 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/683bbdc223d8[17126] org.xdi.exception.ConfigurationException: Failed to load configuration from LDAP. Nov 25 20:05:49 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/683bbdc223d8[17126] #011at org.gluu.oxtrust.ldap.service.AppInitializer.applicationInitialized(AppInitializer.java:203) Nov 25 20:05:49 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/683bbdc223d8[17126] java.lang.RuntimeException: org.xdi.exception.ConfigurationException: Failed to load configuration from LDAP. Nov 25 20:05:49 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/683bbdc223d8[17126] org.xdi.exception.ConfigurationException: Failed to load configuration from LDAP. Nov 25 20:05:49 manager0 oxtrust/nexus-versa.contus.us:8083/gluufederation-oxtrust:latest@sha256:994f9e177b1474347bf8ca8c1270699b2b3aab4dcd8aff5189549e8b8cfc4c26/683bbdc223d8[17126] #011at org.gluu.oxtrust.ldap.service.AppInitializer.applicationInitialized(AppInitializer.java:203) ``` ## Working Scenario The issue does not occurs when we install Consul as standalone (non-cluster), * ***Consul non cluster (Only one consul in manager0)*** * Redis standalone * Only Ldap master * Oxauth global * Oxtrust Global * Oxshibboleth Global * Nginx Global * Update Backup Script * Deployed Ldap Replications ## Error Scenario The issue occurs when we install Consul as cluster, * ***Consul cluster (Cause of issue)*** * Redis standalone * Only Ldap master * Oxauth global * Oxtrust Global * Oxshibboleth Global * Nginx Global Kindly let us know what mistake we are doing in Consul cluster, why oxtrust could not Load configuration from LDAP when consul is clustered. How to track this? Note: Issue persist in both 3.1.3-Master and 3.1.4_01 Thanks in Advance.
Ubuntu 16.04
closed

Answers

By Chris Blanton user 26 Nov 2018 at 8:32 a.m. CST

Copy
Chris Blanton gravatar
> Kindly let us know what mistake we are doing in Consul cluster, why oxtrust could not Load configuration from LDAP when consul is clustered. How to track this? Are you sure you LDAP is started in your secondary scenario?

By Sathish Kumar S . user 27 Nov 2018 at 12:22 a.m. CST

Copy
Sathish Kumar S . gravatar
Chris, Yes LDAP is up, i can even see LDAP access log when Oxtrust trying to load configuration. This is the exact issue the Oxtrust configuration got loaded, but could not convert the JSON string to object, ``` Nov 26 09:27:25 2018-11-26 09:27:25,042 INFO [main] [org.gluu.oxtrust.util.BuildVersion] (BuildVersion.java:82) - Root element :beans Nov 26 09:27:25 2018-11-26 09:27:25,359 INFO [main] [org.xdi.oxauth.model.util.SecurityProviderUtility] (SecurityProviderUtility.java:23) - Adding Bouncy Castle Provider Nov 26 09:27:25 2018-11-26 09:27:25,638 INFO [main] [org.gluu.oxtrust.ldap.service.AppInitializer] (AppInitializer.java:275) - Build date 2018-11-09 20:29. Code revision 7a98d4cb9bbd1ba24f0486fb88c1ef86cdeeab69 on ${git.commit.time}. Build 302 Nov 26 09:27:25 2018-11-26 09:27:25,667 INFO [main] [org.gluu.oxtrust.config.ConfigurationFactory] (ConfigurationFactory.java:134) - Creating oxTrustConfiguration Nov 26 09:27:25 2018-11-26 09:27:25,670 INFO [main] [org.gluu.oxtrust.config.ConfigurationFactory] (ConfigurationFactory.java:420) - ########## ldapFileName = /etc/gluu/conf/ox-ldap.properties Nov 26 09:27:27 2018-11-26 09:27:27,097 INFO [main] [org.gluu.oxtrust.config.ConfigurationFactory] (ConfigurationFactory.java:357) - Loading configuration from LDAP... Nov 26 09:27:27 2018-11-26 09:27:27,325 INFO [main] [org.gluu.oxtrust.ldap.service.AppInitializer] (AppInitializer.java:310) - Created ldapEntryManager: org.gluu.site.ldap.OperationsFacade@2a2632d9 Nov 26 09:27:27 2018-11-26 09:27:27,643 ERROR [main] [org.gluu.oxtrust.config.ConfigurationFactory] (ConfigurationFactory.java:405) - Failed to load configuration from LDAP Nov 26 09:27:27 org.gluu.site.ldap.persistence.exception.MappingException: Failed to convert json value '{ Nov 26 09:27:27 "orgInum":"@!B25B.4280.6C89.D07D!0001!0AB3.44F0", Nov 26 09:27:27 "orgIname":"", Nov 26 09:27:27 "orgSupportEmail":"sathishkumar@contus.in", Nov 26 09:27:27 Nov 26 09:27:27 "applianceInum":"@!B25B.4280.6C89.D07D!0002!D35E.8179", Nov 26 09:27:27 Nov 26 09:27:27 "baseDN":"o=gluu", Nov 26 09:27:27 Nov 26 09:27:27 "baseEndpoint":"https://192.168.7.234/identity/restv1", Nov 26 09:27:27 Nov 26 09:27:27 "idpUrl":"https://192.168.7.234", Nov 26 09:27:27 "applianceUrl":"https://192.168.7.234", Nov 26 09:27:27 Nov 26 09:27:27 "keystorePath":"/etc/certs/shibIDP.jks", Nov 26 09:27:27 "keystorePassword":"w25qnNpFWbXp", Nov 26 09:27:27 Nov 26 09:27:27 "personObjectClassTypes":[ Nov 26 09:27:27 "gluuCustomPerson", Nov 26 09:27:27 "gluuPerson", Nov 26 09:27:27 "eduPerson" Nov 26 09:27:27 ], Nov 26 09:27:27 "personObjectClassDisplayNames":[ Nov 26 09:27:27 "gluuCustomPerson", Nov 26 09:27:27 "gluuPerson", Nov 26 09:27:27 "eduPerson" Nov 26 09:27:27 ], Nov 26 09:27:27 Nov 26 09:27:27 "svnConfigurationStoreRoot":"unused", Nov 26 09:27:27 "svnConfigurationStorePassword":"unused", Nov 26 09:27:27 "persistSVN":false, Nov 26 09:27:27 Nov 26 09:27:27 "allowPersonModification":true, Nov 26 09:27:27 "updateApplianceStatus":true, Nov 26 09:27:27 Nov 26 09:27:27 "clientAssociationAttribute":"inum", Nov 26 09:27:27 Nov 26 09:27:27 "personCustomObjectClass":"gluuCustomPerson", Nov 26 09:27:27 Nov 26 09:27:27 "contactObjectClassTypes":[ Nov 26 09:27:27 Nov 26 09:27:27 ], Nov 26 09:27:27 "contactObjectClassDisplayNames":[ Nov 26 09:27:27 Nov 26 09:27:27 ], Nov 26 09:27:27 Nov 26 09:27:27 "photoRepositoryRootDir":"/var/ox/photos", Nov 26 09:27:27 "photoRepositoryThumbWidth":300, Nov 26 09:27:27 "photoRepositoryThumbHeight":300, Nov 26 09:27:27 "photoRepositoryCountLeveles":3, Nov 26 09:27:27 "photoRepositoryCountFoldersPerLevel":20, Nov 26 09:27:27 Nov 26 09:27:27 "shibboleth3FederationRootDir":"/opt/shibboleth-federation", Nov 26 09:27:27 Nov 26 09:27:27 "velocityLog":"/opt/gluu/jetty/identity/logs/velocity.log", Nov 26 09:27:27 Nov 26 09:27:27 "spMetadataPath":"", Nov 26 09:27:27 Nov 26 09:27:27 "logoLocation":"/var/ox/photos", Nov 26 09:27:27 Nov 26 09:27:27 "gluuSpAttributes":[ Nov 26 09:27:27 Nov 26 09:27:27 ], Nov 26 09:27:27 Nov 26 09:27:27 "configGeneration":None, Nov 26 09:27:27 "ignoreValidation":false, Nov 26 09:27:27 Nov 26 09:27:27 "idpSecurityCert":"/etc/certs/shibIDP.crt", Nov 26 09:27:27 "idpSecurityKey":"/etc/certs/shibIDP.key", Nov 26 09:27:27 "idpSecurityKeyPassword":"FfvuGGLSTUaBVnBNkEVxlA==", Nov 26 09:27:27 "gluuSpCert":"/etc/certs/shibIDP.crt", Nov 26 09:27:27 Nov 26 09:27:27 "idpBindDn":"cn=Directory Manager", Nov 26 09:27:27 "idpBindPassword":"7OjK10yWZ2t5wrML2QW4Ew==", Nov 26 09:27:27 "idpLdapProtocol":"ldaps", Nov 26 09:27:27 "idpLdapServer":"ldap.server:1636", Nov 26 09:27:27 "idpUserFields":"", Nov 26 09:27:27 Nov 26 09:27:27 "ldifStore":"/var/ox/identity/removed", Nov 26 09:27:27 Nov 26 09:27:27 "caCertsLocation":"/usr/java/latest/jre/lib/security/cacerts", Nov 26 09:27:27 "caCertsPassphrase":"", Nov 26 09:27:27 Nov 26 09:27:27 "certDir":"/etc/certs/", Nov 26 09:27:27 "tempCertDir":"/etc/certs/temp", Nov 26 09:27:27 Nov 26 09:27:27 "clusteredInums":[ Nov 26 09:27:27 Nov 26 09:27:27 ], Nov 26 09:27:27 Nov 26 09:27:27 "servicesRestartTrigger":"/opt/gluu/essential_files/trigger_restart_of_services_delete_me_to_do_so", Nov 26 09:27:27 Nov 26 09:27:27 "oxAuthIssuer":"https://192.168.7.234", Nov 26 09:27:27 "oxAuthSectorIdentifierUrl":"https://192.168.7.234/oxauth/sectoridentifier", Nov 26 09:27:27 Nov 26 09:27:27 "oxAuthClientId":"@!B25B.4280.6C89.D07D!0001!0AB3.44F0!0008!0716.CE25", Nov 26 09:27:27 "oxAuthClientPassword":"dQvU9cUyEtuNO72sNPQbYQ==", Nov 26 09:27:27 "oxAuthClientScope":"openid+profile+email+user_name", Nov 26 09:27:27 Nov 26 09:27:27 "loginRedirectUrl":"https://192.168.7.234/identity/authentication/getauthcode", Nov 26 09:27:27 "logoutRedirectUrl":"https://192.168.7.234/identity/authentication/finishlogout", Nov 26 09:27:27 Nov 26 09:27:27 "umaIssuer":"https://192.168.7.234", Nov 26 09:27:27 Nov 26 09:27:27 "scimUmaClientId":"@!B25B.4280.6C89.D07D!0001!0AB3.44F0!0008!1C1B.DACF", Nov 26 09:27:27 "scimUmaClientKeyId":"", Nov 26 09:27:27 "scimUmaResourceId":"0f13ae5a-135e-4b01-a290-7bbe62e7d40f", Nov 26 09:27:27 "scimUmaScope":"https://192.168.7.234/oxauth/restv1/uma/scopes/scim_access", Nov 26 09:27:27 "scimUmaClientKeyStoreFile":"/etc/certs/scim-rs.jks", Nov 26 09:27:27 "scimUmaClientKeyStorePassword":"aSuAqRzNv/vJSe5po4M9/g==", Nov 26 09:27:27 Nov 26 09:27:27 "passportUmaClientId":"@!B25B.4280.6C89.D07D!0001!0AB3.44F0!0008!94CA.10C1", Nov 26 09:27:27 "passportUmaClientKeyId":"", Nov 26 09:27:27 "passportUmaResourceId":"0f963ecc-93f0-49c1-beae-ad2006abbb99", Nov 26 09:27:27 "passportUmaScope":"https://192.168.7.234/oxauth/restv1/uma/scopes/passport_access", Nov 26 09:27:27 "passportUmaClientKeyStoreFile":"/etc/certs/passport-rs.jks", Nov 26 09:27:27 "passportUmaClientKeyStorePassword":"pQvIiLdoU//yHgAywPxoFQ==", Nov 26 09:27:27 Nov 26 09:27:27 "cssLocation":"", Nov 26 09:27:27 "jsLocation":"", Nov 26 09:27:27 Nov 26 09:27:27 "rptConnectionPoolUseConnectionPooling":true, Nov 26 09:27:27 "rptConnectionPoolMaxTotal":200, Nov 26 09:27:27 "rptConnectionPoolDefaultMaxPerRoute":20, Nov 26 09:27:27 "rptConnectionPoolValidateAfterInactivity":10, Nov 26 09:27:27 "rptConnectionPoolCustomKeepAliveTimeout":5, Nov 26 09:27:27 Nov 26 09:27:27 "shibbolethVersion":"v3", Nov 26 09:27:27 "shibboleth3IdpRootDir":"/opt/shibboleth-idp", Nov 26 09:27:27 "shibboleth3SpConfDir":"/opt/shibboleth-idp/sp", Nov 26 09:27:27 "organizationName":"Versa", Nov 26 09:27:27 "idp3SigningCert":"/etc/certs/idp-signing.crt", Nov 26 09:27:27 "idp3EncryptionCert":"/etc/certs/idp-encryption.crt", Nov 26 09:27:27 Nov 26 09:27:27 "clientWhiteList": ["*"], Nov 26 09:27:27 "clientBlackList": ["*.attacker.com/*"], Nov 26 09:27:27 Nov 26 09:27:27 "scimTestMode":false, Nov 26 09:27:27 "ScimProperties": { Nov 26 09:27:27 "maxCount": 1000 Nov 26 09:27:27 } Nov 26 09:27:27 } Nov 26 09:27:27 ' to object ``` Caught Exception code line: https://github.com/GluuFederation/oxTrust/blob/version_3.1.3.1/server/src/main/java/org/gluu/oxtrust/config/ConfigurationFactory.java#L405

By Sathish Kumar S . user 28 Nov 2018 at 1:23 a.m. CST

Copy
Sathish Kumar S . gravatar
@Chris.Blanton Any Luck ?

By Chris Blanton user 29 Nov 2018 at 1:54 p.m. CST

Copy
Chris Blanton gravatar
Are you doing any customizations to the configuration generation? I've never seen this issue before: ``` This is the exact issue the Oxtrust configuration got loaded, but could not convert the JSON string to object, ``` Where the LDAP data is represented in a pretty JSON object.

By Sathish Kumar S . user 03 Dec 2018 at 6:16 a.m. CST

Copy
Sathish Kumar S . gravatar
This is the issue, the JSON has value **"configGeneration":None**, so it becomes Invalid JSON fomat, Note: GLUU_OXTRUST_CONFIG_GENERATION is set as true in ldap-manager.yml Just check this in json-validator, (From the log in my previous post) ``` { "orgInum":"@!B25B.4280.6C89.D07D!0001!0AB3.44F0", "orgIname":"", "orgSupportEmail":"sathishkumar@contus.in", "applianceInum":"@!B25B.4280.6C89.D07D!0002!D35E.8179", "baseDN":"o=gluu", "baseEndpoint":"https://192.168.7.234/identity/restv1", "idpUrl":"https://192.168.7.234", "applianceUrl":"https://192.168.7.234", "keystorePath":"/etc/certs/shibIDP.jks", "keystorePassword":"w25qnNpFWbXp", "personObjectClassTypes":[ "gluuCustomPerson", "gluuPerson", "eduPerson" ], "personObjectClassDisplayNames":[ "gluuCustomPerson", "gluuPerson", "eduPerson" ], "svnConfigurationStoreRoot":"unused", "svnConfigurationStorePassword":"unused", "persistSVN":false, "allowPersonModification":true, "updateApplianceStatus":true, "clientAssociationAttribute":"inum", "personCustomObjectClass":"gluuCustomPerson", "contactObjectClassTypes":[ ], "contactObjectClassDisplayNames":[ ], "photoRepositoryRootDir":"/var/ox/photos", "photoRepositoryThumbWidth":300, "photoRepositoryThumbHeight":300, "photoRepositoryCountLeveles":3, "photoRepositoryCountFoldersPerLevel":20, "shibboleth3FederationRootDir":"/opt/shibboleth-federation", "velocityLog":"/opt/gluu/jetty/identity/logs/velocity.log", "spMetadataPath":"", "logoLocation":"/var/ox/photos", "gluuSpAttributes":[ ], "configGeneration":None, "ignoreValidation":false, "idpSecurityCert":"/etc/certs/shibIDP.crt", "idpSecurityKey":"/etc/certs/shibIDP.key", "idpSecurityKeyPassword":"FfvuGGLSTUaBVnBNkEVxlA==", "gluuSpCert":"/etc/certs/shibIDP.crt", "idpBindDn":"cn=Directory Manager", "idpBindPassword":"7OjK10yWZ2t5wrML2QW4Ew==", "idpLdapProtocol":"ldaps", "idpLdapServer":"ldap.server:1636", "idpUserFields":"", "ldifStore":"/var/ox/identity/removed", "caCertsLocation":"/usr/java/latest/jre/lib/security/cacerts", "caCertsPassphrase":"", "certDir":"/etc/certs/", "tempCertDir":"/etc/certs/temp", "clusteredInums":[ ], "servicesRestartTrigger":"/opt/gluu/essential_files/trigger_restart_of_services_delete_me_to_do_so", "oxAuthIssuer":"https://192.168.7.234", "oxAuthSectorIdentifierUrl":"https://192.168.7.234/oxauth/sectoridentifier", "oxAuthClientId":"@!B25B.4280.6C89.D07D!0001!0AB3.44F0!0008!0716.CE25", "oxAuthClientPassword":"dQvU9cUyEtuNO72sNPQbYQ==", "oxAuthClientScope":"openid+profile+email+user_name", "loginRedirectUrl":"https://192.168.7.234/identity/authentication/getauthcode", "logoutRedirectUrl":"https://192.168.7.234/identity/authentication/finishlogout", "umaIssuer":"https://192.168.7.234", "scimUmaClientId":"@!B25B.4280.6C89.D07D!0001!0AB3.44F0!0008!1C1B.DACF", "scimUmaClientKeyId":"", "scimUmaResourceId":"0f13ae5a-135e-4b01-a290-7bbe62e7d40f", "scimUmaScope":"https://192.168.7.234/oxauth/restv1/uma/scopes/scim_access", "scimUmaClientKeyStoreFile":"/etc/certs/scim-rs.jks", "scimUmaClientKeyStorePassword":"aSuAqRzNv/vJSe5po4M9/g==", "passportUmaClientId":"@!B25B.4280.6C89.D07D!0001!0AB3.44F0!0008!94CA.10C1", "passportUmaClientKeyId":"", "passportUmaResourceId":"0f963ecc-93f0-49c1-beae-ad2006abbb99", "passportUmaScope":"https://192.168.7.234/oxauth/restv1/uma/scopes/passport_access", "passportUmaClientKeyStoreFile":"/etc/certs/passport-rs.jks", "passportUmaClientKeyStorePassword":"pQvIiLdoU//yHgAywPxoFQ==", "cssLocation":"", "jsLocation":"", "rptConnectionPoolUseConnectionPooling":true, "rptConnectionPoolMaxTotal":200, "rptConnectionPoolDefaultMaxPerRoute":20, "rptConnectionPoolValidateAfterInactivity":10, "rptConnectionPoolCustomKeepAliveTimeout":5, "shibbolethVersion":"v3", "shibboleth3IdpRootDir":"/opt/shibboleth-idp", "shibboleth3SpConfDir":"/opt/shibboleth-idp/sp", "organizationName":"Versa", "idp3SigningCert":"/etc/certs/idp-signing.crt", "idp3EncryptionCert":"/etc/certs/idp-encryption.crt", "clientWhiteList": ["*"], "clientBlackList": ["*.attacker.com/*"], "scimTestMode":false, "ScimProperties": { "maxCount": 1000 } } ``` I have checked the oxTrustConfigGeneration value (logger) from entrypoint.py (docker-opndj), everything is fine. This issue occurs occasionally, but worked sometimes ("configGeneration":True). And even sometime i can see the **oxtrust_config_base64** value becomes **None** Currently im checking ldap search manually before deploying OXtrust. As this occurs randomly, its very hard to track.

By Isman Firmansyah staff 04 Dec 2018 at 12:42 p.m. CST

Copy
Isman Firmansyah gravatar
Hi Sathish, > "configGeneration":None, That's the issue as you have pointed out. Can you check the value in consul? i.e. `curl -s <consul-ip>:8500/v1/kv/gluu/config/oxTrustConfigGeneration?raw`

By Sathish Kumar S . user 05 Dec 2018 at 11:27 a.m. CST

Copy
Sathish Kumar S . gravatar
I have checked `curl -s <consul-ip>:8500/v1/kv/gluu/config/oxTrustConfigGeneration?raw` the result is **true**. And now im not getting the above issue (None for configGeneration), but i dont know why. Kindly let me know when the oxTrustConfigGeneration value is updated from ldap-manager to Consul. Will it get updated from entrypoint.py/entrypoint.sh of opendj image. I would check tomorrow as well, and let you know if i get again None for configGeneration. Thanks.

By Isman Firmansyah staff 05 Dec 2018 at 7:51 p.m. CST

Copy
Isman Firmansyah gravatar
> Kindly let me know when the oxTrustConfigGeneration value is updated from ldap-manager to Consul. Will it get updated from entrypoint.py/entrypoint.sh of opendj image. It is updated by `entrypoint.py`. > I would check tomorrow as well, and let you know if i get again None for configGeneration. Try remove the mounted volume for OpenDJ first before trying.

By Sathish Kumar S . user 06 Dec 2018 at 10:41 a.m. CST

Copy
Sathish Kumar S . gravatar
1. Removed all volumes by `docker system prune -a --volumes` and redeployed everything from scratch. But i didnt get the same error. 2. I'm getting **redis cluster error** (oxtrust), when i enable cluster from redis.yml (uncomment the line https://github.com/GluuFederation/gluu-docker/blob/master/examples/multi-hosts/redis.yml#L11) with mode: global. I'm getting "No cluster configuration loaded" in Redis log. I can only run redis as standalone, which will not be useful in multi-host cluster. "`For a Gluu cluster with multiple oxAuth instances, we need a cache storage as a single place to read and write sessions.`" - I guess redis cluster is mandatory.

By Isman Firmansyah staff 06 Dec 2018 at 11:57 a.m. CST

Copy
Isman Firmansyah gravatar
Hi Shatish, We use standalone Redis for example. For clustered Redis, you will need to do it by yourself and change the type of `GLUU_REDIS_TYPE=STANDALONE` to `GLUU_REDIS_TYPE=CLUSTER` when deploying ldap-manager. If you use our v3.1.4 example, we use LDAP instead of Redis to store the cache.

By Sathish Kumar S . user 07 Dec 2018 at 6:43 a.m. CST

Copy
Sathish Kumar S . gravatar
Hi @Isman.Firmansyah, Now i get the error "configGeneration":None, after using 3.1.4 stacks. I can see /tmp/configuration.ldif has base64 value with "configGeneration":None. The result of `sudo docker exec -it $(docker ps --filter name=gluu_consul --format '{{.ID}}') sh -c "curl -s consul.server:8500/v1/kv/gluu/config/oxTrustConfigGeneration?raw"` is **true** ![Screenshot](https://raw.githubusercontent.com/iamsais/screenshot-images/master/screenshot-support.gluu.org-2018.12.07-18-16-09.png "enter image title here") Deployed consul.yml, ./config.sh and then ldap-manager.yml. ldap-manager.yml ``` version: "3.7" networks: gluu: external: true volumes: ldap-opendj: name: ldap-opendj ldap-opendj-flag: name: ldap-opendj-flag services: ldap_manager: image: gluufederation/opendj:3.1.4_02 hostname: ldap.manager environment: - GLUU_LDAP_INIT=true - GLUU_LDAP_INIT_HOST=ldap.server - GLUU_CACHE_TYPE=NATIVE_PERSISTENCE #- GLUU_CACHE_TYPE=REDIS #- GLUU_REDIS_URL=redis.server:6379 #- GLUU_REDIS_TYPE=STANDALONE - GLUU_CONFIG_ADAPTER=consul - GLUU_CONSUL_HOST=consul.server - GLUU_OXTRUST_CONFIG_GENERATION=true - GLUU_LDAP_ADDR_INTERFACE=eth0 - GLUU_LDAP_ADVERTISE_ADDR=ldap.manager # the value must match network alias `ldap.server` because other containers # use this value as LDAP hostname - GLUU_CERT_ALT_NAME=ldap.server networks: gluu: aliases: - ldap.manager - ldap.server deploy: mode: global endpoint_mode: dnsrr update_config: parallelism: 1 failure_action: rollback delay: 30s restart_policy: condition: any delay: 5s window: 120s placement: constraints: - node.labels.deploy.role == manager0 volumes: - type: volume source: ldap-opendj target: /opt/opendj - type: volume source: ldap-opendj-flag target: /flag labels: - "SERVICE_IGNORE=yes" logging: driver: syslog options: syslog-address: "udp://127.0.0.1:614" tag: "ldap/{{.ImageName}}/{{.ID}}" syslog-facility: local2 syslog-format: rfc5424micro ```

By Isman Firmansyah staff 07 Dec 2018 at 10:09 a.m. CST

Copy
Isman Firmansyah gravatar
Hi, It is likely Consul data is not replicated fully and entrypoint pulls the key from one of Consul server that has incomplete keys. Can you try adding `GLUU_CONSUL_CONSISTENCY=default` envvar in `ldap-manager.yml`?

By Sathish Kumar S . user 10 Dec 2018 at 10:15 a.m. CST

Copy
Sathish Kumar S . gravatar
Hi, I have tested adding envvar `GLUU_CONSUL_CONSISTENCY=default` in `ldap-manager.yml`, which worked, i didnt get the error `"configGeneration":None` today. But... I got this issue, not sure this is because of the changes we have done in ldap-manager.yml. Redirecting between http and https, https://192.168.2.28/identity to http://192.168.2.28/identity/ and https://192.168.2.28/identity/error to http://192.168.2.28/identity/error/ (Screenshot attached). I have tried to redeployed may times clearing all docker data (volumes), but no luck :( Kindly let me know which is more stable docker stack 3.1.3.1 (master branch) or 3.1.4, else both should be considered as beta? Another error while redeploying oxauth and oxtrust alone (removed oxtrust volume), ``` Dec 10 10:00:35 manager0 oxauth/nexus-versa.contus.us:8083/gluufederation-oxauth:3.1.4_01@sha256:6a5b59667eb5e63263fbbd00466b024a0a8c7e3740e8fabaee4bfa741589b235/e59194eb17a9[837] 2018-12-10 10:00:35.015:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@520a3426{/oxauth,[file:///opt/gluu/jetty/oxauth/webapps/oxauth/, jar:file:///opt/gluu/jetty/oxauth/webapps/oxauth/WEB-INF/lib/omnifaces-2.6.9.jar!/META-INF/resources, jar:file:///opt/gluu/jetty/oxauth/webapps/oxauth/WEB-INF/lib/oxauth-static-3.1.4.Final.jar!/META-INF/resources, jar:file:///opt/gluu/jetty/oxauth/webapps/oxauth/WEB-INF/lib/javax.faces-2.2.16.jar!/META-INF/resources],AVAILABLE}{/oxauth} Dec 10 10:00:35 manager0 oxauth/nexus-versa.contus.us:8083/gluufederation-oxauth:3.1.4_01@sha256:6a5b59667eb5e63263fbbd00466b024a0a8c7e3740e8fabaee4bfa741589b235/e59194eb17a9[837] 2018-12-10 10:00:35.086:INFO:oejsh.ContextHandler:main: Started o.e.j.s.h.ContextHandler@611b2857{/oxauth/ext/resources,null,AVAILABLE} Dec 10 10:00:35 manager0 oxauth/nexus-versa.contus.us:8083/gluufederation-oxauth:3.1.4_01@sha256:6a5b59667eb5e63263fbbd00466b024a0a8c7e3740e8fabaee4bfa741589b235/e59194eb17a9[837] 2018-12-10 10:00:35.136:INFO:oejs.AbstractConnector:main: Started ServerConnector@406794af{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} Dec 10 10:00:35 manager0 oxauth/nexus-versa.contus.us:8083/gluufederation-oxauth:3.1.4_01@sha256:6a5b59667eb5e63263fbbd00466b024a0a8c7e3740e8fabaee4bfa741589b235/e59194eb17a9[837] 2018-12-10 10:00:35.144:INFO:oejs.Server:main: Started @56762ms Dec 10 10:00:36 manager0 oxauth/nexus-versa.contus.us:8083/gluufederation-oxauth:3.1.4_01@sha256:6a5b59667eb5e63263fbbd00466b024a0a8c7e3740e8fabaee4bfa741589b235/e59194eb17a9[837] 2018-12-10 10:00:36,145 INFO [qtp1514322932-20] [org.xdi.oxauth.service.ApplicationFactory] (ApplicationFactory.java:63) - Cache configuration: CacheConfiguration{cacheProviderType=NATIVE_PERSISTENCE, memcachedConfiguration=MemcachedConfiguration{servers='localhost:11211', maxOperationQueueLength=100000, bufferSize=32768, defaultPutExpiration=60, connectionFactoryType=DEFAULT}, redisConfiguration=RedisConfiguration{servers='localhost:6379', defaultPutExpiration=60, redisProviderType=STANDALONE, useSSL=false, sslTrustStoreFilePath=}, inMemoryConfiguration=InMemoryConfiguration{defaultPutExpiration=60}, nativePersistenceConfiguration=NativePersistenceConfiguration{defaultPutExpiration=60, baseDn=o=@!5947.44B3.4AB3.BF64!0001!EFBD.19E2,o=gluu}} Dec 10 10:00:36 manager0 oxauth/nexus-versa.contus.us:8083/gluufederation-oxauth:3.1.4_01@sha256:6a5b59667eb5e63263fbbd00466b024a0a8c7e3740e8fabaee4bfa741589b235/e59194eb17a9[837] 2018-12-10 10:00:36,154 INFO [qtp1514322932-20] [org.xdi.service.cache.NativePersistenceCacheProvider] (NativePersistenceCacheProvider.java:56) - Created NATIVE_PERSISTENCE cache provider. `baseDn`: ou=cache,o=@!5947.44B3.4AB3.BF64!0001!EFBD.19E2,o=gluu Dec 10 10:00:37 manager0 oxauth/nexus-versa.contus.us:8083/gluufederation-oxauth:3.1.4_01@sha256:6a5b59667eb5e63263fbbd00466b024a0a8c7e3740e8fabaee4bfa741589b235/e59194eb17a9[837] 2018-12-10 10:00:37,690 ERROR [qtp1514322932-20] [org.xdi.oxauth.servlet.OpenIdConfiguration] (OpenIdConfiguration.java:372) - null Dec 10 10:00:37 manager0 oxauth/nexus-versa.contus.us:8083/gluufederation-oxauth:3.1.4_01@sha256:6a5b59667eb5e63263fbbd00466b024a0a8c7e3740e8fabaee4bfa741589b235/e59194eb17a9[837] java.lang.NullPointerException: null ``` Thanks.
Video or screenshot link

By Isman Firmansyah staff 11 Dec 2018 at 1:04 a.m. CST

Copy
Isman Firmansyah gravatar
I'm not exactly sure what was the error by looking at the screenshot and logs. You can check the `/opt/gluu/jetty/oxauth/logs/oxauth.log` and `/opt/gluu/jetty/identity/logs/oxtrust.log` in each container. Can you provide more details?

By Sathish Kumar S . user 11 Dec 2018 at 8:56 a.m. CST

Copy
Sathish Kumar S . gravatar
Hi @Isman.Firmansyah "I got this issue, not sure this is because of the changes we have done in ldap-manager.yml. Redirecting between http and https, https://192.168.2.28/identity to http://192.168.2.28/identity/ and https://192.168.2.28/identity/error to http://192.168.2.28/identity/error/ (Screenshot attached). I have tried to redeployed may times clearing all docker data (volumes), but no luck :(" - Sorry for wasting your time, this is internal issue we have tracked and fixed it. ----- "Kindly let me know which is more stable docker stack 3.1.3.1 (master branch) or 3.1.4, else both should be considered as beta?" - We should take a call on this as it is extremely important for deployment. Kindly help. ---- "OpenIdConfiguration.java:372 - null" - Will this affect? ``` Dec 10 10:00:36 manager0 oxauth/nexus-versa.contus.us:8083/gluufederation-oxauth:3.1.4_01@sha256:6a5b59667eb5e63263fbbd00466b024a0a8c7e3740e8fabaee4bfa741589b235/e59194eb17a9[837] 2018-12-10 10:00:36,145 INFO [qtp1514322932-20] [org.xdi.oxauth.service.ApplicationFactory] (ApplicationFactory.java:63) - Cache configuration: CacheConfiguration{cacheProviderType=NATIVE_PERSISTENCE, memcachedConfiguration=MemcachedConfiguration{servers='localhost:11211', maxOperationQueueLength=100000, bufferSize=32768, defaultPutExpiration=60, connectionFactoryType=DEFAULT}, redisConfiguration=RedisConfiguration{servers='localhost:6379', defaultPutExpiration=60, redisProviderType=STANDALONE, useSSL=false, sslTrustStoreFilePath=}, inMemoryConfiguration=InMemoryConfiguration{defaultPutExpiration=60}, nativePersistenceConfiguration=NativePersistenceConfiguration{defaultPutExpiration=60, baseDn=o=@!5947.44B3.4AB3.BF64!0001!EFBD.19E2,o=gluu}} Dec 10 10:00:36 manager0 oxauth/nexus-versa.contus.us:8083/gluufederation-oxauth:3.1.4_01@sha256:6a5b59667eb5e63263fbbd00466b024a0a8c7e3740e8fabaee4bfa741589b235/e59194eb17a9[837] 2018-12-10 10:00:36,154 INFO [qtp1514322932-20] [org.xdi.service.cache.NativePersistenceCacheProvider] (NativePersistenceCacheProvider.java:56) - Created NATIVE_PERSISTENCE cache provider. `baseDn`: ou=cache,o=@!5947.44B3.4AB3.BF64!0001!EFBD.19E2,o=gluu Dec 10 10:00:37 manager0 oxauth/nexus-versa.contus.us:8083/gluufederation-oxauth:3.1.4_01@sha256:6a5b59667eb5e63263fbbd00466b024a0a8c7e3740e8fabaee4bfa741589b235/e59194eb17a9[837] 2018-12-10 10:00:37,690 ERROR [qtp1514322932-20] [org.xdi.oxauth.servlet.OpenIdConfiguration] (OpenIdConfiguration.java:372) - null Dec 10 10:00:37 manager0 oxauth/nexus-versa.contus.us:8083/gluufederation-oxauth:3.1.4_01@sha256:6a5b59667eb5e63263fbbd00466b024a0a8c7e3740e8fabaee4bfa741589b235/e59194eb17a9[837] java.lang.NullPointerException: null ```

By William Lowe user 11 Dec 2018 at 9:19 a.m. CST

Copy
William Lowe gravatar
Hi Sathish, > We should take a call on this as it is extremely important for deployment. Kindly help. This isn't covered in community support. If you want to purchase a support contract, feel free to [schedule a call](https://gluu.org/booking) and we can continue helping you with troubleshooting. Thanks, Will

By Sathish Kumar S . user 31 Jan 2019 at 5:21 a.m. CST

Copy
Sathish Kumar S . gravatar
Hi Team, We have another issue which is could be related to LDAP, "Login Redirect again to login page" This happening many times. 1. Only after removing LDAP worker1 and worker2, with one LDAP manager the issue gets resolved. 2. When having 3 LDAP - Sometimes its gets login and never gets logged out randomly 3. 3 LDAP - If we clear browser cache, cookies login work sometimes, but doesn't work sometimes. Thanks, Sathish.

Post an answer