By: Alexander Täffner user 28 Aug 2020 at 5:13 a.m. CDT

8 Responses
Alexander Täffner gravatar
This is a bugreport. Since Gluu 4.2 does not correctly run shibboleth on Debian 10 I got suggested to run the development 4.2.1 release which should be working. It is running smoothly except one thing: After some time (sometimes hours, sometimes days) I get a 503 error when I try to authenticate. It is enough to `service identity restart` and hte system starts working again. I think I traced this down successfully to a crash of the LDAP server: ``` [24/Aug/2020:08:43:06 +0000] category=PROTOCOL severity=NOTICE msgID=276 msg=Started listening for new connections on LDAPS Connection Handler localhost port 1636 [25/Aug/2020:08:02:48 +0000] category=org.opends.messages.external severity=WARNING msgID=1 msg=GRIZZLY0013: Exception during FilterChain execution exception=OutOfMemoryError: Java heap space () [25/Aug/2020:08:02:48 +0000] category=CORE severity=ERROR msgID=108 msg=Worker Thread 9 encountered an uncaught exception while processing operation null: OutOfMemoryError: Java heap space () [27/Aug/2020:06:56:48 +0000] category=JVM severity=NOTICE msgID=21 msg=Installation Directory: /opt/opendj ``` My VM hasd 8 GiB of RAM (I upgraded it to 10 GiB for evaluation recently) and I assigned 7 GiB to Gluu applicatinos via setup.py. My Gluu is setup with SAML, SCIM, RADIUS, PASSPORT and CASA. What would you suggest me to do for getting things running stable again?

By Alexander Täffner user 28 Aug 2020 at 5:15 a.m. CDT

Alexander Täffner gravatar
Might also be of interest: ``` [27/Aug/2020:06:56:48 +0000] category=JVM severity=NOTICE msgID=18 msg=JVM Host: auth.example.com, running Linux 4.19.0-10-amd64 amd64, 8261439488 bytes physical memory size, number of processors available 2 [27/Aug/2020:06:56:48 +0000] category=JVM severity=NOTICE msgID=19 msg=JVM Arguments: "-Xms1g", "-Xmx2g", "-XX:+UseCompressedOops", "-Dorg.opends.server.scriptName=start-ds" ``` (changed the domain)

By Mohib Zico staff 28 Aug 2020 at 5:22 a.m. CDT

Mohib Zico gravatar
>> My VM hasd 8 GiB of RAM (I upgraded it to 10 GiB for evaluation recently) and I assigned 7 GiB to Gluu applicatinos via setup.py. My Gluu is setup with SAML, SCIM, RADIUS, PASSPORT and CASA. I would go with 16GB VM for SAML + OpenID Connect + SCIM + Passport + Radius + CASA. This is just a lot for 8 GB. Also, increase WrenDS ( LDAP ) jvm allocation to 5GB.

By Michael Schwartz Account Admin 28 Aug 2020 at 9:41 a.m. CDT

Michael Schwartz gravatar
Also, did you follow the instructions in the VM preparation regarding file descriptors? [https://gluu.org/docs/gluu-server/4.2/installation-guide/#file-descriptors-fd]( [https://gluu.org/docs/gluu-server/4.2/installation-guide/#file-descriptors-fd])

By Yuriy Movchan staff 28 Aug 2020 at 2:12 p.m. CDT

Yuriy Movchan gravatar
Can you try to use latest opendj before running install? For this you should put [this](Can you put https://ox.gluu.org/maven/org/forgerock/opendj/opendj-server-legacy/4.0.0-M3/opendj-server-legacy-4.0.0-M3.zip) file as `/opt/dist/app/opendj-server-legacy-4.0.0-M3.zip` (instead of existing one) before running setup.py

By Alexander Täffner user 31 Aug 2020 at 1:17 a.m. CDT

Alexander Täffner gravatar
@Michael.Schwartz I thought yes but it seems like I actually missed it. I raised the opendj xmx to 4GB for now (I don't have a huge ldap) and fixed the file descriptors. The last times it crashed a few hours after restarting. It's now stable for 32 hours - I'll report.

By Alexander Täffner user 02 Sep 2020 at 2:48 a.m. CDT

Alexander Täffner gravatar
Still stable. I'd consider this as stable. Sorry, seems to have been my fault.

By Alexander Täffner user 09 Sep 2020 at 8:51 a.m. CDT

Alexander Täffner gravatar
Unfortunately the issue suddenly re-appeared so it was not fixed. I just got the issue again and tried to trace it down more. This time a identity restart does not help. Then I restarted opendj (which actually is WrenDS in 4.2 if I got this right) followed by identity. No luck. The jetty logfile of identity says: ``` 2020-09-09 13:35:23,652 INFO [main] [org.gluu.oxauth.model.util.SecurityProviderUtility] (SecurityProviderUtility.java:24) - Adding Bouncy Castle Provider 2020-09-09 13:35:23,779 INFO [main] [org.gluu.oxtrust.util.BuildVersionService] (BuildVersionService.java:80) - Root element :beans 2020-09-09 13:35:23,782 INFO [main] [org.gluu.oxtrust.service.AppInitializer] (AppInitializer.java:253) - Build date 2020-08-19 11:16. Code revision 0f1d70b28433553dc47872fec151b267bd30eae4 on 2020-08-19T11:13:27+0000. Build 2135 2020-09-09 13:35:23,787 INFO [main] [org.gluu.service.config.ConfigurationFactory] (ConfigurationFactory.java:127) - Creating oxTrustConfiguration 2020-09-09 13:35:23,872 INFO [main] [org.gluu.service.config.ConfigurationFactory] (ConfigurationFactory.java:289) - Loading configuration from 'ldap' DB... 2020-09-09 13:35:54,287 ERROR [main] [org.gluu.service.config.ConfigurationFactory] (ConfigurationFactory.java:306) - Failed to create LDAP connection pool! Result code: '91 (connect error)' org.gluu.persist.exception.operation.ConfigurationException: Failed to create LDAP connection pool! Result code: '91 (connect error)' at org.gluu.persist.ldap.impl.LdapEntryManagerFactory.createEntryManager(LdapEntryManagerFactory.java:51) ~[oxcore-persistence-ldap-4.2.1-SNAPSHOT.jar:?] at org.gluu.persist.ldap.impl.LdapEntryManagerFactory.createEntryManager(LdapEntryManagerFactory.java:23) ~[oxcore-persistence-ldap-4.2.1-SNAPSHOT.jar:?] at org.gluu.persist.ldap.impl.LdapEntryManagerFactory$Proxy$_$$_WeldClientProxy.createEntryManager(Unknown Source) ~[oxcore-persistence-ldap-4.2.1-SNAPSHOT.jar:?] at org.gluu.oxtrust.service.AppInitializer.createPersistenceEntryManager(AppInitializer.java:296) ~[classes/:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:95) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:85) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.injection.producer.ProducerMethodProducer.produce(ProducerMethodProducer.java:103) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.injection.producer.AbstractMemberProducer.produce(AbstractMemberProducer.java:161) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.bean.AbstractProducerBean.create(AbstractProducerBean.java:180) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.contexts.AbstractContext.get(AbstractContext.java:96) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:100) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.bean.ContextualInstanceStrategy$ApplicationScopedContextualInstanceStrategy.get(ContextualInstanceStrategy.java:140) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.bean.proxy.ContextBeanInstance.getInstance(ContextBeanInstance.java:102) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:105) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.gluu.persist.EntityManager$PersistenceEntryManager$1037559104$Proxy$_$$_WeldClientProxy.find(Unknown Source) ~[weld-core-impl-3.1.4.Final.jar:?] at org.gluu.oxtrust.service.config.ConfigurationFactory.loadConfigurationFromDb(ConfigurationFactory.java:60) ~[classes/:?] at org.gluu.oxtrust.service.config.ConfigurationFactory$Proxy$_$$_WeldSubclass.loadConfigurationFromDb(Unknown Source) ~[classes/:?] at org.gluu.oxtrust.service.config.ConfigurationFactory.loadConfigurationFromDb(ConfigurationFactory.java:29) ~[classes/:?] at org.gluu.service.config.ConfigurationFactory.createFromDb(ConfigurationFactory.java:291) ~[oxtrust-service-4.2.1-SNAPSHOT.jar:?] at org.gluu.service.config.ConfigurationFactory.create(ConfigurationFactory.java:146) ~[oxtrust-service-4.2.1-SNAPSHOT.jar:?] at org.gluu.oxtrust.service.config.ConfigurationFactory$Proxy$_$$_WeldSubclass.create(Unknown Source) ~[classes/:?] at org.gluu.oxtrust.service.config.ConfigurationFactory$Proxy$_$$_WeldClientProxy.create(Unknown Source) ~[classes/:?] at org.gluu.oxtrust.service.AppInitializer.applicationInitialized(AppInitializer.java:180) ~[classes/:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:95) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:85) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.injection.MethodInvocationStrategy$SimpleMethodInvocationStrategy.invoke(MethodInvocationStrategy.java:168) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:330) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:308) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.event.ObserverMethodImpl.notify(ObserverMethodImpl.java:286) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at javax.enterprise.inject.spi.ObserverMethod.notify(ObserverMethod.java:124) ~[jakarta.enterprise.cdi-api-2.0.2.jar:?] at org.jboss.weld.util.Observers.notify(Observers.java:166) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.event.ObserverNotifier.notifySyncObservers(ObserverNotifier.java:285) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.event.ObserverNotifier.notify(ObserverNotifier.java:273) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.event.ObserverNotifier.fireEvent(ObserverNotifier.java:177) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.bootstrap.BeanDeploymentModule.fireEvent(BeanDeploymentModule.java:93) ~[weld-core-impl-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.module.web.servlet.HttpContextLifecycle.fireEventForApplicationScope(HttpContextLifecycle.java:161) ~[weld-web-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.module.web.servlet.HttpContextLifecycle.contextInitialized(HttpContextLifecycle.java:147) ~[weld-web-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.module.web.servlet.WeldInitialListener.contextInitialized(WeldInitialListener.java:99) ~[weld-web-3.1.4.Final.jar:3.1.4.Final] at org.jboss.weld.servlet.api.helpers.ForwardingServletListener.contextInitialized(ForwardingServletListener.java:34) ~[weld-spi-3.1.SP2.jar:3.1.SP2] at org.jboss.weld.environment.servlet.EnhancedListener.onStartup(EnhancedListener.java:66) ~[weld-servlet-core-3.1.4.Final.jar:3.1.4.Final] at org.eclipse.jetty.plus.annotation.ContainerInitializer.callStartup(ContainerInitializer.java:145) ~[jetty-plus-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.annotations.ServletContainerInitializersStarter.doStart(ServletContainerInitializersStarter.java:64) ~[jetty-annotations-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) ~[jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:341) ~[jetty-servlet-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1445) ~[jetty-webapp-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1409) ~[jetty-webapp-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:855) ~[jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:275) ~[jetty-servlet-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) ~[jetty-webapp-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) ~[jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:46) ~[jetty-deploy-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:188) ~[jetty-deploy-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:513) ~[jetty-deploy-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:154) ~[jetty-deploy-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:173) ~[jetty-deploy-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.deploy.providers.WebAppProvider.fileAdded(WebAppProvider.java:447) ~[jetty-deploy-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:66) ~[jetty-deploy-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:784) ~[jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:753) ~[jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.util.Scanner.scan(Scanner.java:641) ~[jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:540) ~[jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) ~[jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:146) ~[jetty-deploy-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) ~[jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:599) ~[jetty-deploy-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:249) ~[jetty-deploy-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) ~[jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) ~[jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.server.Server.start(Server.java:408) ~[jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) ~[jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) ~[jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.server.Server.doStart(Server.java:372) ~[jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) ~[jetty-util-9.4.31.v20200723.jar:9.4.31.v20200723] at org.eclipse.jetty.xml.XmlConfiguration.lambda$main$1(XmlConfiguration.java:1929) ~[jetty-xml-9.4.31.v20200723.jar:9.4.31.v20200723] at java.security.AccessController.doPrivileged(Native Method) ~[?:?] at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1878) [jetty-xml-9.4.31.v20200723.jar:9.4.31.v20200723] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] at org.eclipse.jetty.start.Main.invokeMain(Main.java:218) [start.jar:9.4.31.v20200723] at org.eclipse.jetty.start.Main.start(Main.java:491) [start.jar:9.4.31.v20200723] at org.eclipse.jetty.start.Main.main(Main.java:77) [start.jar:9.4.31.v20200723] 2020-09-09 13:35:54,289 ERROR [main] [org.gluu.service.config.ConfigurationFactory] (ConfigurationFactory.java:147) - Failed to load configuration from LDAP. Please fix it!!!. 2020-09-09 13:35:54.290:WARN:oeja.ServletContainerInitializersStarter:main: org.gluu.exception.ConfigurationException: Failed to load configuration from LDAP. at org.gluu.service.config.ConfigurationFactory.create(ConfigurationFactory.java:148) at org.gluu.oxtrust.service.config.ConfigurationFactory$Proxy$_$$_WeldSubclass.create(Unknown Source) at org.gluu.oxtrust.service.config.ConfigurationFactory$Proxy$_$$_WeldClientProxy.create(Unknown Source) at org.gluu.oxtrust.service.AppInitializer.applicationInitialized(AppInitializer.java:180) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:95) at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:85) at org.jboss.weld.injection.MethodInvocationStrategy$SimpleMethodInvocationStrategy.invoke(MethodInvocationStrategy.java:168) at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:330) at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:308) at org.jboss.weld.event.ObserverMethodImpl.notify(ObserverMethodImpl.java:286) at javax.enterprise.inject.spi.ObserverMethod.notify(ObserverMethod.java:124) at org.jboss.weld.util.Observers.notify(Observers.java:166) at org.jboss.weld.event.ObserverNotifier.notifySyncObservers(ObserverNotifier.java:285) at org.jboss.weld.event.ObserverNotifier.notify(ObserverNotifier.java:273) at org.jboss.weld.event.ObserverNotifier.fireEvent(ObserverNotifier.java:177) at org.jboss.weld.bootstrap.BeanDeploymentModule.fireEvent(BeanDeploymentModule.java:93) at org.jboss.weld.module.web.servlet.HttpContextLifecycle.fireEventForApplicationScope(HttpContextLifecycle.java:161) at org.jboss.weld.module.web.servlet.HttpContextLifecycle.contextInitialized(HttpContextLifecycle.java:147) at org.jboss.weld.module.web.servlet.WeldInitialListener.contextInitialized(WeldInitialListener.java:99) at org.jboss.weld.servlet.api.helpers.ForwardingServletListener.contextInitialized(ForwardingServletListener.java:34) at org.jboss.weld.environment.servlet.EnhancedListener.onStartup(EnhancedListener.java:66) at org.eclipse.jetty.plus.annotation.ContainerInitializer.callStartup(ContainerInitializer.java:145) at org.eclipse.jetty.annotations.ServletContainerInitializersStarter.doStart(ServletContainerInitializersStarter.java:64) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:341) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1445) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1409) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:855) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:275) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:46) at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:188) at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:513) at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:154) at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:173) at org.eclipse.jetty.deploy.providers.WebAppProvider.fileAdded(WebAppProvider.java:447) at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:66) at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:784) at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:753) at org.eclipse.jetty.util.Scanner.scan(Scanner.java:641) at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:540) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:146) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:599) at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:249) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:408) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:372) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.xml.XmlConfiguration.lambda$main$1(XmlConfiguration.java:1929) at java.base/java.security.AccessController.doPrivileged(Native Method) at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1878) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.eclipse.jetty.start.Main.invokeMain(Main.java:218) at org.eclipse.jetty.start.Main.start(Main.java:491) at org.eclipse.jetty.start.Main.main(Main.java:77) 2020-09-09 13:35:54.290:WARN:oejw.WebAppContext:main: Failed startup of context o.e.j.w.WebAppContext@2c78324b{oxTrust Server,/identity,[file:///opt/jetty-9.4/temp/jetty-localhost-8082-identity_war-_identity-any-13897423309272977458.dir/webapp/, jar:file:///opt/jetty-9.4/temp/jetty-localhost-8082-identity_war-_identity-any-13897423309272977458.dir/webapp/WEB-INF/lib/omnifaces-2.6.9.jar!/META-INF/resources, jar:file:///opt/jetty-9.4/temp/jetty-localhost-8082-identity_war-_identity-any-13897423309272977458.dir/webapp/WEB-INF/lib/oxtrust-static-4.2.1-SNAPSHOT.jar!/META-INF/resources, jar:file:///opt/jetty-9.4/temp/jetty-localhost-8082-identity_war-_identity-any-13897423309272977458.dir/webapp/WEB-INF/lib/jakarta.faces-2.3.14.jar!/META-INF/resources, jar:file:///opt/jetty-9.4/temp/jetty-localhost-8082-identity_war-_identity-any-13897423309272977458.dir/webapp/WEB-INF/lib/richfaces-4.5.17-gluu.Final.jar!/META-INF/resources, jar:file:///opt/jetty-9.4/temp/jetty-localhost-8082-identity_war-_identity-any-13897423309272977458.dir/webapp/WEB-INF/lib/primefaces-8.0.jar!/META-INF/resources, jar:file:///opt/jetty-9.4/temp/jetty-localhost-8082-identity_war-_identity-any-13897423309272977458.dir/webapp/WEB-INF/lib/richfaces-core-4.5.17-gluu.Final.jar!/META-INF/resources, jar:file:///opt/jetty-9.4/temp/jetty-localhost-8082-identity_war-_identity-any-13897423309272977458.dir/webapp/WEB-INF/lib/bootsfaces-1.5.0.jar!/META-INF/resources, jar:file:///opt/jetty-9.4/temp/jetty-localhost-8082-identity_war-_identity-any-13897423309272977458.dir/webapp/WEB-INF/lib/scim-model-4.2.1-SNAPSHOT.jar!/META-INF/resources, jar:file:///opt/jetty-9.4/temp/jetty-localhost-8082-identity_war-_identity-any-13897423309272977458.dir/webapp/WEB-INF/lib/richfaces-a4j-4.5.17-gluu.Final.jar!/META-INF/resources],UNAVAILABLE}{/opt/gluu/jetty/identity/webapps/identity.war} java.lang.RuntimeException: org.gluu.exception.ConfigurationException: Failed to load configuration from LDAP. at org.eclipse.jetty.annotations.ServletContainerInitializersStarter.doStart(ServletContainerInitializersStarter.java:69) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:341) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1445) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1409) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:855) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:275) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:46) at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:188) at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:513) at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:154) at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:173) at org.eclipse.jetty.deploy.providers.WebAppProvider.fileAdded(WebAppProvider.java:447) at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:66) at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:784) at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:753) at org.eclipse.jetty.util.Scanner.scan(Scanner.java:641) at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:540) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:146) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:599) at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:249) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:408) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:372) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.xml.XmlConfiguration.lambda$main$1(XmlConfiguration.java:1929) at java.base/java.security.AccessController.doPrivileged(Native Method) at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1878) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.eclipse.jetty.start.Main.invokeMain(Main.java:218) at org.eclipse.jetty.start.Main.start(Main.java:491) at org.eclipse.jetty.start.Main.main(Main.java:77) Caused by: org.gluu.exception.ConfigurationException: Failed to load configuration from LDAP. at org.gluu.service.config.ConfigurationFactory.create(ConfigurationFactory.java:148) at org.gluu.oxtrust.service.config.ConfigurationFactory$Proxy$_$$_WeldSubclass.create(Unknown Source) at org.gluu.oxtrust.service.config.ConfigurationFactory$Proxy$_$$_WeldClientProxy.create(Unknown Source) at org.gluu.oxtrust.service.AppInitializer.applicationInitialized(AppInitializer.java:180) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:95) at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:85) at org.jboss.weld.injection.MethodInvocationStrategy$SimpleMethodInvocationStrategy.invoke(MethodInvocationStrategy.java:168) at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:330) at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:308) at org.jboss.weld.event.ObserverMethodImpl.notify(ObserverMethodImpl.java:286) at javax.enterprise.inject.spi.ObserverMethod.notify(ObserverMethod.java:124) at org.jboss.weld.util.Observers.notify(Observers.java:166) at org.jboss.weld.event.ObserverNotifier.notifySyncObservers(ObserverNotifier.java:285) at org.jboss.weld.event.ObserverNotifier.notify(ObserverNotifier.java:273) at org.jboss.weld.event.ObserverNotifier.fireEvent(ObserverNotifier.java:177) at org.jboss.weld.bootstrap.BeanDeploymentModule.fireEvent(BeanDeploymentModule.java:93) at org.jboss.weld.module.web.servlet.HttpContextLifecycle.fireEventForApplicationScope(HttpContextLifecycle.java:161) at org.jboss.weld.module.web.servlet.HttpContextLifecycle.contextInitialized(HttpContextLifecycle.java:147) at org.jboss.weld.module.web.servlet.WeldInitialListener.contextInitialized(WeldInitialListener.java:99) at org.jboss.weld.servlet.api.helpers.ForwardingServletListener.contextInitialized(ForwardingServletListener.java:34) at org.jboss.weld.environment.servlet.EnhancedListener.onStartup(EnhancedListener.java:66) at org.eclipse.jetty.plus.annotation.ContainerInitializer.callStartup(ContainerInitializer.java:145) at org.eclipse.jetty.annotations.ServletContainerInitializersStarter.doStart(ServletContainerInitializersStarter.java:64) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:341) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1445) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1409) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:855) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:275) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:46) at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:188) at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:513) at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:154) at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:173) at org.eclipse.jetty.deploy.providers.WebAppProvider.fileAdded(WebAppProvider.java:447) at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:66) at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:784) at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:753) at org.eclipse.jetty.util.Scanner.scan(Scanner.java:641) at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:540) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:146) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:599) at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:249) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:408) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:372) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.xml.XmlConfiguration.lambda$main$1(XmlConfiguration.java:1929) at java.base/java.security.AccessController.doPrivileged(Native Method) at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1878) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.eclipse.jetty.start.Main.invokeMain(Main.java:218) at org.eclipse.jetty.start.Main.start(Main.java:491) at org.eclipse.jetty.start.Main.main(Main.java:77) 2020-09-09 13:35:54.324:INFO:oejs.AbstractConnector:main: Started ServerConnector@29b343c7{HTTP/1.1, (http/1.1)}{localhost:8082} 2020-09-09 13:35:54.326:INFO:oejs.Server:main: Started @42823ms ``` but service opendj status says it i running. Yuriy, could you tell me how I can do this on a already running system? Is there a way to manually "upgrade" it?

By Alexander Täffner user 09 Sep 2020 at 9:07 a.m. CDT

Alexander Täffner gravatar
I think I got another logfile that might be of huge interest. It's too big to share here. Can I email it somewhere?