LDAP Entry Persistence Exception

By: Bruce Gordon Account Admin 20 Dec 2017 at 9:15 a.m. CST

7 Responses
Bruce Gordon gravatar
In a 3 node GLUU replication scenario one of the servers can start failing every 6 minutes with an LDAP Persistence exception. This server then becomes slow and unresponsive. Sometimes the error resolves on one server but then occurs on another server in the configuration. Has anybody got a resolution to this type of error ? Thanks ! Full stack trace :- 2017-12-20 15:09:34,835 ERROR [Thread-168] [org.gluu.oxtrust.ldap.service.StatusCheckerTimer] (StatusCheckerTimer.java:177) - Failed to update current appliance org.gluu.site.ldap.persistence.exception.EntryPersistenceException: Failed to update entry: inum=@!5102.78D2.26B0.8209!0002!5495.4ACB,ou=appliances,o=gluu at org.gluu.site.ldap.persistence.LdapEntryManager.merge(LdapEntryManager.java:185) ~[oxcore-ldap-3.1.1.Final.jar:?] at org.gluu.site.ldap.persistence.AbstractEntryManager.merge(AbstractEntryManager.java:277) ~[oxcore-ldap-3.1.1.Final.jar:?] at org.gluu.site.ldap.persistence.AbstractEntryManager.merge(AbstractEntryManager.java:298) ~[oxcore-ldap-3.1.1.Final.jar:?] at org.gluu.site.ldap.persistence.LdapEntryManager$Proxy$_$$_WeldClientProxy.merge(Unknown Source) ~[oxcore-ldap-3.1.1.Final.jar:?] at org.gluu.oxtrust.ldap.service.ApplianceService.updateAppliance(ApplianceService.java:74) ~[classes/:?] at org.gluu.oxtrust.ldap.service.StatusCheckerTimer.processInt(StatusCheckerTimer.java:175) [classes/:?] at org.gluu.oxtrust.ldap.service.StatusCheckerTimer.process(StatusCheckerTimer.java:125) [classes/:?] at org.gluu.oxtrust.ldap.service.StatusCheckerTimer$Proxy$_$$_WeldSubclass.process$$super(Unknown Source) [classes/:?] at sun.reflect.GeneratedMethodAccessor140.invoke(Unknown Source) ~[?:?] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112] at org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:51) [weld-core-impl-3.0.0.Final.jar:3.0.0.Final] at org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:78) [weld-core-impl-3.0.0.Final.jar:3.0.0.Final] at org.xdi.service.cdi.async.AsynchronousInterceptor$1.get(AsynchronousInterceptor.java:36) [oxcore-service-3.1.1.Final.jar:?] at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) [?:1.8.0_112] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] Caused by: com.unboundid.ldap.sdk.LDAPException: A client-side timeout was encountered while waiting 300000ms for a response to modify request with message ID 7 for entry 'inum=@!5102.78D2.26B0.8209!0002!5495.4ACB,ou=appliances,o=gluu' from server localhost:1636. at com.unboundid.ldap.sdk.ModifyRequest.handleResponse(ModifyRequest.java:908) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0] at com.unboundid.ldap.sdk.ModifyRequest.process(ModifyRequest.java:675) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0] at com.unboundid.ldap.sdk.LDAPConnection.modify(LDAPConnection.java:2745) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0] at com.unboundid.ldap.sdk.AbstractConnectionPool.modify(AbstractConnectionPool.java:1304) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0] at org.gluu.site.ldap.OperationsFacade.modifyEntry(OperationsFacade.java:563) ~[oxcore-ldap-3.1.1.Final.jar:?] at org.gluu.site.ldap.OperationsFacade.updateEntry(OperationsFacade.java:549) ~[oxcore-ldap-3.1.1.Final.jar:?] at org.gluu.site.ldap.persistence.LdapEntryManager.merge(LdapEntryManager.java:179) ~[oxcore-ldap-3.1.1.Final.jar:?] ... 15 more
CentOS 7.0
closed

Answers

By Michael Schwartz Account Admin 20 Dec 2017 at 11:58 a.m. CST

Copy
Michael Schwartz gravatar
Are you using Gluu Cluster Manager? That is going to be the supported way going forward.

By Bruce Gordon Account Admin 20 Dec 2017 at 1:52 p.m. CST

Copy
Bruce Gordon gravatar
No we are not yet ....is this an error you have seen before ? We have come quite a way following your 3.1.1 notes and this feels like the last problem.

By Chris Blanton user 20 Dec 2017 at 2:15 p.m. CST

Copy
Chris Blanton gravatar
Bruce, ``` java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] Caused by: com.unboundid.ldap.sdk.LDAPException: A client-side timeout was encountered while waiting 300000ms for a response to modify request with message ID 7 for entry 'inum=@!5102.78D2.26B0.8209!0002!5495.4ACB,ou=appliances,o=gluu' from server localhost:1636 ``` I haven't seen this before, but OpenLDAP did release a new version citing locking issues, which might be what's going on: ``` Summary of Changes: Packaging: OpenLDAP: Fixed syncprov deadlock with multimaster (ITS#8752) Fixed slapo-syncprov to not clear pending operation when checkpointing (ITS#8444) Added ldapurl binary (SOLD-85) ``` What you can do is stop solserver and then, inside the chroot, `wget http://104.237.133.194/pkg/GLUU/2.4.45-3/RH7/symas-openldap-gluu.x86_64-2.4.45-3.rpm` and `rpm -Uvh symas-openldap-gluu.x86_64-2.4.45-3.rpm` to upgrade to the latest version.

By Bruce Gordon Account Admin 21 Dec 2017 at 5:11 a.m. CST

Copy
Bruce Gordon gravatar
I did upgrade to this version.. now GLUU doesn't start which suggests this version is not compatible ?

By Chris Blanton user 21 Dec 2017 at 10:16 a.m. CST

Copy
Chris Blanton gravatar
What happens when you try to start it? Can you run `service solserver restart -d -1` and tell me what it says at the end of the debug log (which should give an error if it's stopping.

By Bruce Gordon Account Admin 21 Dec 2017 at 10:55 a.m. CST

Copy
Bruce Gordon gravatar
We restarted but not sure of the location of the log ?

By Chris Blanton user 21 Dec 2017 at 11:01 a.m. CST

Copy
Chris Blanton gravatar
`service solserver restart -d -1` should just present you with a log when you run it. It runs the program in debug mode, so it should take up your console.

Post an answer