Errors configuring Cache Refresh to work with AWS Directory Services Simple AD LDAP backend

By: Ryan Kennedy user 24 Oct 2016 at 7:42 p.m. CDT

11 Responses
Ryan Kennedy gravatar
I am attempting to configure the Cache Refresh to work with AWS Directory Service Simple AD LDAP backend. I've confirmed I have the necessary connectivity from the chrooted environment by using the opendj ldapsearch command to query the Simple AD LDAP service hosted in the same VPC. I have tried following the documentation here: https://www.gluu.org/docs/cache-refresh/ including the whitepaper PDF linked at the bottom of the page (https://www.gluu.org/docs/cache-refresh/GluuCache-Refresh.pdf) but it isn't exactly clear to me how I need to fill out the Cache Refresh configuration properties. It is reporting `4 Problems at the last run` in the Cache Refresh main menu. Here is an excerpt from the cache refresh tomcat logs... ``` 2016-10-25 00:01:58,999 ERROR [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-5) Failed to 'add' person '@!REMOVED_ORG_INUM!0001!98D9.414F!0000!E0E9.FA33' org.gluu.site.ldap.persistence.exception.EntryPersistenceException: Failed to persist entry: inum=@!REMOVED_ORG_INUM!0001!98D9.414F!0000!E0E9.FA33,ou=people,o=@!REMOVED_ORG_INUM!0001!98D9.414F,o=gluu Caused by: Connection exception (LDAP config error: schema violation contact LDAP admin.: Entry inum=@!REMOVED_ORG_INUM!0001!98D9.414F!0000!E0E9.FA33,ou=people,o=@!REMOVED_ORG_INUM!0001!98D9.414F,o=gluu violates the Directory Server schema configuration because it is missing attribute sn which is required by objectclass person) 2016-10-25 00:01:59,001 ERROR [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-5) Failed to 'add' person '@!REMOVED_ORG_INUM!0001!98D9.414F!0000!0B62.B367' org.gluu.site.ldap.persistence.exception.EntryPersistenceException: Failed to persist entry: inum=@!REMOVED_ORG_INUM!0001!98D9.414F!0000!0B62.B367,ou=people,o=@!REMOVED_ORG_INUM!0001!98D9.414F,o=gluu Caused by: Connection exception (LDAP config error: schema violation contact LDAP admin.: Entry inum=@!REMOVED_ORG_INUM!0001!98D9.414F!0000!0B62.B367,ou=people,o=@!REMOVED_ORG_INUM!0001!98D9.414F,o=gluu violates the Directory Server schema configuration because it is missing attribute sn which is required by objectclass person) 2016-10-25 00:01:59,003 ERROR [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-5) Failed to 'add' person '@!REMOVED_ORG_INUM!0001!98D9.414F!0000!08BA.1876' org.gluu.site.ldap.persistence.exception.EntryPersistenceException: Failed to persist entry: inum=@!REMOVED_ORG_INUM!0001!98D9.414F!0000!08BA.1876,ou=people,o=@!REMOVED_ORG_INUM!0001!98D9.414F,o=gluu Caused by: Connection exception (LDAP config error: schema violation contact LDAP admin.: Entry inum=@!REMOVED_ORG_INUM!0001!98D9.414F!0000!08BA.1876,ou=people,o=@!REMOVED_ORG_INUM!0001!98D9.414F,o=gluu violates the Directory Server schema configuration because it is missing attribute sn which is required by objectclass person) 2016-10-25 00:01:59,004 ERROR [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-5) Failed to 'add' person '@!REMOVED_ORG_INUM!0001!98D9.414F!0000!E2B8.40FE' org.gluu.site.ldap.persistence.exception.EntryPersistenceException: Failed to persist entry: inum=@!REMOVED_ORG_INUM!0001!98D9.414F!0000!E2B8.40FE,ou=people,o=@!REMOVED_ORG_INUM!0001!98D9.414F,o=gluu Caused by: Connection exception (LDAP config error: schema violation contact LDAP admin.: Entry inum=@!REMOVED_ORG_INUM!0001!98D9.414F!0000!E2B8.40FE,ou=people,o=@!REMOVED_ORG_INUM!0001!98D9.414F,o=gluu violates the Directory Server schema configuration because it is missing attribute sn which is required by objectclass person) 2016-10-25 00:01:59,005 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-5) Updated '0' entries 2016-10-25 00:01:59,005 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-5) Failed to update '4' entries 2016-10-25 00:01:59,006 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-5) Removed '0' persons from target server 2016-10-25 00:01:59,006 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-5) There are '4' entries before updating inum list 2016-10-25 00:01:59,006 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-5) There are '4' entries after removal '0' entries 2016-10-25 00:01:59,006 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-5) There are '4' entries after adding '0' entries ``` Here are the Cache Refresh config screens... ![enter image description here](https://s3-us-west-2.amazonaws.com/gluustuff/cache_refresh_1.png "Cache Refresh 1") ![enter image description here](https://s3-us-west-2.amazonaws.com/gluustuff/cache_refresh_2.png "Cache Refresh 2") ![enter image description here](https://s3-us-west-2.amazonaws.com/gluustuff/cache_refresh_3.png "Cache Refresh 3") ![enter image description here](https://s3-us-west-2.amazonaws.com/gluustuff/cache_refresh_4.png "Cache Refresh 4")
U1404
closed

Answers

By Sahil Arora user 24 Oct 2016 at 8:13 p.m. CDT

Copy
Sahil Arora gravatar
Please set custom LDAP filter to **sn=*** and let me know if it worked.

By Ryan Kennedy user 25 Oct 2016 at 11:14 a.m. CDT

Copy
Ryan Kennedy gravatar
Thanks. I have added `sn=*` as the **Custom LDAP Filter** value, updated, and refreshed. I am still seeing the same 4 errors. Any other suggestions?

By Sahil Arora user 25 Oct 2016 at 10:34 p.m. CDT

Copy
Sahil Arora gravatar
What errors do you see in cache refresh tomcat logs now? Also, make sure that user objects in your AD back-end have a value assigned to cn attribute.

By Ryan Kennedy user 26 Oct 2016 at 12:35 a.m. CDT

Copy
Ryan Kennedy gravatar
so it seems the tomcat `ERROR` entries have gone away but I am still seeing the following set of logs every time the cache refresh attempts... ``` 2016-10-26 05:25:58,979 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-1) Attempting to load entries from source server 2016-10-26 05:25:58,981 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-1) Found '0' entries in source server 2016-10-26 05:25:58,981 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-1) Found '0' unique entries in source server 2016-10-26 05:25:58,981 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-1) Found '0' changed entries 2016-10-26 05:25:58,981 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-1) Loaded '4' problem entries from problem file 2016-10-26 05:25:58,995 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-1) Updated '0' entries 2016-10-26 05:25:58,995 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-1) Failed to update '4' entries 2016-10-26 05:25:58,996 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-1) Removed '0' persons from target server 2016-10-26 05:25:58,996 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-1) There are '0' entries before updating inum list 2016-10-26 05:25:58,996 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-1) There are '0' entries after removal '0' entries 2016-10-26 05:25:58,996 INFO [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (pool-1-thread-1) There are '0' entries after adding '0' entries ``` You will have to forgive my LDAP ignorance but what do you mean by "make sure that user objects in your AD back-end have a value assigned to cn attribute." My understanding was that setting up the Cache Refresh properly would handle everything both in OpenDJ/Gluu as well as the back-end LDAP server. Perhaps I need to run some sort of LDAP configuration on my back-end first? I'm not entirely sure what that process would look like so if you have any pointers that would be much appreciated.

By Sahil Arora user 26 Oct 2016 at 7:02 p.m. CDT

Copy
Sahil Arora gravatar
>>My understanding was that setting up the Cache Refresh properly would handle everything both in OpenDJ/Gluu as well as the back-end LDAP server CR would synch data from backend LDAP server and attributtes are taken from backend as they are. Please make sure cn attribute does exist in your back-end LDAP.

By Ryan Kennedy user 27 Oct 2016 at 10:55 a.m. CDT

Copy
Ryan Kennedy gravatar
Thanks Sahil. How can I tell if the CN attribute exists in my back-end LDAP? And if the attribute is missing... do you know of an ldapmodify command I can use to create it?

By Sahil Arora user 31 Oct 2016 at 9:01 p.m. CDT

Copy
Sahil Arora gravatar
You can use ldapsearch command to query the backend AD LDAP. ``` ./ldapsearch -h <IP> -p <port> -D <Bind DN> -j /tmp/.pw -b <Base DN> "(uid=xxxx)" cn ``` Please make sure all attributes mentioned in "Source attribute to Destination attribute Mapping" must exist in backend LDAP. You can refer online tutorials for ldapmodify command to add/update attribute if doesn't exist.

By Ryan Kennedy user 02 Nov 2016 at 6:09 p.m. CDT

Copy
Ryan Kennedy gravatar
Ok, thanks Sahil. That is what I had been doing. Have been making some progress but still not 100% sure how it is all supposed to work. If I create a user in the Gluu interface, shouldn't the cache refresh then sync that user to the backend ldap server when it runs? I have not been seeing it work that way. But, if I create a user in the backend ldap using the `ldapadd` command from the terminal I _do_ see that user show up in the Gluu interface.

By Aliaksandr Samuseu staff 02 Nov 2016 at 6:20 p.m. CDT

Copy
Aliaksandr Samuseu gravatar
Hi, Ryan. >If I create a user in the Gluu interface, shouldn't the cache refresh then sync that user to the backend ldap server when it runs? I'm almost sure it won't, if it will find out that user with the same `uid` already exists, it should throw some kind of exception, either to its own log, or mb to `wrapper.log`, and fail to update it. What CR does normally should exclude any local user entries' handling. It's possible still to both have users in some backend directory and in the internal directory, and authenticate to both, if advaned custom auth scripts are employed, but that's a bit.. unusual approach to it. And of course you'll still need to ensure they have different `uid`'s. >I have not been seeing it work that way. But, if I create a user in the backend ldap using the ldapadd command from the terminal I do see that user show up in the Gluu interface. That's how it's expected to work. Usually, if backend directory(s) is involved, organizations start to use it both for user storage and authentication. Those entries CR creates internally are still needed to cache/aggregate attributes in one place. They don't even have passwords on them.

By Ryan Kennedy user 08 Nov 2016 at 11:12 a.m. CST

Copy
Ryan Kennedy gravatar
Oh. So if we enable cache refresh OR just point the `Server` option in the `Manage LDAP Authentication` section of the `Configuration` to use a different backend LDAP we will then need to manage the user accounts outside of Gluu? Am I understanding that completely? What are the advantages to using Cache Refresh instead of just pointing the LDAP Auth to a different backend then the local OpenDJ/OpenLDAP LDAP server? I guess with Cache Refresh it just requires mapping attributes between the local OXtrust LDAP and the backend LDAP vs. having to create the Gluu speciifc atttributes in the backend LDAP if you choose to run your own? Is that correct? Any other compelling reasons to use Cache Refresh?

By Sahil Arora user 10 Nov 2016 at 10:11 p.m. CST

Copy
Sahil Arora gravatar
> Oh. So if we enable cache refresh OR just point the Server option in the Manage LDAP Authentication section of the Configuration to use a different backend LDAP we will then need to manage the user accounts outside of Gluu? Am I understanding that completely? You must use CR to manage LDAP authentication from backend source. User accounts will need to be managed outside Gluu which are sync'd with Gluu using CR > What are the advantages to using Cache Refresh instead of just pointing the LDAP Auth to a different backend then the local OpenDJ/OpenLDAP LDAP server? Cache refresh will Sync people and attributes from a backend server into the Gluu Server to speed up authentication transactions.Also, used to perform attribute transformations, changing the name of attributes, or even using an interception script to change the values. one of its main points is agreggation and preparation of user data. As Gluu consists from a lot of different modules, like CAS, Shib, oxAuth, Asimba, it would be a real pain to configure at each of them all possible backends so they could get user attributes when they need it. CR can gather attributes from several backends in one place (with some scripting it's possible to allow it to extract some data from non-LDAP storages too), process them into form that can be released, if needed. > I guess with Cache Refresh it just requires mapping attributes between the local OXtrust LDAP and the backend LDAP vs. having to create the Gluu speciifc atttributes in the backend LDAP if you choose to run your own? Is that correct? All scripts (and oxtrust's basic auth flow) by default will search by uid and check whether gluustatus attribute set to active. If they won't find user entry at all there, auth will fail. So CR helps to add these attributes in local database from backend with help of CR. You can also create Gluu specific attributes at backend to use it.

Post an answer