Can't load Cache Refresh scripts. Using default script

By: Steven Carmody user 26 May 2017 at 9:03 a.m. CDT

32 Responses
Steven Carmody gravatar
I watched the (excellent) videos describing how to configure cache refresh. However, when I enter my info into the cache refresh tabs, and click "update and validate script", I'm presented with the above error msg. I can login to the container and run ldapsearch successfully. I'd note, tho, that ldapsearch asks: The server is using the following certificate: Issuer DN: CN=InCommon RSA Server CA, OU=InCommon, O=Internet2, L=Ann Arbor, ST=MI, C=US Do you wish to trust this certificate and continue connecting to the server? I said YES might this be at all related to the default script problem ? thanks.
CentOS66
closed

Answers

By Mohib Zico Account Admin 26 May 2017 at 9:07 a.m. CDT

Copy
Mohib Zico gravatar
Hi Steve, That's normal message. _"Can't load Cache Refresh scripts. Using default script"_ The translation of above message is: Unable to load any external Cache Refresh script, using default one. Generally we use any Cache Refresh external 'script' ( Manage Custom Scripts --> 'Cache Refresh' tab ) to calculate attributes like ePSA or any other custom attributes which takes some value and do some calculation inside Gluu Server.

By Steven Carmody user 26 May 2017 at 9:43 a.m. CDT

Copy
Steven Carmody gravatar
thanks ! so I have enabled "Cache Refresh". I have a polling interval of 15 minutes. I've waited 30 minutes -- I'm not seeing anything on the cache refresh screens (ie still no info on "Last Run" or "Updates at the last run". I looked at the oxtrust_cache_refresh.log, and there are no recent entries. Thoughts on what to look at next ? btw, the timestamps in that log file are for what time zone ?

By Mohib Zico Account Admin 26 May 2017 at 9:47 a.m. CDT

Copy
Mohib Zico gravatar
Do you have any Cache Refresh custom script?

By Steven Carmody user 26 May 2017 at 9:48 a.m. CDT

Copy
Steven Carmody gravatar
NO

By Mohib Zico Account Admin 26 May 2017 at 10:05 a.m. CDT

Copy
Mohib Zico gravatar
Ok, there are basically lot of things we can check but let's start with first step. - Check the value of 'gluuVdsCacheRefreshEnabled', it should be 'enabled' - Check the value of 'gluuVdsCacheRefreshPollingInterval', it should be '15' for your case - The timezone inside container should be UTC by default, it's better to reconfigure that with your own timezone ( with tzselect etc. ) - Stop identity service with 'service identity stop' - Start tailing oxtrust_cache_refresh.log - Start identity service with 'service identity start' - See if you get any error/message in this log or not. If above step doesn't help, please feel free to share CR configuration screenshots and full oxtrust_cache_refresh.log.

By Steven Carmody user 26 May 2017 at 10:17 a.m. CDT

Copy
Steven Carmody gravatar
Those first two are ENV variables ? OR something else ? I don't see anything in the GUI to display those ? So I did "service gluu-server-3.0.1 login" , and then typed echo $gluuVdsCacheRefreshEnabled i just got a blank line -- printenv doesn't list either of those variables did I go about this correctly ?

By Mohib Zico Account Admin 26 May 2017 at 10:21 a.m. CDT

Copy
Mohib Zico gravatar
Sorry, I should have mentioned that earlier. No, they are ldap attribute. Here are sample commands to get those values: ``` /opt/opendj/bin/ldapsearch -h localhost -p 1636 -Z -X -D "cn=directory manager,o=gluu" -j /tmp/.pw -b 'o=gluu' -T 'gluuVdsCacheRefreshEnabled=*' gluuVdsCacheRefreshEnabled /opt/opendj/bin/ldapsearch -h localhost -p 1636 -Z -X -D "cn=directory manager,o=gluu" -j /tmp/.pw -b 'o=gluu' -T 'gluuVdsCacheRefreshPollingInterval=*' gluuVdsCacheRefreshPollingInterval ```

By Steven Carmody user 26 May 2017 at 12:34 p.m. CDT

Copy
Steven Carmody gravatar
gluuVdsCacheRefreshEnabled: enabled gluuVdsCacheRefreshPollingInterval: 15 this is the only line that appeared after service identity start 2017-05-26 15:37:12,636 INFO [main] [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (CacheRefreshTimer.java:138) - Initializing CacheRefr eshTimer... since then, every hour, lines like this appear: 2017-05-26 13:50:36,746 ERROR [qtp274064559-12] [org.gluu.oxtrust.action.ConfigureCacheRefreshAction] (ConfigureCacheRefreshAction.java:398) - Can't load Cache Refresh scripts. Using default script suggestions ?

By Mohib Zico Account Admin 26 May 2017 at 12:54 p.m. CDT

Copy
Mohib Zico gravatar
Please share screenshots of your Cache Refresh configurations if possible. And see 'Cache Refresh' script of Manage Authentication section is enabled or disabled. [ screenshot attached ]
Screenshot_9.png

By Steven Carmody user 26 May 2017 at 1:30 p.m. CDT

Copy
Steven Carmody gravatar
Your screen snap seems to come from "manage custom scripts". on my system, the ENABLED box is NOT checked -- is that a problem ? screen snaps from cache refresh available at attached url, and manage scripts.
Video or screenshot link

By Mohib Zico Account Admin 26 May 2017 at 2:01 p.m. CDT

Copy
Mohib Zico gravatar
Thanks. 'Server IP Address', this will be the IP address of VM where you Gluu Server is installed.

By Steven Carmody user 26 May 2017 at 2:47 p.m. CDT

Copy
Steven Carmody gravatar
thanks !! Much better ! log file now contains: org.gluu.site.ldap.persistence.exception.EntryPersistenceException: Failed to find entries with baseDN: ou=people,dc=brown,dc=edu, filter: (&(uid=*)(objectClass=person)) from within container I was able to do an ldapsearch with that filter (slight change) (&(uid=stc)(objectClass=person)) and it worked but, got this msg and had to respond: Do you wish to trust this certificate and continue connecting to the server? Please enter "yes" or "no":y if that's the problem, do I need to add a cert to a keystore ? Where's that file, and how do I do that ? thanks !

By Mohib Zico Account Admin 26 May 2017 at 4:19 p.m. CDT

Copy
Mohib Zico gravatar
>> Failed to find entries with baseDN: ou=people,dc=brown,dc=edu, filter: (&(uid=*)(objectClass=person)) This BindDN user you are using has read access to read 'ou=people,...' ? >> Do you wish to trust this certificate and continue connecting to the server? Please enter "yes" or "no":y Did you use '-Z -X' in your ldapsearch command?

By Steven Carmody user 26 May 2017 at 4:53 p.m. CDT

Copy
Steven Carmody gravatar
here's the command I used: /opt/opendj/bin/ldapsearch -h ldapauth.cis-qas.brown.edu -p 636 -Z -s sub -T -D 'cn=stc_auth,ou=special users,dc=brown,dc=edu' -w XXX -b 'ou=people,dc=brown,dc=edu' -z 5 '(&(uid=stc)(objectClass=person))' i did specify a baseDN with ou=people; I did use -Z; I did not use -X I *think* I'm using simple authN, not SASL.

By Mohib Zico Account Admin 26 May 2017 at 4:54 p.m. CDT

Copy
Mohib Zico gravatar
Gotcha. Using -X will 'trust all'.

By Michael Schwartz Account Admin 27 May 2017 at 5:48 p.m. CDT

Copy
Michael Schwartz gravatar
Note: in all our examples we use _ldapsearch_ from opendj `/opt/opendj/bin/ldapsearch` not OpenLDAP _ldapsearch_. That's why you're maybe having some confusion over `-X` Also, I'd recommend using `-j` instead of `-w`. This will enable you to put the password in a file, which can be removed (i.e. don't leave the DM password in the history).

By Steven Carmody user 29 May 2017 at 1:18 p.m. CDT

Copy
Steven Carmody gravatar
I had a thought about why the cache refresh is failing. The cache refresh log contains entries like this, spaced one minute apart (my refresh cycle is 15 minutes): 2017-05-29 00:08:12,681 INFO [pool-2-thread-1] [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (CacheRefreshTimer.java:303) - Attempting to load entries from source server 2017-05-29 00:08:12,952 ERROR [pool-2-thread-1] [gluu.oxtrust.ldap.cache.service.CacheRefreshTimer] (CacheRefreshTimer.java:183) - Exception happened while executing cache refresh synchronization org.gluu.site.ldap.persistence.exception.EntryPersistenceException: Failed to find entries with baseDN: ou=people,dc=brown,dc=edu, filter: (&(uid=*)(objectClass=person)) There's not much detail about why its failing, so from within the container I issued an opendj search with that same filter -- I got back a number of entries, followed by: SEARCH operation failed Result Code: 4 (Size Limit Exceeded) So, while I have a special ldap credential, I'm not able to retrieve all 18K entries. The PROD staff is being very careful about who can dump out all of the ldap entries. Getting permission to clone our PROD ldap servers onto my test machine may take a while .... Is there any way, within GLUU, to specify a list/set of user objects to retrieve ? I can retrieve individual entries, but not ALL entries. Failing that, can I manually enter a number of users into the internal ldap server, and just skip the cache refresh for now ?

By Michael Schwartz Account Admin 29 May 2017 at 2:49 p.m. CDT

Copy
Michael Schwartz gravatar
I see, this filter is bad: `(&(uid=*)(objectClass=person))` In the past, we actually tried to break the searches up by `sn`, for example ``` (sn=aa*) (sn=ab*) (sn=ac*) . . . ``` So as not to overload the backend LDAP server with big requests. I'll have to dig into why this is not happening. Trying to get all the data as one big dump is a bad idea, and may impact the backend LDAP server. I post an update tomorrow.

By Mohib Zico Account Admin 01 Jun 2017 at 3:52 p.m. CDT

Copy
Mohib Zico gravatar
>> Is there any way, within GLUU, to specify a list/set of user objects to retrieve ? I can retrieve individual entries, but not ALL entries. Yes, you can specify the 'Search size limit' from 'Cache refresh' page for your backend searching.

By Mohib Zico Account Admin 02 Jun 2017 at 10:32 a.m. CDT

Copy
Mohib Zico gravatar
Hi Steven, Any luck?

By Steven Carmody user 02 Jun 2017 at 2:01 p.m. CDT

Copy
Steven Carmody gravatar
Hi, I got them to temporarily boost the limit. However, now the cache log is filled with entries like this: Non-deterministic primary key. Skipping user with key: CacheCompoundKey [primaryKeyValues=[tvallant], secondaryKeyValues=null, tertiaryKeyValues=null] that primaryKeyValues value looks like a local uid .... which is what I would expect. I've picked a small number of these people, and tried to look them up via the gluu admin/users/manage users/search function. I haven't yet found anyone ... suggestions ? thanks !

By Mohib Zico Account Admin 02 Jun 2017 at 2:03 p.m. CDT

Copy
Mohib Zico gravatar
This generally happens if there are multiple values in backend, Gluu Server reject them. Can we check if this user 'tvallant' is unique there? Even, let's check if this user's email_address is unique or not.

By Steven Carmody user 02 Jun 2017 at 2:27 p.m. CDT

Copy
Steven Carmody gravatar
Is there an easy way to see if there are *any* user objects in the Gluu server ? I'm pretty sure that uid's and email addresses are unique -- otherwise, our entire IDM system has a major flaw...

By Mohib Zico Account Admin 02 Jun 2017 at 3:15 p.m. CDT

Copy
Mohib Zico gravatar
Pardon me but I think I didn't understand that term 'user object'. Any example?

By Steven Carmody user 02 Jun 2017 at 3:45 p.m. CDT

Copy
Steven Carmody gravatar
If I want to do an ldapsearch against the (internal) gluu server... eg /opt/opendj/bin/ldapsearch -h what do I use as the hostname and the port ? the dns name for the machie is stc-test19.cis.brown.edu thanks

By Mohib Zico Account Admin 02 Jun 2017 at 3:48 p.m. CDT

Copy
Mohib Zico gravatar
localhost:1636

By Steven Carmody user 02 Jun 2017 at 3:50 p.m. CDT

Copy
Steven Carmody gravatar
Another question -- for the source backup server I've specified ou=people,dc=brown,dc=edu but the person mentioned in the above error message (tvallant) is NOT in that OU. They can be found by searching dc=brown,dc=edu, but they are NOT found if you include ou=people in the baseDN do I misunderstand something > ? thanks

By Mohib Zico Account Admin 02 Jun 2017 at 3:55 p.m. CDT

Copy
Mohib Zico gravatar
>> ou=people,dc=brown,dc=edu Yes, Gluu Server will pull users only from this location.

By Steven Carmody user 03 Jun 2017 at 1:29 p.m. CDT

Copy
Steven Carmody gravatar
Hi, I think we've got too many outstanding questions in this thread -- can we go back to the first one, and concentrate on that ? Deal with the other questions later -- so, here's the primary question: the cache log is filled with entries like this: Non-deterministic primary key. Skipping user with key: CacheCompoundKey [primaryKeyValues=[tvallant], secondaryKeyValues=null, tertiaryKeyValues=null] that primaryKeyValues value looks like a local uid .... which is what I would expect. Those are unique within our community, and there will only be one object with that uid value in the OU=People OU I've picked a small number of these people, and tried to look them up via the gluu admin/users/manage users/search function. I haven't yet found anyone in the internal gluu ldap server... suggestions ? What would you like me to do/try next ? thanks !

By Mohib Zico Account Admin 03 Jun 2017 at 1:39 p.m. CDT

Copy
Mohib Zico gravatar
Hi Steven, Here is what I would do in this case: - I would make sure this UID and it's associated entries ( specially email_address ) are unique in backend. - I would ldapsearch for this user in local 'o=gluu' + 'o=site' tree - I would declare those attributes which I really want to pull in Gluu Server ( as Source attribute in CR page ). - I would make sure that those source attribute are properly mapped with Gluu Server associated attributes in Gluu Server. - I would never expect any other people to be pulled other than what I specified as BindDN - I would check the total number of people of baseDN which I specified in Gluu Server and compare the number inside 'ou=people,o=abacd,o=gluu'. If they are same or almost same; I would just ignore those error message for first time. After successful authentication testing. I would dig deep to find out what's wrong for these users.

By Steven Carmody user 12 Jun 2017 at 10:44 a.m. CDT

Copy
Steven Carmody gravatar
My apologies -- the idm team told me to use a specific ldap server. It turns out that server does indeed contain multiple user objects with the same UID value. They didn't know that. I'm now using a different ldap server, and the cache process ran successfully. sorry for the confusion.

By Mohib Zico Account Admin 12 Jun 2017 at 10:49 a.m. CDT

Copy
Mohib Zico gravatar
Hooray!!! You made it!!

Post an answer