2.4.1 to 3.1.1 results in inability to import user data & connection to LDAP

By: Derek Li user 07 Nov 2017 at 9:06 a.m. CST

15 Responses
Derek Li gravatar
I am in the process of upgrading my Gluu 2.4.1 to 3.1.1, but I have seen many issues plaguing my migration. 1) When I run the `./import2431.py backup_2431` to save my configurations, I get issues where folders are missing inside the oxtrust/oxauth directories... ranging from resources, libs, pages folders that I had to create manually for the python script to work. ``` ------------------------------------------------------------ Gluu Server Community Edition Migration Tool ============================================================ INFO Stopping Webapps oxAuth and Identity. INFO Stopping OpenLDAP Server. INFO Copying the Certificates. INFO Updating the CA Certs Keystore. INFO Copying the custom pages and assets of webapps. Traceback (most recent call last): File "./import2431.py", line 722, in <module> migrator.migrate() File "./import2431.py", line 699, in migrate self.copyCustomFiles() File "./import2431.py", line 254, in copyCustomFiles copy_tree(pair[0], pair[1]) File "/usr/lib/python2.7/distutils/dir_util.py", line 128, in copy_tree "cannot copy tree '%s': not a directory" % src distutils.errors.DistutilsFileError: cannot copy tree 'backup_2431/var/gluu/webapps/oxauth/pages': not a directory ``` 2) When I export my data from 2.4.1 and try to import it back into 3.1.1, I get parsing errors: ``` ERROR 59f8b2c0 <= str2entry NULL (smr_normalize gluuVdsCacheRefreshLastUpdate 21) slapadd: could not parse entry (line=476) 59f8b2c0 <= str2entry: str2ad(oxSectorIdentifierURI): attribute type undefined slapadd: could not parse entry (line=13022) 59f8b2c0 <= str2entry: str2ad(oxSectorIdentifierURI): attribute type undefined slapadd: could not parse entry (line=13030) 59f8b2c0 <= str2entry: str2ad(oxSectorIdentifierURI): attribute type undefined slapadd: could not parse entry (line=13038) ``` 3) When I don't import my previous configuration, the server starts up fine and I am able to log in with my default admin account, but I am unable to connect to the LDAP server even after I change the settings to point to my LDAP server properly. ``` 2017-11-06 15:40:07,813 ERROR [qtp985655350-17] [org.gluu.oxtrust.action.ManagePersonAuthenticationAction] (ManagePersonAuthenticationAction.java:353) - Could not connect to LDAP 2017-11-06 15:40:07,811 ERROR [qtp985655350-17] [org.gluu.site.ldap.LDAPConnectionProvider] (LDAPConnectionProvider.java:76) - Failed to create connection pool with properties: {certsDir=/etc/certs, pythonModulesDir=/opt/gluu/python/libs, ssl.trustStoreFile=/etc/certs/openldap.pkcs12, bindPassword=REDACTED ``` 4) I am unable to import any LDIF files into my Gluu UI. The errors always seem to be saying I need to import via XLS only. If there is any other information missing, I will be glad to provide it. Thank you!
Ubuntu 14.04
closed

Answers

By Aliaksandr Samuseu staff 07 Nov 2017 at 12:45 p.m. CST

Copy
Aliaksandr Samuseu gravatar
Hi, Derek. The migration scripts for 2.4.4 -> 3.1.1 process are undergoing additional tests at the moment, there may be still issues with them, so I would suggest to postpone you upgrade for now. Thank you for reporting those issues, I'll notify script's developer about them immediately.

By Derek Li user 07 Nov 2017 at 12:51 p.m. CST

Copy
Derek Li gravatar
Hi Aliaksandr, Would a 2.4.1 -> 3.0.x be stable? And is there any specific documentation on the Upgrade process for that? I do not see anything substantial that sticks out in the Upgrading process docs (https://gluu.org/docs/ce/upgrade/#upgrade-from-2xx-to-311) Thank you. -Derek

By Aliaksandr Samuseu staff 07 Nov 2017 at 1:55 p.m. CST

Copy
Aliaksandr Samuseu gravatar
>Would a 2.4.1 -> 3.0.x be stable? The current aim is to even include 2.3.x instances in migration coverage, so it should work for any 2.4.x. Though I'm not developer of it and can't say for sure. >And is there any specific documentation on the Upgrade process for that? We have the doc [here](https://gluu.org/docs/ce/3.1.1/upgrade/#upgrade-from-2xx-to-311) and it covers procedure completely. As was said, the problem seems to be with the script itself. There are too much architectural differences between 2.x and 3.x, so it's not that easy to account for everything.

By Derek Li user 07 Nov 2017 at 3:01 p.m. CST

Copy
Derek Li gravatar
Would you be able to confirm whether users have successfully backed up, upgraded, and imported configuration from a 2.4.x to a 3.0.x? or would you happen to know if the features in the UI for 3.0.x works fine unlike 3.1.x. This would save me much time instead of trying to upgrade. Thank you.

By Aliaksandr Samuseu staff 07 Nov 2017 at 8:41 p.m. CST

Copy
Aliaksandr Samuseu gravatar
>Would you be able to confirm whether users have successfully backed up, upgraded, and imported configuration from a 2.4.x to a 3.0.x? Do you mean whether migration of existing user entries is possible with it? It's planned to be possible, and so far seems to work fairly ok. >or would you happen to know if the features in the UI for 3.0.x works fine unlike 3.1.x. This would save me much time instead of trying to upgrade. Could elaborate perhaps? In general, web UI of 3.1.1 package is much more refined in terms of stability. If you'll name features you are interested in, I may answer this with more confidence

By Chris Blanton user 08 Nov 2017 at 4:47 p.m. CST

Copy
Chris Blanton gravatar
Derek, Are you using these instructions: [Upgrade from 2.x.x to 3.1.1](https://gluu.org/docs/ce/3.1.1/upgrade/#upgrade-from-2xx-to-311) I think these are the most current.

By Aliaksandr Samuseu staff 08 Nov 2017 at 5:58 p.m. CST

Copy
Aliaksandr Samuseu gravatar
Hi, Derek. Arvind promised today he'll push updated version of the scripts to the repos soon. We still have certain issues with migrating CR-created users, and CR configuration in general. A couple of other issues as well. Till then I would suggest to refrain from trying to conduct the migration.

By Arvind Tomar staff 08 Nov 2017 at 9:58 p.m. CST

Copy
Arvind Tomar gravatar
Hi, Derek. Sorry for delay There is too many schema changes this time for (2.4.4 -> 3.1.1) I am trying to make it easier As much as possible. so we do not have to make it any steps manually. There is few things are still left to solve yet so I will push it by today. Thanks for support Arvind Tomar

By Derek Li user 09 Nov 2017 at 10:24 a.m. CST

Copy
Derek Li gravatar
Chris: Yes, I was trying to migrate using these instructions for both 2.4.1 > 3.0.1/3.1.1. The ./setup.py worked fine, I was able to see everything and setup my LDAP Server connection fine. Just missing the ability to import my LDAP user/group data into the new Gluu. Aliaksandr: That's great news! Luckily they're separate servers so I can just revert to 2.4.1 for now. Arvind: Thank you for the hard work. Please let me know when the script will be updated.

By Derek Li user 10 Nov 2017 at 8:05 a.m. CST

Copy
Derek Li gravatar
Hello, Do you happen to know of any VERY manual way of importing the LDAP configuration/LDIF files into the new glue via terminal? Where in the directory tree in 3.1.1 can I manually replace the LDIF files? Is that possible? or is there a method of importing/exporting the LDAP configs from scratch? Thank you.

By Aliaksandr Samuseu staff 10 Nov 2017 at 8:49 a.m. CST

Copy
Aliaksandr Samuseu gravatar
Hi, Derek. Not quite sure what is "very manual" way is :) But you can import it using the same tools you used in 2.4.4 (we keep old OpenDJ set of tools in addition to OpenLDAP's set). The directory listens on loopback interface at port 1636. Directory tree wasn't changed that much as well. One notable difference is that you need to use `cn=directory manager,o=gluu` as root admin's bindDN now. Some schema changes were introduced as well.

By Derek Li user 12 Nov 2017 at 10:31 p.m. CST

Copy
Derek Li gravatar
Hi Aliaksandr, By manual, I was wondering if any of the LDIF file contents can be copied and pasted into the new upgraded gluu. Any updates on the progress of the script for taking care of the LDAP config imports? Thank you -Derek

By Arvind Tomar staff 13 Nov 2017 at 10:05 p.m. CST

Copy
Arvind Tomar gravatar
Hi Derek Li We are almost done with CR users related fixes for (2.4.4 -> 3.1.1) in upgrade scripts and doing final testings . so It will be available to you very soon. Thanks for support Arvind Tomar

By Derek Li user 20 Nov 2017 at 9:26 a.m. CST

Copy
Derek Li gravatar
Great news to hear! Would the 2.4.1 -> 3.1.1 work as well? Is there a difference between the export feature in 2.4.1 vs 2.4.3?

By Arvind Tomar staff 21 Nov 2017 at 12:15 a.m. CST

Copy
Arvind Tomar gravatar
Hi Derek Li you can run same script , we have common script for all (2.4.x->3.1.1) and yes it will work for (2.4.1 -> 3.1.1) as well. Thanks Arvind Tomar

Post an answer