By: Matt Ferry user 05 Dec 2017 at 11:21 a.m. CST

9 Responses
Matt Ferry gravatar
We are attempting to import users into our gluu server (3.1.1) , and have successfully imported 27,000+ users. We have one final import block that we are trying to import of around 6000 users. Whenever we try to validate the file we receive: Import Failed. There are persons with simular uid(s) in the input file. Using basic sampling (1 user every few hundred) and the Manage People search function I have been unable to locate any of these users from this last import in the database. I have even reduced the import file down to just the first 10 users on the file, and have verified that none of these users are in the database. Yet even this list of 10 users receives this error. Would appreciate any additional insights that may be available regarding this.

By Aliaksandr Samuseu staff 05 Dec 2017 at 11:31 a.m. CST

Aliaksandr Samuseu gravatar
Hi, Matt. >I have even reduced the import file down to just the first 10 users on the file, and have verified that none of these users are in the database. Yet even this list of 10 users receives this error. Please share this reduced list for us to try to reproduce it locally.

By Michael Schwartz Account Admin 05 Dec 2017 at 11:32 a.m. CST

Michael Schwartz gravatar
I'm not sure the import process is meant for more then the initial upload. The docs should probably state this (if it's in fact true). Can you combine the files into one?

By Matt Ferry user 05 Dec 2017 at 11:41 a.m. CST

Matt Ferry gravatar
Reduced file should be available via the following: https://drive.google.com/file/d/1owXWF8dg47MzSTa-y-FKJPXTskpvmpxO/view?usp=sharing

By Michael Schwartz Account Admin 05 Dec 2017 at 11:43 a.m. CST

Michael Schwartz gravatar
But when you import one file (instead of two), does it go ok?

By Matt Ferry user 05 Dec 2017 at 12:04 p.m. CST

Matt Ferry gravatar
All I'm trying to import is one file at a time. Broke 1 large file up into multiple files so that server response time would be improved, and imported one file at a time. Going to do some more digging with ldapsearch in a bit. Wondering if maybe this list was imported, but became corrupt somehow, which is why these users don't show up in manage people.

By Michael Schwartz Account Admin 05 Dec 2017 at 12:46 p.m. CST

Michael Schwartz gravatar
That's what I was wondering... if you just make one big file for all your users, then it will be validated and added.

By Aliaksandr Samuseu staff 05 Dec 2017 at 4:48 p.m. CST

Aliaksandr Samuseu gravatar
I've been able to reproduce it. It's most likely caused by the fact that OpenLDAP in Gluu, in contrary to OpenDJ used before, now requires that "mail" attribute of user entries is also be unique. The error on the page is a bit misguiding. I confirmed with test that even when usernames in a file are guaranteed to be unique, but emails are not, it still tells you that problem is with usernames, for some reason. I also did a test trying to add a user via web UI with an email value already existing in the LDAP tree, and this failed as well. As far as I know, requirement for "mail" to be unique in 3.x is a known issue (I was talking with Yuriy M. about it recently, and he said it's [supposedly] by design). May be Michael will be able to add something.

By Michael Schwartz Account Admin 06 Dec 2017 at 10:35 a.m. CST

Michael Schwartz gravatar
Email has to be unique: it's a Gluu best security practice.

By Matt Ferry user 07 Dec 2017 at 10:04 a.m. CST

Matt Ferry gravatar
Spent some time late yesterday and this morning testing "Unique" , and even the following file: https://drive.google.com/a/4act.com/file/d/1pv0JAvsGbMEl_DVGG2r2W9r5fbHM7Rzi/view?usp=sharing is failing validation with the same error. Also dug through older threads which lead to looking at the oxtrust.log log file , and here are the log results for this morning. 2017-12-07 15:44:26,915 INFO [qtp985655350-45] [org.gluu.oxtrust.action.Authenticator] (Authenticator.java:421) - redirectURI : https://id.4act.com/identity/authentication/authcode 2017-12-07 15:44:27,027 INFO [qtp985655350-45] [org.gluu.oxtrust.action.Authenticator] (Authenticator.java:436) - Session validation successful. User is logged in 2017-12-07 15:44:27,133 INFO [qtp985655350-45] [org.gluu.oxtrust.action.Authenticator] (Authenticator.java:496) - user uid:admin 2017-12-07 15:44:27,156 INFO [qtp985655350-88] [org.gluu.oxtrust.action.Authenticator] (Authenticator.java:117) - Authenticating user 'admin' 2017-12-07 15:44:27,178 INFO [qtp985655350-88] [org.gluu.oxtrust.action.Authenticator] (Authenticator.java:132) - User 'admin' authenticated successfully 2017-12-07 15:44:29,227 INFO [qtp985655350-86] [org.gluu.oxtrust.ldap.service.OrganizationService] (OrganizationService.java:233) - Starting App version 3.1.1.Final 2017-12-07 15:44:54,329 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=710, numFailedCheckouts=0, numReleasedValid=709) 2017-12-07 15:45:04,276 INFO [qtp985655350-85] [org.gluu.oxtrust.action.PersonImportAction] (PersonImportAction.java:411) - Found 5 persons in input Excel file 2017-12-07 15:45:54,329 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=724, numFailedCheckouts=0, numReleasedValid=723) 2017-12-07 15:46:11,373 WARN [qtp985655350-88] [org.richfaces.log.JavaLogger] (JavaLogger.java:213) - Datascroller personsFormId:personsListId:j_idt175: The requested page #2 isn't found in the model containing 1 pages. Paging is reset to page #1 2017-12-07 15:46:54,329 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=741, numFailedCheckouts=0, numReleasedValid=740) 2017-12-07 15:47:54,329 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=753, numFailedCheckouts=0, numReleasedValid=752) 2017-12-07 15:48:54,329 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=766, numFailedCheckouts=0, numReleasedValid=765) 2017-12-07 15:49:16,422 INFO [qtp985655350-51] [org.gluu.oxtrust.action.PersonImportAction] (PersonImportAction.java:411) - Found 5 persons in input Excel file 2017-12-07 15:49:54,329 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=779, numFailedCheckouts=0, numReleasedValid=778) 2017-12-07 15:50:08,015 INFO [qtp985655350-59] [org.gluu.oxtrust.action.PersonImportAction] (PersonImportAction.java:411) - Found 5 persons in input Excel file 2017-12-07 15:50:14,238 INFO [qtp985655350-28] [org.gluu.oxtrust.action.PersonImportAction] (PersonImportAction.java:411) - Found 5 persons in input Excel file 2017-12-07 15:50:54,329 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=794, numFailedCheckouts=0, numReleasedValid=793) 2017-12-07 15:51:10,104 INFO [qtp985655350-85] [org.gluu.oxtrust.action.PersonImportAction] (PersonImportAction.java:411) - Found 5 persons in input Excel file 2017-12-07 15:51:54,329 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=806, numFailedCheckouts=0, numReleasedValid=805) 2017-12-07 15:52:54,329 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=820, numFailedCheckouts=0, numReleasedValid=819) 2017-12-07 15:53:54,330 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=832, numFailedCheckouts=0, numReleasedValid=831) 2017-12-07 15:54:54,329 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=845, numFailedCheckouts=0, numReleasedValid=844) 2017-12-07 15:55:54,329 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=857, numFailedCheckouts=0, numReleasedValid=856) 2017-12-07 15:56:54,328 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=870, numFailedCheckouts=0, numReleasedValid=869) 2017-12-07 15:57:54,329 INFO [ForkJoinPool.commonPool-worker-50] [gluu.oxtrust.service.status.ldap.LdapStatusTimer] (LdapStatusTimer.java:97) - connectionProvider statistics: LDAPConnectionPoolStatistics(numAvailableConnections=2, maxAvailableConnections=10, numSuccessfulConnectionAttempts=3, numFailedConnectionAttempts=0, numConnectionsClosedDefunct=1, numConnectionsClosedExpired=0, numConnectionsClosedUnneeded=0, numSuccessfulCheckouts=882, numFailedCheckouts=0, numReleasedValid=881) 2017-12-07 15:57:59,561 INFO [qtp985655350-698] [org.gluu.oxtrust.ldap.service.OrganizationService] (OrganizationService.java:233) - Starting App version 3.1.1.Final As far as I can tell these users DO NOT EXIST, and are completely unique. There shouldn't be any reason why they are receiving a Simular UID error. There isn't anything similar about them to items that are already in the database.