Additional information / insight: I see this issue reports the same problem, but from a different perspective - it reports that LDAP attributes are not reflecting the null update. https://github.com/GluuFederation/SCIM-Client/issues/21 Related, and potentially fixed? IETF discussion on the use of null values within PUT based requests: https://www.ietf.org/mail-archive/web/scim/current/msg02758.html Sample (from elsewhere) showing the potential to modify using a PATCH verb; and supplying the meta attributes content as outlined above: http://www.simplecloud.info/specs/draft-scim-api-01.html#edit-resource-with-patch

Hi Chris, Thanks for participating here in support forum sharing your insight into the topic > Implementation appears to be correctly aligned with the SCIM standard, in that a blank string, or a null value is ignored and is treated as if it were not passed within the request. Correctly, as per section 2.5 of RFC 7643 > IETF based conversations appear to suggest that in order to remove an attribute's value that a PATCH request method can be used, supplying the attribute name as part of an attributes array based list, under the meta object That's a discussion in the setting of SCIM v1.1 but we are only focused on SCIM version 2 currently. What's more, latest Gluu Server version (3.1) removed support for 1.x so we won't take into consideration stuff from previous standards The "official" way to do removal is using PATCH as described in section 3.5.2.2 of RFC 7644 > I understand from the SCIM service provider config response that a PATCH request type is not supported. Not exactly. PATCH appears as unsupported in the ServiceProviderConfig despite we have already an implementation for it. This is so because it's not production-ready but an experimental feature and sadly, it does not adhere to the spec very well Implementing PATCH thoroughly is extremely time-consuming in my opinion: when you glance at section 3.5.2 you see a lot of conditions like "if ... then ... otherwise ... but ... except" > So my question (plea for assistance) is: is it possible to remove an attribute's value via SCIM? Not officially as you may have noticed. Here is a test in the SCIM-client java project that uses patching capabilities https://github.com/GluuFederation/SCIM-Client/blob/master/src/test/java/gluu/scim2/client/UserPatchWebServiceTestCases.java If you want to remove an attribute, you should: - use op=remove for the Operation - refrain from passing path (it's not supported) - supply a user representation as value Last step is the gotcha. Say you want to remove a (non-required) single-valued attribute: what you have to do is passing a non-empty value in that attribute. This has to do with null values being ignored in our current impl... so just use some dummy value there. For other attrs to remain intact set them to null. This workaround does not hold for multi-valued attributes. In that case you have to pass the current (existing) values you want to be removed. To start, give it a try with `middleName` or `nickName`... Please keep in mind patching is a work in progress (not officialy supported) so it might not fulfil your expectations. > I see this issue reports the same problem... /issues/21 The issue came as a result of an urgent customer need so we resort to deviate a bit from the spec (only for middleName), unfortunately. Regards.

Hi, thank you so much for your response. We've not been using any of the Java SCIM client libraries at all, but have been formulating our HTTP requests to the various SCIM endpoints, with our well formed JSON (as required). So for me your reference into the Java test class is difficult to digest (sorry). I understand your more general explanation from your response and I'm keen to understand more please so that I can create an appropriate SCIM HTTP(s) request. - I understand I should be using the PATCH verb / request type. - I don't know what SCIM end-point I should be hitting with my request. - I expect that I should present this with a JSON payload in the body of the HTTP request. I presume that the "id"="@xxxxx" user element ought to be within the JSON body (I got nervous, worrying that it might remove the user). - I don't know where the op=remove operation ought to be placed, I presume as a URL param? I use postman (Chrome app) to sort out how this all works, before it's then coded into our application libraries. Could you please provide me with a few more pointers? Thank you, very much. Chris.

> I understand I should be using the PATCH verb / request type. I forgot to mention that for the purpose of this workaround you must use PUT. > I don't know what SCIM end-point I should be hitting with my request. `/scim/v2/Users/patch/{id}` Don't be misled by the patch word: stick to PUT > I expect that I should present this with a JSON payload in the body of the HTTP request. Yes... as in most SCIM API methods > I presume that the "id"="@xxxxx" user element ought to be within the JSON body No, it's part of URL after `...patch/` > I got nervous, worrying that it might remove the user I am sure it will not delete your users at all but be careful: do all sort of tests with dummy users first > I don't know where the op=remove operation ought to be placed Once more please see section 3.5.2 of [RFC 7644](https://tools.ietf.org/html/rfc7644), it's the only way to get it. > I presume as a URL param? Inside payload. Here is an example: ``` { "schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"], "operatons":[{ "operationName":"remove", "value":{ "schemas":["urn:ietf:params:scim:schemas:core:2.0:User"], "userName":"bjensen", "emails":[ { "type": "work" }] } }] } ``` Note how the workaround example differs from actual payload of spec. Important notes: * Experimenting with 3.0.x I found you definitely cannot remove single-valued attributes, only multi-valued * In the case shown above I am assuming you are trying to remove the already existing work e-mail entry of the user. Just providing "type" does the job. The same applies for all of its kind: phones, ims, photos, addresses, entitlements, and x509Certificates * Don't be misled by typos you see in the example > Thank you, very much. Good luck with your tests.

Thank you for the extended examples and the end-point listings, also the instruction to use a PUT rather than a PATCH verb. I would never have guessed those details as the standards based spec seems to vary quite considerably from that. Even things like 'operations' being in lower case, the 'operationName' being used and not 'op' etc etc. I wanted to test this and get things working (for others who might be following this and find it useful): The following worked: ``` { "schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"], "operatons":[{ "operationName":"remove", "value":{ "schemas":["urn:ietf:params:scim:schemas:core:2.0:User"], "emails":[ { "type": "other" }] } }] } ``` Using a PUT method to the following URL: https://SITE/identity/seam/resource/restv1/scim/v2/Users/patch/@!UserINUM - Worked, so did: ``` { "schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"], "operatons":[{ "operationName":"remove", "value":{ "schemas":["urn:ietf:params:scim:schemas:core:2.0:User"], "phoneNumbers":[ { "type": "work" }] } }] } ``` The most difficult part of this is that I cannot use this approach or work-around to affect single value attributes. The functional effect I have been trying to have is to reset the oxExternalUid field - to remove the established topt setup from the user profile - via SCIM. This is pretty important to be able to do as a self-serve function for us via a custom built user profile administration system which we have built - external to the Gluu User Admin API. So, testing this with "nickName" : or... ``` { "schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"], "operatons":[{ "operationName":"remove", "value":{ "schemas":["urn:ietf:params:scim:schemas:extension:gluu:2.0:User"], "urn:ietf:params:scim:schemas:extension:gluu:2.0:User": { "oxExternalUid": "totp:whateverwelike" } } }] } ``` Doesn't have any positive effect, it returns an HTTP Status 400 response - check input parameters. I've separately been trying to figure out what creates the formatted 'topt:xxxxxxxxxxxxxxxxxxx' string within the oxExternalID field (but this is a topic for another ticket at some point). It is critical for us to be able to reset this oxExternalUid attribute to a null value in order to reset the workflow for the user who may need to re-establish their TOPT 2FA setup. It's one thing to refer us to https://tools.ietf.org/html/rfc7644 - but when the implementation doesn't follow it, it's a bit tough... Functionally, if we can't reset an attribute to a blank string, and we can't nullify it then how are we expected to reasonably use SCIM to manage accounts - there are justifiable use cases where we need to blank something which already exists? I cannot reasonably delete the user account entirely and re-insert it just to remove an attribute, I'd be concerned about what else is linked to it - groups, etc.. So, is there any other potential way to nullify an existing User's attribute value via SCIM (for a singular type of attribute)?

With the information you have supplied to us it's now more clear where and when you need nullification to take effect. This way we may give you more effective assistance. I think we can leverage the custom script infrastructure to solve this problem. Please await 1-2 days more so I can provide you with a working sample script and instructions on how to set this up. The oxExternalUid is persisted to LDAP once users have typed the 6-digit code after scanning the QR with their mobile devices. This is achieved with a custom script as well. Obviously you already have this script enabled in your installation.

I would like this issue addressing in two parts please. The first is; *how can we nullify an an existing attribute via SCIM*? - This is important in all sorts of circumstances if you're relying on SCIM to perform your user management - which we want to (for various reasons). - Amongst other things we will be using this to reset the TOPT codes on an account as required. *However, this also needs to be available and work for multi-valued attributes too*. It's not a fully functioning system if SCIM can't remove content which was entered against a record. ----- The second issue - which I am happy to park for now relates to establishing, and entering into LDAP (via SCIM), valid and actionable TOPT values. I think this is best left for another support request / ticket, and I will likely need to be told which group within support I should log that one under, it's not strictly Authentication as it's more to do with the establishment of the TOPT attribute values - which feels more like an identity management consideration?? - Let's park this second part until we've got the first bit sorted out (please).

Hi Chris, Quick recap: 1- The mechanism to perform attribute removal in SCIM according to spec is via PATCH. 2- PATCHING is not supported by Gluu server yet: it's in our TODO list, but it won't be available in near future. You can leverage the custom script infrastructure for this task: after every operation is executed in SCIM (create, update, etc.), you can do some post-processing with a python script that will allow you to do removal or any sort of custom business-logic You can code the script yourself. Documentation about custom scripts is at [https://gluu.org/docs/ce/3.1.0/admin-guide/custom-script/](https://gluu.org/docs/ce/3.1.0/admin-guide/custom-script/). In oxTrust there is already a sample script: go to `Custom scripts` > `SCIM`. If you still need more assistance, such as having this feature added ASAP to your Gluu Server or having a script developed for you, you should buy a *Gluu VIP Support* plan: https://www.gluu.org/gluu-server/pricing/ . Kind regards, Jose.

We're not at a commercial point in our programme of work yet, it's still very much a proof of concept, and checking the suitability of the platform (for our needs). The support options which you've pointed me to are out of reach for us. We're a small (and talented bunch) of programmers, but not in python, so this is a little beyond our reach without an initial working example. I've spent most of my day today learning Python syntax and trying to get a working SCIM interception script to push values through into LDAP using the oxTrust LDAP PersonService: org.gluu.oxtrust.ldap.service import IPersonService, PersonService the top of my SCIM intercept script looks like this: ``` from org.xdi.model.custom.script.type.scim import ScimType from org.xdi.util import StringHelper, ArrayHelper from java.util import Arrays, ArrayList from org.gluu.oxtrust.ldap.service import IPersonService, PersonService from org.gluu.oxtrust.model import GluuCustomPerson from org.gluu.oxtrust.model import GluuCustomAttribute ``` The updateUser section looks like this: ``` def updateUser(self, user, configurationAttributes): personService = PersonService.instance() oldUser = personService.getPersonByUid(user.getUid()) listToNullify = [] print "ScimEventHandler (updateUser): interceptor code looking for 'nullValue's to replace" print user.getCustomAttributes() print "Mandatory Attributes:" for thisAttribute in personService.getMandatoryAtributes(): print thisAttribute.toString() print "Looking for items which need to be nullified..." for thisAttribute in user.getCustomAttributes(): if thisAttribute.getDisplayValue() == "nullValue": print thisAttribute.toString() print "SCIM - 'nullValue' string detected within attribute: " + thisAttribute.getName() + " for user: " + user.getDisplayName() print "SCIM - adding it to the list to make truely null " listToNullify.append(thisAttribute.getName()) for itemToNullify in listToNullify: print "Setting to null: [" + itemToNullify + "]" #user.setAttribute(itemToNullify,None) user.removeAttribute(itemToNullify) if user.getAttribute(itemToNullify) == None: print "Done - apparently" else: print "Not done - apparently: " + user.getAttribute(itemToNullify) print "SCIM - Checking through user attributes again..." for thisAttribute in user.getCustomAttributes(): if thisAttribute.getDisplayValue() == "nullValue": print "SCIM - !!!! 'nullValue' string still detected within attribute: " + thisAttribute.getName() + " for user: " + user.getDisplayName() return True ``` This works to a point, I get a status 200 response, with a modified User JSON object returned, with those elements set to null. - It would appear from SCIM that it had all worked, however these changes are not committed through into LDAP. When I do a fresh Get on the User through SCIM, it continues to show the previous field values. I had even tried the commented version which setAttribute to python's None. - This too appeared to modify the user object within the Python script - as shown by the log outputs. - I've attached the segment of the oxTrust_script log which relates to the captured debug output from my script. ``` 2017-09-14 06:09:16,106 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - ScimEventHandler (updateUser): interceptor code looking for 'nullValue's to replace ... Whole Attribute Collection Dump removed ... 2017-09-14 06:09:16,107 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Mandatory Attributes: 2017-09-14 06:09:16,107 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Attribute [name=uid, values=[], metadata=null] 2017-09-14 06:09:16,108 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Attribute [name=givenName, values=[], metadata=null] 2017-09-14 06:09:16,108 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Attribute [name=displayName, values=[], metadata=null] 2017-09-14 06:09:16,108 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Attribute [name=sn, values=[], metadata=null] 2017-09-14 06:09:16,108 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Attribute [name=mail, values=[], metadata=null] 2017-09-14 06:09:16,109 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Attribute [name=userPassword, values=[], metadata=null] 2017-09-14 06:09:16,109 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Attribute [name=gluuStatus, values=[], metadata=null] 2017-09-14 06:09:16,109 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Looking for items which need to be nullified... 2017-09-14 06:09:16,110 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Attribute [name=oxTrustUserType, values=[nullValue], metadata=null] 2017-09-14 06:09:16,110 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - SCIM - 'nullValue' string detected within attribute: oxTrustUserType for user: Chris Wiltshire 2017-09-14 06:09:16,110 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - SCIM - adding it to the list to make truely null 2017-09-14 06:09:16,111 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Attribute [name=oxExternalUid, values=[nullValue], metadata=null] 2017-09-14 06:09:16,111 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - SCIM - 'nullValue' string detected within attribute: oxExternalUid for user: Chris Wiltshire 2017-09-14 06:09:16,112 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - SCIM - adding it to the list to make truely null 2017-09-14 06:09:16,112 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Setting to null: [oxTrustUserType] 2017-09-14 06:09:16,112 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Done - apparently 2017-09-14 06:09:16,113 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Setting to null: [oxExternalUid] 2017-09-14 06:09:16,113 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - Done - apparently 2017-09-14 06:09:16,113 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - SCIM - Checking through user attributes again... 2017-09-14 06:09:16,114 INFO [qtp274064559-19] [org.xdi.service.PythonService$PythonLoggerOutputStream] (PythonService.java:219) - SCIM - Specifically checking those which were supposed to be nullified... ``` In your post, three above this one, it appeared that you had offered to help a little more and provide some form of sample script? Is it still possible to request this assistance from you under the CE model? - I'm also happy to post our final working script example into this thread for the community once done. Your very initial response to me appeared to be aligned with community participation and sharing of insights, I'm still happy to do that. :) Thank you in advance for any further assistance you can lend.

Could you please re-open this ticket based on the information which I've presented above?