Hi, Krishnan.
Please consider points made by Michael above.
I could try to build a proof of concept setup for this use case. Please let me know what kind of ADFS deployment do you use - the more details the better. If it's a stand-alone, on premises kind of deployment (not a cloud solutions), let me know what Windows Server / ADFS version it is.