Hi, Neemesh.
>it is up to each individual SP to have their own list of authorised users to determine access
Yes, that's how it's intended to work. Authorization is usually taken care of by SP/RP. Though UMA (which is supported by Gluu) is capable of doing authorization too, but you'll need to implement its logic through custom scripts yourself.
Regards,
Alex.