Restricting access is called Authorization.
Authorization is actually not great in SAML flow ( it's best in OpenID Connect with the combination of User Managed Access ( UMA ).
However there are couple of way you can implement Authorization in SAML.
- With SAML attribute. Say, you have two applications. Salesforce and Google Workplace. To implement authorization, you can have two "special custom attribute" for these two apps. Whenever user will login to Google, IDP will send that special attribute to SP ( Google in this case ) and if that attribute is missing.... Login to that app ( Google ) won't happen.
- Implement 2FA ( two factor authentication ) for specific SP, say Salesforce. User who will use Salesforce must use 2FA like SuperGluu. If 2FA not available, that user / user group won't be able to log into Salesforce.