You could use a filter on memberOf if you want to use groups. See the "Custom LDAP filter" filed on the "Customer Backend/Key Attributes" tab of the cache refresh config. Set it to something like:
```
(&
(membeof={dn-group1})
(membeof={dn-group2})
(membeof={dn-group3})
(membeof={dn-group4})
.
.
.
)
```