Hey, We use the key rotation functionality of the cluster manager. We have set up and configured the cluster manager in a new environment. However, when we tested the key rotation by clicking on 'Rotate Key', the keys didn't rotate, and I couldn't find any errors. Additionally, the logs of the Gluu servers do not show any notifications regarding this issue. The logs below are from the clustermgr.log ``` 2025-03-03 06:35:49,073 - administrator@local - DEBUG - Making Ldap Connection to ldaps://ip:1636 2025-03-03 06:35:49,073 - administrator@local - DEBUG - BASIC:instantiated Tls: <Tls(validate=<VerifyMode.CERT_NONE: 0>)> 2025-03-03 06:35:49,073 - administrator@local - DEBUG - BASIC:instantiated Server: <Server(host='ip', port=1636, use_ssl=True, allowed_referral_hosts=[('*', True)], tls=Tls(validate=<VerifyMode.CERT_NONE: 0>), get_info='SCHEMA', mode='IP_V6_PREFERRED')> 2025-03-03 06:35:49,074 - administrator@local - DEBUG - BASIC:instantiated <SyncStrategy>: <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - unbound - closed - <no socket> - tls not started - not listening - No strategy - internal decoder - async - real DSA - not pooled - cannot stream output> 2025-03-03 06:35:49,074 - administrator@local - DEBUG - BASIC:instantiated Connection: <Connection(server=Server(host='ip', port=1636, use_ssl=True, allowed_referral_hosts=[('*', True)], tls=Tls(validate=<VerifyMode.CERT_NONE: 0>), get_info='SCHEMA', mode='IP_V6_PREFERRED'), user='cn=Directory Manager', password='<stripped 15 characters of sensitive data>', auto_bind='DEFAULT', version=3, authentication='SIMPLE', client_strategy='SYNC', auto_referrals=True, check_names=True, read_only=False, lazy=False, raise_exceptions=False, fast_decoder=True, auto_range=True, return_empty_attributes=True, auto_encode=True, auto_escape=True, use_referral_cache=False)> 2025-03-03 06:35:49,074 - administrator@local - DEBUG - BASIC:start BIND operation via <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - unbound - closed - <no socket> - tls not started - not listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,074 - administrator@local - DEBUG - BASIC:address for <ldaps://ip:1636 - ssl> resolved as <[<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('ip', 1636)]> 2025-03-03 06:35:49,074 - administrator@local - DEBUG - BASIC:obtained candidate address for <ldaps://ip:1636 - ssl>: <[<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('ip', 1636)]> with mode IP_V6_PREFERRED 2025-03-03 06:35:49,074 - administrator@local - DEBUG - BASIC:try to open candidate address [<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('ip', 1636)] 2025-03-03 06:35:49,086 - administrator@local - DEBUG - BASIC:refreshing server info for <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - bound - open - <local: ip.17:37963 - remote: ip:1636> - tls not started - listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,086 - administrator@local - DEBUG - BASIC:start SEARCH operation via <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - bound - open - <local: ip.17:37963 - remote: ip:1636> - tls not started - listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,090 - administrator@local - DEBUG - BASIC:done SEARCH operation, result <True> 2025-03-03 06:35:49,090 - administrator@local - DEBUG - BASIC:start SEARCH operation via <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - bound - open - <local: ip.17:37963 - remote: ip:1636> - tls not started - listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,137 - administrator@local - DEBUG - BASIC:done SEARCH operation, result <True> 2025-03-03 06:35:49,220 - administrator@local - DEBUG - BASIC:schema read for <ldaps://ip:1636 - ssl> via <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - bound - open - <local: ip.17:37963 - remote: ip:1636> - tls not started - listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,220 - administrator@local - DEBUG - BASIC:done BIND operation, result <True> 2025-03-03 06:35:49,220 - administrator@local - DEBUG - BASIC:start SEARCH operation via <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - bound - open - <local: ip.17:37963 - remote: ip:1636> - tls not started - listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,223 - administrator@local - DEBUG - BASIC:done SEARCH operation, result <True> 2025-03-03 06:35:49,223 - administrator@local - DEBUG - BASIC:start UNBIND operation via <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - bound - open - <local: ip.17:37963 - remote: ip:1636> - tls not started - listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,224 - administrator@local - DEBUG - BASIC:done UNBIND operation, result <True> 2025-03-03 06:35:49,226 - administrator@local - DEBUG - UPDTE[KeyRotation]: {"id": "1", "interval": "1", "rotated_at": "None", "type": "pkcs12", "inum_appliance": "{\"backup\": true}", "enabled": "True"} 2025-03-03 06:35:49,230 - administrator@local - INFO - [2025-Mar-03 06:35] ip.40.100 POST http /keyrotation/settings/? 302 FOUND 2025-03-03 06:35:49,253 - administrator@local - DEBUG - Making Ldap Connection to ldaps://ip:1636 2025-03-03 06:35:49,254 - administrator@local - DEBUG - BASIC:instantiated Tls: <Tls(validate=<VerifyMode.CERT_NONE: 0>)> 2025-03-03 06:35:49,254 - administrator@local - DEBUG - BASIC:instantiated Server: <Server(host='ip', port=1636, use_ssl=True, allowed_referral_hosts=[('*', True)], tls=Tls(validate=<VerifyMode.CERT_NONE: 0>), get_info='SCHEMA', mode='IP_V6_PREFERRED')> 2025-03-03 06:35:49,254 - administrator@local - DEBUG - BASIC:instantiated <SyncStrategy>: <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - unbound - closed - <no socket> - tls not started - not listening - No strategy - internal decoder - async - real DSA - not pooled - cannot stream output> 2025-03-03 06:35:49,254 - administrator@local - DEBUG - BASIC:instantiated Connection: <Connection(server=Server(host='ip', port=1636, use_ssl=True, allowed_referral_hosts=[('*', True)], tls=Tls(validate=<VerifyMode.CERT_NONE: 0>), get_info='SCHEMA', mode='IP_V6_PREFERRED'), user='cn=Directory Manager', password='<stripped 15 characters of sensitive data>', auto_bind='DEFAULT', version=3, authentication='SIMPLE', client_strategy='SYNC', auto_referrals=True, check_names=True, read_only=False, lazy=False, raise_exceptions=False, fast_decoder=True, auto_range=True, return_empty_attributes=True, auto_encode=True, auto_escape=True, use_referral_cache=False)> 2025-03-03 06:35:49,255 - administrator@local - DEBUG - BASIC:start BIND operation via <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - unbound - closed - <no socket> - tls not started - not listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,255 - administrator@local - DEBUG - BASIC:address for <ldaps://ip:1636 - ssl> resolved as <[<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('ip', 1636)]> 2025-03-03 06:35:49,255 - administrator@local - DEBUG - BASIC:obtained candidate address for <ldaps://ip:1636 - ssl>: <[<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('ip', 1636)]> with mode IP_V6_PREFERRED 2025-03-03 06:35:49,255 - administrator@local - DEBUG - BASIC:try to open candidate address [<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('ip', 1636)] 2025-03-03 06:35:49,281 - administrator@local - DEBUG - BASIC:refreshing server info for <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - bound - open - <local: ip.17:52419 - remote: ip:1636> - tls not started - listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,281 - administrator@local - DEBUG - BASIC:start SEARCH operation via <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - bound - open - <local: ip.17:52419 - remote: ip:1636> - tls not started - listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,283 - administrator@local - DEBUG - BASIC:done SEARCH operation, result <True> 2025-03-03 06:35:49,284 - administrator@local - DEBUG - BASIC:start SEARCH operation via <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - bound - open - <local: ip.17:52419 - remote: ip:1636> - tls not started - listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,337 - administrator@local - DEBUG - BASIC:done SEARCH operation, result <True> 2025-03-03 06:35:49,423 - administrator@local - DEBUG - BASIC:schema read for <ldaps://ip:1636 - ssl> via <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - bound - open - <local: ip.17:52419 - remote: ip:1636> - tls not started - listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,423 - administrator@local - DEBUG - BASIC:done BIND operation, result <True> 2025-03-03 06:35:49,423 - administrator@local - DEBUG - BASIC:start SEARCH operation via <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - bound - open - <local: ip.17:52419 - remote: ip:1636> - tls not started - listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,426 - administrator@local - DEBUG - BASIC:done SEARCH operation, result <True> 2025-03-03 06:35:49,426 - administrator@local - DEBUG - BASIC:start UNBIND operation via <ldaps://ip:1636 - ssl - user: cn=Directory Manager - not lazy - bound - open - <local: ip.17:52419 - remote: ip:1636> - tls not started - listening - SyncStrategy - internal decoder> 2025-03-03 06:35:49,427 - administrator@local - DEBUG - BASIC:done UNBIND operation, result <True> 2025-03-03 06:35:49,428 - administrator@local - INFO - [2025-Mar-03 06:35] ip.40.100 GET http /keyrotation/? 200 OK ```