By: Bjoern Schenk user 06 Jul 2017 at 8:46 a.m. CDT

5 Responses
Bjoern Schenk gravatar
Hello, I've had this problem with release 3.0.1 and now also with 3.0.2. After service gluu xxx stop, and subsequently starting the appliance via service gluu start, I am presented with a http 503 Error when trying to log into the appliance. The steps in this thread https://support.gluu.org/outages/3895/503-service-unavailable/ , basically starting and stopping the services in the right order, helped me to get it running again for now. However I gather that this should have been fixed in 3.0.2, which it seems it did not. (I am on a Hyper-V Hosted VM with 4 VCPU and 6 GiB of Ram)

By Michael Schwartz Account Admin 06 Jul 2017 at 8:59 a.m. CDT

Michael Schwartz gravatar
2 CPU Units? What OS are you using? Did you check the logs for apache http, ldap, oxauth and oxtrust? Please attach relevant clues.

By Bjoern Schenk user 06 Jul 2017 at 9:36 a.m. CDT

Bjoern Schenk gravatar
4 CPU Units since I had the same error on 3.0.1 and I read a post where the issue was an insufficient VCPU count. I had it originally at two, but since I have ample resources on my host I have upped this to 4. OS: Ubuntu 16.04 lts Interesting to note is that I seem to recall it always working after fresh install and failing after the first "restart" through gluu service stop-> gluu service start oxauth ``` 2017-07-06 14:21:40,529 ERROR [main] [org.xdi.oxauth.model.config.ConfigurationFactory] (ConfigurationFactory.java:285) - Failed to load configuration from file: /etc/gluu/conf/oxauth-config.json 2017-07-06 14:21:40,530 ERROR [main] [org.xdi.oxauth.model.config.ConfigurationFactory] (ConfigurationFactory.java:130) - Failed to load configuration from LDAP. Please fix it!!!. ``` OxTrust doesn't seem to log anything fresh ldap log ``` Jul 6 14:19:53 idp slapd[4017]: conn=1005 op=365 SRCH attr=gluuApplianceDnsServer oxAuthenticationMode blowfishPassword oxTrustCacheRefreshServerIpAddress oxTrustEmail c description displayName gluuFreeDiskSpace gluuFreeMemory gluuFreeSwap gluuBandwidthRX gluuBandwidthTX gluuDSstatus gluuHTTPstatus gluuSPTR gluuVDSstatus gluuGroupCount gluuHostname iname inum inumFN gluuIpAddress gluuLastUpdate gluuLoadAvg gluuManageIdentityPermission gluuMaxLogSize o oxIDPAuthentication oxLogViewerConfig oxTrustAuthenticationMode gluuPassportEnabled passwordResetAllowed gluuPersonCount gluuAppliancePollingInterval gluuOrgProfileMgt gluuScimEnabled gluuShibAssertionsIssued gluuShibFailedAuth gluuShibSecurityEvents gluuShibSuccessfulAuths oxSmtpConfiguration gluuSmtpFromEmailAddress gluuSmtpFromName gluuSmtpHost gluuSmtpPassword gluuSmtpPort gluuSmtpRequiresAuthentication gluuSmtpRequiresSsl gluuSmtpUserName gluuSslExpiry gluuStatus gluuSystemUptime oxTrustStoreCert oxTrustStoreConf userPassword gluuVdsCacheRefreshEnabled gluuVdsCacheRefreshLastUpdate gluuVdsCacheRefreshLastUpdateCount gluuVdsCacheRefreshPollingInterval gluuVdsCacheRefreshProblemCount gluuWhitePagesEnabled Jul 6 14:19:53 idp slapd[4017]: conn=1005 op=365 SEARCH RESULT tag=101 err=0 duration=0.166ms nentries=1 text= Jul 6 14:19:53 idp slapd[4017]: conn=1004 op=405 SRCH base="inum=@!9931.BE5F.1F67.504F!0002!3135.1E56,ou=appliances,o=gluu" scope=0 deref=0 filter="(objectClass=*)" Jul 6 14:19:53 idp slapd[4017]: conn=1004 op=405 SRCH attr=gluuApplianceDnsServer oxAuthenticationMode blowfishPassword oxTrustCacheRefreshServerIpAddress oxTrustEmail c description displayName gluuFreeDiskSpace gluuFreeMemory gluuFreeSwap gluuBandwidthRX gluuBandwidthTX gluuDSstatus gluuHTTPstatus gluuSPTR gluuVDSstatus gluuGroupCount gluuHostname iname inum inumFN gluuIpAddress gluuLastUpdate gluuLoadAvg gluuManageIdentityPermission gluuMaxLogSize o oxIDPAuthentication oxLogViewerConfig oxTrustAuthenticationMode gluuPassportEnabled passwordResetAllowed gluuPersonCount gluuAppliancePollingInterval gluuOrgProfileMgt gluuScimEnabled gluuShibAssertionsIssued gluuShibFailedAuth gluuShibSecurityEvents gluuShibSuccessfulAuths oxSmtpConfiguration gluuSmtpFromEmailAddress gluuSmtpFromName gluuSmtpHost gluuSmtpPassword gluuSmtpPort gluuSmtpRequiresAuthentication gluuSmtpRequiresSsl gluuSmtpUserName gluuSslExpiry gluuStatus gluuSystemUptime oxTrustStoreCert oxTrustStoreConf userPassword gluuVdsCacheRefreshEnabled gluuVdsCacheRefreshLastUpdate gluuVdsCacheRefreshLastUpdateCount gluuVdsCacheRefreshPollingInterval gluuVdsCacheRefreshProblemCount gluuWhitePagesEnabled objectClass Jul 6 14:19:53 idp slapd[4017]: conn=1004 op=405 SEARCH RESULT tag=101 err=0 duration=0.366ms nentries=1 text= Jul 6 14:19:53 idp slapd[4017]: conn=1005 op=366 MOD dn="inum=@!9931.BE5F.1F67.504F!0002!3135.1E56,ou=appliances,o=gluu" Jul 6 14:19:53 idp slapd[4017]: conn=1005 op=366 MOD attr=gluuFreeMemory gluuLastUpdate gluuSystemUptime Jul 6 14:19:53 idp slapd[4017]: slap_queue_csn: queueing 0x7f29ec125aa0 20170706141953.588131Z#000000#000#000000 Jul 6 14:19:53 idp slapd[4017]: conn=1005 op=366 RESULT tag=103 err=0 duration=5.266ms text= Jul 6 14:19:53 idp slapd[4017]: slap_graduate_commit_csn: removing 0x7f29ec125aa0 20170706141953.588131Z#000000#000#000000 Jul 6 14:19:53 idp slapd[4017]: conn=1004 op=406 SRCH base="inum=@!9931.BE5F.1F67.504F!0002!3135.1E56,ou=appliances,o=gluu" scope=0 deref=0 filter="(objectClass=*)" Jul 6 14:19:53 idp slapd[4017]: conn=1004 op=406 SRCH attr=gluuApplianceDnsServer oxAuthenticationMode blowfishPassword oxTrustCacheRefreshServerIpAddress oxTrustEmail c description displayName gluuFreeDiskSpace gluuFreeMemory gluuFreeSwap gluuBandwidthRX gluuBandwidthTX gluuDSstatus gluuHTTPstatus gluuSPTR gluuVDSstatus gluuGroupCount gluuHostname iname inum inumFN gluuIpAddress gluuLastUpdate gluuLoadAvg gluuManageIdentityPermission gluuMaxLogSize o oxIDPAuthentication oxLogViewerConfig oxTrustAuthenticationMode gluuPassportEnabled passwordResetAllowed gluuPersonCount gluuAppliancePollingInterval gluuOrgProfileMgt gluuScimEnabled gluuShibAssertionsIssued gluuShibFailedAuth gluuShibSecurityEvents gluuShibSuccessfulAuths oxSmtpConfiguration gluuSmtpFromEmailAddress gluuSmtpFromName gluuSmtpHost gluuSmtpPassword gluuSmtpPort gluuSmtpRequiresAuthentication gluuSmtpRequiresSsl gluuSmtpUserName gluuSslExpiry gluuStatus gluuSystemUptime oxTrustStoreCert oxTrustStoreConf userPassword gluuVdsCacheRefreshEnabled gluuVdsCacheRefreshLastUpdate gluuVdsCacheRefreshLastUpdateCount gluuVdsCacheRefreshPollingInterval gluuVdsCacheRefreshProblemCount gluuWhitePagesEnabled Jul 6 14:19:53 idp slapd[4017]: conn=1004 op=406 SEARCH RESULT tag=101 err=0 duration=0.157ms nentries=1 text= Jul 6 14:20:01 idp slapd[4017]: conn=1006 op=83 SRCH base="inum=@!9931.BE5F.1F67.504F!0002!3135.1E56,ou=appliances,o=gluu" scope=0 deref=0 filter="(objectClass=*)" Jul 6 14:20:01 idp slapd[4017]: conn=1006 op=83 SRCH attr=oxIDPAuthentication Jul 6 14:20:01 idp slapd[4017]: conn=1006 op=83 SEARCH RESULT tag=101 err=0 duration=0.312ms nentries=1 text= Jul 6 14:20:01 idp slapd[4017]: conn=1007 op=84 SRCH base="inum=@!9931.BE5F.1F67.504F!0002!3135.1E56,ou=appliances,o=gluu" scope=0 deref=0 filter="(objectClass=*)" Jul 6 14:20:01 idp slapd[4017]: conn=1007 op=84 SRCH attr=oxAuthenticationMode Jul 6 14:20:01 idp slapd[4017]: conn=1007 op=84 SEARCH RESULT tag=101 err=0 duration=0.221ms nentries=1 text= Jul 6 14:20:01 idp slapd[4017]: conn=1000 op=275 SRCH base="ou=oxauth,ou=configuration,inum=@!9931.BE5F.1F67.504F!0002!3135.1E56,ou=appliances,o=gluu" scope=0 deref=0 filter="(objectClass=*)" Jul 6 14:20:01 idp slapd[4017]: conn=1000 op=275 SRCH attr=oxRevision Jul 6 14:20:01 idp slapd[4017]: conn=1000 op=275 SEARCH RESULT tag=101 err=0 duration=0.114ms nentries=1 text= Jul 6 14:20:01 idp slapd[4017]: conn=1008 op=86 SRCH base="ou=scripts,o=@!9931.BE5F.1F67.504F!0001!90C7.C662,o=gluu" scope=2 deref=0 filter="(&(&(objectClass=top)(objectClass=oxCustomScript))(|(oxScriptType=person_authentication)(oxScriptType=client_registration)(oxScriptType=id_generator)(oxScriptType=uma_authorization_policy)(oxScriptType=application_session)(oxScriptType=dynamic_scope)))" Jul 6 14:20:01 idp slapd[4017]: conn=1008 op=86 SRCH attr=dn inum oxRevision oxScriptType oxModuleProperty gluuStatus Jul 6 14:20:01 idp slapd[4017]: conn=1008 op=86 SEARCH RESULT tag=101 err=0 duration=0.559ms nentries=21 text= Jul 6 14:20:23 idp slapd[4017]: conn=1004 op=407 SRCH base="ou=oxtrust,ou=configuration,inum=@!9931.BE5F.1F67.504F!0002!3135.1E56,ou=appliances,o=gluu" scope=0 deref=0 filter="(objectClass=*)" Jul 6 14:20:23 idp slapd[4017]: conn=1004 op=407 SRCH attr=oxRevision Jul 6 14:20:23 idp slapd[4017]: conn=1004 op=407 SEARCH RESULT tag=101 err=0 duration=0.293ms nentries=1 text= Jul 6 14:20:23 idp slapd[4017]: conn=1005 op=367 SRCH base="ou=scripts,o=@!9931.BE5F.1F67.504F!0001!90C7.C662,o=gluu" scope=2 deref=0 filter="(&(&(objectClass=top)(objectClass=oxCustomScript))(|(oxScriptType=cache_refresh)(oxScriptType=update_user)(oxScriptType=user_registration)(oxScriptType=id_generator)(oxScriptType=scim)))" Jul 6 14:20:23 idp slapd[4017]: conn=1005 op=367 SRCH attr=dn inum oxRevision oxScriptType oxModuleProperty gluuStatus Jul 6 14:20:23 idp slapd[4017]: conn=1005 op=367 SEARCH RESULT tag=101 err=0 duration=0.365ms nentries=5 text= Jul 6 14:20:28 idp slapd[4017]: conn=1005 fd=19 closed (connection lost) Jul 6 14:20:28 idp slapd[4017]: conn=1004 fd=18 closed (connection lost) Jul 6 14:20:29 idp slapd[4017]: conn=1006 fd=20 closed (connection lost) Jul 6 14:20:29 idp slapd[4017]: conn=1008 fd=22 closed (connection lost) Jul 6 14:20:29 idp slapd[4017]: conn=1007 fd=21 closed (connection lost) Jul 6 14:20:29 idp slapd[4017]: conn=1002 fd=16 closed (connection lost) Jul 6 14:20:29 idp slapd[4017]: conn=1003 fd=17 closed (connection lost) Jul 6 14:20:29 idp slapd[4017]: conn=1001 fd=15 closed (connection lost) Jul 6 14:20:29 idp slapd[4017]: conn=1000 fd=9 closed (connection lost) Jul 6 14:21:39 idp slapd[8570]: @(#) $OpenLDAP: slapd 2.4.44 (Oct 20 2016 14:08:31) $#012#011root@NY1SCOLUB14BL01:/home/build/sold-2.4.44.20161020/build/openldap.amd64/servers/slapd Jul 6 14:21:39 idp slapd[8571]: slapd starting Jul 6 14:21:40 idp slapd[8571]: conn=1000 fd=9 ACCEPT from IP=127.0.0.1:45288 (IP=127.0.0.1:1636) Jul 6 14:21:40 idp slapd[8571]: conn=1000 fd=9 TLS established tls_ssf=256 ssf=256 Jul 6 14:21:40 idp slapd[8571]: conn=1000 op=0 BIND dn="" method=128 Jul 6 14:21:40 idp slapd[8571]: conn=1000 op=0 RESULT tag=97 err=0 duration=0.065ms text= ``` apache ``` [Thu Jul 06 06:28:25.785593 2017] [mpm_event:notice] [pid 3313:tid 140103752902528] AH00489: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations [Thu Jul 06 06:28:25.785605 2017] [core:notice] [pid 3313:tid 140103752902528] AH00094: Command line: '/usr/sbin/apache2' [Thu Jul 06 10:16:42.459451 2017] [mpm_event:notice] [pid 3313:tid 140103752902528] AH00491: caught SIGTERM, shutting down [Thu Jul 06 10:17:56.154753 2017] [mpm_event:notice] [pid 3643:tid 139948042131328] AH00489: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations [Thu Jul 06 10:17:56.154840 2017] [core:notice] [pid 3643:tid 139948042131328] AH00094: Command line: '/usr/sbin/apache2' [Thu Jul 06 10:18:06.022555 2017] [proxy:error] [pid 3646:tid 139947934086912] (111)Connection refused: AH00957: HTTP: attempt to connect to 127.0.0.1:8082 (localhost) failed [Thu Jul 06 10:18:06.022615 2017] [proxy:error] [pid 3646:tid 139947934086912] AH00959: ap_proxy_connect_backend disabling worker for (localhost) for 5s [Thu Jul 06 10:18:06.022623 2017] [proxy_http:error] [pid 3646:tid 139947934086912] [client 134.155.46.69:51322] AH01114: HTTP: failed to make connection to backend: localhost, referer: https://idp.mbs.education/identity/trustmanager/relationships [Thu Jul 06 10:18:12.843117 2017] [proxy:error] [pid 3647:tid 139947917301504] (111)Connection refused: AH00957: HTTP: attempt to connect to 127.0.0.1:8082 (localhost) failed [Thu Jul 06 10:18:12.843136 2017] [proxy:error] [pid 3647:tid 139947917301504] AH00959: ap_proxy_connect_backend disabling worker for (localhost) for 5s [Thu Jul 06 10:18:12.843143 2017] [proxy_http:error] [pid 3647:tid 139947917301504] [client 134.155.46.69:51330] AH01114: HTTP: failed to make connection to backend: localhost [Thu Jul 06 10:18:35.394830 2017] [proxy:error] [pid 3647:tid 139947793172224] (111)Connection refused: AH00957: HTTP: attempt to connect to 127.0.0.1:8082 (localhost) failed [Thu Jul 06 10:18:35.394860 2017] [proxy:error] [pid 3647:tid 139947793172224] AH00959: ap_proxy_connect_backend disabling worker for (localhost) for 5s [Thu Jul 06 10:18:35.394864 2017] [proxy_http:error] [pid 3647:tid 139947793172224] [client 134.155.46.69:51336] AH01114: HTTP: failed to make connection to backend: localhost [Thu Jul 06 10:18:36.639200 2017] [proxy:error] [pid 3647:tid 139947751208704] AH00940: HTTP: disabled connection for (localhost) [Thu Jul 06 11:03:22.483816 2017] [proxy:error] [pid 3647:tid 139947734423296] (111)Connection refused: AH00957: HTTP: attempt to connect to 127.0.0.1:8082 (localhost) failed [Thu Jul 06 11:03:22.483839 2017] [proxy:error] [pid 3647:tid 139947734423296] AH00959: ap_proxy_connect_backend disabling worker for (localhost) for 5s [Thu Jul 06 11:03:22.483844 2017] [proxy_http:error] [pid 3647:tid 139947734423296] [client 139.162.113.204:38346] AH01114: HTTP: failed to make connection to backend: localhost, referer: https://134.155.46.4:443/ [Thu Jul 06 13:21:28.377432 2017] [proxy:error] [pid 3647:tid 139947675674368] (111)Connection refused: AH00957: HTTP: attempt to connect to 127.0.0.1:8082 (localhost) failed [Thu Jul 06 13:21:28.377450 2017] [proxy:error] [pid 3647:tid 139947675674368] AH00959: ap_proxy_connect_backend disabling worker for (localhost) for 5s [Thu Jul 06 13:21:28.377455 2017] [proxy_http:error] [pid 3647:tid 139947675674368] [client 134.155.46.69:53480] AH01114: HTTP: failed to make connection to backend: localhost [Thu Jul 06 14:20:25.881840 2017] [mpm_event:notice] [pid 3643:tid 139948042131328] AH00491: caught SIGTERM, shutting down [Thu Jul 06 14:21:38.930627 2017] [mpm_event:notice] [pid 8486:tid 139664816613248] AH00489: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations [Thu Jul 06 14:21:38.930704 2017] [core:notice] [pid 8486:tid 139664816613248] AH00094: Command line: '/usr/sbin/apache2' [Thu Jul 06 14:21:46.114376 2017] [proxy:error] [pid 8490:tid 139664610551552] (111)Connection refused: AH00957: HTTP: attempt to connect to 127.0.0.1:8082 (localhost) failed [Thu Jul 06 14:21:46.114396 2017] [proxy:error] [pid 8490:tid 139664610551552] AH00959: ap_proxy_connect_backend disabling worker for (localhost) for 5s [Thu Jul 06 14:21:46.114407 2017] [proxy_http:error] [pid 8490:tid 139664610551552] [client 134.155.46.69:54591] AH01114: HTTP: failed to make connection to backend: localhost ``` Let me know if you need a longer excerpt of the logs

By Aliaksandr Samuseu staff 06 Jul 2017 at 9:58 a.m. CDT

Aliaksandr Samuseu gravatar
Hi, Bjorn (sorry if I misspelled it, seems like there is some charset issue) Correct me, but you are using Debian 8.2, right? Or was it a different issue? Could you reference you previous ticket on this, if any?

By Aliaksandr Samuseu staff 06 Jul 2017 at 10:09 a.m. CDT

Aliaksandr Samuseu gravatar
For Debian [corresponding report](https://github.com/GluuFederation/community-edition-setup/issues/304) is marked as "Closed", so it should be fixed.

By Bjoern Schenk user 06 Jul 2017 at 10:51 a.m. CDT

Bjoern Schenk gravatar
It's bjoern if you dont have the german umlaut for it on your keybourd, but it's totally fine. I'm Using Ubuntu 16.04 LTS, and it wasn't my previous ticket, but rahter a ticket where someone else had the same problem. Upon using the workaound in the first post of this ticket: https://support.gluu.org/outages/3895/503-service-unavailable/ I am able to login to the server again. And as far as I understood it the problem in the referenced ticket should have been caused by a bug in 3.0.1 which should have been fixed in the latest appliance version, on which I am experiencing the issue again (as I had it on 3.0.1 as well) Let me just note that I'm in preproduction so this is not a timesensitive issue (posted this in outages because it seemed the right forum)