Gluu community edition v1.0.2

By: Elizabeth Leong user 21 Oct 2020 at 1:14 p.m. CDT

8 Responses
Elizabeth Leong gravatar
I'm running GLUU Community edition v1.0.2 on an ubuntu 18.04 machine. `pygluu-compose.yml up` runs successfully, however during the final "Launching..." step, nginx shows many connection failures until it finally succeeds. Attaching to the nginx container shows that it tries to connect unsuccessfully several times until it eventually passes. Why is it unsuccessful in connecting? Is there a way to speed this up? Here's the nginx log that I get while running pygluu-compose up 2020/10/21 16:44:31.189500 [INFO] (child) reloading process 10.1.4.49 - - [21/Oct/2020:16:44:36 +0000] "GET / HTTP/1.1" 301 162 "-" "python-requests/2.23.0" "-" 2020/10/21 16:44:36 [error] 32#32: *23 connect() failed (111: Connection refused) while connecting to upstream, client: 10.1.4.49, server: gluu.rapiscansystems.com, request: "GET /identity HTTP/1.1", upstream: "http://127.0.0.1:8082/identity", host: "10.1.4.49" 10.1.4.49 - - [21/Oct/2020:16:44:36 +0000] "GET /identity HTTP/1.1" 502 22307 "-" "python-requests/2.23.0" "-" 10.1.4.49 - - [21/Oct/2020:16:44:46 +0000] "GET / HTTP/1.1" 301 162 "-" "python-requests/2.23.0" "-" 2020/10/21 16:44:46 [error] 32#32: *25 connect() failed (111: Connection refused) while connecting to upstream, client: 10.1.4.49, server: gluu.rapiscansystems.com, request: "GET /identity HTTP/1.1", upstream: "http://127.0.0.1:8082/identity", host: "10.1.4.49" 10.1.4.49 - - [21/Oct/2020:16:44:46 +0000] "GET /identity HTTP/1.1" 502 22307 "-" "python-requests/2.23.0" "-" 10.1.4.49 - - [21/Oct/2020:16:44:56 +0000] "GET / HTTP/1.1" 301 162 "-" "python-requests/2.23.0" "-" 10.1.4.49 - - [21/Oct/2020:16:44:56 +0000] "GET /identity HTTP/1.1" 502 22307 "-" "python-requests/2.23.0" "-" 2020/10/21 16:44:56 [error] 32#32: *27 connect() failed (111: Connection refused) while connecting to upstream, client: 10.1.4.49, server: gluu.rapiscansystems.com, request: "GET /identity HTTP/1.1", upstream: "http://127.0.0.1:8082/identity", host: "10.1.4.49" 10.1.4.49 - - [21/Oct/2020:16:45:06 +0000] "GET / HTTP/1.1" 301 162 "-" "python-requests/2.23.0" "-" 2020/10/21 16:45:06 [error] 32#32: *29 connect() failed (111: Connection refused) while connecting to upstream, client: 10.1.4.49, server: gluu.rapiscansystems.com, request: "GET /identity HTTP/1.1", upstream: "http://127.0.0.1:8082/identity", host: "10.1.4.49" 10.1.4.49 - - [21/Oct/2020:16:45:06 +0000] "GET /identity HTTP/1.1" 502 22307 "-" "python-requests/2.23.0" "-" 10.1.7.24 - - [21/Oct/2020:16:45:10 +0000] "GET /index.html HTTP/1.1" 301 162 "-" "Java/1.8.0_212" "-" 2020/10/21 16:45:10 [error] 32#32: *32 connect() failed (111: Connection refused) while connecting to upstream, client: 10.1.7.24, server: gluu.rapiscansystems.com, request: "GET /identity HTTP/1.1", upstream: "http://127.0.0.1:8082/identity", host: "gluu.rapiscansystems.com" 10.1.7.24 - - [21/Oct/2020:16:45:10 +0000] "GET /identity HTTP/1.1" 502 22307 "-" "Java/1.8.0_212" "-" 10.1.4.49 - - [21/Oct/2020:16:45:16 +0000] "GET / HTTP/1.1" 301 162 "-" "python-requests/2.23.0" "-" 2020/10/21 16:45:16 [error] 32#32: *35 connect() failed (111: Connection refused) while connecting to upstream, client: 10.1.4.49, server: gluu.rapiscansystems.com, request: "GET /identity HTTP/1.1", upstream: "http://127.0.0.1:8082/identity", host: "10.1.4.49" 10.1.4.49 - - [21/Oct/2020:16:45:16 +0000] "GET /identity HTTP/1.1" 502 22307 "-" "python-requests/2.23.0" "-" 10.1.4.49 - - [21/Oct/2020:16:45:26 +0000] "GET / HTTP/1.1" 301 162 "-" "python-requests/2.23.0" "-" 10.1.4.49 - - [21/Oct/2020:16:45:26 +0000] "GET /identity HTTP/1.1" 502 22307 "-" "python-requests/2.23.0" "-" 2020/10/21 16:45:26 [error] 32#32: *37 connect() failed (111: Connection refused) while connecting to upstream, client: 10.1.4.49, server: gluu.rapiscansystems.com, request: "GET /identity HTTP/1.1", upstream: "http://127.0.0.1:8082/identity", host: "10.1.4.49" 10.1.4.49 - - [21/Oct/2020:16:45:36 +0000] "GET / HTTP/1.1" 301 162 "-" "python-requests/2.23.0" "-" 2020/10/21 16:45:36 [error] 32#32: *39 connect() failed (111: Connection refused) while connecting to upstream, client: 10.1.4.49, server: gluu.rapiscansystems.com, request: "GET /identity HTTP/1.1", upstream: "http://127.0.0.1:8082/identity", host: "10.1.4.49" 10.1.4.49 - - [21/Oct/2020:16:45:36 +0000] "GET /identity HTTP/1.1" 502 22307 "-" "python-requests/2.23.0" "-" 10.1.4.49 - - [21/Oct/2020:16:45:46 +0000] "GET / HTTP/1.1" 301 162 "-" "python-requests/2.23.0" "-" 2020/10/21 16:45:46 [error] 32#32: *41 connect() failed (111: Connection refused) while connecting to upstream, client: 10.1.4.49, server: gluu.rapiscansystems.com, request: "GET /identity HTTP/1.1", upstream: "http://127.0.0.1:8082/identity", host: "10.1.4.49" 10.1.4.49 - - [21/Oct/2020:16:45:46 +0000] "GET /identity HTTP/1.1" 502 22307 "-" "python-requests/2.23.0" "-" 2020/10/21 16:45:50.870040 [INFO] (runner) rendered "/app/templates/gluu_https.conf.ctmpl" => "/etc/nginx/conf.d/default.conf" 2020/10/21 16:45:50.870079 [INFO] (child) reloading process 10.1.4.49 - - [21/Oct/2020:16:45:56 +0000] "GET / HTTP/1.1" 301 162 "-" "python-requests/2.23.0" "-" 10.1.4.49 - - [21/Oct/2020:16:45:56 +0000] "GET /identity HTTP/1.1" 302 5 "-" "python-requests/2.23.0" "-"
Gluu Other Ubuntu 18.04
closed

Answers

By Isman Firmansyah staff 21 Oct 2020 at 3:06 p.m. CDT

Copy
Isman Firmansyah gravatar
Hi Elizabeth, > Why is it unsuccessful in connecting? By default pygluu-compose runs all containers at the same time. You're seeing unsuccesful connection because at the time pygluu-compose deploys all containers, the service in question (http://127.0.0.1:8082/identity) is not ready to accept traffic. Eventually the service will be ready and starts accepting traffic. This is the design of our containers healthcheck (readiness and liveness) probe. > Is there a way to speed this up? Yes there is. You can run the containers one by one, but you need to know how these containers are linked. For reference you can refer to https://docs.docker.com/compose/startup-order/.

By Elizabeth Leong user 22 Oct 2020 at 4:58 p.m. CDT

Copy
Elizabeth Leong gravatar
Thanks for the reply. I read the links you sent, and I wonder if I can add the depends_on ordering and wait-for-it instead of starting containers individually. From pygluu-compose's log, it seems that nginx waits for oxTrust server (internal port 8082), which waits for oxAuth, which waits for ldap. ldap is very slow to be totally up. I changed the following 1) pygluu-compose/pygluu/compose/templates/svc.oxauth.yml ``` depends_on: - ldap ``` 2) pygluu-compose/pygluu/compose/templates/svc.oxtrust.yml ``` depends_on: - oxauth ``` How do I make it wait for http://127.0.0.1:8082/identity to be ready ? I tried this: copy wait-for-it.sh from your link and added: command: [ "wait-for-it.sh", "127.0.0.1:8082"] But that doesn't seem right, how do I wait for `identity` ?

By Isman Firmansyah staff 22 Oct 2020 at 6:15 p.m. CDT

Copy
Isman Firmansyah gravatar
Hi Elizabeth, Adding `depends_on` only waits for container being ready (docker internal stuff). Even when the containers ready, they're likely crashed due to external dependencies, i.e. oxAuth needs entries in LDAP. For your info, we designed Gluu containers to have a safe sequence of startup orders for various deployment scenarios, from custom scripts, docker compose, k8s manifests, etc. I would recommend you to follow our Gluu containers design. In other words, applying custom startup orders (i.e. wait-for-it script) need deeper knowledge of our design and docker container in general.

By Elizabeth Leong user 23 Oct 2020 at 5:50 p.m. CDT

Copy
Elizabeth Leong gravatar
I will not attempt to start the containers one by one. Maybe I can achieve faster startup by tuning it. In this document https://gluu.org/docs/gluu-server/4.0/operation/fine-tuning/ under the LDAP section, where can I find/set max-allowed-client-connections? Thank you.

By Isman Firmansyah staff 25 Oct 2020 at 11:49 p.m. CDT

Copy
Isman Firmansyah gravatar
The `max-allowed-client-connections` is OpenDJ global configuration. You can find it using `/opt/opendj/bin/dsconfig`. By default `max-allowed-client-connections` is set to `0` which mean it allows unlimited client connections. Out of curiosity, what requirements you need for your setup? Have you tried using pygluu-compose v1.2.5 and Gluu Server 4.2? Deploying Gluu Server using pygluu-compose v1.2.5 is pretty fast (average time 10-15 mins). The slow process is mostly on generating data for OpenDJ (this only occurs on first deployment of containers though). For your info, if you need custom deployment you may want to consider our Kubernetes distribution and support contract.

By Elizabeth Leong user 26 Oct 2020 at 10:44 a.m. CDT

Copy
Elizabeth Leong gravatar
Thanks Isman. The system we're building has a hard requirement to be entirely up and running in 10 minutes on machine bootup. Right now GLUU is the slowest component, thus I'm trying to find various ways to improve its startup time.

By Isman Firmansyah staff 28 Oct 2020 at 3:34 p.m. CDT

Copy
Isman Firmansyah gravatar
> The system we're building has a hard requirement to be entirely up and running in 10 minutes on machine bootup. Interesting. > Right now GLUU is the slowest component, thus I'm trying to find various ways to improve its startup time. Unfortunately the speed of deployment affected by many variables, i.e. package loading, external network call to Docker hub, the infrastructure used in deployment, etc. hence the result may vary. We will try to improve the deployment time in the future as currently we're improving in other areas. Thanks.

By Isman Firmansyah staff 04 Nov 2020 at 11:51 a.m. CST

Copy
Isman Firmansyah gravatar
Please feel free to re-open the ticket if required.

Post an answer