>> Enter IP Address [172.20.9.30] : localhost.localdomain ERROR: The IP Address is invalid. Try again
This might be the problem; setup script asked if this is the correct IP address or not but you provided hostname.
And btw...don't use 'localhost' / 'localdomain' for your Gluu Server's hostname. localhost can't serve https request.
Also... better not to use 'setup.properties.last' file atm; it's in QA cycle. Grab a fresh VM and run installation from scratch.