Hi, Philip.
As George mentioned above, you need to remove old Apache certificate (the self-signed one) from Java's truststore inside container, and add your new certificate there instead, under the same alias. You'll have to repeat this procedure manually if at any point certbot will update your letsencrypt certificate.
Have you already tried this?