Have you considered adding self-signed SSL certificates to your OJS servers? Without SSL for the last mile, your network traffic is vulnerable between the proxy and your OJS server. Self-signed certificates would resolve that vulnerability, and resolve this issue as well.
Alternately, I’m curious of any debugging you discover on this, as I have very mixed feelings about the base_urls and auto detection of such.