Good news, here is the solution to my problem.
Our webserver is running Red Hat Enterprise Linux 8.4 with SELinux enabled.
SE-linux was blocking php from outbound connections on port 80, and the solution in my case was to allow this traffic with the following command used to configure a running SELinux:
setsebool -P httpd_can_network_connect 1
This change worked immediately without having to restart any services.
After this change I can access the part of the administration interface in OJS which needs web access on port 80 such as Setting > Journal and Setting > Website > Plugins > Plugin Gallery, as well as Administration.
I discovered this by displaying some system messages in realtime, which also had the solution:
journalctl -f
Some relevant output while trying to access Settings > Journal in the backend with a browser:
SELinux is preventing php-fpm from name_connect access on the tcp_socket port 80. For complete SELinux messages run: sealert -l 8e0527b7-c7ee-4f53-8f07-aa85a38b9513
Oct 14 13:53:53 (edited) setroubleshoot[2501589]: SELinux is preventing php-fpm from name_connect access on the tcp_socket port 80.
***** Plugin catchall_boolean (24.7 confidence) suggests ******************
If you want to allow httpd to can network connect
Then you must tell SELinux about this by enabling the 'httpd_can_network_connect' boolean.
Do
setsebool -P httpd_can_network_connect 1