SMTP worked in OJS 2.3.7 and failed in 3.1.1.2 on same server

blstzus · October 16, 2018, 12:09am

Hi,
We have two versions of OJS installed on the same server.
2.3.7 was restored from an old installation, whereas the 3.1.1.2 was installed fresh (i.e. no data etc).

The SMTP configurations (host and port) in config.inc.php for both versions are exactly the same.
When I tried sending email through OJS 2.3.7, it worked well, and I did receive the email.
But when I tried to email myself via OJS 3.1.1.2, it does not work and error log written SMTP connect() failed.

Any idea why is that the case? Any way of overcoming that?
Thanks.

-Aiman-

ctgraham · October 16, 2018, 1:43pm

We moved from an in-house SMTP handler to the more standard PHPmailer tool in 3.x, so there will be some subtle changes, especially in the config.inc.php settings.

Are you reusing the 2.x config.inc.php in your 3.x install, or have you modified a fresh copy of 3.x’s config.TEMPLATE.inc.php?

github.com

pkp/ojs/blob/5e844188ebeee5a3559312df781e250978da198e/config.TEMPLATE.inc.php#L301-L312


; Use SMTP for sending mail instead of mail()
; smtp = On

; SMTP server settings
; smtp_server = mail.example.com
; smtp_port = 25

; Enable SMTP authentication
; Supported mechanisms: ssl, tls
; smtp_auth = ssl
; smtp_username = username
; smtp_password = password

blstzus · October 17, 2018, 4:06am

Hi,
Thanks for the reply.
I use the fresh OJS3 config.inc.php file. Meaning I only uncomment and key in the necessary info to use SMTP (line 302, 305, 306).
the smtp_server and smtp_port information are exactly the same for both installations.

Any suggestion on what to do next?
Thanks.

ctgraham · October 17, 2018, 12:20pm

What was the value for smtp_auth in 2.x, and what is it now?

blstzus · October 17, 2018, 12:50pm

Both versions are left commented. It was never required.

ctgraham · October 17, 2018, 12:57pm

Hmmmm… I would think that should work as-is. Try turning On show_stacktrace in the debug section of config.inc.php, then retry sending a message. This will provide more context in the error log.

github.com

pkp/ojs/blob/bc5eb3c9dd69e729dbfcdcde978086d7650e3346/config.TEMPLATE.inc.php#L474



;;;;;;;;;;;;;;;;;;
; Debug Settings ;
;;;;;;;;;;;;;;;;;;

[debug]

; Display a stack trace when a fatal error occurs.
; Note that this may expose private information and should be disabled
; for any production system.
show_stacktrace = Off

; Display an error message when something goes wrong.
display_errors = Off

; Display deprecation warnings
deprecation_warnings = Off

; Log web service request information for debugging
log_web_service_info = Off

blstzus · October 18, 2018, 12:48am

Thanks for the reply @ctgraham.
Did that and got the following messages in apache2 error log.
I am really sorry, I don’t understand the meaning of the error message

[Thu Oct 18 08:43:11.784306 2018] [:error] [pid 26370] [client 172.16.17.176:56252] Connection: opening to 172.16.3.228:25, timeout=300, options=array (\n), referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.784929 2018] [:error] [pid 26370] [client 172.16.17.176:56252] Connection: opened, referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.785416 2018] [:error] [pid 26370] [client 172.16.17.176:56252] SERVER → CLIENT: 220 Mikail.student.uniten.local Microsoft ESMTP MAIL Service, Version: 7.5.7601.17514 ready at Thu, 18 Oct 2018 08:43:12 +0800 \r\n, referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.785456 2018] [:error] [pid 26370] [client 172.16.17.176:56252] CLIENT → SERVER: EHLO journal.uniten.edu.my\r\n, referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.786027 2018] [:error] [pid 26370] [client 172.16.17.176:56252] SERVER → CLIENT: 250-Mikail.student.uniten.local Hello [172.17.10.241]\r\n250-TURN\r\n250-SIZE\r\n250-ETRN\r\n250-PIPELINING\r\n250-DSN\r\n250-ENHANCEDSTATUSCODES\r\n250-8bitmime\r\n250-BINARYMIME\r\n250-CHUNKING\r\n250-VRFY\r\n250-TLS\r\n250-STARTTLS\r\n250 OK\r\n, referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.786072 2018] [:error] [pid 26370] [client 172.16.17.176:56252] CLIENT → SERVER: STARTTLS\r\n, referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.786629 2018] [:error] [pid 26370] [client 172.16.17.176:56252] SERVER → CLIENT: 220 2.0.0 SMTP server ready\r\n, referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.829836 2018] [:error] [pid 26370] [client 172.16.17.176:56252] Connection failed. Error #2: stream_socket_enable_crypto(): SSL operation failed with code 1. OpenSSL Error messages:\nerror:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed [/var/www/html/newojs/lib/pkp/lib/vendor/phpmailer/phpmailer/class.smtp.php line 375], referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.829975 2018] [:error] [pid 26370] [client 172.16.17.176:56252] SMTP Error: Could not connect to SMTP host., referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.830005 2018] [:error] [pid 26370] [client 172.16.17.176:56252] CLIENT → SERVER: QUIT\r\n, referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.830507 2018] [:error] [pid 26370] [client 172.16.17.176:56252] SERVER → CLIENT: , referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.830533 2018] [:error] [pid 26370] [client 172.16.17.176:56252] SMTP ERROR: QUIT command failed: , referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.830638 2018] [:error] [pid 26370] [client 172.16.17.176:56252] Connection: closed, referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.830712 2018] [:error] [pid 26370] [client 172.16.17.176:56252] SMTP connect() failed. Troubleshooting · PHPMailer/PHPMailer Wiki · GitHub, referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access
[Thu Oct 18 08:43:11.830733 2018] [:error] [pid 26370] [client 172.16.17.176:56252] SMTP connect() failed. Troubleshooting · PHPMailer/PHPMailer Wiki · GitHub, referer: http://journal.uniten.edu.my/newojs/index.php/ijecct/management/settings/access

ctgraham · October 18, 2018, 12:22pm

The critical message is here:
1416F086:SSL routines:tls_process_server_certificate:certificate verify failed

In the handshake, the mailserver requested a STARTTLS action and the client (your OJS server) responded by trying to STARTTLS. The client, however, does not have a certificate chain which allows it to trust the server’s certificate.

You’ll need to work with your system administrator to either:

Import the certificate chain for your mailserver into your OJS server, for use by OpenSSL.
Disable OpenSSL’s requirement to verify peer certificates (not recommended)

I suspect the difference between OJS2 and OJS3 is that when the mailserver requested a STARTTLS to secure the connection, OJS2 just blindly ignored the request.

Alternately, if you want to change the OJS code, you could tell the PHPMailer library to not use TLS at all (also not recommended).

blstzus · October 20, 2018, 1:10am

Thanks a lot @ctgraham. I will work with the admin regarding this matter.
Thanks again.

miki_farman · May 27, 2020, 2:33pm

@blstzus Hi, We just recently upgraded from 3.1.1 to 3.1.2 and found that the mail/notification delivery system isn’t working anymore. we didnt touch the config file. the point is that when we restore the old version everything works well but with new version we get STMP() error.
Did you find any solution for that?

asmecher · May 27, 2020, 3:56pm

Hi @miki_farman,

What specific SMTP error do you get?

Regards,
Alec Smecher
Public Knowledge Project Team

miki_farman · May 27, 2020, 8:10pm

Hi, @asmecher
SMTP connect() failed. Troubleshooting · PHPMailer/PHPMailer Wiki · GitHub
We use the same config file as we had in the old ojs version.

asmecher · May 29, 2020, 8:38pm

Hi @miki_farman,

The only relevant change I can think of between OJS 3.1.1-x and 3.1.2-x is that we upgraded the PHPMailer library we use from version 5.2.26 to 6.0.7. The OJS configuration requirements haven’t changed, so I think it’s probably something within the PHPMailer library that’s not agreeing with your server.

I’d suggest trying PHPMailer with OJS removed from the equation. Set up a test script along these lines and see if it will deliver mail successfully. These lines of OJS code are responsible for setting up the PHPMailer service, for contrast.

Regards,
Alec Smecher
Public Knowledge Project Team

miki_farman · June 2, 2020, 8:08am

@asmecher, Hi, I created a test script with the latest phpmailer as explained in the link. the message has been sent to my email, but sometimes it goes to inbox and sometimes to spam. Anyway phpmailer test script works with our SMTP. Now what is the next step? how have the ojs phpmailer running?

asmecher · June 2, 2020, 7:15pm

Hi @miki_farman,

Did you mirror the same configuration as the lines of code linked above set up when using OJS?

Regards,
Alec Smecher
Public Knowledge Project Team

miki_farman · June 3, 2020, 9:30am

@asmecher Hi, I tried many things and Now forgot what I have done and why now ojs sending email. but anyway the last changes was adding a piece of code to lib/pkp/classes/mail.php in send function.
$mailer->SMTPOptions = array(
‘ssl’ => array(
‘verify_peer’ => false,
‘verify_peer_name’ => false,
‘allow_self_signed’ => true
)
);
Now it seems work but I am not sure if this code will harm the platform and security or not?

asmecher · June 3, 2020, 5:57pm

Hi @miki_farman,

Those are SSL options that are passed into the underlying PHP SSL connection toolset that PHPMailer uses to communicate with the SMTP server securely. They’re documented here: PHP: SSL context options - Manual

Your question is more about SSL and server configuration than OJS or even PHPMailer, but I think broadly speaking that the answer is that you’re disabling some important features related to secure connections between your web server and your SMTP server. Whether that’s a practical risk for you is debatable.

Regards,
Alec Smecher
Public Knowledge Project Team

marc · October 14, 2020, 11:54am

Hi @ctgraham and @asmecher,

Any help thinking this would be very much appreciated, because all this is beyond my knowledge about mailing servers.

the point is, I fall into this same issue this week and yes… the problem is in my server, but I’m wondering what would happen in the docker containers I have on this same server.

Probably you will tell me you are not a docker experts (nor do I) but as an exercise, just imagine the docker-container is an isolated/independent machine connected to the host via an internal network, and going to the world through this host (as far as is the container’s gateway).

So, forget docker and think in 3 different machines.

---------------        --------        --------------
| SMTP server | <----> | host | <----> | ojsMachine |
---------------        --------        --------------

The question is: taking in consideration that “ojsMachine” use “host” as a gateway to arrive to SMTP server, fixing the certificate issue between “smtp” and “host” won’t help “ojsMachine” relay mails, isn’t it? I mean, in the moment “ojsMachine” reach the SMTPserver it still request for a certificate for ojsMachine (not host) and validation will fail…

Do you agree?

I’m asking because the answer to the former question will determine which solution is better here:
a) Let containers relay mails to the “host” (without certificate validation), so “host” will forward to “SMTPserver”, ensuring host-smtp validation is ok.
b) Let containers relay mails “directly” to the SMTP (without certificate validation), that will only be a solution if host as a gatway will means it need to honor smtp-host credentials exchange.
c) Share certificates between the host and the containers and relay to SMTP.

The last one looks like the most secure, but at the same time, the most difficult to implement because each docker container (in same host) will probably have different credentials (different ojs journals with different domain names)… so I’m trying to keep configuration to the minimum.

What do you think?

Thanks in advance for your help,
m.

PD: Adding @lucasdiedrich to the conversation, just in case he also thought about this before.

lucasdiedrich · October 15, 2020, 2:00pm

Hey @marc, when the container try to communicate to anything outside it it will use a bridge connection from the host, so the host doesn’t “relay” the messages, it will bypass directly to the SMTP server.

The SMTP server will see this request as it was directly from the HOST server, not the container. But in network terms this should not interfere in any means with use of SSL certificates.

I didn’t got exactly the problem here, but thinking about the container, it is a dumb machine which doesn’t know or trust anything outside its scope, so or you tell him to blindly send emails to the SMTP server (a), or you tell him to trust some certs (c).

What i don’t get here is that i never saw client certificates to send emails, everything on secure SMTP server in os SMTP server itself. Can you exemplify this client certificates?

Thanks.

marc · October 20, 2020, 10:30am

Hi @lucasdiedrich

Nice to hear about you and sorry for the delay in my answer.

I’m asking because I fall in a similar issue than the described in this post or shown in this thread in stackoverflow.

Summarizing, my institutional mail sever is not asking for any validation (port 25 open to relay whatever they get… no comments) but looks like since php 5.6 they started to force credential validation (at least) so when OJS is calling phpMailer (over mail() function from php) I get connection errors about “certificate verify failed”.

As shown in stackoverflow, it can be easily fixed forcing phpMailer to avoid any validation, but it will make the system vulnerable to “man in the middle” attacks.

So my question is… if I fix the issue between my host and my mail server (ie: exchanging certificates), will my ojs dockers (with multiple domains, and configured to use smtp On against this open institutional server without any authentitication) be able to relay mails or they will be forced to validate credentials?

If I catch you, it won’t happen because “The SMTP server will see this request as it was directly from the HOST server, not the container” but in the other side, mail() will ask for certificate validations “side-to-side” so, from my understanding it means dockers will need a valid certificate to send mails.

If yes… I’m thinking to ask PKP for a fix to allow sending “noCredential” params to phpMailer because I can imagine other scenarios (not only mine, ie: on isolated networks) where you don’t like the smtp sever to validate docker’s credentials.

I’m talking with the mail manager and I’m quite sure we will find a solution, but I’m still interested to find a general solution to ensure OJS plays gently for docker production environments.

Thanks for your help,
m.