OJS 3.6 Patch fails with malformed patch at line X

I have searched this forum and google on this topic with no joy - so hoping I can get some resolution here.

Our OJS installation is repeatedly compromised by “people” uploading .php malware scripts as Journal submissions.

I suspect upgrading will fix the problem but all attempts to apply ojs-2.3.6_to_2.4.8-5 patch are failing. We keep getting malformed patch output on the patch.

I have to assume the patch is good and that I am doing something wrong.

I am running patch out of a Git Bash console on Windows 10.

Any input / guidance on this would be appreciated.

Hi @marcorpsa,

I would recommend following the “full package” instructionss rather than using a patch, particularly if your server has been compromised.

Make sure your files_dir is not directly web-accessible to prevent compromises via malicious author uploads. Otherwise it’ll happen again. (See the recommended configuration area in the README documentation.)

If you really want to use the patch, on Windows based systems you’ll have to convert the patch file’s line endings e.g. by opening and saving it in Notepad.

Regards,
Alec Smecher
Public Knowledge Project Team

@asmecher,
Thank you for your reply

I am running the patch against the last good backup of the site.

The file_dir is accessible - which we will fix however the problem still remains that malicious scripts can be uploaded - this is triggering malware alerts from the server with threats to shut the site down. Need to prevent upload of anything except doc, docx and PDF - is there a way to configure that

I updated the CR => CR/LF - still getting a malfored patch error

patch: **** malformed patch at line 36675: s est▒ dispon▒vel em

Which seems to be some arbitrary content in this file

diff -urN ojs-2.3.6/docs/FAQ_pt ojs-2.4.8-5/docs/FAQ_pt

This is actually round 2 of this update process - we tried to go through it about 18 months back and could not get anywhere - patch kept failing - which is why I am posting here.

Any assistance you can provide in getting this patch to apply would be greatly appreciated.

Hi @marcorpsa,

The best solution will be simply not to use the patch file – use the “Full Package” instructions described in docs/UPGRADE. If your system has been compromised by a malicious file upload, a common thing for a hacker will be to run an automated script that finds and modifies PHP code. This will prevent the patch from applying or possibly even upgrade to a newer version with the malware preserved. Until you do this, you’ll probably find yourself back in the same situation repeatedly.

If you move your files_dir outside of the web root, then there may still be attempts to upload malicious code, but it will be impossible for the attacker to cause it to execute. For that reason, it’s a safe configuration.

If you’re trying to use a patch because you want to preserve local modifications, I’d suggest a different workflow: unpack a copy of the stock version of OJS you’re currently running, and use diff to compare it to the live copy on your site. This will create a patch file describing the current modifications to the system. (I suspect you’ll find both your own modifications, and some malware.) Strip the patch file down to just the parts you want to preserve (i.e. remove the malware and any accumulated cruft) and apply that patch to a newly-unpacked codebase. This will prevent cruft from accumulating, not to mention malware.

Regards,
Alec Smecher
Public Knowledge Project Team

Hi Alec,

As I said before - the code base is not compromised - I am running against the clean (off site) backup.

The issue with the files_dir is not so much the execution of the scripts - our hosting provider runs scans on the site - if they find a malicious package our site gets suspended - we want to limit what files can be uploaded.

I just need to understand your first suggestion regarding the Full Package. Are you saying that from 2.3.6 we can upgrade simply by preserving the config, public and files_dir against a full install of the latest package with no impact from a DB perspective?

I really appreciate your help on this - this has been on my todo for months and has been causing untold stress - being able to resolve it is going to make a huge difference to us.

Hi @marcorpsa,

the code base is not compromised

Definitely good news – sorry, I misread your earlier post.

we want to limit what files can be uploaded

OK, gotcha. In OJS 3.x there is a plugin for this called the “Allowed Uploads” plugin; once you get there I’d recommend installing it. (You’ll still need to ensure that your files_dir is outside the web root or otherwise protected – just using the Allowed Uploads plugin isn’t enough to run a secure OJS install.)

For OJS 2.x, it’s been a little while since I worked with this, so your mileage may vary – but I think the operative check on file uploads is in lib/pkp/classes/file/FileManager.inc.php in the parseFileExtension function. It checks for .php files but not .phtml, which is sometimes configured to execute server-side. You can extend that function to add additional checks, or change the current behavior when a check fails (currently it adds a .txt to the end of the filename when the check is triggered).

I just need to understand your first suggestion regarding the Full Package. Are you saying that from 2.3.6 we can upgrade simply by preserving the config, public and files_dir against a full install of the latest package with no impact from a DB perspective?

Yes – but after you upgrade the code and point it at your old database etc., you’ll have to run the upgrade script (also described in the “Full Package” documentation) to upgrade the database. Of course, take a good backup of everything before you do this.

Regards,
Alec Smecher
Public Knowledge Project Team

Thank you Alec,

Going to give this a go.

Regards

Julian

Alec,
Some feedback on this issue.
We followed the guidelines in the UPGRADE doc as you suggested for doing a full package upgrade.
Everything fine up until the part where we upgrade the DB.
The doc says to use the CLI like so
php tools/upgrade.php upgrade
Did that and received a bunch of deprecation warnings relating to features removed in PHP 7. The Download page specifies that PHP 7 is a requirement, however the script fails on named constructors and the tools seems to want to use the mysql lilbrary instead of the mysqli or PDO.
I then tried running the script with PHP 5.6 and got
DB Error: Unknown column ‘domain’ in ‘field list’
I then followed the instructions to do the upgrade through the Web client - proceed to the Upgrade page - clicked upgrade - white screen of death.

In desperation tried renaming the mysqli to mysql lib and tried the CLI again - no joy.

We really could use some help here. We have been trying for nearly 18 months to update this site and we seem to be blocked at every turn.

Regards
Julian

Just a follow up on this (we are working the problem)
The white screen was caused by the config mysql driver not being set correctly. Changing to mysqli - that is working.

However, when we run the upgrade now we get

Upgrade unsupported. See docs/UPGRADE-UNSUPPORTED for details.

That seems to suggest we cannot upgrade directly from our version to 3.1 - we fist have to go to 2.4

So that is what we will be doing next.

No need to respond to either of the last two pots - if we get stuck I will post back her.

1 Like

Hi @marcorpsa,

Just to clarify a couple of points – yes, when using PHP7, you’ll need to choose the mysqli driver. (The mysql driver is deprecated and often not available in PHP7.)

The deprecation warnings are to be expected, and we gradually work to fix these. Many/most will be fixed with the release of OJS 3.2. However, these are just cosmetic and won’t stop the system from working.

Regards,
Alec Smecher
Public Knowledge Project Team

Hi Alec,
Installed 2.4.8 - tried the update - that failed because default on our system is PHP7
Used the command line to run the upgrade tool using 5.6.4 - that seemed to work nicely until it got to the reviews.xml file then it bombed out.
This was while it was trying to add the following to review_rounds
<field name=“review_round_id” type=“I8”>
<KEY />
<AUTOINCREMENT/>
</field>
Error: Incorrect table definition; there can be only one auto column and it must be defined as a key
I trashed the DB and restored from backup - manually added the auto increment and tried again - this time the review.xml passed.
However, the upgrade bombed again with this

[code: Installer Installer::removeAuthorRevisedFilesFromSignoffs]
[note: docs/release-notes/README-2.3.7]

[code: Installer Installer::installEmailTemplate]
<h1>DB Error: Incorrect string value: ‘\xD8\xB3\xD9\x84\xD8\xA7…’ for column ‘body’ at row 1</h1>

I tried digging around to find out where that string is but it is not clear which table it is in. I dumped the schema for the updated database and searched for all tables with a body field which returned this
comments
email_log
email_templates_data
email_templates_default_data
I checked each of these with a query to try and find the issue - but nothing came up.

Any ideas on where to look?

Trying very hard to not bug you too much but spent almost an hour on this one issue with no resolution so hoping you can point me in the right direction.

Hi @marcorpsa,

The most likely cause for that is a muddle character set configuration (e.g. mixing Latin1 and UTF-8 or something similar). Did you inadvertently change your character set configuration in config.inc.php in the process of trying the upgrade?

Regards,
Alec Smecher
Public Knowledge Project Team

No, what I did was I did a file diff (windiff) on our config and the config that came with the 2.4.8 install. I then changed (in the 2.4.8 version) the values that were set in our config file - so in essence I am using an updated 2.4.8 config - not our original. I thought that would be the best approach to ensure no issues with moving to 2.4.8
I checked the default_charset in both original config files and 2.4.8 - both set to UTF-8 - however in the schema dumps for our 2.3.x db and the 2.4.8 (after running the upgrade) - all are charset=latin1 but after the upgrade there are some tables with a UTF-8 charset - none of them however have a body field.

Hi @marcorpsa,

If your config.inc.php character set configurations are the same, then perhaps the problem is in the database creation itself. Check to see that the database character sets are consistent (e.g. if you’re working with a test database loaded from a dump).

Regards,
Alec Smecher
Public Knowledge Project Team

Hi Alec,

Created a new DB with charset UTF-8 same problem - same place.
Is there a way I can at least find out which table / row is causing the issue?

Hi @marcorpsa,

The problem is more likely that your old database has a misconfiguration, e.g. it’s storing UTF-8 content in a Latin1 database, but that your new database has it right. If that guess is correct, you have two options:

  1. Make sure that your new database mimics the same (potentially wrong) configuration as your old one, or
  2. Figure out your old database’s misconfiguration and transcode it.

Either way, I’d recommend inspecting the character set configuration of your old database to see whether it matches the options set up in config.inc.php. (Checking the character set configuration of your old database is more of a MySQL admin question than an OJS question – there should be lots of guidance on Stackoverflow.)

The body field probably refers to email_templates_data and/or email_templates_default_data.

Regards,
Alec Smecher
Public Knowledge Project Team

I am about to give up this.
I recreated the DB by doing a full dump from the source with create database - used that to create on my side.
Tried the install again and I got this
[pre-install]
[load: upgrade.xml]
ERROR: Upgrade failed: ##installer.installFileError##

No matter what I do - I get the same result. I have removed the folder twice and extracted the 2.4.8 code - updated the DB, tried to go in with the web browser That gives me this
Fatal error : Call to a member function getVersionString() on boolean in E:\Projects\SAJG\website\cache\t_compile%%73^736^73609D70%%install.tpl.php on line 55
The I try an upgrade check
I get this
Failed to load version info from http://pkp.sfu.ca/ojs/xml/ojs-version.xml

Running out of options

Hi @marcorpsa,

The installer.installFileError message suggests that a file (e.g. dbscripts/xml/upgrade.xml) is missing or contains a typo that prevents parsing. Double-check that extracting the .tar.gz file was successful, and that there are no accidental changes to the files.

You say you’re trying the install process, but with your old database in place. Are you perhaps using the installation script instead of the upgrade script?

(Rather than jumping between processes and getting frustrated in the bargain, I’d suggest sticking with the command-line upgrade process. It’s the most reliable and provides the most information if something isn’t working properly.)

Regards,
Alec Smecher
Public Knowledge Project Team

Hi Alec,
Apologies or my last post - it was late and I was getting frustrated.

I have been through the process of unzipping the tar multiple times - same result.

Sometimes a fresh start is needed.

I did a complete new everything. New folder, unzipped archive.
I then imported the DB script exactly as it is from our server - with create database etc.

I started php 5.6.4 local server in the folder in question and used the Web interface to initiate the update process.
It failed on the same \xD8… issue.
I then did a search and replace on the SQL dump of the db changing all the charset’s of the table to UTF8.

Did a complete install again and then ran the updated SQL script.
Repeated the update - this time it completed successfully so we are now at version 2.4

Going to try the 3.1 update.

Hi @marcorpsa,

Not a problem, sometimes the best solution is to put it down for a few hours. I’m glad you’re making progress and good luck with the 3.x update. (OJS 2.3.6 to 2.4.8 is already a big step.)

Regards,
Alec Smecher
Public Knowledge Project Team