OJS upgrade issues (2.4.6.0 -->3.0.2.0)

We are experiencing issues attempting to upgrade OJS. Although several issues have been resolved, we are still getting the error “There are no readable files in this directory tree. Are safe mode or open_basedir active?”. We have copied our ojsdata directory from our production instance several times between upgrade attempts without success. It appears that the files are being moved – although I cannot find the destination. Any suggestions?

php: 5.4.16
apache 2.4.6
RHEL 7.3
MYSQL: 5.6.30

Hi @poulterd,

In what stage of the upgrade process do you receive this message? (The command-line upgrade tool should provide detailed output while it’s running.)

Regards,
Alec Smecher
Public Knowledge Project Team

I am getting the error in both the web upgrade interface and the command line upgrade (refreshing site between upgrades).

……

[code: Installer Installer::migrateFiles]

[code: Installer Installer::convertSupplementaryFiles]

There are no readable files in this directory tree. Are safe mode or open_basedir active?

From the php_errors log

[23-Aug-2017 12:20:07 US/Central] PHP Warning: copy(/apps/ojsdata/files/journals/1//articles/218/attachment/218-1-12024-1-13-20110829.docx): failed to open stream: No such file or directory in /apps/ojs-3.0.2/lib/pkp/classes/file/FileManager.inc.php on line 159
[23-Aug-2017 12:20:08 US/Central] PHP Warning: assert(): Assertion failed in /apps/ojs-3.0.2/lib/pkp/classes/submission/PKPSubmissionFileDAO.inc.php on line 380
[23-Aug-2017 12:20:08 US/Central] ojs2: There are no readable files in this directory tree. Are safe mode or open_basedir active?

It seems that it cannot find the files – although they are present. The “//” is interesting since I cannot find where it is adding the extra slash.

Thanks for your help.

-Dale

Hi @poulterd,

The double-slash is a red herring – operating systems will ignore this. See Upgrade from OJS 2.4.7 to 3.0.1 fails - #18 by asmecher for the likely cause.

Regards,
Alec Smecher
Public Knowledge Project Team

Alec,

Thank you for the pointer. We have walked through the process a few times to be sure we did not miss any articles without an assigned section. Sadly, we get the same error.

-Dale

Hi @poulterd,

Did you check the potential data issue described in the link above?

Regards,
Alec Smecher
Public Knowledge Project Team

Alec,

We did have several articles without sections and reviewed the remainder of the conversation with Oberdan. However, when I turn on loging I still get the same visible error and the stack trace also looks similar – although the fix has been applied.

[code: Installer Installer::convertEditorDecisionNotes]

[code: Installer Installer::convertQueries]

[code: Installer Installer::migrateFiles]

[code: Installer Installer::convertSupplementaryFiles]

There are no readable files in this directory tree. Are safe mode or open_basedir active?

Stack Trace:

File: /apps/ojs-3.0.2/lib/pkp/classes/file/FileManager.inc.php line 157
Function: FileManager->mkdirtree("")

File: /apps/ojs-3.0.2/lib/pkp/classes/submission/PKPSubmissionFileDAO.inc.php line 383
Function: FileManager->copyFile(Null, Null)

File: /apps/ojs-3.0.2/classes/install/Upgrade.inc.php line 1269
Function: PKPSubmissionFileDAO->updateObject(Object(SubmissionFile))

File: (unknown) line (unknown)
Function: Upgrade->convertSupplementaryFiles(Object(Upgrade), Array(1))

File: /apps/ojs-3.0.2/lib/pkp/classes/install/Installer.inc.php line 415
Function: call_user_func(Array(2), Object(Upgrade), Array(1))

File: /apps/ojs-3.0.2/lib/pkp/classes/install/Installer.inc.php line 265
Function: Installer->executeAction(Array(3))

File: /apps/ojs-3.0.2/lib/pkp/classes/install/Installer.inc.php line 186
Function: Installer->executeInstaller()

File: /apps/ojs-3.0.2/lib/pkp/classes/cliTool/UpgradeTool.inc.php line 88
Function: Installer->execute()

File: /apps/ojs-3.0.2/lib/pkp/classes/cliTool/UpgradeTool.inc.php line 64
Function: UpgradeTool->upgrade()

File: /apps/ojs-3.0.2/tools/upgrade.php line 34
Function: UpgradeTool->execute()

-Dale

Hi @poulterd,

Just to verify, after correcting the data (but before running the upgrade), you shouldn’t get any results for the following query:

SELECT a.article_id FROM articles a LEFT JOIN sections s ON (a.section_id = s.section_id) WHERE s.section_id IS NULL;

Is that the case?

Regards,
Alec Smecher
Public Knowledge Project Team

That is correct. The query returns no rows.

-Dale

Hi @poulterd

Could it be that that is a remote supplementary file?
Would it be possible for you to print out the file ID ($submissionFile->getFileId()) an maybe path ($submissionFile->getFilePath()) before this line: ojs/Upgrade.inc.php at ojs-3_0_2-0 · pkp/ojs · GitHub ?
Then to see (in the DB) if that supp file is a remote file i.e. if there is a file on that file path i.e. what could be different/wrong with that file?

Maybe: if that is a remote supp file, you would need this fix/changes: pkp/pkp-lib#2192 migrate remote supp files as remote galleys · pkp/ojs@28fd6f2 · GitHub

Best,
Bozana

Alec,

Thank you for your help. We have been able to get the upgrade to work. We created a new dump, cleaned up some diacritic issues, corrected the empty, and then ran the upgrade. I am not sure what was different but it is now working. Thanks again!

-Dale

Hi @poulterd,

Glad to hear it’s working!

Regards,
Alec Smecher
Public Knowledge Project Team

Good Morning @asmecher

when i ran the query before upgrade " ```
SELECT a.article_id FROM articles a LEFT JOIN sections s ON (a.section_id = s.section_id) WHERE s.section_id IS NULL;

it returns 13 records in phpmyadmin, in this case what does it mean? please help. 

13 records like this:

	
Edit Edit
 Copy Copy
 Delete Delete
30380

Edit Edit
 Copy Copy
 Delete Delete
31796

Edit Edit
 Copy Copy
 Delete Delete
31831

Edit Edit
 Copy Copy
 Delete Delete
58835

Edit Edit
 Copy Copy
 Delete Delete
58848

Edit Edit
 Copy Copy
 Delete Delete
58864

Edit Edit
 Copy Copy
 Delete Delete
58882

Edit Edit
 Copy Copy
 Delete Delete
58883

Edit Edit
 Copy Copy
 Delete Delete
58887

Edit Edit
 Copy Copy
 Delete Delete
58900

Edit Edit
 Copy Copy
 Delete Delete
58903

Edit Edit
 Copy Copy
 Delete Delete
58998

Edit Edit
 Copy Copy
 Delete Delete
59004

Thanks so much Alec.
Dunn.

Hi @dung,

It means you have 13 articles in your OJS 2.x system that aren’t associated with any section, which is a data error. Before you upgrade, make sure that every article has a value in its section_id column that corresponds to a section_id in the sections table (with the section’s journal_id the same as the article’s journal_id, i.e. the article’s section must belong to the same journal as the article).

Regards,
Alec Smecher
Public Knowledge Project Team

Hi @asmecher,

Thanks for your reply, I will correct data and re-upgrade, will post back.

Kind regards,

Dung.

Hi @asmecher,

following your instruction above, and given 1 of my 13 articles has (data error: ‘section_id = 0’) and its’ article_id = 30380’ I ran this query below to find a valid ‘section_id’ value to replace value 0 of ‘section_id’ in articles table. But the query returns 5 different values, should it not return only 1 value?

query:

select a.article_id, a.journal_id, s.section_id, s.journal_id
from articles a
inner join sections s
on (a.journal_id = s.journal_id)
where a.article_id = 30380;

results:

|article_id|journal_id|section_id|journal_id|
|30380|84|488|84|
|30380|84|489|84|
|30380|84|490|84|
|30380|84|491|84|
|30380|84|492|84|

How did I wrongly translate your instruction into sql language?

Thanks in advance for any help.

Dung.

Hello @asmecher

So I ran this query for all 13 bad records.

SELECT a.article_id, a.journal_id, s.section_id, j.path
FROM articles a
INNER JOIN sections s on (a.journal_id = s.journal_id)
INNER JOIN journals j on (s.journal_id = j.journal_id)
WHERE a.article_id = 30380 OR a.article_id = 31796 OR a.article_id = 31831 OR a.article_id = 58835 OR a.article_id = 58848 OR a.article_id = 58864 OR a.article_id = 58882
OR a.article_id = 58883 OR a.article_id = 58887 OR a.article_id = 58900 OR a.article_id = 58903 OR a.article_id = 58998 OR a.article_id = 59004

and manually edit field section_id of each record of articles table using the very first section_id (multiple section_id but same path/slug returned for a given journal as mentioned above) so that I can continue with upgrade process OJS 2.4.8-3 to OJS 3.1.1-4. The upgrade process successfully passed this error but now it ran into another error below:

-----



(mysql): UPDATE review_assignments SET quality = NULL WHERE quality = 0

-----



-----

(mysql): UPDATE users u, (SELECT user_id, GROUP_CONCAT(DISTINCT setting_value SEPARATOR ’ ') AS groupedGossips FROM user_settings WHERE setting_name = ‘gossip’ AND setting_value <> ‘’ GR OUP BY user_id) us SET u.gossip = us.groupedGossips WHERE us.user_id = u.user_id

-----



1260: Row 8 was cut by GROUP_CONCAT()
ADOConnection._Execute(UPDATE users u , (SELECT user_id, GROUP_CONCAT(DISTINCT setting_value SEPARATOR ’ ') AS groupedGossips FROM user_settings WHERE s…, false)% line 1051, file: /var/www/html/ojs-3.1.1-4/lib/pkp/lib/adod b/adodb.inc.php
ADOConnection.Execute(UPDATE users u, (SELECT user_id, GROUP_CONCAT(DISTINCT setting_value SEPARATOR ’ ') AS groupedGossips FROM user_sett ings WHERE s…)% line 446, file: /var/www/html/ojs-3.1.1-4/lib/pkp/classes/install/Installe r.inc.php
Installer.executeSQL(UPDATE users u, (SELECT user_id, GROUP_CONCAT(DISTINCT setting_value SEPARATOR ’ ') AS groupedGossips FROM user_settings WHER E s…)% line 441, file: /var/www/html/ojs-3.1.1-4/lib/pkp/classes/install/Installer.inc.php
Installer.executeSQL(Array[19])% line 399, file: /var/www/ht ml/ojs-3.1.1-4/lib/pkp/classes/install/Installer.inc.php
Installer.executeAction(Array[3])% line 265, file: /var/www/html/ojs -3.1.1-4/lib/pkp/classes/install/Installer.inc.php
ERROR: Upgrade failed: DB: Row 8 was cut by GROUP_CONCAT()

for this error I think I will have to modify query in 3.1.1_update.xml file to set group concat max length to longer such as

SET GLOBAL group_concat_max_len=4294967295;

but not so sure where to put the line?

Some help, tips, or pointers that could help me would be much appreciated.

Dung.

Hi @dung,

I think the best approach to this would be to set it by editing your MySQL configuration – but if that’s not possible, you could add it to the getConn function in lib/pkp/classes/db/DBConnection.inc.php, which is where database connections are created and passed into the rest of the system.

These are ADONewConnection objects, which are part of the ADODB library, so the ADODB documentation may help.

Regards,
Alec Smecher
Public Knowledge Project Team

Hi @asmecher,

Thanks for your reply, before I received your message above I decided to try setting it in “/etc/mysql/mysql.conf.d# vi mysqld.cnf” below section.

[mysqld]
#Basic Settings
group_concat_max_len = 1294967295
user = mysql

To be sure the param is set, I also connected the database using mysql console (mysql -u root -h localhost -p) and successfully issued this command

“SET GLOBAL group_concat_max_len=1294967295”

After that I re-run the migration for 12 hours and it was successfully with message

“Successfully upgraded to version 3.1.1.4”

at the very end.

I would like to sum it up here for others: I ran thru 3 problems:

“1406: Data too long for column ‘contents’ at row 1”

solution in this case I also found somewhere on this forum (too bad did not save url, will update if I can find the page again, here is the link to the conversation of Alec on page 32/52 Upgrade from OJS 2.2.4 to 2.4.8 Error - #33 by RickMath) that you have to change

ojs-3.1.1-4/lib/pkp/xml/schema/notes.xml

from text to long text such as:

from
<field name=“contents” type=“L”/>
to
<field name=“contents” type=“XL”/>

“There are no readable files in this directory tree. Are safe mode or open_basedir active?”

solution for this is to fix data corruption as mentioned above in this post by running query below to find out which records of articles table are corrupted and fix in field section_id

SELECT a.article_id FROM articles a LEFT JOIN sections s ON (a.section_id = s.section_id) WHERE s.section_id IS NULL;

and replace 0 or null in section_id field of table articles with new value where new value can be found as instructed in below quote:

“make sure that every article has a value in its section_id column that corresponds to a section_id in the sections table (with the section’s journal_id the same as the article’s journal_id , i.e. the article’s section must belong to the same journal as the article).” from @asmecher

“1260: Row 8 was cut by GROUP_CONCAT() … ERROR: Upgrade failed: DB: Row 8 was cut by GROUP_CONCAT()”

solution for this is to config your database, or your database engine, or migration script. What I did was:

Set it in “/etc/mysql/mysql.conf.d# vi mysqld.cnf” below section.

[mysqld]
#Basic Settings
group_concat_max_len = 1294967295
user = mysql

Also to be sure I did:

connected the database using mysql console (mysql -u root -h localhost -p) and successfully issued this command “SET GLOBAL group_concat_max_len=1294967295”.

Thanks @asmecher and the people of forum for your time.

Dung.

Hi @asmecher,

I am still troubled by this issue would you please give me a pointer on this:

following your instruction above, and given 1 of my 13 articles has (data error: ‘section_id = 0’) and its’ article_id = 30380’ I ran this query below to find a valid ‘section_id’ value to replace value 0 in ‘section_id’ of articles table. But the query returns 5 different section_id values, should it not return only 1 value?

How do I identify which value? 488 or 489 or 490 or 491 or 492? Is there a table that I need to inner joint to?

query:

select a.article_id, a.journal_id, s.section_id, s.journal_id
from articles a
inner join sections s
on (a.journal_id = s.journal_id)
where a.article_id = 30380;

results:

|article_id|journal_id|section_id|journal_id|
|30380|84|488|84|
|30380|84|489|84|
|30380|84|490|84|
|30380|84|491|84|
|30380|84|492|84|

How did I wrongly translate your instruction into sql language?

Thanks for any pointer Alec, if you can point me to the right direction?

Best regards,

Dung.