Upgrade to OJS 3.0 from 2.4.8.0 - Error while converting supplementary files

Software Support
,
21

Hi @Tarcisio_Pereira,

I think you’ll have to step through Upgrade::migrateUserRoles in the section that’s responsible for migrating Journal Manager roles:

// Managers.
$group = $userGroupDao->getDefaultByRoleId($journal->getId(), ROLE_ID_MANAGER);
$userResult = $journalDao->retrieve('SELECT user_id FROM roles WHERE journal_id = ? AND role_id = ?', array((int) $journal->getId(), ROLE_ID_MANAGER));
while (!$userResult->EOF) {
    $row = $userResult->GetRowAssoc(false);
    $userGroupDao->assignUserToGroup($row['user_id'], $group->getId());
    $userResult->MoveNext();
}

This should be upgrading all your old OJS roles (the roles table) to user groups user_groups and user_user_groups. At a glance, I can’t see why any users who were Journal Managers before shouldn’t be included here, but it looks like in some cases they are being skipped.

Regards,
Alec Smecher
Public Knowledge Project Team

22

Hi @asmecher

I don’t know what else can I test.
I tried delete all roles_id=16 and insert just one user for all the journals.
I tried delete this journal, same problem in other one.
No success. Same problem in every test.

Any idea?

Regards,
Tarcisio Pereira

23

Hi @asmecher

I do not know what else to do.
I’m not able to upgrade to 3.0.1.
What do you think about I wait for 3.0.2?

Regards,
Tarcisio Pereira

24

Hi @Tarcisio_Pereira,

I would suggest e.g. using error_log calls in the above code to see whether/why the user groups aren’t being populated as expected for those journals.

Regards,
Alec Smecher
Public Knowledge Project Team

25

Hi @asmecher

For every journal I have, when I upgrade to 3.0, ojs creates 3 user_groups for role_id = 16.
I checked with:

select context_id, count(*) from user_groups where role_id = 16 group by context_id order by context_id;

Is this the expected behavior? Or should I see just one?

Regards,
Tarcisio Pereira

26

Hi @Tarcisio_Pereira,

Hmm, but I thought we had established that this was not the case in the earlier debugging? (i.e. that there were journals without such a user group.)

Regards,
Alec Smecher
Public Knowledge Project Team

27

Hi @asmecher

My point is;
What exactly is the problem? There is no users in the third group? or 3 groups is also a problem?
From my point of view there should be only one group of managers.

I believe that we have established that there is a problem in the migration of users, but we do not define exactly the expected behavior.

Regards,
Tarcisio Pereira

28

Hi @Tarcisio_Pereira,

What’s supposed to happen is that upon upgrade the script is supposed to migrate permissions/roles from the old model (fixed roles) to the new OJS 3.x one (flexible roles, with defaults installed similarly to OJS 2.x). So creating 3 new user groups per journal is expected, as the defaults specify that. Users should be assigned from the old journal manager role to these 3 groups. When upgrading supplementary files, the code relies upon these newly created groups. So the behavior you’re describing regarding the 3 user groups seems correct to me.

Regards,
Alec Smecher
Public Knowledge Project Team

1 Like
29

Hi @asmecher

I’m testing with one journal:

             while ($journal = $journals->next()) {
                     if ($journal->getId() != 195)
                             continue;
                     // Install default user groups so we can assign users to them.

From what I’ve been looking at, the migration should happen in the following section:

                     // regular Editors.  NOTE:  this involves a role id change from 0x100      to 0x10 (old OJS _EDITOR to PKP-lib _MANAGER).
                     echo("Migrating regular editors\n");
                     $userGroups = $userGroupDao->getByRoleId($journal->getId(), ROLE_ID_MANAGER);
                     var_dump($userGroups);
                     $editorUserGroup = null;
                     while ($group = $userGroups->next()) {
                            echo("Interacting in group_id = ".$group->getId()."\n");
                             if ($group->getData('nameLocaleKey') == 'default.groups.name.editor') {
                                     $editorUserGroup = $group; // stash for later.
                                     $userResult = $journalDao->retrieve('SELECT user_id FROM roles WHERE journal_id = ? AND role_id = ?', array((int) $journal->getId()     , 0x00000100 /* ROLE_ID_EDITOR */));
                                     while (!$userResult->EOF) {
                                             $row = $userResult->GetRowAssoc(false);
                                             echo("Migrating user ".$row['user_id']." to group ".$group->getId()."\n");
                                             $userGroupDao->assignUserToGroup($row['user_id'], $group->getId());
                                             $userResult->MoveNext();
                                     }
                             }
                     }

It was the only place I could find the id of the group without users.
But this section only changes the default.groups.name.editor group. Is there something missing or am I looking in the wrong place? I’m unable to find where is the section for “default.groups.name.productionEditor”.

The output show me only the message:
Interacting in group_id = 4
But no user is migrated.

Regards,
Tarcisio Pereira.

30

Hi,

Can someone please help me?
I’ve been trying to update for two and a half months. Unsuccessfully.

Best regards,
Tarcisio Pereira

31

Hi @Tarcisio_Pereira,

Would you be able to provide me with a database dump for some local testing?

Regards,
Alec Smecher
Public Knowledge Project Team

32

Hi @asmecher

Off course. I will prepare it and send you a private message with link.

Regards,
Tarcisio Pereira.

33

Hi @Tarcisio_Pereira,

I spent some time working through your installation and it looks like you’ve run into a number of pitfalls:

I know you’ve followed the work-arounds for the first two, but I’m not sure if you’ve seen the last one yet?

Regards,
Alec Smecher
Public Knowledge Project Team

34

Hi @asmecher sorry about the delay, we are in a holiday here.

The journal manager put the article in the right place. For upgrade testing only you can put it in any section of the journal.
Let me know about anything you need.

Regards,
Tarcisio Pereira

35

Hi,

I get the same error upgrading from 2.4.8.2 to 3.0.2.

[code: Installer Installer::convertSupplementaryFiles]
PHP Fatal error:  Call to a member function getId() on null in /pathToOjs/classes/install/Upgrade.inc.php on line 1019

1013			$row = $suppFilesResult->getRowAssoc(false);
1014			$suppFilesResult->MoveNext();
1015			if (!$journal || $journal->getId() != $row['context_id']) {
1016				$journal = $journalDao->getById($row['context_id']);
1017				$managerUserGroup = $userGroupDao->getDefaultByRoleId($journal->getId(), ROLE_ID_MANAGER);
1018				$managerUsers = $userGroupDao->getUsersById($managerUserGroup->getId(), $journal->getId());
1019				$creatorUserId = $managerUsers->next()->getId();
1020			}

I work with the current ojs-stable-3_0_2 branch and I’ve checked all suggestions in this topic, but I still get the error.

@Tarcisio_Pereira Do you resolve this issue?

Someone can help me?

Many thanks
Giuseppe

2 Likes
36

Hi @atarix

It seems like one of your journals do not have a journal manager, could that be? – I is not enough to have admin with journal manager rights, there should be a ‘real’ journal manager.
Also, it already happened to some other users that there were some old journals in the DB, that were deleted long time ago, but for some reason they stayed in the DB i.e. were not completely removed, so the upgrade script would find and go through them, but would not find other elements e.g. journal managers. Could you double check if this is maybe in your case too – if you have any journal in your DB table journals, that should not be there any more?

Best,
Bozana

37

HI @atarix

I’m waiting for Alec.
He is testing my database.

Regards,
Tarcisio Pereira.

38

@bozana

thanks I’ll check this possibility

39

Hi @Tarcisio_Pereira,

I tried re-running your upgrade with a new copy of the database. Just in case it’s useful, it’s much quicker to run a test migration if you remove some of the table data from the dump – in particular, the full-text index and metrics. I did this by filtering your database dump and creating a leaner one:

zcat full_database_dump.sql.gz | sed -e '/^COPY metrics/,/^$/d' -e '/^COPY article_search_/,/^$/d' | gzip -9 > database_dump_trimmed.sql.gz

This dropped the database dump down from 2GB to 300MB; testing the upgrade process is massively sped up.

In the process I identified an issue that was slowing down the upgrade and resulting in some incorrect data – the patch there will resolve the issue during upgrade, and I’m working on code that’ll correct any bad data that got introduced. However, I the only effect you’d observe from this is a slow upgrade; patching it should speed up the role migration part of the upgrade.

In my latest test, I no longer get the error you first quoted. Now I get:

PHP Warning:  assert(): Assertion failed in /home/asmecher/git/ojs/lib/pkp/classes/submission/SubmissionFileDAODelegate.inc.php on line 134
PHP Stack trace:
PHP   1. {main}() /home/asmecher/git/ojs/tools/upgrade.php:0
PHP   2. UpgradeTool->execute() /home/asmecher/git/ojs/tools/upgrade.php:34
PHP   3. UpgradeTool->upgrade() /home/asmecher/git/ojs/lib/pkp/classes/cliTool/UpgradeTool.inc.php:64
PHP   4. Installer->execute() /home/asmecher/git/ojs/lib/pkp/classes/cliTool/UpgradeTool.inc.php:88
PHP   5. Installer->executeInstaller() /home/asmecher/git/ojs/lib/pkp/classes/install/Installer.inc.php:186
PHP   6. Installer->executeAction() /home/asmecher/git/ojs/lib/pkp/classes/install/Installer.inc.php:265
PHP   7. call_user_func:{/home/asmecher/git/ojs/lib/pkp/classes/install/Installer.inc.php:415}() /home/asmecher/git/ojs/lib/pkp/classes/install/Installer.inc.php:415
PHP   8. Upgrade->convertSupplementaryFiles() /home/asmecher/git/ojs/lib/pkp/classes/install/Installer.inc.php:415
PHP   9. PKPSubmissionFileDAO->updateObject() /home/asmecher/git/ojs/classes/install/Upgrade.inc.php:1319
PHP  10. SupplementaryFileDAODelegate->insertObject() /home/asmecher/git/ojs/lib/pkp/classes/submission/PKPSubmissionFileDAO.inc.php:390
PHP  11. SubmissionFileDAODelegate->insertObject() /home/asmecher/git/ojs/lib/pkp/classes/submission/SupplementaryFileDAODelegate.inc.php:42
PHP  12. assert() /home/asmecher/git/ojs/lib/pkp/classes/submission/SubmissionFileDAODelegate.inc.php:134
<h1>DB Error: ERROR:  null value in column &quot;file_id&quot; violates not-null constraint
DETAIL:  Failing row contains (128282, en_US, 68234, null, Untitled, 9, RemoteURL, 0).</h1>ojs2: DB Error: ERROR:  null value in column "file_id" violates not-null constraint
DETAIL:  Failing row contains (128282, en_US, 68234, null, Untitled, 9, RemoteURL, 0).

It looks like you have some further data problems that might be related – for example, entries in article_supplementary_files that refer to entries in article_files (by file_id) where the article_files entry is missing:

select count(*) from article_supplementary_files s LEFT JOIN article_files f ON (f.file_id = s.file_id) WHERE f.file_id IS NULL;

I would suggest reviewing and correcting these.

Regards,
Alec Smecher
Public Knowledge Project Team

1 Like
40

Hi @asmecher

Thank you, I will check it.
I’ll try to update again using the base I sent you last.
I come back here to say my results.

Regards,
Tarcisio Pereira;