Migrate and Upgrade Errors From 2.4.5 to 3.1.1

Would anyone be able to assist me in straightening out some errors I’m getting? At a high level, we are trying to migrate from a CentOS 6/OAJ v 2.4.5 to CentOS 7/OJS v3.1.1.

Here are my setup steps and the error I get when upgrading on the CentOS 7 server:

  1. Download and install Apache 2.4, PHP 5.6, and MySQL 5.7
  2. Downloaded OJS zip, extracted it into document root, and created and updated privileges on the appropriate directories
  3. mysqldump the database on the CentOS 6 host.
  4. Initialized my OJS database and DB user with ALL privileges on that DB
  5. Set config.inc.php installed = Off
  6. imported database dump taken from CentOS 6 host

Now here is where it gets odd and I think it may be because the source DB had been upgraded at some point in the past.

  1. Because I get a few errors where [what appears to be migration-related] tables already exist, I took the liberty of deleting a few from the restored DB: article_files_migration, articles_migration, submissions_tmp, review_assignments_tmp, notes_tmp

  2. Now comes the upgrade error that seems to require deleting table rows I don’t understand, so I thought it might be wise to reach out to the forum.

    # php tools/upgrade.php upgrade
    [pre-install]
    [load: upgrade.xml]
    [version: 3.1.1.1]

    [code: Installer Installer::checkPhpVersion]
    [data: dbscripts/xml/upgrade/3.1.0_preupdate_review_assignments.xml]
    [data: dbscripts/xml/upgrade/3.1.0_preupdate_notes.xml]
    [data: dbscripts/xml/upgrade/3.1.0_preupdate_payments.xml]
    [data: dbscripts/xml/upgrade/3.1.1_preupdate_citations.xml]
    [schema: lib/pkp/xml/schema/common.xml]
    ERROR: Upgrade failed: DB: Duplicate entry ‘tinymceplugin-0-enabled’ for key ‘plugin_settings_pkey’

  3. Prior to retrying, I’ll run steps 5-8 again to start fresh with the DB dump.

It’s possible to get past this error by simply removing that db table entry, but seems like dangerous territory to start yanking fields I don’t understand. :-]
Going down that delete path, we end up at this error anyway:

[code: Installer Installer::migrateStaticPagesToNavigationMenuItems]
<h1>DB Error: Unknown column ‘context_id’ in ‘where clause’</h>

Any thoughts? Is there maybe a better option for doing a ‘migration’ rather than ‘upgrade’. I’m happy to provide more info if needed. Thanks in advance!

Hi @vangordd,

Are you dropping and recreating the database before loading the database dump? If not, then the _migration tables might be left lying around even though the dump has been reloaded. I’d suggest dropping and recreating the database before reloading the dump.

Regards,
Alec Smecher
Public Knowledg Project Team

Hey, Alec. Thanks for the reply! I am dropping the database prior to the upgrade. FYI, I’m new to this effort and to OJS. I’m wondering if maybe the CentOS6 (source) database had been upgraded once before and this is a second upgrade. Would that cause this issue or is the upgrade process designed to be done time after time on the same DB?

Thanks again!

Hi @vangordd,

If you’re seeing any …_migration tables inside your pre-upgrade database dump, then unfortunately it’s likely that someone tried an upgrade, failed, and didn’t restore from backup afterwards.

Best case, they did restore from backup (but forgot to drop the database and re-create it), resulting in orphaned …_migration tables but having an otherwise consistent database. In that case you’d drop the …_migration tables before running the upgrade.

Worst case, your database is stranded somewhere between 2.4.5 and some undetermined version of OJS that someone tried to upgrade to once. That can be quite a difficult thing to clean up, mostly because you’re not sure what you’re working with.

The good news is that it seems like your 2.4.5 installation is working OK, which hints towards the former situation.

Regards,
Alec Smecher
Public Knowledge Project Team

Thanks again, @asmecher. I’ll see if I can’t dig in a bit more and clear out some of the extra temp data that happened during the half-upgrade. If I get a clean upgrade, I’ll be sure to update the post. Thanks again!

After much trial and error it appears I have a working upgrade. I figured I’d share in case it could help anyone else trying to migrate and upgrade at the same time.

This was moving from a CentOS 6 (server1) host to CentOS 7 (server2) host.

In short, there were 3 major things that needed to happen because I was moving from one system to another:

  1. Copy all of the ‘upload’ directory from server1 onto server2.
  2. Update to from v2.4 to v3.0 and then to v3.1
  3. Make sure SELinux was set properly.

Upgrade from 2.4 to 3.0:

  1. Download OJS 3.0 zip to server2.
  2. Extracted them to the proper directory.
  3. Built out the new database users and passwords on the server2 as well as granted proper access.
  4. Dumped the database from server1
  5. Piped the dump from server1 into my new database on server2.
  6. Copied all of the “files” directory from my existing installation into the “upload” directory on the new instance.
  7. Updated config.inc.php on server2 to have DB, DB user, and password
  8. Set all of the Apache Document root to be ‘httpd_php_exec_t’ SELinux type
  9. Set cache, upload, files, and plugins directories to be ‘httpd_sys_rw_content_t’ SELinux type
  10. Ran tools/upgrade upgrade
  11. Chowned everything to apache:apache
  12. Restarted apache.

Upgrade from 3.0 to 3.1:

  1. Basically the same steps as 2.4 to 3.0, but I decided to start over by taking a backup of server2’s ‘files’ directory and server2’s database dump.

I’m not sure if it was necessary, but I essentially deleted everything extracted and built in the 2.4 to 3.0 upgrade and started from scratch with the files in the 3.1 download. Instead of using the server1 DB dump and upload dir, I just used the local server2 dump and files.

While doing this upgrade, I did write a script for it if anyone needs to see that. Thanks!

Hi @vangordd,

Thanks for documenting this!

Was there a particular problem you worked around by going from 2.4 to 3.0, then 3.0 to 3.1? The upgrade scripts should be able to handle that in one step.

Regards,
Alec Smecher
Public Knowledge Project Team

Hey, @asmecher.

Thanks for asking! Just to be up front about this, I’m new to OJS and the upgrade process, so please feel free to correct me in any of this. I’m not too proud. :slight_smile:

After trying this again going from 2.4.5 to 3.1.1, it DID work, but with a few manual steps in between. The main thing was that the ‘files’ directory doesn’t exist on 2.4.5; there’s an ‘uploads’ directory instead. When the upgrade was running, I got several errors like saying that it could not find the correct files to rename. What appears to happen is that 2.4.5 → 3.0 moves ‘upload’ to ‘files’ and then 3.0 → 3.1 moves ‘files’ to ‘files’. Unfortunately, the browser doesn’t give a 404 when you go to download the file. You just get a blank screen. Also, I really wanted to keep SELinux enabled, so I kept bumping into Permission Denied errors that got worked through.

To successfully go from 2.4.5 to 3.1.1 (after your prompting) I did this:

  1. Downloaded the 3.1.1 zip and extracted its contents into my document root.
  2. Changed the extracted files’ SELinux context type to ‘httpd_php_exec_t’.
  3. Made directories ‘files’ and ‘reports’.
  4. Updated the context for ‘files’, ‘reports’, ‘cache’, and ‘plugins’ to be ‘httpd_sys_rw_content_t’
  5. Copied everything from server1 ‘upload/journals’ directory into server2 ‘files/journals’ directory.
  6. Took a dump of server1 DB and imported that into a fresh DB on server2
  7. php tools/upgrade.php upgrade
  8. chown -R apache:apache
  9. Toggled “installed” to “On” in config.inc.php
  10. Restarted Apache.

That seemed to do it. Of course, because I was migrated and not doing this in-place, I had to install Apache, Mysql, PHP, and get those all configured first.

Hope that helps. Please let me know if there’s a more appropriate SELinux context type for the directories I modified. Thanks!

-David

Hi @vangordd,

The location of the “files directory” (files or upload in your to configurations) is specified in config.inc.php in the files_dir directive, and it can be whatever you like. Just make sure that you don’t put it somewhere web-accessible or you may be victimized by malicious uploads! See the “Recommended Configuration” section of docs/README.

Regards,
Alec Smecher
Public Knowledge Project Team

Gotcha. Thanks for taking the time to reply, @asmecher. I see that you’re super busy, so it’s much appreciated.

In the end, it reads like my issues were mainly with SELinux then and my ignorance with OJS in general. I think we can mark this thread as resolved. Thanks again!

-David

Hi @vangordd,

Not a problem, glad it’s working. If you do have some recommendations for SELinux users, they would probably be useful to capture here – I don’t have any experience with that environment, myself.

Regards,
Alec Smecher
Public Knowledge Project Team

@asmecher, sure thing. Here is what worked for me:

semanage fcontext -a -t httpd_php_exec_t "$JOURNAL_DIR(/.*)"
semanage fcontext -a -t httpd_php_exec_t "$JOURNAL_DIR"
semanage fcontext -a -t httpd_sys_rw_content_t "$JOURNAL_DIR/cache(/.*)"
semanage fcontext -a -t httpd_sys_rw_content_t "$JOURNAL_DIR/cache"
semanage fcontext -a -t httpd_sys_rw_content_t "$JOURNAL_DIR/files(/.*)"
semanage fcontext -a -t httpd_sys_rw_content_t "$JOURNAL_DIR/files"
semanage fcontext -a -t httpd_sys_rw_content_t "$JOURNAL_DIR/reports(/.*)"
semanage fcontext -a -t httpd_sys_rw_content_t "$JOURNAL_DIR/reports"
semanage fcontext -a -t httpd_sys_rw_content_t "$JOURNAL_DIR/plugins(/.*)"
semanage fcontext -a -t httpd_sys_rw_content_t "$JOURNAL_DIR/plugins"

restorecon -r $JOURNAL_DIR

Again, hope this helps! :slightly_smiling_face: Thanks!

1 Like

Hi @vangordd,

Great, thanks – but please do make sure the contents of files can’t be accessed directly via the web server, or you’re putting yourself at risk!

Regards,
Alec Smecher
Public Knowledge Project Team

1 Like