Import Export a complete journal data in OJS3

[hcl]

Hello,

At first, I wish all of you the best for this new year.

At university of Bordeaux we host several journals in one instance of OJS: currently OJS 3.2.1.3.

There are very different and distinct journals. They publish not in the same field, they have not the same URL, they have their own plugin theme, they don’t share users.

We found convenient to have them in the same instance of OJS essentially for system management reasons. We have only one DB to maintain, one OJS instance to maintain. So when there is an update of OJS to make, migration is realized once.

But on the other hand, if there is a problem with a specific journal, if we want to restore data of a specific day for a specific journal wihtout affecting data of the other journals, or if a journal wants to live its own life separately in an other instance, we can’t export and import the complete data of this one specific journal easily.

When I looked for information on this forum to know if it’s possible to import/export a complete journal in OJS3, I understood it’s not currently possible but there are works in progress that are not available.

Indeed, I read all the threads for Extend native import/export plugin to include additional entities

Can you tell me if I have all understood well ?

In OJS3, it’s not possible to import/export a complete journal into an existing install.

It existed a plugin in OJS 2: fullJournalTransfer, but it’s not planed to support OJS3 in the near future :

In OJS3 there are 2 plugins which can help: Article and Issue export/import and User export/import. But this will not transfer the configuration and workflow. The XML import/export plugin uses the filter framework, which essentially allows each entity to have a matching conversion filter from XML to XML.

In 2018, an issue was opened to plan to extend native import/export plugin to include additional entities: Not-yet-published files, Review assignments, Discussions/queries, Log entries, Journal configuration/settings.

It’s a long-term project, a major feature not already assigned, and the purpose is to achieve it by aligning the import/export tools with an API rather than trying to maintain parallel implementations. PKP hasen’t made steps to improve the coverage of the XML, but want to continue building out the REST API, then add some better command-line tools for interacting with the API outside the UI.

In the same time, an individual thesis project has been carried out where the main goal was to extend the native plugin so it could export/import more information. The thesis NativeXMLExtension has been published and the code for the project is available. This is an independant work.

In our case, what do you suggest to do if we want to easily export/import a complete journal with all data if we want to restore data of a specific day for a specific journal wihtout affecting data of the other journals, or if we want to migrate a journal separately in an other instance ?

Be patient and wait for a future release of OJS which will include plugins and API which will be able to do that ? If so, in how many time do you think it will be available ?

Knowing that our journals are very different and don’t share data or users, to have separate instances of OJS for each of our journals ?

Thanks in advance for your answer.

Best regards.
Helene

[hcl]

Hi @rcgillis,

It’s exactly the link I mentioned and I analysed.

Beyond the work for extending native import/export plugin to include additional entities, we wonder a strategic and technical question :

Even if OJS allow an installation with multiple journals with one shared DB, if we host like us very different journals which don’t share data or users, is it better to have separate instances of OJS for each journal ?

Thanks again for your answer.
Kind regards.
Helene

[rcgillis]

Hi @hcl,

Sorry about that. I missed that when looking for relevant issues.

I know of organizations that do their OJS hosting that way (i.e. run individual instances of OJS for each journal). While it does offer some advantages (e.g. keeping some data distinct and separate from other journals), it does mean that you are responsible for maintaining multiple instances of OJS (e.g. applying patches, upgrades, etc.), which may consume more resources. So, while it is possible, there are both advantages and disadvantages, in my opinion.

-Roger
PKP Team

[NateWr]

Hi @hcl,

Your description of the work done so far, and the work still remaining, is correct as far as I’m aware. Getting to a full journal import/export is a huge task. We want to do it, but we’re still a long way from there. In the meantime, I think we have added some additional data to the XML import/export tools and may add more over the coming releases. Nothing specific is scheduled, though.

In most cases like yours, the recommendation is to do the import/export for published material, and manually bring across submissions that are still in the workflow. A backup of the database/files can preserve the editorial activity of the journal before migration for the historical record, in case this is needed. It’s not ideal, but it is probably the best that can be done for now to extract a journal from a multi-journal instance.

[hcl]

Hi @rcgillis, @NateWr,

I still had a few questions.

1. Would it be possible to consider having a single installation of OJS with several databases, one per journal ?

2. I read there is also now an official docker images for OJS 3.1.2 or greater with a gitHub repository.
   Is such installation allows a separate db-container for each journal (multipleDB) ?
   This repository is still beta ? When is it planed to release a stable version ?
   Maybe @marc could answer to this question ?

3. From what I read in the FAQ of the service of hosting which PKP offers :
   Journals that are hosted via the Basic plan are on a shared OJS installation with other journals.
   For specific needing, it’s Professional hosting plan which includes a dedicated OJS install for the journal.
   From the PKP experience of hosting multiple instances of OJS, do you have a system for maintaining easily those multiple instances of OJS with multiple versions ?

Thanks again for your answers.
Best regards
Helene

[marc]

Would it be possible to consider having a single installation of OJS with several databases, one per journal ?

Yes, you can. This is was the base of a deprecated project called “mojo”.
One single ojs code with symbolic links on each journal.
You can find a detailed explanation here:
https://pkp.sfu.ca/wiki/indexbb49.html?title=Installation:_Multiple_OJS_%26_mOJO#How_mOJO_works

But, this approach is old have some downsides that are resolved in docker.

Is such installation allows a separate db-container for each journal (multipleDB) ?
Yes, you can use the project to build a multi-journal or a single-journal installation.

The docker-ojs project releases a docker image 24-48h after every ojs release, but also a docker-compose.yml file that let you build 2 containers that talk to each other. First include an OJS instance (over a linux/apache/php stack, with all the necessary modules and plugins) and the second is a mariaDB.

You can easily modify this docker-compose or create your own to work with mysql, postgress or whatever you like.

Although it could lead to extra maintenance work (we are working on scripts to manage it easier) I prefer keep my journals isolated. You have a list of pro cons somewhere in this forum (let me know if you don’t find it)

This repository is still beta ? When is it planed to release a stable version ?

I keep it as “beta” because we need more feedback about how is this working (and this won’t change if people don’t adopt it) but I have been eating my owns dog’s food (with 50 jounrals) during last year and the project is stable.

I mean, I have 50 journals (2 millions visits a each year) running in single-tenant dockerized OJS over a 32 RAM server with 8 cpus and the system is very stable.

From the PKP experience of hosting multiple instances of OJS, do you have a system for maintaining easily those multiple instances of OJS with multiple versions ?

I also love to hear about this.

Take care,
m.

[marc]

@hcl you will probably like to take a look to this thread, that includes de pro/cons table about the single-multi-tenant decision:

Hardware requirements for more than 50 Journals

Dear all, as we plan to set up an OJS-infrastructure for supporting a huge amount of journals, we wonder if there is some experience about running such an infrastructure. As we like to set up virtual servers each with: OS: current SLES Apache 2.4 PHP 7.x with fcgi 4 CPU 8 GB RAM 50 GB System-Storage NFS-mounted Storage for the journal data (DB on System-Storage) Our questions are: is this setting valid for our plans? how many journals can be operated within in such an setting in one …

My opinion (and I emphasise it’s an opinion) it’s more or less summarized here:

Reducing the number of journals in a multi-journal instance

Hello ojs_univie, This is a question that I have been thinking about for a long time and to which I would like to arrive at a universal answer… but I think that in the end it all depends on the context in which you are, the resources and your team preferences. I will try to answer your questions quickly, but feel free to ask further is something is not clear enough: 1. Has anyone in the community done something like this before? What has been your experience? Yes. There are several instituti…

About docker project, I hope the readme in the docker-ojs repo to be clear enough (if not, please let me know to improve it).

I you need more context, a diagonal reading of this thread (that was where the project started) could help:

Docker4ojs: A set of docker images for OJS

Hi, It’s a fact that Docker and Vagrant are today mature enough to be taken in consideration. I don’t want to start here a discussion about which one is better… both have their lights and shadows, but I like docker I started a few experiments to see how useful is for DEVELOPMENT but also for PRODUCTION environments. When I started I noticed two existing projects, although a little bit outdated, help me to understand how to start: Docker Hub Docker Hub If the developers are here, they are v…

Cheers,
m.

[NateWr]

Thanks @marc for a very helpful answer. @hcl I’m not involved with our publishing services but I’ll see if someone from the systems team can help with your last question.

[hcl]

Hi @marc,

Thanks for your complete and well-documented answer.
It will be very helpful for our IT Team. We keep in touch.

Thanks also to the PKP team.

Best regards.
Helene