Describe the problem you would like to solve
We need to manage open peer review of the submission in the public journal website, made of submission import (if preprint is already published in a preprint servwer like OPS), immediate publication (or link to the preprint server) and publication of the review stage (all passages)
Describe the solution you’d like
We think that OJS/OMP should allow submitter to import manuscript from preprint server (at least by DOI), then OJS/OMP should publish it in a dedicated part of the journal website. Moreover, OJS/OMP should publish all the steps of the review stage. At the end, if the manuscript pass the peer review the final version should link to the preprint page with review stages, or all the data should be published in the article landing page e.g. in a separate tab called “review”.
Who is asking for this feature?
Milano University Press, Journal Editors, Journal Administrators, Authors, Reviewers
Additional information
Alternative solutions can be implemented, e.g. OPS could manage the open peer review with OJS/OMP importing the manuscript and review and starting the workflow from copyedit
Using the SWORD protocol, submissions can be turned from OPS submissions into OJS submissions. To do this, you’ll need two plugins:
In the OPS installation, the SWORD Client plugin (previously called the SWORD plugin), and
In the OJS installation, the SWORD Server plugin.
We are currently working on some refinements to these that’ll make them more user-friendly, but the basics are already there.
2. Presentation of unpublished content on the OJS website
To present content on the OJS website before it has formally finished the publication process, you can use the Forthcoming plugin (https://github.com/ajnyga/forthcoming). Unfortunately this is not yet compatible with OJS 3.3.0, I don’t think, but updating it would not be a tremendous amount of work.
3. Presentation of OJS reviews on the journal website
This can be achieved using a plugin or even a child theme plugin, which would augment the existing article display to show peer review content.
Does that sound like a comprehensive summary of the pieces that would be required, @bolelligallevi?
Regards,
Alec Smecher
Public Knowledge Project Team
Hi @asmecher, thanks for your answer.
On point 1, thanks for insight on OPS to OJS communication, I have not tried OPS yet but I think the refinements you are working on match the possibility to have an open peer review path involving the two apps.
On point 2, open peer review involves the manuscript before and during the publication process (precisely the review stage), i.e. the preprint, while the forthcoming plugin shows online firsts, i.e. published version before issue publication (maybe @ajnyga can give more details).
In other words, OJS content I need to be published is:
a) the submission manuscript, as provided by the submitter or transfered by OPS
b) all the data (reviewers, decisions, manuscript versions…) of the review stage
Is that possible now?
On point 3, can you confirm that a plugin or a child theme could show all the data I mentioned before in b)?
Finally, trying to be more clear on requirements, I would like OPS/OJS could manage open peer review in a similar way to https://open-research-europe.ec.europa.eu/
That could mean (on a future development point of view) that OJS and OMP should manage manuscript and review data publication by default as open peer review features.
Yes the plugin mentioned is meant for publishing articles before they appear in any issue. So it is not for preprint at all. On the background an issue labeled as “Forthcoming” is still used. The plugin also takes advantage of versioning meaning that version 1 is published in the forthcoming issue and version 2 is then scheduled to the actual issue.
Note that there are always ways the OJS might change that will affect how the plugin works. For example there has been discussion on whether the issue data should be stored in the submission level instead of the publication level. This would affect the way the plugin works. I have not yet considered how it would work after such changes.
The reason I still use issues in the background is that OJS requires an issue for a published article. It is difficult to show article data using the default article handlers without it, impossible that last time I tried. Also things like statistics and DOI registration would require some own solutions in the plugin if there would be no issue set when the article becomes public. With DOIs I have not yet checked the latest main branch of OJS on how it behaves on this.
While the plugin uses the official publishing action in OJS there are limitations on which stage you can publish it (I think you need to be in Copyeditin at least?).
It would be easy to reveal article data using a plugin with its own handler. The Forthcoming plugin used to do this (see GitHub - ajnyga/forthcoming at ojs-3-1-2). As you can see, the article was made public without actually publishing it just by marking it as forthcoming. The plugin then created a public handler and fetched all submissions that had that mark. But as mentioned above, the downside was that you lost statistics and DOIs, probably something else as well.
The big thing to solve would be statistics and DOIs.
With statistics the problem is that the stats features in OJS search for a particular kind of url paths. The paths are used to recognise what was loaded. When you have your own handler path, the stats actions will not notice those in the log file. I know that the stats have been rewritten, so not sure if that could be affected now. But you would also have to solve how to show those stats, because they would be separate for the preprint and for the article.
With DOIs the problem is that for a preprint you would require a separate DOI. It can not bee the same as the one for the final article. So you would have to have an inbuilt action in the plugin for creating and storing DOIs to the metadata AND registering it when the preprint is made public. From the top of my head I would say that this is actually the most work in the plugin.
Hi @ajnyga, thanks for your hints.
What if open peer review of preprint whould be managed in OPS?
I think that this would solve all the problems you mention: it would be possible to assign a DOI there, it would be possible to avoid problems related to issues, it would be possible to develop stats related to preprint in OPS (if not already developed).
In a workflow OPS->OJS, the open peer reviewed preprint could be published in OJS jumping the review stage, with a separate decision (e.g. “already reviewed”) or using the possibility to skip review already built in.
Finally, this solution is probably similar to current best pratice: open peer review of preprints hosted in preprint servers is managed separately and previously from submission to journal.
Best regards
Stefano
Hi @ajnyga, thanks for your answer.
Is not clear to me if OPS can assign DOIs to preprints, do you have any hint?
There is also the problem of review, I think it is missing from OPS now, am I right?
So the problem is: adding review to OPS (maybe a review only devoted to open peer review) is more or less effort than apply changes to OJS to manage early publication of submission files? @asmecher, can you help on this? Below the request rewritten using OPS for preprint DOIs and open peer review.
Best regards
Stefano
Describe the solution you’d like
We think that OPS could assign a DOI to the preprints and then manage the review (assigning DOI to review itself should be considered),publishing all review data.
Then preprint could then be passed to OJS/OMP by sword, and OJS/OMP should publish at least a link to the preprint and open peer review in OPS.
These are examples of plugins that use 3rd party commenting but of course you could create one that has it’s own db tables in the OJS installation. It really depends on what kind of commenting you are thinking of.
Hi, we are also working on a solution for open peer review in OPS.
Our idea is to use the existing sumbmission process for pre-prints also for reviews. So the reviews can enjoy all the features of other kinds of submissions, such as getting a DOI and being referenced in ORCID. The difference is that submitted reviews have a pointer to the reviewed preprint.
We are currently developing a plugin to implement the new type of submission. But I think it would be fruitfull to think of a general open peer review solutiion for OPS/OJS/OMP, as this gains more and more traction.
For a start I added a new field during the submission process with the id of the preprint. This is stored in the “submission_settings” table under the setting_name “isReviewOf”. However, I wonder if this is the correct table to store these relations? Also, I think this information belongs to the metadata section, but I have not so far found an example on how to create custom meta-data fields in OPS/OJS/OMP 3.4: Any ideas where to look are appreciated
Yours,
Felix
You might be interested in COAR Notify, which aims to implement a decentralized approach to adding reviews to preprint repositories.
We also have support for SWORD, as I mentioned above, which could be build on to migrate posted preprints (in OPS) into the review workflow in OJS. The review workflow could be exposed there via the public reviews plugin (which exposes the results of the existing OJS review workflow).
One of the core principles in OPS is “we don’t do reviews here”, so I’d be nervous about merging a change like you propose into the core product; it could be maintained as a plugin, I suppose, but in the past large-scale plugins providing uncommon workflows have been hard to maintain.
Another approach, I suppose, would be to embed a public commenting toolkit onto the OPS front end – using one of these tools, for example.
(In OJS 2.x we used to have our own homegrown commenting toolkit for public comments on articles – arguably an implementation of open peer review. But it was undermaintained and a good example of the kind of feature that’s best implemented through integration rather than implementation.)
Hope that helps!
Regards,
Alec Smecher
Public Knowledge Project Team
Thanks @asmecher for your thorough reply
As always, there is so much more to consider that one does not think of when one starts a project.
Actually we intend to offer two ways for users to react to a preprint: One will be in the way of comments. For this we will include hypothesis (but some of the commenting tools you linked to also sound great). The other will be in the way of full fledged open peer-reviews. To provide an incentive for reviewers, we want to assign DOIs to these reviews and make them accessible as scientific output in their own right. Therefore we had the idea of just using the same submission process as for the preprints themselves.
The public preview plugin for OJS sounds great though! However, we would have to solve the problems DOIs and related metadata
I wonder, if one would combine this with continuous publishing, why would one would need a specialized product for pre-prints such as OPS?
thanks for sharing your considerations about peer review.
I think @felixhelix addressed two huge problems:
publish OJS reviews is not enough: open peer reviews need DOI (moreover: crossref DOIs) and qualified metadata;
when OJS becames capable to manage open peer review, why let the publication of the preprint (i.e. the last step for a fully open workflow, what you call “continuous publishing” if I understood well) to a different software (i.e. OPS)?
Here is our (Milano University Press) point of view: in a strategic development perspective OJS (and OMP) should became THE software to manage all the publication workflow in a fully open way. That means, for example, that should be possible, and fully integrated in the core system:
to publish open access any manuscript as soon as received as preprint (assigning a first preprint DOI)
to manage openly the peer review process (as open research europe does, not only publishing the peer review after has been done) including the publication of the postprint after the peer review process is ended.
OPS can still be usefull for users that have partially closed workflows, or the need to have only one preprint server or a preprint server separated from the publishing softwares (PKP or not), or other scenarios.
We hope that this point of view can be further discussed and that as amany publishers as possible will participate; @asmecher and PKP further considerations are needed and welcome.
In a operative perspective, we will test public reviews plugin as soon as possible, and we think that it should be investigated the effort needed to make the DOI plugin able to include peer reviews. These would be enhancements of the actual OJS features regarding peer reviews regardless any strategic consideration.
Regarding the preprints commenting/peer reviewing, we think that PKP core principle of not doing reviews in OPS is still valid only if OJS will became capable to manage the full open publishing workflow outlined before. Preprint commenting is not enough anymore in a fully open access publishing environment.
Best regards
Stefano
Thanks for the thorough comments – this is very helpful.
This is also touching on the concept of review articles, which historically were for things like literature reviews for content likely published elsewhere – but the same feature set could be used to review content within the same installation, of course. There were previously a few plugins to accomplish this (first Books For Review, then Objects For Review) in OJS 2.x – but they were not carried forward to 3.x (yet). The functionality they provided (IIRC) was to allow the author to identify the literature being reviewed and enter metadata, and I believe also for the editors to invite a review.
Hi @asmecher, thanks for your reply, I hope @felixhelix (and others) will add comments, I think open peer review is critical for the future of open access publishing.
Regarding “continuous publication”, maybe I used the phrase in a wrong way because is connected with version of record, while I wanted to point out that OJS/OMP should manage earlier versions of articles/chapters/books in an open access perspective: talking about articles, in online publishing an increasing number of commercial publishers are starting to publish accepted manuscripts (e.g. Wiley is adding AAM to online firsts, or “First edition” AAMs of Blood Advances in Science Direct like this). Talking about books, my university press has published a postprint of a serie’s book in Zenodo linking to it from OMP: we had to manage the hiding of DOI and chapters (leading to this post), but we thought OMP was an important place to give the notice of the postprint even if not ready to properly manage earlier versions of books.
Best regards
Stefano
We did run into terminology questions during our sprint work; we were hoping that someone like NISO would maintain a set of definitions of these terms but our outreach to them indicated that they did not. If anyone is aware of a good set of accepted, community vetted definitions of terms like Continuous Publication and Online First, I’d love to see them!
Thanks,
Alec Smecher
Public Knowledge Project Team
It seems that there are a lot of options in the wild! The most salient for us are
the objectsForReview Plugin (OJS)
the publicReviews Plugin (OJS)
Scielo (OPS)
a propriatary Plugin developed for our specific needs (OPS)
All have their merits and shortcommings.
The question for me at this point is: what way will pkp go to include open peer review, and which products will be supported? As @bolelligallevi confirmed, this will be more and more asked for. I see open peer review as fully compatible with preprints. While it could well be handled with a plugin, I think it should be developed in close coordination with PKP, as it touches on a lot of issues.
I will meet with my team over the next days to discuss the different options. For now the development of a proprietary plugin for OPS seems the best choice to me. My idea is to base it on the quickSubmit plugin, as we do not want reviewers to go through the full submission process.
To continue on my last post: In the plugin we envision, reviews will be articles/preprints, can get a DOI and be connected to ORCID. In their metadata, their relation to a preprint (hosted on the same server) will be noted. This will allow to create links on the respective preprint detail page, but also to exclude reviews from being listed as preprints and being searched for.
