Open reviews request

piotreba · May 24, 2015, 7:56pm

Hello,

I would be happy to see in the next OJS versions solutions for conducting open reviews. I think that more options for review schema should be offered especially nowadays when publication ethics and transparency have been paid more and more attention to this crucial step of the editorial process.

Peter

asmecher · May 25, 2015, 5:52pm

Hi @piotreba,

We’ve generally got a few community members tinkering with open reviews, and they might be interested to know what kind of open review you had in mind. Could you describe it a little more specifically? OMP also does support open reviews, and that code is being used in OJS 3.0, so this will also be available in that release.

Regards,
Alec Smecher
Public Knowledge Project Team

piotreba · May 25, 2015, 6:52pm

Alec,

I am thinking of the following general schema: the initially accepted manuscript is being published for a given period of time (e.g. set by a section editor) and is exposed for open commenting by non-anonymous reviewers. Reviewers would have to give his/her details obviously. Section Editor should have ability to invite reviewers as well and to comment on the manuscript. It should be possible to set custom review form, as it is currently possible. Reviews would have to be available along with a published paper (and archived via, e.g. PKP Public LOCKSS as well).

This is a kind of general view. Further details would have to be discussed.

Peter

asmecher · May 25, 2015, 6:53pm

Hi @piotreba,

What you describe has already been implemented in OMP and release in OMP 1.1.1, with the exception of review forms, which will be released in OMP 1.2. OJS 3.0b will include all of those features. I’d suggest testing with OMP 3.0b when it’s available, and watching out for OJS 3.0.

Regards,
Alec Smecher
Public Knowledge Project Team

piotreba · May 25, 2015, 6:57pm

That’s cool Alec. Looking forward to see it.

Best,

Peter

joris · September 30, 2015, 4:14pm

Hi there,

There’s arguably another definition of open review which is the reviewers know who the authors are, and the authors know who the reviewers are. Is it feasible on OJS 2.4? So far I have only seen options to blind the authors names.

Thanks

Joris

asmecher · September 30, 2015, 4:17pm

Hi @joris,

That’s not currently available in OJS 2.4.x, but is already coded in OJS 3.0. If you’d like an easy work-around for OJS 2.4.x, you could include the identifying information in the email communication between reviewer and author, since that communication is always mediated by the editor.

Regards,
Alec Smecher
Public Knowledge Project Team

joris · September 30, 2015, 4:23pm

Thanks Alec. I was thinking of asking reviewer to sign their comments if they wish to disclose their name. If they put their name in their comments, that should get passed on to the authors, right?
It would only work on a voluntary basis and won’t automatically add reviewers’ names to the published article, but that’d do for me for the moment. Looking forward to v3!
cheers

Joris

asmecher · September 30, 2015, 4:35pm

Hi @joris,

All communication from reviewer to author is mediated by the editor, so they’d have a chance to modify the email contents before they were passed on. But yes, the reviewer could include their name in the comments; then the Editor could use the “Import Peer Reviews” button to bring in the review contents; then anything there would be passed along to the Author, with maybe a little grooming and fine-tuning of the wording by the Editor.

Regards,
Alec Smecher
Public Knowledge Project Team

piotreba · October 3, 2015, 7:02pm

So I am now testing OJS 3.0b. However, I am not able to set open review for anything more than allowing the reviewer to see whose paper he/she is reviewing, and vice versa. What about allowing for open public review, as was declared to be implemented in this version of OJS? Am I overlooking something?

asmecher · October 5, 2015, 9:33pm

Hi @piotreba,

OJS 3.0 will (as OMP 1.2 already does) support “open reviews” in the sense that the editor can decide to reveal reviewer and author identities to the other party. “Open reviews” in the public/anonymous/etc. sense aren’t yet implemented. I suspect we’ll probably implement this using Hypothes.is or something similar, as that is an interesting option for document-centric workflow but doesn’t yet support the kind of access controls that would be needed for traditional single- or double-blind peer reviews. One of our community members has apparently already integrated Hypothes.is in this way, though I don’t have many details.

Regards,
Alec Smecher
Public Knowledge Project Team

Sleipnir · April 21, 2016, 12:16pm

Hey,
are there any news on Open Review in OJS?
I think it is a great thing. A good example in my eyes is F1000Research (http://f1000research.com/).

asmecher · April 22, 2016, 7:41pm

Hi @Sleipnir,

Nothing specific – can you describe what model you’d hope to be able to use, and what your reader community looks like? We’re definitely interested in open reviews but so far haven’t seen a clearly desired single model arise. Several high-profile experiments with open reviews of various sorts have failed to get the kind of participant engagement that journals need to switch over wholesale, so we’re concerned about investing a lot of development time while a viable model emerges as a clear leader.

Regards,
Alec Smecher
Public Knowledge Project Team

Sleipnir · April 25, 2016, 2:02pm

Hi Alec,

yes, I can imagine that implementing of open peer review within OJS will cost a lot of time for developing. If it is better to wait for leader experiences (and I think those with F1000Research would be quite excellent), what about the opportunity for registered users to comment journal articles by using their real names and to make these comments citable by using DOIs? An important requirement for this would be a check/verification of registrants by the journal managers or editors before sending a confirmation email with access and password.

piotreba · April 25, 2016, 3:53pm

I can imagine that’s challenging, but in my opinion this is a required step towards improving publication quality, especially nowadays when publication transparency is becoming more and more important issue. How could we contribute to establish such model?

asmecher · April 28, 2016, 5:08pm

Hi all,

More information on exactly what model of open peer review you’re interested in would be helpful. I can’t promise we’ll be able to launch an experiment soon, but this has long been a subject of interest for us. We often support experiments like this via specific grants, and if we come across the right opportunity, some interested parties would also be helpful.

Regards,
Alec Smecher
Public Knowledge Project Team

Linda_Vanasupa · November 9, 2016, 5:12pm

Hi,
With respect to an “open review” process, I see that someone has already referenced the F1000 Research platform, which I understand is totally unique.
This platform requires totally transparency on everyone’s part:

The manuscript goes live after a check for meeting the criteria non-duplication and so on, with the author’s names.
Then, reviewers self-select by signing up with their real name (not “user names” ). The reviews themselves are posted live as they are completed. There is a stable URL link to the review itself so that the reviewers are also able to reference their review in service to their discipline.
The authors’ response to the review is itself also transparent;
When the manuscripts are recommended for publication by the reviewers, they are published to a stable DOI index after the Editor “approves” (I think);
Again, all of this happens transparently on the journal web interface.

This process dramatically shifts everyone’s behavior, as you can imagine.