Review Quality Collector (RQC) is an initiative for improving the quality of scientific peer review. Its core is a mechanism that supplies a reviewer with a receipt for their work for each journal year. I am its developer. The receipt is based on grading each review according to a journal-specific review quality definition.
What the RQC plugin is going to be
This plugin will be an OJS adapter for the RQC API, by which manuscript handling systems (such as OJS) report the reviewing data of individual submissions to RQC so that RQC can arrange the grading and add the reviews to the respective reviewers’ receipts.
Find the RQC API description at RQC API description
(not yet public as of today; wait a while; it is being co-developed with the plugin for validation)
What the plugin needs to do
The plugin extends the journal master data forms by two additional fields: rqcJournalID and rqcJournalKey.
If both are filled in the journal master data form, they are checked against RQC
whether they are a valid pair.
If they are accepted, they are stored as additional JournalSettings.
If these settings exist, the plugin will add a menu entry
by which editors can submit the reviewing data for a given
submission to RQC in order to trigger the grading.
This step is optional for the editors.
Depending on how RQC is configured for that journal, the given
editor may then be redirected to RQC to perform (or not)
a grading rightaway.
The plugin will also intercept the acceptance-decision-making
event and send the complete reviewing data for that submission
Where I need help writing it
I have worked my way through an OJS developer installation, through the OJS 2 TechnicalReference.pdf, and through some source code (in particular regarding JournalSettings and the Hooks mechanism).
I found the OpenAIRE plugin, which also extends a form and is hence helpful with that part.
BUT plenty of things are tough to find out in the source code in my view (and it does not help that I am a PHP beginner), so I need help with various details.
Please see my subsequent posts for these – I’ll restrict the present post to the introduction only.
The OpenAIRE plugin shows me how to add a field into the submissionsubmitstep3form: By registering a hook for (for instance) submissionsubmitstep3form::initdata. (I need to do this with the (or a) journal master data form.)
To understand the mechanism more fully, I searched for the point in the code where this hook will be called. It turns out the string submissionsubmitstep3form occurs nowhere else in the codebase; only in the OpenAIRE plugin.
Presumably, the hook name is constructed dynamically?
Where?
Question 2
What I really need is a hook to add fields to the journal master data forms.
I found controllers/grid/admin/journal/form/JournalSiteSettingsForm.inc.php, which looks promising, but does not appear to make any HookRegistry::call calls at all.
Nor are there suitable-looking call_hook calls in any .tpl file.
Is this so (or are they just hidden or are there additional forms what they can look like)?
Does this mean they will need to be added?
Is it problematic to add a number of hooks to OJS?
Are there any rules that guide this (naming, placement, types/purposes, …)?
@ajnyga: Your reference to the googleAnalytics plugin was a breakthrough for me. Thank you very much!
I have now implemented the settings dialog, which is the first of three parts of my plugin.
What I have learned:
To extend the journal master data by a plugin, one should not extend existing journal master data forms by dynamically adding fields to them. Rather, a settings dialog at the plugin level is much more appropriate: Not only is it far easier to understand from a user’s perspective, it is also easier to implement.
The generic plugins googleAnalytics, webFeed, or (soon) reviewqualitycollector are good starting points to learn how that is done.
In contrast to what I had expected, the additional data fields should not be stored in JournalSettings, but rather in PluginSettings because those are journal-specific as well. And again, the latter is even simpler to implement at the same time.
Now I need to catch a moment when all reviewing data is available, to send it to RQC.
There are basically three possibilities:
When the last assigned review comes in
When the editor says s/he would now like to send the data (and immediately rate the reviews)
When the acceptance decision is made
Number 3 is the most reliable and well-defined, so I started looking for suitable hooks.
I had previously taken note of interesting-looking hook names in the OJS 2 Technical Reference (TR) document: ReviewerAction::recordRecommendation, SectionEditorAction::recordDecision, SectionEditorAction::rateReviewer, SectionEditorAction::setReviewerRecommendation, SectionEditorAction::uploadReviewVersion, SectionEditorAction::viewPeerReviewComments, SectionEditorAction::blindCcReviewsToReviewers,
so now I looked for these in the code, but not one of these strings is found in the OJS 3 codebase.
ReviewerAction:: occurs only once, in the form ReviewerAction::confirmReview (which is 1 of 8 ReviewerActions mentioned in the TR); SectionEditorAction:: does not occur at all
Question 3
Could someone knowledgeable about the OJS2/OJS3 differences explain how to understand the disappearance of so many previous hooks?
Big cleanup?
Of what primarily, the roles?
Is there a heuristic what the new equivalent may be named if a previous hook has disappeared?
What I did find in the code instead (when looking for EditorAction:: and ignoring anything that is not a call to HookRegistry::call) is (when ordered process-wise): EditorAction::addReviewer, EditorAction::setDueDates, EditorAction::clearReview (removes a reviewer assignment), EditorAction::recordDecision,
and no others.
This appears to make sense for editors, but
Question 4
If for reviewers there is only ReviewerAction::confirmReview, then which is the hook for when a reviewer submits a review?
And how would/could/should I have found this out myself?
I’ll start working with EditorAction::recordDecision for the moment.
What is the recommended way to get an overview of the data model?:
list of entities
list of attributes for each entity
list of relatiionships for each entity
methods (if any) to access the attributes and to retrieve neighboring entity objects.
ojs_schema.xml is not overly informative.
I have found registry/journalSettings.xml, which appears to be (an approximation to?) an attribute list for Journal, which is helpful, but I don’t see anything similar for other important entiries such as submissions and review assignments.
(And to clarify my priorities: Questions 3 and 4 are only relevant for my overall background understanding, but are not really important; question 5, in contrast, is currently a roadblock for me.)
Hopefully @asmecher will comment more while he has the best overall view, but a few pointers.
Question 3: OJS was pretty much rewritten, including the way roles work, so that is probably the reason.
Question 3/4: the correct moment of giving a review rating and sending the information to RQC is of course something open to debate, but I would lean to the moment when the editor is thanking the reviewer.
After reading a review in OJS3 the editor has the opportunity to thank the reviewer with an email. Clicking the Thank reviewer link opens a modal window with an email template. I think that this same window would be ideal for giving a reviewer rating and sending the thank you message would be ideal for saving the review information to RQC.
edit: a couple of good hooks there would actually enable us to send review information to ORCID as well and possibly some other similar services like RQC.
Here is what is now sort-of working for me to explore the data model. It makes use of the fact that I do not need to create data, only use what existing OJS logic has created:
In config.inc.php, I added a setting activate_developer_functions = On.
If this is true, ReviewQualityCollectorPlugin::register performs HookRegistry::register('LoadHandler', array($this, 'setupSpyHandler')) to set up URLs that allow passing ?submissionId=2 for exploring the content of a particular data object
I work out the logic in the handler incrementally and dump the content of various data objects I want to understand, starting with, in my case, submissions.
This is not always easy, because finding stuff is often not straightforward. For instance, submissions are not called Submission (this class exists in PKP lib) or OJSSubmissions or some such, they are called Article (which I had expected to model published articles and had hence not looked at at first).
To find the relevant things anyway, one should definitely use an IDE, not just an editor, to explore the code. I use PhpStorm and it helps immensely. In the above case, tabulating the subclasses of Submission is the helpful operation.
To access an Article object, we need ArticleDAO and work our way elsewhere from there.
Looking at the list of methods of that class can be unhelpful, once you turn on listing the inherited methods as well, it can be overwhelming. Sort the list alphabetically and focus on the getter methods to get ideas.
Do not only try out calls, explore the code as well. For instance, ArticleDAO::getTitle wants a locale, but if you pass null instead, it will return the titles for all locales, which may be helpful.
You will have to guess which other DAOs may be helpful and explore those as well. For instance in my case, I needed to discover ReviewRoundDAO::getLastReviewRoundBySubmissionId and ReviewAssignmentDAO::getBySubmissionId to retrieve more of the data my plugin needs.
Be prepared that naming can be odd. For instance in ReviewRoundDAO, the promising-looking getCurrentRoundBySubmissionId will only return an integer, it is the above-mentioned. getLastReviewRoundBySubmissionId that returns the actual data. And apparently “current round” and “last round” are just the same thing. At least that is my understanding at the moment…
It sounds like you’re making good progress. We’re working to reduce the size of the codebase, and OJS 3.x makes big steps in that direction, but as it’s a codebase with more than a decade of maintenance there’s still work to be done e.g. on naming conventions, standardization, and de-duplication.
Note that we have a Doxygen reference, though it sounds like PhpStorm might accomplish the same thing.
Regards,
Alec Smecher
Public Knowledge Project Team
Adding an “RQC-grade reviews” editor decision button
In the past year I have now mostly figured out the relevant parts of the data model and am able to create a JSON object to be submitted to RQC. I’ll eventually (but not now) look into EditorAction::RecordDecision to do that implicitly when an acceptance decsion is made.
Right now I’d like to allow for explicit calls to RQC before decision time: OJS should send data to RQC, get back a URL to redirect the user to for grading, and RQC will redirect back to OJS after grading.
To do that, I need to intervene in EditorDecisionActionsManager::getStageDecisions to create an additional decision option object that will show up as a button “RQC-grade reviews” on the editor’s workflow page for the submission.
Q6: There is no HookRegistry::call in that routine for intervention. It appears appropriate to add one. Do you OJS senior folks agree?
Q7: Should I just submit this as a minimal pull request? How fast will that be accepted (just so I can be sure I am on the right track)?
Q8: The actual intervention would then use the LoadComponentHandler hook to register something like plugins.generic.reviewqualitycollector.components.editorDecision.RqcEditorDecisionHandler? Does that sound about right?
Q9: If all that modal dialog needs to do (besides showing some text) is to provide Cancel to do nothing and OK to make a full-page request to some other OJS page, where will I find code showing me how to do that without a form?
Grateful for any help. Reverse-engineering OJS is tough.
I’ll respond to your other post on this – it’s better not to post the same content twice, but if you do, please link the two together. That’ll help keep the forum organized.
Regards,
Alec Smecher
Public Knowledge Project Team