[OJS 3.0.2] CrossRef XML Export Plugin: Status not refreshed properly

ojs3

#1

Hi! Our journal managers say there is some issue with CrossRef status. Mostly they see it as ‘submitted’. Whereas actually some DOIs work OK and some of them may have some errors.

I checked several journals. One of them has either ‘submitted’ (most of the articles) or ‘active’. All the DOIs seem to resolve OK (including ‘submitted’). Another one has only ‘submitted’ articles, while all the DOIs are actually OK.

So I think there really might be some issue with status refreshing.


Automatic CrossRef registration fails
#2

Hi @Ph_We

How did you deposit/registered those articles – do you use the automatic registration with a Cron job or Acron plugin? – If using the automatic deposit to Crossref, the statuses should be automatically updated. Else, one has to use the button “Check status”.
What happens when you click on “Check status” for such an article that has status = submitted, but the DOI is active?
Sometimes the Crossref deposit API does not return the correct status :frowning: Could you thus also test this: What do you see as the last status when you call this Crossref deposit API function with such a DOI as parameter: https://api.crossref.org/deposits?filter=doi: ? (You could also post me the whole result, if possible). If this is the case, it would be good to contact Crossref support.

Best,
Bozana


#3

Hi @bozana,

Thanks for your response. We use the default settings, so it should be Cron in all cases. They tried the button “Check status”, but it did not refresh anything (I checked this too).

As to the CrossRef API, I will ask one of the users to check this.

Best,
Ivan


#4

Hi @bozana,

Here is the API call result for the case when the status is displayed as ‘submitted’, but should actually be ‘active’:

{"status":"ok","message-type":"deposit-list","message-version":"1.0.0","message":{"items-per-page":20,"query":{"start-index":0,"search-terms":null},"total-results":2,"items":[{"handoff":{"status":"completed","delay-millis":2718.2818284590453,"try-count":1,"timestamp":1498817276487},"dois":["10.17323\/727-0634-2017-15-2-333-340"],"parent":null,"matched-citation-count":0,"filename":null,"submitted-at":"Fri Jun 30 06:07:56 EDT 2017","status":"submitted","length":6904,"content-type":"application\/vnd.crossref.deposit+xml","pingback-url":null,"citation-count":0,"test":false,"owner":"hse","batch-id":"567b2eae-e8d1-4d3c-93a4-4f919e3e2de9"},{"handoff":{"status":"completed","delay-millis":2718.2818284590453,"try-count":1,"timestamp":1499279534761},"dois":["10.17323\/727-0634-2017-15-2-333-340","10.17323\/727-0634-2017-15-2-323-332","10.17323\/727-0634-2017-15-2-309-322","10.17323\/727-0634-2017-15-2-297-308","10.17323\/727-0634-2017-15-2-281-296","10.17323\/727-0634-2017-15-2-267-280","10.17323\/727-0634-2017-15-2-251-266","10.17323\/727-0634-2017-15-2-235-250","10.17323\/727-0634-2017-15-2-217-234","10.17323\/727-0634-2017-15-2-201-216","10.17323\/727-0634-2017-15-2-183-200"],"parent":null,"matched-citation-count":0,"filename":null,"submitted-at":"Wed Jul 05 14:32:14 EDT 2017","status":"submitted","length":64523,"content-type":"application\/vnd.crossref.deposit+xml","pingback-url":null,"citation-count":0,"test":false,"owner":"hse","batch-id":"975590cb-7695-4709-bd53-76c9a43dde67"}]}}

This is what I get for the article, which status is displayed as ‘active’:

{"status":"ok","message-type":"deposit-list","message-version":"1.0.0","message":{"items-per-page":20,"query":{"start-index":0,"search-terms":null},"total-results":1,"items":[{"handoff":{"status":"completed","delay-millis":2718.2818284590453,"try-count":1,"timestamp":1493085969595},"dois":["10.17323\/2411-7390-2017-3-1","10.17323\/2411-7390-2017-3-1-102-105"],"parent":null,"matched-citation-count":0,"filename":null,"submitted-at":"Mon Apr 24 22:06:09 EDT 2017","status":"completed","length":2904,"content-type":"application\/vnd.crossref.deposit+xml","pingback-url":null,"submission":{"messages":[{"message-types":[],"message":"Successfully updated in handle","related-doi":"10.17323\/2411-7390-2017-3-1","status":"success"},{"message-types":[],"message":"Successfully updated","related-doi":"10.17323\/2411-7390-2017-3-1-102-105","status":"success"}],"failure-count":0,"warning-count":0,"success-count":2,"record-count":2,"batch-id":"632b21c6-6ac7-4e0e-a9bd-3a2c6aa65ab2","submission-id":"1404838822"},"citation-count":0,"test":false,"owner":"hse","batch-id":"632b21c6-6ac7-4e0e-a9bd-3a2c6aa65ab2"}]}}

I do not know how to read this. But both results have different values for statuses…


#5

Hi Ph_We,

The first result have status=submitted and the second status=completed.

Yes, then it is the Crossref deposit API that does not return the correct status :frowning: That API is unfortunately not “in real production” by Crossref, but currently there is also no other possibility for automatic deposit and their status check :frowning: Crossref is currently working on a new version of the API and as soon this is in production we will use that.
Maybe you can contact Crossref support for now?

Best,
Bozana


#6

@bozana, thank you very much for your help!
We will try to contact their support.


#7

Hello Bozana,

Can you update the status of the new CrossRef API as our 2.4.8 installation of OJS also does not report updated status. I have also queried CrossRef support for their status. Thanks.


#8

Hi @radjr

Everything is the same i.e. it is still the same asynchronous Crossref deposit API that is used in OJS – the new deposit API is still not public/in use.
Also, currently we are integrating the new deposit API first with the OJS 3.x branch.

Have you tested the status using the API URL above – to double check if the problem is on the Crossref or OJS side?

Best,
Bozana


#9

Is there any solution?
a have 3.1.1.2 and described problem still present

for example:
here is article
https://uars.info/index.php/uars/article/view/12
DOI link: https://doi.org/10.32420/1996.1.12
DOI number: 10.32420/1996.1.12
Answer from api.crossref.org: {“status”:“ok”,“message-type”:“deposit-list”,“message-version”:“1.0.0”,“message”:{“items-per-page”:20,“query”:{“start-index”:0,“search-terms”:null},“total-results”:0,“items”:[]}}
Status in OJS: Submitted
mark active button works. Check status button doesnt have any result


#10

Hi @redukr

Could you tell me what you see when calling this Crossref API function/endpoint: https://api.crossref.org/deposits?filter=doi:10.32420/1996.1.12 ?
Then I will maybe have further questions, but for now…
To try to localize the problem source…

Best,
Bozana


#11

@bozana, thanks for answer!

{“status”:“ok”,“message-type”:“deposit-list”,“message-version”:“1.0.0”,“message”:{“items-per-page”:20,“query”:{“start-index”:0,“search-terms”:null},“total-results”:1,“items”:[{“handoff”:{“status”:“completed”,“delay-millis”:2718.2818284590453,“try-count”:1,“timestamp”:1542705173372},“dois”:[“10.32420/1996.1”,“10.32420/1996.1.15”,“10.32420/1996.1”,“10.32420/1996.1.14”,“10.32420/1996.1”,“10.32420/1996.1.13”,“10.32420/1996.1”,“10.32420/1996.1.12”,“10.32420/1996.1”,“10.32420/1996.1.11”,“10.32420/1996.1”,“10.32420/1996.1.10”,“10.32420/1996.1”,“10.32420/1996.1.8”],“parent”:null,“matched-citation-count”:0,“filename”:null,“submitted-at”:“Tue Nov 20 04:12:53 EST 2018”,“status”:“submitted”,“length”:19558,“content-type”:“application/vnd.crossref.deposit+xml”,“pingback-url”:null,“citation-count”:0,“test”:false,“owner”:“uarr”,“batch-id”:“fe660ba3-9df1-4956-a968-2d401661b40e”}]}}

thats what i receive


#12

Hi @redukr

I think this is again at the Crossref site – the status for that submission seems not to be updated at the Crossref site/deposit API yet – s. the part ...“submitted-at”:“Tue Nov 20 04:12:53 EST 2018”,“status”:“submitted”... i.e. the status seems to be “submitted” here.
One another thing that I am wondering about: it seems like you have submitted it today, Nov 20th ? Then maybe to wait a little bit and check later or latest tomorrow…
If it is on the Crossref site, maybe you could contact the Crossref support and explain that you have submitted it and the DOIs seem to be active, but that this deposit API does not update the status…

Best,
Bozana


#13

Hi @bozana I receive answer from crossref, but i amnot sure that i understand what i should wrote to them:

"Hello,
Many thanks for your email.
The DOI is showing as being registered with us as you can see from the metadata here: http://doi.crossref.org/search/doi?pid=support@crossref.org&format=unixsd&doi=10.32420%2F1996.1.12
Can I ask you to elaborate further on your query and I will try and help you with this.
Best regards,
Paul Davis
Product Support Specialist
@CrossrefSupport
"


#14

Hi @redukr

Hmmm… Did you write them that the status of the last deposit is still “submitted” when using the Crossref deposit API i.e. https://api.crossref.org/deposits?filter=doi:10.32420/1996.1.12 ? – Because the support person is pointing to another API call i.e. not mentioning the deposit API at all…

Best,
Bozana


#15

@bozana i have ticked opened for 6 days, at crossref site. no answer…


#16

@bozana i receive answer from crossref. as i understand, that’s problem at crossref side:

"I will pass the query regarding that API call onto our developers and see what comes back.
I will keep you posted on any progress from this and once again apologies for the delay. "


#17

Hi @redukr,

Yes, lets see what they say i.e. if they can do something to actualize the statuses via that deposit API…

Best,
Bozana


#18

We have the same problem with thousands of DOIs. Any update on this?

It is also probably making our scheduled DOI task very slow. The task is trying to register / set a new status for all those thousands of articles that have the “submitted” status and does this every day.


#19

Also, what is the correct crossref::status for an active DOI?

Seems that “active” is just a label, so is it “registered” or “found”?


#20

Ok so @bozana if I am understanding the Crossref export code correctly, this request here https://github.com/pkp/ojs/blob/master/plugins/importexport/crossref/CrossrefInfoSender.inc.php#L63 should only return articles/DOIs that have crossref::status set to EXPORT_STATUS_NOT_DEPOSITED which is basically notDeposited, right?

But the thing is that the filter is not working. It seems the request is returning all articles that have a DOI regardless of the status they have. This means that every day all of the DOIs in the system are registered again.

edit:
the EXPORT_STATUS_NOT_DEPOSITED is not visible in the params here: https://github.com/pkp/ojs/blob/master/classes/article/PublishedArticleDAO.inc.php#L782

edit2:
seems that $pubIdSettingValue does not have a value at all here: https://github.com/pkp/ojs/blob/master/classes/article/PublishedArticleDAO.inc.php#L754

edit3: and that is because this setting here https://github.com/pkp/ojs/blob/master/classes/plugins/DOIPubIdExportPlugin.inc.php#L138 overrides the setting here https://github.com/pkp/ojs/blob/master/classes/plugins/PubObjectsExportPlugin.inc.php#L425
But is that intentional?

edit4: also even if the EXPORT_STATUS_NOT_DEPOSITED param is passed to getExportable from the crossref plugin, the sql that is created does not include it. The logic that creates the sql leaves that out. This would work:

		import('classes.plugins.PubObjectsExportPlugin');
		if ($pubIdSettingName && $pubIdSettingValue && $pubIdSettingValue == EXPORT_STATUS_NOT_DEPOSITED) {
			$params[] = $pubIdSettingValue;
		}

		$result = $this->retrieveRange(
			'SELECT	s.*, ps.*,
				' . $this->getFetchColumns() . '
			FROM	published_submissions ps
				JOIN issues i ON (ps.issue_id = i.issue_id)
				LEFT JOIN submissions s ON (s.submission_id = ps.submission_id)
				' . ($pubIdType != null?' LEFT JOIN submission_settings ss ON (s.submission_id = ss.submission_id)':'')
				. ($title != null?' LEFT JOIN submission_settings sst ON (s.submission_id = sst.submission_id)':'')
				. ($author != null?' LEFT JOIN authors au ON (s.submission_id = au.submission_id)':'')
				. ($pubIdSettingName != null?' LEFT JOIN submission_settings sss ON (s.submission_id = sss.submission_id AND sss.setting_name = ?)':'')
				. ' ' . $this->getFetchJoins() .'
			WHERE
				i.published = 1 AND s.context_id = ? AND s.status <> ' . STATUS_DECLINED
				. ($pubIdType != null?' AND ss.setting_name = ? AND ss.setting_value IS NOT NULL':'')
				. ($title != null?' AND (sst.setting_name = ? AND sst.locale = ? AND sst.setting_value LIKE ?)':'')
				. ($author != null?' AND (au.first_name LIKE ? OR au.middle_name LIKE ? OR au.last_name LIKE ?)':'')
				. ($issueId != null?' AND ps.issue_id = ?':'')
				. (($pubIdSettingName != null && $pubIdSettingValue != null && $pubIdSettingValue != EXPORT_STATUS_NOT_DEPOSITED)?' AND sss.setting_value IS NULL':'')
				. (($pubIdSettingName != null && $pubIdSettingValue != null && $pubIdSettingValue == EXPORT_STATUS_NOT_DEPOSITED)?' AND sss.setting_value = ?':'')
				. (($pubIdSettingName != null && is_null($pubIdSettingValue))?' AND (sss.setting_value IS NULL OR sss.setting_value = \'\')':'')
			. ' ORDER BY ps.date_published DESC, s.submission_id DESC',
			$params,
			$rangeInfo
		);

But I am not sure whether I am just misunderstandin the whole logic here.

In any case, the current code tries to register all created DOIs in the system every day, or at least is trying to check the status of all DOIs. With larger installations this means cron processes that can run for several hours, probably even days.

I understand that there could be a need to register articles even when they are already registered once. I mean you need to update the status and the changes in the metadata. But I am not sure if the current situation where all DOIs are reregistereted all the time is intended to do this?