[OJS 3.0.2] CrossRef XML Export Plugin: Status not refreshed properly

bozana · January 9, 2019, 12:13pm

The function getUnregisteredArticles will look after all articles that does not have a setting ‘crossref::registeredDoi’ i.e. all not successfully deposited and active article DOIs. That setting will only be set/inserted into the DB when the DOI is active i.e. when the last registration/deposit was successfully completed and the DOI metadata can be found in the crossref metadata.

EDIT: the function _getObjectsToBeDeposited (https://github.com/pkp/ojs/blob/master/plugins/importexport/crossref/CrossrefInfoSender.inc.php#L125) then filters those unregistered articles (returned from getUnregisteredArticles): it looks after the new status i.e. updates the status and returns only those that are not deposited (s. https://github.com/pkp/ojs/blob/master/plugins/importexport/crossref/CrossrefInfoSender.inc.php#L129-L135) i.e. that have no ‘crossref::status’ setting name.

The setting value for the setting name ‘crossref::status’ in the DB will be ‘found’ for successfully deposited (last deposit attempt) and active article DOIs.

Does this help?
Best,
Bozana

bozana · January 9, 2019, 12:32pm

It is also probably making our scheduled DOI task very slow. The task is trying to register / set a new status for all those thousands of articles that have the “submitted” status and does this every day.

Yes, that could be – this is probably the updateDepositStatus here https://github.com/pkp/ojs/blob/master/plugins/importexport/crossref/CrossrefInfoSender.inc.php#L129. This code line is needed in order to see if the status has been changed to ‘failed’ and to notify the user in that case.

And now I see the wrong logic there, the line https://github.com/pkp/ojs/blob/master/plugins/importexport/crossref/CrossrefInfoSender.inc.php#L131 should go before calling updateDepositStatus and https://github.com/pkp/ojs/blob/master/plugins/importexport/crossref/CrossrefInfoSender.inc.php#L132-L135 should go after https://github.com/pkp/ojs/blob/master/plugins/importexport/crossref/CrossrefInfoSender.inc.php#L137, and there the $newStatus instead of $currentStatus should be checked.
But, I think this would not change how often the updateDepositStatus has been called.

ajnyga · January 9, 2019, 12:46pm

Hi @bozana

Thanks, I figured over the weekend that it is not just the DOIs with submitted status that are being reprocessed but basically all DOIs with any status.

I am just wondering whether there is a need to keep updating the deposit status of DOIs that have “found” status and are attached to articles that have not been edited lately.

The fact that API is now not returning the correct status is of course a separate issue here and will be hopefully fixed. But when it is fixed it would be maybe good both for us and the Crossref API that the updateDepositStatus is called when it is actually needed.

Or is the logic here that OJS needs to know if a DOI that used to work is changed to failed status? I would thinkt that in those cases Crossref will contact the journal in any case? At least I get a lot of conflict reports etc. straight from Crossref.

bozana · January 9, 2019, 2:00pm

Hi @ajnyga

The DOIs with status ‘found’ should not be considered in the scheduled task – they should have the setting ‘crossref::registeredDoi’ so the function getUnregisteredArticles would not return them. Thus I am wondering why is this happening for you Do those articles have that setting_name? Or is there maybe a bug in the system?
Else, all other DOIs need an automatic status update.
(If an active DOI is manually (this does not happen automatically) re-deposited, e.g. because some metadata have been changed, the old settings will be removed and the status of that last deposit will be considered).

Thanks!
Bozana

ajnyga · January 9, 2019, 2:19pm

Ok, do not waste time on this now! Could very well be that the one’s with found are not among the ones being registestered (looking at the way I was debugging this). Only very few of our 4000 DOIs have that status. About 98% have status submitted. So the original issue here is clearly the important one.

But, would be nice to have OJS reregister DOIs for articles that have been modified. Not talking only about revisions when they come, but maybe just update the metadata whenever the submissions timestamp has been touched or the metadataform saved? Most articles are not ever touched afterwards unless there is a real reason.

ctgraham · January 10, 2019, 2:27pm

^ following this as well.

Our use case is that we moved our registration provider from EZID back to Crossref, and based on enhancements in the plugin (e.g. adding datamining URLs, correcting pagination metadata, etc.) we had interest in resupplying metadata at-large.

Now that I’m looking at this in more detail, it appears that the API endpoint we are using (https://api.crossref.org/deposits) is not documented here:

and is not mentioned here:

Have we checked with Crossref regarding the status (perhaps legacy or EOL?) of the endpoint we’re currently using? If not, I can do so.

Edit: found the deprecated documentation:

github.com

CrossRef/rest-api-doc/blob/master/deprecated/deposit_api.md

# CrossRef RESTful Deposit API (Alpha)

## Versioning

| Date | Version | Changes |
|------|---------|---------|
| 2014-03-10 | v1 | Initial version |
| 2015-01-15 | v2 | Mention PDF deposits |

Note: this code is used by the Open Journals System (OJS) integration. It is not intended for use by other Crossref members, who should use the production deposit system at http://doi.crossref.org

## Background

CrossRef provides a deposit mechanisms for our members and manuscript system vendors.
The current deposit mechanism provides a URL end-point that can accept deposits of
CrossRef metadata. However, this mechanism has some deficiencies, including:

- an inability to programatically track the status of a deposit
- an outdated method of returning deposit results to a user (e-mail responses containing
  success or failure notices)

This file has been truncated. show original

and inquired about support plans.

ajnyga · January 10, 2019, 3:11pm

I think Bozana is working with the new API already? Not sure about the schedule though.

ctgraham · January 11, 2019, 5:02pm

Crossref confirmed that the “deprecated” API is currently supported with no current plans toward end-of-life.

I can confirm that this bug in the Crossref API also affects OJS 2.4.8, preventing the submissions from ever progressing to “complete” and thus preventing the insertion of the “crossref::registeredDoi” setting.

I’m going to raise this with Crossref support as well.

bozana · January 13, 2019, 8:56am

Hi @ctgraham

Thanks a lot for the information!
I think the integration of the new Crossref deposit API will come with the next 3.1.2 release. I will double check with Crossref that this is OK next week.

Best,
Bozana

redukr · January 16, 2019, 7:14pm

@bozana i receive answer:

Paul Davis (Crossref)

Jan 16, 12:55 EST

Hello Georgii,

Apologies for the delay I have been working through various channels to obtain some information for you. Our director of operations has since emailed Bozana as she will be taking the lead on this.

I believe the update with the OJS plugin 3.0.2 obtains the status differently than the older versions of the plugin. Status should now be instant.

If you have any other questions regarding this then please do let me know and once again apologies for the delay.

Best regards,

bozana · January 21, 2019, 9:36pm

Hi @redukr

In the next OJS release OJS 3.1.2 the new Crossref deposit API will be integrated and the status update will work much better.
I will maybe however ask what about the old deposit API and the OJS users that are still using it…

Best,
Bozana

redukr · January 22, 2019, 5:56am

Hello @bozana! That’s great news. But for example,- I cannot update even from 3.1.1.2 to 3.1.1.4 because of using of multi language authors patch. And as I amnot programmer probably, it will not work for me. Is there some news for including multinames in some version of ojs?

bozana · January 22, 2019, 9:46am

Hi @redukr

Yes, the multilingual user names are also coming with 3.1.2.
But, what patch do you use? – If your DB i.e. how the names are saved is different, the DB upgrade will be a problem…

Best
Bozana

redukr · January 22, 2019, 1:44pm

@bozana, I use @litvinovg modification from this topic

Translation of the author's name - #53 by litvinovg

Ainul_Rafiq · April 9, 2020, 3:35am

I am having the same problem here…

while I want to download the xml file or submit, it appaering this massage (We are using OJS 3.0.1.0)
`Validation errors:
Failed to locate the main schema resource at ‘http://www.crossref.org/schema/deposit/crossref4.3.6.xsd’.

Invalid XML:
<?xml version="1.0" encoding="utf-8"?>
<doi_batch xmlns=“http://www.crossref.org/schema/4.3.6” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” xmlns:jats=“http://www.ncbi.nlm.nih.gov/JATS1” xmlns:ai=“http://www.crossref.org/AccessIndicators.xsd” version=“4.3.6” xsi:schemaLocation=“http://www.crossref.org/schema/4.3.6 http://www.crossref.org/schema/deposit/crossref4.3.6.xsd”>

*
<doi_batch_id>_1586403148</doi_batch_id>*
1586403148*
*

 <depositor_name>Sabariah</depositor_name>*

 <email_address>ainul.rafiq@outlook.com</email_address>*

*
Fakultas Kedokteran Universitas Islam Al-Azhar*
*
*
*
```
 <journal_metadata>*
```

   <full_title>JURNAL KEDOKTERAN</full_title>*

   <abbrev_title>kedokteran</abbrev_title>*

   <issn media_type="electronic">2620-5890</issn>*

   <issn media_type="print">2460-9749</issn>*

```
 </journal_metadata>*
```
```
 <journal_issue>*
```

   <publication_date media_type="online">*

```
     <month>02</month>*
```
```
     <day>18</day>*
```
```
     <year>2019</year>*
```
```
   </publication_date>*
```
```
   <journal_volume>*
```
```
     <volume>3</volume>*
```
```
   </journal_volume>*
```
```
   <issue>2</issue>*
```
```
 </journal_issue>*
```

 <journal_article xmlns:jats="http://www.ncbi.nlm.nih.gov/JATS1" publication_type="full_text" metadata_distribution_opts="any">*

```
   <titles>*
```

     <title> INTERVENSI KESEHATAN WASPADA DEMAM BERDARAH (DBD)  DI KELURAHAN TURIDA TAHUN 2017</title>*

```
   </titles>*
```
```
   <contributors>*
```

     <person_name contributor_role="author" sequence="first">*

       <given_name>Diani Sri</given_name>*

```
       <surname>Hidayati</surname>*
```
```
     </person_name>*
```

     <person_name contributor_role="author" sequence="first">*

       <given_name>Nisiaputri</given_name>*

```
       <surname>Rinayu</surname>*
```
```
     </person_name>*
```
```
   </contributors>*
```

   <jats:abstract xmlns:jats="http://www.ncbi.nlm.nih.gov/JATS1">*

     <jats:p>Perubahan musim yang ekstrim dapat memicu timbul dan berkembangnya berbagai macam penyakit yang mungkin dapat menjadi wabah berbahaya bagi masyarakat. Musim hujan menyuburkan datangnya jentik-jentik nyamuk yang menjadi sumber penyakit bila keadaan lingkungan tidak terjaga dengan baik, seperti pada sisi kebersihan serta kerapian lingkungannya. Diperlukan adanya suatu usaha penyadaran masyarakat tentang bahaya penyakit DBD. Usaha-usaha yang dapat dilakukan adalah yang bersifat aktif dan persuasive guna mengajak bersama masyarakat untuk memerangi sertamen cegah penyebaran penyakit DBD untuk meningkatkan pengetahuan dan kesadaran masyarakat KelurahanTurida dalam upaya mencegah penyakit DBD, meningkatkan kepedulian masyarakat terhadap pentingnya kegiatan pencegahan DBD, dan memberikan pengetahuan dan pemahaman yang benar tentang DBD. Metodenya adalah “Waspada DBD“. Bentuk kegiatan yang dilakukan ada tiga, yaitu; penyuluhan, pembagian abate dan program PJB (PemantauanJentikBerkala), dan pemasangan poster. Hasil penyuluhan dihadiri oleh masyarakat Turida dari lima lingkungan yang berjumlah 56 orang. Dalam penyuluhan ini dipaparkan tanda dan gejala DBD,cara pencegahan, kegunaan abate juga memberi sosialisasi cara mengisi kartu jumantik untuk setiap KK. Pembagian Abate dan Program PJB (Pemantauan Jentik Berkala). Program PJB dan pembagian abate ini dilakukan bersamaan untuk tiga lingkungan yaitu Turida Barat Turida Timur dan Lendang Lekong. Poster dipasang di pertengahan lingkungan Turida Barat danTuridaTimur juga di lingkungan Lendang Lekong. Penelitian ini kurang maksimal karena di hadiri oleh 56 warga sehingga belum dapat mewakili populasi Kelurahan Turida selain itu tidak diketahui tingkat efektifitas penyuluhan terhadap pengetahuan masyrakat.</jats:p>*

```
   </jats:abstract>*
```

   <publication_date media_type="online">*

```
     <month>02</month>*
```
```
     <day>18</day>*
```
```
     <year>2019</year>*
```
```
   </publication_date>*
```
```
   <pages>*
```
```
     <first_page>544</first_page>*
```
```
     <other_pages>547</other_pages>*
```
```
   </pages>*
```
```
   <doi_data>*
```

     <doi>10.36679/kedokteran.v3i2.70</doi>*

     <resource>http://e-journal.unizar.ac.id/index.php/kedokteran/article/view/70</resource>*

     <collection property="crawler-based">*

```
       <item crawler="iParadigms">*
```

         <resource>http://e-journal.unizar.ac.id/index.php/kedokteran/article/download/70/52</resource>*

```
       </item>*
```
```
     </collection>*
```

     <collection property="text-mining">*

```
       <item>*
```

         <resource mime_type="application/pdf">http://e-journal.unizar.ac.id/index.php/kedokteran/article/download/70/52</resource>*

```
       </item>*
```
```
     </collection>*
```
```
   </doi_data>*
```
```
 </journal_article>*
```
*
*

</doi_batch>

and the DOI is not active yet.

anyone can give some advice or help?