Our PKP PN Plugin doesn't harvest

Describe the issue or problem

Our journals has previously had working PN configurations and harvesting, but now fails:

PLN has worked before, but didn’t work prior to the update. It is unclear to us exactly when the harvesting broke, but we’re thinking years.

The status and error messages we get are “Unknown Locks State in progress” “harvest-error” or “xml-error”.

We don’t know how to proceed from here, and wonder if there exists something helpful or descriptive in the logs on the harvesting side, or if this looks familiar. We’ve browsed the forum, and looked at the documentation without any luck. We’re grateful for any help.

What application are you using?
For example, OJS 3.3.0.11 (September 2022) and the latest tagged version of PKP/PN 2.0.4.2

Additional information

From our scheduled task logs:

[2023-02-26 13:51:27] boap.uib.no
[2023-02-26 13:51:27] [Notice] Task process started.
[2023-02-26 13:51:27] [Notice] PLN Depositor executeActions started
[2023-02-26 13:51:27] [Notice] Depositor processing for Bergen Journal of Criminal Law & Criminal Justice.
[2023-02-26 13:51:27] [Notice] Getting service document.
[2023-02-26 13:51:28] [Notice] Processing deposit status updates.
[2023-02-26 13:51:28] [Notice] Trying status update for 49 (Issue: 252) (Local Status: [Transferred], Processing Status: [Sent to LOCKSS], Lockss Status: [Unknown])
[2023-02-26 13:51:28] [Notice] Processing status got for 49 -> (deposited)
[2023-02-26 13:51:28] [Notice] Deposit 49 has unknown LOCKSS state inProgress
[2023-02-26 13:51:28] [Notice] Trying status update for 52 (Issue: 256) (Local Status: [Transferred], Processing Status: [Sent to LOCKSS], Lockss Status: [Unknown])
[2023-02-26 13:51:29] [Notice] Processing status got for 52 -> (deposited)
[2023-02-26 13:51:29] [Notice] Deposit 52 has unknown LOCKSS state inProgress
[2023-02-26 13:51:29] [Notice] Trying status update for 207 (Issue: 451) (Local Status: [Transferred], Processing Status: [Sent to LOCKSS], Lockss Status: [Unknown])
[2023-02-26 13:51:29] [Notice] Processing status got for 207 -> (deposited)
...
[2023-02-26 13:51:31] [Notice] Processing status got for 70 -> (xml-error)
[2023-02-26 13:51:31] [Notice] Deposit 70 has unknown processing state xml-error
[2023-02-26 13:51:31] [Notice] Trying status update for 35 (Issue: 20) (Local Status: [Transferred], Processing Status: [Unknown], Lockss Status: [Unknown])
[2023-02-26 13:51:31] [Notice] Processing status got for 35 -> (harvest-error)

Admin page for plugin:

Hi @OyvindLGjesdal,

I’ll leave some generic comments below.

Harvest error

The PKP PN is trying to download the files from the samples below, after retrying for some days, it gives up:

Unknown LOCKSS state inProgress

This was an issue in the plugin, which has been fixed, but a new release of the plugin with the fixes wasn’t made yet, these deposits must be probably completed.

XML error

This is a mix of a problem in your installation with a bug in the plugin (which shouldn’t export failed deposits). About the problem, it could be a missing file or something else.
In order to investigate (I can help to get the issue solved) you can access the Tools > Import/Export > Native XML Plugin > Export Issues, then select the one which failed with a XML error.
After doing that, you should see the error in the interface or perhaps in the PHP/error log of the server.

If you need a sample that failed with this error, here’s one: https://boap.uib.no/index.php/jaf/issue/view/356

Then let me know what happened, I’m interested in solving all the PKP PN related issues.

Best,
Jonas Raoni

Hi @jonasraoni I think your instruction has got me on a better path:

Harvesting Error
The first two links are posted to our test-server, where we restore prod from. We tried to add a test sql statement to prod->test configuration with:

"UPDATE plugin_settings set setting_value = 0 where plugin_name = 'plnplugin' and setting_name = 'enabled'

Could it be that somehow I messed up configuration so that it is actually prod which tries to deposit some items with the boap.ubbe.no url now, or is it still the test server which tries to use the plugin, despite the sql statement?

For the third item, I think the NJSR journal is supposed to have been removed. When I browse in the admin interface I don’t see the NJSR listed anymore, but the item posted belongs to it. I’ll look some more into these. I’ll try to investigate this.

Unknown LOCKSS state in progress
Should I update to use the stable-3_3_0 branch, and then reset those items, or wait for a tagged release? I have now changed to this branch, and reset one single item for testing, which is now in the state In progress.

XML error
I get the following error message when trying to export:

Notice: Undefined offset: 7 in /var/www/boap-ojs/lib/pkp/plugins/importexport/native/filter/SubmissionFileNativeXmlFilter.inc.php on line 85

Notice: Trying to get property 'documentElement' of non-object in /var/www/boap-ojs/plugins/importexport/native/filter/IssueNativeXmlFilter.inc.php on line 204
Validation errors:
Element '{http://pkp.sfu.ca}submission_file', attribute 'stage': [facet 'enumeration'] The value '' is not an element of the set {'public', 'submission', 'note', 'review_file', 'review_attachment', 'final', 'fair_copy', 'editor', 'copyedit', 'proof', 'production_ready', 'attachment', 'query', 'review_revision', 'dependent'}.

Element '{http://pkp.sfu.ca}submission_file', attribute 'stage': '' is not a valid value of the local atomic type.

Element '{http://pkp.sfu.ca}submission_file', attribute 'stage': [facet 'enumeration'] The value '' is not an element of the set {'public', 'submission', 'note', 'review_file', 'review_attachment', 'final', 'fair_copy', 'editor', 'copyedit', 'proof', 'production_ready', 'attachment', 'query', 'review_revision', 'dependent'}.

Element '{http://pkp.sfu.ca}submission_file', attribute 'stage': '' is not a valid value of the local atomic type.

Element '{http://pkp.sfu.ca}issue': Missing child element(s). Expected is ( {http://pkp.sfu.ca}articles ).

Invalid XML:
<?xml version="1.0"?>
<issue xmlns="http://pkp.sfu.ca" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" published="1" current="0" access_status="1" url_path="" xsi:schemaLocation="http://pkp.sfu.ca native.xsd">
  <id type="internal" advice="ignore">356</id>
  <id type="doi" advice="update">10.15845/jaf.v2i2</id>
  <issue_identification>
    <volume>2</volume>
    <number>2</number>
    <year>2018</year>
  </issue_identification>
  <date_published>2018-11-19</date_published>
  <last_modified>2018-11-19</last_modified>
  <sections>
    <section ref="ED" seq="1" editor_restricted="1" meta_indexed="1" meta_reviewed="0" abstracts_not_required="1" hide_title="0" hide_author="0" abstract_word_count="0">
      <id type="internal" advice="ignore">85</id>
      <abbrev locale="en_US">ED</abbrev>
      <title locale="en_US">Editorial</title>
    </section>
    <section ref="FLM" seq="2" editor_restricted="0" meta_indexed="1" meta_reviewed="1" abstracts_not_required="0" hide_title="0" hide_author="0" abstract_word_count="0">
      <id type="internal" advice="ignore">74</id>
      <abbrev locale="en_US">FLM</abbrev>
      <abbrev locale="nb_NO">ART</abbrev>
      <policy locale="nb_NO">##section.default.policy##</policy>
      <title locale="en_US">Films</title>
      <title locale="nb_NO">Artikler</title>
    </section>
  </sections>
  <covers>
    <cover locale="en_US">
      <cover_image>cover_issue_356_en_US.jpg</cover_image>
      <cover_image_alt_text/>
      <embed encoding="base64">...# deleted for brevity</embed>
    </cover>
  </covers>
  <issue_galleys xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://pkp.sfu.ca native.xsd"/>
</issue>

Fatal error: Uncaught Exception: Could not convert selected objects. in /var/www/boap-ojs/lib/pkp/classes/plugins/ImportExportPlugin.inc.php:181 Stack trace: #0 /var/www/boap-ojs/plugins/importexport/native/NativeImportExportPlugin.inc.php(295): ImportExportPlugin->displayXMLValidationErrors() #1 /var/www/boap-ojs/plugins/importexport/native/NativeImportExportPlugin.inc.php(212): NativeImportExportPlugin->exportIssues() #2 /var/www/boap-ojs/lib/pkp/pages/management/PKPToolsHandler.inc.php(94): NativeImportExportPlugin->display() #3 /var/www/boap-ojs/lib/pkp/classes/core/PKPRouter.inc.php(395): PKPToolsHandler->importexport() #4 /var/www/boap-ojs/lib/pkp/classes/core/PKPPageRouter.inc.php(246): PKPRouter->_authorizeInitializeAndCallRequest() #5 /var/www/boap-ojs/lib/pkp/classes/core/Dispatcher.inc.php(144): PKPPageRouter->route() #6 /var/www/boap-ojs/lib/pkp/classes/core/PKPApplication.inc.php(362): Dispatcher->dispatch() #7 /var/www/boap-ojs/index.php(68): PKPApplication->execute() #8 {main} thrown in /var/www/boap-ojs/lib/pkp/classes/plugins/ImportExportPlugin.inc.php on line 181

If I understand the validation errors here, it seems to me to be that there are no submission files in the export, or that articles aren’t picked up? The articles in the issue are regular articles, but with html-galleys only. Under the published article I only find one item under publication/galleys, but I don’t see the html galley in the submission_files (or any other submission files) in the xml export. Is that expected?

Thank you for the very helpful feedback and help!
Øyvind

Thanks for the information! I just started fixing these Preservation Network issues, so any error report is welcome.

The first two links are posted to our test-server, where we restore prod from (…)

As far as I can tell, your update should have worked, it prevents the plugin from running any of its code, but I think you were hit by cache! Take a look at the file /cache/fc-pluginSettings-5-plnplugin.php. If you see an 'enabled' => true, then here’s the culprit.

Still about the production vs test environment, I’ve flagged this issue, which I’m going to fix soon: Test journals can overwrite information in the PKP PN server · Issue #22 · pkp/pln · GitHub

Those are just some examples that I found, if you have any other that sounds suspicious, I can easily check on our side.

Should I update to use the stable-3_3_0 branch, and then reset those items, or wait for a tagged release? (…)

I’ve been pushing a lot of fixes without testing properly, so you might try it with a grain of salt (you’ll just need to run composer install at the plugin folder after the update), if something breaks, it won’t be destructive, so you can get back to the previous version :slight_smile:

I’ll fix 1-2 more bugs, and probably add automated tests before the new release.

If I understand the validation errors here, it seems to me to be that there are no submission files in the export, or that articles aren’t picked up?

I’ve seen this issue before, here are some issues from GitHub:

If this statement returns something, then you’re a victim:
SELECT * FROM submission_files WHERE file_stage IN (7, 8)

This should fix (read the GitHub issue if you’re curious):
UPDATE submission_files SET file_stage = 6 WHERE file_stage = 7;
UPDATE submission_files SET file_stage = 9 WHERE file_stage = 8;

ps: About the missing pieces at the XML, after watching the code, I saw that it’s built by parts. If an inner level fails, then a cascate effect happens and the subsequent outer levels fail as well.

Best,
Jonas Raoni

Thank you, this is very helpful!

I see we should upgrade OJS again to solve some issues, and we also have 100ish submission files in file stage 7 and 8 leftover from OJS 2. I’ve put on notifications on the PLN github repo for releases, so I’ll be notified when you do the release, and I can report back what then happens.

Best,
Øyvind

1 Like