Bepress Import plugin issues/questions

Hi PKP team / @mfelczak,

I’m trying out the bepress import plugin, and seem to be unable to import. It’s telling me that the import was successful, but the issues aren’t showing up in the journal, and none of the files are in the data directory.

I’ve installed the plugin, OJS is up-to-date, and the files are all on the OJS server. The necessary journal, users and roles have been created (as outlined in the docs). I’m running the command on the server as the root user, and the files are in a directory outside of OJS.

Here’s the command that I’m running (with some info removed):

php tools/importExport.php BepressImportPlugin pclc_rpb username editorUsername journalEmail /data/raw-data/bepress_xml/pclc_rpb

I also wanted to clarify what the exact folder structure and naming should look like, as this might be the issue. Here’s a snippet of what I currently have:

bepress_xml
└── pclc_rpb
    └── vol1
        ├── iss1
        │   ├── 1
        │   │   ├── fulltext.pdf
        │   │   └── metadata.xml
        │   ├── 2
        │   │   ├── fulltext.pdf
        │   │   └── metadata.xml
        │   └── 3
        │       ├── fulltext.pdf
        │       └── metadata.xml
        ├── iss2
        │   ├── 1
        │   │   ├── fulltext.pdf
        │   │   └── metadata.xml
        │   ├── 2
        │   │   ├── fulltext.pdf
        │   │   └── metadata.xml
        │   ├── 3
        │   │   ├── fulltext.pdf
        │   │   └── metadata.xml
        │   └── 4
        │       ├── fulltext.pdf
        │       └── metadata.xml

Any insight you can provide is appreciated!

Hi @kaitlin,

Just to confirm, are you running the import plugin with OJS 3.1.0-1 ?

Can you please enable errors in config.inc.php (display_errors = On) to see if that produces any additional info?

Cheers,
Michael

Hi @mfelczak,

Yes, we are on 3.1.0-1 (last updated Dec. 5th). I enabled display_errors, but still got the same result with no additional info. I also don’t see anything in the server logs.

edit: in case it’s helpful, we’re also running this on PHP 7.1.11.

Hi @kaitlin,

Thanks for the additional info. I haven’t tested it with PHP7 but it should work. And confirming your previous question regarding directory structure.

Could you please send me a test issue via email so that I can test locally?

Cheers,
Michael

Hi @mfelczak,

We are working on converting the metadata we have from bepress into the XML format required by the plugin. After some investigating, it appears that it’s going to be easiest to work with an Excel spreadsheet of our metadata from bepress and then convert that from csv to XML (rather than starting with the Dublin Core export from bepress which is what we had been trying). My colleague who is working on the conversion has a few questions:

  1. The bepress output includes the ability to include abstracts in various languages. Assuming that the default language is English, how do we encode multiple abstracts, and tag the language(s) that they are in?

  2. The bepress output has a column called “document_type”, which seems to correspond to the XML field “document-type”. But the XML also has a “type” field, which isn’t mirrored in the CSV file. Do I just say that everything is an “article”, as the example XML file indicates?

  3. The sample XML file encodes both a “submission-date” and a “publication_date” (sic). But neither appear in the bepress output. (At least not for the sample that I have). Where does that data come from, or do I just ignore it?

Thanks,
Kristin

Hi @kristin138,

  1. I’ve filed this as a feature enhancement along with suggested syntax here: Import abstract text in multiple locales · Issue #5 · mfelczak/bepress · GitHub

  2. The document-type and type are used to map to a journal section in OJS. They can be used to group articles by section for imported issues. The import plugin first checks for the existence of ‘document-type’ and then ‘type’ to try to determine the section name. If neither are provided, a default of ‘Articles’ is used as the section for the imported article.

  3. The import plugin checks for the existence of a ‘publication-date’ tag and then for a field tag with a ‘publication_date’ attribute to determine the issue and article publication dates, respectively. If only the former is provided, then the issue publication date is used as the article publication date. The ‘submission-date’ corresponds to the article submission date and defaults to the article publication date if it’s not provided. As a minimum, you will need to include the ‘publication-date’ tag since OJS requires issue and article publication dates.

Cheers,
Michael

Hi @kristin138,

The following CSV to XML script might be useful as a starting point:

Cheers,
Michael

Hi Michael,

Is there a data dictionary for what the migration plugin wants? If so can you share it with me, please?

Thank you,
Kristin

Hi @kristin138,

Sorry we don’t have anything beyond the sample import XML.

Cheers,
Michael

Hi @mfelczak,

I’m working with the same group that has already posted on this thread. We have come across a few additional questions re. the sample import XML:

The sample XML doesn’t seem to include some fields that we would like migrate. How would we go about including/writing tags for:

  1. the volume/issue # in which an article is published
  2. article DOIs
  3. first and last page numbers

Thank you,

Emily

Hi @emilycarlisle,

The volume and issue numbers are actually taken from the import path for the article XML, i.e. bepress_xml/journalName/vol#/iss#/#

I’ve filed 2) and 3) as feature enhancements to the bepress import plugin here:

Can you please review the proposed tagging scheme for the above issues and provide feedback if necessary? We’ll aim to have the additions added shortly.

Thanks!
Michael

Hi @mfelczak,

Those proposed tags look good to me.

By import path, do you mean that the volume and issue numbers are taken care of when the XML is uploaded?

Emily

Hi @emilycarlisle,

Thanks for reviewing those. Yes, the volume and issue numbers are derived from the path to each article import XML file.

Cheers,
Michael

Thanks, @mfelczak

Is there an estimated timeline on the updates? If not, will we be able to know when the updates have been made?

Emily

Hi @emilycarlisle,

I’ll try and get these changes added for you next week.

Cheers,
Michael

That’s great! Thanks @mfelczak!

Hi @mfelczak,

I’m not sure if you’d like a separate thread for new questions, but let me know if that’s the case.

I’m running an import and encountering the following error. Any thoughts on what issue this may point to? Perhaps something in the metadata? I’ve updated the plugin to the latest code in github, and we’re on OJS 3.1.1.

PHP Fatal error: Uncaught Error: Call to a member function getFileId() on null in /data/apache-html/western/plugins/importexport/bepress/BepressImportDom.inc.php:527

Hi @kaitlin,

There was a minor change in OJS 3.1.1 that broke compatibility with the plugin which I’ve patched. Can you please download the latest version of the import plugin and confirm whether the recent fix resolves this issue for you?

Cheers,
Michael

Thanks @mfelczak. I updated the plugin, and am now getting a different error:

PHP Fatal error:  Uncaught Error: Call to a member function getValue() on null in /data/apache-html/western/plugins/importexport/bepress/BepressImportDom.inc.php:304
Stack trace:
#0 /data/apache-html/western/plugins/importexport/bepress/BepressImportDom.inc.php(85): BepressImportDom->_handleIssue()
#1 /data/apache-html/western/plugins/importexport/bepress/BepressImportDom.inc.php(70): BepressImportDom->_handleArticleNode()
#2 /data/apache-html/western/plugins/importexport/bepress/BepressImportPlugin.inc.php(179): BepressImportDom->importArticle()
#3 /data/apache-html/western/tools/importExport.php(63): BepressImportPlugin->executeCLI('tools/importExp...', Array)
#4 /data/apache-html/western/tools/importExport.php(69): importExport->execute()
#5 {main}
  thrown in /data/apache-html/western/plugins/importexport/bepress/BepressImportDom.inc.php on line 304 

It seems to be happening on the same article too.

Hi @kaitlin,

That looks to be related to an error msg dump by the plugin, i.e.

Can you please confirm that you have a volume and issue number set in your import path?

Cheers,
Michael