Tools/harvest.php - 2 issues

Open Harvester Systems Version: 2.3.1.0

Hello, when attempting to run tools/harvest.php one of our archives is returning an error related to SSL.

Below is the full output we receive. I have also attached a screenshot of the certificate info for the domain.
When I access the same OAI request in my web browser there is no issue.
e.g.
running the harvest:
php tools/harvest.php 10 from=2019-09-09 until=2019-09-10 set=DART-Europe

Equivalent in web browser:
https://discovery.ucl.ac.uk/cgi/oai2?verb=ListRecords&metadataPrefix=oai_dc&set=DART-Europe&from=2019-09-09&until=2019-09-10

The second issue is for all other archives. When harvesting, the time taken is very long. For example around 5 hours to harvest approx. 2,000 records. Is there a way to make it quicker?


(full error log relating to SSL)

harvester2 has produced an error
Message: WARNING: Declaration of ZendSearchPlugin::getManagementVerbs() should be compatible with GenericPlugin::getManagementVerbs($verbs = Array)
In file: /var/home/darteu/darteu.publicknowledgeproject.org/www/plugins/generic/zendSearch/ZendSearchPlugin.inc.php
At line: 485
Stacktrace:
Server info:
OS: Linux
PHP Version: 7.2.18
Apache Version: N/A
DB Driver: mysqli
DB server version: 5.7.21-20-log
harvester2 has produced an error
Message: WARNING: Declaration of CustomBlockManagerPlugin::getManagementVerbs() should be compatible with GenericPlugin::getManagementVerbs($verbs = Array)
In file: /var/home/darteu/darteu.publicknowledgeproject.org/www/plugins/generic/customBlockManager/CustomBlockManagerPlugin.inc.php
At line: 181
Stacktrace:
Server info:
OS: Linux
PHP Version: 7.2.18
Apache Version: N/A
DB Driver: mysqli
DB server version: 5.7.21-20-log
harvester2 has produced an error
Message: WARNING: Declaration of StaticPagesPlugin::getManagementVerbs() should be compatible with GenericPlugin::getManagementVerbs($verbs = Array)
In file: /var/home/darteu/darteu.publicknowledgeproject.org/www/plugins/generic/staticPages/StaticPagesPlugin.inc.php
At line: 206
Stacktrace:
Server info:
OS: Linux
PHP Version: 7.2.18
Apache Version: N/A
DB Driver: mysqli
DB server version: 5.7.21-20-log
harvester2 has produced an error
Message: WARNING: Declaration of OAIHarvesterPlugin::manage($verb, $args) should be compatible with PKPPlugin::manage($verb, $args, &$message)
In file: /var/home/darteu/darteu.publicknowledgeproject.org/www/plugins/harvesters/oai/OAIHarvesterPlugin.inc.php
At line: 22
Stacktrace:
Server info:
OS: Linux
PHP Version: 7.2.18
Apache Version: N/A
DB Driver: mysqli
DB server version: 5.7.21-20-log
Selected archive: UCL Discovery
Fetching records…
harvester2 has produced an error
Message: WARNING: fsockopen(): SSL operation failed with code 1. OpenSSL Error messages:
error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed
In file: /var/home/darteu/darteu.publicknowledgeproject.org/www/lib/pkp/classes/file/wrappers/HTTPFileWrapper.inc.php
At line: 77
Stacktrace:
Server info:
OS: Linux
PHP Version: 7.2.18
Apache Version: N/A
DB Driver: mysqli
DB server version: 5.7.21-20-log
harvester2 has produced an error
Message: WARNING: fsockopen(): Failed to enable crypto
In file: /var/home/darteu/darteu.publicknowledgeproject.org/www/lib/pkp/classes/file/wrappers/HTTPFileWrapper.inc.php
At line: 77
Stacktrace:
Server info:
OS: Linux
PHP Version: 7.2.18
Apache Version: N/A
DB Driver: mysqli
DB server version: 5.7.21-20-log
harvester2 has produced an error
Message: WARNING: fsockopen(): unable to connect to ssl://discovery.ucl.ac.uk:443 (Unknown error)
In file: /var/home/darteu/darteu.publicknowledgeproject.org/www/lib/pkp/classes/file/wrappers/HTTPFileWrapper.inc.php
At line: 77
Stacktrace:
Server info:
OS: Linux
PHP Version: 7.2.18
Apache Version: N/A
DB Driver: mysqli
DB server version: 5.7.21-20-log
NOTICE: Only variables should be passed by reference (/var/home/darteu/darteu.publicknowledgeproject.org/www/plugins/harvesters/oai/OAIHarvester.inc.php:78)
harvester2 has produced an error
Message: WARNING: fsockopen(): SSL operation failed with code 1. OpenSSL Error messages:
error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed
In file: /var/home/darteu/darteu.publicknowledgeproject.org/www/lib/pkp/classes/file/wrappers/HTTPFileWrapper.inc.php
At line: 77
Stacktrace:
Server info:
OS: Linux
PHP Version: 7.2.18
Apache Version: N/A
DB Driver: mysqli
DB server version: 5.7.21-20-log
harvester2 has produced an error
Message: WARNING: fsockopen(): Failed to enable crypto
In file: /var/home/darteu/darteu.publicknowledgeproject.org/www/lib/pkp/classes/file/wrappers/HTTPFileWrapper.inc.php
At line: 77
Stacktrace:
Server info:
OS: Linux
PHP Version: 7.2.18
Apache Version: N/A
DB Driver: mysqli
DB server version: 5.7.21-20-log
harvester2 has produced an error
Message: WARNING: fsockopen(): unable to connect to ssl://discovery.ucl.ac.uk:443 (Unknown error)
In file: /var/home/darteu/darteu.publicknowledgeproject.org/www/lib/pkp/classes/file/wrappers/HTTPFileWrapper.inc.php
At line: 77
Stacktrace:
Server info:
OS: Linux
PHP Version: 7.2.18
Apache Version: N/A
DB Driver: mysqli
DB server version: 5.7.21-20-log
Finished:
0 records indexed
2 seconds elapsed
0.00 records per second
11600 records kept from past harvests
11600 records total.

certificate

Hello,
Have you any thoughts on this?
Thanks.

Hi @ucl,

I just tried with the URL above and works for me. I am not sure, just thinking loudly, from the SSL error, it could be that your browser doesn’t trust the certificate root. Can you verify that?

Regards, Primož

Hi,

Yes it works in browser ok. It’s when we try to run tools/harvest.php that we cannot get it to work.

Also, the speed is very slow to harvest. Yesterday took nearly 10 hours to harvest 5k records from an archive.