usageStats task recovery

Hi, we published new journal three weeks ago. But till now we have “Download data is not yet available” for all its papers.

All plugins (Acron, usageStats) are enabled. There are no problems with logs creation – daily logs are in usageEventLogs. However there is a problem with logs processing – folders processing, reject and stage are empty and no logs transferred to archive. I tried to remove the “UsageStatsLoader” from the DB table “scheduled_tasks” and then access the journal pages to start the scheduled task – no effect. Running runScheduledTasks leads only review reminder logs creation in scheduledTaskLogs.

Thus we’ll be very grateful for the help in solving this problem.

Hi @rkhalikov

Which OJS version do you use?
Do you have any schedules task logs in your files_dir > scheduledTaskLogs starting with Usagestatisticsfileloadertask… ? If you have, could you see what it inside of it/them?

If you do not have them (which would maybe be strange, because you had the UsageStatsLoader in the DB), could you maybe then try again i.e. remove the UsageStatsLoader from the DB table scheduled_tasks and then try to run this command (from OJS folder):
If you are using OJS 3.0.2:
php tools/runScheduledTasks.php lib/pkp/plugins/generic/usageStats/scheduledTasks.xml
If you are usign OJS 2.4.8:
php tools/runScheduledTasks.php plugins/generic/usageStats/scheduledTasks.xml

Please backup everything before doing it – to be on the safe side…

Then check the situation with the usage stats log files (where they are) and scheduled task log files…

Then we will see further…
Best,
Bozana

Hi, Bozana.

We use OJS 3.0.2.
Well, what did I do:

  1. Removed the UsageStatsLoader from the DB table scheduled_tasks.
  2. Run the command.
  3. Checked log files - scheduledTaskLogs folder contains one log file (Task process started, Task process stopped), without any files processed.
  4. Moved one stats file from usageEventLogs to stage and run the command again.
  5. This file was processed and moved to archive, but with the following warning in the log file: log contains an url that the system can’t remove the base url from.

Yes, we changed base_url after upgrading OJS from 2.4.8 to 3.0.2, but 2.4.8 installation was not in production and did’t contain the journal launched.

I found Article statistics showing 0 abstract views and 0 PDF downloads - #93 by Ph_We with discussion about base_url. But since OJS upgrade we have the following records in config.inc.php:
base_url = "https://vestnik-journals.spbu.ru"
base_url[s3] = https://biocomm.spbu.ru and so on for all 17 journals.

HI @rkhalikov,

Ah, sorry, in order not to have to move the log file in the folder stage you should use this command:
php tools/runScheduledTasks.php lib/pkp/plugins/generic/usageStats/scheduledTasksAutoStage.xml

Do you maybe see any line number in that warning? Do you see other URL or journal path in that usage stats log file, that is not listed in your config?

Best,
Bozana

Should I get any logs in scheduledTaskLogs after this command? I tried to execute it - no visible errors in shell-client but also no logs about this command.

Yes, it is totally the same as the other, just to use the auto stage parameter, so that the log files do not have to be manually moved into the folder stage – they could stay in the folder usageEventLogs.
Have you removed the UsageStatsLoader from DB table scheduled_tasks or was the scheduled task already executed today? – If the task was executed today, the scheduled task will not be executed again – only once a day…

OK, second round.

  1. I removed the UsageStatsLoader from the DB table and run the command with scheduledTasksAutoStage.xml.
  2. Command completed and stats files processing began.
  3. I found base url warning was because couple of days after upgrade OJS worked via http, not https like now. Thus only old logs produced this warning. All current stats files were processed normally.
  4. Today we finally obtained histograms on articles pages. Thus processed logs show correct statistics.

But the problem with new stats files remains! :frowning: We still have stats files accumulation in usageEventLogs and no more logs in scheduledTaskLogs after my manual command run. It turns out that the problem concerns acron/cron? :frowning:

Hi @rkhalikov

Glad you solved one problem! :slight_smile:

Is you AcronPlugin enabled? What do you see in your DB table plugin_settings, where plugin_name = ‘acronplugin’ and settin_name = ‘crontab’? Do you see an item with “plugins.generic.usageStats.UsageStatsLoader”?

Best,
Bozana

Thanks for your help :wink:

Yes, AcronPlugin is enabled for OJS and journal. But we have two records for acron in DB:

Here is the result of your query:

acronplugin 0 crontab a:5:{i:0;a:3:{s:9:“className”;s:43:“plugins.generic.usageStats.UsageStatsLoader”;s:9:“frequency”;a:1:{s:4:“hour”;i:24;}s:4:“args”;a:1:{i:0;s:9:“autoStage”;}}i:1;a:3:{s:9:“className”;s:48:“plugins.importexport.datacite.DataciteInfoSender”;s:9:“frequency”;a:1:{s:4:“hour”;i:24;}s:4:“args”;a:0:{}}i:2;a:3:{s:9:“className”;s:48:“plugins.importexport.crossref.CrossrefInfoSender”;s:9:“frequency”;a:1:{s:4:“hour”;i:24;}s:4:“args”;a:0:{}}i:3;a:3:{s:9:“className”;s:42:“plugins.importexport.medra.MedraInfoSender”;s:9:“frequency”;a:1:{s:4:“hour”;i:24;}s:4:“args”;a:0:{}}i:4;a:3:{s:9:“className”;s:35:“lib.pkp.classes.task.ReviewReminder”;s:9:“frequency”;a:1:{s:4:“hour”;i:24;}s:4:“args”;a:0:{}}}

Hmmm… I think this is OK, just that I do not understand why is setting_value = 0 in that one line :open_mouth: Could you please change it to 1 and then see, if the scheduled task is executed? – The scheduled tasks should be executed one time per day, and for that the date in the column last_run in the DB table scheduled_tasks is considered. Thus first after a day of that last_run date the scheduled task will be executed. Or you could again delete the UsageStatsLoader from that table and visit journal home page (to trigger the execution via the AcronPlugin), and then check what happened with the files…

Yes, it works now. Today scheduled task was executed properly. So I hope, now we have correctly configured acron/tasks mechanism for our OJS. Thank you very much!

Minor strange problem exists. We get [Error] Error: can't execute "/bin/gzip". Check your gzip setting in config.inc.php. The strangeness is that when I manually run the command, these errors were not found and the stats files were archived normally.

Hi @rkhalikov

Glad that the problem is solved! :-)))
Maybe you can take a look at the log files and DB table metrics for a few more days – just to be sure that everything looks fine…

Concerning that error: Hmmm… Do you have that command /bin/gzip ? And is it executable for everybody i.e. for all users i.e. also for the web user?

Best,
Bozana

Hi bozana

OJS Version 3.1.1

  1. How to activated Acron for each of journals?
    I curious how to activated Acron Plugin on each of journal, since in the acron plugin setting there is no any enable button just reload task scheduler , delete , and upgradeimage

I suspect that OJS is automatically enabled it by default but when I check on the database it is only activated for the index page only, not each of the journals

here the preview :
image

  1. Is there any mechanism or command to import the log to the metrics table?
    I followed all this explanation and run the command
    php tools/runScheduledTasks.php lib/pkp/plugins/generic/usageStats/scheduledTasksAutoStage.xml

The log file created but not the log file content is not send the data to the metrics table.
here is the log preview :
image
but the metrics is still blank or not affected by the command.

  1. Seems like the command with autostage not work .
    I try to copy file from /ojs_files/usageStats/usageEventLogs/xx.log to the /ojs_files/usageStats/stage/xx.log

run this command :
php tools/runScheduledTasks.php lib/pkp/plugins/generic/usageStats/scheduledTasks.xml
command is summarized the log.

the counter log of abstract article read and download is working but the histogram stat is still not shown / still have the word : Download data is not yet available.

But the article counter and download counter is working fine :
image

Here is the metrics preview :
image
any suggestion bozana?

Thanks

2 Likes

Dear @bozana I got these messages too.

does this affect the statistical process or just the archiving process?

[2020-11-29 19:35:11] [Notice] File /home/polgnac/public_html/files/usageStats/processing/usage_events_20201128.log was processed and archived.
[2020-11-29 19:35:11] [Error] Error: can't execute "".  Check your gzip setting in config.inc.php.