10 1 * * * user php tools/runScheduledTasks.php plugins/generic/usageStats/scheduledTasks.xml
With this, the journal statistics worked!
Now in version 3.5.x it’s not working, the access logs are generated correctly in the file_dir/usageStats/usageEventLogs/ folder, but after that they are not processed.
I tried using both the “task_runner = On” option in config.inc.php, and the “task_runner = Off” option and using (* * * * * php lib/pkp/tools/scheduler.php run >> /dev/null 2>&1), none of the options worked.
The only way that worked was by moving the files from file_dir/usageStats/usageEventLogs/ to file_dir/usageStats/dispatch/ and running “php lib/pkp/tools/reprocessUsageStatsMonth.php YYYYMM”, then it triggered the statistics processes.
I thought about using a cron script to move the logs from file_dir/usageStats/usageEventLogs/ to file_dir/usageStats/dispatch/ and execute “php lib/pkp/tools/reprocessUsageStatsMonth.php YYYYMM”, but before implementing this script, I thought I’d ask here on the forum if anyone has experienced something similar or if they have any suggestions. I looked at the documentation at https://docs.pkp.sfu.ca/admin-guide/en/statistics, but it can’t identify where the problem might be.
Also be sure to run jobs, e.g. php lib/pkp/tools/jobs.php run, it creates a few followup jobs processing the log file. I need to call the command several times until the statistics are processed and show up.
The option “job_runner = On” is configured in config.inc.php, but do I still need to run this task “php lib/pkp/tools/jobs.php run” in cron?
Initially I had configured it like this: (10 0 * * * php lib/pkp/tools/scheduler.php run >> /dev/null 2>&1); then like this: (* * * * * php lib/pkp/tools/scheduler.php run >> /dev/null 2>&1); now I configured it like this: (0 0 * * * php lib/pkp/tools/scheduler.php run >> /dev/null 2>&1), I’ll test it to see if it solves the problem, I’ll post here if it does.
it’s a bit confusing. I prefer to also do the call explicit, and more often than the scheduler. There would also be an option process_jobs_at_task_scheduler, that could do that.
Please note that the command php lib/pkp/tools/jobs.php run is not intended for the production usage but for locale dev and testing . There is a dedicated job running mechanism via schedule task that can be run via schedule task runner at CLI via cron or as web based task runner with config setting of process_jobs_at_task_scheduler .
So jobs can be processed in 3 mechanism
via web based job runner with setting the config job_runner
via schedule task runner via setting process_jobs_at_task_scheduler
dedicated worker with daemon monitor (like supervisor)
Initially, I tried using the options “task_runner = On”, “job_runner = On”, and “process_jobs_at_task_scheduler = Off” in config.inc.php, but log files kept accumulating in file_dir/usageStats/usageEventLogs/.
Now, I have set “task_runner = Off”, “job_runner = Off”, and “process_jobs_at_task_scheduler = On” in config.inc.php and scheduled the following cron tasks:
5 1 * * * user find file_dir/usageStats/usageEventLogs/* -ctime 1 -exec mv {} file_dir/usageStats/stage/ \;
*/30 * * * * user php lib/pkp/tools/scheduler.php run >> /dev/null 2>&1
1 * * * * user php lib/pkp/tools/jobs.php run >> /dev/null 2>&1
Plus one more script:
#!/bin/bash
CURRENT_DATE=$(date +%Y%m)
php lib/pkp/tools/reprocessUsageStatsMonth.php ${CURRENT_DATE}
This setup has been working for statistics processing.
I don’t believe this is the best solution, but it worked.