Views and download of articles are not counted anymore since upgrade OJS 3.1.2

Hi @ctgraham,

It seems that no scheduledTaskLogs folder exists. The permissions and users of the files folder are correct (0755), so this can’t be the problem.

Here a short excerpt from the metrics table:

| usage_events_20160512.log | 521 | 8 | 4997 | 4245 | 20160512 | 201605 | 1 | NULL | NULL | NULL | ojs::counter | 1 | NULL | NULL | NULL | 4245 |
| usage_events_20160512.log | 521 | 8 | 4988 | 4247 | 20160512 | 201605 | 1 | NULL | NULL | NULL | ojs::counter | 2 | NULL | NULL | NULL | 4247 |
| usage_events_20160512.log | 521 | 8 | 4987 | 4249 | 20160512 | 201605 | 1 | NULL | NULL | NULL | ojs::counter | 1 | NULL | NULL | NULL | 4249 |
| usage_events_20160512.log | 521 | 8 | 4981 | 4252 | 20160512 | 201605 | 1 | NULL | NULL | NULL | ojs::counter | 1 | NULL | NULL | NULL | 4252 |
| usage_events_20160512.log | 521 | 8 | 5006 | 4253 | 20160512 | 201605 | 1 | NULL | NULL | NULL | ojs::counter | 1 | NULL | NULL | NULL | 4253 |
| usage_events_20160512.log | 521 | 8 | 5003 | 4255 | 20160512 | 201605 | 1 | NULL | NULL | NULL | ojs::counter | 1 | NULL | NULL | NULL | 4255 |

Should I run the command

php tools/runScheduledTasks.php plugins/generic/usageStats/scheduledTasksAutoStage.xml

In the apache log files I can’t find any error logs which indicate problems with the statistic plugin.

thanks for your support,
Anna

Hi @asmecher,

The last log file in “/var/files/usageStats/archive” directory is from the day before the upgrade to OJS 3.1.2
The log files in “/var/files/usageStats/usageEventLogs” directory stayed in this directory since the day of the upgrade to OJS 3.1.2

There are not log files in “/var/files/usageStats/processing”, “/var/files/usageStats/stage”, “/var/files/usageStats/reject” directories when I looked

When I run the command for autostaging:
php tools/runScheduledTasks.php plugins/generic/usageStats/scheduledTasksAutoStage.xml

Nothing happens!

I noticed when I removed the entry in the DB table scheduled_tasks with the class_name = plugins.generic.usageStats.UsageStatsLoader
And then I ran again the command for autostaging, I have the same error when I run the usageStats/scheduledTasks.xml command:
PHP Fatal error: Cannot redeclare geoip_country_code_by_name_v6() in /ojs_home/plugins/generic/usageStats/lib/geoIp/geoip.inc on line 430

I don’t know why I have got this error and if it’s the reason why logs are not moved into the “stage” directory automatically for processing.

I also noticed the time registered in scheduled_tasks for the execution of plugins.generic.usageStats.UsageStatsLoader is not the same that the time server (2 hours difference). I don’t know if it’s relevant. It was maybe the case before.

I don’t know what is wrong, where I have to looking for, and what can I do as tests.

Have you any idea of what the problem could be?

Thanks again for your precious help.

Kind regards.
Helene

The scheduledTasksLogs directory should exist directly under your files_dir, parallel to the usageStats directory, which contains archive, processing, reject, etc.

The lines from the metrics table reference metrics ingested from event logs from 2016-05-12. Do you see recent entries as well?

You generally want to choose to either run the “tools/runScheduledTasks.php” command in cron, or use the acron plugin, which will automatically run the “scheduledTasksAutoStage” task. Do you have the acron plugin enabled?

The usage statistics processing is rate-limited and will not immediately run again unless the time indicated in the scheduled_tasks table has expired. If you clear the entry from the scheduled_tasks table and the command runs again, this is due to the rate limiting.

This is probably related to the value of the PHP date.timezone configuration. It should not cause a failure of processing the usage statistics, but you should set the value in your php.ini (or set time_zone in config.inc.php).

The PHP Fatal error is a concern. Since you have searched for geoip_country_code_by_name_v6 within the OJS codebase and only found the one occurrence, one possibility is that the geoip.inc file is being included twice somehow. If this were the case, however, it should error on the first function in that file (geoip_load_shared_mem), not geoip_country_code_by_name_v6.

Hi @asmecher,

Acron plugin is activated.

The scheduledTasksLogs directory do exists under my files_dir, parallel to the usageStats directory.
The last Usagestatisticsfileloadertask*.log is the day of upgrading to OJS 3.1.2. It’s written inside that the “/var/files/usageStats/processing/usage_events_yyyymmdd.log” was processed and archived.
yyyymmdd is the date of the day before upgrading to OJS 3.1.2

The last records in metrics table are with the last usage_events_yyyymmdd.log file stored in archive directory with the date of the day before upgrading to OJS 3.1.2

Processing usage events logs stopped when I upgraded to OJS 3.1.2 but I don’t know why.

Best regards.
Helene

In the Acron plugin, can you try running the “reload scheduled tasks” action? If Acron is enabled, I would expect it to be automatically running the Usage Statistics File Loader task.

Hello again @asmecher,
I had already thought about it and I already did it but it didn’t change anything.
Helene

Hi @ctgraham,

this afternoon I manually generated the folder

scheduledTaskLogs

and it was immediately filled with serveral log files. The log file for the usage statistics contains several warnings, that the usage_events_log file contains an adress where the system can’t remove the base url. What does that mean?

[2019-05-09 11:25:21] [Warnung] Die Zeile 7914 aus der Datei /opt/ojs-mthz-files/usageStats/processing/usage_events_20190508.log enthält eine Adresse, aus der das System nicht den Basis-URL entfernen kann.

The metrics table doesn’t contain recents entries, although there are daily event log-files in the archive-folder since 2016 up to today.

The acron plugin is enabled. I just restarted it and the event log files was re-generated, but the reports and counter statistics are still empty:

empty_counter

Do you have an more idea what goes wrong? We need theses statistics on monday…

thanks, Anna

Does line 7914 of usage_events_20190508.log look like an OJS URL? If so, it probably means that OJS failed to process the line. This can happen if OJS can’t correctly determine the hostname and any path for the URL. For example, if the line reads:

127.0.0.1 - - "2019-03-25 19:22:53" http://dev.hostname.tld/ojs3/default/index 200 "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0"

But if your config.inc.php references a base_url for something other than http://dev.ojs.hostname.tld/ojs3, then OJS may be failing to process that particular line. If all lines reference this warning, none of the lines are actually processed.

If you can correct the base_url settings in config.inc.php you can re-process the archived log files by moving them back from the “archive” folder to the “stage” folder. Try it with one file first and (if successfully processed) move all of the files from the missing date range. Reprocessing the files will overwrite any existing rows in the metrics table which match that load_id.

Hi @ctgraham,

you are right: the base_url in our config.inc.php differes from the hostname in the usage_events-20190508.log-file! I corrected it and copied the log file again into the staging area.

How can I trigger a new processing of this file?

thanks, Anna

This will happen automatically on the next acron run (1 day from the last run, unless you modify the scheduled_tasks table), or based on a manual run from the command line (as described above).

hi @ctgraham,

it seems to work!! Thank you very very much for your support :smiley:

best regards, Anna

Hi @ctgraham,

I still have the same problems. I tried to looking for what happened when I execute the scheduledTasks commands by displaying some messages in the code step by step:

When I execute the command:
php tools/runScheduledTasks.php plugins/generic/usageStats/scheduledTasks.xml
I have this error:
PHP Fatal error: Cannot redeclare geoip_country_code_by_name_v6() in /ojs_home/plugins/generic/usageStats/lib/geoIp/geoip.inc on line 430

I found the error occurs when in file :
/ojs_home/plugins/generic/usageStats/UsageStatsLoader.inc.php
this command line is executed:
$geoLocationTool = $statsHelper->getGeoLocationTool();
I found that function “getGeoLocationTool()” is called in this file:
/ojs_home/lib/pkp/classes/statistics/PKPStatisticsHelper.inc.php
And the error occurs after this command line is executed:
$geoLocationTool = $plugin->getGeoLocationTool();

When I execute the command:
php tools/runScheduledTasks.php plugins/generic/usageStats/scheduledTasksAutoStage.xml
Nothing happens
I found that in the following file:
/ojs_home/lib/pkp/classes/cliTool/ScheduledTaskTool.inc.php
In function “parseTasks”
When “$frequency” variable is tested, it has got a value, and this value is tested with this function:
$canExecute = ScheduledTaskHelper::checkFrequency($className, $frequency);
The checkFrequency “function” is included in this file:
/ojs_home/lib/pkp/classes/scheduledTask/ScheduledTaskHelper.inc.php
“$frequency” variable is an array which not contains month or day. But it contains an hour value of “0” which is consider not valid:
[attributes] => Array([hour] => 0)
That’s the reason why, “$canExecute” variable is false and nothing is execute after

Do you have any idea why in the 1st command, the call of “plugin->getGeoLocationTool()” command generates the following error ?:
PHP Fatal error: Cannot redeclare geoip_country_code_by_name_v6() in /ojs_home/plugins/generic/usageStats/lib/geoIp/geoip.inc on line 430

And why in the 2nd command, the presence of the “hour” parameter with the value “0” in “$frequency” variable generates an error ?
Where does the “hour” parameter come from? And where can I change it ?

Thanks again for your help.
Kind regards.
Helene

The hour parameter’s 0 value comes from:

This should be checked against:

This checks to see if the current hour has passed the hour of the last runtime. Thus, the scheduledTasksAutoStage.xml can only be run once a day.

I’m not able to reproduce the problem of Cannot redeclare geoip_country_code_by_name_v6() in my install of 3.1.2 stable. I’m very curious about what is causing this. If you search for ‘geoip.inc’ in your OJS files, do you find any references other than:

$ grep -r 'geoip.inc' *
plugins/generic/usageStats/lib/geoIp/README:Just place the 'geoip.inc' file somewhere according to the 'include_path' directive of
plugins/generic/usageStats/lib/geoIp/README:include("geoip.inc");
plugins/generic/usageStats/lib/geoIp/benchmark.php:include("geoip.inc");
plugins/generic/usageStats/lib/geoIp/geoip.inc:/* geoip.inc
plugins/generic/usageStats/lib/geoIp/geoipcity.inc: *              Changed inclusion of geoip.inc file to require_once, so that
plugins/generic/usageStats/lib/geoIp/geoipcity.inc: *                  this library can be used in the same script as geoip.inc.
plugins/generic/usageStats/lib/geoIp/geoipcity.inc:require_once 'geoip.inc';

?

Hi @ctgraham,

I had already looked for an occurrence of “geoip.inc” but I did it again. I have the same results as you.
I kept looking. I found that the error occurs in this file:
/ojs_home/plugins/generic/usageStats/lib/geoIp/geoipcity.inc
After this command line is executed:
require_once(‘geoip.inc’);
So I tried to comment the “require_once” command and no error occured!
Next I removed the entry in the DB table scheduled_tasks with the class_name = plugins.generic.usageStats.UsageStatsLoader
And then I ran again the command for autostaging, ant it worked!
All usage_events_*.log files were processed and metrics table updated.

But I can’t tell why “require_once” command doesn’t work and generates an error. And where the geoip.inc file is called on the first place.
I noticed that:
“geoip.inc” is called in “geoipcity.inc” and “benchmark.php” files
“geoipcity.inc” is called in “GeoLocationTool.inc.php” and “benchmark.php” files
“GeoLocationTool” is imported from several files
Maybe it’s my release of PHP but I upgraded to php7.0 on my debian jessie
Maybe it’s because I didn’t install patches for the OJS 3.1.2 release and I am still with the original release from the official website.
Maybe I have a specific plugin which interfers with the statistic plugin
I will wait for the next release OJS/OMP 3.1.2-1 and will see if the problen is still there

Best regards.
Helene

Are you using PHP 7.0 or one of the PHP 7.x releases like PHP 7.1, 7.2, or 7.3? PHP 7.0 is at end of life, so if it is a PHP 7.0 bug, there isn’t any official support for it. I’m not aware of any OJS changes which would be relevant, except for the relocation of the usageStats plugin, as described earlier. A very odd situation.

Hello again @ctgraham,
If I find something I will let your know.
Helene

Hi @ctgraham,

I think I found the problem,

When I upgraded to php7.0 my Debian Jessie, I followed a documentation and I installed other php7.0 packages including php7.0-geoip package.
And it was the cause of the problem. When I removed this package, the problem was solved and the following error no longer appeared:
PHP Fatal error: Cannot redeclare geoip_country_code_by_name_v6() in /ojs_home/plugins/generic/usageStats/lib/geoIp/geoip.inc on line 430

Kind regards
Helene

2 Likes

how about if there is no update data in /usagestats/archive? in my OJS 3.1.1.4 this is stop since dec 2019

To process usage statistics, the usage statistics plugin must be enabled, and either the Acron plugin must be enabled, or you must setup a system cron job to run the tasks described above.