OJS 2.4.7-1 statistical data

If you look at a file in the usageStats/archive/ folder, you see no URLs of the form [$journalUrl]/index.php/[$journal]/article/view/[$articleId]/[$galleyId] ?

If you access an article galley now, do you see that recorded in your current log in usageStats/archive/ ?

No, all have the form [$journalUrl]/index.php/[$journal]/article/view/[$articleId] 200

I dont have access in this moment to the log in the server (generated today), but I have the log files of this month, and all of then have the same form.

What does the URL in the web browser look like when you view/download an article?

For Abstract view
[$journalUrl]/index.php/[$journal]/article/view/[$articleId]
http://revista.ismm.edu.cu/index.php/revistamg/article/view/1117
For dowload file
[$journalUrl]/index.php/[$journal]/article/download/[$articleId]/[$galleyId]
http://revista.ismm.edu.cu/index.php/revistamg/article/download/1203/677

if you want you can visit the journal home page

Hi ctgraham
Which are your conclusions?

I can’t think of a reason those urls wouldn’t be captured in both your usage stats log and in your Apache access log.

I just downloaded article 1203 twice. Once directly and once with a unique string after the galley id: ā€œw95465MFJ4836Ns87m56e53Kv4N5h9PYā€. You wouldn’t expect to see that unique string in the usage stats log, but you should at the very least see it in your webserver’s access log. The datestamp will be around 11:30 Eastern, today.

Can you provide snippets of your webserver access log and of your usage stats event log for that time?

This is the lines I found in webserver access log
130.49.185.219 - - [22/Mar/2016:11:24:23 -0400] ā€œGET /index.php/revistamg/article/download/1203/677 HTTP/1.1ā€ 200 359769 ā€œOJS 2.4.7-1 statistical data - #9 by aoteroā€ ā€œMozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0ā€
130.49.185.219 - - [22/Mar/2016:11:25:11 -0400] ā€œGET /index.php/revistamg/article/download/1203/677/w95465MFJ4836Ns87m56e53Kv4N5h9PY HTTP/1.1ā€ 200 359686 ā€œ-ā€ ā€œMozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0ā€
130.49.185.219 - - [21/Mar/2016:15:51:18 -0400] ā€œGET /index.php/revistamg/article/download/1203/677 HTTP/1.1ā€ 200 259776 ā€œOJS 2.4.7-1 statistical data - #8 by ctgrahamā€ ā€œMozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0ā€
130.49.185.219 - - [21/Mar/2016:15:52:02 -0400] ā€œGET /index.php/revistamg/article/viewFile/1117/676 HTTP/1.1ā€ 200 40020 ā€œhttp://revista.ismm.edu.cu/index.php/revistamg/article/view/1117/676ā€ ā€œMozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0ā€

But in the log in usage stats there is nothing from your IP

1 Like

I think you are running into this error:

It is fixed in 2.4.8. Can you upgrade?

To catch historical statistics, you will want to process your Apache logs instead of the faulty usage stats logs. See this page for details:
https://pkp.sfu.ca/wiki/index.php?title=PKP_Statistics_Framework#Processing_Log_Files

To ensure accurate statistics, you will need to clear the data from the faulty logs from the metrics table, by load_id (the filename). If you have a load from the usageEventLogs and from the Apache logs for the same day, you will be double-counting some statistics.

Yes, in this moment our system admin is maken some test in a secondary server.
I have another question.
It is possible to have a code, similar to this Abstract views, PDF views and Download Stats Made Public 😶 - #6 by Dein, in order to show pre- and post-upgrade statistic by article.

Thank you for your patience

If the upgrade and the reprocessing of the logs is done successfully, you will only have one set of correct metrics. ā€œPre-upgradeā€ statistics would be the same as ā€œpost-upgradeā€ statistics.

Ok, but in this case, we need to reprocess all logs from november-2011 (first installation of OJS) until today?, the problem is that I have the doubt of the existence of the logs, my questions is because if the old statistics are in the DB we thought to reprocess only the logs from November 2015 to the present, only the range where the download are not registered.

No, you will just need to reprocess the logs during which the bug was active. If you upgraded from 2.3.4 to 2.4.8 in November 2015, delete the metrics associated with the usageEventLogs that have been collected since that time, and process your Apache logs for that same timeframe. You’ll then have correct statistics for before the upgrade (imported by the upgrade) and after the upgrade (from the Apache log processing).

Be sure to export your metrics table before running any manual SQL commands, just in case.

OK, good news.
Thank you

Hi, Graham and Bruno
We upgrade the system to 2.4.8 and reprocess the log…but I have a comment…when I check statistics by article, the values obtained by the code of Bruno Abstract views, PDF views and Download Stats Made Public 😶 - #8 by Dein, are correct for the articles published after upgrade (new metrics) but in the case of the articles published before upgrade (new metric) the values correspond with new metrics noly and, but the total download are the sum of the old estatistics and the values of new metrics.
At this point, my question is…how I can show in the article page the correct value (the old statistics that appear in a report plus the values of new metrics) for example for the article 907 publishes in 2014, the abstract views are 949 and PDF download 312 (in a report), but in the article page the values are abstract 87 and PDF 26 that correspond with the statistic in the log corresponding to november 2015 to present.
Thanks…

@aotero, can you describe a little more specifically what metrics types you are comparing, and what metric types you are wanting to add together? Also, what report are you comparing to the article page? I got a little lost.

Hi Graham
I’m try to be more specific…

  1. In report section the first option

    show these results

    these results are for articles published before the change in the form to collect statistics, as you see, the first article have 438 abstract views and 806 Galley views.
  2. In the article page the results are

    that correspond with the statistic generated after the change to the new concept to collect statistics and are shows in the report obtained with Timed Views Report option in the report section, but this values not correspond with the historical statistics.
  3. The correct value will be 438 + 13 for abstracts views and 806 + 47 for download…
  4. In the case of articles published after the change to the new concept to collect statistics the values are correct, that is logic…

Thanks for patience,

Thanks, that is very helpful.

The ā€œView Reportā€ under the ā€œReport Generatorā€ uses the ā€œViewReportPluginā€, which uses the ā€œojs::legacyDefaultā€ metric type.

The code @beghelli shared in the ā€œā€¦Stats Made Publicā€ post uses the default metric type, which is almost certainly ā€œojs::counterā€.

The discrepancy seems to indicate that the upgrade didn’t convert former counter statistics to new ojs::counter metrics. This could be because of an error in the upgrade, or could be that you weren’t collecting counter statistics before the upgrade.

You could modify the code to explicitly add the ā€œojs::legacyDefaultā€ and ā€œojs::counterā€ types together for display. There is a post in the same thread which does not exactly do this, but is similar in thought:

Better, in my opinion, would be (if possible) to re-process all of your old logs to generate COUNTER-ready statistics for the duration of the journal’s publication history.

Graham, the upgrade process was without problems…and we collect the statistics every time since we use OJS …in the metrics table in the DB we have this.

ojs::legacyDefault

ojs::legacyCounterPlugin

and
ojs::counter

Are there other type of metrics that we need?

Whit respect to the code, we could probe, but this code is for the sum of Galley and abstract.

1 Like

I’ve re-read the PKP Statistics Framework: Statistics Migration documentation. From that, I’m pretty confident that I was wrong about the upgrade converting the old counter statistics to the new ojs::counter metrics. This does not happen automatically.

I still think the best way forward is to re-process all old logs to generate COUNTER-ready statistics. I can help guide through that process if you have questions.

If you did not want to reprocess your logs, you would need to use code like the sum of gallery and abstract views, except the separate metrics call would be for the ā€˜ojs::legacyDefault’ metric type. I can’t code that out for you, but if you wanted to try to code it out and had particular questions, I or someone else here might be able to offer suggestions.

Thanks for your help…there is impossible reprocess log, because not exist for 2011-2014…I will try with code…or maybe putting a few comments in the article page…

Good weekend