If you look at a file in the usageStats/archive/ folder, you see no URLs of the form [$journalUrl]/index.php/[$journal]/article/view/[$articleId]/[$galleyId] ?
If you access an article galley now, do you see that recorded in your current log in usageStats/archive/ ?
No, all have the form [$journalUrl]/index.php/[$journal]/article/view/[$articleId] 200
I dont have access in this moment to the log in the server (generated today), but I have the log files of this month, and all of then have the same form.
I canāt think of a reason those urls wouldnāt be captured in both your usage stats log and in your Apache access log.
I just downloaded article 1203 twice. Once directly and once with a unique string after the galley id: āw95465MFJ4836Ns87m56e53Kv4N5h9PYā. You wouldnāt expect to see that unique string in the usage stats log, but you should at the very least see it in your webserverās access log. The datestamp will be around 11:30 Eastern, today.
Can you provide snippets of your webserver access log and of your usage stats event log for that time?
To ensure accurate statistics, you will need to clear the data from the faulty logs from the metrics table, by load_id (the filename). If you have a load from the usageEventLogs and from the Apache logs for the same day, you will be double-counting some statistics.
If the upgrade and the reprocessing of the logs is done successfully, you will only have one set of correct metrics. āPre-upgradeā statistics would be the same as āpost-upgradeā statistics.
Ok, but in this case, we need to reprocess all logs from november-2011 (first installation of OJS) until today?, the problem is that I have the doubt of the existence of the logs, my questions is because if the old statistics are in the DB we thought to reprocess only the logs from November 2015 to the present, only the range where the download are not registered.
No, you will just need to reprocess the logs during which the bug was active. If you upgraded from 2.3.4 to 2.4.8 in November 2015, delete the metrics associated with the usageEventLogs that have been collected since that time, and process your Apache logs for that same timeframe. Youāll then have correct statistics for before the upgrade (imported by the upgrade) and after the upgrade (from the Apache log processing).
Be sure to export your metrics table before running any manual SQL commands, just in case.
Hi, Graham and Bruno
We upgrade the system to 2.4.8 and reprocess the logā¦but I have a commentā¦when I check statistics by article, the values obtained by the code of Bruno Abstract views, PDF views and Download Stats Made Public š¶ - #8 by Dein, are correct for the articles published after upgrade (new metrics) but in the case of the articles published before upgrade (new metric) the values correspond with new metrics noly and, but the total download are the sum of the old estatistics and the values of new metrics.
At this point, my question isā¦how I can show in the article page the correct value (the old statistics that appear in a report plus the values of new metrics) for example for the article 907 publishes in 2014, the abstract views are 949 and PDF download 312 (in a report), but in the article page the values are abstract 87 and PDF 26 that correspond with the statistic in the log corresponding to november 2015 to present.
Thanksā¦
@aotero, can you describe a little more specifically what metrics types you are comparing, and what metric types you are wanting to add together? Also, what report are you comparing to the article page? I got a little lost.
In report section the first option
show these results
these results are for articles published before the change in the form to collect statistics, as you see, the first article have 438 abstract views and 806 Galley views.
In the article page the results are
that correspond with the statistic generated after the change to the new concept to collect statistics and are shows in the report obtained with Timed Views Report option in the report section, but this values not correspond with the historical statistics.
The correct value will be 438 + 13 for abstracts views and 806 + 47 for downloadā¦
In the case of articles published after the change to the new concept to collect statistics the values are correct, that is logicā¦
The āView Reportā under the āReport Generatorā uses the āViewReportPluginā, which uses the āojs::legacyDefaultā metric type.
The code @beghelli shared in the āā¦Stats Made Publicā post uses the default metric type, which is almost certainly āojs::counterā.
The discrepancy seems to indicate that the upgrade didnāt convert former counter statistics to new ojs::counter metrics. This could be because of an error in the upgrade, or could be that you werenāt collecting counter statistics before the upgrade.
You could modify the code to explicitly add the āojs::legacyDefaultā and āojs::counterā types together for display. There is a post in the same thread which does not exactly do this, but is similar in thought:
Better, in my opinion, would be (if possible) to re-process all of your old logs to generate COUNTER-ready statistics for the duration of the journalās publication history.
Graham, the upgrade process was without problemsā¦and we collect the statistics every time since we use OJS ā¦in the metrics table in the DB we have this.
ojs::legacyDefault
ojs::legacyCounterPlugin
and
ojs::counter
Are there other type of metrics that we need?
Whit respect to the code, we could probe, but this code is for the sum of Galley and abstract.
Iāve re-read the PKP Statistics Framework: Statistics Migration documentation. From that, Iām pretty confident that I was wrong about the upgrade converting the old counter statistics to the new ojs::counter metrics. This does not happen automatically.
I still think the best way forward is to re-process all old logs to generate COUNTER-ready statistics. I can help guide through that process if you have questions.
If you did not want to reprocess your logs, you would need to use code like the sum of gallery and abstract views, except the separate metrics call would be for the āojs::legacyDefaultā metric type. I canāt code that out for you, but if you wanted to try to code it out and had particular questions, I or someone else here might be able to offer suggestions.
Thanks for your helpā¦there is impossible reprocess log, because not exist for 2011-2014ā¦I will try with codeā¦or maybe putting a few comments in the article pageā¦