Understanding the disparity between COUNTER stats and web analytics (OJS 3.3)

Software Support
1

I’m being asked by journal editors why there appears to be such a disparity between the COUNTER stats collected in OJS vs. the number of page views and downloads recorded by our web analytics packages. I understand that the different tool collect statistics differently, but I wonder if someone could speculate as to why the disparity would be so large?

For example, the COUNTER stats from OJS for one of our journals in April shows 16,283 full-text views.

For the same period, our web analytics platform shows 2,572 site visits, 4,154 pageviews, and 60 downloads. The platform does appear to capture views for pages rendered by the PDF.js PDF Viewer, and these constitute the vast majority of pageviews (I’d eyeball it at 90%).

This example shows that the COUNTER stats of full-text article views are roughly 6x greater than the number of article pageviews and downloads captured in web analytics. I haven’t done a deep dive but this looks pretty typical to me of comparisons I’ve done previously.

I also compared the OJS report Journal Main Page Views for April to our web analytics for the same page: 7,843 according to OJS, 228 according to web analytics. I would expect that the web analytics platform does more to filter bots for the page views, but I’m under the impression that COUNTER stats should do some of that when counting readership.

I’m just not sure what the most reliable source is for readership stats when there’s such a disparity!

OJS 3.3.0-22
Matomo 5.10.0

2

Hi @tmrozewski

Oh, yes, the numbers differ a lot.
To be honest, off the top of my head I do not know what is happening.
Recently, the AI bots are massively increasing the usage numbers, that we are not considering yet – we need to figure out how to best solve this within the system – mostly the journals would apply additional tools to filter/forbid such access. Maybe your web analytics package does it already?
We hear from lots of journals that the numbers have increased a lot because of the AI bots…

Maybe you could try to analyze a few things:

  • Was the discrepancy so big earlier, some longer time ago – or have you recently started to use the web analytics package?
  • Would it maybe be possible to chose a day in April to compare it? So that you can see what exaclty is in the OJS log file and if OJS roughly compiles the statistics well from the log file. Or is there maybe a mistake in processing the log files. Also if there are maybe lots of bots there, that maybe OJS does not filter well.
  • Maybe to also double check that writing in the log file is correct, e.g. watch the log file while accessing some article pages and PDFs. Maybe, if possible, to do the same with your web analytics tool.

I would be also interested to know what is happening, and what your further investigation shows.

Thanks a lot!
Bozana

3

See maybe also this thread: Comparison of the results in the statistical tools of the journals .

4

Thanks @bozana ! Perhaps I overestimated the extent to which the COUNTER standard filters for bot traffic.

I suspect there’s a whole rabbit hole of research behind this. I’ll share if I’m able to delve further and discover anything helpful.