Thanks for the feedback. I was nodding along as I read your two replies because it matches the learning I did this month. I’m proud of that, because this area of OJS is both complex and important to institutional subscribers of our journals. I really appreciate you taking the time to provide really explicit technical details about the process.
I wonder if a few more of these details (specifically for administators) might be worth updating in the docs. I also want to cast a vote for the log reprocesser to be expanded to a) run a separate queue from daily tasks so it can be tracked more carefully and clearly with its own log maybe and b) use a temporary table to store a backup of the stats tables in case the reprocessing fails. There’s no guardrails. So if you recover/want to add one day, you have to do the whole month, which entails deleting 29 or 30 others? I understand the rollup processes, but its a huge data risk and I (to my own fault) got burned, I think. Cautionary but also uncessary for it to be so risky to begin with?
Additionally, I think in the setup guide, new users might helpfull be guided even by a single sentence that access logs should be set by default to retain at least one month to match the statistics process. That isn’t set all by default in apache. It’s forever by default, I believe. But if users set their own too low, that can make reprocessing from the logs themselves more challenging or impossible (e.g., you have to pull access logs from far more backups to get days outside your rotatation coverage.)
So now I feel like I have a firm control of it and, I hope, releatedly counter SUSHI API protocals. SIDEBAR especially for @bozana:
I’ve been building a statistical reporting tool. It’s nearly done and we’re beginning to test it. I’ll be using it to manage further stats checking and creating custom reports. As an example, we have a subscriber having trouble with IP access and checking their institutionally affiliated IP addresses with the system is, to me and my team, a diagnostic helper for our journal managers to see a broad picture of our installation’s activity. I’ll DM you access to it and a test user account, since it’s a private server for access to our installation specifically and requires an API key.
In the end my hand was forced a bit because the OJS 3.4 and 3.5 counter report generator remains unuseable and broken and has no functionality to cover multiple subscriptions owned by the same institution. This is an issue if you have to request reports from nearly 4 dozen different journals. And librarians do NOT want to be given json instead of csv, so the API itself needs some kind of custom post-processing… This is circling back to SUSHI access in v51 where you first helped me begin to get access to the backend of the stats reporting.