Ah, I think we figured it out.
A good user and group for the submission files on our server is apache ojs. During upgrade, the upgrade script was run by ojsadmin user. So, when the upgrade script created any directories, for example, new locations for PDFs and other submission and revision files, it assigned ojsadmin user and ojsadmin group to those new directories. Because it’s only directories created by the migration script, that explains why some author revisions can be uploaded (ones where there was no file before upgrade so the application is making the target file during upload), but some can’t (ones where this is an additional file and the upgrade script made a target directory and put an existing file into it). Also, I hadn’t found it in testing, because I clicked through processes where the application created the directories. I was clicking things through the submission process from different points but not from having an author revision then adding a supplementary file to that author revision.
The upgrade script was making new directories like /journals/xxxxxx/articles/xxxxxx/submission/review/revision/ . Those new directories within review/
And, it affects only one journal, because their publishing schedule is such that the upgrade fell right during a 2 week author revision window. So, some authors uploaded, then made additional edits, and want to reupload… to a directory created when the upgrade script moved their first upload.
To fix it, we will recursively change the group and owner on submission files just for that journal before the 4th of July 4 day weekend, and then for all journals after the 4 day weekend so that we are available after changes in case of anything unintended.
By the way, I wish this were more spelled out How should file permissions be set? - #2 by ctgraham . I guess a lot is server specific - what workgroups exist. Or maybe in the https://pkp.sfu.ca/ojs/UPGRADE have some note about setting user and group after the upgrade script does it’s thing, since it subtle enough we didn’t flush it out in testing.