PDF galleys keep getting removed randomly

[supreme]

PDFs of some articles in a published issue keeps getting deleted/removed. This seems to happen randomly, in current and past issues. Even after editing the affected submissions and reuploading the removed PDFs, some still get removed after some time.

When an affected submission is viewed from the dashboard, the galley created is still there but without the PDF file.

Website running OJS 3.4.0.3 with Manuscript child theme.

This has been happening for a year now. How can this be fixed please?
See the attached screenshots below.

&

[abadan]

I have seen cases where PDFs were not displayed in published editions when they were linked to “Article Components” of the “supplementary files” type.

I suggest comparing the article component associated with a PDF galley that is being displayed with one that is not.

[supreme]

We usually make sure to set the “Article Components” type to “Article text” before publishing.

The PDF galleys of the affected articles do show immediately after publishing them, but they somehow get removed after some time (days).

[asmecher]

Hi @supreme,

That’s very odd behaviour and hard to guess what’s happening from here – but as a start, I’d recommend upgrading to the latest 3.4.0-x release. A lot of bugs have been fixed since 3.4.0-3.

Regards,
Alec Smecher
Public Knowledge Project Team

[alexxxmendonca]

We had a similar problem with OPS 3.2 (details here: [OPS] "Galleys" section doesn't load · Issue #5740 · pkp/pkp-lib · GitHub)

I agree that first action point would be to upgrade your OJS to the latest version.

[PaideiaStudio]

We are experiencing the same issue. PDFs of some articles in a published issue keeps getting deleted/remove. We are currently using OPS 3.4.0.8.

I’ve followed this issue: [OPS] "Galleys" section doesn't load · Issue #5740 · pkp/pkp-lib · GitHub

I dont know if this is still a problem (About 80 results here):

SELECT *
FROM publication_galleys as pg1
WHERE pg1.submission_file_id
IN ( SELECT pg.submission_file_id
FROM publication_galleys as pg
LEFT JOIN submission_files as sf ON (pg.submission_file_id = sf.file_id)
WHERE pg.submission_file_id IS NOT NULL
AND sf.file_id IS NULL );

There are no errors in the log file and no detected pattern. At this moment we have uploaded again all the files but it just a matter of time before they disapear again.

Any clues?

[asmecher]

Hi @PaideiaStudio,

This is still too vague for me to do much investigation, but just on a whim, do you use the XML import export tools? Can you check the access log for NativeImportExportPlugin to see if it appears with a POST request?

Regards,
Alec Smecher
Public Knowledge Project Team

[ugp2]

Is that JOIN correct? Shouldn’t it be:

*LEFT JOIN submission_files as sf ON (pg.submission_file_id = sf.**submission_*file_id)

[ugp2]

We have the same issue. The PDF’s and the submission_files-records disappear.

[asmecher]

Hi all,

@ugp2 and @PaideiaStudio, I think @ugp2 is right – the join on submission_files should use submission_file_id, not file_id.

Unfortunately I can’t replicate this behaviour, so I’ll need someone who is experiencing it to investigate further and see if they can determine more about the causes and conditions.

Regards,
Alec Smecher
Public Knowledge Project Team

[ugp2]

We have noticed that when a PDF is deleted (changed date-time of the relevant directory), the following message appears in the other_vhosts_access.log: "POST /$$$call$$$/grid/article-galleys/article-galley-grid/delete-galley?submissionId=99999&publicationId=99999& representationId=99999 HTTP/1.1“ 200 /537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36”

However, I see no connection between the submission or galley mentioned there and the deleted PDF. It concerns a completely different issue and even a different magazine. But maybe it will help you @asmecher find the cause?

Regards,

Pia Spriensma

[asmecher]

Hi @ugp2,

That does help, but I’ve taken a look through the code and still can’t spot where there is a mismatch. Do you by any chance have a database backup taken after that galley was created, but before it was deleted?

Thanks,
Alec Smecher
Public Knowledge Project Team

[ugp2]

Hello @asmecher ,

Yes we do, but they are very large. (zipped > 1 GB) Where can we store it?

And we are running 3.4.0-8. Is it possible that this is the cause?

Kind regards,

Pia Spriensma

[asmecher]

Hi @ugp2,

If you’re willing to send me (privately)…

• A download link to the database (e.g. via Google Drive),
• The submission and galley IDs for the delete-galley request, and
• Any information, e.g. file or pathname, for the galley file that gets deleted,

…I can investigate on a local copy.

Thanks,
Alec Smecher
Public Knowledge Project Team

[revistaiberociencias]

Tambien estamos experimentando perdida de galeradas, en algunos casos pdf , en otros el html y en algunos ambas galeradas, de manera aleatoria en una version actualizada a 3.4.0.8, los he vuelto a subir pero después de un tiempo lo he vuelto a experimentar tanto en volúmenes actuales como en anteriores, puse el jobrunner = off (config.inc.php) y descanse un tiempo pensando que puede ser debido a una limpieza automatica del ojs pero me afecta a la hora de depositar los doi o cosecha de datos, por lo que debo habilitar y desabilitar para depostar los doi

como solucionarlo definitivamente? debo realizar una actualización o instalación limpia?

pensé primeramente que se trataba de un malware ya que había encontrado en la carpeta file de uno de los articulos un archivo php altamente sospechoso, lo depuré y eliminé pero no se realmente si es secuela o un problema de inconsistencias o malas referencias en la bd de ojs el problema actual

ayuda por favor?

[asmecher]

Hi @revistaiberociencias,

I’m working with @ugp2 on this, and others are welcome to jump in too.

So far the challenge has been identifying…

1. The event that triggers the galleys to be removed, and
2. The relationship between that event and the erroneously removed galley.

I think we’ve identified the first part as a POST request to delete a galley. It seems that in some cases, this results in the additional deletion of an unrelated galley.

However, we’re still trying to identify the second part.

What would be extremely helpful to me is the following data:

1. A database dump that was captured before the erroneous deletion,
2. Information about which galley was appropriately deleted (assuming the first part above is correct), and
3. Information about which galley was accidentally deleted

This combination should give me enough to reproduce the issue and debug it. My own attempts run locally have not replicated the problem yet.

Thanks,
Alec Smecher
Public Knowledge Project Team

[KayWP]

We appear to be having a similar problem to @ugp2. I’m currently trying to get out technical team to check if there is a difference between our back ups and our deployed installation (e.g. if something has disappeared in the meantime). That might take a bit of time due to holidays of our technical team.

Do I understand correctly that this query might help us measure how many galleys are ‘missing’?

SELECT * 
FROM publication_galleys as pg1
WHERE pg1.submission_file_id IN (
    SELECT pg.submission_file_id
    FROM publication_galleys as pg
    LEFT JOIN submission_files as sf ON (pg.submission_file_id = sf.submission_file_id)
    WHERE pg.submission_file_id IS NOT NULL
    AND sf.submission_file_id IS NULL
);

[KayWP]

I’m wondering if other installations who have this problem also use the OA switchboard plugin? It was causing errors for us, according to the logs, in at least one of the cases where a galley seemed to have disappeared.

[ugp2]

Hello @asmecher ,

We were able to reproduce the issue.
When a galley is deleted from a journal, it remains on the server. Instead of deleting the intended galley, a random document is deleted.
After implementing pkp/pkp-lib#10758 Delete submission files when a galley is deleted · pkp/pkp-lib@be8e242 · GitHub , it works as intended.
This means we need to upgrade to 3.4.0-9 to solve this problem.

Kind regards,

Pis Spriensma

[asmecher]

Hi @ugp2,

Thanks for the follow-up – that would definitely explain the problems mentioned in this thread, and also why I was unable to reproduce it locally using a newer 3.4.0-x codebase.

Regards,
Alec Smecher
Public Knowledge Project Team