Very Large Files in the PKP PLN

@mjoyce,

If we begin adding Very Large Files (say, for example, an 8 gigabyte video file or dataset) to our journals, will the PKP Private LOCKSS Network plugin be happy transferring such large data?

I don’t think the PKP PLN will be able to handle it. OJS embeds video as base64 elements inside the XML, so the resulting deposit XML is likely to be much much larger. Our tools are unlikely to validate XML of that size, or scan the embedded content for viruses.

Maybe @mjordan can give a better answer.

If failure is expected, will it be a soft fail (recoverable, with other content still captured), or a hard fail (breaking, with other content prevented from capture)?

There’s a good chance it’ll prevent other content getting processed, yes.

So, what is the practical limit of file sizes for the PLN?

^ Bump. @mjoyce, @mjordan?

Hi @ctgraham, there are a lot of variables that will influence the maximum size of a individual file coming out of an OJS instance and moving through the PKP PLN pipeline. @mjoyce will contact you to get some sample data so we can test.

@ctgraham, I think this discussion is still relevant, can you remember something about it? :slight_smile:
As far as I know, the full deposit is constrained by 1GB, other issues might be raised by safety timeouts.

I’ll raise this discussion in another place, but I guess we won’t be able to come out with a proper solution.

We might increase the deposit limit to 2GB, but it’s still a mystery what to do about anything bigger than that, the current behavior is to block them.

I’m not sure if it’s feasible (not technically, but conceptually) to split the deposit in parts, it will require getting in touch with other people :slight_smile:
Anyway, as there are very few of such cases, I don’t think it’s something crucial to be handled at this moment.

Best,
Jonas