[Maybe a bit off topic] My month with OJS, some general feedback to whom it may concern at PKP

Software Support
1

First things first, I’m not a web developer, I’m a social worker. In fact I know very little about coding but have some experience working with WordPress here and there.

As an editor’s assistant at a journal of social work I’m in charge of finding a tool to get our journal into the 21st century. Also I’m looking into starting an open access journal with some friends of mine (of course: on a budget, eg. with no budget at all).

To be very clear: I herewith do not wish to criticise/attack PKP per se nor any of its staff who are all doing an amazing job whilst being ‘on a budget’ (which I guess is much much smaller than WordPress). Also I am very thankful that such a project even exists. I have not found anything else like it.

However there are a few things I noticed in the month of testing OJS and hanging around on this forum I’d like to talk about. Please see these remarks as a ‘contribution’ to improve PKP.

#1 Problems

In this month of using OJS (3.2.0-1 and 3.2.0-3) I have reported a few issues among which

a) List of countries all over the place instead of alphabetical order
b) Javascript error dialogs displaying while reviewing a submission
c) In the discussions reviewers could see the author’s name despite being double-blind review
d) Instructions/Descriptions missing from review forms while reviewing

Again: I’m no web developer so the testing was purely ‘as a user’.

That’s quite a few problems there for one month I think (some where already know, some weren’t). In my opinion all of the above could and should have been caught by the developer team before releasing to the public doing what I did: register as a section editor, as an author/submitting an article, as a reviewer and test the whole process. Which is annoying and time consuming yes but all of the above are basic functionalities and should be quite apparent in a normal testing process.

The PKP Team dealt with my reports impeccably they were all solved within 24-48h after reporting. So great job thanks for that! But the versions shouldn’t be released to the public with such basic issues. Maybe you could create a group of users to test before releasing to general public and I’d be glad to do what I can (should we decide to keep using OJS).

#2 Updating

The process of updating seems to be really daunting for loads of people (or should I say everyone?). Loads of people still with version 2 but even within OJS 3 people do not tend to update. As is quite visible looking at the forum topics.

For one I think it has to do with my #1 here or to put it into movie words “OJS is like a box of chocolate, you never know what you get” (in 3.2.0-1 description/instructions worked, then 3.2.0-3 they didn’t, so you never know).

Another reason might be the updating process itself. I find it to be quite easy - up to a certain point.

Get config.inc.php, favicon.ico and the public folder, throw them in the updated version, turn installed Off in config.inc.php, visit site, hit upgrade and turn installed On again.

That’s the easy part I can do.

These parts however:

  • Synchronize new changes from config.TEMPLATE.inc.php to config.inc.php
  • Be sure to review the Configuration Changes section of the release notes in docs/release-notes/README-(version) for all versions between your original version and the new version. You may need to manually add new items to your config.inc.php file.

I already struggle. Am I supposed to open my config.inc.php and config.TEMPLATE.inc.php side by side and go through all 520 lines one by one to see if something has changed and then copy&paste between them? (if you could answer that I’d be glad)

Wouldn’t it be possible to store these settings somewhere else and have changes applied via Settings as an admin so I wouldn’t have to mind about config.inc.php except maybe install on/off?

Last but not least reason I think for people not updating: just like me, they are not web developers and may be a bit scared of something going wrong. There’s one person on the forum asking if anyone knows anyone who could help updating from OJS 2 to OJS 3 and no answer. That sums it up quite well in my opinion. If I’d ever run into I’d probably be stuck, which is why I tend to have a testing installation where I try things first. Couldn’t you offer paid support?

In WordPress all I have to do is hit update and it does it all by itself (different budgets and coding, I know). Maybe there’s a foundation out there that would happily throw in something?

#3 The way people use it

There seem to be few people testing it with different roles like I did. Phrases like “I’m admin, I have no access as a reviewer or as an author myself” which leads to little feedback for PKP.

Information is usually not passed on from end-user to admin to PKP. Information may get from end-user to editor or section editor and get stuck there.

Of course I do understand that it’s extremely complex (made out of 18’000 files with hundreds of lines of coding each) and “on a budget” it’s hard to manage and there’s always room for improvement. I just wanted to share my thoughts as a hopefully helpful feedback.

Thank you for all your hard work and for offering this product,
Matthias

3 Likes
2

Hi @alaskadream,

I appreciate the feedback. You’ve joined us at a particularly breathless moment – the OJS 3.2.0 release has been atypical, with a very major turn-over in the submission data model (and thus a large portion of the code) due to the addition of metadata versioning, which represents the end of an almost 4-year dev cycle. This introduced a lot of new code to the released product all at once, with a considerable hit to stability.

The 3.2.0-1, -2, and -3 builds represent our efforts to quickly address the elements that are identified and get the 3.2.0-x line into shape. I suspect you would have hit fewer of these speedbumps had you joined us with OJS 3.1.x and I hope (and maintain) that we’re returning to similar territory again.

Compared to a project like Wordpress, we have several extra challenges – the application is simply much larger and more complicated, and our user community has fewer resources to contribute, test, etc. Your experience as a user encountering web developer territory is not uncommon.

Testing has long been a challenge, and we’ve made significant strides via ongoing UI/UX testing and automated testing. But with a large application comes a lot of terrain to cover, and our test coverage is currently limited. Automated testing will only check what it’s told to check, and as you can see, the issues that you’ve encountered made it past those checks. Unfortunately we don’t haven’t had the resources to run a dedicated testing team (pre-release testing is covered as much as we’re able through our dev, hosting, and UI/UX groups) and I don’t think our community is well resourced enough to perform outside testing e.g. via a beta process. Longer term, we are gradually restructuring the application in ways that will facilitate unit testing, such as by building on a REST API dividing the front and back end rather than a less structured monolith.

On upgrades, I’m in the process of overhauling some of the underlying tooling that has been around since OJS 2.0.0 (15 years ago!). This will be released in starting in OJS 3.3 and will very considerably improve the upgrade experience, which has long been a thorn in our community’s side. Our tentative plans are to release OJS 3.3 late this year. OJS won’t then be as painless to upgrade as Wordpress (which is truly a remarkable example) because of its relative size, but it will definitely be a different experience than what you’re seeing on the forum, which is primarily a lot of pain and difficulty getting from 3.1.x to 3.2.x.

On feedback, you’re absolutely right that we often don’t get feedback from users who are encountering problems or proposing improvements or changes; that’s part of the reason this forum is so valuable to us (and why we depend on it to help us improve the software). Please do continue to bring us feedback and we’ll do our best to honour that effort.

One way to spare yourself the care and feeding of an evolving web application is to find a reputable service provider that knows OJS. There are several third parties that do this, and PKP does offers its own paid services. If you go that route, when choosing a provider, I’d encourage you to find one that actively participates as a healthy contributor to the ecosystem – that could mean they’re improving the software via Github, or helping support us financially, helping host a regional user community, etc.

So to summarize…

  • Thanks for reporting the issues you’ve encountered, and I hope we’ve been helpful in getting you through the bulk of them quickly.
  • We hope and expect that the 3.2.0 releases are through their most volatile stage, and we don’t plan to return to this level of volatility soon. (We’ll be publishing updated roadmaps in the near future, so stay tuned for that.)
  • We do have long-term plans in place to address some of the big stumbling points, particularly testing, upgrades, and overall complexity. These will take some time to complete but I think you’ll start seeing the benefits early.

Let me know if you’re curious about any particular aspect of this.

Thanks,
Alec Smecher
Public Knowledge Project Team

4 Likes
3

I didn’t expect a reply at all and certainly not such a long one. Thank you :slightly_smiling_face:

In terms of testing I’m pretty confident you’d find 10-20 people who are willing to test pre-release for free as a contribution to the project.

Kindest regards,
Matthias

4

Hi @alaskadream,

Thanks, we have tried this before with mixed results but I’ll see if we can give it a crack again. We’ve worked out some practices e.g. around code and translation freezes that would allow for some external testing prior to the release.

Regards,
Alec Smecher
Public Knowledge Project Team

5

Regarding this part, there is a very effective way to do it:

  1. Go to portableapps.com/apps and download WinMerge. You don’t have to install it, the executable will only extract it to a folder of your choice. You can click the EXE file to run it.
  2. Click “Open” and select both the old config file and the template config, and then “Compare”.
  3. WinMerge will show you everything that is different between the two files. You can then right-click on the lines that you want to transfer to the other file and select "Copy to right (or left).
  4. In my case I assumed the Template would be the new file and pulled the relevant info from the old config (user names, passwords, etc.). Then I renamed both (the old as a backup and the template as the “official” config).

Thats’ it. I hope it helps.

1 Like
6

Thank you for your answer Renato.
Unfortunately I’m on a Mac so can’t run the application you suggested, however I found this Cross-Platform application which does the same SourceGear | DiffMerge (hasn’t been updated for 7 years but works for me) and also found that BBEdit has the same capability Bare Bones Software | Products
Thank you for the hint!

1 Like
7

Hello @alaskadream,

I think your Mac has a DIFF tool via command line, if you know how to use it…
You should compare the config.inc.php from the old and new versions and check for differences. Since we run it on the server, we use the command line to compare with DIFF on Linux/Ubuntu/Debian. Simply type vimdiff and the file names and two windows will show up, automatically comparing files, hiding repeated lines on both files. I tend to use it also for translations, comparing the English with Portuguese… Allows me to check and add the variables on the same lines as the original.

Ibict has always been a PKP partner, although a new cooperation agreement was just recently signed (2018 or 2019, not sure). A testing cooperation could have been added, if it isn’t in place. When I worked at the Dev department (2002 to 2012), we used to test as much as we could and provide feedback to PKP. Since 2014, I’m currently head of the Editing Section responsible for maintaining our own OJS and our publications, so, testing is going to be difficult for us, as we are users now.

We try our best to collaborate and your feedback is very important, especially to maintain open source software such as OJS.

@asmecher, I’m waiting for those upgrade improvements!!!

8

Hi @alaskadream,

FYI, at your suggestion we’ve put the first release candidates for the forthcoming OJS/OMP/OPS 3.2.1 releases on the Announcements area of the forum: OJS, OMP, and OPS 3.2.1 Release Candidates (Testing Only)

We’ve generally tested these only internally, but there’s no harm opening them for community feedback as well. It’s a bit of a short release cycle, but perhaps we can get in the habit of doing this.

Thanks,
Alec Smecher
Public Knowledge Project Team

2 Likes