Submodule Update Best Practices

After the last few core upgrades to OJS and OMP (currently on 3.3.0-14), we’ve noticed some compatibility issues between the submodules (e.g. the shared PKP library) and the core OJS/OMP code. Our upgrade approach is as follows:

  1. git rebase onto new tag
  2. git submodule update --init --recursive
  3. composer/npm install
  4. database upgrade

These compatibility issues generate various fatal PHP errors, which I’ve been resolving by going into the offending submodule(s) and switching to a more recent branch (often stable-3_3_0).

Is there anything we’re doing wrong here? My assumption was that after rebasing onto a newer tag and running the submodule updates, the submodules would be pointing to the appropriate commit (as bundled into the release).

Any clarification would be greatly appreciated.

Best,
Patrick

Have you made your own submodule updates in your local branch? If so, won’t a git rebase reapply those submodule updates overtop of the latest upstream branch, effectively dropping the upstream submodule updates?

In our environment, when we manage local modifications (which we are trying to minimize in favor of upstream PRs and plugins), we maintain a feature branch with that local which we re-apply over a clean git checkout of the latest upstream branch. If the local lives in a submodule, we do a submodule update from the upstream checkout, and then re-apply the feature branch within the submodule, with a new commit in the parent.

Hey, thanks for the reply.

We haven’t made any custom changes to the submodules. All I’d like to do is to update them to the latest stable commit. My assumption was rebasing our main branch onto the latest tag and then running git submodule update would move the submodule HEADs to the appropriate submodule commit in the superproject for that specific tag.

Is this not correct? Do I need to run a git pull in each submodule after rebasing?

Can you give a specific example of a past rebase command? I think git submodule update --init --recursive should automatically be fetching the latest submodule commits and checking out the most recent submodule hash, so git pull in the submodule is not needed. But, I’m not clear on the nuance of rebasing your branch onto the latest PKP tag, vs rebasing PKP’s tag onto your branch. If you haven’t made your own submodule commits, the rebases might have the same effect, but I’m concerned about your statement “I’ve been resolving by going into the offending submodule(s) and switching to a more recent branch (often stable-3_3_0)”. I interpret that as a submodule commit in your branch that you do not want to propagate to the rebase.

Sure,
Here’s the procedure I followed last time around. For the record, our main branch was forked from PKP about three months ago. I then added our custom themes to it, but haven’t customized any other part of the code.

git checkout <our-main-branch>
git fetch <pkp-repo>
git rebase --onto 3_3_0-14 3_3_0-13
git submodule update --init --recursive

You’re right about my submodule commits. I’d much rather not carry them forward. They were made in desperation when users complained about the site crashing on them. Digging into it, I discovered that the submodules had not fetched the latest submodule commits after taking the steps above.

That’s why I went in to several submodules, switched branches, and pulled down. That solved the code incompatibility problems, but as you pointed out, I now have custom submodule commits in my superproject. Ultimately, I’d prefer to rely solely on git submodule update to keep the submodules in sync whenever I rebase.

I just don’t understand what I’ve done wrong.

Consider:

git clone https://github.com/pkp/ojs.git
cd ojs
git checkout 3_3_0-0
git submodule update --init --recursive
git submodule add https://github.com/ulsdevteam/pkp-clamav.git plugins/generic/clamav
cd plugins/generic/orcidProfile/
git checkout stable-3_3_0
cd ../../..
git add plugins/generic/orcidProfile/
git commit -m 'Local submodule updates'
git rebase --onto 3_3_0-14 3_3_0-0
git submodule update

Here I have checked out 3.3.0-0, added my own submodule (clamav), and updated orcidProfile.

Then, I try to upgrade from 3.3.0-0 to 3.3.0-14 with the rebase command you used. The changes from 3.3.0-0 to 3.3.0-14 get applied as you expected, including the unchanged submodules, but my commit of “Local submodule updates” has been re-applied. This adds clamav back in, but also reverts any changes 3.3.0-14 made to orcidProfile in favor of my manual update.

$ git log --oneline | head
df98973 Local submodule updates
313ada3 Final commits for 3.3.0-14 release
4be9a21 Submodule updates
80520ee Back-port zh_TW from main to stable-3_3_0
45e0212 Scatter zh_TW COUNTER locale keys back to stable-3_3_0
5889797 Merge pull request #3733 from bozana/8587-stable
9d57238 pkp/pkp-lib#8587 Consider new resourceType JournalArticle in DataCite XML
549c69d Submodule update
c88345c pkp/pkp-lib#8579 Fix type error pickiness with PHP8.1
7517d19 Merge pull request #3690 from defstat/pkp/pkp-lib#7337_330

One possibility could be to maintain a commit that represents only your plugin adds, and leave the PKP plugin updates uncommitted. This would allow you to cleanly rebase in a new checkout (or after discarding uncommitted changes).

Another possibility would be to maintain a feature branch with just your plugin adds, and merge that branch into a fresh checkout of the latest PKP release.

My personal approach is to checkout the latest PKP stable and then query the plugin gallery for the latest advertised releases of the plugins, adding them as a commit on the latest stable. This branch/tag is then discarded with the next release.