Lessons Learned Upgrading the Largest Moodle in the Southern Hemisphere.

Cameron Ball, Catalyst IT Australia

After delivering a stand out talk last year (“Performing Brain Surgery on the Beating Heart of Moodle”), Cameron Ball, Catalyst IT Australia’s Senior Developer, was back at the Moodle Moot Global 2023 with yet another impressive presentation – Lessons Learned Upgrading the Largest Moodle in the Southern Hemisphere.

“Upgrading Moodle is a challenge at the best of times,” says Cameron. “With so many “interesting” plugins out there, not to mention core itself, any upgrade is sure to have some degree of challenge.”

“Now, imagine turning that up to about 11, when upgrading the largest site in the Southern Hemisphere.”

“More specifically, it was an upgrade of a monstrous Moodle installation from version 3.9 to 4.1 . We are talking almost 5.5 terabytes of data!”

“With no shortage of unusual plugins, obtuse customisations, and plenty of “fun” database states, a challenge for even the most seasoned Moodle veterans was guaranteed.”

Upgrading large sites often comes with many risks – there’s a lot of code, a lot of content and a lot of DB cruft. Technical debt is another issue and is common.

‘If you ignore your problems long enough, they will go away’ is not the philosophy we believe here at Catalyst.

The four core phases Cameron and his team used to approach this upgrade included:

Phase 1. Core customisation analysis

Manually audit every single customisation to the codebase. For each customisation, work out what it is and decide if it is needed in the new codebase. Identify every piece of code in the codebase which is not part of vanilla Moodle 3.9. Fortunately, there is a really good tool for this. It’s called Git.

Phase 2. Plugin analysis

Manually audit every plugin in the codebase. Again, for each plugin, work out what it is and decide if it is needed in the new codebase.

Phase 3. Configuration migration

Manually audit every single config value (core + plugins) in the DB. For each config, work out what it is and decide if it’s needed in the new DB. We also needed to bring across things like custom User profile fields, Roles (assign, override, switch etc.), Web services, Grade scales, scheduled tasks and more.

Phase 4. Quality assurance

“Of course there’s always a Quality Assurance phase,” says Cameron. “We triage the issues and fix the critical ones first up, followed by everything else.”

The technicality and detail of Cameron’s presentation are beyond the scope of this blog but here are the key lessons and tips in a dot-point summary:

• Only automate things when it makes sense – as programmers it’s tempting to script everything. But sometimes ( and often) that can end up being more work than a manual solution.
• Spreadsheets are still one of the best tools for keeping track of things. So much time was spent looking at and updating spreadsheets for this project.
• Git as a tool is good, but only as good as the commit messages. Always put issue numbers in commit messages if it’s possible! And include as much detail as possible. A lot of time was spent trying to pick apart obscure commits with messages like “Changes.”
• Comparing configuration values between environments is reasonably easy with the right tooling. Thanks to a bespoke script, it was easy to compare every single config value between environments and decide on the right one to use. All without needing input from anyone else.
• Moodle’s role restore functionality is OK, but it’s very limited in that it cannot replicate “role assignment matrices” (i.e. which roles can assign other roles); so be wary that it’s not as simple as “restore roles in the new site”, there is extra work that needs to be done.
• Last but not least, do NOT trust AI with it! Hire some experienced humans to do it.

“We’ve tried asking Chat GPT for help and here’s what it came up with,” says Cameron:

Catalyst IT are proud winners of the Moodle Global Partner of the Year and LMS contributor of the Year Awards 2023. Learn more here.

Our team are regular contributors to the Open Source Community and have developed a great number of improvements to Moodle and other Open Source products by contributing to both the core codes and the development of various plugins for all the community to benefit from.

For your enterprise LMS and CMS implementation, hosting and maintenance needs, contact our team today.

PS: By the time we wrote this blog, Cameron was awarded the Best Presentation at the Moodle Moot Global 2023 and it is now available for you to watch on YouTube. Click here to view.