Managing Software Updates for Hundreds of Websites

Manually maintaining a Drupal website is time-consuming, especially for a small team already busy with new features and bug fixes. When the Drupal Security team announces a new security update, this requires one or more members of the development team to:

1. Stop what they are currently doing
2. Apply the update and create a new pull request
3. Test the security update

Add in other non-critical updates, and a development team will spend a significant amount of time on something that, while necessary, doesn't seem to add much value. If you have multiple websites and codebases, this problem becomes even worse. It could easily become someone's full-time job to do nothing but package updates.

Why you should automate these updates

You need to update your codebase. It's nonnegotiable. It keeps your website secure and might even address bugs your team has been working on or bugs they haven't yet identified. But you can make the process much easier without the huge administrative lift every time.

Automating your updates solves a lot of problems. When set up correctly, automation will update each package independently, one pull request at a time, making identifying and fixing regressions easier. If you combine automated updates with end-to-end testing, you will have more confidence that new problems are not being introduced.

One of our support clients has twelve different Drupal repositories (as well as multiple WordPress websites) with an in-house team of three developers and one designer. Doing only security updates on these websites took us 30 hours per month. When the time came to upgrade from Drupal 8 to Drupal 9, there was a large backlog of non-security updates to perform, which slowed down the process.

For each update, the client also did extensive QA testing and scheduled that testing weeks in advance. If they discovered any problems, all other releases would be blocked. It would take another few weeks to review the next round of fixes because of the scheduling, even if it were a one-line change.

Once these updates were automated using Renovate, it was just a matter of keeping up with all the latest changes, improvements, and bug fixes released with minor version updates. With end-to-end and visual regression testing, a developer only needs to be involved when an automated test or build fails. Having Renovate (properly configured) is like having an extra developer on your team.

What is Renovate?

Renovate is a tool for automating the updating of dependencies in software projects. It can scan your software repositories, identify out-of-date packages, create branches, and submit pull requests for each one. Renovate supports a wide range of programming languages and platforms like PHP and NodeJS.

To use Renovate in its basic form:

• Set Up: Install Renovate from your package manager or the platform-specific distribution.
• Configuration: Create a configuration file, renovate.json, in the root of your repository. This file dictates how Renovate will behave – which dependencies to update, how often, etc. See the example below.
• Running: Execute the Renovate tool, which will scan your repository for outdated dependencies based on your configuration.
• Review & Merge: Renovate creates pull requests (or merge requests) for each dependency update. Review the changes and merge them if everything looks good.

You can leverage many other advanced features and configurations, such as grouping multiple dependency updates into a single PR, scheduling when updates should occur, and more.

Setting Renovate up varies between Git hosting providers. The easiest is GitHub because it's accessible in the GitHub Marketplace.

Are there other options?

We've also tried Violinist.io and Dependabot.

Dependabot's configuration was very limited for our needs, and some features have been removed since being acquired by GitHub. Overall, it's a little harder to work with.

We used Violinist.io on some projects that were not using GitHub. Configuring Dependabot for BitBucket and GitLab takes more work because you must set up custom runners or pipelines. 

Our experience has shown that, over time, Violinist.io has failed to create automated pull requests if/when there are minor issues with composer. Even something like an out-of-date patch, though unrelated to the package it's attempting to update, can halt the entire process. Once it fails, it creates a backlog of pending updates. We haven't had these issues with Renovate because of its flexible configuration.

Our Renovate configuration and preferences

Renovate integration is free, and once it is connected, it will open a pull request for the bare minimum configuration to start looking for package updates. We build upon this base configuration to group some packages together and set up rules around automerging. 

For example, we allow automerging during off-peak hours to avoid rebasing pull requests when others are actively working on the site. We also set branch protection rules that require certain tests to pass before Renovate is allowed to automerge. We only allow minor and patch releases to automerge because we want a developer to review any major version upgrade.

Here's an example of a renovate.json we've been using on our projects.

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": ["config:base", "group:symfony"],
  "timezone": "America/New_York",
  "automergeSchedule": ["every weekend"],

  "rebaseWhen": "auto",
  "platform": "github",
  "baseBranches": ["main"],
  "prConcurrentLimit": 2,
  "rangeStrategy": "bump",
  "branchPrefix": "renovate/",
  "automerge": false,
  "packageRules": [
    {
      "matchManagers": ["composer"]
    },
    {
      "matchManagers": ["npm"]
    },
    {
      "matchPackagePrefixes": ["stylelint"],
      "groupName": "Stylelint packages"
    },
    {
      "matchPackageNames": [
        "drupal/core",
        "drupal/core-recommended",
        "drupal/core-composer-scaffold"
      ],
      "groupName": "Drupal Core"
    },
    {
      "matchPackageNames": ["lullabot/drainpipe", "lullabot/drainpipe-dev"],
      "groupName": "Drainpipe"
    },
    {
      "matchPackagePrefixes": ["gulp"],
      "groupName": "Gulp packages"
    },
    {
      "matchPackagePrefixes": ["jquery"],
      "groupName": "jQuery packages"
    }
  ]
}

Conclusion

There is no end to updating packages. Configuring Renovate allows more time and expertise to be spent on improving your website instead of constantly performing maintenance.