For higher education IT departments, it's a familiar scenario.   

Your team is in charge of managing hundreds of websites that support different mission-critical areas of your organization. You are responsible for updating the sites, keeping them all up and running, and optimizing their performance. Additionally, each site has its own small team updating content, creating new features, and working within the Drupal admin system. They are predominantly non-technical users, and reach out to your team whenever they need help.  

Over the years, we've worked with many teams facing these exact challenges. Through our clients' experiences, it is quite clear that your ability to effectively manage your platform is substantially determined by your architecture. Drupal 8 core includes multisite functionality and is a popular choice that works well up to a certain number of websites. For a large-scale university platform with hundreds of sites, our preference is to use a shared codebase that provides more fine-grained control. This is exactly what Pantheon Custom Upstreams provides.  

Why is that? Read on...

Challenges for Deploying Higher Education Websites

While each of our clients is unique, they also share many of the same challenges. For our higher education clients with large, complex Drupal platforms, we encounter some key pain points time and time again.  

• Redundant implementation. When fixing a bug or developing a new feature, the wrong architecture can leave you re-implementing and testing the solution on a site-by-site basis. This can turn a 5-minute fix into a project that takes several hours.
• Ungainly testing. Testing and deploying Drupal updates across hundreds of sites can get very tedious and time consuming. There are weekly releases of updates for contributed projects, twice monthly core updates, and additional critical security updates when needed, each of which must be tested against every site.
• Difficult system-level updates. System updates like PHP versions are infrequent, but critical for maintaining site security, and can require code adjustments to keep things working correctly. These adjustments must be discovered, implemented, and tested within both the platform code and per-site customizations, making these important updates a potentially daunting task.
• Significant risk of introducing new bugs, conflicts, and inconsistencies. When hundreds of sites share a codebase or installation profile, but also have their own customizations, any change to the platform code has the potential to cause problems at the site level. A new platform feature, for example, might work great on most sites, but conflict with a custom module used on a quarter of the sites, several of which have their own revisions of the module that may cause additional issues...it can get quite messy. 

Possible Architectural Solutions

It's clear that these challenges are big enough to deserve a good solution. There are two main approaches to architecting large-scale Drupal platforms, each with their own advantages and disadvantages.

Multisite Installation

With Drupal's multisite setup, separate, independent sites can be served from a single codebase, each with its own database, configuration, files and domain.   

Multisite has the advantage of being relatively simple and intuitive to manage. With just one codebase, there is less code to administer, and updates are rolled out to all of the sites at once. Additionally, individual sites can incorporate their own themes and modules as needed. 

However, it also comes with some disadvantages, which are amplified when managing large-scale platforms:

• Having a single codebase and single server means you also have a single point of failure. An issue in your server or codebase will affect all of your sites, potentially taking down the entire platform.
• Deploying updates or fixes to selected sites is difficult because all of them share the same codebase. Anything deployed for one site has the potential to affect any other site on the platform.
• Drupal updates deploy to all sites at once, and cannot be reverted on a per-site basis. This means that either all sites must be tested before any can receive the update, or some sites will be in a broken state until they can be fixed individually.
• Teams come up with homemade solutions to circumvent some of the challenges involved with maintaining a Multisite installation. These solutions can be difficult to maintain if documentation is not properly written, or if they function in unexpected ways.

Shared Codebase

A shared codebase can include a set of modules, an installation profile, a Drupal distribution, or other code that is used for multiple sites on your platform. Rather than all running off of the same code, however, each site has its own copy. Individual sites can also have their own customizations if they don't conflict with the shared codebase.   

Some advantages of this approach are:

• Bug fixes, new features or even security updates can be added to the shared codebase and pulled into each site when its team is ready to work on it, rather than all sites updating together on a forced schedule.
• Customizations can be done for each individual site without having unintended effects on other sites.
• Avoiding a single point of failure at the server level eliminates the risk of an error or conflict taking down the entire network of sites.  

Some disadvantages for this approach are:

• Applying shared codebase updates to each site individually requires doing work for each site, and can take more time.
• Multiple servers or hosting environments must be administered and kept consistent across the platform.

Custom Upstreams: The Pantheon Approach to a Shared Codebase

Pantheon hosting platform has introduced the concept of Custom Upstreams to solve scenarios like the one described above. With this solution, you can standardize design and functionality across multiple Drupal sites that share a custom upstream managed by your organization. This governance does not compromise the customization of individual sites. Instead, it offers a sustainable and scalable solution for handling updates across massive site portfolios. When used well, Custom Upstreams frees up developer time to focus on higher-impact work, and makes it possible to spin up new sites quickly, without repeating foundational work.   

Deploying updates to the upstream is as easy as pressing a button in the dashboard to pull the update into the development environment and then deploying this update to test and live environments. Further automation is possible with Terminus, even to the level of running a single script to update and deploy all of the sites at once. There is a lot of good documentation from Pantheon about creating Custom Upstream and related topics.  

Custom Upstreams within Pantheon provides solutions for the challenges presented and also the disadvantages of the architectural solutions explained above:

• There are no servers for you to administer.
• The PHP version could be changed globally and overridden per site if needed using the pantheon.yml file.
• Security updates, new features and global bug fixes can be applied to the custom upstream and pulled into all of the sites either manually or through some automation.
• There is no Single Point of Failure at the server level, and the Pantheon folks are really good at managing their infrastructure.
• A lot of automation can be done by using Terminus and Terminus plugins.  

Final Thoughts

There are always tradeoffs to be made when attempting to balance power with complexity. Drupal's Multisite--a solution that is initially simple and intuitive--can evolve into a quagmire as the platform scales up to meet the requirements of higher education institutions.  

It has been through our years of experience with such institutions that we have established our preferred best practices for architecting and managing these exceptionally challenging platforms with a shared codebase. Pantheon's Custom Upstreams provide tools and automation to relieve some of the complexity of administering independent sites and their environments, while retaining the powerful benefits of such an architecture.   

If you'd like to learn more about our toolbox for large-scale Drupal platforms, check out How to Set Up BLT and GitLab CI to Work with Pantheon Hosting (Part 1) and (Part 2), or just click the green chat bubble to ask one of our team members.