Continuous Deployment, Infrastructure as Code, and Drupal: Part 3

Welcome to the last article of this series, which will take you through a set of Kubernetes objects that define the infrastructure of the sample Drupal site that we have been using. The previous article covered the GitHub Actions workflow that orchestrated the Docker image build, push, and deployment to a Kubernetes cluster. This article will demonstrate the Kubernetes setup. 

Note: If you have never heard of Kubernetes before, the official site has a wonderful tutorial with graphics explaining its main concepts.

An overview of the Kubernetes setup

From the official documentation: "To work with Kubernetes, you use Kubernetes API objects to describe your cluster's desired state.

Kubernetes API objects are YAML files that define the state of the cluster. You can find these files in the directory called definitions within the demo repository. Here is a screenshot with its contents:

Kubernetes Definitions

 There are two resources, one for Drupal and one for MySQL, and a Kustomization file that reference them. Here are the contents of definitions/kustomization.yaml:

resources:
- mysql-deployment.yaml
- drupal-deployment.yaml

In the next section, we will explore the MySQL resource that defines how MySQL gets deployed and another one for Drupal.

The MySQL resource

The file definition/mysql-deployment.yaml defines how MySQL gets deployed and runs within the cluster. It is composed of three Kubernetes objects:

• A service used to expose port 3306
• A persistent volume claim used to request storage to host the database
• A deployment, where you define how the MySQL application gets deployed and started

Taking a look at them one by one, first, here is the service:

apiVersion: v1
kind: Service
metadata:
  name: drupal-mysql
  labels:
    app: drupal
spec:
  ports:
    - port: 3306
  selector:
    app: drupal
    tier: mysql
  clusterIP: None

Notice the following settings listed above:

• drupal-mysql is the hostname that Drupal will use to connect to the database.
• Port 3306 is exposed, which is the default for MySQL to listen for connections.

• This service remains private within the cluster network via clusterIP: None.

And now, here's how to claim storage for the database:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
  labels:
    app: drupal
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi

In the object above, 20GB of storage is claimed to host the database files, and this storage space is connected with MySQL via a volume in the below object where how to deploy MySQL into the cluster is defined:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: drupal-mysql
  labels:
    app: drupal
spec:
  selector:
    matchLabels:
      app: drupal
      tier: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: drupal
        tier: mysql
    spec:
      containers:
      - image: mysql:5.6
        name: mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql
              key: admin-password
        - name: MYSQL_DATABASE
          value: drupal
        - name: MYSQL_USER
          value: drupal
        - name: MYSQL_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql
              key: nonadmin-password
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pv-claim

The deployment object is longer than the other two, mainly because it requires further configuration to define environment variables that Drupal will use to connect with MySQL. Two things are worth highlighting:

• Passwords were obtained from secret objects, which are managed by the Kubernetes cluster. A secret was created to host the root and non-root passwords via the command line by following these instructions from the official documentation.
• We mounted the persistent volume claim that we defined in the previous object at /var/lib/mysql, which is where MySQL stores its data. This will persist the database even while pods are recreated during deployments.

That's it for MySQL! Now it's time to check out the Drupal application.

The Drupal resource

The Drupal resource has the same Kubernetes objects as the MySQL one with a slightly different configuration. Additionally, it defines a CronJob object to run Drupal's background tasks periodically.

You can find the resource with all of the following objects in the repository under definitions/drupal-deployment.yaml. In the service below, port 80 is exposed and handed to the load balancer, which allows incoming requests to the web application.

apiVersion: v1
kind: Service
metadata:
  name: drupal
  labels:
    app: drupal
spec:
  ports:
    - port: 80
  selector:
    app: drupal
    tier: frontend
  type: LoadBalancer

The next object is the storage, which is similar to the MySQL one that we claimed for 20GB. This is attached to the container running the web application via a volume:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: dr-pv-claim
  labels:
    app: drupal
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi

The heart of the Drupal resource is the Deployment object, where the configuration of the web application is defined. Here it is:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: drupal
  labels:
    app: drupal
spec:
  selector:
    matchLabels:
      app: drupal
      tier: frontend
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: drupal
        tier: frontend
    spec:
      volumes:
        - name: drupal-persistent-storage
          persistentVolumeClaim:
            claimName: dr-pv-claim
      containers:
        - image: <IMAGE>
          name: drupal
          imagePullPolicy: Always
          env:
            - name: DB_HOST
              value: drupal-mysql
            - name: DB_NAME
              value: drupal
            - name: DB_USER
              value: drupal
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysql
                  key: nonadmin-password
          ports:
          - containerPort: 80
            name: drupal
          volumeMounts:
          - name: drupal-persistent-storage
            mountPath: /var/www/html/web/sites/default/files
      imagePullSecrets:
        - name: regcred

There are a few things to point out in the above object:

• The image is defined as a placeholder: - image: <IMAGE>. The reason is that a Docker image is built and pushed every time changes are pushed to the GitHub repository. Therefore, the GitHub workflow, at run time, will turn <IMAGE> into something like docker.pkg.github.com/juampynr/drupal8-do/app:foo, where foo is the hash of the last commit pushed to the repository.
• The password is obtained via a Secret object. It's the same object that we saw for MySQL in the previous section.
• The volume was mounted at /var/www/html/web/sites/default/files, the default location to host public files in a Drupal application. This will persist files while Kubernetes recreates pods, which happens during deployments.
• GitHub Packages requires authentication to pull Docker images, even if they belong to public repositories. Kubernetes lets you define a secret with the credentials to authenticate against a Docker image registry. Following these instructions, the secret was created.

The last object for the Drupal resource is a CronJob that runs Drupal cron every five minutes:

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: drupal-cron
spec:
  schedule: "*/5 * * * *"
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: drupal-cron
              image: juampynr/digital-ocean-cronjob:latest
              env:
                - name: DIGITALOCEAN_ACCESS_TOKEN
                  valueFrom:
                    secretKeyRef:
                      name: api
                      key: key
              command: ["/bin/bash","-c"]
              args:
                - doctl kubernetes cluster kubeconfig save drupster;
                  POD_NAME=$(kubectl get pods -l tier=frontend -o=jsonpath='{.items[0].metadata.name}');
                  kubectl exec $POD_NAME -c drupal -- vendor/bin/drush core:cron;
          restartPolicy: OnFailure

The CronJob object uses a custom Docker image that has doctl (DigitalOcean's command-line interface) and kubectl (the Kubernetes command-line interface) preinstalled. Then, it uses these two commands to download the cluster configuration and execute a Drush command against the container running Drupal.

Next steps

Getting the Drupal application up and running in Kubernetes has been a fabulous experience and is a world worthy of continued exploration. Here are some things to figure out:

• Performing rolling updates instead of recreating pods, so there is no downtime during deployments
• Performing load tests against the current infrastructure by doing common tasks in Drupal like saving a node, uploading an image, or requesting a page with cold caches
• Setting up automatic backups for the database and file system
• Triggering automatic or manually rollbacks when a deployment fails
• Setting up secure traffic via an SSL certificate
• Adjusting the architecture to enable scaling the replica set up and down. Here are several pages of the official documentation that explain how to do this:
  • The ReplicaSet object
  • Run a Replicated Stateful Application
  • Running ZooKeeper, A Distributed System Coordinator
• Checking out the existing helm charts for Drupal
• Test running the Drupal 8 application at AWS, Google Cloud, and Linode
• Test hosting platforms that support Drupal over Kubernetes, such as DDEV-Live and Lagoon

Acknowledgments

• Inspiring articles written by Alejandro Moreno and Jeff Geerling.
• Tutorials for getting a mental map of how to fit each of the pieces together in the puzzle:
  • How To Deploy a PHP Application with Kubernetes on Ubuntu 18.04
  • Example: Deploying WordPress and MySQL with Persistent Volumes
• Stellar presentation by Tess (socketwench) 
• Andrew Berry and Salvador Molina Moreno for their technical reviews

As a final note, this is a learning process. If you have any tips, feedback, or want to share anything about this topic, please post it as a comment here or via social networks. Thanks in advance!