Almost a year ago now Development Seed had an article on some of the tools they were developing in order to address a very real, very important problem—that of the whole development to staging to live development process. Up until these tools were available, this process consisted of an archaic—and quite frankly, pain in the butt—method of either duplicating the clicks and changes you made through the Drupal UI in your various environments, or putting all of your changes that needed to be made to the database into update hooks that did very specialized queries, set variables, installed or uninstalled modules etc, etc. This came with a heavy price tag of testing your migration path over and over, resetting your database, and testing again. Oops! Small mistake there... change the code, reset the database, run it again, wash, rinse and repeat. These tools attempt to change all that and now that they're maturing, they've become a godsend for site builders and developers everywhere. Here is the workflow and process we are using at Lullabot and some of the finer points we've picked up along the way.
Version Control
Keeping your code in a revision control system is imperative to the seasoned developer and more than a good idea for those not as experienced. It's more important when you are working on a project with multiple developers so that code conflicts can be resolved instead of accidentally overwriting what your buddy is working on, but also becomes part of a developers toolbox in many other ways.
We're moving to using Git for all of our projects at Lullabot due to the power and flexibility it gives us. Personally I love it so much that I can't stand to use Subversion anymore. When I am involved in a project where everyone is using Subversion I use git-svn so that I can manipulate the Subversion repository on my local computer with Git. The ability to track Subversion branches with Git is my latest candy and super useful for those situations where you need to do frequent merges between branches. Git succeeds in merging where Subversion really falls behind. I don't know if you've ever tried to do this in Subversion, but even the RedBean book introduces it as a 'headache'. And it is. So I use Git for this.
Git is also great for it's quick and easy branching. It's actually become a habit for me to create a branch for each and every issue that I work on for a project. It's so easy to create a branch, and merge it back into HEAD, or the main trunk, that there just isn't a reason to NOT do it. It's quick, it's easy and it keeps your changes separated in a way that if you're not done with the feature branch you're working on and need to fix something in say, the main branch, you can simply switch back to the main branch, fix it, commit it and then go back to what you were doing in your feature branch. With Subversion this is a total PITA because it's such a heavy process. It copies every single file, creating a new physical directory structure and then just try to figure out those merge instructions. With Git it's as simple as
$ git branch [branch-name]
$ git checkout [branch-name]
[make changes]
$ git checkout master
$ git merge [branch-name]
Now that I've beaten that horse to death, let's move on to the actual modules that are making our lives easier these days.
Export it! Export it all!
Getting everything into code has gotten much easier now that Chaos Tools has come and cleared up some of the actual chaos around exporting objects into code. It provides a way to simply add a few keys to your data schema, implement a few hooks, and voilà! Your data object can be read from code, overwritten, re-exported or reverted back to code in an easy to read structure. If you're having trouble visualizing this, think of the way exporting views works and apply that to any arbitrary object that you might be able to think of. Yeah, cool. There are a few prerequisites (such as machine names instead of auto-incremental integer columns) that you need to make sure your tables have, but other then that it's actually pretty straightforward. There's even a nice tag on a lot of modules for Features integration that will give you a list of most of the modules that implement this on d.o - take a look, see how they work, and get your data exportable for goodness sake!
Settings
Have you ever made changes to your local site, or your development site that you then need to make to your live site? Maybe you had to change your default theme, or perhaps the default comment settings for a node type. Instead of clicking these setting on your local and having to go through the UI again and click these settings, checkout Strongarm. Strongarm makes these settings exportable into a custom module.
Layout
Context and Panels have become our layout tools of choice. We tend to choose one or the other based on the project and what the client needs. Personally I can lay out a website much faster with Panels, but there are quite a few occasions where Context is simply a better fit for the situation. Usually this is the case when a client does not need the ability to change the layout around themselves, and it's more of a cleanup job than a brand new site. If I want to rapidly prototype complex layouts I use Panels. If I just need to replicate some basic rules of block visibility, I use Context.
Boxes
Boxes has become a favorite of mine recently. It does two things really well that are a huge improvement over the traditional core Blocks. It allows for easy inline editing of content, and they're exportable. There is a module called fe_block within the Features Extra project that allows you to export blocks, but the problem with this is that they are not very good exports. In this case, it's a core decision to use an auto-incremental key on the boxes table (yes, blocks uses a boxes table, it's confusing, but bear with me) and this has the side effect that if you 'revert' a block and then read it from code, it is reinserted and the id changes automatically which means that anything that referenced that original block (like Context, or Skinr for example) no longer has the correct id to find what you're intending it to find. Boxes gets around this problem by using a machine name for it's unique identifier which plays much better with the exportable mindset. However, one thing that is pretty nice in fe_block is that you can export a block's settings. The actual block itself doesn't work so well, but having those setting is really nice. For instance, if you have a block that is provided by a view and you want to override the title attribute of that block, you can use fe_block to export those settings into code.
Bring it all together
Then we get to Features itself. This is what brings it all together and why it's so important to have all of those other module exportables. Features gives you a nice UI to pick and choose what all you want exported, and then it creates the module for you. That's right, it writes a module for you. Features also gives you an easy way to detect if any of those exported things have changed, and update them if they have. Features' Drush integration is awesome for allowing you to quickly determine what features are overridden ($ drush fr), revert them all ($ drush fra), or update them all ($ drush fu-all). Special thanks to James- aka: q0rban for allowing us to type $ drush fu [feature-name], for those days when things just aren't going too well and you need an expletive to help get you through the day. ;)
And finally...
So now that I've shown you some of the tools, explained the importance of version control, and harped on getting things into code, how does any of this solve the problem of getting all of your changes from development, to staging, to the live site? Well, here's an example workflow from a project I am currently working on. Using the aforementioned tools and the following workflow I was able to make extensive changes to an existing site on my local computer within a new Git branch, export all of my changes to code wrapped within a few features, turn these features on in the staging environment, and have an upgraded copy of the live site without writing a single upgrade path.
Tracking Subversion Branches with Git-svn
The current site I'm working with is in Subversion. It's Phase 2 of the project and this calls for a new branch of the code so that we can provide bug fixes to the current live site which is running on the trunk in our repository. A new Subversion branch was created for the architectural changes we're making to the site. Since we'll be making bug fixes to the trunk, we're going to need to keep merging those fixes into the new dev branch as well. And since merging is such a pain in Subversion, I'm going to use git-svn to work with this code repository. I want to be sure that the new Subversion branch is tracked in Git as a branch, so when I first checkout, or clone the repository I'm going to use the -T and -b operators.
svn repo structure
- branches
-- 2.x-dev
-- original
- trunk
Command to control this svn repo with git $ git svn clone [svn-repo-location] -T trunk -b branches
Resulting git structure $ git branch -r
2.x-dev
original
trunk
Working with Our Subversion Branch in Git
Checkout the dev branch of the project which is now using Git locally. $ git checkout 2.x-dev
This new branch is connected to the Subversion branch and any changes that are committed to it locally with Git can be pushed into this new branch within the Subversion repository. Notice in the screenshot above that it says: "Committing to https://grammys.unfuddle.com/svn/grammys_grammy365/branches/2.x-dev". You can see that I'm clearly not in a 'branches' directory, yet it certainly committed to the 2.x-dev branch. You may also note that the branch indicator in my screenshot says "(context-2.x)". This is actually a branch of the 2.x-dev branch. So it's worth noting that a $ git svn dcommit here is directly connected to the subversion branch "2.x-dev" that I branched off of, and will commit changes to that branch as well. (Whew)
Fixing up the site
The current site I'm working on has a lot of custom blocks with PHP in them doing some things they probably shouldn't. The theme also needs to be redone, and the views and custom blocks need to be exported so that they're not being read from the database constantly. My buddy Jay Wolf is helping me out with the theme, so he is making changes to the development branch in Subversion while I'm using Git on my local. As soon as he had the base theme with all the regions I needed, I got busy with the Context. Once I had the blocks cleaned up and converted to boxes, and started using the Views correctly, I setup separate contexts for the current sections of the site and placed the boxes where their predecessors went. The boxes are using Skinr, and there are a few Quicktabs thrown in. We also want to make sure that the default theme is switched over to the new theme when this goes live.
Merging fixes from trunk
Oh! But then here comes Dave with a bug fix to trunk! The bug fix goes in and we need to merge that change back into our new dev branch. A quick $ git merge master and we have the changes we just made to the master branch merged into our development branch and can continue on our way.
Creating Features
Ok, back to our dev branch. Now that we have everything laid out with context, we're going to create features that correspond to the different contexts, as well as a site feature that will hold some global settings. Let's create our site feature first. This will hold the default theme settings as well as a sitewide context. Let's put this in /sites/all/modules/custom/features. Now that we have our new 'feature' module, we enable it and then take a look at our Features list. You'll see that it is marked 'Default' which means it is reading the feature from code.
And now that we have some basic site settings changed, we create our front page feature. Since the front page of this site holds a lot of the views and boxes referenced through our context, as soon as we tell our new front page feature to include the front page context, it also automatically detects some of the other elements that are referenced through this context. This feature is also wrapped up for us and we put it in the same place and enable it.
Moving the changes to the staging server
Our staging server is running the new Subversion branch already, so we add these new feature modules to the dev branch:
$ git add custom/features
$ git commit -am "adding new features"
$ git svn dcommit
These modules are then on the dev server when we $ svn up. Now we can simply enable our feature modules, and like magic, all of our changes are now working on the staging site!
Oh, and don't forget to clear the caches ;) $ drush cc all