The Swiss Army Knife for Drupal Multisite Docroot Management and Deployment
Introducing Docman: the Swiss Army Knife for Drupal multisite docroot management and deployment. Docman acts as a layer between your docroot - usually a git repository somewhere, but not limited to it- and multiple vendors working on different websites using your standards and predefined sets of modules.
Remember how hosting sales teams always tried to push you to buy more docroots "because otherwise it would be hard to manage websites in a Drupal multisite environment"? Docman can simplify your life so you will be able to stick to one docroot and multiple independent websites in it using the same Drupal core.
Have you ever tried to oblige different vendors to work with one Drupal core for completely different websites, inside a multisite environment, without making them break everything and with clear deployment schema? Find out how docman can make the once daunting task of multisite deployment more efficient.
Does the governance of multiple projects in one multisite environment scare you? Docman has hooks, like Drupal, to launch your tests whenever needed and for each website in multisite environment independently.
Real-world example:
First, make sure you have Ruby installed.
On a Mac, open /Applications/Utilities/Terminal.app and type:
ruby -v
If the output looks something like this, you're in good shape:
ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-darwin13.0.0]
If the output looks more like this, you need to install Ruby:
ruby: command not found
On Linux, for Debian-based systems, open a terminal and type:
sudo apt-get install ruby-dev
or for Red Hat-based distros like Fedora and CentOS, type:
sudo yum install ruby-devel
(if necessary, adapt for your package manager)
Windows, is not supported for now.
Once you've verified that Ruby is installed:
gem install docmanThe important thing to understand - Docman doesn't really care about your directory structure, it scans your master directory for subdirectories and config files and then it will build what you want him to build. In this example we are replicating Acquia docroot structure, but feel free to adapt it to your needs.
This is why the tool is Drupal-version agnostic, you can easily adapt your Drupal 8 websites, because it is you who decide how to build the docroot, not Docman.
First of all you will need a config repository for this configuration. For testing purposes you can just use local repositories and then use external git links when you're ready. Remember, all your repositories (except master config) should have a state_stable branch created as an orphan. Docman uses this branch internally to keep stable version inside. First, lets create a config repository:
$ mkdir docroot-config
Lets init a git repository:
$ cd docroot-config; git init
We now have a local git repository, lets create an initial directory structure and config file:
$ mkdir masterThen we will need your common modules for all the projects:
$ mkdir master/commonThen we will need a directory for Drupal core :
$ mkdir master/docroot Then we will need an empty profiles directory:
$ mkdir master/profiles Then we will need a projects directory, where all your websites will go:
$ mkdir master/projects Then we will need a project 1 directory
$ mkdir master/projects/project1 Then we will need a project 2 directory:
$ mkdir master/projects/project2 Then we will need sites repository, which basically represents Drupal default /sites directory:
$ mkdir master/sites Config files are important files that Docman uses in order to understand what kind of content the directory contain, where he should take the code, and what to run after along with environments.
Config files are .yaml files
The most important configuration file is config.yaml, which is located in the root of the config repository, lets create it:
$ touch config.yamlNow lets fill it with basic config that suits our needs:
---
environments:
dev:
deploy_target: git_target
state: development
target_checker:
handler: :ssh
file_path: /mnt/www/html/docman1.dev
ssh_host: xxxxxxxx.devcloud.hosting.acquia.com
ssh_user: docman1 # Edit this!
test:
deploy_target: git_target
state: staging
target_checker:
handler: :ssh
file_path: /mnt/www/html/docman1.test
ssh_host: xxxxxxxxx.prod.hosting.acquia.com
ssh_user: docman1 # Edit this!
prod:
deploy_target: git_target
state: stable
tagger:
enabled: true
handler: :option
Please note the structure, here we define the list of environments. You can have as many environments as you need, but here you see the default Acquia hosting structure with 3 environments, all of them are managed by git. Variables are pretty self explanatory:
deploy_target: git_target
This is important to say to docman, it set the environment to be built and pushed to git.
state: development
This is the name that you will use for docman build command.
target_checker:
Important part to say to docman about how to check if the code was correctly deployed. In this case after each push to git the tool will use the following parameters to connect to the environnement and check if the code is in place.
handler: :ssh
We are using ssh protocol to check
file_path: :ssh
Where to look for deployed files
ssh_host:xxxxxxx.devcloud.hosting.acquia.com
SSH host. Put the one that is shown to you in your hosting dashboard
ssh_user:docman1
SSH user
tagger:
Subsystem for production environment, create stable docroot tags by checking all the stable tags from all the repositories and merging them into one.
enabled: true
Enabling the tagger
handler: option
@Todo: description
As you can see, your environments can be anywhere, you do not need to stick to the same hosting system, you can get the code from your local development server dev environment and your staging environment will be in the real hosting. Continuous integration!
In this example we have common modules reposiroty which resides in master/common directory. Lets create a config file for it:
$ touch master/common/info.yamlWe are using info.yaml here instead of config.yaml, because docman will search for info.yaml files in each directory to get its config.
Now lets put the basic configuration inside:
status: enabled
type: repo
repo: /Users/Adyax/Code/docman-common-test/
order: 30
states:
development:
type: branch
version: develop
staging:
type: branch
version: master
stable:
source:
type: :retrieve_from_repo
repo: :project_repo
branch: state_stable
file: info.yaml
hooks:
builder:
after_execute:
- type: :script
location: $INFO$/after_build.sh
execution_dir: $PROJECT$
params:
- environment
Important: your remote repository should containt orphaned state_stable branch, it can be empty initially.
Again, see below the quick explanation of what is what here. The config file consists of main settings and states part, states are needed to describe to docman from which part of repository it needs to get the code to build for the specific environment (see config.yaml for the list of environments):
type: repo
Docman needs to know where to look for the code. This time it is a git repository.
repo: /Users/Adyax/Code/docman-common-test/
Git URL which you normally use in git clone command. In this example - local repository.
order: 30
The order in which docman builder gets this repository
states:
Starts the states description part
development:
The state that correspond to the environment in the main config.yaml file
type: branch
Which branch from the original repository should be taken for this state
version: develop
Ddevelop branch for development state in development environment in this example
stable:
Starts the stable part for this state
source:
Starts the source description for the stable version
type: :retrieve_from_repo
In this example it will get the information from the same repository. But guess what - you can get it from other places too, like a specific file somewhere.
repo: :project_repo
Says that we are using the same repository as defined in the beginning of the file
branch: state_stable
Says that we are using the branch state_stable to get the release information (the name of the stable tag for this reposiroty)
file: info.yml
The file in which docman will search for the stable tag name for this repository. Ex. v.0.0.2
hooks:
Starts the hooks part that we be run during the build
builder:
Starts the builder configuration part
after:
Starts the hooks part that we be run during the build
Docman will also search for after_build.sh file, which contains the list of operations to fire after the build is finished:
$ touch master/common/after_build.shPut the following content in it:
#!/bin/sh
set -vx
# Add following files into local git ignore only
if [ "$1" == "local" ]; then
if [ -f .git/info/exclude ]; then
rm .git/info/exclude
fi
echo "info.yaml" > .git/info/exclude
fi
set +vxIn this example we have drupal core repositoty which resides in master/docroot directory. Lets create a config file for it:
$ touch master/docroot/info.yamlThe structure of the config file for this repository is the same as in step 3 ((do not forget to put a proper git link into repo variable)):
type: repo
repo: /Users/Adyax/Code/Adyax/docman-core-test/
order: 1
states:
development:
type: branch
version: master
staging:
type: branch
version: master
stable:
type: branch
version: master
hooks:
builder:
after_execute:
- type: :script
location: $INFO$/after_build.sh
execution_dir: $PROJECT$
params:
- environment
Important: your remote repository should containt orphaned state_stable branch, it can be empty initially.
In this example we have profiles directory empty, and it resides in master/docroot directory. Lets create a config file for it:
$ touch master/profiles/info.yamlWe do not have anything to put here, so lets disable this directory for docman. Put the following into this file:
status: disabled
type: dir
Important: your remote repository should containt orphaned state_stable branch, it can be empty initially.
We have two projects (websites), so lets create two subfolders:
$ mkdir master/docroot/project1
$ mkdir master/docroot/project2And config files for them:
$ touch master/docroot/project1/info.yaml
$ touch master/docroot/project2/info.yamlThe structure of config file for each repository is the same as in step 3, but this time time we will add deployer configuration. Put this into the configuration files you have just created (do not forget to put a proper git link into repo variable):
type: repo
repo: /Users/Adyax/Code/Adyax/docman-project1-test/
states:
development:
type: branch
version: develop
staging:
type: branch
version: master
stable:
source:
type: :retrieve_from_repo
repo: :project_repo
branch: state_stable
file: info.yaml
hooks:
builder:
after_execute:
- type: :script
location: $INFO$/after_build.sh
execution_dir: $PROJECT$
params:
- environment
deployer:
before_deploy:
- type: :script
location: $PROJECT$/tools/deploy/common/before/before.sh
execution_dir: $ROOT$/docroot
params:
- environment
- type: :script
location: $PROJECT$/tools/deploy/$ENVIRONMENT$/before/before.sh
execution_dir: $ROOT$/docroot
params:
- environment
after_deploy:
- type: :script
location: $PROJECT$/tools/deploy/$ENVIRONMENT$/after/after.sh
execution_dir: $ROOT$/docroot
params:
- environment
- type: :script
location: $PROJECT$/tools/deploy/common/after/after.sh
execution_dir: $ROOT$/docroot
params:
- environment
Important: your remote repository should containt orphaned state_stable branch, it can be empty initially.
Sites directory represents Drupal default sites/ directory, lets create it and put the proper config file:
$ mkdir master/docroot/sitesAnd config file:
$ touch master/docroot/sites/info.yamlThe structure of the config file is the same as in step 3. Put this into the configuration file you have just created (do not forget to put a proper git link into repo variable):
status: enabled
type: repo
repo: /Users/Adyax/Code/Adyax/docman-sites-test/
order: 20
states:
development:
type: branch
version: develop
staging:
type: branch
version: master
stable:
source:
type: :retrieve_from_repo
repo: :project_repo
branch: state_stable
file: info.yaml
hooks:
builder:
after_execute:
- type: :create_symlink
target_dir: master/docroot
- type: :script
location: $INFO$/after_build.sh
execution_dir: $PROJECT$
params:
- environment
Important: your remote repository should containt orphaned state_stable branch, it can be empty initially.
Note additional after_execute configuration, we are saying to docman that after getting the repository it needs to create a symlink to sites in master/docroot directory.
Put also the after_build.sh file:
$ touch master/docroot/sites/after_build.shWith the following content:
#!/bin/sh
ln -s ../common all
This will effectively link common directory to /all after the docroot build has beedn finished
Now, as everything is ready, lets build a local development environment. In the directory with your main config.yaml file launch this:
$ docman build local developmentIf you see 'Complete!' in the end of the output of this command - switch to the master directory - your docroot directory will be there and ready to be configured for your local web server.
$ docman build git_target developmentThis will effectively build the docroot and push it to git repository of a target hosting in development environment.
$ docman build git_target stagingThis will effectively build the docroot and push it to git repository of a target hosting in staging environment.
When your changes are in the master branch in one of the used repositories you need to say to docman that there is a new stable version. In the needed repository launch:
$ docman bump stableThis will effectively increase your code version number (or propose one, or ask you for the version, depends on the repository state)
And to generate a new stable docroot tag and push it to remote reposiroty launch this command:
$ docman build git_target stableThis library aims to support and is tested against the following Ruby implementations:
If something doesn't work on one of these Ruby versions, it's a bug.
This library may inadvertently work (or seem to work) on other Ruby implementations, however support will only be provided for the versions listed above.
If you would like this library to support another Ruby version, you may volunteer to be a maintainer. Being a maintainer entails making sure all tests run and pass on that implementation. When something breaks on your implementation, you will be responsible for providing patches in a timely fashion. If critical issues for a particular implementation exist at the time of a major release, support for that Ruby version may be dropped.
The support is happening in GitHub, please create an issue there.