Posts tagged ‘virtual machine’
Configuration Management is an old horse that rarely gets any loving outside of the Microsoft environment.
Generally it’s a mechanism that allows you to control the configuration and software available on machine but it’s usually clunky, brutally inefficient on the network and generally requires total control of the target machines.
Then along comes Opscode and opens up their Configuration Management Kitchen with Chef. Chef is a lightweight approach to Systems Integration & Configuration Management (SI & CM for the light-hearted) built on Ruby/Rails/Gems that allows you to quickly deploy and configure software and services without requiring total domination.
I’ve had my eye on it for a while and with the Virtual Machine environments I’ve been working on for Symfony and Zend – I decided to dig in and give it a spin and I’m impressed; almost beyond words.
Chef depends on having Fully Qualified Domain Names up and running and can be a little quirky without them.
The installation instructions for the Chef-Server and Chef-Client are clear and concise and can be found here.
You start by installing the Chef-Server which provides the core back-bone to support your environment. Once it’s up and running you have Chef running on Rails under Apache providing a web and REST interface for clients (or nodes in the Chef parlance). Here you can view and control the attributes of a node, examine your configuration scripts (Recipes) and authorise clients. The GUI tools in the current (6.2) release are a little raw but functional and the coming 6.4 Release sharpens up the Web UI a lot (and brings with it a whole host of exciting features). I setup the chef server on a stand-alone VirtualBox machine with 256 MB memory and a 3GB disk – which is working well for everything I’ve thrown at it so far. You’ll need to login to the Web UI using OpenID and ensure you use the appropriate domain appended to your login – full details of the OpenID providers and their naming schemes can be found on the OpenID site here.
It can take a few minutes for the registrations to appear in the Chef Web UI.
Once you have the server up and running you’ll need to install the chef-client on a host. Once up and running the client will connect to the server and register itself. You’ll need to fire-up the Web UI on the server and authorise the client before you’ll be able to do anything more with the client.
Once it’s been authorised just run the chef-client again with:
When it completes you’ll see the information about the client in the Web UI in the nodes and status panels.
If you don’t authorise a client on the server then you’ll see a HTTP 403 error when you run the chef-client.
Now you have both the client and server up and running – you can get down to the real business of deploying something.
Open 2 SSH connections – one on the chef-server and another on the chef-client and start simply by following their quick-start guide on the chef-server and in a couple of minutes you’ll have your first chef-recipe complete. Now just drop into the cookbooks folder and copy the quick_start cookbook to /srv/chef/site-cookbooks:
cp –R ./quick_start /srv/chef/site-cookbooks/
Now refresh the Web UI and open the Recipes Panel and you’ll see the quick_start recipe that you just created listed.
To apply the recipe to a node (your client) open up the nodes panel in the Web UI and double click on Recipes for it. In Chef 6.2 you’ll get an awful textbox with the information for the node in JSON format. Scroll down to the bottom and you’ll find the recipes entry – inside the  put “quick_start” (include the “”) and hit save.
The end result should look something like:
If you did it right you’ll see the page update. Another minor issue in the 6.2 release is that if you didn’t update the JSON correctly you’ll see saving that’ll never complete.
All that’s left is to switch to the chef-client SSH terminal and get the client to update itself now with:
A few seconds later the client will find that it has a new recipe and install it. On the client go to the /tmp folder and you’ll see deep_thought.txt from the chef-run J
Now this seems like a lot of effort to get a text file to appear in a folder – but it’s just as simple writing a recipe that installs MySQL, PHP, Redmine, Symfony or Zend Server. But it’s not just about installing packages that’s already pretty simple using bash with apt or yum. Using a recipe allows you to ensure that the installs are idempotent or transactional. If one part fails – then you can ensure that the machine is left in a known reliable state. If you have a failure in a script then you can be left with partial installs or worse – the machine in an unreliable or unworkable state.
One of the exciting aspects to all of this is that it’s very easy to hook things together – not just on one machine but all machines in your environment – regardless of what OS they’re running. A recipe to install Zend Server, Symfony, MySQL or as a single package will work on Ubuntu, Redhat, CentOS or most other variants.
Hooking into the infrastructure allows very simple approaches to things like provisioning, deployment and configuration of environments – in my case this allows:
- Automated creation of a virtual machine instance
- Automatic provisioning of the instance
- Dynamic allocation & changing of the resources available to the instance (Memory, Disk, Drives, etc) although with VirtualBox a reboot is needed for memory changes to take effect.
- Dynamic package and configuration – allowing me (from within the VM instance) to switch it’s mode of operation and determine its role. So within minutes it changes from all in one (complete LAMP on the instance) to the DB Server role
One of the Pro’s in the Symfony Users Google group had some comments on the Virtual Machine for Symfony at Sipx.ws and I wanted to share my thinking about my plans.
Generally when developing you should have an environment that represents that to which you’ll be deploying to – it’ll save you time, effort and much pain to have something as close as possible. There are however, several scenarios for developers:
Ideally your environment matches that to which you’ll be deploying to. If you control the server infrastructure then this is less of a problem – you’ll build the server yourself (ideally via an automated deployment process) and building a VM from this is trivial.
If you don’t however control the server infrastructure then you have a more complex situation to deal with. If the gods are smiling then they’ve built their server completely from public distros and repos and used a package manager for all installs. If this is the case you can dump the package list and server build – and rebase an image yourself. Often however they have a custom OS build (tweaked for whatever reason), local repositories (hopefully mirrored) but sometimes not and a few extras thrown in. This makes building an image that represents the environment you’re going to use, while not impossible, generally non-trivial.
ServerGrove (http://www.servergrove.com/), forward-thinking & proactive, are interested in providing an image to their customers that does just this – allows people to develop locally in an environment that represents where the application will be deployed.
A growing trend with hosting providers is where they allow you to upload your own image to the hosting environment allowing you to build your own OS (subject of course to licensing requirements). One of the aims of the VM was to provide a way for devs to start locally and then upload a copy of the image to the hosting environment. With a few caveats (mostly around networking) you’re assured of 100% success for the deployed project as you’ve been able to put it through its paces in before uploading.
In this scenario the developer is building applications for non-specified specified servers – either because they don’t have or haven’t selected the hosting environment yet, they don’t have complete information from the project sponsor – or some other reason (it’s weird and wacky out there). Another possible deployment is Open Source projects where the deployed application may be any OS – and yet you’d like to have a common “known” environment for developers and end-users.
In this situation the VM helps both the developer and the project sponsor – as it’ll allow the dev to share the VM with the sponsor for testing and signoff. Essentially passing the monkey wrt the hosting environment.
General Approach (now and 1.x)
The current approach I’ve taken is mainly aimed at providing a lean-learning curve, a clean & repeatable environment to the community developing against Symfony and the Zend Framework (the Zend side is mostly a freebie but also aimed at helping people with Lucene search issues). With each build I test to ensure that all sf frameworks work by deploying a test application that covers ORM’s, plugins, routing and the DB/httpd. With the build I try to ensure that it’s portable and therefore works against the major VM Client vendors (VirtualBox, VMWare and Xen currently). The aim of the 1.0 release is to have something built and packaged ready to run – much like the sf sandbox currently works.
While VM’s have been around for a while – and while installing linux has become more user friendly – there’s still a lot of areas you can trip-up building images and installing OS’s. One of the aims was to remove this as a blocker to devs wanting to just get down to developing applications.
With the release of 1.0 there should be the following images and deployments available:
o Stand alone (everything in one box for simple dev projects)
o Load Balanced (built using devSFCore with configuration that puts the server into modes: lb [load balanced], web [web server, memcached & no db], db [db, svn, no httpd but a http management interface])
· Project helpers
o Helpers to aid start-up of projects and development. Things like building the root development folder, linking to the version of the framework you wish to use, creating and configuring the DB, configuring the application to use the DB and running tests on the initial setup. Think a2ensite for creating a symfony application and you’ll get the picture. The intention isn’t so much to dumb down – but to streamline and to facilitate adoption by those not that familiar with symfony. Included will be log creation of the actual steps involved to help devs understand what to do.
With Deployments the general idea is that you’ll be able to run multiple images in modes – to facilitate testing, architecture scenarios, etc. With this you run one image as a DB, several as web servers and drop in a load balancer – and hey-presto you have a way to test how your application performs when scaling out.
With the 1.x branch I’m intending to go with a much lighter approach – still with some base images for various distributions and deployments (there will be standard and live images along the same approach as the live-cd used with some distributions) but using some of the approaches you’ve outlined for providing the packages and for linking in with repositories. This approach however requires some infrastructure to support it – and infrastructure = time + resources and resources = money.
This approach essentially extends the current sf sandbox to a deployed image mode. It’ll work out compatibilities, issues and fixes, deal with things like pear and pecl dependencies, PDO and handle the deployments you’ll see above.
With 1.x comes features for both devs and hosters (and allows for Targeted deployment). Hosters can build their base image and include the needed components into the image – and share it with their customers(the devs). Devs can download and use the image – and it’ll pull all the needed parts down. When they are ready to deploy – then from within the VM they can provision and deploy the application. With the provisioning on the hosting provider side building the image locally, deploying it and then accepting the deployment of the application.
Should the dev decide to move hosting providers to another supporting this model – as it’ll be built using the same components (but probably a different base OS) – then it should be a simple process to download their base image, deploy from the current VM to the new VM, test and redeploy.
I’ve just launched a new website aimed at helping developers have a cleaner environment for developing & testing their Symfony applications at Sipx.ws. Thanks to the great guys at ServerGrove – I managed to get the site up and running in no time.