Generic selectors
Exact matches only
Search in title
Search in content
Search in posts
Search in pages
Filter by Categories
Investor blog
Blog

Choose your region

Selecting a region changes the language and/or content on Nortal.com

Generic selectors
Exact matches only
Search in title
Search in content
Search in posts
Search in pages
Filter by Categories
Investor blog
Blog

Automating Development Environments with Vagrant and Chef

by Priit Liivak, Partner, Head of Engineering, March 23, 2015

What is development environments automation? Why use it? What problem it solves?

Imagine you are a developer just assigned to a new project. There are some new cool ideas to be realized and you are given some good challenges in this project. You are full of excitement to begin coding. Now you just need to set up the environment.

There is a proper guide in the project wiki for doing this, you start to follow it and end up with the following activities:

  • Upgrade your local MySQL server and client.
  • Add that project-specific setting to the MySQL conf.
  • Create your local project database.
  • Create your local project database users and add permissions.
  • import database contents from a shared test environment (and you’re being told to not take the latest contents, its broken at the moment..).
  • Realize that the MySQL database had to be created as UTF-8 encoded. Delete the database and perform again the previous three steps.
  • Upgrade your existing Apache HTTP Server.
  • Re-configure your existing Apache HTTP Server installation for the new project. There is a nice project-specific httpd.conf file available to copy and paste, but you need to remove and install modules for that to work, as well as override some personal settings.
  • Copy some additional configuration files and static content files for Apache HTTP Server.
  • and so on and so on.

So you take a day or two for environment setup. You encounter various issues. Some of the issues are easily solved by that nice senior guy of the new project. Other issues he has never seen. The strangest issue you encounter sounds familiar to him, but he does not remember exactly what they did to overcome this issue. The lady who probably knows more has left the project, works in some other country now, and is unavailable at the moment.

Why is every environment setup for any project a unique artwork? What if you could just get it with a single click?  Why not let the computer do the step-by-step setup automatically for you (and you having a lunch or a table football match with nice colleagues at the same time), and still start being productive in half an hour instead of three and half days? This is exactly where development environments automation, when done properly, can help.

So, development environment automation can make local development environment setups automatic and repeatable. This would save lot of time in situations like these:

  • you are starting work in a new project and need a particular local development environment set up
  • you are switching back and forth between multiple projects that each have a different local development environment
  • your local development environment has broken and it seems to take more than an hour to figure out why and fix it
  • you need a temporary environment for a quick demo or a proof-of-concept experiment

Tools

You can always script development environment automation in your favorite programming language, but that may turn out a long and hard task. Another way would be to build the entire solution in Docker way, which also takes care of development environment automation. Yet the third way is to use Vagrant, which is a tool especially meant for that. Since I like automation and since I happened to have some free time, I started exploring development environment automation the Vagrant way to create a setup of a particular project I would possibly be starting in the future.

Once I got started, I encountered a mess of related tools and technologies that were totally new for me: Ruby, Chef, ChefDK, Chef Solo, ChefSpec, Vagrant, various Vagrant plugins, Berkshelf, Knife, Test Kitchen, RSpec, Serverspec – what are all these? Do I need to learn them all? Do I need to learn a new programming language? Several weeks later, the mess was sorted out and I had a working solution together with automated tests. Here is a brief description of the solution:

The basis for the entire solution is Vagrant (https://www.vagrantup.com/), of course. Vagrant and all the other tools are based on Ruby programming language (https://www.ruby-lang.org). However, I did not have to learn all of Ruby, because the tools generally provide a Ruby-based DSL for configuring them. (I still got quite excited about Ruby)

I used Chef Solo (https://docs.chef.io/chef_solo.html) as a provisioning engine. This is a version of Chef that works without a Chef server.

I started with a generic public vanilla Ubuntu 14.04 box from the Hashicorp’s Atlas box catalog (https://atlas.hashicorp.com/boxes/search) Then i had my Chef cookbook build up everything on top of the box.

I used some public open-source Chef cookbooks for setting up some widely used open-source components: Tomcat, ActiveMQ, MySQL, Java. I used Berkshelf (http://berkshelf.com/) to manage cookbook dependencies. I also used the Berkshelf plugin for Vagrant to handle the dependencies fully automatically – I just had to declare them. This series of blog posts was the most helpful source of information for me: Docker Getting started writing Chef cookbooks the Berkshelf way/.

Since I’m a fan of test-driven development, I developed unit tests for this solution as well. I used Chefspec (https://github.com/sethvargo/chefspec), which is a Chef-specific extension of Rspec (http://rspec.info/) – a unit-testing framework for Ruby. Here is an example of a Chefspec unit test: it ‘creates tomcat-users.xml file with correct admin user and roles

expect(chef_run).to.render_file(users_file)