Making a Vagrant test environment for Drupal 7

Because that’s what I did on my week off! I started making my Kwerey.com website almost two years ago, stopped working on it about a year and a half ago, and since then I’ve upgraded the Ubuntu desktop it was running on. Turns out Apache 2.4 is pretty different to 2.2.

So instead of wrangling the local dev environment I’ve got, I figured I’d make a Vagrant VM, because I’ve since used Vagrant at work and it’s super-handy.

Install the Vagrant basics

Download these two:

– Vagrant (for me, that’s just `apt-get install vagrant`)
– Virtualbox (also available with apt-get)

Grab a relevant-looking Vagrant image by URL from Vagrant’s list of base boxes, or another hosting site like this one. Unless you’ve got VMware or similar set up, you want one where the provider says Virtualbox. (That’s the hypervisor – the thing that mediates between your VM’s operating system and the actual hardware of your computer)

What I did here was look up what my hosting provider was running and installed a Centos 5.x version to match – I picked 5.8 because there was a 64 bit VM available which looked similar to what I wanted. See my footnote below for some detail about that choice.

If you grab an official image, the Atlas site talks you through adding it – if you don’t, use `vagrant box add (name) (url)` to download it, and then make a vagrant file with Vagrant init and edit it to add the name you gave it in this line:

config.vm.box = "(name)"

If you want to get even more specific than this – Packer is an awesome tool for building your own from scratch.

Next up, give your VM an IP address all of its own. This is much better than making it fight for the right to run a webserver at `localhost`! You can give it any IP starting with 192.168.x.x, but I’d go with 192.168.2.20+ to avoid it clashing with any routers, VPN networks, or similar. Uncomment out the config.vm.network block, like so:

# Create a private network, which allows host-only access to the machine
# using a specific IP.
config.vm.network :private_network, ip: "192.168.2.20"

Then run `vagrant up`, and check you can see the VM with ping

shep@harbinger:~/vagrant$ ping 192.168.2.20
PING 192.168.2.20 (192.168.2.20) 56(84) bytes of data.
64 bytes from 192.168.2.20: icmp_seq=1 ttl=64 time=0.404 ms
64 bytes from 192.168.2.20: icmp_seq=2 ttl=64 time=0.208 ms
^C

If so, SSH in (`vagrant ssh`) and you can get to work. On vagrant VMs you can normally use `sudo` without needing a password – it’s useful for automation! – but the convention is to set `vagrant` as the password:

Now the other three letters in a LAMP stack

Install the needed stuff. For Drupal you want the PHP language and various libraries for it to understand what all those modules are talking about, a relational database to store content in, and a webserver to show it off to the world. If you want a LAMP stack, like I did, the things you want are

* Apache, also known as httpd on Red Hat flavoured Linuxes
* MySQL, the server, rather than just the client interface
* PHP 5.3 or higher +

My Vagrant box is Centos-based, and uses the the yum package manager. The below is not actually what I did (or, more accurately, I did this and then undid in order to get the right versions of everything running alongside one another), but I’m learning it here because it’s a reasonable guide to the packages. If you were using an operating system less than 8 years old, this would be all you needed.


yum install httpd
yum install phpmyadmin
yum install mysql
yum install mysql-server
service httpd start
service mysqld start

You don’t strictly need phpMyAdmin to run Drupal, but I find it useful to be able to see a visual version of what’s going on in my database. If you too want to be able to see in there, tweak the settings Apache has for phpMyAdmin so it’ll let you log in from your own machine, not just from within the VM.


[root@centos58 ~]# vim /etc/httpd/conf.d/phpmyadmin.conf

Back this file up (copy it somewhere with .backup on the end, or something) and then open it using your text editor of choice, go to the ‘Directory’ heading, and edit in the IP address of your own computer (not the VM to allow it access to PHPmyAdmin)


<Directory "/usr/share/phpmyadmin">
Order Deny,Allow
Deny from all
Allow from 127.0.0.1
Allow from 192.168.2.1
</Directory>

Then restart Apache. At this point it’ll read the config file and tell you if there’s an error, but what you may look approximately like this:


[root@centos58 ~]# service httpd restart
Stopping httpd:                                            [  OK  ]
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[  OK  ]

That error I got is fine. If you get one that says ‘syntax error’ or similar, you want to go check everything lined up right in that file.

Now, you should be able to navigate to `192.168.2.20/phpmyadmin` and see that it’s up and running. Awesome! But… you need to set a password for ‘root’ on mysql. Luckily, your MySQL install came with this handy mysqladmin client to sort that for you: `mysqladmin -u root password NEWPASSWORD`

Now you’ll be able to log in to phpmyadmin with that root account. Neat: now make the Drupal server user that’s in your settings.php, and give it rights only on the Drupal database.

Now, export your Drupal database from your original site, and import in into here. Drush does a lovely job of this, and I’d thoroughly recommend it, but for some inexplicable reason my shared cloud hosting provider does not want me to be able to install arbitrary software on their servers, so I just scp’d it over then imported it through the MySQL commandline: `mysql -u username -p databasename < filename.sql`, then enter your password.

Fun with Git

Make ssh keys for the user you’ll be making changes from on your new VM (a simple `ssh-keygen` will spit out id_rsa and id_rsa.pub to ~/.ssh in your current user’s directory)

Log in to your code host of choice, and add a key under your own user’s settings if you want to be able to push changes to the repo from the VM, or a read-only deploy key if you don’t. Since I’m using the VM to test code and updates, I added a new key for my own user.

Test the key by changing directory into the place you want your website to be running (this is likely to be /var/www/html), and cloning the repo. Run `git clone git@blah:blah`, as explained somewhere on your repo’s page on the code host site.

Marvel at it in all its glory! Go to `192.168.2.20/reponame`, and you should, if all has gone well for you, be able to see an exact copy of your Drupal site. ‘Grats – you have a dev environment!

EXPORT THE VAGRANTBOX

You can save the current state of your VM pretty easily, so you can easily share it between machines and back it up. I hadn’t done this before so I followed these nice simple steps – http://blog.cloud66.com/saving-your-vagrant-changes/

The output I got from this was:


shep@harbinger:~/vagrant$ vagrant package default --output kwerey-2016-02-26.box
[default] Attempting graceful shutdown of VM...
[default] Clearing any previously set forwarded ports...
[default] Exporting VM...
[default] Compressing package to: /home/shep/vagrant/kwerey-2016-02-26.box
shep@harbinger:~/vagrant$ vagrant box add kwerey kwerey-2016-02-26.box
Downloading box from URL: file:/home/shep/vagrant/kwerey-2016-02-26.box
Extracting box...te: 62.7M/s, Estimated time remaining: 0:00:01)
Successfully added box 'kwerey' with provider 'virtualbox'!

Check that it’s there with `vagrant box list`.

If so, just stick your equivalent of this in a new Vagrantfile, and you can spin it up again in the state you left it with `vagrant up` and tear it down with `vagrant destroy -f`:


# Every Vagrant virtual environment requires a box to build off of.
config.vm.box = "kwerey"
config.vm.network :private_network, ip: "192.168.2.20"

Note that tearing down and rebuilding a VM isn’t something you’ll need to do too much of – git branches are a much better and more convenient way of backing out code based changes.

What Vagrant and Virtualbox are for is making test environments neat and out of the way. If you keep your test environment on a VM you don’t need to be running Apache, MySQL and PHP on your own machine and starting them by default (sounds silly, but I genuinely did this working out web dev skills on my own!). You can back a box up easily as a single file, and execute arbitrary code whenever you boot it.

=====

Coda about PHP versions: I made this process extra complex for myself by deciding get PHP5.3 working on an old enough Centos to be using 5.1 – I decided to use an old version of Centos to match the one my web host uses, and I’m not sure it’s worth it. Trying to run PHPmyAdmin on Red Hat’s packages for php 5.3 complains about unmet dependencies and asks for the 5.1 versions, which cause conflicts and generally kick up a fuss.

If I went back and started this again I’d use Centos 6 and just complain at the hosting provider if anything didn’t work that should. If you fancy it anyway the rough steps I took were this

– remove php-* with Ym
– install php53-common + the mysql and xml parsing packages to go with it
– install PHPmyadmin and MySQL versions to match your hosting peoples’ versions (get tars from the providers)
– check /var/log/httpd and your Drupal report tab for anything funky

Timeago: theming fields in template.php and tpl.php files.

So for a Drupal project sadly unrelated to Kwerey.com, I wanted to grab a date field from a content type and translate it to “time ago”. I found it kinda difficult to find documentation for how to do this kind of coding for themers.

Maybe people don’t normally try this kinda stuff unless they’ve got mad PHP skillz already, but I wish there were more snippets of code and examples out there! In that spirit, here’s what I ended up with. No guarantees it’s best practice, but it works and I’ll talk through my understanding of why and how.

<?php       
/**
 * Implements hook_preprocess_HOOK() for node template
 */
  • Functions go in template.php in your theme’s folder: sites/all/themes/themename/template.php. If you haven’t got a file called that, make one! Copy it off your parent theme if you’re using a subtheme, if not you’re good to just make a blank page and start coding!
  • The <?php means this is the start of a bit of code in the PHP language: the tpl.php files tend to start every line by opening PHP tags and end them all by closing them again, but that’s because they’re a mishmash of different languages. In contrast, your template.php file only needs one <?php at the start.
  • The /**s mean “this is a comment, don’t interpret it as code”. Text editors designed for programmers will grey comments out automatically – I like Sublime, which has a free version for Linux, Mac and Windows.
function kwereyzen_preprocess_node(&$variables){
  $node =& $variables['node'];
  if (isset($node)) {
    if ($node->type != 'datetest') {
      return;
}

Replace kwereyzen with the name of your theme, duh.

Preprocess means this gets run before the node is pieced together, so you can fiddle with the constituent parts before they get turned into static HTML markup.

The (&$variables) means: call up the variables for this node and set them in read/write mode: just writing $variables leaves them read-only.

The next line is getting the variable called ‘node’ out of the big variables we called up in the first line, then setting it locally as $node, and the next lines are saying: now make sure there’s a node here, and check if its content type is ‘datetest’. If not, drop it. End the function. (!= means “if there isn’t”.)

If we’re in luck and the setting called type in the node array does come back as ‘datetest’, we want the function to stick around and do some work for us:

    else {
    $variables['unixtime'] = $node->field_unix_timestamp['und']['0']['value'];
    $variables['time_ago'] = $node->field_date_iso_notime['und']['0']['value'];
    //$variables['date'] = $node->field_date_iso_notime['und']['0']['value'];
    }
  }
}

The lovely Devel module gets you the arrays that make up nodes at this stage in their life, letting you pick out the right machine names and hierarchies of things.

I got stuck for a while, because I was setting variables within the function – just saying $unixtime = $whatever – but the way the template.php file is designed to work is by letting you modify the node’s $variables array: you don’t wanna make a local $unixtime, you want to make it in the node’s existing collection of $variables. So here I’m setting $variables[‘unixtime’] to be the value property of the field called field_unix_timestamp.

So go check there’s the right number of curly brackets at each end of your code, and that there’s a semicolon at the end of every line, or ditto, and then you oughtta be set. Go over to node-tpl.php (or node–datetime.tpl.php, the template file for that specific Content Type) and use that variable there!

<?php echo "Please print out the Unix timestamp ".$unixtime.", computer." ?>

And that’s one way to move information out from the inner workings of Drupal nodes to the happy, easy editable theme layer, where you can mess around to your heart’s content and apply Javascript and everything. Figuring it out taught me an absolute tonne about PHP and Drupal work: I’d recommend it.

Dev Blogging

Heya all – after a month and a bit in public, Kwerey.com’s built up a database with loads of awesome books I’ve never heard of, and gathered some awesome users. Thankyou, all of you. It’s pretty amazing seeing the site grow, and I hope you’ve all found some excellent new stuff to read.

Anyway, the site’s first wobbly steps on its own makes some of the gaps in the infrastructure I put together for it a lot more clear to me.

Here’s some of the bits of functionality I definitely want to implement:

  • favourites lists: a new content type so users can put together a list of book recs with a little space for their thoughts on each
  • a mailing list: there’s a site twitter account and this sideblog, but lots of people don’t have social media and I’d like a way to offer news without disrupting the main page.
  • custom RSS feeds: Kwerey.com has an RSS feed for new books on the archive, but I’m looking in to setting up alerts for books that meet certain criteria, be it Steampunk about lesbians or crime novels about people of colour.
  • a link to the site’s twitter and this blog.
  • a way for users to suggest new genres.

And here’s some maybes – I’d love feedback on them!

  • links to buy books: a couple of people have asked for these, and it’d let me pay for hosting without adverts. I don’t want to fund Amazon, though – where do you buy books online?
  • more complex user pages: is there anything else would you like to see on your user page? Drupal, the software I’m using to build the site, has limited support for this kind of complexity, but as it is there’s not much space for interactivity between users, and in the long term I’d like to change that
  • field for representation of disabled characters: I’ve had mixed feedback about this: would you know how to answer the question “does the book represent disabled characters positively?”

My hunch is the field would be most useful if it was like the LGBT and POC fields: “is the main character disabled? if not is more than half of the book narrated by a disabled character?”.

There are a lot of books out there, and the way I see it is that aiming high from the start means that in the long term the archive will be more useful. But I’m currently non-disabled and I’d welcome lots of input from people with disabilities about this topic.

What would you like to see on Kwerey.com? Have any bugs interrupted your book-adding and site-browsing experiences? Lemme know in comments!

Kwerey: a blog

This is going to be the Kwerey.com blog, documenting site issues and discussing potential changes to the catalogue. Here’s an introduction to Kwerey.com, drawn straight from our About page.

Kwerey.com is a whole new kind of library catalogue: it’s a database of books that you can search not just by their titles and authors, but by what’s on the inside.

Readers of genre fiction will have noticed some trends in their genres of choice: science fiction has grizzled space marines, fantasy has the bastard sons of kings finding their birthright, romance novels have a definite tendency towards straight romance between manly men and dainty ladies, and so on. After a while, those tropes get old, so when you search the Kwerey catalogue, you can ask the site to only display results where the main character is a woman, or where they’re bisexual, or where they’re not white. It’s all about signal-to-noise ratio: bookworms like me have read a lot of books about straight white men by the time we get to fifteen: it’s time for a bit of variety.

This is also a database of books by women. As an undergrad doing a traditional Literature degree at a traditional university, almost all the books were by men. Somewhere between 80% and 90% of the supplementary reading I did was on books by men. I wrote a dissertation on women’s writing during the Second World War, and more than half of the background reading and context and criticism my supervisor suggested to me for that specifically woman-centric project was by men.

The core idea for the site was born from that. Here it is: it’s harder to find books by women.

In the eyes of critics, writing by women is writing about women, and writing by men is about The Human Condition. Women are less visible, they’re less likely to be recommended, they’re not as frequently reviewed or nominated for awards. Last year I made an active choice to only buy books by women, but when I looked at my bookshelves at the end of the year, I still had nearly twice as many novels by men, and only one of the non-fiction books on my shelf was written by a woman. Finding books by women is hard – the default gender balance on our bookshelves is so far off kilter it’ll take a big push on the other side to balance it back out.

I can’t provide that on my own, but I can make a start on it. And if you’d like to come help, you’re very welcome. Join up here and tell us about authors we haven’t heard of, add some books to the catalogue and write a recommendation or two. Together, we’ll find some great writing.