fromSeptember 2013
Feature:

Composer: Sharing Wider

Managing Dependencies
0

Musical notes
Drupal has long had a strong collaborative culture. We share modules, we share development tasks on core and modules, and we share infrastructure on Drupal.org. That's a critical part of the health of our community: Sharing is how Open Source works.

The broader PHP world, however, has long sucked at sharing. Every project is its own island; sharing code between projects has been difficult, and managing third party libraries a pain. Just about the only option was PEAR, but unless you had root access on every server you needed, and were running only a single application per server, it wasn't really useful.

That was then, this is now. Enter Composer, a PHP dependency management tool that works. Composer began life in late 2011 in the Symfony community but was deliberately built to be project-agnostic, and today is being used by thousands of projects large and small, including Drupal.

Composer Basics

Composer consists of two parts. One is Packagist.org, which is a central clearinghouse of Composer-compatible packages. As of July 2013, Packagist offers over 13,000 packages, ranging from simple libraries to complete frameworks. The other part is Composer itself, a command line PHP application that is dead simple to install. By default, Composer will download packages from Packagist.org but you can also set up your own package server, or even just one-off Git repositories, to host Composer-capable code. All you need to make it work is a simple JSON file.

Let's start off with a trivial example. We’ll write a super-simple script that uses the Guzzle HTTP client (now bundled with Drupal 8). To start off, create your project folder. Inside it, create a directory called src. That's where we'll put all of our code. Now create a file called composer.json with the following contents:

{
  "name": "crell/demo",
  "description": "This demo app is amazing.",
  "require": {
    "guzzle/http": "3.4.*"
  },
  "autoload": {
    "psr-0": {
      "Crell\\Demo": "src/"
    }
  }
}

Let's break down what this file does. composer.json files are essentially a Drupal module .info file and a Drush Make file rolled into one. The name property specifies the package name. A package name always consists of two parts: a vendor (the authoring organization or individual) and the project name. For major projects or component-based projects, they are sometimes the same (symfony/symfony, guzzle/guzzle, drupal/drupal, etc.). If in doubt, your GitHub username is a reasonable default for smaller projects. The description is an optional, simple, human-readable description of the project.

The most important parts, though, are the require and autoload keys. The require key lets you specify one or more Packagist-hosted packages that our program requires, including specific versions. Here, we're saying we require the guzzle/http library, version 3.4.anything. It's also legal to specify a minimum version, a minimum and maximum, and various other combinations.

The autoload key specifies how our program's code can be autoloaded by PHP. In this case, we say that any classes in the Crell\Demo namespace can be found in the src directory and follow the PSR-0 autoloading standard. We could also specify classmap instead of psr-0, which would cause Composer to scan every file in the src directory individually to find what classes it offers. More on that in a moment.

Now, let's download the Composer program and run it. The instructions can be found on GetComposer.org, but in most cases it's just a one-liner:

$ curl -sS https://getcomposer.org/installer | php

That does a little setup, and downloads the composer.phar file. (PHAR is a PHP archive format for compressing all of a program into a single executable; very useful for command line tools.) To run it, simply type the following:

$ php composer.phar install

That's it. Composer will examine our composer.json file, see that we need Guzzle, and download the latest 3.4.x version into a vendor directory. Guzzle in turn happens to require the Symfony EventDispatcher library, so that will get downloaded, too. Composer will also generate an autoloader for us that will cover Guzzle, EventDispatcher, and our code in src. All from one command!

Now we can include that autoloader from our own front controller, index.php, and we're done:

<?php
require_once __DIR__ . '/vendor/autoload.php';

// Autoload on demand! Your work: zero.
$client = new Guzzle\Http\Client('https://api.github.com');
$request = $client->get('/user')->setAuth('user', 'pass');
$response = $request->send();
echo $response->getBody();
?>

An important note is that the contents of the vendor directory should not be checked into version control. Instead, Composer generates a composer.lock file, which tracks what exact version of each dependency is installed, down to the specific Git commit. Then, when another developer checks out the project and runs composer.phar install, the system will look at the lock file first, and install exactly that version of everything. That way, there's no mystery bugs from different developers running different versions of a third party library. To update to a newer version of a library or add new dependencies, simply run composer.phar update to ignore the lock file and pull in the latest compatible versions from the composer.json file.

Composing for Drupal

Drupal 8 makes extensive use of Composer to pull in the many third party libraries that have been incorporated into the new system. As of this writing, how exactly contributed modules will leverage Composer in Drupal 8 is still being worked out, but contrib can leverage Composer now in Drupal 7. In fact, in some cases it's better to develop a new module as a Composer library, not as a Drupal module, and then write a simple module to bridge it to Drupal.

As an example, let's look at the Rotten Tomatoes module,i recently released by Palantir.net. We built this module as part of a project for a large media company that wanted to integrate data from the Rotten Tomatoes API into their Drupal site.

Part one is the Rotten Tomatoes library we built, now available through Packagist. This library contains no Drupal-specific code. It leverages Guzzle (a dependency specified through its composer.json file) to provide a clean object-oriented interface to the Rotten Tomatoes REST API (or at least the portions of it that we needed to work with).

Part two is the Rotten Tomatoes module. The Drupal-specific bits of the project reside in the module, and specifically import logic to turn Rotten Tomatoes movies and reviews into Drupal nodes. Movie and Review node types are provided via the Features module. The module itself doesn't do much on its own, but developers can build an import workflow around it as we did for our client. The module includes a composer.json file that specifies the Composer library we wrote as a dependency:

{
  "require": {
   "palantirnet/rottentomatoes": "1.*"
  }
}

Part three is tying it all together. Since Drupal 7 doesn't use a Composer-based autoloader, we need to add one. Enter the Composer Autoload module.iv This simple module scans all available modules for a vendor directory and generated Composer autoloader and includes it from hook_init(), exposing the autoloader to PHP.  (The scan information is cached for performance.) Simply run composer.phar install from within the module directory, clear cache, and all of the dependent code is now available.

But what happens if you have multiple modules with different composer.json files? composer_autoload will handle that just fine, but you may end up with multiple copies of a library; Composer can sort out duplicates from a single composer.json file and its dependencies, but not for independent files. Enter the Composer Manager module.

Composer Managerv is a still in-progress module. It uses Drush to collate all available composer.json files together into a single file in the files directory, which can then be run through composer.phar all at once to generate a single, unified, vendor directory. Unfortunately, it does not automatically register the generated autoloader. The module's author has instead provided a manual function to call to initialize the Composer autoloader, which is less than ideal. The argument is that hook_init() (or hook_boot()) causes potential circular dependencies in cases where a class is autoloaded early in the request. While valid, it makes this module a less-than-suitable solution at the moment. Hopefully, that will change as time goes on.

Composing the Future

The better solution will have to wait for Drupal 8. Drupal is using Composer to leverage 17 different third party libraries (nine of them from the Symfony project). As of this writing, we are checking the vendor directory into Drupal repository due to limitations of our packaging system. We hope to resolve that before release.

Still an open question is how contributed modules can leverage Composer libraries. The adventurous will be able to simply edit the Drupal composer.json file and rerun composer.phar install, but that's not ideal for modules. There is active discussion of how that should be handled.vi If the potential of Composer to produce more generic, loosely coupled code that can be shared between Drupal and other projects excites you (share more, work less!), this is a good opportunity to get involved in core development.

Advanced Composer

Composer has a lot of other options, too. One of the most important is support for dev-only requirements. For example, suppose your project uses PHPUnit for unit tests, and the Symfony BrowserKit component for doing functional testing. (This is a not uncommon approach.) We don't want those libraries in production, but we do need them for development purposes. In that case we can specify them as a require-dev dependency:

{
  "name": "crell/demo",
  "description": "This demo app is amazing.",
  "require": {
    "guzzle/http": "3.4.*"
  },  "require-dev": {    "phpunit/phpunit": "3.7.*",    "symfony/browser-kit": ">=2.3,<2.4-dev",    
"symfony/css-selector": ">=2.3,<2.4-dev",    "symfony/dom-crawler": ">=2.3,<2.4-dev",    
"symfony/finder": ">=2.3,<2.4-dev"  },  "autoload": {
    "psr-0": {

      "Crell\\Demo": "src/"
    }
  }
}

By default, Composer will now download not just Guzzle but also PHPUnit and four Symfony libraries useful for testing. We can then run whatever PHPUnit-based tests we want. For production, though, we don't need to bother with those libraries, so we instead run composer.phar like this:

$ composer.phar install --no-dev

That will skip over all of the require-dev dependencies, saving time and keeping unused code off of our production server.

Another useful option is --optimize-autoloader. That will tell Composer to take extra time to generate a classmap-based autoloader for all packages it downloads, whether they were using a classmap original or not. That takes a while to generate, but in most (although not all) cases it ends up being faster once it's built. That makes it a good candidate for running in production, where classes won't be changing regularly anyway.

Another good dev-vs.-production option is --prefer-source vs. --prefer-dist. By default, if you specify a stable release of a project, then Composer will download only that stable release. If you specify a development version, however, Composer will instead clone the git repository checked out to the release you specify. That allows you to develop on your dependency as well, all from the same workspace.

Of course, in production you don't need a full Git repository. If you must use a dev version of a library in production, use --prefer-dist to always download just the code itself. Alternatively, in development you can specify --prefer-source to always download the full Git repositories. That will take longer, but may make developing a set of related libraries together easier.

Moving Forward, Together

After years of isolation, the PHP community is finally beginning to embrace cross-project collaboration and sharing. Composer is a key part of that transition, and one that will be a cornerstone of many projects before too long.

Composer has other options we haven't covered here. Check out the documentation at GetComposer.org for the full list of command line options and composer.json syntax. And next time you're building a new module or library, take the small extra step to expose it to the world.

Image: ©IStockphoto.com/marsbars

Advertisement

From our blog

Entity Storage, the Drupal 8 Way

In Drupal 7 the Field API introduced the concept of swappable field storage.

The Drupal 6 to 8 Upgrade Challenge - Part 2

Having concluded the readiness assessment, we turn next to migrating the content and configuration. In reality, there’s little chance that we would migrate anything but the blogs from our old site. For the sake of giving Migrate in Core a workout with real conditions, however, we’re going to upgrade with core’s Migrate Drupal module rather than rebuilding.

The Drupal 6 to 8 Upgrade Challenge - Part 1

Nathaniel Catchpole , the Drupal 8 release maintainer and Tag1 Senior Performance Engineer, suggested that Drupal shops everywhere could support the

DrupalCon Austin

The entertainment industry is not for the faint of heart.

Drupal Watchdog Joins the Linux New Media Family
Drupal Watchdog 6.01 is the first issue published by Linux New Media.

Drupal Watchdog 6.01 is the first issue published by Linux New Media. Come see the Drupal Watchdog team at DrupalCon 2016!

Drupal Watchdog was founded in 2011 by Tag1 Consulting as a resource for the Drupal community to share news and information. Now in its sixth year, Drupal Watchdog is ready to expand to meet the needs of this growing community.

Drupal Watchdog will now be published by Linux New Media, aptly described as the Pulse of Open Source.

Welcome to DrupalCon Barcelona - The Director's Cut

For all you schedule-challenged CEOs – and ADHD coders – this Abbreviated Official Director’s Cut is just what the doctor ordered.

Welcome to DrupalCon - The Barcelona Edition

Did we have fun in Barcelona?
OMG, yes!

Did we eat all the tapas on the menu and wash them down with pitchers of sangria?
Yes indeed!

Recursive Closures and How to Get Rid of Them

This came up while manipulating taxonomy children and their children recursively, so it’s as not far from Drupal as you’d think.