Today I will briefly speak about the advantages of both frameworks, and how they can be combined to result in a slicker, powerful testing framework. I will show you how easy testing really can be! And you will be able to try it out, because all the required code has already been released in sfLimeExtraPlugin.

Introduction

In the following sections, I will refer to a set of example classes that I will briefly describe here for your better understanding:

class User
{
  protected $storage;
 
  public function User(SessionStorageInterface $storage)
  {
    $this->storage = $storage;
  }
 
  public function setAttribute($name, $value)
  {
    $this->storage->write($name, $value);
  }
 
  public function getAttribute($name)
  {
    return $this->storage->read($name);
  }
}

The user class offers methods to store data of a user in a session storage that you need to pass to its constructor.

interface SessionStorageInterface
{
  public function write($key, $value);
  public function read($key);
}

Let’s assume that SessionStorageInterface specifies how session storages have to look like. They need methods to write values into the underlying layer, be it the file system or database, and methods to read values.

We will be writing unit tests for the User class. We don’t want to use a real session storage class though, because we don’t want the test to rely on the file system or a database. Thus we will create a fake implementation of SessionStorageInterface:

class StubSessionStorage implements SessionStorageInterface
{
  private $name;
  private $value;
 
  public function write($name, $value)
  {
    $this->name = $name;
    $this->value = $value;
  }
 
  public function read($name)
  {
    return $name == $this->name ? $this->value : null;
  }
}

Fake implementations like this are called “Stubs”. Obviously, they would never make sense in a production environment. Their sole purpose is to help us testing, as in our case, the User class.

PHPUnit and the xUnit Family

Back in 1999, Kent Beck wrote “the mother of all unit testing frameworks”; it was called “SUnit” and supported unit testing for Smalltalk[1]. SUnit lead to a further testing framework, that Beck wrote in cooperation with Erich Gamma: jUnit, for Java. Today, jUnit is the most popular and widely-used unit testing framework for Java and hence it was ported to many other languages: CppUnit for C++, NUnit for C# or PHPUnit for PHP. All those frameworks together are often referred to as the xUnit family[2].

PHPUnit is a high-quality port of jUnit to PHP written by Sebastian Bergmann, currently available in stable version 3.3. A typical test case with PHPUnit looks like this:

class UserTest extends PHPUnit_Framework_TestCase
{
  private $sessionStorage;
  private $user;
 
  public function setUp()
  {
    $this->sessionStorage = new StubSessionStorage();
    $this->user = new User($this->sessionStorage);
  }
 
  public function testAttributesAreReadFromTheSession()
  {
    // fixture setup
    $this->sessionStorage->write('Foo', 'Bar');
    // execute test
    $value = $this->user->getAttribute('Foo');
    // verify results
    $this->assertEquals($value, 'Bar', 'The value was read from the session');
  }
 
  public function testAttributesAreWrittenToTheSession()
  {
  // ...
  }
}

What does this code do? First of all, we see that the test code is organized within a test class. This means that you can use all the power of object-orientation for testing, but it also implies that a certain amount of code overhead is required for defining the class’s structure.

All methods prefixed with “test” are test methods. These methods test that a certain requirement is successfully fulfilled by the tested class. Because most of the test methods need a common set of objects (the “fixture”), these objects are created in the method setUp(), which is executed once before every test method. As a result, each test method works with fresh objects; influences of the previously executed test methods are largely prevented.

We also notice that test methods have descriptive names (which, to be honest, tend to look very weird). The purpose is to allow the reader to quickly scan the method names of a test class to receive an impression of the tested class’s abilities.

In our single, exemplary test method, we first set up the fixture; we tell the fake session storage, which has been instantiated in setUp(), to return the value “Bar” when the method read("Foo") is called. Then we call getAttribute("Foo") on the user, which we expect to read from the session. In the last line of code, we assert that the returned value has indeed been read from the session.

Lime

Lime is a testing framework created by Fabien Potencier for testing the source code of the web framework symfony. It is based on the Test::More Perl library and aims for a very concise and readable test code. The above test in lime looks like this:

$t = new lime_test(1);
 
$t->comment('Attributes are read from the session');
 
  // fixture
  $s = new StubSessionStorage();
  $u = new User($s);
  $s->write('Foo', 'Bar');
  // test
  $value = $u->getAttribute('Foo');
  // assertions
  $t->is($value, 'Bar', 'The value was read from the session');
 
$t->comment('Attributes are written to the session');
 
  // ...

Contrary to PHPUnit, lime tests are written in a procedural way. Thus the code is more concise, because you don’t need to write down structural information of the code.

Initially a lime_test object is created, which tracks the number of expected, successful and failing tests and offers several methods to make testing easier. Each test case is, by convention, introduced by a comment that explains the tests purpose. Therefore the method comment() is called, which does also print the comment on the console when executing the test.

The test fixture has to be created manually for each test case. This is very prone to errors, because you can easily forget to reassign a variable once in a while. Then you’ll suddenly deal with an object left over from a previous test, which may lead to strange and unexpected test results.

Pro and Contra

Let us roughly sum up the advantages and disadvantages of both frameworks:

PHPUnit…

• … is verbose
• … offers magic methods like setUp(), which initiates your test fixture before every test
• … offers other convenient tools not covered in this blog post, like mocking support

lime…

• … is concise and readable
• … requires code repetition
• … requires you to initiate your fixture manually

sfLimeExtraPlugin extends lime and tries to introduce concepts of the xUnit family without making tests more verbose. Quite the opposite, because sfLimeExtraPlugin supports annotations, the written tests are even more concise than without this plugin.

Annotation-Driven Tests

The above test case, written with support of sfLimeExtraPlugin, looks like this:

$t = new lime_test_simple(1);
 
// @Before
$s = new StubSessionStorage();
$u = new User($s);
 
// @Test: Attributes are read from the session
// fixture
$s->write('Foo', 'Bar');
// test
$value = $u->getAttribute('Foo');
// assertions
$t->is($value, 'Bar', 'The value was read from the session');
 
// @Test: Attributes are written to the session
// ...

This test leads to exactly the same test results and console output as the test written with plain lime earlier in this post.

sfLimeExtraPlugin introduces the new class lime_test_simple. When you use that test class, you can mark sections of your code with so-called annotations. The test class knows, for example, that code annotated with @Before must be executed before every test case.

Single test cases are annotated with @Test. You can also add a comment about the purpose of the test. Note that the comment now really is a PHP comment, which is usually highlighted in a different color by code editors and thus disturbs the eye much less than the call to the method comment().

Several other annotations are available. I’ll shortly list all of them:

@Test

A test case

@Before

Executed before each test case

@After

Executed after each test case

@BeforeAll

Executed once before all test cases

@AfterAll

Executed once after all test cases

With these annotations, you can easily structure your test code and avoid code duplication while making your tests easier to read.

Testing for Exceptions

Contrary to plain lime_test, lime_test_simple allows you to automatically test whether exceptions are thrown. With plain lime_test, such a test would look like this:

$t->comment('setAttribute() throws an exception if the data contains <script> tags');
 
  // fixture
  $u = new User(new StubSessionStorage());
  // test
  try
  {
    $u->setAttribute('<script>alert("Evil!")</script>');
    $t->fail('setAttribute() throws an "InvalidArgumentException"');
  }
  catch (InvalidArgumentException $e)
  {
    $t->pass('setAttribute() throws an "InvalidArgumentException"');
  }

In this test, we try to catch the expected exception. If the exception is caught, we mark the test as passed, otherwise as failed.

With lime_test_simple, this is much easier:

// @Before
$u = new User(new StubSessionStorage());
 
// @Test: setAttribute() throws an exception if the data contains <script> tags
// fixture
$t->expect('InvalidArgumentException');
// test
$u->setAttribute('<script>alert("Evil!")</script>');

Mock and Stub Objects

Like PHPUnit, sfLimeExtraPlugin allows you to automatically generate fake objects, which are referred to as “Mocks” and “Stubs”. So far, we had to write our fake StubSessionStorage class by hand. With sfLimeExtraPlugin this is not needed anymore. The component lime_mock will generate such a class automatically:

// @Before
$s = lime_mock::create('SessionStorageInterface');
$u = new User($s);
 
// @Test: Attributes are read from the session
// fixture
$s->read('Foo')->returns('Bar');
$s->replay();
// test
$value = $u->getAttribute('Foo');
// assertions
$t->is($value, 'Bar', 'The value was read from the session');

Before the execution of each test case, we tell lime_mock to generate a new fake instance of SessionStorageInterface. In the test itself, we teach the fake object which methods can be called with what parameters and which value should be returned. We can also specify other constraints, such as how often a method may be called or which exception it should throw.

Then we switch the fake object into “replay” mode. In this mode, the fake object will behave just the way that we configured it before.

You can create stubs for interfaces, classes or abstract classes. You can even create stubs for non-existing classes, which is very convenient if you develop test-driven.

Because this topic deserves a whole blog post of its own, I won’t go into more detail here. For more information about Mocks and Stubs in general and their usage in sfLimeExtraPlugin in specific can be found on the readme page of sfLimeExtraPlugin.

Final Words

I personally think that tests can be written in a much more concise and readable way with sfLimeExtraPlugin. Because it also introduces other powerful features of the xUnit-family, the plugin aims to be an essential tool of every symfony developer who wants to seriously unit test his or her application.

Currently the plugin is available in version 0.2.0alpha. That means that the API may change before the final release (though this is unlikely) and that the code is not being considered 100% stable. I recommend you to try it out nevertheless and give me feedback about its usefulness or shortcomings, report bugs etc.

What do you think about sfLimeExtraPlugin? Do you think you may ever use it?

References

[1] Kent Beck, Donald G. Firesmith: Kent Beck’s Guide to Better Smalltalk. Cambridge University Press, 1998. Page 408

[2] Gerard Meszaros: xUnit Test Patterns. Refactoring Test Code. Addison-Wesley, 2007. Page 75

No related posts.