Navigation

Easy Unit Testing

Unit testing is a very important task of professional, scalable software development. Many tools exist to support unit testing in one or another way. All tools come with advantages and drawbacks. One of the best known test frameworks in the PHP world is PHPUnit. With the release of symfony, Fabien Potencier released another new testing framework for PHP: lime. The biggest advantage of lime over PHPUnit surely is the conciseness of the written test code. There are several disadvantages as well, which include bad test encapsulation due to the lack of support for fixture setup and teardown, and missing support for mock object generation.

Today I will briefly speak about the advantages of both frameworks, and how they can be combined to result in a slicker, powerful testing framework. I will show you how easy testing really can be! And you will be able to try it out, because all the required code has already been released in sfLimeExtraPlugin.

Introduction

In the following sections, I will refer to a set of example classes that I will briefly describe here for your better understanding:

class User
{
  protected $storage;
 
  public function User(SessionStorageInterface $storage)
  {
    $this->storage = $storage;
  }
 
  public function setAttribute($name, $value)
  {
    $this->storage->write($name, $value);
  }
 
  public function getAttribute($name)
  {
    return $this->storage->read($name);
  }
}

The user class offers methods to store data of a user in a session storage that you need to pass to its constructor.

interface SessionStorageInterface
{
  public function write($key, $value);
  public function read($key);
}

Let’s assume that SessionStorageInterface specifies how session storages have to look like. They need methods to write values into the underlying layer, be it the file system or database, and methods to read values.

We will be writing unit tests for the User class. We don’t want to use a real session storage class though, because we don’t want the test to rely on the file system or a database. Thus we will create a fake implementation of SessionStorageInterface:

class StubSessionStorage implements SessionStorageInterface
{
  private $name;
  private $value;
 
  public function write($name, $value)
  {
    $this->name = $name;
    $this->value = $value;
  }
 
  public function read($name)
  {
    return $name == $this->name ? $this->value : null;
  }
}

Fake implementations like this are called “Stubs”. Obviously, they would never make sense in a production environment. Their sole purpose is to help us testing, as in our case, the User class.

PHPUnit and the xUnit Family

Back in 1999, Kent Beck wrote “the mother of all unit testing frameworks”; it was called “SUnit” and supported unit testing for Smalltalk[1]. SUnit lead to a further testing framework, that Beck wrote in cooperation with Erich Gamma: jUnit, for Java. Today, jUnit is the most popular and widely-used unit testing framework for Java and hence it was ported to many other languages: CppUnit for C++, NUnit for C# or PHPUnit for PHP. All those frameworks together are often referred to as the xUnit family[2].

PHPUnit is a high-quality port of jUnit to PHP written by Sebastian Bergmann, currently available in stable version 3.3. A typical test case with PHPUnit looks like this:

class UserTest extends PHPUnit_Framework_TestCase
{
  private $sessionStorage;
  private $user;
 
  public function setUp()
  {
    $this->sessionStorage = new StubSessionStorage();
    $this->user = new User($this->sessionStorage);
  }
 
  public function testAttributesAreReadFromTheSession()
  {
    // fixture setup
    $this->sessionStorage->write('Foo', 'Bar');
    // execute test
    $value = $this->user->getAttribute('Foo');
    // verify results
    $this->assertEquals($value, 'Bar', 'The value was read from the session');
  }
 
  public function testAttributesAreWrittenToTheSession()
  {
  // ...
  }
}

What does this code do? First of all, we see that the test code is organized within a test class. This means that you can use all the power of object-orientation for testing, but it also implies that a certain amount of code overhead is required for defining the class’s structure.

All methods prefixed with “test” are test methods. These methods test that a certain requirement is successfully fulfilled by the tested class. Because most of the test methods need a common set of objects (the “fixture”), these objects are created in the method setUp(), which is executed once before every test method. As a result, each test method works with fresh objects; influences of the previously executed test methods are largely prevented.

We also notice that test methods have descriptive names (which, to be honest, tend to look very weird). The purpose is to allow the reader to quickly scan the method names of a test class to receive an impression of the tested class’s abilities.

In our single, exemplary test method, we first set up the fixture; we tell the fake session storage, which has been instantiated in setUp(), to return the value “Bar” when the method read("Foo") is called. Then we call getAttribute("Foo") on the user, which we expect to read from the session. In the last line of code, we assert that the returned value has indeed been read from the session.

Lime

Lime is a testing framework created by Fabien Potencier for testing the source code of the web framework symfony. It is based on the Test::More Perl library and aims for a very concise and readable test code. The above test in lime looks like this:

$t = new lime_test(1);
 
$t->comment('Attributes are read from the session');
 
  // fixture
  $s = new StubSessionStorage();
  $u = new User($s);
  $s->write('Foo', 'Bar');
  // test
  $value = $u->getAttribute('Foo');
  // assertions
  $t->is($value, 'Bar', 'The value was read from the session');
 
$t->comment('Attributes are written to the session');
 
  // ...

Contrary to PHPUnit, lime tests are written in a procedural way. Thus the code is more concise, because you don’t need to write down structural information of the code.

Initially a lime_test object is created, which tracks the number of expected, successful and failing tests and offers several methods to make testing easier. Each test case is, by convention, introduced by a comment that explains the tests purpose. Therefore the method comment() is called, which does also print the comment on the console when executing the test.

The test fixture has to be created manually for each test case. This is very prone to errors, because you can easily forget to reassign a variable once in a while. Then you’ll suddenly deal with an object left over from a previous test, which may lead to strange and unexpected test results.

Pro and Contra

Let us roughly sum up the advantages and disadvantages of both frameworks:

PHPUnit…

  • … is verbose
  • … offers magic methods like setUp(), which initiates your test fixture before every test
  • … offers other convenient tools not covered in this blog post, like mocking support

lime…

  • … is concise and readable
  • … requires code repetition
  • … requires you to initiate your fixture manually

sfLimeExtraPlugin extends lime and tries to introduce concepts of the xUnit family without making tests more verbose. Quite the opposite, because sfLimeExtraPlugin supports annotations, the written tests are even more concise than without this plugin.

Annotation-Driven Tests

The above test case, written with support of sfLimeExtraPlugin, looks like this:

$t = new lime_test_simple(1);
 
// @Before
$s = new StubSessionStorage();
$u = new User($s);
 
// @Test: Attributes are read from the session
// fixture
$s->write('Foo', 'Bar');
// test
$value = $u->getAttribute('Foo');
// assertions
$t->is($value, 'Bar', 'The value was read from the session');
 
// @Test: Attributes are written to the session
// ...

This test leads to exactly the same test results and console output as the test written with plain lime earlier in this post.

sfLimeExtraPlugin introduces the new class lime_test_simple. When you use that test class, you can mark sections of your code with so-called annotations. The test class knows, for example, that code annotated with @Before must be executed before every test case.

Single test cases are annotated with @Test. You can also add a comment about the purpose of the test. Note that the comment now really is a PHP comment, which is usually highlighted in a different color by code editors and thus disturbs the eye much less than the call to the method comment().

Several other annotations are available. I’ll shortly list all of them:

@Test
A test case
@Before
Executed before each test case
@After
Executed after each test case
@BeforeAll
Executed once before all test cases
@AfterAll
Executed once after all test cases

With these annotations, you can easily structure your test code and avoid code duplication while making your tests easier to read.

Testing for Exceptions

Contrary to plain lime_test, lime_test_simple allows you to automatically test whether exceptions are thrown. With plain lime_test, such a test would look like this:

$t->comment('setAttribute() throws an exception if the data contains <script> tags');
 
  // fixture
  $u = new User(new StubSessionStorage());
  // test
  try
  {
    $u->setAttribute('<script>alert("Evil!")</script>');
    $t->fail('setAttribute() throws an "InvalidArgumentException"');
  }
  catch (InvalidArgumentException $e)
  {
    $t->pass('setAttribute() throws an "InvalidArgumentException"');
  }

In this test, we try to catch the expected exception. If the exception is caught, we mark the test as passed, otherwise as failed.

With lime_test_simple, this is much easier:

// @Before
$u = new User(new StubSessionStorage());
 
// @Test: setAttribute() throws an exception if the data contains <script> tags
// fixture
$t->expect('InvalidArgumentException');
// test
$u->setAttribute('<script>alert("Evil!")</script>');

Mock and Stub Objects

Like PHPUnit, sfLimeExtraPlugin allows you to automatically generate fake objects, which are referred to as “Mocks” and “Stubs”. So far, we had to write our fake StubSessionStorage class by hand. With sfLimeExtraPlugin this is not needed anymore. The component lime_mock will generate such a class automatically:

// @Before
$s = lime_mock::create('SessionStorageInterface');
$u = new User($s);
 
// @Test: Attributes are read from the session
// fixture
$s->read('Foo')->returns('Bar');
$s->replay();
// test
$value = $u->getAttribute('Foo');
// assertions
$t->is($value, 'Bar', 'The value was read from the session');

Before the execution of each test case, we tell lime_mock to generate a new fake instance of SessionStorageInterface. In the test itself, we teach the fake object which methods can be called with what parameters and which value should be returned. We can also specify other constraints, such as how often a method may be called or which exception it should throw.

Then we switch the fake object into “replay” mode. In this mode, the fake object will behave just the way that we configured it before.

You can create stubs for interfaces, classes or abstract classes. You can even create stubs for non-existing classes, which is very convenient if you develop test-driven.

Because this topic deserves a whole blog post of its own, I won’t go into more detail here. For more information about Mocks and Stubs in general and their usage in sfLimeExtraPlugin in specific can be found on the readme page of sfLimeExtraPlugin.

Final Words

I personally think that tests can be written in a much more concise and readable way with sfLimeExtraPlugin. Because it also introduces other powerful features of the xUnit-family, the plugin aims to be an essential tool of every symfony developer who wants to seriously unit test his or her application.

Currently the plugin is available in version 0.2.0alpha. That means that the API may change before the final release (though this is unlikely) and that the code is not being considered 100% stable. I recommend you to try it out nevertheless and give me feedback about its usefulness or shortcomings, report bugs etc.

What do you think about sfLimeExtraPlugin? Do you think you may ever use it?

References

[1] Kent Beck, Donald G. Firesmith: Kent Beck’s Guide to Better Smalltalk. Cambridge University Press, 1998. Page 408

[2] Gerard Meszaros: xUnit Test Patterns. Refactoring Test Code. Addison-Wesley, 2007. Page 75

Posted Tuesday, June 30th, 2009 at 10:06
Written by: | Filed Under Category: Development
You can leave a response, or trackback from your own site.

18

Responses to “Easy Unit Testing”

very nice. i had to write a lot of junit tests in the last weeks and i really learned to love tearDown and setUp. nice to have this features now also in sf.

Very interesting – I’ll try it out!

Leon (lvanderree)

Again a great article Bernhard!

I will keep you informed about my experiences as soon as I will be using it. (I’ll try to do that this week already)

After a week off with my newborn twins I’m back at work and found a little time to play around with sfLimeExtraPlugin.

Here are a few questions from my side:

First question: I’m working with full-blown symfony unit tests, so e.g. autoloading is activated. When I try to mock an existing class (lime_mock::create(‘myClass’)), I get the following error:

Fatal error: Cannot redeclare class myClass in plugins/sfLimeExtraPlugin/lib/mock/lime_mock.class.php(173) : eval()’d code on line 1

Is there a way to to allow mocking existing Objects?

Second question: Your concise way to test exceptions is much more readable due to less code. A downside is that I can’t specify a message for the exception assertion. Is it possible to achive the same console output as below by using the functionality of sfLimeExtraPlugin (Annotations, …)?

Example: the old way:
$t->diag(‘getSomething();
try
{
$object->getSomething(123);
$t->fail(‘Doesn’t throw an exception if the parameter isn’t a string’);
}
catch (Exception $e)
{
$t->pass(‘Throws an exception if the parameter isn’t a string’);
}
$t->is($object->getSomething(‘Jonas’), ‘Baby’, ‘Returns the correct result for a given string’);

$t->diag(‘getSomethingElse();

This test outputs something like:
# getSomething()
ok 1 – Throws an exception if the parameter isn’t a string
ok 2 – Returns the correct result for a given string
# getSomethingElse()

Futhermore I would change the method $t->expect() a little:
I would call it expectException() so it’s clear that we expect an exception, and if no argument is given we just expect any exception.

As I haven’t worked with annotations before the code feels quite unfamiliar and strange in the beginning. I’m not sure which effect annotations will have on debugging etc…
By the way is it possible to comment out annotations ? :-)

One small remark at the end concerning the code block under the “Lime” headline: Using more than a single “one-letter” variable like $t, $u, $s, … makes reading the code a bit harder for me. I would use meaningful names like $user, $session etc.

Bernhard

Hello Klemens,

Mocking existing objects should be possible, but it can be that this is a bug. I think that I already fixed this bug in Lime 2:
http://svn.symfony-project.com/tools/lime/branches/2.0-experimental

The code from sfLimeExtraPlugin has now all been ported to Lime 2 and is being worked on there. Thus sfLimeExtraPlugin will probably not be further developed for a while, but you are warmly welcomed to test Lime 2.

Annotations don’t have much effect on debugging. The line numbers and file names in error messages are not affected by annotations.

I will try to include the possibility to render custom exception messages. And sorry for the short variable names – I will do better next time :-)

Thanks for your response.
Congratulations for the inclusion of sfLimeExtraPlugin code in lime2!

I believe you have a naming error between your StubSessionStorage and when you call it for instantiation: SessionStorageStub.

… class StubSessionStorage implements SessionStorageInterface …

… and …

$this->sessionStorage = new SessionStorageStub();

Thanks,
Sean

Bernhard

@Sean: Thanks, fixed.

[...] a previous post from the Web Mozarts blog they look at unit testing your PHP applications and some of the different testing software alternatives out there to help. [...]

[...] a previous post from the Web Mozarts blog they look at unit testing your PHP applications and some of the different testing software alternatives out there to help. [...]

Nice tutorial, thanks a lot!

[...] Easy Unit Testing – Straipsnelis apie php testavima naudojant įrankius kaip PHPUnit ir pan. Asmeniškai pats nenaudoju tokių dalykų nors savaime aišku, kad reikėtų, tačiau tokiem testam reikia laiko kurio dažniausiai nebūna. [...]

Henning

Bernhard,

I have not tried the plugin yet, but from what I see in this article it is a very good addition to the standard lime test framework. Thanks!
Having a strong background in C++ and being a user of cppUnit (part of xUnit family) I like the features this plugin adds.
However, I noticed several times in the article you stated “Like phpUnit, sfLimeExtraPlugin…”. Since this plugin adds some features that phpUnit has (making it a bit more phpUnit like) and considering that phpUnit has a lot of other really useful features like code coverage metrics that can be tied into systems like Cruisecontrol/PhpUnderControl, skeleton generator, ties into phing and other build tools, part of xUnit family and therefore more familiar to many people, etc., why not use PhpUnit in the first place? (I know it is a bit late for this question since lime is an integral part of sf and has been so from the start)

Thanks,
Henning

[...] A dziś przyszedł czas na odświeżony framework do testowania – lime. I polecam go nie z powodu uwielbienia dla bibliotek wyżej wspomnianej firmy, ale dlatego, że to nieco inne podejście do testów, jakie znamy w PHPWpis długi nie będzie, większość możecie przeczytać na blogu symfony i blogu samego twórc, Berhnarda Schusska [...]

[...] als Lime 2 weiterzuentwickeln. Zwei interessante Blogposts über seine Beweggründe hat er hier und hier geschrieben. Wie man dann mit Lime 2 testet hat er bereits auf dem symfony Day Cologne 2009 [...]

[...] erstaunlicher, dass es bisher keine eigene Website für Lime 2 gibt und die letzten Blogposts [1], [2] zum Thema liegt mittlerweile 3 bis 9 Monate [...]

@Bernhard
http://test.ical.ly/2010/02/24/alles-neu-in-symfony-2-0-jetzt-auch-das-testen-mit-phpunit/
magst du dich vielleicht mal zu den aktuellen Entwicklungen melden? Wäre toll!

Gruss
/Christian

Leave a Reply

 

Additional Resources