PHP & Symfony About PHP and Symfony2 development

Celebrating a year with "A Year With Symfony"

Posted on by Matthias Noback

Almost a year ago, on September 4th, in front of a live audience, I published my first book "A Year With Symfony". Since the first release, well over a 1000 people have read it. In February of this year, I added a new chapter to the book (about annotations).

A third edition?

Flipping through the pages and thinking about how I could best celebrate 1 year and 1000+ readers, I had the following ideas:

  1. I think it's time to rewrite the "Project structure" chapter.
  2. I think it would be nice to expand the book with yet another bonus chapter.

Feedback?

The book itself has proven to be quite popular, though I don't often get feedback about its contents. I would like to know what you really think of it, and I'd like some more (public) reviews.

Notebook

Prizes!

I want all the above things and I want to celebrate the book's anniversary. So this is what I've come up with.

You can win the following prizes:

  • Two of you get free printed copies of "A Year With Symfony" for your entire team (up to 5 copies).
  • Four of you get free digital copies of "A Year With Symfony" for your entire team (up to 10 copies).
  • Two of you get limited edition notebooks (see the image on the right) for your entire team (up to 5 copies).

You only need to fill in the form below in order to apply for the raffle. The winners will be announced near the end of September.

Categories: Book PHP Symfony2

Tags: A Year With Symfony prizes

Comments: Comments

The PHP testing experience: Interview by Fernando Arconada

Posted on by Matthias Noback

Testing para Aplicaciones Symfony2 Fernando Arconada interviewed me about the subject of testing. He is writing a book about testing Symfony2 applications: Testing para Aplicaciones Symfony2. Fernando will translate this interview to Spanish and and add it to his book, together with the articles in my A better PHP testing experience series.

Who is Matthias Noback?

I'm a PHP developer, writer and speaker. I live in Zeist, The Netherlands, with my girlfriend, a son of 9 and our newborn daughter. Currently I have my own business, called Noback's Office. This really gives me a lot of freedom: I work as a developer on one project for about half of the week and in the remaining time I can either spend some time with my family or write blog posts, or finish my second book.

What's your experience with BDD/TDD testing?

As I wrote in the introduction to a series of articles about testing on my blog: I've been writing unit tests for my code for a couple of years now. Unit testing really changed the way I work and it has also made the code I deliver much better. Classes that are well-designed are also easily testable (and vice versa). Equally important: unit testing helped me reduce the bug count and the amount of rework a lot.

I don't always write a test first, though I'm gradually moving towards the test-first practice, since I believe it's the only way I can guarantee that what I'm building serves the purpose which I or my customer had in mind.

Recently I also started writing scenario's (for a private project), running them with Behat. I did this after writing the code, which made the process somewhat harder. Making existing code testable is almost always harder than writing testable code in te first place. Still, seeing all those successful steps in each scenario gave me a lot of confidence that the application provides what I need in terms of my own business (it's a time-tracking, auto-invoicing tool - it's simple but there's money involved, so it must always work...).

Any open source project that you are proud of?

I haven't really worked on any big projects yet, just small libraries. I'm somewhat proud of the Leanpub sampler. You can use it to loop over Leanpub book manuscript files and extract sample text from them. It contains some nice iterators and uses the Aura.Cli command line library.

Should I write tests even for small projects?

The reason you're asking is - I think - that automated tests are presumed to be a replacement for manual checks. When your project is small, why bother with writing those automated tests? The time you need to invest for them really outweighs the time you would need for quickly checking the behavior of your application using the UI (be it a browser or a command prompt).

In my experience small projects become big projects. If you want to introduce tests for the usual reasons, then you will have a hard time adding tests for existing code (like I mentioned above: making untested production code testable probably takes quite some time). On the other hand writing tests for small projects is particularly easy, much easier than writing tests for larger projects. And at such small cost, if you do write tests for a small project it will give you all the usual advantages of an automated test suite:

  • You can be sure that one change here, doesn't break the application there.
  • You don't need to rely on the consistency of your own behavior when you do your manual checks. You can leave it to the computer.

What's your opinion about PHP projects in general?

Looking at the PHP community today and comparing it to a couple of years ago, I've come to the conclusion that PHP developers nowadays care more about open source. The good thing about open source software is: people work really hard to get it right. In particular because the open source code you write may be judged by anybody who looks at your GitHub page. So you make it as good as you can (you may even add some tests!). In reality there is still lots of undocumented, untested rubbish code on GitHub, but in general (although I have no numbers to support my claim), things have been really getting better over time.

Do we need 100% coverage?

No, almost nobody has a 100% unit test coverage. Using a strict TDD workflow, you would automatically have full coverage. But you probably don't do that. Even if you do believe that TDD is the ideal way of developing code (like I do), still you "won't feel like it" all the time. So there goes your 100% coverage.

Do you test from the beginning of the project?

No, not always. Though I always think hard about it before I choose not to. Open source code needs tests because people are going to rely on it. My own code needs tests if I don't trust myself with the subject matter or if (part of) my business depends on it. In other circumstances I may choose not to write tests for my code. Please note however that I almost always regret this decision because not writing tests initially made me go faster, but really slowed me down in the end.

PHPSpec, Behat, Codeception, PHPUnit, which ones do you use?

I mostly use PHPUnit, sometimes also for functional tests. As I mentioned earlier I also use Behat now. I tried Codeception for some time but it "didn't feel good" (I've got no better reason than just this gut feeling). I've experimented with PHPSpec a bit, as well as Prophecy, the test double generation tool it uses. In my experience you can easily use PHPUnit like you use PHPSpec, but if you are new to testing, I definitely recommend PHPSpec since it will more easily guide you in the right direction when it comes to unit testing (or "object spec-ing"). Prophecy is actually really nice and I feel that it makes your tests better if you use it. I personally use PHPUnit and the PHPUnit - Prophecy bridge package, which makes it easy to define test doubles with Prophecy in PHPUnit test cases.

Is there any other quality tool that you use?

I use tools to verify the coding standard of a project (i.e. PHP_CodeSniffer). For open source libraries I use Scrutinizer CI which is a continuous integration service for PHP projects. For each commit it gives you some figures about the code quality and ways you can improve it. Over time you can track the state of the project. It offers some good insight in your code and if you follow (most of) its advice, your project ends up being a lot better than before.

Is TDD dead? And BDD?

No, I don't think that TDD is dead. I think "do everything test-first" is dead (at least when it comes to unit testing), or maybe it was never alive, except in coding dojos.

What started the "TDD is dead" discussion is a post by David Hansson. Though I'm not sure he abandons TDD for all the right reasons, he says some interesting and honest things about his experience with TDD:

[...] At times I got sucked into that fundamentalist vortex, feeling bad about not following the true gospel. [...] It was yoyo cycle of pride, when I was able to adhere to the literal letter of the teachings, and a crash of despair, when I wasn't. It felt like falling off the wagon. Something to keep quiet about. Certainly not something to admit in public.

At first I didn't even get this. I myself was a pretty strong teacher of the "right way", which is test-first or strict TDD. After reading just the title of this post I was blinded already by a feeling of strong opposition towards David's opinions. When my initial revolt was gone, this changed. I got into a Twitter discussion (or "fight") with Manuel Lemos, foreman of the PHPClasses community, after he wrote the article Why TDD failed to become mainstream. I think the article itself is very dangerous for the PHP community and the community of PHPClasses in particular. However, I agree on one thing. He says (in the spirit of David):

Many people followed the preachers of TDD to later realize that it is not right for them. Still some were afraid to admit it and never contested openly.

I now understand both David and Manuel better. Preaching really is a dangerous thing. Yelling things like "What, you don't write tests?!" is even more dangerous. For newcomers to testing this can be very intimidating. They will think that they are doing something very wrong. They won't dare to admit that they don't write tests themselves, because: who would be such a sinner as to not write any test at all? This is actually one of the shortcomings of PHP's most famous proponent of unit testing, Chris Hartjes. As the "Grumpy Programmer" he depicts an angry alter-ego on Twitter. The grumpy programmer, or grumpy tester, will be angry at you for not testing your code.

In real life, Chris is a very friendly person (I never met him, but I know this from witness reports) and he has written some useful books about unit testing that sell well, which is great for world-wide test coverage. Still, there is a bit of a dangerous side on the approach of Chris: someone being angry at you for not testing is definitely not a good reason for you to start testing. In fact, you should not be taught to feel ashamed of your lack of testing experience. Instead you should be encouraged to talk about testing, to try it and be mentored by someone. I know Chris would agree with me, which is why I really like a recent tweet by Chris:

Why am I telling you all of this? Well, for several reasons. First, "TDD" does not equal "testing" (did you notice I already made the huge conceptual jump between these two words somewhere in the preceding paragraphs?). This means that TDD can be dead, but at the same time testing can be very much alive (which I think is the case). Anyway, TDD is a technique you need to master as a developer. And more generally, testing is something you definitely need to do as a developer, and you should learn all about it. But you can't learn about testing from other developers who make you feel ashamed for something you did not yet learn to do (or to do well). The solution? All experienced testers in the world need to become a mentor to someone who is a beginning tester! So here is my call to action:

If you know someone who doesn't know how to write tests, or doesn't do it will, teach them.

And if you are that someone:

Find someone else who knows how to write tests better than you and ask them to teach you how they do it.

Do you have any advise for testing newcomers?

We are all just highly accustomed to quickly slapping something together. We somehow prefer the kick of making big jumps without securing our ropes. If we are moderately good at programming, maybe no accidents happen. But only after you learn to write tests first you will recognize all the little accidents you have caused in the past by writing code while constantly refreshing a page in a browser, or adding var_dumps after each line of code. By following this workflow you introduced countless bugs, and invested lots and lots of hours figuring out why something didn't work. All this debugging was happening at the wrong kind of level (i.e. at the application level), when in fact simply writing some unit tests would have given you smaller amounts of code ("units") to debug, and you would have been able to fix the problem a lot sooner.

Besides writing unit tests, I highly recommend installing XDebug on your machine. This will immediately stop you from being slowed down by tracing down futile, meaningless mistakes. And then, as I just mentioned: find a mentor to help you become a better tester.

Is there a book, post, article about testing that opened your eyes?

Well, I've learned quite a lot from Growing Object-Oriented Software, Guided by Tests. Bridging the Communication Gap: Specification by Example and Agile Acceptance Testing was a very interesting read too. Some of the articles by Robert Martin contain lots of interesting insights about testing.

Do you write tests for your tests? :-P

No, however, depending on the complexity of your fake objects (which is a special kind of test double, it may be wise to add a simple test for them too. They sometimes contain some logic which by itself introduces bugs in test cases. In order to prevent the need for writing tests for tests, don't use any form of even moderately complex code (like for-, or if-statements) in your test cases. Programming logic is likely to introduce problems (even though I do believe you are quite an advanced programmer!), so a test case is no place for it. Instead, move your code out of the test cases and write some tests for it too. This will allow you to then use that code with confidence in your test cases.

Why do all tests evolve to spaghetti code?

There are two situations in which people tend to write less clean code than they usually do: inside test classes or when they first use a micro-framework. Although this happens for different reasons: people are probably more skilled in writing production code than in writing test code. Tests need structure and refactoring of that structure too, but in my experience you only dare to change the structure of code if you know exactly how something works, in other words: when you feel free to move things around a bit. In general I think people feel more free and at ease when writing production code than writing test code. So this would explain why test code often evolves to spaghetti code.

The remedy is to force yourself to clean up test code, just like you clean up production code, following the same little refactoring steps. It also helps to do some kind of peer review for test code. If the reviewer isn't able to quickly read your test code and understand what it does, it's time to refactor it. By the way, this reviewer can also be you, a couple of weeks later.

What do you think about writing tests after writing production code?

Well, this happens all the time and is quite okay. I usually do this myself when I've tested the units of code, but not the bigger system of which they are a part. Such tests however are more like integration tests, or functional tests. Writing unit tests after the fact doesn't really help in my opinion. You will likely feel slowed down because the code may not have been testable in the first place. If you have no unit tests for a particular unit of code, the best time to write unit tests for it is whenever you need to refactor the unit of code (or when you want to add features). Writing a unit test first will give you the confidence required for refactoring the production code. It also ensures that newly added features don't have any unforeseen influence on existing features.

Categories: PHP Testing

Tags: interview TDD PHPUnit Behat Prophecy PHPSpec unit testing

Comments: Comments

A better PHP testing experience Part II: Pick your test doubles wisely

Posted on by Matthias Noback

In the introduction to this series I mentioned that testing object interactions can be really hard. Most unit testing tutorials cover this subject by introducing the PHPUnit mocking sub-framework. The word "mock" in the context of PHPUnit is given the meaning of the general concept of a "test double". In reality, a mock is a very particular kind of test double. I can say after writing lots of unit tests for a couple of years now that my testing experience would have definitely been much better if I had known about the different kinds of test doubles that you can use in unit tests. Each type of test double has its own merits and it is vital to the quality of your test suite that you know when to use which one.

"Mocks" and fragility

If you have followed my advice from the previous article, your unit tests are not assertion-centric anymore. Even better: they describe what is going on and what the outcome of each test should be. The assertions themselves are hidden from sight. But your tests may still be quite unreadable because they contain long lists of set-up code for all the mocks that your Subject Under Test (SUT) requires, like this:

$object = ...;

$validationViolationList = $this->getMock('ValidationViolationList');
$validationViolationList
    ->expects($this->any())
    ->method('count')
    ->will($this->returnValue(1));

$validator = $this->getMock('Validator');
$validator
    ->expects($this->once())
    ->method('validate')
    ->with($object)
    ->will($this->returnValue($validationViolationList);

Code like this is pretty hard to read, although you will get used to it when you use PHPUnit for a long time. Besides readability issues, this code has some more problems:

  • It is not clear which types of test doubles are used.
  • It is not at once clear that the resulting validator "mock" never successfully validates an object, when called it will always return a violation list with one violation.

There are some more concerns or questions that need to be answered about this code:

  • Is it important that the validate() method is called exactly once? Shouldn't it be possible to call it twice and get the same result in both cases?
  • Is it even necessary to check the argument ($object) which is used when the validate() method is called)? And are you aware that by default PHPUnit checks only equality of the argument, not sameness (which in particular matters when the actual object and the expected object should have been exactly the same object).

Why do I have these kinds of concerns? It turns out that if you are not aware of these issues, mocks can badly influence the stability of your test suite. If you are too rigid about the number of times a method is called, the order in which methods are called or the arguments used when calling a method, your test will easily fail whenever you make structural changes to the SUT. When you refactor the code of the class you are testing, it's possible that the order of function calls changes or that maybe one function is called twice. This is why you want to build some flexibility into your test doubles (let's call them "test doubles" from now on, that will make things much more clear).

Different types of test doubles

I already mentioned that there are many different types of test doubles that you can use in your tests (there is a nice article about these types of test doubles by Robert Martin, called The Little Mocker. Let us briefly discuss each of them.

Dummy

When a particular type of object is required as an argument, but it is not used in any significant way, use a dummy. With PHPUnit:

// Logger is an interface
$logger = $this->getMock('Logger');

Whenever a method is called on the $logger dummy, it will return null. Be careful though: this only works when getMock() is called with the name of an interface. When a class name is provided, you also need to specify all the methods that PHPUnit should override to return null, otherwise PHPUnit will just call the existing methods of that class:

// Logger is a class
$logger = $this->getMockBuilder('Logger')
    ->setMethods(array('debug', 'info', ...))
    ->getMock();

Stub

When a test double is supposed to return some fixed values, you need a stub. The characteristics of a stub are:

  • It does not matter which arguments are provided when one of its methods is called.
  • It does not matter how many times a method is called.

With PHPUnit a stub looks like this:

$logger = $this->getMock('Logger');
$logger
    ->expects($this->any())
    ->method('getLevel')
    ->will($this->returnValue(Logger::INFO));

Both of the characteristics of a stub can be recognized in this particular code sample: no list of required arguments has been provided (using ->with(...)), so the arguments that are used will not be validated. Also $this->any() is used to indicate that there is no limit to the number of times this method is called and it will always return the same value: Logger::INFO.

Fake

A fake is a particular type of test double that looks a lot like a dummy, because:

  • It does not matter how many times a method is called.
  • It does not return null, like a dummy does.

But a fake has some simple logic by which it can decide which value it should return. An examples of a fake with PHPUnit:

$urlsByRoute = array(
    'index' => '/',
    'about_me' => '/about-me'
);

$urlGenerator = $this->getMock('UrlGenerator');
$urlGenerator
    ->expects($this->any())
    ->method('generate')
    ->will($this->returnCallback(
        function ($routeName) use ($urlsByRoute) {
            if (isset($urlsByRoute[$routeName])) {
                return $urlsByRoute[$routeName];
            }

            throw new UnknownRouteException();
        }
    ));

There are also simpler fakes, which just return a particular argument:

$urlGenerator = $this->getMock('UrlGenerator');
$urlGenerator
    ->expects($this->any())
    ->method('match')
    ->will($this->returnArgument(0));

A word of advice: you should limit the complexity of your fake's code. Pretty soon you will need unit tests for them too! In my experience, whenever I write code for a fake, I almost always accidentally introduce some kind of obscure bug that takes quite some debugging time to uncover.

Spy

If you want to keep track of method calls being made to a test double, to later examine them and make sure that the methods were called the expected number of times, in a particular order or with the correct arguments, you should use a spy. With PHPUnit this looks like:

$collectedMessages = array();

$logger = $this->getMock('Logger');
$logger
    ->expects($this->any())
    ->method('debug')
    ->will($this->returnCallback(
        function ($message) use (&$collectedMessages) {
            $collectedMessages[] = $message;
        }
    );

// inspect the collected messages

As you can see, the closure merely collects the $message argument whenever the debug() method is called. The closure returns nothing, so the return value of calling debug() will be null. Of course you could combine spy and stub in one test double and return some fixed value here (or anything else really, see also the next section).

Free spies!

There's one nice trick I'd like to share at this point. You don't really need to write spies in this particular way. You can just as well use an undocumented feature of PHPUnit here (read more about it in this article: Spying With PHPUnit).

$logger = $this->getMock('Logger');
$logger
    ->expects($spy = $this->any())
    ->method('debug');

$invocations = $spy->getInvocations();

$this->assertSame(1, count($invocations));

$firstInvocation = $invocations[0];
$this->assertEquals('An error occurred', $firstInvocation->parameters[0]);

Pretty cool, though until now I never felt the need to use a spy like this. The need for a spy can also be a test smell, telling you that the communication between your objects is just too complicated to test with simpler types of test doubles.

Mock

A mock is a type of test double that is actually very much like a spy:

  • It keeps track of method calls and their arguments.

But a mock also:

  • Validates method calls given a certain set of expectations.
$logger = $this->getMock('Logger');
$logger
    ->expects($this->at(0))
    ->method('debug')
    ->with('A debug message');
$logger
    ->expects($this->at(1))
    ->method('info')
    ->with('An info message');

True mocks can be recognized by the usage of $this->at(...), or $this->once(), $this->exactly(...), $this->atLeastOnce(), $this->never(). They often have $this->with(...) which will cause the arguments to be validated.

Mocks don't need to return a value, though they can be configured to do so of course, by turning them into a stub or a fake.

Don't overuse the "mocking" tool

Now that you know all about test doubles and how you can define them using PHPUnit's mocking tool, the advice I'd like to give is to not use it whenever you can. The biggest reason is that mocks created by tools like PHPUnit are very strange things. Test doubles like these:

  • Are created using a combination of code generation and eval() and thus only exist in memory.
  • Are therefore difficult to debug using a tool like XDebug (which can not "step through" them).

Besides, it takes some time to figure out all the special things you need to consider when you define test doubles:

  • with() compares arguments by equality, not sameness.
  • setMethods() requires an array. You will not get a clear error if you don't give it one.
  • getMock() with a class name as the first argument requires you to explicitly specify the methods you want to override.
  • at(...) starts with 0.
  • ...

All of this knowledge is usually only acquired by hard work and lots of debugging.

Manually create your test double classes

It is often much easier to just write a simple class that does exactly what you expect from it. Such a class is not hidden from sight, and it is even testable. These are some of the test doubles from the code samples above, rewritten using plain old PHP classes:

class LoggerDummy implements Logger
{
    public function debug()
    {
        return null; // not even necessary here
    }

    ...
}

class InfoLevelLoggerStub implements Logger
{
    ...

    public function getLevel()
    {
        return self::INFO;
    }
}

class PredefinedUrlsUrlGeneratorFake implements UrlGenerator
{
    private $urlsByRoute;

    public function __construct(array $urlsByRoute)
    {
        $this->urlsByRoute = $urlsByRoute;
    }

    public function generate($routeName)
    {
        if (isset($this->urlsByRoute[$routeName])) {
            return $this->urlsByRoute[$routeName];
        }

        throw new UnknownRouteException();
    }
}

I recommend to choose class names which describe the behavior of the particular test double class.

When it comes to spies and mocks, things will become a bit harder to implement manually. Still, if you need to track just some method calls or arguments, you could implement a spy by yourself and perform validation inside the test method.

The advantages of manually creating test doubles are:

  • You can use XDebug to step through their code because the code of the test doubles can be found in regular PHP files.
  • You make your test code much more readable: instead of lots of set-up code for your test doubles, you just instantiate a class with a descriptive name.
  • If your fakes have complicated code (which is still simpler than the code of the original class of course!), you can even write unit tests for them.

If you don't use test double classes: at least use factory methods

As I mentioned already, not every type of test double is best defined in a separate class. For some types of test doubles, like dummies (in particular dummies of interfaces), mocks and spies, you are still better off with a test double library like PHPUnit's mocking sub-framework. In this case your test classes will still contain mock set-up code that is difficult to read, understand and maintain. The solution is easy: hide the the highly technical set-up code from sight and put it in a private factory method:

public function testSomething()
{
    $logger = $this->loggerDummy();
}

private function loggerDummy()
{
    return $this->getMockBuilder('Logger')
        ->setMethods(array('debug', 'info', ...))
        ->getMock();
}

Conclusion

In this article we looked at test doubles and how they come in different shapes. Whenever you use PHPUnit's $this->getMock() ask yourself what type of test double you really need. Depending on your choice, consider replacing the dynamic in-memory test double with a concrete class manually created for this particular situation. If you don't choose that path, at least hide the construction details of the mocks from sight and use descriptive names.

Another article will be needed to clarify the use of test doubles (or other stand-ins) when testing classes that depend on other, third-party classes.

Prophecy

This article is based on PHPUnit's mocking sub-framework. There are several other libraries out there which can be used to create test doubles on-the-fly. After looking around a bit, the most interesting one is Prophecy. Its README file contains a nice explanation of test double types and how they can be created using the library. The library itself makes a lot of sense. It's also easy to integrate Prophecy with PHPUnit: there is another library which contains a base test class from which you can extend your own test classes.

Categories: PHP Testing

Tags: PHPUnit unit testing

Comments: Comments