PHP & Symfony About PHP and Symfony2 development

Packages: the case for clones

Posted on by Matthias Noback

Don't reinvent the wheel

There is this ongoing discussion in the PHP community (and I guess in every software-related community) about reinventing wheels. A refreshing angle in this debate came from an article by Phil Sturgeon pointing to the high number of "duplicate" packages available on Packagist. I agree with Phil:

Sometimes these are carbon copies of other packages, but often they are feature-weak versions of established packages.

It doesn't make sense to do the same thing over and over again. At least I personally don't try to make this mistake. If I want to write code that "already exists", at least I don't publish it on Packagist.

However, recently I got myself into the business of "recreating stuff" myself. I released a set of packages related to commands, events, command buses and event buses called SimpleBus. Of course, I'm not the first one to write this kind of code. But here's the thing (and I'm quoting Phil again):

The golden rule is: If they are different, then awesome!

So in my defense, just a quick list of things that I want to offer with SimpleBus, which are different from existing packages. I wanted to offer:

  • Small packages with very simple features that can be used separately from each other.
  • Bridge packages to unlock some powerful features when combining some of the packages in one application.
  • Separate packages for framework integration (in this case Symfony).
  • Separate packages for integration with persistence libraries (in this case Doctrine).

Existing solutions combined way too many features in one package, which is to be considered the cause of people like me "reinventing the wheel". I don't want to have all that code in my project. And I don't want to keep upgrading a package when none of its changes are relevant to me.

When I created the SimpleBus packages, my hope was that maybe these would become the reusable components for other people's efforts to create full-fledged CQRS/event sourcing applications. My packages are very abstract, simple and are certainly not "fancy" in any way. Still they offer the amount of flexibility needed to implement any thing you need with it

A lack of good package design

So I admit, I've been rewriting some existing code (still there are subtle differences which I'll in another post). But why did I do it? Because existing solutions lack good package design. Now, what is good package design? Let me quickly explain about the Package Design Principles (they are much less known than the SOLID principles of class design):

  1. The Release/reuse equivalence principle - You can only reuse code that you release. Release only things that you can reuse.
  2. The Common reuse principle - Release classes together which are reused together. If you use one class in a package, you use (almost) all of the other classes too. In other words: if some classes in a package can be used without using all the other classes, they deserve to be in another package (this is the Interface segragation principle for packages).
  3. The Common closure principle - Packages should have one reason to change. If a package has multiple reasons to change, then split the package (this is the Single responsibility principle for packages).

These first three principles are about package cohesion. They tell you what should be in a package and when it's time to split a package. The second set of principles are about package coupling:

  1. The Acyclic dependencies principle - The dependency graph of packages in a project should have no cycles.
  2. The Stable dependencies principle - Packages should only depend on packages which are more stable than itself. Stable packages are less likely to change. A stable package has no dependencies, but is only being depended upon, i.e. it's a responsible package. Instable packages are dependent packages, and no other package depends upon them, i.e. they are very irresponsible.
  3. The Stable abstractions principle - The more stable a package is, the more abstract things it contains (abstract classes and interfaces). The more instable a package is, the more concrete things it contains (classes).

If all package maintainers would follow these package design principles when creating and releasing packages, there would be much more quality in the world of (PHP) packages. In most cases when I did reinvent the wheel, existing packages that provided more or less the same solution had issues with at least two of the abovementioned design principles.

For example, many packages on Packagist provide both library and framework integration code. This makes them violate the Common reuse principle: if I want to reuse it in an application with a different framework, I'm not using half of the classes in the package. These packages also violate the Stable dependencies principle by depending on an entire framework to actually work. A framework is by definition a very instable package because it is such a dependent package itself and therefore likely to change.

Another example: many libraries on Packagist provide some abstract code and a set of concrete classes for multiple persistence libraries (Doctrine ORM, MongoDB, Propel, etc.). These packages violate the Stable dependencies principle because they depend on so many other packages, which are themselves quite unstable. But they also violate the Stable abstractions principle by containing both concrete and abstract things. And, just like the framework-specific packages they violate the Common reuse principle because if I use Doctrine ORM, then I don't use all the MongoDB and Propel-related classes.

Package consumer intuition

I think that even though the package design principles are not well known yet, many of you already knew what I was talking about. And I think the lack of good package design is one of the reasons why people create their own packages instead of using existing ones. There's always something "wrong" about other people's packages and unfortunately, not even part of such a package can be reused, because everything is so tied together.

If at least everybody would only publish small packages, implementing one considerably small feature, or just providing some very generic classes and interfaces, the situation with regard to people reinventing the wheel would be much better.

A connection with PHP-FIG efforts

There is an interesting connection between the world-wide struggle for truly reusable PHP packages and the same kind of struggle for establishing a collection of absolutely generic, always reusable interfaces which several PSR proposals are trying to accomplish. I think one of the main goals here is interoperability - when you use the PSR interfaces, you should be able to switch between logger, cache or event dispatcher libraries, without thinking twice and without the need to adapt any of your existing code.

Establishing these interfaces has however proven to be a long and difficult process. The reason for this has been explained very well by Anthony Ferrara in his An Open Letter To PHP-FIG:

They are trying to create a reasonably generic solution (99% solution). So they need to handle a HUGE range of needs and requirements.

It's very hard to establish the definitive shape of any kind of interface since every project has its own specific characteristics. Just as it is very difficult to find an all-encompassing solution for command and event buses, because you either have to provide all possible alternatives, or be very opinionated. If you choose the first strategy, then your package will violate the Common reuse principle again because nobody needs all of the stuff your package provides and if you choose the second strategy, your package will not be reused often, since people are likely not to agree with opinionated stuff. So the third strategy, which is I'd say the winning strategy, is to offer only very unopinionated code for reuse. Very generic, very simple code.

I'd like to quote Anthony again on a very general recommendation which he directed to PHP-FIG, while I would in fact direct it to anyone who creates reusable PHP packages:

Please stop trying to solve generic problems. Solve the 50% problem, not the 99% problem.

If everybody does this, I'm sure we should be able to do better at preventing duplicate packages and duplicate coding efforts. Until then, I'm not surprised by the amount of package clones out there.

P.S. Also, creating all this stuff ourselves can be fun too, right? ;)

Categories: PHP

Tags: package design reuse dependencies

Comments: Comments

Decoupling from a service locator

Posted on by Matthias Noback

Decoupling from a service locator - shouldn't that be: don't use a service locator?

Well, not really, since there are lots of valid use cases for using a service locator. The main use case is for making things lazy-loading (yes, you can also use some kind of proxy mechanism for that, but let's assume you need something simpler). Say we have this EventDispatcher class:

class EventDispatcher
{
    public function __construct(array $listeners)
    {
        $this->listeners = $listeners;
    }

    public function dispatch($event)
    {
        foreach ($this->listeners[$event] as $listener) {
            $listener->notify();
        }
    }
}

Now it appears that some event listeners are rather expensive to instantiate. And, even though it may never be notified (because it listens to a rare event), in order to register any event listener, we need to instantiate it:

$dispatcher = new EventDispatcher(
    array(
        'some_rare_event'  => new ExpensiveEventListener()
    )
);

Most applications already use a service container (e.g. the Symfony DependencyInjection Container) which only instantiates services when they are requested for the first time. So let's just use the service container to fetch (and instantiate) listeners only when they are actually needed:

use Symfony\Component\DependencyInjection\ContainerInterface;

class ContainerAwareEventDispatcher
{
    public function __construct(
        array $listenerServiceIds,
        ContainerInterface $container
    ) {
        $this->listenerServiceIds = $listenerServiceIds;
        $this->container = $container;
    }

    public function dispatch($event)
    {
        foreach ($this->listenerServiceIds[$event] as $listenerServiceId) {
            // fetch the actual listener from the service container:
            $listener = $this->container->get($listenerServiceId);

            $listener->notify();
        }
    }
}

I consider this a valid approach. However, now my stand-alone library code has a dependency on the Symfony service container. Which is not good since not all people will have it in their projects. They may already use something else for dependency injection. So, let's decouple this!

Of course, we could roll our own service container interface (or service locator actually) for this, e.g.

interface ServiceLocator
{
    public function get($id);
}

class ContainerAwareEventDispatcher
{
    public function __construct(array $listenerServiceIds, ServiceLocator $container)
    {
        ...
    }
    ...
}

In that case we have stand-alone library code again, because we applied the Dependency inversion principle. The initial dependency direction was from ContainerAwareEventDispatcher to the external ContainerInterface. After introducing our own ServiceLocator interface, there is no outward dependency arrow anymore.

However, introducing the extra interface requires users to implement their own service locator classes. In most situations which require some kind of adapter class this makes perfect sense (see a previous post of mine on decoupling your event system). But in this case, the ServiceLocator interface has only one method with one argument. This, I think, allows us to pick an entirely different solution.

Using anonymous callables for dependency inversion

Instead of using adapter classes, we could use adapter closures (or even more general: callables). We rewrite the EventDispatcher to call the "callable" service locator which is called with just one argument; the service id:

class EventDispatcher
{
    public function __construct(array $listenerServiceIds, callable $serviceLocator)
    {
        $this->listenerServiceIds = $listenerServiceIds;
        $this->serviceLocator = $serviceLocator;
    }

    public function dispatch($event)
    {
        foreach ($this->listenerServiceIds[$event] as $listenerServiceId) {
            $listener = call_user_func($this->serviceLocator, $listenerServiceId);
            $listener->notify();
        }
    }
}

Unfortunately we have lost the explicit contract of "being a service locator" here, but instead, we gained a lot of flexibility. Instead of requiring a service locator of a particular type, users can now decide for themselves how they are going to load the event listeners. For example, using the simple PHP dependency injection container Pimple:

$container = new \Pimple();

$container['expensive_event_listener'] = function () {
    return new ExpensiveEventListener();
};

$container['event_dispatcher'] = function ($container) {
    return new EventDispatcher(
        array(
            'some_rare_event' => array(
                // only provide service ids, no instances:
                'expensive_event_listener'
            )
        ),
        // this is the service locator callable:
        function ($serviceId) {
            return $container[$serviceId];
        }
    );
};

For Symfony we still have to create a simple class, because the service container is not defined at runtime, but compiled to a PHP file. This is what it might look like:

class SymfonyServiceLocator
{
    public function __construct(ContainerInterface $container)
    {
        $this->container = $container;
    }

    public function __invoke($serviceId)
    {
        return $this->container->get($serviceId);
    }
}

Using the magic __invoke() method here will turn the entire object into a callable.

And the service definitions required to make this work:

services:
    event_dispatcher:
        class: EventDispatcher
        arguments:
            # provide an array of service ids by event name
            - []
            - @symfony_service_locator

    symfony_service_locator:
        class: SymfonyServiceLocator
        arguments:
            - @service_container
        public: false

By the way, I used this technique in the SimpleBus project (which provides basic implementations for using a command and/or an event bus in your application). There is an event handler esolver which uses a service locator callable to instantiate only relevant event handlers. And of course there is a service locator for Symfony.

Using a synthetic service

There is another approach which allows us to use a closure a service locator in Symfony applications. I'm not sure if it's preferable over the other approach, but you can decide.

First, we make the symfony_service_locator a "synthetic" service, meaning we can set it at runtime:

services:
    ...

    symfony_service_locator:
        synthetic: true

Now in one of our bundle's boot() method (or in the boot() method of the AppKernel method) we can just set the service locator:

class MyBundle extends Bundle
{
    public function boot()
    {
        $this->container->set(
            'symfony_service_locator',
            function ($serviceId) {
                // for PHP < 5.4 you'd have to explicitly "use" the container
                return $this->container->get($serviceId);
            }
        );
    }
}

Conclusion

I think it's the first time I implemented dependency inversion using anonymous callables like this. I like it so far, but I also think you can only do this when it is a surrogate for a very simple interface (one method, one argument). This condition drastically reduces the potential number of use cases, but I think that's a good thing. In most cases it will make much more sense to just introduce another interface, like you're used to.

One last remark

If you want your users to choose between a lazy-loading an eager-loading strategy, using callables for dependency inversion like this comes with a big advantage: you don't have to create an EagerLoadingEventDispatcher. The user only needs to provide something like an "identity service locator":

$dispatcher = new EventDispatcher(
    array(
        'some_rare_event' => array(
            // provide the actual event listener instances:
            new ExpensiveEventListener()
        )
    ),
    function ($listener) {
        // $listener is already an object so just return it!
        return $listener;
    }
);

Categories: PHP Symfony2

Tags: decoupling dependency inversion service container

Comments: Comments

Symfony in Barcelona

Posted on by Matthias Noback

Pizza

I just ate a nice pizza at my hotel room in Barcelona. The funny thing is (at least, to a Dutch guy that is): they wouldn't be able to give me a pizza before 20:00h. At that time in my home country we have long forgotten our desserts, cleaned the dishes and are starting to think about sleeping (just kidding). Anyway, I got the pizza, it was good and now I'm here to write a little report on the things that happened during the last three days.

It all started a couple of months ago when rising Symfony community star Marc Morera (who, by the way, is going to do the Symfony Walk in a couple of weeks) asked me to come to Barcelona and do some consultancy for the company that is working hard on a new Symfony e-commerce product called Elcodi. They were in the process of making some decisions on package architecture and bundle design issues and they asked me to help them with it. Of course I said "yes"! And here we are.

Now you need to know that Elcodi is part of a commercial enterprise. However, the Elcodi team creates open source e-commerce software. A very interesting, and also challenging thing to do. While this premise is already quite fascinating, Elcodi takes some extra steps to really make a difference in the world of companies that open-source their software. Elcodi values its contributors very much and has actually made a great effort to organise a community day for several intimate contributors and developer-friends.

Elcodi Community Day

As you might have guessed by now: I was part of the community day too and was asked to do a workshop for the attendees. The funny thing is, I only prepared a talk on package design and I had some material from a previous High Quality Bundles workshop.

To me personally it was quite hard to actually not have a schedule for the day, but in the end, everything worked out very well and we just followed the natural flow of the conversations, the questions that people had in mind and the subjects upon which the Elcodi team and the other attendees were trying to decide.

Grizzly Bar Barcelona

To finish this story: the night before the Elcodi Community Day I was also asked to do my "The Naked Bundle" talk for the Symfony Barcelona Usergroup. It was just very surprising to see that the event was about to take place right in something called the Grizzly Bar (see the photo on the right).

It turned out to be a very cosy place and lots of people showed up. At this night, I was talking about decoupling your code from the framework code and letting go of its conventions. So you can imagine that it was a big surprise that the organisers had also invited Javier Eguiluz to do his talk on the Symfony Best Practices during the same night. At the same time, this was the best thing the organisers could have done!

It was very nice to finally meet Javier in person. He is a very important Symfony community member and also co-author of the new Symfony Best Practices book. Talking about the best practices and watching Javier's presentation finally made some important things very clear to me:

  1. As I had already previously concluded: the official Symfony Best Practices are to be considered an opinion, just as mine and yours. However, they are intended to be constructive and helpful for developers who start using Symfony. They shouldn't be bothered at first to make all those difficult decisions, driven by discussions about code design.
  2. Looking at the Symfony Best Practices from this perspective really makes them quite good.

Of course, there are some things in the document that I wouldn't recommend, but nevertheless: in a few weeks I'm going to train a group of developers who are starting to work with Symfony and I'm definitely going to resort to the Best Practices to teach them the quick way to get started. Of course, It wouldn't be me if I would not also give them a sneak peek of the road that lies ahead and the numerous options you have to write better code (or at least, "better imho").

Conclusion

So in conclusion I can tell you, this was a very interesting experience. To me this was not only about doing talks and giving advice, it was also about learning about other perspectives, and understanding other people's reasons to do things that I would have personally deemed quite strange.

One last note, if you ever get the chance to meet one of the wonderful people from the Elcodi team, be it Marc Morera, Aldo Chiecchia or Berny Cantos, be sure to talk with them, and probably go out for breakfast, lunch or dinner with them - I promise that you will have a great time.

Categories: PHP Symfony2

Tags: Elcodi Community

Comments: Comments