PHP & Symfony About PHP and Symfony2 development

Backwards compatible bundle releases

Posted on by Matthias Noback

The problem

With a new bundle release you may want to rename services or parameters, make a service private, change some constructor arguments, change the structure of the bundle configuration, etc. Some of these changes may acually be backwards incompatible changes for the users of that bundle. Luckily, the Symfony DependenyInjection component and Config component both provide you with some options to prevent such backwards compatibility (BC) breaks. If you want to know more about backwards compatibility and bundle versioning, please read my previous article on this subject.

This article gives you an overview of the things you can do to prevent BC breaks between releases of your bundles.

Renaming things

The bundle itself

You can't, without introducing a BC break.

// don't change the name
class MyOldBundle extends Bundle
{
}

The container extension alias

You can't, without introducing a BC break.

use Symfony\Component\HttpKernel\DependencyInjection\Extension;

class MyBundleExtension extends Extension
{
    ...

    public function getAlias()
    {
        // don't do this
        return 'new_name';
    }
}

Parameters

If you want to change the name of a parameter, just copy the parameter instead of actually renaming it:

parameters:
    old_parameter: same_value
    new_parameter: same_value
class MyBundleExtension extends Extension
{
    public function load(array $config, ContainerBuilder $container)
    {
        $container->setParameter('old_parameter', 'same_value');
        $container->setParameter('new_parameter', 'same_value');
    }
}

Config keys

The container extension alias can not be changed without causing a BC break, but it is possible to rename config keys. Just make sure you fix the structure of any existing user configuration values before processing the configuration:

class Configuration implements ConfigurationInterface
{
    public function getConfigTreeBuilder()
    {
        $treeBuilder = new TreeBuilder();
        $rootNode = $treeBuilder->root('my');

        $rootNode
            ->beforeNormalization()
                ->always(function(array $v) {
                    if (isset($v['old_name'])) {
                        // move existing values to the right key
                        $v['new_name'] = $v['old_name'];

                        // remove invalid key
                        unset($v['old_name']);
                    }

                    return $v;
                })
            ->end()
            ->children()
                ->scalarNode('new_name')->end()
            ->end()
        ;
    }
}

Service definitions

If you want to rename a service definition, add an alias with the name of the old service:

services:
    new_service_name:
        ...

    old_service_name:
        alias: new_service_name

This may cause a problem in user code, because during the container build phase they may call $container->getDefinition('old_service_name') which will cause an error if old_service_name is not an actual service definition, but an alias. This is why user code should always use $container->findDefintion($id), which resolves aliases to their actual service definitions.

Method names for setter injection

services:
    subject_service:
        calls:
            # we want to change the name of the method: addListener
            - [addListener, [@listener1]]
            - [addListener, [@listener2]]

This one is actually in the grey area between the bundle and the library. You can only change the method names used for setter injection if users aren't supposed to call those methods themselves. This should actually never be the case. You should just offer an extension point that takes away the need for users to call those methods. The best option is to create a compiler pass for that:

namespace MyBundle\DependencyInjection\Compiler;

use Symfony\Component\DependencyInjection\ContainerBuilder;

class InjectListenersPass implements CompilerPassInterface
{
    public function process(ContainerBuilder $container)
    {
        $subjectDefinition = $container->findDefintion('subject');

        foreach ($container->findTaggedServiceIds('listener') as $serviceId => $tags) {
            $subjectDefinition->addMethodCall(
                'addListener',
                array(new Reference($serviceId))
            );
        }
    }
}

You need to add this compiler pass to the list of container compiler passes in the bundle class:

class MyBundle extends Bundle
{
    public function build(ContainerBuilder $container)
    {
        $container->addCompilerPass(new InjectListenersPass());
    }
}

From now on, users can register their own listener services like this:

services:
    user_listener_service:
        tags:
            - { name: listener }

The compiler pass will then call the addListener() method for each of the registered listeners.

From now on you can change the method name used for this type of setter injection at any time, because the user doesn't call it explicitly anymore.

Changing visibility

Parameters

This doesn't apply to parameters, they are all public.

Service definitions

Service definitions can be public or private:

services:
    a_public_service:
        # a service is public by default
        public: true

    a_private_service:
        public: false

When a definition is marked "public" you can't fetch it directly from the container by calling $container->get($id). Private services can only be used as arguments of other services.

If a service definition was previously public, you can't change it to be private, because some users may rely on it to be available by calling $container->get($id). The only option here is to rename the service, make it private, and then add an alias for it with the old name. This will effectively make the service public again.

services:
    a_public_service:
        # an alias is always public
        alias: a_private_service

    a_private_service:
        public: false

If a service definition was previously private, you can change it to public at any time. There are no ways in which a user could have used a private service that won't work when the service becomes public.

Changing values

Service definitions

I think we should agree on arguments of service definitions to be private property of the bundle. Users should not rely on them to stay the same between two releases.

Config values

If you want to change allowed values for bundle configuration, you should fix existing config values, by using the beforeNormalization() method (we discussed this previously).

If you want to remove existing config keys, you will have to unset them using beforeNormalization() too, or they will trigger errors about unknown keys.

If you want to add new required configuration values, you should provide sensible defaults for them, to accomodate existing users:

$rootNode
    ->children()
        ->scalarNode('required_key')
            ->isRequired()
            ->defaultValue('sensible_default')
        end()
    ->end()
;

If the parent key is a new key and is allowed to be missing, you should add ->addDefaultsIfNotSet() to the parent array node:

$rootNode
    ->children()
        ->arrayNode('optional_key')
            ->addDefaultsIfNotSet()
            ->children()
                ->scalarNode('required_key')
                    ->isRequired()
                    ->defaultValue('sensible_default')
                end()
            ->end()
        ->end()
    ->end()
;

New optional config keys should be no problem.

Conclusion

Of course, if the change you want to introduce is not compatible with one of the proposed BC-friendly options, you should release a new major version of your bundle. No problem with that, it just needs to be a conscious decision and you might want to try a bit harder before you decide to do that.

I think this article contains a fairly exhaustive list of ways to prevent BC breaks between bundle releases. If you have a suggestion for this list, just let me know.

Categories: PHP Symfony2

Tags: bundle dependency injection service container package design

Comments: Comments

Semantic versioning for bundles

Posted on by Matthias Noback

A short introduction to semantic versioning

Semantic versioning is an agreement between the user of a package and its maintainer. The maintainer should be able to fix bugs, add new features or completely change the API of the software they provide. At the same time, the user of the package should not be forced to make changes to their own project whenever a package maintainer decides to release a new version.

The most extreme solution to this problem is: the package should never change, the user should never need to upgrade. But this is totally useless, since the user wants new features too, they just shouldn't jeopardize their existing, functioning code. At the same time the package maintainer wants to release new features too, and they may even want to redo things completely every once in a while.

Semantic versioning assumes a package to have a version number that consists of three numbers, separated by dots, like 2.5.1. The first number is called the "major version", the second number is called the "minor version", the last number is called the "patch version". The semantic versioning agreement in short tells the package maintainer to increment the:

MAJOR version when you make incompatible API changes,

MINOR version when you add functionality in a backwards-compatible manner, and

PATCH version when you make backwards-compatible bug fixes.

You can read the full explanation of the concept and what you are agreeing upon if you say that your package "follows semantic versioning" on semver.org.

Symfony and semver

As of version 2.3 the Symfony framework officially uses semver. They also apply some extra rules for parts of the code (classes, interfaces, methods) which they label as being part of the official Symfony API by adding an @api annotation to the respective doc comments. Semantic versioning and public API labeling together this consistutes Symfony's backwards compatibility promise.

In short, Semantic Versioning means that only major releases (such as 2.0, 3.0 etc.) are allowed to break backwards compatibility. Minor releases (such as 2.5, 2.6 etc.) may introduce new features, but must do so without breaking the existing API of that release branch (2.x in the previous example).

Bundles and semver

I was asked by Paul Rogers from the Symfony Live London crew:

How should you version a bundle? Should it be related to the library version, like ElasticBundle does?

As a matter of fact, I had already spent some toughts on this issue. The answer to the first question is: apply semver, just like any package should do. And the answer to the second is: no. It should not per se be related to the library version for which the bundle provides a framework-specific integration layer. I think this second answer requires some more detailed reasoning from my side.

Bundles expose an API themselves

The code in a library package exposes an API. People are going to use that API in a particular way. They are going to instantiate some of the classes from the package with a particular set of constructor arguments. They are going to call some of its methods with a particular set of arguments. This is the reason why semantic versioning should be applied to such packages in the first place: the maintainer should not be allowed to change any of the things the user relies on, like class names, method names, required parameters, etc.

A bundle mostly doesn't expose an API consisting of classes and methods. The API of a bundle consists of *services, or service definitions, and parameters. These are different types of entities. Yet they share some characteristics: service definitions often provide constructor arguments in a particular order, they sometimes contain method calls used for setter injection, they are public or private, abstract or concrete, have a certain name, etc.

Some changes, like making a private service public won't make a difference for an existing user.

services:
    formerly_a_private_service:
        public: true

If a service was formerly a private service, the user could not have relied on it in any way that a public service doesn't support. So that kind of a change should not to be considered a backward compatibility (BC) break.

The other way around - making a public service private - on the contrary should be considered a BC break.

services:
    formerly_a_public_service:
        public: false

Some users may rely on it being public. For example they may call

$container->get('formerly_a_public_service');

somewhere and all of a sudden they would get an exception for that.

The API of a bundle leads a life on its own

The fact that bundles have their own API, which changes in a way that is not related to the changes in the underlying library per se, means that bundles should have their own versioning. When the bundle maintainer introduces a BC break in the bundle's service definitions, they should increment its major version. If the bundle maintainer keeps BC by adding a service alias, or wrapping a service in a way that is transparent to the user, they are allowed to increment just the minor version. And if the maintainer fixes a bug, they can just increment the patch version.

A bundle may reach maturity at a later time than the library

Another reason why bundle versions are not necessarily the same as the version of the library it serves to integrate, is that a bundle may be created much later than the library. In such a situation, the library may be at 2.5.1, while the bundle is only available as a pre-stable development release, e.g. at version 0.3.4. When the bundle is finally stable, it wouldn't make sense to skip some versions and release it as 2.5.1 too.

A library may contain bugs that are totally unrelated to the bundle

Whenever a bug is fixed in the library, its patch version will be incremented. When the library is at version 2.5.1 and a bug gets fixed, the new version will be 2.5.2. Now if there is a bundle at 2.5.1, it should increment to 2.5.2 just to keep up, even though the bundle didn't change in any respect. That doesn't make sense. And even though this may seem a bit weak as an argument, this is what my reasoning really boils down to:

a change in the library doesn't necessarily require a change in the bundle, so it also doesn't make sense to make the bundle strictly follow the library version.

A library may contain features that are not implemented by the bundle

One last argument for separate bundle versioning is that a bundle might not provide integration for all the features that a library offers. Then if the library stays the same, while the bundle starts to expose existing library features, the bundle should increment its minor version, while the library keeps the same version number. This is obviously an unwanted situation.

Conclusion

I think my point is clear and I hope this article can serve as a definite answer to the question: "should a bundle (module, plugin, etc.) follow the versions of the corresponding library?" No, it shouldn't.

Categories: PHP Symfony2

Tags: bundle semver package design

Comments: Comments

Exposing resources: from Symfony bundles to packages

Posted on by Matthias Noback

Syfony bundles: providing services and exposing resources

When you look at the source code of the Symfony framework, it becomes clear that bundles play two distinct and very different roles: in the first place a bundle is a service container extension: it offers ways to add, modify or remove service definitions and parameters, optionally by means of bundle configuration. This role is represented by the following methods of BundleInterface:

namespace Symfony\Component\HttpKernel\Bundle;

interface BundleInterface extends ContainerAwareInterface
{
    /** Boots the Bundle. */
    public function boot();

    /** Shutdowns the Bundle. */
    public function shutdown();

    /** Builds the bundle. */
    public function build(ContainerBuilder $container);

    /** Returns the container extension that should be implicitly loaded. */
    public function getContainerExtension();

    ...
}

The second role of a bundle is that of a resource provider. When a bundle is registered in the application kernel, it automatically starts to expose all kinds of resources to the application. Think of routing files, controllers, entities, templates, translation files, etc.

The "resource-providing" role of bundles is represented by the following methods of BundleInterface:

interface BundleInterface extends ContainerAwareInterface
{
    ...

    /** Returns the bundle name that this bundle overrides. */
    public function getParent();

    /** Returns the bundle name (the class short name). */
    public function getName();

    /** Gets the Bundle namespace. */
    public function getNamespace();

    /** Gets the Bundle directory path. */
    public function getPath();
}

As far as I know, only getName() serves both purposes, since it is also used to calculate the configuration key that is used for the bundle's configuration (e.g. the configuration for the DoctrineBundle is to be found under the doctrine key in config.yml).

There are several framework classes that use the bundle name and its root directory (which is returned by the bundle's getPath() method) to locate resources in a bundle. For instance the ControllerResolver from the FrameworkBundle allows you to use the Bundle:Controller:action notation to point to methods of controller classes in the Controller directory of your bundle. And the TemplateNameParser from the FrameworkBundle resolves shorthand notation of templates (e.g. Bundle:Controller:action.html.twig) to their actual locations.

It's actually quite a clever idea to use the location of the bundle class as the root directory for the resources which a bundle exposes. By doing so, it doesn't matter anymore whether a bundle is part of a package that is installed in the vendor directory using Composer, or if it's part of your project's source code in the src directory; the actual location of resources is always derived based on the location of the bundle class itself.

Towards a better situation for resources

Let me first say, I think the BundleInterface ought to be separated in two different "role interfaces", based on the different roles they play. I was thinking of ProvidesServices and a ExposesResources interface. That would clearly communicate the two different things that a bundle can do for you.

Puli: uniform resource location

Much more important than splitting the BundleInterface is to have a better way of exposing resources located inside a library (or bundle, it wouldn't make a difference actually). This is something Bernhard Sch├╝ssek has been working on. He has created a nice library called Puli. It basically provides a way to locate and discover resources from all parts of the application, be it your own project or a package that you pulled in using Composer.

The core class of Puli is the ResourceRepository. It works like a registry of resource locations. For every resource or collection of resources that you want to expose, you call the add() method and provide as a first argument a prefix, and as a second argument an absolute path to a directory:

use Webmozart\Puli\Repository\ResourceRepository;

$repo = new ResourceRepository();
$repo->add('/matthias/package-name/templates', '/path/to/Resources/views');

Now if you ask the repository to get the absolute path of a particular resource, you can do it by using the prefix you just added to the repository:

$repo->get('/matthias/package-name/templates/index.html.twig')->getRealPath();

This will return the absolute path of index.html.twig, i.e. /path/to/Resources/views/index.html.twig.

Packages exposing their own resources

Though this should seem quite basic to you (it's almost as simple as a string replacement), things will soon get much more interesting when you start using the Puli plugin for Composer. Once you have installed it, you can register any type of resource in the composer.json file of the project it contains. This can be the root project you're working on, or any Composer package that is installed in that project (i.e. any package that you will find in your vendor/ directory).

To expose a specific set of resources, like some Twig template files, you can just list the location of the resources in te composer.json of the package containing them. You can use a location that is relative to the root directory of the package, i.e. where the composer.json file is:

{
    ...
    "extra": {
        "resources": {
            "/matthias/package-name/templates": "Resources/views"
        }
    }
}

Puli automatically adds the absolute location of the package (e.g. /home/matthias/projects/my-application/vendor/matthias/package-name) in front of the relative location that you provide under the resources key (e.g. Resources/views). It thereby makes the package file Resources/views/index.html.twig available anywhere in the application as /matthias/package-name/template.html.twig. Puli will take care of transforming the relative path to the absolute path when necessary.

Integration with other libraries

Of course, most PHP libraries don't know how to work with Puli's ResourceRepository, but Bernhard has already created several Puli extensions, which form a bridge between some popular PHP projects and resources exposed through Puli. For instance there is a Twig extension, allowing you to render Puli-exposed templates using Twig. There's also a Symfony Config extension which allows you to load, for instance, routing configuration files using Puli paths.

Bernhard has also been working on a Puli Symfony bundle, which will make it really easy to have Puli-supported resource exposure in your Symfony project. A small disclaimer though: it didn't work out-of-the-box for me but in defense of the author, there is currently no stable release (yet).

A solution that will always work: stream wrappers

Even though the Symfony eco-system seems to be the first to benefit from Puli and resource-exposing packages, nothing prevents you from using Puli in other types of applications that use another framework or no framework at all. Puli itself is framework-independent and comes with tools to create PHP stream wrappers, which means that any PHP application that uses PHP's built-in file functions like file_get_contents() or fopen() will be able to work with resources exposed by Puli. This is what it looks like:

use Webmozart\Puli\Locator\UriLocator;
use Webmozart\Puli\StreamWrapper\ResourceStreamWrapper;

$locator = new UriLocator();
$locator->register('resource', $repository);

ResourceStreamWrapper::register($locator);

file_get_contents('resource:///matthias/package-name/templates/index.html.twig');

Conclusion

I think with Puli it will become very easy to share resources, like templates, translation files, etc. between applications. They don't need to be in a bundle (or any framework-specific package for that matter) in order to be discovered and used in an application. Resources can be in any package and will even be automatically exposed through their composer.json file.

If you want to know more about Bernhard's ideas behind Puli, you can read about it on his website, even though at the time of writing the site is down.

Categories: PHP Symfony2

Tags: bundles package design Puli

Comments: Comments