Navigation

Recently, Aura is creating some buzz that components of PHP frameworks should not have any dependencies. To quote Paul M. Jones’ recent article on Aura:

The distinction is that all Aura packages (with the exception of the Framework package) are completely independent, and have no cross-package dependencies, whereas at least some of the components from Symfony2 and ZF2 have dependency requirements.

Or, quoting this article on replacing Silex with Aura.Micro:

I was recently working on a small project that used Silex. As I browsed my vendor folder, I realized how much extra “stuff” I had inherited with Silex. There were a bunch of other components required when all I wanted was some quick and easy routing, micro-framework style.

I think this needs some urgent clarification before this way of thinking becomes mainstream and PHP creeps back into its NIH-hole.

Dependencies Are Bad

The general attitude of people arguing against dependencies is that dependencies are a bad thing. Why could they be a bad thing?

Because I have to install them all by hand.

This used to be a problem, but void now that we have Composer.

Because they clutter up my hard disk space.

Hard disk space is cheap, and internet connnections are fast, so whether you upload 500k to your web server or 1M does not really make a difference anymore.

Because they eventually might become unsupported.

That’s a problem, but it’s a problem of the developer of the library, who should pick dependencies that he trusts. You as a user don’t have to care about that.

Because they confuse me.

Composer installs dependencies into a directory “vendor/”, so they are hidden away from you. If you ever browse them, you probably know what you’re looking for.

Because it’s not micro-frameworkish.

I suggest to join the nearest church if you are into religious debates.

Are They Really?

Why do libraries have dependencies at all? Is a library with dependencies less decoupled than one without?

The reasoning for dependencies usually goes like this:

If I need some functionality, and if another library does that, I should reuse that library.

For example, let’s look at the Guzzle HTTP Client’s composer.json:

"require": {
    "php": ">=5.3.2",
    "ext-curl": "*",
    "symfony/event-dispatcher": ">=2.1"
},

Guzzle needs event dispatching and relies on the Symfony2 Event Dispatcher to do so. There’s no point in coding another one if there’s a solid, mature and tested implementation available, right now, for free, that does what you want.

Coupling vs. Cohesion

One major argument for the elimination of dependencies is “decoupling”. So let’s examine two important quality criteria in software engineering: Coupling and Cohesion.

In software engineering, coupling or dependency is the degree to which each program module relies on each one of the other modules.

Coupling can be expressed in various degrees, ranging from high to low:

Content coupling (high)

Content coupling (also known as Pathological coupling) is when one module modifies or relies on the internal workings of another module (e.g., accessing local data of another module). Therefore changing the way the second module produces data (location, type, timing) will lead to changing the dependent module.

[...]

Message coupling (low)

This is the loosest type of coupling. It can be achieved by state decentralization (as in objects) and component communication is done via parameters or message passing (see Message passing).

No coupling

Modules do not communicate at all with one another.

As is generally known, it is preferrable to achieve low coupling, also known as “decoupled code”. What about cohesion then?

In computer programming, cohesion refers to the degree to which the elements of a module belong together.

Like coupling, cohesion can be expressed in various degrees:

Coincidental cohesion (worst)

Coincidental cohesion is when parts of a module are grouped arbitrarily; the only relationship between the parts is that they have been grouped together (e.g. a “Utilities” class).

[...]

Functional cohesion (best)

Functional cohesion is when parts of a module are grouped because they all contribute to a single well-defined task of the module (e.g. tokenizing a string of XML).

High (functional) cohesion is preferrable over low (coincidental) cohesion. Just because you package some code together doesn’t mean that code really belongs together.

To distill the essence of the above quotes:

  • Libraries should be lowly coupled. They should communicate with other libraries through clearly defined interfaces.
  • Libraries should be highly cohesive. They should only implement what they are responsible for (their “core business”).

Reducing the dependencies of a library to zero (no coupling) may be a good thing, but only if the library does One Thing Only. If it does not, complete decoupling leads to low cohesion: The library will do more than it should.

Hurray for Dependencies?

What does this mean in practice? Can we just increase the amount of dependencies as we like?

My personal stance on this topic is that a library should require as little as possible dependencies to run (hard dependencies) – as long as it is highly cohesive – but support as many different libraries as it can (optional dependencies). The limit for the latter really is the library’s developer’s time and what he wants to support. As example, I will quote the composer.json of Behat’s Gherkin parser:

"require": {
    "php":             ">=5.3.1",
    "symfony/finder":  ">=2.0,<2.2-dev"
},
...
"suggest": {
    "symfony/yaml":         "If you want to parse features, represented in YAML files",
    "symfony/translation":  "If you want to use Symfony2 translations adapter",
    "symfony/config":       "If you want to use Config component to manage resources"
},

Because it needs to look for files in the file system, the parser has a hard dependency on the Symfony2 Finder component. But then it also supports optional dependencies. These optional dependencies help you if you want additional support for YAML files or translation functionality. In my opinion, the Gherkin parser is a perfect example of how you should manage your packages’ dependencies.

My Wish

To conclude, I would like to move away from “dependencies are bad per se” to a more differentiated view on dependencies. Dependencies help to achieve high cohesion and to keep DRY. Use what’s there already. Simplify the installation of your packages with Composer. Then try to make as many dependencies optional as you can. But don’t remove all your dependencies just for the sake of it. And stop promoting your libraries as if that was a good thing, please.

Update (December 18th)

For clarification: This blog post is not a rant against NIH. The point is not that we should have one implementation only of everything. Variation is good and competition is healthy.

The point is that you should not put all code into one dependency-free package, if it should rather be in two packages that build upon each other. @sebwebdev on Twitter sums it up quite well:

Loose coupling + high cohesion > No coupling + low cohesion

 

Posted Saturday, December 15th, 2012 at 15:13
Written by: | Filed Under Category: Community
You can leave a response, or trackback from your own site.

31

Responses to “On Libraries and Dependencies”

Johannes

One disadvantage with dependencies that you have not listed, is that you might easily run into conflicts because your favorite library A depends on a certain version of B, and your favorite library C depends on a different version of B.

Then, you need to decide between eiter using A, or C since you can obviously not use both at the same time.

This is also an area where Symfony2 can, and imo should improve by really giving each component its only release cycle and dropping the monolithic repository, see https://github.com/symfony/symfony/issues/5891

Otherwise, I agree with you that there is no point in not adding a dependency if another library already does what you need.

Stan

Dependencies and dependency management are a good thing. However, the more dependencies there are then the more there is to manage. And quite frankly, just because the dependencies are hidden away in our vendor folder does not make it a good thing. There are times when packages seem to fall into that PEAR like mentality where they stack up dependencies without a good holistic plan about what the library needs. Symfony Routing is a good example, it required Config, Yaml, HTTP Kernel and Doctrine Common. That’s a lot of dependencies for just doing routes when all I added in my composer.json was “symfony/routing”. What a surprise the first time I do a composer install on that simple dependency! I think you’ll find the needs of some of those dependencies are fairly small as well. Smaller dependencies provide developers with more choice in the long run. (Disclaimer: I am NOT trying to make a dig at Symfony Router or open a can of worms, I actually like and use it!)

Johannes makes a good point about the release cycle. It’s hard to start treating each component as a stand alone library when they move together as a single unit.

A lot of folks out there will use Symfony end to end, or Zend end to end or even Aura end to end and this dependency thing might not be a big deal to them. But there are some of us out there that have a hybrid of components tooled together. The more dependencies that are forced on us (guilty by assoc) the harder that is to do. So maybe my app that’s routed by Aura, templated by Twig, works with the DB using Doctrine ODM, utilizes Symfony for the Finder and Yaml components and Zend for the console could find itself in a pickle when more baggage is added to the mix through dependencies that might not be necessary.

Perhaps another way of looking at it is like this… What is it that causes a dependency to be acceptable for a package? Are there standards that a team works from to add Package B to the dependency list of Package A? Or is it just because that developer feels like it? This is an architectural question that I think deserves more discussion. Dependencies have implications and they needed to be discussed and considered, not just thrown in because Composer will deal with it for us.

Brian

Hi, one of the counter-points on “Dependencies Are Bad” I don’t feel was given any attention is the thought that more dependencies adds complexity (thus introducing overhead, security threats, etc), and the often encountered in real life “fsck! I can’t use X and Y at the same time because Z is deprecated!”

I would enjoy reading your thoughts on this in a bit more detail.

Thanks!

Bernhard

Thank you all for the interesting comments. I agree that dependencies can lead to problems and that Composer does not solve everything. But the solution is not to have no dependencies, but – as you said – thoughtful dependency management and minimizing hard dependencies and making them optional instead.

@Stan: The Symfony Routing component does not have a single hard dependency. You probably did the same thing as Paul and installed it with the “–dev” switch on.

Einstein already gave us a guideline to use here:

“Everything should be as simple as possible, but no simpler.”

Dependencies should be used when they make the code simpler, overall. They should not be used where it makes the code more complex. That means if you’re using only 1/10th of a given 3rd party library… you’re probably better off reimplementing it because it’s going to be simpler that way. (But yes, model on something proven.) If you’re going to be using 2/3 of it, then having the dependency makes the code simpler because of the cohesion/coupling arguments above.

I’m not 100% sure that Johannes is correct that the Symfony Components should be split up with their own release cycles. On the whole, though, they’re a good example of loosely coupled code, with some notable exceptions. (We’ve run into a number of those while trying to integrate only some of Symfony into Drupal, most of which we’ve tried to argue the case for back upstream to mixed success.)

Stan

@Bernard I disagree. The composer.json “says” Routing does not have a single hard dependency. But if I do not install the the “require-dev” dependencies I get fatals in the component. Consider this:
https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Routing/Loader/AnnotationDirectoryLoader.php

That has a hard dependency on “Config”. There’s no way around it. I would much rather that “symfony/config” be under require and not get the fatal while developing.

That is a documentation issue not a dependency issue. If I use a library for image processing then I would want it to come with implementations for GD, Imagick etc. If I get an library for sending emails I would like it to support different transports. If I have a database package I want it so support different RDBMS.

I see little sense in ensuring that any class be includable without a fatal.

That being said, with composer.json “suggest”, the use of DI and explicit use statements code outside of the subnamespace, one can make figuring out such optional dependencies even in the absence of documentation quite easy.

Bernhard

@Stan This is the distinction I tried to make between “hard” and “optional” dependencies. Hard dependencies are required for a library to be usable at all. Optional dependencies are for extra, non-mandatory features. Annotations, in the Routing component, are not mandatory, but optional.

There is only one reason why dependencies are bad, but it is a huge inescapable one. Dependencies are bad because I, as the developer, am in charge of the security of my code. I must chose between willful ignorance of the security practices of the dependencies or investigate them myself with tools or by hand. If I chose the second I must either keep my changes to myself or return the fixed code to the related repository.

Ignoring the fact that in under a lot of structures I am barred entirely from the last option, if I chose it I quickly stop working on my project and spend all my time fixing other people’s projects. That, or I necessarily am violating most terms of use for open source packages or short-changing my benefactors on security.

That’s it. We can toss all the perfect-world-scenario theory around we like, the final truth is dependencies force us to chose between being ignorant security fools, thieves and terms-abusers, or full-time open source contributors.

Perhaps the last option is the best option. Using only open source tools that you contribute to, and nothing else. If only every workplace were cool with working on open source projects on company time; they are not, and therefore that leaves us with only the other two options. I see no way around it for the vast majority of people.

In a place where “fixing it” cannot mean upstream participation you need full stack frameworks and/or much more proprietary code. Decoupled libraries and dependencies maintained across the web are a methodology and set of tools that preselects for participants the farther in you go.

SampeS8N: It depends on your level of paranoia. as Chris DiBona famously said, “popular open source projects tend to be secure, because insecure open source projects tend to become unpopular fast.”

In *most* cases, looking at a project’s track record with security issues and general coding requirements is sufficient. You don’t need to audit every line of code yourself. I’ve deployed Drupal dozens of times. I have not personally reviewed (manually or with tools) 90% of the code, but the general security track record of Drupal is very good. It’s good enough for the White House.

Similarly, I’ve not read more than 5% of the code base of Symfony, yet Drupal is using much of it in our next release, Drupal 8. In the process we *did* find a security hole, which we reported and was fixed quickly. That makes me more confident that the rest of the code base is secure, because Symfony demonstrated it cares about security and will address issues. (Naturally we’ve done more than just that, but we’ve not audited every line of code.)

If you’re running a nuclear reactor, please audit every line of code. But for 99% of projects it’s a waste of time, money, and effort to do so. Instead audit their security practices. You’ll get a much more meaningful result with less work. And, frankly, the security of an OSS project with a good security process is most likely better than your home grown code in the first place.

@SampeS8N: This sounds like you are absolutely sure, that you will always write completely secure code from scratch and the contributors to a oss-library can’t. That seems to be a little bit shortsightened and that it needs you as “full-time open source contributor” a little bit arrogant. Seriously: That are all developers, who participate in OSS-projects, and you are not the first one, who thinks, that security is important.

Bicou

In composer you only have “require” and “suggest” dependency, “require” are always installed and “suggest” is only displayed.
Maybe it should introduce another level of dependency like “recommend” (as seen in debian dpkg).
Then, you can choose between light (only required), featured (+ recommend) or eager (+ suggest) package installation.

Take the Behat’s Gherkin parser example, readme says “separate project and has no foreign dependencies”, well it’s not true since it depends on a symfony component. But if you install this package without the symfony-finder dependency, the parser is still able to do its job without it (take the “usage” script in the readme).
This would be a perfect “recommend” dependency as the core feature of the package don’t really need it.

Stan

@Lukas I agree that there is a fundamental documentation problem in the situation I noted. I think though, a better way of dealing with the issue would be further sub splitting packages and even more decoupling. I’m just not a big fan of surprises, ie. the fatals. Symfony/Routing/Annotation? That way there could be a distinct dependency config file identifying exactly what is needed to utilize it. Docs in the cookbook, etc. are great too though.

Stan

@Bernhard I understand where you are coming from, and I get the terminology. I just disagree with the approach, the clearest definition of what I need in order to run a package is it’s composer.json file, and with Symfony\Routing it’s not clear at the time of installation. Just my $.02.

Hey @Bernhard,
Nice post on NIH :-) . What I feel is NIH is not a bad idea either ;-).
We have lots programming languages, is that a bad idea?
About the issues :
I am not quite sure whether all the Bridges need to be placed in the library.
I agree many of the Symfony components are standalone, but the problem with the library approach is that many of the Bridges still reside in the components. Can’t we move to itsown Bridge or Bundle what ever you call ? May be its a hard one.
The good thing about NIH is at some point if the developer feels that Router is not working the way I expected can switch it easily.

We can also Benchmark the components. If one component has dependency on development though it don’t have hard coded dependency seems wrong way. I prefer it should have another place. I also agree its hard todo such a way.

Thank you

Bernhard

@Stan Then I guess it’s really just a documentation issue.

@Hari IMO it does not really matter if I extract a bridge out of a component, or leave it in under some namespace “Extensions\”. In both cases, extra setup is required to make the bridge work. In both cases, it boils down to proper documentation. But extracting a bridge also means a lot more work (more packages to maintain, more versions etc.) and in simple cases isn’t really worth the hassle.

nice article, however you have a clear prejudice against churches?

“I suggest to join the nearest church if you are into religious debates.”

Maybe in your experience religious debates don’t get anywhere. I think discussion is healthy when done right. Coders discuss so you need to be open minded and allow discussion. But we all get the point of course if you used it as a figure of speech. It is just not accurate since we do the same.

Everything else on the article gets my ok :)

belleza!

@Sarry Garfield and @SingCrunch I speak from experience. As the developer in charge of security for a government project we have to be paranoid about security. I am under no illusions that I am perfect, which is why I rely on the support of tools in addition to my own knowledge.

These tools, across the board, found security flaws in every open source solution we’ve tried to deploy for the last 4 years. Not a single project we’ve used has escaped the detection of one or more legitimate security holes I could personally exploit.

Almost invariably these flaws were in dependencies that were hangers-on with these projects. It isn’t typically the project we chose to use that has the problem. Only once was it, top-to-bottom, the choice we made. That choice was CodeIgniter.

I have personally been in the position I outlined.

These are off-the-shelf tools. AppScan and Fortify. They aren’t even particularly good scanning tools and they are horrifically expensive and hard to use. I’m not claiming I’m awesome at detecting problems, just that we did in everything from the big to the small. It has lead to us doing more and more internally to reduce the footprint of the code, because less code is inherently more secure than more code.

This is why I believe dependencies are a bad thing in general. If we, as a community, took security more seriously this probably wouldn’t be a problem. But with huge open source projects like Word Press only recently moving away from md5 for password hashing, or the countless other well-known problems with security that many projects still do; I see no end for the problem on the horizon.

Bertrand

Ha, ha, who’s got the NIH syndrom ?
Event_Dispatcher, written in 2005, 3 years before the one in your example…
http://pear.php.net/package/event_dispatcher

[...] Blick auf einige der jüngsten Diskussion über Abhängigkeiten in der PHP-Community Sharing einige seiner Gedanken über das Thema und die Not Invented Here Denkmuster. Die allgemeine Haltung der Menschen [...]

Bernhard

It seems that my post was misunderstood by some. I added a clarification at the end of the post.

Eugene OZ

Guys, I so wondered about all this “noise” about dependencies. Dependencies are absolutely necessary – look at Linux, don’t you know about dependencies here? Or you really think that all your libraries will reinvent the wheel each time just to not use external code? It’s ridiculous.

Iain Potter

Good article. IMHO ridiculous to talk about ‘no dependencies’.

What I would say is that dependencies can make code difficult to unit test, but the use of techniques like dependency injection can help here.

[...] Bertrand lobbed a shot over the net at, I guess, the Symfony crowd, when he said: [...]

@Bertrand: Just FYI .. the Symfony2 event dispatcher is based on the Doctrine implementation. Doctrine itself is almost as old as your implementation, though in general I suspect that Doctrine mostly looked at Hibernate (Symfony2 itself used Spring and Django a lot as a source of inspiration) as the source of inspiration and not at the Cocoa framework (making this potentially an example of different approaches for a similar problem domain).

And in case you are wondering why Symfony2 isn’t just using the Event system from Doctrine Common. We actually looked at it and determined for the flexibility that Symfony2 needs Doctrine would have made changes that would have had a severe impact on its performance. We tried to see if there was a way to share the core logic, but determined that it wasn’t possible.

Furthermore if you look at your code from back then, its using stuff like return by reference. I am sure that it can be rewritten to PHP5 with reasonable effort, but its not like it would have made sense to use as is nor that its a lot of lines of code to write an event dispatcher. It basically boils down to picking one of the algorithms already implemented out there and then making it work in PHP.

[...] О библиотеках и зависимостях — Существует мнение что, меньшее число зависимостей у интрумента означает меньшую связанность. Автор в своем посте аргументирует в пользу того, что само по себе наличие зависимостей совсем не плохо, а их наличие уменьшает связанность и повышает связность. [...]

Bertrand

@Lukas : Look here http://web.archive.org/web/20100907174618/http://components.symfony-project.org/event-dispatcher/

It says : **Symfony Event Dispatcher has its roots in the Apple Cocoa notification center.** So I guess you are misinformed. And I don’t see how this relates to Hibernate or Django…

But I was indeed wrong on one point: they got the NIH syndrom 5 years later, not 3.

Bernhard

@Bertrand: You are both right. Originally, the event dispatcher had its roots in Cocoa. Later on we thought about replacing it with Doctrine’s event dispatcher, but couldn’t, because we would have had to add functionality to it that would have negatively impacted Doctrine’s performance. So we ended up adapting our own implementation instead.

These past few years, I feel increasingly that most components can avoid difficult dependencies by adhering to one simple principle: each component should stick to a single responsibility.

To use a practical example, think about a component that sends e-mail via SMTP – this is the kind of thing where you’re typically going to want some kind of diagnostic facility when it’s running in production. So many people will simply add a $debug flag to the class, and this will toggle some internal logging feature on or off.

The problem with that is, logging diagnostic messages has nothing to do with the sending e-mail – and because logging can be done in 100 different ways, this introduces scope creep and eventually multiple different logging-methods and some way to toggle them, which is way beyond what an e-mail component should do.

Modern PHP has closures, so we can now solve this by simply adding a callback property (e.g. $onLog) to the class – and suddenly, all the SMTP class needs to knows about logging is that messages exist. Everything else can be delegated to a programmer, who can choose to plug in his favorite logger, test-framework, Twitter API, a simple echo-statement during development, or whatever madness you see fit.

Sure, turning on logging now is a bit more work than just raising a boolean flag – but I think we all need to get more comfortable writing a couple of lines in a closure and think of it as “configuration”, even if it is code. It’ll make our lives easier in the long run.

Hello, I think your blog might be having browser compatibility issues.
When I look at your blog site in Safari, it looks fine but when opening
in Internet Explorer, it has some overlapping.

I just wanted to give you a quick heads up! Other then that, superb blog!

Leave a Reply

 

Additional Resources