In my Beyond Frameworks talk, I explained how a component-based architecture can help answer some of the important (i.e. expensive!) questions you might face when creating long-lived apps that rely on a PHP framework. In this series of blog posts, I’m going to look at how to go about creating and working with components.

Working unit tests is only half the battle. If your unit test doesn’t cover 100% of the code, chances are you’re shipping bugs. 100% code coverage doesn’t guarantee bug-free code by any means, but it’s an important step in the right direction. There isn’t much point in shipping code that other developers cannot trust; they will just go elsewhere to get their components.

How Much Of My Code Is Covered By My Tests?

The skeleton component created by the phix php-library:init command makes it very easy for you to see how much of your code is covered by your unit tests. To do this, you need to make sure that you have Derick’s excellent xdebug extension installed. After that, just run your unit tests from the command line, and our skeleton will take care of generating the code coverage data.

stuart:~/Devel/sental/repustateApi$ phing test

This command will generate a HTML-based report in the ‘review/code-coverage’ folder:

This report shows you exactly which lines have been executed during the unit tests, and which lines of code have gone untested, making it very easy for you to see where the gaps in your testing are.

Aim For 100% Code Coverage

I’m the first to admit that this is something I’ve changed my tune over. When I was trained as a professional engineer in the 1990′s, we were taught to do 100% code coverage … and, frankly, the cost of writing all those tests seemed horrendous to me. On one project I worked on, the project manager told me that he budgeted 40% of our time for writing tests. As the dot com bubble came, and everyone rushed to compete on Internet Time (no, I didn’t believe in it then, and I definitely don’t believe in it now!), the emphasis was on shipping code with all possible haste. 100% code coverage seemed a luxury no-one could afford.

Today, with the benefit of 20/20 hindsight, I can clearly see that the costs of shipping partially-working / barely-working code are just too high. Even cobbled-together code ends up surviving for many years longer than anyone anticipated, and that just means generation after generation of programmer having to waste time maintaining the code, or spending their time instead creating and shipping their own barely-working code.

I’ve been experimenting with, and developing, this approach to components in PHP since summer 2010 (annual vacations are great for getting some hacking done!), and there’s no way I’m going to admit how many bugs I found in my own code – code I was actively using – when I went back to add 100% coverage :) Suffice to say it was a lot, and more than enough to convince me of just how important 100% code coverage is to a component.

100% code coverage does not guarantee that your code is bug-free; it is not the magic target that will produce zero-defect code. But it is a good step in the right direction.

One Comment

  1. Simon says:
    March 30th, 2011 at 11:52 am

    Thanks, Stuart. It’ll be interesting to see what comments this will draw, as there has recently been lot of internet-bashing of those of us who aim for 100%, and I think that’s a shame.

    As you point out, the bugs will typically be in the untested portion, plus it’s a lot easier to keep coverage at 100% than it is at 79.84%, simply because you do spot it when some new untested code creeps in.

    I’m glad you qualified this with the caveat that 100% is a step in the right direction rather than a silver bullet full of magic guarantees – I think a lot of the bashers don’t realise that us compulsive testers do appreciate that.