In my Beyond Frameworks talk, I explained how a component-based architecture can help answer some of the important (i.e. expensive!) questions you might face when creating long-lived apps that rely on a PHP framework. In this series of blog posts, I’m going to look at how to go about creating and working with components.

Working unit tests is only half the battle. If your unit test doesn’t cover 100% of the code, chances are you’re shipping bugs. 100% code coverage doesn’t guarantee bug-free code by any means, but it’s an important step in the right direction. There isn’t much point in shipping code that other developers cannot trust; they will just go elsewhere to get their components.

How Much Of My Code Is Covered By My Tests?

The skeleton component created by the phix php-library:init command makes it very easy for you to see how much of your code is covered by your unit tests. To do this, you need to make sure that you have Derick’s excellent xdebug extension installed. After that, just run your unit tests from the command line, and our skeleton will take care of generating the code coverage data.

stuart:~/Devel/sental/repustateApi$ phing test

This command will generate a HTML-based report in the ‘review/code-coverage’ folder:

This report shows you exactly which lines have been executed during the unit tests, and which lines of code have gone untested, making it very easy for you to see where the gaps in your testing are.

Aim For 100% Code Coverage

I’m the first to admit that this is something I’ve changed my tune over. When I was trained as a professional engineer in the 1990’s, we were taught to do 100% code coverage … and, frankly, the cost of writing all those tests seemed horrendous to me. On one project I worked on, the project manager told me that he budgeted 40% of our time for writing tests. As the dot com bubble came, and everyone rushed to compete on Internet Time (no, I didn’t believe in it then, and I definitely don’t believe in it now!), the emphasis was on shipping code with all possible haste. 100% code coverage seemed a luxury no-one could afford.

Today, with the benefit of 20/20 hindsight, I can clearly see that the costs of shipping partially-working / barely-working code are just too high. Even cobbled-together code ends up surviving for many years longer than anyone anticipated, and that just means generation after generation of programmer having to waste time maintaining the code, or spending their time instead creating and shipping their own barely-working code.

I’ve been experimenting with, and developing, this approach to components in PHP since summer 2010 (annual vacations are great for getting some hacking done!), and there’s no way I’m going to admit how many bugs I found in my own code – code I was actively using – when I went back to add 100% coverage 🙂 Suffice to say it was a lot, and more than enough to convince me of just how important 100% code coverage is to a component.

100% code coverage does not guarantee that your code is bug-free; it is not the magic target that will produce zero-defect code. But it is a good step in the right direction.

Be the first to leave a comment »

In my Beyond Frameworks talk, I explained how a component-based architecture can help answer some of the important (i.e. expensive!) questions you might face when creating long-lived apps that rely on a PHP framework. In this series of blog posts, I’m going to look at how to go about creating and working with components.

To be a success, all components need high-quality unit tests, and this applies to the component I am building, a client for Repustate.com’s semantic analysis API. But how do you go about creating the unit tests for a remote web service without the process becoming onerous or even dangerous?

Can You Test The Quick And Nasty Way?

This is the first question that always comes to mind whenever I’m writing an API client. I don’t want to do it the classical way, of creating my own mockup of the web service, if that can be avoided. Mockups are slower to build, and they perpetuate any misunderstandings I have about the API and its behaviour. I want to test directly against the API, so that I can catch errors immediately, and have confidence that my code will work in production. Technically, these are integration tests not unit tests, and there are good arguments out there for also having the mocked up unit tests too, but I’m a pragmatic engineer, and I need to get this done before my app’s requirements change.

Now, this all comes down to whether or not the web service makes life easy for developers or not.

  • Does the web service provide a test API, where I can reset the state of my account to a known position before I start testing?

    This is the holy grail for writing tests against an API. Tests must be deterministic – same data in must always produce the same result out – to be trustworthy. Being able to use a test API, where I can control the starting conditions before I run my tests, gives me everything I need to safely test against the web service’s endpoint.

    Sadly, you’re more likely to see a unicorn than come across one of these, but never give up hope!

  • Is the API destructive in a disruptive way?
  • If I have to test against the live API, can I do so safely? Can I write tests that will not cause trouble for my production systems that are using the API?

    Often, the answer to this is ‘yes’, which is great news. Not as good as our unicorn-test-API above, but something I can work with.

  • Do I have to mock up the API locally?

    This is the testing of last resort, tbh … but sometimes you’re left with no choice on the matter.

    And, it’s worth me mentioning that API mockups are actually the best way to deal with regression tests against your own client, as they allow you to catch badly-formed requests to the remote API perfectly.

At the time of writing, I couldn’t find my unicorn-test-API for Repustate, but because their live API is entirely non-destructive, there is little risk in making my unit tests for this component work against the live API.

Never Commit Your API Key

Like many web services, Repustate’s requires you to apply for an API key, which is then used to authorise your web service requests. This allows Repustate to keep track of who is doing what, and allows them to revoke access if the web service is abused.

To run the tests against the live API, my test code is going to need to know the API key that I’ve obtained from Repustate. But if I commit that to the component’s public GitHub repo, I’m simply asking for every script kiddie to abuse that key. It’s simply not safe to commit this key. And it wouldn’t be safe if the repo was private; how do I know that one day the repo won’t be open-sourced, with the key available to find in the repo’s history?

Never, ever, ever, commit an API key to a component’s source control repo.

But my test code needs this API key. Where can it get the API key from? We could put the key in a config file that’s installed by an uber-secret component, and indeed this is a practical approach when you’re controlling the deployment of apps in the enterprise. (Just don’t make it so secret that your own devs can’t work on it. That kinda brings a lot of work to a standstill). But that won’t work for an opensourced component that hopefully others will find useful.

So we need to put the API key somewhere else, just for the tests to work.

Getting The Key From The Environment

The easiest place for the unit tests to look is in the process’s environment. First, I need to set the key, like this:

stuart:~/Devel/sental/repustateApi$ export REPUSTATE_KEY="<my API key>"

This puts the key into the environment of my terminal’s bash shell. When I run the tests from this shell, the API key will be available for my tests to use:

[code lang=”php”]
$apiKey = getenv(“REPUSTATE_KEY”);
[/code]

This is a handy tip, and it means I can now get on with the quick-and-dirty approach to unit testing my component, without worrying about giving anyone my API key to abuse.

It does cause one problem though: anyone who downloads my component and tries to run the unit tests will simply see them fail. I will mention it in the component’s README.md file, but I cannot rely on anyone actually reading that file in all honesty. For now, I’ve addressed that by testing for the REPUSTATE_KEY environment variable, and if it doesn’t exist, the unit tests die() with clear instructions rather than fail.

Be the first to leave a comment »

This Month

March 2011
M T W T F S S
« Feb   Apr »
 123456
78910111213
14151617181920
21222324252627
28293031