Monday, 30 July 2007

TAP - Test Anything Protocol

TAP is a really cool idea. A log format for the logging of test results coming out of some automated process. It has some great benefits:
  • Really easy for a human to read with tail -f. Hate trying to read XML or some format that is designed for computers to parse easily
  • It is also really easy to to write a parser for it. Just some easy string splitting and you have the data out ready to write to an HTML page or something.
  • Also with the parsing don't need to wait for everything to finish as with something like XML it is only well formed once the whole thing is complete. As some tests runs may take hours or even days. If you are running for that long you really want some type of results along the way so you know that everything is going alright.
I have been playing with a writing a testing framework at work. I am using TAP for the logging the results. TAP is logging the results ok; but I am hitting some limitations in it.
  • Well my testing app is threaded so keeping the TAP log to spec is not easy. So I am currently logging each thread to a separate log file. In this app it isn't actually to bad as each thread is testing a separate input file so having the results is alright as it make some sense. And when making the report it is easy enough to just iterate through all the log files in a directory to make a nice little report.
  • Another thing is it is quite generic and missing things that I would like:
    • Run time. It would be nice if there was a way to mark run time in the file somehow so it is always in the same place or can regex find it in a common way.
    • Also need a way to link it back to a master test plan. How to link a number in TAP to a test in a test script or something. Thinking making a reporting matching the test execution to the test scripts in a requirements in a test matrix style type thing.
    • Also how do you get logging of results across test runs. As some tests might get run in one test execution and not tested in another one. Need some way of bringing all this together.
    • Also searching of the results and generating reports is a bit hard.
So TAP good at logging there and then. But thinking about long term storage, resulting and traceability matrix type stuff need something more substantial and extensible so can add more data than just the generic stuff. All this has to to be done so the extensibility doesn't break the the generic so can use a basic report generator even when have custom data. So that leave me thinking inheritance and databases suspect that is another blog post. Need to do some more thinking first.


Mozilla wants to spin Thunderbird off. My first thought was how can they do that I hate them etc. But sitting around and thinking about it it may be a smart move ... if done right. The main competition is Exchange/Outlook. Currently TB only hits the email side of Outlook. Using Exchange/Outlook at work it provides so much more. Exchange/Outlook provides you with the full PIM thing and integration between your work mates. Using a solution at client site that doesn't Exchange/Outlook it just doesn't work so well. You don't get emails saying that someone is booking time in your calendar and it doesn't remind you that you have a meeting to go to by default. And this client's system doesn't sync with my phone. From my phone I want to see my address book and I want to know when I have an appointment and also want to be able to book new meetings.

So with this spinning off they need to sit down and hook up with some other people and give us the full experience. Lighting is giving us the Calendar, so we just need the server side from somewhere to hook it all together. And some work from the phone to get it talking to my phone via bluetooth and not not just on WIndows but also on Ubuntu. So one will hope that this seperate can give them the freedom to go down this route.

So how will this help the home user well it give TB a market who will pay for support and development.

As to how it revolutionise the web well. Suspect it will push IMAP and iCal so the server and client are tied together but can interchange and also will bring better bluetooth support to Linux, It will also make the phone manufactures think about an OS other than Windows.

So I am wishing the Thunderbird people all the best and hopefully I will be using TB at work as my production PIM in the near future.

Hi there


So who am I? Well I am Test Analyst at large New Zealand IT firm. I have a particular interest in the use of open source and automation. My favorite reason for automation is "I'm lazy and why should I do something when I can get a computer to do it for me. Especially when said computer can do it more often, quicker and more accurately?"

And like most other blogs what I say here is me and is nothing to do with who I work for.