Test, please.

On quite a few of my projects, I need people other than myself to test it. Projects that are more complex than simply displaying a list of phone numbers. Perhaps I have not looked in the right places, but I have not found much in the way of methods of getting people to test applications. I'll explain some of the ways I've tried and their result.

Test this please
Sometimes I send out a link to some people, saying "I've made a few changes please test this when you get a chance". I might get a 5-10% response out of about 10 people. They don't always say anything, but I know they at least went to my test site (from the magic of last login times). Sometimes I give them a deadline, such as let me know by Friday. The problem is I don't know how much they tested and what they did.

Giving out instructions
After watching people doing the above testing and seeing they only did the most simplest of tasks and only spending about 2 minutes and saying "YEP, it works." I came up with a Testing plan. It looks something like this:

Moving a person from one company to another:

  1. Load application.
  2. Search for a person, load that record.
  3. Clear the company and search for a new company.
  4. Load the company and choose a location.
  5. Save.
  6. Go back to basic search and make sure the person's company has changed.

I will usually give 5 or 6 sets of instructions along with a time estimate at the top. This gives busy people an idea of how long this will take and they can plan a slot of time for it. It also helps to give people a deadline, I'll say I need this by Friday. I'll leave blank line to record comments after each set of tasks and additional lines at the bottom for any other comments. I ask them to turn this sheet in to me. I've considered getting some other incentives, like $5 McDonalds gift certificates or from the bagel shop -- but I don't know if management would go for that (personally, if I was a manager I would. Small things like this really boost morale IMHO). I usually get all these back, although not always by the deadline. These require more work on my part and I may not be able to think of all the situations where someone might use the system (in an ideal world, I'd work from a set of requirements / use cases but alas, not so in all my projects).

Having a walk through
In some occasions, I have made a change to my application and gone to someone and asked if they have a few minutes. If so, I will ask them to do this and that, similar to my Testing Plan. This way I can watch them and see if they do things I didn't expect. It may be clear in my mind you click the "book" icon to look things up, but they may press another button. Or they may try and tab to something and it doesn't happen in the order they expect.

Its always good to get fresh eyes on a project, I will often find new employees and give a short explanation of the application and ask them to try it and see what they think. They can often give insight that I, who wrote it or those who have used it through all the incantations and know it like the back of their hand can give.

ALWAYS thank your testers and tell them you will give their comments consideration. They will be more apt to help you in the future.

Be considerate of their time. Most people are trying to get their work done and that doesn't include testing your application. You might talk to a manager or boss and ask them, who would have time to help me with this? you can tell your testers that the boss has approved and they will probably feel more comfortable to spend their time helping you.

Automated testing
I've been looking into testing frameworks that will do some browser emulation, which I would like to do in addition to the user testing. Some of the frameworks I've found:

Anyone know of any others? I'm not in a position to use Java based solutions, but perl, php, ruby or python are possibilities.

I will give at least a couple of these a try and report back next week.

hmm

Yes ... and yes..

hmm.. never thought of an easter egg... course I'd have to explain what one is first..