Web Design and Website Development
Our Promise: More Visitors, More Sales Per Visitor web design melbourne logo

Website Testing
Frequently Asked Questions

Questions

What testing will be conducted on my website?

Testing of your website or application will generally be conducted at two levels

  • System Testing – this is the testing of the software that we perform internally before we release it to you. For smaller projects (e.g. under 10K development budget) this is done by the developer creating the product. For larger projects we use a separate staff member to test your application.

  • User Testing – this is the testing you do to ensure that the software functions as you expect and meets all the agreed points of functionality.

There is a third form of testing carried out on some larger projects

  • External User Testing - this is where we pay external people (e.g. members of the public) to come in and use the application without any training to ensure that it's simple to use.

I'm paying good money for this website, shouldn't I expect a system without any bugs?

After the testing of the application is complete, you should expect to use your site with minimal disruptions or technical bugs. Unfortunately, it is expected that there will be defects (or bugs) discovered during the testing process of your application. These come in two basic forms

  1. Defects - These are where the application simply breaks down and shows an error.

  2. Variations from the agreed functionality - This is where the application works but fails to meet the agreed functionality.

As for variations, we generally pride ourselves in delivering what we promise, however, this issue does come around occasionally where we have different perceptions of what was agreed upon. Our use of prototyping in development typically reduces the incidence of such problems. In all cases, the written quote is the basis for decision making as to what changes are required.

As far as defects are concerned, we again strive to minimise these, however, the reality is that they are a common part of software development and should be managed as such. Even the worlds biggest software developers Microsoft and IBM haven't managed to eradicate bugs from their software.

What is my role as a client in terms of testing?

You, as the client, will be expected to play a key role in the testing of your website.

You will be expected to allocate dedicated time (typically in portions of a few hours at a time) to thoroughly test the entire application to ensure it performs according to the quoted functionality and will make sense to your users.

Why am I as the client doing testing? Shouldn't that be the job of Alliance Software?

We will conduct thorough testing of your system prior to releasing it to you - so yes, testing your system is something we take seriously.

There are however good reasons why we require your assistance. You see, the people who build a system are generally the worst people to thoroughly test it. The developers of your applications get so 'close' to it, that they often struggle to think the way that a new user would. That means that to really be confident that an application is going to work properly, we need someone other than the developer to do the testing.

We are happy to organise members of the public to test your application, however most clients prefer to save the cost of this and do the testing themselves.

It should be noted that even for clients without budgetary constraints, they should still expect to do some testing, simply to ensure that they got what they paid for.

Also, once you've completed testing of the system you'll know it thoroughly and be confident in using it. Clients who take the time to test their websites generally get the most value from them.

What is the "testing period" for?

We will agree in advance on a set period of time for testing (called the "testing period"). Typically this lasts for between one and three weeks depending on the complexity of your application.

The testing period is a time where the application is thoroughly tested and we allocate staff to immediately fix all bugs.

You should allocate time to test your application on the first day of the testing period. Often small applications can be thoroughly tested in under an hour, whilst larger ones may take a full day of testing.

As soon as the testing is done, our developers will review all your reported defects (see below for how reporting works) and begin work fixing them. Typically we can do this in 1 to 3 days. Once the defects have been fixed, you'll be required to test the application again, to subsequently check that all issues have been rectified.

Any further issues are again reported and the process continues until no more defects are present. Based on experience, even complex applications can be tested (and bugs fixed) within test periods of 1 to 3 weeks providing that the client makes testing a priority.

During the testing period, the developer(s) on your project will be 100% focused on rectifying defects. The testing period is the critical time to get defects rectified.

How should I go about doing the testing?

The key to good user testing is simply to be thorough and consistent. We find that a written checklist is a great help.

When creating your checklist, you should aim to note down each point in the workflow of your system. Most of the applications we build are based either off a written specification or a PowerPoint presentation. You should review both of these documents when compiling your list of functions to test.

Using two forms such as the following works well. The first form is simply a checklist of what works and what fails, the second is where you record the comments of what changes are required for test cases that fail

Example Test List
 
Success
Works, but needs changes
Fails
1, Login using correct login details
x
 
 
2, Login using incorrect login details
 
x
 
3, View list of new orders
x
 
 
4, View order details
 
 
x
etc...      

Note - for complex applications, it may help to group functions under subheadings.

Example Test Comments
Test # Comment
2
 Page: www.yoursite.com/login.php
Example Data: Any login account used
Problem: Login failure message to harsh
Solution: Should read "Sorry, your login details are in correct, please try again".
4
 Page: www.yoursite.com/view-orders.php
Example Data: Viewing order number 28
Problem: Page fails, just displays "Internal error"
Solution: Show the order!

How do I tell you about defects or bugs that I discover?

Once you’ve completed testing, you’ll need to email us a full list of all defects or issues you’ve discovered. Two key points to note.

Firstly, please send all defect reports to support@alliancesoftware.com.au (not to a specific developer).

Secondly, to help you out, we need the following information about each defect:

  • Page: The web page where the bug was found (you can see this in the address bar of your web browser)
  • Example Data: Any data relevant to the bug. So if for example, the bug relates to a specific person or product in the system or you triggering the bug happens when you enter certain information, we need to know.
  • Problem: What is the problem? What is the system doing wrong?
  • Solution: How should the system work?

If the problem is that a page doesn't work, obliviously the solution is simply 'get page to work', however, at other times the solution might be more specific, e.g. "the subtotal of the column is wrong when you have a large number of entries".

How quickly will you fix defects I find and what will I have to pay?

As a client, you need to be aware that the way we respond to defect reports differs over time as follows:

  • Defects reported during the testing period – at this stage we are actively seeking defect reports and allocate staff to work on these to immediately resolve all defects. At this state, your defects have absolute priority. Defects detected in this stage are fixed free of charge.

  • Defects reported after the testing period, during the support period – defects are assessed based on system impact and competing workloads and queued accordingly. Defects detected in this stage are fixed free of charge.

  • Defects reported after the support period – defects are again assessed based on system impact and competing workloads and queued accordingly. Work done at this point is charged at our standard hourly rate or according an agreed support contract.

It is critical that you make full use of the testing period. As a development company, we need to schedule this (e.g. we can't allow it to be open ended) as we need to organise developers to also service other clients.

Experience has found that clients who allocate time to test during the testing period easily iron out problems in their systems. Those who fail to test their systems during the allocated testing period may find frustrations in having to wait for issues to be fixed.