What is Software 'Quality'?

Quality software is reasonably bug-free, delivered on time and within budget, meets requirements and/or expectations, and is maintainable. However, quality is obviously a subjective term. It will depend on who the 'customer' is and their overall influence in the scheme of things. A wide-angle view of the 'customers' of a software development project might include end-users, customer acceptance testers, customer contract officers, customer management, the development organization's management/ accountants/ testers/ salespeople, future software maintenance engineers, stockholders, magazine columnists, etc. Each type of 'customer' will have their own slant on 'quality' - the accounting department might define quality in terms of profits while an end-user might define quality as user-friendly and bug-free.

Software Testing

Testing involves operation of a system or application under controlled conditions and evaluating the results (eg, 'if the user is in interface A of the application while using hardware B, and does C, then D should happen'). The controlled conditions should include both normal and abnormal conditions. Testing should intentionally attempt to make things go wrong to determine if things happen when they shouldn't or things don't happen when they should. It is oriented to 'detection'.

Organizations vary considerably in how they assign responsibility for QA and testing.

Sometimes they're the combined responsibility of one group or individual. Also common are project teams that include a mix of testers and developers who work closely together, with overall QA processes monitored by project managers.

Automated Testing

Automated Testing is automating the manual testing process currently in use. This requires that a formalized "manual testing process" currently exists in your company or organization. Minimally, such a process includes:

• Detailed test cases, including predictable "expected results", which have been developed from Business Functional Specifications and Design documentation

• A standalone Test Environment, including a Test Database that is restorable to a known constant, such that the test cases are able to be repeated each time there are modifications made to the application

If your current testing process does not include the above points, you are never going to be able to make any effective use of an automated test tool.

So if your "testing methodology" just involves turning the software release over to a "testing group" comprised of "users" or "subject matter experts" who bang on their keyboards in some ad hoc fashion or another, then you should not concern yourself with testing automation. There is no real point in trying to automate something that does not exist.  You must first establish an effective testing process.

The real use and purpose of automated test tools is to automate regression testing . This means that you must have or must develop a database of detailed test cases that are repeatable , and this suite of tests is run every time there is a change to the application to ensure that the change does not produce unintended consequences.

An "automated test script" is a program . Automated script development, to be effective, must be subject to the same rules and standards that are applied to software development. Making effective use of any automated test tool requires at least one trained, technical person - in other words, a programmer .

Defect prevention

The objective of defect prevention is to identify the defects and take corrective action to ensure they are not repeated again over the iterative cycles. Bug reports from the implementation phase, incorrect test cases, incorrect functional requirements, functional requirements incorrect source code form the need for defect prevention. Defect prevention can be done by preparing an action plan to minimize or eliminate defects, defect metrics, defining corrective actions, and to have a root cause analysis of the defects.

Defect prevention can be accomplished by following the following guidelines:

1. Defect data would be calculated from periodic reviews from various phases, test logs from the execution phase of the application.
2. The defect data would be analyzed to perform a root cause analyses where the defects would be segregated and classed under root causes of the problem.
3. The problems should be categorized to form various root causes. Defects should be clubbed under the root causes to see which root cause is causing the defects. Accordingly, those root causes should be classified as per priority and severity. Defect metrics would be generated.
4. Improvement opportunities would be identified and prioritized.
5. Higher management would be notified if some actions need to be taken or some issues needs to be escaled.
6. An action plan should be drawn to reduce defects and proper improvement opportunities should be implemented. They should be monitored to see whether the improvement opportunities results in reduction of defects, otherwise the actions need to be modified.
7. Periodic reviews, peer reviews should be ensured to see whether the action plans and the action points are being followed. After a certain interval of time, defect data would be again generated to see whether the defect metrics are within acceptable limits or modifications needs to be done.
8. The defect age of the defects should be monitored. If the defect age for a particular defect is high, proper action needs to be taken to resolve that defect, like escalation and monitoring of that defect.

QA Documentation

QA practices should be documented such that they are repeatable. Specifications, designs, business rules, inspection reports, configurations, code changes, test plans, test cases, bug reports, user manuals, etc. should all be documented in some form. There should ideally be a system for easily finding and obtaining information and determining what documentation will have a particular piece of information. Change management for documentation should be used if possible.