Software development Quality Assurance
(get it right the first time – every time)

In the 1980's Quality Assurance was the new idea to come from Japan. Everyone was doing it, everyone wanted to be able to show off their high levels of quality and be able to prove that they deserved the accolade.

The company I worked for at that time was a traditional engineering company manufacturing for the Food and pharmaceutical industries. They mad a decision in the early 80's to get into computer controlled manufacturing systems. I was part of the 20+ team they assembled to develop control systems for the machinery they made.

Coming from a computer systems and electronics background I was surprised to find the company sending out complex equipment untested , mostly the manufacture overran and they ran out of time but there was a culture in the company that the field engineers would sort things out on the customers site when the commissioned the equipment.


After a poor experience in Australia I returned from a proposed 10 week commissioning some 6 months later. The software had been a nightmare, the plant design hadn't suited the customers needs and the whole project had been a shambles. I determined that something should be done about this or I would have to leave the company. I did a lot of talking and held post project meetings to analyse what went well and what went wrong. In the end I was given the task of developing a quality control system to control the development of our software which would fit into our on going ISO 9000 certification.

Much thought and discussion between myself and other engineers resulted in a development system that when applied to a full contract resulted in a project that was developed on time, below budget (don't tell the customer) and when installed worked first time exceeding the performance requirements. Pretty much everything the customer and the manufacturer wanted.

So how did it work? Well to some extent I reinvented the wheel and put in place things that were well knows but rarely used in practice.

The first thing I realized was the manufacturer and the client didn't communicate very well. The client wasn't an expert in software systems and didn't know what could or could not be done and the manufacturer made assumptions that had no foundation.

Software is very difficult to control. As a product you can't “see it”, touch it, weigh it or generally fetch it off the shelf. Control requires a firm and controlled environment where everyone involved understands the need and reason for the controlling system.

I developed the idea that we would write a USER REQUIREMENT SPECIFICATION (URS) this would describe exactly in great detail what the customer wanted the system to do. Because of the nature of software you have to include hardware details as well as the software needs.

This document was written in a form that unskilled and none software people could read it and understand what it was saying, largely for the software side we used flow charting to describe the system and software flow. It became obvious during this phase that often the client didn't really know what their process was doing. Many people were drawn into the meetings to finalise the requirement specification and some times the client found this very revealing for their manufacturing process.
To write such a document required a skilled engineer with a logical mind who can break down a problem into the component parts and document each part. Such engineers are few and far between it appears. However this document is a powerful tool.

The quality assurance control required that the document was signed off by the client and by the manufacturer. It then became the master document for the project. When, (not if), changes arose once the requirement specification had been signed off those changes were costed, given a time schedule and the client advised of the impact they made to the project both in terms of cost and time schedule.
This is important because tight control at this stage allows the sales people to quote much more competitively. Often seeing the cost and time effects customers would decide they didn't need the change after all.

In a manufacturing business you can't just do something for nothing and assume that things will turn out OK. To deal with this most sales departments lay a contingency amount over the project costs making the project seem more expensive that it should be.

The URS leads to the development of a SYSTEMS REQUIREMENT SPECIFICATION (SRS). This document is totally aimed at the software. And is used to control the development process.
The control system is broken down into small sections. Each section is such that it is a completely contained module . That is it has a definable functionality. Defined inputs and defined outputs. Each module is organised such that it is assessed to be a man week worth of work from start to finish.
This breaking sown into a man week of work means that the project control has a resolution of 1 week + or -. The project manager can tell precisely to within 1 week if the project is on time or not.
Finished is defined as operating to the specification, Documented, tested and handed over to the project manager. The project manager then issues the next module and freezes the finished module. NO further work will be done on that module unless there are changes to the specifications that require it. Such work will be costed and time allowed and signed off by the client before it is undertaken.

Testing software is difficult at best. To maintain a level of certainty that the software system will do what it is supposed to do and will continue to do that for all foreseen circumstances is a big task. Fortunately the documentation helps with this. Module inputs and outputs are defined and testable. The concern of the system is the functionality of each module. As long as the inputs circumstances are understood and controlled then you are putting a boundary around the software module where you only need to be concerned about what it does rather than how it does it.

Much of our module testing was done by creating a test system which provided the correct signals at the right timing simulating the plant the module would be connected to. Writing these test systems was also my job. With a good, complete, specification the task is much easier and often creating the test system showed up weaknesses in the specification that could be caught and corrected before harm was caused.

It is critical that all modules are documented as they are finished because software engineers forget how it works, they leave or go sick and you have a chunk of software and no one knows how it works. This becomes critical when at a later date modification is needed.

Once all the modules are completed and tested they can be assembled into the operating system. A decision can be made to test the whole system in a virtual environment with another computer pretending to be the plant or it can be tested with the appropriate hardware when it is completed.
It is essential that there is close cooperation between hardware development and software development although they can for much of the time happen separately from each other. The hardware need to be manufactured in a controlled environment much as the software is. Working to the URS and a suitable SRS for the hardware. It is no good if changes are made adhoc and at test or worse on the clients site we find the software and the hardware don't match up. It happens.
It is important to consider what happens if a module fails a test. There has to be a return to the specification. If the document is correct, and much effort should be put into ensuring it is, then the module was not created correctly to the requirements of the specification.

Should it ever show that the specification is wrong, either because the software can not meet the demands of the specification or through error then the specification must be changed, checked against the URS and if the URS is wrong then changes must be agreed with the client. The software must always be in agreement with the specifications.

In the Pharmaceutical world many manufacturing systems require validation before the USA Federal Drug Association will allow their product to be sold in the USA. That validation requires complete and total documentation of the manufacturing system so that the manufacturer can prove that their system is doing what it was designed to do and will continue to do so n the future.

Much of this paper work was seen as restrictive and time consuming to produce. BUT the end result is a project that is IN control. We have all seen government IT projects that stray wildly over budget and time scale. Usually it is the developer who bears the pain and, if they haven't maintained control they are to blame in my opinion. But often the delays and poor functionality is the result if the client change their minds, not understanding what is required in the first place or hopelessly optimistic sales people promising the earth without worrying how the developers will be able to fulfil their promises.

To make such a system work the project team need to understand why they are operating like this. They need to buy into the system and work with it trusting the documentation and the team that created it.

It is no good in a competitive world saying that's OK - BUT . The old adage that if you fail to plan then your planning to fail is a good one.

I know from practical experience that this structure works, projects developed in such a controlled manner are on time, they are in or under budget and most importantly from both manufacturer and client point of view they work, first time.

It is easy to say why isn't this done everywhere if it is so good. Well to be honest it is quite hard to get right and you need the right people in place to create the documents and control the project. In addition there is a culture is free spirits in software development where it is seen as more of an art than science. Rubbish. Manufacturing can not afford free spirit when money is involved.
At a software conference I asked the assembled experts to define quality software for me. Much laughter ensued and the best I got was “it's like good sex, you recognise it when you see it”. I kid you not.

My definition of quality software is that it does what the client wants no more no less. To get there requires discovering what the client wants of course.

Richard Harris
Bsc Hons.