Reliability HotWire: eMagazine for the Reliability Professional
Reliability HotWire

Issue 3, May 2001

Reliability Basics
Developing Good Reliability Specifications

While most organizations are concerned with the reliability of their products, many will not develop good reliability specifications. This is a serious flaw that can result in unreliable products reaching the field. Even the most vigorous reliability testing program will be of little use if the product being tested has a poorly-defined reliability requirement. Sometimes, reliability requirements will be developed by a marketing organization or other group with little or no experience in life data analysis. This can result in vague or unrealistic requirements that do little to aid in the development of the product.

One type of poor reliability specification is the non-quantitative (or "qualitative") specification. This type of specification may be useful as a general guideline or goal for the project, but often specifications of this type will become codified in the requirements documentation for the project as the reliability specification. One such example of this is the goal that a new product will have a reliability that is "no worse" than that of the previous model. While this is certainly a laudable goal, it is not very useful as a reliability requirement, particularly if there is no quantitative information about the previous model. Another common qualitative reliability specification is that the product will "meet or exceed customer expectations." While this is once again an admirable aim, if there is no numerical measure of "customer expectations," the requirement is essentially useless. Ideally, customer measurement programs and input should be used to develop reliability goals, but in a more quantitative manner.

Another common pitfall when it comes to specifying product reliability is the use of the mean life or MTTF (mean time to failure) metric. This is widely popular due to its perceived ease of calculation, and due to the fact that it sounds very scientific. However, there are numerous drawbacks to using the MTTF as a reliability metric. Essentially, the MTTF represents only one point in the lifetime distribution of the product and, as such, does not adequately characterize the overall reliability behavior. For example, the three products in the following plot have identical MTTF values despite having quite different reliability curves.

 

Clearly, these three products have vastly different reliability characteristics, yet they all have a value of 100,000 for the MTTF. For more information on the limitations of the MTTF as a reliability specification, see ReliaSoft's Reliability Edge newsletter article on the MTTF at http://www.reliasoft.com/newsletter/2Q2000/mttf.htm.

We have seen that non-quantitative statements and MTTF values are not adequate for reliability specification. What, then, should be used to specify reliability in a way that will adequately address reliability engineering needs? At a minimum, a reliability specification should consist of three components:

  • A specified reliability

  • A time associated with the specified reliability

  • A desired confidence level 

For example, we could specify that a product should have a 90% reliability at 1000 hours of operation with a 95% confidence level. In simpler terms, this means that we want to be 95% confident that 90% of the population will survive at least 1000 hours.

This type of reliability specification can be taken a step further with a tool known as "Planguage." Planguage (a contraction of the words "planning language") was developed by Tom Gilb, and is a useful and flexible tool for developing qualitative and quantitative specifications. Planguage is a keyword driven language that can be applied in many areas of business, but we will limit our discussion to its use for specifying reliability. Some of the keywords that will be of interest in our reliability specification are:

  • TAG - a unique identifier for the measurement in question

  • GIST - a short description of what is being specified

  • SCALE - the scale of measurement of the quantity or quality of interest

  • METER - the method by which the scale is measured

  • MUST - the absolute minimum acceptable level of the scale

  • PLAN - the desired level of the scale

  • STRETCH - the best-case level of the scale

  • PAST - a value of previous results for purposes of comparison

While these are not all of the keywords that can be used in Planguage, they are enough to apply to our reliability specification example. Care must be taken to distinguish between scale and meter, as they can be easily confused. A suitable analogy would be the electrical service running to a house. The scale would be the actual amount of electricity flowing through the wires while the meter would be the electrical meter which records how much electricity is being used. This should become more clear as we apply Planguage to our reliability specification example:

  • TAG - product reliability

  • GIST - the probability of a product operating for a given amount of time without failing

  • SCALE - reliability at 1000 hours, at 95% confidence

  • METER - reliability testing in QA facilities

  • MUST - 87%

  • PLAN - 90%

  • STRETCH - 95%

  • PAST - 85% reliability measured on previous model just prior to release

This gives just a very quick thumbnail sketch of Planguage and its application to reliability specification. Obviously, this type of specification is much better than a statement like "the product must be no worse than the previous model." Planguage is a very elegant and simple method to develop specifications for qualitative measures and can be applied to many areas of business, including marketing goals, mission statements, business objectives, etc. For more information on Tom Gilb and Planguage, go to http://www.result-planning.com. (Note: This link is provided for your information only. ReliaSoft has no affiliation with this organization.)

ReliaSoft would like to thank Erik Simmons of Intel Corp. for his contribution to this article.

 

ReliaSoft Corporation

Copyright © 2001 ReliaSoft Corporation, ALL RIGHTS RESERVED