|
Sources of Reliability Data
Part 2 - Field Data
Last month in "Part
1: Reliability Testing Basics," we discussed reliability testing
as a source of reliability data. While reliability
testing is vital to the implementation of a reliability program, it is not
the sole source of product reliability performance data. Indeed, the data
received from the field is the "true" measure of product
performance, and is directly linked to the financial aspects of a product.
In fact, a great deal of field data may be more finance-related than
reliability-related. However, given the importance of the link between
reliability and income, it is important to insure that adequate
reliability information can be gleaned from field performance data. In
many cases, it is not too difficult to adapt current field data collection
programs to include information that is directly applicable to reliability
reporting.
Field Data Examples
Some of the most
prevalent types of field data are discussed below. These discussions will
tend towards generalizations, as every organization has different methods
of monitoring the performance of its products once they are in the
field. However, the descriptions below give a good general view of how
different types of field data may be collected and put to use for a
reliability program.
Sales and
Forecasting Data
Sales and forecasting information is a sort of general-use data type that is necessary as a basis for many
other analyses of field data. Essentially, this information provides you
with a figure for the population of products in the field. Knowing how
many units are being used at any given time period is absolutely
vital to performing any sort of reliability-oriented calculations. Having
an accurate measurement of the number of failures in the field is
basically useless if there is not a good figure for the total number of
units in the field at that time.
Warranty Data
The warranty data type is somewhat of a catch-all category that may or may not include the
other types of field data listed below, and may not contain adequate
information to track reliability-related data. Since most warranty systems
are designed to track finances and not performance data, some types of
warranty data may have very little use for reliability purposes. However,
it may be possible to garner adequate reliability information based on the
inputs of the warranty data, if not the actual warranty data itself. This
of course is a case of "garbage in, garbage out," and a poorly
set-up warranty tracking system will yield poor or misleading data
regarding the reliability of the product. At the very least, there should
be a degree of confidence regarding the raw number of failures or warranty
hits during a particular time period. This, coupled with accurate shipping
data, will allow a crude approximation of reliability based on the number
of units that failed versus the number of units operating in the field in
any given time period.
Field Service
Data
The field service data type is connected with field service calls where a repair
technician has to physically repair a failed product. This is a
potentially powerful source of field reliability information, if a system
is in place to gather the necessary data during the service call. However,
the job of the service technician is to restore the customer's equipment
to operating condition as quickly as possible, and not necessarily to
perform a detailed failure analysis. This can lead to a number of
problems. First, the service technician may not be recording information
necessary to reliability analysis, such as how much time the product
accumulated before it failed. Second, the technician may take a
"shotgun" approach to repair. That is, based on the failure
symptom, the technician will replace all of the parts whose failure may
result in that particular system. It may be that only one of the parts
that were replaced had actually failed, so it is necessary to perform a
failure analysis on all of the parts to determine which one was actually
the cause of the product failure. Unfortunately, this is not always done,
and if it is, the parts that have had no problem found with them will
often be returned to field service circulation. This may lead to another
potential source of error in field service data, which is that used parts with
unknown amounts of accumulated time and damage may be used as replacement
parts on subsequent service calls. This makes tracking and characterizing
field reliability very difficult. From a reliability perspective, it is
always best to record necessary failure information, avoid using the
"shotgun" approach to servicing failed equipment, and always use
new units when making part replacements.
Customer Support
Data
The customer support data type comes from phone-in customer support services. In many
cases, it may be directly related to the field service data in that the
customer with a failed product will call to inform the organization. In
some circumstances, it may be possible to solve the customer's problem
over the phone, or adequately diagnose the cause of the problem so that a
replacement part can be sent directly to the customer without resorting to
a service technician having to make an on-site visit. It would be hoped
that the customer support and field service data reside in the same
database, but this is not always the case. Regardless of the location,
customer support data must always be screened with care, as the data does
not always reflect actual problems with the product. Many customer support
calls may concern usability issues or other instances of the customer not
being able to properly use the product. In cases such as this, there will
be a cost to the organization or warranty hit, even though there is no
real fault or failure for the product. For example, a product that is very
reliable, but has a poorly-written user manual may generate a great deal
of customer support calls. This is because, even though the product is
working perfectly, the customers are having difficulty operating the
product. This is a good example of one of the sources of the
"disconnect" between in-house and field reliability data.
Returned
Parts/Failure Analysis Data
As was mentioned earlier, failed parts or systems are sometimes returned
for a more detailed failure analysis than can be provided by the field
service technician. Data from this area is usually more detailed regarding
the cause of failure, and is usually more useful to design or process
engineers than to reliability engineers. However, it is still an important
source of information regarding the reliability behavior of the product.
This is especially true if the field service technicians are using the
"shotgun" approach to servicing the failed product. If this is
the case, it is necessary for all of the returned parts to be analyzed to
determine the true cause of the failure. The results of the failure
analysis should be linked to the field service records in order to provide
a complete picture of the nature of the failure. In many cases, this does
not occur or the returned parts are not analyzed in a timely fashion.
Even if they are, there tend to be a significant proportion of returned
parts with which no problem can be found. This is another example of a
potential cause of the disparity between lab and field reliability data.
However, even if the failure analysis group is unable to assign a cause to
the failure, a failure has taken place, and the organization has most
likely taken a warranty hit. In the field, the performance the customer
experiences is the final arbiter of the reliability of the product.
Field Data
Collection
Depending on the
circumstances, collection of field data for reliability analyses can
either be a simple matter or major headache. Even if there is not a formal
field data collection system in place, odds are that much of the necessary
general information is being collected already in order to track warranty
costs, financial information, etc. The potential drawback is that the data
collection system may not be set up to collect all of the types of data
necessary to perform a thorough reliability analysis. As mentioned
earlier, many field data collection methodologies focus on aspects of the
field performance other than reliability. Usually, it is a small matter to
modify data collection processes to gather the necessary reliability
information.
For example, in one
instance the field repair personnel were only collecting information
specific to the failure of the system and what they did to correct the
fault. No information was being collected on the time accumulated on the
systems at the time of failure. Fortunately, it was a simple matter to
have the service personnel access the usage information, which was stored
on a computer chip in the system. This information was then included with
the rest of the data collected by the service technician, which allowed
for a much greater resolution in the failure times used in the calculation
of field reliability. Previously, the failure time was calculated by
subtracting the failure date from the date the product was shipped. This
could cause problems in that the product could remain unused for months
after it was shipped. By adding the relatively small step of requiring the
service technicians to record the accumulated use time at failure, a much
more accurate model of the field reliability of this unit could be made.
Another difficulty in
using field data to perform reliability analyses is that the data may
reside in different places, and in very different forms. The field service
data, customer support data, and failure analysis data may be in different
databases, each of which may be tailored to the specific needs of the
group recording the data. The challenge in this case is in developing a
method of gathering all of the pertinent data from the various sources and
databases and pulling it into one central location where it can easily be
processed and analyzed.
The
"Disconnect" Between In-House and Field Data
It should be noted at
this point that there may be a "disconnect," or seeming lack of
correlation, between the reliability performance of the products in the
field and the results of in-house reliability testing. A typical rule of
thumb is to expect the unreliability in the field to be twice what was
observed in the lab. Some of the specific causes of this disparity have
already been discussed, but in general the product will usually receive
harsher treatment in the field than in the lab. Units being tested in the
labs are often hand-built or carefully set up and adjusted by engineers
prior to the beginning of the test. Furthermore, the tests are performed
by trained technicians who are adept at operating the product being
tested. Most end-use customers do not have the advantage of a fine-tuned
unit and training and experience in its operation, thus leading to many
more operator-induced failures than would be experienced during in-house
testing. Also, final production units are subject to manufacturing
variation and transportation damage that test units might not undergo,
leading to yet more field failures that would not be experienced in the
lab. Finally, the nature of the data that goes into the calculations will
be different. In-house reliability data is usually a great deal more
detailed than the catch-as-catch-can type of non-parametric data that
characterizes a great deal of field data. As can be imagined, there are
any number of sources for the variation between field reliability data and
in-house reliability test results. However, with careful monitoring and
analysis of both sources of data, it should be possible to model the
relationship between the two, allowing for more accurate prediction of
field performance based on reliability testing results.
|
