What Experts Do: RPN for QbD Risk Assessment

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

This is important because some folks get stuck on the rating system. Please share your rationale behind the different ordinal (ranking) scales.

Nitin

Research & Development at Pharma

Dear Mr. Sun, In risk assessment, for risk ranking Low(1)-Medium(2)-High(3) is perfect. In DOE this rating system is depending on various factors and what model we are considering / applying. This ranking system includes “with center point ” or “without center point” consideration in DOE during model application considering input and response factors/variables. This is what I know… please flash it more… Thank You.

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

Mr. Nitin, Thanks for sharing. There seem to be 2 arguments around 1-2-3 scale: 1) not enough resolution; or 2) need more separation. How would you respond to it?

Research & Development at Pharma

Mr. Sun, these arguments again depends on how much critical factor is for quality attributes and risk level of the same. We are always considering all things with two opposite extremes with neutral center point. So I would say 3-point scale is acceptable for quality outcomes. We can’t scale it in multiple points as its not practically feasible to get desired outputs.

Risk assessment is nothing but the ‘Brain Storming’

During the brainstorming session, we make a list of all the factors that could possibly influence any of the responses we want to measure. We don’t try to evaluate factors as they are listed, just listed them all down. It is likely that our list may end up with 30 or more factors!

Next, group the factors into categories such as – (rank 1) most likely, (rank 2) somewhat likely and (rank 3) least likely to influence the responses. Also note if the factor is easily controllable or difficult to control. From this list, we decide how many and which factors we want to include in the experiment. For the other factors,we decide if they will be held constant, monitored, or ignored.

Remember, it is not necessary to include absolutely everything in one giant experiment. Smaller, sequential experiments teach us about the process quickly and generally cost less.

Once we have determined the factors to study, we must decide on a range for each one. The range should be large enough so that we expect to see a difference in the response(s), but not so large that we “fall off the cliff” and don’t get any measurable results (thats why we cant scale in multiples). Think about the change in the response that we want to find. The factor range should be set so that it could cause this amount of change in the response. Ideally, the experiment will produce some good and some bad product so we can learn what causes the change in response. So bottom line is all this is decided by our understanding of critical quality attributes.

Research & Development at Pharma

Above example of risk assessment is in the form of DOE…

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

i like the way you explained your answer.

Nitin Kadam likes this

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

I also like 1-3-9 scale. Anyone else use this scale?

Olivier MICHEL

Senior Consultant chez ALTRAN AG

Dear Sun , I’ve seen the 1,2,3 scale quite often , which does not means it is the more relevant. Indeed most companies are stuck by procedural ranking values and sometimes there are endless arguments to pass below the resulting RPN with the criticality green / yellow or red light. In such instances it can be useful to …re-design the scale, at least based upon a visual view of what was obtained as first /e.g. Pareto chart of RPN. I know there are many articles about risk ranking and RPN values ; best guess is that there should be some statistical possible evaluation of selected scale and resulting RPN to assess significance ? Sorry not to help further.

Manager – Quality by Design at Philip Morris International

We use a 1-2-3-5-7 scale for severity and occurrence, and a 1-3-5-6-7 scale for detectability.

Each of the scales is described with several examples from different areas of development. Our experience has been that this gives us just about the right balance between simplicity and detail …. and now that the system has been in repeated us for several years we only rarely have problems coming to a ranking.

One thing, however, that we do require (and which I suspect helps us avoid the problems Olivier describes), is that we do all the assessments for a given factor (e.g. severity) and these results are then hidden whilst we are doing occurrence, once occurrence is complete it is also hidden while we are doing detectability. This helps us to avoid people mentally calculating the RPN and deciding ‘Oh, that’s too high’ and then seeking to ‘correct’ one of the individual factors.

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

Since our industry does not have an established criteria like automotive or aerospace we usually generate our own. This may be alarming to inspectors since “the scale” is different for every company and every product. Of course the scale is a relative ranking (or normalized) — so in the end, it may turn out fine. Thanks Oliver and Gawayne for sharing your approaches. Very helpful. Please share more of your practices and tips.

Manager – Quality by Design at Philip Morris International

Sun …. on your last comment …. I believe that whilst the regulators will accept variation in the risk-assessment from company to company (providing there is a clear definition) … they are, however, looking for consistency of practice within a company, and this includes consistency in the way we rank our risk assessment. We currently have 4 very different product which all use the same ranking structure. Yes it takes a little more time, but it does then have the benefit for the regulator of consistency, and for the company of being able to directly compare products.

Nitin Kadam, Sun Kim and 1 other like this

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

Gawayne, this is true. Companies have SOP’s for risk assessments for commercial products. This should provide some consistency within a company. However this is not common for development projects, where many things are still fluid. How about for development projects? Could you share some practices you’ve seen with R&D?

Atul Gunjal likes this

Inna Ben-Anat

Director, Global QbD Strategy and Product Robustness at Teva Pharmaceuticals

The farther the product is within the lifecycle, the higher resolution of RPN may be applicable. For example, at early R&D stage I think simple 1 (least risk), 2, 3 (most risk) or green, yellow, red are sufficient. Often even as a holistic evaluation, without dividing officially for severity, detectability and occurrence. Since at R&D stage it is less about failure mode and more about identifying and then understanding scientific link and relationship of CPPs and CMAs to CQAs. Then as we gain more knowledge and understanding about a product, higher resolution may be required: 5 levels scale, for S, D and O, is more applicable for process design and commercial stages.

I think what most important is the justification we provide for level of risk, whether it is initial risk assessment or revised after additional development/enhancement was done and risk can be reduced as a result of mitigation. Clear justification and definition of risk ranking will facilitate better understanding from regulator’s end and within the company.

Nitin Kadam, Sun Kim and 3 others like this

Senior Application Specialist at Umetrics AB

Gawayne

Can you please explain while the scales are differently “loaded” 1,2,3,5,7 for occurrence and 1,3,5,6,7 for detectability.

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

Inna, insightful as always. I like: “R&D stage is more about identifying and then understanding scientific link and relationship of CPPs and CMAs to CQAs.”

Erik, Good point, if the scales are balanced, it could bring a “bias” effect to either occurrence or detectability. Gawayne?

Research & Development at Pharma

Almost same thought I wanted to explain in my comment what Inna mam wants to say… completely agree… multiple ranking we can apply where we need more precise optimization and that we need in to process optimization… at the understanding stage… i.e. early phase of development three rank is system definitely serves the purpose..

Sun Kim likes this

Manager – Quality by Design at Philip Morris International

Erik … the loading is based upon the way we use the detectability criterion and perform the calculation … in practice the impact is the same …the gap widens at the high-risk end of the scale in the same way it does for Severity and occurrence.

On Inna’s observation … I have no problem with using a less sensitive scale in early development, however we have chosen to adopt an approach where we use a consistent set of scales throughout development and manufacturing risk assessments. We did this because we felt that, on balance, that we preferred to reduce the risk of confusion between stages at the cost of increased effort in early development. On combination products this ensures that we treat device and consumable the same way – even although the speed and complexity of development are significantly different. It’s not an issue of right or wrong, it’s just a question of choice.

Dr.S. Gangula likes this

Erik

Senior Application Specialist at Umetrics AB

If I understand Donald J Wheeler http://www.qualitydigest.com/inside/quality-insider-column/problems-risk-priority-numbers.html correctly he is VERY critical to the idea to calculate RPN numbers. As I myself like multivariate I kind of agree with him that when you have spent time defining 3 numbers why then throw away information by making them into one number. Comments from Inna and Gawayne and others with experience is really appreciated.

Sun Kim likes this

Manager – Quality by Design at Philip Morris International

Erik … I’ve discussed this before elsewhere, but just to describe to you how we choose to perform our assessments we apply the following :

> for severity, occurrence, and detectability we calculate 2 numbers each …. the first is the actual value, and the second is our confidence in the assessment (based on a detailed ranking table).

> the conventional S,O &D are then used to calculate an RPN the sole function of which is to determine a risk category …. i.e. it determines whether we have to take action, and if we do then to what extent we have to reduce the RPN. Any assessment of actual action is based on the S, O &D numbers, not on the RPN

> the 3 ‘second’ numbers are used to calculate a ‘confidence ‘ value (it’s a bit like creating a mechanism to allow is to assess the quality of our process understanding). This second composite value also has trigger points, but this time its not to change the product, but to increase our understanding of that particular aspect of the product … it could be through experiment, through literature search, or through modeling for example … We also have targets for reduction here.

> the net effect is that if we think the product is safe, but we’re not sure, then we have to do more work to show that it is safe. It we think there is a risk, then we have to reduce it. And finally, if we think there is a risk but we’re not sure, then we have to increase the certainty, and if the risk stands-up, then we have to reduce it.

I would certainly not claim it is perfect, but it does give us a degree of confidence that we are covering the bases that the regulator would expect us to cover …. and our experience in using it for 3 years has been overwhelmingly positive.

We’re actually currently in the process of extending use of this approach, which currently covers consumable and hardware development and manufacturing, to also cover software and firmware development.

I have some real sympathy with Wheeler’s view, but I do think that much of his criticism can be addressed by intelligent use of the technique.

Hope this helps.

Dr.S. Gangula, Sun Kim like this

Senior Application Specialist at Umetrics AB

Gawayne

Thanks for the explanation. I can see that you and your colleagues have put quite a lot of effort and thoughts into your system.

Heath Rushing

at Adsurgo LLC

In my experience, a 1-3-5-10 scale works well because it provides a clear differentiation between the truly critical and non-critical process parameters. There are very few high RPNs in this system which provides a clear separation for your parameters while removing the inefficiency of long discussions over the difference between a 4 and 5 (like you have with the 1-10 scale).

Nitin Kadam, Sun Kim like this

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

Heath, I’m glad to see a non-linear scale. 80-20 rule. At development, we should focus on the top CPP’s.

Nitin Kadam likes this

Ramadas Bhat

Executive QA Director at Isis Pharmaceuticals

Is it absolutely necessary that one should use RPN scoring system?. The bottom line is to come up with overall quality risk. If there are major risks then remove or reduce them via appropriate / suitable corrective actions. The scoring system often puzzles me, especially if you have a window/range of 1-3 to asisgn low classification. The simplest (I call it “FMEA Light” (!!!) approach could be a non-quantitative one and assess impact (severity), probability and detectability either as Major / High or Minor / Low. Do not even use the in-between i.e. Moderate category although in few cases this could benefit. Come up with overall risk based primarily on impact / severity. This means if the impact is high and probabality is low and the detectability is high, the overall risk is High.

Sun Kim likes this

Manager – Quality by Design at Philip Morris International

Ramadas …. I think the simply answer to your question is ‘No’ …. you are required to assess risk, and the regulator will want to be convinced that you have an effective, repeatable method for doing that ….after that the method is up to you. The advantage of the FMEA approach is that it is fairly widely adopted, so the regulators are used to seeing it (and its variations) and will accept it more quickly. The choice is still yours.

Olivier

Olivier MICHEL

Senior Consultant chez ALTRAN AG

To add on Gawayne usefull comments (thanks for sharing your field-proof experience), I think that whatever the scale there also should be some common sense applied to the RPN calculation and score ; whereby I favor a picture/scheme/graphical presentation of RPN after calculations [and also because I am aging so can no longer play with endless tables with numerous columns and lines `-) ] ; I understand there should be some procedural and fixed scoring so that one has not to re-invent the wheel at each risk analysis occurence. Some additional view with non linear and 3 scale scoring together with an interesting approcah to ponder scoring in the following:http://www.nnepharmaplan.com/Global/Downloads/Expert-articles/Risk-analysis-and-mitigation-matrix-(RAMM)-2012.pdf

Senior Application Specialist at Umetrics AB

Reading about RAMM and other similar methods (FMEA) it seems that all of these methods talk about getting a group of judges/referees together that then jointly discuss until they have a agreed on a score. Comparing this with sensory analysis (taste of food, wine…) where one also get a group together but ENSURE that each member gives his/her own score without influence from others and then uses the average or weighted average as a measure.

As I see it this idea of a group discussion might very often ensure that the views of the alfa male/female will be used. I am doubtful that this is the optimal way forward.

Have any of you tested to let each member give their own independent score?

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

Erik, Good observation. I tried both. I learned process owner/expert has the final say on rating. It’s not a democratic process (nor should it be). Here’s how I do it: (https://qbdworks.com/2-uncommon-qbd-practices/)

To minimize waiting, facilitator should ask the process owner rate individually and then have the group review. This tip alone will cut your risk assessment group meetings by half.

HyunSung Kim

R&D Quality Assurance at Green Cross Corp.

The risk priory number is related the previous data. First, the interval of each numbers is related a discrimination. The discrimination is originated the previous information; preclinical data, toxicology study, stability study. and early research study and so on. Second, The more data, the more large mumber. And the highest number (3, 5, 10 and so on) of ranking numbers is related the impact of among factors; severity, occurence and detectibility. Normally at the research and development stage, the 5 is chosen as the highest number. At continuous process verification stage, the 10 is selected as the highest scaling number.

Sun Kim likes this

Yvonne Ayasse

Técnico de Garantía de Calidad y Técnico Responsable

Hi Sun, there are many opinions for risk ranking. I think it depends on the life cycle stage of the product.

At the development stage, if you have no consistent quantitative data about the process, I would propose a qualitative approach 3 point scale low (1), medium (2) and high (3) for each one of the factors.

You would argue: Not enough resolution! In order to get more resolution you need a quantitative approach with more levels and therefore you need data; regulators require that you demonstrate the quantitative level of probability of each risk rating (by statistical means, for example). At the developing stage this is only possible, if the process is similar to another well studied (and not allways because the process equipment could be similar, but no the same).

At this early stage you have defined your product profile,selected CQAs from all other quality attributes, done FMEA and you are checking by DoE how CPPs are linked with CQAs. A quantitative approach you will achieve by validation and later by experience.

Sun Kim, Ramadas Bhat like this

Comments from FMEA users outside the QbD context

Please note that the lifecycle stage of this group is different – mostly post-commercialization. The big difference is the amount of data and information available. For development QbD projects, information and data may be scarce.

Some industries (i.e. automotive) already have RPN scale standards (i.e. AIAG). I forsee in a few years, our industry may pursue such an effort.

Interestingly even with the standards, common theme seems to be: “Make your own scale.”

27 comments

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

This is important because some folks get stuck on the rating system. Please share your rationale behind the different ordinal (ranking) scales.

Philip Stunell

Engineering Management Consultant

If you are a component supplier to major OEM you may be required to adopt their rating preferred scales (for example AIAG / SAE etc).

If you have the freedom to choose your own, then the rating scale should have enough resolution to compare alternative product or process designs and identity the best (lowest risk) option – but not be so fine grained that it becomes too complex. This means that you need some clearly defined ‘bench-marks’ to calibrate your rating scales – and although not perfect these are provided by AIAG / VDA / SAE etc

In my experience, people argue for ‘simplified scales’ (L-M-H) or (1-2-3) when they have no objective or quantified data to guide them – and the correct score has become a ‘matter of opinion’ rather than evidence based.

The danger with a ‘low resolution’ and ‘subjective’ rating scale is that it does not encourage people to understand “why” the score is high or low – or “what” the difference is between alternative proposals in sufficient detail.

To improve the design or production process we need a better understanding of the factors that modify Risk / RPN / Hazard levels – and the purpose of the FMEA is develop that understanding – and document the results for future reference.

Fabrício X., Sun Kim and 5 others like this

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

Philip, Thanks! For companies who do not have an established scale (and have to establish one) what approach worked well for you? Any examples?

Philip Stunell

Engineering Management Consultant

I use (and teach) the AIAG rating scales, and encourage people to “calibrate the scales” in a practical way – so for example’

If we have a severity 9 or 10 failure in service – then we are saying this might require a product recall – so we must if possible address the cause or break the chain of cause and effect to protect the consumer.

8 means the car won’t get me home..

7 means I might get home, but would not go out again.until it is fixed

Etc

Then getting people to give specific examples that fall into each category – as this helps to create a sense of proportion and consistency in scoring.

What people find most difficult is judging the ‘occurrence rating’ and ‘detection scores’ in the Design FMEA and the use of Design Validation as a control to identify design errors. Although.the current edition (4th) of the AIAG standard is ‘more helpful’ – this is area of most debate and confusion.

Andreas K., Sun Kim and 3 others like this

Deborah Ybarra

Quality Manager at Batesville Casket Company

Sun,

When I began working for my current company, they did not have a standard for FMEAs. A few odd PFMEAs had been done, with little to no impact, as it was not well understood how to use them. Certainly DFMEAs, EFMEAs, etc. had not been attempted on any scale. They were using an AIAG format and using the scales given in the FMEA guide. These scales, however, did not translate well into their environment. Their customers were funeral homes, not OEMs. The customer base did not require FMEAs to be done, but the impact of not evaluating the risk was evident in our processes. We revamped the entire way we evaluated risk. We utilized the same numerical scales, but applied new definitions to the scales to determine risk based on our particular industry. It made more sense overall and has worked well. In many cases we were able to substantially reduce risk and were well prepared when problems occurred.

Stephen M., Fabrício X. and 1 other like this

Luis Antonio Arruda Batista

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

Deborah, very interesting application! So in summary, you used 1-10 scale with new definitions for each level of casket failures?

Deborah Ybarra

Quality Manager at Batesville Casket Company

Yes, exactly.

Eduardo Santos, MSc.

Management Business – Quality & Safety Engineering at ESCD – ES Consulting & Development

Dear Sun Kim, I suggest You take a look at FMEA Handbook from AIAG. Very usual approach in automotive industry.

Daniel Silea Sut

Supplier quality management, SQM leader plant Sibiu_Continental Automotive

i would recommend you to use AIAG or VDA 4. also something which could help you.

Management Engineering Student at Università degli Studi di Roma “Tor Vergata”

Hi Kim!

That’s a interesting question. In my time evaluating PPAP’s, I’ve only seen the suppliers using the scale 1-10, as suggested by 4th. Edition FMEA manual. I totally agree with Phillip, if you have freedom for define it, you can see what is the more applicable for you. Be flexible, don’t follow religiously a scale, because the scales can’t be too adequate for you evaluation.

The important thing using the scales in FMEA risk evaluation in my opinion, is that you should use it together your good sense during the evaluation. Any scale will work right if you give the punctuations with coherence.

I have an example of coherence and wrong use of scales. I saw a FMEA once that it had a severity score “9” for the failure mode “burrs in the hole”. The justification for this score was that “burrs can hurt the operators in the next operations”. And another failure mode with severity “6” or “7” with the failure mode “hole higher than the specified” with effect “counter part doesn’t assemble”. So, let’s think: the burrs can be removed with sanding in the next operation, and the burrs doesn’t affect the healthy of final customer. But if you have a hole higher than specified, you’d lose your part (it won’t assemble, and any rework can’t be possible). What failure is the most severe? You need to pay attention for this kind of situation when you use your scale: My evaluation is coherent?

Best regards Kim!

Luis Antonio Arruda Batista

Management Engineering Student at Università degli Studi di Roma “Tor Vergata”

Another thing I forgot to mention, take care with the scales of Manual, or with any scale. They are only for reference, but you can fit them for your reality in your company. If you need to modify the criteria of the scale for make it adequate to your process, don’t hesitate to make it!

Best regards!

Vivek Madakshira

Quality Assurance at Volvo Bus Corporation

Scale for calculating RPN, Severity: 1 to 5 Very minor to Low, 6 to 10 Moderate to hazardous, Occurence : 1 to 3 Very low to low, 4 and 5 : moderate, 6 to 10 : high to very high, Detectability: 1 to 4 : very high to moderate high, 5 Moderate, 6 to 10: low to almost impossible..,

Andrew Walsh

Industry Professor at Stevens Institute of Technology and Principal, PharmaClean Group, LLC

Hi Sun,

I’ll share my experiences with FMEA scales from the Pharmaceutical industry, where they are fairly new.

About 10 years ago I was leading a site-wide team that was trying to re-evaluate the way we did Computer Systems Validation and Equipment Qualification. We were trying to initiate a “risk-based” approach that the FDA had recently advocated and we were trying to come up with an SOP for performing FMEAs.

The definitions used by the automotive industry did not fit with pharmaceutical manufacturing and had to be replaced. Without belaboring it, we could not come up with a scale that everyone could agree on new definitions for. We found scales of 1-10 were not resolvable (that is, we could not make clear distinctions between 5 and 6, or 7 and 8). Also, different disciplines would rate items differently. For example, QA always rated things at a significantly higher risk than Operations (an 8 vs. a 6).

Other scales, such as 1-3 (Low/Med/High) led to most things being rated “High” and led to little improvement over the current practice of just validating everything. Changing to a scale of 1,3,5,7,9 tended to result in compromises between groups leading to many 5 ratings.

An article published about that time suggested not having a mid point (0, 2, 4, 6, 8, 10) so that items had to be either on the “High” side or “Low” side. Again, the ratings moved everything to the “High” side.

We spent 3 years working on this SOP (among many other things) and never completed it (the facility is now closed).

Fast forward to today and I am currently working on an FMEA approach for evaluating Pharmaceutical Cleaning Processes for Cleaning Validation. I am now working on a system that uses real measured values for Severity and Likelihood and avoiding using arbitrary ordinal scale values. While reading Douglas Hubbard’s book “The Failure of Risk Management – Why it’s Broken and How to Fix it” I had an epiphany and recognized the many errors that were being made in our efforts 10 years ago. Ordinal Scales do not work.

I am also looking at redefining the RPN value to something reliable considering how the current approach to calculating RPN (SxOxD) is not justifiable (see Donald Wheeler’s article athttp://www.qualitydigest.com/inside/quality-insider-column/problems-risk-priority-numbers.html).

Personally, I still really like the concept of the FMEA and I am using with my clients, but the typical Ordinal Scales used for SOD and the calculation of the RPN are seroius problems and need to be improved.

I hope you find this helpful.

Best regards,

Andy

Stephen M., Andreas K. and 1 other like this

Owner of DAZONT Consulting, former lead auditor for AFNOR CERTIFICATION, TUV Rheinland and NSAI (retired)

sorry guys, but in automoyive industry hou are seldom free to chose:

“7.3.6.3 Product approval process

The organization shall conform to a product and manufacturing process approval procedure recognized by the customer.

NOTE Product approval should be subsequent to the verification of the manufacturing process.

This product and manufacturing process approval procedure shall also be applied to suppliers.”

Must bé understood as a way to push All levels of suppliers to use the rules defined by the final OEM Customer.

what if you work for several OEMs ? have a look on their portals, they give information about the équivalence they accept

Andreas Kefalidis

Dipl.Ing. Electrical Engineer, ASQ CQE

Hi Andy

I totally agree with you,

Donald’s Article is also very informative.

Thank you.

Andreas

Andrew Walsh likes this

Antonio Perez

Plant Manager at Chrysler Mexico S.A. de C.V.

60 is the best to assure qualiry

Carl Carlson

Reliability and FMEA Consulting and Training

Hello Sun,

I agree with Philip in his posts on this topic. In my own words . . .

The criteria for the severity scale must be reviewed and agreed upon, clearly showing needed differentiation between safety and regulatory risk, loss or degradation of primary and secondary functions, and lower severity such as annoyance. For Process FMEAs, in addition to the criteria for the effect on the product, the criteria for the effect on the manufacturing process must also be established.

The criteria for the occurrence scale must be reviewed and agreed upon, clearly differentiating the full range of anticipated failure rates, from very low to very high, and where possible identifying ranges of failure frequency for each occurrence level that makes sense for the system, product or process being analyzed.

If detection risk is ranked on a detection scale, the criteria for the detection scale must be reviewed and agreed upon, clearly differentiating the likelihood of detection from very remote to almost certain. For System and Design FMEAs, risk related to detection can also be differentiated based on the timing opportunity for detection, the type of test used for detection, in addition to likelihood of detection. For Process FMEAS, risk related to detection can be further differentiated based on the manufacturing stage or type of operation.

If the risk ranking scales are not mandated and the team has the flexibility to establish their own risk ranking scales, there is a simple rule to follow: use the minimum number of ranking levels for each scale that adequately differentiates the risk criteria. In other words, if the team can manage with five ranking levels and the needed differentiation of risk for a given application is adequately defined, then use ranking scales with five levels. If ten ranking levels are needed to adequately differentiate and define the risk, use scales that have ten ranking levels. It is worthwhile to spend the time needed to define properly the scales with the correct resolution and criteria. Using scales that have too many ranking levels for a given application can result in the FMEA team spending excessive time deciding which level on the scale represents the risk without adding value. Using scales with too few ranking levels can result in the FMEA team missing important risk differentiation.

It is essential for the FMEA team to understand the limitations of RPN (which have been written about elsewhere), and avoid a numbers game. In addition, don’t forget that high-severity must be addressed regardless of RPN value.

Carl

Like (4)

Stephen M., Philip S. and 2 others like this

Marc Schaeffers

CEO at DataLyzer International

Most companies still use the standard RPN. Switching to SOD is a nice idea but what if the limit to take action is let’s say 150. Severity never changes so if a severity is higher than 1 you always have to take actions. You could set the limit to 500 but that means 499 will not require an action nad again if severity is higher than 4 you can never finish so both RPN and SOD have some serious disadvantages

A new approach which is slowly coming up is the approach of matrix based risk priorities because it overcomes the disadvantages of RPN and SOD.

You make 3 matrices (SxO), (SxD) and (OxD) and for each combination you give a colour red, orange or green. Then you create a new matrix with the 3 columns and the possible color combinations which will result in 27 possible combinations. The 3 combinations give a new colour (red, orange or green) and the colour indicates if action is required.

This takes the disadvantage of numbers completely away and is easy to understand.

Like (3)

Melania Marilena N., Andreas K. and 1 other like this

Owner of DAZONT Consulting, former lead auditor for AFNOR CERTIFICATION, TUV Rheinland and NSAI (retired)

This discussion Is about ranking.

The topic “RPN versus rest of the world” Is deeply documented elsewhere on This forum

Andrew Walsh

Industry Professor at Stevens Institute of Technology and Principal, PharmaClean Group, LLC

Hi Jean-Jaques,

The title of this discussion is “In calculating RPN’s for FMEA’s, which Scale Type do You Use? “, so discussion of RPN is not only applicable, but essential since it is the final goal of the ranking systems.

Besides, I did a search for RPN in the Posts and it returned only 3 posts that contained RPN and only 1 of them was about RPN in risk assessments and contained only 5 replies. Where is RPN deeply documented on this forum so we may reference it? Thanks!

Best regards,

Andy

Andreas K. likes this

Andrew Walsh

Industry Professor at Stevens Institute of Technology and Principal, PharmaClean Group, LLC

Hi Marc,

I have seen the systems you describe and they are an improvement over the standard RPN (which as you note is in wide use by companies blissfully ignorant of its issues).

I have been toying with a system that simply maintains the original scores and simply displays them side-by-side. So a ranking of S = 9, O = 7 and D = 9 would appear as 979. The value of each score is clearly evident in the final value. These can be color coded as well so that numbers, let’s say, 7 and above are Red, 4-6 are Yellow and 3 or less are Green. It is clear at first glace where the problem is(are) and where work should be started first. Either eliminating the hazard if S is too high, reducing the Occurrence is O is too high or increasing the Detection if D is too high.

After Risk Reduction efforts have been taken, new scores can be generated and documented. In a glance, it is now also clear where efforts were made to reduce risk and by how much. So in the case of the 979 used above a new score of 931 would show that major efforts at improving Detection where taken and that a good effort at reducing Occurrence was achieved. For a Severity of 7-9 the goal should be to reduce the O and D scores to “0” so 900 would be the ultimate (but probably unachievable) goal.

This would be used for an initial analysis where there is insufficient data to use scales with real values. In my case (Cleaning Validation) I have real scales for Severity and Occurrence so when I have that data available I will use these values instead of arbitrary ranking scales.

I hope this was helpful,

Best regards,

Andy

Andreas K. likes this

Owner of DAZONT Consulting, former lead auditor for AFNOR CERTIFICATION, TUV Rheinland and NSAI (retired)

The question is about the scale type, not about what to do with RPN. The answer to This question os in FMEA 4th and also in guidances of OEMs.

It Is disapointing to read about tresholds set to 60 or 250 after so many years of explanations done by IATF about the intent of FMEA

Julie V. likes this

Owner of DAZONT Consulting, former lead auditor for AFNOR CERTIFICATION, TUV Rheinland and NSAI (retired)

Andy

Your method is simple and valuable, It looks to me like meeting the intent

Andreas K. likes this

Ganesh Sundaresan

Quality Professional

I believe the risk identification and prioritization is as good as the FMEA team and the Engineer who owns the Design. Even the most accurate of scaling methods, can many times play-down or play-up the real risk on the ground. That is not to discount the use of scaling for prioritization, but there are some limitations that separate figures from facts. While RPN without doubt misleads, SOD certainly gives the Engineer some food for thought.

Donald Wheeler’s article is informative. Thanks.

Lori Gray

Corporate Business Systems Coordinator

1-10, 1 being virtually no chance, and 10 being near certainty of occurrance..

S 1-10, 1 means the user will unlikely notice, and 10 meaning safety is in jeopardy.

1-10, 1 it will certainly be caught, 10 shows a design weakness wii make it too inial production without detection.

That being saud; 1 minor, 2-3 slight annoyance, 4-6 moderate failure causing dissatisfation to the customer, 7-8 high degree of customer dissatisfaction and this does not involve safety or noncompliance to government regulations, 9-10 Very high severity rank and is safety related failures along with failures to regulations and standards.

hope this helps

Like (2)

Luis Antonio A., Andreas K. like this

Master Trainer 3rd Party Automotive & Aerospace CB Auditors

In my view, please follow something simple with which you are comfortable and which gives precision in estimation. For this, in my opinion, rating of 1-10 is meeting both the requirements of easiness and precision since the results give a gradual (only 10% at a time) increase or reduction of rating. I have tried all the variations and finally arrived at 1-10 scale. Main focus of FMEA should be on PREVENTION AND DETECTION CONTROLS besides searching for opportunities to give positive feedback to Product or Process Designers to do everything to reduce the SEVERITY rating.

Zahirrudin Mahmud

Senior Process Engineer at Flextronics

1-10 . Ranking description was provided by customer because we are doing their product. 🙂

Comments from Lean Six Sigma Community

As long time lean six sigma practitioners, you’ll see that their practices have evolved over time. Take note of the theme: “simplify the decision-making process.”

8 comments

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

This is important because some folks get stuck on the rating system. Please share your rationale behind the different ordinal (ranking) scales.

Rod Morgan

Consultant and Advisor, Process Improvement Strategies, Lean Six Sigma Master Black Belt

Hi Sun! Great question. I believe the first key element is consistency within and across the FMEAs being conducted so that “apple to apples” comparisons can be made. The standard 1-10 scales for severity, occurrence (ex. PPM) and detection should always apply to manufacturing processes and equipment, but I have found with my work in services (ex. Healthcare), 5-point scale is easier to work with and gets the job done. For example, you might consider;

Severity

1-No Effect

2-Slight Effect

3-Moderate Effect

4-Major Effect

5-Severe/Catastrophic

Occurrence

1-Yearly

2-Monthly

3-Weekly

4-Daily

5-Hourly

Detection/Control

1-Always

2-More Likely

3-50% of the time

4-Less Likely

5-Never

For “one offs” where I am working on a project with a team and these types of tools have not been institutionalized by their organization, I might even have them discuss the ratings and come up with their own description and tables, thereby making them a part of the rating system and, hopefully, better at completing the FMEA since they helped to develop the “standard”. When it comes to RPN “debates”, a quick decision to “meet in the middle” and agree is always best.

At the end of the day, this is a highly subjective exercise in services but fabulous in driving out the hidden (buried) factory. I think that each tool or method is there to tell a story and if that story gets told, does it really matter whether we crossed every “t” and dotted every “I”?

Thank you for starting this thread… great topic and FMEA is one of my favourite tools… you can teach a team to use it in 30 minutes and then “let them loose” on the process map and the “chase” begins… what fun!

Like (2)

Evans W., Rai Chowdhary like this

Rai Chowdhary

Business Executive

Sun,

You have received some very good comments already; I will only add that the scales for each – Severity, Occurrence, and Detection – should be balanced; i.e. all should be 1 to 5, or 1 to 10, etc. I have seen some teams use scales of 1 to 7 for Detection, and 1 to 10 for Severity and Occurrence. This practice can lead to confusion and misleading comparisons.

Hope that helps.

Rai.

Rod Morgan likes this

Sr. Manager, Master Black Belt in Quality-by-Design, Agile Lean Six Sigma and Design Thinking

Rod, Very Practical approach! Rai, Thanks for the important bias issue that may arise.

Rod Morgan likes this

Muhammad Siddique

Lean Six Sigma

Numbers make sense if one can justify through granularity or resolution of differences. What I mean if you can differentiate between two numbers like 1 & 3, use it if you cannot then use higher numbers so that numbers represent the difference in severity,Occurrence & Detection like 1,5,9

Sun Kim, Rod Morgan like this

Nabil Gharbieh

Corporate Lean Six Sigma Account Leader for Xerox Global Accounts at Xerox

After over 20 years of conducting and teaching different kinds of FMEAs I found that most of the time if the scale level description does not have a clear effect such (if gap is less than a certain value or if we got less than 15 responses, or ..), then there is a big area of disagreement on how sever or not is the effect. The scale that always worked was :

1- No effect

3- Measurable effect

7- Sever affect

9-Catastrophic effect (system shut down).

With this scale you remove a lot of the non value add discussions about the size of the effect and utilize the team time in a more productive way.

Like (3)

Evans W., Rod Morgan and 1 other like this

Ashok Motwani

Lean Sigma Champion at Plexus Corp.

Sun,

Just saw this discussion, here are my 2 cents worth.

Rod has correctly mentioned that the numbers in ranking severity, occurance and detection should relate to your environment. I have personally used 1-10 ranking and 1,3,7,10 ranking scale in different industries depending on what product/process we were developing FMEA for.

I generally have always started the FMEA meetings by inducing the conversation regarding the RPN calculation to prioritize the action on improving controls. This leads into developing an operational definition for each level of ranking scale based on practical examples directly related to the issue or indirectly system being evaluated. (I kind of said the same thing in different words what Rod & Nabil mentioned above).

Hope this helps. Comments appreciated.

Ashok