Category Archives: Improvement

Auditing, Improvement, ISO, Training

Imitation Game – Faking 9001

Leave a comment
Faking 9001
Highest Form of Flattery

Faking 9001 or, “How to Fake a Management System”.

Yes, you read that right. I could say,”How to fool an ISO auditor”, or “How to pretend to have a management system” – but I think you get the idea.

I strive to disambiguate. I should also strive to speak more plainly. To you I present an introduction to my latest book. It is a work of comedic fiction (a flavor of fiction perpetuated solely in my own mind).

Ostensibly, it is a “How To” book. I’ve considered and rejected several titles with numbers. People like that kind of thing and I’ll put in more naming effort once I know what kinds of numbers are involved. Maybe, “9001 Ways to Fake It”.

I hope it isn’t more than seven.

In the real world, things happen; cogs turn and pistons pist. An auditor steps into the world for a day or three. She never sees the real world – she sees the manifestation of history in the forms of records, documents, an array of physical objects and people, and through discussion.

All of this is interpreted through a human brain riddled with genetic propensities toward cognitive bias. And a desire to get home.

Faking it well is helped by a solid understanding of the audit process.  That, and only slightly more than a vague familiarity with the standard.

Records

Quality records are the means by which an auditor sees much of the system. We put an inordinate amount of credibility in them, most of us.

So, there’s a minimum set we expect to see. The standard refers to them clearly.

From them we deduce the  existence of things in the past. Who knows if they’ve actually taken place.

  • Management Reviews
  • Internal Audits
  • Corrective Actions
  • Preventive Actions
  • Nonconforming material slips
  • Letters from customers; quotations
  • Purchase orders

There are more, the book has them. Actual management systems have them. Fake ones have them too.

But, a record is not an actual event. It is toner on paper. Or it consists of electrons bouncing on a liquid crystal display – all as real as crystal meth dreams behind blinking eyes.

Documents

Process and product documents sufficient to satisfy the sample size. Keep it simple; a few customers, a few simple designs.

Process documentation

A quality manual, six basic procedures, maybe a few more “tier twos” to make it look good. A work instruction or three.

Product documentation

Some drawings, product specifications, customer specifications. Cosmetic criteria…

Metrics

The coup de grâce. Charts, baby – charts. Bar charts, Pareto Charts and, the real killers – control charts.

See the previous posts about metrics – Customer satisfaction is vital. Get that – well, make that.

It’s got pretty colors and goal lines, right? Must be true. Don’t forget the goal lines.

Packaging it all

Binders – lots of binders. And logs, your going to need logs. And a few file folders. Just a few.

The idea is to provide easy access to the information in an organized appearing way. They are to be delivered to the auditor in a conference room. Stacks of binders.

Keep that person in the conference room for as long as possible. Time is your friend – use it up. Make sure there is a fridge in there, and snacks.

Maybe a nice mini-bar.

Records binder – this is where you’re going to assemble your management reviews and internal audit records. All of that stuff. Might need two of those.

Documents binder – Quality manual and the tier twos. Create a master list. Don’t make everything revision A. Higher revision levels indicate a more healthy system. Depending on your document control procedure you might need records to support the other revisions, like a DCN form – try not to over-complicate your life for no reason.

A file drawer of Purchase orders to vendors. Less vendors is going to be easier than more vendors. Have an approved vendors list. Don’t make it too hard to get on that list! Download ISO certs from those vendors you happen to have in the file drawer. Or a survey that they filled out (use a couple different colored pens at least) – do both, why not. Approve those P.O.’s!

A file drawer of completed sales orders. What should be in there? I don’t know… a quotation, a purchase order (with approvals from production saying it is possible to fulfill it), a job traveler with completed and signed-off steps, an inspection report, shipping documents. What? A letter from the customer saying how happy they were?? How did that get in there?

Throwing the Bones

Nobody is perfect. Don’t be a nobody; be a gambler – be someone that survives an audit. In this case, survival means getting a certificate on the wall.

A nonconformance isn’t going to keep you from it. You’re going to get one (technically you’ll be giving them) and then you’re going to analyze it, find the root cause of how it happened, you’re going to address that cause with some action to keep it from happening again, and finally, you’re going to verify that what you did, worked.

Try not to let anyone find all of that before the nonconformance is issued…

You’ll put all that into a package, send it to your Registrar, they will love it – and send you a Certificate.

What bones?

Minor bones, of course. Toss out Phalanx and phalanges – not femurs. Offer a femur and you’ll feel like you’re on the receiving end of “9001: A Space Odyssey”.

The auditor needs to feel like something was accomplished.

To an auditor, a nonconformance is like a souvenir ear they can bring back to their underground lair, “See, I was there…” he holds the trophy up for the other trolls to see by the light of their sooty fire – “And I got this!”

They grunt appreciatively and slap his hunched back, sharing in the victory.

Some examples:

  • A document in your document binder that has a revision that isn’t the same as what is in your master list (at the front of the document binder). Put two – no, three – maybe he’ll find one. But don’t put a rev 3 on a document and a rev 2 in the master list – that indicates a bigger problem and might earn you more digging and possibly a re-audit. You only want to put on one show.
  • A non-conformance that doesn’t have root cause filled out. Make it a simple one – something with an obvious cause. Two of these should do it.
  • Find an inspection record and re-copy it and in so doing obscure a diagonal half of it.
  • Assuming the impact is minimal, a piece of test equipment with a date that is unreadable. Don’t overdue that, it might earn a visual re-audit.
  • Also to avoid – unidentified items on your (hopefully very small) production floor. An accumulation of these will earn you a visual re-audit. Avoid anything that to verify that it is all fixed and fine would require someone to physically see the result in action.
  • Get creative. I am often surprised at peoples’ creativity.

What else?

Well, I haven’t really addressed how to stage a production area with it’s product and operators and all of it’s 7.5 implications. It is possible to do it. I can’t give away the whole show here!

Or interviews with the rank and file regarding the Quality Policy. Piece of cake.

Design. That seems tricky but it really isn’t.

Fooling one of us isn’t terribly hard – I wouldn’t be surprised if it happens quite often. But the point is, in the process of packaging and obfuscating one often implements the thing to good effect despite the frivolous ambition. It is neither the best nor the worst that is lost in the ambiguities of translation; truth is somewhere in the middle.

And isn’t it true that success is 80% perspiration and 20% making things up? I am pretty sure that’s how it goes.

Give it a shot. What could go wrong? Hrm – well, it might be against some law. Fraud maybe. I stayed in a Holiday Inn once… I’d ask your legal team about the idea first (then do it anyway, of course).

But if you pull it off, and it’s all in place – maintenance isn’t such a stretch, is it?? I mean, you went through all that trouble.

Next up: “How to fake Continual Improvement”

😉

 

 

Improvement, Metrics

Quality Metrics Monday – June 2015

Leave a comment

Quality Metrics
Quality metrics are a vital part of any management system. They allow for data-driven decision making. They allow the individuals within an organization to be able to answer the question, “How’s it going?” – without resorting to “gut feel”.

Gut feel is horribly unreliable, our brains lie to us (read the very good You Are Not So Smart: Why You Have Too Many Friends on Facebook, Why Your Memory Is Mostly Fiction, and 46 Other Ways You’re Deluding Yourself, for examples).

I’ve covered quality metrics in earlier posts, particularly with regard to goals and objectives. That two-parter grew into some specific metrics for a Human Resources department and I’d like to add more departments as well.

Frankly, it got away form me a little. Hey, it happens… Let us begin again.

What are Quality Metrics?

Quality metrics, to summarize and perhaps pull a different tack, are the basis for continual improvement. Continual improvement is a large component of what compliance to the standard means. To use an oft quoted phrase, “If you don’t know where you are, a map won’t help. ” (attributed to software engineering legend Watts Humphrey).

  • Metrics let you know where you are – goals, objectives tell you where you want to be.
  • Metrics are measurable, and they are often numeric.
  • Metrics are sometimes called KPI’s (Key Process Indicators).
  • Metrics can be called Measurables (because they should be measurable).

The Minimum Set

When we audit, we’re looking for a minimal set of these measurables. I want to let you know that set, so your life – and mine, can be a little easier. And no, these aren’t set in stone, most of them aren’t anyway – but you should have something like these.

You should have something.

The other influence on the minimum set is going to be the Quality Policy of the company. Generally however, the most basic quality policy invokes the necessity for the metrics I will discuss below.

If your quality policy has something particular, like “… happy employees” then part of the set should have some way to measure that (Employee Turnover, for example). Or if the policy addresses “… a vital part of the community”, then your collection of metrics should include a measurement for that (participation in community outreach programs, perhaps).

To be fair, the standard is only specific about one metric, and metrics are only one  way to measure an objective. And where 9001 isn’t specific, it is so general that justifying the absence of a metric or the objective it measures get’s very… convoluted.

And this measuring of objectives is important because the gospel, according to 5.4.1, says that objectives shall be established at all relevant functions and levels within an organization. Here’s the chapter and verse:

5.4.1 Quality objectives
Top management shall ensure that quality objectives, including those needed to meet requirements for product…, are established at relevant functions and levels within the organization. The quality objectives shall be measurable and consistent with the quality policy.

Objectives shall be measurable… okay, I can play devil’s advocate too; “accomplished or not accomplished” – that’s a measurement.

Fine, that works – and sometimes it makes complete sense. But sometimes it is, at best, a short-cut, and as my dear-old-dad used to say, “You’re only hurting yourself”.

Metrics can heal. I said the standard only specifies one metric, so here it is:

8.2.1 Customer satisfaction
As one of the measurements of the performance of the quality management system, the organization shall monitor information relating to customer perception as to whether the organization has met customer requirements. The methods for obtaining and using this information shall be determined.

NOTE Monitoring customer perception can include obtaining input from sources such as customer satisfaction surveys, customer data on delivered product quality, user opinion surveys, lost business analysis, compliments, warranty claims and dealer reports.

So, an auditor wants to see that. What is your customers’ perception of how your company is meeting their requirements?

Not, “Our customers love us!” verbally expressed.

Not, “We hardly get any complaints.”

And no, not the blank deer-in-the-headlights stare followed up with, “So, would you like another doughnut?” (Yes. Yes, I would very much like another doughnut).

It can be a list of customers with a percentile score next to their name.

It can be a list of customers with a stoplight metric; green/yellow/red. Or little smiley faces. But behind the pretty lights and the unblinking eyes should be data.

How do you know? Like the notes says after the requirement in 9001:2008, “input from sources such as customer satisfaction surveys, customer data on delivered product quality, user opinion surveys, lost business analysis, compliments, warranty claims and dealer reports” – whatever your creative genius comes up with. But something.

I’ve seen Survey Monkey put to good use for this. Reply cards sent with the product generally have a horrible return rate and they are almost not worth the effort. On-line options tend to do better.

Have an overall score covering all customers, or individual scores – but be prepared to justify your choices. If you exclude some customers then explain why.

Action Plans to Meet Goals

Along with that metric there should be a Goal. And – this is important – if the goal is not being met then there should be an action plan to achieve the goal.

The requirement for this is manifold and at the core of the standard. Here’s one:

8.2.3 Monitoring and measurement of processes
The organization shall apply suitable methods for monitoring and, where applicable, measurement of the quality management system processes. These methods shall demonstrate the ability of the processes to achieve planned results. When planned results are not achieved, correction and corrective action shall be taken, as appropriate.

Here’s another:

8.5.1 Continual improvement
The organization shall continually improve the effectiveness of the quality management system through the use of the quality policy, quality objectives, audit results, analysis of data, corrective and preventive actions and management review.

One more:

8.4 Analysis of data
The organization shall determine, collect and analyse appropriate data to demonstrate the suitability and effectiveness of the quality management system and to evaluate where continual improvement of the effectiveness of the quality management system can be made. This shall include data generated as a result of monitoring and measurement and from other relevant sources.

I could go on…

Back to the Minimum Set

So, customer satisfaction is one. “Check.”

Here are some others we expect to see (depending on your business, of course).

On Time Delivery – I could be part of customer satisfaction. Have a way to measure it that is consistent.  Often MRP systems have a problem with making a sensible number out of the data it collects. There is absolutely no problem with normalizing the data afterwards. Do what you have to do to get a number that is consistent. It’s value is that it is a means toward continual improvement.

And, please, don’t tell me it is 100% unless you’re selling unrealistic expectations as an actual business. I will be happily surprised if you can achieve that goal – though yes, it does happen routinely for some companies. Make it a stretch goal, but start low and change it gradually as your process can consistently reach it.

Scrap – how much material are you not using. Some folks use this along side a “re-use” metric. This is typically a dollar value, but for some commodities it is a weight or volume metric to save frequent currency or value conversions.

Machine down-time – Can be tied with a “Preventive Maintenance tasks on time” report as they are often related.

The rest of the organization.

So far I’ve listed mostly production and operation-related metrics and those should be a “slam dunk” for most organizations.

The other departments, such as Sales, Purchasing, QA and HR have a typical set as well (covered HR pretty well in a  previous post).

Again, I am pointing out basic and typical minimum expected sets.

Sales

Conversion Rate – converting quotations into purchase orders

Time to quote – in days, usually.  Add your own fine print.

Purchasing

Time to place – Time between purchase request and placing of the order. This might include a sub-metric; Time to approve.

HR

I provide an extensive list here, but at a minimum, there should be a focus on the timing-related items, such as Time to Indoctrinate/Perform initial training.

QC

Hold times – inspection wait times, for example – we often find Cp, Cpk, etc. – but I think the product metrics are pretty well understood. I could cover them in another post.

QA

All of the the QMS related items, particularly:

Internal Audits on Time

Open NCR/CA/PA aging

Until Next Time

Please, don’t get the idea that all of these metrics are expected all of the time, other than customer satisfaction metrics.

But, most organizations could use a few more, rather than a few less. Certain metrics come and go. They get crushed, and another one takes its place.

At the end of the day they should provide value to the company, and generally that can’t be determined until they are tried on for a while.

Quality metrics are the lenses by which we see how the organization is working; make them work for you. Empty frames don’t do anyone any good.

 

 

 

Improvement, Metrics, Quality Tools

Metrics – How To’sday

1 Comment
W. Edwards Deming

Collecting Metrics and tracking them is widely considered to be key to improvement – but the specifics are often elusive, particularly for relatively small organizations.

And, naturally, acting appropriately to the results is critical to improvement as well. We’ll touch on that, too. Eventually.

These posts (there will be at least two of them on the topic), while not providing an exhaustive list of departmental metrics, will provide at least a starting point for many.

The terms themselves are often used interchangeably, even if they clearly aren’t synonymous; metrics, KPIs, measurables, goals and objectives, dashboards…  where to begin?

Define Your Terms

Beyond dictionary and accepted terminology, ISO 9000:2005 “Quality management systems — Fundamentals and vocabulary” has formal definitions for a few of the key terms we’ll be using.

3.2.5 quality objective – something sought, or aimed for, related to quality (3.1.1)

3.1.1 quality – degree to which a set of inherent characteristics (3.5.1 – “distinguishing feature”) fulfils requirements (3.1.2 – “need or expectation that is stated, generally implied or obligatory”)

The next question is, “How does one measure progress against an objective?”

Fortunately, the 2nd note in ISO 9000:2005 under the term (3.5.1) distinguishing feature (a component of an objective) says, “A characteristic can be qualitative or quantitative.”

Helpful or not, Note 3 goes on to state:

NOTE 3 There are various classes of characteristic, such as the following:

  • physical (e.g. mechanical, electrical, chemical or biological characteristics);
  • sensory (e.g. related to smell, touch, taste, sight, hearing);
  • behavioral (e.g. courtesy, honesty, veracity);
  • temporal (e.g. punctuality, reliability, availability);
  • ergonomic (e.g. physiological characteristic, or related to human safety);
  • functional (e.g. maximum speed of an aircraft).

Essentially, you can measure performance against an objective any way you wish – that performance measurement is what I am calling a metric.

Why have Metrics?

If an organization is going to spend time doing anything, particularly with metrics, then there should be a reason.

So, as with most Management System realities the two main reasons tend to be “Because it’s required” and “Because it makes sense”.

Let’s look at the latter first.

Because it makes sense.

Does it? You may be surprised to learn that not everyone agrees with managing via metrics, or at least, they would say that not everything of importance can be measured.

W. Edwards Deming (October 14, 1900 – December 20, 1993) was one who said it, in fact. Don’t know who Deming is?  Complicated man – almost legendary, possibly self-contradictory (the lens of history often does that to great men). He is thought to be one of the key figures helping Japan rise from the literal ashes after WWII. He gained popularity by expanding and championing concepts of Statistical Process Control, Plan-Do-STUDY-Act (PDSA) – and much, much more. I’d need a book to do him justice.

Side note – Deming was not a fan of the now common “Plan-Do-Check-Act” (PDCA). Notice the difference; “Study” vs “Check”. He called PDCA a “corruption”. His PDSA takes Study literally as deductive and inductive learning – it is built into a learning and improvement cycle. We’re getting a little far from the main topic, but I could expand on this more if anyone would like me to – leave a comment.

Here is a list of Key Deming Books. If you only have time to read one, many would recommend “Out of the Crisis“.

While Deming understood that not everything is measurable, and was quick to point out the dangers of slogans, thoughtless quotas and targets (such as “zero defects”) he still believed a system must be managed and that “There is no substitute for knowledge.” 

For the mere mortals among us, however, a reasonable place to start is by carefully selecting a set of solid process measurements to determine if something is improving – or not.

Another popular way to put it, “If it can’t be measured, it can’t be improved.” (sounds a bit like a slogan, doesn’t it? My apologies to Mr. Deming). That quote, by the way, isn’t attributed to Deming – it is a quote from Lord Kelvin (actually “William Thomson” of Belfast, 1824-1907). He was a mathematical physician and father of the first and second Laws of Thermodynamics. Next time you hear someone say Deming said it, please do let them know how unlikely that would be.

Anyway, what I’m saying is, “A map doesn’t help you if you don’t know where you are.” Another slogan… That one, by the way is probably based on a similar quote by Lewis Carroll, an English writer, mathematician, logician, Anglican deacon and photographer. His went, “If you don’t know where you are going, any road will get you there.”

Did I make my point? What was the point? Do metrics make sense? I’m leaning toward “Probably”. Let’s circle back after we look at some metrics – see if you think they’d make sense.

We’ll address the first reason, “Because it’s a Requirement” – next Tuesday. I’ve got more Deming to read – I can’t decide if he was a genius or if he was insane… typical.

Thanks.

Sal