Category Archives: Training

Auditing, Improvement, ISO, Training

Imitation Game – Faking 9001

Leave a comment
Faking 9001
Highest Form of Flattery

Faking 9001 or, “How to Fake a Management System”.

Yes, you read that right. I could say,”How to fool an ISO auditor”, or “How to pretend to have a management system” – but I think you get the idea.

I strive to disambiguate. I should also strive to speak more plainly. To you I present an introduction to my latest book. It is a work of comedic fiction (a flavor of fiction perpetuated solely in my own mind).

Ostensibly, it is a “How To” book. I’ve considered and rejected several titles with numbers. People like that kind of thing and I’ll put in more naming effort once I know what kinds of numbers are involved. Maybe, “9001 Ways to Fake It”.

I hope it isn’t more than seven.

In the real world, things happen; cogs turn and pistons pist. An auditor steps into the world for a day or three. She never sees the real world – she sees the manifestation of history in the forms of records, documents, an array of physical objects and people, and through discussion.

All of this is interpreted through a human brain riddled with genetic propensities toward cognitive bias. And a desire to get home.

Faking it well is helped by a solid understanding of the audit process.  That, and only slightly more than a vague familiarity with the standard.

Records

Quality records are the means by which an auditor sees much of the system. We put an inordinate amount of credibility in them, most of us.

So, there’s a minimum set we expect to see. The standard refers to them clearly.

From them we deduce the  existence of things in the past. Who knows if they’ve actually taken place.

  • Management Reviews
  • Internal Audits
  • Corrective Actions
  • Preventive Actions
  • Nonconforming material slips
  • Letters from customers; quotations
  • Purchase orders

There are more, the book has them. Actual management systems have them. Fake ones have them too.

But, a record is not an actual event. It is toner on paper. Or it consists of electrons bouncing on a liquid crystal display – all as real as crystal meth dreams behind blinking eyes.

Documents

Process and product documents sufficient to satisfy the sample size. Keep it simple; a few customers, a few simple designs.

Process documentation

A quality manual, six basic procedures, maybe a few more “tier twos” to make it look good. A work instruction or three.

Product documentation

Some drawings, product specifications, customer specifications. Cosmetic criteria…

Metrics

The coup de grâce. Charts, baby – charts. Bar charts, Pareto Charts and, the real killers – control charts.

See the previous posts about metrics – Customer satisfaction is vital. Get that – well, make that.

It’s got pretty colors and goal lines, right? Must be true. Don’t forget the goal lines.

Packaging it all

Binders – lots of binders. And logs, your going to need logs. And a few file folders. Just a few.

The idea is to provide easy access to the information in an organized appearing way. They are to be delivered to the auditor in a conference room. Stacks of binders.

Keep that person in the conference room for as long as possible. Time is your friend – use it up. Make sure there is a fridge in there, and snacks.

Maybe a nice mini-bar.

Records binder – this is where you’re going to assemble your management reviews and internal audit records. All of that stuff. Might need two of those.

Documents binder – Quality manual and the tier twos. Create a master list. Don’t make everything revision A. Higher revision levels indicate a more healthy system. Depending on your document control procedure you might need records to support the other revisions, like a DCN form – try not to over-complicate your life for no reason.

A file drawer of Purchase orders to vendors. Less vendors is going to be easier than more vendors. Have an approved vendors list. Don’t make it too hard to get on that list! Download ISO certs from those vendors you happen to have in the file drawer. Or a survey that they filled out (use a couple different colored pens at least) – do both, why not. Approve those P.O.’s!

A file drawer of completed sales orders. What should be in there? I don’t know… a quotation, a purchase order (with approvals from production saying it is possible to fulfill it), a job traveler with completed and signed-off steps, an inspection report, shipping documents. What? A letter from the customer saying how happy they were?? How did that get in there?

Throwing the Bones

Nobody is perfect. Don’t be a nobody; be a gambler – be someone that survives an audit. In this case, survival means getting a certificate on the wall.

A nonconformance isn’t going to keep you from it. You’re going to get one (technically you’ll be giving them) and then you’re going to analyze it, find the root cause of how it happened, you’re going to address that cause with some action to keep it from happening again, and finally, you’re going to verify that what you did, worked.

Try not to let anyone find all of that before the nonconformance is issued…

You’ll put all that into a package, send it to your Registrar, they will love it – and send you a Certificate.

What bones?

Minor bones, of course. Toss out Phalanx and phalanges – not femurs. Offer a femur and you’ll feel like you’re on the receiving end of “9001: A Space Odyssey”.

The auditor needs to feel like something was accomplished.

To an auditor, a nonconformance is like a souvenir ear they can bring back to their underground lair, “See, I was there…” he holds the trophy up for the other trolls to see by the light of their sooty fire – “And I got this!”

They grunt appreciatively and slap his hunched back, sharing in the victory.

Some examples:

  • A document in your document binder that has a revision that isn’t the same as what is in your master list (at the front of the document binder). Put two – no, three – maybe he’ll find one. But don’t put a rev 3 on a document and a rev 2 in the master list – that indicates a bigger problem and might earn you more digging and possibly a re-audit. You only want to put on one show.
  • A non-conformance that doesn’t have root cause filled out. Make it a simple one – something with an obvious cause. Two of these should do it.
  • Find an inspection record and re-copy it and in so doing obscure a diagonal half of it.
  • Assuming the impact is minimal, a piece of test equipment with a date that is unreadable. Don’t overdue that, it might earn a visual re-audit.
  • Also to avoid – unidentified items on your (hopefully very small) production floor. An accumulation of these will earn you a visual re-audit. Avoid anything that to verify that it is all fixed and fine would require someone to physically see the result in action.
  • Get creative. I am often surprised at peoples’ creativity.

What else?

Well, I haven’t really addressed how to stage a production area with it’s product and operators and all of it’s 7.5 implications. It is possible to do it. I can’t give away the whole show here!

Or interviews with the rank and file regarding the Quality Policy. Piece of cake.

Design. That seems tricky but it really isn’t.

Fooling one of us isn’t terribly hard – I wouldn’t be surprised if it happens quite often. But the point is, in the process of packaging and obfuscating one often implements the thing to good effect despite the frivolous ambition. It is neither the best nor the worst that is lost in the ambiguities of translation; truth is somewhere in the middle.

And isn’t it true that success is 80% perspiration and 20% making things up? I am pretty sure that’s how it goes.

Give it a shot. What could go wrong? Hrm – well, it might be against some law. Fraud maybe. I stayed in a Holiday Inn once… I’d ask your legal team about the idea first (then do it anyway, of course).

But if you pull it off, and it’s all in place – maintenance isn’t such a stretch, is it?? I mean, you went through all that trouble.

Next up: “How to fake Continual Improvement”

😉

 

 

9001:2015, ISO 9001, Training

First Things First: Implementing 9001

2 Comments

Sky is the LimitAre you in the midst of implementing 9001? Are we ever not in the midst of it?

I’d like to begin a series of posts on implementing 9001 – or any management system, from scratch. It will be focused on 9001, based on the draft of the next version of it in fact, but it should be well applicable to any management system; quality, environmental or information security – pick your flavor.

Where should one begin? Document control? Internal audits? The bones of a nice corrective action system? So many choices…

I audit these systems on a daily basis, I finding the folly alongside the fabulous. I’ll do what I can to make it a worthwhile read.

If you’re a current implementor, please – let us know any challenges you are having.

I should probably make apologies to Stephen Covey for using his book title (but it does provide me an opportunity to plug a very good resource – click it!).

18001, 9001:2015, ISO 14001, ISO 27001, ISO 9001, Training

Training Effectiveness Guide

Leave a comment

An Apple a DayVerification of training effectiveness is required by most of the frequently used management system standards. Even companies with well-established training programs struggle with how to evaluate and moreover, how to realize value from their efforts.

Naturally, I see many methods of verification of training effectiveness in my audits – sometimes inspiring, sometimes – not so much. At the very least, it all makes me think. Without giving away anyone’s secrets, I thought I’d share my own thoughts.

The Requirement for Training Effectiveness

Verification of training effectiveness shows up in several of the popularly implemented standards; 9001:2008  (and in the DIS of the 2015 version), 13485:2003 (Medical devices), OHSAS 18001:2007 (Safety), and 27001:2013 (Information Systems).

You may note the omission of ISO 50001:2011 (Energy management) – verification of training effectiveness isn’t there, at least not directly.

And 14001:2004 (Environmental management), the current version, does not have a requirement to evaluate training effectiveness (old school), but the second draft (CD2) does have it. So, if you’re in that world – best start considering how you’re going to meet that need.

I’ll use the text from 9001 except where there’s a noteworthy difference in one of the other standards:

Competence, training and awareness
The organization shall

a) determine the necessary competence for personnel performing work affecting conformity to product requirements,

b) where applicable, provide training or take other actions to achieve the necessary competence,

c) evaluate the effectiveness of the actions taken,

d) ensure that its personnel are aware of the relevance and importance of their activities and how they contribute to the achievement of the quality objectives, and

e) maintain appropriate records of education, training, skills and experience…

OHSAS 18001 (Safety) is particularly succinct (note how it also addresses risk):

The organization shall identify training needs associated with its OH&S risks and its OH&S management system. It shall provide training or take other action to meet these needs, evaluate the effectiveness of the training or action taken, and retain associated records.

The main point for our discussion today is that people get trained, and the effectiveness of that training must be evaluated.

Something as simple as this:

Basic Requrement

Ideally, though, if the training wasn’t effective one might question the training method. Another natural response, not unwholly unwarranted, is to assume the person being trained is at fault; that he or she just didn’t “get it”. This is, in my experience, the typical reaction –  and usually without justification.

But it need not be so. This is a larger topic, and mostly beyond the detail level of this post, but books and careers are made on the study of learning. If you’re interested in a fairly detailed work on that topic there’s How Learning Works: Seven Research-Based Principles for Smart Teaching. It’s geared more toward a university environment but it gives a fairly thorough understanding of the challenge.

For those of us juggling the job you were hired for plus the task of training, a great book choice is Design For How People Learn (Voices That Matter) by Julie Dirksen. It is certainly a practical and useful guide to the fundamental concepts of instructional design.

Failing that, simply showing someone a powerpoint or loading a VHS tape doesn’t necessarily have any chance to provide an effective learning experience for some people – even if you do make them sign a piece of paper afterwards saying their eyes were mostly open most of the time.

A better flow diagram may look like this:

Slightly more advanced training flowchart

Or, how about this? We’ve added a “lessons learned” that will modify the materials or methods for next time.

More enlightened Essentially, what I’m saying is before you train – evaluate what types of training would be suitable for both the task and the individual. Then, execute the training and make the evaluation of effectiveness. With this information in hand, go back and tweak the materials while providing any feedback to the training methodology for next time – and provide any needed retraining.

This is simply an extension of the Plan, Do, Check Act (PDCA) methodology and is at the core of these standards.

Some Specific Methods to Evaluate Training Effectiveness

Let’s look at some common ways to evaluate training effectiveness.

Testing

Tried and true, and most corporations’ go-to method of determining training effectiveness. Written tests do lend themselves well to safety-related training, or clear requirements-based content such as ITAR or even general policies with “dos and don’ts”.

Just be sure that if testing is used that there is a minimum score needed. I’ve reviewed test results and have found individuals that have received fairly close to a zero and still “pass”.

What happens in this situation is that if there is a low hurdle to jump it soon leads to corrections needed or non-conformances found in the process or product.

Typically these initial non-conformances have found the root cause to be related to training and this type of method defect is weeded out early.

Having said that, Testing is a simple solution and is likely going to be in your bag of evaluation options, if done properly.

An On-going Review of Process Metrics

Essentially, the rationale is saying, “Hey, we have trained people, we measure the important metrics and our trends are good; within limits and we are improving where possible.”

It definitely can work. Naturally, the challenge is twofold; defining the meaningful metrics and measuring them in a consistent manner. Goals should be established already in response to The Standards’ other requirements.

If this method is used, be sure to clearly define how it works and have data to back it up. Remember, one of the other requirements is to maintain records to show the verification of training effectiveness – so clearly define what the record is.

Inquiry

It could be as simple as asking the individual who was trained, “How do you think the training went?” – this could be coupled with a short “trip report” (or not). This does tend to have limitations in terms of when it can be used, however. It is well-suited to external trainings or trainings to executive or higher-end technical positions, though it likely can be adapted to any situation.

A Review of an External Certification

Specific to externally performed trainings where the attendee completes a course and is given a post-course test or other plausible evaluation. Again, records would be needed to show this.

At Employee Reviews

A bit tricky in practice since these reviews often take place once a year but it is quite a common solution. Works particularly well for procedure or work-instruction-based trainings.

Similar to the “review of ongoing process” method described above, this technique would predetermine  criteria for success; coupling training that has occurred over the period.

The rationale being that successful completion of tasks and assignments; the day-to-day job would indicate effective training. Again, defining this process in detail, including records, is key to success.

Note that claiming effective training through the lack of problems or defects is not the same thing, especially if no one is formally looking at metrics in the first place.

So, That’s five methods for evaluating training effectiveness to consider:

  • Testing
  • An on-going review of process metrics
  • Inquiry
  • A review of external certification
  • At employee reviews

There are certainly more, particularly software-driven solutions as part of an HRIS application (Human Resource Information Systems) – but typically these need a little coaxing to fit the need (and are generally expensive).

I’m quite interested in other methods you might know of.

In Search of Value

The key though, is that your methods of training effectiveness, whatever they are, give something back to the organization.

The whole point is that it is a waste of resource to take up someone’s time with a training that isn’t absorbed and incorporated into the position.

Question Everything

What is the training trying to accomplish? If it is training to a procedure or work instruction – is the document needed in the first place?

Is it actually a training? Sometimes what companies call “training” is really only a communication – there is a difference, even if subtle.  Typically a training teaches, while a communication informs – the distinction is up to the company to decide. It’s important to make that distinction since if it is training, the hoops come up (as I like to say). Training means evaluate training effectiveness; they go hand-in-hand.

If a process must be documented in order to ensure things go smoothly, then it is worth training people to follow that process. To ensure the training is valuable, ensure that the process is doing what it is supposed to be doing.

From the Top Down

The business is there for a reason; a policy is stated and goals and objectives support that policy.

People are hired with the needed competencies. When there is a gap between what they know coming into the organization and what they need to know to meet the goals and objectives – then training is needed.

Processes and documents are defined to realize the policy – these are often the things that must be learned. And when people know what they are doing – it all works according to plan.

It all points back to monitoring the processes, goals and objectives – this is the key to providing training that provides value. Verifying that the training does what it is supposed to do – evaluating training effectiveness – is simply protecting that investment in time and money.

 

Thanks for reading, I hope it helps – but don’t worry, it won’t be on the test.