Achieve the Holy Grail of Training Effectivene
A systematic approach to educating pharmaceutical employees
As a pharmaceutical industry training professional, I am regularly
asked, “How do we make training more effective?” It’s an important
question, but it doesn’t have a simple answer.
Training touches most of what we do as an industry. Our operations
are based on two systems—training and documentation—and everything we do
hinges on our people appropriately executing our procedures. Training
is what ties them together. It affects everyone in our businesses, how
well our businesses run, and what results we’re able to obtain. And we
know it’s not working when we see large numbers of deviations and
increasing manufacturing costs.
Training effectiveness results from an effective training
process—probably the least understood process in our industry. Our
industry values meaningful numeric data, and we use it to define,
describe, and understand our processes, but it’s difficult to identify
and provide meaningful numeric data that defines and describes a
training process—especially if the process is suboptimal. Worse, many of
the outcomes of training hinge on an individual’s motivation and
ability to learn, characteristics for which numeric data doesn’t exist.
It’s not impossible to generate data around the training process, but it
tends to be difficult. Anecdotal data from around the industry on
evaluation practices shows that we typically focus on happiness with the
training and/or whether the trainees possess knowledge, rather than on
the success and effectiveness of our process.
The regulations don’t help us much, either. The FDA provides sparse
guidance on training in the regulations, and what the agency does
provide, while helpful, are only criteria to be met, like:
- “Each manufacturer shall have sufficient personnel with the
necessary education, background, training, and experience to assure that
all activities...are correctly performed” 21CFR820.25(a)1;
- “Each manufacturer shall establish procedures for identifying
training needs and ensure that all personnel are trained to adequately
perform their assigned responsibilities” 21CFR820.25(b);
- “Training shall be in the…operations the employee
performs…including the cGMP regulations and written procedures…as they
relate to the employee’s functions” 21CFR211.25(a)2;
- “Training...shall be conducted by qualified individuals on a continuing basis” 21CFR211.25(a); and
- “Training shall be documented” 21CFR820.25(b).
Even if we meet these requirements, there’s no guarantee that our
training is effective. And regulators don’t often look deeply into our
training effectiveness, as long as our process runs without many
deviations and product safety is assured. Training even receives a
somewhat cursory review in inspections unless a particular weak spot is
identified.
In reality, “training effectiveness” results from a combination of
two things: a training system that includes all of the elements of an
effective training process and processes that support the training
system. Both must be present and working together—training effectiveness
suffers if either of these elements is missing.
Elements of a Training System
To develop role-based curricula and training plans,
we need to know who is responsible for performing which roles and what
baseline knowledge, experience, and skills are required.
To ensure its effectiveness, we must first know what training is
needed—our curricula. In simplest terms, curricula define groups of
requirements to be met. We should define curricula for all employees,
including requirements for onboarding (corporate, site, and
departmental), ongoing GMP and safety training, and task training.
Curricula should also define professional development requirements
appropriate for specific positions or levels, such as leadership
development training for management or specific certifications required
for IT, engineering, and project management positions. Requirements for
specific tasks and roles should be relatively standard; we should not
define requirements based on a specific person. Finally, we should
express these as role-based curricula for all levels of employees,
including senior management and contract employees.
Next, we need to determine when and how to provide training and
capture this information in training plans. Our departmental training
plan should document the progression of tasks in the development of an
employee in that role and department, and it should be structured such
that training builds on previous skills acquired earlier in the
development process. Then, in alignment with our defined curricula and
training plans, but based on the individual’s specific experience and
needs, we define an individual training plan, tailored to meet these
needs. (In many cases, tailoring requirements to an individual happens
through performance management and/or a personal development plan.) The
combination of departmental and individual training plans allows for
flexibility while individuals learn the skills to perform the
department’s tasks. What each individual needs to be able to meet the
requirements may vary, but the requirements and expectations should be
the same for anyone performing the same task.
In order to develop role-based curricula and training plans, we need
to know who is responsible for performing which roles and what baseline
knowledge, experience, and skills are required. Combine that with the
information used to build curricula and training plans, and we’ve
identified the need for job descriptions—truly the starting point of our
training system.
A job description should exist for each position, defining the role
and tasks performed, as well as describing the level of performance
required. Defining these expectations enables all the other elements of
the training system. Job descriptions also allow us to assess an
individual’s knowledge, skills, and experience against defined criteria,
which helps to avoid putting individuals into positions for which they
are unprepared. Organizational charts visually structure job
descriptions to show reporting relationships, for the purpose of
determining accountability for functions and operations. Because neither
of these are currently GMP requirements in the U.S., we often consider
them a non-GMP HR function; however, the EU GMPs require job
descriptions and organizational charts—and a company marketing products
in the EU is expected to have them.
3
Because our curricula identifies who to train in what areas, and
because job descriptions describe what employees need to be able to do
when trained, we can now provide appropriate training activities.
Training effectiveness discussions typically focus on training
activities because they’re the most visible piece of the process, and we
usually base our decisions on the delivery of the training activity and
whether or not it met expectations. But basing judgment of our training
system on this element alone overlooks the many other factors critical
to our ability to provide effective training! The quality of the
training activities we design, develop, and deliver directly affects
training effectiveness, but the quality of training results from the
combined outputs of three different process elements: instructional
design, accurate content, and qualified trainers.
click for larger view
FIGURE 1. Components of an effective training process.
Instructional design allows us to define and build appropriate
training. It is a specialized skill set that enables an instructional
designer to break a task into its multiple components and determine how
to effectively teach and assess each one. It begins with a needs
analysis, which identifies what the training must include and
accomplish—and whether training is actually needed at all. Then, the
instructional designer, usually in conjunction with a subject matter
expert, builds the content and practice activities and determines the
most appropriate type of assessment based on the intended use of the
content.
Effective training must provide accurate content. During the
development process, a subject matter expert, and quality assurance
where appropriate, should review training materials for accuracy. This
should be part of a training content approval process, which should also
include a system to control the approved content and any changes that
have been made to this content. Where approved content is used on a
continual basis, it should undergo periodic review and should also be
reviewed when system or process changes are made, to ensure continued
accuracy and appropriateness.
Next, qualified trainers must deliver our training. Qualified
trainers should be subject matter experts who are able to properly
perform the task, answer questions about it, and assess trainee
performance. They also need interpersonal and training delivery skills.
Where they deliver training will define the skills sets they need;
providing one-on-one task training in an operational environment is very
different than providing group training in a classroom. Trainers
without training skills are just as detrimental to training
effectiveness as those who have no knowledge of the content or process
for which they’re providing training.
Once we’ve delivered the training, we need to ensure that it
accomplishes our desired outcomes. To do this, we assess and evaluate
both the trainee and the training program itself. When we assess and
evaluate, we want trainees to demonstrate that they have the desired
knowledge, have developed and can use the skills required to perform
tasks, and, where necessary, can react appropriately to different
situations. The most common assessment is a test that demonstrates
knowledge, but those don’t allow trainees to demonstrate their ability
to perform appropriately. The training program is typically evaluated
using Donald Kirkpatrick’s Kirkpatrick Model, which helps us determine
whether or not we obtained the desired results, and if not, what areas
to investigate to understand why we didn’t and what to do about it.4
Last, we need to be able to provide records for our training system.
We commonly think of training records as proof of training completion,
but we should also include content approvals and change control,
assessment results, curricula, training plans, and more. These records
help demonstrate that the system is effective and training is happening
appropriately, as well as providing proof of qualification to auditors.
What Processes Support the Training System?
If there is any doubt regarding management’s role
in and effect on training, the FDA’s “Guidance for Industry: Quality
Systems Approach to Pharmaceutical CGMP Regulations” clearly assigns
responsibility for all aspects of training to management.
The most basic support to any GxP system is a standardized process
documented in our operating procedures, because an effective system
requires consistency of application and standards for operating. For
training, our procedure should include a standard approach for how we
perform and document training, how we qualify trainers, how we process
and maintain our training documentation, and what we define as passing
scores on knowledge assessments, along with a standardized on-boarding
process through which all GMP employees are processed. Where
appropriate, we should also include standardized time frames for
completion in our training procedure.
Appropriate operational procedures are critical to an effective
training process, because we derive much of our training content from
them. Procedures should clearly define processes, responsibilities, and
tasks to an appropriate level of detail, providing standard performance
requirements for each process. Ambiguity of performance requirements or
responsibilities, or the lack of documented processes, causes training
effectiveness to suffer—it’s difficult to teach, learn, or assess a
process that everyone does differently. However, if procedures provide
on-the-job reinforcement of items learned in training—even wording as
simple as “at this point in the investigation, perform a root cause
analysis to identify the root cause of the problem”—they automatically
improve training effectiveness, because they tie directly to what was
learned and define when to use it.
Systems and repositories—preferably electronic—ensure that we can
manage the process and the associated materials and records, and access
the information we need. While this can be done in a well-managed paper
system, electronic learning management systems and learning content
management systems make this process much easier to manage. For example,
the ability to easily access records enables us to quickly verify an
operator’s qualification on a specific task. Having a location where
people can access controlled, approved training materials and
information provides the same benefit as a controlled documentation
system: access to current, consistent information.
The Most Critical Factor
Of all the system elements, management support has the largest impact
on training effectiveness. So what does “management support” mean? In a
word: expectations. Our management teams should expect the following:
- Training occurs, is done well, and is completed on time;
- Results are measured, reported, and acted on appropriately;
- Training is meaningful, and skills learned in training are appropriately applied to operations;
- All parts of the training system and support processes are in
place and functioning appropriately, and people are accountable for
their parts of the process;
- Training helps meet the goals of the business; and
- Training is part of the job and part of ongoing employee performance management processes.
If there is any doubt regarding management’s role in and effect on
training, the FDA’s “Guidance for Industry: Quality Systems Approach to
Pharmaceutical CGMP Regulations” clearly assigns responsibility for all
aspects of training to management:
“Under a quality system, managers are expected to establish training programs that include the following:
- Evaluation of training needs
- Provision of training to satisfy these needs
- Evaluation of effectiveness of training
- Documentation of training and/or re-training.
When operating in a robust quality system environment, it is
important that managers verify that skills gained from training are
implemented in day-to-day performance.”
5
The guidance also requires senior leadership and management to define
and provide policies, procedures, and desired work culture, and
prescribes these topics for inclusion in our training courses and
content. So, the FDA expects management to have a vested interest in the
content, the quality, and the process of training.
Finally, consider this: The attitude the organization’s leadership
takes toward training—particularly their own training requirements—sets a
powerful example, either good or bad, for the rest of the organization.
Training takes time away from operations and is often viewed as a
necessary evil. But, when employees see leadership attending GMP
refreshers or completing applicable procedural training, they understand
that training is a business activity that has value and is worth the
time. However, if people see their leadership regularly excused from or
habitually late in completing training, they think, “Training isn’t
important enough for management to spend their time on.” Training, in
their eyes, is now a punishment for those at lower levels, one that can
be bypassed by management in favor of other activities.
Achieving the Holy Grail
So this is why improving training effectiveness doesn’t have a simple
answer. We’re dealing with a fundamental business process that touches
many parts of the organization, along with other processes built upon or
around it.
At heart, improving training effectiveness is about problem solving
and continuous improvement, and it will likely lead to changes in
systems, processes, and procedures, as well as training practices. But
finally achieving effective training “nirvana” provides measurable
benefits to the business: Imagine having competent and motivated
employees making fewer errors and better decisions, along with fewer
wasted resources, increased efficiency, and, ultimately, more compliant
and more productive operations. Who’d have thought a necessary evil
could be so positive?
References
- U.S. Government Printing Office. United States Code of Federal
Regulations, 21 CFR 820.25: Quality System Regulation: Personnel.
Updated April 1, 2011. Available at: www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?fr=820.25. Accessed June 7, 2012.
- U.S. Government Printing Office. United States Code of Federal
Regulations, 21 FR 211.25: Current Good Manufacturing Practice for
Finished Pharmaceuticals: Personnel Qualification. Updated April 1,
2011. Available at: www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?fr=211.25. Accessed June 7, 2012.
- European Commission Health and Consumers Directorate-General.
EudraLex Volume 4: EU Guidelines to Good Manufacturing Practice
Medicinal Products for Human and Veterinary Use, Part 1, Chapter 2.
Personnel, Section 2.2. Dec. 2010. Available at: http://ec.europa.eu/health/files/eudralex/vol-4/pdfs-en/cap2en200408_en.pdf. Accessed June 7, 2012.
- Kirkpatrick Donald L. The Kirkpatrick Philosophy. Kirkpatrick Partners website. Available at: www.kirkpatrickpartners.com/OurPhilosophy/tabid/66/Default.aspx. Accessed June 7, 2012.
- U.S. Food and Drug Administration. Guidance for Industry: Quality
systems approach to pharmaceutical CGMP regulations. September 2006.
Available at: www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM070337.pdf. Accessed June 7, 2012.