A Seven-Step Process for Evaluating the Impact of Performance Improvement Interventions
by Ingrid Guerra-López, PhD

For evaluation to be worth the resources it consumes, it must enable decision makers to make sound decisions based on relevant, reliable, and valid data that lead to improved performance. It is from here that all evaluation efforts stem. All components of the evaluation must be aligned with those objectives and expectations that the organization values and the decisions that will have to be made as a result of the evaluation findings. The key ingredient for successfully collecting relevant, reliable, and valid data is alignment. Below is a brief overview of this seven-step evaluation process, as illustrated by Figure 1.

Figure 1. Alignment Between All Elements of Evaluation (Guerra-López, 2007)

1. Identify Key Stakeholders and Expectations
The stakeholder groups include those who will be making decisions either throughout the evaluation process or directly as a result of the evaluation findings. Also important are those who will be affected by the evaluation—either in the process or potentially as a result of the findings. Including this group will make the implementation of the evaluation plan much easier.

Stakeholder expectations are the basis for your contract, whether verbal or written, and should explicitly articulate what is expected of you (as well as of the stakeholders!). If you feel expectations are unreasonable, this is the time to discuss, educate, and come to a consensus—not after you have completed, what you believe is, a successful evaluation.

2. Determine Key Decisions and Objectives
Ask the stakeholders to articulate what decisions will be made as a result of your findings. This discussion should be tied to the objectives that must be reached—everything within the organization must contribute toward those objectives. The relative worth of any intervention or solution is primarily contingent on whether it is helping or hindering the achievement of organizational objectives. If stakeholders have difficulties focusing on results, ask “If that happens, what would be the result? How would you know?”

3. Derive Measurable Indicators
The heart of your evaluation plan will be to gather the data required to answer the evaluation questions. People often end up making perilous judgments based on wrong or incomplete data, particularly when they try to force connections between inappropriate data (just because it happens to be available) and the decisions to be made. The data you will seek to collect are essentially about key performance indicators.

4. Identify Data Sources
With a list of specific indicators for which to collect data, you must first determine where you can find those data. The data drives the appropriate source. You can likely find the data that you are looking for right in your own organization. Some excellent sources include strategic plans, annual reports, project plans, consulting studies, and performance reports, to name a few. The Internet and other technologies also offer unprecedented access to useful data.

5. Select Data Collection Instruments
The right data collection tools are a function of the data you are seeking. Likewise, the utility of data collected is a function of the methods you select. When evaluators limit the data they collect by employing an overly narrow set of observation methods because they do not know how to use others, their data set will not be complete and, in turn, their findings will not be valid. Be sure to make your selection based on the pros and cons of collection instruments, regarding important criteria such as appropriateness of the instrument for the required data, time, characteristics of sample, comprehensiveness of tool, previous experience with tools that are being considered, and feasibility among others.

6. Select Data Analysis Tools
The analysis of data is more than number crunching, it is the summary and organization of large volumes of data to discover patterns and fortify arguments used to support conclusions or evaluative claims that result from your evaluation study. If you have quantitative data, various statistical operations can help you organize your data as you sort through your findings. Qualitative data is also subject to analytical routines. Qualitative observations can be ordered by source and by impact, or sorted according to general themes and specific findings. Checking the frequency of qualitative observations will begin to merge qualitative into quantitative data.

7. Communicate Results and Recommendations
A rigorous evaluation does not speak for itself. Communicating with key stakeholders throughout the evaluation process keeps them aware of what you are doing and why, which increases the amount of trust they place in you and your efforts. In addition, it allows them the opportunity to participate and provide valuable feedback. By the time the final report and debriefing come along, these products will not be seen as something imposed on them, but rather as something they helped create. With this type of buy-in, application of recommendations will be more likely.

Conclusion
Remember, evaluation findings must enable decision makers to make sound decisions that result in improved performance. Thus, the evaluation purpose must be tied to improving performance and, thereby, promoting useful evaluation questions. These, in turn, set the stage for deciding what data to collect, how you collect it, and then how to make sense of it so you come up with justifiable recommendations for action.

About the Book
Evaluating impact: Evaluation and continual improvement for performance improvement practitioners. (2007, HRD Press) is the fourth volume in the Defining and Delivering Successful Professional Practice series co-published with the International Society for Performance Improvement. The series editors are Roger Kaufman, Dale Brethower, and Richard Gerson. Other available books include Change Choices and Consequences; Achieving High Performance; Evaluation and Continual Improvement of Results, and Performance By Design.

Ingrid Guerra-López, PhD, is an assistant professor at Wayne State University, is an associate research professor at the Sonora Institute of Technology in Mexico, and consults for public and private organizations, specifically in the areas of performance measurement and tracking. Her most recent book is Evaluating Impact: Evaluation and Continual Improvement for Performance Improvement Practitioners. Ingrid may be reached at iguerra@wayne.edu.

return to top

Advertisement  Would you like to advertise in this space? Contact marketing@ispi.org

TrendSpotters: Performance-based Management System
by Carol Haig, CPT, and Roger Addison, CPT, EdD

Please join us in welcoming Carol Panza, CPT, to TrendSpotters and congratulating her on receiving ISPI’s 2007 Distinguished Service Award. Carol, cmp@orgmap.com, is a past ISPI Board Director and a founding member of ISPI’s Europe chapter. She currently serves on the board of the IFTDO, ISPI’s conference partner this April 30-May 3, in San Francisco. Carol leads CMP Associates, a management consulting firm specializing in the analysis and custom design of improvement strategies for business. With analysis as her professional focus, Carol happily says she “gets paid for being nosy.” She graciously contributed her Performance-based Management System to the TrendSpotters Open Toolkit.

Genesis of This Model
Some years ago, Carol was asked to help a client replace an ineffective performance appraisal system heavily skewed toward personal characteristics. The organization was an amalgam of four cultures in a shipboard environment. Carol built the Performance-based Management System model to focus on job accomplishments in a work context, thus producing a culturally neutral process for measurably evaluating performers.

Model Description
The Performance-based Management System is driven by the four questions in the boxes across the top:

• What Performance is expected?
• What Performance is happening? Is it a trend?
• Where and how is improvement needed?
• What was overall performance?

Each question is supported by one or more documents used to collect information and roll it up to the individual documents that ultimately display the answers to each of the questions. So, for example, the supporting document for the fourth question—What Was Overall Performance?—is a completed Performance Evaluation form.

In HPT models, performance is typically displayed as happening from left to right (beginning with What Performance Is Expected?) and analyzed from right to left (beginning with the end—What Was Overall Performance?). The Performance-based Management System model is flexible because you can ask the questions in any order—depending on the challenge you are asked to resolve—and ultimately answer all of them by building the appropriate documents to support each one. You can design the documents in this system in a way that works for your organization.

How to Use the Model
When Carol developed the Performance-based Management System, the goal was to produce an overall individual performance summary that also promoted performance improvement for the employee being evaluated. Thus, the appraisal was to be used for the good of the organization as well as for the good of the individual.

As always in HPT, Carol suggests that you start with the end in mind—overall performance—and then go back to the beginning to determine what the expected performance is. If you can identify the driver for your analysis, you will know which of the four questions to start with. However, performance expectations that are defined as accomplishments and are also clear, complete, relevant, and properly aligned to the organizations results form the essential foundation that makes the entire system work.

Success Story
A government agency was involved in a lawsuit that was resolved with a consent decree mandating the design and implementation of a new performance appraisal system. Carol was invited to bring in the Performance-based Management System under the client’s assumption that it was a plug-and-play solution. Fortunately, the client quickly realized the intensive and highly specific nature of the analysis required. Carol was able to use the model to create an appraisal system that met the consent decree requirements and enabled the client organization to use it with great success.

Advice to Users of the Performance-based Management System
To get started with this model, Carol suggests that you focus on answering the four questions highlighted earlier. As you research the answers, you may discover that the information in the job descriptions in your organization is activity based rather than accomplishment based. Since the critical question is What Performance is Expected?, your first task may be to re-write job descriptions to clearly answer this question. Accomplishment- or results-based job descriptions enable the measurement of performance; activity-based job descriptions do not. With a solid foundation of well-constructed job descriptions, you will be ready to proceed with developing your new appraisal system.

Links to the Performance Technology Landscape
The Performance-based Management System supports these principles of Performance Technology:

Application Exercise
Here are two ways to get started with the Performance-based Management System:

• For your next appraisal-related project, establish clear expectations for the outcome: state the achievements that are to result from using the system you are building.
• Review job descriptions affected by your project to determine if they are performance based and update if they are not.

With these two elements in place, you will be positioned to do what HPTers are so good at: getting behind the job accomplishments to identify what skills, knowledge, tools, and resources employees need to develop and grow professionally. Look for ways to build in prescriptive suggestions to improve individual performance and, ultimately, the processes and the performance of your organization.

To review past contributions to the TrendSpotters Open Toolkit and find all the models and tools featured in this column, click here.

If you have an HPT model or tool that supports you in your performance improvement activities, contact Carol Haig, CPT, at carolhaig@earthlink.net or http://home.mindspring.com/%7Ecarolhaig, or Roger Addison, CPT, EdD, at roger@ispi.org.

return to top

HPT: The World’s Best Kept Secret
by Roger Chevalier, CPT, PhD

After many presentations at ISPI and ASTD conferences, I get the distinct feeling that our human performance technology (HPT) message is only reaching other consultants and trainers. We seem to have overlooked the greatest potential market for our systematic and systemic approach to improving performance—line managers and supervisors.

One of the critical roles that managers and supervisors have is that of improving the performance of their divisions, departments, and workgroups. They routinely assess the present level of performance and compare it to a desired or planned level of performance. They then identify the potential causes for performance shortfalls and come up with solutions they implement. Next, they evaluate the results to see if the difference between “where they are” and “where they want to be” has been closed. Sound familiar? This is the systematic approach defined by the last six Standards of Performance Technology.

The first four Standards of Performance Technology have value to managers and supervisors as well. They to need to focus on outcomes, take as systems view, add value to their organizations, and establish partnerships, if they are going to be able improve performance. These principles apply as much to line managers and supervisors as they do to performance consultants.

There is an old Chinese proverb that most of us are familiar with: “Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime.” All too many of us have been giving HPT to managers and supervisors when we should be teaching HPT to them.

Imagine a workforce where all managers and supervisors have a consistent approach to identifying performance gaps; determining causes; selecting, designing, and developing a range of solutions; implementing those solutions; and then evaluating the results. One organization, the Royal Canadian Mounted Police (RCMP), does not have to imagine any more.

The RCMP has been using HPT to improve operational readiness in police units and detachments since 2003 and currently has the HPT process in active use in more than 150 sites cross Canada . The standards have been applied throughout the operational management structure down to the individual police officer level to achieve measurable results while ensuring consistent application as HPT was introduced in different regions and business lines.

Part of any HPT implementation strategy should include embedding the Standards of Performance Technology throughout the organization. It is not enough that a select few know and use HPT. Line managers and supervisors are the ones ultimately responsible for improving performance, and it is time that we focused on delivering our systematic HPT approach to them. It is time that we teach them how to fish.

Dr. Roger Chevalier, CPT, will present a Masters’ Series session, Applying the Standards of Performance Technology, with a representative from the Royal Canadian Mounted Police. He will also present a conference workshop, A Manager’s Guide to Improving Workplace Performance, reflecting on his new book of the same name. Both the Masters’ presentation and the conference workshop are part of the ISPI/IFTDO 2007 International Performance Improvement Conference in San Francisco, April 30-May 3.

return to top

Advertisement  Would you like to advertise in this space? Contact marketing@ispi.org

Potomac Chapter Members Present HPT to Iraqi Delegates
by Molly Wankel, PhD

On November 7, 2006, several members of the Potomac Chapter (Washington, DC Metro Area) had the honor of presenting information about our profession and ISPI to 13 representatives of Iraqi ministries. There were delegates from the Ministry of Municipalities and Public Works and the Minister’s Office and the Municipality of Erbil/Khurdish Regional Government. The delegates, most of whom were new to training and HPT, were in the United States for training and site visits to agencies with similar missions.

The Potomac Chapter members hosting the delegation were June Fair, Phil Harris, Elaine Rand, and Molly Wankel. After the welcome and introductions, Molly and Elaine briefly described the history of ISPI. June led the representatives in an HPT analysis based on Tom Gilbert’s Behavior Engineering Model and Peter Dean’s research. Individuals’ assessment of whether training is the most effective solution to a performance problem fed into a lively discussion of whether it is more effective to fix the system or the individual, which was the point of the presentation: Human Performance Technology (HPT) Is the Means for Achieving the Goal. The group discussed the HPT profession, characteristics of the discipline, the HPT process, and how ISPI supports the field of performance improvement. Phil presented on the ISPI Certified Performance Technologist (CPT) program, and how he has used HPT in his work, which the Iraqis found most interesting. Both Phil and June are CPTs. The ISPI members offered a list of resources they have found particularly helpful throughout their careers in HPT.

Elaine Rand, longtime ISPI member, discusses human performance
technology with the Iraqi delegates.

The Iraqis were very engaged, asked many questions, and were most curious about the process for forming an ISPI chapter in their country. However, the term “chapter” had to be further defined as a local branch of the organization and not part of a book.

The ISPI members were heartened by the Iraqis’ interest in the topic and found the relationship-building event to be particularly meaningful as it occurred on Election Day.

Molly Wankel, a longtime member of ISPI and past president of the Potomac Chapter, is manager, learning and performance for SI International. Currently, she is serving as an external and internal consultant to HPT efforts for enhancing performance of federal and corporate employees. Molly consistently presents at ISPI and other conferences, has written many articles for publication, and wrote a chapter in Sustaining Distance Training edited by Zane L. Berge. She may be reached at molly.wankel@si-intl.com.

return to top

From the Board
Performance Beyond Borders…and All Expectations
by Clare Elizabeth Carey, CPT, EdD, ISPI President

“Zoom” describes the calendar’s ops tempo. ISPI’s Annual Conference is just weeks away! It was only a blink ago that I was outlining my presidential vision in this column. As I begin the transition of my tenure, I am delighted to share some coming attractions and our Board’s accomplishments over this past year.

Conference 2007: Good, Fun, Cool, Enlightening, Provocative, and “S”!
Please join me in celebrating the incredible work of the 2007 Conference Committee, led by Paul Cook and Monique Mueller. Working with staff over the last two years (no kidding!), the conference committee volunteers have invested lots of energy, expertise, and enthusiasm to produce a great program for San Francisco. The theme, Performance Beyond Borders, continues to attract positive attention from our membership, IFTDO, and global colleagues. There were more than 300 proposals submitted. With the 31 workshops, 6 masters’ series speakers, Lynn Lancaster’s opening and Klaus Wittkuhn’s closing keynotes, and 15 encore presentations, we can expect the 200+ sessions to have depth, breadth, and quality. Conference is going to be good!

In keeping with ISPI’s spirit of inclusion, the conference Community Center will celebrate a “World Bazaar” of ideas and engagement. With 80 volunteer “Ambassadors” to host scheduled and spontaneous events, support creative displays, and facilitate interactive and inviting forums, this area will serve as a friendly and supportive hub for attendees. Conference is going to be fun!

If you have not already, take note of the Conference Blog. Dawn Papaila and Craig Grimm have collaborated to move ISPI forward into the wonderful adventure of blogospheres. This is an amazing site where 139 presenters agreed to share their session abstracts and 123 presenters are already exchanging emails. Conference is going to be “cool”!

Given our global audience and IFTDO partners, our conference keynote presentation will be inclusive and informative. If ever a topic has currency and world appeal, it is Lynn Lancaster’s presentation on the multi-generational workforce, Bringing Out the Best in Every Generation. Organizations in every country and in every domain are experiencing “ClashPoints” to some degree. (Come on, admit it…you have witnessed the combustions of that techno-savy new hire with your seasoned, traditional, techno-phobic boss!) How do we bridge attitudes, values, and work styles? As performance practitioners, how do we apply our expertise to leverage the diverse talents of our boomers, X-ers, Y-ers, and Net-gens? In my discussions with Lynn, I outlined our RSVP performance mindset (results-focused, systematic view, value-added, and partnership advocacy). I am confident that Lynn will connect with our performance pulse and push our perspectives to new levels. Conference will be enlightening!

The “last shall be a first,” you might say, as we conclude our conference with the notable Klaus Wittkuhn, who will present a unique, first-ever general summary session, Process—A Concept that Changed the World. We are assured of an international celebration of art, science, and technology with Klaus. Conference will be provocative!

But wait, there is more…Do not miss our closing banquet celebration with Allison Rossett. This is sure to be a stimulating and joyous event where we share good food, great company, some laughs, and poignant stories. For those of you who have passed on this tradition, I suggest you reconsider. Come recognize our colleagues, friends, and new acquaintances and celebrate the S in ISPI. Conference will be about uS!

So, what are you waiting for? Register today and join us in San Francisco, April 30-May 3.

Year 2006-07 in Review
In brief, this has been a challenging yet a good year. The Board quickly blended their talents and expertise and tackled several strategic initiatives. Our executive director, Rick Battaglia, adroitly executed ISPI business and established key tactical partnerships. The staff demonstrated empowered creativity in their strategies and solutions. Collectively, we embraced the strategic thrusts of inclusion, innovation, and integrity by strengthening the connectivity with members, expanding technology applications, and advancing the strategic directions of ISPI.

We implemented our budgets set in the September meeting and, for the most part, are on track. This has been a challenging economic year. However, any setbacks were mitigated with Rick’s business acumen and the staff’s success with institutes, special programs, and creative application of resources.

The Directors have worked diligently throughout the year on their respective initiatives. We completed the initial SWOT and then narrowed the focus for our collective energy and attention. We committed to investing seriously in the branding initiative, and we are pleased with the results so far. We will provide more details of the staff’s and Board’s accomplishments during the Annual Business Meeting scheduled for Wednesday, May 2, 12:15-1:15 pm. We welcome your participation!

Special Recognitions
One of the Board’s most important and inspiring responsibilities is the selection of annual award recipients. We get to share stories and describe the many positive contributions of our ISPI colleagues. With so many deserving individuals, it is a challenge to pick one awardee for each category. It was a joy to contact the respective recipients and share in their surprise and delight. Please know our ISPI awards mean a great deal to our awardees and our general membership. Please extend your warm acknowledgements and congratulations to:

• Carol Panza, CPT, Distinguished Service Award
• Dale Brethower, PhD, T. F. Gilbert Professional Achievement Award
• Klaus D. Wittkuhn, CPT, Honorary Life Member Award

For this conference, the Board agreed to present a special recognition to Marilyn Gilbert for her enduring contributions to the practice of human performance technology. We know many members have been influenced, inspired, and mentored by Marilyn, who continues to teach and promote the field of performance improvement. It will be heartwarming to celebrate one of ISPI’s great treasures at Conference.

A Final Thought…
As I prepare to turn the presidential reins over to the very capable President-elect, Jeanne Farrington, I want to thank everyone for their ongoing feedback. I have valued and responded to all of your comments. I could not have asked for greater support from the members.

The accomplishments of this year are due to the incredible dedication and work of Rick, the ISPI staff, wonderful volunteers, and our exceptional Board of Directors. I remain in awe of these Directors, Bob Bodine, Matt Peters, Jim Fuller, Darlene Van Tiem, and Miki Lane. In spite of major life changes, new jobs, new locations, real estate adventures, and personal challenges, the members of this Board continued to honor their responsibilities and complete additional tasks with full commitment. Their dedication and service to ISPI goes beyond all expectations! It has been an honor and privilege to serve with them. Be assured, our Society is in good hands.

There is a lot more to share and we will at the conference. Here’s to seeing everyone in San Francisco!

Note from ISPI: In 2004, the Board of Directors expanded the Board to include the immediate Past-president as a non-voting, ex-officio member of the Board. We are pleased Clare will continue her service to the Society and Board for one more year.

return to top

ISPI Announces Election Results!

The votes have been tallied, and the following candidates have been elected to serve as members of ISPI’s 2007-2009 Board of Directors.

Matt Peters, CPT President-elect	Timm J. Esque, CPT Director	Mary Norris Thomas, CPT, PhD Director

The following Board members retain their seats: Jeanne Farrington, CPT, EdD, (who becomes President in May), Jim Fuller, CPT, Miki Lane, CPT, Darlene Van Tiem, CPT, PhD, Immediate Past President Clare Elizabeth Carey, CPT, EdD, and Richard D. Battaglia, CAE (ex officio).
 
A special thanks to departing Board member: Robert L. Bodine, CPT, PhD for his hard work and dedication to ISPI.

return to top

ISPI Invites Chapter Contest Winners to
San Francisco

The International Society for Performance Improvement wants to thank our chapters and their members for participating in this inaugural drawing for a free registration to the 2007 International Performance Improvement Conference. We received nearly 500 contest forms from 19 chapters, including Canada and Australia. We hope the chapters found this to be a beneficial tool for generating new chapter members and promoting ISPI’s Annual Conference. Congratulations to the chapter members listed below who won this year’s contest.

Chapter Drawing Winners

• Arizona, Tiffany Temple
• Chicago, Rebecca Stiner
• Front Range, Colorado, Michelle Williams
• Los Angeles, Michael Dorn
• Massachusetts, Virginia Hallman
• Michigan, Peggy Bennett
• Minnesota, Deb Elliott
• New Jersey, Carol Panza
• New Mexico, Carrie O’Hara
• Ohio Heartland, Susan Stasiak
• Orange County, California, Rhonda Askeland
• Potomac, Maryland/Virginia/DC, Gary Powell
• St. Louis, Jeanette Wallace
• San Antonio, Lydia Garza
• San Diego, Sandy Troya
• Seattle, Gail Rosenbaum
• Sydney, Australia, Rada Millwood
• Utah, Rich Reitter
• Vancouver, British Columbia, Canada, Ian Beardsell

If you are interested in improving workplace performance and expanding your professional network, you won’t want to miss this unique opportunity to join your colleagues from around the world. For more information on this year’s conference, visit www.ispi.org/ac2007.

return to top

Selling T&D in the Organization
by Terrence L. Gargiulo

Listening is an essential skill for selling T&D in an organization. Listening demonstrates T&D’s ability and willingness to adapt to the needs of an organization. We must go out of our way to understand our customers in order to determine how to help them achieve their objectives. We cannot expect people to come to us. Only after we build strong inroads with people and establish our credibility will they proactively seek our help. Until then, we need to reach out. When we listen to people and observe the work they are engaged in, we will discover opportunities where we can make a difference. They may not be what we expect. Going in with an open mind and attentive eyes is what is necessary. We also need to be careful not to jump to conclusions or take what we hear from people at face value. What people ask for and what people need may be very different. Working together we can explore solutions.

The ability to think from other people’s organizational and personal perspectives is central to building institutional support and selling T&D. Each area of an organization represents a frame of reference. Because T&D serves all areas of the business, we have to become skilled at moving in and out of various perspectives. Our perspective becomes secondary to understanding, appreciating, and thinking from other perspectives. Pacing with people, meeting them on their turf, and making an effort to speak the language inherent to their functional area is our greatest challenge. When we succeed in temporarily abandoning our perspective and immersing ourselves in other people’s organizational worldviews, tremendous results become possible. It’s a simple truism; when people feel you have listened to them, they are more inclined to listen to you.

We have to be careful to avoid the temptation to immediately fit a functional area’s needs into our collection of prefabricated solutions. People need to feel that their unique needs have been understood and that we haven’t rushed to offer them a cookie-cutter solution. Selling T&D as a strategic asset demands that we help our customers feel unique even when we may realistically slot them into tried-and-true interventions. We stand to learn a lot in the process. Our customers offer us wonderful opportunities to continuously develop, grow, and stretch in new directions. Learning is dynamic, so it should come as no surprise that we need to model these principles in the way we do our work.

As we become more skilled at listening, we also become better at anticipating what our customers will need. We start viewing our customers’ new projects and initiatives as opportunities for T&D. We come to the table with ideas and recommendations. Our customers begin viewing T&D as a strategic partner. Powerful things happen when our customers realize we are looking at the world from their organizational perspective. In the beginning, it will be impossible for you to reach out to all the areas of your organization, so start with a small win. Small wins add up to bigger ones. As you build momentum and institutional support, other areas will hear your T&D success stories and begin coming to you.

Focusing on the building of strong relationships is critical to the success of selling T&D. We need to allot greater amounts of time to cultivating relationships. How people perceive us is directly related to the amount of time and effort we spend in reaching out to others and taking an active interest in them. Achieving results is only one part of the equation. Broadcasting our achievements and accomplishments in the form of hard facts is not convincing enough to engender trust and widespread support of our activities. We have to reach out and make an emotional connection with people. One of the best ways to do this is to share a story (see The Strategic Use of Stories in Organizational Communication and Learning [Gargiulo, 2005], and Stories at Work: Using Stories to Improve Communications and Build Relationships [Gargiulo, 2006]). Our customers need to realize we have their best interests at stake.

There is an interesting paradox in people’s perceptions of T&D. People require ongoing learning to succeed. Learning implies that the learners are missing some information or experience. However, most people avoid parading their deficiencies in front of others. No one is accustomed to exhibiting personal vulnerabilities and people often fail to see their blind spots. So on the one hand people have a built-in resistance to T&D’s generic charter of promoting learning, yet on the other hand people possess an intrinsic desire for receiving learning help and support. Our T&D customers need to believe we can help them and be willing to trust us. When we become too focused on the formalities of T&D we are apt to weaken our relationships with those customers. Marching people through our procedures, processes, and standard course offerings may hinder rather than aid our cause. Not that these things should be eliminated—it’s just that they are not enough to build new relationships and nurture existing ones.

In order to build long-term institutional support and sell the benefits of T&D, we need to leverage every person of our team as an ambassador of goodwill. It takes a critical mass of people working together to effectively reach out to all areas of an organization. The foundation for promoting T&D stewardship is the culture we develop within our own teams. We must endeavor to create an ethos for our T&D team that exudes warmth, fascination, tireless interest in others’ well-being and development. These are qualities that generate excitement, commitment, and ownership. There is nothing soft and fuzzy here. Beyond the obvious humanistic merits of such an approach, this advice is couched in utilitarian benefits. Our customers and employees alike will embrace and respond to the positive energy we create. Encouraging our team members to be ambassadors and building a positive working environment for the team will help T&D to become an indispensable partner at the organization’s strategic table. We want people to seek our participation and ideas when planning key organizational initiatives. In fact, we will know we have really succeeded in selling the potential of T&D when we begin to help the organization uncover new opportunities, when we are not just responding to support it in achieving the goals and objectives it has already set.

Note: This article is an excerpt from Building Business Acumen for Trainers: Skills to Empower the Learning Function and is reprinted with permission by the publisher, Copyright 2006, John Wiley & Sons.

Terrence L. Gargiulo, MMHS, is a six-time author, international speaker, organizational development consultant, and group process facilitator specializing in the use of stories. He holds a Master of Management in Human Services from the Florence Heller School, at Brandeis University, and is a recipient of Inc. Magazine’s Marketing Master Award. He is co-founder of the Storytelling Organization Institute (STORI). Highlights of some of his past and present clients include GM, HP, DTE Energy, Dreyers Ice Cream, UNUM, U.S. Coast Guard, Boston University, Raytheon, City of Lowell, Arthur D. Little, KANA Communications, Merck-Medco, Coca-Cola, Harvard Business School, and Cambridge Savings Bank. Terrence may be reached at terrence@makingstories.net.

return to top

Center for Systems Management Joins
ISPI Advocate Program

CSM, the Center for Systems Management, is proud to be the newest ISPI Advocate. Since 1989 CSM has engaged in value-based partnering relationships with both private and public sector clients, globally providing consulting and training services. CSM believes that its systems engineering approach complements ISPI’s HPT approach and is looking forward to building on these synergies. CSM brings together organizational human capital needs, integrating performance improvement with project management and systems engineering (or systems of systems), which enables CSM to provide holistic solutions to achieve corporate performance goals. Our team contributes at all levels of the organization, enhancing the knowledge, skills, and abilities needed to excel and meet or exceed return-on-investment expectations. Contributing to the HPT community the CSM team brings a rich practitioner experience base, together with a proven background of innovation in the systems management arena. CSM not only offers customized consulting to help solve problems, but also provides training and process improvement services.

Professionals from complex organizations count on CSM to deliver lasting performance improvement and management solutions that build the leadership and technical skills of their diverse workforce. CSM is proud of its 18 years of experience and the critical role it has played in the performance and success of our clients including Raytheon, Northrop Grumman, General Dynamics, SAIC, Booz Allen Hamilton, BAE Systems, AT&T, Motorola, Ball Aerospace, Nokia, NASA, and several intelligence agencies.

CSM is a Project Management Institute (PMI) Registered Education Provider (REP), a transition organization of Practical Systems and Software Measurement (PSM), and is a member of the Software Engineering Institute (SEI) Partner Network. CSM is an active participant in the International Council on Systems Engineering (INCOSE), providing process and practitioner expertise in maturing the discipline. CSM’s founders, Hal Mooz and Kevin Forsberg, together with Howard Cotterman, authored two foundational project management books, Visualizing Project Management and Communicating Project Management, that present CSM’s Dual Vee model of the components and processes required to successfully implement a project of any size and kind, from building an in-house human resources system to designing an airplane to building a communication system. Through this model and other intellectual contributions, CSM continues to advance the disciplines of systems management with developing human capital to support the performance improvement needs of its clients. Working with their clients as a trusted partner and change agent, CSM develops and implements sound performance improvement processes that support the needs and goals of an organization. Like other ISPI advocates, the CSM team thrives on innovation and results.

return to top

ISPI Career Center: Find Your Next Job or Improve Your Current One While in San Francisco

The Career Center at ISPI’s upcoming 2007 International Performance Improvement Conference in San Francisco, April 30-May 3, is more than just a job fair. This year, ISPI is offering programs and services designed to meet the career needs of all conference attendees, whether you are actively seeking a new position or finding ways to enhance a current position.

Interview Program
Conference attendees will have the opportunity to meet with employers seeking strong candidates with skills in human performance technology. We also welcome employers who are interested in posting a position and/or interviewing candidates. Limited space may be available to present your organization’s opportunity in a group format. For more information on available positions or posting your open positions, contact Amy Connelly at 317-769-4310 or careercenter@ispi.org.

Workshops
Experienced career development professionals will present a number of workshops during the three-day career event:

Prioritizing Career Values and Identifying Transferable Skills
Richard Knowdell, President, Career Research & Testing, Inc.

Most of us need help to make successful short-term job transitions and long-term career transitions. Often, these transitions are unwelcome and unplanned. In a hands-on workshop, participants have an opportunity to prioritize their career values and discover other career assessment tools used by leading coaches to facilitate and guide clients to success.

Search for a Job (or Clients) Better and Faster
Sally Gelardin, Principal, The Job Juggler, Marilyn Harryman, Associate Counselor, Bay Area Career Center, and Roberta Floyd, Career Development Counselor, Oakland Community College

Fill in the gaps in your job hunting or client recruitment strategy. Discover what works in the new economy. Learn how to organize your job/client search on your computer. Explore e-profiles, e-portfolios, and e-resumes. Discover e-networking techniques. Determine how, where, and with whom you want to work and how you’re going to make it happen. Become aware of the most up-to-date employability skills needed in today’s marketplace.

Employee Career Development in Organizations—Employer-focus!
Richard Knowdell, President, Career Research & Testing, Inc.

Learn a four-step process to enable employees to develop clear goals and strategies to develop their careers within the organization. This process has been proven to reduce attrition rates for motivated employees and assist employees who don’t fit within the organization to move out.

“15 Minutes to a Better…” Career Series
During the networking breaks offered during the conference, members of the Career Center staff will offer short presentations on topics related to job search and career development.

Careers Over Lunch
Bring your lunch to the Career Center and join other professionals for informal networking and consultation with Career Center staff.

Complimentary Career Coaching
Thirty-minute confidential career coaching sessions will be available at no charge to registered conference attendees. These sessions are provided by National Certified Counselors and are appropriate for individuals entering the workforce, at mid-career, or considering retirement. Our coaches are available to review resumes, listen to attendees’ career issues, offer solutions, and refer attendees to career counselors and career coaches near their homes. Conference attendees can pre-register for these sessions by emailing Amy Connelly at careercenter@ispi.org.

For more information on the Career Center in San Francisco, click here.

return to top

Project Proven Tools and Techniques for ISD:
The Enabling K/S Matrices
by Guy W. Wallace, CPT, Representative of the Instructional Systems (IS) ProComm

They are used to capture the enabling “awareness/knowledge/skills” that the master performers and subject matter experts agree are needed to perform at a mastery level. And, of course, that does not always make them 100% right. But who else would you have determine them—especially in complex jobs and processes where there is more cognitive and covert behavior going on than the physical and overt behavior that a trained observer could see?

Call the captured K/Ss the “enabling competencies” versus the “performance competencies” captured on the Performance Model. The data in the columns of the Knowledge/Skill Matrix is captured live by the facilitator during the same three- to four-day meeting in which the Performance Model is developed.

To develop Knowledge/Skill Matrices, knowledge/skill items are identified and listed on a matrix chart. A list (typically customized for a project) of predefined knowledge/skill categories allows the analyst to control the brainstorming session. By structuring the brainstorming effort in this way, the analyst can keep individual knowledge/skill items closely linked to performance as described in the Performance Model.

Additional data points are gathered for each knowledge/skill item on the matrix; these are captured in the columns on the right-hand side of the matrix.

K/S Categories
K/S Matrices are organized by the “appropriate to your effort” K/S Categories from this master list:

1. Company Policies/Procedures/Practices/Guidelines
2. Laws, Regulations, Codes, Agreements, and Contracts
3. Industry Standards
4. Internal Organizations and Resources
5. External Organizations and Resources
6. Marketplace Knowledge
7. Product/Service Knowledge
8. Process Knowledge
9. Records, Reports, Documents, and Forms
10. Materials and Supplies
11. Tools/Equipment/Machinery
12. Computer Systems/Software/Hardware
13. Personal/Interpersonal
14. Management/Supervisory
15. Business Knowledge and Skills
16. Professional/Technical
17. Functional Specific        

K/S Matrices Charts
The Knowledge/Skill Matrices link each knowledge/skill item to the performance that it enables (as described in the Performance Model). Thus, the Performance Model ensures that the discrete knowledge/skill enablers in the Knowledge/Skill Matrices are performance relevant. And, in turn, the performance orientation is passed on to the design of instructional products, for which the Knowledge/Skill Matrix data, among other data, are a key input.

An example “typed-up” KSM chart is presented in Figure 1.

Figure 1. KSM Chart

The columns on the right side of the Knowledge/Skill Matrix give a richer insight into the enabling knowledge/skill items.

The select/train column differentiates those items that are attended to (if indeed they are) by the selection process, not the training process. If the knowledge/skill item is supposed to be taken into account during the candidate selection process, the item should not be an issue for the T& D system. The analyst marks training items with a “T” and selection items with an “S.”

The criticality column gives an assessment of the importance of the knowledge/skill item to one’s ability to perform at a level of mastery. The analyst marks each item as H, M, or L (high, medium, or low criticality) as the Analysis Team dictates.

The difficulty column indicates how hard the Analysis Team assesses it will be for the typical member of the target audience to grasp the awareness, knowledge, or skill related to this knowledge/skill item. The analyst and team mark each item H, M, or L.

The volatility column is an assessment of the amount of maintenance required by the content covering the knowledge/skill item. Volatility impacts packaging and deployment and distribution strategies. The analyst and team mark this column H, M, or L.

The final column, depth, indicates the depth of coverage needed for the eventual or potential training at the awareness level (A), the knowledge level (K), or the skill level (S).

Additional, in-depth content on this tool and technique is available in Chapter 11 of the third edition of The Handbook of Human Performance Technology, edited by James A. Pershing, CPT, PhD.

If you would like to participate further in a discussion about Performance Models, make a comment about the article, or ask Guy a question, click here to visit the IS Discussion Forum.

Next month: Existing T& D Assessments for Re-Use.

Guy W. Wallace, CPT, has been an external ISD and HPT consultant since1982, is the president of EPPIC Inc., has been a member of ISPI since 1979, is a past president of ISPI, is the author of lean-ISD, and is a recipient of an ISPI 2002 Award of Excellence. He may be reached via guy.wallace@eppic.biz and related resources may be obtained at his website: www.eppic.biz.

return to top

Volunteer Opportunities: ISPI's Annual Conference

Are you interested in attending ISPI’s 45th Annual International Performance Improvement Conference in San Francisco, California? Are you willing to attend pre-assigned sessions or workshops, open to monitoring sessions you may not have selected, and able to distribute and collect evaluation forms and assist ISPI presenters? If you answered yes to these questions, you may want to apply to be a Workshop or Conference Volunteer.

Workshop Volunteers
Workshops will be held Sunday, April 29, and Monday, April 30, 2007. Workshop Volunteers receive complimentary registration to the workshop they are assigned; however, if you wish to attend the conference, you must register at the appropriate rate. Workshop Volunteers do not receive a reduced conference registration fee. Workshop Volunteers must meet the following criteria:

• Must report to meeting room at least 45 minutes before start of the workshop
• Monitor one 2-day workshop or two 1-day workshops
• Must be available for the full day, on both days
• Collect meeting room information as requested
• Distribute speaker handout material as needed
• Check with presenters to ensure they have everything necessary for their presentation prior to the start time

Conference Volunteers
The conference dates are Sunday, April 30-Wednesday, May 3, with volunteers starting on May 1, 2007. Conference Volunteers shall receive the following discounted registration fees, which include one ticket to the Opening Reception, Bagel Barrel, and the Extravaganza.

• ISPI members: $399
• Non-members: $599                                                  
• ISPI student members: $199

Conference Volunteers must meet the following criteria:

• Attend the volunteer orientation
• Will report to volunteer coordinator(s)
• Must report to meeting room at least 45 minutes before start of session
• Must work 3-4.5 hours per day monitoring concurrent sessions as assigned
• Distribute and collect evaluation forms and capture requested information
• Check with presenters to ensure they have everything necessary for their presentation prior to the start time
• Distribute speaker handout material as needed

Volunteers will be responsible for their own travel, hotel, and other cost associated with attending the conference. Volunteer positions are assigned on a first-come, first-served basis and will be filled with a preference given to ISPI student members, ISPI members, and then non-members, in this order. For more information or to apply, contact Kim Akoto at conference@ispi.org.

return to top

Performance Marketplace

Performance Marketplace is a convenient way to exchange information of interest to the performance improvement community. Take a few moments each month to scan the listings for important new events, publications, services, and employment opportunities. To post information for our readers, contact ISPI Director of Marketing, Keith Pew at keithp@ispi.org or 301.587.8570.

return to top

ISPI Membership: Join or Renew Today!

Are you working to improve workplace performance? Then ISPI membership is your key to professional development through education, certification, networking, and professional affinity programs.

If you are already a member, we thank you for your support. If you have been considering membership or are about to renew, there is no better time to join ISPI. To apply for membership or renew, simply click here.

return to top

Newsletter Submission Guidelines

ISPI is looking for Human Performance Technology (HPT) articles (approximately 500 words and not previously published) for PerformanceXpress that bridge the gap from research to practice (please, no product or service promotion is permitted). Below are a few examples of the article formats that can be used:

• Short “I wish I had thought of that” Articles
• Practical Application Articles
• The Application of HPT
• Success Stories

In addition to the article, please include a short bio (2-3 lines) and a contact e-mail address. All submissions should be sent to april@ispi.org. Each article will be reviewed by one of ISPI’s on-staff HPT experts, and the author will be contacted if it is accepted for publication. If you have any further questions, please contact april@ispi.org.

return to top

About PerformanceXpress

Feel free to forward ISPI’s PerformanceXpress newsletter to your colleagues or anyone you think may benefit from the information. If you are reading someone else’s PerformanceXpress, send your complete contact information to april@ispi.org, and you will be added to the PerformanceXpress emailing list.

PerformanceXpress is an ISPI member benefit designed to build community, stimulate discussion, and keep you informed of the Society’s activities and events. This newsletter is published monthly and will be emailed to you at the beginning of each month.

If you have any questions or comments, please contact April Davis, ISPI’s Associate Executive Director, at april@ispi.org.

ISPI
1400 Spring Street, Suite 260
Silver Spring, MD 20910 USA
Phone: 301.587.8570
Fax: 301.587.8573
info@ispi.org
http://www.ispi.org

return to top

turn to top