Our Big Bang Moment
by James J. Hill, CPT, EdD

Organizational health, the diagnosis of ailments, and the prescription of performance improvement solutions are following the path of our human health counterparts. However, we are adopting the lessons from that field very slowly.

A Brief History (drawn heavily from The Great Influenza, J. M. Barry, 2005)
Around 430 B.C. Hippocrates identified his “Four Humors” (blood, phlegm, bile, and black bile). From that point through the 1500s, little changed in the medical field. Then, Paracelsus (observation of nature, experimentation), Vesalius (research through dissection), and Fracastorius (who hypothesized that diseases had specific causes; the “imperceptible particle”) offered new ways to approach diagnosis and discovery; new ways of thinking. However, they had little impact on practice.

Not until the mid-1700s (Lind, Locke, Hume) was there a movement toward empirical research. This was followed by the emergence of experimental processes led by Edward Jenner and his smallpox work—2,000 years after Hippocrates!

The fundamental flaw of the medical profession—one that was not fully accepted until the early 1900s—was that it relied on observation and reason alone versus experimentation. Said another way, “We apply a treatment, we observe, things get better, the treatment must have worked.” That same logic is what gave us the rain dance.

As Barry points out in his superb book, the 1800s were a time when practically every other field of science was blooming. In contrast, one historian called medicine “the withered arm of science” (p. 25). Then, in France and Germany the equivalent of a medical experimentation Big Bang occurred. Dissection, the invention of research tools like the stethoscope, and the precise measurement of bodily functions were the order of the day. Unfortunately, that changed very little in the United States. Physicians still applied a regimen of therapeutics.

To protect their ignorant positions, they intentionally shied away from using even the simplest medical equipment. To wit, there were only six thermometers in the largest Union army and no American school taught the use of the microscope, as Barry also notes. In the late 1800s, anyone could self-proclaim the title “Doctor”; and, to sum it up, as late as 1900, 41 states licensed pharmacists and only 34 licensed physicians.

The Relation to Human Performance
Rather than the four Hippocratic humors, the profession of human performance has two: knowledge and motivation (here I draw from Brethower). Knowledge systems consist of one’s ability to handle the routine—automated expertise; and one’s ability to handle novelty—conceptual understanding. The motivational “system,” the need for a person to be effective and be in control, is driven by a combination of what one chooses to do and the effort he or she puts forth toward actually doing it. Somewhere hidden in our two humors is the environment—the goals, tools, and resources people need to perform.

To address the performance humors, performance improvement practitioners have neatly identified two general prescriptions: training and non-training interventions. And this is where the problem begins.

Because of an oversimplified analytic approach, practitioners mostly get the prescription wrong—meaning they have no significant impact on organizational health or growth. In fact, just as in the late 1800s, anyone can proclaim himself or herself an organizational performance expert. “I’ve been in a lot of organizations, so I pretty much know how to fix things when they go wrong” is the equivalent of saying, “I’ve been a human for a long time, so I pretty much know what medicine to take when I don’t feel good.”

The response to both statements is, “Maybe.” When we have a cold, multiple approaches (even simultaneously) are OK. But, when things go really wrong, we need physicians—human-focused and organizational—who can tell us with a high degree of confidence what we need to do. The alternative is for us to guess at a bundle of solutions—an organizational performance speedball—that, if it does not kill us, is likely to make us a lot sicker. From an organizational perspective, the data below (adapted from M.E. Smith, 2002, and other sources) overwhelmingly drive home the point:

Success Rates for Common Improvement Approaches

Why Is This Important?
What we need is the organizational equivalent of the Prescription Drug Benefit Manual. What solutions are best for what ailments?

To get to the right solution set, we have to start at the beginning—the diagnosis of performance. The challenge is sorting through the marketing hype to get to a core diagnostic process. In our organization’s research, we have found that many practitioners attempt to “hook” their clients on a set of terminologies and buzzwords, making a change of course difficult. They are, in effect, building dependency versus independence.

This is important to organizational leaders because they are bombarded daily with self-proclaimed, and often well-meaning, “doctors.” These purveyors of solutions appear in the form of consultants, other vendors of “prescriptions,” and people who just know they are right. Their diagnostic process, such as it is, has long been known to be focused on the procurement and implementation of “things” such as: training, process improvement, and occasionally organizational redesign. The acquisition of any of those is sometimes the right approach. Regardless of impact, they require a lot of meetings, people move around, emails fly, charts are developed, and in general they provide the illusion of progress.

However, because we deal with complex organizational organisms, a holistic approach is required. A thing or a one-dimensional solution will only have the desired impact if other all elements of performance are considered, budgeted, and implemented. As Roger Addison, ISPI’s senior director of performance improvement notes, “A performance architecture is required. If a training program is needed, then it is likely that other changes in management guidance and supervision, incentives, and on-the-job support aids will also be needed. The key is to look at the whole system.”

IOU
There is a market for independent, objective, and unbiased organizational diagnostics. We require it from our financial advisors and we ought to hold our high cost cadre of consultants to the same standards. Based upon technology advancements in the past 15 years, these diagnostics can and should be easily accessible and mostly self-administered. For the most part, consultants take too long, cost too much, and generally get in the way. Other professions have made similar moves. Medical self-diagnosis? WebMD. Tax diagnosis? TurboTax. If things get too hard or too dangerous, we can rely on experts, but mostly we find that we can do the work ourselves.

Impact?
The costs of business—whether public sector or private, for profit or not—are only going to rise. Organizations do not have the time or money for more training; in fact, they ought to be striving for less…maybe none. They also do not have the fiscal wherewithal to buy every piece of equipment they want. They need only the things that provide the biggest market impact. As a natural result, executives are on a constant search for costs to cut. In the September 2007 issue of CFO magazine, the former Gillette CEO Jim Kilts says, “I’ve never seen an organization that didn’t have a whole bunch of unnecessary costs in it.” The objective is to put money back into the business and increase top line revenue, operating margin, and the metrics that drive specific industries.

Just like there are many drug stores that can fill prescriptions, there are many vendors and internal support organizations among us that can adequately respond to the need for a wide range of performance solutions. The few that are really valuable are those with the ability to provide systematic, pinpoint diagnoses that add to organizational income statements. Their advice and counsel can be accepted with a high degree of confidence and, as a result, they consistently put their clients on the path to making the best possible resource decisions for achieving real finance growth. That is the unique offering of our profession!

When we provide a consistent level of dependable analytical performance using a common language and proven scientific methods, we will witness a monumental change in our industry. That will be our Big Bang moment.

Jim Hill, CPT, EdD, is the CEO of Proofpoint. Prior to launching his company, Jim had notable careers as a corporate executive at Sun Microsystems and as a Marine Corps officer. He is a past president of the International Society for Performance Improvement and received his CPT designation in 2003. Jim has been recognized by Training magazine as one of 14 visionaries in the field of organizational performance and is a frequent author and speaker on issues of organizational performance. He received his doctorate in Human and Organizational Performance from University of Southern California. Jim may be reached at jim.hill@proofpoint.net.

return to top

Advertisement  Would you like to advertise in this space? Contact marketing@ispi.org return to top

TrendSpotters:
The Organizational Evaluation Process Inventory
by Carol Haig, CPT, and Roger Addison, CPT, EdD

This month TrendSpotters welcomes Dawn Snyder, CPT, PhD, of Dawn Snyder Associates, Inc. She is currently focused on consulting with clients in business, industry, and government to develop curricula and evaluate training programs. In addition, Dawn, dawn@dawnsnyderassoc.com, is teaching several graduate courses in Workforce Development and Education at The Ohio State University. She joins us to share her insights and expertise in evaluation with the Organizational Evaluation Process Inventory tool.

Genesis of This Tool
When it comes to evaluation, some clients have very limited experience, some have a good structure in place, and others believe they are highly skilled at evaluation. Dawn tells us that regardless of a client’s experience, evaluation is a challenge in most organizations. She needed a way to start the evaluation conversation with clients so that everyone could share concepts and approaches productively for the benefit of the organization and the project. Dawn developed the Organizational Evaluation Process Inventory to help launch the evaluation discussion. It adds value to all levels of evaluation experience, helping to put a structure in place as well as to diagnose any missing components. The Organizational Evaluation Process Inventory is effective with any performance improvement solution.

Tool Description
The tool assumes that a complete analysis was done at project inception and that there is some evaluation infrastructure in place. The Organizational Evaluation Process Inventory contains 20 components of an effective evaluation system expressed as actions. The first 10 actions take place at project launch, the next 10 during the course of the project. The client rates each action on a frequency scale of 1—Never to 5—Always and tallies the responses. Scoring criteria interpret the results and suggest corrective action where needed.

How to Use the Tool
The Organizational Evaluation Process Inventory produces a snapshot of evaluation as it is practiced in the organization. It leads naturally to an exploration of current practices and their effectiveness as well as pointing out missing elements that would help to complete the evaluation system. The Inventory is a versatile and flexible tool. Use it as a conversation starter when assembling a team during the initial project planning activities. Use it also to help determine why a past evaluation did not go as planned or failed to produce useful data. The Inventory is appropriate in any situation where you will do evaluation.

Success Story
Dawn Snyder Associates, Inc. had a client with considerable analysis and curriculum development experience. The client wanted to fully explore evaluation to help move its analysis and curriculum development to the next level. Rather than continue the practice of evaluating some individual courses, the client wanted to create an evaluation strategy for its entire curriculum with a replicable process to be used to evaluate all courses. The Organizational Evaluation Process Inventory provided a comprehensive entry for this project. In particular, the tool helped identify some basics of good evaluation design and paved the way for a discussion that raised critical questions:

• Who in the organization will receive the data produced by the evaluation?
• How will you report your data?
• What do you want to include in your reporting?
• How will you administer your evaluation?
• What resources will you need (databases, people, other records) and how will you access them?

Using the Inventory enabled the client to build its evaluation strategy and replicable process successfully.

Advice to Users of the Organizational Evaluation Process Inventory
Most clients approach evaluation by determining the questions they should ask. Dawn suggests that, as with other components of performance improvement, they begin evaluation planning with the end in mind: What is the purpose of the evaluation? What are stakeholders’ expectations, and how will we meet them?

One caution: Do take the time to find out what an acceptable evaluation result will be to your stakeholders before you begin. As Dawn says, “There is no such thing as a values-free evaluation.” Different stakeholders have different agendas for evaluation. We must find out what they value—what they care about—so we build an evaluation that meets their expectations and that measures what they expect to see, regardless of how favorable or unfavorable the results may be.

Links to the Performance Technology Landscape
The Organizational Evaluation Process Inventory supports these principles of Performance Technology:

Application Exercise
Try the Organizational Evaluation Process Inventory at your next opportunity to discuss evaluation with a client. Or, revisit a previous evaluation experience to diagnose any difficulties you encountered.

We invite you to explore Dawn Snyder Associates, Inc. for more helpful tools and models.

TrendSpotters Open Toolkit
Visit the TOT to view a valuable array of tools and models that you can download for your use. In addition, you may browse all the past TrendSpotters interviews published since March 2002.

You may contact Carol Haig at carolhaig@earthlink.net or at http://home.mindspring.com/~carolhaig; and you may contact Roger Addison at roger@ispi.org.

return to top

Ten Reasons to Use Performance Tests Instead of Knowledge Tests
by Peter R. Hybert, CPT

Many certifications today are based on completion of a knowledge test. A knowledge test samples some of the information required (it is hoped) for performance, and measures whether the learner can recall the majority of it (typically, 80% or more of the sample).

Put that way, knowledge testing does not sound too impressive as a way to accurately predict someone’s capability to perform something. I think a better approach, especially when trying to measure performance, is a performance test.

What Is a Performance Test?
A performance test is when capability to perform is verified either by observing the actual performance of a task or evaluating its output.

For example, cooking a Thanksgiving turkey (or filtering a parenteral, or injectable, pharmaceutical in a sterile environment) requires that the task be observed so that the assessor can verify that safe food handling (or sterile) techniques were used during the process. Just checking the end product, or even testing the countertop (or lab) for contamination, is not sufficient.

On the other hand, reviewing a computer program may simply require a test of the functionality and comparison of the code with standards. You should not have to watch the programmer type it in.

Here are 10 reasons to use performance tests wherever you can.

A performance test measures the right things.
Assuming that your focus is on performance, a performance test is going to tell you what you should want to know. It does not tell you the learner “knows” the rules of the road; it requires the learner to demonstrate that he or she can follow the rules of the road. It shows that the learner can steer and stop the car appropriately, use lanes correctly, follow traffic signals and signs, monitor other drivers’ movements, or any other criteria built into the test.

If the performance test is done well, it is a more comprehensive test of capability than a knowledge test.
Actually performing almost always requires more than simple recall of information or even application of rules. It requires doing a series of things to produce a result. It requires that the performer can deal with information, use tools and resources, adjust for situational factors, and even work with “noise” in the environment. Knowing the traffic laws (the “written test”) is not a good test for whether a teenager can navigate through traffic and make good decisions in the moment.

A performance test specifically defines the work and the criteria for performance.
In almost every case where we have developed performance tests (including in work environments where detailed procedures and SOPs exist for every task), we have created new knowledge. That is, we have identified or clarified tasks, techniques, or sequences that were missing or incorrect in existing documentation or in practice. Often, the missing piece is the criteria for performance. This knowledge is valuable to the business.

Clarifying performance requirements often leads to simplification of the performance by focusing on key criteria. It takes some of the mystique out of “mastery” but makes it easier for all performers to perform effectively.

Performance tests connect training to performance.
Defining the performance requirements provides a clear linking point for training content. This level of definition around performance requirements will guide decision making about the amount and type of practice exercises, simulations, and even knowledge content needed in the training.

You do not have to hide the answers.
A performance test, including the key performance criteria, can be freely published. With a performance test, just because you know what is expected does not mean you can do it. So there is no need to hide or randomize the questions and answers. (This is why the test instrument can be designed to be used by learners as a job aid.)

You can get work done during the testing process.
In a business situation, the performance test is usually administered by a “master performer” (who has been trained and qualified to administer performance tests). So while the learner is being tested, he or she is doing real work, but under qualified supervision. Though it may be at a slower rate, it still results in output.

You can manage learner expectations for “going solo.”
Instead of the learner “watching and learning” with the master performer for some undetermined period of time, a specific date is identified and the learner, master performer, and supervisor have a clear point in time for when the learner is, or should be, ready to go solo.

Performance tests are inherently valid.
Assuming that the test is based on the actual work requirements, it will automatically meet a key requirement for validity—that it matches work requirements. Knowledge tests require a great deal more work to prove they are valid, non-biased, and so forth.

Performance tests can be developed rapidly.
Part of any testing or training development effort should include analysis of the work; after all, you cannot build very much content until you know what people are supposed to be doing and how to recognize it when you see it. From that point, creating the performance test is incremental effort; the hard work is the analysis, which you would have to do anyway.

Performance tests serve multiple purposes.
Performance tests are “re-usable objects.” They can be used for testing in the field or in a simulation setting. They can be used as coaching tools because they describe specifically what to do. They can be used as job aids for the same reason. They can even be built into procedure documents, either as the actual procedure or as supplementary information to specific tasks.

Peter R. Hybert, CPT, is principle consultant of PRH Consulting Inc., a human performance consulting firm that focuses on “leveraging know-how for performance” through training, knowledge management, tools, and process interventions. He has been an ISPI member since 1984 and has presented and published in various training and HPT venues. He may be reached at pete@prhconsulting.com. For more information or other articles, please check out the library at www.prhconsulting.com or visit www.prhconsulting.com/blog.

return to top

Advertisement  Would you like to advertise in this space? Contact marketing@ispi.org return to top

Bright Lights, Big City, ISPI: Join Us in NYC for The Performance Improvement Conference

The International Society for Performance Improvement is proud to bring you The Performance Improvement Conference, April 5-8, 2008, in New York City. The conference theme, Enhancing Knowledge, Know-How, & Results, reflects how we put our knowledge to work, and increase our ability to accomplish valued results.

Why should YOU attend The Performance Improvement Conference?

• Learn from educational sessions by presenters who are leading the way in workplace performance improvement and development
• Obtain timely research, real-world case studies, and successful strategies
• Acquire knowledge to align workforce development with overall business objectives
• Become a part of a network of performance improvement practitioners discussing best practices
• Takeaway relevant tools and job aids to share with your clients and colleagues
• Participate in an experience that lasts long after the conference ends

The Performance Improvement Conference begins with a premier line-up of half-, one-, and two-day workshops (April 4 & 5), along with two three-day HPT Institutes (April 3-5). The conference expands on Sunday, April 6, into three days of nearly 150 educational sessions and forums. As an attendee, you will enjoy many opportunities to learn from current and future leaders in the field of performance improvement from around the world.

You may download the conference brochure and see for yourself all the opportunities available to enhance your knowledge, know-how, and results by attending The Performance Improvement Conference!

Reserve Your Sleeping Room Now
ISPI’s Special Hotel Rate at the New York Marriott Marquis Times Square is one of the hottest in the city. Reserve your room today while space is available: $242 Single/Double and $271 Triple. The hotel is within walking distance of the Broadway theater district, Fifth Avenue/Madison Avenue shopping, Radio City Music Hall, Rockefeller Center and the NBC Studios, St. Patrick’s Cathedral, and Central Park. Also nearby, you will find Carnegie Hall, Lincoln Center, the United Nations, and the Empire State Building.

Attention 2007 Conference Attendee
If you attended ISPI’s Annual Conference in San Francisco earlier this year, you could come to NYC for as low as $125. Call ISPI for details: 301.587.8570.

For more information, visit www.ispi.org/AC2008, or register today.

return to top

From the Board
Online Communities: Only Connect?
by Timm J. Esque, CPT

One purpose of ISPI and any professional organization is to connect the members of its community. One simple example of how ISPI partly accomplishes this is through this publication, which is available monthly to Society and Chapter members and thousands of others in the larger performance improvement community. The one-way broadcast continues to be a useful impetus for connecting but, of course, falls far short of ensuring it. Chapter meetings, conferences, committee meetings, and a variety of other face-to-face forums exist partly to complete these connections.

Anyone digitally plugged in probably has a pretty good idea that connecting means much more than face-to-face meetings these days. We are living in the network age and all signs are that it is going to revolutionize the ways we connect, communicate, organize, and, it is hoped, the way we get things done. Some would say it already has. As we are frequently reminded these days, the younger generations consider conventional means of connecting and collaborating rather slow (and dull). Of course, you do not have to be young to be inpatient with email and teleconferencing, but it seems to me they are leading the way in inspiring and testing connection, communication, and collaboration tools that take full advantage of our rapidly evolving information technology.

I bring this up because ISPI will continue honing in on just the right online community and connecting tools this coming year. Last year a conference blog was set up to help communicate and market what to expect at the conference in San Francisco. Some of the people who worked on the conference blog are leading an effort to implement even more connective and collaborative online tools leading up to the conference in New York. Stephanie Wawrykowicz led the task force that looked at the feasibility of an ISPI “wiki” and produced a proposal to the Board over the last few months. Brian Grant, who participated on last year's blog effort, is heading up the team that will evaluate and implement online tools in support of this year’s conference. Brian has written and presented extensively on the topics of social networks and social capital. He believes that an online identity is an important step in the connection process and that the focus needs to be on social networking. Among other things, his task force will be getting some experienced online community users to help evaluate and break in specific capabilities.

So what is the purpose of these efforts? My title for this column plays on the oft-quoted E.M. Forster epigram, “Only Connect” (from his novel Howard’s End set in Victorian England). I have always interpreted Forster’s message to be about the importance of really connecting to (seeking to understand) our fellow human beings. Just shy of 100 years since Howard’s End was first published, I think to connect today (especially online) means both more and less than what Forster had in mind. It means less in the sense that when we are connecting with others in the network sense, we often have less than total and mutual understanding in mind. Sometimes online networks are handy tools to find others who have information we need and that is about it. Social networking, as I am just beginning to understand it, seeks to connect those that have some sort of direct or indirect affiliation (could be professional, hobby-based, geographical, and so forth) and then maximize the benefits of the resulting connections. But what really intrigues me about these connecting tools is how they can produce more than just connections.

As I have attempted to get literate on these ideas and tools, I have come to realize that one of the key purposes of many of these online tools (social networks, wikis, and so forth) is a shortcut to accomplishment. When these tools are used correctly, they are not just facilitators of communication and group process, they produce outputs. There is a demonstration of what I mean in a brief explanatory video on wikis at www.commoncraft.com. In this example, the wiki is not just a repository for ever more refined and critiqued knowledge, it is tool that produces collaborative output in support of any sort of project or process (in this case an assignment of camping equipment responsibilities). As the early adopter members of the first task force pointed out to me, this means that it becomes possible for those members of our community who cannot dedicate 90 minutes of their workday to ISPI meetings to still participate and get involved in a meaningful way (helping get stuff done).

But I think the purpose of ISPI implementing these tools is even bigger than society connectedness and maintenance. At the moment, the outputs produced with these tools are mostly restricted to ideas (plans, reports, conceptual problem solving) but who is to say that will always be the case. I do not think it is too big of a leap to suggest that someday tangible, three-dimensional outputs will be produced the same way by people who make their contributions from different locations and when it is convenient for them to do so. I do not claim to have special insight into the direction online connectivity is going to go. I have not been an early adopter of these tools, and I suspect I am doomed to stay in the “adapt if I have to” group. But there is no doubt in my mind that a professional society that is dedicated to the understanding of how people best produce valued outputs needs to take a very keen interest in online connectivity.

If you would like to contact Brian Grant in regard to our online communication efforts, he may be reached at brian_s_grant@raytheon.com. Or, please feel free to contact me, Tjesque@ensemblemc.com. So, watch for news on this year’s conference and please connect!

return to top

ISPI Executive Director Retires

At the end of this month, Richard Battaglia will be stepping down from his position as Executive Director & CEO of the International Society for Performance Improvement after nearly 14 years of service. When Rick first arrived, it was a somewhat turbulent time for the Society. His strong fiscal management was a necessary and much-appreciated approach to steering the organization through some financially-tenuous waters. Rick was instrumental in transforming ISPI from one that was primarily staffed by volunteers to one that has a professional staff that services most of the administrative and logistical planning and tasks. This allows our members to concentrate on creating, sharing, learning, and networking.

In Rick’s early years, ISPI mainly relied on membership and conference fees. Now, with help from him and some insightful former Boards, we have a more diverse revenue stream. Our educational offerings now include the HPT Institutes (public, online, and in-house), the ProSeries, and the Fall Conference. Rick was also a supporter of our name change from NSPI to ISPI, restructuring the Awards Program to make it more criterion-referenced, developing our certification program, initiating the partnership with IFTDO, creating ISPI's Book Program and then transitioning it to Wiley, moving News & Notes to PerformanceXpress online, and also for any number of additional initiatives that are too numerous to list here.

We are grateful for his years of service and wish him well.

April Davis, CAE, who has been with ISPI for nearly 10 years, most recently as Associate Executive Director, will be taking the helm as our Interim Executive Director. The Board of Directors and Rick have confidence in April, and she is enthusiastic about working with the Board, staff, and members to steer ISPI through this important transition.

return to top

CPT News from Around the World

The deadline for submitting an application to become a CPT is November 15. If you would like to apply, consider participating in one of the monthly conference calls with Judy Hale, director of certification. You may also contact her directly at 630.427.1304 or at judy@ispi.org.

In preparation for the coming season of reviewing applications, Judy has trained more than 20 CPTs in the last month on how to be a reviewer. This brings the number of trained reviewers to 90. All applications are reviewed by two trained CPTs.

Special Work by a CPT—A New Perspective on How We Do Our Work
While Ken Silber, CPT, is consulting with a client, Dr. Kenneth H. Silber, PhD, is watching him and wondering how he is thinking about identifying and solving the client’s HPT problem. This is less an indication of Ken's schizophrenic state, and more an indication of his interest in how instructional designers and human performance technologists think, learn, and work.

In a 1981 article, Dr. Silber speculated that instructional design was not a procedure but rather was a meta-cognitive strategy. In his just published article in Educational Technology, he updated this thesis using an analogy with architectural design and described ID as ill-structured problem solving.

Ken interpreted this finding for us as follows. When we perform a procedure, we know the current state, the desired state, the possible actions, and the constraints. A procedure is also known as “well-structured problem solving.” On the other hand, when we perform ill-structured problem solving, the current state we are given is unlikely to be the real problem, the desired state is ambiguous, the possible actions are ones that we have to create, and the constraints are ones that we have to discover.

It seems, given the kinds of problems IDs and HPTs are called on to solve, that we work in the domain of ill-structured problem solving. This means that a step-by-step procedure, such as that described in a flowchart of the ID or HPT process, does not describe what an expert in our field does.

Instead of a flowchart, Dr. Silber developed an ID model that consisted of a series of principles which make up the mental model of an instructional designer. Ken explains that what experts use to solve problems in our field is a set of principles that guide their thinking as they work their way through a problem. In HPT, for example, those principles might be ISPI’s 10 Standards of Performance Technology.

He had help in the development of the ID model from his 2006 and 2007 sessions at the ISPI conference, during which he presented his preliminary ideas and engaged in vigorous debate with participants.

This new way of thinking about ID also has implications for how we teach it. Traditionally, ID is taught as a series of steps that one must follow (the ADDIE Model). First, one learns all the steps in the procedure, and then one learns that one does not do all the steps all the time in the same sequence. Not only is this a wasteful approach to teaching, but research indicates that it does not work. The approach that does work is to teach the principles and then to teach guidelines for using them. Dr. Silber is using this approach in the ID class he is now teaching at NIU.

Over the summer, Ken (or was it Dr. Silber) did some light some reading of the bestseller How Doctors Think. It argues that medical doctors also do not use algorithms to diagnose, but rather use principles as guidelines. Encouraged by the fact that another field like ours, involved in diagnosis and solution of problems, is thinking along the same lines, Ken and Dr. Silber are currently collaborating on the development of a similar principle-based model of HPT for presentation at the 2008 ISPI conference. The model will attempt to identify the principles that expert HPT practitioners use to define and solve performance problems. Because they are familiar to all ISPI members, the 10 Standards will serve as the organizing principle for the model. Under each performance standard, the model will list the principles identified in the HPT literature.

For example, under Take a Systems View, one can identify the following principles an HPT expert uses:

• Identifying the current work, workplace, or market environment in terms of how it affects organizational and group performance leads to a more complete and accurate diagnosis of the organizational problem (ISPI Std. 2)
• Identifying if there is a lack of alignment among goals and objectives, performance measures, rewards and incentives, and so forth, leads to a better diagnosis of organizational problems and underlying causes (ISPI Std. 2)

Although these may look like the behaviors in the CPT performance standards, Ken explains the difference. First, they are actually not things HPT experts do, but rather are things they keep in their mind as they solve problems. Second, they are not done in any sequence, but are principles HPT experts may apply at any point in the problem-solving process, and in fact will probably apply many times throughout the process. Third, the principles are applied only when appropriate, and only those principles appropriate to any problem are applied to that problem.

Fourth, experts in HPT are better at deciding which principles apply in a given situation and have a larger repertoire of principles from which to choose when faced with a new problem. Dr. Silber points out that this difference was demonstrated by Hardre et al. in a recent article in ISPI’s PIQ journal.

This notion has implications for how we teach others to do HPT. Just as with teaching ID, teaching a procedure is neither efficient nor ineffective. Instead, one must teach the HPT principles that underlie our technology, and then teach guidelines for deciding which principles to apply to which problem situations.

To learn more, you may contact Dr. Silber at ksilber@niu.edu or Ken at wiseoldken@aol.com, or read A Principle-Based Model of Instructional Design: A New Way of Thinking About and Teaching ID (Silber, K.H. [2007]. Educational Technology, XLVII (5), 5-19).

Dr. Ken Silber is the co-author of Silber, K.H., & Kearny, L. (in press). Seeing Organizations through Business Glasses: A Guide to Understanding and Adding Value to an Organization. San Francisco: Jossey-Bass/Pfeiffer; and Foshay, W. R., Silber, K. H., and Stelnicki, M. B. (2003). Writing Training That Works. San Francisco: Jossey-Bass/Pfeiffer. He is co editor of Silber, K.H., and Foshay, W.R. (Eds.). (In press). ISPI Handbook of Training and Improving Workplace Performance: Volume I: Instructional Design and Training Delivery. San Francisco: Jossey-Bass/Pfeiffer.

Your Story
If you have a story to tell that you think others would value, send it to judy@ispi.org.

return to top

Project Proven Tools and Techniques for ISD:
Pilot-Testing Guidelines and Tools for ISD
by Guy W. Wallace, CPT

The 11th topic we are covering in this 12-part series is Pilot-Testing Guidelines and Tools for ISD.

Pilot-Testing
One phase of my ISD effort is the pilot-test phase. Instructors, facilitators, and administrators prepare materials and themselves to conduct the pilot test.

As part of the preparation, the learning experience is described to managers of pilot participants. The managers are also informed of any post-pilot activities that must be completed to ensure successful transfer of knowledge and skills to the actual job.

The Pilot-Test Deployment Team
The purpose of the pilot-test deployment team is to conduct a pilot test following the initial development of the T&D. The pilot-test deployment team includes instructors, facilitators, and administrators who conduct the pilot session. The pilot-test deployment team is used in levels 2 and 3 of my approach to ISD.

ISD team members coordinate all logistics for facilities, equipment, media, food and beverages, invitations, and confirmations for the attendees. They also deliver the instruction or oversee the instructional delivery and participation for the purposes of pilot testing. Finally, they conduct written and verbal evaluations and debriefings to gather feedback for revision purposes.

Along with the roles of facilitators and instructors, another role is crucial for the conduct of a pilot test: the role of the pilot-test participants. Participants attend and evaluate the initial delivery of the T&D for the purpose of generating evaluations and revision recommendations; the project steering team considers these evaluations and recommendations.

The Pilot-Test Team
Pilot-test participants are handpicked by the project steering team to create a balance between

• Target audience representatives
• Management representatives

Target audience representatives are from the pool of eventual learners who will participate in the T&D after the pilot. They are used to measure the amount of learning that occurs. Management representatives (aka management spies) are handpicked by the project steering team to participate in the test. They are used to determine whether the right learnings are taught to an acceptable level. In combination, the two perspectives give the ISD team the right data to determine what happened well and what did not.

Conducting the Pilot-Test
During the pilot-test session, written and verbal evaluations are collected and debriefings are conducted. After the session, the feedback is assessed and revision recommendations are generated for consideration by the project steering team. The project steering team may accept, modify, or reject the revision recommendations. The final results constitute the revision specifications used in the final phase of an ADDIE-level ISD project: Revision & Release.

Evaluations
The types of evaluations I use in pilot-testing come from this family of evaluations:

• End-of-lesson written evaluations—used after every two or three lessons
• End-of-day debriefings—done at the end of each day or PT period
• End-of-event written evaluations—used after every T&D event
• End-of-event debriefings—done after every T&D event

Of course, how you implement the above evaluations will depend on the delivery platforms you must pilot-test.

Summary
The purpose of the pilot-test is to conduct a “full destructive test” following the initial development of the T&D content, before it is updated and the released— “pushed to” and/or “pulled by” the various target audiences.

The last article in this series will be Tips for Recruiting and Working with a Powerful ISD Project Steering Team.

Note: An expanded version of this article and the prior articles of this series may be found in the documents area of the IS ProComm. Click here to visit the website.

Guy W. Wallace, CPT, has been an external ISD and HPT consultant since 1982, is the president of EPPIC Inc., has been a member of ISPI since 1979, is a past president of ISPI, is the author of lean-ISD, and is a recipient of an ISPI 2002 Award of Excellence. He may be reached via guy.wallace@eppic.biz, and related resources may be obtained at his website, www.eppic.biz, including his three most recent books available as free PDFs: lean-ISD, T&D Systems View, and new in 2007, Management Areas of Performance.

return to top

ISPI Member Raymond E. Urgo Honored with Distinguished Service Award

Raymond E. Urgo was recently honored with the Distinguished Service Award by the Board of Directors of the Society for Technical Communication in recognition of his dedicated leadership and service to the Policies and Procedures Special Interest Group. Urgo founded STC’s Policies and Procedures Special Interest Group in 1993 and led its growth to 1,200 members in 24 nations—the world’s only nonprofit group of policies and procedures professionals. He holds STC’s highest membership rank of fellow for his distinguished contributions to the society and the profession.

return to top

Listed below are the most recent Job Postings that have been added to ISPI’s newly designed Job Bank, click here to view all jobs.

Center for Creative Leadership
Sr. Learning Design Faculty
Job Location: United States, North Carolina
Job Type: Permanent, Full-Time

Insignia Training Partners, LLC
Instructional Designers
Job Location: United States, All
Job Type: Contract, Part-Time

Franklin Templeton Investments
Sr. Performance Consultant
Job Location: United States, Florida
Job Type: Permanent, Full Time

Premera Blue Cross
Manager, Workforce Development
Job Location: United States, Washington
Job Type: Permanent, Full Time

Textron
Training Consultant
Job Location: United States, Rhode Island
Job Type: Permanent, Full-Time

return to top

San Diego : Sun, Fun, and ISPI

Before “lean”, Six Sigma, knowledge management, or one-minute cures, our presenters were breaking new ground in the principles and practices of performance technology. Participants will work with proven tools and techniques and learn new approaches to enhance value for both the organization and the client.

	Geary Rummler’s Introduction to Serious Performance Consulting will take you beyond job-level performance improvement for individual workers to an exploration of the process and organizational levels of performance improvement where HPT practitioners can make a lasting contribution to their organizations.
	Margo Murray’s Managing Mentoring Processes for Measured Results will provide guidelines and practice in the front-end strategies, needs/readiness assessment, and back end evaluation, and continuous improvement, to create sustainable mentoring processes.
	No matter what your role in any organization, Carl Binder’s Six Boxes™ Performance Improvement and Introduction to FluencyBuilding™—a combination of practical, results-driven methodologies—will give you new perspectives, insights, research-based tools, and quick start methods for immediate application.
	Donald Tosti’s Organizational Performance: Focus on Results will draw on the years of research and development within HPT to introduce a common framework which facilitates the understanding of the relationships between HPT and virtually all other forms of organizational consulting.

These workshops are limited in size to ensure participants receive individual attention from presenters and quality time to interact with other attendees. Each attendee will have the opportunity to participate in a pre- and post-workshop conference call. Upon completion of the program, you will receive a Certificate of Completion. In addition, you can earn up to 12 CPT points toward re-certification. For more information, visit www.ispi.org/ProSeries.

return to top

Performance Marketplace

Performance Marketplace is a convenient way to exchange information of interest to the performance improvement community. Take a few moments each month to scan the listings for important new events, publications, services, and employment opportunities. To post information for our readers, contact ISPI Director of Marketing, Keith Pew at keithp@ispi.org or 301.587.8570.

return to toprn to top

ISPI Membership: Join or Renew Today!

Are you working to improve workplace performance? Then ISPI membership is your key to professional development through education, certification, networking, and professional affinity programs.

If you are already a member, we thank you for your support. If you have been considering membership or are about to renew, there is no better time to join ISPI. To apply for membership or renew, simply click here.

Newsletter Submission Guidelines

ISPI is looking for Human Performance Technology (HPT) articles (approximately 500 words and not previously published) for PerformanceXpress that bridge the gap from research to practice (please, no product or service promotion is permitted). Below are a few examples of the article formats that can be used:

• Short “I wish I had thought of that” Articles
• Practical Application Articles
• The Application of HPT
• Success Stories

In addition to the article, please include a short bio (2-3 lines) and a contact e-mail address. All submissions should be sent to april@ispi.org. Each article will be reviewed by one of ISPI’s on-staff HPT experts, and the author will be contacted if it is accepted for publication. If you have any further questions, please contact april@ispi.org.

About PerformanceXpress

Feel free to forward ISPI’s PerformanceXpress newsletter to your colleagues or anyone you think may benefit from the information. If you are reading someone else’s PerformanceXpress, send your complete contact information to april@ispi.org, and you will be added to the PerformanceXpress emailing list.

PerformanceXpress is an ISPI member benefit designed to build community, stimulate discussion, and keep you informed of the Society’s activities and events. This newsletter is published monthly and will be emailed to you at the beginning of each month.

If you have any questions or comments, please contact April Davis at april@ispi.org.

return to top