n a v i g a t i o n
m a i n
Insights to Performance Excellence
Choose a link:
Self-assessments of organization performance and management systems take several forms, ranging from rigorous and time intensive, to simple and somewhat superficial. This section discusses the various approaches to organizational self-assessment and the pros and cons of each. Curt Reimann, the first director of the Malcolm Baldrige National Quality Award Office and the closing speaker for the 10th Quest for Excellence Conference, spoke of the need to streamline assessments to get a good sense of strengths, areas for improvement, and the vital few areas to focus leadership and drive organizational change. Three distinct types of self-assessment will be examined: the written narrative, the Likert scale survey, and the behaviorally anchored survey.
Full-length Written Narrative
The Baldrige application development process is the most time-consuming organizational self-assessment process. To apply for the Baldrige-Award, applicants must prepare a 50-page written narrative to address the requirements of the Performance Excellence Criteria. In the written self-assessment, the applicant is expected to describe the processes and programs it has in place to drive Performance Excellence. The Baldrige application process serves as the vehicle for self-assessment in most state-level quality awards. The process has not changed since the national quality award program was created in 1987(except for reducing the maximum page limit from 85 to 50 pages).
Over the years, three methods have been used to prepare the full-length, comprehensive written narrative self-assessment:
1) The most widely used technique involves gathering a team of people to prepare the application. The team members are usually assigned one of the seven Categories and asked to develop a narrative to address the Criteria requirements of that category. The category writing teams are frequently subdivided to prepare responses Item by Item. After the initial draft is complete, an oversight team consolidates the narrative and tries to ensure processes are linked and integrated throughout. Finally, top leaders review and scrub the written narrative to put the best pin on the systems, processes, and results reported
2) Another technique is similar to that previously described. However, instead of subdividing the writing team according to the Baldrige categories, the team remains together to write the entire application. In this way, the application may be more coherent and the linkages between business processes are easier to understand. This approach also helps to ensure the consistency and integrity of the review processes. However, with fewer people involved, the natural "blind spots" of the team may prevent a full and accurate analysis of the management system. Finally, as with the method previously described, top leaders review and scrub the written narrative.
The third method of preparing the written narrative is the least common
and involves one person writing for several days to produce the application.
Considering the immense amount of knowledge and work involved, it is easy
to understand why the third method is used so rarely.
organizations report that the discipline of producing a full-length written
self-assessment (Baldrige application) helped them learn about their organization
and identify areas for improvement before the site visit team arrived.
The written narrative self-assessment process clearly helped focus leaders
on their organization's strengths and areas for improvement--provided
that a complete and honest assessment was made.
Because the application is closely scrutinized and carefully scrubbed and because of page limits, it may not fully and accurately describe the actual management processes and systems of the organization. Decisions based on misleading or incomplete information may take the organization down the wrong path.
Although the written self-assessment provides information to help guide a site visit, examiners cannot determine the depth of deployment because only a few points of view are represented in the narrative.
Finally and perhaps most importantly, the discipline and knowledge required to write a meaningful narrative self-assessment is usually far greater than that possessed within the vast majority of organizations. Even the four 1997 Baldrige winners hired expert consultants to help them prepare and refine their written narrative.
Short Written Narrative
Two of the most significant obstacles to writing a useful full-length written narrative self-assessment are poor knowledge of the Performance Excellence Criteria and the time required to produce a meaningful assessment. If people do not understand the Criteria, it takes significantly longer to prepare a written self-assessment. In fact, the amount of time required to write an application/assessment is inversely related to the knowledge of the Criteria possessed by the writers. The difficulty associated with writing a full-length narrative has prevented many organizations from participating in state, local, or school award programs.
To encourage more organizations to begin the performance improvement journey, many state award programs developed progressively higher levels of recognition, ranging from "commitment" at the low-end, through "demonstrated progress," to "achieving excellence" of the top of the range. However, even with progressive levels of recognition, the obstacle of preparing a 50-page written narrative prevented many from engaging in the process. To help resolve this problem, several state programs permit applicants who seek recognition at the lower levels to submit a 7- to 20-page "short" written narrative self-assessment. (Most states still require applicants for the top-level award to complete a full-length written self-assessment.) The short form ranges from requiring a one-page description per category to one-page per Item (hence the 7- to 20-page range in length).
clearly takes less time to prepare the short form.
The short form provides significantly less information to help examiners prepare for the site visit. Although it does take less time to prepare than the full-length version, the short form still requires several hundred hours of team preparation.
short form is usually closely scrutinized and carefully scrubbed just
as its full-length cousin. This reduces accuracy and value to both the
organization and examiners.
The Likert Scale Survey
Just about everyone is familiar with a Likert Scale survey. These surveys typically ask respondents to rate, on a scale of 1 to 5, the extent to which they strongly disagree or strongly agree with a comment.
The following is an example of a simple Likert scale survey Item:
leaders effectively communicate values and customer focus.
A variation on the simple Likert scale survey Item has been developed in an attempt to improve consistency among respondents. Brief descriptors have been added at each level as shown in the following descriptive Likert scale survey Item:
leaders effectively communicate values and customer focus.
The Likert scale survey is quick and easy to administer. People from all functions and levels within the organization can provide input.
Both the simple and the descriptive Likert scale survey Items are subject to wide ranges of interpretation. One person's rating of 2 and another person's rating of 4 may actually describe the same systems or behaviors. This problem of scoring reliability raises questions about the accuracy and usefulness of both the simple and the descriptive survey techniques for conducting organizational self-assessments. After all, a quick and easy survey that produces inaccurate data still has low value. That is the main reason why states have not adopted the Likert scale survey as a tool for conducting the self-assessments, even for organizations in the beginning stages of the quality journey.
The Behaviorally Anchored Survey
A behaviorally anchored survey contains elements of a written narrative and a survey approach to conducting a self-assessment. The method is simple. Instead of brief descriptors, such as "strongly agree/strongly disagree" or "none-few-some-many-most," a more complete behavioral description is presented for each level of the survey scale. Respondents simply identify the behavioral description that most closely fits the activities in the organization.
A sample follows:
Making Improvements Based on Progress Reviews
1F) How do leaders use the results of progress reviews?
= Not evident
= Basically Effective
= Role Model