October 2000
Shortell, SM, Bennett, CL, & Byck, GR. (1998). Assessing the impact of continuous quality improvement on clinical practice: What will it take to accelerate progress? The Milbank Quarterly 76: 593-624.
Usually this feature highlights a recently published article. This month the feature is hot because of its relevance to PainLink’s mission. I came across the article in doing my dissertation and it has informed much of my work on PainLink. This is a must read for any Pain Management Champion who has ever felt frustrated, even momentarily—whether with the pace of your initiative, backsliding after reaching a goal, lack of resources or support, or other barriers. It is a rich article; these are simply highlights. PainLinkers can go to the private pages to view a slide presentation on institutional change that I made incorporating the content from this article.
Article Summary
The purposes of the article were to examine the evidence of clinical uses of continuous quality improvement (CQI), outline its strengths and weaknesses as an approach to improving practice, and to discuss the role of CQI as part of an integrated strategy for improving practice.
The authors define CQI "as a philosophy of continual improvement of the processes associated with providing a good or service that meets or exceeds customer expectations" (p. 594). To determine the impact of CQI, they reviewed literature that met author-defined criteria for best evidence: strength of the research design, quality of the data collected, and the relevance of the findings to improving clinical practice. They reviewed 42 single-site studies and 13 multi-site studies. Of the 55 studies, 44 had been conducted in inpatient settings and 11 in outpatient/primary care settings. They selected studies that addressed three sorts of quality problems: overuse (15 studies), underuse (7 studies), and misuse (30 studies). Three studies addressed both underuse and overuse.
Analyses of these studies focused on the following areas:
- settings and problems addressed (e.g. surgical procedures, emergency room care)
- types of problems (usually interdisciplinary though physician involvement varied widely)
- study designs (two single-site studies were randomized trials, the rest were pre/post designs)
- interventions (problem-focused, most often provider training and education and information dissemination)
- results (mainly positive findings).
Only one of the studies was focused on pain and this is not discussed in detail. The analyses inform the conclusions drawn by the authors which are:
- reports (that meet the authors’ criteria) documenting the impact of CQI in clinical practice are few
- the design limitations of CAI studies make it difficult to know whether CQI efforts were responsible for the improvement or whether other explanations account for positive results
- most studies address misuse; overuse or underuse are less often the focus
- most studies report favorable results
While recognizing the design limitations of most of the studies analyzed, the authors identify the following factors that seem to be correlated with successful CQI efforts: involvement of several physicians, feedback to individual clinicians, and a supportive organizational culture. Factors associated with failed CQI efforts: choosing a problem less likely to be affected by CQI because of the difficulties involved in implementation, nonacceptance by local physicians of national guidelines, insufficient or ineffective dissemination, and vague feedback to clinicians.
Drawing on work by O’Brien et al., (1995) and their own analyses, the authors urge consideration of four interrelated dimensions of organizations that must be addressed to ensure that CQI results in significant, institution-wide improvement: strategic, cultural, technical, and structural. The strategic dimension refers to an organization’s mission, and priorities—what is most important to the organization. The cultural dimension includes the beliefs, values, norms and behaviors—characteristics of the organization (and the people within) that can promote or undermine CQI efforts. The technical dimension refers to the preparation of staff and the information support systems needed to support CQI. The structural dimension includes the mechanisms needed to promote learning and ensure adoption of best practices throughout the organizations such as task forces, standard communication strategies, and other methods. What I found interesting is the authors’ identification of the "type of failure" that results when just one of these dimensions is not addressed. Pain Management Champions will identify with these:
- no strategic dimension, no impact on anything important
- no cultural dimension, no one notices anything, there may be temporary effects but nothing of lasting value
- no technical dimension, lots of frustration and false starts
- no structure, no dissemination
The authors go on to explain what some of the barriers are to making headway in addressing these dimensions of organizations effectively. Strategically, the barrier may arise form an organization’s failure to make CQI a central part of its planning (e.g., budget, staff, organizational priority). Two common barriers to managing the cultural dimension are a focus on professionals’ needs rather than the customers’ needs or when physicians do not become involved (regardless of the reason—time constraints, inexperience, resistance). The most common technical barriers are the lack of team-based, problem-oriented training and the absence of opportunities for ongoing training and development. The authors conclude with a diagram of a "capability wheel" and strategies for incorporating CQI in an integrated approach to improving the health of individuals and communities.