Innovations in End-of-Life Care
an international journal of leaders in end-of-life care

Professor Irene J. Higginson, B.Med.Sci., B.M.B.S., F.F.P.H.M., Ph.D. is a palliative medicine physician who has been researching audit and outcome measures in palliative care for more than fifteen years. For the past three years she has been based at King's College School of Medicine and Dentistry and St. Christopher's Hospice in London, England. In this interview with Innovations associate editor Anna L. Romer, she discusses the purpose of audit and the evolution of two quality of care measures she has designed: the Support Team Assessment Schedule (STAS) and the recently developed Palliative Care Outcome Scale (POS). The nine core items of the STAS are Pain Control, Other Symptom Control, Patient Anxiety, Family Anxiety, Patient Insight, Family Insight, Communication between Patient and Family, Communication between Professionals, and Communication Professional to Patient and Family. Staff members grade each of these domains using a forced choice severity scale of 0 to 4. The actual instrument just lists the item, for example, "Pain Control" and a few words to cue the person using the form "Effect of his/her pain on the patient." The POS, or Palliative Care Outcome Scale is a new tool that represents an evolution in practice. A key difference between the two tools is that the POS has two parallel instruments one for a staff member to complete and the other for the patient to complete. In terms of content, the POS includes many of the same domains, but frames the questions more specifically and so has the potential to elicit more detailed and useful information.

Dr. Higginson describes the evolution of the two measures and reflects on the interface between quality of care, i.e., a measure of the care palliative care teams provide based on their own assessments of patient needs and experience, and quality of life, a term grounded in patients' subjective experience. Dr. Higginson details her hopes for the Palliative Care Outcome Scale, a tool that attempts to address this gap between health care professionals' perspectives and patients' perspectives by using two instruments with similar items for staff and patients to complete separately. [Citation: Higginson I. Measuring the Quality of Care in Palliative Care Services: An interview with Irene Higginson, by AL Romer, Innovations in End-of-Life Care, 2000;2(1), www.edc.org/lastacts]

How do you understand audit and why audit is important? Is audit different from evaluation?

We would define clinical audit as the systematic assessment of the quality of care a service is delivering. In simple terms, audit is finding out whether you are doing the right thing well. Evaluation is sometimes finding out whether you are doing the right thing at all. It is slightly different in that way. Another difference is that in audit you have a cycle, in which you assess whether things work, and whether you are doing the right thing well, and then, feed back the results to improve care. The idea is that once you identify what is not working, you make a change in practice to improve things, and then observe the outcomes. It is a continuous cycle. That is the main way in which audit differs from evaluation. If you are evaluating a service, you would ordinarily stop at finding the result and would not do anything further.

Are you suggesting that audit includes evaluation but goes further?

Yes, but they also differ in scale. Audit is something that everyone can do on a small scale. You can audit the care of one person or ten people or what a service is doing. The idea is you are determining whether a particular intervention or treatment works for the group of people that you are considering and then once you know, you can improve things. Whereas, in evaluation studies, you are trying to decide whether you should be doing any treatment at all for a given problem, so you usually need a lot of cases and a lot of people.

Audit sounds similar to what we call Continuous Quality Improvement (CQI) in the United States. Can you build a bridge between these terms?

In fact, there is a great deal of overlap between the terms. In many instances, you would regard a clinical audit as one particular section of a larger quality improvement program. So, a quality improvement program might have six or seven audits included within it. For example, you might have an audit looking at access to a service, another audit looking at the clinical management of care, and another audit looking at bereavement care. They would all be part of one continuous improvement program.

In 1980, Charles Shaw published a series of articles in the British Medical Journal about this concept.^1,2,3,4,5 In that series, he used a number of terms, which add up to the same concept, i.e., a cycle of activities designed to improve practice. You decide what you think you should be doing, find a way to review it, and then feed the results back into practice to improve patient and family care.

That cyclical approach is generally what people call "audit," or sometimes "quality improvement." I could have renamed my book "Quality Improvement in Palliative Care."⁶ If I had, I might have gone wider and looked at some of the non-clinical aspects of care. In other words, it might then have included some of the more organizational things, such as how you train staff or where you put staff. These activities do not tend to come under the more narrow rubric of clinical audit.

Please tell us about the particular tool you designed almost ten years ago to audit palliative care services: the Support Team Assessment Schedule (STAS). What were your goals when you began?

We came up with the idea of finding ways to review what the clinical services were doing, looking at whether their goals were achieved or not, and then looking at how we might improve those results. If you're deciding how to review a service, the most standard way of doing that is to use an approach that was originally developed by Avedis Donabedian, in Michigan.^7,8 He approached care by looking at structure, process and outcomes. The structure of care refers to the staff and the resources that you have; the process of care refers to what you do with the resources, i.e., how many staff visits you do, how many patients you see, and so forth. The outcome of care or the result of care refers to such things as pain, quality of life, how people feel or how satisfied they are with the service they've had, and whether they get better or worse. I felt that the most important aspect of Donabedian's model is outcome, because it refers to what the person who receives the service actually feels at the end of the day.

I felt that we should try and break down what palliative care services were trying to do by looking at the components of the outcomes that they should include. So, we worked with a group of palliative care teams and interviewed some patients and families about what they hoped to get from their care. We also observed the teams in action. We originally came up with a pilot list of domains that we thought palliative care services should be addressing. These original items included pain and symptom control, anxieties and family support needs, and some information on co-ordination needs. There were some outcomes that were not directly related to patients and families but were related to helping and advising other professionals because in the United Kingdom, the services were also about supporting other services. We then said, "Can we turn this list of things into something that can be assessed?" After a review of the literature turned up nothing suitable, nor easy to use, we began to work on devising rating scales for each item. It took years. We played around with them, changed them, revised them. In the end, we came up with a five-point scale for each item, which would be a grading of how severe the problem was for the patient or family member whose experience the staff member was assessing. We called this tool the Support Team Assessment Schedule, (STAS) because it gave a staff assessment at a particular point in time of how much of a problem a patient or family member had with each of those items. The team could then look at how the scores on each item changed over time and whether things got better or worse: so, in a sense, you could "see" the results of care in terms of these particular outcome measures. The nine core items on the STAS are: Pain Control, Other Symptom Control, Patient Anxiety, Family Anxiety, Patient Insight, Family Insight, Communication between Patient and Family, Communication between Professionals, and Communication Professional to Patient and Family.

That brings us to a very interesting conundrum that you have talked about in a number of your papers: How do you train people to assess something from the patient's perspective if they're not actually asking the patient? How do staff assess patient experience?

Well, there is a conundrum in that. We did quite a lot of work in trying to understand the different perspectives that patients, family members and staff bring to completing this assessment. I think that every person brings his or her own particular biases and viewpoints to an assessment. People come with different perspectives, so even when patients record an assessment, they assess things within the frame of their own life experience.

Perspectives also vary from patient to patient. So patients might have their own particular perspective, and similarly family members will bring not only their life experience, but their own personal feelings at that moment in time to the assessment. I think you have to accept that all of these perspectives are slightly different and all are important. Whenever we use an assessment, we need to be aware of who makes it, and what effect that has.

For example, if you use staff, they will tend to assess a patient within the range of experience of patients that they have seen, and with their own particular biases about what they want to see and what they don't want to see. Now, one of the important things to get staff to do, actually, is to get them to ask patients and family members about some things which staff might otherwise prefer to ignore. One of the things the palliative care staff are very good at doing is asking about things that other staff on other services do not ask about.

So in a way, the assessment schedule actually changed practice because it made staff ask about things that they would not otherwise ask about. We trained them to try to record how the patient felt or considered the problems themselves, rather than the staff's own perception of the items. The other problem with staff making assessments is that standard training encourages you to measure the length of something or the size of something rather than how much of a problem it is to the patient. So, we really tried to get staff to record how much of a problem something was to the person.

How much training does it take to implement the STAS?

It is a process of role training and practicing. You can give people the concept of it quite quickly. But it is usually only after people have been using the tool on a regular basis for a period of a few months that they would be really trained, because they need to practice and discuss what the items on the STAS mean.

Do you encourage the same member of the palliative care team to do the assessments, or do different people assess each patient, each time?

In order for the audit to be effective, we feel it needs to be part of routine practice. The system needs to work how the team works, and not the other way around. The team needs to fit the assessment in with that practice.

We have some suggestions and we can give some guidance as to how other people do it. Often palliative care teams do use a key worker to do the assessments, because they have a key worker assigned to a patient, but it really depends on how the team works. Sometimes the discrepancies are interesting. For example, if from one week to the next, different people are making an assessment of the same patient, and if they get a very different assessment of severity for an item for the same patient, it becomes quite interesting to discuss it and understand why.

We start off the training on imaginary patients, so people begin with descriptions of patients and then go into role play. During the initial day of training, people practice using the measure, interpreting the scores and so on, and then they go on and practice it in their work settings. Usually, the first month of use is really a month of training, in which people are learning to understand the measure and assessment. Then, the assessments can be included in the patient record.

The challenge in palliative care is that often patients cannot complete assessments. Or even if they can complete forms, they may not want to. Therefore, I think that training staff to be able to interview a person and to record information that could then be used in a standardized assessment, almost like what psychiatrists do in the present state examination, is very important.

In the future, we want to have a kind of gold standard assessment of a person's issues or concerns based on a particular format of interview. In this interview, the staff member would ask for information from the patient, so that the patient would not have to fill out a form.

In this scenario, the tool would be a form that initiates the interview process and then allows the staff member to record the person's experience. Would that be more time consuming than what you've described as the current use of the STAS?

Yes, more time consuming in one way, but on the other hand, an initial palliative care interview may take a long time anyway. If it is done at a person's home or it is a first interview, it may be three-quarters of an hour or more. Later assessments are as long as the person needs, or the number of problems warrant. If you are looking after someone in home care, you might well have a fairly long talk with that person at each visit. It takes time for the person to tell you what he or she feels about any problems and what's happening.

What we're suggesting is that we should try to find a way to capture and write down the results of what some of the best palliative care services already do. In other words, they do use this longer interview and explore not only a person's symptoms but how the person is feeling and how the family's feeling and how the whole family's coping during that home visit.

So, good palliative care services, in the UK are already doing this kind of in-depth interview as part of practice. Is the challenge, then, to get staff to accurately record what they're learning from the patient in a format like the STAS?

Exactly. In fact, we've moved on, from the STAS into a newer measure called the Palliative Care Outcome Scale (POS), which is a bit shorter and has a form where you can get patients to complete their own assessments. The patient form is similar, but slightly more user-friendly in format. In addition, I'm increasingly moving towards a checklist of some items that should be regularly recorded, particular symptoms such as dyspnea, or nausea, because that helps remind health care professionals and screens for problems clinically. Staff might not otherwise ask about these symptoms, and the person might not raise them.

You also need a section that is free-field, where the patient can raise whatever is bothering them. One would elicit this information by asking an open-ended question such as, "What are the main things that are concerning you?" We have such a question in the latest version of the patient-oriented POS. In the staff version there is an additional open-ended question in which staff are asked "If any, what have been the patient's main problems in the last 3 days?"

How does the Palliative Care Outcome Scale (POS) differ from the STAS? What did you learn from implementing the STAS that led you to do things differently with the POS?

There were several things that were wrong with the STAS that might be better in the Palliative Care Outcome Scale. There is overlap between the two, as they are similar in some ways. First, we wanted to have patients complete a separate questionnaire. Second, in the STAS there are some items around insight and planning which were tricky. A patient's insight into his or her disease and prognosis is useful for staff to know clinically, but technically insight is not an outcome, because you don't necessarily want to make people have a certain type of insight. You tend to work with where people are. Particularly given all the research done on cancer patients, in which there appears to be some evidence that "denial" has been associated with longer survival, it would actually be a bit of a problem to start fiddling around with people's insight. Unlike pain, which you would want to relieve, you don't necessarily want everybody to have full insight, so we thought those items in the STAS should be dropped from anything we would call an outcomes scale. Some staff, however, still like to use the insight items because they provide an understanding of where the patient is, which is of value for communication.

The actual wording used in the two instruments is rather different. The POS grounds each question in a very particular time frame, namely, "over the past 3 days." The STAS considered a week, which didn't work so well in inpatient settings. Furthermore, in the POS each probe is in the form of a specific question that in some instances explicitly names whose perspective is being elicited. For example, question # 9 of the staff questionnaire reads, "Over the past 3 days, how much time do you feel has been wasted on appointments relating to the healthcare of this patient, e.g. waiting around for transport or repeating tests?" Both tools follow each probe with a forced choice severity scale from zero to four. The words to clarify the severity level are more concise in the POS. For example, item #1 in the staff POS reads,

"Over the past 3 days, has the patient been affected by pain?"

[ ] 0 Not at all, no effect

[ ] 1 Slightly—but not bothered to be rid of it

[ ] 2 Moderately—pain limits some activity

[ ] 3 Severely, activities or concentration markedly affected

[ ] 4 Overwhelmingly, unable to think of anything else"⁹

The staff POS has ten items, followed by an open-ended question and a question on performance status. Most of the nine core STAS items are covered in the ten POS items. We did move the communication and information items around a little bit. We decided to have something called "information" rather than "communication" because the end result was whether people felt they had the information that they wanted to have.

We also looked at the "McGill Quality of Life Questionnaire"¹⁰ and were quite impressed with the ideas that they had tried to look at, including self-worth and well-being. So, we wanted to try to include some items in those areas. I have to say we are reexamining that domain because it doesn't always work well in an English population. For example, people were asked about how they would rate their well-being and how they would rate their life as they looked at it overall. Question # 7 of the staff POS reads, "Over the past 3 days, do you think they have felt life was worth living?" We've had mixed reviews on this question and a related question about how patients feel about themselves. Some patients felt the questions were a very abrupt way of asking about this sensitive issue. A small number of patients and staff found these questions uncomfortable, although in other instances these questions opened a conversation. However, I think the concerns are enough to make us want to do some more testing on this area, and see if there is an alternate wording that works better in the UK.

Another change is that in the POS, we wanted to have almost exactly the same questions in the patient and staff questionnaire so we could compare them.

People tend to think that quality of life is something that only the patient can tell you about. What do you think of that idea?

The more I think about quality of life, the more I think it's interesting and complicated. The simple answer to your question is, yes, only the patient can tell you about his or her quality of life. But on the other hand, in palliative care, unless you find ways to use proxies (that is other people telling you about the patient's quality of life), you will never get any information on your weakest group of patients, because they may be unable to tell you. Using proxies is necessary but more difficult.

However, I'm also slightly skeptical of people who say, "Well, of course the quality of life questionnaire must be filled in by the patient." Most of these questionnaires have been designed by professionals. So, while the tool elicits the patient's rating of how he or she is feeling on a whole series of items, the weighting, the underlying concepts, the range of questions that are included and the factor analysis are all determined by professionals. This takes us into the realm of person-centered measures of quality of life, which is a newer field in palliative care.

We recently submitted a research proposal to try to understand how people perceive quality of life differently in different cultures, and how they perceive it differently at different stages of illness. Your perception of what makes up good quality of life at end-stage illness and what is important to you at that stage, may not be the same as earlier. We also need to understand how change over time is perceived because in the clinical setting it is crucial to understand what changes in quality of life are important to the patient. It's not enough just to say, "The patient has a score of X," the point is what that score means to the patient and how an individual patient weights the changes in scores.

Can you give me an example?

In palliative care, for example, if you have a quality of life assessment on week one of care and you're using a standardized measure, like the EORTC quality of life measure¹¹, or the FACT¹² and that patient has a particular score, and then three weeks later, the patient has a different score, the big questions are: Does the team feel that the change in score is clinically important? Are the components in that change important for the person? Perhaps the patient is pleased that he or she has moved to a lower pain score, but the quality of life measure looks lousy because in fact the patient became functionally less mobile.

What is important is how you add up these different components. How do patients weight the changes that happen? In general, people say that quality of life is influenced very much by expectation. One of the problems you have in many of the scales is you're not taking into account expectation. I can think of a woman who was quite frail and had not much physical ability who commented on a quality of life measure by saying, "Well, according to this measure, I have a lousy quality of life because I can't do my washing up. I actually think I have a good quality of life if somebody comes and does my washing up for me." I think that one of the weaknesses of many of the quality of life measures used among frail people is that we exclude the fact that you can have service interventions that get around some of that level of frailty or disability.

Patients' understandings—what particular symptoms or limitations mean—inform the concepts of quality of life and quality of care, so the boundaries between these two concepts can become less clear. Can you say more about this potential confusion?

For me, an outcome is the beginning. That's the result of an intervention. Quality of life is one of the potential outcomes. In order for care to be successful, the professionals have to understand patients' experience including perceptions of quality of life. The professionals are the ones who are doing things to make changes and so improve overall quality of care. If the staff do not understand the patients' perceptions, or if their perception is completely different and they are not aware of that gap, then that is not helpful. So, one of my objectives is to try to bring the two sets of perceptions closer together. Because it is the gap between what the professional thinks and what the patient thinks that is often a major problem in dealing with the service, and what we need to do is close that gap.

High quality of care, is also a result or outcome of care, but it's more about what the staff do, and how they deliver services. How and what services are delivered, such as communication, information giving, and practical support, have a major effect on quality of life, often as much as some symptoms.

This new instrument, the POS does give an opportunity to be more aware of the potential gap in staff and patient perspectives. It allows you to gather the two perspectives on these items and compare them. In fact, at the moment in St. Christopher's Hospice where both the patient and the staff questionnaire are now being used routinely, staff are picking up on when, for example, the patient says something that they have not identified.

For example, sometimes staff think that communication is terrible, and the patient thinks it's much better than the staff do, and that is interesting in itself because this difference highlights the patient's priorities.

What about the social acceptability concern, patients gliding over the negatives because of being afraid to be labeled "difficult patients?" Have you noticed any of that as you are working with a patient assessment and a staff assessment?

Well, some patients want to please when they're completing assessments. In some ways, having the open questions at the beginning about a person's main concern helps to draw them away from that. It is one of the new things that I like.

Are there other areas where you are learning right now?

There are two main areas—extending the POS beyond the specialist palliative care setting into other settings, and also looking much more at how it's useful in the clinical setting. We're exploring these questions as part of a new project, Promote, which is funded by the National Lotteries Research Board. Palliative care is relevant in many areas of care. It is relevant not only when people get into hospital or specialist palliative care services, but when they are with their family doctors or with other people, maybe even when people are thinking about when to refer somebody to a palliative care service. I actually think the assessment could be quite useful for doctors to help make decisions about referral to palliative care services. What's more, I think it could help other health care professionals, maybe even more than palliative care people, to explore what might be important to patients and families.

The other area I'm interested in right now is trying to look more at the utility of these types of tools in clinical practice. These instruments were developed as audit tools to measure how well a service works, but they can also be useful in the individual clinical encounter. I want to learn whether the tool actually helps communication between the individual and the health professional, or helps the health professional sort and prioritize problems and plan treatment or interventions. Or can it be used to look at how well an individual is responding to a particular course of treatment or intervention?

Are the STAS and POS used in that way right now?

I'm quite interested in trying to understand how it could be used more in that way. Remember, these tools were originally designed for the purpose of audit, which is for quality management of groups of patients. Their purpose was to identify groups of patients who did well or who did badly or pick out individuals who did well or did badly, but not to plan individual care. Although it wasn't the original plan, using it in this way had value for the staff and patients because it was clinically based. A lot of people are using the STAS in that way. The tools don't tend to get used if they're not put in the individual charts, so the places that have continued to use the STAS are the ones that use it in the individual charts. In fact, I would say the STAS is being used more in individual charts than not. I'm interested in how to push the clinical use further. With regard to the POS, I'm not yet sure how that will go. We'll have to see.

It seems as though the POS may be even more important for individual care, because it is designed to capture any gap between clinician perceptions and patient perceptions.

Yes, I think so. I'd be interested in looking at barriers to its implementation, what makes it work, what doesn't make it work in individual care.

How many people are now using the POS?

It's new. We originally started in eight centers and most of them are still using it in one form or another. Since then, we've done some training days, so I don't have an accurate count of how many there are. It is primarily being used in the United Kingdom. I would say approximately 15 palliative care teams are using it at the moment. We had one general practice that was using it originally. At present, 96 teams have registered an interest in using the POS. The sites involved in Promote, our new project, will also be using this tool. We are just now recruiting the sites in London and the West Midlands.

How long does the training take for the POS?

At the moment, we are running a half-day training.

Is the POS available for clinicians to use yet?

Yes. We have just published a paper in Quality and Health Care, which included the staff version of the instrument.¹³ This first paper reports on the psychometric properties of the POS. We are in the process of developing the manual and booklet for the POS, so we have only been releasing the tool to those people who have attended a training. We are registering people who want to use it so we can give them the updates. We are encouraging people, who want to use it to contact us, and we can send them a pack.¹⁴ A charity, Help the Hospices, has funded us to write the booklet. We've given a bit more thought to how we develop the project and how to make the tool itself more user-friendly. We are pleased to let people have access to it on the web, here in Innovations. But I do ask that people register with me to let us know they are interested in using it. We don't charge for it, but if they register, it's very helpful because we know who is and isn't using it, we can follow up with users, so that we can get their views about what they've done, and send them updates and new information.

Post a Comment

1. Shaw CD. Aspects of audit.1. The background. British Medical Journal. 1980;280:1256-8.[Return to Featured Innovation]

2. Shaw CD. Aspects of audit. 2.Audit in British hospitals. British Medical Journal. 1980;280:1314-6.[Return to Featured Innovation]

3. Shaw CD. Aspects of audit. 3.Audit in British general practice. British Medical Journal. 1980;280:1361-3.[Return to Featured Innovation]

4. Shaw CD. Aspects of audit. 4. Acceptability of audit. British Medical Journal. 1980;280:1443-6.[Return to Featured Innovation]

5. Shaw CD. Aspects of audit. 5. Looking forward to audit. British Medical Journal. 1980;280:1509-11.[Return to Featured Innovation]

6. Higginson I (ed.). Clinical Audit in Palliative Care. Oxford: Radcliffe Medical Press. 1993. [Return to Featured Innovation]

7. Donabedian A. Evaluating the quality of medical care. Milbank Memorial Fund Quarterly. 1966;44:166-203. [Return to Featured Innovation]

8. Donabedian A. Explorations in Quality Assessment and Monitoring, 3 vols. Vol.1, The Definition of Quality and Approaches to Its Assessment. Vol.2, The Criteria and Standards of Monitoring. Vol. 3, The Methods and Findings of Quality Assessment and Monitoring: An Illustrated Analysis. Ann Arbor, Michigan: Health Administration Press, 1980,1982,1985. [Return to Featured Innovation]

9. Hearn J, Higginson IJ. Development and validation of a core outcome measure for palliative care: the palliative care outcome scale. Quality and Health Care. 1999;8:p. 226.[Return to Featured Innovation]

10. Cohen SR, Mount BM, Strobel MG, Bui F. The McGill Quality of Life Questionnaire: A measure of quality of life appropriate for people with advanced disease. A preliminary study of validity and acceptability. Palliative Medicine. 1995; 9:207-219.[Return to Featured Innovation]

11. Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, Filiberti A, Flechtner H, Fleishman SB, de Haes CJM, Kaasa S, Klee M, Osoba D, Razavi D, Rofe PB, Schraub S, Sneeuw K, Sullivan M, Takeda F, for the EORTC Study Group on Quality of Life. The European Organization for Research and Treatment of Cancer QLQ-C30: A quality of life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute. 1993; 85:365-376.[Return to Featured Innovation]

12. Cella DF, Tulsky DS, Gray G, Sarafian B, Lloyd S, Linn E, Bonomi A., Silberman M, Yellen SB, Winicour P, Brannon J, Eckberg K, Purl S, Blendowski C, Goodman M, Barnicle M, Stewart I, McHale M, Bonomi P, Kaplan E, Taylor S, Thomas C, & Harris J. The Functional Assessment of Cancer Therapy (FACT) Scale: Development and validation of the general measure. Journal of Clinical Oncology. 1993;11(3): 570-579. [Return to Featured Innovation]

13. Hearn J, Higginson IJ. On behalf of the Palliative Care Core Audit Project Advisory Group. Development and validation of a core outcome measure for palliative care: the palliative care outcome scale. Quality and Health Care. 1999;8:219-227.[Return to Featured Innovation]

14. You may download the POS at this website, but please complete the registration form and return it to Prof. Irene Higginson at Dept. of Palliative Care and Policy, King's College London and St. Christopher's Hospice, 51-59 Lawrie Park Rd. Sydenham, London SE26, United Kingdom. Fax: +44(country code for England)- 20-776-9345. [Return to Featured Innovation]