HRSA/MCHB 2005 FEDERAL/STATE PARTNERSHIP MEETING

PUBLIC HEALTH ACROSS THE LIFESPAN

CAHPS Title V Assessment

CHUCK DARBY: My name is Chuck Darby, I’m with the agency for Healthcare Research and Quality and I’m one of the Project Officers on the project that we’ll talk to you today about, called the Consumer Assessment of Health Providers and Systems or CAHPS, for short, and Carla Zema is with me, she and I have worked together for some time. Carla is an expert in the area of patient’s perception of care and also in quality improvement.

Just as an overview, we’ll talk about the CAHPS program and its products, the use of CAHPS surveys and tools by Title V programs, collaboration between Title V and State Medicaid CHIP agencies on CAHPS projects. How many of you when you first came in thought CAHPS was a misspelling of CHAPS, and you were here for something about clothes designer, who knew about CAHPS already? If you could raise you hand. Okay. Any of you in the room use the survey before for some purpose? Okay. Just to save time we want to see if we could cut out 101, but maybe we shouldn’t.

CAHPS stands as I said for the Consumer Assessment of Healthcare Providers and Systems. It is a program for the agents for healthcare research and quality. It is actually a family of surveys and tools that is comprehensive and is currently evolving, I’ll talk to you a little bit about some of the surveys we have. It includes, in addition to the surveys themselves, support, technical assistance that those who use them and also other tools for reporting the data out and covers all aspects of the project planning through administration, evaluation, and reporting.

This is the growing family of surveys, we started out just throwing health plans in the beginning, in 1995, but now have expanded beyond that so we now have facility level surveys for hospitals, dialysis facilities and also nursing homes, a resident survey for nursing homes, which is just about complete and one that’s under development for family members who have loved ones in nursing homes. At the ambulatory level, we still have the health plan survey that we are modifying and working on now, in addition, a survey that we are developing for concisions of medical groups and then finally, a behavioral health survey of behavioral health and substance abuse services called ECHO.

Included in the CAHPS products are standard questionnaires for different ages and groups and coverage populations, the idea of a CAHPS survey is that there are a core set of items that could apply to someone whose on Medicaid, commercially assured, insured, or in Medicare. Also there are optional supplemental questions that we add to the Core, there are protocols for sampling survey administration data collection and data preparation. There are also data analysis programs that are provided, a software package that could be used to analyze the CAHPS data to get scores. There are also reporting guidelines that sponsors can use to help them in figuring how to report the data out. For example, if you are going to report it out to the public, there are templates and guidelines for how best to report that out, based on research that we have been doing with consumers about what they understand about statistical data about their health care.

Why measure the patient’s perspective of care? It is critical to make sure that we have a comprehensive quality measurement strategy and as part of that, it would include measuring the patient’s perspective of care. Also, the surveys that we develop ask questions for which the patient is the only source, or maybe the best source of information. We don’t ask patients about the technical quality, well do you think they did the brain surgery properly or not. What we ask them is, was the communication, the personal aspects of care, was the communication with your provider, how did that go? Did they answer your questions, did they listen to the kinds of questions that you had? Did you get the access to the health care that you needed, etc., and we’ll talk about some of the demands.

Over time, as we have been working on it, we revolved some principles as we developed the surveys. One is that the emphasis should be on consumers of the health care or patients, whichever term you want to use. Report on actual experiences, we ask people to tell us about their experience, only if they have had it, so that if someone has not had a particular interaction with their health care system, we don’t ask them to tell us about it because they are not going to be informed respondents in that case.

We attempt to do standardize measurement as much as possible. The idea is that there have always been a lot of good surveys out there for measuring different health care settings, but not a single survey that was standardized that would allow you to compare one health care facility or one health plan to another. And that’s how we got into it originally, to allow people for example, to choose a health plan so that we provided data to them about what other consumers said about the health care so that they could make a choice. First with health plans, now we are moving to other providers.

Also, as part of this, by being a standardized survey, we can provide a benchmark on a large number of providers that are in the database so there’s someone, one provider wants to compare themselves to other providers of a common type, they can do that. Also, applicability across heterogeneous populations as I mentioned, a single set of core items that can apply regardless of the type of insurance that you have, extensive testing with consumers to find out what it is, what is important to them to focus groups that we do. Providing meaningful information we have found that in the work that we have done that consumers really resonate with the kinds of data that we provide them, more so than the clinical data because they understand it and it is the kind of thing that they think about when they are trying to choose a health provider.

We also do a lot of work with stake holders, anybody that is going to be measured as part of the survey, for example when we work with health plans, we involved a lot of health plans in the design of the survey, to make sure that they thought it was fair and well done, because later down the road if they don’t embrace it, then it’s probably not going to fly. Also we make sure that everything is made, put it in public domain, all of us, I hope all of you have paid your taxes, as tax payers we’re paying for the development of these surveys so that they should be put in the public domain to let anyone use them that wants to.

As far as our measurement strategy, I wanted to talk a little bit about our approach. We make sure that we do both what we call reports and ratings of care. An example of rating of care would be, in the last six months how would you rate or how satisfied are you with how promptly you were seen when you went to the doctor’s office or clinic; excellent, very poor, poor, fair to poor. So the idea is that someone is sort of evaluating or grading their experience. We have those kinds of items but in addition, the bulk of our items are experienced items, such as, in the last six months, did you ever have to wait in the waiting room for more than fifteen minutes past your appointed time – yes or no. So they report on the experience. The little bit of work that we have done we want to do more, indicates that that’s a little closer to actually measuring the quality of the care. With satisfaction items or rating items, the level of expectation that someone has tends to affect their care. We found that particularly true in Medicaid programs, that over a period of time when someone was in the program that sometimes their expectations were at a point that they lowered their expectations. We found that in other settings also. So that in this case the consumers are reporting on and with satisfactions their rating, they’re affected more by expectations. This is the team of public and private research team that we have put together to work on this project; by and large these organizations have been involved with the project since 1995.

The Harvard Medical School, the American Institute for Research, this organization does not like to be called AIR, RAND, my own agency, the Centers for Medicare, Medicaid services, the Centers for Disease Control and also the National Institute of Disability and Rehabilitation Research, a combination of research organizations and private organizations and the Federal Agency. We get a lot of input into the development process that is standard, sort of scientific process of getting, designing a survey, including looking at the literature and existing instruments. We normally have a call for measures where we look to see what’s already out there that might help and form our development. We do something called cognitive interviews where we administer the survey to a respondent and then ask them to give us feedback on the questions to see if they mean what we thought we wanted them to mean. And we find from respondents when they do this we learn a lot about what their interpretations of the items are. We do the field testing, focus groups to get input and then state caller input.

Just as an example for the health plan survey, there are a number of versions. There’s an adult survey with commercial, Medicare, Medicaid, Managed Care and Fee for Service. I noticed a mistake on this slide which we didn’t have a chance. There are no children’s surveys for Medicare, but for a Medicaid, commercial environment, both Managed Care and Fee for Service. This includes the children with chronic conditions survey, short, CCC that is for Children with Special Health Care Needs.

Here are some of the composite measures of remains that we were measuring in that instrument, getting needed care, do you get the care that you need, getting care quickly if you, or you’re able to call after hours and get medical advise, how well doctors communicate, we find that communication composite is at the key of everything, courtesy and respect, helpfulness of the office staff and then things about, in the case of health plans, about the customer service. We also have rating items about all the health care you get, the health plan, personal doctor and nurse and then specialists that someone has seen.

Additional composites that are included in children with special health care needs survey are items on prescription medicine, getting specialized services, family centered care and coordination of the child’s care. By the way, in the blue binders, if you haven’t already gotten one with the folders, there are some handouts that include the items that are included in the questionnaire.

If one were constructing such a questionnaire, you would start with the Core Health Plan Survey that has well-tested, valid composites and then allows us, with that Core, to have comparable results across populations and would also facilitate benchmarking. With the supplemental items we are provided with some flexibility to provide more detailed information in certain areas that people want. We allow people to add supplemental items in the questionnaires as they go and you could add as many as you like but one has to think about what affect it would have on the response rates. We also do this to provide, someone may want very specific state data of interest only to that state or a handful of states and also to help provide more detailed information for quality improvement. Here are some of the topics that we have for supplemental items, I won’t go into these in detail. There are items that exist for these, we’re actually in the process of looking at some of these items and seeing if we can’t expand them and improve them.

A typical Medicaid version of the survey might be the CAHPS Health Plan Core Survey, starting with a, then adding a set, it would be specific for children with special health care needs. Other supplemental items that might be added then if you wish you could add your own items. So the idea is of modularity to the survey that allows you to maintain the Core so that you could compare yourselves to others but at the same time add supplemental items.

These data go into something called the National CAHPS Benchmarking Database. I think we are in the process of actually changing the name but you can still find it under this name if you were to go on the Web. This allows people to submit their data and then to get a report back, free of charge, that compares their data to other Medicaid programs or commercial health plans and others. Here’s just some contact information as indicated, this is part of something called the CAHPS User Network, which we’ll talk a little bit more about. We provide a full range of technical assistance for anyone who wants to administer the survey on how to draw the sample, how to report out the results, etc., etc. And the data, NCBD, has been in existence and we have data from 2000, moving forward.

Some of the products are an annual chart book that is an aggregation of all the data that has been submitted by types of plans, individual sponsor reports that I mentioned, files for researchers who may want to do an analysis and we have had a number of researchers who have analyzed the CAHPS database. Then we will do custom analyses and reports and provide detailed information through fact sheets and other research briefs.

Also, a tool that was developed with funding from the Centers for Medicare and Medicaid Services by the Harvard Medical team is something called the CAHPS Improvement Guide. It’s one thing to get all of this information and sort of know where you stand, see where your strengths and weaknesses are, it’s another thing to say, well what do I do to make it better. This Improvement Guide is also now available on the Website, it was prepared, even though it was prepared for a Medicare agency, it is applicable across types of plans, so it is something that you might want to take a look at. It provides over two-dozen strategies for mapping the CAHPS scores and working them. Now I am going to turn it over to Carla who will be a little bit more precise about what implication this might be for Title V.