AMCHP 2006 ANNUAL CONFERENCE
EARLY CHILDHOOD: BUILDING THE FOUNDATION FOR LIFELONG HEALTH
March 4-8, 2006
NEVA KAYE: Now, I think, if I could just slide that back a little bit so I don’t have it—we are going to shift in our focus from this extremely well documented best of possible practice for outcome model that I was privileged to be a part of in site number 16, to a brand-new tool that is an outcome measure. Ad I’d like to refer you to your handouts. There is a flier that describes the book for the life skill progression. It’s a Brooks publication, and this will tell you how you can get it. And so you know that it was published, this is what you would be looking for on their Website. And then, I’m not going to be using the PowerPoint because I’d like you to spend a lot of your time while I’m talking listening to me, but looking at the life skill progression tool. And as I talk, I’d like you to notice the differences between this tool for outcomes that was designed by the field with the consultation of national experts, as opposed to designed in a university or a research center and passed downstream. There is a considerable difference in the design because of that bottoms-up approach.
The life skill progression was intended to be a tool to define progress and outcomes for low-income parents. That is both mothers and fathers, if you have programs that are able to serve fathers. And for children under age three. Because the child scales are based on developmental screenings that are done using standardized tools other than the LSP to determine a child’s developmental level, you, if you are working in a Head Start type setting that are serving families that are preschool or older, if you’re using developmental screening tools that go up to age five or six, many of the scales for the child can continue on up into that age range.
So, remember that we’re talking about in general, a tool designed primarily for home visitation services and for families with pregnant or parenting very young children. As such, it’s the first tool of its sort nationally. So, unlike the Ole’s Research, which has such beautiful longitudinal results, we have had about five years of experience with pilots with this tool. It is used now in about 11 states. It’s used in a wide range of programs. Public health nursing programs. There are a couple of nurse family partnerships programs that had been using it. The early Head Start was one of our pilot sites, and Parents as Teachers was another pilot site. And interestingly enough, in our pilot program, we were able to use all of the home visitation programs in the county having this as their outcome tool. So we were able to look at outcomes across intervention models and across different types of staffing.
So one of the things we feel with confidence that we can say now is that this LSP tool, with training, can be used by any type of home visitor that we have in the field, whether they’re paraprofessional or parent educators or public health nurses or social workers. The other environments that it’s starting to move into are the infant mental-health arenas, where there’s an intimate relationship and the family’s life crisis is interfering with the mental health, or helping precipitate it. There is potential for use in social services for court systems.
The tool is designed to capture sequential progress. And one of the concepts that was important in the design of it is that when you walk into one of these extremely chaotic families, you almost don’t know where to begin with the problems that the family is facing. So this was intended to be a tool that is a utilization focused evaluation product, or a tool that will allow the visitor to get as much good from using it for intervention planning and clarity and seeing what’s going on by organizing the chaos that they encounter as it will for documenting program outcome tools and providing sequential progress in a definitive way.
The other issue that is unique about the LSP is that many of our outcomes and Pat’s list in her slides was an excellent example of the outcomes that many programs have been hoping to achieve, are defined as prevent childbirth. Some of you HRSA folks are working with the early Healthy Starts, rather, and that is prevent low birth weight. So the goals are prevent this or prevent that. And prevention obviously in public health is hugely important. But if you’re the visitor that is walking into a chaotic home situation, the long-term goal of preventing a low birth weight baby is not particularly, it’s in the back of your mind, but it’s not particularly useful in the intervention planning where the rubber meets the road to be able to support the family’s change process and to identify with the family in an ISP what issues they need to deal with first, although medical is usually high on our list of intervention. So, I found that Neal Halfon’s model from UCLA that’s called Critical Pathways, and if you’re not familiar with it, I do have a copy with me. Rand has published it. It’s on their Website. And you can get it from the Rand folks. And what is interesting about it is that his critical pathway of outcomes seems to capture the complexity in creating outcomes. He talks about structural determinants, which include things like the impact of poverty on outcomes, or maternal education on outcomes. The next item is a process determinants, and that would be things like access or use of prenatal care. In order to affect outcomes, those issues need to be addressed. Most of our current outcome targets, our program goals, are stated in ultimate outcomes, prevent child abuse, prevent low birth weight, prevent prematurity. And what I think the LSP is filling in is a blank spot in what we haven’t measured, and that is what Halfon refers to as intermediate outcomes.
If you look at what influences the factors that go into a low birth weight baby, those of you that are in HRSA could probably spit out maybe 20 of them fairly quickly and in terms of the type of data that you are interested in getting. So in order to create an outcome down the road, a home visitor needs to be able to intercede with a family in a way that taps their own motivational volition and intention to create the outcome change. That means that the outcomes, sorry. That means that the outcomes need to be closer to the area entered—thank you—where the visitor is actually working with the family and useful to her in planning the intervention. Before I go through the tool with you, I want to say that Brad Richardson with the University of Iowa School of Social Work did the preliminary reliability study, and it came in at about 90%. We’ve had a private evaluator, who was using the tool in Orange County do a separate study, and it looks like it runs between 80 and 90%, as far as we can tell for inter- and intra greater reliability. This assumes that they have gone through an eight-hour training in the use and scoring, and that they have materials to reference if they have questions about how to score a family.
So, if you—oh, and the validity studies, before we went to print, I was able to use the zero to three fellows. Catherine Bernard was my primary mentor in the design of the tool, if you’re familiar with Catherine Bernard’s work. She has a wonderful mind for outcomes. And I had people like Walter Gilliam from our presentation at lunch yesterday as part of my review team for the validity study. We were able to do multi-program, multi-ethnic, multi-discipline group of about 70 reviewers. All but the dentists at UCSF reviewed all of the scales voluntarily. The dentists just reviewed their own. So we feel at this point that there certainly is more that can be done with reliability and with validity, but it looks like we’re good to go and start using it at a much broader base out in the field.
If you’ll reference the document and look at the first page, the top of the page is the heading material. This is the material that you would need for a relationship database. It includes the family identifiers, and if we are doing Web-based entry, there may be a Web number involved. There is an indicator for whether this is the initial LSP, an ongoing LSP, or a closing LSP. I should say that once you’ve been trained, the time commitment on this is that it is, our recommendation is that it’s done at intake, within the first two visits. It takes about five minutes to do, once you been trained, and it’s done twice a year. So that is a total time commitment of staff for yet one more piece of paper in their data pile of 10 minutes a year. That’s doable.
Uh, anybody who is foolish enough to suggest to field staff that they’d like one more data form should prepare to be shot at. But after the initial, “You’ve got to be kidding response,” they embrace this. And Joanne will be talking shortly about her experience in Indiana. We would then do a closing one whenever it happens. What that is allowing us to do is see progress in six-month leaps. And we built in markers for months of service so that you could factor dose, dose of intervention received. So we have months of service and we have numbers of attempted visits and number of completed visits. So you can see dose in either of those ways. You can run reports by the visitor, you can sort it by the program, and as I mentioned, there are communities where the entire community of visitation across multiple programs are using it. There are programs where an entire state is using it.
In the Healthy Starts, there is a long list of medical codes that are trapped. If you want to sort the scale outcomes by linking them to medical codes, either for the parent or the child, or for DSM4, mental health codes, like substance use or depression, or if you want to use codes for to babies that have special needs, you can uniquely create numbered identifiers for those medical codes, and then you can sort your data by those codes as well, so that you can get to psychosocial profiles on the parents to match the medical conditions that are interfering with your outcomes.
There is a reminder in our database of when the next LSP is due. And race and ethnicity we left blank because, near as I can tell, every program has a slightly different requirement. And it allows programs to do their own coding for their community and their funding requirements as well. The categories that the LS P. covers, if you’ll follow along with me the dark headings as we walk through the scale, there are relationships with family and friends, there are relationships with children, which is the parenting scale section, there are relationships with supportive resources, so that you can look at what would be functional health literacy skills and how families are using not only your program but other programs, and the visitor.
There is an education and employment, and that includes immigration issues as well as language, and there are some political implications in that one. Health and medical care for both the parent and the child, I said the scale could be used for fathers. There is one scale in here that they don’t qualify for, and that would be in the medical care section. They are not getting prenatal care, and it doesn’t count if they just go along for the ride. And then there is a mental health section. We had broken out of depression and a substance use, even though they are in the DSM4 book, because we are seeing so much of that and we need that information separately.
The last section is the parents scale would be the ability to provide for basic essentials. This is where you’d see your housing, your nutrition, and use of WIC would be in that one. Transportation, and that varies, depending on rural and urban settings, medical health insurance, and that’s where you can track what their coverage, or lack of it is, and income and childcare. That’s the snapshot that we are suggesting is most useful to home visitors in looking at long-term outcomes. It’s a Likert scale. Zero means they didn’t tell me or I don’t know. One is as low as it gets. And that means that all of the violence indicators, whether it’s domestic violence or child abuse or child neglect, would be a score of one. And five is as good as it gets. As you read across, in scoring this, a visitor would circle the words that apply, and because it is a clinical planning tool, they can write on this form. For example, in the first scale on relationship with their family or extended family, if they have different relationships with a mother or their own father, and you’d get two scores for that one the scale. You would blend those skills as an average for the data, but you could write in “Father out of the home due to sexual abuse,” and Mother, you might give her a score of a four where she’s available but not really emotionally invested.
So that you would have the capability of clinically capturing what’s real and still assigning a numerical value where you could see progress and movement across it. The numerical score is then written in. One of the concepts, this is another one that’s unique, is that there is not an IQ for this. The scales unzip, they stand-alone, they’re analyzed alone. Because they’re separate and distinct life skills. And they have something attached to them conceptually that is called a target range. A target range means that they have reached a life skill for that scale that is adequate to optimal. If you looked at the first scale, and you scan across from one until five, where would you say that family relationships would be at a level that are adequate or optimal?
We set it at four and five for that scale. And the target scale varies for a scale. For example, if you look at Scale 33, medical and health insurance, we’re saying that a target range is two or up. So it’s two to five. And the reason is that any coverage, even if it’s only for a pregnancy, is better than none at all, in terms of being able to receive care relative to pregnancy and prenatal care.
So that’s an example of the concepts that are in the LSP. The child scales were designed to match the ages and the stages developmental questionnaire ASQ and ASQSE, so that you can use a DDST or some other screening scale, although those are the two main ones that I’m encountering in the field. What is unique about the child scales for development is that it’s not enough for an outcome to demonstrate that you screened them, or that you referred them, or who you referred them to for early intervention. If they have a developmental delay, you want them referred buying in, and going to and participating on a regular basis with early intervention. So what you have here is the ability to capture whether you refer them, whether they have delays first of all, whether those delays meet early intervention criteria, whether they’re enrolled, and the degree to which they’re participating. So that’s a different measure than we normally see.
If you are looking at less than early intervention eligible delays, that would be a 3.5 as a score. And that’s in the instructions. And then you have a category that allows you to indicate that they’re average or above average. Those developmental scales, 36 through 40, are the ones that you can expend up into the preschool years if you’re using a screening tool that goes, say up to age five. And there are Head Start programs that are using the LSP as an outcome tool. The other scales, there is one for social emotional that the ASQSE would fit with, and the regulation and breastfeeding are there as well. That’s the tool. We have created training that teaches supervisors and staff to use this tool by transferring the numerical scores to a summary sheet, so that if you have a person in the program like NFP for two and a half or three years, you would have probably about six LSPs, or seven, if you had the closure one.
And if you want to see progress, if you have to flip through five pages times six, it’s cumbersome. So we’ve done a summary sheet where if they transfer the numerical scores and are used to knowing what the scoring means, they can see what has moved into a target range in a flash. We haven’t known when people change or what they change first, or what they change in order to be able to get to a change in another scale. We’re starting to be able to see that with the data that we have on this. In the book, we have a disk at the back of the book that allows a program to print the forms onsite and maintain them themselves. The database is an access database, and that is available through me, and you have my cards with the email. It is an access 2000 or more XP model. What we’re finding is that programs, even if they have their own system, are better off to start using this separate database, seeing what the material looks like, getting used to working with the outcomes before they integrate it with their larger system and have the facilitation of data just dropping in so that it doesn’t have to be filled out each time. That’s true for programs where the staff have their own computers and do their own data entry. We’re also recommending that the LSP be used for reflective supervision and there is a structured form for practicing to use the LSP to reflect on identifying needs, identifying strengths, identifying what has changed in the last in the last six months, identifying what you want to work on next, and supporting the development of a reflective question.
And we have a separate curriculum that supports the teaching of reflective questions that would go along to support this. So there is a longer package than what I’m describing now. I’m going to pass the microphone to Dr. Martin. And she has been piloting the LSP in her Healthy Families Indiana programs.