AMCHP 2005 ANNUAL CONFERENCE
DELIVERING RESULTS, IMPROVING PREGNANCY & BIRTH
February 19-23, 2005
JODI DRISKO: Over the social marketing piece I was just going to briefly go over what we did during our social marketing research piece. So I don’t think I ever introduced myself, I’m Jodi Drisko. I’m the Director of Research and Evaluation in our Health Statistics Section at the State Health Department. So I’ve been working with this program from an evaluation perspective but also helping using the data to drive program planning, which we’ll talk about a little later, and how you can use the research and the data in an evaluation framework to actually drive your program planning versus doing evaluation at the end as an afterthought, which is what I think most people do. So just to give you an overview of--kind of an overall how did we even start the campaign. So this is even before the campaign. This is our fact finding. How are we going to develop our message? What is our message? Who should be our target audience?
So we started out with doing some focus groups with consumers, which were women who had just delivered for the most part. Or women in their--who were pregnant at the time, in the provider’s office at the time. And then also providers and all of--I think we did some midwives and then family practioners and Ob/Gyn’s for the healthcare providers. I don’t--we--as we get further into explaining the campaign, we talk about provider much more broadly. But these focus groups were with healthcare providers. So then after we did the focus groups, which we’ll talk about in a few minutes, we selected a advertising agency who was Praco and they had helped us--they had worked in this area before and we could contract with them fairly easily through I’m sure all of you know the laborious state contracts process.
So we worked with them to create the materials and we--Stephanie will talk a little bit more in detail, but we did multiple mockups and then we retested them with a similar audience, sometimes the exact same people who we did the focus groups with. So we retested things, changed them, retested again to try to find something that would resonate with as many different groups as possible without having to have 40 different sets of materials. So then the campaign promotion and next steps is kind of where we’re at right now. So I thought I would just from an evaluation perspective--this--I’ll be talking mostly from an evaluation data user perspective that I think in public health we use the term focus group very loosely. And what I really want to reiterate is that focus group interviews are a research method, and a lot of times because we don’t have enough money, we’re not necessarily trained to do focus groups, we go out, find a convenient sample, and do a “focus group”. And just ask a couple questions and I tend to call those discussion groups, more fact-finding.
But a focus group interview--I mean, here’s a kind of a definition. They allow for group interaction and greater insight into why certain opinions are held. So you can really get a lot of very rich information out of it if they’re done properly. You can use them for multiple purposes, planning and designing new programs, which this would fall into that. Evaluating existing programs, which we might end up doing focus groups later down the road to see if people are understanding the materials, if they’re the right materials, if they’re reaching the intended audience correctly. And then developing marketing strategies, which is what this whole thing has really been about. So focus group interviews are really the ideal way to understand how people regard an experience, an idea or an event. But you want to construct your group carefully because you want people to have some shared experience or event or an idea that you’re going after.
So, like, with our groups we did them with pregnant women or newly postpartum women so they had all just shared the same experience and we were talking to them about something that had to do with this experience. And I think a lot of times--and I don’t want to knock us for not having enough money to hire people to do them right, but it really helps if you have some training about it. So kind of just how they work is--the biggest part is that they promote self-disclosure, and it really allows you to understand people’s perceptions and attitude related to whatever concept or idea or whatever you kind of want to throw at them. And when you construct questions, say you do have a contractor, which hopefully you can find one that has some expertise around the issue that you’re interested in. You really need to work with them to develop the questions. You don’t want to say this is what I’m interested in and let them develop the questions and you don’t necessarily want to develop them by yourself.
Because although many focus group questions seem deceivingly simple because they need to be really open-ended, they’re quite complex to do justice to them and really pull off the group appropriately. So with this whole self-disclosure thing, that’s where you really get to the meat of the issue, because if the focus group is run right, people will self-disclose or often, as I’ve seen in a lot of teen focus groups, they won’t self disclose but they’ll say things like, “My friend did this,” or, “I know someone who says this about that.” So they won’t necessarily say, “I feel that way.” But they’ll say they know someone who does even though they’re really talking about themselves. So just a couple characteristics that are important for successful focus groups are careful recruitment of participants. You really need to have some kind of systematic procedure of how you’re going to recruit people.
They need to have that shared experience. You need--a good group size is usually five to 10 people. You don’t want any more because that gets out of control. And depending what kind of audience you’re recruiting for you, probably need to recruit twice as many people as you want to show up because the majority of people will not show up unless you’re giving them a lot of money. And my rule of thumb is the harder the group is to reach the more money you should give them. So if you’re trying to do some kind of focus groups with Medicaid moms you want to give them a good chunk of cash, you know? You don’t want to give them $5. That’s insulting. But, you know, if you can $50 or some kind of gift certificate somewhere. But the harder the group is to recruit and the more valuable their information is the more you should offer them, because they’re offering you a lot.
You usually want to do at least three to four groups. I think a lot of times we end up doing one group then maybe another one and say, “Oh, it was exactly like that other one, so I’m sure all of them will be the same.” So you really want to try and do three to four groups to get a good idea of the different types of opinions that are out there. And also if you’re trying to reach different populations you’ll want to do a couple groups around each of those. We did some--what--what groups did we do? We did some monolingual Spanish groups—
STEPHANIE BEAUDETTE: Yeah.
JODI DRISKO: --and then we did Hispanic but not—
STEPHANIE BEAUDETTE: Bilingual, right.
JODI DRISKO: --bilingual or English-speaking—
STEPHANIE BEAUDETTE: We had low-income Caucasian, mid-income and high-income Caucasian as well.
JODI DRISKO: We didn’t do a lot of research with the African American Community because in Colorado we don’t have a lot of African American women or African American people in general. So you want similar types of people in your group. But you don’t want them to be friends with each other because they’ll end up having their own communication and possibly alienating some of the other people in the group. So then you want to set a proper meeting environment, which is usually a neutral setting. So you don’t want to invite them to come to the State Health Department because even though as we all work in the State Health Department and know that it really is just another building that people from the outside will think this State Building and probably not consider that as neutral. Rec. centers, community spots are really some of the best places to do it. And if you want to know about healthcare issues it’s often not ideal to do it in a healthcare clinic, community health center.
Do it outside of that because people won’t be able to be critical when they’re in that environment that they’re talking about. Usually you want to set it up in a circle so people can communicate with each other and tape record it if possible, asking people’s permission all the time if that is acceptable. And then the skillful moderator. You really want someone who has--who’s trained to do this, who uses your predetermined questions that hopefully you’ve worked with them to develop, and someone who can establish a safe environment that will allow for open communication. So a skilled moderator--I’ve done focus groups before, I took a weekend seminar or something from Richard Krueger, who’s focus group guru so--I haven’t done them in a couple of years though so I would feel a little rusty doing them. But I remember before doing them I would be more nervous before a focus group than anything else, because you always want it to go right, you want to make sure that people are happy, and it’s way more nerve wracking than any kind of presentation, that’s for sure.
And then the appropriate analysis and reporting. I think you really need to--hopefully the consultant or whoever you pay to do this kind of thing will know how to do the appropriate, qualitative analysis and report back to you the main themes and quotes are always great. If you can get the transcripts yourself, too, because you may pick up on some issues that other people may not, as well. And that is just a little kind of two cents worth on just some very basic stuff about focus groups. I think this afternoon is a workshop on how to do focus groups, which I’m sure you’re all gonna go to after sitting through this one for three and a half hours. So Stephanie will talk to you about the results that we got from our focus group with physicians and with consumers.
STEPHANIE BEAUDETTE: We--as--as Jodi mentioned, we did focus group testing with physicians and with pregnant women just to get an idea of, you know, what are their attitudes and beliefs about weight gain and pregnancy and what are the main themes before we even get started with materials and direction of--with our campaign. And so we worked with about 20 Ob/Gyn’s, family practioners, they were private practice and public health physicians. And I can’t recall specifically if--if the focus group moderator had a big focus group. I know there were a lot of one-on-one calls, so to speak, with these providers to get the information that they needed for a focus group and I believe they were given about $100 honorarium for taking the time to--to give us this information. So it was--it was very enlightening for me although it shouldn’t have been shocking to kind of see what some of their responses and comments were, which I’m--I’m going to go through in a moment. And with pregnant women, that group was held a couple months afterwards. And as mentioned earlier, recruiting double the group that you would like to have is a good idea.
We--at several points in the time when they were doing the focus group testing with the pregnant women is when I was just coming onboard with the State Health Department, kind of taking over with this campaign from where it was started. And I remember hearing about how extremely difficult it was to get these pregnant women to come and participate in the focus groups. You know, as a pregnant woman whether they have other children or not, already their time is very busy and so trying to find one more hour of time to dedicate to something that they may or may not believe in was quite challenging. However, these same women, as--as we had mentioned earlier, a lot the same women were ones that we went back to and were able to have them participate in some repeat focus groups once we actually had some materials for message testing.
This was most often postpartum for these women and at that point in time they were so excited and thought that, you know, there should be focus groups and meetings all over the place because, you know, just like a lot of women, we get together and we get on a common subject and we can’t stop talking. And so these women were so excited to share their pregnancy stories and delivery stories and wanted forums to--to network even more after this. We did give them gift certificates to Babies-R-Us, and I can’t remember what the amount was but I--I thought that it was somewhere in the range of at least $25. We thought that would be something tangible for them to kind of make it worth their while and‑-and something that they could use. So our women were, actually they were all pregnant women. They were in their second and third trimester, all income groups. We do have a very high Hispanic population in Colorado , and within that group we have a large migrant population who come to Colorado specifically to deliver, oftentimes late in their pregnancy.
So that is our primarily monolingual Hispanic group, and we had translators for that population. Again, all income levels and our ages were 18 to 37. So with our women, and again this is where I’m spending the bulk of time kind of talking about what we found. Some of the themes were similar with what we saw with healthcare providers. There’s a lot of skepticism between the correlation of mom’s weight gain and infant’s birth weight. It’s very hard to kind of put those things together given the recommendations that the healthcare community has been giving over the last decades. When I was pregnant, my mom was very surprised when my providers were saying or--or I knew that I should gain 25 to 35 pounds, because that was unheard of, you know, back in the ‘70s and ‘60s when I was born to--to recommend weight gain over 20 pounds. And so there’s that thought of, you know, you shouldn’t be gaining too much weight because you’re never going to lose that weight afterwards. There is a lack of education from healthcare providers. This was very common with our mid- to upper-income women.
There is the assumption out there from healthcare providers that this group has the financial resources and the education that they should either know better or have access to the “What to Expect When You’re Expecting” and the myriad of websites that are out there giving information about pregnancy. So this group received the least amount of education and information from their healthcare provider during their pregnancy on this subject and our PRAM’s data shows from some other subjects as well. It was the low-income group who has heard from their healthcare provider, be it a nurse midwife or practioner or even their physician how much weight to gain, what things to eat. This group is also tied into the WIC Community where they’re getting nutrition education in addition to their food resources. The Food Stamps Program also gives nutrition education, so they’re getting tons and tons of information, this group that we’re assuming, this poor group, they’re not getting any resources, are actually getting the bulk of education in our community.
And again, our biggest concern, which I actually kind of overlooked, the number one concern that women cited in all of our focus groups except for the monolingual Hispanic women, was fear of becoming overweight or not being able to lose weight postpartum. That was across the board. Our monolingual Hispanic group did not make any mention of concern of gaining too much weight during their pregnancy. We will also see for other health indicators during pregnancy this group has usually the best birth outcomes. The monolingual Hispanic women coming straight from Mexico , eating well, gaining well, lower stress levels surprisingly, less smoking, less drinking, less drug use during pregnancy, having the healthy weight babies. Nonetheless they’re--they’re a big concern in some areas as well in our group too. The longer they’re in the States the more problems they have.
Some of the comments that we received from women during our focus groups. I know three pregnant women who have pictures of models on the inside of their kitchen cupboards so that when they open the cupboard they don’t eat so much. This is really scary how much of a concern this is. The Hollywood stars give a false impression of what a pregnant lady should look like. Now, I mentioned earlier that one in four women in Colorado do not gain enough weight. Also in Colorado , we’re one of the few States on the CDC maps who has not changed to the dark color for the obesity. One in five women in Colorado are underweight prior to pregnancy. So we’re a pretty fit, healthful State in that regard, which is also probably working to our detriment with this specific indicator as well. However, there was some information that I’d seen--I have a whole list of different email newsletters that kind of come through on different topics and that. And I was reading an article somewhere in--I can’t remember if it was the Chicago Tribune or something like that, but it was talking about some women in--in Vermont area and the obsession with weight gain during pregnancy.
And the desire to stay in their pre-pregnancy jeans or just not going to maternity clothes at all, it was fine going up in sizes in jeans but not going into maternity. And that’s really scary that we’re not embracing this time of change in our body. I mean, we’re certainly not advocating pregnancy as a free pass that you can just eat whatever you want, because this is where we’re coming into problems with our healthcare providers in our state is there is that concern that people go crazy and the providers always remember the women who gain 80 pounds and they’re not thinking of the women who only gain 20 pounds. And that in their mind is not a red flag, and this is something that we want to--to change. Again our providers, same thing, skeptical about the correlation. Providers are very evidence-based driven. And so they want to see direct cause and effect. And inadequate weight gain is not a direct contributor to guaranteeing you’re going to have a low birth weight baby. But as I mentioned before, we found in Colorado increases your risk by 65 percent. And this is not uncommon nationwide when you look at some of the other research that’s been out there for years.
Again, physicians, even though they had some very strong statements about the--the weight gain in pregnancy. And when we started raising the issue of inadequate weight gain they were also very candid in telling us they’re really not trained in nutrition during their residency and medical training. Which as a healthcare provider and a dietician I know this, but I know the general public probably has the assumption that the physician is the end-all, be-all and they’re supposed to answer all of their questions and know everything. But most providers have on average about two hours of nutrition education and training throughout their entire program. Some maybe a little bit more and hopefully these trends are changing. But they were very candid in saying they don’t have the information and so therefore they’re not comfortable talking about it with the women. Also, inadequate weight gain is not simply not eating enough food, it’s Pandora’s box. And so when you’ve got limited time, 15 minutes to work with this woman, you don’t kind of want to open up the whole can of worms if you can’t address it. And so they were feeling inept and--and having a lack of tools to deal with this and address it. So what did we do?
We hired Praco to help us develop some materials from the information we got out of these focus groups. And it was a huge learning curve for me learning all of the language and the lingo and, you know, what’s a tagline, what’s a logo, what’s a header? And so they gave us all of this information and--and the logos, for example, you see the *Amship logo on the lower right hand corner and our campaign logo is in the upper left hand corner. If you can’t see it very well, some people thought it looked like the mom’s belly was moving. Actually it’s the silhouette of a pre-pregnancy shape and then a pregnancy shape on the woman, the change there. And so we had this, we had several other logos that they had drawn. And so we took these logos out there.
Our campaign tagline is “A Healthy Baby is Worth the Weight”. We had this and several other taglines that they pitched. We took them out on boards and I didn’t do this, our focus group moderator did it. And tested these messages and the logos again with women to see what resonated. We took photos through our campaign materials, all of our photos, all of the curves, all of the text that was used in all of our materials and the brochures were tested with all of our target groups again and with healthcare providers to see what resonated well with them and what they wanted to see. From our end, what we thought would be appropriate is not what the general public wanted to see at all. We were coming in, myself and other colleagues were coming in thinking we can’t have too affluent a looking woman on the poster because they’re not going to be able to resonate with this woman. Unh-unh.
They wanted to look at someone who was attractive and affluent and maybe who they aspired to be like. We were concerned about how do we hit and cover the cultural differences in Colorado between Caucasian and Hispanic. That tended not to be as big of an issue as we thought. I’ve had comments but only from healthcare providers about the woman in our--in our photos not being Hispanic looking enough. There’s not a lot of pregnancy photos of Hispanic women out there that we could use to begin with, but the Hispanic community resonated with and selected every photo and tagline and header in our campaign.
The only thing that didn’t hit on all marks with that group was the choice of colors with the campaign. We have the blue and the green and that group wanted more vibrant tones. But as you know working for a health department, two-color is big cost saving, so all of our materials except for things that can go through a color printer and or wheels, two colors, huge cost savings. We again took the materials to the--when we did with healthcare providers we went with the nursing community, and that’s the registered nurses, CNM’s, nurse practioners, because we really believe they’re the ones who are doing the education in the office, not the physicians.
My ultimate message is for the physicians to be educated on why we were doing this campaign. What the accurate information is. And don’t negate what the nursing staff already told the--the patient. Be there and be supportive. So we revised, we edited, we went back, finally selected a printer. This was another big challenge within the state agency. We--we found a way to kind of get around some things. We have a $25,000 purchase order threshold, $25,000 to the cent works well and you just have to manage a lot of different people that way rather than going out for a competitive bid. We did have to do a competitive bid for our printer. Rather than trying to, you know, take a course on how you can evaluate different printers and qualities and that for myself to evaluate all of these printers coming in, we went with low bid. And that worked well in all aspects except for we did ask in our low bid for a timeframe to be met with our printing deadline and that was not met. But the quality in that we were--we were pleased with.
Reviewing blue lines, again when you’re really looking into the costs to do these campaigns I think Kelly and Allison mentioned, don’t forget about the staff time. I’m 20 hours a week and do this all the time, eat, sleep and breathe it. It was quite exhausting doing all of this as we were going through it, learning what a blue line is and going through and looking for every dust speck on the blue lines before it goes back to the printer and you approve it. So please factor in the--the time that it’s going to take for your staff to--to work through all aspects of this. Is there any question with just that brief piece that I went through about materials and that? Yes?
UNIDENTIFIED SPEAKER: You were--you were talking about the physicians not having the training. What about nurses? How much nutrition comes with—
STEPHANIE BEAUDETTE: You know, I don’t know the level and I think it would vary on, you know, where they receive their training, how long they’ve been in practice. Just my experiences with some of the nurses that I’ve worked with on this is it’s--it’s all across the board and even within the practice you’ll have a difference of opinion. I think, you know, there’s standards of care are kind of out there and pretty common even though people don’t always follow the standards of care for a particular condition. It would be nice to see that there--it would at least be unanimous within a practice, but one nurse can advocate one thing for weight gain and the other nurse who’s seeing people on a different day could have a complete different knowledge base that she’s coming with. There isn’t a lot of consistency even within a small office setting, which I think is something that we’re hoping to address and--and get some accuracy out there. But I think it’s definitely more than physicians, but I can’t answer specifically how much they get, yeah. Any other questions that--yes?
UNIDENTIFIED SPEAKER: You mentioned the perception or attitude of women who don’t want to gain very much weight in pregnancy. Does that also link to their attitude about their attitude about the baby’s weight? Is there an attitude that they don’t want to have a high birth weight baby or a normal birth weight?
STEPHANIE BEAUDETTE: With our teens--with our teen moms there is the perception out there that having a smaller baby is easier to deliver. I’m hoping that March of Dimes Prematurity Campaign can debunk some of those myths. I’ve seen a lot of their TV spots through different promotional meetings I’ve gone to for this campaign. I have not seen them on TV myself but it was--it was very compelling once you see all the complications of these low birth weight babies and--and knowing the reality they’re not easier to deliver. And if that would be the case they’re certainly not easier to care for afterwards. So that was the only issue there. Ironically, even though these women were telling us they’re--they’re very concerned and kind of can‑-focused on themselves during their pregnancy, they did tell us at the end ultimate goal is that they want to have a healthy baby. They want to do whatever it takes to have a healthy baby. So at least that piece was there. And so I think that’s why, you know, on the cover of our brochure, you know, when I was working with our--our graphic artist, we were trying to figure out, you know, what photo are we going to use. And the photo on the cover of our brochure and on the cover of our folder has a mom looking down at the baby. And, you know, he is really looking into things in--in detail and hidden messages and he’s like, well, this is kind of the outcome and do you want to be focusing on the outcome, you know, of the baby or do you want to be focusing on the mom during her pregnancy? And so shouldn’t we have a pregnant woman there? Well, everybody wanted to see the baby and so I think that that’s where we’re going to get our biggest bang for our buck is--is bringing people into what the outcome of their pregnancy is. Louisiana, and I’m going to talk a little bit about this more at the end, Louisiana is doing a--have been doing a weight gain campaign and their TV spots and ads, it’s a clip of the ultrasound and the baby’s talking to the mom. And that was the way that the women in the state from their focus group testing really resonated with and were able to connect with--with that issue and do something about it, because they also have a very, very high inadequate weight gain and low birth weight rate as well. Any other questions? Yes?
UNIDENTIFIED SPEAKER: You said that you did the focus groups and then hired the PR firm and brought the focus group information to the firm?
STEPHANIE BEAUDETTE: Yeah.
UNIDENTIFIED SPEAKER: What kind of organization did the focus groups for you?
STEPHANIE BEAUDETTE: I’m sorry, what?
UNIDENTIFIED SPEAKER: What kind of organization did the focus groups for you?
STEPHANIE BEAUDETTE: It was an independent contractor. She had her own private practice--or not private practice but group. How would you describe Susan’s?
JODI DRISKO: Yeah, she’s a consultant. She’s done a lot of qualitative research and has done a lot of focus groups in the past. She’s also worked with underserved communities a lot and knows maternal child health issues.
STEPHANIE BEAUDETTE: Okay. Well, I am going to let Jodi go back to the evaluation. She insisted that she come before the campaign rollout because she’s more important.
JODI DRISKO: Well, we’ve done this presentation a couple times at different conferences in Colorado , not the exact same presentation but similar, and I’m always last since I’m the evaluation and I said, “You know, I’m sick of being last because we developed the evaluation before we developed the program. So I should really be first because that’s how we developed the program and it kind of drives home the idea that you should plan for the evaluation and let your data drive the program.” So since this was a--a workshop-type environment I thought I would start with just a generic logic model.
Since I--I’m fairly new to the State Health Department. I’m been there for almost three years, and I have a background in evaluation and research. So when I came to the Health Department I had already known some people because I’d worked with them prior, especially in women’s health. And I’ve had so many different programs who now pay for my salary because we are so poor at evaluation that I’ve had to help all these different programs. And I think in general we all could do a much better job of evaluating all of our programs and setting up evaluation as a planning tool and not just as this afterthought of one more thing I have to do. I really don’t know how to do it and I hate it. So I thought I would just go over kind of a generic logic model that I came across, I don’t know, about a year ago. I’ve done a lot of different trainings on how to do a logic model and what are all the pieces. And somebody gave me this and I thought this is great because it basically has everything in you need you just to fill in increase, decrease and your topic specific information. So logic models can look really different. There isn’t one right way to do it or any necessarily wrong ways to do it, but they all consist of the same basic sets of information: inputs, activities, outputs sometimes are on there sometimes they’re not--sorry this slide is messed up I think it’s--we had to—
STEPHANIE BEAUDETTE: It’s fine in the handbook so that—
JODI DRISKO: --switch computers and who knows what happened. And then there’s usually three levels of outcomes, which are short-term, intermediate and your long-term outcome. And usually your long-term is your big goal, not your small program goal but your more long-term 5-10 year vision of what you want to happen. So with this model I liked it so much because it basically kind of spoon-fed everything from my perception at least as an evaluator. So the inputs are what we invest into a program. This is staff time, volunteer time, your funding, research, your materials, equipment, your partners, coalitions, that kind of thing. So this just kind of says what has gone into the program to begin with? And then the activities, the what we do, the workshops, meetings, services, training, counseling, assessment, partnering, working with media, just all the different activities you do for any kind of program.
So with, say, a WIC Program your activities would be educational activities with the women and the financial piece of it as well. So then outputs really measure participation and that’s what--where we often refer to as kind of the bean counting pieces. How many people did we reach? How many classes were there? The number of people who were involved in the effort. It’s all the basic numbers, which often people really confuse with the outcomes. These are really outputs. It’s that participation, the who did we reach? Those aren’t outcomes. And often these are really hard to track. And then when you were talking earlier about the computer system and how you can’t change anything without it taking three years, I can really know how you feel there because I’ve worked in a system--I was trying to help evaluate a case management program for special needs kids and, like, how many clients do you serve? They said, “I don’t know.” And I said, “What do you mean you don’t know?” “We don’t know. We have some we have at this level, some at this level, some come every once in a while.” And, like, “Well, don’t you have any computer?” “No.” “Don’t you have files?” “Well, yeah.” I said, “Well, you can go and count your files in your file drawer.”
But they just had no idea on even the basic thing of how many clients they serve. So if you don’t even know how many people you’re serving, it’s really hard to figure out if you’re making any kind of change in that population. So then with outcomes or what kind of longer term is impact. The short-term outcomes are usually result in learning. So this is things like increasing awareness, increasing knowledge about something, changing attitudes, which is part of what we’re doing is increasing awareness and trying to change attitudes, improving skills, changing opinions, aspirations, motivations, those kinds of things. Those are short term. So I think with a lot of programs we’re always trying to change somebody’s behavior or awareness about something. So an awareness needs to happen before a behavior can change, which is a medium-term result, which are action-oriented.
So I tried to bold some of these words like the learning, the action and conditions, although I can’t--I don’t think you can see it on the slide. But the medium or intermediate outcomes are these action related things like behavior change, changing some kind of practice, decision making changes, policy changes, which are huge, and some kind of social action where it drives people to do something different. And then the long-term outcomes are more conditions of social, economic, environmental, biological conditions, which in our case--which I’ll go over our logic model in a moment. But the reason it’s called logic model, and I’ve tried to work with people on this. And to me it all makes sense, but to a lot of people it doesn’t. But it’s called the logic model because it should logically progress. So if we do this we can count this but we should expect this, which will lead to this, which will then lead to this. And you can work on them from left to right or from right to left. So there isn’t any one way that is right. So if you know--like, I work with the Steps to a Healthier U.S. Program in Colorado . And we know our goal is to decrease obesity in the state and then to decrease complications from diabetes and asthma.
So we started with that is our goal, that’s--that’s what we have to do in five years no less, hah hah hah. And then we work backwards. So what are the best evidence-based activities we can put in place to affect this? So it we want these things we know that are intermediate outcomes are behavior change and then before behavior comes awareness and knowledge. So we need to put programs into place that will change knowledge and awareness, hopefully impact behaviors by having options and opportunities available for people to change their behavior, and then hopefully someday that will change these long-term goals. Does anyone have questions about logic model? It’s really great to do this as a program-planning tool and not as necessarily an evaluation tool.
Because I’ve worked with programs that they say they’re doing--their activities are doing one thing and their outcomes are really doing something else. And I can give you just a brief example, is what I called their illogic model was they were doing a social case management program for pregnant teens, helping them get cribs, clothes, maybe they’d take them to a prenatal care visit every once in a while, but really trying to help them get what they needed, enrolling them in Medicaid. And their outcomes were reducing low birth weight. So I said, “What does low birth weight have to do with getting a crib and baby clothes? You know, this--if you want to impact low birth weight read the literature first to find out that really not that much in a--social case management will not impact low birth weight, first of all. In that most programs that can impact low birth weight are much more intensive, more home visitation type models, even though their outcomes are two years down the road at age two.
So it’s--you either need to change your outcome or you need to change your program.” So they ended up changing their outcome because that was a little easier and more realistic. So for this campaign our goals or our what will become long-term outcomes were to decrease inadequate weight gain among pregnant women and to ultimately decrease low birth weight in the state. That’s our ultimate goal, but we know that’s not going to happen in five years, especially since we are on a curve of increasing low birth weight rate. Every year it goes up another point one percent and we don’t understand because--why, because our elevation is still remaining the same. So our main two pieces of program activities were to motivate prenatal care healthcare providers, to discuss the issue first and foremost and then to monitor weight gain over time, and then to increase awareness among the consumers as well. So our logic model came up, our inputs--and this is where we started while we were doing focus groups. So we knew that we were going to do some kind of social marketing campaign, but we really didn’t know what it would look like at the time. And Stephanie and I used to work together at our employment place before the health department and then she went--came on to the State Health Department and I came probably, like, eight months later.
STEPHANIE BEAUDETTE: Yes.
JODI DRISKO: So then we end up working on this, which we had worked on some of this stuff at our old place as well a little bit. So we have our PR materials we figured we would develop at some point and the educational materials, our staff, we will talk to you about our pilot project that we did with local health department staff. And in Colorado , I know not all states are like this, I think most of the states here are. We have a State Health Department and then we have large--our large counties have County Health Departments and then our small counties have either Regional Health Departments or nursing services that do just a subset of the functions of public health in the community. And this--we had an inadequate weight gain advisory group as well and then we had money from our MCH Block Grant. So the activities that we decided to do were the ed materials for both providers and patients and then doing trainings with providers in different agencies around the importance of this issue, understanding the research behind it because they, as Stephanie said, are evidence-based for the most part and want to see that research link and then try and tell you that it’s not true. So we knew that was going to be interesting.
And then we have our website, which we’re still working on right now. So our outputs are that we’re tracking are number of providers trained, number of brochures given out, number of patients counseled, which is difficult to measure, and the easiest thing to measure is our number of website hits. So those are outputs. As you can see, that’s not going to tell us how effective our program is. That’ll just tell say how many people we’re reaching. But it doesn’t tell us that our message is getting out there the way that we want to hear it. So our short-term outcomes are to increase awareness, can go back to that generic logic model under awareness and you can write in your own thing: increase, decrease, whatever. Increase awareness among healthcare providers about the importance of talking to patients about weight gain. And preferably they would do this on an ongoing basis instead of just one time. And then to increase awareness among pregnant women about the importance of weight gain during pregnancy.
So then our intermediate outcome is to decrease inadequate weight gain among pregnant women in Colorado that will ultimately lead to hopefully decreasing our low birth weight rate in the state. And I have the data source. So what are we going to do? How are we going to measure these things? Because that’s the other thing. When you come up with your outcomes you have to figure out how are you going to measure this? Because just having them out there is great, but if your computer system can’t change or if you can’t build the infrastructure into place to measure them then it’s just oh, well, maybe it worked, I don’t know. And this whole campaign thing for us has cost quite a bit of money, so we have to tell our funders what we’ve done, who it’s impacted and what has happened. So we can say we’ve made good use of your money.
So our targeted actions, this is more of just an evaluation matrix, is the campaign and all the stuff that’s in the packets that you guys have: the posters, brochures, our little gestational BMI wheel, which has been the best seller ever. Stephanie and I wish we would’ve patented this. We’d make a fortune by now. All the ed materials and our website. So the way we’re measuring that is to look at the prevalence of providers talking to women about weight gain. We’re measuring that from our PRAM survey. We’ve--in the past for all of you who--I know some of you are PRAM states and some of you aren’t--that we just went through a revision of the PRAMS questionnaire last year and so we took the opportunity to change our questions and we were working on this. I also happen to be the PRAMS Director so we can make changes more easily.
STEPHANIE BEAUDETTE: Good connection.
JODI DRISKO: But that shows the importance because our PRAMS is in a health statistics section so its important to make those relationships with the proper stakeholders to make sure that you can get the change that you want. So we’ve had a question for years on, did a doctor, nurse or other healthcare worker talk to you about weight gain during pregnancy? And we had about 80 percent of women say yes. But we have no idea what that meant. What did they talk to them about weight gain? Maybe it was they talked to them at six months saying you haven’t gained enough weight yet or you’ve gained too much. But we just had no idea what they were talking about. So we changed the question on our new version, which I’ll show you in a minute. The proportion of pregnant women who gain an inadequate amount of weight we can monitor through PRAMS but we’re probably among most of you having to do our birth certificate revision, which will start in 2006 for us. We’re a little slow in that area, but I think most states have been.
So we added pre-pregnancy weight and height on there so we can calculate pre-pregnancy BMI and then look at total pounds gained to see if it’s within the IOM recommendations. And we can look at low birth weight due to inadequate weight gain with our PRAMS data and soon--well, in 2008 when our 2006 data will be ready we can look at it on the birth certificate, and then number of brochures given out and our number of website hits. We can look at our brochures. We have an order database so people who--they can fill out this little card which I think you guys may have a sample of as well. And we log that into a database so we can see who’s ordering, how much are they ordering, how often, that kind of thing, how widespread is it throughout the state. And then the website hits is easy. So this is the question that we added to our PRAM survey. It says, how much weight did your doctor, nurse or other healthcare worker tell you to gain during your pregnancy?
And then they can write in exactly how many pounds or kilos they were told. Or they can check, they didn’t tell me how much weight to gain, I don’t remember or other. We had tested this because before you can add--this is considered a state-added question for PRAMS. It’s not one of their core questions or one of their standard set of questions. So if you have a state-added question, which they consider that you just made up yourself, you have to go out and test it. So this was my payback to Stephanie is that I said, “Go out and test this question.”
STEPHANIE BEAUDETTE: Very unusual.
JODI DRISKO: So she went to some WIC Clinics and some other places that talked to women, have them fill out all of our questions that we needed tested, which were probably about eight, and then talked to--look at--we looked at the‑-she looked at the answers while she was there and then asked them, “What do you think this means? What does--what does this question mean to you? Did you understand what it said?” To try and figure out if people knew and that’s actually how we put the I don’t remember and the other on there, because when we tested it we just had pounds or kilos and they didn’t tell me. And then people would write stuff, weird stuff. They told me to gain exactly 18 pounds. It’s like where does that come from? And then people just didn’t remember. And I’m guessing that they probably weren’t spoken to about it but since it was in more of a clinic setting where we were doing the testing of the questions, they may have felt kind of weird about saying that because it’s kind of like, well, they should have and I don’t remember so I don’t know if that happened or not. So we’ve added this question.
We also do a PRAMS-like survey for Wyoming and I think we have a similar question on there as well or we will. We’ll change it so they can monitor their campaign as well. And then as I said earlier that we’ve added the two fields for our 2006 birth certificates, so that will allow us to look at this issue for every woman who delivers in this state instead of our PRAMS survey we get about 2,000 completes a year. We do about a five percent sample where statewide we have about 68,000 births. So this will give us a little richer data.
STEPHANIE BEAUDETTE: That’s your break.
JODI DRISKO: And I guess it’s a break time. How about that? Do you guys want a break or do you want to keep going? Can take a quick break? What do you think?
UNIDENTIFIED SPEAKER: Can we just fill up the coffeepot?